If only Mozart had this

Music to Your Eyes A new computer system can convert sound files into sheet music by creating a harmonic dictionary. It could conceivably convert this Global Symphony Orchestra production of "Star Wars" back into written notes. via Flickr/ Paraflyer

Spanish telecommunications engineers have devised a new method to generate sheet music based on the sounds of individual notes, which it can identify regardless of musician, instrument, and venue.

The research team, from the University of Jaen in Jaen, Spain, describes an automated system that determines the spectral pattern of an instrument's musical notes. The pattern is used to create a harmonic dictionary, which is paired with a pattern algorithm. The system then determines which note is which, and converts the information into a readable format. Given a WAV file of a recording, the software can produce a MIDI transcription.

Automatic music transcription could help musicologists analyze sound samples, recover musical content and separate varying audio sources, according to Julio José Carabias, co-author of the paper and a researcher from the Department of Telecommunications Engineering at the University of Jaen.

The method's details were published in IEEE Transactions on Audio, Speech, and Language Processing. The system is adaptable, meaning it can interpret any instrument, from dulcimers to didgeridoos. As of now, it only works for one instrument at a time, but the researchers think the method can be scaled to include many instruments playing at once. Other musical transcription devices use databases and are trained to recognize specific notes -- much like a spectrometer is trained to recognize the spectra of certain chemical compounds. But the Spanish device learns on its own, by creating its own dictionary.

A sound spectrum is a representation of a sound in terms of the amount of vibration at each individual frequency. The distribution of a note's harmonic energy defines its spectral pattern. Using that information, the system creates a dictionary of sound. It identifies the notes even when the type of instrument, musician, type of music or recording studio conditions vary.

"Another advantage of this method is that it does not require prior training with a musical database," Carabias said.

8 Comments

As a violinist, I understand that this must have been extremely difficult to achieve. Nice job!

bdhoro87

from coral gables, fl

I don't think being a musician gives one an accurate idea of this audio engineering solution. Also, I thought I've seen software in Guitar Center that does a very similar task called Celemony Melodyne, I don't think it works with just any recording, but it's able to recognize things like chords, and discern individual notes, and breaks down full recordings into separate tracks.

This could be very useful to do true transcriptions of great performances. You can buy a book of classical piano, but what if I want to learn to play a classical song exactly the way a certain performer played it? Many similar such applications could change the way we view musical compositions.

Could anyone explain why non-musicians have much more 'accurate' ideas about music than musicians do? Must be that we go about doing things the wrong way. We study and practice instead of passively absorbing all of that knowledge via popular media.

Well, there have been midi implementations of this for decades. Extracting note data from a wav file is different, though not impossible. In fact, I'm surprised that this is "news."

The problem with this sort of computer transcription is that the computer has trouble quantizing the rhythms and beat durations, so what you end up with is pretty much unreadable, unless you adjust quantization settings. But even then, something like a triplet can screw up the transcription. And also, the computer would most likely transcribe everything enharmonically, unless the user specified what key signature the song (or portion of the song was in).

Conductors make the best S/W Engineering Leaders! Unlike sports where the goal is competitive, music creates and focuses on the synergies and qualities of the end product, not the glory of individuals or even the group. A conductor deals with people, their differences, their strengths, their needs, their different sensitivities and communication styles, (their parents, too!). The conductor must get to the essence with few words and gestures, always keeping everyone on the same beat (not just the same page).

A few weeks ago I critically examined the work cited in this popular science piece on my research blog at Aalborg University Copenhagen, Medialogy. (I cannot cite my discussion here because it is flogged as spam.)

There is not much novel in their work, and they even show that their approach is not as accurate as others. So does that mean the others can transcribe any performance never recorded?

That's crazy that it can transcribe free music from performances. Technology is getting so good these days.

http://www.ez-tracks.com

ya it's an important milestone in music field's devolopment......


138 years of Popular Science at your fingertips.

Innovation Challenges



Popular Science+ For iPad

Each issue has been completely reimagined for your iPad. See our amazing new vision for magazines that goes far beyond the printed page



Download Our App

Stay up to date on the latest news of the future of science and technology from your iPhone or Android phone with full articles, images and offline viewing



Follow Us On Twitter

Featuring every article from the magazine and website, plus links from around the Web. Also see our PopSci DIY feed


February 2012: The Future of Fun

Science is reinventing play, from extreme sports to gamification to ridiculous roller coasters to the playgrounds of tomorrow, and this issue is chock full of fun. Also, on a less fun note: Did global warming destroy my hometown?


circ-top-header.gif
circ-cover.gif
bmxmag-ps