The Great Gig
Dr. Ching Hua Chuan is creating a software program that will make things a lot easier for aspiring rock stars everywhere
By: Julianna M. Klose
Call it the merging of art and science. Assistant Professor of Computer Science Dr. Ching-Hua Chuan had long been interested in both engineering and music, studying towards her master’s degree in electrical engineering by day, while playing in rock bands in her native Taiwan by night. But, it was combining these two far-ranging interests while working on her doctorate at the University of Southern California (USC) that led her to develop a new software, automatic style-specific accompaniment, or ASSA, that some say will change the way musicians compose music.
“I’m very fortunate,” Chuan says. “I’ve combined my interest with my professional specialty. Before USC, it was always a struggle for me. I played in bands after work and was still an engineer during the day. It was like two different lives.”
ASSA generates musical accompaniments to new or existing melodies but, unlike similar programs already on the market, it can “learn” the style of a specific band, and generate harmonies and accompaniment to match this band’s musical style. Chuan developed the software while working on her doctorate under Dr. Elaine Chew at USC and is continuing her research since joining Barry’s faculty in fall 2008.
Her interest in music began early, learning guitar at age 12 and, by the time she was in college, Chuan found herself playing guitar for several rock/pop bands in Taiwan. While performing with these groups, she began to notice a common problem.
“When you’re in a band, most of us didn’t have formal musical training,” she said. “Sometimes we’d have a hard time communicating using actual musical terms. If you wanted the drummer to accompany you in a certain style, sometimes it would be hard to communicate how. We would find the best way was to grab a CD and let them hear a musical style.”
Her software operates on this same principal – allow the system to hear the style, and then imitate accompaniment in the same style. It’s a process already familiar to many musicians who play by ear; Chuan simply automated it.
It has been tested with original melodies, as well as melodies already written by other bands. When developed, the program was tested using five rock albums from Chuan’s favorite artists, including Radiohead, Green Day and Keane. In the testing stage, the system resembled the original accompaniment in anywhere from 70 to 85 percent of the accompaniment it generated.
In essence, ASSA has found a way to capture something more elusive than simply matching chords – an artist’s musical style. In fact, when tested against similar software that operates off basic musical theory, it was able to match more chord tones to the band’s original accompaniment.
“We emulate the style and the accompaniment part of that group – the style of harmonization,” Chuan says. “To harmonize, you have the melody and the chords. The system analyzes this relationship between the melody and the chords…one chord follows another and different chords give you different feelings. It can make you feel tension or make you feel relaxed. The system looks at what kind of transition the band would use under what kind of musical context.”
According to Chuan’s original advisor, Dr. Elaine Chew, associate professor of engineering and music at USC, ASSA stands out because, “of the way it treats each song as an individual.”
"Often when we talk about style, we talk in generalizations, such as the 'classical style,' which lasted 70 years or hundreds of years, depending on whether you refer to a historical period in music or the iTunes genre label," she said. "I think that the software is different because it doesn't treat any song as a generic entity, it allows the style of each song to be very individualized."
This type of system can sometimes have unexpected results – results not found in a music theory textbook. Chuan cites a test she performed while attending a recent academic workshop on rock music. At the event, she tested her software before an audience of musicians.
“There were 15 other graduate students there, but everyone except for me was involved in music,” she says. “I was the only engineer. I was a little nervous.”
To showcase her software, she trained it to Fiona Apple’s musical style, and then generated accompaniment in this style to Radiohead’s song “Creep.” Chuan claims the feedback was interesting – where the chords matched with music theory rules, listeners were pleased, but where the chords didn’t match, they were confused. The irony was that these rule-breaking chords actually matched with Fiona Apple’s more experimental style – the style the software was trained on.
“In terms of these ‘different’ parts, the system was actually imitating Fiona Apple’s style,” Chuan says. “So it wasn’t actually a bad result, because the system was doing what it was supposed to and generating results based on that style. So actually, the ‘bad parts’ still met the goal.”
Chuan is in the process of licensing ASSA, and is already fielding offers from companies looking to tap into its commercial potential. With the mainstream success of other musical software, such as Guitar Hero and Rock Band, there are certainly commercial implications for software that automates both style and accompaniment.
Before reaching this point, however, Chuan is continuing to research and refine ASSA. At this stage, the software outputs only chord names or the written notations for the appropriate chords. Her goal is to reach a point where it will also output rhythms and instrumental parts.
“There’s no ending for research,” Chuan says. “There’s always more to learn and improvements that can be made...For me, if I work on something I’m really interested in then I’m highly motivated. I don’t have to make myself work; I really want to figure it out.”
This motivation is something she tries to instill in her students – a drive to find their interests, however far-ranging, and merge them into a career.
“I guess I was born with this,” she says. “Even when I was composing, I would do it systematically. I was born to do this type of thing, using both sides of my brain.”
And, while this prospect may seem intimidating – computerized creativity – supporters insist that ASSA does not eliminate art from musical composition, it only builds on it.
"The whole point is not to replace composers," Chew says. "Composers have a place in this world that is really special. The software is only as good as the input - it simply learns the style that a human composer has already created. When the output is good, the computer is only using that knowledge, in a sense standing on the shoulders of excellent song writers."