Augustana Orchestra integrates contemporary music-making technology

In a recent performance, the Augustana Orchestra showcased an innovative project exploring modern production performance, using virtual instrument accompaniment and digital mixing to emulate the illusion of a full, symphonic orchestra.

Senior Gilbert DeWitte, a music and multimedia entrepreneurship major, first proposed the project during a J-term independent study in 2024 with Peter Folliard, the dean of the School of Music and Augustana Orchestra conductor. 

The orchestra showcased the project at its April 7 concert by playing the pieces “Signs of Life II: I. Allegro” and “Overture to Prince Igor” with a recorded accompaniment.

“I want to expose our students and audience to what's already happening in Hollywood. I want to foster curiosity and understanding of how these technologies work and what their potential might be,” Folliard said.

Advances in modern music-making practices include the widespread use of virtual instruments in recordings. The use of virtual instruments is used often in Hollywood soundtracks by notable contemporary composers, including Hans Zimmer and Ludwig Göransson.

“You can listen to any Hollywood film right now and the odds that it’s an actual live orchestra that they recorded playing it is pretty slim,” DeWitte said.

The multimedia entrepreneurship program, which launched in fall 2021, prepares students to succeed as musicians in an increasingly tech-savvy industry. Students in the program learn how to use programs like Pro Tools and Sibelius to understand modern music-making technologies.

As the orchestra leans into embracing this technology, all members of the Augustana orchestra responded to an anonymous survey to gauge interest and willingness to participate in the project. Folliard said he cannot share the survey results until the project has institutional review board approval. 

“I will share that we wouldn't be able to do any of this without Dr. Shannon Proksch, assistant professor in the psychology department and principal oboist in the Augustana orchestra,” Folliard said. “Dr. Proksch’s experience in conducting research about music in a metacognitive context is fascinating and will help us gain a much more scientific understanding of people’s reactions to this project.”

After the survey, orchestra members and Folliard discussed their views on the project, voicing both excitement and concern.

“I wouldn’t want to use this for the majority of our pieces for our concerts,” freshman Ryan Stapish, principal violinist of the orchestra, said. “They’re here to hear us, not the virtual instruments.”

After the concert, audience members were invited to share their thoughts on the project.

“It’s surprisingly hard to tell that they’re using virtual instruments even when you’re actively listening for it,” Aidan Christensen, a senior percussionist, said.

Christensen subbed in for another percussionist on one piece that night and did not play in any of the virtual-instrument-assisted pieces. 

According to Folliard, using virtual instruments also saves money for the School of Music.

“There is a huge money-saving aspect to leveraging digital samples to add what we don't have in people or equipment,” Folliard said. “Just wait for the next time an orchestra needs to buy a new celesta, a beautiful instrument that is used sparingly throughout the year. A new [celesta] is approximately $16,000 and will last for anywhere between 10 to 20 years and requires maintenance, or consider buying a virtual instrument that contains a collection of the best celestas in the world, recorded with the best microphones in the best halls and the total cost is $99.”

Ensembles are able to use virtual instruments to bolster a small chamber ensemble or to add in missing parts. This addition can be significant for community orchestras, which often have difficulty ensuring ensembles have all parts covered and balanced.

While virtual instruments can help fix balance issues, Stapish explained how musicians account for balance without the use of such technology and use unbalanced sections as a chance to learn.

“It’s a learning opportunity to get the other section to play out that’s smaller, and, then, they may get the chance to play quieter, and everyone listens to each other,” Stapish said. “With the playback track, it just throws it all together, and I feel like people won’t listen to the whole orchestra as much.”

Folliard said the conversations around the inclusion of virtual instruments have been the most rewarding part of the project. 

“We're talking about really important and deep concepts like what it means to make music and to share our gifts with our audiences,” Folliard said.

As opposed to synthesizers, which generate audio through programming and mathematical formulas called waveforms, virtual instruments rely on many recordings of world-class musicians playing real instruments with different deliveries, including volume, articulation and even microphone position. 

Once these recordings are compiled into a sample library, individuals can use a musical instrument digital interface controller — a device often with piano-like keys, knobs, buttons and sliders — to trigger sounds from the sample library. These samples create a recording of the instrument’s part, which users can manually adjust to match the desired sound.

DeWitte said a creative opportunity of the digital-instrument MIDI tracks includes instrumentalists’ ability to listen to the MIDI tracks of their instruments and adjust the tempo to their liking when practicing.

While virtual instruments can enhance the listening experience, the use of such technology comes with its own limitations. 

“Tempo does get tricky,” DeWitte said. 

The live ensemble must remain synchronized with the recordings. To achieve this, conductors listen to click tracks through an earpiece or headphones during performances.

Despite the challenges associated with using this technology, Folliard said he remains optimistic about the future use of virtual instruments.

“Overtime, I don’t see how orchestras will be able to avoid integrating virtual instruments into regular practices,” Folliard said. “It has already begun in compositions where composers specifically call for a specific electronic sound, or even a laptop, such as Mason Bates’ ‘Mothership.’ We'll have to ask ourselves these questions about aesthetics, finances, ethics, composer's intent, flexibility and convenience more and more.”