Tag Archives: robots

Echoes from the Future of Opera: A Mashup of Music and Technoloy at the MIT Media Lab

On Saturday, my classmate Emily Elert and I visited the Opera of the Future research group at the MIT Media Lab. Graduate student Elena (Elly) Jessop was kind enough to spend the day with us and demonstrate some really neat fusions of music and technology.

Now, I can’t talk in detail about absolutely everything we saw, because some of it is not yet ready for the public’s eye. To respect the Media Lab and the researchers who work there, I will only discuss what can already be found online in some form or another. I’ll be linking out a lot in case you’re interested in reading more.

The Opera of the Future group, headed by innovative composer Tod Machover, “explores concepts and techniques to help advance the future of musical composition, performance, learning, and expression.” Their work includes Hyperinstruments, traditional musical instruments enhanced by technology to actively interact with performers: as the musicians play, sensors and computers allow the instruments to interpret the nuances of live performance and accordingly change the sounds they produce, creating a new spectrum of sonic power, range and finesse. Hyperscore is a computer program that translates complex musical concepts into intuitive and colorful visual representations, making relatively sophisticated music composition a possibility for those with no formal training.

Tod Machover playing a hypercello (Credit: Tod Machover, MIT)

A large ongoing project is Death and the Powers, a new opera that makes unprecedented use of technology to modify live musical performance.  The creative team includes Machover (composer), former US Poet Laureate Robert Pinsky (librettist), Artistic Director of the American Repertory Theater Diane Paulus (director), and Hollywood’s Alex McDowell (production designer), who has worked on such visually stunning films as Fight Club, Minority Report and Watchmen.

Proposed Set for Death and the Powers

Proposed Set for Death and the Powers (Credit: Opera of the Future)

The opera – currently scheduled to premiere in Monaco in September, 2010 – centers on the rich and powerful businessman Simon Powers, who dies in the second act. Powers, however, has uploaded his consciousness onto a massive computer called The System, so his essence persists throughout the performance. The stage, which largely represents the Powers household, comes alive as Simon Powers learns to manipulate the external world in his new computerized existence. Power’s daughter Miranda, his wife Evvy, and his research assistant Nicholas struggle to cope with his posthumous influence and understand the choices he made.

To achieve this transcendent effect, researchers at the Opera of the Future Group – in particular Peter Torpey - have developed what they call Disembodied Performance, a collection of technological innovations that allow an absent performer to remain actively involved in a live performance. Although Powers won’t be directly visible, his voice will be heard, his emotions will be felt, his reactions will be known. All kinds of sensors on the singer and actor playing the role of Simon Powers will wirelessly transmit data to a central computer, where the data will be used to modify the operatic performance in real time. Large bookcases onstage will present the audience with waves of color to reflect Powers’s mood, bristling in anger when a character walks by that Powers doesn’t like, for example. Ambisonic sound – which reverberates in any and every direction – will wrap the audience in song, placing them at the very center of Powers’s voice. Many-legged furniture – inspired by the beautiful kinetic sculptures of Theo Jansen – will scurry across the stage.

Elly Jessop has been working on a remarkable device for the opera called the Vocal Augmentation and Manipulation Prosthesis (VAMP): a glove that gives its wearer the power to modify their singing voice in real time using a gesture-based vocabulary.

VAMP

Vocal Augmentation and Manipulation Prosthesis (Credit: Elly Jessop)

Currently, VAMP is a long fabric glove outfitted with pressure sensors and accelerometers that wirelessly transmit data to a computer, which interprets the gestures and – working with a microphone and speakers – changes the singer’s voice. If you want to preserve a note you’re singing for example, you use a plucking gesture to pull the note from your mouth, which signals the computer to continue playing that note as you sing something else. This allows for vocal layering and textured singing. Bending of the elbow controls volume and shaking your hand induces vibrato. You can watch Elly’s fantastic demo on her web site.

VAMP is specifically designed for Nicholas, Simon Power’s research assistant, who – according to the script – has a prosthetic arm than endows him with special abilities. Since their work is for an opera, Elly and her colleagues decided that the arm should give Nicholas special vocal abilities: thus, VAMP. Eventually the glove will look far more like a robotic arm than it does now.

Other technological innovations designed for Death and the Powers include a group of advanced animatronic-like robots, ranging in height from four to seven feet, with giant triangular lights for heads. These robots, which will be partly pupeteered, can sense and avoid one another, perform semi-autonomous movement, and may eventually have their own voices. Within the opera’s narrative framework, they function as a kind of Greek chorus, perpetually retelling the story of Simon Powers in a choreographed pageantry. The also serve as sculptural set pieces and, since they’re so bright, as stage lights.

Stage Robot

Stage Robot for Death and The Powers (Credit: Opera of the Future)

Another example of music and technology working in harmony is a giant stringed instrument called The Chandelier, which can be stimulated by both a human player and by electromagnets, allowing for a much greater range of notes than a standard string instrument. The Chandelier provides the opportunity for a powerful duet between Powers and his wife, after his physical death – another form of disembodied communication.

The Chandelier

The Chandelier (Credit: Opera of the Future)

After the 2010 premiere in Monaco, a US and World tour is planned. I for one can’t wait to see how it turns out.

New Scienceline Blog

You can now see a newer version of my last blog post published on Scienceline.org!

Coming up next: robots that want to be human.