When artist David Li partnered with Google Arts & Culture on a machine learning experiment in late 2020, he didn’t realize the project would make a return at this week’s I/O. “Opera singers use their voices as expressive and beautiful musical instruments and I was interested in the idea of making interactive versions of these voices, so that anyone could play with them,” he says. At the time, he simply wanted to explore what technology could do with the human voice — but the Blob Opera quickly became a hit.
“I’m definitely not a musician,” says Pamela Peter-Agbia. “I stopped playing piano a very long time ago.” Pamela is a producer for Google, and despite her non-musical background, she’s part of a very special, very experimental musical project: bringing Blob Opera to I/O.
“We were very excited to get the invitation to bring Blob Opera to I/O with new features,” says Laurent Gaveau, who’s the head of Google Arts & Culture lab. Laurent is a musician who studied at a conservatory in France and worked in music for 10 years. “Opera is one of my passions,” he says. “The magic of performing art is one of the most difficult things to grasp from a technical standpoint. So it’s been a pleasure to invite David Li to create this experiment that so playfully guides people to learn more about the voice and music. But we weren’t really expecting how much people would enjoy it when we first introduced Blob Opera.”
Before the experiment’s initial release in December 2020, the team needed samples of opera voices. They turned to four talented freelance opera singers who went into the recording studio alone for hours at a time, using their voices and talent to unearth a huge range of sounds. They explained it to the singers as teaching a computer to sing. “Imagine trying to create a kind of map of sound with different elements — vowels, consonants, all the different notes — and then bringing it all together to train an algorithm,” Pamela says. “We fed 16 hours of singing in a Convolutional Neural Network, which learned how to reproduce each voice type based on what it heard,” David explains. In fact, the team created two models: One that taught the computer to sing, and another that taught it to harmonize. The end result being, of course, the Blob Opera.
There was an added bonus to all that time in the studio. “The singers really enjoyed using their voices in a different way — I remember our soprano, Olivia, was excited to see how young people would use this.”
“Until I saw the Blobs, I couldn’t quite understand it — but when I finally saw it, I loved it!” says Olivia ( whose fiance was the tenor). “The sheet music we received was basically parts of an opera, but all written on one note,” she explains. “So we sang those pieces as one note, in a range of pitches, for hours.” Olivia says that while it could be tricky, she thoroughly enjoyed the process — especially since she’d hardly been able to sing live all year.
Pamela hardly minded logging hours in the studio with them. “It made me realize how much I miss live music!”
Fittingly, Blob Opera’s return (which comes with a new look and new features) had its own performance moment on stage with Tune-Yards at Google I/O 2021. Pamela and Laurent mentioned that Tune-Yards’ lead vocalist Merrill Garbus’s voice was a perfect fit for the Blobs’ operatic sounds and that it was incredible to see a real artist working with the blobs as colleagues.
While the Blob Opera will be a part of the Tune-Yards live performance at I/O, they’ll also be available online, where anyone can interact with it. The interactive Blobs, which have new, colorful styles, can also leave their digital opera house and go on tour to places like Cape Town, London, Mexico City, New York, Paris or Seoul. Each location comes with its own set of traditional, local songs – like “Frére Jacques” or La Bamba. David calls the custom version of Blob Opera, used to create the show, a “massively optimized version.” “We used the new WebAssembly SIMD feature so 16 blobs could all move and sing at the same time,” he says. “It was great to get to push web browsers in this way.”
Laurent says it was a joy to bring the Blobs back in this new way, and both he and Pamela describe the process as one of the smoothest collaborations they’ve worked on. Of course, it’s had something of an effect on them as well. “I have to say, what I’ve found really funny,” Pamela says, “…is that I’m literally eating, sleeping, breathing Tune-Yards and opera songs these days!”
To learn more about this or other musical experiments — such as AR Synth or Assisted Melody – visit the Google Arts & Culture experiments website or get our free app for Android or iOS to discover more about performing arts.
