Meet the composers using AI to create music

Performers creating music from thin air with a wave of the hand, controlling musical software using brain scans, or jamming with AI musicians—it all sounds like the stuff of science fiction.

Not for pianist, composer and technologist Zubin Kanga. The celebrated musician is at the forefront of a wave of tech-savvy talents harnessing technology to make the improbable everyday.

For over a decade, Kanga’s fearless attitude towards technology has redefined the perimeters of musical performance.

He has both curated and performed shows that have pushed barriers, with motion sensors, artificial intelligence, live-generated 3D visuals and virtual reality among the technological advancements used to unlock new possibilities in music and performance.

In 2020, after his appointment as lecturer in Musical Performance and Digital Arts at Royal Holloway University, London, Kanga was awarded a UK Research and Innovation Future Leaders Fellowship to fund his Cyborg Soloists project.

“That’s funding for four years of work for the moment,” Kanga tells Reader’s Digest, “possibly another three years after that. It’s a huge range of composers, performers and industry partners, who work with different technologies and researchers.

"We’re creating new works together which explore things like new digital instruments, motion sensors, AI and machine learning, and new approaches to visuals.”

Credit: ThirdMan Productions. Zubin Kanga performs Steel and Bone with MiMu sensor gloves

Kanga’s approach to harnessing cutting-edge technology was first informed by the relative limitations of his chosen instrument.

“The piano is a very sophisticated technology,” he says. “But it is an old technology. It had a long period of development through the 19th century. And then from the early 20th century till now it hasn’t really changed at all. It’s an amazing instrument, but it does have certain limitations in terms of the type of sound you can create.”

Once Kanga began experimenting with technology, there was no looking back.

“There was so much more I could do, as a solo performer on stage. Once you have visual elements you can do all sorts of things. I’ve performed works which deal with film history, others that have used animated visuals or live visuals, and I’ve worked with stop-motion animation.

"Electronics also help with the theatre of the performance. Once you attach sensors to yourself, you can control things live on stage, and the audience can see the relationship between the electronic sound and what I’m doing.”

One of the early works in the Cyborg Soloists project is Steel And Bone, composed by Kanga himself. He performs the piece using MiMu sensor gloves.

“I can put up one finger, and that’ll produce a particular sound,” Kanga explains. “And then I can manipulate that sound just by moving my wrist through the air—I can do that with lots of different gestures.

"In Steel And Bone, I’m actually playing inside the piano with these steel knitting needles, and getting all these interesting, almost percussive effects on the strings. Then I’m using samples of them. Sometimes I’m using live delays and manipulating them. The sound can morph depending on how my hands are moving.

“It allows me to make a very theatrical piece, and people can see this immediate connection between how I’m moving—these very big, almost conductor-like gestures through the air—and the way the sound is changing.

"So I think it’s great for the audience as well. Rather than, say, effects being produced behind a laptop, which can be very opaque to the audience.”

Zubin Kanga premieres Answer Machine Tape, 1987 at Time of Music Festival, Finland, 2022

Composer Philip Venables’ Answer Machine Tape, 1987 is another Cyborg Soloists work using pioneering tech to moving effect. It focuses on New York visual artist David Wojnarowicz and the turbulent period leading up to the death of Wojnarowicz’s close friend Peter Hujar from an AIDS-related illness in 1987.

In the piece, Kanga uses the KeyScanner to transcribe Wojnarowicz’s answering machine tape in the days leading up to Hujar’s death. The piano effectively works as an instrument and typewriter and controls electronic sound and light to create an immersive solo multimedia performance.

Kanga enthuses about the KeyScanner. “It’s a MIDI scanner on the keyboard. It’s been made by Andrew McPherson from the Augmented Instruments Laboratory at Queen Mary University of London.

"The key scanner sits at the back of the piano keys and basically it can tell when a piano key is being pressed. It converts that into a MIDI signal which is taken by a programme called Maximus, which allows you to type onto the screen. And it folds up so I can tour with it.

“I’m transcribing what’s happening on the tape and the piano provides the harmony to what’s happening. Together they give a textual comment on the tape, give context with this interior monologue of what’s happening to David as he might have been listening to this tape, and form character studies of each person who’s calling—galleries, artists, famous New Yorkers like Fran Lebowitz, the hospital.

"And so, it gives you a window into his life, the 1980s in New York and the AIDS crisis.”

This is just the start, Kanga goes on to enthuse about the use of EEG brain scanners to make music, the possibilities that AI offers composers as a tool, how virtual reality could transform performances and more.

It’s obvious how excited Kanga is by the possibilities that Cyborg Soloists works have already opened up and how his work might change the way music is performed and composed. 

“My hope is that a lot of the tools that we create as part of the project can be used by a wide range of musicians. There are a lot of people wanting to know more and get their hands on the gear and to be involved in the project.” Watch this space.

Zubin Kanga performs the UK premiere of Philip Venebles' Answer Tape Machine, 1987 on November, 19 at Huddersfield Contemporary Music Festival.

Read more: Meet the people preserving music

Read more: How music NFTs are shaking up the music industry for fans and artists

Keep up with the top stories from Reader's Digest by subscribing to our weekly newsletter