On February 5, in the Svetlanov Hall of the Moscow International House of Music, classical works created by human composers and generated by a neural network were performed together on stage for the first time in Russia. The State Kremlin Orchestra, conducted by Merited Artist of Russia Igor Shevernev, presented a musical synthesis of the Baroque era and modern technologies as part of a project initiated by the Opera Foundation under the leadership of Vera Kononova, with the support of Skoltech.

In cooperation with the Skoltech Artificial Intelligence Center, not only was music produced using generative AI, but two “virtual” voices were also recreated: the baritone “Phantom” (named after the ominous hero of The Phantom of the Opera) and the tenor “Farinelli” (named after the legendary 18th-century singer known for his extraordinary vocal range).

For the research group in generative artificial intelligence at the Skoltech AI Center, this marked their first experience in creating musical compositions. According to Milena Gazdieva, an engineer-researcher at the Center, working on this project required configuring a number of additional parameters and using open-source libraries adapted for generating both musical material and vocal parts:

“Despite Skoltech’s extensive experience in generative AI research, music is a new direction for us, and we are pleased that we succeeded in bringing our ideas to life. Judging by the audience’s response, they appreciated how AI interacts with the richest heritage of Baroque music.”

A significant contribution to the development of the neural network compositions was made by Skoltech master’s student Vladislav Gromadsky.

The project with the State Kremlin Orchestra is of particular interest to Skoltech as it demonstrates the capabilities of generative AI beyond the traditional sphere of scientific tasks.

“The combination of Baroque tradition and modern technologies is a precedent for further research and the development of cross-disciplinary projects uniting engineers, scientists, musicians, and composers,” remarks Professor Evgeny Burnaev, the director of the Skoltech AI Center.

The researchers received assistance in formulating requests for the neural network from Duan Yu Qian, a young Chinese composer who inspired the AI to create soundscapes using the konghou harp, the di flute, and the sheng — ancient national instruments whose sound resembles that of an organ. The concert featured Baroque arias by Johann Sebastian Bach, works by Duan Yu Qian, and experiments by the neural network, aimed at showcasing the potential of artificial intelligence in music.

“A musical duel between neural networks and timeless classics is highly relevant when it comes to attracting a new target audience — young people enthusiastic about AI technologies, many of whom are trying their hand at composing,” explains the project’s creator, soprano Vera Kononova, whose Opera Foundation promotes classical music, increases its public visibility, and fosters new forms of collaboration between musicians and scientists. “However, in my view, the performance of a ‘live’ artist will always surpass that of a synthesized voice, since in addition to mastery, there is an energy that a machine cannot replicate. Yet when a human being, with all their artistry, performs music generated by neural networks — created on the basis of the finest works—then something truly incredible can emerge, delighting the audience.”