On December 15, 11 winners of the fifth international Yandex ML Prize were awarded in Moscow. Skoltech PhD students Nikita Gushchin, Anton Razzhigaev, and Denis Kuznedelev, who study at the Computational and Data Science and Engineering program, received awards in three nominations. Experts recognized them as one of the best among 200 applicants from 11 countries. All winners will receive awards from 500 thousand to 1 million rubles, depending on the nomination, access to Yandex 360 services, and a grant of 500 thousand rubles to use the Yandex Cloud platform, which can help perform calculations and process experimental data.

Denis Kuznedelev studies acceleration and compression of neural networks: “My colleagues and I have proposed a new effective method for assessing the importance of parameters in the network and determining the optimal configuration of compressed weights. We conducted most of the experiments on neural network architectures used in the field of computer vision, but the method itself is universal and can be applied to almost any field of artificial language or audio processing. A number of modern neural network architectures can now be run on devices with limited computing power even without Internet access — for example, a neural network for face recognition or speech on a smartphone, a multilingual translator. I also do machine learning on heterophilic graphs. This area is interesting from an applied point of view: It allows us to develop algorithms to identify scammers on social networks and suspicious banking transactions.”

Denis received an award in the “Yandex Researchers” nomination. He shared that it was a great honor and luck for him: “I’m very excited to receive recognition for my scientific results, especially considering the number of contestants. This event was also a great opportunity to get to know and network with other researchers in the field.”

Anton Razzhigaev is working on three research areas: the integration of proven knowledge sources with language models to solve the problem of hallucinations, multimodal research combining various data formats, and the study of large models’ learning mechanisms, including the analysis of their activations and internal representations. The student received an award in the “Researchers” nomination. 

“As for the first area, I’m trying to combine structured information from Wikipedia and language models so that they don’t make up facts (hallucinations). As for the second area, it seems that strong AI can be built on not only through text, but also on the integration of all major data formats: texts, images, video, audio. And the amount of textual information is very limited — in a couple of years, we will exhaust it and teach language models on everything that is available, so we need to expand the area of knowledge available to them. In the third area, I’m extremely interested in how and why large models learn so well. To understand this, I delve into these models, study their activations, the dynamics of their embeddings. During such dissection of trained (and learning) models, it becomes clear how to make them more efficient — faster, more compact and, hopefully, even smarter,” commented Anton.

Nikita Gushchin is developing new generative neural networks for images and biological data: “In the case of images, I’m developing neural networks capable of improving the quality and resolution of images or redrawing images in a different style. In biology, networks model the development of stem cells and the response of cells to medicines.” 

Nikita received an award in the “First publication” nomination. “This year, I’ve two papers to the NeurIPS 2023 conference, and one of the papers was in the top 2.5%. Since these were my first articles, I applied for this nomination. During the hour-long interview, I expanded on my work and future plans. As a result, Yandex revealed that I’m one of the three winners in the “First Publication” nomination, which of course makes me very happy,” shared Nikita.