Parallel Algorithms for Artificial Intelligence

The purpose of the laboratory is to develop alternative methods for executing parallel programs in order to reduce resource consumption in artificial intelligence projects. We focus on two main areas: breaking down large operations into smaller, more manageable subtasks, and planning the order in which these subtasks should be executed on available hardware resources. The lab develops its own machine learning framework NNTile.

Deliverables

To achieve a highly scalable performance for current AI needs we will focus on:

Asynchronous parallel training and inference
Task-based parallel programming model
Scheduling algorithms
Communication reduction by quantization
Numerical stability of operations
Alternatives to linear layers
Impact of data ordering on performance of linear layers in mixed precisions
Writing custom low-level kernels using OpenAI Triton
Контакты

Сколковский институт науки и технологий

Территория Инновационного Центра «Сколково»

Россия, Москва, 121205, Большой бульвар д. 30, стр. 1

Офис E-A4-3028

Руководитель лаборатории