A research team from the Skoltech AI Center and Samara University have developed a system for automatically separating the stages of production processes from video streams. Industrial cameras will detect deviations in the production process themselves and even prevent accidents. By employing the self-supervised learning approach, the cost of manual data markup can be reduced while the model’s stability in real conditions can be increased. The research results are presented in the IEEE Access Q1 journal, one of the leading international platforms in the field of engineering and computer science.

The technology is designed for time segmentation of video streams from production sites. The system understands the stage of an operation, such as oil change or component assembly, and automatically highlights key points in the video.

“The introduction of such systems provides real savings: Now there’s no need to manually process hundreds of hours of videos to train a neural network to recognize production stages,” explains Maxim Aleshin, a leading machine learning engineer at the Skoltech AI Center. “The model will independently identify patterns in large volumes of raw material. This allows industrial cameras to detect deviations from the normal course of the process in real time and help prevent emergencies.”

The neural network is trained on a large array of unlabeled video recordings, independently identifying key features without the human contribution. Then it undergoes further training on a small marked-up sample and adapts to specific tasks (for example, to classify such events as “wheel change”, “oil change”, and “static state”). The system has shown high video stream processing speed, which makes it suitable for real-time use in industrial environments.

According to Svetlana Illarionova, who heads a research group at the Skoltech AI Center, the technology will be part of broader solutions to ensure industrial safety and optimize production processes.

In the near future, the team plans to expand the number of supported scenarios and types of production operations, test the system on real-world facilities with continuous monitoring of a large number of processes, and integrate the approach into systems for smart video surveillance on industrial sites.

“It is precisely these projects that make production safer and more intelligent. We are confident that the proposed technique will find application beyond the classic assembly lines,” emphasized Svetlana Illarionova.