Zakhar Yagudin, a master's student in the Engineering Systems program from the Intelligent Space Robotics Laboratory (Skoltech Center for Digital Engineering) shared the conference outcome:

"I presented four papers from our laboratory at the IEEE FLLM 2024 conference in Dubai:

A. Lykov and D. Tsetserukou, 'LLM-BRAIN: AI-driven Fast Generation of Robot Behavior Tree Based on Large Language Model' (more details here),

Z. Guo, Z. Yagudin, A. Lykov, M. Konenkov, and D. Tsetserukou, 'VLM-Auto: VLM-Based Autonomous Driving Assistant with Human-Like Behavior and Understanding for Complex Road Scenes' (more details here),

A. Lykov, M. Altamirano Cabrera, K. Fidele Gbagbe, and D. Tsetserukou, 'Robots Can Feel: LLM-Based Framework for Robot Ethical Reasoning' (more details here),

V. Berman, A. Bazhenov, and D. Tsetserukou, 'MissionGPT: Mission Planner for Mobile Robot Based on Robotics Transformer Model' (more details here).

Overall, the conference went great. There were postdocs from MIT, students from Cambridge, that is, top universities in the world. And overall, there was a lot of positive feedback about our articles."

Zakhar talked about an article he co-authored:

"Our project 'VLM-Auto: VLM-Based Autonomous Driving Assistant with Human-Like Behavior and Understanding for Complex Road Scenes.'"

The problem is that classical autonomous navigation systems are too dependent on rules. If certain scenarios are missing from the initial dataset, their behavior becomes unpredictable. However, by using a large language model, thanks to the generative approach, the system can adapt more flexibly to situations and choose appropriate behaviors. For example, even if certain situations are not present in the training data, the large language model still finds an appropriate action template. In our project, the model receives an image as input and determines environmental parameters: weather conditions, type of road surface (slippery or dry). Based on this data, it adjusts the car’s control. So, if the car is moving through a city in rain and fog, among many pedestrians, it is necessary to reduce speed and acceleration accordingly."