Overview

The Computer Vision for Intelligent Mobility Systems (CVIMS) research group at AImotion Bavaria develops deep learning methods for analyzing and generating image data across two- and three-dimensional sensor modalities. Our overarching goal is to achieve super-human perception for automated vehicles, aircraft, rail systems, and other means of transport. Beyond mobility, we are also committed to applying computer vision for environmental protection and sustainability.
A central focus of our research is cooperative perception, the fusion of sensor data from multiple agents, especially vehicles and infrastructure. By combining diverse imaging modalities, such as cameras and LiDAR sensors, we aim to provide a richer, more reliable view of dynamic traffic environments. This work is part of Ingolstadt’s transformation into a living testbed for autonomous driving, where real-world intersections and dedicated test tracks serve as benchmarks for innovation.

Objectives

  • Improving the reliability and completeness of environmental perception through diverse multi-sensor data integration.
  • Investigating the trustworthiness, alignment, and synchronization of shared data from other traffic participants.
  • Developing robust fusion algorithms capable of handling uncertainties and inconsistencies in sensor and communication data.
  • Enabling a shared, collective understanding of the driving environment through 3D object detection.
  • Building reliable models that enhance situational awareness for safe navigation and real-time decision-making.
     

Research Highlights

Our research in cooperative perception focuses on developing methods that enable multiple agents to interpret complex urban scenes in a consistent and reliable manner. We investigate how temporal modeling, multi-modal fusion and simulation-based analysis can strengthen perception systems as traffic environments vary across time, viewpoint and layout.

Cooperative Perception Across Vehicles and Infrastructure:
The UrbanIng-V2X dataset paper, created jointly by Karthikeyan Chandra Sekaran, Dominik Rößle, Markus Geisler, Adithya Mohan, Daniel Cremers, Wolfgang Utschick, Michael Botsch, Werner Huber and Torsten Schön, provides a diverse set of sensor modalities from multiple agents (vehicles and infrastructure sensors) across multiple intersections for studying cooperative perception.
Its key scientific contribution lies in revealing how strongly the performance of V2X models depends on intersection geometry: many approaches work well in familiar layouts but degrade on unseen configurations. This insight guides the development of more generalizable perception methods.

Temporal Reasoning in BEV Perception:
Research by Dominik Rößle, Jeremias Gerner, Klaus Bogenberger, Daniel Cremers, Stefanie Schmidtner and Torsten Schön investigates how past scene information can be selectively incorporated into Bird’s-Eye-View representations.
By reusing relevant temporal cues, models become more stable in dynamic traffic, more robust to occlusions and more consistent across agents.

Multi-modal and Multi-instance Fusion:
Studies by Dominik Rößle, Daniel Cremers and Torsten Schön develop adaptive fusion mechanisms capable of integrating heterogeneous sensor modalities and varying numbers of detected objects.
These methods address key challenges in cooperative perception, where viewpoints, sensing conditions and contextual information differ across agents.

Simulation-based Scenario Reconstruction:
Research by Jeremias Gerner, Dominik Rößle, Klaus Bogenberger, Daniel Cremers, Stefanie Schmidtner and Torsten Schön advances the reconstruction of vehicle behavior in microscopic traffic simulation.
More accurate scenario reconstruction enables controlled and repeatable evaluation of cooperative perception approaches, particularly for cases that are difficult to capture consistently in the real world.
 

Funding

The research is funded by the Hightech Agenda Bayern, and receives additional funding from the Bavarian Academic Forum – BayWISS.

Impact

This work strengthens Ingolstadt’s position as a hub for autonomous driving development and contributes to creating transparent, trustworthy, and sustainable AI-based perception systems. The cooperative perception developed at CVIMS forms a cornerstone for safer, explainable, and scalable autonomous mobility.

THI Contact

Prof. Dr. Torsten Schön
Computer Vision for Intelligent Mobility Systems
Prof. Dr. Torsten Schön
Phone: +49 841 9348-2335
Room: K201
E-Mail:

Funded by