3 min read
A paper, Navigation under uncertainty: trajectory prediction and occlusion reasoning with switching dynamical systems, published within the last year was submitted to 2025 IEEE International Automated Vehicle Validation Conference and following is a breakdown revisiting its significance.
See the Unseen
Occlusions, or objects hidden from view, are one of the biggest hurdles to making self-driving vehicles safe on the road. Imagine a delivery truck is blocking your view at a crosswalk. What are the odds that someone or something might emerge from behind the truck? Should you swerve, slow down, or prepare to slam on the brakes?
Human drivers rely on experience, context, and intuition to anticipate what might emerge from behind the truck but how do self-driving cars currently predict what might happen next and how to react?
Oceans of data.
The problem with this data-driven approach is that the historical training data only represents the statistically most likely information, or norms, which by definition doesn’t properly account for anomalies or rare events that fall outside the norms such as a kid chasing a ball into the street. Ironically, these outlier scenarios are perhaps the most critical to safety. What’s more, many current models only react to what they can see directly and are essentially oblivious to hidden objects.
The paper from VERSES research team in collaboration with researchers from Volvo Cars Safety Centre proposes a radically different approach using what are called switching dynamical systems.
Imagine an autonomous car with a system that has a set of driving behaviors at its disposal. The system continuously evaluates other cars and pedestrians around it, including what it can and cannot see, and selects the driving behavior that makes the most sense given the context. It both predicts the speed and velocity of objects around it and reasons about what may be hidden. Even better, this unified approach could be far more computationally efficient than traditional methods that treat trajectory prediction and occlusion reasoning as separate problems.
Take a pedestrian crossing the road. The system factors in things like how fast the pedestrian is moving and if they're looking straight ahead or checking their phone – these nuances help predict the right course of action. Environmental contexts may be weighed as well such as the time of day, the flow of foot traffic, or the bus stop just ahead.
Researchers put their model to the test via simulations using real-world driving data from Waymo’s OpenMotion dataset. They were able to not just validate whether the model selected the optimal action but also how it arrived at its conclusions. They also tested how varying amounts of sample driving footage impacted performance.
Vehicle Trajectory Prediction with Uncertainty and Confidence
Another crucial factor measured is the model's confidence level about its predictions and reasoning. A self-driving system that’s always overconfident is just as risky as one that's constantly second-guessing itself. For example you wouldn't want a car slamming on the brakes because it thinks it saw a squirrel out of the corner of its eye. But you also wouldn't want it to ignore a real pedestrian because it was too sure of itself. The ability to account for uncertainty is crucial for safety as circumstances are almost never black and white but rather a continuum of probabilities.
The approach outperformed simpler models that were also tested. Researchers generated visualizations of the system’s beliefs as it navigated a scene which depicts how its understanding evolves as it observes new information.
- After 50 inputs SLDS was 57x (0.82) more certain about Vehicle Trajectory Prediction than cGMM (4.74) and 138x more certain GMM (11.34)
- After 50 inputs SLDS was approximately 3x (0.34) more certain about Pedestrian Trajectory Prediction than cGMM (0.97) and 4x more certain GMM (1.57)
Figure 1: Visualizations of predicted vehicle trajectories.
Figure 2: Visualizations of predicted vehicle trajectories by context length.
Figure 3: The concentric circles represent the models beliefs about the existence and position of pedestrians behind occlusions and the reduction in the number, size, and opacity of them represents the reduction of uncertainty.
The research is still in its early stages but could have significant implications for the future of self-driving. While this model was primarily focused on testing against cars and pedestrians, future work could expand to encompass other motor vehicles, cyclists and even animals running across the road.
This ability to “see the unseen” has applications far beyond just self-driving cars. Other situations that require navigating around obstacles and anticipating the motion of other actors and where information is limited could benefit: search and rescue missions, in-home robot assistants, financial markets, medical diagnoses and more.
