Rapid Ghost Pedestrian Detection: Spike Vision for Safer Autonomous Driving

<title>Abstract</title> Autonomous driving and robotics are increasingly focusing on improving safety, particularly in collision avoidance systems. Detecting unexpected obstacles quickly is essential for preventing accidents, especially in high-speed driving scenarios where the sudden appearance of pedestrians presents a significant challenge. Conventional systems often rely on prediction, but they struggle with obstacles emerging suddenly from blind spots, introducing delays in detection. Here, we show that spike cameras, can significantly reduce these delays by enabling rapid object detection. Our system integrates perception, reasoning, and action into an end-to-end, ultra-low-latency closed-loop that allows for faster reactions. Our system reduces the overall latency of detecting sudden appearance of objects to under 5 ms: in real-world ghosting pedestrian scenarios, it achieves a 97% success rate in obstacle avoidance tests. Additionally, even at an equivalent real vehicle speed of 118 km/h, the system maintains a success rate of 92.5%. These results represent a meaningful improvement in autonomous driving safety, providing faster, more accurate obstacle detection in high-speed environments, where quick reactions are crucial for avoiding accidents and ensuring the safety of all road users.