Quadrupedal robots are conquering various applications in indoor and outdoor environments due to their capability to navigate challenging uneven terrains. Exteroceptive information greatly enhances this capability since perceiving their surroundings allows them to adapt their controller and thus achieve higher levels of robustness. However, sensors such as LiDARs and RGB cameras do not provide sufficient information to quickly and precisely react in a highly dynamic environment since they suffer from a bandwidth-latency tradeoff. They require significant bandwidth at high frame rates while featuring significant perceptual latency at lower frame rates, thereby limiting their versatility on resource constrained platforms. In this work, we tackle this problem by equipping our quadruped with an event camera, which does not suffer from this tradeoff due to its asynchronous and sparse operation. In levering the low latency of the events, we push the limits of quadruped agility and demonstrating high-speed ball catching with a net for the first time. We show that our quadruped equipped with an event-camera can catch objects at maximum speeds of 15 m/s from 4 meters, with a success rate of 83%. With a VGA event camera, our method runs at 100 Hz on an NVIDIA Jetson Orin.

Reference:
Benedek Forrai, Takahiro Miki, Daniel Gehrig, Marco Hutter, Davide Scaramuzza
Event-based Agile Object Catching with a Quadrupedal Robot
PDF: https://rpg.ifi.uzh.ch/docs/ICRA23_Fo...

Our research page on event-based vision: http://rpg.ifi.uzh.ch/research_dvs.html

For event-camera datasets, see here:
1. https://dsec.ifi.uzh.ch/
2. http://rpg.ifi.uzh.ch/davis_data.html
3. https://github.com/uzh-rpg/event-base...

For an event camera simulator: http://rpg.ifi.uzh.ch/esim

For a survey paper on event cameras, see here:
https://rpg.ifi.uzh.ch/docs/EventVisi...

Other resources on event cameras (publications, software, drivers, where to buy, etc.):
https://github.com/uzh-rpg/event-base...

Affiliation:
B. Forrai, T. Miki and M. Hutter are with the Robotic Systems Lab, Department of Mechanical Engineering, ETH Zurich, Switzerland.
D. Gehrig and D. Scaramuzza are with the Robotics and Perception Group, Dept. of Informatics, University of Zurich, and Dept. of Neuroinformatics, University of Zurich and ETH Zurich, Switzerland http://rpg.ifi.uzh.ch/