On November 2, 2023, at an event hosted by the Swiss Association of Aeronautical Sciences at ETH, Prof. Davide Scaramuzza presented a comprehensive overview of our latest advancements in autonomous drone technology aimed at achieving human-level performance. Autonomous drones hold immense potential in fields like inspection, agriculture, and search-and-rescue operations, offering prospects for enhanced productivity. However, the current capabilities of these drones lag behind human pilots in terms of speed, adaptability, and reliability. The key question addressed in this talk is: How can autonomous drones outperform human pilots? The presentation delves into the complexities of autonomous, agile navigation in environments without GPS, highlighting the challenges in perception, learning, planning, and control in robotics research. Prof. Scaramuzza illustrates how the integration of deep learning techniques with fast-response sensors, such as event cameras, enables drones to attain remarkable levels of speed and resilience, relying exclusively on onboard computational resources. This advancement is a significant stride toward elevating the efficiency and safety of future autonomous aerial vehicles.

00:00 - Introduction
00:28 - Global commercial drone market
02:23 - Vision-based drone navigation
05:53 - Why we want agile drones
13:34 - Previous works on agile flight
15:14 - Challenges of vision-based agile flight
18:36 - Vision-based drone control architecture
21:00 - Key questions to incorporate deep learning into drone control
22:08 - Our vision-controlled drone: "SWIFT"
23:28 - Acrobatics and high-speed flight in the wild
28:09 - Drone racing
30:22 - Autonomous drone racing
31:28 - Formal definition of the autonomous drone-racing problem
33:29 - Autonomous drone racing control approaches: MPC, MPCC, RL
36:56 - Unmodeled effects in autonomous drone racing
37:26 - State estimation of SWIFT (Nature paper)
38:36 - How SWIFT learns to race with reinforcement learning (Nature and SciRob papers)
40:52 - SWIFT against the world champion drone-racing pilots (Nature paper)
44:00 - Simulation-to-reality gap in SWIFT
46:17 - Reinforcement learning vs. optimal control (SciRob paper)
47:38 - SWIFT vs. human flight behavior
53:09 - The future of drones: how birds fly and see
54:00 - Fast sensors: event cameras
59:05 - The evolution of event cameras in academia and industry
1:00:24 - Application 1: keeping drones flying when a motor fails
1:01:04 - Application 2: Dodging dynamic objects with drones
1:01:49 - Application 3: Catching dynamic objects with legged robots
1:02:14 - Combining event cameras with RGB cameras
1:02:59 - The future of event cameras for robotics: low-latency, high-bandwidth control
1:08:51 - Q&A
1:08:56 - User-conditioned neural network control policies

About the presenter:
Davide Scaramuzza is a Professor of Robotics and Perception at the University of Zurich. He did his Ph.D. at ETH Zurich, a postdoc at the University of Pennsylvania, and was a visiting professor at Stanford University. His research focuses on autonomous, agile microdrone navigation using standard and event-based cameras. He pioneered autonomous, vision-based navigation of drones, which inspired the navigation algorithm of the NASA Mars helicopter and many drone companies. He made major contributions to visual-inertial state estimation, vision-based agile navigation of microdrones, and low-latency, robust perception with event cameras, which were transferred to many products, from drones to automobiles, cameras, AR/VR headsets, and mobile devices. In 2022, his team demonstrated that an AI-controlled, vision-based drone could outperform the world champions of drone racing, a result that was published in Nature. He is a consultant for the United Nations on disaster response and disarmament. He won many awards, including an IEEE Technical Field Award, the IEEE Robotics and Automation Society Early Career Award, the European-Research-Council Consolidator Grant, a Google Research Award, two NASA TechBrief Awards, and many paper awards. In 2015, he co-founded Zurich-Eye, today Meta Zurich, which developed the world-leading virtual-reality headset Meta Quest. In 2020, he co-founded SUIND, which builds autonomous drones for precision agriculture. Many aspects of his research have been featured in the media, such as The New York Times, The Economist, and Forbes.

Affiliation:
Robotics and Perception Group, University of Zurich, Switzerland: https://rpg.ifi.uzh.ch/