russtedrake PRO
Roboticist at MIT and TRI
Doing research in robotics
Russ Tedrake
Image credit: Boston Dynamics
Q: Autonomous or Driver-controlled?
DARPA Robotics Competition
2015
Robots are dancing and starting to do parkour, but...
what about loading the dishwasher?
The Machine Learning Revolution
(in robotics; in a few slides)
What's my rule?
What's my rule?
Input
ImageNet: 14 Million labeled images
Released in 2009
Transfer learning
Something we couldn't have expected...
(Pre-)Training on ImageNet makes it easier to "learn" to recognize other objects
A sample annotated image from the COCO dataset
"Self-supervised learning"
Example: Text completion
No extra "labeling" of the data required!
Now robots are starting to manipulate objects they've never seen before...
and the programmers never considered / tested!
More complex tasks...
Going out into the real world...
"some interesting failure cases"
We still need "common sense"
Put the plate in the dishwasher...
- and don't break the plate,
- throw any mugs out the kitchen window,
- ....
"Foundation models" in machine learning
Example: Image + Text
Humans have put lots of captioned images on the web
...
"Foundation models" in machine learning
Example: Dall-E 2
"A painting of a professor giving a talk at a robotics competition kickoff"
Input:
Output:
"Foundation models" in machine learning
Example: Dall-E 2
"An FTC competition robot with green plants growing from its metal body"
Input:
Output:
"Foundation models" in machine learning
Some of the big questions:
- Have humans put enough random information on the web for robots to learn what they need to know about operating the world?
- As we get more and more robots, how should they pool their experience?
- How do we balance robot learning / exploration with safety?
We need your ideas, better theories/algorithms/robots, even more scalable machine learning systems, ...
Do you love robotics?
What can you do right now?
- Programming => Software engineering
- Physics
- Math (it's extremely important!)
- Calculus
- Linear Algebra (the foundations of machine learning)
- Machine learning tutorials online are becoming very accessible
- FIRST robotics!
Work hard and good luck this season!
FTC Kickoff 2022
By russtedrake
FTC Kickoff 2022
Fall 2022 FTC Northeastern Region Kickoff (at NHS)
