DARPA Robotics Competition

2015

Robots are dancing and starting to do parkour, but...

what about loading the dishwasher?

The Machine Learning Revolution

(in robotics; in a few slides)

What's my rule?

What's my rule?

Input

ImageNet: 14 Million labeled images

Transfer learning

Something we couldn't have expected...

(Pre-)Training on ImageNet makes it easier to "learn" to recognize other objects

A sample annotated image from the COCO dataset

"Self-supervised learning"

Now robots are starting to manipulate objects they've never seen before...

and the programmers never considered / tested!

More complex tasks...

Going out into the real world...

We still need "common sense"

"Foundation models" in machine learning

"Foundation models" in machine learning

Example: Dall-E 2

"A painting of a professor giving a talk at a robotics competition kickoff"

Input:

Output:

"Foundation models" in machine learning

Example: Dall-E 2

"An FTC competition robot with green plants growing from its metal body"

Input:

Output:

"Foundation models" in machine learning

Some of the big questions: 

• Have humans put enough random information on the web for robots to learn what they need to know about operating the world?
• As we get more and more robots, how should they pool their experience?
• How do we balance robot learning / exploration with safety?

Do you love robotics?

What can you do right now?

• Programming => Software engineering
• Physics
• Math (it's extremely important!)
  • Calculus
  • Linear Algebra (the foundations of machine learning)
• Machine learning tutorials online are becoming very accessible
• FIRST robotics!