[data science] CVPR 2022: Robot Learning Tutorial (Vision, Text)

[Undiscussed Horrific Abuse, One Victim of Many]

Pick-and-place robotics are the primary thing missing for makerspaces
to be able to provide systems that automatically produce prototype
circuit boards.

[for me and likely others on this list, building the robotics and
establishing with the makerspaces is the hard part there; this
tutorial is on the software end]

The simple tutorial moves blocks according to human language.

https://sites.google.com/view/cvpr2022-robot-learning

Conference on Computer Vision and Pattern Recognition 2022 Tutorial on
Vision-based Robot Learning

When: June 20th, 9am to 12pm.
Where: Both in-person and online.

From:
- FlingBot
  - site: https://flingbot.cs.columbia.edu/
  - paper: https://arxiv.org/abs/2105.03655
  - code: https://github.com/columbia-ai-robotics/flingbot
- Implicit Behavioral Cloning
  - site: https://implicitbc.github.io/
  - paper: https://arxiv.org/abs/2109.00137
  - code: https://github.com/google-research/ibc
- NeRF-Supervision
  - site: http://yenchenlin.me/nerf-supervision/
  - paper: https://arxiv.org/abs/2203.01913
  - code: https://github.com/yenchenlin/nerf-supervision-public

Objective

(1) Introduce research on Computer Vision for Robotics.

(2) Provide hands-on experience for participants to directly run a
simulated robotics environment via Colab notebook in a browser.

Program / schedule
(times are in New Orleans time -- CDT, GMT /UTC - 06:00)

9:00		Introduction

9:15 		Interactive Tutorial (with colab walkthrough!) -- Learning
Visuomotor Policies Together with Language and Visual-Language Models

Part 1: Introduction to models, environments, data, and begin training.

10:00		Coffee Break

10:15  		Talk: "All the Good Stuff" -- A Toolkit for Vision-Based Robot Learning

10:45		Talk: "Memoirs from a Vision Researcher" -- What I Wish I Knew,
Starting to Work on Robotics

Self-supervised learning and reinforcement Learning

11:15 		Talk: "We Don't Even Need to Train Anymore?" -- Zero-Shotting Robotics

11:45		Interactive Tutorial (with colab walkthrough!) -- Learning
Visuomotor Policies Together with Language and Visual-Language Models

Part 2: Deploy policies, interactive, Q&A

12:15		End!


Animated GIFs

Example Colab Tutorial Walkthrough: Interactive Visuomotor Policies
with Language Models and Visual-Language Models

https://lh5.googleusercontent.com/UTKjqbzrZ8Oj_iMaKEcFahfdLeKgHNMi7nXc-gLAZs5LrWqeBN-maRP1CA5B6V9tORaTZJs-NmfI068wjnpttuxg2qRVeo03e9GGDaaAbyAG2oit2IkOAnwuWY6-N0zSAw=w1280

FlingBot

https://lh6.googleusercontent.com/S3PSGoE-xNaeTkYqaMVNUREM3Zf3yz_ZweTMbA40ZnrEU7XvEKpb_KIb7EJTI7_j9tuWf3YBoER-38Peq4oFBfEkGCpztnx2UQvjWHJgFINnT2MEjLrLgg7YdGzuDJzojw=w1280

Implicit Behavioral Cloning

https://lh5.googleusercontent.com/oBhhKBJ5fgRscdr2BRZwHix4YkOGEizkm11Is6QGyCGsf8n6of87xR62PMHmsWC5cVbY85bG8HPvvvgzpasDjuCDhSDFPHDpuJX84oIFPWufOvVdeWEgxnGIFYkLXOv5Kg=w1280

NeRF-Supervision

https://lh6.googleusercontent.com/x5KGeK_3CH5jLlXewY9IGT_8DYrRCk1NxWPjTGGZeoy6ru5hAbtTCq6lnY4T61pMXIQ3YIel3iZEvjYC91bHxaeY_UadYfjSSQqWwCs_TKEKUpLIN1DEo3sWLp3V-c4J6w=w1280