Robotics Foundation Models and Physical AI Building Blocks

Darryl Palmer Control System Team Lead at Caterpillar and IEEE IES Industrial Activities Committee

Abstract:

This talk showcases the various phases of the Physical AI pipeline for training and testing affordable, open-source SO101 robotic arms using open-source robotic foundation models, such as Action Chunking Transformers, within the LeRobot Framework (from Hugging Face). Integrating generative AI models (like those powering ChatGPT) into physical systems allows these models to demonstrate their capabilities in the physical world, assisting with challenging routine tasks such as folding laundry. Robotic Foundation Models and affordable open-source robotic arms have democratized innovation in Physical AI, enabling individuals to explore intelligent automation without requiring large laboratories. The audience will learn the building blocks of intelligent automation to explore novel use cases.


Brief Bio:

Darryl was CTO co-founder of Janus Choice, a healthcare AI company founded in 2012. Janus Choice used AI to inform and engage patients to make informed decisions that reduced hospital readmissions and improved outcomes. The company was in multiple hospitals and had their first pilot in Cleveland Clinic. He brings experience in raising money and enterprise selling in healthcare. After Janus Choice Darryl was co-founder of an engineering services company and served as the architect and technical lead of IoT and software development. He has spent more than 30 years as a software engineer and is a Senior Member of IEEE. Darryl also served as Technology Leader and Lead Software Engineer for numerous companies. His previous employers and clients include NASA, GM, Rockwell Automation, and Danfoss. He currently is Control Systems Team Lead for Caterpillar.