Engineering:Leonardo (robot)

Leonardo's body by Stan Winston Studios

Leonardo is a 2.5 foot social robot, the first created by the Personal Robots Group of the Massachusetts Institute of Technology. Its development is credited to Cynthia Breazeal. The body is by Stan Winston Studios, leaders in animatronics. Its body was completed in 2002. It was the most complex robot the studio had ever attempted as of 2001. Other contributors to the project include NevenVision, Inc., Toyota, NASA's Lyndon B. Johnson Space Center, and the Navy Research Lab. It was created to facilitate the study of human–robot interaction and collaboration. A DARPA Mobile Autonomous Robot Software (MARS) grant, Office of Naval Research Young Investigators Program grant, Digital Life, and Things that Think consortia have partially funded the project. The MIT Media Lab Robotic Life Group, who also studied Robonaut 1, set out to create a more sophisticated social-robot in Leonardo. They gave Leonardo a different visual tracking system and programs based on infant psychology that they hope will make for better human-robot collaboration. One of the goals of the project was to make it possible for untrained humans to interact with and teach the robot much more quickly with fewer repetitions. Leonardo was awarded a spot in Wired Magazine’s 50 Best Robots Ever list in 2006.

Construction

There are approximately sixty motors in the small space of the robot body that make the expressive movement of the robot possible. The Personal Robot Group developed the motor control systems (with both 8-axis and 16-axis control packages) that they've used for Leonardo. Leonardo does not resemble any real creature, but instead has the appearance of a fanciful being. Its face was designed to be expressive and communicative since it is a social robot. The fanciful, purposefully young look is supposed to encourage humans to interact with it in the same way they would with a child or pet.

A camera mounted in the robot's right eye captures faces. A facial feature tracker developed by the Neven Vision corporation isolates the faces from the captures. A buffer of up to 200 views of the face is used to create a model of the person whenever they introduce themself via speech. Additionally, Leonardo can track objects and faces visually using a collection of visual feature detectors that include color, skin tone, shape, and motion.

The group plans that Leonardo will have skin that can detect temperature, proximity, and pressure. To accomplish this, they are experimenting with force-sensing resistors and quantum tunnelling composites. The sensors are layered over with silicon like is used in makeup effects to maintain the aesthetics of the robot.

Purpose

The goal of creating Leonardo was to make a social robot. Its motors, sensors, and cameras allow it to mimic human expression, interact with limited objects, and track objects. This helps humans react to the robot in a more familiar way. Through this reaction, humans can engage the robot in more naturally social ways. Leonardo's programming blends with psychological theory so that he learns more naturally, interacts more naturally, and collaborates more naturally with humans.

Learning

Leonardo learns through spatial scaffolding. One of the ways a teacher teaches is by positioning objects near to the student that they expect the student to use. This same technique, spatial scaffolding, can be used with Leonardo, who is taught to build a sailboat from virtual blocks, using only the red and blue blocks. Whenever it tries to use a green block, the teacher pulls the “forbidden” color away and moves the red and blue blocks into the robot's space. Leonardo learns, in this way, to build the boat using red and blue blocks only.

Leonardo can also track what a human is looking at. This allows the robot to interact with a human and objects in the environment. Naturally, humans will follow a pointing gesture and/or gaze and understand that what is being pointed at or looked at is the object the other human is concerned with and about to discuss or do something with. The Personal Robots Group has used Leonardo's tracking ability and programmed the robot so it can act human-like, bringing its gaze to an object the human is paying attention to. Matching the human's gaze is one way Leonardo seems to exhibit more natural behavior. Sharing attention like this is one of the ways that allows the robot to learn from a human. The robot's expressions, being able to give feedback on its “understanding” is also vital.

Another way that Leo learns is by mimicry. The same way infants learn to understand and manipulate their world is helpful for the social robot. By mimicking human facial expressions and body movement, Leo can distinguish between self and other. This ability is important for humans in taking each other's perspectives, and it's the same for a social robot. Being able to understand that “others” don't have the same knowledge it has lets the robot view its environment more accurately and make better decisions based in its programming of what to do in a given situation. It also allows the robot to distinguish between a human's intentions and their actual actions, since humans are not exact. This would allow a human without special training to teach the robot.

Leonardo can explore on its own, in addition to being trained with a human, which saves time and is a key factor in the success of a personal robot. It must be able to learn quickly using the mechanisms humans already use (like spatial scaffolding, shared attention, mimicry, and perspective taking). It also cannot require an extensive amount of time. And finally, it should be a pleasure to interact with, which is why aesthetics and expression are so important. These are all important steps in bringing the robot into a home.

Interacting

Shared attention and perspective taking are two mechanisms Leonardo has access to that help it interact naturally with humans. Leonardo also can achieve something like empathy, however, by examining the data it gets from mimicking human facial expressions, body language, and speech. In a similar way, humans can understand what other humans might be feeling based on the same data, Leonardo has been programmed according to the rules of simulation theory, allowing it to render something like empathy. In these ways, social interaction with Leonardo seems more human-like, making it more likely humans will be able to work with the robot in a team.

Collaborating

Leonardo can work together with a human to solve a common problem as much as his body allows. He's more effective at working shoulder-to-shoulder with a human because of the theory of mind work that is blended with his programming. In a task where one human wants cookies and another crackers from two locked locations and one of them has switched the locations, Leonardo can watch the first human trying to get to where he thinks the cookies are and open a box with cookies, helping him achieve his goal. All of Leonardo's social skills work together so it can work alongside humans. When a human asks it to do a task, it can indicate what it knows or doesn't know and what it can and cannot do. Communicating through expression and gesture and through perceiving expression, gesture, and speech, the robot is able to work as part of a team.

Contributors

• Professor Cynthia Breazeal
• Stan Winston
• Lindsay MacGowan (Artistic Lead)
• Richard Landon (Technical Lead)
• The Stan Winston Studios Team
  • Jon Dawe
  • Trevor Hensley
  • Matt Heimlich
  • Al Sousa
  • Kathy Macgowan
  • Michael Ornealez
  • Amy Whetsel
  • Joe Reader
  • Grady Holder
  • Rob Ramsdell
  • John Cherevka
  • Rodrick Khachatoorian
  • Kurt Herbel
  • Rich Haugen
  • Keith Marbory
  • Annabelle Troukens
• Fardad Faridi (Animator)
• Graduate Students
  • Matt Berlin
  • Andrew “Zoz” Brooks
  • Jesse Gray
  • Guy Hoffman
  • Jeff Lieberman
  • Andrea Lockerd Thomaz
  • Dan Stiehl
• Matt Hancher (Alumni)
• Hans Lee (Alumni)

See also

• Kismet (robot)
• Paro (robot)
• Robonaut
• Social robot
• Robotics

Bibliography

• A “Sensitive Skin” for Robotic Companions Featuring Temperature, Force, and Electric Field Sensors
• A “Somatic Alphabet” Approach to “Sensitive Skin”
• Action parsing and goal inference using self as simulator
• An Embodied Cognition Approach to Mindreading Skills for Socially Intelligent Robots
• An Embodied Computational Model of Social Referencing
• Applying a “Somatic Alphabet” Approach to Inferring Orientation, Motion, and Direction in Clusters of Force Sensing Resistors
• Collaboration in Human-Robot Teams
• Learning From and About Others: Towards Using Imitation to Bootstrap the Social Understanding of Others by Robots
• Learning from Human Teachers with Socially Guided Exploration
• Perspective Taking: An Organizing Principle for Learning in Human-Robot Interaction
• Robot Learning via Socially Guided Exploration
• Robot Science Meets Social Science: An Embodied Model of Social Referencing
• Robot’s Play: Interactive Games With Sociable Machines
• Spatial Scaffolding for Sociable Robot Learning
• Teaching and Working with Robots as a Collaboration
• The dynamic lift of developmental process
• Tutelage and Socially Guided Robot Learning
• Understanding the Embodied Teacher: Nonverbal Cues for Sociable Robot Learning
• Working Collaboratively with Humanoid Robots

Further reading

• Brooks, Andrew; Cynthia Breazeal (2006). "Working with Robots and Objects: Revisiting Deictic Reference for Achieving Spatial Common Ground". Human Robot Interaction. doi:10.1145/1121241.1121292. 
• Breazeal, Cynthia; Matt Berlin; Andrew Brooks; Jesse Gray; Andrea Thomaz (2006). "Using perspective taking to learn from ambiguous demonstrations". Robotics and Autonomous Systems 54 (5): 385–393. doi:10.1016/j.robot.2006.02.004. 
• Breazeal, Cynthia; Cory Kidd; Andrea Thomaz; Guy Hoffman; Matt Berlin. "Effects of Nonverbal Communication on Efficiency and Robustness in Human-Robot Teamwork". MIT Media Lab. 
• Stiehl, Walter Dan (2005). Sensitive Skins and Somatic Processing for Affective and Sociable Robots Based upon a Somatic Alphabet Approach. Massachusetts Institute of Technology. 

External links

• Personal Robots Group (Leonardo home page)
• TED Talk: Cynthia Breazeal
• The Real transformers (New York Times article)