Our speaker tonight is Dr Cynthia Breazeal and she’s visiting us from the Massachusetts Institute of Technology where she’s the Director of the Personal Robotics Group at the Media Lab. We’re very grateful for Cynthia travelling on a whistlestop tour. She arrived yesterday on a plane and she’s going out tomorrow. So she’s maybe a little bit tired and thank you very much. We’re delighted to have such an honourable speaker and we’re looking forward to the talk. Also this evening, our Master of Ceremonies here is Derek Mooney. And Derek will firstly introduce Cynthia and give us some more information on her profile and background. He’ll also be conducting a Q&A session which we’ll have after Cynthia’s presentation. Derek, as you know, is one of Ireland’s bestknown TV presenters and he’s a prolific broadcaster particularly in the area of natural history, and has worked on shows such as Habitats, The Nature Line, Nature Trails for RTE Radio 1 and he’s also involved in the very popular Winning Streak TV show. And maybe if we play our cards right he might actually give us a ticket on the show, a chance on the show. Any chance? I’ll keep my fingers crossed. So, thanks again very much. You’re all very welcome and I’ll introduce you to Derek.
Thanks very much indeed. Well, good evening everybody. My name is Derek Mooney, as Oonagh has said and I’m a broadcaster, producer and presenter with RTE Radio here in Dublin. For those of you not from Ireland, RTE is Ireland’s national broadcasting service. And I present a daily programme on Radio 1 called Mooney which is a magazine programme. It’s a mixed bag of everything and anything. Today we had the pleasure of having Cynthia on talking about robots and I have to say, by the time she left, I actually was convinced I was going to end up marrying a robot. I thought we’d civil marriages, what next? Because she managed to convince me that in time robots will have emotions and feelings and they’ll even have a sixth sense. You can make your own decisions on that. But to formally introduce Cynthia: Dr Cynthia Breazeal is an Associate Professor of Media Arts and Sciences at the Massachusetts Institute of Technology where she founded and directs the Personal Robots Group at the Media Lab. She’s a pioneer of social robotics and human/robot interaction. Her research programme focuses on developing personal robots that interact with humans in human-centric terms, work with humans as partners and learn from people via tutelage. She is author of the book Designing Sociable Robots. Tonight the talk is titled The Personal Side of Social Robots. So will you please welcome Dr Cynthia Breazeal.
Good evening, it’s a pleasure to be here and to participate in Science Week. Thank you for the introduction. So I am going to talk to you about social robots. And clearly a lot of the real-world applications for this work concerns bringing robots into the everyday lives of you, of real people, not just the specialists and so forth. But to set the stage, I’m often asked, you know, how did I get started in this field. And I think, as perhaps many of you who may have decided to pursue careers in science and engineering, for me it started when I was a 10yearold little girl and I saw Star Wars and I fell in love with these robots. And I think the thing that was so captivating to me about the droids, these two droids, was not that they were, of course only incredibly capable and intelligent in their interactions, and of course throughout the movie there are many instances of their getting their human counterparts out of various predicaments, but you know, these robots were fullfledged characters in their own right. They had personalities, they had emotions, they had a relationship with one another, they had relationships with the humans they interacted with. They were more like friends than just tools that people used and we cared about them and they cared about us. And I think as an impressionable 10-year-old kid I saw that. And in the sort of work that I do had the sort of indelible impression of how I aspire to what robots could be in the future, it was a sort of vision that was seared in my 10yearold consciousness that I carry with me today.
But then you might ask, so where are all the robot sidekicks today? You know, we all fell in love with these robot sidekicks and so forth. Where are they today? We’re entering a really exciting time in robotics in that, for a long time the people said the robots are coming, they’re coming and it was like ‘Where are they?’ They’ve been in factories and so forth, you’ve seen them, they go into the depths of the oceans and off to planets and exploration. And it’s like, in many ways I think the final frontier is your living room. And the reason why is, if you think about it, our society, our interaction, it’s much more complex, much richer, much more diverse than a robot that’s navigating, say, an inanimate world. And many of the research challenges that I’m trying to tackle and I’m going to share with you today, really has to do with bringing robots into the everyday human environments that we take for granted but are incredibly rich and incredibly complex and incredibly dynamic.
So this is just a Scientific American cover where Bill Gates even wrote an article on the dawn of robots, comparing the sort of nascent robotics, personal robotics industry to where the personal computer industry was in the 1970s. A lot of people anticipate that this is going to grow and become very big. And I think we’re at, again, this very significant time and I think we’re just at the threshold between the first personal robots we see today in terms of toys and vacuum cleaners and so forth to what they could be in the years to come.
So, robotics is a field. When you think about it, the 40- or 50year history, the work has focused on robots that are capable in their interactions with things. So it's objects that are governed by the laws of physics. And through that you see many applications and a lot of research efforts are doing just that. So I mentioned planetary exploration really to navigate these sort of inanimate worlds, manufacturing where you, again using tools, that sort of repetitive task again and again. And now you are starting to use highly dexterous robots. But fundamentally this is about robots interacting with things.
And of course the new challenge is what robotics will be about if you talk about personal robots, it's interacting with people to do things. And this is a fundamental difference that has questions raised because it’s really about trying to understand, building robots that can interact with entities that have minds, namely you or animals and the fact that our behaviour is governed by having a mind means that there’s a whole different set of ways of thinking and understanding that these robots are going to need in order to really interact with us in the rich way and a capable way that is going to be required for their acceptance and functionality.
So today if you look at top research labs such as NASA, Johnson Space Center, you see examples here of, this is Robonaut. Robonaut is a humanoid robot who’s designed to work as a team mate for astronauts. And the idea here is that the robot can use the same tools as the human astronauts do and fully suited so you don’t have to retro-fit space shuttles and so forth in order to have Robonaut interacting. Now Robonaut, of course, doesn’t need oxygen, can operate on the outside of the space shuttle space station for a much longer period of time. So the question here is in this highlytrained sort of situation the robot is actually interacting with the person much more like a partner or a team-mate rather than a tool that’s operated.
Then of course, I don’t know how many of you have seen this advertisement, this is obviously from Honda, portraying the sort of vision of the robot in the home and I get a kick out of this picture because first of all it’s a family portrait basically and the dog Fido’s been kicked to the kerb. He’s still part of the family but there’s kind of some status here that suggests that while Asimo is here with the family, you know, poor Fido is down here. You know, but clearly these images are starting to come to us not only through science fiction but from advertising from real companies. So again, this is a very exciting time.
So a lot of this work, then, is just informed and just appreciates the fact that we as humans are profoundly social. And when we bring these robots into the environment we find that our ways of trying to think about them, understand them, is also profoundly social. So even in the case of something like the Roomba which is just a sort of disc that vacuums your floors, it sends off these empathy cues. And this is in 1946: there was a set of psychological experiments showing that even simple moving shapes on the screen, if you showed these to people and asked them to describe what’s going on they don’t describe in terms of vectors and spatial relationship. They talk about the mental states of these shapes, you know, that the little triangle is scared of the big triangle and is defending the circle and they’re escaping, so again this is just appreciating that social intelligence is just a fundamental way we have of understanding and thinking about the world and when entities, even as abstract as these shapes, push those buttons, we can’t help but unleash this way of thinking about it.
So perhaps it’s not surprising that people anthropomorphise even things like the Roomba so they make these sort of tea cosies with eyes, and kind of, you know, anthropomorphising them. I like this little, I don’t know who did this but there’s, ‘All work and no play makes Roomba a dull toy’. The sort of playfulness that I think is another aspect of the appeal of these kinds of systems is that it’s fun, it’s whimsical, it’s playful, and my hope is that these robots are not just going to be these drudging sorts of things that are useful but that we really enjoy them. They put a smile on our faces; well I think that’s something we don’t want to neglect. So the implication there is perhaps personal robots are going to also have to be profoundly social in a way that they have also the social intelligence; this way of thinking and understanding us as part of a social world.
So I’m going to present a variety of research that we’ve been doing and try to crack that nut. And again, these are fundamentally new questions in robotics, questions that we’ve never posed before because we haven’t really thought seriously about robots in the context of everyday people. So, I’m going to be talking today about robots that are really designed not only to recognise patterns of activities and actions and what you’re doing but to actually try to relate to you. And I’m going to talk about that in more concrete but actually try to coordinate their internal states which you might think of akin to you know, things like beliefs, intent desires, your sort of mental psychological states. And coordinate those states with your mental states in order to work as partners to do things together, to do joint actions together. And in doing that, you know, in psychology this term is called theory of mind or sometimes it’s called mind reading. It’s not about ESP, it’s really about the way that as we watch each other move around we don’t have to say a word but we understand the internal states, the motivations and goals that are going through your head in order to allow us to predict and understand your actions.
And I think a challenge of social robots is to essentially give them, endow them the sort of theory of other mind competence. And in humans this social cognitive ability underlies our whole swathe of social behaviour from communication to many forms of social learning such as tutelage to collaborative team work. Our ability just to coordinate ourselves in partnership, this social intelligence is fundamental.
So to highlight a variety of aspects of this work I’m going to talk about four different themes. The first again is this idea of social intelligence. The second one, I’m going to talk a little bit about these robots; in this case the Leonardo robot. It’s able to learn from human tutelage, to learn from natural human interaction in a way that people are naturally inclined to teach. So rather than expecting robots to learn new interfaces and new ways of communicating and interacting, how do you build robots that just understand our behaviour, that we can just be us and they can learn from that? Then I’m going to start talking a little bit about the actual social affordances of robots. So you talk about, say, sociallyinfluencing behaviour. We know from human social psychology these non-verbal cues like interpersonal distance, eye contact, facial expressions, gestures, postural mirroring all have a very important and often subconscious effect on how you perceive other people in terms of how trustworthy you see them, how persuasive, how much you like them. And it turns out that if robots exhibit these same behavioural patterns, you make the same kinds of inferences. So a robot, simply by the way it makes eye contact with you, can impact how trustworthy you perceive it to be. And that has interesting implications and possible applications that we might have not imagined 10 years ago. And then I’m going to end, pushing on that theme of communication and thinking about robots as this embodied systems that share these novelties. Maybe one killer application of robotics in the future is just a new kind of communication medium in its own right to facilitate human-human communication cost distance, but embodied into a physical artefact rather than just a cell phone or Skype or so forth. So just kind of taking all of that to the next level.
So the first thing I’m going to highlight is the social intelligence, the social way of thinking and understanding. And a lot of this work – this is my middle child, Nathan, he was just a baby – much of this work was in fact heavily inspired and guided by insights from developmental psychology. And if you think about people, we are tremendously socially complex and sophisticated. It takes us years to develop that competence. The thought of being able just to program decisive skills and understanding in a machine is daunting, if not impossible. So early on I thought that maybe one way to grapple with this question is to look at developmental psychology, social development and how the robots acquire these abilities and these ways of understanding through interacting with people. So the challenge, in some sense, is to build a sort of robotlike infant where people played the role of care-givers that try to bootstrap that kind of intelligence. So you’re going to see a variety of works that look at a diverse range of literature on developmental psychology and try to glean lessons and principles that can be used to try and bootstrap the social intelligence of robots.
And one of the key insights if you look to developmental psychology is just the appreciation of, you know, infants. They’re human, they’re anthropomorphic. So being in a body with a mind, in a world of like bodies with like minds yields a lot of multi-modal associations. Between the self and appearance of other agents, between the behaviours and events in time of the self and the behaviours in other agents, and based on those correlations, your ability to start inferring the internal workings of the states of mind of the self and the other agents. So basically using yourself as a simulator, to be able to take the other’s perspective as a way or a mechanism or strategy we have of understanding others. And how can you put that way of thinking and understanding into a robot?
So the first story I want to tell looks at one of the earliest forms of social learning and interaction that human infants exhibit which is imitation, in particular early facial imitation. And scientifically, this is a really intriguing question because, you know, infants, as much as just hours, even minutes, old can imitate in simple ways the expressions of adults, so how can you do this? Because you think about the problem: you're a newborn infant, you can see something like somebody stick out their tongue and a sort of like facial gesture, facial expression. And that’s a visual stimulus but then you do a motor response. So it’s a crossmodal association. So the question is how do they do that, because they haven’t seen mirrors on themselves. You know, how do they make this crossmodal association?
And one of the influential thinkers, developmental psychologists in this area, is Andrew Meltzoff. He and Andrew Moore proposed what was called an A Model, active inter-modal mapping hypothesis, by which this is sort of the substrate that infants come into the world with that allow them to do this basic form of imitation. Of course if you look at imitation in infants it goes through a very rich, very sophisticated development trajectory in its own right. But looking just at facial imitation, the idea here is that infants are born into the world with a basic motor repertoire and they babble. They can basically kind of go through this little repertoire and they don’t necessarily have meaning behind it. They’re just exercising their motor system but once you put the infant in front of a parent or a caregiver or another person, we are prolific imitators of infants. We are prolific imitators of others that we are trying to form that connection to. So the idea is it’s not as if infants come into the world right off the bat necessarily being real sophisticated imitators. But it’s the fact that our parents imitate us, that we learn these correlations and learn, you know, maybe we start off with a rough mapping of eyes, kind of map eyes to eyes and kind of mouth to mouth. But through these interactions of having adults mimic us we gather these associations and can start learning them, mapping between 'Oh, if I move my mouth in this particular way it becomes more nuanced and subtle so that I can map that now.' And eventually I can flip the tables and do that imitation back at you.
So I’m going to show you a video of a computational model of that, that we did on… this is a simulation actually of just training of the mouth region of our Leonardo robot. So these are the basic repertoire, the four mouth poses. The idea here, again, is Leo’s playing the role of the infant. He’s motor babbling, he’s just going through the repertoire of facial movements of the mouth here. And the human is actually imitating that, you know, so the human’s playing the role of a parent. And then, based on that, we have a vision system that’s tracking the facial features of the person. The robot is sensing a contingency between which region the human is moving to, which region it’s moving to in order to basically gather this training set and samples of 'When I move my mouth in this way this is what I see.' And then it uses a neural network to basically train up what’s called an inter-modal mapping. So this thing here is basically what Meltzoff proposes, it’s called an inter-modal mapping that goes from the visual coordinates of what you see on a face to a re-representation of that within the motor coordinates of the imitator.
So this representation is actually in the motor coordinates of the robot that then can it do a basic samples and match process. So based on a repertoire of basic postures you can find a way to blend those postures to approximate the inter-modal visitation. So it’s a search-to-match process basically in order to do the imitation. And this is what you would see in the observable world. This is essentially a depiction of the internal representation in the motor coordinates. And then based on that you can even have the training model imitate novel facial expressions.
So this is the idea here, is that the robot is basically modelling the sort of key principles and insights for maybe, I mean any theories that you know can be debated, but maybe is how infants actually do this. So from this simple interaction if you extrapolate that, you can think about inter-modal representations as being these sorts of mirror neuron micro-processes. The motor knowledge of the robot in this case can be used not only to produce these actions but also to help it recognise these actions in others. So, and the way it learned these mappings is through imitation, so it’s learning this sort of body mapping. How does my body map onto your body? And if you have that ability to mimic, this now is a way of interacting and learning that can bootstrap now more sophisticated and other forms of social learning.
a. Description
So another example then could be something like social referencing. So social referencing, if you look at babies, it starts to peak right around sort of one year of age. So it relies on a number of precursors, the ability to share attention, the ability to do this sort of imitation. But social referencing, you do this as adults too. You encounter a sort of novel situation, you don’t really know, you know, is this thing good or bad, what should I, what should my attitude towards this thing be? And so you often look to your trusted friend or if you’re the case of a child you might look to your parent. And you look at their assessment and if they’re acting encouraging you might go, ‘Oh, this is okay’ and if they’re looking fearful then maybe you’ll back off. So in the classic set of psychological experiments this is called the – oops; did I just turn that off? I think I just turned off the projector.
b. Visual cliff test
Anyway so this is a visual cliff experiment. So the idea is for these like roughly oneyearold infants, they have this sort of chequered pen where there’s actually a drop off. But there’s a glass over it so if the infant actually walks across they’re not going to fall over. But they have the appearance then of a visual cliff. And this is at an age where infants, they’re not really quite sure, you know, what’s safe and what’s not safe. So as the infant approaches the visual cliff, the experiment is to have mom on the other side either saying, ‘Come on, come on; it’s okay.’ You know, to have the infant kind of learn, ‘Oh maybe this is an okay situation’ and to walk across or to go, ‘Don’t do it’ so the infant will back off. So that’s social referencing, that’s kind of the classic psychological experiment that people look at, for infants to see at what age infants are able to do this sort of learning.
Alright I’ll just keep talking. So, I’m going to wave my hands through this. So, we’ve talked about imitation, we’ve talked about this basic pathway by which, you know, the robot can play the role of the infant and do this motor babbling, you see those interactions, you imitate it. That’s the way in which the robot starts to learn the mapping and which it can actually mimic you. So that’s kind of the first step of the process.
a. Psychological explanation
Now it turns out that in other psychological experiments, there’s a lot of dual loops between motor response, motor emotional response and internal affective states. So even if you look at, say, college students, it turns out that even just the act of putting your face in a particular configuration like a smile or a frown can induce the corresponding internal affect. So if you smile you tend to feel more positive; you frown you tend to be more negative.
So one mechanism potentially by which infants may learn how to do the social referencing is, given this imitative interaction, ‘I see you smile, I mimic that’, that can perhaps induce a positive affective state within the infant. So then it can start to learn correlations between, ‘I see your smile and I feel more positive.’ And then to learn the loop between ‘now I don’t have to actually explicitly have to do the facial expression. Now I just see you and that induces an affective state within myself’. So this is a strategy that Leonardo has been modelling, a sort of empathetic experiential way of trying to coordinate these internal affective states with that of the person that it’s watching. And then, once you can do that, again this is the point, the direct connection between when you see the facial expression and you respond to it.
b. Demonstration
So I’m going to show a video then of the social referencing interaction with Leonardo. Leonardo now has the fur on so we can show the robot. And I don’t know if you guys know the Sesame Street characters. But this is a novel thing; Leo’s never seen it. It’s clear that Leo has some basic speech recognition and this is basically to show that he can see Big Bird; he can associate the name Big Bird with it. Now, Mattie is giving you an attitude. He’s acting, he’s speaking in a very positive tone of voice, his facial expressions are very positive and Leo is doing this sort of empathetic mechanism by which he’s inducing a positive affective state within himself. And sharing attention with Mattie so understanding that his reaction is about this thing and starting to form an affective tag, essentially a positive effect with this novel stimulus called Big Bird that’s this yellow thing. And now he’s picked up in that and now he’s like, it’s like, I want the thing so when he reaches for it he’s basically saying ‘I like it, give it to me, I want it.’ So, you know, basic emotional response, give the good, avoid the bad. So he’s like, ‘Good, I want it, I want it, I want it.’ He’s like gesturing; ‘I want it, give it to me.’ Okay, so that’s positive.
And now of course you can imagine doing the other side as well. So Cookie Monster, it’s a blue thing. So again, this is just verifying that he can see Cookie Monster, he knows the label, and he knows what it is. So Cookie Monster, none of that, none of that. So, now you can take this away, you can show that these are actually memories now and that when you bring these things back he still has the affective reaction to them. So this is a form of, sort of affective, a simple form of affective memory. So ‘I want the yellow thing, I want it, I want it, I want it’ and you bring back the Cookie Monster thing and he’s like, ‘Uh, no. I don’t want that thing; take it away.’ So again, a very early, very important form of social learning that we take for granted but is really critical especially if you can imagine infants and the complexity of the human environment which has so many things that you don’t know if it’s safe or dangerous. This is just a fundamental way of learning about the world with and from others.
a. Scientific explanation
So if you keep pushing this idea now, we’ve talked about the sort of empathetic or experiential process by which the robot’s trying to coordinate its internal states with you. You can push that beyond effect to look at other cognitive states such as belief states or goal states. So again, this is inspired by psychology which basically says, there’s a theory called simulation theory, which says one way, one strategy that we have of understanding these other mental states of others is that we basically use ourselves as a simulator for them, saying, ‘If I were perceiving the world from your vantage point, if I were doing those actions in that context, what would I be, what would my goals be, what would my motives be?’ And that’s a starting point for inferring that that might be what the other person might be motivated, what their goals might be.
So it’s the notion of this dual use again, of our cognitive system, not only to generate our own behaviour but then to simulate and infer the behaviour of others, the internal states of others. And Andrew Meltzoff hypothesises that we learn to simulate others through imitation. So this is kind of this evolving story. So if you, you know, there’s a very rich literature now in neuroscience especially, you know, social neuroscience, looking at fRMI scans of human brains where you have people do tasks such as read a story where they’re asked to take their own perspective in this story or the perspective of another and finding that very similar brain regions are active when you’re taking your own perspective versus when you’re asked to take the perspective of another person.
b. Demonstration
So again, it’s an idea of re-use and sharing of representations. So I’m going to show another video of Leo doing this sort of process in a collaborative task. So those markers on the people’s heads are basically a way that Leo has of tracking head pose and hand positions, the state of the boxes and the state of the food items, the chips and cookies. So this is just talking about the cognitive architecture. You can imagine, you know, human brains have all these systems that are interacting with one another and we’re trying to model that, essentially, within the robot. So Leo is using his light system to trap all of these moving entities and is essentially trying to model those entities, belief states, in terms of who’s seeing what, who’s doing what, what are the state of the objects, where are the locations? So this are the belief states.
Now this is a manipulation based on a classical psychological task called the false belief task, that’s used to probe children’s developing theory of other minds. Which is when one person’s gone, there’s a trick that’s played on them which is to switch the objects. So Jesse has seen something, Jesse’s the one in the red, that Matt is not privy to. So Jesse’s beliefs should be different to Matt’s beliefs and Leo has seen the trick so Leo also should be privy to this information. So he’s going to seal the boxes, so Leo is going to be helpful. You’re going to come back and you’re going to want either chips or cookies and Leo has to infer from your actions, and your belief states, what the desired food item is. And then Leo has a little control box item over here that he can operate to open one or two food boxes here to help you get what you, he’s locked the boxes so it’s hard to get what you want.
So this is a collaborative task. So Matt comes back again and of course, he didn’t see what happened. He’s going for this box but we all know, except for Matt, that the wrong thing is in there. So this is called medi-cognition, this is thinking about thinking. Now, even though Matt has false beliefs and invalid plans so to speak, Leo understands what his true intention is and is going to operate this control panel to get Matt what he actually wants even though he’s going for the completely wrong box. So there’s the chips. Alright so now Jesse comes back and of course he does the exact same actions but he has different belief states and Leo understands that. And yay Leo, okay. So that’s all I want to talk about now about social intelligence. Of course you can imagine extending those ideas and pushing them in more complicated situations and other kind of mental states.
a. Setting up the experiments
But now I want to look at a different question which is the tutelage and learning question.
next »
« Back to Lecture