Our speaker tonight is Dr Cynthia Breazeal and she’s visiting us from the Massachusetts Institute of Technology where she’s the Director of the Personal Robotics Group at the Media Lab. We’re very grateful for Cynthia travelling on a whistlestop tour. She arrived yesterday on a plane and she’s going out tomorrow. So she’s maybe a little bit tired and thank you very much. We’re delighted to have such an honourable speaker and we’re looking forward to the talk. Also this evening, our Master of Ceremonies here is Derek Mooney. And Derek will firstly introduce Cynthia and give us some more information on her profile and background. He’ll also be conducting a Q&A session which we’ll have after Cynthia’s presentation. Derek, as you know, is one of Ireland’s bestknown TV presenters and he’s a prolific broadcaster particularly in the area of natural history, and has worked on shows such as Habitats, The Nature Line, Nature Trails for RTE Radio 1 and he’s also involved in the very popular Winning Streak TV show. And maybe if we play our cards right he might actually give us a ticket on the show, a chance on the show. Any chance? I’ll keep my fingers crossed. So, thanks again very much. You’re all very welcome and I’ll introduce you to Derek.
Thanks very much indeed. Well, good evening everybody. My name is Derek Mooney, as Oonagh has said and I’m a broadcaster, producer and presenter with RTE Radio here in Dublin. For those of you not from Ireland, RTE is Ireland’s national broadcasting service. And I present a daily programme on Radio 1 called Mooney which is a magazine programme. It’s a mixed bag of everything and anything. Today we had the pleasure of having Cynthia on talking about robots and I have to say, by the time she left, I actually was convinced I was going to end up marrying a robot. I thought we’d civil marriages, what next? Because she managed to convince me that in time robots will have emotions and feelings and they’ll even have a sixth sense. You can make your own decisions on that. But to formally introduce Cynthia: Dr Cynthia Breazeal is an Associate Professor of Media Arts and Sciences at the Massachusetts Institute of Technology where she founded and directs the Personal Robots Group at the Media Lab. She’s a pioneer of social robotics and human/robot interaction. Her research programme focuses on developing personal robots that interact with humans in human-centric terms, work with humans as partners and learn from people via tutelage. She is author of the book Designing Sociable Robots. Tonight the talk is titled The Personal Side of Social Robots. So will you please welcome Dr Cynthia Breazeal.
Good evening, it’s a pleasure to be here and to participate in Science Week. Thank you for the introduction. So I am going to talk to you about social robots. And clearly a lot of the real-world applications for this work concerns bringing robots into the everyday lives of you, of real people, not just the specialists and so forth. But to set the stage, I’m often asked, you know, how did I get started in this field. And I think, as perhaps many of you who may have decided to pursue careers in science and engineering, for me it started when I was a 10yearold little girl and I saw Star Wars and I fell in love with these robots. And I think the thing that was so captivating to me about the droids, these two droids, was not that they were, of course only incredibly capable and intelligent in their interactions, and of course throughout the movie there are many instances of their getting their human counterparts out of various predicaments, but you know, these robots were fullfledged characters in their own right. They had personalities, they had emotions, they had a relationship with one another, they had relationships with the humans they interacted with. They were more like friends than just tools that people used and we cared about them and they cared about us. And I think as an impressionable 10-year-old kid I saw that. And in the sort of work that I do had the sort of indelible impression of how I aspire to what robots could be in the future, it was a sort of vision that was seared in my 10yearold consciousness that I carry with me today.
But then you might ask, so where are all the robot sidekicks today? You know, we all fell in love with these robot sidekicks and so forth. Where are they today? We’re entering a really exciting time in robotics in that, for a long time the people said the robots are coming, they’re coming and it was like ‘Where are they?’ They’ve been in factories and so forth, you’ve seen them, they go into the depths of the oceans and off to planets and exploration. And it’s like, in many ways I think the final frontier is your living room. And the reason why is, if you think about it, our society, our interaction, it’s much more complex, much richer, much more diverse than a robot that’s navigating, say, an inanimate world. And many of the research challenges that I’m trying to tackle and I’m going to share with you today, really has to do with bringing robots into the everyday human environments that we take for granted but are incredibly rich and incredibly complex and incredibly dynamic.
So this is just a Scientific American cover where Bill Gates even wrote an article on the dawn of robots, comparing the sort of nascent robotics, personal robotics industry to where the personal computer industry was in the 1970s. A lot of people anticipate that this is going to grow and become very big. And I think we’re at, again, this very significant time and I think we’re just at the threshold between the first personal robots we see today in terms of toys and vacuum cleaners and so forth to what they could be in the years to come.
So, robotics is a field. When you think about it, the 40- or 50year history, the work has focused on robots that are capable in their interactions with things. So it's objects that are governed by the laws of physics. And through that you see many applications and a lot of research efforts are doing just that. So I mentioned planetary exploration really to navigate these sort of inanimate worlds, manufacturing where you, again using tools, that sort of repetitive task again and again. And now you are starting to use highly dexterous robots. But fundamentally this is about robots interacting with things.
And of course the new challenge is what robotics will be about if you talk about personal robots, it's interacting with people to do things. And this is a fundamental difference that has questions raised because it’s really about trying to understand, building robots that can interact with entities that have minds, namely you or animals and the fact that our behaviour is governed by having a mind means that there’s a whole different set of ways of thinking and understanding that these robots are going to need in order to really interact with us in the rich way and a capable way that is going to be required for their acceptance and functionality.
So today if you look at top research labs such as NASA, Johnson Space Center, you see examples here of, this is Robonaut. Robonaut is a humanoid robot who’s designed to work as a team mate for astronauts. And the idea here is that the robot can use the same tools as the human astronauts do and fully suited so you don’t have to retro-fit space shuttles and so forth in order to have Robonaut interacting. Now Robonaut, of course, doesn’t need oxygen, can operate on the outside of the space shuttle space station for a much longer period of time. So the question here is in this highlytrained sort of situation the robot is actually interacting with the person much more like a partner or a team-mate rather than a tool that’s operated.
Then of course, I don’t know how many of you have seen this advertisement, this is obviously from Honda, portraying the sort of vision of the robot in the home and I get a kick out of this picture because first of all it’s a family portrait basically and the dog Fido’s been kicked to the kerb. He’s still part of the family but there’s kind of some status here that suggests that while Asimo is here with the family, you know, poor Fido is down here. You know, but clearly these images are starting to come to us not only through science fiction but from advertising from real companies. So again, this is a very exciting time.
So a lot of this work, then, is just informed and just appreciates the fact that we as humans are profoundly social. And when we bring these robots into the environment we find that our ways of trying to think about them, understand them, is also profoundly social. So even in the case of something like the Roomba which is just a sort of disc that vacuums your floors, it sends off these empathy cues. And this is in 1946: there was a set of psychological experiments showing that even simple moving shapes on the screen, if you showed these to people and asked them to describe what’s going on they don’t describe in terms of vectors and spatial relationship. They talk about the mental states of these shapes, you know, that the little triangle is scared of the big triangle and is defending the circle and they’re escaping, so again this is just appreciating that social intelligence is just a fundamental way we have of understanding and thinking about the world and when entities, even as abstract as these shapes, push those buttons, we can’t help but unleash this way of thinking about it.
So perhaps it’s not surprising that people anthropomorphise even things like the Roomba so they make these sort of tea cosies with eyes, and kind of, you know, anthropomorphising them. I like this little, I don’t know who did this but there’s, ‘All work and no play makes Roomba a dull toy’. The sort of playfulness that I think is another aspect of the appeal of these kinds of systems is that it’s fun, it’s whimsical, it’s playful, and my hope is that these robots are not just going to be these drudging sorts of things that are useful but that we really enjoy them. They put a smile on our faces; well I think that’s something we don’t want to neglect. So the implication there is perhaps personal robots are going to also have to be profoundly social in a way that they have also the social intelligence; this way of thinking and understanding us as part of a social world.
So I’m going to present a variety of research that we’ve been doing and try to crack that nut. And again, these are fundamentally new questions in robotics, questions that we’ve never posed before because we haven’t really thought seriously about robots in the context of everyday people. So, I’m going to be talking today about robots that are really designed not only to recognise patterns of activities and actions and what you’re doing but to actually try to relate to you. And I’m going to talk about that in more concrete but actually try to coordinate their internal states which you might think of akin to you know, things like beliefs, intent desires, your sort of mental psychological states. And coordinate those states with your mental states in order to work as partners to do things together, to do joint actions together. And in doing that, you know, in psychology this term is called theory of mind or sometimes it’s called mind reading. It’s not about ESP, it’s really about the way that as we watch each other move around we don’t have to say a word but we understand the internal states, the motivations and goals that are going through your head in order to allow us to predict and understand your actions.
And I think a challenge of social robots is to essentially give them, endow them the sort of theory of other mind competence. And in humans this social cognitive ability underlies our whole swathe of social behaviour from communication to many forms of social learning such as tutelage to collaborative team work. Our ability just to coordinate ourselves in partnership, this social intelligence is fundamental.
So to highlight a variety of aspects of this work I’m going to talk about four different themes. The first again is this idea of social intelligence. The second one, I’m going to talk a little bit about these robots; in this case the Leonardo robot. It’s able to learn from human tutelage, to learn from natural human interaction in a way that people are naturally inclined to teach. So rather than expecting robots to learn new interfaces and new ways of communicating and interacting, how do you build robots that just understand our behaviour, that we can just be us and they can learn from that? Then I’m going to start talking a little bit about the actual social affordances of robots. So you talk about, say, sociallyinfluencing behaviour. We know from human social psychology these non-verbal cues like interpersonal distance, eye contact, facial expressions, gestures, postural mirroring all have a very important and often subconscious effect on how you perceive other people in terms of how trustworthy you see them, how persuasive, how much you like them. And it turns out that if robots exhibit these same behavioural patterns, you make the same kinds of inferences. So a robot, simply by the way it makes eye contact with you, can impact how trustworthy you perceive it to be. And that has interesting implications and possible applications that we might have not imagined 10 years ago. And then I’m going to end, pushing on that theme of communication and thinking about robots as this embodied systems that share these novelties. Maybe one killer application of robotics in the future is just a new kind of communication medium in its own right to facilitate human-human communication cost distance, but embodied into a physical artefact rather than just a cell phone or Skype or so forth. So just kind of taking all of that to the next level.
So the first thing I’m going to highlight is the social intelligence, the social way of thinking and understanding. And a lot of this work – this is my middle child, Nathan, he was just a baby – much of this work was in fact heavily inspired and guided by insights from developmental psychology. And if you think about people, we are tremendously socially complex and sophisticated. It takes us years to develop that competence. The thought of being able just to program decisive skills and understanding in a machine is daunting, if not impossible. So early on I thought that maybe one way to grapple with this question is to look at developmental psychology, social development and how the robots acquire these abilities and these ways of understanding through interacting with people. So the challenge, in some sense, is to build a sort of robotlike infant where people played the role of care-givers that try to bootstrap that kind of intelligence. So you’re going to see a variety of works that look at a diverse range of literature on developmental psychology and try to glean lessons and principles that can be used to try and bootstrap the social intelligence of robots.
And one of the key insights if you look to developmental psychology is just the appreciation of, you know, infants. They’re human, they’re anthropomorphic. So being in a body with a mind, in a world of like bodies with like minds yields a lot of multi-modal associations. Between the self and appearance of other agents, between the behaviours and events in time of the self and the behaviours in other agents, and based on those correlations, your ability to start inferring the internal workings of the states of mind of the self and the other agents. So basically using yourself as a simulator, to be able to take the other’s perspective as a way or a mechanism or strategy we have of understanding others. And how can you put that way of thinking and understanding into a robot?
So the first story I want to tell looks at one of the earliest forms of social learning and interaction that human infants exhibit which is imitation, in particular early facial imitation. And scientifically, this is a really intriguing question because, you know, infants, as much as just hours, even minutes, old can imitate in simple ways the expressions of adults, so how can you do this? Because you think about the problem: you're a newborn infant, you can see something like somebody stick out their tongue and a sort of like facial gesture, facial expression. And that’s a visual stimulus but then you do a motor response. So it’s a crossmodal association. So the question is how do they do that, because they haven’t seen mirrors on themselves. You know, how do they make this crossmodal association?
And one of the influential thinkers, developmental psychologists in this area, is Andrew Meltzoff. He and Andrew Moore proposed what was called an A Model, active inter-modal mapping hypothesis, by which this is sort of the substrate that infants come into the world with that allow them to do this basic form of imitation. Of course if you look at imitation in infants it goes through a very rich, very sophisticated development trajectory in its own right. But looking just at facial imitation, the idea here is that infants are born into the world with a basic motor repertoire and they babble. They can basically kind of go through this little repertoire and they don’t necessarily have meaning behind it. They’re just exercising their motor system but once you put the infant in front of a parent or a caregiver or another person, we are prolific imitators of infants. We are prolific imitators of others that we are trying to form that connection to. So the idea is it’s not as if infants come into the world right off the bat necessarily being real sophisticated imitators. But it’s the fact that our parents imitate us, that we learn these correlations and learn, you know, maybe we start off with a rough mapping of eyes, kind of map eyes to eyes and kind of mouth to mouth. But through these interactions of having adults mimic us we gather these associations and can start learning them, mapping between 'Oh, if I move my mouth in this particular way it becomes more nuanced and subtle so that I can map that now.' And eventually I can flip the tables and do that imitation back at you.
So I’m going to show you a video of a computational model of that, that we did on… this is a simulation actually of just training of the mouth region of our Leonardo robot. So these are the basic repertoire, the four mouth poses. The idea here, again, is Leo’s playing the role of the infant. He’s motor babbling, he’s just going through the repertoire of facial movements of the mouth here. And the human is actually imitating that, you know, so the human’s playing the role of a parent. And then, based on that, we have a vision system that’s tracking the facial features of the person. The robot is sensing a contingency between which region the human is moving to, which region it’s moving to in order to basically gather this training set and samples of 'When I move my mouth in this way this is what I see.' And then it uses a neural network to basically train up what’s called an inter-modal mapping. So this thing here is basically what Meltzoff proposes, it’s called an inter-modal mapping that goes from the visual coordinates of what you see on a face to a re-representation of that within the motor coordinates of the imitator.
So this representation is actually in the motor coordinates of the robot that then can it do a basic samples and match process. So based on a repertoire of basic postures you can find a way to blend those postures to approximate the inter-modal visitation. So it’s a search-to-match process basically in order to do the imitation. And this is what you would see in the observable world. This is essentially a depiction of the internal representation in the motor coordinates. And then based on that you can even have the training model imitate novel facial expressions.
So this is the idea here, is that the robot is basically modelling the sort of key principles and insights for maybe, I mean any theories that you know can be debated, but maybe is how infants actually do this. So from this simple interaction if you extrapolate that, you can think about inter-modal representations as being these sorts of mirror neuron micro-processes. The motor knowledge of the robot in this case can be used not only to produce these actions but also to help it recognise these actions in others. So, and the way it learned these mappings is through imitation, so it’s learning this sort of body mapping. How does my body map onto your body? And if you have that ability to mimic, this now is a way of interacting and learning that can bootstrap now more sophisticated and other forms of social learning.
a. Description
So another example then could be something like social referencing. So social referencing, if you look at babies, it starts to peak right around sort of one year of age. So it relies on a number of precursors, the ability to share attention, the ability to do this sort of imitation. But social referencing, you do this as adults too. You encounter a sort of novel situation, you don’t really know, you know, is this thing good or bad, what should I, what should my attitude towards this thing be? And so you often look to your trusted friend or if you’re the case of a child you might look to your parent. And you look at their assessment and if they’re acting encouraging you might go, ‘Oh, this is okay’ and if they’re looking fearful then maybe you’ll back off. So in the classic set of psychological experiments this is called the – oops; did I just turn that off? I think I just turned off the projector.
b. Visual cliff test
Anyway so this is a visual cliff experiment. So the idea is for these like roughly oneyearold infants, they have this sort of chequered pen where there’s actually a drop off. But there’s a glass over it so if the infant actually walks across they’re not going to fall over. But they have the appearance then of a visual cliff. And this is at an age where infants, they’re not really quite sure, you know, what’s safe and what’s not safe. So as the infant approaches the visual cliff, the experiment is to have mom on the other side either saying, ‘Come on, come on; it’s okay.’ You know, to have the infant kind of learn, ‘Oh maybe this is an okay situation’ and to walk across or to go, ‘Don’t do it’ so the infant will back off. So that’s social referencing, that’s kind of the classic psychological experiment that people look at, for infants to see at what age infants are able to do this sort of learning.
Alright I’ll just keep talking. So, I’m going to wave my hands through this. So, we’ve talked about imitation, we’ve talked about this basic pathway by which, you know, the robot can play the role of the infant and do this motor babbling, you see those interactions, you imitate it. That’s the way in which the robot starts to learn the mapping and which it can actually mimic you. So that’s kind of the first step of the process.
a. Psychological explanation
Now it turns out that in other psychological experiments, there’s a lot of dual loops between motor response, motor emotional response and internal affective states. So even if you look at, say, college students, it turns out that even just the act of putting your face in a particular configuration like a smile or a frown can induce the corresponding internal affect. So if you smile you tend to feel more positive; you frown you tend to be more negative.
So one mechanism potentially by which infants may learn how to do the social referencing is, given this imitative interaction, ‘I see you smile, I mimic that’, that can perhaps induce a positive affective state within the infant. So then it can start to learn correlations between, ‘I see your smile and I feel more positive.’ And then to learn the loop between ‘now I don’t have to actually explicitly have to do the facial expression. Now I just see you and that induces an affective state within myself’. So this is a strategy that Leonardo has been modelling, a sort of empathetic experiential way of trying to coordinate these internal affective states with that of the person that it’s watching. And then, once you can do that, again this is the point, the direct connection between when you see the facial expression and you respond to it.
b. Demonstration
So I’m going to show a video then of the social referencing interaction with Leonardo. Leonardo now has the fur on so we can show the robot. And I don’t know if you guys know the Sesame Street characters. But this is a novel thing; Leo’s never seen it. It’s clear that Leo has some basic speech recognition and this is basically to show that he can see Big Bird; he can associate the name Big Bird with it. Now, Mattie is giving you an attitude. He’s acting, he’s speaking in a very positive tone of voice, his facial expressions are very positive and Leo is doing this sort of empathetic mechanism by which he’s inducing a positive affective state within himself. And sharing attention with Mattie so understanding that his reaction is about this thing and starting to form an affective tag, essentially a positive effect with this novel stimulus called Big Bird that’s this yellow thing. And now he’s picked up in that and now he’s like, it’s like, I want the thing so when he reaches for it he’s basically saying ‘I like it, give it to me, I want it.’ So, you know, basic emotional response, give the good, avoid the bad. So he’s like, ‘Good, I want it, I want it, I want it.’ He’s like gesturing; ‘I want it, give it to me.’ Okay, so that’s positive.
And now of course you can imagine doing the other side as well. So Cookie Monster, it’s a blue thing. So again, this is just verifying that he can see Cookie Monster, he knows the label, and he knows what it is. So Cookie Monster, none of that, none of that. So, now you can take this away, you can show that these are actually memories now and that when you bring these things back he still has the affective reaction to them. So this is a form of, sort of affective, a simple form of affective memory. So ‘I want the yellow thing, I want it, I want it, I want it’ and you bring back the Cookie Monster thing and he’s like, ‘Uh, no. I don’t want that thing; take it away.’ So again, a very early, very important form of social learning that we take for granted but is really critical especially if you can imagine infants and the complexity of the human environment which has so many things that you don’t know if it’s safe or dangerous. This is just a fundamental way of learning about the world with and from others.
a. Scientific explanation
So if you keep pushing this idea now, we’ve talked about the sort of empathetic or experiential process by which the robot’s trying to coordinate its internal states with you. You can push that beyond effect to look at other cognitive states such as belief states or goal states. So again, this is inspired by psychology which basically says, there’s a theory called simulation theory, which says one way, one strategy that we have of understanding these other mental states of others is that we basically use ourselves as a simulator for them, saying, ‘If I were perceiving the world from your vantage point, if I were doing those actions in that context, what would I be, what would my goals be, what would my motives be?’ And that’s a starting point for inferring that that might be what the other person might be motivated, what their goals might be.
So it’s the notion of this dual use again, of our cognitive system, not only to generate our own behaviour but then to simulate and infer the behaviour of others, the internal states of others. And Andrew Meltzoff hypothesises that we learn to simulate others through imitation. So this is kind of this evolving story. So if you, you know, there’s a very rich literature now in neuroscience especially, you know, social neuroscience, looking at fRMI scans of human brains where you have people do tasks such as read a story where they’re asked to take their own perspective in this story or the perspective of another and finding that very similar brain regions are active when you’re taking your own perspective versus when you’re asked to take the perspective of another person.
b. Demonstration
So again, it’s an idea of re-use and sharing of representations. So I’m going to show another video of Leo doing this sort of process in a collaborative task. So those markers on the people’s heads are basically a way that Leo has of tracking head pose and hand positions, the state of the boxes and the state of the food items, the chips and cookies. So this is just talking about the cognitive architecture. You can imagine, you know, human brains have all these systems that are interacting with one another and we’re trying to model that, essentially, within the robot. So Leo is using his light system to trap all of these moving entities and is essentially trying to model those entities, belief states, in terms of who’s seeing what, who’s doing what, what are the state of the objects, where are the locations? So this are the belief states.
Now this is a manipulation based on a classical psychological task called the false belief task, that’s used to probe children’s developing theory of other minds. Which is when one person’s gone, there’s a trick that’s played on them which is to switch the objects. So Jesse has seen something, Jesse’s the one in the red, that Matt is not privy to. So Jesse’s beliefs should be different to Matt’s beliefs and Leo has seen the trick so Leo also should be privy to this information. So he’s going to seal the boxes, so Leo is going to be helpful. You’re going to come back and you’re going to want either chips or cookies and Leo has to infer from your actions, and your belief states, what the desired food item is. And then Leo has a little control box item over here that he can operate to open one or two food boxes here to help you get what you, he’s locked the boxes so it’s hard to get what you want.
So this is a collaborative task. So Matt comes back again and of course, he didn’t see what happened. He’s going for this box but we all know, except for Matt, that the wrong thing is in there. So this is called medi-cognition, this is thinking about thinking. Now, even though Matt has false beliefs and invalid plans so to speak, Leo understands what his true intention is and is going to operate this control panel to get Matt what he actually wants even though he’s going for the completely wrong box. So there’s the chips. Alright so now Jesse comes back and of course he does the exact same actions but he has different belief states and Leo understands that. And yay Leo, okay. So that’s all I want to talk about now about social intelligence. Of course you can imagine extending those ideas and pushing them in more complicated situations and other kind of mental states.
a. Setting up the experiments
But now I want to look at a different question which is the tutelage and learning question.
We know of course, you know, as people interact, especially when you’re teaching kids, we do a lot to structure the learning experiences and learning environment of the learner because, again, a human environment is incredibly complicated, to help the learner learn. So how do we as the teachers kind of structure the interactions, use our body, use space, in order to help the learner learn what you want to teach them? So the question I want to look at here is how do we use space and our bodies to socially structure a learning episode for people? So we basically looked at two human participants studies. We looked at humans learning from humans as well as people trying to teach the robot. So in some sense, you know, you do an experiment and to verify you can actually, you know, remove a person and replace it with a robot and then see if you get the same kind of behaviour. So, the task that we looked at, it’s a collaborative task in that one person, the learner, knows information for the desired task which is you’re going to construct a figure using at least eight blocks and the teacher actually knows there’s a secret constraint. There’s a secret constraint that the learner has to learn from you in order to succeed at the task. So, we call it collaborative because each person comes to the interaction with partial knowledge and then of course the goal is something like this, where you want to create this figure. So the teacher in this case, the constraint is the figure must be constructed using all of these triangular blocks and none of the square blocks. So how do people do this? How do we teach each other the sort of constraint without using language? So the only constraint we told people is you can’t talk, because if you talk then it’s obvious, but we wanted to see how people use face and their bodies in order to communicate this.
b. Isolating the key cues
So, we didn’t know what cues would be relevant so we captured a lot of data. So we used that Vicon optical motion capture system we talked about that you saw how to track people’s head pose and hand trajectories. We built a special table called a light table that basically has a camera underneath that sees how people are moving the blocks dynamically in time. And then we also had an external video camera to capture more data. Because, again, we didn’t really know what cues were going to matter. We developed a lot of technologies to essentially capture all of this data, which is a mountain of data as you can imagine, and synchronise it in time and even label it and do some basic feature extraction to automate the analysis process to try to understand what are the patterns of activity that are really significant in doing this task. So here you’re seeing the Vicon system if people are moving their hands and gestures, external camera and this is the light table looking up to see who is moving what objects at what time. You can track colour and orientation, block type and so forth.
So, you know, if you look at the data, people do a lot of stuff, you know. So, again, human behaviour is really complicated. They’re doing all these things so there’s various, you know, hand and head gestures, pointing, tapping, nodding, shaking the head, facial expression, ways that you’re moving the blocks, you know, combinations of these cues, you know. So the question is, is there a sweet spot, is there a sub-set of these cues that ideally that could be very simple yet really prevalent and really reliable in helping the teacher and learner, helping the learner learn the right cue?
So, I’m going to let you, you know, think about this for a little bit because there’s a lot of things it could be. But it turns out that the key, there’s a couple, but one of the key signals that’s used is block movement towards and away from the learner. So this is colour coded data of all the block movements happening in a space in one set of interaction and what you see here is the teacher is on one side of the table and the learner is on this side of the table. Teachers tend to move the bad blocks, the irrelevant blocks away from the learner and they tend to push all the good blocks near the learner. And clearly learners are picking up on this because they tend to only use those good blocks. So here’s the very simple but very prevalent cues that we use to physically structure our learning environment for a learner.
c. Data
And it turns out here’s another graph showing that even the amount of movement indicates how good or bad, so the further you move it towards or away from the learner indicates like how completely irrelevant this thing is or how far you push it towards the learner signals how useful that block is to you. So it’s just another way of showing that data. It turns out that for this task, if you add up all of the translations that happen across the table well, the shape is the length of a football field. So people are moving these things very quickly all of the time. There’s a lot of activity going on. And of course you can imagine this is a challenging problem to build a robot that can actually learn from these interactions but if you’re armed with these cues, you can help a robot do these sorts of tasks by having it be savvy to these cues.
d. Puzzle block task
So this is a video that actually combines two learning tasks together. So the first task is, the robot again that has the puzzle blocks is actually learning how to operate this puzzle block in order to reveal two hidden shapes on this table. So I haven’t talked about this kind of learning and I’m just throwing it in here because it’s kind of cool. But, you know, so Leo, the self-motivated learner when you’re not there he explores on his own, he’s kind of internally driven to be curious and to master these things so he’s learning how to operate the box. But of course the point is if a person’s present, Leo’s exploration should be guidable so the person can help suggest actions for Leo to try, to try different combinations, to highlight significant cues like, you can see these lights are changing on the boxes so to signify important states. So Leo, through collaborating with the person, is starting to learn how to operate these two boxes to reveal a blue shape and a yellow shape that he’s actually going to need in the next task.
So, you know, we fast forward these videos because everybody wants robots to learn but learning videos are really boring to watch. So this is obviously kind of time lapse in order to give you a kind of sense of, now there’s practice and rehearsal, the robot’s learning sub-goals, hierarchies of tasks, how to master those higher order of tasks. There’s a lot of sort of task knowledge that’s being built up incrementally over time.
Okay, now we’re going to do the other interaction. So this is a secret constraint task where the person has a preference of wanting Leo to only use the blue and the yellow blocks. So this is a second kind of learning task for now. Leo, he knows the figure, the robot knows the figure to build which is a sailboat in this case. The person knows the secret constraint or the preference in this case and without talking, Jesse in this case, is trying to get the robot to learn how to do this task. So we have actually brought in human participants to teach the robot. We find that the robot can learn from people off the street which is the point; we want the robot to be able to learn from anyone. So when we use these demonstrations we have to use people in our group because we’re not allowed to show video from people who are human participants.
So, Leo’s starting to pick up on the idea that it’s the yellow and the blue blocks that matter. So now you see he’s starting to use the yellow and the blue blocks to build the sailboat figure and he realises he needs a shape and it’s not there and he’s just learned the skill for how to get it. So this is actually really cool because the robot is immediately applying what it learned in a different context in a new context in order to perform the task.
You don’t see robots doing this a whole lot. This is a big thing. And so he’s continuing to build a figure and then he realises that there’s a yellow shape that he needs and it’s not there but he knows how to get it because he just learned that skill from another interaction and he pulls that into the figure. So, yay Leo. So, I mean the reason why this video, I think, is really significant is it’s showing the robot’s integrating multiple forms of learning through multiple kinds of interactions to immediately apply that knowledge to solve a problem. And again, I mean you don’t see machines doing that a whole lot. So this is really, I think, a milestone in these kinds of systems.
a. Achieving Companionable Robots
Okay, so that’s a lot about Leo. I’m going to start talking about some other robots now. In particular I want to talk about again the social cues and how they can impact human perceptions along these sorts of social dimensions. So looking at social influence. So, you know, one challenge of robotics, personal robots in particular, is longterm interaction. All these studies I’ve shown you are very cool, you know, they’re based on these, sort of these classic psychological experiments. They take about five minutes to do. But robots, personal robots are going to be in your house for a long time, right? So this challenge is how do you build robots that can sustain an engaging, rewarding, interesting longterm interaction with you is a real challenge. And I’m sure many of you, you know, remember Clippy the Microsoft paperclip. You know, when this goes wrong it’s not a good thing, right? You don’t want Clippy to be your robot right? So, longterm interaction is a really important issue.
b. Weight Management Dimension
And the question that we asked in this work I’m going to talk about is, you know, can a robot help a person to achieve some song term behavioural change goal that has, say, positive, enduring outcomes along the health dimension? So, the domain we looked at was weight management. And weight management in the United States, it’s a huge issue. 65% of Americans are either overweight or obese. We know that a lot of chronic diseases later in life are tied to being obese when you’re younger. So in terms of burden on the health care system and your quality of life, managing your weight is really, really important.
This is a longterm endeavour where it’s been found that social support by your friends and your family is a very, you know, successful way of helping a person stick to a diet and exercise programme. So it’s not losing the weight, it’s keeping it off is the challenge. Social support is valuable and important. So the question is can you design a robot that can help you maintain a diet and exercise programme by building a longterm rapport with you?
So in this case, this is the Autumn robot. You know, we actually built a number of these robots and put them in people’s homes in the Boston area. They had to be really simple just because they had to actually work so much more simply than Leonardo robot. So, I mean, just, it has just a couple of cues. It can make eye contact with you, there’s a camera in the forehead and so it can maintain eye contact and can basically do sort of shared joint exercises, looking at you when it’s talking to you, looking at information on the screen when it’s talking to something on the screen. It uses speech synthesis to speak to you but there’s, it’s not doing speech recognition. All the sort of language interactions are based on text on the screen and again we do that for robustness issues.
a. Social network
But the idea here again is, robots and social robots are maybe an intriguing technology because not only can they be useful to you in helping you manage your network of technological devices like a Bluetooth scale or a pedometer to help you with just the logistics of keeping track of your weight in terms of how many calories you’re burning off or even perhaps how many calories you’re eating. But it could potentially also play a role in your social network. So a big theme in a lot of our work is, you know, a lot of old work in AI was about robots are going to replace us, you know. And this newer, more enlightened viewpoint is that robots are going to empower us and support our social networks. They have to be integrated into our existing human networks. So in the case of this robot, if it’s interacting with you everyday, like just maybe five minutes, that’s not someone you’re going to hire someone to do, just to do for five minutes. So this is something that makes sense to have some sort of robot do for you.
b. Tracking progress
But it can help you keep track of your progress and share that or have you choose to share that with your doctor and your trainers. So everyone has much more detail and much more accurate information on how you’re doing so they can provide you with better advice as well. So we want these robots to be empowering not only for you to achieve your goals but also to empower the other people who are helping you to achieve those goals as well.
So I’m going to show a quick video then of this robot, it’s called Autumn. Synthesised voice because it’s got to generate the speech on the fly. So you’ll notice, I mean a lot of the dialogue that the robot uses, it’s based on patient-therapist dialogue so there’s a lot of talk about, ‘Help us to help you achieve your goals.’ So there’s a lot of team building, social rapport building dialogue as well as these non-verbal cues. And there’s actually a lot of other cues, the robot actually has a model of the state of the relationship, so there’s a sort of initial phase where you’re getting to know one another and the explanations may be more detailed. As you interact with the robot longer, it understands that you already know all that stuff so the interactions become much shorter. So there’s a lot of modelling in the relationship and the knowledge that the robot is using to incorporate into its dialogue.
So, we did a study where we built, again, a number of these robots and we put them into people’s homes in the Boston area. People ranged in age from roughly 18 to 72 years of age, so a wide demographic of people, looking at a sixweek study. So we told them they had to interact with the robot mandatorily for four weeks and then they could have an optional two weeks if they chose to keep the robot. And that’s one way to assess engagement and liking is that if they choose to keep the robot an additional two weeks and they don’t have to, then clearly they want to keep interacting with it.
And we looked at three conditions. So the three conditions were the Autumn robot that you saw, a computer that ran the exact same software, meaning it gave you the exact same advice as the Autumn robot and had the same touch interface. So basically you’re just removing the social embodiment. The dialogue and the information is identical. And then just pen and paper logs because that’s typically what you’re given in a weightloss clinic. So these are the kind of control interventions that we looked at. And again, what we were trying to understand, the question was longterm interaction and also quality of the working alliance. So, is the embodiment or the social cues a contributing to a positive working alliance between the human and the robot because that would be indicative of longterm interaction and achieving your longterm weight loss goals.
Okay, so I’m going to let you think about that. Do you think it matters that it’s a robot? Because the computer is giving you the exact same advice, same quality advice. And, you know, the reason why I’m presenting this work is because the robot did a lot better, right? So, it turns out that if you just look at, you know, length of time interacting with the robot, people chose to interact with the robot almost twice as long as they did the computer. Same advice, same everything. There’s something about the embodiment and I think the social rapport essentially of the human and the robot that was built that was just different than the computer or of course, using the pen and paper logs. So people chose to stay and interact with the robot significantly longer.
The other thing is, if you look at sort of standard measures like questionnaires that are used to measure quality of working alliance between say patients and therapists you can apply the same questionnaire to assess the working alliance between the human and the robot. And again you see the robot is scoring much higher on the working alliance inventory. So it seems that people also perceive a much stronger working alliance with the robot even though the advice was identical.
Trust is really interesting. People trusted the robot more than the computer and it turns out they also saw the robot to be more credible. And then the emotional relationship between the robot and the other conditions was incredibly different. So, people would clothe the robot, they would name the robot, when we would pick up the robot they’d come out to the car and say goodbye to the robot. They did not do this with the computer. So the emotional engagement of people to this technology was markedly different.
So, you know, when I started this work with robotics, you know, there was a lot of animated agents, you know, screen agents and so on. People would always ask, ‘Why build a robot? Does it really matter?’ And if all that matters was seeing these visual cues and clearly there’s something going on with human psychology that it really matters that it’s in your world, it’s of your world, it’s in your space, it’s not just a sort of thing on a screen. In ways I think we still have to really understand it matters to people that, I think, this thing is physical. So there’s a lot more, I think, to be understood.
So, one other thing that robots can do that animated agents can’t do is touch via physical interaction. So what about the role of, say, of social touch in human-robot interaction? Now, we know, again, from the human-human literature, social touch such as the handshake can be seen as a warm and open gesture, it can contribute to persuasiveness and liking of people. But it’s complicated because depending on who’s shaking your hand and in what context it could have a negative result as well. If you’re a woman and you’re feeling threatened by this in some way, you’re going to have the exact opposite reaction.
And as you can imagine, there’s a lot of really complicated gender stuff between shaking someone’s hand if you’re same gender versus a different gender. So touch is very complicated in humanhuman interaction. Now, you can imagine, one of the intriguing things about robots is you can use them, in essence, like a controlled experimental tool to control these non-verbal interactions like eye contact, interpersonal distance, handshakes and so forth in a way that’s impossible to do with another person. So we know all of these cues matter, we know all of these cues shape our perception of trust and liking but we don’t know exactly in a quantitative way how and by how much. You can’t tell a person, disable your eyes now and interact with this person and then we’ll see how they perceive you in terms of how likable you are. Or disable your, you know, mouth so you can’t do that with people but you can do that with a robot.
a. Bringing the robot to the people
So, we did a study at the Boston Museum of Science. This is our latest robot, Nexi here which is a mobile humanoid robot interactive with over 300 people in the computer, Conners Place, which is kind of the Science/Technology area of the museum. And a lot of our work now is trying to get these robots out into the real world with real people because, as you can imagine, people interacting with a robot in a public space, I mean, people have a completely different mindset than if they’re going to your lab to interact with a robot at MIT versus bringing the robot to people’s turf, so to speak, and having the robot interact with them in their space. So it’s really important, I think, to get these robots out there in human environments to really kind of understand, you know, what the impact is on people.
b. Interaction with participants
So this is just some snapshots for computers, Conners Place. You know, we try to control the interaction as much as while there’s a lot of distraction, so human participants were recruited. They were brought into an enclosed area so there wouldn’t be a lot of visual distraction so they could kind of focus on the interaction with the robot. People would come in groups potentially or alone. You know, you might be with your friends or your family, so in that case we would choose one person to be the participant in the study and the other people were requested to kind of stand, you know, behind a certain line in the background. So people sometimes were alone, sometimes they weren’t.
c. Donation test
Often when you do human participant studies you have to compensate people for their time so we gave the people upfront $5 in $1 bills. And the reason why you did that is because one of the measures you want to look for in terms of persuasiveness, the role of say, touch and gender and persuasiveness was that the robot was going to ask them for a donation. They didn’t know that going in, that it was going to ask them for a donation. And people like their money; they don’t so give up their money. So, how much money they gave the robot might be an interesting measure in terms of, and whether they gave money, of the persuasiveness of the robot as you bury gender or interpersonal distance or touch or so forth.
So the interaction was basically, they should bring people into this space, you know, it’s a museum setting so the robot gives an educational message, it was a sort of mildly interactive, you know, the robot wasn’t, I’d say, a richlysophisticated activator like Leonardo. It was a simple interaction. At the end of the interaction the robot would thank you and shake your hand in the case of the touch condition say. And then it would ask you for the donation. And the question is, like how much money would people give and this is the donation box shown here. After the whole experiment they could go to another area and do a questionnaire and so forth. So that’s basically the study.
Protocol. Now, you know, again, we talked about gender as interesting implications for touch, so we had two gendered robots. Of course robots have no gender but you can manipulate that to some extent by just the voice that you give the robot. So, in the female robot condition [voice plays on video]. So this is a little interaction here. So this is the beginning of the interaction, the robot goes into, you know, a little more technical description and again, it’s an educational message. Now this is you know, the male condition here. So you can see the robot’s making eye contact here so it’s looking up higher here but it’s basically just changing the voice. So it’s exact same animations, exact same everything, you’re just changing the voice. So that’s how you control, in some sense, the gender, vary the gender of the robot.
a. Effect of handshake on donation
What we found were some interesting things. It turns out that being in a group or alone had a significant impact on how much money people gave the robot. So one thing that you see is, you know, in this situation where people didn’t, the robot didn’t shake the person’s hand there’s really not that much difference so to speak in how much money people gave. But if the robot shook your hand and you were with other people that saw you shake the robot’s hand you gave a lot more money. A lot more money. All you sales people, you get a lot more money. But if you were alone people tended to give less so that might say something about kind of social expectations or expectation violence. The robot’s handshake, frankly, wasn’t the most compelling handshake in the world. I bet if we had a better handshake that was warmer or whatever, I bet we could have pushed this up. But the intriguing thing here is the group dynamics had a real significant impact.
b. Effect of gender
Now it also turns out that gender was really interesting. So, if you didn’t have a handshake condition, and this was women interacting with the female robot or a male robot and men interacting with either the female robot or the male robot, you see a crossgender preference. And you see this in human-human interaction as well. Women prefer the male robot or found the male robot more persuasive and men found the female robot to be more persuasive. Okay and this is with no handshake. But if the robot shook your hand it flipped and women found the woman, the female robot to be more persuasive and men found the male robot more persuasive.
So what does this mean? Touch in robots is really complicated and there’s a lot to be understood here. And it’s typical because when you do a study like this it just turns out that nobody has done this exact study yet for humans to know what humans do. So the intriguing question here is, you know, one theme is that, you know, what happens when robots really do mirror what you see in human-human but the other question is what happens when they don’t? And what’s going on when they don’t? So, this, sort of begs the follow on study in terms of what would people do in a situation if it’s a human-human study? There hasn’t been one done yet and maybe we’ll do that.
But again, I think the bottom line here is that touch is complicated in human-human situations and it’s probably going to be interesting and complex in human-robot interactions.
So, the last thing I want to touch on very briefly, drawing on a lot of this work that we learned from the social influences, is just now thinking about robots presups themselves as a new communication media. So all the work about Leo was all about the intelligence of Leo and it’s autonomous and it’s learning and it’s doing all this stuff. But what if the robot really is just like a cell phone? So you are communicating to another person through the robot but in this very embodied, co-located, multi-modal kind of way and why is that interesting? Well we all know, you know, from cell phones and teleconferencing systems, you miss a lot of what goes on in facetoface interaction.
And your trust and liking and engagement and persuasiveness is often compromised if you interact with someone, especially for the first time, when you’re not facetoface versus when you make that first contact facetoface. So this is an example of posture mirroring and the role of touch and gesture. You lose a lot of that in our current communication technologies that we use to communicate with each other across distance, but with a robot you can bring a lot of this back.
So this is the Huggable, this is a robotic teddy bear that we’ve been developing at the Media Lab, looking at a robot as a distance learning technology for children. So, an educator, a parent, a grandparent can sort of jack into this robot à la, you know, Diamond Age ractor style to interact. So Dan is playing the role of a child. Imagine Dan is eight, you know. But playing the role of a child where you can read a book together, you could have tacto-playful interactions. I mean, basically robots in your world. And especially for very young children, that’s how they learn. I mean they learn by co-located, co-present, facetoface interactions that are very concrete in the world. So a robot can bring that to you.
So the Huggable, in many ways is a very sophisticated robot. This is basically the ractor idea here where this little video is giving you a flavour of the kinds of interfaces we’ve been developing in the interactions. So, you know, a lot of people have built like these robotic toys and they’re really boring. They just don’t have very interesting behaviour. The power of this idea, it could be Grandma on the other side and you have all of this shared experience and knowledge of Grandma that can be challenged through this cuddly little form factor. In the case of an educator, there’s been lots of studies showing a difference between if children are learning from an adult it might be intimidating or they might feel restricted in their exploration because they don’t want to look dumb. You know, versus peer-to-peer learning.
So this robot puts all of that human expertise in this cute, cuddly little form factor that could really, as you can imagine, change the social dynamic between a child and how they learn with the Huggable versus a child and how they learn with an adult. We could explore totally different learning paradigms. So it’s more of a peer-to-peer, maybe the child would be much more open to exploration, making mistakes and so forth, than they might with an adult. So it’s intriguing to think about, you know, in this case robots, it’s just a new kind of communication medium applied to a lot of different applications I think.
a. Roomba
So, I think then to recap I want to just talk about some various trends that are kind of alluded to in the talk. But what robots might be, what roles they might be in the future. And I think, you know, the first lesson that we’re already starting to see now is, I think robotics, like a lot of computer technologies, you can imagine that will be ubiquitous in the case that things that you interact with you might not necessarily think of as being a robot but it’s incorporating robotic technology. So the Roomba’s a key example of that. If you talk to Roomba owners they don’t think of this as being a robot, they just think of it as being a vacuum cleaner but an autonomous vacuum cleaner.
b. Autonomous Cars
Similarly when you think about cars in the United States, you know, we have these Defense Advanced Research Projects Agency (DARPA) grand challenge and DARPA urban challenge in these autonomous driving vehicles. Arguably cars are becoming more and more like robots everyday. They have a lot of computation, a lot of sensing. Things like automatic braking are happening kind of beyond what the human is doing. And now with autonomous vehicles, there’s a lot of major car manufacturers starting to see autonomous cars as the future of urban driving. And that could fundamentally change that relationship of those cars to the driver when the car is no longer the tool that you operate but is a partner that you interact with. A car that might understand your driving goals, your, what you’re trying to accomplish by being in the car or your preferences of music and so forth. So, it could it could transform our relationship that we have with this sort of robot that’s basically your car. So I think this is one trend that you’re going to continue to see.
c. iMAC Commercial
So this is just a funny movie of, 2004, you remember these iMAC commercials when the iMAC first came out? You know, maybe your computer will be a robot, you know.
So again, just to highlight, it doesn’t have to be anthropomorphic, there’s a lot of anthropomorphic robots. Does not have to be anthropomorphic to get this sort of social thing going and I think that’s important to remember.
d. InTouch help telepresence robot
The other, I think, application, you’re already starting to see it. This is the InTouch help Telepresence Robot which a doctor can visit a hospital remotely as a robot essentially, to visit patients and nurses and other doctors at a remote facility through this sort of robotic telepresence. In Japan, people like Hiroshi Ishiguro are taking this to an extreme where he’s created an android in his likeness, and he tells stories. So Hiroshi actually holds two appointments – he is a professor at the University of Osaka and he works at the Advanced Telecommunications Research Institute (ATR) which is a big kind of Japanese technology lab in Japan. And he talks about using; he calls it the Geminoid, to interact with his students while he’s still at ATR. So your advisor never leaves, he’s always there. You’re starting to see, you know, the makers of Roomba have now stuck a camera on the Roomba to look at sort of the remote surveillance so of course, telepresence.
e. Keepon
And even these little, this is Keepon. I love this robot. This is a robot that was developed in Japan by Hideki Kozima that’s being applied as a therapeutic tool in autism. So, for children that are on that autism spectrum having the children interact with this little cute, unintimidating thing in order to learn skills like joint attention and so forth. So, robots as a therapeutic tool to help people learn and inquire and improve their social communication skills I think is another really intriguing area.
And then, you know, a lot of my talk is really talking about this, about the partner robot. And I think of course, you know, this is further off. I think you’re already starting to see, you know, examples of course of the other two. You know, Pero in some sense is maybe the simplest kind of social robot technology. This is a seal robot developed in Japan that they’ve been deploying in elder care facilities. Showing that this sort of soft, fluffy, kind of animate like seal robot can help improve the sort of emotional experience of elders with dementia.
So looking again as a sort of therapeutic quality of life application where there’s people with dementia they can’t take care of a real pet but they can interact and get a lot of the benefits of touch and so for the companion animals with a robot. So this is a simple version of that but of course you can imagine the trajectory going up from there. So I’m going to do some wrap up here. And I think one thing that’s an important take away from the work is not just thinking about what robots will be in the future in terms of applications and how they’re going to affect your life. But really if you think about the scientific side of this work it’s really through the process of trying to build these robots and taking inspiration from psychology, neuroscience and natural systems we learn a lot about ourselves as well.
So, you know, this is a very old human story, we’ve been trying to build robots and machines in our image for hundreds of years, thinking about it for thousands of years. And I think it’s just a profound human quest of going through this process in order to think about and reflect upon ourselves and what it means to be human and what it means to be human in the future. So, robots as a scientific tool and a tool for reflection, I think, continues to be a theme and a strong theme of this work and I think a role of robotics in many years to come.
And then, finally I just want to wrap up, you know, I started this talk talking about Star Wars and how I was inspired to do this largely because I saw this movie as a young kid. You know, and life has a way of coming back in loops. So, because I saw Star Wars as a kid, I developed Kismet. And Kismet was a robot that was finished right around the time of the movie AI and it was actually an article about Kismet in TIME magazine and one of the producers saw that article and called me up and invited me to consult on the movie as a sort of scientific bridge during the promotional phase. So I didn’t actually advise on the movie itself but I was basically there to help field questions from the press about the themes in the movie. I don’t know how many people saw this movie but this was the Kubrick-Spielberg film about a robot boy who could love.
And in that movie of course the robot that I loved was Teddy. Teddy was a great character. And through that I met Stan Winston and when I met Stan; I told Stan I’d just finished building Kismet, I said we should build Teddy but we should really build Teddy. And that’s what became Leonardo. So, in many ways Stan and I share the same vision, the same dream in that Stan, coming from performance and Hollywood and special effects, his dream was to build a real character.
What he meant by that is that robots in the movies like Teddy here, I mean they’re basic, they’re just puppets. They’re only brought to life on the screen by a whole team of puppeteers who are controlling all these degrees of freedoms so Teddy had like 40 points of motion. Stan’s dream was to build Teddy that had a life and existence off the set too. That’s exactly the vision I had for Kismet, C3PO and R2D2. Robots that had personality, persona that transcended the set.
So, I started this great collaboration with Stan. Of course, he is a legend in Hollywood and unfortunately we lost him to cancer way too early. But the dream continues, the dream continues.
But then it turns out a couple of years ago Lucas films teamed up with the Boston Museum of Science to create a show called Star Wars: Where Science Meets Imagination. So, many of the artefacts in the original Star Wars movies were brought to science museums so there was a touring show across different cities and I think different countries where I actually was asked to consult and participate in this exhibition. And there was this one thing called the robot theatre where I basically was like a character in this theatre. But I had this sort of interaction with C3PO so I actually met Anthony Daniels who’s the actor who played C3PO. And incidentally, Anthony Daniels is the only human who actually appeared in every single movie. The only human actor to be in every single one so it was a great thrill for me to meet Anthony Daniels and you’ll see in that theatre performance there was Kismet. So this wasn’t the actual robot but it was a mock-up but it was kind of like again this sort of closing the loop of seeing Star Wars as a little girl and actually consulting on this movie and actually getting to meet George Lucas himself. So it’s just sort of a nice story of how these things come around to circle.
And then the last thing I want to highlight is, you know, who are building these robots with me? They’re students. They are undergraduate and graduate students; they’re not, you know, people who are 18 years old, 20 years old, they are people who are in there who are not [inaudible] and I think again, if you’re willing to pursue your dreams and use your memories to create your dreams you’re capable of so many things so this is just to acknowledge all of the really brilliant, hardworking and creative students that I have. Work that I couldn’t have gotten through without their help.
I’m going to end.
Derek Mooney
I think you’ll agree that was absolutely fascinating. What would you say if I told you that actually Cynthia is an android? And that the real Cynthia is going to walk through that door any minute? You can see what I mean, when I spoke to her on radio today, why I believe that in the not too distant future I imagine we probably will not quite marrying robots but certainly be living with them. Several things, we’re going to take questions so if anyone has questions please put your hand up, lámha suas and we’ll get a microphone to you. As I was watching that, as you went on and on and on you were answering every question I wanted to ask you.
When you were talking about the obesity I was wondering, did the people who were taking part in this experiment, were they all women first of all or were they women and men? And then, were all of the voices on the robots that you put into these homes in Boston, were they all male voices or were they all female voices?
Cynthia Breazeal
Yeah, so the subject pool that we got for the weight management study were recruited at the Boston Weight Loss Clinic so Dr Caroline Apovian is a pre-eminent authority in weight management at Boston University who is an advisor on the project. It just turned out that 80% happened to be women. You know, we didn’t, we would have preferred a 50% split between men and women but it just turns out for that study, 80% happened to be women but the age range was a nice spread in age from 18 to 72 years of age.
For the control you heard the synthesised voice. That was the voice that was used in all of the studies which was a sort of quasi-female-mechanical robotty kind of sounding voice.
Derek Mooney
The reason I ask you is because women are more conscious about their appearance, I think, than men are when they get to a certain age. The followon question was going to be about the age range you used and that women are more conscious and they’re more willing to make that effort and if you had another woman there beside them helping them along well then it was kind of girl power sort of thing, if you know, what I mean. And I was just wondering did that have any impact on the study you were doing?
The other thing you spoke about was touch and how important is touch and you were talking about the handshake particularly. Because I was interested in this because there was research done by a group of researchers in Iowa University and they had 200 students who were going through an interview process. And they had handshake experts watching the students as they went through the interview process. And they hadn’t been told about this. And they were watching to see how and when they shake hands and who would shake hands. And they found that the people who gave the firm handshake, the men, all got the jobs. And the women who gave the firm handshake all got the jobs. And the ones who had the weak handshakes at the interviews, no matter what else they did, just for some reason didn’t get the jobs and they concluded touch was very important and handshake was very important etc, etc. And I’m sure as you go down the road with your research and you’re looking at this already because it was leaping to my mind as I was seeing it on the screen there, how important this sense of touch is and how important the human figure is. Which is why you don’t just put in the PC, which is why it has to have some kind of form and shape. But why didn’t you dress them? The people had dressed them, why didn’t you dress them?
Cynthia Breazeal
Yeah, you know, people have been raising that question about, you know, are these robots naked? And, you know, we haven’t yet, we’re just starting to now appreciate how much people want to adorn these systems. So you meet people, they talk about cell phones being personalised and you putting your different covers and so forth. People like to adorn these robots as well. So, I guess for us we put these cosmetic shells on in some sense we kind of see that as some form of donning, you know, covering on them. But I think you’re right. I think clothing is a really, I mean culturally it’s a really, it’s another really fascinating topic.
Derek Mooney
Because I’m assuming, you showed some snaps of people
Cynthia Breazeal
Yeah, robot with a tie, you know.
Derek Mooney
who had put a hat on and stuff.
Cynthia Breazeal
Versus a robot with a [inaudible] baseball cap.
Derek Mooney
They were the people who had these in their homes
Cynthia Breazeal
Those were the people who were dressing them.
Derek Mooney
so they wanted to relate to them. You could see that was screaming out that ‘Actually, this has become my mate’ or ‘I want to make this more humanlike so I’m just going to do this.’
Cynthia Breazeal
A lot of Boston Red Sox caps on the robots.
Derek Mooney
But I just wondered why, you could have dressed them and sent them out dressed.
Cynthia Breazeal
We could have. But then people
Derek Mooney
But that’s why I’m just wondering why you didn’t.
Cynthia Breazeal
Right. Well, basically because we didn’t even think about it. But I think the advantage of letting people do it themselves is that they could personalise it. You know, so that’s probably something that you will actually want to let people do for themselves. It’s almost kind of like when you look at these human-human interaction studies and then look at personality types, that people tend to prefer people with similar personality traits and so forth. Robots might have greater acceptance if you’re allowed to customise them in a way that fits into your home décor, you know, the culture you’re coming from and so forth, so.
Derek Mooney
That’s what I was going to ask you about next, was personality traits. Because there’s a programme on, I think it’s Discovery Living or something which is When Animals Go Bad and you see it when the animals turn on the keepers, whatever. And I was just wondering what happens when robots go bad. And when these robots move onto a stage when they all develop their own little personalities and some of them will be a little bit nasty. I’m thinking of Brenda who I work with has this sonic device in the car that makes her aware when she’s going to back up and reverse into something and it goes beep, beep, beep when she’s going to collide with something. So she turned it off one day and what happened the next day? She actually hit the wall when she was reversing in. Okay, stupid thing. She said she can’t reverse anyway but I’m just wondering, you know, will robots in time develop that kind of trick sense of humour? Or if you like, ‘Okay if you’re not interested in what I have to say, okay sh***.’ You know, what I mean?
Cynthia Breazeal
I mean I think, again it’s what we’re finding from human psychology as well as from human computer interaction that, you know, people pick up on these cues in a really, you know, nuanced way. Whether you intended to design them in or not so I suspect that people assigned a personality to Autumn that we didn’t necessarily attempt to try to craft that model in people’s heads. But just to enact, they develop a personality for that robot. Again, I mean, depending on the goals you have for interacting with the system, you might be the kind of person, or you want the drill sergeant robot because that’s what’s going to help you get your results. Whereas other people might want the sort of empathetic robot because that’s what’s going to connect with them more. I think, you know, these are all intriguing questions that we’re just starting to look at.
Derek Mooney
But the robot learns that the guy has put the biscuits over there or the cookies over there or he’s done this or he’s done that. So then it can develop almost a mind of its own. It sounds almost ridiculous. I find it hard to believe that I’m saying it myself but if it can figure out, if it can reason as you’ve demonstrated, and I have to believe what I’m seeing, then that’s not such a huge leap of the imagination is it?
Cynthia Breazeal
I don’t think it’s a, I mean we certainly have been imagining it, again for thousands of years, so it’s not that leap of imagination. I think, you know, a lot of this work, I think a lot of what makes robotics intriguing is that because of science fiction movies, people have very strong opinions about what robots are and what they could be before they ever really even existed. So, you know, in the United States, you know, the first thing people think about is Terminator you know. If you go to Japan, the iconic robot figure is Atom Boy which is a fundamentally benevolent robot. So you go to Japan and there’s tremendous acceptance of robots. Of course they see robots as being benevolent entities.
So, I mean your question is a very typical question for Western culture where we’ve grown up with this sort of suspicion and this sort of fear of these technologies and I just want to highlight that this is a cultural thing. It’s not a universal thing. I think in a lot of movies, you know, if you talk about inspiration of science fiction, it’s because the machines can’t relate to us and don’t care about us. It’s that because we’re kind of the superfluous or annoying thing that’s just getting in their way. That’s why they, you know, discard us whereas my work is really taking the opposite view which is if robots can actually relate to us and empathise with us and sympathise with us then they can become more socialised in society. And that’s a very kind of science fiction kind of idea.
Derek Mooney
And learn to get the better of us. No, I’m joking.
Cynthia Breazeal
Well, you know, I think a lot of that’s socialisation too because kids, if you look at children, you know, right at the same time where they’re exhibiting real like heart-warming forms of empathy and thoughtfulness they’re also learning how to trick you. I mean it’s the same cognitive competence that’s being played out in two different ways so that’s where the teaching of values and norms becomes important. But that’s a very social process so, you know, I think there’s a bigger question of, you know, what kinds of robots are we going to have in the future. I think we’re going to have many, many, many different kinds. And I can just throw up some slides showing you many, many different kinds of robot. I think, you know, it’s going to be us as a society to try to understand what is the appropriate relationship of people to these kinds of technologies.
What’s the appropriate kind of responsibilities that you want to assign to them and vice versa? And have that be an informed discussion because I think from the scientific side we have to understand these things in order for us to start to be able to make, you know, informed judgements of what’s the appropriate, appropriate roles. So in even things like trust, you know, the fact that you can vary non-verbal cues that affect how much people trust these robots. The reason why that’s important is because you want to design them so that people have appropriate trust. Right, so it’s all about trying to understand it first so that you understand how to responsibly design them and I think that’s a really important thing.
Derek Mooney
The android I saw for the first time, James May on BBC2 who presents a programme called Top Gear, you may not be familiar with it. But he’s done this kind of journey of discovery as to how far robots have come and that was the first time I saw the android, it was like two months ago on TV. But at what stage does control come in? I mean, you’re working away. I mean I couldn’t believe when I saw that programme two months ago how far it’s come and watching this today is another eye opener for me. What stage do governments step in and control or do you have carte blanche, I mean is there an endless kind of pocket there of money for you?
Cynthia Breazeal
I mean, so in the United States stem cell research is one of the big hot areas so I think basically when it’s raised as many technologies, it’s something that needs to be carefully considered. And it’s starting to get out to affect, you know, products and so forth. And then I think, yeah it has to enter into the dialogue. I think right now, I mean we’re still a long way off frankly. I mean progress is exciting but I mean we’re still way off. And it’s really much more about the basic science and the research, to try to understand it. And understand how people interact with things so that we can make an informed decision about how it makes sense to design in and what kind of roles and so forth these machines should have in the future if any.
You know, so I’ve shown you research from robots that’s just a pure communication medium all the way to stuff like Leonardo which is the fully autonomous robot. And you can imagine there’s a whole range and more of possible, yeah, roles and applications in between. But what’s important is to have the dialogue and I think what’s important is to communicate the questions and what we’re learning. And again, I think it’s significant to highlight what we’re learning about people too because again, we’ve been trying to build these machines in sort of our image for hundreds of years and every new technology we up the ante so when you talk about, when the first people are starting to be able to build these very fine gear mechanisms. That’s when you start seeing the first ancient automaton like the Vaucanson Duck and so forth.
And then, over time the mechanical computer came into play and you started to see these more sophisticated and then the electric computer came and so it’s almost like with every new technology we’re able to push this area of inquiry even further. And I think the reason why we do that, a lot of it is because is, through this process we’re trying to understand ourselves. We’re trying to think about ourselves and use this endeavour, I think as a scientific pursuit. And in some cases probably for the last philosophical pursuit to try to grapple with what does it mean to be human and what are humans becoming, what do we want to be in the future? And science fiction does the same thing, right? Science fiction uses robots as a sort of foil.
Derek Mooney
I think science fiction now has to catch up to be honest with you.
Can we just take some questions? Just say your name please if you would. Thanks.
Question [Jan]
Hello, my name is Jan. I have a whole bunch of questions that I’ll just try and pick a couple. So, one of the ones I particularly wanted to ask is have any of your research, has any of it focused on what attributes of the robot interaction walk into the ‘uncanny valley’ and make things abhorrent and completely break that sensation of trust and relationship building?
Cynthia Breazeal
So I think, I mean with respect to the uncanny valley, I think a lot of the research right now that’s working in that space is actually trying to understand and characterise the uncanny valley. So, the uncanny valley is this conjecture that was originally posed by a Japanese researcher called Mori, where the hypothesis was that if you look at affective response, so, liking or just like positive of negative affect and human characteristics, that the idea is as you make various artefacts more anthropomorphic in nature, you get positive affects. And like Kismet is kind of in that area, it’s kind of cute mechanical cartoon sort of thing. But as you get close but aren’t close enough, you plunge into the uncanny valley. You know, and a lot of people probably say like the androids of today are probably in the uncanny valley.
I think if you look at, you know, computer animation, you know, if you look at Tin Toy which is a predecessor of Toy Story and maybe even Toy Story itself, I mean the human characters are in the valley, they’re looking kind of zombie-ish in some sense. But then, as you get closer and closer you can pull out of the valley. And so like Shrek for instance, you might argue that computer animation is finally out of that valley where you can tell they’re not real people but they have an aesthetic onto their own. So this is called the uncanny valley.
And then, I think, you know, and again this is a conjecture and people kind of throw this around like this is a fact but it’s really a conjecture. And it’s applied not only to the way the artefact looks but also to its movement, how it moves. And a lot of the research, some of the research that’s being done say with the android robots is trying to really characterise what is this uncanny valley? How does it vary across age, across culture, across expectations? So, I think it’s much more they’re trying to understand it at this stage.
And there’s probably just a lot of kind of personal preferences and biases as well. I mean somebody might see the androids and get really creeped out and other people might be okay with it. That might be cultural and personal preference and so forth. So, you know, people are starting to look at fMRI studies and things like that to try and even understand what’s going on in the brain when you’re observing these kinds of stimuli. So, there’s an effort to try to understand the uncanny valley right now.
Derek Mooney
A lot of that will have to do with conditioning as well, won’t it? I mean if you’re around something long enough you’ll get used to it.
Cynthia Breazeal
Very possibly.
Derek Mooney
Anybody else want to ask any questions? If you just put your hands up. This gentleman here, if you just hang on until we get a mic to you please so everybody can hear you. Thanks.
Question
Hi, yeah, I was at the lecture on Monday night titled Learning to Live with our Planet, right? And there was less than half the number of people here. And I’m just trying to make the, just trying to figure out that, how it is that maybe can robots help us solve our economic crisis or can they help us solve our environmental issues? I mean, it’s all very fascinating; it is fascinating because I’m interested enough to be here but how does it, how, where are we going? Where do you see robots in our lives?
Cynthia Breazeal
Sure. So, I was talking about the sort of personal robot spectrum. I mean, certainly if you look at the whole field of robotics there’s many, many, many applications from scientific instruments that we send to planets to understand other planets to environmental robots that are being sent out in order to measure and characterise our environment so robots today, I mean, you know, there have been famous under, deep ocean exploration robots that try to understand and characterise the ocean. So I mean, absolutely robotics is playing a profound role in how we understand our planet and enabling us to capture data in a way that we couldn’t do previously through, you know, large networks of robots or sensor networks and so I think they absolutely playing a role today in trying to understand our planet and informing courses of action. So if you implement a course of action, trying to understand the implications of that in, you know, ideally like even in real time to be able to see do we need to correct and so forth. So, robots are absolutely playing a role in helping our planet and all kinds of things.
I mean, see robots as, they’re a technology you know, technology can be applied in all kinds of different contexts. Fundamentally the reason why I do it and why people in robotics do this whole work is because we’re trying to improve the human condition, we’re trying to help people, you know, so that’s really what’s it’s always been about.
Derek Mooney
But will you design an environmentallyfriendly robot? How are they fuelled, how are they powered currently?
Cynthia Breazeal
Yeah, many of them are batterypowered. Not all of them but many of them are batterypowered, sure. You know, rechargeable batterypowered, yeah. And in fact, I mean, there’s a lot of synergy right now going on between things that are driving mobile computing and alternative energy sources for cars and so forth that are feeding back and helping robotics as well. So there’s a lot of kind of synergistic technologies that are being of help in a lot of different contexts that are helping advances in robotics.
Derek Mooney
Sorry, go ahead.
Question [Jamie]
Hi, I’m Jamie and as a hobbyist I’m wondering which robot you think we should get for Christmas?
Cynthia Breazeal
As a hobbyist? You know, it depends on how old. Twenty? Well, you know, Lego Mindstorms is still a classic at toy of the century; I mean that’s pretty good. Toy of the century, so you know.
Jamie
But maybe a bit more advanced so you can do some soldering and programming.
Cynthia Breazeal
Well some Lego Mindstorm you can do a lot of programming yourself. I mean there’s a new, so one of my colleagues Mitch Resnick at the Media Lab was actually a co-inventor of Lego Mindstorms. And he’s gone on his next big project is called Scratch and Scratch is a new programming language that’s designed to make programming accessible to people of all ages. And what it allows kids to do is to create interactive games and tell stories and so forth through programming computers but in a way that it’s like building blocks. It’s kind of like doing Legos but it’s computational blocks. And there are hundreds of thousands of kids and adults too actually using these systems and there’s an automatic translator so you can build a programme in Scratch in English and then you can do a translation and it could be translated into Chinese.
And you can, basically like it’s kind of leveraging off of like, you know, when the internet allowed you take a webpage and copy the code to design your own webpage. These projects are being shared and building this big communities of people. Children or people can take one project and download it, adapt it and make a new project. So there’s a very kind of creative community going on and I think the exciting thing about a lot of that is not just the community but the sort of co-learning creative process that these kids are getting introduced to these technologies in a way that fosters exploration and creativity. That, I think has appeal for people of all ages actually and it’s a very big community right now. So I would encourage you to look at Scratch and design video games and interactive experiences and I think it’s very cool stuff.
Derek Mooney
I was interested also when you were giving your talk you said that the robots are going to be there to help us achieve our goals. Do you not think though that there is a very strong possibility that we’re going to become redundant ourselves? And that, who wants to go to work when you can send a robot out to do it for you?
Cynthia Breazeal
Well I mean, I think you know, that’s a very sort of, you know, 1960s notion of AI and robotics.
Derek Mooney
No, it’s a very modern notion.
Cynthia Breazeal
I don’t actually think so and especially when you talk about, you know, the elderly. It’s really important that you know, people maintain their sense of dignity and independence so when I get old the last thing I want is a robot that I can just sit on my couch and have it do everything for me. I still want to cook my own meals, I still want to garden, I still want to do all the things that I do and enjoy in life.
Derek Mooney
That’s a different thing to working.
Cynthia Breazeal
But I might want something to be able to help me. So my point is just, I don’t see these robots really as things that, you know, replace people and I try to keep stressing that in my talks. It’s really about empowering people and the synergy and the partnership and the fact that in many cases, the fact that these robots aren’t human is what makes them so interesting because they have strengths and abilities that can compliment our own abilities. So I really view it much as this partnership. How do you design robots to be partners for people in a way that they have their strengths and abilities and you have your strengths and abilities and it’s how can those come together to again, you know, fundamentally it’s about addressing human goals and human values.
Derek Mooney
I wonder because I wish I could remember now where this research was done but there was some work done not so long ago, 90% of people hate their jobs.
Cynthia Breazeal
Well that’s really too bad because I love my job.
Derek Mooney
Why on earth would you go to work if you hate your job? Send the robot.
Cynthia Breazeal
So that might be an example of if there’s dull, drudgery type aspects of your work.
Derek Mooney
90%. It’s a huge number.
Cynthia Breazeal
Maybe that makes sense that, you know, you offload those aspects to a machine and you let people do what people are good at which is creativity and collaboration and so forth. So, I mean people talk about computers in the same way so thinking about what machines are good at, thinking about what people are good at and just making sure that it makes sense what people, you know, are doing I think. You know, leveraging our capacities so we feel, you know, more human and experience our humanity versus kind of downtrodden so we feel like machines. You know, I think that’s often a criticism in technological processes.
Derek Mooney
Do you look at a robot as a machine? How do you view it?
Cynthia Breazeal
Yeah. But I think what I’m trying to challenge is our cultural
Derek Mooney
I saw your little eyes light up when you were showing the pictures at the end.
Cynthia Breazeal
Of what a machine is. Right, I mean I think there’s
Derek Mooney
I think you showed us Leonardo.
Cynthia Breazeal
Yeah I know I think, yeah. I mean fundamentally, yeah it’s a machine but it’s not a machine like your toaster. You know, I think what I’m trying to do is challenge our notions of what machines are and what they could be. You know, I mean a lot of what my work is and a lot of what you know, academia is trying to do is encourage us to think, to broaden the way we think, to broaden our ideas, to explore new territories. So I see these robots as being, yeah I mean in a technical sense they’re machines but they’re extending, I think, our notions of what machines are and what they can do.
Derek Mooney
Because when you had the learning robot. What was his name again?
Cynthia Breazeal
The Leonardo robot.
Derek Mooney
That was Leonardo, yeah. Excuse me. So Leonardo was there and when everybody saw him, when somebody walked into the room and said ‘Hello’ everybody laughed here, everybody because the cuddly little image of oh it could be another little person there.
Cynthia Breazeal
Yeah.
Derek Mooney
That could easily be, that area there could be very grey matter altogether, couldn’t it? I mean in time?
Cynthia Breazeal
Meaning?
Derek Mooney
I mean, at the beginning it’s a robot because it’s clearly a robot. And then as time moves on, well it’s not a robot anymore it’s my mate, and as time moves on, I fall in love with the bloody thing.
Cynthia Breazeal
You’re really in this kind of mate thing actually. I think there’s something going on.
Derek Mooney
Did you not see the Woody Allen movie all those years ago? Do you remember?
Cynthia Breazeal
You’re fixated on this.
Derek Mooney
No, I’m just curious. I’m curious to know where all this is going. Is there anyone else with a question before we come back to this? Just to get as many people as we can.
Question
The Professor presented a problem at a philosophy of computing class that I took and it was, the question was, is it that we are computers that we think? And I realised only recently that that was an Alan Turing notion. Is it because we are computers that we think? So that’s what you’re saying.
Derek Mooney
We’re very complicated kind of computers. This was something I was with Cynthia today. I mean, you know, for these things to go all the way they’d have to become human almost. Because you break a leg and all the knock-on effects that has. If you break a leg it’s not just a broken leg so physically it has to be fixed up. So, ‘Oh Christ I broke my leg, I won’t be able to go to work tomorrow, I have to pay the bill, I have to go to the hospital, I have to queue up, da, da, da, da, da.’ And where do the robots go about this?
Cynthia Breazeal
Well I think, you know, I think there’s a shift happening where, you know, what we want to do. We’re building the systems to try to understand people and looking at this seriously. We’re trying to design systems to be compatible with people. We’re not necessarily trying to build these systems to be people, you know, and it’s to state the obvious I mean robots are not human and they’re never going to be. Now when we talk about things such as the way that we think and our emotions, I mean robots are not going to have human emotions because they’re just, they’re not human.
Derek Mooney
Or human flaws perhaps?
Cynthia Breazeal
If you look at other species like dogs and so forth, people attribute genuine emotions to dogs but they’re not human emotions; they’re dog emotions. So I think the question that’s raised here, the new question is: if robots can have emotion what are robot emotions in the sense of what makes sense within the context of the fact that these are robots? And certainly there’s interpersonal aspects of that and there’s internal processes aspects of that. I mean people in psychology are learning that emotion and cognition are deeply intertwined and that human intelligence is characterised by that, you know, deep intermeshing of what we consider to be kind of logical cognition and this sort of emotional intelligence.
And I think from that standpoint you could argue that it makes sense for robots to have these ways of thinking and understanding the world because they’re really useful. So, but what does that mean for a machine and I think similarly you keep going to the relationship.
Derek Mooney
Obviously I’m joking; just in case you’re wondering.
Cynthia Breazeal
We don’t know, we don’t know what the human-robot relationship is really going to be because they’re not really among them. And we understand the domesticated animal, dog-human relationship because they’ve been with us for thousands and thousands of years. We tend to map them onto relationships that we know because that’s kind of the obvious thing to do. But I think in reality, you know, it’s going to be different kinds of relationships, you know, and you’re not going to have a human-robot that makes you feel like a human-dog relationship. You know, it’s different.
Derek Mooney
But you’re taking it a stage further. What animals lack, which is what humans have, is the ability to communicate vocally, to be able to speak to each other and understand each other. And what you’re doing with these computers is you’re giving them that ability and that facility and that’s where you’ve taken this a stage further. Because we know that the dog can fetch the paper, the dog knows when you’re home, the dog can tell you when it wants to go for a walk because it looks at the door or it looks at the tap if it wants a drink of water.
Cynthia Breazeal
In many ways dogs are a lot smarter than these robots. I just got to; you know, give a call out to the dogs. Dogs are really smart.
Derek Mooney
But this is where you’re taking this that stage further because it’s going to lift its eyebrows when you lift your eyebrows as a dog does. A dog can look sad too, a dog can dream, this has been proven already.
Cynthia Breazeal
Dogs are very social animals so yeah.
Derek Mooney
So this has gone beyond that already from what I’ve seen on the screen. Anyway, is there somebody else up there? Yeah.
Question
Yeah, you said earlier about that you couldn’t do tests with humans. Like disable eyes and mouths and all that. Are you planning on doing that with robots to find out what makes people like robots, like if you take out the eyes would they like robots more?
Cynthia Breazeal
Yeah, so I mean again if you look at the human-human psychological literature you know, that these cues such as eye contact and interpersonal distance and facial expressions all matter but we don’t know to what extent and how much. But, yeah, we’re starting to do a series of studies to kind of systematically vary those parameters and seeing how they impact people’s perception of us. And that’s going to be interesting not only for robot designers I think but also for people to think about, you know, the impact of these cues and kind of like how much, you know, impact they have on our perceptions. So, yeah.
Derek Mooney
Do you want to ask a question? Can we get a microphone down to, what’s your name? Philip wants to ask a question.
Question [Philip]
What subjects do people have to learn in school to become a robot scientist?
Derek Mooney
You were looking at the robot were you?
Cynthia Breazeal
You know, so if you look at the nature of work that I do. I mean clearly there’s a lot of technical aspects. There’s a lot of engineering, mathematics, science. But you’ll also notice there’s a lot of sort of performance sides of it too. So the fact that I collaborated with Stan Winston on the creation of Leonardo, that it’s facial expressions, it’s a way of expressing itself. I mean that could be speaking to classical animation and so forth. So, you know, all that you can know and learn is going to help you and inform your ideas and how you think about and understand these sorts of problems. And allow you to be creative and go beyond what’s been done in the past. So, I would never want to limit your curiosity in the subject matter that you’re passionate about or want to learn about. But certainly a lot of robot people are coming at it from a technical perspective. But you’re starting to see and especially in this area of human-robot interaction, psychologists are coming into it as well from the scientific affective in the case of like, you know, Stan’s stuff, he’s coming at it from an artistic perspective so I think all of these things, you know, play an important role of how the different perspectives and way of thinking about this sort of very rich space of possibilities.
Derek Mooney
Anybody else? This gentleman here. Thanks Philip.
Question [Mike]
My name’s Mike. I have a question about the mirroring effect because you mentioned mirroring a lot in the beginning of the presentation. And I was wondering because robots think using code of course so they get the stimuli and they react, then they learn. Have you tried for example copying Leonardo or like, you know, making a more advanced version of Leonardo sit in front of the less advanced version of Leonardo and try the more advanced robot to make him teach the less advanced robot?
Cynthia Breazeal
So we’re starting, I mean we have a very new project basically trying to look at that transfer question right now. So, and just, you know, not necessarily having a robot explicitly sort of do pedagogy to teach another robot but what can you transfer from one robot which might have a different embodiment even to another system. So, I mean there’s a lot of interesting hard problems in succeeding in doing that but we’re starting to look at these questions as well.
Derek Mooney
And when they start recognising their own images, robots?
Cynthia Breazeal
There are people who have actually already started doing that.
Derek Mooney
Really?
Cynthia Breazeal
Oh yeah. Recognising stuff in a mirror, sure.
Derek Mooney
That’s extraordinary stuff.
Cynthia Breazeal
Sure.
Derek Mooney
I know certainly when birds do it they attack.
Cynthia Breazeal
Yeah, well you know, it’s fascinating because in most species eye contact is viewed as an aggressive thing. It’s usually it’s like either you run because something’s going to eat you or it’s a kind of challenging-
Derek Mooney
Fight or flight.
Cynthia Breazeal
Yeah or it’s a challenging sort of thing. I mean humans are, you know, we’re very different in that eye contact is seen as this kind of pro-social sort of things in many cases. Of course it doesn’t have to mean that but many, many, many species are very sensitive to at least being watched. And there’s a really good reason for that.
Derek Mooney
Anybody else? Up the back?
Question
Hi, sorry I was just interested in your, when you had the robot in the medical environment. I was just wondering would this kind of reflective or empathetic, you know, personality that’s being built up. Would you envisage their involvement becoming greater there? That maybe they could recognise distress or maybe they could collaborate?
Derek Mooney
You’re talking about in the hospital situation?
Question
Yeah, or in any medical environment. In an ambulance or something that they could collaborate with the medical professionals. We could do with some help in this country with our health care.
Derek Mooney
They couldn’t be any worse than some surgeons I’ve come across in my time.
Cynthia Breazeal
There’s a whole field that’s called affective computing that basically is looking at that; giving computers and machines the ability to perceive human affective states either through observable facial expressions, tone of voice but also through five physical sensing. And that’s really relevant in a lot of medical applications, it’s real relevant, you know, in the case of say, of people who are on autism spectrum disorder where in many cases they may have these very aroused internal states but they’re not expressing it through the same channels that we would expect from another person to see if you’re stressed out and so, you know, you hear examples of a teacher who is trying to teach someone something but they’re really stressed out and suddenly the person has an outburst. And they’re like, suddenly it was an outburst and it’s like what they’re telling you once they become maybe later in life verbal it was never a sudden outburst, it was like building and building and I didn’t know why you wouldn’t back off. And it’s just, you know, if you had systems that could interpret those signals and then have some, maybe an artefact, a robot or something that could communicate that so a person would understand what’s going on inside them. I think there’s a lot of applications for stuff like that. Medical, therapeutic, just communication, I think there’s a lot of interesting applications for stuff like that.
Derek Mooney
Including sensing danger. I mean robots already can sense danger.
Cynthia Breazeal
Sure.
Derek Mooney
Like the little warning device on the car – ‘You’re going to crash, stop, I’m telling you, I’m screaming at you, stop.’ Turn it off you crash, you made a mistake. Anybody else? Sorry, this lad down here. If we got a mic, please Liam. Or you could shout it out if you want.
Question
So when you have one intelligence learning from another intelligence, you have a robot say learning from a human, did you ever find that when you’re doing your experiments that the non-human intelligence did something quite intelligent but not just very human?
Cynthia Breazeal
Did something quite intelligent but not human.
Question
I mean something that humans might not actually think of doing but turns out to be quite intelligent.
Cynthia Breazeal
Yeah, you know, I think there are certainly examples of that so I’m going to talk about some different work.
Derek Mooney
That’s when you get worried. When it starts doing something you don’t expect.
Cynthia Breazeal
But there have been some really, you know, kind of funny stories you know, when people were doing a lot of work with genetic algorithms and they were these sort of virtual worlds and involving these kind of virtual creatures. And what they found was that sometimes these virtual creatures would find a bug in your code, could exploit some artefact if your virtual world might like you didn’t model gravity exactly right and would exploit that to be able to do the task. So I mean there are cases of, you know, when these systems learn and they find these irregularities. Sometimes they find stuff that you didn’t expect or predict and start using it. So, in the case of these virtual creatures in the sort of realm they would start using the violations of your laws of physics in order to solve the task of how to locate from point A to point B in really funny ways because it’s completely violating, yeah, what we know as the physical laws.
But I mean I think that’s certainly, it’s certainly possible, you know. If it can sense it, if it’s within its space of possibilities that it can search over and discover then yeah, you could expect it to find the things that might not necessarily be intuitive to us but you could take advantage of.
Derek Mooney
We’ll just take a couple more questions. So that man there.
Question
I’ve got one question but it’s sort of half been answered by all the other questions that have come before. The first half of your talk effectively was focused on psychology and the learning of how we learn because you had to learn that first to try and apply those rules to help your robots to learn. At some point have you got a measure of what it is to learn and can you get a way of measuring when your creations can learn on their own beyond yourself, what you’ve taught? Effectively, when will androids dream of electric sheep, to paraphrase a famous author. And what have you learned about yourself that you wouldn’t have thought of from what you’ve been working on?
Cynthia Breazeal
Yeah, so I mean from the lot of the work that we’re doing I mean, so the one video I showed was Leo showing that, you know, when a person’s not nearby that he actually does have these internal drives to explore, has essentially this sort of curiosity drive to try to discover the things that are new so you know, if you look at, I mean, even Piaget, he talks about these multiple drives and motivations we have for learning. So we’re both attracted to novelty and we also have this sort of pleasure of mastering. Sort of a push ball motive so to speak to discover the thing you don’t know yet but also to master what’s familiar. And, you know, that’s basically a process that the robot’s engaging in, in that video. Now, that being said, children learn in all kinds of ways that this robot is not learning. And I think there’s still a lot we have to understand about how children learn.
So, you know, machine learning is this huge field of inquiry and the vast majority of that work is not inspired by how humans learn at all. It’s a lot of statistical kind of numbercrunching sort of things. And I mean there’s applications and there’s, you know, these are things that computers are good at and so it makes sense. I think a lot of the reason why we’re building robots in the way that we are is because we want them to learn from people. So if you look at these sort of machine learning techniques, if you understand the algorithms you can design the algorithm and kind of frame the whole problem. That the algorithm basically at that point just kind of turns its crank and out comes the result if you as the human designer were smart enough to structure that appropriately.
We’re looking at a very different problem which is someone who knows nothing about the underlying machine learning algorithms but has a lifetime of experience of teaching and learning from each other. What does that look like when you try to teach a machine? So that’s why we draw so much from how people learn because we’re trying to make this process intuitive for people and also effective that people bring so much insight and constraint and help basically to help people learn, to help children learn. You know, and the spatial scaffolding which is one example of this simple interaction that people automatically understand what the meaning of that is. Trying to design robots that can appreciate all of these cues that I threw up on the slide. I mean that’s a pretty daunting task. We’re kind of highlighting this, this one particular one.
Just appreciating that, you know, learning on your own even as people, we don’t do that, you know, if you leave an infant in a room alone they don’t learn. They can’t.
Derek Mooney
Would they explore?
Cynthia Breazeal
Well it depends how young, you know, and there’s cases of like the Romanian orphanages where the children were just basically given the kind of biological needs but they weren’t socially interactive so they grew up to be, just irrevocably, to be severely retarded. So appreciating that our mind goes through all these critical periods, I mean even these kinds of interactions and many of them are profoundly social interactions to develop normally. So the social is critical for a cognitive and continued social development as well. So, you know, the world is a really complicated place. I keep saying that but I think we need a lot of help beyond just our own exploration to make headway.
And that’s why, you know, ants basically don’t have to learn a lot in that they kind of are born, you know, with this certain knowledge and that’s what’s going to get them through their ant day. Humans are what is called an altricial species meaning that we go through a very long period of development where we are utterly dependent on our care givers because we need that time in order to learn and acquire these abilities through the interaction with others. So, again I think, you know, machines can learn in very inhuman, non-biological ways but I think it’s intriguing and potentially relates significant to build robots that can also learn in these more human inspired ways.
Derek Mooney
I think that’s to do with the lifespan though. I mean if you’re a human you’re not going to, you’re going to live longer than an ant, you’re going to live longer than a mayfly which lasts for a day. It comes up, it goes up, it gets mated with, it’s gone, it’s all over. So it only needs to know what it has to do for that moment if it even knows it. Humans have a much longer lifespan, so we have to do a lot more so it takes us a lot longer but we learn it and that’s how we get through it. But what about robots? How long, what’s the kind of lifespan or have you even thought about that?
Cynthia Breazeal
I think that’s one of the big questions. Is that right now, I mean the reality of it is, most robots, especially research robots, aren’t ‘on’ that long. You know, there’s manufacturing robots that are on 24x7 but they’re just doing the same thing again and again but learning robots, they’re on, you know, they don’t really go through, I think, a life cycle. And I think longterm interaction and longterm learning, I mean that’s a grand challenge in the field right now. Is how do you build a system that can go through these extended months, years types of interaction and really build and accumulate.
Derek Mooney
Because if you go into one room where the robot is kind of like this and then you go into another room where the robot is kind of like that. And that it’s different stages and if you’ve, depending on what your issue is that day or what the task is you say these robots are designed to help you fulfil your goals. I mean is it the baby one? Is it the kind of adolescent, is it the adult or is this one that you grow up with? Or that grows up with you or comes in readymade, already designed?
Cynthia Breazeal
Yeah, I think these are all, you know, very fair questions that I don’t think people, you know, know the answers to yet. But I think, certainly a grand challenge right now of robotics is looking at this longterm interaction and longterm learning because it really hasn’t been done yet.
Derek Mooney
Because if the baby is growing, Leonardo is teaching the kid in the room and ‘What did you learn today?’ ‘Well we learned this.’ And then does that robot grow as the baby grows?
Cynthia Breazeal
To build… absolutely. You know, people call, you know, the ability to scale, you know, to become more complex over time without losing ability. Because you can imagine these systems become much more complicated I mean they could just get unwieldy. I mean they could just, you know, there gets to be a point where you’re not constructively building upon things but it’s becoming, you know, too sort of you know, scattered.
Derek Mooney
Well it’s kind of there in the obesity one isn’t it? Because you say initially the conversation is more instructive and then as it gets to know you almost -
Cynthia Breazeal
So based on, right. So the robot is actually taking, I mean this is very different kind of, it’s not learning in the way that we’re talking about children or anything. It’s more of an adaptation. So everyday the robot as basically part of your interaction is asking you a couple of questions in some sense, that is trying to measure what’s the state of the relationship, meaning is the working alliance working? Do we feel like it’s going through disrepair? And then based on these measures it’s adapting the dialogue to either try to maintain the interaction or also just based on just a clock, an internal clock knowing how many times you’ve interacted with it. It has a measure; it has ways of basically computing how familiar are you with the robot. Knowing what it said to you in the past and sort of appreciating the past interactions in order to gauge what it should be saying now.
So when you talk about this life cycle and sort of relationships, I mean the thing that it’s bringing into it is time. You know, when you build a relationship with something you have a set of shared experiences, a whole history of past experiences that shape and inform what you do in the moment and shape and inform what you think about in the future. You know, and the weight management robots are really simple example of that. But, you know, we should think about scaling that to more complicated systems. I mean this idea of this longitude of relationship is a really fascinating question.
Derek Mooney
One last question.
Question
I’m just wondering how far we are from installation shown in the Bladerunner movie?
Cynthia Breazeal
In Bladerunner? So, you know, this is funny because you’re the third person who’s mentioned Bladerunner. And Bladerunner is a different technology. Bladerunner is genetic engineering and maybe eventually I’ll go to genetic engineering to do this kind of stuff but yeah, I mean it’s a different technology that people are using there. Yeah, so it is work that’s going on, it absolutely, genetic engineering is going on but it’s not my area of expertise and you know, I couldn’t even venture to guess, you know, how close we are to doing that. But, yeah that’s a science that’s advancing very rapidly.
Derek Mooney
Well Cynthia, on behalf of everybody here I’d like to thank you for a very informative presentation. You’ve got a fabulous mind and a fascinating imagination, a fantastic imagination I have to say. So let’s here it for Dr Cynthia Breazeal and thank you very much indeed for coming along.
« Back to Lectures