Jacqueline Kory-Westlund

Click here to see the video showing this project!

Study Overview

For my master's thesis at the MIT Media Lab, I created a social robotic learning companion that played a storytelling game with young kids.

Children’s oral language skills in preschool can predict their academic success later in life. Helping children improve their language and vocabulary skills early on could help them succeed later. Furthermore, language learning is a highly social, interactive activity. When creating technology to support children's language learning, technology that leverages the same social cues and social presence that people do—such as a social robot—will likely provide more benefit than using technology that ignores the critical social aspects of language learning.

As such, in this project, I examined the potential of a social robotic learning companion to support children's early long-term language development.

Study

The robot was designed as a social character, engaging children as a peer, not as a teacher, within a relational, dialogic context. The robot targeted the social, interactive nature of language learning through a storytelling game that the robot and child played together. The game was on a tablet—the tablet showed a couple characters that the robot or child could move around while telling their story, much like digital stick puppets. During the game, the robot introduced new vocabulary words and modeled good story narration skills.

Furthermore, because children may learn better when appropriately challenged, we asked whether a robot that Matched the “level” of complexity of the language it used to the general language ability of the child might help children improve more. For half the children, the robot told easier or harder stories based on an assessment of the child’s general language ability.

17 preschool children played the storytelling game with the robot eight times each over a two-month period.

I evaluated children's perceptions of the robot and the game, as well as whether the robot's matching influenced (i) whether children learned new words from the robot, (ii) the complexity and style of stories children told, and (iii) the similarity of children’s stories to the robot’s stories. I expected that children would learn more from a robot that matched, and that they would copy its stories and narration style more than they would with a robot that did not match. Children’s language use was tracked across sessions.

Results

I found that all children learned new vocabulary words, created new stories during the game, and enjoyed playing with the robot. In addition, children in the Matched condition maintained or increased the amount and diversity of the language they used during interactions with the robot more than children who played with the Unmatched robot.

Understanding how the robot influences children’s language, and how a robot could support language development will inform the design of future learning/teaching companions that engage children as peers in educational play.

Links

• This work is also on the Personal Robots Group site and the MIT Media Lab site
• New Scientist covered this work: Kindergarten bots teach language to tots
• I wrote about this project for the MIT Media Lab blog: Making new (robot) friends: Understanding children's relationships with social robots.
• That blog post was republished on IEEE Spectrum: Robots for Kids: Designing Social Machines That Support Children's Learning.

Publications

• Kory, J. (2014). Storytelling with robots: Effects of robot language level on children's language learning. Master's Thesis, Media Arts and Sciences, Massachusetts Institute of Technology, Cambridge, MA. [PDF]

• Kory, J., & Breazeal, C. (2014). Storytelling with Robots: Learning Companions for Preschool Children’s Language Development. In P. A. Vargas & R. Aylett (Eds.), Proceedings of the 23rd IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE: Washington, DC. [PDF]

• Kory-Westlund, J., & Breazeal, C. (2015). The Interplay of Robot Language Level with Children's Language Learning during Storytelling. In J. A. Adams, W. Smart, B. Mutlu, & L. Takayama (Eds.), Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction: Extended Abstracts (pp. 65-66). [on ACM]

• Kory-Westlund, J. (2015). Telling Stories with Green the DragonBot: A Showcase of Children's Interactions Over Two Months. In J. A. Adams, W. Smart, B. Mutlu, & L. Takayama (Eds.), Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction: Extended Abstracts (p. 263). [on ACM] [PDF] [Video] Winner of Best Video Award.

• Kory-Westlund, J. M., & Breazeal, C. (2019). Exploring the effects of a social robot's speech entrainment and backstory on young children's emotion, rapport, relationships, and learning. Frontiers in Robotics and AI, 6. [PDF] [online]

{: .img-responsive .img-rounded .center-block}

Click here to see the video showing this project!

Concept: Glowing Orbs

Glorbs. Glowing orbs. Interactive balls that help you visualize and explore the dynamics of motion&mash;acceleration, velocity, rotation. Colored lights embedded in the balls give immediate visual feedback about how the balls are moving. Throw one up in the air, and watch the colors change from blue (high acceleration) to purple to red (little acceleration) and back to blue on impact as you catch it. Learn about physics!

The idea for glorbs evolved from a project I did in the Media Lab's Tangible Interfaces class the previous semester. My group came up with an idea for a wearable interface to motivate and guide collaborative, synchronous motion—i.e., pairs of wristbands containing LEDs that would change color and intensity as you move synchronously with other people wearing similar wristbands. We made a stop motion video showing off the concept, and a prototype wristband that lit up.

Why not make that class project come to life for The Other Festival? Randy, my husband, thought it'd be fun to do an electronics project, so away we went. Several brainstorming sessions later, the concept had evolved into something doable in the span of one semester—you know, not requiring a bunch of sensors and complex algorithms for measuring synchronous movement between humans...

Glorb Design

We prototyped on a breadboard. Wires, xbees, microcontrollers, LEDs... We used a triple-axis inertial measurement unit (IMU) to measure acceleration and rotation, which was streamed to an AVR microprocessor. There, it calculated average values and set the LED colors and brightness levels appropriately.

A wireless transmitter (xbee radios!) let us stream the data to a computer, so we could show a real-time graph.

Once we had settled on a design, Randy laid out a PCB. We ordered parts. We used the Media Lab's shop to machine the boards. We soldered.

We cut plastic whiffle balls in half, stuffed them with the electronics, a battery pack, and some bubble wrap, and tied them up with thin nylon rope. Bam! Glorb.

Besides the glorb that changed colors from red to purple to blue to reflect its acceleration, we also made a glorb that got brighter under high rotation, and dimmer under low rotation.

The Other Festival

We set four glorbs on a table in a dark room. We wanted to see what people did with them— play? explore physics?—so we did not provide any instructions or guidance, just four glowy balls.

People came in. While we didn't hear anyone explicitly discussing physics, people did pick up the balls, tossed and caught them, played around, and generally found them entertaining. Which was the goal!

Video

I made a video showing the development of the glorbs, featuring yours truly juggling three of them. Watch it here!

Acknowledgments

Thanks to Jonathan Speiser, Adina Roth, and the 2013 MAS.826 class for their support on earlier versions of this project!

Alive and not alive

At the core of this project is the idea that new technologies are not alive in the same way as people, plants, and animals -- but nor are they inanimate like tables, rocks, and toasters. We attribute perception, intelligence, emotion, volition, even moral standing to social robots, computers, tutoring agents, tangible media, any media that takes -- or seems to take -- a life of its own.

Sometimes, we relate to technology not as a thing or an inanimate object, but as an other, a quasi-human. We talk to our technology rather than about the technology, moving from the impersonal third-person to the personal second-person, moving into social relation with the technology.

So, given that we perceive and interact with these technologies as if they are alive... are they? At what point do they become alive?

What does it mean for a technology to be alive?

How much does whether they are “actually” alive matter, and how much is our categorization of them dependent on how they appear to us?

Maybe they will not fit into our existing ontological categories at all.

Not things.

Not living.

Something in between.

Story

I explored the question of how to encounter the "aliveness" of new technologies through a set of life-size sequential art pieces.

The story followed several robots in the human world. Life-size frames filled entire windows. The robots ask about their own aliveness, self-aware and struggling with their own identity. They try to fit in, but don't. A wheeled robot looks sadly up at a staircase. A shorter wheeled robot sits in an elevator, unable to reach the elevator buttons. A stained-glass robot draws our attention to the personal connections we have with our technology.

Social robots. Virtual humans. Tutoring agents.

They are here. They are probably not taking over the world. They are game-changers and they make us think.

Perhaps they cannot replace people or make people obsolete. Perhaps they are fundamentally different. Perhaps they will be a positive force in our world, if done right. If viewed right. If understood as what they are. As something in between.

How will we deal with them? How will we interact? How will we understand them?

Medium

The story was created as a life-story story that the reader could walk through, so reading would felt more like walking down the hall having a conversation with the character than like reading.

I read Scott McCloud's great book, Understanding Comics, around the same time as doing this project. (Perhaps you can see the influence. Perhaps.) Comic-style, sequential art to promote a dialogue. An abstract character, because if you had an actual robot tell the story, something would be lost. Outlining the robot character in less detail, as more abstract, drew more attention to the ideas being conveyed, and let viewers project more of themselves onto the art.

The low-tech nature was partially inspired by ancient Chinese cut paper methods, as well as by some comics styles. The interaction between the flat, non-technological medium through which the story is told and the content of the story -- questions about technology -- calls attention to the contrast between living and thing. What is the role of technology in our lives?

Installation

Select frames from Am I Alive? were installed at the MIT Media Lab during The Other Festival.

Video

I made a short video showing the concept, making of the pieces for the installation, and photos of the installation. Watch it here!

Relevant research

If you're curious about the topic of how robots are perceived, here are a couple research papers you might find interesting:

• Coeckelbergh, M. (2011). Talking to robots: On the linguistic construction of personal human-robot relations. Human-robot personal relationships (pp. 126-129) Springer.

• Kahn Jr, P. H., Kanda, T., Ishiguro, H., Freier, N. G., Severson, R. L., Gill, B. T., Ruckert, J. H., Shen, S. (2012). “Robovie, you'll have to go into the closet now”: Children's social and moral relationships with a humanoid robot. Developmental Psychology, 48(2), 303.

• Severson, R. L., & Carlson, S. M. (2010). Behaving as or behaving as if? Children’s conceptions of personified robots and the emergence of a new ontological category. Neural Networks, 23(8), 1099- 1103.

Boston isn't cheap—so how does a poor student make do?

You're a grad student living in one of the most expensive cities in the country. Living can and is often expensive. Not sure what corners to cut? Read on.

Food

1. Learn to cook. You don't have to cook complicated and fancy dishes. Learn how to make pasta. Learn how to make oatmeal. Maybe stir-fry. Steamed vegetables? Smoothies? Sandwiches? Salads? There are a lot of simple things you can cook at low prices. Check out food blogs like Budget Bytes for ideas. This will, incidentally, allow you to eat out less often, cutting costs there too!

2. Get your grocery store's discount card, if it offers one. Sure, the grocery store will be able to track your purchases... but your food will be cheaper. Make sure to get the coupons they mail out, too. Coupons, you ask? Aren't those a bit old-fashioned? Don't diss coupons—I got a free gallon of ice cream once! Also, if there's a produce store around, find it—it'll often be way cheaper than produce at a big grocery store. For example, if you live out toward Medford or Somerville, try Roberto's.

3. When faced with two options at the grocery store.... pick the cheaper one. When looking down the snack food aisle, think to yourself, "Should I spend $5 on a bag of chips... or $5 on two or three pounds of delicious apples?" Did you know that bags of dried beans are at least two-thirds cheaper than canned beans? But then there's this tradeoff between time and money. Do you buy the cheaper option, like dried beans instead of canned, but spend more time cooking them? Or is time spent boiling beans actually equivalent to the time spent wrangling cans with your old busted can opener? (Maybe that's just me. My fiancé has an old Russian can opener from WWII, which we use whenever we remember that the normal can opener doesn't work that well.)

Not Food

1. Utilities. If you pay for heat, turn down the thermostat by a couple degrees and put on a sweater. If you pay for electricity, remember to turn off the lights. If you pay for water, take a shorter shower. Fairly straightforward.

2. In the transportation realm: Walk when you can. Did you know there's usually only a mile or less between T-stops? If you don't have a monthly T-pass, walk if you only have one stop to go! If you do have a monthly pass, make sure to take advantage of MIT's partially subsidized train and bus passes. Combine errands and other trips out so you spend less time and money on transportation, even if means you carry a bit more stuff at once.

3. Get a laundry drying rack. Instead of spending money on a dryer, drape your clothes over the rack. They'll dry themselves! This works better in warm weather or if the drying rack is set up near a heater. Be careful not to set your clothes on fire, though.

4. Buy used stuff. Shop at thrift stores and dollar stores. You can find great deals on furniture, clothes, silverware that doesn't match, kitchen and cleaning supplies, decent quality dishes, and much more.

5. Be inventive! Buy less stuff in general. Need an end table? Have a box of random junk, books, computer parts, or summer clothes that you have to store anyway? Take the box. Set it next to your couch. Drape a nice-looking piece of fabric or a blanket over it. Bam! End table and storage, all-in-one! Even Ikea can't beat that. Don't have a good desk chair? Take a pile of textbooks, stack them on your desk, and make it an ergonomic standing desk instead!

—

This article originally appeared in the MIT Graduate Student Council publication The Graduate, February 2013

So, what do new grad students need to know?

I'm a new graduate student.

As such, I just spent the past week being properly oriented for the journey I'm about to undertake. It'll be (in the words of various orientation presenters) amazing, hard, depressing, enlightening, enriching ... basically, a grab bag of adjectives! In between the heartwarming-if-cliche welcome speeches, excited conversations with fellow newbies, and getting lost in the tunnels under MIT, I'd like to think I picked up some useful tidbits of information.

Expectations and communication

The biggest thing is to communicate. Surprise! Who would've thought that the key to successfully working with your colleagues, classmates, labmates, and advisor would be to communicate with them? The top three pieces of advice:

1. Tell your advisor/classmates/colleagues what to expect of you.
2. Ask what to expect of your advisor/classmates/colleagues.
3. Be your own advocate.

For example, if you run marathons and thus go for a long run every day at noon, tell your advisor and labmates this. That way, they don't expect to find you in the lab when you're out running. They might tell you that they have three kids and leave work every day at 6pm sharp -- so don't schedule meetings after 5pm. Or that they're so not a morning person, so never expect to see them working before noon -- but if you need something at 3am, they're the person to contact.

It's not just about when to expect to see people in the lab. Ask about communication styles. Does this person like emails? Phone calls? Meetings? Texts? Some people prefer a quick five-minute conversation in person to a lengthy email exchange. Ask what this person's expectations are about you. Does your advisor expect to see you in the lab eight hours a day? Does your labmate expect you to help out on project XYZ? Ask questions whenever you're unsure of something. After all, every relationship is different. So what works for this relationship?

The key is to share enough relevant information with each other to know what to expect. Be up front about who you are, what you do with your time, and what you want to get out of the situation or the relationship. This way, no one's left wondering. If everyone knows what to expect, you won't get into a situation where someone's upset because they didn't get what they were expecting.

Communicate both when things are going well and when they're not. If you're working on a project with someone, give regular updates on your progress -- whether you've achieved awesome results, or are stuck in a rut. Sometimes, the person you're working with can help you out of the rut. I worked with someone once who said, if you don't update me, I'll assume you're not working. While that's not true of everyone, make sure the relevant people know what you're up to.

If you remember one thing, remember this: People assume too much. People will build up their own image of you whether or not you tell them anything. So be proactive. Be your own advocate. Make sure they build up an image that correctly reflects reality.

Other advice

1. Leave your lab. Make a point of getting out of your lab, out of your department, and meeting people. Meet people from everywhere! You can meet people through campus-wide events, lectures, your classes, clubs, outside activities... pretty much anywhere there are people, really.

2. Leave your comfort zone. Try new things. Try hard things. Learn.

3. It'll be hard, but that's okay. The orientation events I attended had a common theme—grad school is hard. Grad school is supposed to be hard. You may not be motivated every step of the way. The key is persistence and perseverance. Find ways of keeping yourself on track. And:

4. Take care of yourself. Don't put the rest of your life on hold. Leave the lab once in a while. Do outside activities—whether that's walking your dog, spending time with your family, or backpacking in Kenya. What do you enjoy besides your research? Make time for it. It'll help keep you sane.