Designing Conversational AI Media

Humans talk with humans. From our birth to death, we all talk with others everyday. I attempt to design socially intelligent conversational AI media to support and entertain human lives in the century.


ASTRO – Artificial Socio-Task Assistant Robot

Needs of Mixing Voice and Touch Inputs Voice user interface is now common, however, it’s not always the best way to achieve certain tasks. Users sometimes may want to take their initiative to explore information themselves through their preferable moderates. In this project, we explore mixed-initiative user experiences where a user collaborates with a conversational AI agent on a large touchscreen. In this project, we design a conversational user interface […]


HARP – Holographic Archive of Research Projects

Holographic Archive of Research Projects @ Carnegie Mellon University (HARP@CMU) is a digital library of several hundred past research projects from all schools and disciplines, guided by a socially-aware humanoid virtual agent, displayed on a large custom 360 holographic display, where students, faculty, and guests can explore the rich history of interdisciplinary academic research conducted at the university. Leveraging state-of-the-art social AI technologies that the ArticuLab, Language Technologies Institute, and […]


SARA – Socially Aware Robot Assistant

Socially Aware Robot Assistant SARA is a Socially-Aware Robot Assistant that interacts with people in a whole new way, personalizing the interaction and improving task performance by relying on information about the relationship between the human user and virtual assistant. Rather than taking the place of people, Sara is programmed to collaborate with her human users. Rather than ignoring the socio-emotional bonds that form the fabric of society, SARA depends […]


InMind – Rapport-Building Personal Assistant on Mobile Devices

Intelligent assistants of 2025 In 2014, Yahoo! and Carnegie Mellon University (CMU) have announced a five-year, $10 million partnership, called the InMind project. The goal of the InMind is to invent, implement, experiment, and iterate with an intelligent assistant for mobile devices. Major companies are already exploring intelligent assistants such as Siri, and Google Now, M, and Cortana, and we expect they will continue to develop these. To make an […]


Expressive Opinion Generation

Beyond the Cooperative Conversation Principle Let us begin by looking at an example of a question asked by one person and answered by another: A : “Do you have any favorite actor or actress?” B : “Yes, my favorite actress is Audrey Hepburn. Audrey is, just as one would expect, a charming and beautiful woman even in her private life!“ In this example, person A asked about person B’s favorite […]


Facilitation Strategies

Regulating A Four-Participant Conversation We present a framework for facilitation robots that regulates imbalanced engagement density in four-participant conversation as the forth participant with proper procedures for obtaining initiatives. Four is the special number in multiparty conversations. The three-participant conversation is the minimum unit where the participants autonomously organize a multiparty conversational situation. The fourth participant is the first person who can objectively observe the conversational situation. In three-participant conversations, […]


NANDOKU – Conversation Robots for Elderly Care Application

We proposed a robot that promotes an enjoyable party game as a facilitation robot system. In this task, a robot participates in a quiz game as one of participants and tries to promote the other participants’ enjoyment of the game. The functions implemented in the robot are: (1) The robot participates in the group’s communication using its basic group conversation functions; (2) The robot performs the game according to the […]

Design, Projects


Concept As a robotic platform for multiparty conversation facilitation, we present the design of SCHEMA robot. In order to participate in multiparty conversational situations, and be recognized as a ratified participant, a robot needs to have capabilities to exchange conversational protocols, which include organizing participation structure, transmitting messages, and turn-taking. Such protocols essentially needs robots’ embodied functions, including facial expressions, head gestures, and directional control of torso. Based on our […]