Toward an Emotionally Intelligent AI

Google grant advances UAlbany professor’s work on automatic emotion recognition

CEAS Assistant Professor Yelin Kim has been awarded a highly competitive 2018 Google Faculty Research Award for her work in automatic emotion recognition. (Photo by Mark McCarty)

ALBANY, N.Y. (April 3, 2018) -- What if your computer could understand and adapt to your emotional state? That’s the subject at the heart of assistant professor Yelin Kim’s research: developing artificial intelligence systems that can better understand and analyze human behaviors, especially emotion.

Kim has just been awarded a highly competitive 2018 Google Faculty Research Award for her work in automatic emotion recognition. Her proposal, entitled “Towards Emotionally Intelligent AI Systems: Robust and Adaptive Multimodal Emotion Recognition,” aims to advance multimodal (audio-visual) emotion recognition, a technology that can provide emotional intelligence to AI systems.

The objective is to develop algorithmic and statistical methods for analyzing audio-visual human behavior, particularly focusing on emotional and social signals inferred from speech and facial expressions.

Her work has the potential to dramatically improve human-centered and interactive systems and transform  intelligent assistants (such as Google Assistant, Apple Siri, Microsoft Cortana, and Amazon Alexa) into systems that can understand what we are saying as well as our emotional state.

“An emotionally intelligent Google Home, for instance, could be used to help individuals better monitor their emotional landscapes over a long period of time,” said Kim, an assistant professor in the Department of Electrical and Computer Engineering at the College of Engineering and Applied Sciences (CEAS).

But the project has the potential to improve AI systems across the spectrum.

“In security, intelligent surveillance systems can detect irregular behaviors based on machine recognition of nervousness or anxiety,” continued Kim. “When it comes to marketing, emotion recognition technology can help providing more personalized and affect-aware services to its users based on their estimated emotion, attitude, and preference. And in education and entertainment, the technology can automatically analyze user engagement based on machine perception of joy, excitement, or boredom.”

Google received 1033 proposals covering 46 countries and over 360 universities for this round of funding. After expert reviews and committee discussions, Google ultimately funded 152 projects. The subject areas that received the most support this year were human computer interaction, machine learning, machine perception, and systems.

The Google Faculty Research Awards program is a competitive worldwide competition designed to support academic research that is aligned with Google's mission and provide both faculty and students the opportunity to work directly with Google researchers and engineers. The awards are for one year, and are structured as unrestricted gifts to universities to support the work of world-class full-time faculty members at top universities from around the globe.

Kim received her master’s degree and Ph.D. in Electrical Engineering-Systems from the University of Michigan, Ann Arbor, and her bachelor’s degree in Electrical and Computer Engineering from Seoul National University, South Korea.

She is the recipient of several awards, including a Faculty Research Award from UAlbany, and Best Student Paper Award from ACM Multimedia. She is the director of the University’s Inspire Lab (Interaction Sensing and Perception in Real Environment), where her primary research interests are in human-centered speech and video processing, affective computing, and multimodal signal processing.

For more news, subscribe to UAlbany's RSS headline feeds