Toward an Emotionally Intelligent AI

Google grant advances UAlbany professor’s work on automatic emotion recognition

CEAS Assistant Professor Yelin Kim has been awarded a highly competitive 2018 Google Faculty Research Award for her work in automatic emotion recognition. (Photo by Mark McCarty)

ALBANY, N.Y. (April 3, 2018) -- What if your computer could understand and adapt to your emotional state? That’s the subject at the heart of assistant professor Yelin Kim’s research: developing artificial intelligence systems that can better understand and analyze human behaviors, especially emotion.

Kim has just been awarded a highly competitive 2018 Google Faculty Research Award for her work in automatic emotion recognition. Her proposal, entitled “Towards Emotionally Intelligent AI Systems: Robust and Adaptive Multimodal Emotion Recognition,” aims to advance multimodal (audio-visual) emotion recognition, a technology that can provide emotional intelligence to AI systems.

The objective is to develop algorithmic and statistical methods for analyzing audio-visual human behavior, particularly focusing on emotional and social signals inferred from speech and facial expressions.

Her work has the potential to dramatically improve human-centered and interactive systems and transform  intelligent assistants (such as Google Assistant, Apple Siri, Microsoft Cortana, and Amazon Alexa) into systems that can understand what we are saying as well as our emotional state.

“An emotionally intelligent Google Home, for instance, could be used to help individuals better monitor their emotional landscapes over a long period of time,” said Kim, an assistant professor in the Department of Electrical and Computer Engineering at the College of Engineering and Applied Sciences (CEAS).

But the project has the potential to improve AI systems across the spectrum.

“In security, intelligent surveillance systems can detect irregular behaviors based on machine recognition of nervousness or anxiety,” continued Kim. “When it comes to marketing, emotion recognition technology can help providing more personalized and affect-aware services to its users based on their estimated emotion, attitude, and preference. And in education and entertainment, the technology can automatically analyze user engagement based on machine perception of joy, excitement, or boredom.”

Google received 1033 proposals covering 46 countries and over 360 universities for this round of funding. After expert reviews and committee discussions, Google ultimately funded 152 projects. The subject areas that received the most support this year were human computer interaction, machine learning, machine perception, and systems.

The Google Faculty Research Awards program is a competitive worldwide competition designed to support academic research that is aligned with Google's mission and provide both faculty and students the opportunity to work directly with Google researchers and engineers. The awards are for one year, and are structured as unrestricted gifts to universities to support the work of world-class full-time faculty members at top universities from around the globe.

Kim received her master’s degree and Ph.D. in Electrical Engineering-Systems from the University of Michigan, Ann Arbor, and her bachelor’s degree in Electrical and Computer Engineering from Seoul National University, South Korea.

She is the recipient of several awards, including a Faculty Research Award from UAlbany, and Best Student Paper Award from ACM Multimedia. She is the director of the University’s Inspire Lab (Interaction Sensing and Perception in Real Environment), where her primary research interests are in human-centered speech and video processing, affective computing, and multimodal signal processing.

RSS Link For more news, subscribe to UAlbany's RSS headline feeds

About the University at Albany
A comprehensive public research university, the University at Albany-SUNY offers more than 120 undergraduate majors and minors and 125 master's, doctoral, and graduate certificate programs. UAlbany is a leader among all New York State colleges and universities in such diverse fields as atmospheric and environmental sciencesbusiness, public health, health sciences, criminal justice, emergency preparedness, engineering and applied sciences, informatics, public administration, social welfare, and sociology taught by an extensive roster of faculty experts. It also offers expanded academic and research opportunities for students through an affiliation with Albany Law School. With a curriculum enhanced by 600 study-abroad opportunities, UAlbany launches great careers.