How Haptic Technology is Bringing Cell Research to our Fingertips

One student standing in a yellow UAlbany shirt watches another student mirror movements from gloves on a laptop screen indoors.
Students Isaiah Williams and Liz Kidd experiment with haptics gloves in the CEHC Makerspace at ETEC. (Photo by Aishwari Talhan)

By Michael Parker

ALBANY, N.Y. (March 9, 2023) — From the buzzing of our mobile phones to the clicking feedback as we type our text responses, haptics serve an increasingly sophisticated role in how we interact with modern technologies.

Now, researchers at the University at Albany, the University of Illinois Urbana-Champaign and the University of Iowa are collaborating on a project that will provide students a way to engage on a tactile level with a biological cell.

The project, “Collaborative Research: Haptic Technology Innovations to Guide Student Gesturing with Dynamic STEM Visualizations,” is providing $850,000 through the National Science Foundation (NSF) to create haptic tools for use in educational contexts. The grant is funded by the NSF’s Research on Emerging Technologies for Teaching and Learning (RETTL) program.

"Haptics has been considered as important for many fields such as surgical training, but its role for education has not been fully understood,” said Vice President for Research and Economic Development Thenkurussi (Kesh) Kesavadas, who serves as co-principal investigator on the project and has been working in the field of haptics for over 25 years.

“This project aims to determine how sensory feedback delivered to students’ hands helps them understand the meanings of pictures, graphs, and other representations displayed in computer simulations,” he added.

UAlbany Vice President for Research and Economic Development Thenkurussi (Kesh) Kesavadas
Vice President for Research and Economic Development Thenkurussi (Kesh) Kesavadas is serving as a co-PI on the haptics project. (Photo by Patrick Dodson)

The researchers are designing and developing a custom haptic glove that interacts with a computer visualization of a biological cell.

Aishwari Talhan, who recently joined UAlbany as a research scientist after completing her postdoctoral work at McGill University, is working on the haptics portion of the project at UAlbany.

“Unlike past research on haptics, our project aims to structure learning by guiding students to think about the meaning of what their hands feel when they move across the visualization of the cell,” said Talhan.

The goal for the UAlbany project team will be to develop the haptic glove that will deliver force feedback and guide students’ gestures as they interact with the simulation.

The context of biology is important to the project because it situates the learning within STEM disciplines where relatively little research exists on how haptics can impact the student academic experience.

“Many past studies have demonstrated that using the body and hands to learn can help in similar science contexts, but this project aims to additionally test the claim that the body itself helps students learning,” said Kesavadas.

This is accomplished by using a novel neuroimaging method (functional Near Infrared Spectroscopy or fNIRS), which allows students to move their bodies while learning and then determine if motor areas of the brain remain active during later problem solving.

“The project delivers added value to science education because it will determine when and how specific hand movements such as gestures, combined with feedback from the haptic glove enable science learning above and beyond gestures or sensory feedback (i.e., touch) alone,” said Kesavadas.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

“Right now, haptics is a little known and underused technology for most people in education,” said Robb Lindgren, an associate professor of Curriculum & Instruction and Educational Psychology at Illinois. “If anything, people think of it as the bells and whistles of certain video games that can make them more exciting. They’re not thinking about how can I potentially use this technology to help my students understand things?”

Lindgren’s role on the project is to design and develop a biochemistry simulation that undergraduate students will interact with using a haptic glove. The specific learning objective of the simulation is understanding the multiple forces that determine whether and how many ions cross the cell membrane — a type of phenomenon referred to as “dynamic equilibrium.”

Meanwhile at Iowa, a research team led by Assistant Professor of Educational Psychology and Learning Sciences Matthew Lira will study the learning and neuropsychological effects of students using the technologies. This includes using the fNIRS technology.

“I have witnessed first-hand how even the best performing students fail to understand electricity in the body. I was one of them once. That’s what motivates me to do this work,” said Lira. “The learning sciences has so much to offer in encouraging students to use their bodies and innovative technologies to learn. But we cannot just hand them to students, we have to understand how students are most likely to interpret feedback from visual or haptic technologies.”