Emergency and Risk Communication Message Testing Laboratory

From backchannels on the front lines to the siren in your pocket, we can build better warnings, one message at a time.

About

The Emergency and Risk Communication Message Testing (ERC) Laboratory is an interdisciplinary laboratory within the College of Emergency Preparedness, Homeland Security and Cybersecurity at the University at Albany.

 

What We Do

At the ERC Laboratory, we conduct applied research with a focus on short fuse, imminent threat hazards. Using a variety of methods, including interviews and focus groups, experiments and surveys, and user interaction, we identify strategies to increase attention, comprehension, and behavioral intent when action can’t wait.

 

Why We Do It

Effective emergency and risk communication messaging can make the difference between life and death. Well-constructed messages have the potential to alert individuals and persuade them to take timely action; poorly constructed messages can lead to confusion and information seeking when moments matter.

 

Funding and Outcomes

The funding for research conducted in this lab comes from the National Science Foundation (NSF), the United States Geological Survey (USGS), the Federal Emergency Management Agency (FEMA), and the National Oceanic and Atmospheric Administration (NOAA). Findings from our research have been implemented by local, state, and federal agencies including NOAA, USGS, the Federal Emergency Management Agency (FEMA), and the Centers for Disease Control and Prevention (CDC).

The Primary Goals of ERC

To improve the communication of imminent threat messages by:

  • applying knowledge from sociological and communication theories
  • measuring attention, perception, and behavioral intent
  • translating findings to practical approaches for risk communicators
Natural Disasters
This file is licensed under the Creative Commons Attribution-Share Alike 4.0 International license.
Contact ERC Lab
ETEC, 3rd Floor
Harriman Campus

1200 Washington Avenue
Albany, NY 12226
United States

Methods

In the ERC Lab, we approach the study of alerts and warnings as an applied research question bound directly to prior empirical research on warning response (Mileti & Sorensen), and theories derived from persuasive communication such as fear appeals (Rogers, 1975), linked to decision making (Lindell and Perry, 2012) and risk information seeking (Dunwoody & Griffin 1999) to test message effectiveness for imminent threat hazards. ​

Message testing studies may include measures of prior experience, risk perception (affect, susceptibility, exposure, and severity), message understanding, message believing, message deciding, message personalization, information seeking, self-efficacy, response efficacy, and behavioral intent. Investigations may also include think-aloud interviews, focus groups, and bio-physiological measures including eye-tracking.  ​

​The focus of message testing includes the visual display of information (the style of the message including the use of colors); the message text (content, information being conveyed within the message), and how information is organized (the structure of the message, including the use of text, visual, and audio information).

Eye Tracking Lab

Eye tracking analysis is used by researchers to understand how messages are being used or interpreted by audiences. Eye movements can be detected using eye-tracking technology, measuring pupil size, gaze patterns, attention to stimuli, and duration of attention. Attention to visual information can serve as an indicator of an area of interest and/or more complex processing of information.​

Eye-tracking is especially helpful in understanding how to best design graphics displayed for decision making, such as by identifying areas of interest that result in initial or sustained attention from the viewer. When coupled with think-aloud interviews and post-session surveys or cognitive recall, investigators can identify parts of messages that result in greater comprehension, leading to behavioral intent, or confusion, resulting in information seeking and delayed action.

Our equipment

The ERC Lab includes two eye-tracking stations using Tobii Pro Fusion 120Hz Hardware

Usability Lab

The Usability Lab is a shared lab resource in ETEC for the research activities of Carol Anne Germain and Jeannette Sutton. It includes a desktop computer and a wall-mounted large-screen display, and AV to collect audio and video data from multiple angles in the observation room during usability testing. The desktop computer facilitates usability activities such as interaction and search online; the wall-mounted large-screen display can represent larger, interactive digital spaces such as watching television or engaging in gaming activities conducted on large screens.

Research

Message Design Dashboard for Wireless Emergency Alerts

ERC MDD Text Alert

Hazardous events, ranging from toxic materials leaks to mass casualty incidents, and seasonal weather events compounded by changing climate conditions and crumbling infrastructure, pose a significant risk of harm to populations around the globe. Communication of warnings, potential impacts, and the protective actions that can be taken to reduce injuries or prevent loss of life is vital at the onset of potential disaster. Governments at all levels have recognized the need to communicate to their constituents, resulting in a nationwide policy: the Warning, Alert, and Response Networks Act which led to the development of the Wireless Emergency Alert (WEA) System and the coordination of participating wireless carriers to deliver alerts and warnings to mobile devices to those at risk. This nationwide approach means that regardless of location, alerts and warnings can be delivered to those at risk. Technological advances such as WEA have emphasized and prioritized capabilities to broadcast geo-targeted emergency alerts to populations throughout the United States. Research on human perceptions, behavioral intent, and protective actions taken in response to these mobile-delivered messages is limited.

Collaborating with CTG, this project will address this gap by developing a tool - the Message Design Dashboard (MDD) - to help emergency managers write effective messages for public alert and warnings, as well as develop presentations, training materials, and workshops to educate alerting authorities on developing effective public alerts and warning messages. At the conclusion of the MDD design, project team members will conduct training seminars/workshops at events and conferences as directed by FEMA.

 

Researchers

 

Collaborators

Michelle Wood, Professor in the Department of Health Science at Cal State Fullerton

 

Funding

Federal Emergency Management Agency - Integrated Public Alert & Warning System (FEMA-IPAWS)

 

Publications

Alerts and warnings on short messaging channels: Guidance from an expert panel processSutton, J. & Kuligowski, E. (2019). Natural Hazards Review. 20 (2).

Sutton, J., & Kuligowski, E.(2019). Guidance on Creating Alerts and Warnings for Short Messaging Channels.

Sutton, J., & Kuligowski, E. (2019). Alerts and warnings on short messaging channels: Guidance from an expert panel process. Natural Hazards Review. 20(2)

Bean, H., Liu, B. F., Madden, S., Sutton, J., Wood, M., & Mileti, D. (2015). Disaster warnings in your pocket: A qualitative study of how audiences interpret wireless emergency alerts. Journal of Contingencies and Crisis Management, 24(3), 136-147.

Bean, H., Liu, B., Madden, S., Mileti, D., Sutton, J. and Wood, M. (2014) Comprehensive testing of imminent threat public messages for mobile devices. National Consortium for the Study of Terrorism and Responses to Terrorism (START). College Park. 

DeYoung, S.E., Sutton, J., Nichols, K., Farmer, A., and Neal, D.  (2019). “Death was not in the agenda for the day”: Emotions, behavioral reactions, and perceptions in response to the 2018 Hawaii Wireless Alert. International Journal of Disaster Risk Research. 36, 101078. 

Wood, M., Mileti, D., Bean, H., Liu, B. F., Sutton, J., & Madden, S. (2017). Milling and public warnings. Environment and Behavior.

Sutton, J., & Kaufmann, R. (2017). Designing imminent threat messages for an unfamiliar hazard. Communication Teacher.


 

Forecasting and Communication of Extreme Heatwaves in NYC 

UAlbany Heat Index NYC

When one imagines deadly weather conditions, hurricanes, snowstorms, or tornadoes are generally the first phenomena to come to mind. However, extreme heatwaves (and extreme temperature in general) are significantly more deadly on an annual basis.  Improving the accuracy and communication of extreme temperature products is vital to reducing the impacts of extreme heat on vulnerable populations in urban settings.  

Working with collaborators in the UAlbany Center of Excellence and the Atmospheric Science Research Center, Co-PIs Sutton and Germain will conduct usability testing in the ERC lab leading to the design, development, and testing of a static webpage that will provide decision-aids for National Weather Service core partners to effectively communicate risk to vulnerable populations.

 

Researchers

 

Collaborators

  • Nick Bassill, Principal Investigator
  • Chris Thorncroft
  • Eric Stern

 

Funding

National Oceanic and Atmospheric Administration (NOAA)

 


 

Eye Tracking the Storm

UAlbany display featuring Mesonet and advanced weather predictor database

The goal of a tornado warning message is to alert individuals to impending threats and offer guidance on the actions to take to reduce harm and loss. However, warning messages must compete for attention in a busy information environment and a single warning message may serve as the sole alert to persuade a person that the threat is real and danger is imminent. Limited research has been conducted on how to best display messages on newer technologies, especially those that include the technological affordances for graphics and other visual elements. Evidence-based research can provide guidance on how to maximize the effectiveness of messages that communicate visual risk information. ​

In this project, we investigate how members of the public attend to and interpret warning messages that include visual imagery. We conduct mixed-methods, experiments using a combination of eye-tracking, think-aloud interviews, and post-test survey to investigate how individuals attend to, perceive, and process visual risk information. When combined with robust theory, interviews, and self-reported survey items, eye tracking allows us to measure behavioral outcomes of cognition and decision making, including the nature of information seeking and processing which leads to self-protective actions under conditions of threat.

 

Researchers

Jeannette Sutton, Principal Investigator

Laura FischerCo-Principal Investigator

 

Funding

National Oceanic and Atmospheric Administration (NOAA) and Vortex-SE

 

Publications

Sutton, J., Fischer, L. M., & Wood, M. M. (2021). Tornado Warning Guidance and Graphics: Implications of the Inclusion of Protective Action Information on Perceptions and Efficacy. Weather, Climate, and Society.  

Sutton, J., & Fischer, L. (2020). Understanding visual risk communication messages: An analysis of visual attention allocation and think aloud responses to tornado graphics. Weather, Climate, and Society. 13 (1) 173-188.


 

Public Responses to the 2019 Ridgecrest Earthquake Sequence and ShakeAlert

Earthquake heat map of Ridgecrest, California
USGS. (2020). M 7.1—2019 Ridgecrest Earthquake Sequence. USGS Earthquake Hazards Program. 

Studies of protective actions have been undertaken for a range of hazards that have significantly longer lead times/forewarning than an earthquake including hurricane, tsunami, flood, tornado, and technological hazards such nuclear threat and chemical spill. Research on protective actions in response to earthquake events has been conducted internationally (New Zealand, Japan) but little attention has focused on those behaviors in the United States (Nakayachi, Becker, Potter, & Dixon, 2019) . ​The novelty of earthquake early warning (EEW) in the US, and its ability to warn populations at risk, as well as the infrequency of shaking events that would trigger the system, has meant most ShakeAlert research to date have been experiments (see for example, Sutton, Fischer, James, & Sheff, 2020). Recent earthquake events have prompted the delivery of earthquake alerts providing the first opportunity of its kind to collect data on public perceptions and behavioral responses prompted from a warning message.    ​

This research investigates how people made sense of environmental cues, such as felt earth movement, and social cues, including the alerts received, to make decisions about taking protective action. We draw from theories of warning response (Mileti & Sorensen, 1990; Sutton et al., 2020; Wood et al., 2018), risk perception, and trust in automated systems to measure individual perceptions and other outcomes in response to EEW messages.  ​

Data are collected using two methods. First, we conduct a series of qualitative interviews with persons who experienced the Ridgecrest earthquakes in 2019 followed by a false earthquake alert in 2020. Second, using a representative panel of residents in Southern California, we collect survey data about public responses to earthquakes and earthquake early warnings.

 

Researchers

 

Funding

National Science Foundation (NSF)

 

Publications

McBride, S. K., Bostrom, A., Sutton, J., de Groot, R. M., Baltay, A. S., Terbush, B., Bodin, P., Dixon, M., Holland, E., Arba, R., Laustsen, P., Liu, S., & Vinci, M. (2020). Developing post-alert messaging for ShakeAlert, the earthquake early warning system for the West Coast of the United States of America. International Journal of Disaster Risk Reduction, 101713.


 

Earthquake Hazard Program (EHP) Message Testing

emergency alert notification on cellphone device

The design of earthquake early warning (EEW) messages is a priority research area for the USGS and ShakeAlert community. Although EEW provides a tremendous opportunity to reduce human, property, and economic losses, critical research gaps remain. Despite decades of research about warnings and human response following the onset of earthquake shaking, little research has been conducted on the unique context of EEW. Understanding the best ways to craft EEW messages to motivate people to take action is essential to realizing the potential benefits afforded by ShakeAlert and other EEW systems. ​

ShakeAlert technology is currently limited to sending a single, identical message to all recipients, regardless of users’ location or situation despite research demonstrating that personalized, detailed messages are more effective at motivating protective action than protective actions appropriate for more general warnings. The purpose of this study is to examine the benefit of including additional specificity in EEW messages by including information about the earthquake epicenter, countdown to shaking arrival, and anticipated shaking intensity.​

This research will use a mixture of focus groups and online experiments to measure the effects of message specificity on perceptions and behavioral intent in response to earthquake early warnings.

 

Researchers

 

Funding

United States Geological Survey (USGS)

 

Publications

Sutton, J., Fischer, L., James, L. E., & Sheff, S. E. (2020). Earthquake early-warning message testing: Visual attention, behavioral responses, and message perceptions. International Journal of Disaster Risk Reduction, 101664.

Sutton, J., Vos S. C., Wood, M., & Turner, M. (2018). Designing effective tsunami messages: The role of short messages and fear in warning response. Weather Climate and Society. 10(1), 75-87.


 

COVID-19

A collage of images representing the COVID-19 pandemic
Blue Planet Studio/Shutterstock.com

Public health and emergency management agencies are on the front lines of informing and educating the public about the science of virus transmission and prevention. In response to a threat such as COVID-19, their mission requires them to communicate accurate and credible information to local populations using all means of information delivery.  Increasingly, social media is a critical component of this communication toolbox but using it rapidly and effectively to  inform the public in a crowded media environment remains a significant challenge.

In prior work on online communication in the recent Zika and Ebola outbreaks, we established that effective messaging depended upon employing a combination of content, style, and structure features but that the right mix seemed to depend upon properties of the disease event (including the uncertainty and ambiguity of the threat, the nature of the consequences involved, and the need for public information). COVID-19 poses a distinct risk profile, with a disruption potential to the American public and the built environment not seen by any threat within decades. In this project, we propose to identify the key drivers of effective messaging in an emerging pandemic, and specifically to rapidly identify strategies for improving effectiveness in social media communication involving COVID-19 by public agencies.

Our specific focus will be on the outcomes of message retransmission (an essential outcome for both high levels of message penetration and ensuring the multiple exposures needed for behavioral influence) and engagement (a critical indicator of attention and a driver of trust), both of which are measurable and established as core outcomes in prior studies of effective social media communication.  By establishing evidence-based guidance for agencies to effectively warn, inform, and engage with the general public during an emerging pandemic, we will provide the tools they need to mount effective interventions that save lives, reduce economic losses, and protect the security of the nation against health threats, in alignment with the broader mission of the NSF.​

 

Researchers

  • Jeannette Sutton, Principal Investigator
  • Carter Butts, Co-Principal Investigator

 

Funding

National Science Foundation (NSF)

 

Publications 

Renshaw, S. L., Mai, S., Dubois, E., Sutton, J., & Butts, C. T. (2021). Cutting Through the Noise: Predictors of Successful Online Message Retransmission in the First 8 Months of the COVID-19 Pandemic. Health security, 19(1), 31-43.​

Sutton, J., Renshaw, S., & Butts, C. T. (2020).  COVID-19: Retransmission of official communications in an emerging pandemic.  PLOS ONE 15(9): e0238491.  ​

Sutton, J., Renshaw, S., & Butts, C. T. (2020).  The first 60 days: American public health agencies’ social media strategies in the emerging COVID-19 pandemic. Health Security. 18(6). DOI: 10.1089/hs.2020.0105

Sarah C. Vos, Sutton, J., Yue Yu, Scott Leo Renshaw, Michele K. Olson, C. Ben Gibson, and Carter T. Butts (2018). Retweeting Risk Communication: The Role of Threat and Efficacy. Risk Analysis, Vol. 38, No 12.

Our Team

The research team at the ERC Lab comprises scholars with expertise and interests in sociology, information science, public administration, communication, cybersecurity, and library science. Research collaborators come from the fields of public health, agricultural communication,  mathematical sociology, and atmospheric and geological sciences. We welcome inquiries about lab affiliations and research assistantships.

 

Michele Olson

Senior Researcher & Project Manager, Emergency and Risk Communication Message Testing Laboratory

Michele ("Micki") Olson is a senior researcher and project manager in the Emergency and Risk Communication Message Testing Laboratory at the University at Albany. She has over 10 years working as a communication researcher, with expertise in risk communication, alerts and warnings, and persuasive message design.

She applies concepts, theories, and methods from the communication discipline to better understand and design warning messages, including Wireless Emergency Alerts (WEAs) and risk communication on social media. She uses both quantitative and qualitative methods in her work, including surveys, experiments, interviews, focus groups, and content analysis. Her research has resulted in over 60+ conference presentations at regional, national, and international conferences and numerous publications, fellowships, and awards.

Previously, Olson worked as a social scientist in the National Oceanic and Atmospheric Administration's (NOAA) Weather Program Office, where she provided technical and scientific leadership in social science, risk and weather communication, and qualitative and quantitative methods. She focused on translating social science interdisciplinary research findings to non-expert audiences, evaluating qualitative and quantitative social science research protocols, and providing project management level oversight and scientific guidance in social science research funding.

[email protected] | LinkedIn

Michele Olson

 

Graduate Research Assistants

Nick Waugh

Research Assistant, Ridgecrest & EHP

Nick Waugh is a second-year graduate student in the College of Emergency Preparedness, Homeland Security & Cybersecurity where he is pursuing a Master's in Information Science -  Cybersecurity. He is a research assistant on NSF-funded research investigating wireless emergency alerts and disasters.

[email protected] | LinkedIn | CV

Nick Waugh
Savanah Crouch

Research Assistant, Ridgecrest & EHP

Savanah Crouch is a second-year graduate student in the College of Emergency Preparedness, Homeland Security & Cybersecurity where she is pursuing a Master's in Information Science -  School Libraries. She is a research assistant on NSF-funded research investigating wireless emergency alerts and disasters.

[email protected]

Savanah Crouch
Elisabeth Dubois

Research Assistant, COVID-19 Project

Elisabeth Dubois is a fourth-year Information Science Ph.D. student in the College of Emergency Preparedness, Homeland Security, and Cybersecurity. She completed a B.S. in Digital Forensics and an MBA from the University at Albany, in 2018 and 2020 respectively. Elisabeth is a research assistant on NSF-funded research investigating message transmission during COVID-19.

[email protected] | LinkedIn

Elisabeth Dubois_ETEC

Projects & Publications

Publications

FEMA Wireless Emergency Alerts

Alerts and warnings on short messaging channels: Guidance from an expert panel processSutton, J. & Kuligowski, E. (2019). Natural Hazards Review. 20 (2).

Sutton, J., & Kuligowski, E.(2019). Guidance on Creating Alerts and Warnings for Short Messaging Channels.

Sutton, J., & Kuligowski, E. (2019). Alerts and warnings on short messaging channels: Guidance from an expert panel process. Natural Hazards Review. 20(2)

Bean, H., Liu, B. F., Madden, S., Sutton, J., Wood, M., & Mileti, D. (2015). Disaster warnings in your pocket: A qualitative study of how audiences interpret wireless emergency alerts. Journal of Contingencies and Crisis Management, 24(3), 136-147.

Bean, H., Liu, B., Madden, S., Mileti, D., Sutton, J. and Wood, M. (2014) Comprehensive testing of imminent threat public messages for mobile devices. National Consortium for the Study of Terrorism and Responses to Terrorism (START). College Park. 

DeYoung, S.E., Sutton, J., Nichols, K., Farmer, A., and Neal, D.  (2019). “Death was not in the agenda for the day”: Emotions, behavioral reactions, and perceptions in response to the 2018 Hawaii Wireless Alert. International Journal of Disaster Risk Research. 36, 101078. 

Wood, M., Mileti, D., Bean, H., Liu, B. F., Sutton, J., & Madden, S. (2017). Milling and public warnings. Environment and Behavior.

Sutton, J., & Kaufmann, R. (2017). Designing imminent threat messages for an unfamiliar hazard. Communication Teacher.

 

Earthquake & Tsunami Messaging 

McBride, S. K., Bostrom, A., Sutton, J., de Groot, R. M., Baltay, A. S., Terbush, B., Bodin, P., Dixon, M., Holland, E., Arba, R., Laustsen, P., Liu, S., & Vinci, M. (2020). Developing post-alert messaging for ShakeAlert, the earthquake early warning system for the West Coast of the United States of America. International Journal of Disaster Risk Reduction, 101713.

Sutton, J., Fischer, L., James, L. E., & Sheff, S. E. (2020). Earthquake early-warning message testing: Visual attention, behavioral responses, and message perceptions. International Journal of Disaster Risk Reduction, 101664.

Sutton, J., Vos S. C., Wood, M., & Turner, M. (2018). Designing effective tsunami messages: The role of short messages and fear in warning response. Weather Climate and Society. 10(1), 75-87.

 

HEROIC/ Twitter/ COVID Messaging

Renshaw, S. L., Mai, S., Dubois, E., Sutton, J., & Butts, C. T. (2021). Cutting Through the Noise: Predictors of Successful Online Message Retransmission in the First 8 Months of the COVID-19 Pandemic. Health security, 19(1), 31-43.​

Sutton, J., Renshaw, S., & Butts, C. T. (2020).  COVID-19: Retransmission of official communications in an emerging pandemic.  PLOS ONE 15(9): e0238491.  ​

Sutton, J., Renshaw, S., & Butts, C. T. (2020).  The first 60 days: American public health agencies’ social media strategies in the emerging COVID-19 pandemic. Health Security. 18(6). DOI: 10.1089/hs.2020.0105

Sarah C. Vos, Sutton, J., Yue Yu, Scott Leo Renshaw, Michele K. Olson, C. Ben Gibson, and Carter T. Butts (2018). Retweeting Risk Communication: The Role of Threat and Efficacy. Risk Analysis, Vol. 38, No 12.

 

VORTEX 

Sutton, J., Fischer, L. M., & Wood, M. M. (2021). Tornado Warning Guidance and Graphics: Implications of the Inclusion of Protective Action Information on Perceptions and Efficacy. Weather, Climate, and Society.  

Sutton, J., & Fischer, L. (2020). Understanding visual risk communication messages: An analysis of visual attention allocation and think aloud responses to tornado graphics. Weather, Climate, and Society. 13 (1) 173-188.

 

Presentations

Calls to action in short warning messages: A comparative study of 3 hazards. A presentation from the AMS Conference by Jeannette Sutton, Laura Fischer, and David Huntsman in 2022.

Communicating about Extreme Heat: A Usability Study. A presentation from the AMS Conference by Jeannette Sutton, Nick Waugh, and Carol Anne Germain in 2022. 

Writing Effective Warning Messages: The Message Design Dashboard Project. A presentation from the A&W Conference by Jeannette Sutton in 2022. 

Visual Risk Communication: Attention, Understanding, and Decision Making in Response to a Changing Tornado Scenario A recording from the AMS 5th Conference on Weather, Warnings and Communication by Dr. Jeannette Sutton on June 14, 2019

Eye-Tracking the Storm: The Effect of Variation in Presentation of Visual Risk Information A recording from the American Meteorological Society annual meeting, Phoenix, AZ by Dr. Jeannette Sutton on January 8, 2019

NOAA Seminar (April 2019) Dr. Jeannette N Sutton and colleague Dr. Erica Kuligowski discuss "Creating Alerts and Warnings for Short Messaging Channels"

Flash Briefing (March 14,2020) The Coronavirus, also known as COVID-19, has been declared a pandemic, which means there’s a worldwide spread of a new disease. Dr. Sutton speaks to Eric Singer from the PAST Fusion Cell at Argonne National Laboratory.

 

News

2022

2021

 

2020