Uncertainty, Confidence and the Lens Model
This year several graduate students and I have been engaged in a variety of projects related to decision making.
Richard Finger completed an M.S. thesis in which he investigated the use of degraded and blended icons to convey uncertainty regarding an object's identity. The thesis consisted of two studies in which icons were used to convey the probability that the identity of a radar contact was hostile or friendly.
A pilot study first investigated whether participants could sort, order, and rank icons from seven sets intended to represent different levels of uncertainty. For example, one set used an "X" to indicate a 100 percent chance of the entity being hostile, and an "0" to indicate the opposite. As the chance approached 50 percent, the icons used were more and more distorted (made more "fuzzy" using a pixelating function in a graphics package).
Results from the pilot study indicated that sets of degraded or blended icons intended to represent levels of situational uncertainty could be ordered and rated in a manner similar to expectations. Additional results from the pilot study indicated a framing effect on performance: Participants' interpretations of displayed information became less ideal in a negatively framed context (that is, when those icons were described as representing hostile rather than friendly entities). Finally, results from the pilot study were similar across icon sets, indicating that experimental results were not necessarily specific to a particular icon form.
Three icon sets were selected for further study in a decision making experiment, in which participants had to identify objects as either hostile or friendly. Participants saw a simulated radar screen in which unidentified contacts and probabilistic estimates of their identities were depicted in one of three ways: with degraded icons and probabilities, with non-degraded icons and probabilities, and with degraded icons only.
Results showed that participants using displays with only degraded icons performed better on some performance measures, and as well on other measures, than conditions where degraded icons were annotated with numeric probabilities, where information regarding uncertainty was conveyed only via numeric probabilities, or where numeric probabilities were mapped to the icons in the task instructions.
These results are significant because they indicate both that people are able to understand uncertainty conveyed through such a manner and, thus, that the use of distorted or degraded images may be a viable alternative to convey situational uncertainty.
This work will continue through both an NSF-funded grant, and an association with the Center for Multi-source Information Fusion at the University of Buffalo.
Chang-soo Nam also completed an M.S. thesis in which he studied the relationship between task characteristics and confidence in judgments. Nam used a pavement judgment task, in which participants had to make a decision regarding a type of repair given characteristics and a photograph of a pavement crack. Participants were provided with cognitive feedback regarding their performance across four sessions.
A common effect in studies of confidence is that participants tend to be more overconfident in their judgments (that is, more confident than their performance indicates) on tasks that are more difficult. This result was confirmed in the present study - but in this case, assessments of task difficulty were made a priori, and objectively, by creating experimental conditions across which judgment predictability (Re) varied. That is, tasks were grouped into two conditions with high and low Re. Additional analyses indicated that participants who were more overconfident had lower values of G (linear knowledge) indicating poorer adaptability to the linear structure of the environment than those who were not overconfident, but had similar levels of consistency (Rs).
One interpretation of these results is that participants who are overconfident may be reflecting, in their self-assessments of confidence, their belief that they are making judgments consistently.
Additionally, two other students are pursuing lens-model related work. Gordon Gattie is beginning data collection for a study applying aspects of cognitive feedback to train dental students in classifying oral cancers, and Pratik Jha is applying the multi-variate lens model to describe fault diagnoses in process control.
Contact Ann Bisantz