Article #26
Newsletter list 


Previous Article 

Next Article 


Judgment Analysis, Think-Aloud Protocols, Cause Mapping, Image Theory and Neural Network Simulation

Ray Cooksey
Armidale, Australia

In the past year or so, I have been involved in four different areas of Brunswikian research with various colleagues and students at the University of New England, as well as in Europe.

The first research study involved a fairly standard judgment analysis approach to understanding job applicant shortlisting judgments made by a university job selection panel. Shortlisting here was defined as the decision to allow an applicant to proceed to the interview stage of the selection process (otherwise the applicant is not considered further).

However, we coupled the standard judgment analysis methodology (using actual job applicants' curriculum vitae where the shortlisting outcome was known) with a think-aloud protocol methodology, applied on a judgment-by-judgment basis. This permitted us not only to model judgment policies using the cues established by university policy, but also to trace the potential dynamic intrusions of other information into the judgment process for each job applicant, both at the individual and group levels.

We are in the process of gathering data for a second judgment analysis investigation that also employs a think-aloud protocol method and qualitative cause mapping methodology. This time, we are examining university student judgments about engaging in risky sexual behavior (i.e., unprotected sex) using a series of representative scenarios (hypothetically generated using a broad-based survey of students to establish the 'population' parameters for sampling cue values and intercorrelations).

Part of this project is to get at students' subjective judgments of the risks associated with engaging in unprotected sex in a variety of different situations. Another goal is to tap into students' mental models regarding the factors they see as contributing to engaging in unprotected sex. Here is where we see the value of marrying judgment analysis, think-aloud protocols, and cause mapping methods in a coherent triangulated approach.

We are currently planning an investigation designed to provide insights into the dynamics of the commons dilemma, using a combination of judgment analysis methodology, image theory principles, and cause mapping methods. Here we plan to employ dynamic simulation software, specifically designed to run commons dilemma scenarios related to ocean fish harvesting, to look at fish harvesting judgments over time, made on the basis of exposure to key cue information about the resource pool and other external conditions and constraints. We also hope to map participants' mental models for the commons dilemma in terms of key factors that influence resource availability.

Finally, I am assisting a colleague, Hubert Bruins in the Department of Oral and Maxillofacial Surgery at the University Medical Center Utrecht, in his efforts to test a new methodology for judgment analysis. He has designed a neural network approach to judgment analysis that shows good potential for dealing with non-standard judgment tasks (such as those involving dichotomous judgments and categorical cues) where a multiple regression approach may be difficult to defend (i.e., in cases where assumptions are not met or where dynamic nonlinear relationships exist).

The specific judgment being modelled relates to the prophylactic extraction of teeth in patients with cancers of the neck and/or head prior to subjecting the patients to radiation therapy. The neural network approach models conditional probabilities as the basis for estimating cue-judgment and cue-cue relationships over a series of patient cases - the neural network is essentially grown and evolved as one progresses through judgments made on the patient cases. This study is currently being finalized in a paper to be submitted to Medical Decision Making.

Contact Ray Cooksey

Previous Article                Next Article

Newsletter list