Professor Emeritus, Department of Psychology
University of Colorado
Now that the previous three
or four decades have made it obvious that judgment and decision making
can be studied, the primary methodological question that has emerged
over the past decade is whether the research methods conventionally employed
will permit the generalization of results beyond the laboratory conditions
within which the results have (usually) been obtained. It must be recognized,
however, that there are those who will dispute that statement, and will
argue that generalization is not the point; they will say that establishment
of empirical regularities, or laws of behavior, within the laboratory is
the point. I will not address that dispute here; I believe it to be unresolvable.
The argument that is critical, and I hope, is resolvable, is between those
who believe that current laboratory research practices do provide
generalization to conditions outside the laboratory, and those who do not
believe that such generalization is (ordinarily) possible.
that is critical, and I hope, is resolvable, is between those who believe
that current laboratory research practices do provide generalization
to conditions outside the laboratory, and those who do not believe
that such generalization is (ordinarily) possible.||
current centrality of this dispute can be seen in the title of an article
by Wiseman and Levin (1996). The authors make the extraordinary
jump from their laboratory situation to the shootdown of the Iranian jetliner
by the USS Vincennes. They acknowledge that "no laboratory task
could duplicate the conditions faced by the decision makers in this case"
but then go on to say: "Nevertheless (italics added), decision-making
experts brought in to provide expert testimony to a U.S. House Armed Services
Committee were able to draw parallels to the type of decision making task
so often studied in the laboratory, and to describe plausible biasing effects
in the Vincennes incident based on heuristics demonstrated in laboratory
settings employing hypothetical consequences" (p. 249). Two points should
be noted: (1) no justification for the generalization from the laboratory
to the Vincennes situation was offered aside from the assertion that "experts
were able to draw parallels" between the laboratory situation "so often
studied." A footnote reference to the testimony of Paul Slovic is offered
as support for this statement; Slovic's testimony itself, which was in
fact quite different, was not presented, however. Note that the entire
generalization hinges on the word "Nevertheless," for which no justification
whatever is presented.
these remarks by Wiseman and Levin to indicate the superficial manner in
which this important topic is currently treated, and the manner in which
an unjustified generalization can be perpetuated. Other researchers will,
no doubt, state that "As Wiseman and Levin have shown" to support the claim
of generalization. In what follows I provide a brief history of Brunswik's
efforts to address this topic and his challenges to the methodological
status quo during the 1940s and 1950s. In later pieces I will take up the
matter of representative design and its current confusion with the concept
of ecological validity.
research by psychologists has always been treated with some suspicion by
nonpsychologists, as well as some psychologists—the general idea being
that the life of the mind is simply too complicated to be examined in the
restricted conditions of the laboratory which, the critics often claim,
"trivialize" the topic. That question comes down to what is now treated
as the generalization of results: Can the results obtained in narrow,
highly controlled conditions be generalized to conditions outside the laboratory?
The question was introduced in a prominent, detailed and scholarly manner
by Brunswik in several articles beginning in 1943 when he called for "a
fundamental, all-inclusive shift in our methodological ideology regarding
psychology" (p. 261; see also Brunswik, 1952, 1956; Hammond, 1966, 1996a,
1996b, Cooksey, 1996). Note Brunswik's use of the term "ideology." That
in itself would have been enough to evoke wrath from the academic establishment;
it bordered on the insulting to charge that members of the establishment
were following an "ideology" (which connotes a cherished belief, a dogma,
or a doctrine), rather than straightforward scientific method. That came
close to an insult because the experimental psychologists of the day believed
that their methods were establishing psychology as a science, and
that they were simply extensions of logic, not part of a doctrine
or dogma of any sort.
not all, however. Brunswik had chosen the worst possible time to challenge
the use of the new research methods in psychology. For during the period
immediately following World War II, psychology discovered R. A. Fisher's
(1925) methods of "experimental design" as well as the methods introduced
by Neyman and Pearson (1933). Both offered powerful new ways of designing
experiments that were being applied with great success in the field of
agriculture (For an interesting description of the development of research
methods in psychology prior to World War II, see Gillis & Schneider,
1966). In short, just as psychologists were finding themselves in possession
of new and impressive methods for designing experiments that would bring
them new knowledge—and new status as scientists—Brunswik was trying to
tell them that it was all a mistake.
had chosen the worst possible time to challenge the use of the new research
methods in psychology.
The new methods of designing experiments had been quickly taken up by researchers
in agriculture in the 1930s and textbooks were soon written with specific
examples of their application to problems in that field. As a post World
War II graduate student in psychology, 1945-1948, I stumbled on these new
principles of experimental design in textbooks in agricultural research,
and soon found myself in the odd position of explaining experimental designs
to my professors in terms of "fertilizer, alternating crop rows, and plant
growth" because I wanted to make use of the new methods in my dissertation.
The first textbook for psychology students that included the new methods
did not appear until 1949 (McNemar, 1949). Several textbooks appeared soon
after, however, and psychologists—students and faculty alike—took hold
of these ideas with enthusiasm. It was not long before every graduate
student in psychology was required to pass examinations on "experimental
design" and to become sufficiently competent in the application of these
methods to construct their experiments according to the (unquestioned)
principles of "the systematic design of experiments." If the reader is
a graduate student in psychology anywhere in the world, she or he is undoubtedly
familiar with, and has been examined for, competence in these methods (often
known as "analysis of variance") and modern variations thereof.
reason for this enthusiasm was that Fisher's experimental designs not only
offered psychologists much more powerful ways to carry out their research,
but a much greater opportunity to claim the mantle of "scientist". The
Fisherian methods, and the (very different, and opposing) methods from
Neyman and Pearson, offered a form of rigor and scope that was very respectable
indeed, and moreover, had already demonstrably aided agricultural research
and production considerably. With new rigor came a new vigor and a new
respectability, and experimental psychology rapidly grew into a thriving
discipline. (For an interesting description of the growth of psychology
during this period by an historian, see Herman, 1995). Unmentioned during
this period of optimism was the fact that the Fisherian and the Neyman/Pearson
methods are completely at odds; psychologists cheerfully used both without
acknowledging the contradictions between them (see Gigerenzer et al., 1989).
During this period of the expansion of knowledge and skill these research
methods dominated the prestigious research journals, and the research papers
that did not employ them rarely got into print. That situation is not much
was precisely in these circumstances that Brunswik put forward his criticisms
of the methodological "ideology" of the day. His main charge was that it
was bringing more harm than good to psychology. To say this was poor timing
would be a gross understatement. But, of course, Brunswik did what he had
to do, namely, give voice to methodological blasphemy and thus strike a
blow at what many saw as the very foundation of psychological science.
Therefore, his views received exactly the kind of response one might expect
in such circumstances; they were ignored (no textbook in statistical methods
or experimental design ever mentioned them), and when not ignored, they
were harshly denigrated. All of this had an unfortunate consequence; Brunswik
committed suicide in 1955 at age 52.
reader should know that I am not a dispassionate observer of these events.
Because I have long been an admirer of Brunswik' s work my account of these
developments may be biased. Therefore I feel constrained to explain briefly
what my relationship with him was.
met Brunswik at a distance in 1938, shortly after he arrived in Berkeley
from Vienna. Although only an undergraduate at the time, I had heard of
the remarkable new professor and so I sat in on a few of his classes to
see what a European professor was like. He made an impression on me, despite
his less-than-perfect English, and when I returned to Berkeley as a graduate
student in 1945 I sought him out. Indeed, I sat at his feet at every opportunity.
He was both forbidding and friendly at the same time; forbidding because
of his obvious store of knowledge (you thought twice about striking up
a conversation with him, or going to his office), friendly because once
you did he was unfailingly courteous and then you were glad you did, and
thought just a bit better of yourself for having done it. Within moments
of talking with him you were aware of the fact that you were in the presence
of a true scholar (I was not alone in thinking that he knew everything);
yet his demeanor was so pleasant (even to graduate students) that one was
disarmed, and fright (but not awe) quickly disappeared. I felt that he
took me seriously and listened carefully to what I had to say, despite
the naiveté of my remarks. And he had a great deal to say about
whatever topic you brought to him. He was not appreciated by undergraduates,
however—he was always over their heads—and the chairman of the department,
Edward Tolman, had to write a strong letter to the Dean pleading for his
reappointment after his first year.
completely convinced by Brunswik's arguments, and even published one article
applying his views about experimental design while still a graduate student
(Hammond, 1948). And because I was able to present his point of view in
simpler, and perhaps more comprehensible, terms than he used, I was dubbed
the "poor man's Brunswik" by my fellow graduate students. It may be that
I have always occupied that role (see Hammond, 1986; Hammond, 1990), although
I would like to think that I have extended his ideas as well (see, for
example, Hammond, 1997). I maintained correspondence with him from the
time I left Berkeley in 1948 until his death in 1955. After his death,
at the request of Edward Tolman I edited a memorial volume The Psychology
of Egon Brunswik (Hammond, 1966) and published several books
and articles that were intended to further his contributions (see e.g.,
Hammond, 1954, 1955, 1996b; Hammond, Hamm, Grassia, & Pearson, 1987;
Hammond, Stewart, Brehmer, & Steinmann, 1975; Hursch, Hammond, &
Hursch, 1964). His general views of psychology have dominated my thinking
about psychology since I was a student. I founded a "Brunswik Society"
that has held annual meetings for the past 13 years.
dubbed the "poor man's Brunswik" by my fellow graduate students.
placed that caveat before the reader, in my next note on this web site
I will offer a brief exposition of Brunswik's challenge to the establishment
of experimental psychology that eventually appeared posthumously in his
1956 book Perception and the Representative Design of Psychological
Brunswik, E. (1943). Organismic achievement
and environmental probability. Psychological Review, 50,
Brunswik, E. (1952). The conceptual framework
of psychology. In International encyclopedia of unified science
(Vol. 1, no. 10, pp. 4-102). Chicago: University of Chicago Press.
Brunswik, E. (1956). Perception and the representative
design of psychological experiments (2nd ed.). Berkeley, CA: University
of California Press.
Cooksey, R. (1996). Judgment analysis:
Theory, methods, and applications. New York: Academic Press.
Fisher, R. A., Sir. (1925). Statistical methods
for research workers. Edinburgh, Scotland: Oliver & Boyd.
Gigerenzer, G., Swijtink, Z., Porter, T., Daston,
L., Beatty, J., & Krueger, L. (1989). The empire of chance: How
probability changed science and everyday life. New York: Cambridge
Gillis, J., & Schneider, C. (1966). The historical
preconditions of representative design. In K. Hammond (Ed.), The psychology
of Egon Brunswik (pp. 204-236). New York: Holt, Rinehart, and Winston.
Hammond, K. R. (1948). Subject and object sampling:
A note. Psychological Bulletin, 45, 530-533.
Hammond, K. R. (1954). Representative vs. systematic
design in clinical psychology. Psychological Bulletin, 51(2),
Hammond, K. R. (1955). Probabilistic functioning
and the clinical method. Psychological Review, 62, 255-262.
Hammond, K. R. (Ed.). (1966). The psychology
of Egon Brunswik. New York: Holt, Rinehart, and Winston.
Hammond, K. R. (1986). Generalization in operational
contexts: What does it mean? Can it be done? IEEE Transactions on Systems,
Man, and Cybernetics, 16, 428-433.
Hammond, K. R. (1990). Functionalism and illusionism:
Can integration by usefully achieved? In R. M. Hogarth (Ed.), Insights
in decision making: A tribute to Hillel J. Einhorn (pp. 227-261). Chicago,
IL: University of Chicago Press.
Hammond, K. R. (1996a). Human judgment and
social policy: Irreducible uncertainty, inevitable error, unavoidable injustice.
New York: Oxford University Press.
Hammond, K. R. (1996b). Upon reflection. Thinking
and Reasoning, 2, 239-248.
Hammond, K. R. (1997). Expansion of Egon Brunswik’s
psychology. In K. Fischer & F. Stadler (Eds.), Wahrnemung und gegenstandswelt:
Zum lebenswerk von Egon Brunswik (1903-1955) (pp. 79-105). Vienna:
Hammond, K. R., Hamm, R. M., Grassia, J., &
Pearson, T. (1987). Direct comparison of the efficacy of intuitive and
analytical cognition in expert judgment. IEEE Transactions on Systems,
Man, and Cybernetics, 17, 753-770.
Hammond, K. R., Stewart, T. R., Brehmer, B., &
Steinmann, D. O. (1975). Social judgment theory. In M. F. Kaplan &
S. Schwartz (Eds.), Human judgment and decision processes (pp. 271-312).
New York: Academic Press.
Herman, E. (1995). The romance of American
psychology: Political culture in the age of experts. Berkeley: University
of California Press.
Hursch, C. J., Hammond, K. R., & Hursch, J.
L. (1964). Some methodological considerations in multiple-cue probability
studies. Psychological Review, 71, 42-60.
McNemar, Q. (1949). Psychological statistics.
New York: Wiley.
Neyman, J., & Pearson, E. S. (1933). On the
problem of the most efficient tests of statistical hypotheses. Philosophical
Transactions of the Royal Society of London. Series A, 231,
Wiseman, D. B., & Levin, I. P. (1996). Comparing
risky decision making under conditions of real and hypothetical consequences.
Organizational Behavior and Human Decision Processes, 66,