The forty-five literature applications reviewed were produced by twenty-three different publishers, and evenly split between those designed for elementary and those designed for secondary/post-secondary populations. The majority accessed multimedia from a CD ROM disk (31), although some used a combination of CD ROM and laserdisc (10), and a few (4) used only floppy disks. Most (24) were offered for both MacIntosh and PC computers, with the remainder evenly split between applications designed exclusively for one or the other platform. In general, the cost of the programs was moderate, ranging between $25.00 and $100.00 for straight CD ROM or floppy disk offerings, and between $200.00 and $300.00 for programs including a laserdisc.

Program content was also found to be well suited to the populations for whom the applications were designed. At the elementary level, content centered on fairy tales, fables, and published picture books commonly found in elementary classrooms. At the secondary/post-secondary level, all but two applications were based on works and authors from among those most frequently taught in secondary/post-secondary classrooms (Applebee; 1989, 1991, 1993). General descriptions of the kinds of programs found at both the elementary and secondary/post-secondary levels follow.

Almost all the applications designed for elementary students could be best described as talking books. At their most basic, these applications presented stories as illustrated text in a linear, page-by-page fashion, with the text highlighted as it was read to the students. Many also defined words on request, both in text and speech, and almost half included a non-English option. Only one, however, offered a choice of readers other than by language. Most elementary applications also included sound effects and/or music, and animated illustrations

Nine of the talking book programs included a print function which allowed students to print text or, more commonly, pictures. Six contained interactive quizzes which tested students' comprehension with multiple choice, single-correct-answer questions, and five included "interactive pages" -- illustrations which students could explore by clicking on their different elements to find hidden animations. Three talking books further encouraged students to manipulate the stories they were reading by allowing them to cut and paste text and pictures, to color pictures, and to add text and sound.

None of the elementary applications reviewed included video clips. Only one contained any background information about the works presented. None included on-line features that encouraged student comments or interpretations of the works, although a few encouraged off-computer interpretive activities.

All of the talking books, then, were uniformly centered on the reading of highlighted text, indicating that publishers view the teaching and learning of literature at the elementary level as little more than the teaching and learning of reading. Although some of the applications we reviewed seemed well suited to such a task, this ubiquitous association of sound and text tends to focus students on decoding processes, rather than on thinking and responding to literature. A common focus on content comprehension and the lack of interest in interpretation, literary devices, authors, and/or background information are further indications of a bias toward skills-based reading instruction.

The three "non-talking book" elementary applications we looked at were quite various and therefore defy classification. One was an adventure-type game in which players explored an imaginary environment and picked up objects that they were then supposed to return to appropriate nursery rhyme characters. When an object was returned to the correct character, the nursery rhyme was recited. The other two applications might best be described as story makers, which, although they included story examples, were primarily devoted to student assembly of a variety of elements to create their own stories. One of these was primarily text-based, a kind of word processor containing predefined story elements. The other was also focused on assembling predefined elements, but was more oriented toward sound and animated elements linked with text.

While the elementary applications we looked at made more extensive use of the computer's sound and graphics capabilities than did the secondary/post-secondary applications we reviewed, the latter made greater use of its nonlinear linking capabilities and interactive video technologies. They also exhibited a difference and a greater diversity in pedagogical approach. Some (6) of these applications could best be classified as books on computer. Like their elementary level counterparts, these programs presented the full text of collected or single works on the computer screen, and were essentially linear, with student interaction limited, for the most part, to electronic page-turning. Although a few of these books on computer included audio readings of selected passages, however, none offered a complete reading and none highlighted the text as it was read. They were also more likely to at least minimally value student interpretations by providing on-line note-taking capabilities, and less likely to provide high quality illustrations and/or animations.

A second common type of secondary/post-secondary application was databases (7). These programs provided book notes or the complete texts of collected or single works, background information, and a variety of database functions for searching, collecting, and printing the information they contained. Most also included note-taking capabilities, and a few contained interactive questions and answers and/or off-line exercises.

Like the databases, the two (2) secondary/post-secondary applications we categorized as hypertexts were essentially text-based, but they differed from the latter in that they did not include typical database functions, but rather, extensive built-in links between entries. Both of the applications in this category were focused on background information about a single author and his works, and, although they included selected passages from such works, did not provide the complete texts of any.

Another category of secondary/post-secondary programs, hypermedia (6), linked the complete texts of particular works to background information and video segments presented via laserdisc, one of which provided multiple readings by various actors and multiple interpretations by various scholars of the five works it covered. All of the hypermedia applications we reviewed contained extensive on-line background information, and most included open-ended questions presented on-line but designed to be answered off computer.

The final two (2) secondary/post-secondary hypermedia applications we reviewed were problem solving games, ostensibly linked to literary works. The games were highly interactive and contained excellent graphics and sound. In both, students were asked to explore simulated environments and collect clues to solve a mystery, but the mysteries were not related to the works on which the games were based, and, although they encouraged a kind of critical thinking, that thinking was convergent and focused on single correct solutions to the mysteries. In general, then, the secondary/post-secondary applications were much more concerned with literature -- with interpretations, with context, with authors, with literary devices, and with analyses -- than were their elementary level counterparts, and they were more likely to provide for at least note-taking on the part of students. In tone, however, and more importantly perhaps, in form, these applications focused on single "correct" interpretations and analyses. They shared the "scientific," text-centered approach to literature teaching and learning Applebee (1993) found common in high school literature classes.

Indeed, the most prevalent design paradigm for these commercial products seemed to be the "transmission of knowledge" model that once dominated both instructional technology and the teaching of literature. When response-based criteria were applied to commercial multimedia/hypermedia products, they faired poorly. Although software products were rated quite positively as multimedia, when examined closely for features that were pedagogically grounded in response-based theory, they fell a good bit short of what participating teachers deemed desirable within response-based contexts.

In particular, average ratings on response-based criteria for the applications we reviewed were 4.69 (on a scale of 1 to 10), while the same software packages averaged 7.26 (on a scale of 1 to 10) on technical criteria relating to multimedia design. If one considers programs with ratings of 4 or below as "poor" with respect to such criteria, those with ratings of 5 to 7 as "adequate," and those with ratings of 8 or better as "good" to "excellent," fully 23 of the 45 programs we reviewed were rated as "poor," and only 5 were considered "good" to "excellent" from a response-based perspective. On the other hand, from a technical point of view, only 5 applications were considered "poor," while 22 were rated as "good" to "excellent." Figure 1 below graphically summarizes these ratings.

Table 1
Comparison of Ratings on Technical Concerns and Response-Based Criteria

Such results are, as previously mentioned, quite discouraging from a response-based perspective, indicating as they do that these applications, rather than breaking new ground in literature teaching and learning, have generally adopted older and more traditional reading and text-centered pedagogical approaches. McLuhan (1963) suggests that new media generally mimic old forms before exploiting their uniqueness, as in the cases of, for example, the Gutenberg Bible or early movies. Perhaps it is too early to be overly discouraged. Perhaps, on the other hand, it is a good time to become proactive on such issues. Specific findings concerning each response-based criterion are discussed below.

What counts as knowledge? is concerned with the formal representation of knowledge within a program. In general, evaluators found that the hypermedia applications we reviewed tended to represent knowledge as static and "canonical" (overall rating, 4.65). If one considers applications with ratings of 4 or below as "poor," those with ratings of 5 to 7 as "adequate," and those with ratings of 8 or higher as "good" to "excellent" in response-based knowledge representation, the greatest number of programs (21) were seen as "poor" in this regard. Nineteen programs were viewed as "adequately" representing knowledge in a response-based fashion, and only 5 were seen as providing "good" to "excellent" representations of knowledge from such perspective. Table 2 shows these rankings.

Table 2
"What Counts as Knowledge" Results

Elementary level applications were rated a good deal lower (3.82) than secondary/post-secondary applications (5.48) on this criterion, mostly due to their propensity to provide a single reading of the text and to give single-correct-answer comprehension questions. The more highly rated elementary applications provided multiple voices, open-ended questions, and/or "interactive pages". Evaluators who observed students using the latter commented that these pages encouraged an exploratory approach to literature and elicited both questions and links to personal experience from their users. Elements in secondary/post-secondary programs that evaluators thought represented knowledge in a more reader-based fashion included multiple representations of the same knowledge search capabilities, provisions for note-taking, open-ended questions, and construction tools.

Table 3
"Role of Text" Ratings

The role of the text (Table 3 above) refers to the way a program represents meaning in relationship to a text. A response-based perspective assumes that there will always be multiple defensible interpretations of a text because readers will always bring varied experiences to their readings. Evaluators felt that the majority of applications they looked at did not support such assumption (overall rating, 4.83). Applying standards as above to this criterion, evaluators ranked the majority of programs (25) as representing text "poorly" from a response-based perspective. Fifteen programs were found to "adequately" represent text, and only 4 were seen as providing "good" to "excellent" representations of text from a response-based perspective.

Elementary applications were again rated lower (4.18) in this category than secondary/post-secondary applications (5.48), mostly due to their lack of provision for student responses, but also because of their frequent use of pop-up definitions which evaluators thought created a very concrete impression of meaning residing in text. The two elementary applications most highly rated on this criterion, in contrast, offered multiple representations of meaning and/or alternative definitions of words from which students could choose. Secondary/post-secondary applications' higher ratings were mostly on the strength of two features commonly found in them -- note-taking capabilities and open-ended questioning. While evaluators thought all note-taking capabilities were at least minimally a positive feature from a response-based perspective, they preferred notes linked to text or written in the margins of a text to the more common drop-down notes. For similar reasons, reviewers favored the rarer programs that provided spaces for answering open-ended questions on-line to the more frequent use of questions asked on-line but designed to be answered off computer. They also thought that video representations linked to text supported a response-based perspective because they offered alternative interpretations of it.

Table 4
"Role of Students" Ratings

The role of the students (Table 4 above) is concerned with whether and how a program validates students' responses to a literary work. Evaluators gave this criterion the highest ratings in the response-based category (overall rating, 5.59; secondary/post-secondary, 6.09; elementary 5.09), indicating that they felt that students were somewhat empowered by the hypermedia literature programs we reviewed. Applying standards as above to this criterion, evaluators ranked slightly fewer programs (18) as assigning "poor" roles to students from a response-based point of view, and a good deal more programs (22) as assigning them "adequate" roles, then on the other response-based considerations. By these standards, however, only 5 applications were viewed as providing "good" to "excellent" opportunities for students to interact with text.

Features that evaluators found empowering of students included interactive pages, construction tools, note-taking capabilities, nonlinear access to background information, and open-ended questioning, but it was noted that all of these could be improved upon from a response-based point of view. With regard to the latter, evaluators felt that not only is the agenda of such questions set by the application and not by the student, but the common practice of presenting questions on-line to be answered off-line tends to value the questions (i.e., the "expert") over the answers (i.e., the student). It should also be noted that the programs we reviewed, if they provided them at all, provided opportunities for individual student responses rather than spaces in which discourse among students was encouraged, and none encouraged student-generated questions.

The role of the teacher is concerned with whether a teacher is empowered or constrained by a program, and with whether or not a program promotes student-teacher interaction. Evaluators gave this criterion the lowest ratings in any category (overall, 3.68; secondary/post-secondary, 4.22; elementary 3.14), indicating that they felt that teachers were essentially disregarded by the hypermedia literature programs we reviewed. Applying standards as above to teacher empowerment, fully 29 programs were seen as "poor" in this regard. Twelve programs were viewed as providing "adequate" roles for teachers, and only 4 applications were seen as providing "good" to "excellent" roles for teachers from a response-based point of view. Table 5 below shows these results.

Table 5
"Role of the Teacher" Ratings

Indeed, few of the applications we looked at had well designed teacher guides, and most teacher guides offered no teacher materials at all. Very few of the applications we looked at included any provision for teacher input other than the ubiquitous "notes," and only one included program management tools. In addition, as noted above, none of the applications we reviewed provided public discourse spaces which might be seen as the primary areas in which teachers could interact with students in a variety of ways.

In terms of response-based features (Table 6), only two -- transparent navigation and intertextuality and juxtaposition -- were found in more than half the software packages reviewed. Fully five of the features identified as supportive of response-based teaching and learning -- the facility to share responses, the facility to support non-text responses, support for envisionment, access to multiple perspectives, and the promotion of student ownership -- were found in less than a quarter of them. A sixth feature -- support for discourse -- was found, if at all, as an off-line, rather than an on-line, feature.

Indeed, the pedagogical approaches taken by the vast majority of the commercial applications we reviewed mirrored the approaches commonly found in schools. At the elementary level, literature teaching and learning was equated with reading instruction. At the secondary/post-secondary level, teaching and learning was almost exclusively text-centered. What was sorely missing, from a response-based perspective, in most of the commercial applications we reviewed, was any provision for constructive roles for learners. In the next phases of the "Multimedia and Literature Teaching and Learning" project, we set out, therefore, to design and test programs containing features that addressed this shortcoming. Based on the strengths, weaknesses, and potentiality found in commercial products, two prototype applications, Kidspace, for elementary students, and the Beats, for secondary/post-secondary students were designed and pilot-tested in active classroom settings.