Chapter V. Developing a departmental plan for Summative use of Peer Observation
Summative assessment of teaching effectiveness requires a more methodical and procedural approach than does a system for formative assessment. The two should be developed together and should be coordinated, but a summative assessment needs to be formalized and procedurally strict so as not to compromise its integrity at any point during the process. Keep in mind, too, that careers are at stake, so the system must be designed to accomplish its goals fairly and steadily even in the face of controversy.
What are the key problems in using peer observations summatively?
|A careful administrator can minimize the problems inherent in evaluation of teaching by creating the conditions for effective summative assessment through his/her leadership:
- by making the departmental learning outcomes and teaching mission public
- by publicly participating in and offering to model being evaluated
- by making the teaching mission and evaluation forms available prior to classroom visits
- by arranging meetings with faculty prior to an evaluation
- by clearly articulating accepted standards for performance, tied to the values and mission
- by achieving departmental consensus on the objectives of classes that will be evaluated
- by meeting with faculty after an assessment to answer questions and by asking faculty to develop personal plans of action based on the teaching evaluations (Return to top)
How can a fair system be developed?
The single most important factor in successful peer observation of teaching is the careful preparation (via orientation and training) of the faculty evaluators. This may seem counter-intuitive, since the faculty can cite their great amounts of teaching experience as sufficient training, but it is difficult to over-estimate
|the risk of launching a program without making sure everyone practices certain procedures and techniques and understands why they are essential. Assessment specialists (check with your teaching center) can be invited into the department to do group sessions for the entire faculty; or those who will conduct peer observations can be asked to attend workshops addressing this set of skills. Practice evaluations, role plays, and case studies are particularly useful in preparing faculty to observe their colleagues in the classroom.
There are several other considerations that need to be addressed in developing a plan for summative assessment of teaching.
- Consider your department’s mission and teaching values. If you are going to build a sound teaching evaluation plan, the values that guide the system should be explicit and directly connected to things that are most important to your faculty. Otherwise, the criteria for evaluation will constantly slide, thus invalidating the process and creating political problems.
- If you have not already done so, you will need to identify specific outcomes for your graduating students. This means that you will need to record clearly, through faculty consensus, what you want students to become during their time under the influence of your academic program. Your outcomes will indicate what you value most as teachers, and therefore which approaches to teaching are most likely to generate the results you target in your students.
- Consider the political climate of your department and college. The personalities of the key administrators as well as the unofficial opinion leaders will surely have an effect on both the function of the process as well as the perceived validity of the results. Depending on the make-up of your department, it may not be enough to simply start with the enthusiasts and hope to pull along the rest. Even the most recalcitrant should be invited into the discussion at some point early in the process. It is often a good idea to invite in a facilitator from outside the department to structure a discussion on the process of developing a summative evaluation plan.
- Consider how the evaluation process will be administered. Will there be a committee that takes responsibility for the process? Or will the whole department share in that responsibility? Even if you go with a committee structure, the whole department will need to be included in the conversation. Either way, the faculty who perform observations for summative purposes need to undergo preparation. This preparation should include developing an appreciation for how the department’s teaching and learning values can be supported through a wide range of teaching strategies and styles. Another element of the preparation should be “norming,” in which a group of would-be observers watch a videotaped teaching moment or review a document and then compare and discuss their respective evaluations.
- Consider the type of instrument to be used. In the next chapter there is a summary of this topic with suggestions that could be adapted to nearly any peer observation plan. Some use a “yes-no” scheme, in which the teaching values of the department have been represented as a set of teaching behaviors that are either present or not in a given instructor’s practice. Others allow for scoring on a Likert scale. Others are more flexible still, inviting observers to look for classroom phenomena that represent preferred teaching values. Any of these can be effective as long as the goals, criteria and standards are clear. It will be necessary for your departmental faculty to decide how narrowly it wants to define effective teaching practices, and how much interpretation to leave to the observer.
- Consider how evaluations will be recorded and communicated both to the person being evaluated and to the administrators who need to know. Summative evaluations can be sensitive, and therefore should be handled with care and confidentiality. The process for who gathers the results of the evaluation process, compiles them, and reports them to the chair should be made explicit. The chair or chief administrator then communicates the conclusions to the evaluated faculty member as part of a larger evaluation process. Contrast the formal handling of summative assessment with the more personal handling of formative assessments. (Return to top)
Can the models used for formative assessment also work in summative assessment?
The two processes should work in tandem, so that the focal points of formative assessment are consistent with the focal points of summative assessment. The feedback a faculty member receives from colleagues during formative assessment should square with the criteria being used in summative assessment for the same department. The person being evaluated needs to see that the criteria and standards do not shift, so that the work he/she does as a result of formative feedback can
be measured in the summative assessment process. For example, if a department puts great effort into encouraging its faculty members to adopt collaborative learning models because this model encourages student development consistent with the mission of the department, then the formal evaluation of teaching should be sure to address that primary value. It makes no sense to foster collaborative learning only to follow up with drop-in peer observations that measure only the quality of lectures. (Return to top)
Should the peer observers who participate in a faculty member's formative assessment also be involved in his/her summative assessment?
This overlap is not ideal, but sometimes it works just fine. Practically, depending on the size of your department, it may be unavoidable. At a minimum, every member of the department should be aware of the teaching and learning goals and practices expressed in the departmental mission, and receive training in evaluation that addresses those goals and practices. It is possible for the same peer to conduct both formative and summative assessment as long as the process that generates formative recommendations is separated from the summative assessment. The faculty member needs to be confident that the formative assessment is not a definitive judgment. (Return to top)
Why can't the two processes take place simultaneously, using the same structures and formats?
The two processes have conflicting goals and therefore, at some point, diverge completely. The role of a faculty observer for the purpose of formative assessment is different from the role of a faculty observer for the purpose of summative evaluation. While the former is more closely and consciously aligned with the specific needs of the particular colleague being observed, the latter is more closely and consciously aligned with the teaching values of the department. In the best of all worlds, these two perspectives are very closely aligned, but administratively discrete. In politically healthy departments the distinction is less pronounced because
|the high level of faculty-to-faculty trust allows for the processes to overlap. But that’s not the best case upon which to develop procedures to manage a high-stakes decision process. The summative evaluation process needs to be able to function even in an environment of political dysfunction and distrust. The careful separation of formative and summative processes aids in this purpose.
|Summative evaluation applies criteria and standards that are shared by the professional community of the practitioners, for the purpose of making a definitive judgment about a faculty member’s teaching effectiveness. Formative processes are guided by those same criteria and standards; however, their purpose is not to generate a final judgment, but rather to point to aspects of teaching that could be further developed. (Return to top)
If we've never done peer assessment of teaching before, how do we start?
- Start with a departmental discussion of the teaching mission, based on student learning outcomes for your curriculum.
- Identify the criteria and standards for effective teaching in your program or department.
- Verify that a formative peer observation plan is in place (see Chapter Four), to ensure that the faculty have an opportunity to develop their
|| teaching toward the mission. If there is not such a plan in place, stop consideration of a summative peer assessment process until a formative process exists.
|Select or design mission-specific instruments and procedures to guide the evaluation process.
||Prepare and train the faculty on how to conduct peer observations.
|Decide on a procedure for collecting and communicating the results of the evaluations. (Return to top)
What difficulties will a department need to address in setting up a peer observation program for summative assessment?
- Consensus on what constitutes “good teaching” will be difficult.
- Even well-meaning peers may not be consistent in their evaluations of colleagues.
- Faculty reputations can bias an observation.
- Peer Observation should be only one part of a larger evaluation of teaching.
- It is difficult to get faculty to undergo training on observing teaching. (Return to top)