Senate Resolution No. 0304-05R

University at Albany RESOLUTION
on General Education Assessment

Introduced by SUNY Faculty Senator Richard Collier, 4/19/04

Whereas the University at Albany has already taken a campus position against any system-wide assessment (last year's Senate Resolution No. 0203-02R, attached) and finds nothing in the two attached documents from the “Campus-based Assessment Committee” that persuades us to change this position,

Whereas our campus-based assessment, developed in accordance with SUNY's guidelines and category definitions and fully approved by GEAR, infuses the Written Communication and Critical Thinking requirements throughout our curriculum and embeds their assessment in the full structure of our courses,

Whereas we are completing the second year of our three-year cycle of General Education assessment, with embedded assessment of Written Communication and Critical Thinking that does not involve or lend itself to one or more examinations,

Whereas the latest version of System-wide Assessment speaks of "instruments" (which we deem to be examinations) and requires our results to be correlated statistically with the results from nationally-normed external testing, and

Whereas we do not find either compelling or intellectually valid the argument that this latest version should be supported because portions of it may be less objectionable than prior (or future) versions,

BE IT RESOLVED THAT: the faculty of the University at Albany will continue to follow the General Education Assessment Plan of its own, and not support, embrace, or willingly participate in a SUNY-mandated standardized General Education Assessment Plan such as that recently proposed by the “Campus-based Assessment Committee.”



RATIONALE:

The documents “Strengthened Campus-based Assessment” and “Proposed [GEAR] Revisions (Spring 2004)” were produced by the ad hoc “Campus-based Assessment Committee” that was charged with modifying System-wide assessment proposals previously rejected by the University Faculty Senate, the Faculty Council of Community Colleges, and the Student Assembly. The committee included representatives of the Senate, the Faculty Council, the Student Assembly, SUNY System, and one Trustee (the student member). The committee completed its work during its two meetings (March, 2004).

The University Faculty Senate has requested each SUNY campus to review and consider endorsing the two attached documents and report its decision back to the University Faculty Senate, which will decide whether or not to endorse the proposed documents at its next Plenary Session, at Syracuse Health Sciences, April 22nd-24th.

The Undergraduate Academic Council carefully studied the proposal at its March 25th meeting and determined that its requirement of a nationally-normed “instrument” to assess Writing and Critical Thinking or a “locally developed instrument…demonstrated to correlate statistically (i.e., have concurrent validity) with nationally-normed measures” directly conflicts with our on-going, GEAR-approved assessment of these two important General Education categories.

It was strongly emphasized we do not have an “exam” and will not develop a local instrument for critical thinking or writing. We do not want an instrument as a test of something not necessarily well tested by such a device. Our faculty members have participated extensively in designing the assessment process and have agreed that at the beginning of the semester the instructor will inform the students how the various components will count into their final grade. The instructor is also required to explain to them the connection between the class objectives and SUNYGER category objectives. At semester end, the instructor reports to Albany's General Education Assessment Committee students' achievement on those portions of course requirements relevant to the general education category or categories being assessed.

The UAC notes that our learning objectives/outcomes are based on the SUNY definition for the “Critical Thinking” category: “Students will identify, analyze, and evaluate arguments as they occur in their own or others’ work; and develop well-reasoned arguments.” In contrast, the proposal documents seem to imply the existence of generally agreed upon national “norms” for critical thinking.
While the conflict with our assessment of writing and critical thinking is sufficient grounds to reject the latest proposal, the UAC also questioned the usefulness of a System-wide effort to assess the base level mathematics ability common to the many hundreds of disparate SUNY courses that have been approved to satisfy SUNYGER Mathematics. The campus-based assessment of each calculus, statistics, etc., course is both feasible and may improve the course, in contrast to a statewide testing of such minimal high school level mathematics as may be common to all those courses.
The UAC concluded this proposal for implementing System-wide assessment, although arguably less objectionable than previous versions, remains unacceptable and urges the University Senate to reject the proposal.

Senate Resolution No. 0203-02R

University at Albany RESOLUTION

WHEREAS an email sent from Associate Provost Donald Steven on the SUNYGED Listserv on March 21, 2003 indicated that the Provost's office "would like to begin discussing the Task Force recommendations on university-wide assessment…"; and
WHEREAS the purpose of general education assessment has been understood to be an evaluation of a campus' programs, not an assessment of individual students; and
WHEREAS it is our belief that assessment needs to be a campus-based activity for the following reasons:

BE IT RESOLVED THAT: the faculty of the University at Albany will continue to follow the General Education Assessment Plan of its own, and not support, embrace, or willingly participate in any SUNY-mandated standardized General Education Assessment Plan.

Strengthened Campus-based Assessment

Summary

Background

It is not the intent of this summary to retrace the University’s many efforts over the years to develop a high quality assessment program. As we all know, colleagues have been engaged for a long time in serious and conscientious efforts to develop a State University Assessment Initiative that represents both “best assessment practices” and our own “best thinking.” We can be very proud of what has been accomplished to date.

In its final report, the Provost’s Advisory Task Force on the Assessment of Student Learning Outcomes underscored its belief that assessment serves two complementary functions in higher education today: ‘Assessment as improvement’ and ‘Assessment as accountability,’ and that both of these functions have an appropriate place in the SUNY Assessment Initiative and can strengthen the University’s institutions and the System as a whole.

In recent years, we have struggled to try to find a way to address both of these goals in a way that faculty and stakeholders agree meets their concerns and expectations. After many attempts, we believe that we have discovered an approach that is based on good assessment practice, that addresses our faculty’s many legitimate concerns, and that meets System goals as expressed by the Chancellor.

The Elements of a Proposal

We believe that proposal for a strengthened campus-based assessment process could be structured as follows:

1. Existing campus-based assessment plans need only be revised and incorporated in the proven and ongoing GEAR process. There would be no need for a second, System layer of assessment.
2. Only the learning outcomes in three foundational areas: Mathematics, Basic Communication (Written), and Critical Thinking (Reasoning), should be assessed incorporating external measures and the assessment of these “building blocks” of general education could be done in various ways that would provide real flexibility for campuses.
3. An instrument such as the National Survey of Student Engagement (NSSE), the Community College Survey of Student Engagement (CCSSE), or a revised, extended Student Opinion Survey (or similar measures) could be used to understand the campus academic climate and how that might relate to the assessment results. This analysis of assessment results could be incorporated into campus procedures.
4. System Administration should cover the cost of all externally referenced measures and surveys for a representative sample of students.
5. Value-added assessment should be optional.
6. The proposed policy should contain language which will safeguard assessment data from misuse.


Proposed Revisions to the GEAR Guidelines

To incorporate these elements into the GEAR process, very few changes are actually needed:

Re “externally-referenced measures”, to GEAR Criterion #3, add the following:

Every effort has been made to describe “externally referenced measures” in a way that provides for meaningful external referencing while providing maximum flexibility for campuses. Not only will this address the longstanding concern of faculty that no proposal should promote a “one size fits all” approach, but it also renders any attempt to engage in inter-campus comparisons essentially meaningless. Campuses will continue to use their current approved assessment plans for all of the SUNY-GER learning outcomes. For the learning outcomes in the three “building blocks,” there are a number of ways in which campuses can meet the requirement for “externally-referenced” measures: for example, through the use of nationally-normed tests such as the Academic Profile, the California Critical Thinking Skills Test or the Quant-Q, or by using SUNY-normed measures in lieu of nationally-normed measures, using one of two approaches:

i. A locally developed instrument that measures the learning outcomes in one or more of these three areas and that is demonstrated to correlate statistically (i.e., have concurrent validity) with nationally-normed measures.
ii. A locally developed instrument that measures the learning outcomes in one or more of these three areas that is reviewed and approved by GEAR. As part of this process, GEAR will rely on discipline-based panels of distinguished SUNY faculty that will develop standards and rubrics campuses may use to assess student performance. Campuses choosing to use their own standards and rubrics must demonstrate to GEAR that their standards and rubrics are essentially equivalent to those developed by the discipline-based panel.

Campuses opting for this approach would be expected to periodically provide GEAR with samples of student work for each standard of student performance, to be reviewed to ensure ongoing validity and reliability of the measure. Note that it is also possible to combine these approaches: for example, a campus might choose to use the CCTST for Critical Thinking and SUNY-normed measures for Mathematics and Basic Communication (Written).

Re the “campus academic environment”, add a new GEAR Criterion (as #6) to:

Re student involvement, to address the need for greater student involvement:


Safeguards re the Utilization and Reporting of Assessment Results

To ensure that assessment results are used appropriately, it is essential that this process be guided by a set of principles that will safeguard assessment data, along the following lines:

Costs

The costs of the purchase and scoring of nationally-normed measures — as well as for the NSSE or CCSSE — should be paid for by System Administration for State-operated/funded institutions and community colleges, based on a sample size of up to 20% of the undergraduate student body on a three-year cycle.

Next Steps

System Administration has indicated that its goal is to move assessment forward through consensus and the Chancellor has said that if a faculty proposal met his expectations and has the support of governance, he would be willing to consider revisions to the June 17 Board resolution.

 

Proposed Revisions (Spring 2004)

General Education Assessment Review (GEAR) Group

Review Process Guidelines

I. Introduction

The General Education Assessment Review (GEAR) Group has been established upon the recommendation of the Provost’s Advisory Task Force on the Assessment of Student Learning Outcomes and has been formed jointly by leadership from the University Faculty Senate, the Community College Faculty Council, System Administration and the Student Assembly.

Comprised primarily of faculty from throughout the University, GEAR also includes students, campus chief academic officers, and campus professional staff (particularly from Institutional Research). GEAR is co-chaired by Dr. Patricia Francis, Professor of Psychology and Executive Assistant to the President at the College at Cortland and Dr. Donald Steven, Executive Vice Provost for Academic Affairs. GEAR’s Web page, which includes a summary of its activities as well as many useful resource and reference materials, may be accessed at http://cortland.edu/oir/gear/.

II. Goals

The GEAR Group’s goal is to work with campuses as they develop and implement their campus-based plans for assessing student learning outcomes in General Education, following the guidelines contained in the Task Force report as well as subsequent discussions involving faculty and campus and System leadership. GEAR intends to function as a resource and a colleague, making itself available to campuses to the extent that they would welcome and in ways that they feel would be helpful, engaging them in a dialogue as they develop and carry out their assessment plans. In its “process review” of campus General Education assessment plans, GEAR will focus exclusively on the campus’s assessment processes and procedures, not the assessment outcomes themselves.

III. Process

Each campus is responsible for determining the particular structure and content of its campus-based General Education assessment plan, following its own existing governance processes.

The task of developing and implementing a campus-based assessment plan for General Education should fall primarily to the faculty members who teach in the program, with the assistance of professional staff and students when appropriate. (Indeed, it may well be the case that on some campuses a full-time staff and/or faculty assessment person may be in a leadership role.) Campus-based assessment plans should be submitted to, and approved by, the campus’s Faculty Senate or Faculty Council prior to being submitted to the GEAR Group for formal review.

GEAR’s Expectations of Campus General Education Assessment Plans

In its initial review of campus assessment plans, the GEAR Group will use eight nine criteria in evaluating a plan’s comprehensiveness and rigor. In addition to reflecting widely recognized best assessment practices in higher education, these criteria are consistent with the general guidelines included in the Task Force Report and subsequent discussions, the expectations for assessment of the Middle States Commission on Higher Education, and regulations proposed by the New York State Education Department as part of its Quality Assurance Initiative in Higher Education.

In its initial review, the GEAR Group will seek to ascertain for each campus plan that:

1. The objectives for student learning in General Education relate directly to the student learning outcomes defined in the Implementation Guidelines of the Provost’s Advisory Task Force on General Education. The GEAR Group is likely to agree that this criterion is met if all outcomes from the Implementation Guidelines are reflected in the campus’ statement of General Education learning objectives for its program. (It is important to note that campuses may also include additional learning objectives that are specific to their own program.)

2. Programmatic activities intended to accomplish the campus’ objectives for student learning in General Education are described. The GEAR Group is likely to agree that this criterion is met by the campus providing GEAR with its guidelines or procedures for designating courses as General Education courses.

3. The measures developed to assess student learning are designed to provide credible evidence of the extent to which students have achieved the learning outcomes or skills stated in the objectives. The GEAR Group is likely to agree that this criterion is met if, for each learning objective, appropriate assessment measures have been established for determining the degree to which students have mastered the objective. In judging the appropriateness of a specific measure, the GEAR Group will rely on answers to the following questions:

4. The plan proposes standards to which student performance relative to the learning outcomes in the objectives can be compared. The GEAR Group is likely to agree that this criterion is met if campus assessment plans include, for each learning objective, the standard defining what level of student performance the faculty considers as “exceeding,” “meeting,” “approaching,” and “not meeting” standards.

5. The anticipated results of the assessment are able to affirm the degree to which the learning objectives have been achieved and thus make it possible to identify areas that need to be addressed in order to improve learning. The GEAR Group is likely to agree that this criterion is met if it is clear from the assessment plan that mechanisms exist for sharing assessment results with appropriate faculty and staff and for making programmatic improvements based on the assessment results (if necessary).

6. Mechanisms for assessing the campus academic environment are described. The GEAR Group is likely to agree that this criterion is met if it is clear that the assessment plan provides for the periodic administration of a survey that yields indicators reflecting the campus academic environment (e.g. the National Survey of Student Engagement, the Community College Survey of Student Engagement or a revised, extended SUNY Student Opinion Survey or similar instrument.) and a report on what has been learned from the campus’s consideration of the possible relationship between academic assessment results and these environmental influences.

7. The assessment plan has been reviewed and approved through the appropriate curriculum and faculty governance structures and shows evidence of student involvement in the development of revisions to the assessment plan. The GEAR Group is likely to agree that this criterion is met if the assessment plan includes a section describing the process through which the plan was developed and approved on the campus prior to being shared with the GEAR Group, as well as the efforts made to include students in the process of revising the initial plan.

8. The plan adheres to the timetable established by the GEAR Group and agreed to by the University Provost. The GEAR Group is likely to agree that this criterion is met if it is clear that the assessment of all of the General Education learning objectives in the Knowledge and Skills Areas and Competencies takes place within a three-year cycle. (The campus plan should include the schedule for the assessment cycle.)

9. The assessment process includes provisions for evaluating the assessment process itself and disseminating assessment results to the appropriate campus community. The GEAR Group is likely to agree that this criterion is met if processes are described in the assessment plan for evaluating the assessment process once complete, making changes in the process if necessary, and sharing assessment results with the appropriate campus community.

Initial Review

GEAR will receive and critique campus assessment plans and approve those that meet its expectations for effective assessment; campuses will be advised in writing of revisions that would likely lead to approval, as appropriate. GEAR will place a strong emphasis on the extent to which campuses demonstrate they will use assessment results to improve their General Education programs.

Ongoing Review

After the initial review process, the GEAR Group will review campus General Education assessment plans on a biennial, staggered basis, applying the same criteria as above, with greater emphasis on how campuses are using assessment data to improve their General Education programs.

IV. Reporting

GEAR will establish a clear protocol and a standardized reporting format—consistent with the recommendations of the Provost’s Advisory Task Force on the Assessment of Student Learning Outcomes—for campuses to use to report assessment results in General Education to System Administration for the purpose of accountability. This annual report, to be submitted by the Chief Academic Officer at each campus directly to the Office of the Provost, will include specific information on its students’ progress in mastering the learning outcomes outlined in the General Education Implementation Guidelines. System Administration will use these data—in accord with the Utilization and Reporting of Assessment Results principles in the Task Force report—in the preparation of summary reports to external stakeholders for accountability purposes.

V. Summary

The GEAR Group will continue the long tradition of involving existing faculty governance and curriculum review structures on individual State University campuses in the process of assessment. This involvement of SUNY faculty was central in the early 1990’s when the State University was playing a leadership role nationally in the assessment movement, and it has certainly characterized the deliberations of the Task Force on the Assessment of Student Learning Outcomes that has provided the raison-d’être for GEAR.

Approved by GEAR: October 16, 2001, updated: December 5, 2003
Proposed revisions, Spring 2004

Appendix


Nationally-normed Measures

There are many nationally-normed measures available that are designed to assess learning outcomes in Mathematics, Basic Communications (Written), and Critical Thinking (Reasoning). The table below shows a number of examples of some which campuses may find useful:

Discipline
AP
CAAP
CCTST
CRA

QUANT-Q

Mathematics
x
x
x
Basic Communications (Written)
x
x
Critical Thinking (Reasoning)
x
x
x
x
x

 

AP Academic Profile, Educational Testing Service (www.ets.org/hea/acpro/)
CAAP Collegiate Assessment of Academic Proficiency, ACT (www.act.org/caap/)
CCTST The California Critical Thinking Skills Test, INSIGHT Assessment
(www.insightassessment.com/test-cra.html)
CRA California Reasoning Appraisal, INSIGHT Assessment (www.insightassessment.com/test-cra.html)
QUANT-Q Quant-Q, INSIGHT Assessment (www.insightassessment.com/test-cra.html)

Costs

The costs of the purchase and scoring of nationally-normed measures—as well as for the National Survey of Student Engagement or the Community College Survey of Student Engagement—will be paid for by System Administration for State-operated/funded institutions and community colleges, based on a sample size of up to 20% of the undergraduate student body on a three-year cycle.

SUNY-normed Measures

Campuses wishing to include SUNY-normed measures in lieu of nationally-normed measures in Mathematics, Basic Communication (Written), and Critical Thinking (Reasoning) may use one of two approaches:

iii. A locally developed instrument that measures the learning outcomes in one or more of these three areas and that is demonstrated to correlate statistically (i.e., have concurrent validity) with nationally-normed measures, including those listed above.

iv. A locally developed instrument that measures the learning outcomes in one or more of these three areas that is reviewed and approved by the GEAR Group. As part of this process, GEAR will rely on discipline-based panels of distinguished SUNY faculty that will develop standards and rubrics campuses may use to assess student performance. Campuses choosing to use their own standards and rubrics must demonstrate to GEAR that their standards and rubrics are essentially equivalent to those developed by the discipline-based panel.

Campuses opting for this second approach would also be expected to periodically provide GEAR with samples of student work for each standard of student performance, to be reviewed by GEAR to ensure ongoing validity and reliability of the measure. GEAR will provide campuses with feedback and, possibly, recommendations regarding any scoring adjustments that may be required.

Mixing and Matching

It is also possible to combine these approaches: for example, a campus may choose to use the CCTST for Critical Thinking and SUNY-normed measures for Mathematics and Basic Communication (Written).