|
Doctoral Handbook
Research Tools Exam
(PLEASE NOTE: Prior
to beginning the Research Tools Exam, a student must
have completed a minimum of four inquiry
courses.)
The University at Albany requires that doctoral
students demonstrate competence in “research
tools” appropriate to their professional specialties.
This demonstration of skill with research tools
must be in addition to formal coursework applied
to the graduate program. To demonstrate these skills
for the doctoral degree in Educational Theory and
Practice, students will complete the Research Tools
Examination, which is designed to give students
an opportunity to demonstrate their skills in reading
and critically evaluating educational research,
including assessing that research along pragmatic,
technical, theoretical, ethical, and philosophical
lines. Three examples follow: Example
#1 (2002), Example
#2 (2006) and Example
#3 (2009).
The Research Tools Examination will be conducted as follows:
| 1. |
The student will notify the Department when she or he is ready to begin the exam. |
| 2. |
The Department will assign an Examination Committee consisting of two faculty members, one of whom will be designated as Chair. |
| 3. |
The Committee will identify 4 research-based empirical articles from recent issues of the American Educational Research Journal, selected to represent substantial treatments of both qualitative and quantitative methodologies. From among those four articles, the student will select 2, with approval of the Chair, and will develop a written critical evaluation of each article, according to the Criteria described on the following pages. The selection of two out of the four articles must maintain a balance of methodological approaches. |
| 4. |
The student may use any relevant
resources in developing the critique, including
consultation with faculty and other students.
However, the written work must reflect the student's
own thinking and writing. |
| 5. |
The Examination must be submitted within three months of when the two research articles are agreed upon by the student and the Chair. |
| 6. |
The Examination Committee will assess the adequacy of the Exam in addressing the Criteria defined below. The Committee will meet with the student for an oral examination to discuss the student's written critiques. Based on the written critiques and the Oral Examination, the student can earn a grade of "Pass", "Revise" or "Retake". |
| |
|
"Pass" indicates that the student has demonstrated critical facility with the understanding and critical evaluation of research methodology. |
| |
|
"Revise" indicates that the student has shown uneven competence, adequately addressing some issues but inadequately addressing others. Students will be asked to revise identified weaknesses within a specified period of time (to be determined depending on the nature of the revisions). |
| |
|
"Retake" indicates that the student has failed to demonstrate a critical understanding of the research methods and their critical evaluation; exhibited misunderstandings or errors; did not respond completely to the criteria; or did not note important strengths and weaknesses across the articles. |
| |
|
A student who revises her or his exam can receive a grade of "Pass", "Retake", or "Fail". |
| |
|
A student may take the exam no more than 3 times (2 “Retakes”) before receiving a “Fail". |
Criteria for the Research Tools Examination
This examination is intended to entail critical
evaluation of research articles, not just a summary
of them. The members of the Examination Committee
will assess a student's depth of understanding of
research methodologies, including the sophistication
of the student's evaluations along pragmatic, technical,
theoretical, ethical, and philosophical lines.
Each article should be addressed in a separate
paper, written in complete APA style (or another
accepted scholarly style, with previous approval
of the Examination Committee Chair). The length
of the papers should be a maximum of ten pages (not
including the reference page) and double-spaced
with 1" margins.
Each paper should be written in two parts. The first part will describe the research, in about one paragraph. The bulk of the paper should be the second part, providing a critical evaluation of methods and interpretations. Using the lists below as a general guide, each paper should be written in coherent, continuous prose, addressing issues that are pertinent to the article. (Do not write a paper that addresses these points in order, using the list as if it were a checklist. Do write a thoughtful, professional critique of the article that addresses as many of these issues, or other issues not on the list, as are relevant.)
Part I: Describe the overall study.
In one page or less, describe the following aspects of the study. Using your own words, interpret and summarize what is presented in the paper and, if necessary, translate it into terms that are accessible to the non-specialist.
| 1. |
Identify the research question. |
| 2. |
Summarize the theoretical framework guiding the study. |
| 3. |
Describe and characterize the research methodologies (both design and data analysis) employed. |
| 4. |
Identify the authors’ conclusions. |
Part II. The critical evaluation.
Evaluate and critique the design and implementation of the study. In developing this part of the paper, you should consider at least the following issues and discuss ways they are addressed appropriately or, if they are not, identify and discuss any gaps and concerns.
| 5. |
Have the research questions been adequately addressed? |
| 6. |
Do the authors provide adequate justification for their choice of design, selection of sites and/or participants, and data collection and analysis techniques? |
| 7. |
How do the design, site and participant selection, data collection and analysis methods, conclusions, and interpretations relate to the research questions and theoretical framework?Are each of these aspects of the research appropriate? Are they well-implemented? |
| 8. |
Is evidence appropriate to the research methodology provided of the reliability and validity of the measures and conclusions? Explain. |
| 9. |
Are there other possible interpretations of the results? |
| 10. |
Discuss any ethical issues raised by this work. As appropriate, include: |
| |
a. |
Human subjects and other issues concerning the research participants. |
| |
b. |
Honesty and completeness of the report. |
| |
c. |
Sensitivity to inclusion or exclusion of particular classes of people. |
| |
d. |
Sensitivity to ways that the research might be used inappropriately or harmfully. |
| |
e |
Fairness in the treatment of work of colleagues. |
| |
f. |
Controversies (either theoretical, methodological, or other) addressed or ignored by the authors. Are controversies treated in a balanced, fair manner? |
| 11. |
Reflect on the completeness and importance of this paper. |
| |
a. |
What open questions remain? |
| |
b. |
Are there any alternative methodologies or methodological issues that you see as missing? |
| |
c. |
Do the authors identify weaknesses in their own work? |
| |
d. |
What other issues/concerns/questions do you have with this work?
|
| |
e. |
What is the importance of this work? |
Click to get Research Tool Exam Form
|
|
