Because course evaluations are used by reviewers to make important decisions regarding promotion and tenure, they should be interpreted appropriately. Deans, department chairs, and promotion and tenure committee members should keep the following in mind while conducting reviews:
- Course evaluations represent student perceptions.
- Course evaluations are not faculty evaluations or measures of student learning.
- Research indicates that despite some indications of bias, course evaluations remain the best representation of teaching effectiveness, and are highly correlated with other measures of teaching quality (Linse, 2017).
Recommendations for reviewers:
- Look for patterns, and examine distributions rather than means, which are sensitive to outlier ratings. Course evaluation data usually do not conform to a bell curve, and means are not useful for data that are not normally distributed.
- Ideally, course evaluations should be reviewed in conjunction with other measures of teaching (peer or other observations, course materials, teaching portfolios, evidence of scholarly teaching). See this article for recommendations.
- A complete history of a faculty member’s ratings should be considered rather than an averaged score from all their classes. Even good teachers can have an "off" semester.
- Small differences between mean scores among semesters are common and don’t mean much.
- To reduce the effect of bias (e.g., Mitchell & Martin, 2018), avoid comparing faculty to each other. Also avoid comparing faculty to unit means. If the department has many good teachers, there will be some good teachers who will fall below the artificially high mean.
- Focus on common ratings and comments rather than emphasizing outliers. Contradictory comments are not unusual, and occur in most groups of ratings. If the distribution of ratings is bell-shaped or lower ratings dominate, there may be a problem. Instructors should always be given the opportunity to address problems, and supervisors should be ready to suggest solutions. For example, you can reassure the instructor that “behaviors practiced by excellent teachers can be learned” (Linse, 2017, p. 96). Remind them that they can get support through CETL (without making CETL seem like a punishment!). Consider having the instructor work with a mentor, especially if the problems seem to be a pattern. Finally, follow up at a later date to see how the instructor's teaching methods have changed.
For more information, see this excellent article by Angela Linse.
Linse, A.R. (2017). Interpreting and using student ratings data: Guidance for faculty serving as administrators and on evaluation committees. Studies in Educational Evaluation, 54, 94-106.