Fine-Tuning Student Perception Surveys

In this Chronicle of Higher Education article, Dan Berrett reports on the pros and cons of college students evaluating their instructors. Having students fill out questionnaires at the end of a course is not an effective way “to encourage, guide, or document teaching that leads to improved student learning outcomes,” says Stanford professor Carl Wieman (a Nobel laureate) in a recent issue of Change Magazine. Why? Four concerns have surfaced:

• Poorly designed questions – For example, many surveys ask students about things they’re not in a position to know (Is your instructor knowledgeable about course content?).
• Process versus outcomes – Surveys measure student satisfaction versus whether they actually learned; the relationship between the two may be tenuous at best.
• Unfairness – Students sometimes don’t remember key aspects of a course and rate instructors inaccurately.
• Misuse of survey results – Instructors and administrators take numerical averages as numerical gospel, obsessing over small differences in ratings, for example, 4.3 versus 4.4. 

[See Marshall Memo 589 for an article on problems with high-stakes use of student surveys.]

Ken Ryalls of the nonprofit IDEA Center in Missouri believes surveys can be effective tools for improving teaching and learning if the quality of questions is improved and the surveys are part of a broader strategy. “We’re the first ones to say that student ratings are overemphasized,” he says, and advocates looking at survey results in conjunction with peer observations and instructors’ self-reflection. Carl Wieman believes the best way to measure classroom effectiveness is having instructors submit  an inventory of the research-based teaching practices they used.

Despite his concerns, Ryalls still believes surveys contain valuable information – after all, students spend more time than anyone else with their instructors. “What drives me crazy,” he says, “is this notion that students don’t know what they’re talking about. Student voice matters.” The IDEA Center has developed questions that are being used in hundreds of colleges and universities, and Ryalls believes the results are helping instructors reflect on and continuously improve their practice. One key feature in IDEA student surveys is that instructors can include questions about the extent to which their specific course objectives were achieved. (For more on the Center, see http://ideaedu.org.)

“Can Student Course Evaluation Be Redeemed?” by Dan Berrett in The Chronicle of Higher Education, December 4, 2015 (Vol. LXII, #14, p. 9), no free e-link available

From the Marshall Memo #615