Student Survey Data As Part of Teacher Evaluation

Student Survey Data As Part of Teacher Evaluation

        In this important Kappan article, Harvard senior lecturer Ronald Ferguson describes a scenario in which a principal peeks into a classroom and likes what she sees (students are busy and well-behaved) and the teacher and principal are pleased with his test-score results (they’re almost always above average). But the students, if asked, would have told a very different story: lessons are uninteresting, assignments emphasize memorization more than understanding, and the teacher seems indifferent to their feelings and opinions. In short, it’s not a happy place and there is no love of learning.

        Universities routinely survey students on how professors are performing, but until recently, K-12 students have not been given the chance to evaluate their teachers. This is because, although students spend hundreds more hours in classrooms than any administrator, people doubt that students can provide valid, reliable, and stable responses about the quality of teaching.

        The Measures of Effective Teaching (MET) project has put those doubts to rest. Comparing value-added analysis of test scores, classroom observations, and student perception surveys (using Ferguson’s Tripod questions), researchers have found that students provide accurate, helpful information on their teachers’ performance. “[S]tudents know good instruction when they experience it as well as when they do not,” says Ferguson. The research design was careful to control for students’ family background and isolate each teacher’s characteristics and impact on learning.

These robust findings notwithstanding, Ferguson offers two caveats about using student survey results to evaluate teachers:

• Any method of assessing teacher effectiveness is prone to measurement error.
• Teachers may temporarily alter their behaviors to improve their survey results, especially if students’ opinions have high stakes.

These concerns lead Ferguson to say, “No one survey instrument or observational protocol should have high stakes for teachers if used alone or for only a single deployment.” He supports the idea of student surveys being one of several measures used to evaluate teachers.

        Over the last eleven years, almost a million K-12 students have filled out anonymous Tripod surveys on their teachers, and Ferguson and his colleagues have refined the questions to the point where they pass muster with other researchers. The survey questions are grouped under seven headings, and students respond by rating their agreement or disagreement with each statement on a 5-4-3-2-1 scale:

        • Care. This goes beyond a teacher’s “niceness” to encompass demonstrated concern for students’ happiness and success. A sample question: My teacher really tries to understand how students feel about things.

        • Control. These questions measure management of off-task and disruptive behaviors in the classroom. A sample question: Our class stays busy and doesn’t waste time.

        • Clarify. This addresses the teacher’s skill at promoting understanding, clearing up confusion and misconceptions, differentiating, and helping students persevere. A sample question: My teacher has several good ways to explain each topic that we cover in this class.

        • Challenge. This covers effort and rigor and measures whether the teacher pushes students to work hard and think deeply. Sample questions: In this class, my teacher accepts nothing less than our full effort and My teacher wants us to use our thinking skills, not just memorize things.

        • Captivate. Do teachers make instruction stimulating, relevant, and memorable? Sample questions: My teacher makes lessons interesting and I often feel like this class has nothing to do with real life outside school.

        • Confer. This covers teachers seeking students’ points of view and allowing them to express themselves and exchange ideas with classmates. A sample question: My teacher gives us time to explain our ideas.

        • Consolidate. This measures whether teachers check for understanding and help students see patterns and move learning into long-term memory. A sample question: My teacher takes the time to summarize what we learn each day.

        Ferguson notes that five of these areas measure teachers’ support of students – Care, Clarify, Captivate, Confer, and Consolidate – and two measure “press” – Control and Challenge.

        What have the survey results revealed about teachers? Even lower-elementary students express clear distinctions among teachers, with greater variation within schools than between schools. Overall, the MET study has shown Tripod survey results to be valid and reliable predictors of student learning in math and ELA – in fact, more reliable than administrators’ classroom observations. Students whose teachers scored in the top quarter on Tripod questions learned the equivalent of 4-5 months more per year than students whose teachers scored in the bottom quarter. The differences in ELA were about half as large as in math.

        Not all the Seven C items are equally predictive of student achievement. When Ferguson asks audiences which of the Seven C’s they think are most important to student achievement, most pick Care. But that’s not what the MET data show. Here are the seven survey questions that correlate most strongly with achievement gains:

• Students in this class treat the teacher with respect.
• My classmates behave the way my teacher wants them to.
• Our class stays busy and doesn’t waste time.
• In this class, we learn a lot every day.
• In this class, we learn to correct our mistakes.
• My teacher explains difficult things clearly.

However, the difference between these and other Tripod items is not large, says Ferguson: “Educators should keep all of them in mind as they seek ways to improve teaching and learning.”

        What about student outcomes beyond test-score gains? “We also want attentiveness and good behavior, happiness, effort, and efficacy,” says Ferguson. The good news is that he and his colleagues have found “the same teaching behaviors that predict better behavior, greater happiness, more effort, and stronger efficacy also predict great value-added achievement gains.” It’s not either-or; it’s both, and student survey results, used wisely, can help give teachers and administrators valuable data to improve teaching and learning.

 

“Can Student Surveys Measure Teaching Quality?” by Ronald Ferguson in Phi Delta Kappan, November 2012 (Vol. 94, #3, p. 24-28), http://www.kappanmagazine.org; Ferguson can be reached at ronald_ferguson@harvard.edu

 

Stephen Anderson

Principal,