The Challenge of “Calibrating” Teacher Observations

When districts strive to provide great training for their administrators on doing high-quality observations and evaluations, I’m delighted.

But there’s one goal the process can never achieve, and it bothers people to no end.

The unreachable goal? Calibration.

“But without calibration, how can teachers be evaluated fairly?” the concern goes. “If one administrator’s assessment would be different from another’s, how is this a defensible system?”

Of course we don’t want teacher evaluations to be capricious or unreliable. And using multiple evaluators for high-stakes evaluations isn’t a bad idea at all.

But what most districts try to do doesn’t work.

Arrows

Calibration Training

Here’s how calibration training typically works: One school (or a vendor) provides a video of a lesson, and all the administrators watch and do a practice write-up.

Then, the calibration begins.

“Why did you give her a 3 for ‘monitoring student engagement’?”

“I gave him a 2 for ‘having a clear objective for the lesson.'”

And the debate begins…but it never goes anywhere.

And it can’t, because the observers don’t have the information they need.

Missing Context

I don’t believe teaching is like auditioning for a musical, where you can do a scene or a number and give the casting director a good sense of whether you’d be right for the show.

When you teach, you’re not just showing up and doing a little song and dance. Some of the most important work happens behind the scenes, when you’re phoning parents, planning lessons, reviewing student work, collaborating with other educators, and doing the million other things that go into great teaching.

Good evaluation frameworks account for this behind-the-scenes work, but even when it comes to understanding what’s happening during a lesson, there’s essential context that a stranger won’t know.

As a supervisor and colleague, you know things about your teachers and their students that are essential for conducting a fair evaluation, and even for truly understanding what’s happening in a lesson.

Ten Things Administrators Know About Their Own Classrooms

Here’s what’s missing when we do practice observations from videos, or even when we observe other schools’ teachers during site visits. We don’t know:

  • What the teacher taught yesterday, and what’s happening tomorrow
  • What went well and what didn’t go as planned yesterday
  • Which students are having a hard time lately
  • Which students have IEPs, behavior plans, or other systems in place
  • What the teacher’s team decided jointly to do for this unit
  • When the teacher’s dog died (true story…)
  • What routines and procedures the teacher has in place
  • How today compares to the typical day in this classroom
  • What the teacher learned about students’ understanding yesterday
  • How today’s lesson connects to upcoming plans

You can’t know this context if you’re watching a video.

But you also can’t know it if you aren’t in classrooms regularly, and if you aren’t involved in planning and collaboration meetings.

(If you’re ready to develop the habit of visiting classrooms regularly, so you know what’s going on, check out the 21-Day Instructional Leadership Challenge, which currently has more than 1,400 participants).

The bottom line? Evaluation can only be done well in the context of a teacher-supervisor relationship. It can’t be simulated, at least not very well. And it can’t be calibrated, except among administrators who actually work in the same school and have the same information.

How To Help Administrators Do Better Observations

If you want to do an exercise of this type, forget about calibration, and focus on the quality of the written evaluation.

Don’t worry if one principal thinks the lesson was great and another thinks it was terrible. Focus on the quality of evidence the principal provides in the written report.

Share good and bad examples, talk about the difference, and give feedback on how to make the final product stronger.

As a side benefit, principals will develop a clearer understanding of the evaluation criteria you’re using, and this will directly aid the calibration goal.

Share:Email this to someoneTweet about this on TwitterShare on Google+Share on FacebookShare on LinkedIn
About Justin Baeder

Justin Baeder helps school administrators increase their productivity through the High-Performance Instructional Leadership Network. Learn More »

Speak Your Mind

Google+