VAM and Merit Pay

One of the most popular elements of the current business-model education reform movement (a movement to which I am vehemently opposed) is the use of Value-Added Models (VAM) to measure teacher effectiveness. Some states and districts are proposing the use of VAM to determine merit pay.

The simple version is this: The student is tested at the end of each school year. You take the kid’s score at the end of Year 1 and compare it to her score at the end of Year 2 and that tells you how much value the teacher has added, supposedly independent of factors like family, poverty, etc.

Diane Ravitch recently linked to an excellent piece written by mathematician John Ewing, president of Math for America, an organization dedicated to helping those non-Emmy Noethers among us to better understand — and apply — mathematics.

You should read his entire piece, because it gives an excellent rundown of where VAM comes from, how it was connected to education (and some of the early warnings that came with it!), and why it is problematic to link it to high stakes (like harsh penalties for kids, merit pay, or sanctions for schools).

The bit which jumped out most to me is from a report by the Economic Policy Institute, “Problems with student scores to evaluate teachers“. (Full disclosure: Ravitch is one of the authors of the report.) Among other things, the report notes:

For a variety of reasons, analyses of VAM results have led researchers to doubt whether the methodology can accurately identify more and less effective teachers. VAM estimates have proven to be unstable across statistical models, years, and classes that teachers teach. One study found that across five large urban districts, among teachers who were ranked in the top 20% of effectiveness in the first year, fewer than a third were in that top group the next year, and another third moved all the way down to the bottom 40%. Another found that teachers’ effectiveness ratings in one year could only predict from 4% to 16% of the variation in such ratings in the following year.

Think about that next time some politicians starts bloviating about how we need merit pay in order to get rid of all those “bad teachers”. (For the record: Yes, of course, some teachers suck. But merit pay and high-stakes testing are not good ways to fix that problem!)

2 comments to VAM and Merit Pay

  • Dillon

    Merit Pay does not have to be directly linked to test scores or student performance. You may argue that teaching has intangibles that cannot be accurately measured for performance, but the same is true for almost all jobs and yet administrators still find ways to set a salary for these employees. A combination of student results, and evaluation by colleagues, students, administrators, and parents, as well as numerous other factors could be considered to introduce a merit pay system.

  • esp

    There are obviously shortcomings in any system of compensation. The main argument against the current model of seniority- and education-level-based pay for teachers is that no matter how crappy the teacher’s teaching is, s/he will still get paid the same amount (and get more every year). I understand this concern.

    However — even though I know a few teachers who don’t put in good work in the classroom — this does not seem like a huge problem to me. No one stays in a job that requires so much stress, paperwork, energy, and time unless s/he has a tremendous passion for education. And when teachers DO screw up and get on the Slacking Boat, there are many ways to deal with it other than docking pay.

    Merit pay opens the door to punishing good teachers based on many, many factors beyond their control. (And of course you’re right — every job suffers from this problem. But imagine we paid cops less if they didn’t prevent crimes from happening. Or paid dentists less if their patients got more cavities.)