GFBrandenburg shows “value-added” teacher measures cannot work


You wanted evidence that Michelle Rhee’s plans in Washington, D.C., were not coming to fruition, that the entire scheme was just one more exercise in “the daily flogging of teachers will continue until morale improves?”

G. F. Brandenburg ran the numbers. It isn’t pretty.

Click the top line, which should be highlighted in your browser, to see the original post; or click here.

See more, next post.

GFBrandenburg's Blog

It all makes sense now.

At first I was a bit surprised that Bill Gates and Michelle Rhee were opposed to publicizing the value-added data from New York City and other cities.

Could they be experiencing twinges of a bad conscience?

No way.

That’s not it. Nor do these educational Deformers think that value-added mysticism is nonsense. They think it’s wonderful and that teachers’ ability to retain their jobs and earn bonuses or warnings should largely depend on it.

The problem, for them, is that they don’t want the public to see for themselves that it’s a complete and utter crock. Nor to see the little man behind the curtain.

I present evidence of the fallacy of depending on “value-added” measurements in yet another graph — this time using what NYCPS says is the actual value-added scores of all of the many thousands of elementary school teachers for whom they have…

View original post 255 more words

9 Responses to GFBrandenburg shows “value-added” teacher measures cannot work

  1. Ed Darrell says:

    My point was that 20,000 kids have already graduated, Morgan — can’t help ’em because they’re gone.

    Plus, getting the DVD players does nothing to staunch the waste of time, money and people spent in continuing to try to find the “bad apples” to fire them, as if a few bad teachers were all that’s holding back the district of 400 schools and tens of thousands of teachers from working for excellent student achievement.

    But I appreciate the offer.

    Like

  2. Do you think this observation means you are astute?

    No, I didn’t say any such thing.

    What I think is, to figure out what the chart “proves,” you need to know what the Y axis is (beyond what’s spec’d in the chart title, which, contrary to your claims, doesn’t define this).

    Now unless there’s something I’ve missed, and all I’ve seen to suggest that is a lot of empty posturing, this proves something else: People who agree with what the chart is supposed to “prove,” aren’t really following it. They’re just taking orders about what to think, and then doing the “Emperor’s clothes” thing where they call anyone who doesn’t similarly obey, dim or dull or slope-foreheaded or what-not. It is clear the chart is intended to be circulated only among those who already agree with the point “proven.” In other words, it isn’t good.

    For what it’s worth, the shape formed by the scatter points reminds me of another chart I saw — amount spent on education by states, along X, with resulting test score improvement along Y. The correlation is very, very weak, hardly there at all…if one does take the trouble to compute and draw a line, it’s very similar to this one. Barely sloped, upward to the right, but with lots of exceptions, almost indiscernible.

    But I’m actually more interested in the rejection of my offer to personally buy & ship DVD players. Five classrooms — no, that wouldn’t help, none of the 20,000 can enjoy the benefits of DVD until they all can, so let’s keep bellyaching about Perry until the $4 billion is restored. Classic!

    Like

  3. Bill Michaelson says:

    Yes, Mr. Freeburg, the chart creator omitted an axis label. That is an obvious oversight. Do you think this observation means you are astute?

    Hardly. The meaning of the axes is easily inferred from the title of the scatterplot. If you don’t understand this, it’s because you don’t want to.

    If you don’t know how to interpret a correlation scatterplot, you need to educate yourself first. Otherwise you have no business commenting. But the concept of requiring appropriate qualification for policy-makers is anathema to the current crop of education reformers. So your criticism falls right in line.

    Like

  4. Interesting comments thread, illustrating a dictum of Abraham Lincoln, whose advice on anything at all, alas, is unlikely to be adopted by the resident dissenter here: It is better to remain silent and be thought a fool than to speak and remove all doubt.

    Yes, it really does take all of the common elements and mold them neatly into a nice concise mish-mash, much more abbreviated than the result usually is around here…

    “Doom, despair, excessive misery” straight out of Hee Haw. All Rick Perry’s fault, and here’s a chart saying so. It’s a blob with two axes, one of which nobody even bothered to label.

    “Resident dissenter” points it out, just like the little boy out of “Emperor’s New Clothes”…Aw, it just shows my lack of savoir-faire. Give-me-an-address-I’ll-send-five, that’s straight out of the way things used to be done before the liberals put themselves in charge, people helping each other out instead of relying on some pencil-pushing pencil-neck bureaucrat — or, recalcitrant Republican governor. Just solve the problem. (Since, as a matter of record, I sympathize with Mr. Darrell’s complaint, if materials are available by video VHS would pare down the field of options rather uselessly.)

    Just more bitchin’. And I’m a simpleton. Suave, nuanced, capable people are just supposed to complain until the government benevolently and generously drops something in their hands. Complain and draw graphs that aren’t complete…but the graphs are not only complete, but wonderful, if they’re supposed to say the right things. If anyone points out the clear and obvious deficiencies, just do what Tom Sawyer did with the white picket fence, aw it’s too bad you’re slope-foreheaded to see how cool this is. Just keep it up and who knows? Maybe I’ll grab a brush!

    What a pickel. Half among us won’t maintain any optimism about anything, whatsoever, unless a democrat is in charge — in which case, we see at the national level, we can no longer rely on the press to do their jobs.

    Like

  5. Porlock Junior says:

    Interesting comments thread, illustrating a dictum of Abraham Lincoln, whose advice on anything at all, alas, is unlikely to be adopted by the resident dissenter here: It is better to remain silent and be thought a fool than to speak and remove all doubt. It’s surprising at first to see that there could be anyone in this century, not to mention the previous one, who totally doesn’t know what graphs and trend lines are about. While the math stuff about correlations is kinda advanced, requiring high-school algebra and all, the failure of the blob to show a trend is pretty clear without all of that requires-some-education stuff about statistics.

    Meanwhile, thanks for this item. The Brandenburg posting is a really nice demo. I had to look up this meaning of “value added” because I don’t follow the edu-bs literature at all, but with that simple job done, it’s highly enlightening. Also, not really very suprising, given the tenuous statistical abstractions the whole thing is based on, but the lack of significance seems to exceed my expectations.

    Like

  6. Ed Darrell says:

    That’s a very kind thought, Morgan. I’ll wager you can find a public school classroom close to you that needs something. I may have to dig for the program that allows teachers to post what they need, and have people donate it. (I have my own DVD — and computer, and internet connection, and black and white printer, and sound system, and I provide supplemental texts I have acquired, as well as the software to accompany our textst that the district has lost, or something.)

    Sending a DVD player now, five years late, won’t help roughly 20,000 students who graduated in the interim. But at least you’re only five years late and $5 million short — Rick Perry came up $4 billion short on education this year, and $4 billion for next year, so the official “Fire the Teachers Until the Kids Learn What’s Good For Them” campaign continues at least one more year.

    Research indicates student achievement rises when class size drops to 18, and rises dramatically when it drops below 15. Texas Lege and Texas Education Agency grant waivers and now want to change the law to put 36 or more student in each class. We can’t even fit all the desks in.

    So, yeah, it ticks me off that we have a multi-million-dollar program designed to smoke out the bad eggs, and it doesn’t work.

    Like

  7. Is Dynex good enough?

    Give me an address. I’ll send five.

    I’m still not clear on what I’m supposed to get from this graph; the clustering around 0,0 is supposed to demonstrate non-correlation? How am I supposed to get that, if the Y axis is undefined?

    Like

  8. Ed Darrell says:

    That was quick.

    Yeah, there’s more. Click through to the article at G. F. Brandenburg’s blog. If Rhee, and Gates, and the Texas Poobahs and countless others, were correct that we should fire the bad teachers to improve education, the dots on that chart shouldn’t all be clustered around 0,0.

    Mr. Brandenburg explained:

    In any introductory statistics course, you learn that a graph like the one below is a textbook case of “no correlation”. I had Excel draw a line of best fit anyway, and calculate an r-squared correlation coefficient. Its value? 0.057 — once again, just about as close to zero correlation as you are ever going to find in the real world.

    In plain English, what that means is that there is essentially no such thing as a teacher who is consistently wonderful (or awful) on this extremely complicated measurement scheme. How teacher X does one year in “value-added” in no way allows anybody to predict how teacher X will do the next year. They could do much worse, they could do much better, they could do about the same.

    He explained it more clearly in an earlier post:

    So, one year, a teacher might be around the 90th percentile. The next year, she might be around the 10th percentile. Or the other way around. Did the teacher suddenly get stupendously better (or worse)? I doubt it. By the time they are adults, most people are pretty consistent. But not according to this graph. In fact, if somebody is in the 90th to 100th percentile in school year 2006/07, then the probability that they would remain in the same 90th-to-100th-percentile bracket is roughly 1 in 4. If they are in the 0th to 10th percentile in 2006-2007, the chances that they would remain in the same bracket the following year is about 7%!!

    What this shows is that using value-added scores to determine if someone should keep their job or get a bonus or a demotion is absolutely insane.

    Here in Dallas the district has spent several millions of dollars on such a rating system, taking staff away from curriculum development, delaying implementation of important updates of technology — DVD players instead of VHS, for example, to run the instruction videos that we pay millions for from the book publishers, because no one does anything in VHS any more, or like updates of classroom projection systems.

    And it’s all money down the toilet. These programs have damaged the education system, hurt the education of millions of children, destroyed effective teaching teams and destroyed careers — and they don’t work at all. Not even a little bit.

    Not only was Obama’s “Race to the Top” program a bit of a waste (it saved a lot of teacher jobs and, therefore, a lot of kids’ education), but the foundation ideas behind No Child Left Behind are right out of the science of Trofim Lysenko and the Cardiff Giant, completely, utterly and totally bogus.

    Like

  9. Forgive my density, by now you shouldn’t be expecting anything better from me. But what’s the message here? So much huffing and puffing and so little hard data, I don’t see it.

    I have a graph, on which X axis is something to do with test scores, Y axis is “Axis Title.” Okay…guess I don’t need to study this too hard…there’s a big blob in the middle, looks like a shotgun blast to some unfortunate guy’s shirt…and there’s one dot way off to the right, I guess that means a really high test score. So all the other dots need to be emulating that one dot and not the other way around.

    Any other lessons to be gleaned from this?

    Like

Please play nice in the Bathtub -- splash no soap in anyone's eyes. While your e-mail will not show with comments, note that it is our policy not to allow false e-mail addresses. Comments with non-working e-mail addresses may be deleted.

This site uses Akismet to reduce spam. Learn how your comment data is processed.