National Public Education

Measuring School Success

I recently read a good article by NPR on whether money matters in public education.  Much as I respect National Public Radio, and am very pleased that they set out to show that it does matter, I would like to say a few things about their approach.  They seem to pass right over what I think is the main reason there is so much confusion about this topic.

The article starts with the findings of researchers, like James Coleman and Eric Hanushek, that, in general, increasing the amount of money available to a school has little effect on student outcomes such as test scores and graduation rates.  This is true.  There is ample evidence that, particularly in low-performing schools, increases in funding have produced almost no changes in these outcomes.

But, there’s a problem with the kinds of outcomes we are looking at, those test scores and graduation rates.  They are not a good indication of what the school is all about.  As a teacher, I never felt my job was to raise test scores or graduation rates.  I had more important things to think about, such as educating the kids.  It was their lives that I wanted to change, not their scores.  I wanted them to find themselves; to grow up and discover ways to work hard and be happy doing this.  I really believed that learning my subject would help them do this.  As Thomas Jefferson said, education can show students that “their own happiness does not depend on the condition of life which chance has placed them.”  That's what education is all about.

And besides, what teacher ever set out to have every student pass a test?  That’s not the point of testing.  You give the tests so you can find out who is having trouble.  It’s the failures that you are interested in, not the “A’s”.   And my goal wasn't that what they learned in my class would somehow help them in other classes, or on some standardized test of general knowledge.  I just wanted them to learn what I was teaching them.  That was hard enough.

Well, you might say, you should have been more concerned with their test scores.  Should I?  Is that what you would want for your own child?  A teacher whose main emphasis was on the data produced by the child rather than the child himself; some kind of classroom bureaucrat who was focused on whether he, the teacher, was succeeding?

So it is perfectly logical that more money doesn’t change those test scores.  The teachers don’t want it to.  They want to use the money to help make the kids more mature, more self-confident, more successful as adults.  And this often does work.  There are studies, as the NPR article noted,  that better resources in a grade-school classroom do affect things like adult earnings, even though they don’t raise test scores.  Of course, though, such things are rarely measured by the schools, since they have no way to track the kids into their adult lives.

A good way to understand this whole topic is to consider what we do in other public programs.  Suppose we wanted to see whether more money for the fire department was producing better results.  But the results we chose were those we could easily measure, such as response time to the fire, or the amount of water used, or the cost of the property damage.  But are these things what firemen care about?  Aren’t they in this profession because they want to save lives, and actually do this regularly?  Wouldn’t they spend the money, then, on how to save those lives, not on how to improve water use?  So it might well happen that more money did not change the kind of outcomes we were measuring, but not because money didn’t matter, but just because we were trying to measure the wrong outcomes.

To its credit, the NPR story does raise this question of whether we are looking at the right outcomes, but only at the very end, and only then as one of many issues to be resolved.  They miss the importance of this issue.   We are all using the wrong metric to evaluate school success; a metric that no one in the schools takes very seriously, and that we don’t want them to take seriously.  We want them to improve the kids, not the testing data.

If we did want to evaluate the schools more seriously, I would suggest: a) longitudinal data on the students on into their adult years, and b) data on the growth of individual students, not average scores.  Many schools get in new kids with low scores every day, and we want them to; someone has to educate those kids.  But their average scores, then, stay low no matter how well they raise the scores of each kid.  It is quite possible to be a very good school with low average scores.

Peter Dodington

June 17, 2017