Semi-Convincing Tests

Talking Wire below I mentioned that I'd always found the "teaching to the test" critique of No Child Left Behind to be fairly unpersuasive. In one of his rare bits of persuasive-to-me rhetoric, George W. Bush observed "I've heard people say you're teaching the test; if you teach a child to read, they'll pass the test." That seemed right to me. In some subjects -- history comes to mind -- I can imagine an effective "teach the test" method that doesn't actually impart any historical knowledge. For reading and basic math skills, however, the easiest way to teach kids to pass a test seemed to me to be teaching reading and basic math. Indeed, I recall that my AP Physics class involved a hefty test-prep element, but fundamentally this was accomplished by . . . teaching me Newtonian Mechanics.

That said, I thought Episode 9 of The Wire did, in fact, successfully dramatize an example of "teaching the test" in a plausible way. Craig Jerald at Education Sector steps up to the plate to try and un-worry me. He's only semi-convincing. He brings good evidence to bear that real teaching is a more effective way of improving test scores than is simple test prep. That, however, isn't evidence that schools are not, in fact, doing what The Wire portrays them as doing. What's more, the presumption behind the whole fix-the-schools drive is that the schools were doing a bad job ex ante of teaching reading and math. So you have a bunch of people who have not, historically, hit upon good methods of imparting reading and math scores to their kids. Now you tell them there will be consequences unless test scores go up. Sure, the best way to get them to go up would be to start teaching reading and math better. But if you're talking about a bad school, then presumably the teachers and administrators haven't found a way to get this done. So, instead they adopting the semi-effective method of doing narrow test prep. And the scores go up -- at least somewhat. Then we proclaim ourselves cured of the bad schools problem. And yet, nobody's learned.

Now, on the other hand, as true as that might be, it's not clear how not testing would make things any better.

Comments

Not testing would free up more time to teach stuff, for one thing.

Posted by teofilo | November 13, 2006 6:19 PM

"it's not clear how not testing would make things any better."

Better for whom? Better for what?

Not testing means that the teacher's union can pretend that there latest demand will fix schools without ever allowing us to find out if any of their previous demands have worked. Not testing means that it is difficult to identify and fire bad teachers--a plus for the current formulation of the union.

Posted by Sebastian Holsclaw | November 13, 2006 6:32 PM

Matt sez: "Sure, the best way to get them to go up would be to start teaching reading and math better. But if you're talking about a bad school, then presumably the teachers and administrators haven't found a way to get this done."

You're getting close to the truth. The reality, based on my experience with 3 kids currently in some pretty good public schools in So. Cal., is that even good schools are so afraid of losing Federal money (and State money tied to the Federal money) that they're playing it safe and doing narrow test prep. And then there's the matter of the larger school district, which decides on a curriculum, orders books, etc. on a district-wide basis. So if a school district (like ours) has a mix of underperforming and overperforming schools, the *district* as a whole will mandate materials and curriculums aimed at the underperforming schools. Hence, every school, good and bad, gets the same narrow test prep materials as the lowest-common-denominator means to the end.

We've seen this over and over again, from my fourth-grader getting mind-numbing reading worksheets of factoids that are devoid of social, historical, scientific, or any other kind of context, to my highly literate sixth-grader forced to sit through two separate English classes, leaving no periods free for "electives" like music, foreign language, art, etc.

Now, really good, experienced teachers and administrators find a way to bootleg real knowledge into the classroom anyhow. But they're fighting the top-down NCLB system to do so.

Posted by Steve | November 13, 2006 6:55 PM

I think the concern over teaching to the test in grounded in the same motivation that makes doctors complain about evidenced-based algorithms. Doctors think that using the weight of their own experience will produce better results than some following some rote technique or simple rule. For excellent doctors, it's true: their judgment is better than a standard form. But for most doctors, and on average, following a standard is much better than leaving it up to individuals.

For teachers, the same thing holds true. A great teacher is going to be better if given freedom to teach what they want. But students will get better results from a mediocre or even good teacher when material is standardized.

I'm sure, though, that standardization feels like a personal insult, which is too bad.

Posted by guilty | November 13, 2006 6:56 PM

One problem with "teaching to the test" is that it only allows you to teach testable things like reading and math. That cuts out arts and performing arts in particular, which have been shown to take the hardest-to-reach kids and bring them to a higher level than any other kind of program. Link

Posted by Antid Oto | November 13, 2006 6:58 PM

But students will get better results from a mediocre or even good teacher when material is standardized.

That's an argument for a standardized curriculum, not necessarily standardized testing.

Posted by teofilo | November 13, 2006 7:01 PM

I am just a few weeks into being a volunteer after-school math tutor for kids in the DC public schools. It is already clear to me that their math education consists of learning how to solve certain kinds of problems so that they can solve similar types of problems on exams. There does not seem to be any attempt to explain the intuition behind any of what they're doing, nor to explain what any of it is good for (by this I don't mean what it is good for in the student's daily life, I mean why it is useful for *anyone*).

Posted by David J. Balan | November 13, 2006 7:25 PM

Another major problem with high-stakes testing is that it's a terribly blunt, ineffective way to pretend to measure teacher performance. Standardized tests are designed to gauge where an individual student is, not where groups of them are. NCLB measures "progress" by comparing this year's third graders to last year's. Different samples, of course there will be different results. That doesn't really tell you anything about the teachers, or even about the school. The only way to measure the job the school is doing would be to track the same kid's scores from one year to the next, but as I understand it that's not how it's done. It would be a lot more difficult record-keeping, for one thing, to track every single student in every school in a district.

In addition, it's simply ridiculous to pretend to hold school systems accountable for bad teachers when they can't fire them. Worse, NCLB demands constant improvement, so schools are faced with a ratchet effect: if they do improve, they have to beat that improvement in future years.

Posted by Antid Oto | November 13, 2006 7:29 PM

"Teaching to the test" isn't such a bad thing if you're actually dealing with a serious test of skills, but quite often you aren't. The test Bush used in Texas is a perfect example, take a look at it sometime.

What you'll find is a very strangely organized test, with questions that seem bizarrely off-the-wall. But after a while you'll realize there's a dozen or so patterns that are used over and over. And the best way to take the test is to skip the reading passage, look at the questions, recognize the patterns of correct answers, and perhaps skim the reading passage for keywords.

With tests like this, and they're pretty common, you "teach to the test" not by teaching the subject matter, but by repetitiously giving kids test-taking tips for the specific test. And since the test scores go up, parents assume their kids are learning something, and don't find out differently until a few years later when everybody flunks the SAT (which is what happened in Text).

Posted by david | November 13, 2006 7:40 PM

"Test", at least in the abstract is meant in the strict empirical sense. Like a scientific experiment, you're supposed to be testing to see how a student's performance is affected by the teaching going on. Now students throughout the ages have discovered and refined techniques for gaming this system to their benefit. This makes tests somewhat less than scientific experiments. Teachers have to be ever more clever to get a true measure of their student's progress because they have to be able to distinguish the actual learning from the test-taking skills on display.

These "teach to the test" issues boil down to the same game played at the school administration level. Now it's the teachers employing the same techniques they learned as students to squeeze every last point out of their class' performance.

One of the problems with this is that those very techniques are themselves valuable. Figuring out how to game the system can mean the difference between success and failure in a whole host of real life situations. NCLB is removing the benefit that students might otherwise get in learning how to take tests.

How's that for an argument?

Posted by stand | November 13, 2006 7:40 PM

I'm in the side, "teach for the test" is bad medicine. Most of my points have been well made by others. But here is one more.

Metrics based on cross-sectional measures of progress are prone to huge errors and rigging the system. So, for example, if this year's 4th graders are a better group than last year's, scores will go up; if worse, scores will go down. A much much better methods would be to longitudinal. To see if individual student's scores, on average, improve or go down. Longitudinal assessment is the gold-standard in epidemiology and should be in school assessment. With the cross-sectional method, getting weak students not to show is a time-honored method of improving scores. With a longitudinal method, this doesn't work. Perhaps, more significantly, longitudinal assessment is a much greater aid to individual students.

Posted by John | November 13, 2006 7:50 PM

I agree with John & Antid: the best way to measure educational success is to follow individual kids as they move through the system. This is the only way to show what value the school is adding to the kids' education.

Another aspect of gaming the system that NCLB encourages is a focus on those kids closest to the proficiency line. The law is obsessively focused on getting X% kids over the proficiency bar, defined as Y score on the standardize test. It does not care where those kids came from. Thus, schools get credit for pushing kids near the line over the top but zero credit for making perhaps much greater progress in moving a kid from the very bottom to near the proficiency line or for moving a kid from proficient to advanced proficient.

In our district, this led the school board member to advocate focusing resources on those kids closest to the proficiency line. To hell with the rest, including those already scoring above proficiency. This type of gaming, that effectively denies equal education effort to kids already above proficiency or too far below to matter, can only be eliminated by adopting a year-to-year progress model for individual students. Then, schools will get credit for making progress for EVERY student.

Posted by mert7878 | November 13, 2006 8:06 PM

Teaching to the test is the least of NCLB's problems - though teaching to the test, as someone up the thread pointed out, limits the subjects our students are taught to language and math. Say goodbye to art, music, science, history, social science and foreign languages. The subjects schools are forced to care about are the ones that keep them open, and those are language and math. (I think I read that there's a science test being added somewhere down the pike, though if it turns out to be creationist science, we'd be better off to skip it.)

The real problem with NCLB is that it's a time bomb. It establishes a schedule of gradually increasing percentages of a school's student body who must score at the 'proficient' level on standardized tests. (This applies not just to the student body as a whole, but to each individually-defined subgroup as well. So, if the required goal is 28%, then the student body, English language learners, economically disadvantaged, special ed, and whatever other groups are identified must all produce a 28% success rate on the SAME test.) Once a school fails to meet its defined goal for two years running, it is required to divert part of its budget to various added expenses, like busing students to other schools and providing tutors. With each additional year that the school fails to meet its goal, the sanctions become more severe. Within five years or so of that first failure, the school is closed.

Here's the kicker: by school year 2013-14 (unless it's 2012-13, I can't find my schedule), NCLB requires that 100% of the student body of every school accepting Title I funds must score 'proficient' on standardized tests. This is simply impossible, for a lot of reasons which have nothing whatsoever to do with the quality of our teachers or their teaching. There will always be some percentage of kids who will not be able to score at that level, no matter what anyone does for them.

Even if all the schools in the country manage to eke out passing scores to that point (which many of them are doing now by lowering standards for proficiency and forcing failing students to drop out), five years after that 100% goal hits, we will have no more public school system. Teaching to the test? That's just a distraction. This law is an abomination. It should be gutted or, better yet, repealed.

I've had kids in public schools since 1975 and my youngest won't graduate until 2010. I'm a long-time PTA member, a volunteer math tutor at the middle school level, and a member of various school and district parent-advisory groups. I'm not saying there aren't reforms that are needed - but I am saying that NCLB isn't about reforming or improving our schools. It's about closing them.

Posted by cmac | November 13, 2006 8:08 PM

I am a graduate student at Middle Tennessee State University, and I am currently in the middle of writing a thesis; for my research, I have designed an assessment model for a tutoring program on campus, which has me plunging into some deep issues about writing assessment. In particular, researchers have found, quite consistently, that aside from mechanical proficiency, such as grammar, spelling, and sentence structure, you can't measure much with tests. Most essays are judged based on a rigid, 5-paragraph or thesis-driven format, which simply does not detect things such as creativity and critical thinking skills; in addition, essays tests often collect short, impromptu samples, which goes against almost three decades of composition research that shows students produce the best writing when they get the opportunity to create multiple drafts and revise. This sends the wrong message to students -- we ask them to engage in the writing "process," then we determine their proficiency based on a 30-minute sample produced under stressful conditions.

In other words, tests generally suck at measuring writing proficiency. In addition, they carry a "hidden curriculum," an underlying message about writing: they encourage students to imagine writing as the boring, formulaic process of responding to a prompt. (Actual prompt from a writing test: "Imagine you are a raindrop. Describe what happens to you.") I am not sure about reading and mathematics, but in composition research, most scholars are generally agreed that writing tests are a poor way to measure student aptitude.

Kennie Rose

Posted by Kennie Rose | November 13, 2006 8:25 PM

One problem with NCLB is it leaves the choice of test up to the states. Each state gets to make up its own test with little federal oversight. There's an obvious conflict of interest problem here -- but perhaps in the minds of George W. Bush and Ted Kennedy, that's not a bug, that's a feature. Typically, a state introduce a test that starts out very hard. Everybody flunks. Then, in subsequent years, amazingly enough, scores rise rapidly. All the politicians take credit. Everybody is happy!

Meanwhile, on national tests put together by disinterested third parties, such as the NAEP, SAT, and ACT, scores are rising ever so slightly.

Posted by Steve Sailer | November 13, 2006 8:27 PM

Why are we emphasizing testing individual students via the NCLB anyway? The common defense of it always refers to some practical function that education will prepare students to fullfil in the real world. Actually, however, they are being prepared for the real world of the 18th century, where you carried information around mostly in your head. Since then, though, the knowledge skill you need is how to access knowledge - and to do that requires groups, teams, communities. Just the kind of things students are not trained in at all in testocentric environments.

The whole testing culture should be redesigned to manage what it bars, at present, as cheating - that is, the social practice of learning. The forced individualisation of learning, and the testing system in particular, has nothing to do with education and everything to do with accumulating crazy numbers so that middle aged kids can wave around transcripts that will get them into 'good' schools. It is meritocracy gone haywire, in which merit doesn't match the real knowledge environment.

Anyway, the NCLB is, perhaps, Bush's most pernicious gift to the country, even taking into account his other acts of criminal negligence, corruption, and plain dumbness.

Posted by roger | November 13, 2006 8:27 PM

One thing about Newtonian mechanics is that it's a closed system, and thus "teaching to the test" is just teaching the subject. Teaching reading--above the level of word recognition--is quite different: the test must find out whether a student is integrating the material with other things s/he has encountered. (It is of little value to recount what one has read if one can't say what it MEANS.)

Posted by dj moonbat | November 13, 2006 8:46 PM

The problem in underperforming schools (at least at the one I was at) is the students. Far too many just don't care. That's not going to be solved by the teachers or curriculum. It's not a pleasant thought, and the most successful way to change this would be wholly unacceptable to Americans. So, it will never get better. Blame the kids

Posted by a | November 13, 2006 8:46 PM

I'm going to leave the subject of testing and ask Matt a personal question.

What did your parents pay in tuition for your high school? for college?

As I remember, Matt went to a New York City day school and Harvard.

I'm not trying to be nasty, but I'm raising the possibility that a quality education costs more than most are willing to pay. Conventional wisdom is that the costs of New York private schools are way too high. Flip that around and suggest that we're simply not spending enough, in general, on k-12 education.

The consensus is that the US as a third-rate k-12 educational system and world's best college system. We seem to be willing to spend much more per kid-year for college than k-12. Perhaps there is a connection.

I'm not at all convinced of this argument, but its a serious one. Testing is an attempt at a quick fix (or a smokescreen) to avoid dealing with more difficult and costly issues.

Posted by John | November 13, 2006 8:54 PM

"quality education costs more than most are willing to pay."

You could spend $20,000 a student and it wouldn't make a difference unless you're willing to get rid of the troublemakers. Simple as that. You can't teach a class when 4/5 of the class is fucking around.

Posted by a | November 13, 2006 8:56 PM

Obviously, NCLB is pretty ridiculous and corrupt. The only way every student in the U.S. is going to be "proficient" at everything by 2014 is by massive, shameless cheating on the part of all concerned.

On the other hand, it may also be that NCLB is better than what it replaced. Educational policy in the United States is hilariously stupid, ripe for a "Catch 22" satire.

The fundamental problem is that, except in Lake Wobegon, half the kids in the United States are below average in intelligence, but nobody in the educational industry wants to admit this publicly. Thus, educational policies are based on wishful thinking rather than realism.

Posted by S | November 13, 2006 9:24 PM

The problem is not the intelligence or the unruliness of American kids. The problem is a system mired in the 19th century. From poor teachers to poor practices to poor curruicula. No child left behind is an 18th century solution to a 19th century problem. I don't think we value education sufficiently to put sufficent money and effort into fixing things.
Personal experience. I go to the school board meetings in my suburban town frequently. I go to about half the meetings and find out what's missing in the others. Many issues have motivated parents to attend meetings and speak to the board. The biggest issue was whether the school should have a hockey team. Next came the issue of adding artificial turf to the football field. There have been about 6 major issues over the past 2 years. Not a single one has involved an academic issue. When I talk to parents, the principal issue is what it takes to get into college, never what it takes to have a sound high school education. High school should be about high school. Students, like my daughter, focus on trying to decode each teacher's intentions rather than eduation. I could go on. For example, as a scientist I can see that high school science education is not science.

I do get angry when I see posts like the ones above that say that the problem is that half of the students are below average intelligence or that even with $20,000 per kid per year you couldn't teach unruly american kids. These are the worst kinds of cop outs. A serious look at American education shows a systems that is not working. this is true in the middle-class suburbs and the urbs. Its not the kids, its us. We owe a much better effort.

Things like 'no child left behind' are mindless attempts at a fix -- like the silly phonics/whole language controversy.

Let me give 2 examples of what I think a adequate high school should have:

1. foreign language selection. How many high schools offer Chinese, Arabic or Russian? Very few. My local school offers French, German, Italian and Spanish. Why? principally, because that's the way it was 50 years ago, and things change very very slowly.

2. computers. How many high school kids have their own laptop computers? How many high schools have 1:1 laptop programs? A few minutes of casual thought makes it obvious that computers are essential tools for education, at least at the middle school level and beyond. $1000/kid/3 years is not a lot. People are begining to work on a $100/computer laptop program for the poorest regions of Africa. Yet I would guess that less than 1% of high schools have laptop programs. This is inexplicable.

In short, the educational system sucks because we really don't want to fix it.

Posted by John | November 13, 2006 10:18 PM

NCLB is just our Prussian school system taken to its logical end.

I hope it goes over the edge.

Posted by Linus | November 13, 2006 10:29 PM

"I do get angry when I see posts like the ones above that say that the problem is that half of the students are below average intelligence or that even with $20,000 per kid per year you couldn't teach unruly american kids. These are the worst kinds of cop outs. A serious look at American education shows a systems that is not working. this is true in the middle-class suburbs and the urbs. Its not the kids, its us. We owe a much better effort."

A school board meeting is not an urban school. When a typical reaction to, "Jesus, will you please take your headphones off?", is, "Fuck you", more computers, better books and more qualified teachers are not going to make a difference. It has nothing to do with commitment.

At my school, the Latin Kings burned a the books belonging to four different sections and left their symbol all over the room. And this happened in the middle of the school day. When a speaker came into the classroom to talk about the kids future, one student made a fairly convincing argument that drug dealing offered him the best chance of happiness. Anecdotal evidence like this isn't the reason to have zero faith in the current educational system. It's the naive belief that teachers can change a whole society.

The kids aren't stupid and they've heard every line that you can throw at them. They simply don't care.

The best solution is to create more charter schools and allow students who are not disruptive to enroll. I would think that in the long term this might exert a social pressure to succeed in school, with more and more minorities valuing education. But that 5-10% of students can't possibly learn when they are surrounded by peers who have no intention of learning, or allowing learning to take place.

Posted by a | November 13, 2006 10:55 PM

The real solution to the education problem is to test someone else: prospective parents. People that will be horrible parents should not be allowed to reproduce. Their offspring will be a huge drain on society and no amount of schooling will fix them.

Posted by miguel | November 13, 2006 11:51 PM

I'd like to point out that we took standardized tests all the time when I was growing up outside D.C. in the 70s-it was called the SRA back then. What NCLB has done in my new home state of Colorado is dramatically increase the frequency of testing. And this costs a lot of money! So is way, way more testing a good use for that money, compared to say reducing class sizes?? NO WAY. There's clear research on this, the best way to get better results is a lower student:teacher ratio. Even a crappy teacher can do better with 20 kids than 32! Why don't the politicians want to do this? This is really, really expensive, because you need way more classrooms, or you need to go to year-round schools, which are a great idea. You can fit 1/3 more students by going to a staggered year-round schedule.

While I was in my doctoral program in educational psychology at UC Berkeley, I asked every teacher I had, "Does teaching spelling teach kids to spell?" And the best answer I got was, "We have a lot of practices in education that haven't been empirically validated." I think that part of NCLB is what got Democratic support: we do need research-validated practices.

Posted by Laurie Roberts | November 14, 2006 12:07 AM

Does anyone know which European country has the highest per-pupil spending?

Linus does.

(Drum roll.)

It's Denmark kids. And do you know what kind of education system those funky Danes have?

Not only do they directly fund public schools, and directly fund state-sponsored private and parochial schools (like Holland and Sweden); they provide partial tuition remission for independent schools as well.

Let me suggest to you that the level of funding for education in Denmark and the nature of the school system in that country are related.

You're never going to have a richly funded k-12 education system in America without enfranchising people of faith (who have felt increasingly disenfranchised by our public school system for the last 40 years) as well as the wealthy elites (who unlike in say Japan don't send their kids to public schools [and we wouldn't want our schools to be especially Japanese anyhow]). And the only way you're going to enfranchise people of faith and the rich is by adopting some variation on the Dutch, Swedish, or preferably Danish model (which has the most extensive level of choice in Europe).

In my modest view, a four-tiered system would be preferable. The first tier would be the public and charter schools, which would continue to be overseen by local school boards and regulated by the states and federal government. The next tier would be foundational schools, which could be founded by groups of parents or teachers along any denominational lines or educational philosophy. They would receive both foundational and developmental funding, and be overseen by independent school boards (as similiar schools are in the Netherlands), as well as regulated by the states and federal government. The third tier would be (progressive) per-student funding (as in vouchers) for pre-existing independent schools. The only prerequisite for receiving taxpayer funding would be accreditation by that national organization which accredits independent schools, and in exchange for otherwise staying out of the business of these schools they would be ineligible for foundational or developmental funds. The fourth tier would be a modest tax credit (perhaps $1500 to start) for homeschool parents. It would cover seminar costs, educational materals, educational travel, etc.

Northern European schools are among the most successful in the world, and even at a time of limited national pride in these countries (an astonishingly low percentage of Dutch people are proud to be Dutch), their citizens are wildly enthusiastic about their schools. I see no reason to think the same wouldn't happen here. If America adopted this kind of system people - even conservatives - would allow their elected representatives to cut education funding over their dead bodies.

Posted by Linus | November 14, 2006 12:26 AM

I was a bit disturbed by the idea that a kid could clean up at dice gambling with some cursory teaching. That really struck an idiot box sitcom note for me, as there's no way in hell that's reasonable. Even biggest idiot louts in the world will go a long way for an edge when it comes to gambling, and some trivial math seems well within that domain.

Posted by Turkey | November 14, 2006 1:13 AM

First of all, it is very possible to teach test prep without actually effectively teaching the underlying subject. This is especially true on multiple-choice tests. It is even probably true on essay tests that are graded in a centralized location on narrow criteria, if the preparator knows what the criteria are. Kaplan and Princeton Review, for instance, both have methods of preparing for the SAT that have nothing to do with learning the material and everything to do with exploiting the vagarities of multiple-choice tests.

Secondly, even if the best way to prep for standardized tests was indeed to teach the material, what about the material that isn't included on the tests? Most of the NCLB-style tests seem to focus on reading and math alone. While these are certainly very important areas of study, are they really all that is needed? Do we want our students to know nothing of art, music, logic, philosophy, or, for that matter, the system of government under which we live?

I've heard too many horror stories about "teaching to the test" to believe NCLB is a good idea. And these stories are from good teachers in good suburban public schools. I hate to even imagine what horrors are being perpetrated against urban youngsters in the name of standardized testing. I seem to recall reading that one district was actually ordering the teachers to read from a *script*. Brr.

Posted by Josh G. | November 14, 2006 8:04 AM

For reading and basic math skills, however, the easiest way to teach kids to pass a test seemed to me to be teaching reading and basic math.

Easiest? Why? It's much easier to teach somebody how to (for instance) write an essay that satisfies a narrow rubric than how to write a good essay. And I don't believe it's possible at all to create a math test where the _best_ way to teach kids to pass it is to teach them generally applicable math skills.

This is a basic, natural process. The test becomes the only indicator of progress, so teachers who teach test-taking skills the best become defined as the best teachers. Don't take it from me, take it from my mother. The 'best teacher' in her school spends 3 weeks a month having her kids drill previous editions of The Test.

Posted by neil | November 14, 2006 8:11 AM

Readin', 'ritin' and 'rithmatic.

So, Bush is advocating that we quit schoolin' our kids around the age of 9 once they get the basics down. Personally, I'm a big fan drastically truncating the the mandatory parts of public education. But I'd firebomb TV too, so it's part of a larger scheme.

Posted by Jeffrey Davis | November 14, 2006 10:20 AM

My mother is an early childhood educator and I am a HS Mathematics teacher. One of the things my mother explained to me when she was going through college (a late lifer) is that there are studies that show that early childhood education methods show lower test results at the early ages up through middle school. However, in middle school, the early childhood kids take off and achieve well above those who had a more traditional elementary education approach. However, given the current system where I am accountable for kids succeeding in my year only, it makes sense for everyone to drill their kids unceasingly in the old school elementary education model even if you realize this might hurt performance down the line.

This insight has only been reinforced in my time as a HS teacher. Our kids have learned that the easiest way to pass standardized tests is to learn the process of mathematics ("How do I do this?") for a limited set of skills instead of learning the concepts (Why do I do this?") for a much broader understanding of Mathematics. The only standardized tests that I have seen overcome this type of thinking are the AP exams. It always comes as a bit of a shock to my kids that I really can't teach them the rules on how to do everything they will be asked to do, that they will actually be asked to think mathematically using the concepts they have covered. For any who haven't had me at an earlier level, this as an amazingly alien concept and hurts their performance. Essentially, by the time I get them, I have to try and undo 10 or so years of being trained the "steps" of mathematical problems without any real concetual groundwork. I have no doubt that at every step this improved their test scores that year. I also have no doubt that this hurt their mathematical scores in the long run.

Posted by socratic me | November 14, 2006 10:40 AM

Linus- just for the record; the Japanese elite does not send it's children to public schools prior to university, though the top colleges here are public.
BTW, the school system here is an extreme example of 'teaching to the test', and although basic literacy and mathematics skills are taught very well, the finer points of later education are lost in an ultra-rigorous memorization of facts without context.

Posted by darragh | November 14, 2006 10:49 AM

I think socratic is on to something: the perverse incentive that NCLB creates to educate only on a year-to-year basis. In our diverse district we have an elementary curriculum that has been developed to lay the foundation for higher-level learning and thinking required in middle and high school. As a result, our african american students in the secondary schools are the highest achieving group of AA students in the state at least based on standardized tests. One side-effect of this, however, is that our elementary school test scores tend be be all over the place -- primarily because the curriculum does not emphasize the drill-and-kill instruction that leads to higher scores on the elementary tests. We are now under pressure, however, to throw the elementary curriculum and teach to the test [though of course, the advocates do not use that term] in elementary school just to improve the test scores.

A side note: in the burbs in particular, NCLB is not the only fueling thing the over-emphasis on standardized tests. In the real estate market, tests scores have become the most important quality measure of a school system. So, the pressure to keep up real estate values also promotes teaching to the test.

Posted by mert7878 | November 14, 2006 11:24 AM

So you have a bunch of people who have not, historically, hit upon good methods of imparting reading and math scores to their kids. Now you tell them there will be consequences unless test scores go up. Sure, the best way to get them to go up would be to start teaching reading and math better. But if you're talking about a bad school, then presumably the teachers and administrators haven't found a way to get this done. So, instead they adopting the semi-effective method of doing narrow test prep.

Is it a realistic assumption that schools were doing a poor job teaching ex ante would be able to succesfully teach narrow test prep either? Insofar as the difficulties in teaching at the bad schools aren't dependent on the nature of the material taught, you'd expect similar problems.

Posted by MattXIV | November 14, 2006 11:34 AM

"Linus- just for the record; the Japanese elite does not send it's children to public schools prior to university, though the top colleges here are public."

Thanks. Maybe someday my comments will be merit not only fact checking but spell checking as well. I don't know about you, but there's no question in my mind that the larger argument Linus is making isn't worth dignifying.

Posted by Linus | November 14, 2006 1:14 PM

But - you know - just for the record Linus's misguided perception of public schooling in Japan came not only from myth and rumor but from the fact that less than 1% of Japanese students are enrolled in private elementary schools. (Apparently only about 5% of Japanese students overall are enrolled in private schools. The number for America is above 10%.) It is probably more accurate to say that very few upper middle class people send their kids to private schools in Japan (which is not quite the case in America).

Posted by Linus | November 14, 2006 1:44 PM

There is a long comment thread in which I try to explain "teaching to the test" here; basically, it's a problem of bad test design, not with testing per se.

Posted by SamChevre | November 14, 2006 3:33 PM

This is a bit of a side issue, but I thought I'd point out that many states are moving to individual-level value added assessment (i.e. measuring improvement in the same kid's scores year to year), and the rest will likely follow soon. So the criticism some people made in the early comments will probably end up obsolete.

Posted by MQ | November 14, 2006 3:55 PM

Q: Are we helping individual kids with standardized tests that are a state secret until sometime in the next school year?

We've got mainframe, command and control data models when what would help kids, parents, teachers, and administrators most would be interactive feedback. The whole standardized testing paradigm is based on bureaucratic centralization. We're stuck in a 1950s data model for education reform.

This is doing really well for the big business of testing, where we keep adding grade levels and subjects and splitting out the results as a plethora of different ways central bureaucrats can label schools as failing.

But isn't it time to shift some of the focus back to kids, by putting modern, interactive data tools directly in the hands of teachers and local administrators, and scaling back the emphasis on punitive command and control testing?

Posted by MassParent | November 15, 2006 2:02 AM

There are many valuable classroom activities that are not testable. Show and tell for example. In second grade my kid's class had show and tell three days a week (six kids went every day), and the session was led by another kid, whoever was "Student Spotlight of the Week."

It was practice in organizing a presentation, public speaking, leading a group discussion, and participating in a group discussion. If this sounds silly and trivial to you, you probably don't know any eight year olds. Knowing how to make a presentation and make relevant comments are skills you have to learn and practice.

Another example: committee projects. In first grade, every table in my kid's class had to agree upon, and build, an "invention" together (typical product: homework machine). Again, knowing how to work as a group is a teachable skill that needs to be practiced.

Now he is in third grade and the testing has begun in earnest. They just sit in their seats and do written work, all of it geared toward test-taking.

Posted by Ohio Mom | November 15, 2006 9:21 AM

Probably you should read some of Alfie Kohn's criticisms of testing and other bad educational practices. His website has a number of his articles available.

Posted by Mike Bruce | November 15, 2006 1:57 PM

Many thanks for the Alfie Kohn link -- maybe we should all chip in and buy one of his books on testing for Matt.

Posted by Ohio Mom | November 16, 2006 10:05 AM