Differences

Kevin Carey has a post with some data that was desperately in need of chartification so here we see the percentage of poor fourth graders in various cities who test as having achieved proficient (red) or basic (green) levels of competence according to NAEP scores. As you can see, though educating poor children is challenging everywhere, some of these cities are having much more success than are others. Obviously, there's more to "demographic factors" than whether or not you qualify for the school lunch program, but as far as quick arguments go it's pretty convincing evidence that there are things LA, Chicago, Cleveland, and DC could be doing to obtain the kind of better results that New York and Boston are getting with poor children.

Comments

LA had been making gains until the teacher's union and mayor cooperated to oust the superintendent; the new one is a joke.

Also note that this chart should include information about language proficiency; it stands to reason that places with more native English speakers should do better than those with fewer.

Posted by J | November 16, 2007 10:00 AM

Not to suggest that there is a causal connection between the two and perhaps I am mistaken, but aren't Cleveland and DC the only two cities on the list with a voucher program?

Posted by Dungheap | November 16, 2007 10:13 AM

it stands to reason that places with more native English speakers should do better than those with fewer.

That would make New York's good results quite surprising.

Posted by Peter | November 16, 2007 10:27 AM

Wasn't Houston notorious for having their kids not take tests like these, back when W was the governor of Texas and Rod Paige was in charge of Houston's schools? One way this statistic would be massaged is that lower-performing kids who dropped out would simply not be counted at all in test "averages". It's hard for the average itself to be meaningful, if the dropout rate is ignored.

Posted by Whispers | November 16, 2007 10:31 AM

One needs to look at the spectrum of poverty among the different sets of students - maybe the NYT distribution is richer than the DC distribution.

Posted by rilkefan | November 16, 2007 10:34 AM

Wow, didn't take long for the voucher and anti-union people to jump in. New york has some of the strongest teacher unions and Houston the weakest and their scores were pretty close. Could be that fixing education requires a more complex approach.

Posted by exlitigator | November 16, 2007 10:39 AM

One thing to point out is that some states have school districts that do not fall on city lines. Houston and Austin, being in Texas, have indepdent school districts. In houston, several parts of the city have schools that are not in the Houston Indepdent School District. Many of those schools are in the whiter, western neighborhoods.

DC is a school that is almost totally devoid of white students. I would guess that Cleveland is similar.

Posted by superdestroyer | November 16, 2007 10:48 AM

% of students in district who are english-language learners (2001)
Source:
http://www.cgcs.org/pdfs/surveyfinalreport.PDF

NYC 16.9%
Boston 21.0
Charlotte 5.4
Austin 17.8
San Diego 27.8
Houston 27.2
LA 43.2
Chicago 13.3
Cleveland 4.1
DC 12.5

Back to you, Peter.

Also, not against any union except teachers' unions, which fail to recognize that teachers are not just employees, but also must act in kids' interests even as they work for their own.

And vouchers are bad. Who was advocating for vouchers, exlitigator? Do agree it's a much more complicated question than this bar chart suggests.

Posted by J | November 16, 2007 11:10 AM

J:

Gee, I guess Cleveland, Ohio must be filled with non-English speaking students. Hardly surprising, since it's right on the Mexican border.

Whispers:

Yes, I'd heard the same thing about Houston's notorious policy of getting better test scores by not testing the students who'd probably do worse. It's known as the "Enron Method" of raising your test scores...

Posted by RKU | November 16, 2007 11:15 AM

Question: Does DC actually have any real control over their schools, or does Congress run them? Someone's doing a bad job, I just want to know who to blame.

Posted by fostert | November 16, 2007 11:16 AM

"One needs to look at the spectrum of poverty among the different sets of students - maybe the NYT distribution is richer than the DC distribution."

Great point. When you see Charlotte, Austin and Houston near the top, something interesting has to be at work.

BTW, the Chicago Sun-Times says Chicago "has the second-largest percentage of poor kids among the 11 districts..."

Posted by leo | November 16, 2007 11:29 AM

There is an interesting book on the complexities of education in the inner cities called "Black Social Capital". There are intangible factors like the school's administration and the curriculum itself that can play as much an influence as vouchers and English fluency.

If you are looking for an easy correlation, you are assuming there is an easy fix, which is why you aren't finding that simple correlation.

Posted by freddiemac | November 16, 2007 11:32 AM

Is there a place where I can send MY a copy of Tufte's The Visual Display Of Quantitative Information? This graph may have scarred my retinas.

Posted by alkali | November 16, 2007 12:16 PM

In terms of relative poverty levels, it's worth noting that New York and DC, first and last on the chart, have almost exactly the same percentage of school-age children living below the poverty line, 29.4% vs 29.1%.

As for test administration and dropouts in Houston, the NAEP test is administered by an independent organization under contract to the federal government, so a lot less opportunity for cheating than with a locally-administered test. And the results are for 4th graders in math, so probably not a lot of dropouts.

Posted by Kevin Carey | November 16, 2007 12:31 PM

Kevin,

You really have to find a way of gauging the extent to which high poverty concentrations within schools are more or less prevalent in these districts for this analysis to be persuasive. In the extreme case, if all the poor kids in one city are concentrated in a subset of schools while they are spread out evenly among all the schools in another city, that would influence the results (in favor of the latter). It's way too simplistic to just say NYC "does a better job" than DC, because NYC has a higher share of schools with relatively lower concentrations of poverty, and poor kids in middle class schools get higher test scores.

--Greg

Posted by Greg Anrig | November 16, 2007 12:47 PM

Looking at the Head Start programs of the two at the bottom...DC their HS program is run by Parks department and Cleveland's is also not part of the city's education department which I must add does not list a dept of ed on their official site. Looked at LA and their HS is a strickly a half day program. Chicago has a big mission statement and all kinds of info...but no real details of the program. The NY HS program appears to be a top shelf program...

Posted by tin foil | November 16, 2007 12:47 PM

Several reports of "questionable" results from New York and Houston schools. Seems too much is riding on the teachers and principals to allow their students to do poorly.

Posted by Gary Denton | November 16, 2007 1:16 PM

The higher the percentage of non-Asian minorities, the worse the school's performance. It's not difficult, people.

Posted by Gredas | November 16, 2007 1:19 PM

There are lots of immigrants in many cities. Do the immigrants also do without fresh vegetables? I know that in some places the Korean or Asian grocery is the best thing going, but I'm not sure how this works out in central urban areas.

Posted by ed | November 16, 2007 2:28 PM

Greg,

If NYC and DC have identical overall poverty rates, and NYC has a higher share of schools with relatively low poverty concentrations (good for poor kids) wouldn't it also have to have a higher share of schools with relatively high poverty concentrations (bad for poor kids)? That sounds like a wash at best, since super-high poverty concentrations are arguably more of a bad thing than low concentrations are a good thing.

I'm not "just" saying NYC does a better job than DC, as if that's all that matters. But the fact that NYC does a better job than DC is indisputable, and I think it's pretty clear that the difference is reflected in test scores

- Kevin

Posted by Kevin Cary | November 16, 2007 3:14 PM

In response to an early point, there are charter schools in all these cities. A higher percentage of D.C. and Cleveland students are enrolled in charters because parents are so eager to try alternatives to the district-run public schools.

According to Education Trust, Cleveland didn't test a relatively high percentage of students, which means their very low scores probably are inflated.

Posted by Joanne Jacobs | November 16, 2007 3:15 PM

Kevin,

Here's another way of putting it: Suppose in DC, all poor kids attended high poverty schools while in NYC only half did. Then you would be very likely to see higher aggregate test scores for poor kids in NYC relative to DC. I'm saying the inverse of what you are: school administration is definitely superior in NYC compared to DC, but I bet demographic dispersion is more significant than you do in accounting for the numbers in your graph. It's a researchable question, so I strongly encourage you to pursue it -- just focusing on those two cities.--Greg

Posted by Greg Anrig | November 16, 2007 3:24 PM

I've been looking at NAEP results for years, and the bottom line is that race/ethnicity breakdowns are by far the biggest driving factor. I've recently been looking into how to remove the influence of race on NAEP scores so we can measure institutional competence, and I'll link to that in another post. So, a better way to do this chart would be to compare poor students by each race in each city, such as just African-Americans compared to other African-Americans.

Unfortunately, I don't believe the NAEP has asked for parents' birthplace since 1992, so it's tough to break out immigrant status, which can be important, since a lot immigrant poverty is temporary. For example, there are a lot of African immigrants from the far right edge of their home country's bell curves who are just scraping by economically in America as they pursue their advanced degrees while driving a cab. It's not at all surprising that their children tend to do better on tests than typical African-Americans. (That's why, as Henry Louis Gates has complained, Harvard's black students are increasingly non-African Americans,Barack Obama-types with no American slaves in their family trees.)

Back in the 1990s in Chicago, I consulted for a charity that handed out prep school scholarships to high-potential poor children. Among the directors, we had lots of arguments over just how in need the students who scored highest on the tests were, since they were usually immigrants. For example, one year, there were 3 Russian Jewish 8th graders who had hit the 99th percentile on the tests. Their current incomes were low, but that was because their parents, who were doctors in the USSR, weren't working here because they were studying to pass the American medical exam. Similarly, I was skeptical of giving limited scholarship money to high-scoring Korean immigrants whose claim to be poor was the tax returns filed by their parents who worked in extended family-owned cash businesses. I argued, in these cases, that the kids weren't at risk for failing in life, and we'd be better off choosing an African-American youth from the projects who scored at the 65th percentile and send him to St. Mel's, an all-boys Catholic high school with a charismatic, take-no-crap black principal.

Posted by Steve Sailer | November 16, 2007 6:35 PM

Audacious Epigone has done a more sophisticated job at looking at how good a job each states' public schools are adding relative to what they had to work with by comparing 2003 NAEP scores for 4th graders to 2007 NAEP scores for 8th graders.

On this value added list, the much maligned Washington DC public school system comes out best, followed by Massachusetts (traditionally, the smartest state in the country), North Dakota, Montana, and Maryland.

The worst state is West Virginia, with North Carolina second.

http://anepigone.blogspot.com/2007/11/state-rankings-by-naep-improvement-from.html

This isn't a perfect measure yet, but it's nice to see an NAEP ranking of states that's not obviously explained by the racial makeup of the student bodies.

Posted by Steve Sailer | November 16, 2007 6:43 PM

A supply teacher reports from the frontline:

http://www.worldnetdaily.com/news/article.asp?ARTICLE_ID=58713

Posted by Killl | November 16, 2007 7:59 PM

"I argued, in these cases, that the kids weren't at risk for failing in life, and we'd be better off choosing an African-American youth from the projects who scored at the 65th percentile and send him to St. Mel's, an all-boys Catholic high school with a charismatic, take-no-crap black principal."

Is it possible that Steve Sailer has more hands-on experience helping real-life African Americans than the folks who regularly hurl insults at him here? That would be interesting.

Posted by Fred | November 17, 2007 12:00 AM

It was an interesting experience interviewing scores of low income 8th graders and their families looking for scholarships to get them into private high schools. I eventually took the lead in formulating the charity's philosophy that they were most looking to help students with the largest gap between potential and likely outcome if they stayed in public school in the 'hood.

Thus, if we had a choice between a kid who might be a doctor if he gets a scholarship whereas if he stayed in public school might wind up a pharmacist, well, swell. But a kid who might wind up a career sergeant in the Air Force versus doing time - that was a better use of our money.

One thing we found was that the really high potential poor kids (e.g., 90th percentile or higher on tests) were in good shape without us. There was lots of scholarship money chasing them, plus they could often attend the fine Whitney Young public high school, which requires an entrance test to get in. It was the kids below them, the ones who were way above average in intellectual potential for their neighborhood, but not for the State of Illinois as a whole who were at risk.

Posted by Steve Sailer | November 17, 2007 12:31 AM

The numbers are meaningless without taking into account race. New York has a much higher percentage of white students below the poverty level, probably because of a much higher number of eastern European immigrants.

Atlanta and Washington D.C. are relatively comparable, and Atlanta does a much better job of educating disadvantaged kids, and does it for a lot less money when per pupil school funding is taken into account.

Posted by Rory Hester | November 17, 2007 1:15 AM