Innumeracy

Kevin Drum reads a New York Times article about holiday retail sales and bangs his head against the wall as he observes the story citing nominal sales figures: "Question: why does this happen so routinely?"

It almost certainly happens so routinely because many reporters and editors don't really understand what they're doing. Reputable colleges hand out degrees to people who have almost no understanding of quantitative methods. I recall that Larry Summers observed in his inaugural speech that "We live in a society, and dare I say a University, where few would admit—and none would admit proudly—to not having read any plays by Shakespeare or to not knowing the meaning of the categorical imperative, but where it is all too common and all to acceptable not to know a gene from a chromosome or the meaning of exponential growth." Journalists, being basically a species of writer, tend to come from humanities backgrounds even though we deal with quantitative issues all the time. Journalism schools might help close the gap by making people take "math for journalists" classes (the concepts of statistical significance and margins of error in polls come up constantly, for example, and are often dealt with very poorly) but as best I can tell they normally don't.

Comments

Then again writing stories about economics and related statistics can nearly ALWAYS be constructed with the caveat "adjusted for inflation" (or some other such benchmark). Should there be a journalistic rule all writing incorporate such adjustments? Or a rule each story is preceded with the qualifier "Statisitics herein may or may not bear any relation to pertinent economic indicators"? I don't think there are enough fingers for this dike.

Posted by steve duncan | December 26, 2007 1:44 PM

"We live in a society, and dare I say a University, where few would admit—and none would admit proudly—to not having read any plays by Shakespeare or to not knowing the meaning of the categorical imperative"

Was Summers being serious? I went to Duke and Duke Law, and I freely admit I just Googled "categorical imperative."

I think most in our society would have no problem admitting that they have no idea what is the categorical imperative is and that they faked their way through Shakespeare in high school

Posted by Oberon | December 26, 2007 1:47 PM

Well, I'd certainly agree that lots of journalists are lazy and innumerate. But I'm not sure that's the central dynamic here.

Suppose the NYT reporter had gotten things right and wrote something like "retail Holiday sales actually fell this year for the first time since 1992"...

Bam! All the ignorant individual investors who read the NYT immediately sell their retail stocks, which drop like a rock and many billions of dollars of market value are lost. Disaster!

So maybe all the endless PR people hired by the retail companies---who are basically the ones releasing the data---do absolutely everything they possibly can to "persuade" the NYT reporter to present things in a really stupid way.

After all, who has a vested interest in retail sales looking terrible? Just the hedge-funds which are shorting those stocks, and they tend to hire far fewer PR people to "spin" the reporters in the other direction.

Posted by RKU | December 26, 2007 1:49 PM

Basic math ignorance is not just a problem limited to journalism, it is a fundamental flaw among the vast majority of the population. It is the proximate cause (but not root cause) of the current subprime mess, for instance.

On a seperate note, I wonder if Larry Summers will have a position in the new Democratic Adminstration, or if will continue to be persona no grata due to that "chicks can't do math" thing.

Posted by Kolohe | December 26, 2007 1:50 PM

Well, I went to Harvard and I Kant explain the categorical imperative. (Although it was for grad school in science so my Harvard education doesn't count.)
I forget if you linked to it previously, but Howie Kurtz' defense of his innumeracy in reporting polls as a "statistical tie" was jaw-dropping. Here's a guy who has his error spelled out for him and he still insists he's right- and he's supposed to be the media critic for the Post:

Gainesville, Va.: Howard, I think it is time to send political reporters and commentators to a remedial course in statistics. You wrote that Obama could as easily be tied with Hillary because his lead of 28 percent to 25 percent is within the margin of error. What the confidence interval means is that if the confidence interval is smaller than the difference between two means we can be 95 percent (not 100 percent) sure that the opinions held in the universe will reflect the higher response to be greater than the lower. If the difference of the two poll results are within the confidence interval, it may mean we can only be sure that one is greater than the other with 80 percent or 75 percent confidence. It doesn't mean they are tied. It really would be helpful if you and your colleagues boned up on this question before you made statements that aren't true.
Howard Kurtz: I stand by what I said. Obama's lead is within the margin of error. In fact, the poll's margin of error was over 4 percent, which makes the 3-point lead even more of what I would regard as a statistical tie.

Posted by SP | December 26, 2007 1:51 PM

For backcountry papers, I understand this error. But, uh, for a big paper like the NYT, someone there DOES understand the math, so I don't understand why these kinds of articles don't get run by a "numbers editor" or somesuch.

It's like that embarrassing Kit Seelye article a month or two back where she compared Obama and Clinton's heath care policies and said the uninsured rate of Massachusetts was 20%, and a laid out a whole host of other wildly inaccurate health policy points. The NYT has Robert Pear and Gardiner Harris on staff, so when the political reporters are reporting on health policy, why don't they run these articles by them?

We keep trying to build a better reporter, but that seems pointless considering the fact that it's already a pretty demanding job.

Why don't we focus on building a better newspaper? The answer isn't a crash course in economics for line reporters, which may or may not be understood, but to create an environment in which it's actually DIFFICULT in our pre-eminent papers to write a story rife with innumeracy. One where it's at least as hard to put in an innumerate figure as it is an inappropriate homophone.

Posted by anonymiss | December 26, 2007 2:02 PM

But do they really write these stories? I'm sure all they do is just slightly rephrasing some press-release of some government agency or business association.

Posted by abb1 | December 26, 2007 2:14 PM

Confirming his own argument, I guess, Larry Summers not only doesn't know it's poor style to split infinitives, he also doesn't know the difference between "to" and "too."

Posted by Farinata X | December 26, 2007 2:23 PM

It's not a lack of training that accounts for reporters' innumeracy. I'll give a small anecdote, that illustrates my point.

I went to Yale Law School, and sat through several classes that required a bit of numeracy. Not much: just basic accounting principles. The students just didn't get it. Certainly not the ones with literary backgrounds. But many of them had economics backgrounds. Still duh. Given the way Yale Law courses are graded, they weren't harmed by their ignorance. (To those who aren't in on the secret, the credo of the Ivy Leagues is that their admissions offices never make a mistake, at least the kind shown on report cards.)

So the same bunch of effete innumerates get jobs as Wall Street brain dogs. All of a sudden--whoosh! They quickly become quite skilled with accounting concepts. Otherwise, they might lose their high-paying jobs.

The moral of this story? It's not the education; it's the incentives. Our Glorious Press doesn't really care much about getting the story right.

Posted by Joe S. | December 26, 2007 2:28 PM

SP,

Your example is terrible.

If the margin of error is 4%, all we can say is, with 95% confidence, the true value of a particular candidates support is within 4% of the reported value. Assuming the distribution is Gaussian, there is only a 50% chance that the true measure of support is within 1.5% of the measured value. If the candidates are off by 3%, then, based on this one poll, there is a 50 chance that they're tied.

Posted by br | December 26, 2007 2:47 PM

Medill requires one math and one economics class for graduation, although yeah, I would have liked a more specific course than the microecon-taught-by-an-80-year one I took.

Posted by Dave Weigel | December 26, 2007 2:47 PM

"Journalism schools might help close the gap by making people take "math for journalists" classes (the concepts of statistical significance and margins of error in polls come up constantly, for example, and are often dealt with very poorly) but as best I can tell they normally don't."

Here's a more direct approach: make a B.S. in Engineering, Economics, Finance, etc. a prerequisite
for writing a story/column even remotely related to Engineering, Economics, Finance, etc.

Posted by kafka | December 26, 2007 2:48 PM

Matt --- I think your degree is showing through :)as the only reason why I, after going through both undergrad and graduate education at an elite US university, know anything about the Kantian categorical imperative or pretty much anything else about German philsophy is that I was screwing a very cute philosophy major while she was taking her continental philosophy sequence. Otherwise I would have had minimal reason to care --- As another poster said more cleanly, incentives matter on what things deserve our limited attention span, and the broader question is why do we as a society place such a low incentive on basic quantative literacy?

Posted by fester | December 26, 2007 2:49 PM

I've always (seriously) wondered: what do they teach people in J-school?

Posted by Jason C. | December 26, 2007 2:52 PM

C'mon Matt,

The "categorical imperative"? I'll give you "The Bard" to define the boundaries, but now the public needs to grasp categorical imperative?

I don't think so. The dry concept and its author are, at best, intriguing to the soi disant intelligentsia, career-challenged esoteric types, and a species of failed opium smokers that never did discover the secret of the dark abyss.

What say we settle on having to know, distinctly, categorical and imperative-and award bonus points to those who understand the meaning of both.

Besides, it's the goddamn newspaper that's being read and written for, and I don't want you intellectualizing or interfering with the one thing that allows me to take a dump mindlessly....

Posted by resh | December 26, 2007 3:11 PM

br- Yes, there's a chance they're tied or that Hillary's really ahead, but I think it's a 25% chance since they could be on the bell such that they're further apart than the 3% reading. Furthermore, since there's a 50% chance they're inside the +/- 1.5% interval, that does indeed tell you that it's still more likely than not that Obama's lead would be real if you could query all respondents instead of a sample.
Kurtz is saying that if you have a lead of 4% and the MOE is 3% then the lead is real, while if you have a lead of 3% with a MOE of 4% then it's a tie. Statistical sampling doesn't work that way, it's about probabilities of a subset of respondents correctly representing the whole, and Kurtz is an idiot for not knowing that. (Anyway, voter model error in polls is almost always more significant than sampling MOEs.)

Posted by SP | December 26, 2007 3:41 PM

Oberon:

Yeah, I think you're right. Summers is a bit optimistic about the number of people who care about -- or have heard about -- the categorical imperative, or who've read Shakespeare. Matt's complaint seems to be that there's too much attention focused on the decline of reading standards, and not enough focused on the decline of math skills: "We're not illiterate! Actually we're innumerate!" Well, why can't we be both?

Posted by Shoshana | December 26, 2007 3:46 PM

There actually have been rudimentary statistics classes taught in journalism schools for at least 30 years. Back in my day it was called something like scientifically accurate reporting.

No joke - half of my class dropped the first day after some expressed bewilderment over x being the square root of x squared. Given that it was an elective I have no idea where they went.

Posted by pops | December 26, 2007 3:57 PM

Obviously they were protesting the fact that the answer is actually +/- x; The course was too simplistic for them.

Posted by SP | December 26, 2007 4:04 PM

Plenty of good journalists never go to J-school. Much better to get a degree in math, or chem, or some pre-law thing, or political science, so you'll know something about the topic you're covering.

Posted by too many steves | December 26, 2007 4:04 PM

What could be more embarrassing than a fussy, pendantic comment that is totally wrong? While few people object to splitting infinitives under all circumstances, the only infinitive in the Summers quotation, "to know," is unsplit.

By the way, Summers is the first person I ever heard use the work innumeracy, back in 1975 or 76.

Posted by Paul Turner | December 26, 2007 4:09 PM

This is an example of something that is a little more complex then it appears.

Exactly what is the article talking about when it reports retail sales?

Generally they are talking about department store type merchandise -- think of what Wal-Mart sells.

But the credit card sales also include gasoline and restaurant sales that are included in the CPI but not the deflator for department store type merchandise like clothing, electronics, furnishings, etc., where prices are actually falling.

So what deflator do you use. Clearly not the overall CPI that is heavily weighted to housing and services and so would overstate the relevant inflation rate for this series.

Generally one should use the deflator for GAFO retail sales -- again think department stores.
This series has actually fallen 2% to 3% every year since 1993 and is on target for a similar drop this year. This deflator implies that a 2% to 3% nominal increase is a 4% to 6% real increase. Too many people -- including Fed reserve board members & Greenspan -- tend to use the overall CPI to deflate same store sales data and conclude that real retail sales were down. But that overstates the weakness.

I do not know what weight to give the restaurant and gasoline sales in the credit card data so I'm not sure how to convert this data into a real series.

But on the other hand few reporters are even aware of these problems in determining what deflator to use.

Posted by spencer | December 26, 2007 4:22 PM

I agree that nominal figures are used too often, and inflation-adjusted figures not often enough. I also agree that a lot of journalists suffer from innumeracy. I'm not sure Kevin Drum's example, though, is particularly apt.

It seems to me retail sales figures -- like year to year corporate revenue figures in general -- are almost never adjusted for inflation. Not even the Wall Street Journal or The Economist cite inflation-adjusted figures in this manner. I suspect there's a good reason for this: when you're talking about the narrow universe of a single company or industry over a fairly short period of time (a year or less) it's probably not all that important to take inflation into consideration, and it may even be misleading.

After all, if, say, Macy's same store holiday sales are up 2% in nominal terms over last year's, you as the reader are perfectly free to reference the inflation rate in your own mind when you read the story reporting this bit of news. But in this instance, citing inflation in the story itself would, perhaps, needlessly complicate things. After all, Macy's isn't the economy as a whole. It's only one very little piece of it. For all we know, maybe Macy's has been able to slash its labor costs, or perhaps it has found a number of ultra low cost suppliers in India and China, and so its costs have actually gone down over the last year.

Real figures are absolutely a must when talking about the macroeconomy - especially over longer periods of time. But at the single firm or industry micro level, especially over short periods of time, nominal figures are perfectly fine, and evidently preferable, as even reputable financial media outlets use them.

Posted by Jasper | December 26, 2007 4:32 PM

Interesting that Summers tucked the more recondite Kantian reference in with general knowledge Shakespeare - He was getting a semi-cryptic dig in at colleagues who lack his Eurocentric frame of reference.

Posted by Comment | December 26, 2007 4:55 PM

Journalistic ignorance is not unknown in other areas -

Who among you did not utter a smug chuckle when watching a lightweight like "Tucker" make fun of that young lady who got nervous aswering a geography question durng the Miss Teen USA pageant?

Posted by Comment | December 26, 2007 4:59 PM

Fester's manly admission above is just a more vivid way of articulating the point about Kant. Joe S. is correct about icentives - Consider the medicore product of the Intelligence Community! - Imagine if it was better - they would have lost their jobs and they would have been punished by ideologues for reporting tough truth. So the incentive system created Iraq experts who don't speak Arabic and never travel - Not to mention, suspicion of left wing views , drug tests, and social liberalism - made recruiting at the better schools problematic.

Posted by Comment | December 26, 2007 5:18 PM

spencer and jasper are on the right track, and, sadly, kevin drum is not, in this case at least.

note a couple of additional points:

the offical CPI numbers for december aren't available until january, so it isn't clear what kevin would have them do. if journalists aren't getting the concept in general, i'm not sure if we'd be happy with their interpolation or estimation of a december CPI number.

it's possible, perhaps even probable, that more merchandise was moved this year (ie, more sweaters and shoes and ipods, etc.) than last year.

Posted by Thomas | December 26, 2007 5:24 PM

I speak from experience. It's not that the reporters and editors don't know what they're doing, it's that they're just repeating what's in the press release. MasterCard didn't adjust for inflation because it knew it would make the numbers look worse. The Times didn't do it because it wasn't in the release.

Posted by Seth Davis | December 26, 2007 5:33 PM

Jasper:

I agree in part to the fact that Retail Sales need not be adjusted for inflation, given the fact that US GAAP does not require such adjustment when it comes to financial statements. So we are left then with the question as to whether or not the NY Times article is providing a fair documentation of what is really going on with retail sales. I would argue you that it could have been helpful to throw in both numbers just to give some perspective to someone not familiar with why a 3.6% increase in sales is considered a disappointment. I mean, really, most people receive around a 3% raise annually (especially if your raise is based on COLA adjustments versus incentive adjustments). Or they could have at least thrown a sentence or two discussing the fact that these numbers fail to take into account inflation, so the reader at least realizes that the 3.6% is not as rosy at it appears.

Secondly – as for this article, I am completely on the same page with a few other posters wondering whether the current metrics for determining Christmas is getting outdated. In fact, if the New York Times wanted to really do some decent reporting, they would investigate trends in spending, and the impact on traditional Christmas dependent retailers such as Toy R’ Us, Macy’s and Nordstrom. I think as people become more and more fashion conscious (and hence picky about what they wear), the old gift of clothes for Christmas is starting to fade. I know I will not dare buy anything for my Wife, and quite frankly, most people buy me outfits which I end-up returning anyways. But spa days and other non-traditional gifts (gift cards to restaurants for example) seem to be gaining in popularity.

However – this makes me wonder why it seems writing, especially at large news organizations or even on most television news oriented programming is so weak. My hypothesis would be based on a number of factors, but each contributing to this:

(a) Journalism = Education Degree – journalism may teach you about being a journalist, but just like teachers, you need to have some underlying knowledge about the topics you write-on if you are going to be able to write effectively on the subject matter. As news organizations have cut costs, trimmed staff, and forced out knowledgeable writers through early – retirement packages, who do you think they are going to hire: someone fresh out of college or someone who has worked, successfully, in their respective fields such as science, law, business etc… But this leads to why it is they can get away with this, or at least think they can:

(b) Divergence of the Public: as put forth by Jerry Skurnick (via Slate’s Kausfiles) the public is increasingly either becoming even more informed due to the web or, due to the decline of network news, even less informed. In addition, this happened as all the major newspapers were cutting staff and trying to emulate the USA Today, with short stories void of details or complexities. This is why the only paper I actually pay for is the Sunday New York Times (ironic given this article was from the Times). I have tried to read the Denver Post and Rocky Mountain news, and quite frankly, I always feel that the articles are written for the lowest common denominator. Yet, it is the lowest common denominator which has little interest in news in general. So they are chasing the unattainable girl, all the while ditching the “Plane Jane” who is all too willing to take a chance on them if the papers only put in some effort.

Posted by Brad | December 26, 2007 6:03 PM

It happens because most 'journalists' are pretty dumb. Americans are in general probably dumber than they have ever been. Sure you can dig up the quotes pointing out they have been dumb all along but I don't think they ever used to be willfully dumb. I mean they still have a hard on for the most willfully dumb man ever to be president. They identify with him, strongly.

Maybe they were right about fluoridation. Either that or modern mass media, particularly TV, has been doing the damage to Americans critical thinking ability. That's my vote.

Posted by rapier | December 26, 2007 6:42 PM

Neither "categorical imperative" or Shakespeare are necessary to get through life - even in urban America.

Math - at least on a basic level - is.

That said, statistics is one of the hardest basic maths to get your head around. The stat class I took in college HAD to be graded on a curve or everyone including me would have flunked.

It didn't help, of course, that our stat teacher was some Southern bozo who insisted on pronouncing the word "histogram" as "histiogram" - until I couldn't take it any more and corrected him.

My only real connection with a journalist was with writer John A. Keel - the guy they did the "Mothman Prophecies" movie about with Richard Gere playing his part. That's right, the "UFO nut".

But we was a damn good journalist. He had a syndicated column for the North American Newspaper Alliance and was geography editor for a major encyclopedia since he'd traveled everywhere.

But what made him a good journalist was that he didn't take the usual answers as gospel. He would actually try to FIND OUT what the hell was going on. And if he didn't know an answer to something, he'd find someone who was an expert and ask. And he'd do the footwork necessary to get the whole picture.

This led him to question the whole "UFOs are alien spaceships" crap and resulted in his coming up with a lot of data that led to even more outre theories on his part - which theories were, however, more logical than the conventional spaceship theory - or the theory that it was all hoaxes and hallucinations and swamp gas.

His problem - and he knew it - was that his theories - and the data itself - was unprovable and unreproducible given the human level of technology back in the 70's. That pretty much left him between a rock and a hard place. That got him relegated to the status of nutjob by the skeptics and even many more "conventional" UFO theorists.

So thirty years later we're still stuck in the same place - either UFOs are bullshit or they're alien spacecraft.

Neither of which is likely to be true.

The job of a journalist is to learn enough about a subject to be able to see the "big picture" and then explain it to his readers. Most journalists and journalism schools concentrate on the second part of that - the writing - whereas it's the first part that's hard and absolutely necessary.

Posted by Richard Steven Hack | December 26, 2007 7:03 PM

" ... a substantial number of journalists are ignorant, lazy, opinionated and intellectually dishonest. The profession is heavily cluttered with aged hacks, toiling through a miasma of mounting decrepitude and often alcoholism."
~Conrad Black

Posted by Comment | December 27, 2007 12:09 AM

"Math is hard!"
~Barbie

Posted by rickhavoc | December 27, 2007 9:38 AM

I've got to agree with those who vote for 'deliberate error'. Remember that this is a NYT article - not the Daily Podunk Dog Droppings and Fresh Gossip. The NYT can afford to hire experienced reporters who are among the elite, whether they are j-school/liberal arts grads who learned the quant stuff on their own, or quant guys who learned journalism on their own.

And the particular topic was adjusting for inflation, which should be very familiar to anybody writing about the economy. We're not talking about Bayesian shrinkage here.

Posted by Barry | December 27, 2007 10:20 AM

My 2 cents:

I teach sophomore intro economics in a large public research university, the flagship university of our state, with a large and reasonably well-regarded Journalism school that requires my course of all of its students.

When I look at the Journalism majors on my rolls, I am also looking at my D and F students. OK, a fair number are also C students. But in any given class (and these are large classes of 200+ students), over many years, only one or two of my journalism students make A's.

You don't have to know anything, or have any critical acumen, to be a TV anchor. These kids know that, and they resent being asked to think critically. And they are right, of course. Journalism is not about providing information, it is about selling product.

Posted by StratDouglas | December 27, 2007 3:19 PM