« Stimulating | Main | Failed Hit »

Is Our College Students Learning?

25 Feb 2008 02:42 pm

Everyone in Washington says they want to bring down the cost of college. But as Kevin Carey writes, throwing more money into tuition subsidies isn't going to make college affordable over the long run as long as we keep in place structural incentives for ever-higher costs, most notably the total absence of any measure of quality. Unfortunately, America's colleges and universities are very good at creating a situation where nobody can get any real sense of which schools are doing a good job of educating people and which aren't. Under the circumstances, it's no wonder you don't see any institutions trying to find innovative and more efficient ways to deliver services. After all, nobody knows what the numerator is in the productivity equation, so a cheaper school just looks less fancy and prestigious.

For further reading on this issue I'd recommend Ben Adler's Washington Monthly article on the higher ed lobby and any of the many writeups of Alan Kruger's research indicating that professional success of graduates of highly selective colleges is almost all selection effect with little value-added.

Share This

Comments (48)

No offense, but this is not exactly a groundbreaking conclusion. Next we'll be told that people enroll in MBA programs for the networking rather than the pursuit of knowledge.

It is true that universities cannot drop costs as cost is seen as synonymous with quality. But many schools make up for this on the back end with financial aid offers that are actually tuition discounts and do not come from the endowment. So the true price of any school is its tuition combined with its discount rate.

Many smaller liberal arts schools have little to no endowment. You will find that they do a lot of streamlining in costs because tuition has to cover operating costs, and the discount rate that they have to charge to get students through the door eats into that directly.

Larger NSF R1 schools often don't care. They have a guaranteed student base, and a significant part of their revenue stream is grant overhead.

Unfortunately, America's colleges and universities are very good at creating a situation where nobody can get any real sense of which schools are doing a good job of educating people and which aren't.

Noooo!

It's almost as if the "best" universities are only going to admit students who're already the recipients of someone else's "good job of educating people" anyway, whom they can then pass along into the stratosphere of the workforce.

I have an axe to grind concerning college costs.

If you are rich then you pay the full amount.

If you are middle class then you pay a certain amount and get loans and grants for the balance.

As you get richer, you are expected to pay more and get fewer grants.

This means that college costs act as a fairly regressive tax for virtually everyone in the middle and upper class.

The net effect of college costs is that I in a far higher tax rate than the richest people in the country. And I know I am not alone.

It is absurd.

As you get richer, you are expected to pay more and get fewer grants.

This means that college costs act as a fairly regressive tax for virtually everyone in the middle and upper class.

Huh? If out-of-pocket costs go up as you go up in income, that sounds pretty progressive to me. You can argue that it's not sufficiently progressive. Or I suppose you could object to the capping of out-of-pocket costs at, you know, the actual stated price of the education. But regressive? That's a stretch.

After all, nobody knows what the numerator is in the productivity equation, so a cheaper school just looks less fancy and prestigious.

Really? Does anyone really think that the George Washington University is a better school than Berkeley? That the University of Miami is better than Michigan?

Really? Does anyone really think that the George Washington University is a better school than Berkeley? That the University of Miami is better than Michigan?

They are, if you measure them as schools. Whether you can measure their finished products (the students) as reliably, is his point.

Also, he said cheaper school, not better.

I buy "selection effect"--but there's more to it than that. Because of the selection effect, these schools can conceivably give you a better result than not because of what might be described as a competition effect. If you are a bright student who now has to compete with a lot of other bright students, you might push yourself more and have higher standards for yourself than if you were with less bright students. On the other hand, this doesn't really help the less bright students, who would tend to be stuck with their own kind as well.

But it's easier to see the effect if you recast it in terms of athletics. An elite athlete training with and competing with other elite athletes will turn in better performances than an elite athlete training with someone like me.

Just so you know, Krueger's paper isn't really taken seriously amongst economists.

If I remember correctly his coefficients aren't even statistically significant and he does some really weird stuff with his data.

He might still be right, but I don't think anyone thinks his data set gives him the power to make the claim he is making.

Bill - They are? Since when?

US News & World Report ranks Berkeley and Michigan at 21 and 25, and Miami and GW at 52 and 54. Certainly for people form California and Michigan, the public schools are much cheaper, and seem to have better reputations (and are thus, more prestigious), than the much more expensive private schools.

These are somewhat special cases, but note that the US News rankings probably discriminate considerably more on the basis of how much money a school has than most normal things do. So far as I can gather, someone with good grades from a decent, but not superlative, state school is at no particular disadvantage to someone who went to an expensive, but non-Ivy League class private school.

On what basis do you claim that the University of Miami is a better school than Michigan?

All I know is that I'm going to North Seattle Community College (at the age of 29, getting the degree I dropped out on at 19) and feel like I'm getting a quite solid education, in the sense that I know things about programming, writing, english comp, and mathematics that I didn't know last summer.

On what basis do you claim that the University of Miami is a better school than Michigan?

I didn't, and don't.

Whoops. and now that I reread my first post, I see I failed to connect my pronoun to the right antecedent. Or rather, I answered the question backwards. So, Mich and Berkeley ARE better schools on paper.

Is Our College Students Learning?

Remember, when Bush makes this kind of English mistake, it is hilarious and shows how stupid he is. When Matt makes this kind of mistake, it shows... What does it show again? Oh, well, Bush went to Harvard, too, so maybe it is Harvard's fault.

Umm ... It shows that Matt has mastered the literary device known as the allusion?

Re Matthew's comment "professional success of graduates of highly selective colleges is almost all selection effect with little value-added. "
------------
Still paying off that Harvard loan, Matthew?

Oh well, spending $160,000 --no, make that $180,000 and 4 years -- and ending up as an Atlantic blogger IS an Education.

of a sort. hee hee.

See "The Big Con" by David Maurer.

I don't think it's so hard to find out which institutions are doing the best job of educating students. Ignoring the "prestige factor", I'd say average class size is the best measure of what kind of education you're going to get. After that, it depends on the quality of the professor, and the emphasis the administration places on education over other demands on a professor's time. The seriousness of the other students also makes a difference. But what's the big problem with measuring educational quality? I'd say a lot of high school seniors are pretty good at it.

Measure of quality: Careful what you wish for. A desire for measures of quality got us into the standardized testing mess that we're in with public schools. All we need are our colleges teaching to the test.

Selection effect: Even if true, it strikes me as a terribly simplistic observation. How does one measure whether someone at the top of the scale has learned less, as much, or more as someone near the middle? And do you think Harvard or Berkeley could teach what it teaches if it had the same student profile as Southwestern State U?

Look, I suppose it's possible that our colleges and universities are perpetrating a huge scam, stealing money from unsuspecting students and their parents while teaching them nothing that they couldn't get from an online correspondence course. But if you really believed that, then go ahead and hire for your own company someone with one of those degrees advertised in spam email. Through selection effects and through education, our colleges really are teaching something. Honest.

1) The above discussion shows a rather fuzzy-headed approach at analysis --must have a bunch of Bowdoin graduates here.

Obviously, one needs to define the evaluation criteria --which probably vary greatly among students.

2) It's of little value to go to a highly rated University if it is weak in your subject of interest -- the Ivy League is very weak in Engineering, for example. And even Engineering schools tend to excel in some departments and not in others.

3) I've never seen much point in comparing total Universities against one another -- only departments. Harvard, MIT, Stanford etc are unusual in that they do well across multiple disciplines (but not all ) but that diversity is of value only if you don't know what you want to do and want to spend a lot to cover your bets.

4) The idea of wanting schools that are good at Handholding may be of value to some but not to others. If you're interested in being an entrepreneur , for example, you may highly value a school which wins a lot of money in the Federal COMPETITION for research dollars. Because you want a good library and the most current info on the leading edge of technology.

Only be sure to look at DOLLARS per department member -- some small schools win research funds out of proportion to their size.

5) In a few geo areas, you may want to look at COMMERCIAL funding for research as well. In Silicon Valley for example, the money flows to the relatively little known University of California at San Francisco --not so much to Berkeley or Stanford. Which is a sanity check on where the really smart money thinks value lies.

I will grant that the market for higher education is seriously screwed up. But the top schools really do offer a far better learning environment. From class selection to peer groupings to administrative respect for students, the experience difference between an elite private school and a really good private/public school isn't even comparable. In my experience I'm talking about the difference b/w MIT/Harvard and USC/UCLA/Boston University type schools. But if you live in CA, you can't beat the cost of Berkeley or UCLA.

It's of little value to go to a highly rated University if it is weak in your subject of interest -- the Ivy League is very weak in Engineering, for example.

Cornell engineering is top 10. Certain areas (e.g. Operations Research, Computer Science) are ranked even higher.

Note that the reputation of many universities is based on their GRADUATE Professional Schools -- Medicine, Business or Law. But that's irrelevant to an undergraduate. You can study at Timbuktu U and apply to Harvard's Medical School later.

Plus I've always wondered what's going through the mind of people who major in the Liberal Arts at MIT instead of Middlebury or Williams.

The top 10% of high school students do much more difficult work in high school than most of them will ever do in college.

The average high performing high school senior(say SAT and Subject test scores averaging above 700) is able to roughly describe photosynthesis, accurately identify the Corrupt Bargain, differentiate and integrate a function, and be comfortable analyzing a short story or novel (with dreadful prose, as high school English teachers could care less about quality of writing).

Average college graduates might be able to do one of those tasks, depending on their major, but quite possibly won't be able to do any of them.

So the whole notion that colleges should be measured on "teaching" is absurd, because the best high school students come in with more skills than most will have upon graduation. (That's why high performing students are less and less likely to go to average,cheaper schools, because all their peers are at the best schools.)

I also find it amusing that the notion of a college exit exam is getting credence, when the real problem is that we don't have a college entrance exam. So long as you don't demand standards coming in, you can't demand standards coming out.

And we'll never require standards on either side, because it would cause admissions to plummet with an unacceptable skew in certain demographics.

In a few geo areas, you may want to look at COMMERCIAL funding for research as well. In Silicon Valley for example, the money flows to the relatively little known University of California at San Francisco --not so much to Berkeley or Stanford. Which is a sanity check on where the really smart money thinks value lies.

Commercial grants are usually very small compared to government ones, especially military grants. Larger schools that have ready access to these grants often don't consider the commercial sector worth their while. Even good grants from Microsoft pale in comparison to an AFOSR or even an NSF grant.

The grant game is very complicated. Schools like Berkeley know how to play this game extremely well. It also has very little to do with education, so you should not use it as an educational metric.

Plus I've always wondered what's going through the mind of people who major in the Liberal Arts at MIT instead of Middlebury or Williams.

Generally, what's going through their head is, "At the age of 20, I realized that I will never succeed at/hate the sciences and engineering, but I still want to graduate in 4 years."

The quality of the graduate schools is advantageous to an undergraduate because it gives the undergraduate an opportunity to do research with a highly-regarded research professor and get a good recommendation for graduate school.


The quality of the graduate schools is advantageous to an undergraduate because it gives the undergraduate an opportunity to do research with a highly-regarded research professor and get a good recommendation for graduate school.

But for a lot of the metrics that I consider important to student learning this is not as much the case. Consider things like class availability or the relationship b/w students and the administration. These things will tend to be constant across the university.

Re Tyro's comment "The quality of the graduate schools is advantageous to an undergraduate because it gives the undergraduate an opportunity to do research with a highly-regarded research professor "
------------
Ha ha ha. Good one --you had me going there for a moment before I caught the irony. You English majors are cunning little devils at that -- it's the one marketable skill you learn , which is probably why you use it to the utmost.

NOTE that ,in my earlier comment, I was referring to PROFESSIONAL graduate schools.

I don't see what value an undergraduate English major gets from Harvard's rep in Medicine, Law, and Graduate Business (MBA).

(I was going to suggest "reflected glory" but I didn't want to trigger severe retching among the crypto-Trotskyites. Google "Larry Summers, public castration of ". Just after "Larry Summers, REAL Trotskyites, friend of" )

Re Walker's comment "Cornell engineering is top 10. Certain areas (e.g. Operations Research, Computer Science) are ranked even higher."
-----------
Hmmmm.

hee hee

[snicker]

ha ha ha ha ha ha ha stop it! Please! cough choke gasp

Although I concede that if you want to know how to harvest a good crop of corn, Cornell's the place to go.

What drives me crazy is this stupid assumption that the same student, given the opportunity to go to an Ivy League school verses their own state university will ALWAYS receive a better education at the Ivy.

Within every college and university, we have excellent teachers and we have lousy teachers. Too often, tenure means that the lousiest teachers hold the highest rank in departments, arrogantly disdaining their confused or low performing students, while the most creative, enthusiastic ones are regularly stuck at the bottom of the food chain, with no guarantees of permanency. Why? It's cheaper--and more importantly, we don't reward good teaching as the and its results in the highest priority in evaluating higher educators.


Another irony is, many of the best teachers actually seek out schools where the pace of life in the college community is sane, the student body is interesting and diverse in background and preparation, and take the opportunity to really invest in giving their students a superior quality education--even if they aren't "top tier".

And all of us from "no name" schools are much better for it, and some of us hope the rest of the country doesn't catch on and start shipping their little "geniuses" out to take up space in our terrific local colleges and universities. Please, stay focused on prestige, vanity, and superiority complexes, you folks.

"Too often, tenure means that the lousiest teachers hold the highest rank in departments, arrogantly disdaining their confused or low performing students, while the most creative, enthusiastic ones are regularly stuck at the bottom of the food chain, with no guarantees of permanency. Why? It's cheaper--and more importantly, we don't reward good teaching as the and its results in the highest priority in evaluating higher educators."

As someone who graduated from named schools, but now is tenured at a no-name school, I think that this is exaggerated. I've known very few professors whose bad teaching is rationally attributable to their having tenure. On the other hand, I know I'm a better teacher with tenure; I don't have to pander to numbers on student evals so administrators with no imagination have some way to "evaluate" me. (Actually, those numbers have gone up since not worrying about them, for whatever that's worth.)

There are many dirty little secrets in the ratings game. Brian Leiter at Texas has exposed many of those in the US News rankings. I have a number of colleagues at a school which caps its class size in the fall at a very low number (this is when the rankings information is gathered) and boosts is significantly in the spring. There is a great deal of this sort of thing.

I must say, though, that the same dispositions I have that make me very liberal politically cause me great consternation when I see people assessed by the name of the college they went to.

Re MattD's comment "the same dispositions I have that make me very liberal politically cause me great consternation when I see people assessed by the name of the college they went to."
----------
Console yourself in what I've noted before: If you look at Forbes' list of our 400 richest plutocrats, the list has a lot of college dropouts who never even applied to the Ivy League. About 10 percent of the list as I recall.

Of the top ten, 3 are Walmart heirs to Sam Walton-- who studied at the University of Missouri.

Many of the self-made billionaries in the computer industry --Paul Allen, Bill Gates, Larry Ellison, Michael Dell, Tom Waite, etc are dropouts. I don't see too many PhDs from Berkeley , Stanford, MIT or Harvard.

A little thought explains why: Spending $160,000 and 4 years of enormous effort just for a piece of paper is NOT the judgement of a successful entrepreneur.

The reality also is that students at the same university will have vastly different experiences. I go to Rutgers and am in the honors program. In honors classes the average class size is about 12. The average class size for a normal class is probably closer to about 70 and may not even be taught by a professor depending on the subject.

My brother went to Cornell and felt the quality of the education there (as an engineer) was rather poor. But it got him a fantastic job upon graduation.

So that raises one of the many many questions on the subject, do we measure schools by the product (if so, what is makes a good product?) or by the overall quality of the experience?

Nice Republican talking point, Matt. Lynn Cheney and many conservatives agree that there need to be measurements of quality in higher ed, and schools are currently bending over backwards trying to measure outcomes and please the regional college accrediting bodies who are requiring this thanks to such pressures. What this means in practical terms is increased standardization of courses, increased scrutiny and interference in the freedoms teachers have to teach classes they way they see fit. NCLB has had a powerful effect in narrowing k-12 curricula, and this standardization perfectly fits the conservative ideal of a limited, standard curriculum that produces a limited, standard student, all the while doing away with messy, diverse, individualized forms of learning that suit particular communities and regions. Cobra gets it, in other words.

For another perspective, see James Altucher's recent column in the FT: College a waste of time and money for kids

"All I know is that I'm going to North Seattle Community College (at the age of 29, getting the degree I dropped out on at 19) and feel like I'm getting a quite solid education, in the sense that I know things about programming, writing, english comp, and mathematics that I didn't know last summer."

You're getting more bang for your buck than most college students. Two years at a community college + two years at the best public 4-year college in your state that accepts you = good chance for high ROI on your tuition dollars.

Re Fred's "see James Altucher's recent column in the FT: College a waste of time and money for kids"
------------
A very good link, Fred.

Glad you liked it, Don Williams.

As a student at Berkeley...

Our reputation isn't as great as it should be, and I think a lot if has to do with how much cheaper our tuition is compared to the Ivies. We have some pretty awesome professors ala Bob Reich and A.J. Gregor-- there's no question that both students and professors are top notch.

I think all of the students here are pretty high achievers in the first place. There's no question that the students here can compete with Harvard or Stanford, or our other expensive counterparts.

But I also took a few courses at a community college, and found that I was getting a great education there, too (the greatest difference I saw right away was that there is less emphasis on research than there is at Berkeley).

Iz we learning? Sure! But there's no way to explain why an excellent education at UC Berkeley costs about $20,000 (including state subsidies per student) and is $40,000 elsewhere, and why my $400 community college education was just as good.

Re Walker's comment "Commercial grants are usually very small compared to government ones, especially military grants. Larger schools that have ready access to these grants often don't consider the commercial sector worth their while. Even good grants from Microsoft pale in comparison to an AFOSR or even an NSF grant."
-------------
Sigh. Must be a liberal arts major.

Doesn't anyone ever look at QUANTITATIVE EVIDENCE?
Or did that exotic idea die out with Galileo?

From the 2006 Lombardi Report "The Top American Research Univerisities" at
http://mup.asu.edu/research2006.pdf :

1) Stanford: total 2006 research $671 Billion ,
federal funded research component $541.7 Bil ,
other = $ 129.3 Bil

2) Berkeley: total 2006 research $525.6 Bil,
federal funded research $269.8 Bil ,
other = $255.8 Bil

3) University of California at San Francisco: total 2006 research $728 Bil ,
federal funded research $418.9 Bil ,
other = $309.1 Bil

Did anyone ever do a regression analysis to see the correlation in the US News "Peer rankings" of universities versus the size of the Universities' endowments (and hence, ability to pay lavish salaries to professors)???

hee hee hee

Working in the Learning Technologies department at a large public research university, I feel a constant pressure from above to be thinking about how we're going to deal with what's seen as an inevitable move by the federal government to start demanding institutional assessment.

I understand the desire for this type of reporting. But the fact is that there's only so far you can go in measuring what makes a good education. And to such extent, there are already numerous accreditation systems that assess a certain baseline level of quality. I don't think it's possible to get a really poor education at an accredited institution (assuming you put in the work and pass your courses).

In the end, I don't really understand what the goal is here. The only thing we could get out of some kind of federal standards for higher ed assessment is a set of rankings that are just as meaningless and arbitrary as the US News and World Report. What value is that to anyone?

The USNews rankings are absolutely meaningless for undergrad. I believe they're very important for law school - an exclusive law firm may interview, say the top quarter at a top 10 school, the top 10 percent at a top 10-25 or the top five percent or less at a tier 2 (26-50). That's just what I've heard from a couple law school friends, though. I assume MBa programs are the same way - some employers only hire from a certain class rank and top schools. Just another way of distinguishing between people to try and figure out who will put in the best 70-hour week.

I had some Princeton Review "The Best 300-something Colleges" book early in high school that I apparently forgot about. It was based on student surveys and had ratings for "campus life" and "academic experience." I found it awhile back and was looking through it. The "academic experience" ratings were in no way tied to reputation but seemed to really resemble the experiences of those I know who went to different schools.

I still do SAT tutoring to pad a meager salary, and I always recommend small liberal arts schools like Williams, Amherst, Colby, Pomona, etc., to the parents of bright kids. I can't bother to tell them that the most consistently good courses I took in college were not at the Ivy or top public but at a community college at ($150/course).

Matt: "Unfortunately, America's colleges and universities are very good at creating a situation where nobody can get any real sense of which schools are doing a good job of educating people and which aren't."

Since nobody knows what the content-free term "education" means in terms of actual knowledge, abilities, etc. - i.e., what the GOAL is - as opposed to pablum about being "consumers", "responsible citizens", blah, blah - clearly that's no surprise.

I remember reading a sci-fi short story years ago. Some guy gets sent to another recently discovered planet where the inhabitants were mostly close to human. They have an "educational establishment", so this guy goes there to find out what it's like.

He finds kids who, among other achievements such as understanding advanced physics by their teenage years, can kill animals approximating a Cape Buffalo with their bare hands by the age of 15 or so... He remarks in his report to his superiors something to the effect that "the average human child would be reluctant to attempt such a feat."

Great story that was apparently intended to contrast the stupid farce current human civilization calls "education" vs what would be considered a "real" system for training offspring to be able to function to their full capability.

The "education complex" is as much a farce as the "military-industrial complex", and the "law-enforcement-security complex" - and for the exact same reasons. Someone is being paid to, as Burroughs once said, "do everything in the stupidest, most incompetent way possible."

"Many of the self-made billionaries in the computer industry --Paul Allen, Bill Gates, Larry Ellison, Michael Dell, Tom Waite, etc are dropouts. I don't see too many PhDs from Berkeley , Stanford, MIT or Harvard."

Well, just to be fair:

Bill Gates dropped out of Harvard. Microsoft's CEO, Steve Balmer, graduated from Harvard. And the co-founders of Google were both Ph.D. candidates at Stanford. And billionaire Amar Bose, Ph.D. founder of the eponymous company, is an emeritus professor of electrical engineering at MIT.

Although it's true that in America people can and do become fabulously wealthy without elite educations, it's also true that schools such as MIT and Stanford produce far more of these super-rich than your typical state school (although Marc Andreeson was at the University of Illinois when he invented the precursor to Netscape).

"(although Marc Andreeson was at the University of Illinois when he invented the precursor to Netscape)"

Illinois is an example of a very good public school that is top-notch in a few areas (engineering, computer science, some natural sciences). What keeps it from being a Michigan or Berkeley is its programs in the humanities. Eudora and Telnet came out of UIUC, as well.

No small number of technological items we value came out of places like Illinois (and Wisconsin, Purdue; and ag products out of places like Iowa and Nebraska).

As a student at Berkeley... Our reputation isn't as great as it should be

Buck up, your graduate programs are top notch (I was rejected from one of Berkeley's PhD programs twice). What drags down UCB's reputation as an undergraduate institution is that the are a large number of merely "above average" students who dilute the impact of the truly brilliant ones that are there. Berkeley has something like 20,000 undergraduates, and it's hard to cultivate an elite reputation with that many students.

I always recommend small liberal arts schools like Williams, Amherst, Colby, Pomona, etc.

Blah. I see their appeal, but many years after graduating high school, I picked up an old course catalog from Williams that I had when I was applying to college in the 1990s. Looking through the list of computer science classes, I realized that they offered every single computer science class that an undergraduate should know to acquire a computer science degree. And that's all. By contrast, at the large research university I went to, I was able to experience a much larger range of different classes and opportunities in my field and work with professors in a wide variety of different areas. I'm not saying that the education you get at a liberal arts college is bad, but looking back on it, I can tell how I would have been limited by it.

And billionaire Amar Bose, Ph.D. founder of the eponymous company, is an emeritus professor of electrical engineering at MIT

His son, also, got a Ph.D. and started his own company. Generally, people with Ph.D.s from Berkeley aren't doing it because they want to be billionaires. There are far more direct ways of making lots and lots of money if that's what you want to do with your life.

The USNews rankings are absolutely meaningless for undergrad. I believe they're very important for law school - an exclusive law firm may interview, say the top quarter at a top 10 school

That's kind of my point. It's meaningful to the 2% of law students who are aiming for these exclusive firms. For the rest of the world, they can have meaningful and rewarding careers regardless of what law school they went to. If the school is accredited then they are getting a decent education.

Beyond that, I don't see what a new ranking scheme could do besides adding yet another layer of overhead to college bureaucracy and distorting the experience into something aimed at doing well on the government rankings instead of educating the students.

I think of college as part of your career, as for most people it's the primary barrier of entry to getting a job they want. We don't have federal rating systems for the quality of one's work experience, why do we need it for colleges?

umm, why am i dubious when someone posting as Jar Jar Binks claims that:

"Just so you know, Krueger's paper isn't really taken seriously amongst economists."

evidence, at all? krueger is a pretty big heavyweight in the labor economics field. his minimum wage paper being another example of his work.


Comments closed March 10, 2008.

Copyright © 2008 by The Atlantic Monthly Group. All rights reserved.