Economics and similar, for the sleep-deprived

A subtle change has been made to the comments links, so they no longer pop up. Does this in any way help with the problem about comments not appearing on permalinked posts, readers?

Update: seemingly not

Update: Oh yeah!


Wednesday, September 05, 2007

 
Freakiology

yes folks it's part 4 of the "Freakonomics" review, now entering into its third glorious year!

Welcome people who came here via direct links! If you are British, would you mind reading this post too? Thanks awfully. If you aren't British, as you were.

Part Four of this review is meant to deal with the odd and slightly patronising relationship that Freakonomics has with normal sociology. I've kind of pussyfooted around this subject so far (although there is a bit about it in the crime gangs section), and include by citation Henry and Kieran (yer actual sociologists) from their contributions to the CT book seminar. I think Henry and Kieran were both wildly too generous in their assessment of the book - you can see that they made a number of caveats and criticisms, but basically welcomed it as being both healthy economics and an interesting contribution to sociology. I take a much more cynical view of both these possibilities - note however that a lot of Henry and Kieran's comments are about Levitt's more general research program and this review is of the specific book "Freakonomics". Let's start off by asking an interesting question - how did the book itself come to be written?

Well I think it happened this way. A publisher found a hole in their autumn schedule where a "big ideas" pop academic book ought to be. They thought that with the right amount of publicity, they could replicate the wildly successful Malcolm Gladwell phenomenon. Normally, the New Yorker is where you'd go to for someone with the right combination of Big Ideas and easy-readin' prose, but they were out of luck. James Surowiecki would have obviously filled this gap just perfectly, but he'd just signed to a different publisher, so they started casting the net a bit wider. Someone had told them that there was this wacky economist guy in Chicago generating Gladwell-style ideas at a rate of knots, but nobody liked the idea of putting a big publicity budget behind an untried writer. So they brought in a safe pair of hands in the form of Steven Dubner, a workmanlike journalist on New York magazine (Edit: Felix reminds me that "New York" magazine is not the same thing as the New York Times magazine, which Dubner actually wrote for, thanks) who was unlikely to set the world on fire on his own, but who could be trusted to translate Levitt's prose into readable English and add enough gee-willikers analogies and anecdotes to convince the airport crowd that they were understanding what they were reading. And thus was a popademic sensation born!

No, hang on, that makes it sound like the whole thing was churned out like a sausage factory. Let's try again.

Well, I think it happened this way. Dubner was a magazine journalist who was doing all right on New York the NYT magazine, but was well below the exalted level of a Malcolm Gladwell or a James Surowiecki. Realising that he was basically nowhere near as talented or intelligent as those guys, and therefore had no chance of coming up with enough ideas himself to write a "The Tipping Point" or a "The Wisdom Of Crowds", but nevertheless wanted to make a load of money off a potboiler book, he decided that half of sumthin' beat all of nuthin and started looking around for someone with a load of ideas. He got commissioned to write the magazine profile of Levitt and bingo, a cash-in popademic sensation was born!

No, no, hang on, that's much too cynical and unpleasant. Let me start again …

Levitt is a Beckerite, a particularly virulent subspecies of economist prevalent at the University of Chicago and specialising in applying economic analysis to other areas of human behaviour. Most other Beckerites haven't made much of an impression on the field, mainly because they are in general really quite nasty people. Levitt had one unusual property, which was that despite being a Beckerite, he wasn't an asshole and was therefore able to create a career by coauthoring Beckerite papers with people who had important skills that he didn't, therefore lifting them far above the run of the mill of Beckerites. For example, he managed to get Ian Ayres to do his calculus for him, John Donohue to do his statistics for him, Suhdir Venkatesh to do his primary research for him and so on. This got him a degree of academic fame, but John Bates Clark medals ain't going to buy the Jacuzzi. One day, however, he met this magazine journalist called Steve Dubner, and realised that he could pull the same trick of coauthoring. Dubner did his job of hyping Levitt's research to a parodic degree, and a popademic sensation was born!

No, no, hang on, that's really mean-spirited and nasty. Let's try one last time.

Dubner was a good writer who had an interest in social and policy issues, who found someone with really interesting ideas but no audience. Levitt was a personable economist who wanted to get a wider public audience for what he considered to be intrinsically interesting research. Together, they thought that an extended version of Dubner's magazine profile might make a really good book. The publisher, mindful of the Gladwell sensation, decided to publicise the hell out of it, and it took off and they both enjoyed a more or less deserved success.

The point here is that you can always tell any one of a dozen stories about anything, and all of them often have an element of truth to them. John Kay even goes further and says that for a lot of important stories, there ain't no single truth (the untruest thing you will ever hear at business school will come from one of your classmates, and it will be the great falsehood "I Was There And It Didn't Happen Like That"). In Freakonomics, Levitt and Dubner have a real habit of always picking a single story, and usually one which reflects really badly on the people involved.

So, for example, sumo wrestlers are "cheating" because wrestlers who need to win a bout to stay in the top league do statistically significantly better when fighting wrestlers for whom the match is a dead rubber. Not "taking sensible steps to minimise the risk of injury". Not "following unwritten social conventions of the sport". Not anything else, but "cheating", and most likely doing so because of bribery and corruption from betting syndicates. Perhaps one day Levitt and Dubner will look at the behaviour of Tour de France cycle riders, and it will just totally blow their minds.

Similarly when it turns out that estate agents' houses typically sell for about 10% more than comparable houses owned by non-estate agents, then this shows that those rascally agents are ripping you off!!1!. That's it! Personally, I don't think 10% is that much. If I went to Aubergine and got a meal 90% as good as Gordon Ramsay gets when he eats there, I'd be pretty damn pleased. Furthermore, estate agents selling their own houses don't have to deal with their horrible emotionally needy clients making all sorts of irrational decisions, and are more likely to be moved by financial considerations than by which buyer was nice to their dog and more likely to keep their gazebo. And so forth, and so on. It's a version of what George Orwell was talking about in bemoaning the habit of self-styled "sophisticated" Marxists to scout around for the lowest and meanest possible motivation for anything and then proclaim it the "real" one. I've often joked that neoclassical economics and evolutionary psychology are unusual among social science fields because they're the only areas where you can gravely insult someone by agreeing with their theory (by telling an economist that he's only saying what he's saying because he's well paid for doing so, or telling an EP type that his latest paper is simply an attempt to draw attention to himself and gain status). Freakonomics is fairly and squarely in that tradition.

I think a large part of the trouble here is that the whole project of attempting to apply the methodology of economics to the kind of behaviour studied by sociologists is that the "rigour" of classical economics comes from a reductionist approach, and for this reason it really doesn't handle "thick" concepts, with non-nugatory social meaning to them, at all well. For example, sumo wrestlers on the brink of relegation do well against superior wrestlers who are sure of staying in their own division. Certainly sounds like evidence of cheating.

On the other hand, as I've noted above, there are a lot of other potential explanations. First, of course somebody is going to fight harder in an important match than in a dead rubber. Second, there are all sorts of unwritten conventions in sports which can be analysed as repeated games, where people pass up potential short term gains to maintain an arrangement which suits the long term interest of all (in cricket, bowlers pass up a lot of Mankading opportunities, for example) and the wrestlers might be following one of these. I actually think that this is the more likely explanation for the pattern of behaviour seen in Levitt's work, as it doesn't make much sense to me for this pattern to be the result of match-fixing by gambling syndicates. If you're carrying out betting coups, you want a match with a clear favourite and with a lot of betting on it, and you do it as infrequently as possible[1]. Fixing nearly every single one of a large category of evenly matched battles between no-marks is about the opposite of economically sensible behaviour[2].

But third, and most importantly, what's "cheating", within the world of sumo? Fixing title fights certainly is (there is a lot of evidence that this happened, although it's tangential to the work in Freakonomics). But how about "not using your best moves on Joe-Bob who has had a bit of bad luck in the tournament, in return for which he will lie down for you on your next match en route to the title"? After all, carry out even a cursory analysis of mountain stage times in the Tour de France and you'll pick up evidence of the "charabanc" - the arrangement whereby the sprint specialists ensure that the gap between the leader and the back of the field does not get too large to disqualify the sprinters. This will certainly show that some riders go much slower on mountain stages than they otherwise would be capable of. But publish a paper demonstrating that you've found evidence of collusion between teams at the back of the pack on Alpe d'Huez and the team from L'Equipe will probably ask if you're someone's simple brother.

There are a hell of a lot of "you scratch my back and I'll scratch yours" arrangements in the world. Some are illegal, some are disreputable, some are more-or-less standard and some are pretty much constitutive of what it is to behave honourably. I think you'd have to know a hell of a lot about the culture of sumo wrestling in order to be 100% sure that this mapped unambiguously onto the word "cheating". I even bet there would be a lot of disagreement between people within that world about the nature of the implicit contracts there. That's what sociologists exist to tell us about, and one is not doing a service to the world of knowledge by pretending that these thick categories can be flattened out. Economics' big claim to being a science is at least partly based on its formalism, but if you try to investigate a lot of social phenomena without paying sufficient attention to what your data points mean then … well, then you're going to end up concluding that a crime gang is more or less like a capitalist firm with an "up or out" promotion structure [3].

But hang on … surely the special status of the claims of Freakonomics is underwritten by the fact that they're driven by the data? After all, there are plenty of bits of research in Freakonomics that don't really fit into the economic paradigm at all - the childrens' names, or the cheating teachers. So maybe we should read through the verbiage about "incentives" this and "optimisation" that, as being sort of vestigial Beckerism that will fall away as we develop a brand new field of "data-driven social analysis" (to quote the CT seminar).

Well I think not. The problem is, which data are driving Freakonomics? The answer certainly appears to be "whatever data happened to pass across Levitt's desk". There is no guarantee that the Micawber approach to data-gathering is a valid one. Sociologists in general ask a hell of a lot more questions about the validity of their data, what exactly they are measuring, what implicit assumptions are made in the data gathering process, etc etc ad nauseam, than economists do. There's a reason for this.

The reason is that economists are in general used to working with standard data that is gathered by official bodies. Twas not always the case. Sorting out what Gross Domestic Product was meant to measure, how to measure it and all the related issues in the system of national accounts was carried out in the 1930s and there is a reason why the third Economics Nobel ever awarded was given to Kuznets for doing it. But for a period of time longer than a generation of economists, the basic material of economics has been hard numbers, collected in annual statistical abstracts, checked, collected and audited by Someone Else. Wasily Leontief was complaining about the atrophying of data-gathering skills in the profession back in the 1950s, and it's got worse since.

But anyway, the point is that "the data" by and large has a fixed referent in the context of economics - it's the same data which everyone else is using, collected by and large for a known purpose, by people without much axe to grind. And also, in economics, the question that the data has been collected to answer is the same as the question you're asking, because economics has intentionally restricted its subject matter to the organisation of production, consumption and exchange. There's no question about other aspects to the problem because economics isn't interested in the non-widget significance of Amalgamated Widgets (or for that matter, in widgets except in as much as they are bought or sold for the numeraire).

You just can't say that about "the data" when the referent is "this dataset that I've just come across", or you're going to make two categories of mistakes. First, you're going to make mistakes of data quality control unless you're very careful, because there's no equivalent of the BLS underwriting the quality of "the data" that you're using. This is a widespread problem in actual economics, by the way - it's genuinely frightening how many otherwise sensible economists use the Freedom House or similar "Indices of Democracy" as if they were useful numerical proxies for the state of social and political institutions, despite the horrendously tendentious way in which those scores are compiled. And even when the data aren't being compiled by someone putting their thumb on the scales, the greater degree of freedom that you have in picking up your "quirky" datasets means that there's much more due diligence required in making sure that you haven't picked up a lemon (and much more self discipline needed to make sure that you're not data-dredging, a subject covered in previous sections).

And second, there's a big danger of mistaking the map for the territory - assuming that you've got the answer, that "the data" tell you, rather than an answer which the combination of the data you happened to pick up and the model you happened to impose on it told you. I said in part one that Levitt & Dubner have a really bad habit of saying:

"Whichever way you look at the numbers, X"

when all they can really justify is:

"Whichever way I look at the numbers, X".

but in fact, I should have said that they could only really support:

"Whichever way I look at these numbers, X".

So there's a big danger of coming up with a partial, misleading explanation, and "the data" will not warn you when this is the case; there's a big danger that you'll end up chasing a lot of charabancs, in the sense of finding explanations of social phenomena as solutions to short-term optimisation problems while ignoring the long-term context in which they are placed. The best you can hope for is that a passing sociologist, or someone with specific institutional knowledge, will set you right (in which case the game theory textbook is certainly big enough for you to come up with a "just so" story fitting the new explanation into your theoretical framework as being "really" an iterated game; the Davies-Folk Theorem applies here). But even this best case is not really adding anything to the sum of human knowledge; it's just an example of one discipline trying to colonise another.

Which brings me back to my fundamental problem with this book - the attempt to simultaneously hunt with the quantitative hounds and run with the qualitative hare. The jacket of the book advertises a universal, general toolbox that gives definitive answers to important questions. It advertises this in a way that, say, the latest Gladwell, or "The Wisdom of Crowds" doesn't. Those books are meant to be loosely themed compilations of the ideas of clever journalists, showing you one way of thinking about some interesting questions, which sometimes works. The big selling point of Freakonomics was that this was meant to be science. This was meant to be the one in which you got the real answers, if you were only smart enough to follow the true path of incentives and data. And it doesn't, because it can't, because good economics is bad sociology.

Quite why this expedition to try and make economics do the work of sociology was first carried out, and what it means for the current state of economics, will be the subject of part Five.

[1] Or alternatively, there is the Malaysian method on cricket, where you fix the betting on something other than the actual result (overs bowled, wides, minutes of play, etc). But this would depend on the existence of a much wider and deeper betting market than the fairly nugatory underground one which exists on sumo.

[2] This is not the first time I've had cause to question the thoroughness with which the "thinking like an economist" model is being applied. Remember in part One of the review, where I was asking why it was that the sales force of that crack gang were, according to the accounts, being paid flat wages with no element of sales commission? More about this in part five, on the general theme of there really being no substitute at all for specific institutional knowledge.

[3[ I think that the crack gang chapter is the worst mistake in the book and the one which really proves that the "Freakonomics" methodology leads to serious errors. It is quite likely that the average sumo fan (although perhaps not the average wrestler) would regard tacit collusion on the final day of a tournament as being cheating. But a crack gang (even a really weird one in which all the members are paid salaries and don't take sales commission, sorry to bang on about this but I mean really) isn't like McDonalds, or like McKinsey, or like any other recognisable capitalist organisation. Proof; even really primitive cultures have groups recognisable as youth gangs, but they don't have groups recognisable as corporations. Again, social meaning. The purpose of a gang is not just making money. Fifty Cent should not be taken as a reliable witness on this issue.
31 comments this item posted by the management 9/05/2007 08:35:00 AM


This page is powered by Blogger. Isn't yours?




Links:

Bitch : Lab
Aaronovitch Watch
Balkanalysis
Perfect.co.uk
Maxspeak
Brad Delong
The Robert Vienneau blog

Political and philosophical heroes

Subcomandante Marcos
Will Rogers
Boris Vian
The English Svejk

RSS Feed:
This seems to matter to a lot of people

If you liked this "Daniel Davies" website, you might be interested in

"Danux", the web developer
The martial artist (and fan of extremely annoying Flash intros) from Blackburn
The Welsh political journalist
A Scouse computer programmer who collects Soviet cameras
"Danimal", the heavy metal drummer
Canada's finest recorder of radio jingles
More of the same, at the Guardian
A tailor's in Lampeter where Jimmy Carter once bought a hat
An advertising man who has written a novel about dogging (I think we sometimes get each other's email)
An award-winning facilities manager in Dubai
The son of the guitarist from the Kinks Update: he is apparently "balls-out motherfucking shit-dicked exxxstatic" to be included on a Kerrang magazine giveaway CD of Iron Maiden covers, which is nice.
"Fritz Gretel" from the Ramones film "Rock 'n' Roll High School"
The former presenter of the leading politics talk radio show on the Isle of Man, now a business change manager in the Manx government secretary's office
An aquarium curator in Sussex who keeps on scoring home runs like this (this is the first stable link I've found, but he is constantly kicking ass in acquarial terms)

If you didn't like this "Daniel Davies" website, then don't give up on the Daniel Davies industry completely!

An American "Christian Political Analyst" who has the same name as me
A student at Patrick Henry College
these two might be the same guy ...
"Scatter", the deceased Liberian gangster
A naked man stuck in a chimney in Wigan
A thug in Barrow



This blog has been going downhill since ...

August 2002
September 2002
October 2002
November 2002
December 2002
January 2003
February 2003
March 2003
April 2003
May 2003
June 2003
July 2003
August 2003
September 2003
November 2003
December 2003
March 2004
April 2004
May 2004
May 2005
June 2005
July 2005
August 2005
September 2005
October 2005
November 2005
December 2005
January 2006
February 2006
March 2006
April 2006
May 2006
June 2006
July 2006
August 2006
September 2006
October 2006
November 2006
December 2006
January 2007
February 2007
March 2007
April 2007
May 2007
June 2007
July 2007
August 2007
September 2007
October 2007
November 2007
December 2007
January 2008
February 2008
March 2008
April 2008
May 2008
June 2008
July 2008
August 2008
September 2008
October 2008
November 2008
December 2008
January 2009
February 2009
March 2009
April 2009
May 2009
June 2009
July 2009
August 2009
September 2009
October 2009
November 2009
December 2009
January 2010
February 2010
March 2010
April 2010
May 2010
June 2010
July 2010
August 2010
September 2010
October 2010
November 2010
December 2010
January 2011
February 2011
March 2011
April 2011
May 2011
June 2011
July 2011
August 2011
September 2011
October 2011
November 2011
December 2011
January 2012
February 2012
March 2012
April 2012
May 2012
June 2012
July 2012
August 2012
September 2012
October 2012
December 2012
February 2013
April 2013
June 2013
July 2013
August 2013
March 2014
April 2014
August 2014