How undependable are the experts?

March 26, 2009 | By | 3 Replies More

We are in the middle of a huge economic crisis.  Should we listen to the experts?   Of course we should, because the economy and the financial sector are horrifically complicated.

What happens when the experts disagree, however?   To which experts should we listen?  I took a stab at that question recently, but I remain unconvinced that any of the economics experts can be trusted.   Yes, there are people like George Soros who have made a phenomenal amount of money during the crisis, but this makes me wonder whether he (and all of the other recent success stories) are smart or whether they are lucky.

Today, Nicholas Kristof (in the NYT) reminds us that many experts (at least political experts) have a terrible track record.  His opening sentence:  “Ever wonder how financial experts could lead the world over the economic cliff?”   He warns us of the “Dr. Fox effect,” named for a “pioneering series of psychology experiments in which an actor was paid to give a meaningless presentation to professional educators.”  Despite the fact that the lectures consisted of gibberish, they were well received.   He mentions a study showing that “clinical psychologists did no better than their secretaries in their diagnoses.”  He also mentions a study by Philip Tetlock which determined that “The [82,000] predictions of [284] experts were, on average, only a tiny bit better than random guesses — the equivalent of a chimpanzee throwing darts at a board.”  Those experts who were the most impressive to most people “provided strong, coherent points of view, who saw things in blacks and whites.”

I’m reminded of Alan Sokal’s intentionally nonsensical article that he submitted to the postmodern journal, Social TextSee here for more of the details.  BTW, if you want to generate your own postmodern bullshit, use this postmodernist bullshit generator (every time you hit the link, more impressive-sounding bullshit will be assembled automatically into an article).

How far astray are we led by “experts”?   Consider investment “experts.”  There are none worse.  Entire industries are built on the thoroughly disproved notion that a stock-picker can consistently beat the market.   Dan Smolin has made a career of proving that stock-picker experts are thoroughly and demonstrably terrible at what they claim to be.    But many of us still run to these financial “experts” to help us pick the “right” stocks.

Just think of the hundreds of political military experts who were similarly awful at their recommendations and predictions regarding the invasion of Iraq.   They appeared hundreds of time on network TV during the few weeks prior to the invasion, all of them confident in their assessments and advice.  Consider, also that fewer than 1% of them took anti-war stances. Consider, also, that many of these “experts” were secretly in positions to financially benefit from an invasion of Iraq.

Consider the thousands of religious experts, from coast to coast, who loudly and confidently tell their religious followers that there is a heaven and that they will go there, without the tiniest big of evidence in support.  The followers of fundamentalist preachers continue to listen to these guys even when they attack evolutionary biologists, even though these religious leaders have no training in science and no basic understanding of the principles of evolutionary biology.

Everyone loves weather forecasters, right?  These guys are wrong so incredibly often that no station dares to post their track records for those five-day forecasts they confidently present night after night.

The list goes on and on.  We insist on listening to the experts, medical experts, beauty experts, psychologists, their track records be damned.    That’s because they are the best that we’ve got, no matter how wrong they are how often.

The bottom line is that we crave experts because we crave certainty, even where there isn’t any.  The confirmation bias causes us to rely heavily on experts hawking our own opinions, even when there is no evidence in support, as long as the expert dishes out those opinions with a loud confident voice.   And a fancy business suit doesn’t hurt either.


Tags: , , , , , , , , ,

Category: Current Events, ignorance, Media, Politics, Psychology Cognition, Religion, snake oil

About the Author ()

Erich Vieth is an attorney focusing on consumer law litigation and appellate practice. He is also a working musician and a writer, having founded Dangerous Intersection in 2006. Erich lives in the Shaw Neighborhood of St. Louis, Missouri, where he lives half-time with his two extraordinary daughters.

Comments (3)

Trackback URL | Comments RSS Feed

  1. Niklaus Pfirsig says:

    I think meteorologists may be considered more accurate that any of the others, since they base their forecasts on a statistical analysis of regional conditions and historical data. Which is why the forecasts list probabilities for rain, instead of saying that it will rain. Part of the problem is that the general population, from being mentally numbed by a steady diet of video games and "reality" TV programming, are not capable on understanding statistical theory. The lack of understanding is used and abused by those with an agenda.

    Many of the experts are simply no better than the hair tonic salesman in "Sweeney Todd"

  2. Erich Vieth says:

    Dan Smolin finds the bogus claims of many investment funds distressingly predictable:

    There was one index fund in the mix, which the plan administrator proudly pointed out to me. It was the Dreyfus S&P 500 Index Fund (PEOPX).

    I checked the data on this fund from the Dreyfus web site. Here's what I found: The fund has over $2.4 billion in assets. It has total expenses of 0.50%, including a management fee of 0.25% and a "service fee" of 0.25%. The goal of this fund is simply to track the performance of the S&P 500 index. How difficult can this be?

    Quite difficult, when you have an expense ratio of 0.50%.

    The fund underperformed the index by 0.42% in the last year; 0.38% over 3 years and 0.46% over 10 years.

    He pointed out several low expenses that actually can track the S&P. The Dreyfus fund describes itself as follows: "The investment seeks to match the performance of the Standard & Poor's 500 Composite Stock Price index."

    The fix to this misrepresentation would seem to be simple. Rewrite the fund description: "This Dreyfus fund cannot track the S&P because of the high expenses we charge." Problem fixed. As Smolin points out this problem is ubiquitous in the world of finance.

  3. Jim Razinha says:

    Good thing I found this thread, because I had a fun bit on the postmodern generator mostly drafted in my head, and it would have been redundant. I updated my Firefox to Chrome, uh Firechrome, and was shuffling bookmarks when I tripped across it again. I had forgotten where I saw it first (thought Shermer for some reason) until I tracked down my copy of "A Devil's Chaplain" – along with a bunch of other "oh yeah!" books.

    …there is also language designed to be unintelligible in order to conceal an absence of honest thought.

    – Dawkins

Leave a Reply