How undependable are the experts?

We are in the middle of a huge economic crisis. Should we listen to the experts? Of course we should, because the economy and the financial sector are horrifically complicated. What happens when the experts disagree, however? To which experts should we listen? I took a stab at that question recently, but I remain unconvinced that any of the economics experts can be trusted. Yes, there are people like George Soros who have made a phenomenal amount of money during the crisis, but this makes me wonder whether he (and all of the other recent success stories) are smart or whether they are lucky. Today, Nicholas Kristof (in the NYT) reminds us that many experts (at least political experts) have a terrible track record. His opening sentence: "Ever wonder how financial experts could lead the world over the economic cliff?" He warns us of the “Dr. Fox effect,” named for a "pioneering series of psychology experiments in which an actor was paid to give a meaningless presentation to professional educators." Despite the fact that the lectures consisted of gibberish, they were well received. He mentions a study showing that "clinical psychologists did no better than their secretaries in their diagnoses." He also mentions a study by Philip Tetlock which determined that "The [82,000] predictions of [284] experts were, on average, only a tiny bit better than random guesses — the equivalent of a chimpanzee throwing darts at a board." Those experts who were the most impressive to most people "provided strong, coherent points of view, who saw things in blacks and whites." I'm reminded of Alan Sokal's intentionally nonsensical article that he submitted to the postmodern journal, Social Text. See here for more of the details. BTW, if you want to generate your own postmodern bullshit, use this postmodernist bullshit generator (every time you hit the link, more impressive-sounding bullshit will be assembled automatically into an article). How far astray are we led by "experts"? Consider investment "experts." There are none worse. Entire industries are built on the thoroughly disproved notion that a stock-picker can consistently beat the market. Dan Smolin has made a career of proving that stock-picker experts are thoroughly and demonstrably terrible at what they claim to be. But many of us still run to these financial "experts" to help us pick the "right" stocks. Just think of the hundreds of political military experts who were similarly awful at their recommendations and predictions regarding the invasion of Iraq. They appeared hundreds of time on network TV during the few weeks prior to the invasion, all of them confident in their assessments and advice. Consider, also that fewer than 1% of them took anti-war stances. Consider, also, that many of these "experts" were secretly in positions to financially benefit from an invasion of Iraq. Consider the thousands of religious experts, from coast to coast, who loudly and confidently tell their religious followers that there is a heaven and that they will go there, without the tiniest big of evidence in support. The followers of fundamentalist preachers continue to listen to these guys even when they attack evolutionary biologists, even though these religious leaders have no training in science and no basic understanding of the principles of evolutionary biology. Everyone loves weather forecasters, right? These guys are wrong so incredibly often that no station dares to post their track records for those five-day forecasts they confidently present night after night. The list goes on and on. We insist on listening to the experts, medical experts, beauty experts, psychologists, their track records be damned. That's because they are the best that we've got, no matter how wrong they are how often. The bottom line is that we crave experts because we crave certainty, even where there isn't any. The confirmation bias causes us to rely heavily on experts hawking our own opinions, even when there is no evidence in support, as long as the expert dishes out those opinions with a loud confident voice. And a fancy business suit doesn't hurt either.

Continue ReadingHow undependable are the experts?

How to weed out junk science when discussing climate change.

George Will's recent journalistic malpractice has inspired much discussion by many people concerned about climate change. It's a critically important issue given that 41% of Americans currently think that the threat of global warming is being exaggerated by the media. The intellectual energy runs even deeper than criticism of George Will, though, leading us to the fundamental issue of how journalists and readers can distinguish legitimate science from sham (or politicized) science. The Washington Post recently agreed to publish a precisely-worded response to Will by Christopher Mooney. Here's Mooney's opener:

A recent controversy over claims about climate science by Post op-ed columnist George F. Will raises a critical question: Can we ever know, on any contentious or politicized topic, how to recognize the real conclusions of science and how to distinguish them from scientific-sounding spin or misinformation?

Mooney methodically takes Will to task on point after point. For instance, weather is not the same thing as the climate. The state of the art in 1970s climate science has been superseded by 2007 climate science. You can't determine long-term trends in Arctic ice by comparing ice thickness only on two strategically picked days. The bottom line is not surprising. If you want to do science well you have to do it with precision, measuring repeatedly, crunching the numbers every which way and then drawing your conclusions self-critically. What is not allowed is cherry picking.

Readers and commentators must learn to share some practices with scientists -- following up on sources, taking scientific knowledge seriously rather than cherry-picking misleading bits of information, and applying critical thinking to the weighing of evidence. That, in the end, is all that good science really is. It's also what good journalism and commentary alike must strive to be -- now more than ever.

Mooney has given considerable thought to these topics. His byline indicates that he is the author of "The Republican War on Science" and co-author of the forthcoming "Unscientific America: How Scientific Illiteracy Threatens Our Future." I would supplement Mooney's well-written points, borrowing from our federal courts. They have long been faced with the struggle to determine what is real science and what is junk science, and they have settled on what is now called the "Daubert" test, (named after the case first applying the test, Daubert v. Merrell Dow Pharmaceuticals, 509 U.S. 579 (1993)). The Daubert analysis is applied many times every day in all federal courts (and many state courts) all across America. The problem facing judges is that the parties to law suits often produce experts who express scientific theories and explanations that are never heard outside of courtrooms. This justifiably makes judges suspicious. Is the witness doing "real" science or his he/she doing sham science to further the interests of the party paying his/her bills? The Daubert test asks the judge to serve as gatekeeper, to make sure that only legitimate science sees the light of day in courtrooms. Here are the relevant factors:
  • Does the method involve empirical testing (is the theory or technique falsifiable, refutable, and testable)?
  • Has the method been subjected to peer review and publication?
  • Do we know the error rate of the method and the existence and maintenance of standards concerning its operation?
  • Is the theory and technique generally accepted by a relevant scientific community?
Positive answers to each of these factors suggests that the witness is doing real science. Astrology would fail this test miserably. Applied to climate science, the Daubert test would require that we listen carefully to what the scientists talk about with each other, in person and in their peer-reviewed journals. Daubert would require that we know enough about the techniques of climate science to know how it makes its measurements and conclusions. Daubert would certainly require that we know the difference between the weather and the climate. Applying Daubert is not simply a matter of listening to the scientists. Quite often, the scientists are bought and paid for (e.g., scientists working for tobacco companies and corrupt pharmaceutical companies). Applying Daubert requires taking the time to understand how the science works to solve real-world questions and problems and then taking the time to see that its methodology is being used with rigor in this application. There are no shortcuts, expecially for outsider non-scientists. No shortcuts. No cherry-picking.

Continue ReadingHow to weed out junk science when discussing climate change.

Linking to Wikileaks could cost you $11,000

Well, it will if the Australian government gets its way on its internet censorship bill. That's right. The ACMA seems to have placed Wikileaks on its potential web blacklist and seems set on throwing fines of up to $11,000 at anyone who links to it. I'd happily go all out on this one, but a fellow Antipodean has already got this one in his sights:

I'm posting this on my American blog because the Australian government, through the Australian Communications and Media Authority is fining people on Australian sites who give the links below the fold $11,000/day. Pretty well everything I feared about censorship by the internet filter and heavy handed government action is coming true. First of all, it transpires that only one bureaucrat at ACMA is required to block and ban a site, with no further oversight or redress. Second, it turns out that yes, ordinary and popular pornography sites are being blocked, so that if the filter becomes mandatory, these legal sites will effectively become censored for no apparent reason (other than political whim or special privileges). Thirdly, the whistleblower site Wikileaks is blocked by the ACMA blacklist.
John follows with the excerpt from a Crikey article:
Like New Labour in the UK, the ALP has now abandoned that [civil liberties movement], for a number of reasons. Once it committed itself to neoliberal economics ("social capitalism") Labo(u)r became freaked about the social dissolution and rupture, the desocialisation created by turning the polis into a giant market of winners and losers. The tough answer to this is genuine social democracy, in which people have a social being not entirely defined by whether they're a "winner" or a "loser". The easy answer is to let the market rip, allow it to change the culture, and then seek to control and reshape people's behaviour, selling it to them as "protecting the many against the few".

Continue ReadingLinking to Wikileaks could cost you $11,000

A 24-hour news network lineup I’d like to see

6:00 AM The Overnight News The morning show. Discusses significant events that happened in America the night before, as well as important stories that developed during the day in other time zones around the world. 9:00 AM Science Today Expert analysis of some of this week's more significant peer-reviewed publications.…

Continue ReadingA 24-hour news network lineup I’d like to see

Obama Backlash Growing Quickly

For the last eight years, the fear and doubt crowd have had one of their own in the head office. Comforted that science will be suppressed and church programs will be federally funded, the vocal conservatives fell into relative complacency. Or did I miss it? But now, the flood gates are opening. The "gun nuts" are buying up weapons and ammunition creating a price spike never before seen. Why? Because they are convinced that Obama is a communist who will outlaw their guns and tax their ammo. The anti-family-planning crowd is staging bigger sit-ins at health clinics. They are also submitting bills to local, state, and federal government to restrain the feared upsurge in availability of contraception and related information. He overturned the stem cell funding ban. Now discarded blastocysts are again eligible research subjects rather than just trash. The humanity! After all, they quail, if we don't respect trashed cell clusters as people, how can we possibly value adult citizens? Anti-science groups are pushing ever more vocally for science in schools to be properly tempered by religious counterpoints. Bills appear in state after state calling for "Academic Freedom," meaning to give the Bible equal weight as proven science in schools. Yes, this started long ago, but now they have greater urgency. The conservative media is calling Obama's initial tax cuts a hike, and his pushing through of the stimulus package (that had been in the works for months under his predecessor) as typical unrestrained Democrat unilateral spending. They can get away with it because they knew that the necessary spending will pass whether or not they approve it. Looking tough with no teeth is all they have, at present. Recall the political fury during the 1930's depression. A Democrat prevailed over strong Republican objections and turned the country around doing basically what Obama is now doing. He kept getting re-elected because he got results. Results directly opposite of all the dire predictions of the Republicans. Basically, the same arguments being made now against Obama.

Continue ReadingObama Backlash Growing Quickly