Tag Archives: probabilities

We Reckon Scientists Feel 95% Certain, But Don’t Ask Us How

28 Sep

The UN IPCC released its 5th Summary For Policy Makers yesterday.

From TIME magazine:

95%. That’s how certain the hundreds of scientists who contribute to the Intergovernmental Panel on Climate Change (IPCC), which released the first chapter of its fifth assessment on global warming this morning, are that human beings are the “dominant cause of observed warming” that’s been seen since the mid-20th century.

In science, which almost always speaks in probabilities, that’s about as clear as you get.

Hmmm.

But how exactly is that “95%” certainty figure arrived at?

Professor Judith Curry explains (underline/red is mine):

I tried to figure out how the IPCC AR4 came up with the ‘very likely’ (90%) confidence level for the attribution statement…

[TBI: that is, how did they come up with the % “confidence” level for the previous IPCC assessment]

The IAC Review of the IPCC recommended the following:

Chapter Lead Authors should provide a traceable account of how they arrived at their ratings for level of scientific understanding and likelihood that an outcome will occur.

The IPCC uncertainty guidance urges authors to provide a traceable account of how authors determined what ratings to use to describe the level of scientific understanding (Table 3.1) and the likelihood that a particular outcome will occur (Table 3.3). However, it is unclear whose judgments are reflected in the ratings that appear in the Fourth Assessment Report or how the judgments were determined. How exactly a consensus was reached regarding subjective probability distributions needs to be documented.

Yesterday, a reporter asked me how the IPCC came up with the 95% number.  Here is the exchange that I had with him:

Reporter:  I’m hoping you can answer a question about the upcoming IPCC report. When the report states that scientists are “95 percent certain” that human activities are largely to cause for global warming, what does that mean? How is 95 percent calculated? What is the basis for it? And if the certainty rate has risen from 90 in 2007 to 95 percent now, does that mean that the likelihood of something is greater? Or that scientists are just more certain? And is there a difference?

JC:  The 95% is basically expert judgment, it is a negotiated figure among the authors.  The increase from 90-95% means that they are more certain.  How they can justify this is beyond me.

Reporter:  You mean they sit around and say, “How certain are you?” ”Oh, I feel about 95 percent certain. Michael over there at Penn State feels a little more certain. And Judy at Georgia Tech feels a little less. So, yeah, overall I’d say we’re about 95 percent certain.”  Please tell me it’s more rigorous than that.

JC:  Well I wasn’t in the room, but last report they said 90%, and perhaps they felt it was appropriate or politic that they show progress and up it to 95%.

Reporter:  So it really is as subjective as that?

JC:  As far as I know, this is what goes on.  All this has never been documented.

Got that?

The UN’s own official InterAcademy Council (IAC) review of the IPCC’s procedures and processes for their previous assessment report admitted that even though the IPCC “urges authors to provide a traceable account of how authors determined what ratings to use“, nevertheless this did not happen —

it is unclear whose judgments are reflected in the ratings that appear in the Fourth Assessment Report or how the judgments were determined.

And the IAC admitted that the IPCC provided no documentation to show how the supposed “consensus” view of the subjective judgments of hundreds of individual scientists with regard to “certainty” was actually reached —

How exactly a consensus was reached regarding subjective probability distributions needs to be documented.

In effect, this is what the IPCC is really saying about the “certainty” of scientists contributing to its assessment of the present state of climate science:

“Trust us. We reckon the scientists — collectively — feel 95% certain. But don’t ask us how.”

josh

%d bloggers like this: