Download IPCC Guidance Note on Consistent Treatment of Uncertainties for Fifth Assessment Report and more Summaries Decision Making in PDF only on Docsity!
Guidance Note for Lead Authors of the
IPCC Fifth Assessment Report on
Consistent Treatment of Uncertainties
IPCC Cross-Working Group Meeting on Consistent Treatment of Uncertainties
Jasper Ridge, CA, USA
6-7 July 2010
Core Writing Team:
Michael D. Mastrandrea, Christopher B. Field, Thomas F. Stocker,
Ottmar Edenhofer, Kristie L. Ebi, David J. Frame, Hermann Held, Elmar Kriegler,
Katharine J. Mach, Patrick R. Matschoss, Gian-Kasper Plattner, Gary W. Yohe,
and Francis W. Zwiers
The Guidance Note for Lead Authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties
is the agreed product of the IPCC Cross-Working Group Meeting on Consistent Treatment of Uncertainties.
This meeting was agreed in advance as part of the IPCC workplan.
At its 32nd session, the IPCC Panel urged the implementation of this Guidance Note.
Supporting material prepared for consideration by the Intergovernmental Panel on Climate Change.
This material has not been subjected to formal IPCC review processes.
T REAT I SSUES OF U NCERTAINTY
At an early stage, consider approaches to communicating the degree of certainty in key findings in your chapter using the calibrated language described below. Determine the areas in your chapter where a range of views may need to be described, and those where the author team may need to develop a finding representing a collective view. Agree on a moderated and balanced process for doing this in advance of confronting these issues in a specific context.
Be prepared to make expert judgments in developing key findings, and to explain those judgments by providing a traceable account: a description in the chapter text of your evaluation of the type, amount, quality, and consistency of evidence and the degree of agreement, which together form the basis for a given key finding. Such a description may include standards of evidence applied, approaches to combining or reconciling multiple lines of evidence, conditional assumptions, and explanation of critical factors. When appropriate, consider using formal elicitation methods to organize and quantify these judgments.^6
Be aware of a tendency for a group to converge on an expressed view and become overconfident in it. 7 Views and estimates can also become anchored on previous versions or values to a greater extent than is justified. One possible way to avoid this would be to ask each member of the author team to write down his or her individual assessments of the level of uncertainty before entering into a group discussion. If this is not done before group discussion, important views may be inadequately discussed and assessed ranges of uncertainty may be overly narrow. 8 Recognize when individual views are adjusting as a result of group interactions and allow adequate time for such changes in viewpoint to be reviewed.
Be aware that the way in which a statement is framed will have an effect on how it is interpreted (e.g., a 10% chance of dying is interpreted more negatively than a 90% chance of surviving). 9 Consider reciprocal statements to avoid value-laden interpretations (e.g., report chances both of dying and of surviving).
Consider that, in some cases, it may be appropriate to describe findings for which evidence and understanding are overwhelming as statements of fact without using uncertainty qualifiers.
R EVIEW THE I NFORMATION A VAILABLE
Consider all plausible sources of uncertainty. Experts tend to underestimate structural uncertainty arising from incomplete understanding of or competing conceptual frameworks for relevant systems and processes.^7 Consider previous estimates of ranges, distributions, or other measures of uncertainty, their evolution, and the extent to which they cover all plausible sources of uncertainty.
Assess issues of uncertainty and risk to the extent possible. When appropriate probabilistic information is available, consider ranges of outcomes and their associated probabilities with attention to outcomes of potential high consequence. Additional value can come from information that supports robust decisions for a wide range of climate and socio-economic futures.^10
E VALUATE AND C OMMUNICATE AT THE A PPROPRIATE L EVEL OF P RECISION
The following process and language should be applied to evaluate and communicate the degree of certainty in key findings. Paragraph 8 explains the basis of confidence in terms of level of evidence and degree of agreement. Paragraph 9 defines the confidence scale. Paragraph 10 discusses quantified measures of uncertainty. Finally, Paragraph 11 provides criteria for communication of uncertainty at different levels of precision.
- Use the following dimensions to evaluate the validity of a finding: the type, amount, quality, and consistency of evidence (summary terms: “limited,” “medium,” or “robust”), and the degree of agreement (summary terms: “low,” “medium,” or “high”). Generally, evidence is most robust when there are multiple, consistent independent lines of high-quality evidence. Provide a traceable account describing your evaluation of evidence and agreement in the text of your chapter.
- For findings with high agreement and robust evidence, present a level of confidence or a quantified measure of uncertainty.
- For findings with high agreement or robust evidence, but not both, assign confidence or quantify uncertainty when possible. Otherwise, assign the appropriate combination of summary terms for your evaluation of evidence and agreement (e.g., robust evidence, medium agreement).
ipcc guidance note
- For findings with low agreement and limited evidence, assign summary terms for your evaluation of evidence and agreement.
- In any of these cases, the degree of certainty in findings that are conditional on other findings should be evaluated and reported separately.
A level of confidence is expressed using five qualifiers: “very low,” “low,” “medium,” “high,” and “very high.” It synthesizes the author teams’ judgments about the validity of findings as determined through evaluation of evidence and agreement. Figure 1 depicts summary statements for evidence and agreement and their relationship to confidence. There is flexibility in this relationship; for a given evidence and agreement statement, different confidence levels could be assigned, but increasing levels of evidence and degrees of agreement are correlated with increasing confidence. Confidence cannot necessarily be assigned for all combinations of evidence and agreement in Figure 1 (see Paragraph 8). Presentation of findings with “low” and “very low” confidence should be reserved for areas of major concern, and the reasons for their presentation should be carefully explained. Confidence should not be interpreted probabilistically, and it is distinct from “statistical confidence.” Additionally, a finding that includes a probabilistic measure of uncertainty does not require explicit mention of the level of confidence associated with that finding if the level of confidence is “high” or “very high.”
Likelihood , as defined in Table 1, provides calibrated language for describing quantified uncertainty. It can be used to express a probabilistic estimate of the occurrence of a single event or of an outcome (e.g., a climate parameter, observed trend, or projected change lying in a given
range). Likelihood may be based on statistical or modeling analyses, elicitation of expert views, or other quantitative analyses. The categories defined in this table can be considered to have “fuzzy” boundaries. A statement that an outcome is “likely” means that the probability of this outcome can range from ≥66% (fuzzy boundaries implied) to 100% probability. This implies that all alternative outcomes are “unlikely” (0-33% probability). When there is sufficient information, it is preferable to specify the full probability distribution or a probability range (e.g., 90- 95%) without using the terms in Table 1. “About as likely as not” should not be used to express a lack of knowledge (see Paragraph 8 for that situation). Additionally, there is evidence that readers may adjust their interpretation of this likelihood language according to the magnitude of perceived potential consequences. 11
- Characterize key findings regarding a variable (e.g., a measured, simulated, or derived quantity or its change) using calibrated uncertainty language that conveys the most information to the reader, based on the criteria (A-F) below. 12 These criteria provide guidance for selecting among different alternatives for presenting uncertainty, recognizing that in all cases it is important to include a traceable account of relevant evidence and agreement in your chapter text.
A) A variable is ambiguous, or the processes determining it are poorly known or not amenable to measurement : Confidence should not be assigned; assign summary terms for evidence and agreement (see Paragraph 8). Explain the governing factors, key indicators, and
ipcc guidance note
High agreementLimited evidenceHigh agreementRobust evidenceLow agreementLimited evidenceLow agreementRobust evidence Evidence (type, amount, quality, consistency)Agreement Low agreementMedium evidenceHigh agreementMedium evidenceMedium agreementMedium evidenceMedium agreementLimited evidenceMedium agreementRobust evidence ConfidenceScale
High agreement Limited evidence
High agreement Robust evidence
Low agreement Limited evidence
Low agreement Robust evidence
Evidence (type, amount, quality, consistency)
Agreement Low agreement Medium evidence
High agreement Medium evidence
Medium agreement Medium evidence
Medium agreement Limited evidence
Medium agreement Robust evidence
Confidence Scale
Figure 1: A depiction of evidence and agreement statements and their relationship to confidence. Confidence increases towards the top-right corner as suggested by the increasing strength of shading. Generally, evidence is most robust when there are multiple, consistent independent lines of high-quality evidence.
Table 1. Likelihood Scale Term* Likelihood of the Outcome Virtually certain 99-100% probability Very likely 90-100% probability Likely 66-100% probability About as likely as not 33 to 66% probability
Unlikely 0-33% probability Very unlikely 0-10% probability Exceptionally unlikely 0-1% probability
- Additional terms that were used in limited circumstances in the AR4 (extremely likely – 95-100% probability, more likely than not – >50-100% probability, and extremely unlikely – 0-5% probability) may also be used in the AR5 when appropriate.
