In “Combining probability with qualitative degree-of-certainty metrics in assessment,” Helgeson et al. present a mathematical model of the confidence-likelihood relationship in the IPCC uncertainty framework. Their goal is to resolve ambiguities in the framework and clarify the roles of “confidence” and “likelihood” in decision-making. In this paper, I provide a conceptual evaluation of their proposal. I argue that the IPCC cannot implement the model coherently and that adopting it could result in unclear and potentially misleading communication of uncertainty.

Robustness analysis (RA) is the prescription to consider a diverse range of evidence and only regard a hypothesis as well-supported if all the evidence agrees on it. In contexts like climate science, the evidence in support of a hypothesis often comes in the form of model results. This leads to model-based RA (MBRA), whose core notion is that a hypothesis ought to be regarded as well-supported on grounds that a sufficiently diverse set of models agrees on the hypothesis. This chapter, which is the first part of a two-part review of MBRA, begins by providing a detailed statement of the general structure of MBRA. This statement will make visible the various parts of MBRA and will structure our discussion. We explicate the core concepts of independence and agreement, and we discuss what they mean in the context of climate modelling. Our statement shows that MBRA is based on three premises, which concern robust properties, common structures, and so-called robust theorems. We analyse what these involve and what problems they raise in the context of climate science. In the next chapter, which is the second part of the review, we analyse how the conclusions of MBRA can be justified.

Robustness analysis (RA) is the prescription to consider a diverse range of evidence and only regard a hypothesis as well-supported if all the evidence agrees on it. In contexts like climate science, the evidence in support of a hypothesis often comes from scientific models. This leads to model-based RA (MBRA), whose core notion is that a hypothesis ought to be regarded as well-supported on grounds that a sufficiently diverse set of models agrees on the hypothesis. This chapter, which is the second part of a two-part review of MBRA, addresses the thorny issue of justifying the inferential steps taking us from the premises to the conclusions. We begin by making explicit what exactly the problem is. We then turn to a discussion of two broad families of justificatory strategies, namely top-down and bottom-up justifications. In the latter group we distinguish between the likelihood approach, independence approaches, and the explanatory approach. This discussion leads us to the sober conclusion that multi-model situations raise issues that are not yet fully understood and that the methods and approaches that MBRA has not yet reached a stage of maturity. Important questions remain open, and these will have to be addressed in future research.

I critically assess an argument put forward by Kuorikoski et al. (2010) for the epistemic import of model-based robustness analysis. I show that this argument is not sound since the sort of probabilistic independence on which it relies is unfeasible. By revising the notion of probabilistic independence imposed on the models' results, I introduce a prima-facie more plausible argument. However, despite this prima-facie plausibility, I show that even this new argument is unsound in most if not all cases of model-based robustness analysis. This I do to demonstrate that the epistemic import of model-based robust analysis cannot be satisfactorily defended on the basis of probabilistic independence.

This paper has two related objectives. The first is to offer a thorough diagnosis of some conceptual problems in the IPCC uncertainty framework. I believe that any successful attempt to revise and improve the framework will have to start from a clear understanding of the current conceptual problems, their implications for the IPCC authors’ treatment of uncertainties, and the quality of the information provided in IPCC reports. Accordingly, the first objective of this paper is to contribute to this first step. I begin by showing that there is no interpretation of ‘confidence’ and ‘likelihood’ that is compatible with the IPCC uncertainty guide’s recommendations - and thus with the resulting practice of the IPCC authors in their communication of uncertainty. I argue that the lack of a conceptually valid interpretation of ‘likelihood’ and ‘confidence’ has worrying implications for both the practice of the IPCC authors in their treatment of uncertainties and the quality of the information provided in IPCC reports. Finally, I show that an understanding of the reasons behind the decision to include two uncertainty scales in the IPCC uncertainty framework can give us some interesting insights into the nature of this problem. Its second objective is to critically reflect on what an adequate IPCC uncertainty framework could look like. I critically assess two strikingly different proposals for a new IPCC uncertainty framework (Winsberg, 2018; Bradley et al., 2017) and I argue that in both proposals the interpretation of ‘confidence’ is conceptually problematic. Finally, I offer my own tentative proposal that meets what I take to be two basic desiderata for an adequate uncertainty framework: conceptual clarity and decision relevancy.

In scientific inquiry, a “robust” result is often viewed as supporting a hypothesis, but the Bayesian logic behind this requires careful examination. Schupbach’s (2018) explanatory account of robustness analysis (RA) provides valuable insights, particularly for empirically driven cases of RA. However, this paper challenges the claim that this account equally applies to model-based RAs. I argue that applying this account to model-based RAs is significantly more complex than recognized by Schupbach and others and that it fails to apply to a significantly broader range of model-based RAs than previously assumed.

Debates on the role of values in science, particularly in relation to inductive risk, have remained largely orthogonal to disputes over the legitimate use of probability in inference. This separation is surprising, as both concern the conditions under which scientific reasoning produces reliable and epistemically justified conclusions. In this paper, I argue that although discussions on values have been invaluable in highlighting the role of non-epistemic considerations in scientific practice, they have often overshadowed an equally pressing issue: the uncritical acceptance of probabilism as the default framework for representing uncertainty. This oversight has shaped philosophical debates on inference in ways that obscure important methodological concerns. A more critical engagement with probabilistic reasoning is essential to upholding the epistemic rigor and integrity of scientific reasoning.

The aim of this thesis is to improve our understanding of how to assess and communicate uncertainty in areas of research deeply afflicted by it, the assessment and communication of which are made more fraught still by the studies’ immediate policy implications. The IPCC is my case study throughout the thesis, which consists of three parts. In Part 1, I offer a thorough diagnosis of conceptual problems faced by the IPCC uncertainty framework. The main problem I discuss is the persistent ambiguity surrounding the concepts of ‘confidence’ and ‘likelihood’; I argue that the lack of a conceptually valid interpretation of these concepts compatible with the IPCC uncertainty guide’s recommendations has worrying implications for both the IPCC authors’ treatment of uncertainties and the interpretability of the information provided in the AR5. Finally, I show that an understanding of the reasons behind the IPCC’s decision to include two uncertainty scales can offer insights into the nature of this problem. In Part 2, I review what philosophers have said about model-based robustness analysis. I assess several arguments that have been offered for its epistemic import and relate this discussion to the context of climate model ensembles. I also discuss various measures of independence in the climate literature, and assess the extent to which these measures can help evaluate the epistemic import of model robustness. In Part 3, I explore the notion of the ‘weight of evidence’ typically associated with Keynes. I argue that the Bayesian (or anyone who believes the role of probability in inductive inference is to quantify the degree of belief to assign to a hypothesis given the evidence) is bound to struggle with this notion, and draw some lessons from this fact. Finally, I critically assess some recent proposals for a new IPCC uncertainty framework that significantly depart from the current one.