1

Use of agro-climate ensembles for quantifying uncertainty and informing adaptation

2
3

Andrew Challinor1,2, Mark Stafford Smith3, Philip Thornton4,2

4
5
6
7

1

8
9

2

10
11

3

12
13

4

Corresponding author. Institute for Climate and Atmospheric Science, School of Earth and
Environment, The University of Leeds, LS2 9JT, United Kingdom. a.j.challinor@leeds.ac.uk Tel. +44
(0)113 3433194
CGIAR-ESSP Program on Climate Change, Agriculture and Food Security, International Centre for
Tropical Agriculture (CIAT), A.A. 6713, Cali, Colombia.
CSIRO Climate Adaptation Flagship, GPO Box 1700, Canberra ACT 2601, Australia.
Mark.Staffordsmith@csiro.au
International Livestock Research Institute (ILRI), PO Box 30709, Nairobi 00100, Kenya.
P.Thornton@CGIAR.ORG

14
15

Abstract

16
17
18
19
20
21
22
23
24
25
26
27
28

Significant progress has been made in the use of ensemble agricultural and climate modelling, and
observed data, to project future productivity and to develop adaptation options. An increasing
number of agricultural models are designed specifically for use with climate ensembles, and
improved methods to quantify uncertainty in both climate and agriculture have been developed.
Whilst crop-climate relationships are still the most common agricultural study of this sort, on-farm
management, hydrology, pests, diseases and livestock are now also examined. This paper introduces
all of these areas of progress, with more detail being found in the subsequent papers in the special
issue. Remaining scientific challenges are discussed, and a distinction is developed between
projection- and utility- based approaches to agro-climate ensemble modelling. Recommendations
are made regarding the manner in which uncertainty is analysed and reported, and the way in which
models and data are used to make inferences regarding the future. A key underlying principle is the
use of models as tools from which information is extracted, rather than as competing attempts to
represent reality.

29
30
31
32
33
34

Keywords: Climate models, Crop models, Ensembles, Climate change, Adaptation, Food security,
Climate variability, Uncertainty, Crop yield

35

1. Introduction

36
37
38
39
40
41
42

The use of climate ensembles with agricultural models, particularly crop models, is an increasingly
common method for projecting the potential impacts of climate change (see e.g. reviews by
Challinor et al., 2009a,b). These developments are timely, given the significant societal interest in
both the implications of climate change and the uncertainty surrounding predictions. Ongoing
increases in greenhouse gas emissions will continue to alter climate for some decades. Climate and
impacts ensembles provide a tool for predicting the implications of these changes and for
developing adaptation options.

43
44
45
46
47
48
49
50
51
52
53

This special issue demonstrates the maturity of this field by highlighting recent progress in
methodologies for the design and use of ensembles and in the agricultural modelling that is used in
such studies. The word ensemble is used here to indicate any multiple model simulations that seek
to quantify uncertainty. This includes both ensembles that quantify parametric uncertainty using one
model and ensembles that quantify structural uncertainty by using a number of models. Ensemble
agricultural and climate modelling, or more briefly agro-climate ensemble modelling, refers here to a
set of directly comparable agricultural simulations generated using one or more climate projections
with one or more agricultural models in one or more configurations. The direct comparability of the
simulations makes the ensemble a tool for quantifying and exploring uncertainty. An ensemble crop
simulation, for example, seeks to quantify uncertainty due to some or all of: climate, crop response
to climate, and other determinants of crop productivity.

54
55
56
57
58
59
60
61
62

The papers in the special issue reflect the growing breadth of topics that are being assessed using
ensemble techniques. They also suggest a parallel with the development of ensemble methods
within climate change science itself, whereby a “new era” in prediction was identified as a result of
the increasing use of ensembles (Collins and Knight, 2007). The increase in the use of ensemble
techniques in agriculture has been largely enabled by this development in climate science. The
influence of climate science is evident from the common use of multiple climate realisations in agroclimate ensembles, compared to the far rarer use of multiple crop models. Thus agro-climate
ensembles are often the result of the use of an agricultural model as a tool for interpreting climate
ensembles in an agriculturally relevant way.

63
64
65
66
67
68
69
70
71
72
73

The generation of robust projections of agricultural production requires adequate account of
uncertainty in future atmospheric composition and climate, the subsequent response of agricultural
systems, and the range of non-climatic drivers that affect agriculture. Only in this way can
appropriate adaptation and mitigation actions be determined. The question of how much account
of uncertainty is adequate for any specific adaptation and mitigation action is not trivial. This
important question is discussed briefly in section 3.2, but falls largely outside the scope of this
special issue. Our starting point here is the recognition that, in an effort to ensure that treatments of
uncertainty are at least adequate, the climate impacts community is putting increasing efforts into
improving the methods used to assess impacts and adaptation, and understanding the associated
uncertainties. This includes assessing, intercomparing and improving tools and methodologies (see
Rosenzweig et al. 2012) and asking: what do our models tell us about the real world?

74
75
76

The choices in climate impacts modelling regarding model complexity, ensemble size and spatial
resolution, whether made explicitly or resulting from the inherent trade off forced by limited
computer power, affect the way in which the model results need to be interpreted (Challinor et al.,

77
78
79
80
81
82
83
84
85
86
87

2009a). Computing power limits the potential for studies to employ complex models over a large
spatial domain and systematically sample uncertainty, so that modelling work tends to focus on one,
or maybe two, of these three characteristics. The agricultural simulation studies in this special issue
demonstrate this trade off: they vary in their sampling of uncertainty and can broadly be divided into
those that have relatively high spatial resolution (Ewert et al. 2012, Gouache et al. 2012, Graux et al.
2012, Robertson et al. 2012, Teixeira et al. 2012, Ramirez et al. 2012, Kroschel et al. 2012) and those
that use relatively complex models and/or simulate a number of different agricultural processes and
practices (Ruane et al. 2012, Tao et al. 2012, Hemming et al. 2012, Osborne et al. 2012, Fraser et al.
2012, Berg et al. 2012). The studies also reflect the increasing ability to simulate agricultural
responses across large or multiple regions, including global assessment (Berg et al. 2012, Fraser et al.
2012, Hemming et al. 2012, Kroschel et al. 2012, Osborne et al. 2012, Ramirez et al. 2012).

88
89
90
91
92
93
94
95

Due to the focus on the use of climate ensembles, either to achieve large geographical coverage, or
to capture uncertainty through the use of many ensemble members, relatively few studies here
employ downscaling techniques (Gouache et al. 2012, Graux et al. 2012, Hoglind et al. 2012,
Ramirez et al. 2012, Kroschel et al. 2012). Efforts to produce coordinated ensembles of regional
climate model simulations (e.g. ENSEMBLES, COREDEX) are likely to lead to an increasing potential to
sample uncertainty at higher spatial resolution. Downscaling is not covered explicitly in this
introductory paper, except to note that two studies in this special issue (Hawkins et al. 2012, Hoglind
et al. 2012) are relevant to weather generation.

96
97
98

Every approach to climate impacts assessment has its pros and cons. In the development of each
approach, a number of questions are addressed, either implicitly or explicitly. The following list is
drawn in part from a workshop on climate impacts held in April 20101:

99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117

1. What is the appropriate degree of complexity for simulation? This is relevant both to the
biophysical model (section 2.1) and in considering the influence of, and interactions
between, the range of other drivers of agricultural productivity, such as pests and diseases
and management practices (section 2.2.2.).
2. What are appropriate methodologies for quantifying and representing uncertainty (section
2.2.1)? There are an increasing number of sets of climate ensembles produced from a range
of research programmes. How are impacts modellers and, more broadly, users of climate
information to choose between these? Which uncertainties in climate and its impacts
dominate under which circumstances? Given that complete sampling of uncertainty using
ensembles is not possible, can objective probabilities be determined? How should
uncertainty in agricultural models be represented and evaluated?
3. How should uncertainty be presented and communicated? How do these choices affect the
methods used to quantify uncertainty? These questions have implications for the design and
use of ensembles (section 3.2).
In addition to introducing and framing the special issue, this opening paper seeks to identify
methodologies for making effective use of agro-climate ensembles. Thus, the summary of progress
in section 2 is used as a basis for a discussion of knowledge gaps (section 3.1) and some brief
reflections on the utility of agro-climate ensembles (section 3.2). Conclusions are presented in
section 4. Throughout the manuscript, the word uncertainty, where used without further
1

See the report on the EQUIP user meeting at http://www.equip.leeds.ac.uk/user-workshop-3-269.html

118
119
120

qualification, is used to denote a lack of predictive precision due to either inherent limitations to
predictability (e.g. due to unknown future greenhouse gas emissions) or to a lack of predictive skill
(e.g. errors in the design of a model).

121
122

2. Progress in agro-climate modelling

123
124
125
126
127

Here we highlight progress in the models used for agricultural impacts assessment (section 2.1)
and improvements in the methodological design of studies that use those models, both in terms
of the quantification of uncertainty (section 2.2.1) and the use of modelling studies to inform
adaptation, which necessarily implies simulating crop yield but also a range of other quantities
and processes (section 2.2.2).

128
129

2.1 Agricultural models designed for use with climate ensembles

130
131
132
133
134
135
136
137
138
139
140

Judicious choices of both agricultural model and the technique used for calibration are crucial for the
development of robust conclusions regarding the impacts of climate change. Implicit in this choice is
a judgement on the appropriate degree of complexity for simulating biophysical and agricultural
processes. Insufficient complexity, by definition, renders a model incapable of simulating the
processes that result in observed quantities. Excess complexity in a model results in sufficient
degrees of freedom to reproduce observations, but this will often require parameter values that
cannot be adequately constrained – thus increasing the chances of getting the right answer for the
wrong reason (Challinor et al., 2009b). In practice, use of a range of approaches, with associated
recognition of the pros and cons implicit in the assumptions made, is a way of assessing the
robustness of results. This observation has been developed and labelled in a number of research
fields and in a number of ways, e.g. equifinality (Beven, 2006) and consilience (Wilson, 1998).

141
142
143
144
145
146
147
148
149
150
151
152
153

The use of a range of approaches within agricultural modelling is perhaps most evident with crops,
as is indicated by the papers in this special issue, which range from detailed process based models
(e.g. Ruane et al. 2012) to empirical models (Lobell 2012) and diverse models of intermediate
complexity (e.g. Ramirez et al 2012, Osborne et al 2012, Watson et al 2012). Model complexity is
inherently linked to the spatial scales at which crop responses are being simulated (for a full
discussion, see e.g. Challinor et al., 2009a,b). Ramirez et al (2012) integrate the FAO-EcoCrop
database with a basic mechanistic model that uses environmental ranges as inputs to determine the
main niche of a crop and then produces a suitability index as output. Ruane et al. (2012) investigate
the ability of empirical models of crop yield to reproduce the results from more complex processbased crop model simulations and infer pros and cons of each approach. The range of models now
available is increasingly enabling spatially explicit global assessments of the actual (Osborne et al.
2012) and potential (Berg et al. 2012) productivity of crops and the impact of specific processes such
as heat stress (Teixera et al.2012).

154
155
156
157

The studies collected here also demonstrate the relatively recent increase in the use of non-crop
simulation models for climate impacts studies. The simulations of Hoglind et al. (2012) indicate
increased grass yields into the future, mainly due to increased temperatures; Graux et al. (2012) find
new opportunities for herbage production in spring and winter, although future conditions show

158
159

increased interannual variability in production. Section 2.2.2 highlights progress in other non-crop
simulations, for example socio-economic processes and pests and diseases.

160
161

2.2 Improvements in the design of agro-climate ensembles

162
163

2.2.1 Improved quantification of uncertainty

164
165
166
167
168
169
170
171
172
173
174

The papers in this special issue present advances in both the methods used to assess uncertainty and
the knowledge resulting from agro-climate ensembles. Methodological improvements address the
inability to associate occurrence of events across an ensemble with the probability of those events
occurring. More broadly, methodologies are required that enable the calibration and evaluation of
ensemble prediction systems in order to better constrain ensemble outputs. Tao et al. (2012)
applied Bayesian probability inversion and a Markov chain Monte Carlo (MCMC) technique to a
large-scale crop model in order to attempt to make probabilistic predictions. This study, which
focuses on the use of statistical tools to constrain ensembles, contrast with approaches that focus on
specific processes such as heat and/or water stress (e.g. Teixida et al. 2012, Challinor et al. 2010),
sometimes constraining ensembles using relatively simple techniques (e.g. Challinor and Wheeler,
2008a).

175
176

New knowledge on sources of uncertainty contained in this special issue can be divided into two
categories:

177
178
179
180
181
182
183
184
185
186

(i) Uncertainty in specific processes such as CO2 fertilisation and pest occurrence. Gouache et
al. (2012) simulate the occurrence of Septoria tritici blotch on winter wheat and find that the
contribution of the disease model to total uncertainty was greater than that of the climate
model. Ruane et al. (2012) used the positive and monotonic relationship between CERESMaize yield and carbon dioxide concentrations as a metric for the uncertainty associated with
CO2 fertilisation and found this uncertainty to be significant (10 to 20%). This issue may be
addressed by constraining the response of crops to increased CO2 using observations
(Challinor et al., 2009c). However, interactions between water stress and CO2 can add
significantly to the uncertainty in the response of crops to changes in CO2 (Challinor and
Wheeler, 2008a).

187
188
189
190
191
192
193
194

Model simulations with fully coupled vegetation and climate also provide evidence of the
magnitude of the CO2 fertilisation effect. Hemming et al. (2012) examine both direct and
indirect plant physiological responses to CO2 using such a model. The direct effects of
elevated CO2 account for a 75% increase in net primary productivity (NPP), whilst indirect
effects (i.e. the sum of effects mediated through the associated change in climate) account for
a 21% decrease in the ensemble average. The extent to which results for NPP can be directly
compared to results from calibrated and/or constrained crop model simulations is not yet
clear.

195
196

(ii) Assessments of the impact of uncertainty in agricultural model inputs, including climate
model data. It is clear from the analysis above, and from a broader reading of the studies

197
198
199
200
201
202
203
204
205
206
207
208
209
210

presented here, that the uncertainty resulting from simulation of a climate impact (such as
crop yield or disease occurrence), and the fraction that this contributes to total uncertainty,
varies across studies. Studies using crop and climate models have suggested that uncertainty
in climate is a significant, if not dominant, contribution to total projected uncertainty (e.g.
Challinor et al., 2009c). The broader issue of error in the inputs to climate impact models is
therefore an important one. Lobell (2012) finds, using an empirical crop model, that studies
that ignore measurement errors are unlikely to be biased for estimating the temperature
sensitivity of yields, but can easily underestimate sensitivity to rainfall by a factor of two or
more. Watson et al. (2012) examine the impact of error in rainfall, temperature and yield data
(used for calibration) on process-based crop model, by randomising and perturbing observed
data. For their study case, errors generated by randomising the temporal sequence of
seasonal total precipitation produced an error in simulated yield of approximately three times
that of temperature or yield. However, perturbing input data to values beyond those found in
the current climate increased all yield errors significantly and to comparable values.

211
212
213
214
215
216

The above studies all focus on the importance of input data from the perspective of
agricultural models themselves. An important exception is the study of Craufurd et al. (2012),
which highlights the role of crop science experiments in providing high quality data to inform
crop modelling. In particular, the authors note that the diversity of genotypic responses is not
well represented by existing crop science experiments, since responses have only been
quantified for a limited number of genotypes.

217
218
219
220
221
222
223
224

The importance of weather and climate inputs in determining the predictive skill of
agricultural models implies that appropriate effort should be made to ensure that these inputs
are as accurate as possible (without introducing false confidence through unwarranted
precision). After reviewing the methods available for post-processing climate model output,
Hawkins et al. (2012) employ these methods using a ‘perfect sibling’ framework, which is
similar to the perfect model approach, and find significant variation in results. Whilst that
study does not employ a weather generator, the results are relevant for the on-going
development of weather generators.

225
226

2.2.2. Going beyond biophysical crop yield impacts

227
228
229
230
231
232
233
234
235
236
237

Much of the progress in agricultural modelling using ensembles has occurred with crop models.
However, in order to inform adaptation, information is needed not just on likely future crop yields as
influenced by biophysical processes, but also on the influence of a broader range of processes. Many
of the studies discussed in section 2.1, and those presented elsewhere in this special issue, address
adaptation in some way. These studies aim for a more complete description of the system through
accounting for socio-economic drivers of productivity (Fraser et al. 2012), on-farm management
such as choice of crop variety or planting date (Osborne et al. 2012; Ruane et al. 2012), or the
impact of pests and diseases (Garrett et al. 2012; Kroshel et al. 2012; Gouache et al. 2012). For
example, Fraser et al. (2012) use socio-economic data to model adaptive capacity and hydrological
data to model exposure to drought, without the use of a crop model (though such work has been
combined with biophysical models: Challinor et al., 2010). Garrett et al. (2012) provide a framework

238
239

for integrating models of livestock, crops, pests and disease, whilst Kroschel et al. (2012) present a
specific tool for adaptation planning in the integrated management of potato tuber moth.

240
241
242
243
244
245
246
247
248

As the use of ensembles is extended to increasingly complete descriptions of agro-climatic processes
(including biotic stresses and human actions), the complexity of the associated models and/or model
chains will increase. Since the number of interactions between physical, agricultural and biological
systems increases as the number of processes simulated increases, the uncertainty in the
interactions will likely result in greater total uncertainty. Thus additional complexity brings with it
demands for increased ensemble size in order to adequately sample uncertainty. If such models and
model chains are carefully calibrated and have appropriate complexity then we may expect to see
increasingly accurate representations of agro-climatic processes that in turn can be used to inform
adaptation.

249
250
251
252

3. Discussion

253

3.1 Remaining science questions and challenges

254
255
256
257
258
259

If projections based on agro-climate ensembles are to be robust, then a number of questions remain
to be answered. Crop modelling relies on measurements for development, calibration and
evaluation. How can field experiments, such as those that assess crop phenotypes, be best targeted
towards modelling? Without addressing this question and others like it, agricultural models will at
best make sub-optimal use of environmental data, and at worst they will be relied upon in lieu of
that data, thus likely misleading adaptation efforts.

260
261
262
263
264
265
266
267
268

A second challenge is to better understand the relationship between model complexity, measured
uncertainty and actual uncertainty, and the manner in which this varies across spatial scales.
Repeated projections for the near future, such as seasonal forecasts of crop yield, produce
uncertainty ranges that are verifiable using standard techniques (e.g. Challinor et al., 2005). No such
techniques can exist for projections of changes in the mean and variability of agricultural
productivity on longer timescales, since there will be only one evolution of climate. Where climate
change predictions are repeated many times, e.g. for multiple locations, ranges can be verified; but
the extent to which these ranges can be compared to assessments of structural and parametric
uncertainty is not clear.

269
270
271
272
273
274
275
276

The move from emissions scenarios to Representative Concentration Pathways (van Vuuren et al.,
2011) facilitates improved understanding of the consequences of uncertainty for prediction: by
separating the uncertainty in future greenhouse gas emissions from uncertainty in the subsequent
response of the climate system, the new framework has the potential to identify the component of
future climate change that we can control. However, it is not yet clear whether or not this change
will lead to more robust projections. Bayesian theory demonstrates that prior assumptions, whether
made implicitly or explicitly, affect uncertainty estimates. Whilst some authors (e.g. Berger 2006)
maintain that this does not preclude objective quantification of uncertainty, other authors question

277
278
279
280
281
282
283

the potential for objective uncertainty assessment, both within ( O’Hagan, 2006) and beyond (Yohe
and Oppenheimer, 2011) the Bayesian framework. Given this conceptual difficulty, and given that
attempts to quantify uncertainty in agro-climate modelling can lead to very large ranges, and that
ranges that can rarely be inter-compared (Challinor et al., 2007), it may be that new frameworks for
quantifying and managing uncertainty are needed (sections 3.2 and 4). Studies that aim to compare
and improve agricultural models, notably AgMIP (Rosenzweig et al., 2012), should do so in a manner
that permits direct inter-comparison.

284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299

Uncertainty in projections can be reduced by detailed examination of processes (see section 3.2)
and/or by using observations to constrain simulations (e.g. Watson et al. 2012). Observational data
for calibration and evaluation are critical to both of these methods of reducing uncertainty. For
example, the yield simulations of Ewert et al. (2012) where the crop model is calibrated for
individual regions using phenology and growth parameters are more skilful than those without this
calibration, leading the authors to argue for region-specific calibration of crop models when
conducting pan-European assessments. Similarly, the bivariate yield emulator tested by Ruane et al.
(2012) for maize in Panama underestimated the potential yield impacts of extreme seasons and
revealed errors due to the omission of additional crucial metrics including the number of rainy days
and the standard deviation of temperatures. Thus, at least in some cases bivariate yield emulators
are not sufficient for the prediction of yield in current or future climates. This work demonstrates
the need for sufficient complexity in the development and calibration of agricultural models.
Similarly, Watson et al. (2012) demonstrate the importance of yield data for the calibration of
regional-scale models. Crop experiments relevant to future climates are also important (Craufurd et
al. 2012), for example in evaluating the performance of crop varieties under climate change and in
assessing crop response to elevated CO2.

300
301

3.2 Effective use of agro-climate ensembles

302
303
304
305
306
307
308
309
310
311

The issues outlined in section 3.1 regarding data, model complexity, and simulated and actual
uncertainty, make it clear that validated, definitive probabilistic ensembles of impacts are difficult, if
not impossible, to produce. This implies the need for significant thought in the way that uncertainty
and prediction are framed. It also implies a need to recognise that different models may be needed
for different parts of the decision cycle. Depending on the aims of any given study, one of two
approaches is usually taken to developing agro-climate ensembles. Projection-based approaches use
models and data to increase understanding and view decision-makers as end users. Utility-based
approaches focus on the decisions that need to be made, rather than projections of impacts. For a
broader discussion of these two approaches to managing uncertainty in climate and its impacts, see
Mearns et al. (2010) or Dessai et al. (2007).

312
313
314
315
316
317

Projection-based approaches map out the cascade of uncertainty from climate through to impact.
Their success may be contingent on a degree of consilience (see section 2.2.1), which is something
that the research process is apt at achieving, albeit at a speed limited by the publication cycle. Model
inter-comparisons and combinations (Rosenzweig et al. 2012) – including the synthesis of
information from process-based and statistical approaches – are likely to be particularly useful
techniques for achieving consilience. Since attempts to combine both climatic and socio-economic

318
319

drivers of agriculture (e.g. Challinor et al., 2010) are relatively few in number, it is not yet clear
whether or not consilience can be achieved across the biophysical and socio-economic domains.

320
321
322
323
324
325
326
327
328

Projection-based approaches are particularly well-suited to research and this is perhaps the
approach most commonly found in the literature. Over time, new knowledge about agro-climatic
systems is generated and this knowledge can then be used wherever and however the opportunity
arises. Projections with well-bounded and uncertainty ranges are more likely to be useful in this
context than those with wide ranges. Robust outcomes may emerge by focussing on underlying
processes. For example, Ruane et al. found that avoided water stress from rapid maturity offsets the
effect of temperature increases. Thornton et al. (2009) found that maize and bean yields in the
drylands of East Africa responded in a similar fashion to climate change under both increased or
decreased rainfall, due to the relationship between temperature and rainfall.

329
330
331
332
333
334
335
336
337
338
339
340

Utility-based approaches hypothesise that taking into account how information is used can improve
its utility. Thus research design is informed by the decision-making process, for example the chain of
decisions around investment in new crop varieties. Since decisions naturally involve social and
economic systems, utility-based approaches usually involve the social sciences (Raymond et al.,
2010; Twyman et al., 2011). The specific nature of the decisions examined in a utility-based
approach may make it difficult to generalise the results from different studies. However, the
embedding of information and learning within decision-making processes can provide an alternative
framework within which to seek consilience: synthesising sources of information in to a decision
may, in spite of some individually weak elements, enable a decision that is more robust, due to other
elements being stronger in the full decision context. For example, Ash et al. (2007) and McIntosh et
al. (2005) found that an integrated plant growth index was both more predictable and more relevant
to farm decision-making than the rainfall and temperature data on which that index depends.

341
342
343
344
345
346
347
348
349

Whether a projection or utility based approach is used in any given study will depend on a range of
factors. The nature of the specific agro-climatic system studied, and the ability (skill) of the tools
developed to reproduce the properties of this system, may in part determine the likely success of a
utility-based approach. Model skill in turn is underpinned by the development of models for
understanding and for prediction. As agro-climatic ensembles are developed and applied to a range
of systems, the skill and utility of these tools needs to be carefully assessed. Promising areas for
future work include the use of household models of agricultural activity as part of ensemble
systems, in order to assess the impact of human responses to climate change at the local scale; and
ensembles of integrated assessment tools and economic models (Rosenzweig et al., 2012).

350
351

4. Conclusions

352
353

In addition to providing an introduction to this special issue, some recommendations for research
may be drawn from the analysis above.

354
355
356
357

1. Analysis of processes as a tool for navigating uncertainty. The use of models as black
boxes, with the associated focus on model outputs, places a significant burden on the model
to correctly reproduce the interactions between processes. The examination of processes
across a series of models can identify research gaps in both modelling and field data

358
359
360
361
362

(Challinor and Wheeler, 2008b). Such analyses are not routinely applied; indeed, it is often
unclear which processes have been simulated within a given study (White et al., 2011).
Model intercomparison projects – notably AgMIP (Rosenzweig et al. 2012) – provide
opportunities to clearly document which processes are simulated and synthesise the results
of numerous models.

363
364
365
366
367
368

2. Explicit reporting on sources of uncertainty. When seeking either to improve
understanding or to produce decision-relevant information, it is important to distinguish the
sources of uncertainty. For example, climate change can be affected by policies to alter
greenhouse gas emissions, but there is no political control over the response of the climate
system to any given greenhouse gas forcing. Thus uncertainty in these two contributions to
climate change has different implications for decision making.

369
370
371
372
373
374

3. Strategies for combining diverse models and datasets. Agro-climate ensemble modelling
rarely uses ensembles of agricultural models. Techniques for using multiple agricultural
models could be targeted at projection- or utility- based approaches. In the latter case,
different models may be needed for different parts of the decision cycle. In either case,
there is likely to be a role for the development of field experiments that are targeted
towards modelling, such as those that assess crop phenotypes.

375
376
377
378
379
380
381
382
383
384
385

Underpinning all three of these recomendations is a methodology that treats models (and also data)
as tools from which information is extracted, rather than as competing attempts to represent reality.
This methodology could be used to improve understanding of the role of complexity, utility, spatial
scale and uncertainty in agricultural prediction and adaptation. For example: how can net primary
productivity from climate models (as analysed by Hemming et al. 2012) be used as part of crop yield
assessments?; what are the relationships between model complexity, measured uncertainty and
actual uncertainty, and how do these vary across spatial scale?; and can utility-based and projectionbased approaches to agricultural prediction be combined by explicitly simulating the decision making
process in projection-based agro-climate modelling (e.g. Garrett at al. 2012)? One approach to this
final question is to develop methods for combining analysis of uncertainty from projections with an
assessment of the accuracy needed for a specific decision.

386
387

Acknowledgements

388
389
390
391

The authors gratefully acknowledge support from NERC, though the EQUIP project, and the CGIAR,
through their programme on Climate Change, Agriculture and Food Security. AJC thanks Tom Fricker
for sharing his knowledge of Bayesian literature. The authors are grateful to two anonymous
reviewers for their particularly insightful comments.

392
393

References

394

All 2012 papers refer to other papers in the special issue

395
396
397

Ash A E, Peter McIntosh B, Brendan Cullen A, Peter Carberry C and Mark Stafford Smith D (2007).
Constraints and opportunities in applying seasonal climate forecasts in agriculture. Australian
Journal of Agricultural Research 58, 952–965.

398

Berger, J., 2006. The case for objective Bayesian analysis. Bayesian Analysis 1 (3), pp 385—402.

399

Beven, K. (2006). A manifesto for the equifinallity thesis. Journal of Hydrology 320, 18-36.

400
401
402

Challinor, A. J., E. S. Simelton, E. D. G. Fraser, D. Hemming and M. Collins(2010) Increased crop
failure due to climate change: assessing adaptation options using models and socio-economic data
for wheat in China. Environ. Res. Lett. 5 (2010) 034012

403
404
405

Challinor, A. J., T. Osborne, A. Morse, L. Shaffrey, T. Wheeler, H. Weller (2009a). Methods and
resources for climate impacts research: achieving synergy. Bulletin of the American Meteorological
Society, 90 (6), 825-835

406
407
408

Challinor, A. J., F. Ewert, S. Arnold, E. Simelton and E. Fraser (2009b). Crops and climate change:
progress, trends, and challenges in simulating impacts and informing adaptation. Journal of
Experimental Botany 60 (10), 2775-2789. doi: 10.1093/jxb/erp062

409
410
411

Challinor, A. J., T. R. Wheeler, D. Hemming and H. D. Upadhyaya (2009c). Ensemble yield simulations:
crop and climate uncertainties, sensitivity to temperature and genotypic adaptation to climate
change. Climate Research, 38 117-127

412
413

Challinor, A. J. and T. R. Wheeler (2008a). Use of a crop model ensemble to quantify CO2 stimulation
of water-stressed and well-watered crops. Agric. For. Meteorol, 148 1062-1077

414
415

Challinor, A. J. and T. R. Wheeler (2008b). Crop yield reduction in the tropics under climate change:
processes and uncertainties. Agric. For. Meteorol, 148 343-356.

416
417
418

Challinor, A. J., T. R. Wheeler, P. Q. Craufurd, C. A. T. Ferro and D. B. Stephenson (2007). Adaptation
of crops to climate change through genotypic responses to mean and extreme temperatures.
Agriculture, Ecosystems and Environment, 119 (1-2) 190-204

419
420

Challinor, A. J., J. M. Slingo, T. R. Wheeler and F. J. Doblas-Reyes (2005). Probabilistic hindcasts of
crop yield over western India. Tellus 57A 498-512

421
422

Collins, M. and Knight, S., Eds. (2007). Ensembles and probabilities: a new era in the prediction of
climate change. Papers of Theme Issue, Phil. Trans. Roy. Soc. A 365 (1857), 1955-2191.

423
424

Dessai , S., M. Hulme, R. Lempert and R. Pielke, Jr. (2009) Do we need more precise and accurate
predictions in order to adapt to a changing climate? Eos, 90(13), 111-112.

425
426
427
428

van Vuuren, D. P. , Jae Edmonds, Mikiko Kainuma, Keywan Riahi, Allison Thomson, Kathy Hibbard,
George C. Hurtt, Tom Kram, Volker Krey and Jean-Francois Lamarque, Toshihiko Masui, Malte
Meinshausen, Nebojsa Nakicenovic, Steven J. Smith and Steven K. Rose (2011). The representative
concentration pathways: an overview. Climatic Change DOI 10.1007/s10584-011-0148-z.

429

Mearns et al., 2010. The Drama of uncertainty. Climatic Change (2010) 100:77–85.

430
431

O’Hagan, A., 2006. Science, subjectivity and software (comment on articles by Berger and by
Goldstein. Bayesian Analysis 1 (3), pp, 445—450.

432
433
434

Raymond, CM; Fazey, I; Reed, MS; Stringer, L; Robinson, GM; Evely, AC (2010) Integrating local and
scientific knowledge for environmental management, Journal of Environmental Management, 91,
pp.1766-1777. doi:10.1016/j.jenvman.2010.03.023

435
436
437

McIntosh, Peter C., Andrew J. Ash, Mark Stafford Smith, 2005: From Oceans to Farms: The Value of a
Novel Statistical Climate Forecast for Agricultural Management. J. Climate, 18, 4287–4302. doi:
10.1175/JCLI3515.1

438

Thornton, P. K. et al. Global Environmental Change 19 (1), 54-65 (2009).

439

Wilson, E.O., 1998. Consilience: The Unity of Knowledge. Knopf, New York.

440
441
442

Twyman, C; Fraser, E; Stringer, LC; Quinn, C; Dougill, AJ; Ravera, F; Sallu, SM (2011) Closing the Loop:
Climate Science, Development Practice and Policy Interactions in Dryland Agro-Ecological Systems ,
Ecology and Society, 16,

443
444
445

Yohe, G. and M. Oppenheimer (2011). Evaluation, characterization, and communication of
uncertainty by the intergovernmental panel on climate change—an introductory essay. Climatic
Change (2011) 108:629–639

446