ARTICLE
https://doi.org/10.1038/s42003-021-02463-w OPEN
Data-driven decentralized breeding increases
prediction accuracy in a challenging crop
production environment
Kauê de Sousa 1,2, Jacob van Etten2, Jesse Poland 3, Carlo Fadda 4, Jean-Luc Jannink 5,6,
Yosef Gebrehawaryat Kidane 4,7, Basazen Fantahun Lakew4,8, Dejene Kassahun Mengistu4,7,
Mario Enrico Pè7, Svein Øivind Solberg1 & Matteo Dell’Acqua 7✉
Crop breeding must embrace the broad diversity of smallholder agricultural systems to
ensure food security to the hundreds of millions of people living in challenging production
environments. This need can be addressed by combining genomics, farmers’ knowledge, and
environmental analysis into a data-driven decentralized approach (3D-breeding). We tested
this idea as a proof-of-concept by comparing a durum wheat (Triticum durum Desf.)
decentralized trial distributed as incomplete blocks in 1,165 farmer-managed fields across the
Ethiopian highlands with a benchmark representing genomic prediction applied to conven-
tional breeding. We found that 3D-breeding could double the prediction accuracy of the
benchmark. 3D-breeding could identify genotypes with enhanced local adaptation providing
superior productive performance across seasons. We propose this decentralized approach to
leverage the diversity in farmer fields and complement conventional plant breeding to
enhance local adaptation in challenging crop production environments.
1 Department of Agricultural Sciences, Inland Norway University of Applied Sciences, Hamar, Norway. 2 Digital Inclusion, Bioversity International,
Montpellier, France. 3 Department of Plant Pathology, Kansas State University, Manhattan, KS, USA. 4 Biodiversity for Food and Agriculture, Bioversity
International, Nairobi, Kenya. 5 College of Agriculture and Life Sciences, Cornell University, Ithaca, NY, USA. 6 Agricultural Research Service, United States
Department of Agriculture, Ithaca, NY, USA. 7 Institute of Life Sciences, Scuola Superiore Sant’Anna, Pisa, Italy. 8 Ethiopian Biodiversity Institute, Addis
Ababa, Ethiopia. ✉email: m.dellacqua@santannapisa.it
COMMUNICATIONS BIOLOGY |           (2021) 4:944 | https://doi.org/10.1038/s42003-021-02463-w |www.nature.com/commsbio 1
1234567890():,;
ARTICLE COMMUNICATIONS BIOLOGY | https://doi.org/10.1038/s42003-021-02463-w
The big data revolution in genomics has transformed plant variety to each of 1,165 farmers located in the same breedingbreeding with inexpensive sequencing methods, enabling mega-environment. We tested 3D-breeding against a competitivegreatly accelerated variety development1–3. At present, benchmark that represents breeding based on a genomic pre-
plant breeders use data-driven methods, including genomic pre- diction model trained on centralized stations to predict varietal
diction, to increase selection intensity while reducing the time of performance in farmers’ decentralized fields. We focused on grain
the breeding cycle and deriving greater genetic gain4. Most con- yield (GY) and farmers’ overall appreciation (OA) of wheat
ventional breeding programs still rely on a centralized scheme genotypes, which were both recorded in centralized and decen-
aimed at maximizing genetic diversity (G) in the early stages of tralized trials. To establish the benchmark, we used a genomic
selection and then identifying superior germplasm based on prediction model trained on data measured in stations to predict
phenotypic observations made in a limited number of research wheat GY and OA in farmer fields (Fig. 1a). We then developed
stations with explicit environmental (E) and management (M) 3D-breeding to move the selection to farmer fields, predicting
conditions. In this setting, genomic prediction may be used to wheat performance in farmers’ fields using a decentralized
predict the performance of untested new genotypes but is bound approach (Fig. 1b). Comparing side by side the accuracy of the
to the G ´E ´M interactions captured by the research stations two methods, we found that that 3D-breeding could increase
that are used to train the selection models5. This limitation of prediction accuracy in challenging environments and thus com-
centralized breeding approaches may result in suboptimal plement genomics assisted breeding.
development and deployment of crop varieties for use by farmers
seeking local adaptation in challenging environments6. This is Results and discussion
especially relevant in smallholder farming systems, which involve
7 Performance of centralized breeding based on genomic pre-about 80% of the world farmers and call for tailored solutions to diction and farmers’ traditional knowledge. Heritability (H2),
support food security. the proportion of phenotypic variance explained by genotypic
To respond to local cropping needs impacted by climate variance, was 0.55 and 0.42 for GYSTATION across locations for 2012change, breeders need to find new ways to accelerate variety and 2013 respectively (Supplementary Data 1). To capture farmers’
development while directly addressing G ´E ´M interactions to
3,8,9 traditional knowledge regardless of gender, farmer scores werethe fullest . Mobilizing farmers’ traditional knowledge of crop combined across men and women respondents: the H2 of
varieties and local adaptation can address this challenge and
enhance adoption of improved varieties6,10–12 in a coherent, OASTATION was 0.78 across locations. Narrow sense heritability (h
2)
decentralized breeding program relying on farmer-participatory was calculated considering genetic co-variance of genotypes and
selection13–15. A crowdsourced citizen science approach has provided more conservative estimates for all traits, yet OASTATION
demonstrated the feasibility of a data-driven decentralized variety was consistently more heritable than GYSTATION (Supplementary
evaluation16 that enables on-farm variety testing in a digitally Data 1 and 2). We validated the centralized benchmark by pre-
supported and cost-ef cient way17. Predictive accuracy of farmer dicting on-station performance from one season to the next,fi
selection criteria may outperform breeder evaluations even in a focusing on a subset of 41 genotypes that were later distributed in
context of modern agriculture18. decentralized farmer fields. This led to accuracies up to τ ¼ 0:248
Crowdsourced citizen science further integrates the E and M in predicting GYSTATION in the following season (Supplementary
components into breeding by performing selection directly in Fig. 1). Previous studies showed that men and women may prior-
target environments and using environmental data to analyze itize different traits depending on their role in the farming activity,
genotypic responses. Thus, the citizen science approach scales E from cropping to marketing of products
22,23. In our study, gender
2
and M data collection to generate a volume of data that matches differences in OASTATION scoring are reflected by different H
the big data dimension of G. Combining genomic prediction with achieved by men (0.84) and women (0.67), with a more marked
citizen science opens the possibility of simultaneously capturing difference in Hagreselam (Supplementary Data 2). Still, men and
the three dimensions of crop performance, G, E, and M, in a data- women provided consistent evaluations (Supplementary Fig. 2).
driven way. Here, we describe and demonstrate potential bene ts This is in line with tricot observations reporting that gender havefi 17
of this approach that we call data-driven decentralized breeding, low overall effect on varietal choice and shows that farmer
or 3D-breeding, for short. Potentially, 3D-breeding could bene t scores are reliable measures of genotypes performance. Indeed, wefi
the ~500 million smallholder farmers around the world who often found that OASTATION was a better predictor than GYSTATION to
produce in challenging, low-input environments and work with capture both OASTATION and GYSTATION , including when dis-
diverse cropping and farming systems and respond to local aggregated by gender (Supplementary Fig. 3). Previous studies
consumption preferences7. explored the relation between OA and agronomic performance of
We applied the 3D-breeding approach in the Ethiopian high- wheat, showing that farmers’ appreciation was positively correlated
lands, where many smallholder farmers grow durum wheat to yield, seed size, biomass, and negatively correlated with time to20,21
(Triticum durum Desf.) and select landraces following criteria flowering and time to maturity .
related to environmental adaptation, food culture, and market
demand19,20. Rich local wheat diversity has co-evolved with local Benchmark: using centralized measures to predict performance
cultures and landscapes over millennia. Consequently, Ethiopian in farmer fields. The benchmark had a low prediction accuracy
farmers still often select and cultivate local landraces, which when using GYSTATION to predict GYFARM in individual seasons,
under local conditions tend to outperform modern varieties with an average of τ ¼ 0:046. When using OASTATION to predict
produced by centralized breeding21. In this context, 3D-breeding OAFARM , the average was τ ¼ 0:141 (Table 1). Indeed, GY and
can leverage local wheat diversity and knowledge and bring OA collected in stations were poorly correlated with on-farm
breeding closer to the target environments cutting through the performance (Supplementary Fig. 4). Accuracy remained low
complexity of G ´ E ´M. when GYSTATION was used to predict measures of GYFARM and
Here, we collected data from the genotyping and phenotyping OAFARM combined across seasons and in alternative scenarios
of 400 wheat varieties in centralized stations commonly used for considering different subsets of training and test populations
varietal selection in Ethiopian highlands. We then selected and (Supplementary Data 3). Interestingly, OASTATION had consistent
distributed a subset of 41 genotypes as packaged sets containing positive accuracy in predicting GYFARM and OAFARM (Supple-
incomplete blocks of three genotypes, plus one commercial mentary Fig. 5). This confirmed that genomic prediction can be
2 COMMUNICATIONS BIOLOGY |           (2021)4 :944 | https://doi.org/10.1038/s42003-021-02463-w |www.nature.com/commsbio
COMMUNICATIONS BIOLOGY | https://doi.org/10.1038/s42003-021-02463-w ARTICLE
Fig. 1 A comparison of centralized versus decentralized breeding approaches. Centralized breeding (a) derives recommendations from breeders’
evaluation and possibly participatory assessments in a limited set of stations, using genomics to accelerate the production of varieties that are eventually
recommended with coarse spatial resolution. The plot shows the broad recommendation space of two hypothetical varieties, Var A and Var B. This system
may become more efficient if complemented by 3D-breeding (b), a decentralized approach where the best candidate genotypes are tested by farmers in
small, blinded and randomized sets. 3D-breeding produces scalable solutions that can be linked to genomics, farmers’ knowledge and environmental data,
to enhance the local adaptation of the resulting varieties and tailor their recommendation to the landscape. This is represented in the plot to the right by the
precise recommendation space of hypothetical varieties Var A, Var B, Var C and Var D.
Table 1 Performance of the 3D-breeding compared with the benchmark of a centralized genomic prediction.
Approach OA GY
Centralized GS
Season 1 (n=179) 0.134 −0.012
Season 2 (n=651) 0.105 0.076
Season 3 (n=335) 0.183 0.073
0.141 (± 0.039) 0.046 (± 0.049)
3D-breeding
Season 1 (n=179) 0.270 0.160
Season 2 (n=651) 0.276 0.078
Season 3 (n=335) 0.203 0.119
0.251 (± 0.040) 0.109 (± 0.041)
3D-breeding provides higher across-season goodness-of-fit (Kendall τ) than centralized genomic prediction on overall appreciation (OA) and grain yield (GY) derived from farmer rankings on
decentralized fields.
Prediction accuracy combined across seasons is given in bold.
enhanced by farmers’ traditional knowledge whereas selection farming sites goes beyond that captured by temperature variation
based only on GY could result in reduced appreciation by farmers (Supplementary Fig. 10). Regardless the fact that both stations and
(Supplementary Fig. 6). farms were located in the same agroecological zone (Supplementary
GYSTATION provided a more accurate prediction of GYFARM when Fig. 11), the benchmark failed to predict performance under
restricting the model to cold-tolerant genotypes (Supplementary production conditions, showing that the small-scale variation in
Fig. 7). This was likely due to the partial representation of the climate and management may hamper the success of centralized
climatic variation that can be provided by a centralized approach breeding decisions.
with a handful of stations (Supplementary Fig. 8), as farms could
experience lower temperatures than stations (Supplementary Fig. 9). 3D-breeding provides higher prediction accuracy than the
Still, centralized predictions of increasingly distant farm environ- benchmark. Model predictions from 3D-breeding consistently
ments shown an erratic pattern, showing that variation at the provided higher accuracy than the benchmark for GYFARM and
COMMUNICATIONS BIOLOGY |           (2021) 4:944 | https://doi.org/10.1038/s42003-021-02463-w |www.nature.com/commsbio 3
ARTICLE COMMUNICATIONS BIOLOGY | https://doi.org/10.1038/s42003-021-02463-w
OAFARM with τ ¼ 0:109 and τ ¼ 0:251 (Table 1). When sup- of high quality, making the comparison realistic. We have
ported by smaller sets of observations (from 5% to 75% of the explored whether the superiority of 3D-breeding was sensitive to
available data), 3D-breeding maintained superior accuracy than the influence of data availability, the geographical placing of the
the benchmark, with a mean accuracy spanning from τ ¼ 0:162 centralized selection environments or the variable of focus
to τ ¼ 0:230 for OAFARM and from τ ¼ 0:076 to τ ¼ 0:106 for (overall appreciation or grain yield) and found that its superiority
GYFARM (Supplementary Data 4). The prediction accuracy of the was robust. This has important implications for breeding pro-
3D-breeding approach was not biased towards specific environ- gram design.
mental conditions, suggesting that it could capture the environ- Genomic prediction is a well-known approach to accelerate
mental diversity of test sites better than the benchmark breeding programs, but current implementations in plant
(Supplementary Figs. 12 and 13). breeding have not yet been combined with a decentralized
Overall appreciation of genotypes in 3D-breeding pro- approach. The earliest and most successful implementations of
vided higher prediction accuracies than GYFARM in all farmers’ genomic prediction have arguably occurred in dairy cattle
fields (Supplementary Fig. 14). Previous studies showed that breeding25. The accelerated evaluation of bull net merit was key
farmer evaluations are able to capture agronomic performance of to this26, but that success also depended on the fact that breeders
genotypes in untested locations18,20, as confirmed by the high H2 had access to phenotyping data from a broad range of
observed for OASTATION (Supplementary Data 2). Farmers environments in the form of milking records, which farmers
provided OA according to their own experience and preferences, record for their own management benefit. In conventional crop
and it presumably depended on a combination of traits of which breeding, all of the phenotyping costs fall on the breeding
GY represented only one dimension21. By eliciting traditional program and limit the number of target environments that can be
knowledge of men and women farmers at cropping sites, 3D- represented in the selection process. 3D-breeding seeks to
breeding successfully predicted varietal performance under local complement and expand the flow of information from a few
growing conditions (Supplementary Fig. 5). GYFARM is objectively centralized locations to the whole mega-environment where
and independently measured at each plot and therefore it could results from numerous decentralized observations and farmer
not be biased by OAFARM . It is possible that GYSTATION and knowledge may converge to inform breeding decisions.
GYFARM failed to capture secondary traits with high heritability In centralized breeding, the environmental variation of target
(Supplementary Data 1) that were observed by farmers and that environments is factored through experimental control or indirectly
were correlated to the GYFARM of genotypes under field as an average response across breeding stations as in our
conditions20,21. As OAFARM is directly related to the probability benchmark. This makes extrapolation to real farming conditions
of variety adoption it is an important complement to GY in challenging. G ´E affects yield and its components27,28 and calls for
driving varietal development for challenging environments. selection models to explicitly account for it29. These models,
however, are bound to the observations that can be made in
resource-intensive breeding trials. The scope and size of the
Superior genotype selection with 3D-breeding is consistent benchmark in this study was representative of a regional variety
across seasons. We extrapolated the 3D-breeding model predic- trial, an advanced stage in breeding focusing on a set of genetic
tions to assess the probability that the genotypes selected by 3D- materials and target environments with the aim of selecting the best
breeding based on OA will outperform currently recommended
24 genotypes for varietal release and recommendation. Even when theyvarieties . We found that the best three genotypes in each terminal are place din relatively representative locations, centralized stations
node of the 3D-breeding model splits had a genetic background cannot represent the entire pedoclimatic space occupied by target
markedly separated from that of varieties currently recommended farmer fields (Supplementary Fig. 9). Data from crowdsourced
for the region, and consistently higher worth (Fig. 2a). Indeed, the citizen science, like 3D-breeding, may further our understanding of
model selected genotypes derived from landraces over improved the G ´ E interactions that are observed in farmer fields and allow
varieties. We estimated the reliability, i.e. the probability that the the integration of increasingly accurate seasonal prediction
model recommendation exceeds the current recommendation in models30 in breeding and germplasm recommendation pipelines.
terms of OAFARM . In this assessment, predictions from 3D-breeding The 3D-breeding approach addresses the low correlation between
outperformed the current varietal recommendations in most of the performance in selection environments and production environ-
farmers’ fields, with consistent high reliability (0.83–0.91), including ments, while taking a step forward to fully data-driven breeding. In
in challenging areas for which the centralized breeding approach this, 3D-breeding is a promising approach that could add to
could not provide accurate predictions (Fig. 2b). To provide an conventional breeding increasing varietal performance in small-
agronomic measure, we also predicted the increase in GYFARM and holder agriculture, which accounts for the largest share of the global
tested to see if the yield advantage could be maintained by selecting farms7. In those settings, the adoption rate of current breeding
the best three genotypes indicated by 3D-breeding under 15 dif- innovation may be suboptimal due to socioeconomic and
ferent growing seasons simulated on target farms. We found that environmental factors9,21,31–33. Climate change is pushing these
3D-breeding ensured consistent recommendations over years farming systems to the edge of their adaptation capacity with
with expected increases in yield of about 20% (Fig. 2c). Thus, increasing pressure from pest and diseases34,35, threats of yield
3D-breeding accurately identified the best performing genotypes to loss36,37 and increased seasonal climatic variability38,39, calling for
be advanced in breeding efforts targeting local growing conditions, tailored solutions. 3D-breeding may speed up the turnover of
to be developed into suitable new varieties, and to be promoted with varietal release to address these challenges. As farmers are at the
environmental-specific recommendations. center of the experimental design, varieties deriving from 3D-
breeding are more likely to be adopted and suited to local
Implications for rethinking breeding programs. Our results cultivation11,40, increasing the effectiveness of breeding efforts.
show that 3D-breeding is superior to a benchmark that represents Indeed, we found that farmers’ OA was a better predictor than GY
a centralized breeding approach. The genomic prediction in predicting yield realized both in centralized and decentralized
benchmark and 3D-breeding rely on different statistical designs trials (Table 1). Likewise, varieties derived from landraces
and methods, yet they have the same aim: providing accurate consistently outranked the performance of improved varieties
prediction of phenotypes in untested environments. We believe (Fig. 2a) derived from centralized breeding19. Beyond varietal
that the implementation of the two approaches was realistic and recommendations, 3D-breeding can direct the choice of parents to
4 COMMUNICATIONS BIOLOGY |           (2021)4 :944 | https://doi.org/10.1038/s42003-021-02463-w |www.nature.com/commsbio
COMMUNICATIONS BIOLOGY | https://doi.org/10.1038/s42003-021-02463-w ARTICLE
Fig. 2 Selection of durum wheat (Triticum durum Desf.) genotypes based on 3D-breeding. a Principal component coordinates of the genetic diversity of
tested genotypes. Pink dots represent the varieties currently recommended for the area of study. 3DB Cold tolerant (blue) represents the top 3 genotypes
selected by 3D-breeding in cold areas (minimum night temperature <11.5 °C). 3DB Warm tolerant (red) represents the top 3 genotypes selected by
3D-breeding in warm areas (minimum night temperature >11.5 °C). Size of dots represents the performance of genotypes in farmer fields as overall
appreciation (OA). b Probability of outperforming improved varieties currently recommended by using genotype selection generated by 3D-breeding with
OA. The panel shows the probability of the top 3 genotypes in a given location in outperforming the improved variety recommended for that location.
c Expected increase in yield across 15 consecutive growing seasons (2001 to 2015) for genotype selection from 3D-breeding. n= 1,165 observations.
crosses aiming at the production of recombinant lines to provide that farmers perceive as beneficial the interaction with experts
higher and more stable yields in local agriculture. and the sharing of information42. Benefit to farmers may exceed
the immediate access to improved technology, if the deeds to
Potential of 3D-breeding for challenging cropping environ- reconcile farmers’ and breeders’ rights in plant variety protection
ments. It has been advocated that scientific research and inno- succeed43.
vation must decidedly focus on small-scale farming systems to In this study, farmers evaluated top performing varieties
move towards a world with zero hunger by 203041. 3D-breeding chosen from a larger set, but future studies may focus on larger
makes smallholder farmers innovation drivers as well as reci- collections of germplasm to be evaluated through 3D-breeding in
pients, supporting the sustainable intensification of challenging combination with evaluations performed in research stations.
environments. However, 3D-breeding is useful beyond small- These may include new genetic materials prioritized by speed-
holder farming agriculture, and the citizen science approach on breeding44 and haplotype-based selection45. Our results show that
which it relies has already been applied to several crops to already the current replication level of the experimental design
enhance the selection of climate-adapted varieties16. Its general may support more diversity (Supplementary Data 4). 3D-
scheme may also be useful in high-input, yield maximizing breeding may be most effective as a complement to a centralized
agriculture to enhance local adaptation and support sustainability breeding system providing a high-throughput evaluation of
and food security, where the usefulness of farmers’ evaluations in correlated traits to support earlier varietal selection to be tested
a genomic setting was already demonstrated18. In these settings, in farmer fields46. Our method may complement and enhance
3D-breeding could contribute to the identification and develop- trait prioritization and speed-breeding methods currently used to
ment of varieties with higher local adaptation, reducing the need reduce the need of extensive, resource-intensive multilocation
of external inputs to achieve desired yields. trials47. Accuracy is just one among the factors controlling genetic
There are a number of open questions in relation to gain48, thus our findings should be integrated in the broader
decentralized crop breeding, including how to best motivate picture of modern breeding. Multi-trait models may increase
new farmers to participate in the evaluation of materials, how prediction accuracy by measuring correlated traits with higher
much planting material each farmer needs, the logistics of heritability46,49,50. These models could be employed in centra-
providing farmers with the genetic material, and how to share lized stations and used to narrow down the set of varieties to be
benefits deriving from the utilization of farmers’ knowledge to distributed to farmers in the 3D-breeding approach aiming to
produce new varieties. Both in centralized stations and in fine-tune local adaptation. Moreover, our findings support the
decentralized fields, we found that farmers were eager to need to further explore the challenge to model farmers’
participate without material compensation. Farmers seek access appreciation at the genomic level to improve the effectiveness
to new genetic materials that they could not access otherwise, in of genotypes evaluation trials18.
exchange for the minimum investment of running small plots and The advantages provided by the approach are clear: phenotyp-
providing a concise evaluation at the end of the season in the case ing costs would be divided in much smaller packets, supporting
of the tricot evaluation17. This happens even if some may not be the modular expansion of the breeding effort towards new genetic
adapted to their growing environment. Previous studies showed materials or new locations. In return, each generated datapoint
COMMUNICATIONS BIOLOGY |           (2021) 4:944 | https://doi.org/10.1038/s42003-021-02463-w |www.nature.com/commsbio 5
ARTICLE COMMUNICATIONS BIOLOGY | https://doi.org/10.1038/s42003-021-02463-w
would be a better representation of the true farming conditions to Evaluation of genotypes in centralized trials. Centralized trials were performed
which varieties are directed. Previous research found that the in 2012 and 2013 in the districts of Geregera (Amhara) and Hagreselam (Tigray)
involvement of farmers in selection experiments has negligible (Supplementary Fig. 15). The experimental stations were chosen to represent the
51 highland agroecology of Ethiopia and are often used as varietal testing sites foreffects on costs . In 3D-breeding the costs are shared by farmers, local agriculture. The trial was laid out in a replicated alpha lattice design with the
who would in exchange obtain access to the best materials for full set of 400 genotypes as entries, for a total of 800 plots in each field. Field
their farm. Farmer preference would be collected directly on managements were conducted as per local guidelines with manual weeding.
farms rather than derived from correlated metrics that come from Accessions were sown in four rows 2.5 m long, at a seeding rate of 100 kg ha
−1. At
sowing, 100 kg ha−1 diammonium phosphate and 50 kg ha−1 urea were applied,
on-station evaluations in centralized breeding. In terms of with additional 50 kg ha−1 urea at tillering.
absolute costs, an implementation of 3D-breeding based on OA In each location, 15 men and 15 women who were experienced smallholder
would require additional investments in seed multiplication, seed farmers growing durum wheat were invited to evaluate plots during the
distribution and telecommunications to obtain feedback from 2012 season. After being informed on the study, its aims and methods, farmers
farmers. These costs are generally lower per data point than in provided a verbal informed consent that was recorded on paperwork. Theevaluation was conducted at flowering time in each experimental station, for a total
on-farm evaluation trials using conventional approaches. Geno- of 60 farmers involved. The farmers had no previous knowledge of the genotypes
typing costs are negligible thanks to ever increasing sequencing included in this study to prevent bias in the evaluations. The participants provided
capabilities1. appraisal with Likert scales57 to genotypes for overall appreciation (OA)20,21, with
1 being worse and 5 the best. Prior the experiment, farmers were involved in focus
group discussions and trained on how to perform the evaluation21. During the
Conclusion evaluation, farmers were divided in gender-homogenous groups of 5 people, were
The data-driven focus of 3D-breeding enables embracing the introduced in the field from random entry points, and were accompanied plot by
plot by a researcher who guided the evaluation and collected OA values from
complexity of real-world G ´E for the benefit of breeding. Such a individual farmers. Farmers did not use half-values to streamline the evaluation
multidimensional, collaborative approach calls for best practices effort. After harvesting, technicians measured grain yield (GY) as grams of grain
in data management and sharing52. 3D-breeding is based on a produced per plot, then converted into t  ha1. Other agronomic traits were also
documented set of methods, from experimental design17 to data collected as detailed in Mengistu et al. (2016)19.
curation and analysis53,54. While our demonstration of these
methods relied on a large dataset, we believe that much larger Evaluation of genotypes in decentralized trials. A total of 1,165 decentralized
field sample designs and genomic variant datasets are quite fea- field, each with 4 plots, were established between 2013 and 2015 during three
sible and will provide additional power, as is also much in evi- growing seasons across the regions of Amhara (471), Oromia (399) and Tigray(295) (Supplementary Fig. 15) using a subset of 38 purified landraces accessions
dence in livestock genetics. The expansion of the design with the identified through farmer evaluation in centralized trials21 and three modern
addition of further testing seasons and local management con- cultivars, for a total of 41 wheat genotypes (Supplementary Fig. 15). Farms were
ditions may allow to highlight drivers of local performance of selected in areas representative for wheat growing in Ethiopia, based on previous
genotypes beyond temperature55. Further 3D-breeding studies history of cultivation of the crop (Supplementary Fig. 16). Individual farmers wereengaged via local agricultural offices and selected based on their willingness to
may opt to stratify participants for socioeconomic features of participate and of the following criteria: (i) being wheat growers, (ii) owning the
interest, including gender, age, or income, to fully characterize land, (iii) living in the village all year. No financial incentive was given to farmers
traditional knowledge in its many dimensions. Ideally, 3D- besides the opportunity to test new varieties and keep the harvest from the
breeding could be combined with conventional, centralized decentralized varietal plots. Farmers were fully informed of the study and provided
a verbal informed consent that was recorded on paperwork. Selected farms were
breeding to improve the training of prediction models to address representative of the agroecological zones of the centralized fields (Supplementary
local adaptation. Once new varieties are developed though the Fig. 11). Season 1 (2013) comprised 179 fields, Season 2 (2014) comprised 651
crowdsourced combination of breeders’ and farmers’ knowledge, fields, and Season 3 (2015) comprised 335 field. Differences in number of fields by
future research shall focus on the potential impact of these season are due to availability of farmer communities. Trials (farmer-managed
plots) followed the triadic comparison of technologies (tricot) approach17. Sets of
methods on conservation and use of traditional agrobiodiversity three local genotypes plus an improved variety were allocated randomly to farmers
both in situ and beyond the local environments in which it was as incomplete blocks, maintaining spatial balance by assigning roughly equal fre-
developed. The crowdsourced citizen science approach associated quencies of the genotypes. Each farmer also received an improved variety (Asassa
with open-source digital tools makes it possible for breeders and in Tigray and Amhara, and Hitosa and Ude in Oromia), for a total of four plots per
farmers to apply 3D-breeding in new contexts and crops, farmer. Trial size ranged from 0.4 m
2 to 1.6 m2 depending on season and location.
Field technicians provided guidance to farmers on the tricot approach prior the
dependent only on creativity in identifying untested production experiment. Farmers planted, managed and evaluated their own experiments. At
niches, potentiating a culturally driven co-evolution between the end of the growing season, farmers were visited by an enumerator and indi-
farming systems and data-driven breeding to complement tradi- cated the OA of genotypes by ranking the four entries that they received from best
tional breeding. to worst, using pre-defined answer forms. Field technicians collected GY measuresin farmers’ plots after harvesting. Differently from the centralized trials, the OA
was derived from the relative rankings of genotypes, as each farmer evaluated a
Materials and methods different set of materials.
Plant materials and DNA extraction. We selected 400 durum wheat (Triticum
durum Desf.) genotypes from a representative collection of landraces accessions Centralized trait data analysis. All analyses were done in R58. GYSTATION and
maintained at the Ethiopian Biodiversity Institute (EBI) and improved lines cul- OASTATION measured in centralized trials were used to derive best linear unbiased
tivated in Ethiopia. Landrace accessions were purified to derive a uniform genetic prediction (BLUP) values using the R package ASReml-R59, treating locations as a
background to undergo all subsequent analyses, so that all seeds derived from a fixed factor and all other factors as random. Full model details are reported in
single spike representative of the EBI accession as described in Mengistu et al. Supplementary Note 1. For the central comparison between benchmark and 3D-
(2016)19. Genomic DNA was extracted from fresh leaves pooled from five seedlings breeding, we used measures of GYSTATION combined across seasons and locations
for each of the purified accessions with the GenEluteTM Plant Genomic DNA (Eq. S1). Similarly, OASTATION in the central comparison represents OA values
Miniprep Kit (Sigma‐Aldrich, St Louis, USA) following manufacturer’s instructions combined across genders and locations (Eq. S3). When relevant, GYSTATION and
in the Molecular and Biotechnology Laboratory at Mekelle University, Tigray, OASTATION measures were split by location, season or gender (Supplementary
Ethiopia. Genomic DNA was checked for quantity and quality by electrophoresis Note 1). Broad sense heritability (H2) and narrow-sense heritability (h2) were
on 1% agarose gel and NanodropTM 2000 (Thermo Fisher Scientific Inc., Wal- derived for agronomic traits (Eq. S2) and farmers’ OA (Eq. S4). Agreement
tham, USA). Genotyping was performed on the Infinium 90k wheat chip at between farmer gender groups in evaluating centralized station data was derived
TraitGenetics GmbH (Gatersleben, Germany). Single nucleotide polymorphisms from a linear model fit. Spearman correlations between location specific BLUP
(SNPs) were called using the tetraploid wheat pipeline in GenomeStudio V11 values and farm performance were also computed.
(Illumina, Inc., San Diego, CA, USA). SNP calls were cleaned for quality by filtering
positions and samples with failure rate above 80% and heterozygosity above 50%.
Full details on the genotyping are given by Mengistu et al.19. The SNP calls for the Decentralized trait data analysis. For the analysis of the decentralized data, we
genotypes included in this study and the details on the provenance of genotypes used the Plackett–Luce model60,61, using the R package PlackettLuce54. The
tested are given as part of the full dataset on Dataverse56. implementation of Plackett–Luce model to analyze data from decentralized crop
6 COMMUNICATIONS BIOLOGY |           (2021) 4:944 | https://doi.org/10.1038/s42003-021-02463-w |www.nature.com/commsbio
COMMUNICATIONS BIOLOGY | https://doi.org/10.1038/s42003-021-02463-w ARTICLE
variety trials is demonstrated by van Etten et al.16. Plackett–Luce is a rank-based Generalization of the 3D-breeding. To evaluate if the model obtained with the
model that follows the Luce’s axiom of choice61, which assumes that ranking order variable selection procedure retained predictive power across seasons, we simulated
between every pair of options does not depend on the presence or absence of other untested future seasonal climate with representative seasonal scenarios of past
options. The model estimates the worth parameter α which related to the prob- climate conditions by extracting the last 15 years of daily climate data derived from
ability (P) that one genotype i wins against all other n genotypes in set, and are NASA POWER (2001–2015). We determined three windows for sowing dates in
obtained using the following equation: each growing season as the midpoints of equiprobable quantile intervals estimated
from the observed planting dates in the data set. We predicted genotype perfor-
Pði  f a aj; :::; ngÞ ¼ i i
a þ þ ¼ ¼ aa 1 i ð1Þ
mance for 15 seasons ´ 3 sowing dates (45 seasonal scenarios) for 1,200 random
1 ::: n points generated across an alpha hull area within the range of the trials’ coordi-
nates. We averaged genotype probability of winning across these scenarios for each
planting date interval, excluding the seasons used as testing data.
Implementation of the genomic prediction benchmark. We established a We calculated the reliability, the probability of outperforming a check variety71.
benchmark that represents a centralized breeding approach enriched with farmer We used the worth parameters from Plackett–Luce to determine the values of
evaluations. We believe that this benchmark represents a realistic and competitive positive-valued parameters αi associated with each genotype i, by comparing the
alternative to 3D-breeding. On-station involvement of farmers is not common worth from the check variety (Asassa, Hitosa and Ude, currently recommended for
practice but is increasingly conducted in association with breeding14,18 and makes the mega-environment24) with the worth of the selected genotypes from 3D-
the benchmark more competitive. The stations selected for the benchmark were breeding. These parameters (αi) are related to the probability (P) that genotype i
commonly used as breeding field trials for Amhara and Tigray regions of Ethiopia, wins against all other n genotypes in a set as shown in Eq. 1. To calculate the
and differ in altitude, temperature, rainfall, and soil21. Additional multilocation reliability of a genotype, we used Equation 2:
trials would typically occur in earlier stages of the breeding cycle. Centralized a
stations and farmer fields belong to the same agroecological zones of Ethiopia Pði  jÞ ¼ i
a þ a ð2Þ(Supplementary Fig. 11). i j
The benchmark was based on genomic prediction models and marker-based
genetic relationship matrices computed on BLUP data with the package rrBLUP62, Environmental characterization of test sites and genotypes. The agroecolo-
a method widely used in breeding programs worldwide. To measure accuracy of gical zonation of Ethiopia was obtained by the Ethiopian Institute of Agricultural
genomic predictions, we calculated the Kendall’s tau coefficient (τ), a measure of
63 Research (EIAR)
72. GPS coordinates of centralized stations and decentralized
similarity of rankings , between predicted values and observed values. The use of farmer fields were used to retrieve climatic data from NASA POWER. Tem-
the τ metric, uncommon in breeding64, allowed to compare accuracies with the perature indices for covariates used in the PL model were retrieved for the
3D-breeding approach. A Pearson’s correlation, the standard metric for genomic growing seasons object of the study in the time span from sowing date and
prediction accuracy, was also computed but did not show any relevant difference
flowering dates as measured on-site. Climatic variables considered were the
with the Kendall τ. Also to provide a more coherent comparison with 3D-breeding, maximum night temperature (°C) during reproductive growth and the minimum
the benchmark was trained with ordinal rankings derived from absolute values of night temperature (°C) during the vegetative growth, which showed to be the
GY and OA measured in centralized trials, without showing any relevant difference most relevant for the sampled data. A principal component analysis (PCA) was
from the training performed with absolute values. used to summarize and depict variation at test sites. Climatic distance of test
The benchmark considered two main prediction scenarios. In the first scenario, sites was derived from a multidimensional scaling (MDS) of the multivariate
prediction was restricted to the centralized experiment. In this scenario, the climate dataset. For each of the two stations, climatic distance was computed
genomic prediction model was trained on GYSTATION and OASTATION measured on with all farm sites. Wheat genotypes were split in cold adapted and warm
the full set of 400 genotypes evaluated in 2012, and the training dataset was adapted according to the altitude of their original sampling site with a one-tailed,
GYSTATION measured in the same locations in 2013 on the subset of 41 genotypes unequal-variance t-test.
that were also included in the 3D-breeding. In the second scenario, the benchmark
was trained on combined GYSTATION and OASTATION data in centralized trials and
used to predict the test population of 41 genotypes measured in decentralized fields Statistics and reproducibility. Centralized experiments were run in two locations,
for GY and OA . Mirroring the approach used in the 3D-breeding, the for two seasons, on replicated plots for 400 genotypes for a total of 3,200 plots. TheFARM FARM
accuracy of genomic prediction in the second scenario was derived from a cross- benchmark was run with different prediction scenarios considering separated and
validation approach averaging Kendall τ specific for Season 1, Season 2, and Season overlapping training and test populations and specified in the methods. Decen-
3 using the square root of the sample size as weights65. tralized trials were performed on 1,165 farmer fields, with four plots per farmer
The benchmark was tested with additional prediction scenarios considering field evaluated in ranking, for a total of 4,660 plots. Organizing the datasets relied
different training and test populations, including: (i) without overlap between on R packages data.table73, caret74, gosset75, janitor48, magrittr76 and tidyverse77.
training and test samples, (ii) restricting the training to the subset of 41 genotypes Climatic variables were obtained using the packages climatrends68 and
selected for 3D-breeding, (iii) predicting GY and OA in decentralized nasapower67. Statistical analysis was performed using packages PlackettLuce54,FARM FARM
elds strati ed by their environmental distance from centralized stations. gosset75 78fi fi and qvcalc . Spatial visualization was performed with the packages
dismo79, raster80, sf81 and smoothr82. Charts were produced using packages
corrplot83, ggplot284 and patchwork85.
Implementation of the 3D-breeding. The model representing the 3D-breeding
approach was built with the data generated by the citizen science decentralized
trials using Plackett–Luce Trees (PLT). This model includes covariates through Reporting summary. Further information on research design is available in the Nature
recursive partitioning (successive binary splits based on covariate thresholds)66. We Research Reporting Summary linked to this article.
used PLT to analyze OAFARM and GYFARM . DNA data from SNPs was added into
the model as a prior using an additive matrix. Agroclimatic indices were used as
covariates in the PLT model. Daily temperature and precipitation data were Data availability 56
obtained from the NASA LaRC POWER Project (https://power.larc.nasa.gov/), Data is available through Dataverse .
using the R package nasapower67. The set of agroclimatic covariates was extracted
for the vegetative, reproductive and grain filling phases and the whole growth Code availability
period (from planting date to harvesting) in each observation point using the R Code is available through Dataverse56.
package climatrends68. This resulted in 110 covariates.
To create a model that provides generalizable predictions across seasons with
few covariates, we used blocked cross-validation (with seasons as blocks) combined Received: 11 January 2021; Accepted: 16 July 2021;
with a forward selection69. We used the deviance values of each validation season
to calculate an Akaike weight, which is the probability that a given covariate
combination represents the best model70. We performed forward selection, using
this combined Akaike weight as our selection criterion. The PLT models had a cut-
off value of α ¼ 0:01 and a minimal partition size of 20 percent of the total dataset.
The covariates selected under this procedure were the maximum night temperature References
(°C) during reproductive growth and the minimum night temperature (°C) during 1. Poland, J. Breeding-assisted genomics. Curr. Opin. Plant Biol. 24, 119–124
the vegetative growth. To compare the accuracy of the model representing 3D- (2015).
breeding with the benchmark, we calculated the Kendall τ between observed
2. Hickey, J. M., Chiurugwi, T., Mackay, I., Powell, W., & Implementing
rankings and predicted coefficients in farmer fields. To accommodate for the
Genomic Selection in CGIAR Breeding Programs Workshop Participants.
different number of observations derived from the benchmark and from
decentralized fields, we run additional 3D-breeding scenarios trained with subsets Genomic prediction unifies animal and plant breeding programs to form
of 75%, 50%, 25%, 15% and 5% of the decentralized plots to explore the prediction platforms for biological discovery. Nat. Genet. 49, 1297–1303 (2017).
accuracy attainable by 3D-breeding with fewer observations. Details on the 3. Eshed, Y. & Lippman, Z. B. Revolutions in agriculture chart a course for
procedure are given in Supplementary Note 1. targeted breeding of old and new crops. Science 366, eaax0025 (2019).
COMMUNICATIONS BIOLOGY |           (2021) 4:944 | https://doi.org/10.1038/s42003-021-02463-w |www.nature.com/commsbio 7
ARTICLE COMMUNICATIONS BIOLOGY | https://doi.org/10.1038/s42003-021-02463-w
4. Hickey, L.T. et al. Breeding crops to feed 10 billion. Nat Biotechnol 33. Tesfaye, S., Bedada, B. & Mesay, Y. Impact of improved wheat technology
37, 744–754 (2019). adoption on productivity and income in Ethiopia. Afr. Crop Sci. J. 24, 127–135
5. Heffner, E. L., Sorrells, M. E. & Jannink, J. L. Genomic selection for crop (2016).
improvement. Crop Sci. 49, 1–12 (2009). 34. Deutsch, C. A. et al. Increase in crop losses to insect pests in a warming
6. Ceccarelli, S. & Grando, S. Decentralized-participatory plant breeding: an climate. Science 361, 916–919 (2018).
example of demand driven research. Euphytica 155, 349–360 (2007). 35. Carvajal-Yepes, M. et al. A global surveillance system for crop diseases. Science
7. Lowder, S. K., Skoet, J. & Raney, T. The number, size, and distribution of 364, 1237–1239 (2019).
farms, smallholder farms, and family farms worldwide. World Dev. 87, 16–29 36. Challinor, A. J., Koehler, A.-K., Ramirez-Villegas, J., Whitfield, S. & Das, B.
(2016). Current warming will reduce yields unless maize breeding and seed systems
8. Tester, M. & Langridge, P. Breeding technologies to increase crop production adapt immediately. Nat. Clim. Change 6, 954–958 (2016).
in a changing world. Science 327, 818–822 (2010). 37. Ray, D. K., Ramankutty, N., Mueller, N. D., West, P. C. & Foley, J. A. Recent
9. Acevedo, M. et al. A scoping review of adoption of climate-resilient crops by patterns of crop yield growth and stagnation. Nat. Commun. 3, 1293 (2012).
small-scale producers in low- and middle-income countries. Nat. Plants 6, 38. Cai, W. et al. Increasing frequency of extreme El Niño events due to
1231–1241 (2020). greenhouse warming. Nat. Clim. Change 5, 1–6 (2014).
10. Witcombe, J. R., Joshi, A., Joshi, K. D. & Sthapit, B. R. Farmer participatory 39. Holmgren, M., Hirota, M., van Nes, E. H. & Scheffer, M. Effects of interannual
crop improvement. I. Varietal selection and breeding methods and their climate variability on tropical tree cover. Nat. Clim. Change 3, 755–758 (2013).
impact on biodiversity. Exp. Agriculture 32, 445–460 (1996). 40. Ceccarelli, S. Plant Breeding with Farmers: A technical manual(International
11. Rhoades, R. E. & Booth, R. H. Farmer-back-to-farmer: a model for generating Center for Agricultural Research in the Dry Areas (ICARDA), 2012).
acceptable agricultural technology. Agric. Adm. 11, 127–137 (1982). https://hdl.handle.net/20.500.11766/7745.
12. Fadda, C. et al. Integrating conventional and participatory crop improvement 41. Ending hunger: science must stop neglecting smallholder farmers. Nature.
for smallholder agriculture using the seeds for needs approach: a review. https://doi.org/10.1038/d41586-020-02849-6 (2020).
Front. Plant Sci. 11, 1 (2020). 42. Beza, E. et al. What are the prospects for citizen science in agriculture?
13. Ceccarelli, S. Efficiency of plant breeding. Crop Sci. 55, 87 (2015). Evidence from three continents on motivation and mobile telephone use of
14. Ceccarelli, S. & Grando, S. Participatory plant breeding: who did it, who does resource-poor farmers. PLoS ONE 12, e0175700 (2017).
it and where? Exp. Agriculture 2019, 1–11 (2019). 43. de Jonge, B. Plant variety protection in Sub-Saharan Africa: balancing
15. van Eeuwijk, F. A., Cooper, M., DeLacy, I. H., Ceccarelli, S. & Grando, S. commercial and smallholder farmers’ interests. J. Politics Law 7, p100 (2014).
Some vocabulary and grammar for the analysis of multi-environment trials, 44. Watson, A. et al. Speed breeding is a powerful tool to accelerate crop research
as applied to the analysis of FPB and PPB trials. Euphytica 122, 477–490 and breeding. Nat. Plants 4, 23–29 (2018).
(2001). 45. Brinton, J. et al. A haplotype-led approach to increase the precision of wheat
16. van Etten, J. et al. Crop variety management for climate adaptation supported breeding. Commun. Biol. 3, 1–11 (2020).
by citizen science. Proc. Natl Acad. Sci. USA 116, 4194–4199 (2019). 46. Lado, B. et al. Resource allocation optimization with multi-trait genomic
17. van Etten, J. et al. First experiences with a novel farmer citizen science prediction for bread wheat (Triticum aestivum L.) baking quality. Theor. Appl.
approach: crowdsourcing participatory variety selection through on-farm Genet. 131, 2719–2731 (2018).
triadic comparisons of technologies (tricot). Exp. Agriculture 55, 275–296 47. Cobb, J. N. et al. Enhancing the rate of genetic gain in public-sector plant
(2019). breeding programs: lessons from the breeder’s equation. Theor. Appl. Genetics
18. Annicchiarico, P., Russi, L., Romani, M., Pecetti, L. & Nazzicari, N. Farmer- 132, 627–645 (2019).
participatory vs. conventional market-oriented breeding of inbred crops using 48. Firke, S. janitor: simple tools for examining and cleaning dirty data. R package
phenotypic and genome-enabled approaches: a pea case study. Field Crops Res. version 1.2.1. Available at: https://CRAN.R-project.org/package=janitor
232, 30–39 (2019). (2020).
19. Mengistu, D. K. et al. High-density molecular characterization and association 49. Runcie, D. & Cheng, H. Pitfalls and remedies for cross validation with multi-
mapping in Ethiopian durum wheat landraces reveals high diversity and trait genomic prediction methods. G3: Genes Genomes Genet. 9, 3727–3741
potential for wheat breeding. Plant Biotechnol. J. 14, 1800–1812 (2016). (2019).
20. Kidane, Y. G. et al. Genome wide association study to identify the genetic base 50. Ibba, M. I. et al. Genome‐based prediction of multiple wheat quality traits in
of smallholder farmer preferences of Durum wheat traits. Front. Plant Sci. 8, multiple years. Plant Genome https://doi.org/10.1002/tpg2.20034 (2020).
1230 (2017). 51. Mangione, D., Senni, S., Puccioni, M., Grando, S. & Ceccarelli, S. The cost of
21. Mancini, C. et al. Joining smallholder farmers’ traditional knowledge with participatory barley breeding. Euphytica 150, 289–306 (2006).
metric traits to select better varieties of Ethiopian wheat. Sci. Rep. 7, 9120 52. Leonelli, S., Davey, R. P., Arnaud, E., Parry, G. & Bastow, R. Data management
(2017). and best practice for plant science. Nat. Plants 3, 1–4 (2017).
22. vom Brocke, K. et al. Participatory variety development for sorghum in 53. van Etten, J. et al. ClimMob: Software to support experimental citizen science
Burkina Faso: farmers’ selection and farmers’ criteria. Field Crops Res. 119, in agriculture. version 3.1. Available at: https://climmob.net (2020).
183–194 (2010). 54. Turner, H. L., van Etten, J., Firth, D. & Kosmidis, I. Modelling rankings in R:
23. Sthapit, B. R., Joshi, K. D. & Witcombe, J. R. Farmer participatory crop the PlackettLuce package. Comput. Stat. https://doi.org/10.1007/s00180-020-
improvement. III. Participatory plant breeding, a case study for rice in Nepal. 00959-3 (2020).
Exp. Agriculture 32, 479–496 (1996). 55. Kehel, Z., Crossa, J. & Reynolds, M. Identifying Climate Patterns during the
24. Wheat Atlas, Available at: http://beta.wheatatlas.org/. Accessed March 2020. Crop-Growing Cycle from 30 Years of CIMMYT Elite Spring Wheat
25. Meuwissen, T. H. E., Hayes, B. J. & Goddard, M. E. Prediction of total genetic International Yield Trials. in Applied Mathematics and Omics to Assess Crop
value using genome-wide dense marker maps. Genetics 157, 1819–1829 Genetic Resources for Climate Change Adaptive Traits (eds. Bari, A., Damania,
(2001). A. B., Mackay, M. & Dayanandan, S.) 151–174 (CRC Press, 2016).
26. Hayes, B. J. & Daetwyler, H. D. 1000 bull genomes project to map simple and 56. de Sousa, K. et al. Replication data for: Data-driven decentralized breeding
complex genetic traits in cattle: applications and outcomes. Annu. Rev. Anim. increases genetic gain in a challenging crop production environment.
Biosci. 7, 89–102 (2019). https://doi.org/10.7910/DVN/OEZGVP (2020).
27. Quintero, A., Molero, G., Reynolds, M. P. & Calderini, D. F. Trade-off 57. Likert, R. A technique for the measurement of attitudes. Arch. Psychol. 22, 55
between grain weight and grain number in wheat depends on GxE interaction: (1932).
a case study of an elite CIMMYT panel (CIMCOG). Eur. J. Agron. 92, 17–29 58. R. Core Team. R: A language and environment for statistical computing.
(2018). version 4.0.2. (2020).
28. Ceccarelli, S., Erskine, W., Hamblin, J. & Grando, S. Genotype by environment 59. Gilmour, A. R., Gogel, B. J., Cullis, B. R., Welham, S. J. & Thompson, R.
interaction and international breeding programmes. Exp. Agriculture 30, ASReml User Guide Release 4.1. (Structural Specification, VSN International
177–187 (1994). Ltd, Hemel Hempstead, HP1 1ES, 2015).
29. Eeuwijk, F. Avan et al. Modelling strategies for assessing and increasing the 60. Plackett, R. L. The Analysis of Permutations. J. R. Stat. Soc. Ser. C. 24, 193–202
effectiveness of new phenotyping techniques in plant breeding. Plant Sci. 282, (1975).
23–39 (2019). 61. Luce, R. D. Individual Choice Behavior: A Theoretical Analysis. New York:
30. Bauer, P., Thorpe, A. & Brunet, G. The quiet revolution of numerical weather Wiley (1959).
prediction. Nature 525, 47–55 (2015). 62. Endelman, J. B. Ridge regression and other kernels for genomic selection with
31. Dixon, J. et al. Adoption and economic impact of improved wheat varieties in R package rrBLUP. Plant Genome J. 4, 250 (2011).
the developing world. J. Agric. Sci. 144, 489–502 (2006). 63. Kendall, M. G. A new measure of ranking correlation. Biometrika 30, 81–93
32. Jalleta, T. Participatory evaluation of the performance of some improved (1938).
bread wheat (Triticum aestivum) varieties in the Jijiga plains of eastern 64. Simko, I. & Piepho, H.-P. Combining phenotypic data from ordinal rating
Ethiopia. Exp. Agriculture 40, 89–97 (2004). scales in multiple plant experiments. Trends Plant Sci. 16, 235–237 (2011).
8 COMMUNICATIONS BIOLOGY |           (2021) 4:944 | https://doi.org/10.1038/s42003-021-02463-w |www.nature.com/commsbio
COMMUNICATIONS BIOLOGY | https://doi.org/10.1038/s42003-021-02463-w ARTICLE
65. Whitlock, M. C. Combining probability from independent tests: the weighted Acknowledgements
Z-method is superior to Fisher’s approach. J. Evolut. Biol. 18, 1368–1373 We thank the farmers who evaluated the genotypes in both centralized and decentralized
(2005). trials. We also thank Olga Spellman (Alliance of Bioversity International and CIAT) for
66. Zeileis, A., Hothorn, T. & Hornik, K. Model-based recursive partitioning. English editing, and Rebecca Nelson, Geoffrey Morris, Roberto Buizza and Martina Occelli
J. Comput. Graph. Stat. 17, 492–514 (2008). for the critical discussions and valuable insights. We thank the three anonymous reviewers
67. Sparks, A. H. nasapower: a NASA POWER global meteorology, surface for their thoughtful revisions. This work was implemented as part of the CGIAR Research
solar energy and climatology data client for R. J. Open Source Softw. 3, 1035 Program on Climate Change, Agriculture and Food Security (CCAFS), which is carried out
(2018). with support from the CGIAR Trust Fund and through bilateral funding agreements. For
68. de Sousa, K., van Etten, J. & Solberg, S. Ø. Climatrends: climate variability details, please visit https://ccafs.cgiar.org/donors. The work of K.d.S. and S.Ø.S. was sup-
indices for ecological modelling. R package version 0.1.6. Available at: ported by The Nordic Joint Committee for Agricultural and Food Research (grant num.
https://CRAN.R-project.org/package=climatrends (2020). 202100-2817). The work of M.D.A., M.E.P., Y.G.K. and B.F.L. was supported by the
69. Meyer, H., Reudenbach, C., Hengl, T., Katurji, M. & Nauss, T. Improving Doctoral School for Agrobiodiversity at Scuola Superiore Sant’Anna.
performance of spatio-temporal machine learning models using forward
feature selection and target-oriented validation. Environ. Model. Softw. 101,
1–9 (2018). Author contributions
70. Wagenmakers, E. J. & Farrell, S. AIC model selection using Akaike weights. M.D.A., J.v.E., J.P., K.d.S., S.Ø.S., J.L.J., C.F. and M.E.P. designed research; C.F., Y.G.K.,
Psychonomic Bull. Rev. 11, 192–196 (2004). D.K.M., B.F.M., performed field research; M.D.A., J.P. and K.d.S. analyzed data; K.d.S.,
71. Eskridge, K. M. & Mumm, R. F. Choosing plant cultivars based on the M.D.A., J.v.E. and J.L.J. wrote the paper; S.Ø.S. and J.P. commented on all versions of the
probability of outperforming a check. Theor. Appl. Genet. 84-84, 494–500 manuscript and contributed by suggesting novel additional analyses and interpretations.
(1992).
72. Ministry of Agriculture of Ethiopia. Agro-ecological Zonations of Ethiopia. Competing interests
(2020). The authors declare no competing interests.
73. Dowle, M. & Srinivasan, A. data.table: extension of data.frame. R package
version 1.12.8. Available at: https://CRAN.R-project.org/package=data.table
(2019). Additional information
74. Kuhn, M. caret: classification and regression training. R package version 6.0- Supplementary information The online version contains supplementary material
85. Available at: https://CRAN.R-project.org/package=caret (2020). available at https://doi.org/10.1038/s42003-021-02463-w.
75. de Sousa, K., van Etten, J., Dumble, S., Greliche, N. & Steinke, J. gosset:
modelling metadata and crowdsourced citizen science. R package version Correspondence and requests for materials should be addressed to M.D.
0.2.1. Available at: https://agrobioinfoservices.github.io/gosset/ (2020).
Peer review information Communications Biology thanks Abdulqader Jighly and the
76. Bache, S. M. & Wickham, H. magrittr: a forward-pipe operator for R. R
other, anonymous, reviewer(s) for their contribution to the peer review of this work.
package version 1.5. Available at: https://CRAN.R-project.org/
= Primary Handling Editors: Brooke LaFlamme and Caitlin Karniski. Peer reviewer reportspackage magrittr (2014).
77. Wickham, H. et al. Welcome to the Tidyverse. J. Open Source Softw. 4, 1686 are available.
(2019).
78. Firth, D. qvcalc: Quasi Variances for Factor Effects in Statistical Models. R Reprints and permission information is available at http://www.nature.com/reprints
package version 1.0.1. Available at: https://CRAN.R-project.org/
package=qvcalc (2019). Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in
79. Hijmans, R. J., Phillips, S., Leathwick, J. & Elith, J. dismo: Species Distribution published maps and institutional affiliations.
Modeling. R package version 1.1-4. Available at: https://CRAN.R-project.org/
package=dismo (2017).
80. Hijmans, R. J. et al. raster: Geographic Data Analysis and Modeling. R package Open Access This article is licensed under a Creative Commons
version 2.5-8. Available at: https://cran.r-project.org/package=raster (2015). Attribution 4.0 International License, which permits use, sharing,
81. Pebesma, E. Simple Features for R: Standardized Support for Spatial Vector adaptation, distribution and reproduction in any medium or format, as long as you give
Data. R. J. 10, 439–446 (2018). appropriate credit to the original author(s) and the source, provide a link to the Creative
82. Strimas-Mackey, M. smoothr: Smooth and Tidy Spatial Features. R package Commons license, and indicate if changes were made. The images or other third party
version 0.1.2. Available at: https://CRAN.R-project.org/package=smoothr material in this article are included in the article’s Creative Commons license, unless
(2020). indicated otherwise in a credit line to the material. If material is not included in the
83. Wei, T. & Simko, V. R package “corrplot”: Visualization of a correlation article’s Creative Commons license and your intended use is not permitted by statutory
matrix. R package version 0.9. Available at: https://github.com/taiyun/corrplot regulation or exceeds the permitted use, you will need to obtain permission directly from
(2021). the copyright holder. To view a copy of this license, visit http://creativecommons.org/
84. Wickham, H. ggplot2: Elegant Graphics for Data Analysis. (Springer-Verlag licenses/by/4.0/.
New York, 2016).
85. Pedersen, T. L. patchwork: The Composer of Plots. R package version 1.0.0.
Available at: https://CRAN.R-project.org/package=patchwork (2019). © The Author(s) 2021
COMMUNICATIONS BIOLOGY |           (2021) 4:944 | https://doi.org/10.1038/s42003-021-02463-w |www.nature.com/commsbio 9