Technology in Society 67 (2021) 101741 Contents lists available at ScienceDirect Technology in Society journal homepage: www.elsevier.com/locate/techsoc Artificial intelligence, systemic risks, and sustainability Victor Galaz a,b,*, Miguel A. Centeno o, Peter W. Callahan c, Amar Causevic a,r, Thayer Patterson c, Irina Brass d, Seth Baum e, Darryl Farber f, Joern Fischer g, David Garcia h,m,q, Timon McPhearson a,b,i,p, Daniel Jimenez k,n, Brian King k, Paul Larcey l, Karen Levy j a Beijer Institute of Ecological Economics (Royal Swedish Academy of Sciences), Sweden b Stockholm Resilience Centre (Stockholm University), Sweden c Princeton Institute for International and Regional Studies (PIIRS), Princeton University, USA d Department of Science, Technology, Engineering and Public Policy (STEaPP), University College London, United Kingdom e Global Catastrophic Risk Institute, New York, USA f College of Engineering and the School of International Affairs, Pennsylvania State University, USA g Faculty of Sustainability, Leuphana Universitaet Lueneburg, Germany h Complexity Science Hub, Medical University of Vienna, Austria i Urban Systems Lab, New School, New York, USA j Department of Information Science, Cornell University, USA k CGIAR Platform for Big Data in Agriculture, Cali, Colombia l Department of Engineering, University of Cambridge, United Kingdom m Faculty of Computer Science and Biomedical Engineering, Graz University of Technology, Graz, Austria n Universidad Icesi, Cali, Colombia o School of Public and International Affairs (SPIA), Princeton University, USA p Cary Institute of Ecosystem Studies, Millbrook, New York, USA q Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Vienna, Austria r Stockholm Environment Institute (SEI), Stockholm, Sweden A R T I C L E I N F O A B S T R A C T Keywords: Automated decision making and predictive analytics through artificial intelligence, in combination with rapid Artificial intelligence progress in technologies such as sensor technology and robotics are likely to change the way individuals, Climate change communities, governments and private actors perceive and respond to climate and ecological change. Methods Sustainability based on various forms of artificial intelligence are already today being applied in a number of research fields Systemic risks Anthropocene related to climate change and environmental monitoring. Investments into applications of these technologies in Resilience agriculture, forestry and the extraction of marine resources also seem to be increasing rapidly. Despite a growing Social-ecological systems interest in, and deployment of AI-technologies in domains critical for sustainability, few have explored possible Automation systemic risks in depth. This article offers a global overview of the progress of such technologies in sectors with Digitalization high impact potential for sustainability like farming, forestry and the extraction of marine resources. We also identify possible systemic risks in these domains including a) algorithmic bias and allocative harms; b) unequal access and benefits; c) cascading failures and external disruptions, and d) trade-offs between efficiency and resilience. We explore these emerging risks, identify critical questions, and discuss the limitations of current governance mechanisms in addressing AI sustainability risks in these sectors. * Corresponding author. Beijer Institute of Ecological Economics (Royal Swedish Academy of Sciences), Sweden. E-mail address: victor.galaz@su.se (V. Galaz). https://doi.org/10.1016/j.techsoc.2021.101741 Received 5 May 2021; Received in revised form 31 August 2021; Accepted 6 September 2021 Available online 17 September 2021 0160-791X/© 2021 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). V. Galaz et al. T e c h n o l o g y i n S o c i e ty 67 (2021) 101741 1. Introduction of research, with analysis of new data and ask: Technological change is a fundamental component of scientific and a) Where in the world, and into which sectors directly relevant for economic breakthroughs [1], and has the potential to dramatically in- biosphere-based sustainability, is AI and associated technologies fluence global efforts toward sustainability [2,3]. As the pressure of diffusing? human activities increasingly shapes the biosphere and the climate b) Which systemic risks from a sustainability perspective could emerge system, so does the hope that artificial intelligence (AI)1 and associated as the result of this diffusion? technologies such as robotics and the Internet of Things (IoT), will be c) To what extent do current notions and principles related to able to increase societies’ capacities to detect, adapt and respond to “responsible AI” acknowledge systemic risks related to climate and environmental change [4–6]. Numerous reports highlight sustainability? how applications of AI and automation may help address climate change d) Which possible governance mechanisms could be developed to help and biodiversity loss, contribute to more effective monitoring and uses mitigate these risks? of natural resources, and further progress towards the achievement of the Sustainable Development Goals (SDGs) (e.g. Refs. [4,7,8]. Our ambition is not to conduct a systematic literature review, but to While increased applications of AI and associated technologies could bring together previously disconnected research fields (i.e. studies of the lead to more effective uses of land- and seascapes, augmented envi- wider social and economic implications of AI, research on systemic risk, ronmental monitoring capacities, and improved transparency in supply and the sustainability sciences) to help guide future research, and inform chains, it could also create new systemic sustainability risks as AI current policy debates about the governance of AI and its potential to technologies diffuse into new social, economic, and ecological contexts. help accelerate climate action. We conclude by posing broadly formu- Some recent syntheses have discussed these risks briefly (e.g. Refs. [7,9, lated research questions as a way lay the foundation for multi- and 10], yet their potential allocative harms [11] and unexpected social and transdisciplinary work across these diverse, and until now poorly con- ecological effects [12] are poorly elaborated, and more often than not, nected strands of research. overlooked. Prominent agenda-setting reports about the social impacts of AI, for example, either ignore sustainability dimensions altogether (e. 2. The growing importance of artificial intelligence for g. Ref. [13], or underemphasize their possible social, economic and sustainability ecological risks (e.g. Refs. [6,14–16]. In this article, we offer an overview and elaborate possible systemic AI-based technologies are gaining increased interest applied in a risksfor sustainability2 created by the diffusion of AI and associated number of research fields related to the environmental, sustainability technologies. Systemic risks – i.e. risks that evolve from networked in- and climate sciences. Examples include AI applications in climate and teractions in complex systems [17,18] – are of particular interest since Earth system modeling [25,26]; AI-augmented environmental moni- the application of AI-technologies in combination with globalization toring [27]; autonomous underwater marine conservation interventions processes, are likely to create novel connections between humans, ma- and data collection [28,29]; AI-supported tracking of illegal wildlife chines and the living planet including ecosystems and the climate sys- trade [30]; and “smart” urban planning for sustainable development tem. Such poorly understood human-nature-machine interactions [31–33]. increase the possibilities for disruptions that propagate through conta- The Royal Society in addition, has identified “digital twins” gion in key sectors of society, such as food, energy and commodity augmented through AI-analysis as key components of potentially plan- production systems dependent on resilient ecosystems and associated etary digital “control loops” for effective climate mitigation action, and ecosystem services [19]. more robust farming practices [22]. The ability of “digital twins” – that Here, we do not focus on known direct impacts such as the energy is, advanced digital replications of complex and evolving systems using consumption or the carbon footprint of deep learning and data-mining “big” real-time data - has gained increased attention in the sustainability [20], nor on opportunities for AI in helping address climate change domain. Such tools allow its users to simulate, explore, optimize and [21,22]. Our focus is instead on complementing this literature by help identify risks in various sectors related to sustainability ambitions, exploring networked risks that result from an increased connectivity including in infrastructure development, and resource consuming sys- between humans, machines and social-ecological systems. tems of various forms (e.g. energy and water) in e.g. cities [34–36]. Our empirical analysis and discussion focus exclusively on early The potential for AI and associated technologies also seems to be applications of AI and associated technologies in domains critical for driving a growing interest from the private sector. According to esti- what some have denoted biosphere-based sustainability [23]. That is, mates, nearly 12 million IoT sensors will be installed and in use on farms we focus on critical ecosystems such as agriculture and forestry along around the world by the year 2023 [37]. Agricultural technology with the technical infrastructure underpinning resource management (agtech) investment reached a new record of $1.5 billion in 2017, and and extraction. These living systems are often overlooked in current venture capital investment in the space has grown 80% annually since analyses of the connection between AI and sustainability, despite their 2012 [38]. The precision forestry market could grow from USD 3.9 fundamental importance for the climate system and human develop- billion in 2019, to reach USD 6.1 billion by 2024 [39]. With goals to ment [24]. Here, we combine literature from seldom connected strands improve urban livability and sustainability, planners could increasingly rely on AI for traffic management, smart policing, lighting control, facial recognition, and smart waste disposal systems [32,33]. The smart city market is expected to reach USD 460 billion by 2027 [40], smart city AI 1 Here we use the term “artificial intelligence/AI” to refer to technologies that software alone is projected to total USD 5 billion annually by 2025 [41], employ machine learning (ML) including “deep learning” (DL) methods (see and the market for robotics and autonomous systems in cities is expected Ref. [13]. We write “AI and associated technologies” in cases where AI is an to grow from 6.2 billion USD in 2018, to 17.7 billion USD in 2026 [33]. integrated part of a technology, such as a “smart tractor” or Unmanned Aerial Applications of AI and other associated technologies for sustain- Vehicles that employ computer vision. Hence our main interest in this paper is in the social and ecological impacts of AI and associated technologies, rather ability could be viewed as examples of technological “niche-in- than the underlying ML or DL technique per se. novations” capable of rapid upscaling and diffusion if followed by 2 By risk, we refer to the possibility of harm, commonly quantified as the increased investments, enabling legal conditions, and growing public product of the probability and severity of the harm [130]. By ‘sustainability’ we and consumer interest [42]. The COVID-19 pandemic seems to have refer specifically to the importance of the biosphere and a stable Earth system triggered a growing interest from the private sector and governments to for ongoing human development and prosperity [23,131]. accelerate digitization and automation in supply chains and other parts 2 V. Galaz et al. T e c h n o l o g y i n S o c i e ty 67 (2021) 101741 of the economy [43,44]. The diffusion of AI-technologies unfolds not prominent sector for the development and deployment of AI and asso- only through increased direct investments however, but also by the ciated technologies through digital farming/precision agriculture. This much less visible infusion of e.g. deep learning systems into existing is not surprising considering the very strong push internationally to- technologies [45]. wards increased production and reduced uses of scarce resources such as These converging trends suggest that the development and deploy- water through the application of new technologies and “digitalization” ment of AI and associated technologies are likely to not only have social [47–49]. and economic consequences, but will very likely impact climate, The differences in access to funding between different regions in the biodiversity, and ecosystems around the world [46]; Wynsberghe, world is notable, and follows the same pattern as other studies of the 2021). Their diffusion hence merits increased attention from a sustain- “digital divide” [50–52]. The prominent position of China in terms of ability perspective. investments (Fig. 1B) also seems to follow AI-investment patterns in Fig. 1 shows the geographical distribution of AI and associated general [53]; see also [54] for digital agriculture). technologies (here including applications of IoT, robotics and analysis supported by artificial intelligence) with a focus on companies and in- 3. Artificial intelligence, systemic risks and sustainability vestments in sectors linked to the management of the living planet, i.e. land- and seascapes. The data has been extracted from the international As we discussed in the previous section, there seems to be a growing technology company and investor database Crunchbase, with a specific interest, and increased investment in the development and deployment focus on companies operating in the selected sectors (see Supplementary of AI and associated technologies in sectors critical for sustainability. Information for methodological details). The technologies’ effectiveness and broader social, economic, and As the data shows, the agricultural sector seems to be the most ecological impacts however, unfold within a wider social, technological Fig. 1. Global distribution of AI technologies and investments in farming, forestry and the marine/aquaculture sectors. Fig. 1A. Geographical and sectoral distribution of companies that develop applications of IoT, sensors, robotics and AI-supported analytics for aquaculture, forestry and agriculture. Total number of companies N = 1114. Fig. 1B. Geographical distribution of investments in companies listed in 1A. See Supplementary Information for details about methods and data. 3 V. Galaz et al. T e c h n o l o g y i n S o c i e ty 67 (2021) 101741 and environmental context [55] making their distributional conse- bias. In this type of bias, an AI-system might be working as intended by quences and sustainability risks difficult to predict with specificity [56]. its designer, but the user does not fully understand its utility, or tries to In the following sections, we identify and explore four areas where infer different meaning that the system might not support. Developers of the use of AI and associated technologies in the pursuit of sustainability AI-support systems for digital agriculture, as an example, are still unable goals could give rise to systemic risks. These risks could, if not managed to convert complex geospatial information into appropriate crop man- proactively, unravel the progress and even decrease elements of sus- agement actions, resulting in misinterpretation and misuse of data. For tainability. These are related to a) algorithmic bias and allocative harms; example, many farmers utilize precision technology incorrectly to apply b) unequal access and benefits; c) cascading failures and external dis- more (instead of less) nitrogen (N) fertilizer in the hope of increasing ruptions; and d) trade-offs between efficiency and resilience. We also yields [49]. identify a number of important research questions to help advance our A contributing element to these bias types is a lack of appropriate understanding of sustainability risks created by AI and associated data. Data gaps can partly be tackled using satellites, drones, mobile technologies. devices, sensors and social media, and can be combined with various AI- While not an exhaustive list of the potential systemic risks from AI techniques to help overcome challenging scarce or incomplete data [31, technologies in this space, we view these as important starting points 71]. Increased data collection about systems and individuals result in that should be addressed by academia and policy-makers alike. their own challenges however. Urban sustainability scholars have already raised a number of issues related to AI and tentative threats to 3.1. Algorithmic bias and allocative harms privacy, research ethical challenges, and the risk of building decisions on spurious correlations [72]. For example, location-tracking systems The risks and impacts of possible algorithmic biases and their allocative via smartphones and vehicles make it possible to not only extract data harms (as defined by Ref. [11] has gained considerable attention in the that is helpful for urban planning purposes, but can also allow for the last years. As has been shown in other domains such as policing and the triangulation of a person’s identity and other personal information, even health sector (e.g. Refs. [57,58], inconsistencies and biases in training with sparse data. This highlights the need to match data collection for data, security breaches leading to corrupted data capture and sustainability goals with robust and transparent data management pol- decision-making systems, and flawed AI-models can have detrimental icies [31], and responsible innovation approaches [32]. impacts as AI-systems are applied. Whether from inappropriate training data, unsuitable contexts, or Growing volumes of environmental, social and ecological data are a user interpretation errors, algorithmic biases are common, and need to fundamental prerequisite for the application of artificial intelligence in, be thoughtfully considered in the sustainability domain. In the fields of for example, conservation and digital farming (e.g. Refs. [10,50]. agriculture, environment, and sustainability, such biases can result in Environmental and ecological data have well known limitations how- for example, risks to critical elements of food security and ecosystem ever, both in their temporal coverage, and geographical spread [59–61]. resilience. While the rapid growth of data from mobiles and satellites offer vast Key future questions: opportunities to map and respond to social vulnerabilities such as poverty and malnutrition, it has become increasingly clear that solutions • To what extent are insights and risk management solutions about algo- supported by “big data” and AI-analysis can be strongly skewed since the rithmic biases from other domains applicable to sectors such as digital “most disadvantaged people tend to be the least represented in new farming, digital forestry, urban planning and marine extraction and sources of digital data” [62]. management? Algorithmic biases of this sort can have a number of sources [63], • How is the predictive potential and efficacy of AI-models affected by the and may very well emerge in the sustainability domain in the following fact that ecosystems such as land- and seascapes are changing rapidly due ways: to e.g. climate change? Training data bias could emerge if AI-systems are designed with • Which social, economic and ecological impacts may result from these poor, limited, or biased data sets. For example, AI systems developed for biases, and how should these be prevented? precision agriculture in data poor contexts could - if not validated properly with local knowledge and expert opinion - result in incorrect 3.2. Unequal access, benefits, and impacts management recommendations to small-scale farmers who would struggle to maintain high, stable yields [64]. Resource constraints, and unequal access to information and Transfer context bias could emerge when AI-systems are designed communication technologies [51,52] create additional risks as for one ecological, climate, or social-ecological context, and then AI-technologies start to diffuse into new sectors. The growing interest in incorrectly transferred to another. While the training data and the digital, data-driven or precision farming is a good example of this. resulting model may be developed and suitable for the initial social- At present, smallholder farmers account for a considerable propor- ecological situation (say, a large industrial farm in a data rich tion of global food production [73], and especially in less wealthy context), using it in a different setting (e.g. a small farm) could lead to countries, many people depend on small-scale family-farms to meet their flawed and damaging results. Such bias may emerge, for example, as nutritional needs [74]. While applications of AI in combination with individuals and companies use off-the-shelf AI-software for their pur- increased automation for farming have been suggested to contribute to poses [65]. The use of simpler forest monitoring and carbon sequestra- increased yields and resource efficiency [47], the equitable distribution tion models has already led to controversies partly due to their tentative of such benefits cannot be taken for granted. Even non-AI technologies transfer context bias [66]. for intensifying agriculture are often deemed unaffordable by members The fact that ecosystems both on land and in the ocean are changing of poor local communities [75]. In addition, there is a clear “digital rapidly as the result of climate and ecological change [67] also pose divide” in data-driven farming with small-scale farmers facing serious serious challenges as AI-models, and lead to a type of concept drift [68]. obstacles to access big data and mobile technologies, which is likely to AI-systems built on historical ecological conditions hence are likely to distribute the benefits of these technologies in unequal ways [76]. fail as the ecosystems on land- and seascapes shift surprisingly and at Similar concerns and uncertainties about the tentative loss of times irreversibly. This latter phenomenon is well-known in ecology as employment opportunities resulting from increased automation [77] are “regimes shifts” which may emerge without prior warning with large present in these sectors as well of course [78]. While it might seem repercussions on ecosystems and those who depend on them [69,70]. premature to raise this as a possible risk, early studies indicate that the Even if both the training data, and the context in which the algorithm economic benefits of AI applications in farming appear to be greatest for is used is appropriate, their application can still lead to interpretation larger farms that can spread their fixed costs over many acres, and that 4 V. Galaz et al. T e c h n o l o g y i n S o c i e ty 67 (2021) 101741 can reduce labor costs through automation [49]. As a result, critics have as a whole (this issue is explored in more detail in the next section), argued that the growing interest in “digital agriculture” by influential especially if the components of the system are optimized and managed international actors such as the World Bank and the UN Food and properly (say, a regional network of IoT-connected farms). Agriculture Organization (FAO) overemphasize the need to increase Malicious external attacks can expose such endogenous vulnerabil- aggregate food production for a growing population, while ignoring ities as well, and even the most advanced AI-systems based on deep underlying well-known socio-political issues driving food insecurity neural networks are vulnerable to sabotage [93]. Connectivity and flows such as poverty and social inequalities [49,79], and the detrimental of information are prerequisites for the operation of AI-technologies in impacts on technological development resulting from corporate con- digital farming, forestry, and aquaculture, but also represent potentially centration in the food sector [80]. serious weak points in the system’s security. For example, digital Equal access to AI-technologies does not guarantee equal or fair farming systems and applications of AI for “smart cities” rely on data outcomes however. Even if farmers are able to optimize their specific transfer, sensor access to wireless and other communication networks, operations cost-effectively, widespread use of AI in farming may still remote transmission and system actuation, typically in real time [94]. result in concentration of capital and deepened inequality. As many Each of these can be disrupted intentionally and thus affect the opera- traditional input and equipment providers are increasingly positioning tion of e.g. semi-automated farming systems with both detrimental so- themselves as data companies, this accumulated information might be cial and ecological impacts [95,96], some of which may involve serious put to use to extract rents, lock farmers into unfavorable contracts, or data-breaches [97]. Box 1 elaborates this issue in more detail. price discriminate across services [48,81]. There are also concerns about These endogenous and exogenous risks created by novel human- the impacts of automation replacing jobs in these sectors, especially as it machine-ecological interactions have gained limited attention so far, could prove detrimental for vulnerable social groups such as migrant despite a growing interest and investments in these technologies. workers [38]. Small-scale fisheries and coastal communities (estimated Key future questions: to employ some 37 million people [82], and small-scale enterprises in the forestry sector (providing employment for an additional estimated • What cybersecurity risks could emerge in digital farming, forestry and 41 million people [82], may face similar challenges related to allocative other extractive sectors as AI-enhanced technologies gain prominence in harms, and unequal distribution of benefits as applications of these sectors? AI-technologies make their progress into their domains [28,83]. • What are the most important features of resilient infrastructures that Key future questions: would minimize the risks of cyberattacks and “normal accidents”, while also securing the integrity of production ecosystems such as • What are the possible distributional impacts that result from the increased agroecosystems? adoption of AI-technologies and automation in farming, forestry and other sectors related to the extraction of natural capital? 5. AI, efficiency and resilience • Which legal, economic and/or governance mechanisms can help prevent such distributional risks, and support the deployment of AI that is of Technological advances play a key role as societies strive for benefit to vulnerable groups in these sectors? increased control and productivity of ecosystems in both land- and seascapes as a means to secure human development [103]. The use of AI 4. Shocks, cascading failures and attacks and associated technologies in farming and other forms of extraction of natural resources such as sea food and biomass may very well lead to AI and associated technologies create numerous new complex in- increased efficiency and productivity, as often noted by prominent in- teractions not only between humans and machines, and machines and ternational organizations and think-tanks such as the World Bank [47]; machines [84], but also increasingly with machines and ecosystems, and Microsoft and Price Water House Cooper [16]; and the World Economic with the Earth system as a whole [2,55]. The addition of AI and asso- Forum [15]. Such efficiency gains could happen through data-driven ciated technologies into the worlds of agriculture and resource man- temporal and site-specific farm management, reduced waste, and the agement could be seen as adding more nodes and connections to these use of autonomous seeding or weed control, just to mention a few [104]. already complex social-ecological and socio-technical systems. While increased efficiency in resource use is not dangerous in and of The growing interactions between humans, machines, and ecology itself, and may well be desirable for engineered systems like energy and could be viewed through the lens of complex adaptive systems [85]. traffic systems, there are several potential downsides for living systems Such systems may through the use of AI and associated technologies, such as agricultural landscapes, forests, and marine ecosystems. The key contribute to the emergence of “distributed AI” (DAI) - decentralized issue is that optimizing system performance to maximize efficient gen- systems with the ability to bring together information and agents across eration of a small set of goods (say, a particular crop), often undermines levels and domains, at the same time as they (partly) autonomously overall system functioning and resilience over the long term [105]. As react, adapt and learn pro-actively to changing circumstances [86]. these systems become increasingly optimized and efficient, they also Applications of DAI for industrial purposes are well-known (e.g. become more brittle and vulnerable to undesirable so-called “regime Ref. [87], including in technical infrastructure such as energy systems shifts”, which are characterized by abrupt, unwanted, and sometimes [88,89]. These processes of decentralized adaptive problem-solving irreversible changes in a given ecosystem [70]. have also been observed for astonishingly complex yet resilient indige- Thus, for example, industrial agricultural landscapes around the nous farming systems in Bali [90], and could as proposed by some, be world now generate high yields of a few crop species, but have led to augmented and automatized through the extensive use of AI and asso- declines in many other ecosystem services also valued by societies, ciated technologies to support artificially intelligent curation of wild including biodiversity, scenic beauty, and climate or flood regulation places and nature (e.g. Ref. [91]. DAI could also, at best, help interpret [106]. Biodiversity in particular provides many functions directly rele- and respond to the complex systems properties and the continuous vant for the sustainable production of food, fuel and fiber, such as the changes that characterize farming, forestry and marine systems under decomposition of organic matter, pest control or pollination. Even when rapid change due to human activities and climate change. key species are maintained, declines in the diversity of crop and wild However, increasingly nested and complex systems are also suscep- species reduce the resilience of ecosystems making them increasingly tible to unexpected shocks, and cascades that develop endogenously, vulnerable to shocks such as a drought, or a newly introduced pest [19]. also known as “normal accidents” [92]. This implies that internal fail- Applications of AI and increased automation – including AI-systems ures can emerge unexpectedly and ripple and amplify across network that prioritize efficiency over redundancy and diversity - could accel- links (e.g. a regional food supply chain) and create failures in the system erate such loss of resilience. Since the economic benefits of automation 5 V. Galaz et al. T e c h n o l o g y i n S o c i e ty 67 (2021) 101741 Box 1 Cyberattacks in farming, food systems and ecosystem management Using sensors and other technologies to create increasingly accurate models of farms and ecosystems can produce valuable information for management and monitoring. “Virtual farms,” based on data from sensors, can be analyzed with AI algorithms for meaningful insights from management strategies to yield predictions [98]. These analyses require considerable amounts of computational power, which is rarely housed on the farm itself. Instead, valuable information is often transmitted, stored, and interpreted offsite using cloud storage and data analytics, and can be susceptible to data breaches at multiple stages [95,99]. These risks have been raised the last years (e.g. Ref. [100], and became highly visible in June 2021 when ransomware cyberattacks forced the shutdown of numerous meat plants in the U.S. [101]. The data and algorithms used in digital agriculture are also vulnerable to more traditional security risks. As recently as November of 2019, for example, an ex-employee of Monsanto with plans to sell information to a foreign government was indicted for economic espionage after being caught at the airport with copies of a software technology known as the “Nutrient Optimizer” [102]. This predictive algorithm is a critical component of an online platform, which collects, stores, and visualizes farming data from the field to increase productivity. While these pro- ductivity increases are important to seek out, it is critical to remember that using complex, remote, and potentially insecure technological networks can make valuable agricultural information available to nefarious actors around the globe. In the wrong hands, this information could have significant economic consequences, and the systemic risks of cybersecurity need to be managed effectively. and associated applications of AI and automation seem to be the greatest security. Principles-based guidelines have thus become the dominant for larger farms [50], investments in these technologies could create approach to governing AI systems. strong incentives for both larger and more simplified agricultural However, such principles have at least historically consistently landscapes [49], despite evidence that smaller farms tend to be most overlooked climate, sustainability and environmental dimensions. Owe productive and biodiverse over longer time periods [107]. The latter and Baum [117] for example, argue that AI ethics in general, have failed have proven to be key for the food security of communities in the most to give serious moral consideration to “nonhumans”, i.e. nonhuman fragile regions of the world [108]. animals and the environment. Fig. 2 summarizes our analysis of 186 In addition, local farming strategies, as well as social and ecological publicly available documents exploring principles for the benevolent use knowledge are often developed over generations. These contain of AI (see Supplementary Information for details about methodology). numerous social, cultural and even spiritual practices, some of which The data builds on strategic searches of keywords in the documents to have proven key to support the resilience of communities and the eco- assess the frequency of mentions of key dimensions of “responsible AI”, systems they manage in the face of changing social and environmental and sustainability respectively. We realize that this is a rough and circumstances [109,110]. Such tacit sources of knowledge and are not imperfect metric, but can still be used as an indication of the strong easily captured by data-driven approaches [111]. emphasis on social rather than environmental sustainability dimensions Simplification of ecosystems such as agricultural and forest land- of current discussions on “responsible” or “ethical AI”. scapes has been suggested to affect social relationships among people, Many of the principles related to algorithmic bias and transparency with the possible loss of local knowledge, which could lead to acceler- are nevertheless applicable for some of the sustainability risks identified ated loss of ecosystems [112,113]. These processes could undermine the in previous sections. If effectively implemented, these principles could foreseen benefits created by the use of AI-technologies. The economic help mitigate the risks of algorithmic biases such as transfer context bias, and technological logic of AI and their associated technologies could by incentivizing companies and governments to make sure AI-systems in hence be in conflict with the logic of resilient ecosystems. Assessing e.g. forestry are explainable and adaptive to changing climate condi- whether AI-applications lead to additional simplification empirically tions. Some of these principles have come to even include environmental however, will be challenging as changes in land-use and forest cover are and sustainability dimensions, such as the High-Level Expert Group on driven by a number of factors, many of which are not related directly to AI (HLEG) and their recommendation that “AI systems should be sus- technology [114]. tainable and benefit all human beings, including future generations” Key future questions: [44]. The fact that climate and environmental risks and costs tend to • Does the increased adoption of AI and associated technologies lead to systematically be externalized and challenging to quantify [118], may additional simplification, which may lead to a loss of resilience, of living very well undermine the economic and legal incentives of AI-developers systems such as agricultural landscapes and forest ecosystems? and users to implement such principles in practice if they are associated • How are local strategies and ecological knowledge likely to be affected by with costs. Critics of the current principles-based approach to AI an increased deployment of AI-technologies such as predictive analytics governance have emphasized a number of limits to operationalizing and automation? fairness, the practical limits of providing algorithmic explainability or • How can AI and associated technologies be developed and deployed in transparency, and the lack of professional accountability mechanisms ways that prioritize resilience over efficiency and simplification? needed to ensure their consistent implementation [119–121]. We sug- gest that issues of environmental sustainability pose distinctive chal- 6. “Responsible AI”, sustainability and governance lenges in what Weernart denotes “high-stake settings” (Weernart, 2021) for both people and nature, thus warranting dedicated attention and As we have discussed in previous sections, the development and further refinement of existing AI principles-based governance frame- deployment of AI and increased automation entail both opportunities for works, as well as more precise guidelines for how to implement and sustainability, but also numerous poorly explored systemic risks as continuously monitor their performance. humans, machines and ecosystems interact in new ways. Some of these These mechanisms could, at least in principle, evolve through sectors- risks could potentially be ameliorated through the application of prin- specific guidelines, product and process standards, or through new or ciples defining “ethical AI”, “responsible AI” or “AI for Good” that have amended legal-regulatory frameworks. emerged in the last years [115,116], especially those that address fair- Sector-specific guidelines, for example, are emerging in areas such as ness, non-discrimination, accountability, transparency, privacy and medical technology and digital manufacturing, but there have been 6 V. Galaz et al. T e c h n o l o g y i n S o c i e ty 67 (2021) 101741 Fig. 2. Summary of analysis of ethical principles of AI, or responsible AI from the public and private sector, including international organizations. Comment: Visu- alized numbers show frequency of mentions of key words found in published “responsible AI” principles. Selected keywords are related to core ethical princi- ples (gray columns), compared to key words related to sustainability (green columns). Number of docu- ments analyzed N = 186, see Supplementary Infor- mation for details about methods. (For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.) relatively few guidelines for areas related to sustainability. International either amendments to existing legal-regulatory frameworks in data organizations and the EU have expressed a commitment to responsible protection, safety and/or cybersecurity, new regulations to protect and trustworthy AI in the context of sustainable development through, consumers against algorithmic bias and provide transparency and for example, the proposed Artificial Intelligence Act [122]. These com- accountability, or increased oversight powers for existing or new regu- mitments however, are related to principles of non-discrimination, di- latory agencies [123], including independent audits of AI-systems versity, and inclusivity, rather than on responding to the specific [119]. Until now however, these regulatory proposals focus largely on dynamics between AI-based technologies and environmental sustain- individual risks (e.g. product safety regulations protecting the con- ability. For example, climate change and sustainability are only sumer), as opposed to systemic risks [124] that characterize the complex mentioned in passing in the Act, with “environmental sustainability” human-machine-ecological systems described here. In addition, ‘safety’ being suggested as one possible and voluntary “additional requirement” is consistently viewed from the perspective of individuals rather than by those developing AI systems (see p. 36, paragraph 81 in the Artificial from a wider environmental sustainability perspective (e.g. Ref. [119]. Intelligence Act). This creates a problematic governance gap that should be addressed. Standards-making organizations have also looked at ways to trans- The lack of adoption, enforcement, and commitment to govern sys- late ethical principles into product and process standards that ensure the temic sustainability risks created by AI becomes particularly problem- responsible development, deployment, and monitoring of AI systems. atic in the climate and environmental domain where strong regulatory Recent examples include: ISO/IEC TR 24028:2020 ‘Trustworthiness in and enforcement capacities cannot be taken for granted. Even though a Artificial Intelligence’; the IEEE ‘Ethics Certification Program for few industrialized countries see some reductions in e.g. climate emis- Autonomous and Intelligent Systems’; ISO/IEC 24028 ‘Bias in AI systems sions [125], neither the capacities of international institutions nor of and AI aided decision-making’; or BS 8611:2016 ‘Robots and robotic national governments have been able to address the continued erosion devices: Guide to the ethical design and application of robots and robotic of ecosystems, biodiversity and other critical natural capital [126]. systems.’ Again, these initiatives focus mostly on organizational gover- Existing legal frameworks and governance mechanisms in the environ- nance mechanisms and procedural guidance for managing known social mental and sustainability domain hence cannot be assumed to AI risks – such as lack of transparency and accountability – rather than compensate for the lack of robust and responsible governance of AI broader systemic considerations linked to the impact of these technol- systems and technologies. ogies on sustainability. Key future questions: In addition, these organizational procedures and considerations need to be further incorporated in emerging sectoral standards for smart 1. How can existing principles of “responsible AI” and similar, be leveraged farming, agricultural electronics or greenhouse gas management stan- to also advance sustainability ambitions? dards, such as ISO/TC207 - Environmental Standards or ISO/TC23 - 2. What governance mechanisms could support synergies between environ- Tractors and machinery for agriculture and forestry. Thus, systemic risk mental and technological regulation in ways that minimizes systemic considerations pertaining to the complex dynamics between AI tech- sustainability risks? nologies, ecological and environmental safety, supply chain resilience 3. How can such mechanisms be developed in ways that are adaptive to both and their wider distributional consequences for sustainability are rarely technological and environmental change, including climate disruptions featured in current standards packages. and surprises, at the same time? As AI and associated technologies continue to develop, proposals for their regulation have increased in recent years as well. These include 7 V. Galaz et al. T e c h n o l o g y i n S o c i e ty 67 (2021) 101741 7. Conclusion International and Regional Studies (Princeton University) for funding and hosting the workshop "Human-Machine-Ecology: A Workshop on Artificial intelligence, digitization and automation seem to be gain- the Emerging Risks, Opportunities, and Governance of Artificial Intel- ing traction in sectors of fundamental importance for sustainability. The ligence" at Princeton University on January 11th-12th, 2019, and the driving forces behind the diffusion of these technologies are the result of Consulate General of Sweden in New York for hosting the second both technological advancements, and societal and environmental workshop “Artificial Intelligence, People, and the Planet" in New York, pressures. On the technological side, leaps forward in predictive analysis on October 15th, 2019. We would also like to thank participants of these through various forms of AI-methods, IoT-systems, satellite technolo- events for their valuable input, the four anonymous reviewers for their gies, increasing computational capacity, and new developments in ro- constructive comments on earlier versions of this article, and Emilia botics industries, have paved the way for new approaches to efficiency, Arens for supporting the work with data extraction and analysis for productivity, and decision making under uncertainty. Secondly, de- Figure 1A and B.V. Galaz’s work was funded by the Beijer Institute of mands from society to better manage scarce natural resources and un- Ecological Economics (Royal Swedish Academy of Sciences) and the derstand the scope and impacts of rapid climate and environmental Stockholm Resilience Centre (Stockholm University) with support from change have also spurred research and development in this promising Zennström Philanthropies. D. Garcia’s work was supported by the field. As we have discussed here, however, this progress could (and Vienna Science and Technology Fund (Grant No. VRG16-005). K. Levy’s should) be matched with a growing recognition of not only opportu- work was supported by Microsoft. D. Farber’s work was supported by nities, but also possible systemic risks for sustainability. the College of Engineering, Penn State University. T. McPhearson was Our analysis shows that the most rapid development of AI and supported by the U.S. National Science Foundation through grants associated technologies in the sustainability domain, seem to be #1444755, #1934933, and #1927167 as well as the SMARTer Greener unfolding in farming, with substantial investments in these technologies Cities project through the Nordforsk Sustainable Urban Development in China and the United States in particular. As we discuss, such diffu- and Smart Cities grant program. sion could lead to new types of systemic risks resulting from various forms of algorithmic biases, distributional effects, and tentative net- Appendix A. Supplementary data worked vulnerabilities. These risks can partly be addressed through a growing number of principles and standards that govern the deployment Supplementary data to this article can be found online at https://doi. of AI, but need to be complemented with governance mechanisms that org/10.1016/j.techsoc.2021.101741. are able to integrate sustainability dimensions explicitly. Many of the risks discussed here are tentative, and difficult to Author statement quantify with precision. System risks that evolve out of complexity and poorly understood system interactions between humans, machine, and Conception or design of the work: VG, MC, TP, PC. Data collection: ecology are particularly challenging. In addition, the fact that both the VG, AC, PC, TP. Data analysis and interpretation: VG, AC. Drafting the development and use of these technologies are nascent makes it difficult article: VG, MC, TMP, PC, IB, SB, DF, DJ, BK. Critical revision of the to assess to what extent the risks identified are intrinsic to AI and article: IB, SB, DF, JF, DG, TMP, KL, DJ, BK, PL. Final approval of the associated technologies themselves, or the result of “pacing problems” version to be published: all. [127] created by novel uses of AI-technologies in new social and envi- ronmental contexts. References Our limited predictive abilities as these AI-risks diffuse into the sustainability domain requires what Shannon Valor calls “technomoral [1] B.W. Arthur, The Nature of Technology: what it Is and How it Evolves, Simon and humility” [128], but also to strike a balance between stringent and Schuster, New York, NY, 2009. [2] V. Galaz, Global Environmental Governance, Technology and Politics: the adaptive modes of governance. We suggest that governing AI risks for Anthropocene Gap, Edward Elgar Publishing, Cheltenham, 2014. sustainability due to the limited predictability created by systemic risks [3] F. Westley, P. Olsson, C. Folke, T. Homer-Dixon, H. Vredenburg, D. Loorbach, that emerge through human-nature-machine interactions are likely to J. Thompson, M. Nilsson, E. Lambin, J. Sendzimir, B. Banerjee, V. Galaz, S. van der Leeuw, Tipping toward sustainability: emerging pathways of transformation, require hybrid and highly adaptive approaches [129]. These need to be Ambio 40 (2011) 762–780, https://doi.org/10.1007/s13280-011-0186-9. developed with the capacity to respond to changes in the climate system, [4] J. Campbell, D. Jensen, A. Kim, D. Theresa, Building a Digital Ecosystem for the ecological systems, and advances in AI-technologies at the same time. Planet, 2019. Such governance approaches should in similar ways, as for other chal- [5] C. Herweijer, D. Waughray, Fourth Industrial Revolution for the Earth Harnessing Artificial Intelligence for the Earth, World Economic Forum, 2018. lenges characterized by complexity, bring together governmental and [6] L.N. Joppa, AI for Earth, Nature 552 (2017) 325–328, https://doi.org/10.1038/ private actors, as well as self-regulatory and mandatory regulatory in- d41586-017-08675-7. terventions to secure polycentric and flexible responses. Investors, [7] Future Earth, Digital disruptions for sustainability (D^2S) agenda– cross-cutting actions agenda, Environment 62 (2020) 30–41, https://doi.org/10.1080/ governments and the private sector should take these issues seriously as 00139157.2020.1750924. AI-augmented technologies are increasingly being promoted as a key [8] R. Vinuesa, H. Azizpour, I. Leite, M. Balaam, V. Dignum, S. Domisch, solution to a turbulent climate future. A. Felländer, S.D. Langhans, M. Tegmark, F. Fuso Nerini, The role of artificial intelligence in achieving the Sustainable Development Goals, Nat. Commun. 11 Future discussions about how to best govern these technologies from (2020) 233, https://doi.org/10.1038/s41467-019-14108-y. a sustainability perspective need to acknowledge the complex features [9] A. van Wynsberghe, Sustainable AI: AI for Sustainability and the Sustainability of of ecosystems, their fundamental importance for human development, AI, AI and Ethics, 2021, 0123456789, https://doi.org/10.1007/s43681-021- 00043-6. and the pressures they face under accelerating climate change. One key [10] O.R. Wearn, R. Freeman, D.M.P. Jacoby, Responsible AI for conservation, Nat. issue is the possible negative distributional implications of increased Mach. Intell. 1 (2019) 72–73, https://doi.org/10.1038/s42256-019-0022-7. applications of AI-technologies on social groups that depend directly on [11] S. Barocas, K. Crawford, A. Shapiro, H. Wallach, The problem with bias: from allocative to representational harms in machine learning, in: Special Interest the resources and services provided by these ecosystems on land- and Group for Computing, Information and Society, Philadelphia, 2017. seascapes. Hopefully this article can contribute to future discussions [12] V. Galaz, A.M. Mouazen, ‘New Wilderness’ Requires Algorithmic Transparency: a about how to better understand and govern AI risks for sustainability. Response to Cantrell et al, Trends Ecol. Evol. 32 (2017) 628–629, https://doi.org/ 10.1016/j.tree.2017.06.013. [13] House of Lords, AI in the UK: Ready, Willing and Able? Report by the UK House of Acknowledgement Lords, 2018. [14] ITU, Turning Digital Technology Innovation into Climate Action, International Telecommunication Union, Geneva, 2019. We would like to thank the Beijer Institute of Ecological Economics [15] World Economic Forum, Harnessing Artificial Intelligence for the Earth, Report, (Royal Swedish Academy of Sciences), and the Princeton Institute for 2018. 8 V. Galaz et al. T e c h n o l o g y i n S o c i e ty 67 (2021) 101741 [16] Microsoft and PricewaterhouseCoopers, How AI can enable a sustainable future, [45] E. Engström, P. Strimling, Deep learning diffusion by infusion into preexisting Available at: https://www.pwc.co.uk/sustainability-cli- mate-change/assets/pdf technologies – implications for users and society at large, Technol. Soc. 63 (2020) /how-ai-can-enable-a-sustainable-future.pdf, 2019. (Accessed 9 December 2020). 101396, https://doi.org/10.1016/j.techsoc.2020.101396. [17] M.A. Centeno, M. Nag, T.S. Patterson, A. Shaver, A.J. Windawi, The emergence of [46] P. Dauvergne, AI in the Wild: Sustainability in the Age of Artificial Intelligence, global systemic risk, Annu. Rev. Sociol. 41 (2015) 65–85, https://doi.org/ MIT Press, 2020. 10.1146/annurev-soc-073014-112317. [47] World Bank Group, Harnessing Digital Technologies to Improve Food System [18] D. Helbing, Globally networked risks and how to respond, Nature 497 (2013) Outcomes (Washington D.C), 2019. 51–59, https://doi.org/10.1038/nature12047. [48] J. Clapp, S.L. Ruder, Precision technologies for agriculture: digital farming, gene- [19] M. Nyström, J.B. Jouffray, A.V. Norström, B. Crona, P. Søgaard Jørgensen, S. edited crops, and the politics of sustainability, Global Environ. Polit. 20 (2020) R. Carpenter, V. Bodin, Galaz, C. Folke, Anatomy and resilience of the global 49–69, https://doi.org/10.1162/glep_a_00566. production ecosystem, Nature 575 (2019) 98–108, https://doi.org/10.1038/ [49] A. Lajoie-O’Malley, K. Bronson, S. van der Burg, L. Klerkx, The future(s) of digital s41586-019-1712-3. agriculture and sustainable food systems: an analysis of high-level policy [20] E. García-Martín, C.F. Rodrigues, G. Riley, H. Grahn, Estimation of energy documents, Ecosyst. Serv. 45 (2020) 101183, https://doi.org/10.1016/j. consumption in machine learning, J. Parallel Distr. Comput. 134 (2019) 75–88, ecoser.2020.101183. https://doi.org/10.1016/j.jpdc.2019.07.007. [50] B. Basso, J. Antle, Digital agriculture to design sustainable agricultural systems, [21] D. Rolnick, P.L. Donti, L.H. Kaack, et al., Tackling Climate Change with Machine Nat. Sustain. 3 (2020) 254–256, https://doi.org/10.1038/s41893-020-0510-0. Learning, 2019, pp. 1–111, arXiv 1906.05433. [51] K. Salemink, D. Strijker, G. Bosworth, Rural development in the digital age: a [22] The Royal Society, Digital Technology and the Planet - Harnessing Computing to systematic literature review on unequal ICT availability, adoption, and use in Achieve Net Zero, The Royal Society, United Kingdom, 2020, ISBN 978-1-78252- rural areas, J. Rural Stud. 54 (2017) 360–371, https://doi.org/10.1016/j. 501-1. jrurstud.2015.09.001. [23] C. Folke, R. Biggs, A.V. Norström, B. Reyers, J. Rockström, Social-ecological [52] United Nations Development Programme, Human Development Report 2019, resilience and biosphere-based sustainability science, Ecol. Soc. 21 (2016) 41. United Nations Development Program, New York, 2019. [24] C. Folke, S. Polasky, J. Rockström, et al., Our Future in the Anthropocene [53] D. Castro, M. McLaughlin, Who Is Winning the AI Race: China, the EU, or the Biosphere, Ambio, 2021, pp. 1–36. United States? - 2021 Update, Center for Data Innovation, 2021. Report). [25] S. Rasp, M.S. Pritchard, P. Gentine, Deep learning to represent subgrid processes [54] R. Birner, T. Daum, C. Pray, Who drives the digital revolution in agriculture? A in climate models, Proc. Natl. Acad. Sci. U. S. A 115 (2018) 9684–9689, https:// review of supply-side trends, players and challenges, Appl. Econ. Perspect. Pol. doi.org/10.1073/pnas.1810286115. (2021) 1–46, https://doi.org/10.1002/aepp.13145. [26] M. Reichstein, G. Camps-Valls, B. Stevens, M. Jung, J. Denzler, N. Carvalhais, [55] S.A. Markolf, M.V. Chester, D.A. Eisenberg, D.M. Iwaniec, C.I. Davidson, Prabhat, Deep learning and process understanding for data-driven Earth system R. Zimmerman, T.R. Miller, B.L. Ruddell, H. Chang, Interdependent infrastructure science, Nature 566 (2019) 195–204, https://doi.org/10.1038/s41586-019- as linked social, ecological, and technological systems (SETSs) to address lock-in 0912-1. and enhance resilience, Earth’s Futur 6 (2018) 1638–1659, https://doi.org/ [27] M. Hino, E. Benami, N. Brooks, Machine learning for environmental monitoring, 10.1029/2018EF000926. Nat. Sustain. 1 (2018) 583–588, https://doi.org/10.1038/s41893-018-0142-9. [56] P. Olsson, V. Galaz, W.J. Boonstra, Sustainability transformations: a resilience [28] P. Girard, T. Du Payrat, An Inventory of New Technologies in Fisheries, Oecd perspective, Ecol. Soc. 19 (2014) 1, https://doi.org/10.5751/ES-06799-190401. Green Growth and Sustainable Development Forum, Green Growth and [57] S. Barocas, A.D. Selbst, Big data’s disparate impact, Calif. Law Rev. 104 (2016) Sustainable Development (GGSD) Forum, Paris, 2017. 671–732. [29] J.A.C.C. Nunes, I.C.S. Cruz, A. Nunes, H.T. Pinheiro, Speeding up coral reef [58] Z. Obermeyer, B. Powers, C. Vogeli, S. Mullainathan, Dissecting racial bias in an conservation with AI-aided automated image analysis, Nat. Mach. Intell. 2 algorithm used to manage the health of populations, Science 366 (2019) (2020), https://doi.org/10.1038/s42256-020-0192-3, 292–292. 447–453, https://doi.org/10.1126/science.aax2342. [30] E. Di Minin, C. Fink, T. Hiippala, H. Tenkanen, A framework for investigating [59] L.N. Joppa, B. O’Connor, P. Visconti, C. Smith, J. Geldmann, M. Hoffmann, J.E. illegal wildlife trade on social media with machine learning, Conserv. Biol. 33 M. Watson, S.H.M. Butchart, M. Virah-Sawmy, B.S. Halpern, S.E. Ahmed, (2019) 210–213, https://doi.org/10.1111/cobi.13104. A. Balmford, W.J. Sutherland, M. Harfoot, C. Hilton-Taylor, W. Foden, E. Di [31] R.T. Ilieva, T. McPhearson, Social-media data for urban sustainability, Nat. Minin, S. Pagad, P. Genovesi, J. Hutton, N.D. Burgess, Filling in biodiversity Sustain. 1 (2018) 553–565, https://doi.org/10.1038/s41893-018-0153-6. threat gaps, Science 352 (2016) 416–418, https://doi.org/10.1126/science. [32] T. Yigitcanlar, J.M. Corchado, R. Mehmood, R.Y.M. Li, K. Mossberger, aaf3565. K. Desouza, Responsible urban innovation with local government artificial [60] A.A. Siddig, Why is biodiversity data-deficiency an ongoing conservation intelligence (AI): a conceptual framework and research agenda, Journal of Open dilemma in Africa? J. Nat. Conserv. 50 (2019) 125719. Innovation: Technology, Market, and Complexity 7 (1) (2021) 71. [61] T. Poisot, et al., Environmental biases in the study of ecological networks at the [33] Mark A. Goddard, et al., A global horizon scan of the future impacts of robotics planetary scale, BioRxiv (2020), https://doi.org/10.1101/2020.01.27.921429. and autonomous systems on urban ecosystems, Nat. Ecol. Evol. 5 (2) (2021) [62] J. Blumenstock, Don’t forget people in the use of big data for development, 219–230. Nature 561 (7722) (2018) 170–172, https://doi.org/10.1038/d41586-018- [34] B. Ketzler, V. Naserentin, F. Latino, C. Zangelidis, L. Thuvander, A. Logg, Digital 06215-5. twins for cities: a state of the art review, Built. Environ. 46 (4) (2020) 547–573. [63] D. Danks, A.J. London, Algorithmic bias in autonomous systems, in: IJCAI [35] X. Zhang, J. Shen, P.K. Saini, M. Lovati, M. Han, P. Huang, Z. Huang, Digital twin International Joint Conference on Artificial Intelligence, 2017, pp. 4691–4697, for accelerating sustainability in positive energy district: a review of simulation https://doi.org/10.24963/ijcai.2017/654. tools and applications, Front. Sustain. Cities 3 (2021), https://doi.org/10.3389/ [64] D. Jiménez, S. Delerce, H. Dorado, J. Cock, L.A. Muñoz, A. Agamez, A. Jarvis, frsc.2021.663269. June. A scalable scheme to implement data-driven agriculture for small-scale farmers, [36] Y. Ham, J. Kim, Participatory sensing and digital twin city: updating virtual city Glob. Food Sec. 23 (2019) 256–266, https://doi.org/10.1016/j.gfs.2019.08.004. models for enhanced risk-informed decision-making, J. Manag. Eng. 36 (3) [65] A. Chouldechova, A. Roth, The Frontiers of Fairness in Machine Learning, 2018, (2020), 04020005. pp. 1–13, arXiv 1810.08810. [37] A. Meola, Smart Farming in 2020: How IoT Sensors Are Creating a More Efficient [66] R.M. Ochieng, The Role of Forests in Climate Change Mitigation: A Discursive Precision Agriculture Industry, 2021-08-31, Business Insider, 2021, 2nd of Institutional Analysis of REDD+ MRV, Wageningen University, 2017. February 2021, available online: https://www.businessinsider.com/smart-far [67] R.J. Hobbs, E. Higgs, J.A. Harris, Novel ecosystems: implications for conservation ming-iot-agriculture?r=US&IR=T. and restoration, Trends Ecol. Evol. 24 (2009) 599–605, https://doi.org/10.1016/ [38] S. Rotz, E. Gravely, I. Mosby, et al., Automated pastures and the digital divide: j.tree.2009.05.012. how agricultural technologies are shaping labour and rural communities, J. Rural [68] A. Tsymbal, The Problem of Concept Drift: Definitions and Related Work, vol. Stud. 68 (2019) 112–122, https://doi.org/10.1016/j.jrurstud.2019.01.023. 106, Computer Science Department, Trinity College Dublin, 2004, p. 58, 2. [39] Markets and Markets, Precision Forestry Market by Technology (CTL, Geospatial, [69] A. Hastings, D.B. Wysham, Regime shifts in ecological systems can occur with no Fire Detection), Application (Harvesting, Silviculture & Fire Management, warning, Ecol. Lett. 13 (4) (2010) 464–472. Inventory & Logistics), Offering (Hardware, Software, Services), and Geography - [70] J.C. Rocha, G.D. Peterson, R. Biggs, Regime shifts in the anthropocene: drivers, Global Forecast to 2024, 2019. risks, and resilience, PLoS One 10 (2015), e0134639, https://doi.org/10.1371/ [40] Grand View Research, Smart Cities Market Analysis Report by Application journal.pone.0134639. (Governance, Buildings, Utilities, Transportation, Healthcare, Environmental [71] J.E. Blumenstock, Fighting poverty with data, Science 353 (2016) 753–754, Solution), by Region, and Segment Forecasts, 2019, pp. 2019–2025. https://doi.org/10.1126/science.aah5217. [41] Tractica, Artificial Intelligence Applications for Smart Cities - 23 Use Cases across [72] F. Creutzig, S. Lohrey, X. Bai, A. Baklanov, R. Dawson, S. Dhakal, W.F. Lamb, Six Smart City Sectors: Governance, Safety & Security, Mobility & Transportation, T. McPhearson, J. Minx, E. Munoz, B. Walsh, Upscaling urban data science for Energy & Resource Management, 2020 (Infrastructure Management, and global climate solutions, Glob. Sustain. 2 (2019) 1–25, https://doi.org/10.1017/ Healthcare). sus.2018.16. [42] F.W. Geels, B.K. Sovacool, T. Schwanen, S. Sorrell, Sociotechnical transitions for [73] B.E. Graeub, M.J. Chappell, H. Wittman, S. Ledermann, R.B. Kerr, B. Gemmill- deep decarbonization, Science 357 (2017) 1242–1244, https://doi.org/10.1126/ Herren, The state of family farms in the world, World Dev. 87 (2016) 1–15, science.aao3760. https://doi.org/10.1016/j.worlddev.2015.05.012. [43] World Economic Forum, The Future of Jobs Report 2020, World Economic [74] S.K. Lowder, J. Skoet, T. Raney, The number, size, and distribution of farms, Forum. Report, 2020. smallholder farms, and family farms worldwide, World Dev. 87 (2016) 16–29, [44] European Commission, Ethics Guidelines for Trustworthy AI [Text], 2019, April https://doi.org/10.1016/j.worlddev.2015.10.041. 8. https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trust [75] T.S. Jiren, I. Dorresteijn, J. Hanspach, J. Schultner, A. Bergsten, A. Manlosa, worthy-ai. N. Jager, F. Senbeta, J. Fischer, Alternative discourses around the governance of 9 V. Galaz et al. T e c h n o l o g y i n S o c i e ty 67 (2021) 101741 food security: a case study from Ethiopia, Glob. Food Sec. 24 (2020) 100338, [105] C.S. Holling, G.K. Meffe, Command and control and the pathology of natural https://doi.org/10.1016/j.gfs.2019.100338. resource management, Conserv. Biol. 10 (1996) 328–337, https://doi.org/ [76] Z. Mehrabi, M.J. McDowell, V. Ricciardi, C. Levers, J.D. Martinez, N. Mehrabi, 10.1046/j.1523-1739.1996.10020328.x. H. Wittman, N. Ramankutty, A. Jarvis, The global divide in data-driven farming, [106] J.A. Foley, R. DeFries, G.P. Asner, C. Barford, G. Bonan, S.R. Carpenter, F. Nat. Sustain. (2020), https://doi.org/10.1038/s41893-020-00631-0. S. Chapin, M.T. Coe, G.C. Daily, H.K. Gibbs, J.H. Helkowski, T. Holloway, E. [77] D. Acemoglu, P. Restrepo, Automation and new tasks: how technology displaces A. Howard, C.J. Kucharik, C. Monfreda, J.A. Patz, I.C. Prentice, N. Ramankutty, P. and reinstates labor, J. Econ. Perspect. 33 (2) (2019) 3–30. K. Snyder, Global consequences of land use, Science 309 (2005) 570–574, https:// [78] T. Daum, Farm robots: ecological utopia or dystopia? Trends Ecol. Evol. (2021). doi.org/10.1126/science.1111772. [79] A. Sen, Poverty and Famines: an Essay on Entitlement and Deprivation, Clarendon [107] V. Ricciardi, Z. Mehrabi, H. Wittman, D. James, N. Ramankutty, Higher yields and Press, Oxford, UK, 1982. more biodiversity on smaller farms, Nat. Sustain. (2021), https://doi.org/ [80] J. Clapp, The problem with growing corporate concentration and power in the 10.1038/s41893-021-00699-2. global food system, Nature Food 2 (6) (2021) 404–408. [108] C. Queiroz, A.V. Norström, Downing, et al., Investment in resilient food systems [81] A. Mateescu, M.C. Elish, AI in Context: the Labor of Integrating New in the most vulnerable and fragile regions is critical, Nature Food 2 (8) (2021) Technologies, Data & Society Research Institute, New York, 2018. 546–551. [82] Food and Agriculture Organization, United Nations Environment Programme, [109] F. Berkes, C. Folke, in: Linking Social and Ecological Systems: Management The State of the World’s Forests 2020, 2020, https://doi.org/10.4060/ca8985en. Practices and Social Mechanisms for Building Resilience, Eds, Cambridge [83] K.M. Bayne, R.J. Parker, The introduction of robotics for New Zealand forestry University Press, Cambridge, UK, 1998. operations: forest sector employee perceptions and implications, Technol. Soc. 34 [110] S. Barthel, C. Folke, J. Colding, Social–ecological memory in urban (2012) 138–148, https://doi.org/10.1016/j.techsoc.2012.02.004. gardens—retaining the capacity for management of ecosystem services, Global [84] I. Rahwan, M. Cebrian, N. Obradovich, J. Bongard, J.F. Bonnefon, C. Breazeal, J. Environ. Change 20 (2) (2010) 255–265. W. Crandall, N.A. Christakis, I.D. Couzin, M.O. Jackson, N.R. Jennings, E. Kamar, [111] D. Jiménez, H. Dorado, J. Cock, S.D. Prager, S. Delerce, A. Grillon, M.A. Bejarano, I.M. Kloumann, H. Larochelle, D. Lazer, R. McElreath, A. Mislove, D.C. Parkes, A. H. Benavides, A. Jarvis, From observation to information: data-driven ‘Sandy Pentland, M.E. Roberts, A. Shariff, J.B. Tenenbaum, M. Wellman, Machine understanding of on farm yield variation, PLoS One 11 (2016), e0150015, behaviour, Nature 568 (2019) 477–486, https://doi.org/10.1038/s41586-019- https://doi.org/10.1371/journal.pone.0150015. 1138-y. [112] M. Riechers, Á. Balázsi, L. Betz, T.S. Jiren, J. Fischer, The erosion of relational [85] T. McPhearson, D. Haase, N. Kabisch, Å. Gren, Advancing understanding of the values resulting from landscape simplification, Landsc. Ecol. 35 (2020) complex nature of urban systems, Ecol. Indicat. 70 (2016) 566–573, https://doi. 2601–2612, https://doi.org/10.1007/s10980-020-01012-w. org/10.1016/j.ecolind.2016.03.054. [113] S. ̌Sūmane, I. Kunda, K. Knickel, A. Strauss, T. Tisenkopfs, I. des I. Rios, M. Rivera, [86] F. Bouquet, S. Chipeaux, C. Lang, et al., Introduction to the agent approach, in: T. Chebach, A. Ashkenazy, Local and farmers’ knowledge matters! How Agent-based Spatial Simulation with NetLogo, Elsevier, 2015, pp. 1–28. integrating informal and formal knowledge enhances sustainable and resilient [87] H.V.D. Parunak, Applications of distributed artificial intelligence in industry, in: agriculture, J. Rural Stud. 59 (2018) 232–241, https://doi.org/10.1016/j. G.M.P. O’Hare, N.R. Jennings (Eds.), Foundations of Distributed Artificial jrurstud.2017.01.020. Intelligence: 139–164, Wiley Interscience, 1996. [114] Patrick Meyfroidt, et al., Middle-range theories of land system change, Global [88] A. Imteaj, M.H. Amini, J. Mohammadi, Leveraging decentralized artificial Environ. Change 53 (2018) 52–67. intelligence to enhance resilience of energy networks, in: 2020 IEEE Power & [115] J. Fjeld, N. Achten, H. Hilligoss, A. Nagy, M. Srikumar, Principled Artificial Energy Society General Meeting (PESGM), IEEE, 2020, August, pp. 1–5. Intelligence: Mapping Consensus in Ethical and Rights-Based Approaches to [89] V. Robu, D. Flynn, M. Andoni, M. Mokhtar, Consider ethical and social challenges Principles for AI, SSRN Electronic Journal, Cambridge, MA, 2020, https://doi. in smart grid research, Nat. Mach. Intel. 1 (12) (2019) 548–550, https://doi.org/ org/10.2139/ssrn.3518482. 10.1038/s42256-019-0120-6. [116] A. Jobin, M. Ienca, E. Vayena, The global landscape of AI ethics guidelines, Nat. [90] J.S. Lansing, Perfect Order - Recognizing Complexity in Bali, Princeton University Mach. Intell. 1 (2019) 389–399, https://doi.org/10.1038/s42256-019-0088-2. Press, 2012. [117] A. Owe, S.D. Baum, Moral Consideration of Nonhumans in the Ethics of Artificial [91] B. Cantrell, L.J. Martin, E.C. Ellis, Designing autonomy: opportunities for new Intelligence, AI and Ethics, 2021, 0123456789, https://doi.org/10.1007/s43681- wildness in the Anthropocene, Trends Ecol. Evol. 32 (3) (2017) 156–166. 021-00065-0. [92] C. Perrow, Normal Accidents: Living with High Risk Technologies, Updated [118] S. Polasky, K. Segerson, Integrating ecology and economics in the study of Edition, Princeton University Press, Princeton, NJ, 2011 https://doi.org/ ecosystem services: some lessons learned, Annu. Rev. Resour. Econ. 1 (1) (2009) 10.5465/amr.1985.4278477. 409–434. [93] D. Heaven, Why deep-learning AIs are so easy to fool, Nature 574 (7777) (2019) [119] G. Falco, et al., Governing AI safety through independent audits, Nat. Mach. Intel. 163–166, https://doi.org/10.1038/d41586-019-03013-5. 3 (7) (2021) 566–571. [94] J. West, A prediction model framework for cyber-attacks to precision agriculture [120] L. Haas, S. Gießler, V. Thiel, In the Realm of Paper Tigers – Exploring the Failings technologies, J. Agric. Food Inf. 19 (2018) 307–330, https://doi.org/10.1080/ of AI Ethics Guidelines [Internet], AlgorithmWatch, 2020 [cited 2021 Jan 19]. 10496505.2017.1417859. Available from: https://algorithmwatch.org/en/ai-ethics- guidelines-inventor [95] C. Cooper, Cybersecurity in food and agriculture, in: J. LeClair (Ed.), Protecting y-upgrade-2020/. Our Future, Hudson Whitman Excelsior College Press, Albany, NY, 2015. [121] B. Mittelstadt, C. Russell, S. Wachter, Explaining explanations in AI, in: [96] M. Gupta, M. Abdelsalam, S. Khorsandroo, S. Mittal, Security and privacy in Proceedings of the Conference on Fairness, Accountability, and Transparency, smart farming: challenges and opportunities, IEEE Access 8 (2020) 34564–34584, 2019, pp. 279–288. https://doi.org/10.1109/ACCESS.2020.2975142. [122] European Commission, Proposal for a Regulation of the European Parliament and [97] L. Cheng, F. Liu, D. Yao, Enterprise data breach: causes, challenges, prevention, of the Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial and future directions, Wiley Interdisciplinary Reviews: Data Min. Knowl. Discov. Intelligence Act) and Amending Certain Union Legislative Acts, 2021. Brussels, 7 (5) (2017) e1211. 21.4.2021 COM(2021) 206 final. Brussels. [98] K. Bronson, I. Knezevic, Big Data in food and agriculture, Big Data Soc 3 (2016) [123] O.J. Erdélyi, J. Goldsmith, Regulating artificial intelligence proposal for a global 1–5, https://doi.org/10.1177/2053951716648174. solution, in: J. Furman, G. Marchant, H. Price, F. Rossi (Eds.), ArXiv. Association [99] H. Chi, S. Welch, E. Vasserman, E. Kalaimannan, A framework of cybersecurity for Computing Machinery, 2020, pp. 95–101. approaches in precision agriculture, in: A.R. Bryant, J.R. Lopez, R.F. Mills (Eds.), [124] J. Black, A. Murray, Regulating AI and Machine Learning: Setting the Regulatory Proceedings of the 12th International Conference on Cyber Warfare and Security, Agenda, European Journal of Law and Technology, 2019. ICCWS 2017, Academic Conferences and Publishing International, 2017, [125] Le Quéré Corinne, et al., Drivers of declining CO 2 emissions in 18 developed pp. 90–95. economies, Nat. Clim. Change 9 (3) (2019) 213–217. [100] U.S. Department of Homeland Security, Threats to Precision Agriculture. 2018 [126] United Nations Development Programme, Human Development Report 2020 - Public-Private Analytic Exchange Program, 2018. https://www.dhs.gov/sites/de Human Development and the Anthropocene, United Nations Development fault/files/publications/2018%20AEP_Threats_to_Precision_Agriculture.pdf. Program, New York, 2020. [101] R. McCrimmon, M. Matishak, Cyberattack on Food Supply Followed Years of [127] L. Downes, The Laws of Disruption: Harnessing the New Forces that Govern Life Warnings, 2021. Politico, 2021-06-05, online: https://www.politico.com/news/ and Business in the Digital Age, Hachette, UK, 2009. 2021/06/05/how-ransomware-hackers-came-for-americans-beef-491936. [128] S. Vallor, Technology and the Virtues: A Philosophical Guide to a Future Worth [102] USDOJ, Chinese National Who Worked at Monsanto Indicted on Economic Wanting, Oxford University Press, 2016. Espionage Charges, OPA | Department of Justice.” United States Department of [129] I. Brass, J.H. Sowell, Adaptive governance for the Internet of Things: coping with Justice, 2019 https://www.justice.gov/opa/pr/chinese-national-who-worked- emerging security risks, Regul. Gov (2020), https://doi.org/10.1111/rego.12343. monsanto-indicted-economic-espionage-charges. [130] S. Kaplan, B.J. Garrick, On the quantitative definition of risk, Risk Anal. 1 (1) [103] L. Rist, A. Felton, M. Nyström, et al., Applying resilience thinking to production (1981) 11–27. ecosystems, Ecosphere 5 (2014), https://doi.org/10.1890/ES13-00330.1 art73. [131] W. Steffen, K. Richardson, J. Rockstrom, S.E. Cornell, I. Fetzer, E.M. Bennett, [104] R. Finger, S.M. Swinton, N. El Benni, A. Walter, Precision farming at the nexus of R. Biggs, S.R. Carpenter, W. de Vries, C.A. de Wit, C. Folke, D. Gerten, J. Heinke, agricultural production and the environment, Annu. Rev. Resour. Econ. 11 (2019) G.M. Mace, L.M. Persson, V. Ramanathan, B. Reyers, S. Sorlin, Planetary 313–335, https://doi.org/10.1146/annurev-resource-100518-093929. boundaries: guiding human development on a changing planet, Science 347 (80) (2015), https://doi.org/10.1126/science.1259855, 1259855–1259855. 10