Terms of Reference: Evaluability Assessments o f the Regional Integrated Initiatives, CGIAR July 2023 Photo: Crop Trust/CGIAR Genebanks Initiative Correct citation: CGIAR Independent Advisory and Evaluation Service (IAES). (2023). Evaluability Assessment of Regional Integrated Initiatives of CGIAR (RIIs), Terms of Reference. Rome: IAES Evaluation Function. https://iaes.cgiar.org/ Cover image: Sanjiv de Silva/IWMI ©2020 Contents List of Acronyms ............................................................................................................................................................ 5 Executive Summary ....................................................................................................................................................... 1 1 Background .............................................................................................................................................................. 4 1.1 Introduction to Regional Integrated Initiatives ............................................................................................................................... 4 1.2 Evaluability Assessments .............................................................................................................................................................................. 5 1.3 The Evaluability Assessment study and selection of RIIs to participate ....................................................................... 6 1.4 Organization of EA Team and the TORs ................................................................................................................................................ 7 2 Evaluability Assessment Purpose, Objectives, Scope, and Users ................................................................... 7 2.1 Purpose and Objectives ................................................................................................................................................................................... 7 2.2 Users of Results ..................................................................................................................................................................................................... 9 2.3 Scope and Timing .............................................................................................................................................................................................. 10 3 Evaluability Assessment Approach and Method ............................................................................................. 10 3.1 Approach ................................................................................................................................................................................................................. 10 3.2 Method ........................................................................................................................................................................................................................ 11 3.2.1 Step 1: Identify Purpose and Scope .......................................................................................................................................... 12 3.2.2 Step 2: Identify and Involve Stakeholders ........................................................................................................................... 12 3.2.3 Step 3: Apply the EA framework ................................................................................................................................................. 13 3.2.4 Step 4: Collect Evidence .................................................................................................................................................................. 14 3.2.5 Step 5: Assess readiness and make Recommendations ........................................................................................ 15 3.2.6 Step 6: Communicate and use of EA Results ................................................................................................................... 17 3.3 Management Response ................................................................................................................................................................................ 17 3.4 Consideration and Expected Limitations of the EA Process ................................................................................................. 18 4 Deliverables, Knowledge Management and Dissemination .......................................................................... 18 4.1 Sub-Reports from EA of each RII .............................................................................................................................................................. 18 4.2 Synthesis Evaluability Assessment Report ....................................................................................................................................... 19 4.3 Meta-Review Exercise of EA Process and Tool ............................................................................................................................. 20 5 Management, Roles and Responsibilities in the Evaluability Assessment Process .................................. 21 5.1 CGIAR Management Engagement and Response ..................................................................................................................... 21 5.2 IAES Management and Responsibilities ............................................................................................................................................. 21 5.3 Evaluability Assessment Team Role and Responsibilities .................................................................................................... 22 5.4 RII Team Participation and Responsibilities ................................................................................................................................... 24 5.5 Contractual and Payment Considerations .................................................................................................................................... 25 Annex 1: Background on Six Regional Integrated Initiatives ............................................................................... 26 Annex 2: GENDER Platform Evaluability Assessment – Executive Summary ................................................... 30 Annex 3: Evaluability Assessment Framework ..................................................................................................... 37 Annex 4: List of Stakeholders Consulted Toward TOR Development ................................................................. 40 Annex 5: Inventory of Documents and Reference Materials ................................................................................ 41 Annex 6: Requested points of Engagement and Tasks for the MEL or a Designated Focal Point in Evaluability Assessment ........................................................................................................................................... 42 List of Tables Table 1. Regional Integrated Initiatives of CGIAR, under the RAFS Science Group .................................................................. 4 Table 2: The four RIIs to undergo an EA .................................................................................................................................................................. 7 Table 3. Objectives of the Evaluability Assessment of the RII ................................................................................................................ 8 Table 4. User Groups to EA Results ........................................................................................................................................................................... 9 Table 5: Decision Support Framework for RIIs using Stoplight System, full EA Framework .............................................. 16 Table 6. Roles and Responsibilities of the EA Team, including IAES Management Function ......................................... 22 List of Figures Figure 1: Phases of the project cycle for timing EAs of RIIs ........................................................................................................................ 7 Figure 2. Six-step Process for Conducting Evaluability Assessments of RIIs in CGIAR ........................................................ 10 Figure 3. Six Domains of Evaluability Assessment Framework of CGIAR ....................................................................................... 11 Figure 4. Phases and Steps, along the Indicative Timeline for Evaluability Assessments of RIIs, 2023 ................... 12 List of Acronyms AMD CGIAR Initiative on Asian Mega-Deltas CRP CGIAR Research Programs EA Evaluability Assessment EF Evaluation Framework EP Evaluation Policy ESA Initiative on Diversification for resilient agribusiness ecosystems in East and Southern Africa FGD Focus Group Discussion CWANA CGIAR Initiatives on Fragility to Resilience in Central and West Asia and North Africa GI Genetic Innovation IAES Independent Advisory Evaluation Services KM Knowledge Management LAC CGIAR Initiative on AgriLAC Resiliente M&E Monitoring and Evaluation MEL Monitoring, Evaluation and learning MER Management Engagement and Response MELIA Monitoring, Evaluation, Learning and Impact Assessments MR Management Response PRMS Performance Results Management System QA Quality Assurance QoR4D Quality of Research for Development PCU Project Coordination Unit PPU Portfolio Performance Unit R4D Research-for-Development RAFS Resilient Agrifood Systems RIIs Regional Integrated Initiatives SIMEC Strategic Impact Monitoring and Evaluation Committee SDG Sustainable Development Goal ST Systems Transformation TAFFSA CGIAR Initiative on Transforming Agrifood Systems in South Asia ToC Theory of change ToRs Terms of Reference WCA CGIAR Initiative on West and Central African Food Systems Transformation Terms of Reference: RII Evaluability Assessments Executive Summary Background The CGIAR Independent Advisory and Evaluation Services (IAES) 2022‒24 multi-year workplan provides for the evaluability assessments (EAs) of Regional Integrated Initiatives (RIIs) in 2023. These Terms of Reference (ToRs) respond to the mandate of the CGIAR IAES Evaluation Function to execute the evaluability study, as well as set out the purpose, objectives and a process to follow, in accordance with the CGIAR Evaluability Assessment Guidelines. The CGIAR 2030 Strategy sets the stage for doing business differently to ensure that research provides real solutions for development. Towards meeting Sustainable Development Goal (SDG) targets, CGIAR strives for global and regional impact by organizing its work along three Action Areas: Systems Transformation (ST), Resilient Agrifood Systems (RAFS), and Genetic Innovation (GI). Under the RAFS Action Area, the RIIs aim to address unique national and regional challenges prioritized by stakeholders, and function as a key vehicle for the co-design and co-delivery of innovations, capacity development, and policy change with local and regional partners. The following six RIIs are deeply connected with local and regional partners and stakeholders, and are a mechanism for South-South learning exchange: • Fragility to Resilience in Central and West Asia and North Africa (F2R-CWANA) • AgriLAC Resiliente (LAC) • Asian Mega-Deltas (AMD) • Transforming Agrifood Systems in South Asia (TAFFSA) • Diversification in East and Southern Africa (ESA) • West and Central African Food Systems Transformation (WCA) Aligned to the central positioning of regional agrifood system challenges in the CGIAR 2030 Research and Innovation Strategy are four types of targeted users for EA process and results and the purposes for use: • CGIAR System Council (Commissioner)–Accountability, Steering • Management of RAFS Science Group and RIIs–Accountability, Learning • Internal RII stakeholders (CGIAR Project Coordination Unit (PCU)–Learning, Steering • External regional partners to RIIs, including bilateral projects–Learning, Accountability The goal of the EA is to support the RIIs in preparing for successful evaluations with the following specific objectives of the RIIs: • Objective 1: To assess the quality of the Monitoring, Evaluation and Learning (MEL) approach (framework) for the targeted RII, including theory of change (ToC), logic models, results framework, plans for evaluation and learning processes; and to assess the extent to which the MEL budget in place is sufficient to support the MEL framework implementation. • Objective 2: To facilitate reflection with RII staff on MEL framework and capacity within each RII, readiness for sound evaluation, and areas for improvement, including resource needs, based on EA results. • Objective 3: To gain clarity on requirements and resources needed to facilitate and support cost- effective and impactful evaluation of three Science Groups, including Quality of Research-for- Development (QoR4D) and their work on regional/national level RII programmes. • Objective 4: To synthesize learnings across all four RII EA reports to promote organizational learning, specifically for other RIIs in their effort to adjust their MEL framework and align with the ToC of RAFS. • Objective 5: To meta-review the EA process, and specifically the EA tool used to further develop and improve upon it for future use. 1 Terms of Reference: RII Evaluability Assessments The IAES/Evaluation workplan for 2023, reconfirmed by the System Council, included EAs of two RIIs. By design, the intent was to (1) facilitate learning and evaluability of the RIIs’ new implementation modality in the CGIAR portfolio; and (2) pilot the EA tool and EA process. Initial engagement between IAES and RAFS Action Area director helped establish the understanding of IAES/evaluation EA concepts, and potential uses for RAFS AA and wider CGIAR, as well facilitate their engagement with RIIs to discuss teams’ availability, workload, and interest for EAs processes. Originally, three initiatives out of six RIIs were recommended by RAFS Director [1]; the 4th RII was recommended for addition by IAES/Evaluation based on explicit interest assessed from bilateral engagement, funding, and capacities-see Table 4. To summarize, guided by IAES budget considerations and the RAFS Action Area director, and subsequently RII coordinator (appointed July 2023), the following criteria were applied: (1) status of the Initiative In project cycle and funding levels; (2) explicit interest and/or availability from the RIIs; and (3) geographical representation, with potential for deeper learning at regional level. Four selected initiatives include: 1. Fragility to Resilience in CWANA–Central and West Asia and North Africa: this Initiative aims to respond to the climate, nutrition and agrifood challenges most affecting the CWANA region. This is done by applying, scaling, and supporting effective, resilience-focused solutions, reducing fragility and conflict, and empowering all stakeholders for change, while minimizing and/or mitigating any trade- offs. Countries: Morocco, Egypt, Lebanon, Sudan and Uzbekistan. 2. AMD-Southeast Asia and the Pacific: this Initiative aims to create resilient, inclusive, and productive deltas, which maintain socio-ecological integrity, adapt to climatic and other stressors, and support human prosperity and wellbeing. This is achieved by removing systemic barriers to the scaling of transformative technologies and practices at community, national and regional levels. Countries: Vietnam, Bangladesh and Philippines. 3. TAFFSA–South Asia: Working across South Asia, this Initiative aims to deliver a coordinated program of research and engagement across the food production–to–consumption continuum to support equitable access to sustainable healthy diets, improve farmer livelihoods and resilience, and conserve land, air, and groundwater resources. Countries: India, Bangladesh, Nepal and Pakistan. 4. West and Central African Food Systems Transformations (WCA)-West and Central Africa: This Initiative aims to improve nutrition, income and food security within the context of climate change in WCA through nutritious, climate-adapted, and market-driven food systems. Countries: Cote d’Ivoire, Nigeria, and Congo. EA Approach and Method The EA approach will be utilization-focused and collaborative, as well as focus on accountability and learning. In line with the scope and EA objectives, and within the timeline provided, the expanded EA Framework of CGIAR will be used: across six assessment domains, to answer 40 questions about the following: intervention logic, MEL systems and resources, gender, diversity, and inclusion; long-term evaluability; context and environment; and management and key stakeholder engagement and support. The suggested approach will use mixed methods of data collection and analysis: (1) documents and data reviews: a desk review of secondary data sources, documents and related resources; and (2) primary data collection: key informant interviews, and focus group discussions, both in-person and remote. As a process- oriented evaluative exercise, the EA has a method, which would involve six steps: (1) identify purpose and scope; (2) identify and involve stakeholders; (3) apply the EA framework; (4) collect evidence; (5) assess readiness and make recommendations; and (6) communicate and use of EA results. The key deliverables of the EA process are the following: (1) four EA sub-reports, for each RII assessed; (2) EA synthesis report, combining information from the four sub-reports above; and (3) Meta-review paper, adjusted EA tool and other relevant documents. 2 Terms of Reference: RII Evaluability Assessments In response to the EA purpose and objectives, throughout the implementation of EA, IAES has and will continue to liaise with RII management and leadership, as well as the Project Coordination Unit (PCU and Portfolio Performance Unit (PPU) to coordinate the development of the Management Response (MR) within a stipulated timeframe. EA Team Role and Responsibilities: Under the oversight, management, and guidance of IAES, the EAs will be conducted by an EA team of evaluation experts, each with specific roles and responsibilities assigned for the realization of the smooth execution of this mandate. Quality Assurance (QA) will be provided in line with IAES processes ensuring the quality, consistency, and soundness of all EA deliverables to IAES. 3 Terms of Reference: RII Evaluability Assessments 1 Background The CGIAR Independent Advisory and Evaluation Services (IAES) 2022‒24 multi-year workplan (2021 Decision Reference SC/M14/DP4, re-confirmed in 2022) provides for the selected evaluability assessments of Regional Integrated Initiatives in 2023. The CGIAR IAES Evaluation Function executes the evaluative study consistent with its mandate set in the 2018 IAES ToR. 1.1 Introduction to Regional Integrated Initiatives Research and innovation targeted at complex regional agrifood system challenges are central to the CGIAR 2030 Research and Innovation Strategy. CGIAR strives for global and regional impact by organizing its work along three Action Areas in which accelerated innovation is required to create sustainable and resilient food, land, and water systems and to meet SDG targets. The three Action Areas, which build on the firm foundation of CGIAR’s traditional strengths in genetics and farming systems with a more ambitious agenda around food, land, and water systems, are: Systems Transformation (ST)ST, Resilient Agrifood Systems (RAFS)RAFS, and Genetic Innovation (GI)GI. The 2030 Strategy sets the stage for doing business differently to ensure that research provides real solutions for development. CGIAR is changing the way it works, following seven new implementation approaches,1 including 'Positioning regions, countries, and landscapes as central dimensions of partnership, worldview, and impact.’ Under RAFS Action Area, the aim of the RIIs is to address unique national and regional challenges prioritized by stakeholders, and function as a key vehicle for the co-design and co-delivery of innovations, capacity development, and policy change with local and regional partners. Building on experience and lessons from systems research, RIIs in their first year of implementation (2022) laid a foundation for on-the-ground research, coordination, and synergies across CGIAR to amplify impact at scale. As they are deeply connected with local and regional partners and stakeholders, RIIs are a mechanism for South-South learning exchange.2 Table 1 lists the six RIIs, and additional information on the RII portfolio is provided in Annex 1. Table 1. Regional Integrated Initiatives of CGIAR, under the RAFS Science Group Initiative Initiative title Intervention countries with activities (per proposals) code INIT-10 1. Fragility to Resilience in Central Egypt, Lebanon, Morocco, Sudan and Uzbekistan and West Asia and North Africa (F2R-CWANA) INIT-14 2. AgriLAC Resiliente (LAC) Colombia, El Salvador, Guatemala, Honduras, Mexico, Nicaragua, and Peru INIT-18 3. Asian Mega-Deltas (AMD) Bangladesh, India, Cambodia, Myanmar, and Vietnam INIT-20 4. Transforming Agrifood Systems Bangladesh, India, Nepal, and Pakistan in South Asia (TAFFSA) 1 Other six ways of working include: (1) Embracing a systems transformation approach, seeking multiple benefits across five SDG-linked Impact Areas; (2) Leveraging ambitious partnerships for change in which CGIAR is strategically positioned (3) Generating scientific evidence on multiple transformation pathways (4) Targeting risk-management and resilience as critical qualities for food, land, and water systems; (5) Harnessing innovative finance to leverage and deliver research through new investment and funding models; and (6) Making the digital revolution central to our way of working 2 Regional Integrated Initiatives - CGIAR 4 Terms of Reference: RII Evaluability Assessments Initiative Initiative title Intervention countries with activities (per proposals) code INIT-21 5. Diversification in East and Eswatini, Ethiopia, Kenya, Madagascar, Malawi, Mozambique, Southern Africa (ESA) Rwanda, Uganda, South Africa, Tanzania, Zambia, and Zimbabwe INIT-22 6. West and Central African Food Burundi, Democratic Republic of the Congo, Ghana, Ivory Systems Transformation Coast, Nigeria, and Rwanda. (WCA) Source: https://www.cgiar.org/research/initiatives-dashboard/ (July 2023). 1.2 Evaluability Assessments The evaluation guidelines Conducting and Using Evaluability Assessments in CGIAR operationalize CGIAR- wide Evaluation Framework and Policy (2022), specifically evaluability-one of 15 principles of the CGIAR Evaluation Framework. The Framework is a reference point for professionalism within research-for- development (R4D) evaluation that underpins how evaluation is conducted in CGIAR. Evaluation is defined in the Framework and Policy as the systematic and objective assessment of an ongoing or completed project, program, initiative policy or operational modality in CGIAR, as well as its design, implementation, and results. Rigorous, independent, external evaluations are foundational to CGIAR’s effort to inform the design of interventions, provide actionable evidence to support management and governance decisions, and ensure a high level of accountability to donors. The inclusion of the evaluability principle recognizes that meeting this potential requires advanced planning and appropriate evaluative inputs. At CGIAR, EAs contribute to learning and steering, by providing evidence-based recommendations to enhance the evaluability of initiatives, and the portfolio at large. An EA conducted early in project development and implementation helps to ensure that the subsequent formative and/or summative evaluation3 of an intervention (independent or decentralized) will be cost-effective, result in a useful learning process and in findings that inform decisions and improve performance. Often an EA will focus on supporting sound portfolio and/or intervention logic, the further development of a monitoring and evaluation (M&E) framework for a project, a sound approach to data collection for the purpose of M&E, and the potential usability of evaluation results by stakeholders. EAs are relatively new to CGIAR. However, they have been increasingly recognized in the evaluation profession internationally over the past 30 years and are considered a best practice in preparing projects and programs for effective and impactful evaluations. The Performance Results Management Study (PRMS), which was jointly conducted by the Evaluation Function of IAES and internal audit of CGIAR (to be published Q3 2023), aimed to address evaluability. One of the two objectives of the study was to independently identify and highlight inherent and emerging risks to ability of the redesigned PRMS to meet stakeholder expectations in each phase of design and implementation. Targeted sub-objectives also addressed the suitability of the PRMS products to facilitate evaluability, e. g., the CGIAR Results Dashboard. The study was concluded with internal and external practices and mechanisms to improve evaluability of CGIAR initiatives and portfolio. Furthermore, recommendation seven in the PRMS Study addresses evaluability: “Assess existing Monitoring, Evaluation, Learning and Impact Assessments (MELIA) structures, processes, and resources to leverage strengths and lessons learned from the CGIAR Research Programs 3 Michael Scriven coined the terms “formative evaluation” and “summative evaluation” in his 1967 publication, The Methodology of Evaluation. Scriven used the term “formative evaluation” to refer to evaluation activities conducted during the development process of a programme or product, with the aim of improving its quality. The term “summative evaluation” was used to refer to the evaluation of the programme or product at the end of the development process, with the aim of determining its overall value, merit, significance and worth. 5 Terms of Reference: RII Evaluability Assessments (CRP) reporting towards meeting commitments and overall evaluability of the portfolio. Formalize identification and documentation of good and inhibiting practices and factors to reinforce the agility and usefulness of the PRMS in line with One CGIAR governance arrangements.” Box 1: How is Evaluability understood within the Evaluation Field? The evaluation field broadly accepts the definition of evaluation put forward by theorist Michael Scriven: “Evaluation is the systematic process to determine merit, worth, value or significance” (1991). Another notable theorist in the evaluation field, Michael Quinn Patton, describes a certain kind of evaluation- program evaluation as “the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future programming” (Patton, 1997, p. 23). The term evaluability was first coined by Joseph Wholey and his colleagues at the Urban Institute in Washington, DC, in the 1970s, as a response to the low quality of evaluations of US Government programs. EAs became a means by which to examine a program’s structure to determine whether it could lend itself to generating useful results from an outcome evaluation. EAs were also viewed as a pre-formative evaluation activity that was part of a cost-effective strategy in determining readiness for evaluation and enhancing use. As Scriven (1991) commented, evaluability is analogous to requiring serviceability in a new car, and that it may be thought of as “the first commandment in accountability” (Scriven, 1991, p. 138). In CGIAR, the Evaluation Function under IAES has conducted two EAs to date, in accordance with the core domains of the EA Framework in the EA Evaluation Guidelines. EAs were conducted for the GENDER Platform in 2022 (see CGIAR GENDER Platform Evaluation: Inception Report (2023), and Annex 2: GENDER Platform Evaluability Assessment–Executive Summary (in this document); and the Genebank Platform in 2023 (see reference in Evaluation of the Genebank Platform: Inception Report (2023). 1.3 The Evaluability Assessment study and selection of RIIs to participate The EA study described in this ToRs is in three parts: 1) The conduct of EAs for chosen RIIs; 2) the synthesis of the EA findings and lessons learned as a group of CGIAR interventions; and 3) a meta-review of the CGIAR’s EA framework exploring its use and effectiveness as experienced by stakeholders, along with suggestions for further refinement of the approach and tool. IAES/Evaluation workplan for 2023, reconfirmed by System Council, included evaluability assessments of two RIIs. By design the intent was to (1) facilitate learning and evaluability of the RIIs- new implementation modality in CGIAR portfolio; and (2) pilot the EA tool and EA process. for the. Initial engagement between IAES and RAFS Action Area director helped establish the understanding of IAES/evaluation EA concepts, and potential uses for RAFS AA and wider CGIAR, as well facilitate their engagement with RIIs to discuss teams’ availability, workload and interest for EAs processes. Originally, three initiatives out of six RIIs were recommended by RAFS Director[1]; the 4th RII was recommended for addition by IAES/Evaluation based on explicit interest assessed from bilateral engagement, funding and capacities - see all outlined in Table 4. To summarize, guided by IAES budget considerations and the RAFS Action Area director, and subsequently RII coordinator (appointed July 2023), the following criteria were applied: (1) status of the Initiative In project cycle and funding levels; (2) explicit interest and/or availability from the RIIs; and (3) geographical representation, with potential for deeper learning at a regional level. [1] See Annex 1: Background on Six Regional Integrated Initiatives and RII portfolio on Regional Integrated Initiatives 6 Terms of Reference: RII Evaluability Assessments Table 2: The four RIIs to undergo an EA Initiative title Countries of activities Lead/co-lead (Center, Coordinator/MELIA location) 1. F2R- Morocco, Egypt, Lebanon, Lead (ICARDA-MENA Morocco), MEL (IA) CWANA Sudan, Uzbekistan. Co-Lead (IWMI-Jordan) (ICARDA -Ireland and Uganda) 2. AMD Bangladesh, Cambodia, Lead (IRRI-Vietnam), MEL Coordinator (IRRI- India, Myanmar, Vietnam. Co-lead (WorldFish- Philippines) Bangladesh) 3. TAFFSA India, Bangladesh, Nepal, Lead (CIMMYT-Bangladesh), MEL (IA) Pakistan. Co-lead (IFPRI-New Delhi) (CIMMYT-Bangladesh) 4. WCA Cote d’Ivoire, Nigeria, Lead (Africa Rice-Cote d’Ivoire), MEL(IA) Ghana, Congo, Rwanda and Co-lead (IITA-Nigeria) (IITA-Congo) Burundi. The other two initiatives (ESA and LAC, Table 1) will be subject to a facilitated application of EA framework, as part of 2024 evaluation of Science Groups. 1.4 Organization of EA Team and the TORs This ToR facilitates the implementation of the EAs of Regional Integrated Initiatives at CGIAR by setting out purpose, objectives and a process to follow, in accordance with the CGIAR Evaluability Assessment Guidelines. It also supports the implementation of an evaluability synthesis to identify lessons learned from the assessments; and a meta-evaluative process to reflect on the results of the four EAs, lessons learned about their implementation, stakeholder experience and observations, and to formulate any recommendations for further refinement of the IAES EA tool. This ToR is organized into five sections. Section two outlines the EA purpose, objectives scope and users for the ToR. Section three maps out the approach and methods to be used, while Section four details learning and dissemination of results. The EA team implementing the ToR, in cooperation with the RIIs, is comprised of three persons. Two team members are external evaluation consultants (an EA advisor and an EA lead), supported by evaluation analyst from IAES/Evaluation Function. The Evaluation Function lead of IAES will guide, manage and provide QA to the EA process. The roles and responsibilities of each team member is detailed in section five. 2 Evaluability Assessment Purpose, Objectives, Scope, and Users 2.1 Purpose and Objectives Aligned to the EA Guidelines for CGIAR, the assessment’s primary purpose is to ensure evaluability and preparedness for an evaluation, aligned to different placing in the project cycle. Figure 1: Phases of the Project Cycle for Timing EAs of RIIs 7 Terms of Reference: RII Evaluability Assessments Source: Evaluability Assessment Guidelines (2022). The EAs for the four RIIs will focus on: 1) carrying out the EA, resulting in findings, conclusions, and recommendations; 2) supporting RII teams in understanding and implementing the recommendations (EA results); and 3) assisting RII leadership consider resourcing of MEL going forward. Table 2 presents the objectives, which reflect these three areas of focus, with Objective 2 indicating the kind of participatory process and focus on capacity development to be adopted. These ToR also has several organizational- level objectives for the four RII EAs. These include: (1) a focus on promoting learning across the four RII EA results, in the form of a synthesis; and (2) a meta-evaluative exercise aimed at evaluating the EA approach, specifically the EA tool used. The RII EA experience will support further reflection on EA tool use and possible adjustments going forward. Table 3. Objectives of the Evaluability Assessment of the RIIs by Target Group4 Targeted Objective N Narrative audience To assess the quality of the MEL approach (framework) for the targeted RII, including ToC, logic models, results framework, plans for evaluation, learning Objective 1 processes, as well as the extent to which the MEL budget in place is sufficient to support the MEL framework implementation. Four RIIs, RII To facilitate reflection with RII staff on MEL framework and capacity within each RII, coordinator Objective 2 readiness for sound evaluation, and areas for improvement, including resource needs, based on EA results. To gain clarity on requirements and resources needed to facilitate and support Objective 3 cost-effective evaluations5 of three Science Groups, including QoR4D and their work RIIs, on regional/national level RII programmes. 4 Regional Integrated Initiatives 5 By design, the main stated purpose of EAs is to support programs to better establish a program logic and a means for monitoring and evaluating outcomes and long-term impact, critical for a process and performance, as well as impact evaluation: EA Framework’s Domain A: Intervention logic: To be evaluable, an intervention must clearly describe what it hopes to achieve and how. Intervention logic represents the overall logical integrity of the intervention. This logic should be supported by a robust ToC and a body of evidence that lends credibility/ plausibility to the ToC. As part of applying EA framework, reviews would consider the M&E plans, MELIA plans if applicable, to which extent they relate to TOC and how they would support evaluations at a later stage. Successful implementation of evaluations that are effectively planned for and designed to identify long-term outcomes and impact can then support programs to understand how well they are progressing toward achieving their desired impact (formative evaluation) and the extent to which they have made an impact, and the nature of that impact (summative and ex ante evaluations). 8 Terms of Reference: RII Evaluability Assessments Targeted Objective N Narrative audience To synthesize learnings across all four RII EA reports to promote organizational RAFS Science Objective 4 learning, specifically for other RIIs in their effort to adjust their MEL framework and Group, PCU, align with the ToC of RAFS. To meta-review the EA process and specifically the EA tool used to further develop CGIAR-wide, Objective 5 and improve upon it for future use. IAES, PPU As discussed above, the goal of the assessment is not to make an evaluative judgment but to support the RIIs in preparing for successful evaluations, which can involve assessing the adequacy of the ToC, establishing the interests of the stakeholders, and determining whether available data would make it possible to formulate a judgment given the initiative timeframe and resources. An assessment will help determine which evaluation type is warranted, the status of implementation, and the availability of evidence used to make evaluative judgments. 2.2 Users of Results There are four types of targeted users for EA process and results. The commissioner of EAs, the CGIAR System Council, is a major user of EAs, as is the RII management who will use the results for furthering their approach to evaluation and increasing levels of evaluability within programming. Additional users who have a stake in the EA and the ability of RIIs to be evaluable include RII partners within CGIAR and external partners who are affiliated with and have a stake in the outcome of RIIs. CGIAR PCU may also be among key users, given its roles in project management and monitoring tools. Based on the scoping phase, stakeholder and user needs will be elaborated and articulated in the inception note (see EA Methods Section below) as part of a stakeholder mapping processing. Table 4. User Groups to EA Results Purpose vis-à-vis EF Targeted user Rationale for prioritizing user-groups /EP A. CGIAR System - The System Council commissions and funds all Accountability, Council independent and external evaluations and Steering (commissioner) evaluative activities that IAES executes. B. Management of - Enhance systematic approaches across MEL for Accountability, RAFS Science RIIs, and within RAFS Learning Group and RIIs - The evaluative activity will support evidence- based advocacy and decision-making. C. Internal RII - For management and leadership of CGIAR beyond RAFS, Learning, stakeholders including two other action areas and other initiatives in Steering CGIAR D. External regional - Provide learning around roll-out of RIIs at a country or Learning, partners to RIIs*, regional levels and measurability of results Accountability including bilateral - Foster systematic exchanges of knowledge and information projects along ToC and RF - Formulate mutual understanding and co-development of MEL for learning for exchange, evaluation and use Note: IAES* policymakers, national governments and National Agricultural Research and Extension Systems (NARES), researchers 9 Terms of Reference: RII Evaluability Assessments To support learning and accountability among the key user and stakeholder groups, the EA will collect, analyze, and present the information to meet their diverse needs. To the extent feasible, given the resource and time allocated to the EA, key stakeholders will be consulted and engaged through relevant channels and using the appropriate engagement tools. 2.3 Scope and Timing The scope of EA process will cover the design from launch to the present (see Figure 2 from the EA Guidelines). Each RII is at a different stage of implementation, thus, each of the EAs will take on a slightly different focus in its own alignment of EA objectives at the RII level, as illustrated below in Figure 1. Out of the six RIIs (Table 1 above), four have been selected, namely AMD, CWANA, WCA and TAFFSA. These RIIs are detailed in Annex 1, and their rationale is discussed below in Section 3. The full EA framework (shown in Annex 3) will be implemented by an independent evaluation team (EA lead; see Section 5 on team roles and responsibilities) in a participatory approach. The EA Framework will cover six domains addressed with a total of 40 questions. The six-step framework shown in Figure 2, will be used to map out the process for implementation (ref CGIAR EA guideline, 2023). Assessment of each RII is expected to take maximum five weeks, including pre-analysis of data and travel when agreed. Figure 4 provides additional detail on timelines and activities for each step. Figure 2. Six-Step Process for Conducting EAs of RIIs in CGIAR, with timelines6 1. Identify the purpose and 2. Identify and involve 3. Apply the EA Framework scope of EA exercise stakeholders to selected RIIs (April-June) (April - Sept.) (Aug.- Nov.) 4. Collect key documents, 5. Decide on evaluation identify potential evidence readiness; make 6. Use EA results gaps, engage evaluand recommendations (Dec. 2023- 2024+) (July- Nov.) (Nov.-Dec.) 3 Evaluability Assessment Approach and Method 3.1 Approach The CGIAR Evaluation Framework and Policy (2022) and Evaluation Guidelines, and Conducting and Using Evaluability Assessments in CGIAR will guide design and implementation of EAs, and use of their results. The 6 Conducting and Using Evaluability Assessments in CGIAR: CGIAR Evaluation Guidelines 10 Terms of Reference: RII Evaluability Assessments EA design will build on the two EAs referenced above, the 2022-2023 PRMS study, with learning from the 2021 Synthesis of Learning. The EA design and implementation will draw on evaluation industry standards. The EA approach will be utilization-focused and collaborative, as well as focus on accountability and learning. In line with the scope and EA objectives, and within the timeline provided, the expanded EA analytical framework will be used: across the following six assessment domains, to answer 40 questions (see Figure 3 and Annex 2). For all six assessment domains, Annex 3 describes each assessment domain, defining and explaining why they are being measured, and providing guidance for evaluators on how to use them. Figure 3. Six Domains of EA Framework of CGIAR7 A. Intervention logic: To be evaluable, an intervention must clearly describe what it hopes to achieve and how. Intervention logic represents the overall logical integrity of the intervention. This logic should be supported by a robust ToC and a body of evidence that lends credibility/plausibility to the ToC. B. MEL systems and resources: To be evaluable, an intervention must have a credible plan in place to track its contribution to outcomes. The plan should include a unified vision of how M&E activities will fulfill accountability, delivery, and learning needs. The MEL system must generate relevant and quality data. Most often, this data is defined by an intervention’s indicators. Having appropriate indicators that are aligned with desired results is essential. A baseline is a necessary starting point against which to assess intervention performance and results. C. Gender, diversity, and inclusion: CGIAR is committed to the inclusion of women, youth, and socially excluded and vulnerable groups. To be credible and legitimate, CGIAR research must be based on the inclusion of the end-users it hopes to reach. D. Long-term evaluability: Many of the impacts of CGIAR research will not be recognized until long after intervention delivery. E. Context and environment: To be evaluable, an intervention must be accessible to evaluators and key stakeholders in the evaluation. F. Management and key stakeholder engagement and support: To be evaluable, management and stakeholders must be active participants in the evaluation process. The suggested approach will rely on mixed methods of data collection and analysis: (1) documents and data reviews: a desk review of secondary data sources: documents and related resources; and (2) primary data collection: key informant interviews, and focus group discussions, both in person and remote.8 The EA team would use innovative approaches in data collection, analysis, and dissemination (as applicable) to enhance the value of real-time learning. 3.2 Method 7 Conducting and Using Evaluability Assessment in CGIAR: CGIAR Evaluation Guidelines 8 Primary and secondary are distinguished by data collected by the evaluator (key informant interviews, observations) and data provided to the evaluator for review and analysis (monitoring data, financial data), respectively. 11 Terms of Reference: RII Evaluability Assessments As a process-oriented exercise, evaluations are typically guided by phases, aligned to the steps. The phases include preparatory/scoping work; development of approach and method during an inception phase, including field trip; data collection and analysis; presentation of preliminary findings involving briefings and validation; and reporting involving the presentation of findings, conclusions and recommendations, both orally and in writing. Similarly, the EA has a method involving a series of steps, as shown in Figures 2 and detailed in Figure 4 in checklist format. The scoping for EA of the RIIs began in April 2023 and is expected to finish in September 2023. It has included endorsement by the Strategic Impact, Monitoring and Evaluation Committee (SIMEC), for sharing with the System Council, System Board, and CGIAR Executive Management by end of 2023. The intent is the EAs of RIIs are conducted early enough for necessary uptake during portfolio and initiative evolution in 2024. Figure 4. Steps with activities along the Indicative Timeline for EAs of RIIs, 2023 May-July 2023 Sept-Nov Nov.-Dec. Aug.-Nov. Dec. 2023- Steps 1-2 2023 2023 Step 3 2024+ Step 4 Step 5 Step 6 •Preliminary •Stakeholder desk review •Desk review •Validation workshops •Synthesis consultations •TOR review/ •EA team endorsement •Primary data report collection: •QA identification •EA team development key informant •ToR induction •EA report •MR finalized interviews ( finalization development •EA inception KII) and process •Meta-review note Focus Group of the •Management Discussions process/ tool Response (FGDs) development •Development •Field trips to of KM selected products initiatves •Use of EA •Preliminary resuts findings-for individual RIIs from EAs: validation and draft reports 3.2.1 Step 1: Identify Purpose and Scope The EA scoping commenced in March 2023, with initial consultations with RAFS Action Area director and coordinator. The purpose was to explain the EA and understand the RII context and concerns, and align the EA with the implementation cycle of the CGIAR portfolio and funders priorities. Additionally, such consultations paved the way for the assessment, which involved initial planning of the scope, purpose, and expected outputs for diverse stakeholder groups. This initial planning step has been taken with an interest in transparency in EA design and implementation, which facilitates credibility and legitimacy, and helps mitigate for largely unnecessary concerns among the RII stakeholders of being assessed. Consistent with standards and principles set out in the Evaluation Framework (including relevance, use, transparency, legitimacy, and participation), conducting an EA begins by identifying the needs of potential evaluation participants, what they want to know and how they will use the EA results. 3.2.2 Step 2: Identify and Involve Stakeholders 12 Terms of Reference: RII Evaluability Assessments Several engagements with pre-selected four RII stakeholders fostered early buy-in for the EA and for the development of this ToR. Specifically, the following activities have been implemented during the early planning phase: • Identification of the core stakeholders across CGIAR and RIIs- expected to continue as an iterative process; • Classification and mapping of expectations and roles in the EA processes; establish who should participate, their interest in the EA results, and their role in the decision-making process; • Provision of initiative-level documentation by the four RIIs, with initial analysis by IAES- see Box 2; • Preliminary Analysis of RII monitoring Data for Y1 – See Box 3 The EA lead will scope and define stakeholder interests and expectations as part of the Inception phase. This engagement will contribute toward the development of the inception note to guide the EA exercise. 3.2.3 Step 3: Apply the EA framework Towards preparing the ToR, in Step 1, RIIs shared several documents during the planning phase, as shown in Box 2 below and Annex 5: Inventory of Documents and Reference Materials. Additional documents will be requested and analyzed in Step 3, for a more in-depth review by the EA team towards developing the inception note, and subsequently at Step 4 in implementing the EA. Box 2: Example Documentation provided by selected RIIs during initial planning Phase • Proposal or strategy documents • ToC (at all intervention levels, and including historic and current versions, if updated annually) • Results Framework or other documents with articulated inputs, activities and outputs, desired outcomes, impacts and indicators • Plan of Work and Budget, Annual (and other) reports • Project lists (with related documentation) • Contact lists for internal and external stakeholders and key informants • Standard indicator description sheets • ISDC reviews, funder review/comment on proposal as applicable • Previous independent or other evaluations, studies, and impact assessments • Access to key databases and data sources with relevant information, e.g., monitoring data and staff lists, access to Web of Science for bibliometrics, dashboards • Internal audit or other assurance-related documents • Governance-related information (independent advisory bodies, structure, ToRs, composition, meeting minutes) Items in bold indicate the minimum, core required list, with other documents to be collected if available. The inception note, which the EA external team lead (see Management, Roles and Responsibilities in the Evaluability Assessment Process) will draft and finalize toward the end of the inception phase, will serve to ground application of the EA framework, mapping out the final approach, method, and presentation of results. The purpose of the inception note is to further refine the objectives, approach and method for each RII EA, towards individual RII reports and a synthesis EA report; as well as to help ensure a shared understanding of the EA objective between the commissioner (with its needs represented by SIMEC), executing office (IAES) and the external EA team, and among key stakeholders. The inception note will also serve as a roadmap and as a guiding document in the EA process for all the stakeholders involved, 13 Terms of Reference: RII Evaluability Assessments including the EA team members themselves and the evaluand9, steering proper implementation, monitoring, and mutual quality assurance in all its steps.10 The IN will focus on the following elements: • Deeper dive into the EA Framework and steps: validation of objectives and sub-objectives as needed., refinement of EA methodology • A stakeholder analysis identifying key stakeholders for selected RIIs through desk analysis of main documents and discussion with the teams • Prioritization of strategic issues of importance for emphasis during the inquiry phase, should they have changed • EA report outlines for individual RIIs and overall synthesis. The EA team will collect and analyze data and evidence according to the EA framework in the inception phase, and prepare a preliminary list of findings and conclusions. 3.2.4 Step 4: Collect Evidence Data collection will follow mixed methods, leveraging both qualitative and quantitative data from primary and secondary sources to understand operating environments and to track contextual and programmatic assumptions. This step aims to compare the ‘implementation reality’ to the ‘design of the object of evaluation’, using, at a minimum, the following methods: Document reviews: The EA lead will continue requesting and review of all documentation provided during the data collection phase, using this and other documentation as part of the assessment. Semi-structured KIIs: During the data collection phase, the evaluation lead will conduct semi-structured interviews with select key informants. The EA external Lead will put forth a rationale for all key informants to engage through semi-structures in the inception note. FGDs: During the data collection phase, FGDs will be conducted with different segments of the stakeholders identified in the stakeholder analysis and mapping. Maximum 15 interviews per RIIs are envisioned, with additional interviews extending to RAFS and CGIAR-at large, and country-base partners. The EA team lead will put forth a rationale for all stakeholders to engage through FGDs in the inception note. Use of monitoring data: A CGIAR Performance Results Management System (PRMS) or related modules (i.e. Results Dashboard or other CGIAR dashboards) should contain key performance-related information.11 However, outdated or incomplete data are not uncommon, so it is good to review key documents (Box 3) during the EA (Domain B: MEL systems and resources) to ensure they are up-to-date and complete within the planned time scope of the evaluation. Box 3 presents preliminary analysis of the results dashboard for year 1 (2022). 9 The term “evaluand” is defined within the evaluation field as that which is to be assessed, as defined by its parameters and its scope. 10 See also blog on IAES’s approach to evaluation inception reports. 11 PRMS and its modules are under redesign during the elaboration of this document. The CGIAR dashboards and decision registers provide access to information on CGIAR results, financials, Trust Fund and governance decisions. 14 Terms of Reference: RII Evaluability Assessments Box 3: Preliminary Analysis of RII monitoring Data for Y1 Preliminary analysis of the results dashboard1 shows that in Y1, RIIs have achieved a total of 690 results to date (July 2023). Furter information at a glance is available in Image 1, and the number of indicators with evidence on the dashboard for the RIIs is summarized below (Table 2): Image 1: RII) in numbers (screenshot from CGIAR Results Dashboard, July 2023). • Data completeness: Not all RIIs report for all indicators. One of the most significant gaps identified during the analysis concerns the geographic coverage. Most of the time the ‘region’ is missing even if the ‘country’ is known, but frequently both ‘region’ and ‘country’ are missing. However, the region can be easily derived when the country is known. Another indicator which often shows gaps is the ‘lead contact person’. Many gaps were also found among the ‘evidence’ links both at the output and outcome levels, as well as in the KRSs. • Custom indicators: RIIs report for the same set of indicators. Obser vation: During the data collection and inquiry phase, field visits will be carried out, as proposed and agreed upon in the inception phase. Sampling strategy for site visits will be elaborated, with tentative criteria such as geographical representation and balance of RII team members and core partners. The EA external team lead will carry out analysis of data collected, and based on the findings, will assess the extent of evaluability of each of the RIIs and recommendations for improvement. 3.2.5 Step 5: Assess readiness and make Recommendations The EA external team lead will develop four draft sub-reports, one for each of the RII EAs. As further discussed below, these four sub-reports will be synthesized into one report to provide greater scope and utility at the organizational level. The RII draft sub-reports will be shared with the management of the RIIs, and RII coordinator for review and comment. Thereafter, the EA team will present preliminary cumulative findings by EA domains, to debrief the IAES and the CGIAR management and seek validation (via a validation workshop) and provision of factual corrections and feedback.12 Engaging on the preliminary findings and results of the EA will be a cyclical and iterative process, embedding learning on the actual EA implementation. In applying the EA framework, questions will be asked about how the findings about each of the domains aggregate and fit with each other, as findings from one area of analysis are likely to have implications for others. Stakeholders are likely to provide information about relevant documents, relevant documents can provide information about relevant stakeholders, both will provide information about information systems, and vice versa. A summary judgment derives from aggregating the assessment of the individual questions and judgments by domain. The use of a checklist (Excel tool provided to the EA team) with a narrative explanation is suggested to 12 See also CGIAR Guidelines on Evaluation Final Report. 15 Terms of Reference: RII Evaluability Assessments facilitate the triangulation of results. The checklist should be considered as a guide rather than a determinant to move forward with an evaluation. At this stage, sufficient information has been collected to decide on evaluation readiness. If the EA criteria have not been met, the results from an EA can be used to develop recommendations that will lead to a higher state of evaluation readiness in the future, meaning an evaluation can be conducted once appropriate mitigation measures to potential limitations are in place. An EA decision making framework is presented in Table 5, (below) and Annex 2 includes an example from the GENDER Platform. Table 5: Decision Support Framework for RIIs using Stoplight System, full EA Framework Indicator Decision Follow-up steps/recommendations Yes, to all 14 core Feed-in lessons and evidence from EA in programming questions and/or evaluation design: adapt ToC and measurements Proceed with EA criteria met and convene stakeholders in a decision-making evaluation committee to contribute to ToR for evaluation of RAFS science group. Yes, to all 14 core questions + Yes to Address issues raised, reassess and refine key evaluation EA criteria Postpone the at least half of the design parameters (e.g., criteria/questions, timing, partially met evaluation other questions, resources). even if with caveats The RII is not ready for an evaluation until deficiencies are No, to all 14 core remedied, e.g., institutional bottlenecks that could questions + No to undermine the evaluation, data quality that is insufficient more than half of to support meaningful evaluative judgments, or external None of the EA Cancel the the other questions conditions that make conducting an evaluation too criteria are met evaluation difficult or dangerous. In the meantime, the recommendations can inform MEL-related adjustment and any other aspects aligned to the lines of inquiry as per the framework. Source: CGIAR IAES. Consistent with the purpose and objectives of EA of RIIs, weighting of EA framework domains can be considered. Furthermore, cumulative scoring for four RIIs can be considered in lieu of individual scores, to amplify strength and representativeness of learning for the other two RIIs. Either way, a written justification would accompany the scoring matrix. The EA RII reports and an overall synthesis will follow CGIAR evaluation reporting guidelines and QA processes. The first reviewer of the draft report will be IAES in line with CGIAR’s guidelines on the final evaluability assessment report.13 The EA team will be obliged to provide a revised version of the draft reports if the quality is not acceptable. If the quality of the draft report is satisfactory (in form and substance), the EA manager at IAES will circulate it to: (i) the respective RII team for comments and factual corrections; and (ii) external peer reviewers and the evaluation reference group (ERG) members for review and comments. With the feedback from the relevant stakeholders, the EA team will finalize the draft reports considering comments according to the team’s judgment, availability of evidence- all to be documented in the Response Matrix. A discussion version of the synthesis report will be circulated by IAES to SIMEC for acceptance via its secretariat. 13 Old guidelines on the final evaluation report are currently under revision. IAES to provide final version to the evaluation team when published. 16 Terms of Reference: RII Evaluability Assessments 3.2.6 Step 6: Communicate and use of EA Results The use of the EA results and learning from stakeholder experience conducting the EAs will be done at both the RII programming level and at the CGIAR broader organizational level. Complying and complementing the implementation of MR (see Management Response), in fulfillment of the EA objectives outlined above the following three distinct ways are envisioned: 1) Use of EA results at the RII level: Following implementation of the EA, in 2024, individual reports from RIIs and the EA synthesis report, the EA brief, and other knowledge products along with the MR, will be published on the IAES website. The EA will result in concrete recommendations in accordance with its objectives of supporting readiness for quality evaluation at the RII programming level. The results and recommendations will be presented with relevant stakeholders, and learning events organized to discuss implementation of the recommendations. In line with the dissemination and knowledge management strategy to be developed at the inception phase, tailored presentations will be made to targeted stakeholders, and learning events organized with internal and external stakeholders. 2) Use of EA RII results to learn about extent of evaluability across programmes within CGIAR: EA team will develop a synthesis report of the results of each of the four RII EAs as a means to support organizational learning across CGIAR portfolio of RIIs and RAFS science group, specifically about common areas of evaluability challenges and common recommendations. The approach to this review will involve a detailed synthesis of the EA reports and analysis across reports. 3) Use of EA results and experience to improve upon the EA approach and tool: As part of a meta- evaluative exercise, IAES will undertake a review of the EA process, resulting in possible adjustments to the EA tool and the overall EA approach based on the stakeholder experience implementing the Genebanks and RII EAs. The method for this meta-evaluative exercise will involve collection of data from stakeholders on their experience and learning from the exercise (both during and after), and a systematic analysis of the EA results and how they compare to intended outcomes of the exercise. A report will summarize the findings, conclusions and the recommendations and will be shared with those stakeholders who engaged in the exercise, as well as in a webinar for interested parties for discussion. More information on each of the deliverables is found below in Section 4. 3.3 Management Response In line with the CGIAR Evaluation policy, MRs are mandatory for all System Council-commissioned evaluations in CGIAR.14 The PPU oversees the procedure for tracking, monitoring and outyear reporting against the implementation of evaluation recommendations when the evaluand is CGIAR. In line with the MER guidelines15, and in response to the EA purpose and objectives, IAES will liaise with RII management and leadership, as well as the Project Coordination Unit (PCU) and PPU to coordinate the MR preparation within a stipulated timeframe.16 The draft synthesis report will be circulated to SIMEC for endorsement. Once formally presented to the CGIAR System Council, the MR to the Synthesis are considered final and will be published on the IAES website. 14 The co-development of the Management Engagement and Response guidelines is ongoing at the time of this evaluation. 15 Under development at the time of this TOR development, to be available at the IAES Evaluation Guidelines page 16 See also CGIAR Guidelines on Evaluation Engagement and Management Response. 17 Terms of Reference: RII Evaluability Assessments Box 4: Management Response Recognizing the responsibility of CGIAR management to encourage a robust culture of accountability, learning and continuous improvement, the CGIAR Evaluation Policy states: “A management response is required for every evaluation where CGIAR has had a partial or complete decision-making power in the evaluation process, or CGIAR has fully or partially financed the evaluation.” 3.4 Consideration and Expected Limitations of the EA Process Understanding of the necessary consideration and expected limitations is grounded in the scoping exercise towards developing this ToR. The following should be considered during the evaluability assessment design, implementation, and analysis of results: • The EAs are conducted two years into initiative implementation, with senior management changes in the RAFS Science Group effected mid- 2023; and • With a focus on real-time learning on the sample of four RIIs, only selected lessons and recommendations would be generalizable across all six RIIs, and extendable to RAFS science group, and wider CGIAR portfolio, including in the forthcoming evaluations of science groups in 2024; and onwards in the 2025-2027 CGIAR portfolio. These and other considerations and limitations should be elaborated upon in the inception note, with suggested mitigation strategies. 4 Deliverables, Knowledge Management and Dissemination This section provides greater detail on the nature of the deliverables themselves as well as the process for their development, finalization, and dissemination. The following section details the EA sub-reports and presentation; the synthesis EA report and presentation; and the meta-review paper (Box 5). 4.1 Sub-Reports from EA of each RII Box 5: Knowledge Management and Deliverables from the EA As will be further defined in the inception note by the EA Team, the process deliverables will include four sub-reports, one for each RII, detailing -Four EA sub-reports methods, findings, conclusions, and recommendations; and a -Presentations and brief -Other deliverables and KM PowerPoint presentation based on the sub-reports and geared towards products based on results use of evidence by single RIIs, detailing the results for each of the four -EA synthesis report EAs. The recommendations will be evidence-based, relevant, focused, -Presentation and brief clearly formulated and actionable, and will form the basis of continued -Meta-review paper work done by the RIIs in working toward impactful evaluation in their -Adjusted EA tool and other interventions. relevant documents a) Four EA sub-reports EA reports for each of the four RII EA exercises will be drafted in alignment with CGIAR’s guidelines. Reports will describe findings and conclusions, based on evidence collected as defined in the inception notes, and make recommendations logically in alignment with the conclusions. The recommendations will be 18 Terms of Reference: RII Evaluability Assessments evidence-based, relevant, focused, clearly formulated, and actionable. They will be prioritized and addressed to the different stakeholders responsible for their implementation. The main findings and recommendations will be summarized in an executive summary for each report, and then annexed to the synthesis report. The RII EA reports are described as sub-reports as they will contribute toward and inform the final product - synthesis EA report described below. The EA sub-reports should be concise (no longer than 15 pages excluding the Executive Summary and Annexes) and written in plain English. The EA team lead will submit the final EA report for an individual RII by email to the evaluation manager in electronic editable form (MS Word) aligned with IAES’s Style Guide. The EA report will follow a standardized structure and template to be provided by IAES. The draft report will be quality assured by IAES; then the draft report will be shared with (i) external peer reviewers with relevant expertise called up from IAES’s vetted roster; and (ii) some members of IAES’s evaluation reference group. The first draft report will also be shared with the RII teams for their review and comments, including to check for any factual errors and to highlight the significance of any such errors in the conclusions. The EA team will integrate the collective feedback received into a discussion version of the report which will be professionally copy-edited. The sub- reports will be shared with SIMEC as reading materials. b) Presentations Throughout the EA implementation, the EA external team lead, where there is opportunity, will present and share the EA-related results to targeted audiences via various communication channels upon request by IAES, e.g., events, conferences and social media. The presentations will target RIIs, to enable knowledge sharing by the EA team (both EA advisor and EA lead) to further their understanding of EA use going forward in MEL practice. In support of this effort, IAES may decide on specific deliverables to be developed in cooperation with RII staff. Other deliverables and KM products based on EA results In line with MR and broader learning, there will likely be specific pieces of work recommended for each of the RII teams, based on the EA results. In consultation with the RII teams, there may be additional agreed upon deliverables identified to support the RII teams in furthering the evaluability of their projects. IAES will establish whether and under which conditions the EA team may take on these pieces of work, e.g., absence of MEL staff at the level of RIIs.17 4.2 Synthesis Evaluability Assessment Report While the sub-reports are actionable specific to each of the RIIs, the synthesis EA report will synthesize findings, conclusions and recommendations across the four EA RII sub-reports for the purpose of supporting organization-wide learning. a) Synthesis EA Report The main output of this evaluative activity, the final synthesis report, will be in line with CGIAR’s guidelines and aligned with IAES’s Style Guide. It will describe findings and conclusions, based on evidence collected in the EA framework as defined in a prepared inception note, based on the four RII EAs, and make recommendations logically in alignment with the conclusions. The recommendations will be evidence- based, relevant, focused, clearly formulated, and actionable. They will be prioritized and addressed to the different stakeholders responsible for their implementation. The main findings and recommendations will 17 The roles and responsibility and content of such activities would be subject to alignment to the mandate of the Evaluation Function and alignment to the Evaluation Framework and Policy (2022). Potential deliverables may include a revised MEL framework, including ToC and results framework. 19 Terms of Reference: RII Evaluability Assessments be summarized in an executive summary. The main report should be concise (no longer than 25 pages excluding the executive summary and Annexes) and written in plain English. The executive summaries of four EA sub reports will be annexed to the report. IAES will share this first draft report with a team of (i) external peer reviewers with relevant expertise called up from IAES’s vetted roster; and (ii) some members of IAES’s evaluation reference group. The draft report will also be shared with the RIIs and RAFS leadership team for their review and comments, mostly with the aim for validating conclusions and recommendations. After integrating feedback from peer-reviews and QA, the EA advisor, who has final responsibility for the deliverable, will submit the final EA report by email to the IAES evaluation manager in electronic editable form (MS Word) aligned with IAES’s Style Guide, and a standardized structure and template to be provided by IAES. The EA team will integrate the collective feedback received into a discussion version of the report which will be professionally copy-edited. Subsequently, the discussion version will be presented to SIMEC for acceptance. With the feedback of SIMEC integrated, the discussion version of the report will be presented to System Council for their endorsement. The final synthesis report with the Management Response will be published on IAES’s website. b) Presentation and Brief In consultation with IAES/evaluation, the EA team will present and share the EA-related results to targeted audiences via various communication channels, e.g., conferences and social media. The EA team will be expected to produce a 2-3 page brief of key findings and lessons, following a template provided by IAES.18 4.3 Meta-Review Exercise of EA Process and Tool a) Meta-review report As EAs are new to CGIAR, the four RII EAs offer an important opportunity for reflection and learning. The EA advisor will implement a systematic approach to gathering data on stakeholder observations and reflections about the EA process, particularly on their experience using the EA tool. As a meta-evaluative exercise, the EA advisor will develop an approach to evaluate the EA process that will result in findings, conclusions, and recommendations for improving upon the process and the EA tool. The recommendations will be evidence-based, relevant, focused, clearly formulated, and actionable. The main findings and recommendations will be summarized in an executive summary. The length of the main text will be determined, based on the proposed scope. The draft meta-review paper will be submitted by the EA advisor to IAES for review. Based on comments to the draft, the EA advisor will review and finalize the meta-review paper. The paper will be submitted to IAES, aligned with IAES’ Style Guide and following an agreed upon structure. The EA report will follow a standardized structure and template to be provided by IAES and will include an executive summary. b) Presentation The EA advisor will develop an accompanying PowerPoint presentation for the meta-review report. Where necessary, the EA advisor will present and share the results to targeted audiences via various communication channels upon request by the IAES, e.g., conferences and social media. c) Adjusted EA tool and other relevant documents 18 See CGIAR Guidelines on Final Evaluation Report. 20 Terms of Reference: RII Evaluability Assessments The EA advisor will adjust the EA tool and any other relevant documents, based on the results of the meta- evaluation paper that details findings, conclusions, and recommendations. These adjustments will be made based on clear understanding with IAES, documented in a short inception note to guide the agreed upon changes, as well as any process for their adaptation to be adopted. All deliverables, except presentations, that will be published on the IAES website are subject to proofreading revision by an editor engaged by IAES. The names of the EA advisor and EA lead will feature in publications they have drafted or participated in the drafting of, with their titles, as consultants to the IAES. 5 Management, Roles and Responsibilities in the Evaluability Assessment Process The CGIAR System Council as evaluation commissioner via IAES takes accountability for the EA of RII. In line with the CGIAR Evaluation Framework and Policy (2022), CGIAR management share leadership and mutual responsibility for the conduct and use of results of independently conducted evaluation activities. 5.1 CGIAR Management Engagement and Response Consistent with the principles and standards in the CGIAR Evaluation Framework, utility and use guide the evaluative engagement with stakeholder groups (the evaluand, CGIAR management and other key intended users) throughout the evaluability assessment design and implementation. To stimulate the uptake of the evaluation results and learning, early management engagement began in the scoping and design phase, which facilitated the co-development of the priority evaluation questions (Annex 1). In line with the CGIAR Evaluation policy, and CGIAR EA guidelines, MRs are mandatory for all System Council-commissioned evaluations in CGIAR.19 Following the guidelines under development for MR, IAES will liaise with RAFS and RII management as well as PPU to coordinate the MR preparation within a stipulated timeframe.20 The draft EA report will be circulated to SIMEC. Once formally presented to the CGIAR System Council, and the System Council has endorsed the synthesis report, the MR and EA synthesis report are considered final and will be published on the IAES website. PPU oversees the procedure for tracking, monitoring and outyear reporting against the implementation of EA recommendations in CGIAR. Towards mutual responsibility, throughout the implementation of the assessment, the management of selected RII, leadership of RIIs and RAFS, management, core focal points (actors) would respond to the EA team’s requests: documentation and data, access to partners and staff for engagement with the EA team, and information on partners and stakeholders. These actors will also be responsible for giving factual feedback on the draft RII reports, and full EA synthesis report, as required. 5.2 IAES Management and Responsibilities IAES is responsible for planning, conducting the initial design, managing evaluator selection and contracts, and initiating and managing the workflows of the evaluability assessment. The latter is done in a way that ensures the quality and independence of the evaluative process and EA reports, as well as the timely 19 The co-development of the MER guidelines is ongoing at the time of this evaluation. 20 See also CGIAR Guidelines on Evaluation Engagement and Management Response. 21 Terms of Reference: RII Evaluability Assessments delivery of high-quality key outputs. IAES is also responsible for ensuring the compliance of processes and products with the CGIAR Evaluation Framework and Policy (2022) and the IAES’s mandate (see IAES TOR). The lead of IAES/Evaluation Function, in the capacity of EA evaluation manager, is specifically responsible for (i) selecting, contracting, and convening the EA team; (ii) contractual arrangements; (iii) monitoring and supervision of the EA team against agreed ToR and contracts; (iv) facilitating access to the evaluand for data collection by the EA team; (v) coordinating QA and validation; (vii) guidance and support on documentation (e.g., required templates, editorial services and graphics); and (viii) developing a knowledge management approach and products. The EA team and the manager from IAES/Evaluation Function will ensure they undertake adequate consultation with EA stakeholders throughout the process. IAES will facilitate a validation workshop on preliminary findings with core stakeholders. IAES will follow its layered QA system to assure for EA process and EA outputs, which involves: (i) a review by IAES; and (ii) an external peer review by the advisor and selected member of the IAES evaluation reference group. Both groups will be called upon to interrogate the EA methodology. For validation, IAES will circulate the draft inception note to SIMEC via its secretariat, for comments–particularly to flag if the questions posed will meet the needs of System Council, being the commissioner of the evaluative activity. IAES will ensure QA and that the EA team incorporates the relevant feedback. The final inception note subsequently represents the contractual basis for the EA team’s work and deliverables, and it will be published on IAES’s website. 5.3 Evaluability Assessment Team Role and Responsibilities Under the oversight, management, and guidance of IAES, the EAs will be conducted by an EA team of evaluation experts. The team members will be drawn from the IAES’s standing SME and evaluator roster, supported by an analyst, and each carefully vetted for any conflicts of interest. The responsibilities of the team members will be strategically and operationally split by EA steps to best leverage their expertise and availability. See Table 6 for the roles and responsibilities of each of the team members, including IAES management and oversight. Table 6. Roles and Responsibilities of the EA Team, including IAES Management Function IAES/Evaluation Evaluation Function EA advisor EA lead analyst management (IAES) and oversight Provide inputs Scoping and Support IAES in drafting the EA as needed to Draft the EA ToR - planning ToR support ToR development Conduct inception Support the EA lead with any Provide inputs phase, including Provide technical aspects; provide as needed to document review, Inception management, guidance as needed; review support inception interviews with guidance; QA concept notes and provide concept note RII staff, and producing comments preparation concept note Support the EA lead with any Conduct EA, travel to RII Implementation technical challenges; serve as Provide inputs Provide guidance; sites as needed, draft of the EA a resource as needed; review as needed to quality control four EA sub-reports; process draft EA report and provide the EA report finalize EA sub-reports comments 22 Terms of Reference: RII Evaluability Assessments IAES/Evaluation Evaluation Function EA advisor EA lead analyst management (IAES) and oversight based on feedback from stakeholders - Co-develop a synthesis - Co-develop a report of learnings based synthesis paper; on the four RII EA sub- presentation on reports; presentation on Provide guidance; findings. Provide inputs findings. quality control; - Prepare and deliver as needed to Dissemination - Lead formative meta- perform or support presentations for assembly and and learning, evaluative exercise (draft facilitation of learning webinars, development use of EA results short paper with findings webinars/learning along with IAES of EA learning and recommendations for events evaluation lead, and materials adjusting the EA tool participate in other and/or process). dissemination - Support dissemination efforts as needed and KM efforts as needed Validation and Liaise with RII Provide inputs Support to RII teams in operationalizing pathway for application of teams, RAFS as needed to implementing recommendations. Additional activities and learning and leadership, PPU the revised and deliverables to support the RII teams. IAES will establish recommendatio and PCU, to further whether and under which conditions the EA team may take ns in the EA socialize results developed on these pieces of work21 reports and tool and EA tool tools The EA lead and EA advisor will both be evaluators with a strong background working internationally, and with experience conducting and advising on EAs. The analyst will have knowledge of CGIAR data, including familiarity with the CGIAR Results Dashboard. In addition to the overall areas of responsibility outlined in Table 6 above, the IAES/Evaluation Management Function will more specifically assume the following: • Guiding and managing the EA team during the phases and steps • Overseeing the preparation of, and quality-assuring, data collection outputs by the team • Contracting an analyst to support the EA design and implementation, including all the deliverables and intermediary products (inception note, individual RII sub-reports as needed, and the synthesis evaluation report) • Support convening of the team towards a jointly authored and agreed set of findings, conclusions and recommendations • Facilitating and engaging the EA team in learning and governance meetings with CGIAR and other stakeholders The EA lead has the final responsibility for the four EA sub-report deliverables to IAES, subject to adherence 21 The roles and responsibility and content of such activities would be subject to alignment to the mandate of the Evaluation Function and alignment to the Evaluation Framework and Policy (2022). Potential deliverables may include a revised MEL framework, including ToC and results framework. 23 Terms of Reference: RII Evaluability Assessments to CGIAR Evaluation Framework and Policy. Ultimately, the EA Lead is responsible for ensuring the quality, consistency, and soundness of all EA deliverables to IAES. Specifically, the primary responsibilities of the EA lead will be: • Develop the methodology and approach in the inception note with the support of the EA advisor • Submit four deliverables in adherence with CGIAR Evaluation Framework and Policy: o Four inception reports (one for each RII EA): further development of the ToR will ensure clarity on the evaluator’s approach and methodology, and roles and responsibilities for the evaluator and stakeholders. o Four PowerPoint presentations of inception reports: to accompany the inception reports. o PowerPoint presentation of preliminary findings of the EA: to be delivered for validation by RII leadership and IAES; comments to be incorporated into the draft EA report. o Four draft EA sub-reports: presentation of findings, conclusions and recommendations for IAES and RII stakeholder review. o Four final EA sub-reports: finalized reports with executive summaries. o Four PowerPoint presentations of the final EA results: accompanying PowerPoint presentations to the sub-reports, to be used for presentation and discussion. The EA advisor, in addition to the overall responsibilities above in Table 6, will have the following specific role and responsibilities as part of the EA team: • Advise and provide support to the EA lead as required; serve as a sounding board in the development of the approach and methods; and review draft deliverables developed by the EA lead as determined helpful. • Support IAES/Evaluation Management Function as needed and determined helpful. • Co-develop the following deliverables for the synthesis report, with input from EA lead as necessary: o Short methods paper outlining the proposed method for developing the synthesis report for the four EA sub-reports, in alignment with IEAS policy; o Draft synthesis report for the four EA sub-reports with EA team lead submitted for review. o Final synthesis report for the four EA sub-reports, with executive summary. o PowerPoint presentation to accompany the final synthesis report. o Contribute to public dissemination as required, both in written form (e.g., blogs), and orally (e.g., meetings and workshops). • Develop the following deliverables for the meta-review paper: o Short methods paper outlining the proposed method for the meta-evaluative study, in alignment with IEAS policy. o Draft review paper outlining findings, conclusions and recommendations for changes made to the EA tool, based on the RII EA experience, or EA process as outlined in the EA Guidelines. o Upon addressing comments on the draft, submit a final meta-review document, with a two- page executive summary. o Contribute to public dissemination as required, both in written form (e.g., blogs), and orally (e.g., meetings and workshops). The Evaluability Assessment lead and an EA Advisor will have a minimum of 25 years of experience between them, including EA experience. Both the EA lead and advisor will have experience in leading teams, excellent analytical, synthesis and communication skills (written and verbal), and demonstrated skills in mixed qualitative and quantitative data collection and analysis techniques in evaluative exercises. 5.4 RII Team Participation and Responsibilities 24 Terms of Reference: RII Evaluability Assessments The EA will be delivered in a participatory approach, requiring the active engagement of the RII in providing logistical support and planning to the process, in collaboration with IAES and the Evaluability Assessment team. The RII staff will make available relevant documents for review. Importantly, the RII staff will participate in the EA as a learning exercise, actively engage with the EA team, and reflect and build initiative staff evaluation capacity going forward in planning and preparing for impactful evaluation. 5.5 Contractual and Payment Considerations CGIAR’s IAES is hosted by CGIAR System Organization through an arrangement with the Alliance of Bioversity International and CIAT at its offices in Rome, Italy. Contracting will be carried out by its hosting entities and under their name on behalf of the IAES/Evaluation Function. The EA team members are expected to sign and abide by the Conflict of Interest policy of the IAES and must maintain independence in fact and appearance from the RIIs throughout the assignment. All contracting fees and conditions will be administered in line with the approved policy for consultants. Confidentiality and non-disclosure provisions are covered in these contracts. All collected data must be anonymized and kept within the IAES SharePoint repository. Informants should be duly notified to adhere to ethical evaluation principles. 25 Terms of Reference: RII Evaluability Assessments Annex 1: Background on Six Regional Integrated Initiatives22 Proposed 3 year outcomes 2022 Highlights- Reports Research Emphasis/ (individual RIIs page) Transforming Agrifood Systems in South (Table 1) Objectives Asia - CGIAR TRANSFORMING AGRIFOOD SYSTEMS IN SOUTH ASIA (TAFFSA)- SOUTH ASIA Objective: Working across 1) Sub-national governments, donors, the private sector and/or Working with more than South Asia, this Initiative aims to development partners co-develop knowledge systems and 160 partners, raised the deliver a coordinated program engage with networks reaching at least 5,000 stakeholders capacity of 7,078 field- of research and engagement and decision-makers to inform at least four policies/programs level extension across the food production–to– and/or market interventions supporting agrifood systems agents to provide real- consumption continuum to transformation. time, high-resolution support equitable access to 2) Data informed actions supporting agrifood systems are climate adaptation sustainable healthy diets, implemented by sub-national governments, donors, the farm advisories. improve farmer livelihoods and private sector and/or development partners, encouraging resilience, and conserve land, agrifood systems change in at least eight learning locations. Enabled shift from air and groundwater resources. 3) Farmers implement improved farming practices and/or water-intensive rice to diversify production systems on at least 0.71 million hectares more profitable and less Research Emphasis: Catalyze averting greenhouse gas emissions by 8.21 million tons resource consumptive research and engagement to CO2 equivalent. crops in India, resulting enhance equitable access to 4) Innovations in entrepreneurial rural service provision markets in farmers intensifying sustainable, healthy diets while and public and private extension systems are supported to crop management improving farmer livelihoods accelerate uptake of improved farm management practices practices and and climate resilience, and and production diversification by at least 0.58 million farmers diversifying production conserving land, air, and including 0.20 million women. on over 35,930 hectares groundwater resources. 5) Business models supporting farm product aggregation, better by more than 700,000 pricing for farmers at the farm gate and/or shortened value farmers. chains benefit at least 95,000 farmers (47,500 of whom will be women). 6) At least two food product supply chains reduce food waste and/or financial losses for food distributors, processors and/or retailers. 7) At least five local governments engage in efforts to reshape rural food environments to support access to affordable and nutritious food. 8) At least two nutrition behavior change programs operated by governments and/or NGOs provide evidence-based guidance to consumers on sustainable and healthy diets, benefiting 0.24 million people (all women). 9) Gender and equity focused nutrition approaches are included in at least two agrifood systems linkage and/or social protection programs operated by governments and/or NGOs in South Asia. Diversification of resilient agribusiness ecosystems in East and Southern Africa (ESA) - East and Southern Africa 22 Regional Integrated Initiatives 26 Terms of Reference: RII Evaluability Assessments Proposed 3 year outcomes 2022 Highlights- Reports Research Emphasis/ (individual RIIs page) Transforming Agrifood Systems in South (Table 1) Objectives Asia - CGIAR Objective: This Initiative aims to 1) At least 30 agribusinesses (40% run by women and 40% by Delivered 25 innovations support climate-resilient youth) scale climate-smart solutions supporting that involved 102 agriculture and livelihoods in 12 diversification, intensification and risk management of maize partnerships across 11 countries in East and Southern mixed systems through at least US$5 million of new finance in countries. Africa by helping millions of debt, equity and grants. smallholders intensify, diversify 2) At least 50,000 farmers, value chain actors and consumers Scaled improved farm and reduce the risks in maize- (40% women and 40% youth) in maize mixed systems start to practices in 11 countries, based farming through use climate-smart intensification and diversification practices impacting 30,000 improved extension services, with improved water and land management. farmers. Several of these small and medium enterprise 3) Investments of US$100 million enable, and two practices are showcased development, supporting strategies/policies help support, collaborative governance in 198 mother trials, governance frameworks and and management of multifunctional landscapes. 4,722 baby trials and 21 increased investment with a 4) At least 1 million farmers and other value chain actors (40% innovation trials in four gender and social inclusion women and 40% youth) access bundled digital agro-advisory countries reaching lens. and agricultural risk management products and services that 5,055 farmers. support their response to climate risks and management of Research Emphasis: Enable land and water systems for climate resilience. Initiated an accelerator smallholders to intensify, program with ten African diversify, and reduce the risks in SMEs, which were 70% maize-based farming systems women-led, to amplify through improved extension innovations in services, small- and medium- mechanization, irrigation, sized enterprise (SME) and climate-smart development, and robust agriculture. governance frameworks, with a gender equality and social inclusion lens. AgriLAC Resiliente (LAC) - Latin America and the Caribbean Objectives: This Initiative aims 1) Nutrition-sensitive innovations co-designed with local actors Scaled the Local to increase the resilience, enable agrifood systems in four regional countries to Technical Agroclimatic sustainability and effectively align the technical aspects of transition processes Committees approach competitiveness of Latin with the socio-ecological needs of at least 200,000 farmers. and assisted over 20,000 American and Caribbean 2) A digital ecosystem spanning three Latin American and rural individuals in agrifood systems and actors by Caribbean countries empowers producer associations, Guatemala, empowering better equipping them to meet agritech companies, government agencies, NGOs and public them to build resilience urgent food security needs, extension services to offer digitally enabled agro-advisory against climate change. reduce climate threats, stabilize services to at least 200,000 farmers. conflict-vulnerable 3) Low-emission strategies with development goals across Facilitated the co- communities and reduce out- agroecosystems, landscapes and value chains, reaching at production and migration. least 300,000 hectares, are integrated by national and local dissemination of governments in three Latin American and Caribbean accessible local Research Emphasis: Co- countries. agroclimatic bulletins, develops scalable 4) InnovaHub learning, knowledge management and evidence in which are now efficiently socioecological and technical four Latin American and Caribbean countries accelerate on- used by farmers to innovations that enhance local farm uptake and scaling of innovations, making them more inform their agricultural to regional climate resilience, gender-responsive, production-friendly and context-specific, decisions, driving increase food-related incomes, reaching at least 200,000 farmers. progress and fostering promote nutritious diets, and 5) CGIAR science, evidence, and tools are used by public and resilience. potentialize regional private institutions in three Latin American and Caribbean environmental services and countries to inform and shape more transformative, 27 Terms of Reference: RII Evaluability Assessments Proposed 3 year outcomes 2022 Highlights- Reports Research Emphasis/ (individual RIIs page) Transforming Agrifood Systems in South (Table 1) Objectives Asia - CGIAR critical challenges, including sustainable, mitigation-comprehensive and climate out-migration and poverty. adaptation-friendly policies, incentives and initiatives. These are then mainstreamed and scaled throughout three countries. Asian Mega-Deltas (AMD) - South East Asia and the Pacific Objectives: This Initiative aims 1) A network of inclusive learning alliances comprising at least Influenced local and to create resilient, inclusive and 200 stakeholder communities and 30 organizations in each national policies in productive deltas, which delta, supported by at least three national/sub-national Vietnam and maintain socio-ecological policies, that can scale up diversification of agrifood systems Bangladesh. In Vietnam, integrity, adapt to climatic and to accelerate adaptation by 150,000 smallholders and the National Green other stressors, and support improve management of 100,000 hectares. Growth Strategy human prosperity and 2) At least three national governments or international incorporated research wellbeing, by removing development partners work with grassroots actors, including outputs, while the systemic barriers to the scaling marginalized women and youth, to co-design gender- Department of Crop of transformative technologies equitable and socially inclusive joined-up natural resources Production included and practices at community, and food systems governance policies or strategies. data and knowledge national and regional levels.  3) Digital climate advisory and bundled services, both public and generated by the private, are used by at least 0.8 million smallholders (at least Initiative in its regional Research Emphasis: Secures 29% women) with at least one financing partnership plan directive and seasonal the globally important food established.  work plans. Alternate baskets of Asian Mega-Deltas 4) Ministries and major NGOs in at least three countries use co- wetting and drying against climate change, produced knowledge and evidence to table at least three technology in rice develops approaches that nutrition-sensitive interventions to reduce mortality, disability- cultivation was enhance the nutrition sensitivity adjusted life years and micronutrient deficiencies for 20,000 highlighted in the of deltaic food systems, and Asian mega-delta producers (80% women) and 100,000 Nationally Determined supports the development of consumers. Contributions for climate-resilient livelihood 5) High-level policymakers and development partners in at least climate mitigation in options in deltas. two deltas that are involved in knowledge integration networks Bangladesh. make public statements on the importance of One CGIAR and Asian mega-deltas and engage with the Initiative to design climate adaptation-oriented policies and projects, and investment plans with the nominal value of at least US$1.8. billion are developed. Fragility to Resilience in Central and West Asia and North Africa (CWANA)- Central and West Asia and North Africa Objectives: This Initiative aims 1) Government, civil society, private sector and international Produced 173 impactful to respond to the climate, NGOs jointly develop strategies and policies to create more results. The nutrition and agrifood efficient, inclusive and resilient national agrifood systems. AgriTech4Morocco challenges most affecting the 2) Government supports and facilitates the use of “best-bet” Innovation Challenge CWANA region by applying, genetic innovations developed for the region. trained 32 teams to scaling and supporting 3) Government supports and facilitates the on-farm and ex-situ propel science-based effective, resilience-focused conservation of agrobiodiversity. agrifood solutions. This solutions, reducing fragility and 4) Government, civil society and private sector scale up bundled involved capacity conflict, and empowering all solutions to decompose yield gaps. building, mentorship, stakeholders for change, while 5) Government, civil society and private sector put into practice and networking in the integrated management of food, land, water and energy partnership with the systems. Accelerate for Impact 28 Terms of Reference: RII Evaluability Assessments Proposed 3 year outcomes 2022 Highlights- Reports Research Emphasis/ (individual RIIs page) Transforming Agrifood Systems in South (Table 1) Objectives Asia - CGIAR minimizing and/or mitigating 6) Government, civil society and private sector scale up Platform and national any trade-offs. innovations and digital tools for food value chain climate risk partners in Morocco. management. Research Emphasis: Formulates and implements customized climate adaptation and mitigation strategies to address the unique needs of dryland farming, integrating the management of fragility risks. West and Central African Food Systems Transformations (WCA) - West and Central Africa Objectives: This Initiative aims 1) At least 80,000 smallholder households will have access to Developed a multi-crop to improve nutrition, incomes climate-resilient nutrient-dense crop varieties, and at least household-level survey and food security within the 16,000 of them will use five climate resilient, nutrient-dense to assess social context of climate change in crop varieties and 6 good agronomic practices. constraints to gender West and Central Africa through 2) At least 10 key partners in the next phase implementation and generational nutritious, climate-adapted and plans consistently use three validated scaling tools. equality in agribusiness market-driven food systems. 3) At least four governments use inclusive approaches to across West and Central landscape management and 100 rural communities develop Africa. Research Emphasis: Provides informed and inclusive land and water management plans coordinated partnership in the that will diversify income from agriculture and increase Facilitated the transformation of agrifood production to create jobs and stability. establishment of systems in communities 4) Timely climate information and early warning systems for agribusiness hubs affected by climate change improved decision-making are in use by 3 million farmers, 30 by cataloging 15,285 focusing on building the value chain actors and three governments. value chain actors. entrepreneurship capacities of 5) The Women Empowerment in Agriculture Index shows a 20% women and youth. increase in the target countries. 6) Household dietary diversity scores increase by at least 30% in the target countries. 7) At least 20,000 youth and 15,000 women are engaged in value-added activities related to agriculture, and at least 50% of these will have access to credit. 29 Terms of Reference: RII Evaluability Assessments Annex 2: GENDER Platform Evaluability Assessment – Executive Summary See full Inception Report To operationalize the ‘evaluability’ principle, and in line with other relevant standards and principles, the IAES recently launched Conducting and Using Evaluability Assessments In CGIAR: CGIAR Evaluation Guidelines. As per SIMEC’s request (October 2022), an Evaluability Assessment (EA) was conducted for the evaluation of the GENDER Platform, to assure them of its readiness for the independent external evaluation by IAES, given that it formally completed its first year in 2021. An EA served as a pre-evaluation tool, which allowed the presence and status of key MELIA components related to performance to be checked. The specific objectives of the evaluability assessment of the GENDER Platform were to: • Clarify the feasibility and scope, and thus ensure a cost-effective and timely evaluation • Provide a reflective learning process on the importance of MEL-related inputs for the evaluative process • Enable consensus building among the core stakeholders, i.e., GENDER Platform management and funders, to manage expectations of what an evaluation will produce • Inform evaluation design and solidify lines of inquiry; i.e., evaluation criteria and ToR • Map and identify the availability of additional documentation for evaluation • Ensure alignment with the Evaluation Framework’s standards • Provide management evidence to advocate for necessary MEL resourcing and capacity. Methodology The overall EA approach was collaborative and learning-focused. In line with the objectives and the estimated timing of the GENDER Platform evaluation, the EA was guided by a core analytic framework. Across the following five (out of 6) assessment domains, fourteen (14 out of 4023) questions were answered: 1) Intervention logic; 2) MEL systems and resources; 3) Gender diversity and inclusion; 4) Context and environment; 5) Management and stakeholder engagement and response. The EA exercise took one week, including integration into the evaluation TOR. The EA was jointly carried out by the GENDER Platform team, with facilitation and Quality Assurance (QA) from IAES. Self- Evaluability Assessment was conducted the GENDER platform team, including the Director, Consultant - Program Management Specialist (the Consultant), and the MEL focal point. IAES conducted three scoping interviews towards EA. The principles of triangulation and internal coherence of the GENDER Platform and external IAES perspectives were employed during analysis to build responses to the EA questions, make assessments and identify areas for attention and further inquiry during the evaluation. Limitations of the exercise were 23 A fully-fledged comprehensive EA (either fully or largely independent) would cover 40 questions across 6 domains: A. Intervention logic; B. MEL systems and resources; C. Gender, diversity, and inclusion; D. Long-term evaluability; E. Context and environment; F. Management and key stakeholder engagement and support. 30 Terms of Reference: RII Evaluability Assessments associated with the newness of the MEL expert to the GENDER Platform, and the newness of Evaluability Assessment in the CGIAR context. Given that the EA was conducted as part of evaluation scoping, it was also a challenge to guide consultations away from evaluation-oriented issues of achievements and challenges to evaluability-related questions of data and systems for monitoring; and the larger context of CGIAR reform. Findings and conclusions The objectives of the evaluability assessment were met. The EA results show readiness of the evaluand (the GENDER platform) for evaluation. More specifically: A. Across the 5 domains, 11 out of 14 questions were assessed as Yes/green; and 3 questions were assessed ‘May be’/orange (Figure A1 and Table A1). B. Three ‘May be’/orange ratings related to the quality of indicators, data disaggregation, and accessibility – with caveats, to be considered in the evaluation design and data collection (Table A1). C. Across the 14 responses the following overarching needs were identified: • To strengthen the capacity on understanding and use of the EA • To explicate key MEL-related and broader inputs and processes, for a common understanding i.e., records of and typology of partners of the GENDER platform • To enhance understanding of the CGIAR Evaluation criteria as per the CGIAR Evaluation Framework and Policy, aligned to the QoR4D framework. This evaluation does not cover the evaluation of the QoS criteria based on scoping exercise. • Further inquiry and recommendations related to TOC revision, the profile of a full-time MEL expert, and guidance on how to more specifically monitor continued comparative – in response to specific requests from the Platform. D. The EA process encouraged reflection by the GENDER Platform and learning by IAES on the utility of the tool; towards assuring the cost-effective and smooth design and implementation of the evaluation. Main Report from the Evaluability Assessment Overview In 2022, the CGIAR System Council and Board approved a CGIAR Evaluation Framework and a revised Evaluation Policy. Rigorous, independent, external evaluations are foundational to CGIAR’s effort to inform the design of interventions, provide actionable evidence to support management and governance decisions, and ensure a high level of accountability to donors. ‘Evaluability’ is one of 15 standards and principles of the Evaluation Framework – a reference point for professionalism within research-for- development evaluation that underpins how evaluation is conducted in CGIAR. To operationalize the ‘evaluability’ principle, and in line with other relevant standards and principles, the IAES recently launched Conducting and Using Evaluability Assessments In CGIAR: CGIAR Evaluation Guidelines. As per SIMEC’s request (October 2022), an Evaluability Assessment (EA) was conducted for the evaluation of the GENDER Platform, to assure them of its readiness for the independent external evaluation by IAES, given that it formally completed its first year in 2021. An evaluability assessment served as a pre- evaluation tool, which allowed the presence and status of key MELIA components related to performance to be checked. The specific objectives of the evaluability assessment of the GENDER Platform were to: • Clarify the feasibility and scope, and thus ensure a cost-effective and timely evaluation • Provide a reflective learning process on the importance of MEL-related inputs for the evaluative process 31 Terms of Reference: RII Evaluability Assessments • Enable consensus building among the core stakeholders, i.e., GENDER Platform management and funders, to manage expectations of what an evaluation will produce • Inform evaluation design and solidify lines of inquiry; i.e., evaluation criteria and ToR • Map and identify the availability of additional documentation for evaluation • Ensure alignment with the Evaluation Framework’s standards • Provide management evidence to advocate for necessary MEL resourcing and capacity. Methodology The overall EA approach was iterative, collaborative, and learning-focused. In line with the objectives and the estimated timing of the GENDER Platform evaluation, the EA was guided by a core analytic framework of the guidelines Conducting and Using Evaluability Assessments In CGIAR: CGIAR Evaluation Guidelines. The assessment was limited to five (out of 6) domains: 1) Intervention logic; 2) MEL systems and resources; 3) Gender diversity and inclusion; 4) Context and environment; 5) Management and stakeholder engagement and response, which covered 14 questions (see table A2). The whole exercise took one week, including integration into the evaluation TOR24. The EA was jointly carried out by the GENDER Platform team, with Quality Assurance (QA) from IAES. The process they followed included: • The Director, Consultant - Program Management Specialist (the Consultant), and the part-time MEL focal point were provided with the template to fill in Y/N answers, specify supporting documented evidence; and provide a narrative explanation of their scoring. • The MEL expert and Consultant filled in designated areas. • The Platform Director validated their work and complemented it with additional narrative-based explanations on key aspects. • Consultation between the Gender Platform’s MEL expert and Consultant and two representatives of IAES; an evaluation team leader observed. The discussion focused on the clarification of concepts and precise requirements for documentation. Subsequently, the Gender Platform team shared a prefilled template with IAES. • IAES analyzed responses and triangulated against selected background information based on scoping. The principles of triangulation and internal coherence of the GENDER Platform and external IAES perspectives were employed during analysis to build responses to the EA questions, make assessments and identify areas for attention and further inquiry during the evaluation. Limitations of the exercise were associated with the newness of the MEL expert to the GENDER Platform, and the newness of Evaluability Assessment in the CGIAR context. Given that the EA was conducted as part of evaluation scoping, it was also a challenge to guide consultations away from evaluation-oriented issues of achievements and challenges to evaluability-related questions of data and systems for monitoring; and the larger context of CGIAR reform. Findings and conclusions The EA presents findings and conclusions against the 14 questions by the 5 domains, related to the core EA-specific objectives concerning the results framework, the planning, monitoring, and evaluation system, and human and financial resources for monitoring and evaluation. The EA made it possible to make set objectives and broadly assess the adequacy of the theory, with potentially available data enabling an evaluative judgment to be formulated. 24 For subsequent integration into Inception report for the evaluation. 32 Terms of Reference: RII Evaluability Assessments E. Across the 5 domains, 11 out of 14 questions were assessed as Yes/green; and 3 questions were assessed ‘May be’/orange (Figure A1 and Table A1) F. Three ‘May be’/orange responses related to the quality of indicators, data disaggregation, and accessibility – with caveats, to be considered in the evaluation design and data collection (Table A1). G. Across the 14 responses the following overarching needs were identified: To strengthen the capacity on understanding and use of the EA • To explicate key MEL-related and broader inputs and processes, for a common understanding i.e., records of and typology of partners of the GENDER platform • To enhance understanding of the CGIAR Evaluation criteria as per the CGIAR Evaluation Framework and Policy, aligned to the QoR4D framework. This evaluation does not cover the evaluation of the QoS criteria based on scoping exercise. • Further inquiry and recommendations related to TOC revision, the profile of a full-time MEL expert, and guidance on how to more specifically monitor continued comparative – in response to specific requests from the Platform. H. The EA process encouraged reflection by the GENDER Platform and learning by IAES on the utility of the tool; towards assuring the cost-effective and smooth design and implementation of the evaluation. Figure A1. GENDER Platform - EA result using stoplight system Indicator ‘Yes’ to 11 core Evaluability assessment criteria questions met ‘May be’ to 3 with caveats Evaluability assessment criteria partially met None of the evaluability assessment criteria are met None Source: CGIAR IAES Conclusion: The EA results show readiness of the evaluand (the GENDER platform) for evaluation. Recommendation: In response to the EA findings and conclusions, in proceeding, the evaluation should deploy a participatory approach to promote buy-in and learning and ensure a repository of documents to support the evaluation process to meet its objectives and the needs of the three stakeholder groups. Table A2. GENDER Platform - EA results and recommendations, decision support framework using stoplight system Y/N Follow-up needed/ inquiry integrated Core Evaluability Domains and Assessment Criteria; by Com into the evaluation design/limitation Key findings ment noted A. Intervention logic: To be evaluable, an intervention must clearly describe what it hopes to achieve and how. Intervention logic represents the overall logical integrity of the intervention and should be supported by a robust ToC and a body of evidence that lends credibility/plausibility to the ToC. 33 Terms of Reference: RII Evaluability Assessments Y/N Follow-up needed/ inquiry integrated Core Evaluability Domains and Assessment Criteria; by Com into the evaluation design/limitation Key findings ment noted 1. Theory of change: A clear logic across the proposal's Y Theory-based evaluation approach. narrative in relation to figures and annexes diagrams and frameworks, including a ToC and a result framework impact pathway linked to indicators. 2. Quality and quantity of evidence base: ‘Final’ Annual Y Eval design and potential learning/ report (AR2022) includes relevant summaries, evidence, recommendation: Exploration of how to and the GENDER Platform website and newsletter cover update ToC due to expansion context to relevant deliverables for outputs and outcomes. ToC needs include youth and other socially updating and given the expansion context to include youth disadvantaged groups. and other socially disadvantaged groups. 3. Clarity of intervention additionality, comparative Y Potential learning/ recommendation: advantage25 , and spheres of control: Partners as part of guidance by the evaluators on how to more the bid and their extended partnerships are the Platform's specifically monitor continued comparative comparative advantage in terms of experience and advantage. expertise. No framework developed to track or measure. 4. Feasibility: Feasibility of the ToCs’ causal logic should be Y Eval design: Elaborate context, explication framed in the context of funding, the type of funding, and under efficiency, and facilitator and expenditure percentage. Notably, the GENDER Platform inhibitors to implementation. proposal was developed with a longer-term vision in mind, and so primary outcomes are expected to be reached as far into the future as 2028 and 2030. 5. Complexity: There are complex relationships as with most Y Attention to complexity is one of the CGIAR-wide interventions, especially as gender research principles of the Eval Framework. Analysis has not been taken on board as clearly as with climate and sampling framework to contextualize change and other platforms. Any guidance on how to and capture. enhance the platform's impact within this complex situation would be appreciated. B. MEL systems and resources: To be evaluable, an intervention must have a credible plan to track its contribution to outcomes. It should reflect a vision of how monitoring and evaluation activities will fulfill accountability, delivery, & learning needs. The MEL system must generate relevant and quality data, most often by an intervention’s indicators. A baseline is a necessary starting point against which to assess intervention performance & results. 6. Quality of MEL Framework: The Platform has a results Y Potential learning/ recommendation: any framework, (proposal Annex 2). Each year the POWB was suggestions emerging from the evaluation developed according to the CGIAR format and reported on as to whether these documents were using the CGIAR Annual Report format. The development of sufficient to monitor and assess progress a MEL plan for the GENDER Platform was to be the first against the results framework. activity of the MEL staff to be hired (as noted in the MELIA section of the AR 2020 and 2021) but that hire has not yet taken place. 7. Information resources and system in support of MEL: The Y Potential learning/recommendation: designated MEL person is recruited for 50 days specifically prioritize MEL and advice on the type of for the entire evaluation. A 2021 round of recruitment was person and role would be appreciated unsuccessful, and a second round is starting up, advice on the type of person and role would be appreciated 8. Quality of indicators or other measures: Annual Reports May Evaluation design: In the M&E continuum were generated from MARLO Data entry. The results be assessing pathway form outputs to impacts framework includes indicators, which the evaluators along the TOC is key. Uptake and quality of should review and consider, taking into account that many 25 Towards the development of the related concept note, at the time of finalizing the EA guidelines the following blog was considered https://iaes.cgiar.org/isdc/news/effectively-using-concept-comparative-advantage-within-cgiar 34 Terms of Reference: RII Evaluability Assessments Y/N Follow-up needed/ inquiry integrated Core Evaluability Domains and Assessment Criteria; by Com into the evaluation design/limitation Key findings ment noted of the outputs are intended to be achieved in the years to selected outputs would be subject to inquiry: come, far beyond the period of this evaluation. A relevance for example. 9. Quality of Baseline: Doing research through grants for Y Eval design: Module-specific baseline partners should follow those gaps for example. The documents exist; to be considered in narrative of the proposal describes gaps to be identified interview guides and online survey design by the Platform, in terms of evidence (module 1), methods for comparability and measurement. and tools (module 2), and capacity needs (module 3). Reviews of these needs and gaps have taken place and can be considered as baselines in each area C. Gender, diversity, and inclusion: The CGIAR is committed to the inclusion of women, youth, and socially excluded and vulnerable groups. 10. Clarity of partners and end-user groups: Partners are Y Eval design/implementation: Typology is not defined in the proposal, with new partners also added clear, related to the Platform’s comparative each year in the annual reports, listed in two tables (CGIAR advantage and value – added to internal and non-CGIAR partners) and internal partners and stakeholders. 11. Data disaggregation (gender, youth, other): Related to May Eval design: Availability of sex- largely gender-disaggregated data, one of the roles of the be disaggregated data to be established, as Platform is to foster an environment where gender is a high well as other relevant dimensions of priority and therefore the disaggregation of data by inclusion, i.e., in partnerships. gender would be the norm across the whole CGIAR. This question does not seem as relevant for the Gender Platform itself but for example, where training has taken place, we can identify numbers of male/female participants. E. Context and environment: To be evaluable, an intervention must be accessible to evaluators. 12. Accessibility: Only certain interviews would happen May Eval design: The timing of evaluation and between 14-16 November face-to-face in Nairobi. The be accessibility considered for in-person with availability assessment will take longer then by Nov 4 and remote interviews, to enhance accessibility we would like to reach out with the final ToR and possibly of stakeholders. Due to the set-up of new other explanatory document to make the discussions platforms, and considering Christmas break, fruitful. Otherwise online as needed by agreed and realistic evaluation timing is of the essence. It will be deadlines. Before the evaluation officially starts to considered as a limitation, with mitigation schedule accordingly with sufficient time for organization being a slightly extended data collection provided. timeline. F. Management and key stakeholder engagement and support: To be evaluable, management and stakeholders must be active participants in the evaluation process. 13. Return to management and other key stakeholders: Y Eval design/inception report: Evaluation Evaluation criteria and key questions address the issues of criteria are guided by the CGIAR Evaluation importance to stakeholders. The criteria of relevance, Policy, which is framed by QoR4D and effectiveness, efficiency, and sustainability portrayed OECD/DAC evaluation criteria. The decision through the questions is being reviewed and a final version not to assess Quality of Science, aligned is pending. We understand QoR4D criteria as a global with the QoR4D framework has evolved component with very specific definitions will not be part of during the review of the questions. the evaluation criteria but rather some partial terminology (relevance and effectiveness) somehow differently defined in addition to efficiency and sustainability. 14. Demand from and Participation of Key Stakeholders: Y Eval design: Scoping exercise has confirmed During the latest workshop after mid-October, the GENDER openness to the process, i.e., module lead Platform Director made a clear statement addressed to all interview. The snowball method would need staff subject to this evaluation to take their time and effort to be used to identify selected partners to respond well and honestly. The partners of the Platform internally and externally when needed. 35 Terms of Reference: RII Evaluability Assessments Y/N Follow-up needed/ inquiry integrated Core Evaluability Domains and Assessment Criteria; by Com into the evaluation design/limitation Key findings ment noted are spread across the world so most interviews will be virtual. Please do make allowances for the fact that the evaluation spans the end of the year period and set realistic deadlines accordingly. 36 Terms of Reference: RII Evaluability Assessments Annex 3: Evaluability Assessment Framework Indicator/ Stop Light Domain Assessment Criteria Core Scoring criterion System Theory of change: a. Is there an explicit ToC (or logical Framework) that describes the intervention’s expected results and 1 impact pathways?26 b. Are the ToC model and narrative well aligned? (e.g., the narrative explains the model, and elaborates the causal logic with examples). c. Have ToC assumptions that are essential to the working of the ToC been explicitly stated, in sufficient detail that they can be assessed? d. Are there sufficient causal linkages to plausibly suggest that intervention activities are sufficient to produce desired outcomes (e.g., End of Initiative outcomes)? A. Intervention logic: Quality and quantity of evidence base: To be evaluable, an e. Is the ToC supported by a credible body of evidence 2 intervention must (primary or secondary)? clearly describe what it hopes to achieve and Clarity of intervention additionality, comparative how. Intervention logic advantage27 and spheres of control: 3 represents the overall f. Has evidence been provided around specific claims logical integrity of the made about the intervention’s comparative intervention. This logic advantage and contribution, and that of other should be supported by actors, to the achievement of the desired results a robust ToC and a (Spheres of control and influence of an intervention) body of evidence that which could be assessed? lends credibility/ Feasibility: plausibility to the ToC. g. Are the ToCs’ causal logic realistic and feasible to 4 achieve within the timeframe and resources allocated? Complexity: h. Are there complex relationships between different 5 intervention components that will make the attribution of results difficult to assess? i. Is the ToC appropriately aligned and linked with other nested ToCs to realize broader strategic goals (e.g., organization, country)? j. Is there an explicit written acknowledgment of how complex change processes are expected to occur (i.e., nonlinearity, emergence, adaptation, feedback loops, etc.)? 27 Towards the development of the related concept note, at the time of finalizing the EA guidelines the following blog was considered https://iaes.cgiar.org/isdc/news/effectively-using-concept-comparative-advantage-within-cgiar 37 Terms of Reference: RII Evaluability Assessments Indicator/ Stop Light Domain Assessment Criteria Core Scoring criterion System Quality of MEL framework: 6 a. Does the intervention have a MEL framework in place that is fit to generate evidence to support all the key events in the ToC and generate the data in support of the Results Framework and PRMF? b. Does the MEL framework specify a monitoring plan for indicators (specifying who, when and how indicator data will be collected)? B. MEL systems and Information resources and systems in support of MEL: 7 resources: To be c. Are there designated MEL personnel at the level of evaluable, an evaluand? intervention must have d. Is the/a MEL system generating the data in support a credible plan in place of the Initiative/intervention Results Framework and to track its contribution PRMF (information system, excel)? to outcomes. The plan Quality of approach to learning: --- should include a e. Are there mechanisms in place for making use of unified vision of how findings from MELIA products (reviews, evaluations, M&E activities will fulfil impact assessments) for decision-making? accountability, delivery, f. What evidence is there about quality (strategy, and learning needs. uptake) and use of learning opportunities in the The MEL system must past? generate relevant and Quality of indicators or other measures: 8 quality data. Most g. Does the intervention results framework include often, this data is indicators that are appropriate to evidence its ToC defined by an from outputs to impacts? intervention’s h. Do indicators include both human/social and indicators. Having environmental qualities? appropriate indicators i. Are there indicators around science delivery and that are aligned with quality? desired results is j. Are the indicators SMART?28 essential. A baseline is Quality of baseline: 9 a necessary starting k. Does the intervention have baseline evidence point against which to against its targets towards its objectives? assess intervention l. Is baseline data related explicitly to the results performance and framework of an intervention (i.e., indicators)? results. Quality of results statements? m. Are results statements for the evaluand (output, outcome, et al., statements) sufficiently clear for success to be recognizable? n. Do the result “types” make sense? o. Are elements framed correctly? (e.g., outcome statements are actor-specific and presented in active language: i.e., answers the question ‘who is doing what differently?’) C. Gender, diversity, Clarity of partners and end-user groups: 10 and inclusion: The a. Are partners and other stakeholders clearly defined, CGIAR is committed to within spheres of control and/or influence, along the inclusion of women, with how their interests may coincide or conflict? youth, and socially 28 S-specific, M-measurable, A-achievable, R-realistic, T-timebound. 38 Terms of Reference: RII Evaluability Assessments Indicator/ Stop Light Domain Assessment Criteria Core Scoring criterion System excluded and b. Have important differences between end-user vulnerable groups. To groups been identified, concerning differences in be credible and their expected roles and results? legitimate, CGIAR Data disaggregation (gender, youth, other): 11 research must be c. Do existing data allow for data disaggregation based on the inclusion according to targeted cross-cutting groups? of the end-users it d. For assessing inputs to QoS - is evidence in place hopes to reach. about young and mid-career researchers? Sustainability: --- D. Long-term a. Have the expectations about the nature and evaluability: Many of duration of the sustainability of the intervention the impacts of CGIAR and/or its effects been made clear enough to be research will not be evaluable? recognized until long b. Is there clarity on the linkages across the ToC after intervention towards the potential and actual sustainable delivery. development impact, as appropriate? Accessibility: 12 a. Is there anything about the timing of a planned E. Context and evaluation that would make it difficult/impossible to environment: To be conduct (e.g., seasonality, budget allocations, public evaluable, an holidays, local elections)? intervention must be b. Are there security or political issues that would make accessible to a planned evaluation difficult/impossible to evaluators and key conduct? stakeholders in the c. Are there any geographical constraints on evaluation. accessibility, either by the evaluation team or by local stakeholders in the intervention Return to management and other key stakeholders: 13 a. Do evaluation criteria and potential questions address the issues of importance to stakeholders? b. Is there a commitment to learning from evaluation F. Management and findings by any of the stakeholders? key stakeholder c. Do stakeholders have mechanisms and the engagement and capacity to learn from potentially negative support: To be evaluation findings? evaluable, d. Is there likely to be a process for ensuring management and Management engagement and response to the stakeholders must be evaluation findings? active participants in Demand from and participation of key Stakeholders: 14 the evaluation process. e. Do stakeholders understand expectations about their role and potential contribution to an evaluation? f. Are stakeholders available and ready to participate in an evaluation? 39 Terms of Reference: RII Evaluability Assessments Annex 4: List of Stakeholders Consulted Toward TOR Development Name Sex Initiative Affiliation CGIAR Center (M/F) 1. Martin Kropff M One CGIAR Director, RAFS Science Group CGIAR 2. Maya F CGIAR Managing Director, Africa; RII Coordinator Alliance Biodiversity & Rajasekharan CIAT 3. Roberto Rocha M CGIAR Senior Manager CYMMIT Correa 4. Michael Baum M CWANA Lead/Director ICARDA 5. Rhiannon F CWANA Project Coordinator ICARDA Crichton 6. Innocent Bikara M CWANA MEL (IA) ICARDA 7. Maha Al-Zu’bi F CWANA Regional Researcher IWMI 8. Bjoern Ole Sander M AMD Lead/Director IRRI 9. Yuji Enriquez M AMD MEL Manager IRRI 10. Timothy Joseph M TAFFSA Lead/Director CIMMYT Krupnik 11. Purnima Menon F TAFFSA Senior Director IFPRI 12. Aminou Arouna M WCA Program Leader & Impact Assessment Africa Rice Economist 13. Regina Kapinga F WCA Country Representative and Head of IITA Advocacy & Resource Mobilization 14. Robert Asideu M WCA Senior Manager/Coordinator IITA Sub-Total M = 9; CWANA=4; F=5 AMD=2; TAFFSA=2; WCA=3 40 Terms of Reference: RII Evaluability Assessments Annex 5: Inventory of Documents and Reference Materials Reference Materials Independent Science for Development Council. 2022. Identifying and Using CGIAR’s Comparative Advantage. Rome: CGIAR Independent Advisory and Evaluation Service. Identifying and Using CGIAR’s Comparative Advantage | IAES | CGIAR Independent Advisory and Evaluation Services Inventory as of July 31 # Document Title Year Initiative 1. From Fragility to Resilience in Central and West Africa and North Africa 2021 CWANA (F3R-CWANA) 2. ISDC Review of 12 Initiative Proposals 2022 CWANA 3. From Fragility to Resilience in Central and West Africa and North Africa, 2022 CWANA ISDC Feedback Response 4. CWANA POR Version History Documentation 2022 CWANA 5. CWANA Results Framework 2022 CWANA 6. Output Reporting Innovation Development 2022 CWANA 7. Standard Indicator Description Sheet, Reporting of Core Indicators 2022 CWANA Categories at Output Level 8. Standard Indicator Description Sheet, Reporting of Core Indicators 2022 CWANA Categories at Outcome Level 9. Plan of Results and Budget 2023-24 2022 CWANA 10. Evidence Examples to Justify Innovation Readiness Level 2022 CWANA 11. The Reflect Process: Adaptive Initiative Management and Reporting 2022 CWANA 12. TAFS-WCA overview: Women and Youth digital based agribusiness 2023 WCA 13. Briefing Note: Inclusive digital agribusiness models for Jobs and 2023 WCA innovation scaling in WCA 14. CGIAR Initiatives on Central African Food Systems Transformation 2022 WCA 15. WCA Change Management TOC 2023 WCA 16. Monitoring, Evaluation and Learning and Impact Assessment (MELIA) 2023 WCA Framework: E-registration and Baseline survey of TAFS-WCA initiative in six countries 17. Policy Innovation Program and Impact Assessment Program 2023 WCA 18. TAFS-WCA End of Initiative Outcomes 2023 WCA 19. Proposal. Transforming AgriFood Systems in West and Central Africa 2021 WCA (TAFS-WCA). 20. Pause, Reflect and Stakeholders Workshop of TAFS-WCA. 2022 2023 WCA Achievement Overview. 21. Policy, Innovation Systems and Impact Assessment Program. Technical 2022 WCA Report. 22. Asian-Mega Delta Consolidated Budget 2022 AMD 23. CGIAR Initiative on Asian Mega-Deltas, Technical Report 2022 AMD 24. CGIAR Initiative on Asian Mega-Deltas, Annual Technical Report 2022 AMD 25. Securing the food system of Asian Mega-Deltas for climate and 2021 AMD livelihood resilience 41 Terms of Reference: RII Evaluability Assessments Annex 6: Requested points of Engagement and Tasks for the MEL or a Designated Focal Point in Evaluability Assessment Steps Phase MEL focal point key tasks 1-2 Scoping • Review the CGIAR guidelines on conducting an evaluability assessment (2022) • Upload preliminary relevant and reliable extant program documentation and data for the evaluation against the requested detailed list of required documentation. This will constitute the evaluation repository. • Review draft TORs with a focus on EA objectives towards utility for the Initiative • Review the Inception note for clarity and fact checking • MEL staff facilitate stakeholder involvement 3-4 Application of EA • Participate in the facilitated engagement on filling in the EA tool - the framework and spreadsheet based on the condensed core parameters Collecting Evidence • Fill in the gaps and complement with supporting documentation where necessary.29 • Serve as a key informant about the MEL system for an interview and respond to online surveys if applicable. • If needed, support/facilitate access to interviewees/key informants to answer questions from the evaluation team. 5. Development and • Participate in the validation of preliminary findings, conclusions, and validation of EA results recommendations for an individual RII report. • Coordinate comments from the RII team on the draft EA report for initiative from the RII team and ensure they are sent to IAES within the stipulated time. 6 Engage with EA results: • Review Synthesis report across four RIIs for fact-checking. communication and • Contribute to the development of the Management Response, e.g., from Big Data facilitating use Platform Evaluation. • Upon finalizing EA results and MR, support the use of findings to ensure that key actions are implemented and learning is woven into the programming. 29 In line with the CGIAR Evaluability Assessment guidelines, this follows if the evaluability assessment was conducted as an integrated part of the inception phase. 42 Independent Advisory and Evaluation Service Alliance of Bioversity International and CIAT Via di San Domenico, 1 00153 Rome, Italy IAES@cgiar.org https://iaes.cgiar.org/