Evaluation of the CGIAR Research Program on Wheat Volume 2 – Annexes April 2015 Wallace Beversdorf (Team Leader) Sylvie Brouder Paramjit (Pammi) Sachdeva Rasheed Sulaiman Deborah Templeton This evaluation has been commissioned by the Independent Evaluation Arrangement (IEA) of CGIAR. The Independent Evaluation Arrangement (IEA) of CGIAR encourages fair use of this material provided proper citation is made. Correct citation: CGIAR-IEA (2015) Annexes to the Evaluation of CGIAR Research Program on Wheat. Rome, Italy: Independent Evaluation Arrangement (IEA) of CGIAR iea.cgiar.org Wheat Evaluation, Annexes, April 2015 Table of contents ANNEX A. - Evaluation Team Members – Short Bios ........................................... 2 ANNEX B. - Country visits ..................................................................................... 3 ANNEX C. - List of people met .............................................................................. 5 ANNEX D. - List of sample projects ..................................................................... 18 ANNEX E. - Assessment of publications ............................................................. 21 ANNEX F. - WHEAT researcher survey ............................................................... 26 ANNEX G. - Sample project assessment - TEMPLATE ......................................... 49 ANNEX H. - Peer review scoring of a random sample of publications – TEMPLATE 54 1 iea.cgiar.org Wheat Evaluation, Annexes, April 2015 ANNEX A. - Evaluation Team Members – Short Bios Wallace (Wally) Beversdorf - Evaluation Team Leader. Since 2005 Wally has been an independent consultant, involved in several reviews of research, including CSIRO Flagship program reviews and 2007 mid-term review of the Generation Challenge Program. Before 2005 he worked over 10 years in the private seed sector, including positions as Head of Novartis world-wide R&D, Head of Syngenta Plant Science and Vice President for Syngenta Biotechnology. He has also worked in the academia, holding positions at the Guelph University. He has a PhD in plant breeding and genetics. Sylvie Brouder - Evaluation Team Member. Since 2005 Sylvie has served as Professor of Agronomy at Purdue University where she has been working for nearly 20 years. She is also Director of the Purdue University Water Quality Field Station. Her research interests include crop nutrient use efficiency, agroecosystem viability and sustainability and ecophysiology linked to abiotic stress tolerance. Through her teaching, outreach and consultancy activities she has been involved in international agricultural research and science agenda development. She has PhD in ecology. Deborah (Debbie) Templeton - Evaluation Team Member. Until September 2013 Debbie worked for ACIAR for over 10 years primarily managing the Impact Assessment program. Between ACIAR employment, Debbie also worked for 3 years as social scientist (impact assessment specialist) at IRRI. She has experience in capacity building in research evaluation and impact assessment and in managing and conducting impact assessments. She has PhD in economics. Rasheed Sulaiman - Evaluation Team Member. Rasheed is the Director of the Centre for Research on Innovation and Science Policy in India (since 2006). Prior to that, he worked as senior scientist at the National Centre for Agricultural Economics and Policy Research at ICAR. He has expertise in agricultural extension systems and policy, and agricultural innovation system. His research also covers the role of private sector and public-private partnerships, developing new approaches to reaching rural women and evaluation of ICTs in agriculture. He has a PhD in agricultural extension. Paramjit (Pammi) Sachdeva - Evaluation Team Member. Pammi is specialized in program and institutional assessment and HR management with expertise also in capacity development, systems analysis and organizational design. Since 2001 he has worked as an independent consultant and been involved in a number of external reviews of CGIAR Centers and programs, and in international development project and human resource management consultancies. Previously he worked at the World Bank as senior management specialist and advisor and earlier in his career at ISNAR as senior research officer. He has a PhD in social systems sciences. 2 iea.cgiar.org Wheat Evaluation ANNEX B. - Country visits Date Country Team Main schedule members 21-24 Sep Turkey WB, DT Ankara visit CIMMYT/ICARDA office Visit research institutes Izmir Visit Regional Cereal Rust Research Center in the Aegean Agricultural Research Institute 24-26 Sep Lebanon WB, DT Beirut Terbol, AREC, Kfardan LARI ICARDA offices in Beirut 26 Sep – 1 Ethiopia WB, DT Addis Ababa Meeting with ICARDA & CIMMYT staff and presentation on Collaborative Activities Oct with Ethiopian NARS WB, DT Debre Zeit Debre Zeit ARC, Green house, single race & international nurseries, breeding activities, seed producers & farmers seed producers, on farm trials and Gonde seed farm WB, DT Asella KARC, irrigation facility, international and single race nurseries, seed labs, Breeding and Seed Activities WB, DT Addis Ababa EIAR Quality Lab and Meeting with EIAR management 1-4 Oct Morocco WB, DT Casablanca INRA campus (greenhouse, biotech of INRA, quality lab and future building of ICARDA) Meeting with ICARDA staff Bread wheat presentation, Durum wheat presentation 3 iea.cgiar.org Wheat Evaluation Date Country Team Main schedule members Marchouch Marchouch station (all wheat personnel attending) Annoceur Annoceur station, visit of durum off-season. Focus on RWA screening. Gigu Arrival to farmers communities in Gigu together with professor Boulif (ENA Meknes), presentation of farmers school (ICARDA's contribution) and of IFAD activities on-farm in Meknes area, visit of the farm and lunch with farmers 8—13 Nov Bangladesh RS WRC Dinajpur meet with WRC Scientists and DAE (Department of Agricultural Extension) and BADC (Bangladesh Agriculture Development Corporation RS Rangpur Discuss with CSISA-CIMMYT, SRFSI staff and OFRD/BARI Scientists NGO partners (RDRS, etc.) RS Dhaka Discuss with BARI authorities/Seed Certification Agency 14-17 Nov India RS Arra and Zero tillage wheat sowing, interaction with service providers, farmers , KVK and other SB Patna partners RS Samatipur, Visit fields with bed planting, zero tillage intercropping of maize and interaction with SB Pusa women farmers in Bandra Visit to BISA farm-Pusa Interaction with Head, IARI Research Station of RAU officials Interaction with ICAR and other partners in ICAR complex-Patna 4 iea.cgiar.org Wheat Evaluation ANNEX C. - List of people met SURNAME, Name Position Organization Country visit Abeyo, Bekele Wheat Breeder, Global Wheat Program CIMMYT Ethiopia Abinasa, Mohammed Wheat Breeder ICARDA Ethiopia Adhikari, Mani Krishna ADO CIMMYT Bangladesh Lebanese Agricultural Research Afram, Michel Antoine President – Director General Institute Lebanon Research Extension Specialist – Technology Promotion Debre Zeit Agricultural Aliy, Sheriff Research Center EIAR Ethiopia Amil, Rola PhD Student ICARDA Lebanon Amri, Ahmed Head, Genetic Resources Section ICARDA Morocco Andaloussi, Abbad Head , Plant Protection Department INRA Morocco Asaad, Siham Head, ICARDA’s Seed Health Laboratories ICARDA Lebanon Director, Technology Multiplication Directorate and Coordinator, Deployment of Atilaw, Abebe Rust Resistant Wheat Varieties in Ethiopia EIAR Ethiopia Chief Executive Officer of ACIAR, Pacific Countries representative on the CGIAR Fund Austin, Nick Council ACIAR Australia Badstue, Lone Gender Specialist, SEP CIMMYT Mexico Bakshi, Elahi Applied Ag Economist CIMMYT Bangladesh Bandyopadhyay, Ranajit Pathologist ICARDA Mexico Bänziger, Marianne Deputy Director General Research & Partnership CIMMYT Mexico 5 iea.cgiar.org Wheat Evaluation SURNAME, Name Position Organization Country visit Barma, Naresh Ch Deb Chief Scientific Officer BARI, Gazipur Bangladesh Bassi, Fillipo Durum Wheat Breeder ICARDA Morocco Baum, Michael Director - BIGMP CIMMYT Mexico Bennani, Sahar INRA Morocco ICAR Research Complex for Bhatt, B. P Director Eastern Region India Bishaw, Zewdie Head, Seed Section ICARDA Ethiopia Wheat Research Center, Bodruzzaman, Md Principal Scientific Officer (Soils) Dinakpur Bangladesh Braun, Hans Program Director Global Wheat Program CIMMYT Mexico Briones, Ernesto Senior Systems Developer IITA Mexico Chand, Ramesh Professor-cum-Plant Pathologist Banaras Hindu University Obrégon Chanyalew, Solomon Center Director, Debre Zeit Agricultural Research Center EIAR Ethiopia Directorate of Wheat Research Chatrath, R Principal Scientist (DWR), Karnal, ICAR India Chikoye, David R4D Director, IITA-Southern Africa IITA Mexico Chowdhury, S. Consultant Borlaug Institute for South Asia India 6 iea.cgiar.org Wheat Evaluation SURNAME, Name Position Organization Country visit Coffman, Ronnie Principal Investigator, Durable Rust Resistance Prog Cornell University Obrégon Dababat, Amer Plant Pathologist and Nematologist CIMMYT Turkey Global Wheat Program Datta, Swapan K Deputy Director General (Crop Science) Indian Council of Ag Research Obrégon De la O Elizagaray, Marisa Manager, Risk Management & International Policy CIMMYT Mexico Debele, Tolessa SARD-SC Coordinator for ESA ICARDA Ethiopia Ethiopian Seed Enterprise (ESE) Degite, Gemechu Chair Gonde-Iteya Basic Seed Farm Ethiopia Technical Coordinator, East African Agricultural Productivity Project Wheat Regional Center of Excellence Dessalegn, Tadesse Kulumsa Agricultural Research Center EIAR Ethiopia Dixon, John Principal Adviser Research and Cropping Systems and Economics (CSE) program ACIAR Australia manager. Duveiller, Etienne Director of Research for South Asia CIMMYT Obrégon Dwivedi, P K Programme Coordinator Krishi Vigyan Kendra, Buxar India El Bouhssini, Mustapha Entomologist ICARDA Morocco El Hassouni , Khaoula sponsored by GRDC Morocco Erenstein, Olaf Program Director, Socioeconomics Program CIMMYT Mexico Figueroa, Pedro Plant Pthologise in National Wheat Program INIFAP Obrégon Firke, Asnake Director, Crop Research EIAR Ethiopia Fisher, Tony Honorary Research Fellow at CSIRO, Member of WHEAT Stakeholder Committee CSIRO Australia 7 iea.cgiar.org Wheat Evaluation SURNAME, Name Position Organization Country visit Freymond, Jean Secretary SASAKAWA Africa Association Obrégon Fulss, Richard Head, Knowledge Management CIMMYT Mexico Gathala, Mahesh Cropping System Agronomist, Leader SRFSI CIMMYT Bangladesh Gérard, Bruno Program Director, Global Conservation Agriculture CIMMYT Mexico Resilient Farming Systems Intensification (SRFSI) Project, Ghosh, Anup Kumar Senior Agricultural officer RDRS Bangladesh Girma, Bedada Durable Rust Resistant Program, Kulumsa Agricultural Research Center EIAR Ethiopia Mexico (Texcoco Govaerts, Bram Associate Director, Global Conservation Agriculture CIMMYT and Obrégon) Guertin, Michelle Senior Monitoring, Evaluation & Learning Specialist CIMMYT Skype Gupta, Pryianka sponsored Borlaug Scholarship Individual Morocco Directorate of Wheat Research Gupta, R K Principal Scientist (DWR), Karnal, ICAR India Gutterson, Neal President, CEO Mendel Biotechnology Obrégon Haddad, Nicolas Manager, Agricultural Research and Education Center (AREC) American University of Beirut Lebanon Department of Agricultural Hannan, Abdul Deputy Director Extension Bangladesh Haque, Anarul Extension Agronomist, CSISA, Hub Manager, Rangpur CIMMYT Bangladesh ICAR Research Complex for Harris, Abdul Principal Scientists Agronomy Eastern Region India 8 iea.cgiar.org Wheat Evaluation SURNAME, Name Position Organization Country visit Hassen, Haloma Consultant in small ruminant research ICARDA Morocco Hearne, Sara Senior Scientist, Maize Molecular Geneticist/Pre-breeder CIMMYT Mexico Herremans, Anna (former) Director, International Finance CIMMYT Mexico BARI, On Farm Research Hassan, Selima Sr Scientific Officer Division, Rangpur Bangladesh Hobbs, Huntington Leader, Strategic Planning and Research Coordination, MasAgro CIMMYT Mexico GIS and Decision Support Systems Hodson, David SEP CIMMYT Ethiopia Homma, Shitaye Durum Wheat Breeder, Debre Zeit Agricultural Research Center EIAR Ethiopia Wheat Research Center, Hossain, Akbar Senior Agronomist Dinakpur Bangladesh Bangladesh Agricultural Hossain, Mofazzul Deputy Director Seed Marketing Development Corporation Bangladesh Wheat Research Center, Hossain, Monwar Scientific Officer (Plant Breeding) Dinakpur Bangladesh Wheat Research Center, Hossain, Zakir Scientific Officer (Agricultral Engineer) Dinakpur Bangladesh Hundie, Bekele Plant Pathologist, Kulumsa Agricultural Research Center EIAR Ethiopia Obrégon and Huttner, Eric Research Program Manager, Crop Improvement & Management ACIAR Australia ICAR Research Complex for Idris, Mohammad Principal Scientist (Entomology) Eastern Region India 9 iea.cgiar.org Wheat Evaluation SURNAME, Name Position Organization Country visit Iraqi, Driss Coordinator, Biotechnology Unit INRA Morocco Jahan (Belal), Abu Hena Sorwar Principal Scientific Officer BARI, Gazipur Bangladesh Jakobi, Nina WHEAT Program Assistant CRP Mexico Agricultural Economist Jaleta, Moti SEP CIMMYT Ethiopia Central Soil Salinity Research Jat, H S Cropping Systems Agronomist, CSISA Institute India Senior Cropping Systems Agronomicst & CIMMYT-CCAFS South Asia Coordinator, Jat, M L CIMMYT CIMMYT India Jat, Raj Kumar Cropping Systems Agronomist Borlaug Institute for South Asia India Kaan, Kathy Program Manager BMGF Obrégon Kan, Mustafa Agricultural Economist, Department of Economy, Statistic and Extension Bahri Dagdas International Turkey Agricultural Research Institute Department of Agricultural Karim, Rezaul Upzilla Agricultural Officer Extension Bangladesh Kassie, Grima T Agricultural Economist, SEPRP ICARDA Morocco Keil, Alwin Senior Agricultural Economist CIMMYT India Kemal, Seid Ahmad Pathologist ICARDA Ethiopia SURNAME, Name Position Organization Country visit Keser, Mesut IWWIP, Coordinator for ICARDA ICARDA Turkey 10 iea.cgiar.org Wheat Evaluation Kircalioğlu, Gün Research Coordinator Aegean Agricultural Research Turkey Institute Kiwan, Pierre Assistant Researcher , Terbol Research Station ICARDA Lebanon Kommerell, Victor WHEAT Program Manager CIMMYT Mexico Rajendra Agricultural Kumar, Mithilesh Director of Research University, Pusa, Samastipur India Kumar, Virender Cropping Systems Agronomist CIMMYT India Legesse, Wasihun Wheat Breeder , Debre Zeit Agricultural Research Center EIAR Ethiopia Listman, Michael G. Senior Science Writer, Corporate Communications CIMMYT Mexico Lopez Saavedra, Victor Manager of Institutional Relations TTF-MasAgro CIMMYT Mexico Lopez, Diana Project Management Unit CIMMYT Mexico Lumpkin, Thomas DG CIMMYT Mexico Machlab, Hassan Country Manager/Lebanon ICARDA Lebanon Wheat Research Center, Malaker, Paritosh Kumar Director Dinakpur Bangladesh Mallari, Sally MAIZE Program Assistant CIMMYT Mexico Mallik, R. S. Senior Agronomist And Csisa Objective 1 Leader CIMMYT India Memher Agera Seed Multiplication and Marketing Mamo, Beditu Chair Cooperative Ethiopia 11 iea.cgiar.org Wheat Evaluation SURNAME, Name Position Organization Country visit Medina, Richard Director, Internal Audit CIMMYT Mexico Members of the Mahila Kisan Sakhi CSISA - Bihar, Mahila Samaykhya, Muzaffarpur n/a India Mendoza, Fernando P Senior Internal Auditor IITA Mexico Menkir, Abebe Team leader for maize improvement research at IITA, Focal Point for CRP at IITA IITA Mexico Mert, Zafer Phytopathologist / Plant pathologist, Department of Crop Diseases and Resistance The Central Research Institute Turkey Breeding for Field Crops (TARM), Ministry of Food, Agriculture and Livestock (MFAL) Mir, Patricia V Risk Management Analyst CIMMYT Mexico Mishra, Satya Prakash Officer VST Tillers Tractors Ltd India Mishra, V.K. Professor, Dept. of Genetics and Plant Breeding Banaras Hindu University Obrégon AMUARI High Yield Varieties Mitiku, Demessie Seed Agronomist, Supporting Seed Demonstration for Private Seed Growers and Agricultural Products PLC Ethiopia Mittal, Surabhi Economist CIMMYT India Miyamoto, Masaaki Chief Executive Officer SASAKAWA Africa Association Obrégon Momin, Abdul Cropping Systems Agronomist, Manager-CSISA Jessore Hub CIMMYT Bangladesh Former Durum Wheat Breeder and now Consultant with focus on pre-breeding and Nachit, Miloudi on farm trials ICARDA Morocco Nasserlehaq, Nsarellah Durum Wheat Breeder INRA Morocco Nawar, Fawzy Senior documentation specialist, Genetic Resources Section ICARDA Morocco 12 iea.cgiar.org Wheat Evaluation SURNAME, Name Position Organization Country visit Nazari, Kumarse Plant Pathologist, Cereal Rust Regional Center ICARDA Turkey Ortiz Monasterio, Ivan Agronomist, Wheat Harvest Coordinator CIMMYT Mexico Ozdemir, Fatih Director, Bahri Dagdas International Agricultural Research Institute, IWWIP BDIARI Turkey Coordinator for Turkey Ozkan, Isa Head, Field Crops Research Department MFAL Turkey Pandit, D B Cropping Systems Agronomist, Manager-CSISA Mymensingh CIMMYT Bangladesh Payne, Thomas S. CIMMYT Board Secretary CIMMYT Mexico AEGEAN Agricultural Research Peksuslu, Ali Director, Institute Turkey Pfeiffer, Wolfgang H Deputy Director, Operations HarvestPlus Obrégon Pixley, Kevin Program Director, Genetic Resources Program CIMMYT Mexico Powell, Wayne Chief Science Officer CGIAR Obrégon Rahman, Ataur Specialist Conservation Agriculture BARI, Gazipur Bangladesh Rajasekharan, Nellooli P. Director, International Human Resources CIMMYT Mexico Resilient Farming Systems Intensification (SRFSI) Project, Rashid, Mamunur A&E Coordinator RDRS Bangladesh Renard, Geneviève MAIZE and WHEAT Communication Specialis GCP Mexico Reynolds, Mathew Head Wheat Physiology CIMMYT Obrégon Wheat Research Center, Reza, Mustafa Alir Senior Scientific Officer, (Wheat Pathology) Dinakpur Bangladesh 13 iea.cgiar.org Wheat Evaluation SURNAME, Name Position Organization Country visit Ribaut, Jean-Marcel Director, Generation Challenge Program CIMMYT Mexico Riis-Jacobsen, Jens Director of Int. Systems and Information Technology CIMMYT Mexico Rodriguez, Horacio MasAgro Extension Coordinator CIMMYT Mexico Guchi Habe Badosa Bread Wheat and Seed Multiplication Rorisa, Bejiqa Cooperative Farmer and Value Chain Cooperative Ethiopia Rossi, Frederik Agricultural Economist, Manager-CSISA Faridpur Hub CIMMYT Bangladesh Rugema, Hilary Coordinator, Crop Productivity Improvement SASAKAWA Africa Association Obrégon Saidi, Seddik Head, Plant Breeding Department INRA Morocco Saint Pierre, Carolina Scientist, Wheat Phenotyping Coordinator SeeD CIMMYT Obrégon Sanchez –Garcia, Miguel Spring Wheat Breeder ICARDA Morocco SanVicente, Felix Breeding lead for the tropics in Mexico CIMMYT Mexico Wheat Research Center, Sarker, Abu Zaman Princial Scientific Officer (Agronomy) Dinakpur Bangladesh Agricultural Information Service Sayem, Abu Regional Director (Ministry of Agriculture) Bangladesh Schulthess, Urs Crop modeler, Global Conservation Agricultural program CIMMYT Mexico Sefera, Tadessa Center Director, Kulumsa Agricultural Research Center EIAR Ethiopia Segovia, Vanessa Post Doctorial Fellow ICARDA Turkey Directorate of Wheat Research Senthil, R. Principal Scientist (DWR), Karnal, ICAR India 14 iea.cgiar.org Wheat Evaluation SURNAME, Name Position Organization Country visit Shahjahan, Md Croping Systems Agronomist, , Manager-CSISA Khulna Hub CIMMYT Bangladesh Directorate of Wheat Research Sharma, Indu Director and Member, Wheat Management Committee (CRP-Wheat) (DWR), Karnal, ICAR India Central Soil Salinity Research Sharma, P C Head, Crop Improvement Institute India Directorate of Wheat Research Sharma, R K Principal Scientist (DWR), Karnal, ICAR India Short, Thomas W. DDG Support Services CIMMYT Mexico Country Manager ICARDA Ethiopia and Coordinator Sub-Saharan Africa Regional Silim, Said Nahdy Program ICARDA Ethiopia Sim, Graham Director, International Finance CIMMYT Mexico Singh, Ravi P Head, Bread Wheat Improvement & Rust Res CIMMYT Obrégon Directorate of Wheat Research Singh, Satyavir Principal Scientist (DWR), Karnal, ICAR India Sohail, Quahir Spring Wheat Breeder ICARDA Morocco Indian Agricultural Research Solanki, I S Head Institute India Solh, Mahmoud DG ICARDA Lebanon Souihka, Allah Manager, Marchouch Station ICARDA Morocco Sugandhi, ? Gender Cordinator, CSISA CIMMYT India Tadessa, Zerihun National Coordinator, Wheat Program EIAR Ethiopia Tadesse, Wuletaw Bread Wheat Breeder ICARDA Morocco SURNAME, Name Position Organization Country visit 15 iea.cgiar.org Wheat Evaluation Tafesse, Mekasha Driver and Logistical Assistant ICARDA Ethiopia Tanwir, Niayer Secretary Creation Welfacer Society India Tawkaz, Sawsan Head, ICARDA’s Double Haploid Laboratory ICARDA Lebanon Memher Agera Seed Multiplication and Marketing Teklu, Kifle Participating Farmer Cooperative Ethiopia AMUARI High Yield Varieties Tesfaye, Bezabih Seed Specialist and Agricultural Products PLC Ethiopia Tessema, Alemu Chair Denkaka Megertu Cooperative Ethiopia Thornton, Matthew Hub Coordinator CIMMYT Mexico Directorate of Wheat Research Tiwari, Ratan Principal Scientist (DWR), Karnal, ICAR India Tiwari, T. P. Cropping System Agronomist & Country Representative CIMMYT Bangladesh Tovar Mondragon, Jose Ramiro Manager, Financial Planning CIMMYT Mexico Trachsel, Sam Scientist, Global Maize Program CIMMYT Mexico Tsivelikas, Athanasios Genebank Manager ICARDA Morocco Uddin, Mohdi Cropping System Agronomist, Manager-CSISA Faridpur Hub CIMMYT Bangladesh Udupa, Sripada Senior Scientist, Biotechnology, ICARDA Morocco van Ginkel, Maarten DDGR ICARDA Lebanon Wakjira, Adunga Deputy Director General EIAR Ethiopia 16 iea.cgiar.org Wheat Evaluation SURNAME, Name Position Organization Country visit Ward, Rick Principal Scientist, Global Wheat Program CIMMYT Obrégon Watson, David MAIZE Program Manager CIMMYT Mexico Willcox, Martha Senior Scientist working with the Seeds Discovery (SeeD) project CIMMYT Mexico Worku, Zewuditu Woman Farmer n/a Ethiopia Guchi Habe Badosa Bread Wheat and Seed Multiplication Yai, Kebede Cooperative Farmer and Value Chain Cooperative Ethiopia Yasmin, Samina Cropping Systems Agronomist, Manager-CSISA Barisal Hub CIMMYT Bangladesh Yazbek, Mariana Post Doc Fellow, Genetic Resources ICARDA Lebanon Zaim, Meryem sponsored by GRDC n/a Morocco Zehr, Usha Barwale Chief Technology Officer, Maharashtra Hybrid Seeds Co. Ltd. MAHYCO Obrégon 17 iea.cgiar.org Wheat Evaluation ANNEX D. - List of sample projects A total of 40 projects were selected from the project database. These included the 10 largest projects (according to the budget allocated to WHEAT) as well as 30 projects which were randomly selected by Flagship. Projects had to be replaced from the initial selection and some projects utlimately could not be reviewed because information was found insufficient or was only available in Spanish, or treated as confidential. The evaluation team looked at a total of 34 projects listed below: No Title FP Lead A4031.09.19 Accelerating adoption of yellow rust resistance winter varieties in Central Asia 4 Other The Potential for Wheat Production in Sub-Saharan Africa: Analysis of Biophysical Suitability and A4031.09.24 Economic Profitability 1 CIMMYT Enhanced quality and healthy seed testing system for International Winter Wheat Improvement A4031.0918 Program (IWWIP) 3 Other Enhancing cotton germplasm, improving resistance to cotton leaf curl virus and supporting cotton B 1198 best management practices for small farmers. 4 ICARDA B1146 Joint ICARDA-ARC Wheat Improvement Program 2 ICARDA B1202 Pakistan Wheat production enhancement 3+4+5 ICARDA B1202 Pakistan Wheat Production Enhancement Initiative 2+3+5 ICARDA B1287 Support to Agricultural Research for Development of Strategic Crops in Africa (SARD-SC)‘ 5 ICARDA B1313 Wheat Productivity Improvement towards Food Self-sufficiency 5 ICARDA B1355 and 1356 Mobilizing Novel Genes from Wheat 2 CIMMYT/ICARDA C0035 Cereal Systems Initiative for South Asia (CSISA)-Phase II 4 CIMMYT 18 iea.cgiar.org Wheat Evaluation No Title FP Lead C0037 Cereal Systems Initiative for South Asia (CSISA)-India 4 CIMMYT C0040 MASAGRO-Desarrollo sustentable con el agricultor 4 CIMMYT C0064 Africa Rising: Research in Sustainable Intensification for the next generation 4 CIMMYT Wheat Addressing the challenge of climate change for sustainable food security in Turkey, Iran, Pakistan, Uzbekistan, Morocco and Spain through the exploitation of wheat Genetic Resources, n/a adoption of resilient improved varieties on farmers fields and their socio-economic impact. 2 CIMMYT R0148 MASAGRO-Descubriendo la diversidad genética de las semillas 4 CIMMYT T0083 Expansion of the Cereal Systems Initiative for South Asia (CSISA) in Bangladesh 4 CIMMYT W 0286 Development of Heat Tolerant Wheat for South Asia 3 CIMMYT W0101 Desarrollo de nuevas variedades de trigo duro, trigo harinero y triticale a partir de germoplasma procedente del CIMMYT-Phase III 3 CIMMYT W0204 Identification and utilization of novel sources of resistance against soil borne pathogens in wheat (CIM00014) 3 CIMMYT W0205 Enhanced Delivery of CIMMYT Germplasm to Australia 3 CIMMYT W0212 Development of Cereal Germplasm and the Screening for Disease Resistance and End-Use Quality: CIMMYT - Triticale Improvement Component 3 CIMMYT W0235 Identifying new genetic sources and evaluating United States wheat germplasm for resistance to stem rust in Eastern Africa (25856) 3 CIMMYT W0239 Pakistan Wheat production enhancement 3+4+5 CIMMYT 19 iea.cgiar.org Wheat Evaluation No Title FP Lead W0250 Durable Rust Resistance in Wheat (Phase II 3 CIMMYT Genetics and Physiology of Wheat Development to Flowering: Tool to breed for improved W0252 adaptation and yield potential: 2 CIMMYT W0265 Sustainable Wheat & Maize Production in Afghanistan ( 5 CIMMYT W0266 Agricultural Innovation Programme 4 CIMMYT W0269 KAI-Agricultural Technologies in Kazakhstan 3 CIMMYT W0273 Joint Development of Korean Winter Bread Wheat Lines with heat tolerance and high protein n/a CIMMYT W0276 Russia-Omsk Institute 3 CIMMYT W0281 Field evaluation of Genetically Modified Wheat 2 CIMMYT W0289 Rapid Development of Climate Resilient Wheat Varieties for South Asia using Genomic Selection 2+3 CIMMYT W1/2 activity Socioeconomics & policies for wheat futures, 1 CIMMYT 20 iea.cgiar.org Wheat Evaluation ANNEX E. - Assessment of publications Selection of individual publications for assessment The team intended to assess 37 publications in total. The publications were categorized according to the Center that issued them (CIMMYT and ICARDA) and to discipline (i.e. whether they related to agronomy, breeding or socio-economics). Target numbers were then developed based on how many publications each Center had published in the 3 disciplines (see Table 1 below). Publications for assessment were then selected randomly from the database according to these target numbers. Table 1 - Criteria for selection of sample publications SAMPLE ACTUAL CIMMY CIMMY CIMMYT ICARDA TOTAL T ICARDA T ICARDA Agronomy 85% 15% 10 9 1 6 1 Socio- economics 71% 29% 6 4 2 5 2 Breeding 75% 25% 21 17 4 19 3 Number of Publications 30 7 30 6 After all publications had been identified, an assessor was assigned for each one, bearing in mind the areas of competence, expertise and experience of the assessors. Some of the publications were re-classified (especially some of the agronomy publications which related more to breeding). One publications by 21 iea.cgiar.org Wheat Evaluation ICARDA was not assessed, as it could not be accessed (Genome-wide association mapping for five major pest resistances in wheat, Molecular Breed). The total of 36 publications assessed by the evaluation team are shown in the table below. Team CENTER member YEAR Title JOURNAL Type SB 2012 Achieving yield gains in wheat Plant, Cell & Environment (5.14 in CIMMYT 2012, year of publication Breeding DT 2012 Agricultural Extension, Collective Action and Innovation Systems: Lessons on The Journal of Agricultural CIMMYT Network Brokering from Peru and Mexico Education and Extension Socio econ RS 2012 Analysis of genotypic variation for NDVI (normalized difference vegetation index) and its relationship with grain yield in winter wheat under terminal heat stress Plant Breeding 131(2012) ICARDA Breeding WB 2012 Analysis of leaf and stripe rust severities reveals pathotype changes and multiple minor QTLs associated with resistance in an Avocet × Pastor wheat population TAG 2012 CIMMYT Breeding RS 2012 Breeding progress for yield in winter wheat genotypes targeted to irrigated environments of the CWANA region Euphytica (2013) ICARDA Breeding RS 2012 Conservation Agriculture in Maize- and Wheat-Based Systems in the (Sub)tropics: Journal of Sustainable Agriculture CIMMYT Lessons from Adaptation Initiatives in South Asia, Mexico, and Southern Africa (2012) Socio econ RS 2012 Conservation agriculture, increased organic carbon in the top-soil macro- aggregates and reduced soil CO 2 emissions Plant Soil (2012) CIMMYT Agronomy Food sovereignty or the human right to adequate food: which concept serves CIMMYT DT 2012 better as international development policy for global hunger and poverty Agric Hum Values (2012) Socio econ reduction? 22 iea.cgiar.org Wheat Evaluation Team CENTER member YEAR Title JOURNAL Type WB 2012 From genotype × environment interaction to gene × environment interaction by Crossa, J Current Genomics Current Genomics 2012 CIMMYT Breeding RS 2012 Genetic structures of the CIMMYT international yield trial targeted to irrigated environments Mol Breeding CIMMYT Breeding WB 2012 Genetic yield gains and changes in associated traits of CIMMYT spring bread wheat in a "Historic" set representing 30 years of Breeding Crop Science, 2012 CIMMYT Breeding WB 2012 Genetic yield gains of the CIMMYT International semi-arid wheat yield Crop Science (2012) CIMMYT Breeding SB 2012 Global Crop Improvement Networks to Bridge Technology Gaps J. of Exp. Botany (5.242 for 2012 CIMMYT – year of publication) Agronomy RS 2012 Global incidence of wheat rusts and powdery mildew during 1969-2010 and durability of resistance of winter wheat variety Bezostaya 1 Eur J Plant Pathol Breeding WB 2012 Lr68: A new gene conferring slow rusting resistance to leaf rust in wheat Theoretical and Applied Genetics CIMMYT Breeding WB 2012 Multi-trait and multi-environment QTL analyses for resistance to wheat diseases Open Access PLOS-One (2012) CIMMYT Breeding DT 2012 Performance of biofortified spring wheat genotypes in target environments for grain zinc and iron concentrations Field Crops Research 137 (2012) CIMMYT Breeding WB 2012 Phenotyping transgenic wheat for drought resistance (2012) Journal of Experimental Botany CIMMYT Breeding SB 2012 Physiological Traits for Improving Heat Tolerance in Wheat Plant Physiology (6.56 in year of CIMMYT publication, 2012) Breeding 23 iea.cgiar.org Wheat Evaluation Team CENTER member YEAR Title JOURNAL Type SB 2012 Quantifying how winter wheat crops accumulate and use nitrogen reserves during Field Crops Research (2.47 in year CIMMYT growth of publication, 2012) Agronomy Stay-green in spring wheat can be determined by spectral reflectance SB 2012 measurements (normalized difference vegetation index) independently from J. of Experimental Botany (5.242 CIMMYT phenology for 2012 – year of publication) Breeding DT 2012 The Production Functions of Wheat Production in Turkey Bulgarian Journal of Agricultural ICARDA Science, 18 (No 2) 2012 Socio econ SB 2012 The yield correlations of selectable physiological traits in populations of advanced spring wheat lines grown in warm and drought environments Field Crops Research; CIMMYT Breeding WB 2012 Virulence of Beauveria bassiana against Sunn pest, Eurygaster integriceps Puton Journal of Entomology and ICARDA (Hemiptera: Scutelleridae) at different time periods of application Nematology (2012) Agronomy RS 2013 Assessing the impacts of GAFTA on selected members’ bilateral agricultural trade: Am-Euras. J. Agric. & Environ. ICARDA an application of the Gravity Model. Science (2013) Socio econ DT 2013 Crops that feed the world 10. Past successes and future challenges to the role played by wheat in global food security Food Sec. (2013) CIMMYT Socio econ RS 2013 Earliness in wheat: A key to adaptation under terminal and continual high temperature stress in South Asia Field Crops Research (2013) CIMMYT Agronomy WB 2013 Genetic analysis of adult plant resistance to yellow rust and leaf rust in common spring wheat Plant Disease (2013) CIMMYT Breeding 24 iea.cgiar.org Wheat Evaluation Team CENTER member YEAR Title JOURNAL Type SB 2013 Historical changes in grain yield and quality of spring wheat varieties cultivated in Siberia from 1900 to 2010 Canadian Journal of Plant Science CIMMYT Breeding SB 2013 Identification and characterization of international Fusarium head blight J. Plant Pathology (1.71 in 2013, CIMMYT screening nurseries of wheat at CIMMYT, Mexico year of publication) Breeding RS 2013 Meta: A suite of sas programs to analyze multienvironment Breed trials Agronomy Journal Volume 105, CIMMYT Issue 1, 2013 Agronomy Mineral nitrogen dynamics in irrigated rice-wheat system under different CIMMYT SB 2013 irrigation and establishment methods and residue levels in arid drylands of European Journal of Agronomy Agronomy Central Asia (2.92) WB 2013 QTL mapping of slow-rusting, adult plant resistance to race Ug99 of stem rust fungus in PBW343/Muu RIL population TAG 2013 CIMMYT Breeding WB 2014 Association Mapping of Resistance to Yellow Rust in Winter Wheat Cultivars and Elite Genotypes Crop Science (2014) ICARDA Breeding From adoption claims to understanding farmers and contexts: A literature review DT 2014 of Conservation Agriculture (CA) adoption among smallholder farmers in southern Agriculture, Ecosystems and CIMMYT Africa Environment 187 (2014) 116–132 Socio econ DT 2014 Meeting demands for increased cereal production in China Journal of Cereal Science 59 CIMMYT (2014) 235e244 Breeding 25 iea.cgiar.org Wheat Evaluation ANNEX F. - WHEAT researcher survey Sent out: 13 Nov 2014 Reminder sent by CIMMYT: 2 Dec 2014 Closed: 11 Dec 2014 Total CIMMYT ICARDA Other (competitive partner grants) Survey sent to 109 58 32 19 Responses 66 43 15 9 Response rate 61% 74% 47% 47% Complete Reponses 43 28 9 6 Completed response rate 39% 48% 28% 32% 26 iea.cgiar.org Wheat Evaluation Survey population Other (please Program/FP specify) 19.7% Position (Q3) leader/theme leader , 12.1% Research Fellow FEMALE Gender (Q2) , 0.0% 18.2% Associate Principle scientist/Post Investigator/Se doctoral fellow nior scientist , 22.7% 45.5% MALE 81.8% 27 iea.cgiar.org Wheat Evaluation Working with org since (Q5) since 2012, 20, Other (please Research area (Q4) 30% specify) 9.1% pre 2008, 25, Informatics 38% 1.5% Social sciences , 7.6% Natural resource sciences 4.5% Crop sciences 77.3% 2008-2011, 21, 32% Afghanistan , 1.5% Uzbekistan 3.0% Bangladesh , 4.5% China 3.0% Turkey 10.6% Ethiopia , 3.0% Tunisia 1.5% Germany, 1.5% Syria, 3.0% South Africa , 3.0% India , 9.1% Pakistan , 3.0% Jordan 1.5% Nepal 4.5% Lebanon , 3.0% Duty station (Q6) Morocco 10.6% Mexico 33.3% 28 iea.cgiar.org Wheat Evaluation CRP engagement 30 For which CRP(s) do you currently 25 25 work? (Q7) 20 17 MAIZE 15 14 WHEAT 11 10 10 Other CRPs 8 5 4 2 3 0 less than 30% 40-70% More than 80% 29 iea.cgiar.org Wheat Evaluation Working for WHEAT..... (Q7) More than 80%, 25, less than 30%, 17, 45% 30% 40-70%, 14, 25% 40 34 35 SI mentioned in top 3 (Q8) 30 26 25 21 20 18 15 17 15 10 9 9 10 8 4 5 0 30 iea.cgiar.org Wheat Evaluation FP mentioned in top 3 (Q8) 45 39 40 35 29 30 28 25 20 18 15 13 11 10 5 0 FP1 Maximizing FP2 Novel FP3 Global FP4 Sustainable FP 5 Engaging DON'T KNOW Benefits for Diversity and Breeding Intensification and Poor Women, Tools to Adapt Partnerships to of Wheat- strengthening Children, and to Climatic and Meet Future based system Men Resource Food Demands Constraints 31 iea.cgiar.org Wheat Evaluation What entity/organization do you primarily identify with? (Q10) Other (please specify) 1.8% WHEAT, 24.6% My home organization 73.7% 35 What entity/organization do you primarily identify 30 29 with? (Q10) 25 20 before 2011 15 13 after 2011 10 8 6 5 0 1 0 My home organization WHEAT Other (please specify) 32 iea.cgiar.org Wheat Evaluation How many different projects in WHEAT are you currently working on? (Q11) 18 17 16 14 13 12 10 8 8 6 6 6 5 4 2 2 0 1 2 3 4 5 6 more than 7 33 iea.cgiar.org Wheat Evaluation 25 How well do you know WHEAT? (Q13) 20 20 17 15 11 10 8 5 1 0 34 iea.cgiar.org I have participated in designing WHEAT including its program structure, objectives, main theories of change and Intermediate Development outcomes (IDOs). I have not participated in designing the WHEAT but have read key program documents and know the program structure, objectives, main theories of change, IDOs and key organizations involved. I have some knowledge of the program structure, objectives, main theories of change, IDOs and key organizations involved. I have some knowledge of the WHEAT, but know only the SIs/FPs that my work contributes to. I know very little of nothing about WHEAT Wheat Evaluation RESEARCH Q 14. What is your perception of the factors influencing the choice of research topics in the WHEAT Flagship you mostly contribute to? 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Expressed needs of clients or beneficiaries 3 2 2 5 12 24 Funding availability 0 4 1 12 11 20 1 - No influence at all 2 Donor priorities 1 3 5 8 13 15 3 4 WHEAT strategy 1 1 7 9 14 13 5 6 - Primary influence Scientific interest/knowledge gap 01 8 13 10 10 Center strategy 1 4 3 11 20 7 35 iea.cgiar.org Wheat Evaluation Q 15. According to you, which are currently the three most important research activities within WHEAT and where you think the emphasis should be? Current most important research activities (top rated) 0 5 10 15 20 25 30 Strategic research to produce International Public Goods 26 Adaptive research 12 Local research to produce feed-back and synthesis for global… 7 Scaling-out of research results 3 “Blue sky”, high risk research 1 Technical assistance 0 Other 0 Where emphasis should be (top rated) 0 5 10 15 20 25 30 35 Strategic research to produce International Public Goods 32 “Blue sky”, high risk research 5 Local research to produce feed-back and synthesis for… 4 Adaptive research 4 Scaling-out of research results 3 Technical assistance 1 Other 0 36 iea.cgiar.org Wheat Evaluation Q 16. The CRP receives funding from different sources where the Windows 1 and 2 are of least restricted type. What is your view of how W1/2 funds are used in WHEAT? 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Leverage bilateral funding 01 9 10 8 9 Fill gaps in research funding 0 5 8 8 9 12 Increase integration between different areas of research 2 4 6 9 11 9 1- Not at all Enhance desirable partnerships through competitive grants 4 4 7 7 12 5 2 Enhance research quality through competitive grants 1 7 9 10 6 7 3 4 Increase gender relevant research 3 4 6 10 10 2 5 Provide opportunities for long-term, high risk research 4 7 8 11 1 8 6- Main Purpose Provide accountability through ex post studies 1 5 10 7 2 4 Provide relevance through ex ante studies 1 4 11 8 2 3 Cover overhead costs 4 8 2 6 3 4 37 iea.cgiar.org Wheat Evaluation Q 17. In your view, how well are the following aspects for enhancing the effectiveness of WHEAT managed? Please score using a scale of 1-6 where 1=not well at all and 6=very well. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Development of impact pathway 01 6 9 16 10 Engagement with partners 02 5 10 19 10 1- Not well at all 2 Annual progress monitoring 01 8 11 15 7 3 4 Capacity building 1 3 6 10 16 8 5 6- Very well managed Analysis of target groups 0 4 8 15 13 2 Feed-back between W1/W2 and bilateral projects 1 4 7 15 5 1 38 iea.cgiar.org Wheat Evaluation Q 18. In your view, what are important pathways through which research in WHEAT aims to have impact? Please score using a scale of 1-6 where 1=not an impact pathway at all and 6=primary impact pathway. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Influencing food security through global germplasm development 010 6 10 31 Influencing rural livelihoods through enhanced productivity 0 3 8 14 21 1-Not an impact pathway at all Enhancing varietal use through better seed dissemination 0 4 6 20 17 2 3 Enhancing environmental sustainability through improved farming practices 01 5 9 17 14 4 5 Enabling equitable benefits from research 1 2 4 12 13 12 6-Primary impact pathway Influencing funding of cereal research 1 3 5 8 16 10 Influencing partner research agendas in developing countries 1 2 6 12 10 12 39 iea.cgiar.org Wheat Evaluation Q 19. In your view, how effectively are the measures listed below managed in your Center/CRP for assuring and enhancing high quality of research? Please score using a scale of 6 where 1=no attention at all and 6=very effectively. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Infrastructure needed for high quality science 2 2 9 11 15 5 Strategic use of grants 1 4 9 10 12 7 1 - No attention Availability and quality of technical facilities/ equipment 0 4 10 12 13 4 Learning and knowledge management 0 4 9 19 6 6 2 Availability and quality of research support 1 5 10 9 15 4 3 Acceptance and encouragement of innovative thinking 2 4 9 13 10 5 Researcher performance assessment 1 7 9 8 12 5 4 Research data management 3 5 10 13 8 6 5 Quality assurance processes such as internal peer feed-back 2 5 10 15 8 4 Allocation of competences and appropriate skill mix to research 6 - Very effectively teams 2 9 8 14 6 5 Personal incentives for high research quality 5 6 10 9 10 3 Encouragement for learning from “ failure” 2 9 15 6 8 2 40 iea.cgiar.org Wheat Evaluation PARTNERSHIPS Most involved partners (top rated) 0 5 10 15 20 25 National research institutions 20 Other CGIAR centers 5 National governments 5 Universities in developing countries 4 Universities in developed countries 3 Other CRPs 2 Other 2 Farmer organizations 1 Civil society organizations 1 National agricultural extension systems 0 Donor implementation agencies 0 International NGOs 0 Local NGOs 0 Ultimate beneficiaries 0 41 iea.cgiar.org Wheat Evaluation Top 3 most involved partne0 rs (Q21) 10 20 30 40 Other CRPs Other CGIAR centers National governments Universities in developing countries Universities in developed countries National research institutions National agricultural extension systems Donor implementation agencies International NGOs Local NGOs Farmer organizations Civil society organizations Ultimate beneficiaries Other 42 iea.cgiar.org Wheat Evaluation Q 22. In your view, to what extent do the current partnerships increase the likely effectiveness of your WHEAT-related research in areas listed below? Please score in scale of 6 where 1 -= not at all; and 6 = very much 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Help in outreach of the research results 01 10 15 17 Increase the relevance of your research 1 2 7 17 16 1 - Not at all 2 Help scale out your research results 0 4 6 16 16 3 4 Build on synergies which result in more streamlined research implementation 1 4 9 12 16 5 6 - Very much Increase the state-of-art in research 1 6 8 13 13 Help in attracting funding 3 5 12 10 13 43 iea.cgiar.org Wheat Evaluation Q 24: In your view, to what extent are the partners in WHEAT involved in Program activities as listed below? (Please score in scale of 6 where 1 -= not at all; and 6 = very much involved) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Research prioritization 1 6 5 15 12 Planning of projects 3 7 5 12 11 1- Not at all 2 3 Publishing research result 3 8 8 13 7 4 5 6- Very much involved Receiving mentoring in research 3 8 9 9 8 Feed-back from clients/users back to research 6 6 10 12 5 44 iea.cgiar.org Wheat Evaluation GENDER Q 25. Please indicate your agreement with the following statements that relate to mainstreaming of gender issues in your work and WHEAT. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Gender disaggregated data on results are collected when ever appropriate. 0 2 3 12 12 3 The WHEAT gender strategy has been well communicated to teams and researchers. 1 2 6 16 10 4 Strongly disagree The WHEAT gender strategy influences the way my team plans Disagree and conducts its work. 1 7 4 10 16 2 Somewhat disagree Somewhat agree There is too much emphasis in gender even in research where gender does not matter. 0 9 5 10 6 7 Agree Strongly Agree There is sufficient funding to implement the gender-related activities 1 4 5 13 7 3 The WHEAT gender strategy influences the way most teams in the CRP plan and conduct their work. 1 4 4 12 7 1 45 iea.cgiar.org Wheat Evaluation Capacity development Q 26. Please indicate your agreement your agreement with the following statements that related to capacity development (CD) in your work and WHEAT. Please score in a scale of 6 where 1=very dissatisfied and 6=very satisfied. 0% 20% 40% 60% 80% 100% My Center is addressing institutional and organizational CD needs in developing 0 8 4 20 7 countries very well, WHEAT is addressing well institutional and organizational CD needs in developing 02 5 5 14 9 Strongly countries very well. disagree Training and mentoring are equitably Disagree targeting men and women. 02 5 9 14 10 CD activities are designed so as to enhance Somewhat uptake of research results and likely 02 5 7 16 8 disagree effectiveness. Somewhat agree CD activities are well integrated to research 02 4 11 14 9 Agree My Center is addressing CD capacity needs Strongly Agree of individuals very well. 1 5 5 12 10 8 WHEAT is addressing CD needs of individuals very well. 0 6 3 10 10 6 There is sufficient funding available for CD activities 2 8 14 8 1 2 46 iea.cgiar.org Wheat Evaluation Satisfaction/value added Q 27. Please indicate how satisfied you are with the following working conditions for your work. Please score in a scale of 6 where 1=very dissatisfied and 6=very 0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100% Share of time required for training and mentoring 0 3 9 7 14 4 Share of time required for coordinating among partners 02 8 14 13 3 Share of time required for travel and meetings (whether for Center or CRP) 0 6 9 12 10 4 1- Very dissatisfied Incentives for cooperating with non-CGIAR 2 partners 0 7 12 9 10 3 3 Share of time allocated to research (compared to time for administration and reporting; whether… 2 6 10 11 7 5 4 Reliability and predictability of bilateral project 5 funding 1 7 11 10 12 1 6 - Very satisfied Incentives for working across themes and disciplines in the CRP 2 3 14 8 9 2 Incentives for cooperating with other CGIAR Centers or CRPs 0 9 12 10 7 1 Reliability and predictability of CGIAR funding through Window1/Window2 6 7 9 8 7 0 47 iea.cgiar.org Wheat Evaluation Q 28. Please indicate your agreement with the following statements related to the value WHEAT has had or is likely to have influencing the success of your research compared to past Center-based implementation of the research. Please score in a scale of 6 where 1=very dissatisfied and 6=very satisfied. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% WHEAT is enabling research to be better aligned to beneficiary needs. 01 6 9 18 4 The results framework in WHEAT will improve the value-added from research. 01 6 8 17 3 Research is becoming strategically better focused on development outcomes. 01 10 8 15 7 WHEAT provides a better framework for guiding and focusing research planning. 0 9 11 17 4 1 - Very dissatisfied Due to WHEAT, capacity development is now addressed more strategically to improve research effectiveness 01 8 12 12 3 2 WHEAT has improved the way in which gender issues are integrated 3 to research 0 2 7 8 13 2 My research is better rationalized in the broader context of program 4 objectives. 0 3 11 7 14 5 5 WHEAT encourages accountability for development outcomes (not 6 - Very satisfied just research outputs) 01 8 13 13 1 WHEAT is creating or enhancing synergies between participating Centers. 0 2 11 12 9 3 Research is becoming better integrated across disciplines and teams towards results. 0 5 10 9 14 2 WHEAT has good potential to help streamline administrative procedures 1 2 9 11 9 0 WHEAT has good potential to help streamline monitoring and reporting 2 1 11 11 7 1 48 iea.cgiar.org Wheat Evaluation ANNEX G. - Sample project assessment - TEMPLATE PROJECT NUMBER: (if applicable) PROJECT TITLE: FLAGSHIP PROJECT NUMBER: TYPE OF PROJECT: RESEARCH <-> more downstream DEVELOPMENT ORIENTED – please explain briefly LEAD: CIMMYT/ICARDA/other Filled out by: Date: Documentation used for assessment: RELEVANCE AND COHERENCE: Alignment with WHEAT objectives Project alignment means that project objectives and activities to achieve those objectives are clearly defined and match with those of the SI/FP • Is the project presented as part of WHEAT CRP? • To what extent is the project/activity aligned to the FP (or specific SI) objectives and WHEAT IDOs? - Very well - Moderately - Poorly Comments: 1. Relevance of project • Are the main target beneficiaries of the project clearly defined? 49 iea.cgiar.org Wheat Evaluation • Does the project reflect needs and priorities of target group? • Does the project address globally/regionally important issues where CGIAR has comparative advantage? 2. SCIENCE QUALITY • Is the research addressing researchable issues? 3. • Are research hypothesis clearly formulated and testable? 4. • Does the project use state of the art methodology? 5. 6. To what extent do the choices of research topics and research designs reflect a high quality of scientific thinking, state-of-the-art knowledge of the scientific literature and complements research done elsewhere? 7. • To what extent does the research reflect an iterative process between downstream and upstream research? Has feedback from downstream work been incorporated in research design? LIKELY EFFECTIVENESS: • Is there an impact pathway described and is it realistic? 50 iea.cgiar.org Wheat Evaluation 8. • How has progress been in terms of research results and outputs? 9. • Are limitations and delays explained? 10. • How does the project design address the scaling-up and out of research outputs? 11. • Does the project include an implementation modality (partnerships, management) which reflects an appropriate role for CIMMYT/IITA? 12. • What has been the main success so far? (MENTION ONE MAJOR ACHIEVEMENT) Assess its significance. 13. Are constraints to research outputs as well as uptake/impact and challenges mentioned and addressed? 14. • Is the reference to when impact (adoption) is expected and in what scale? How likely do you see this realizing? PARTNERSHIPS • What types of partners are included in the project? • Are the partners’ roles clearly defined? 15. • How is the work divided between CIMMYT/ICARDA and the partners? 51 iea.cgiar.org Wheat Evaluation • To what extent are these partners relevant and likely effective in relation to the project's objectives and design -- at the project's conceptual stage? - High - Substantial - Modest - Low - Not mentioned GENDER 16. • Does the project/research strategy address gender specific issues? 17. • Does the project report on gender disaggregated results? • From the point of view of achieving the objectives of the project, has there been an under-emphasis, appropriate emphasis, or over- emphasis on gender analysis and gender-specific research in terms of achieving the Flagship IDO? - Under-emphasis - Appropriate emphasis - Over-emphasis - Not mentioned CAPACITY DEVELOPMENT 18. 52 iea.cgiar.org Wheat Evaluation • Are there any capacity building measures mentioned and who do they target? • What capacity building activities have been implemented? • From the point of view of achieving the objectives of the project, has there been an under-emphasis, appropriate emphasis, or over- emphasis on capacity development in terms of achieving the Flagship IDO? - Under-emphasis - Appropriate emphasis - Over-emphasis - Not mentioned CONCLUSION/OBSERVATIONS please briefly explain any qualifiers to the above questions (issues that could not be reflected) as well as briefly state the main strengths as well as issues relating to this project 53 iea.cgiar.org Wheat Evaluation ANNEX H. - Peer review scoring of a random sample of publications – TEMPLATE Number of publication (according to our list and file name): Name of publication: Journal name: Center: CIMMYT/ICARDA Area: Agronomy/Breeding/Socio econ Team member to review publication Team members shall assess each publication according to the following standardized criteria and approach: Criterion Assessment approach methodological rigor and coherence of data Scale1 analysis comprehensiveness of research narrative Scale innovativeness; novelty Observation: would novelty be expected, if yes what kind of novelty was observed quality (and appropriateness) of publication Observation of low-quality or inappropriate venue venue relative to subject and quality of paper collaboration (especially co-authorship) evident Observation of extent of authorship and with whom “fit” with CRP objectives Observation of outliers overall quality of publication (including Brief narrative additional criteria at evaluator discretion) 19. When there are other outputs than published work – such as germplasm - systematic assessment of quality may be difficult and assessment of quality can be based on project reports, volume of output, extent of use/uptake and user perceptions. 1 Scale of 4 (1=poor; 2=mediocre; 3=good; 4=excellent) or 6 (1=poor; 2=quite poor; 3=adequate; 4=quite good; 5=good; 6=excellent). Eventually evaluation findings should not have too many scales. 54