More meat, milk and eggs by and for the poor Applying a theory of change based approach to livestock research for development: Vietnam country report Froukje Kruijssen Mona Dhamankar Boudy van Schagen December 2021 CGIAR is a global partnership that unites organisations engaged in research for a food-secure future. The CGIAR Research Program on Livestock provides research-based solutions to help smallholder farmers, pastoralists and agro-pastoralists transition to sustainable, resilient livelihoods and to productive enterprises that will help feed future generations. It aims to increase the productivity of livestock agri-food systems in sustainable ways, making meat, milk and eggs more available and affordable across the developing world. The Program brings together five core partners: the International Livestock Research Institute (ILRI) with a mandate on livestock; the International Center for Tropical Agriculture (CIAT), which works on forages; the International Center for Research in the Dry Areas (ICARDA), which works on small ruminants and dryland systems; the Swedish University of Agricultural Sciences (SLU) with expertise particularly in animal health and genetics and the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) which connects research into development and innovation and scaling processes. The Program thanks all donors and organisations who globally supported its work through their contributions to the CGIAR Trust Fund © 2021 This publication is licensed for use under the Creative Commons Attribution 4.0 International License. To view this license, visit https://creativecommons.org/licenses/by/4.0. Unless otherwise noted, you are free to share (copy and redistribute the material in any medium or format), adapt (remix, transform, and build upon the material) for any purpose, even commercially, under the following conditions: ATTRIBUTION. The work must be attributed, but not in any way that suggests endorsement by the publisher or the author(s). CITATION: Kruijssen, F., Dhamankar. M. van Schagen, B. 2021. Applying a theory of change based approach to livestock research for development: Vietnam country report. Nairobi, Kenya: ILRI, Amsterdam, Netherlands: KIT. Contents 1 Introduction .................................................................................................................................... 4 2 Methodology ................................................................................................................................... 4 2.1 Realist evaluations .................................................................................................................. 4 2.2 Contribution analysis .............................................................................................................. 6 2.3 Implementation of the methodology ..................................................................................... 7 Step 1: Validating the initial Theory of Change .............................................................................. 7 Step 2: Reflections on Theory of Change ........................................................................................ 7 Step 3: KAP outcome harvesting ..................................................................................................... 8 Step 4: Validation and contribution analysis .................................................................................. 9 3 The priority country project in Vietnam ......................................................................................... 9 3.1 Background ............................................................................................................................. 9 3.2 Theory of change and assumptions ...................................................................................... 10 3.3 Project activities .................................................................................................................... 13 4 Results ........................................................................................................................................... 13 4.1 Reflections on the Theory of Change .................................................................................... 13 4.2 Results of the KAP survey ..................................................................................................... 15 Livestock producers ...................................................................................................................... 15 Vets and extension workers .......................................................................................................... 19 Policy makers ................................................................................................................................ 21 4.3 Contribution analysis ............................................................................................................ 23 4.4 Triangulating with Photovoice .............................................................................................. 25 5 Conclusions ................................................................................................................................... 27 References ............................................................................................................................................ 28 Annexes ................................................................................................................................................. 29 Annex 1: Survey questions KAP survey Li-chăn project .................................................................... 29 Annex 2: Output of MURAL of ToC reflections Li-chăn, February and October 2021 ...................... 32 Farmers ......................................................................................................................................... 32 Vets and extension workers .......................................................................................................... 36 Partnerships and policy makers (only Oct 2021 reflections) ........................................................ 39 1 Introduction The CGIAR Research Program on Livestock Agri-food systems (CRP Livestock for short) provided research-based solutions to support smallholder farmers, pastoralists and agro-pastoralists transition into sustainable, resilient livelihoods and to productive enterprises that will help feed future generations. An important component of the CRP Livestock research agenda was to demonstrate how livestock research can translate into impact through livestock value chain transformation in four selected priority countries, Ethiopia, Tanzania, Uganda and Vietnam. These projects built on work started by the CRP on Livestock and Fish, which ran from 2012 to 2016. In both research programs, an important component of the research agenda was to deliver impact through livestock research. Starting in 2019, a more concerted investment was made to package the research outputs and pilot an integrated livestock development intervention in each country that could take the research outputs to scale. A key notion was to ‘accelerate’ research to outcomes and impact by transforming the entire value chain, working with development partners from the start. This was done by building on ‘best-bet interventions’ already identified, involving all thematic teams of the program (covering the topics of animal health, genetics and breeding, feeds and forages, marketing and business skills, and the environment), and establishing in-country project leadership and coordination. The ambition was to integrate a range of research outputs into an intervention package (or a basket of intervention options) and pilot these as an integrated livestock development intervention in each site, with the ultimate goals of taking research outputs to scale. KIT Royal Tropical Institute partnered with the CRP Livestock to support the four priority country projects in monitoring and learning on livestock research for development (LR4D). The support provided by KIT consisted of two main components: 1. Facilitate mainstreaming a Theory of Change (ToC) approach in the priority country projects, which involved further development of the country projects’ ToCs, and supporting their use for reflection and review, establishing an evidence base to assess the cause and effect logic, and documenting lessons learned. 2. Monitor and document the lessons learned on the process of developing and delivering integrated intervention packages through the priority country projects, and document the lessons learned. This report describes the main results from the first component for the project in Vietnam. It also served as an input into a synthesis with lessons learned across the four country projects (Kruijssen et al., 2021). The remainder of this document is structured as follows, Section 2 describes the methodology used for the analysis, including the theoretical underpinnings of the methodology. Subsequently, Section 3 provides a short description of the country project in Vietnam and its ToC. Section 4 presents the results and finally in Section 5 some conclusions are presented. 2 Methodology 2.1 Realist evaluations The methodology applied for lessons learning and assessing the project’s ToC against evidence on its outcomes is based on realist evaluation, and contribution analysis. Realist evaluation is a sub-stream of theory-based evaluation approaches. Theory-based evaluation has been popularised in recent years as a response to the inability of impact evaluation methods (e.g. Randomised Controlled Trials) to determine the mechanisms by which interventions and research for development can achieve impact, and generate institutional lessons on research and innovation processes (Hall et al., 2003). While (semi-) experimental evaluation methods can determine whether an intervention works, it does not explain why and how the intervention works. However, this knowledge is crucial to scale innovation beyond test locations (Maru et al. 2016). Impact assessments thus need to be complemented with analytical frameworks that allow for institutional learning (Hall et al. 2003). Theory-based evaluation approaches aim to develop a ‘program theory’, that clarifies “how program activities are understood to cause (or contribute to) outcomes and impacts” (Westhorp, 2014; pp 4). Impact pathways describe the results chains (linkages from outputs to outcomes and impact), but a ToC adds the causal assumptions behind these links, i.e. what has to happen for the causal linkages to be realised (Mayne and Johnson, 2015). Theory-based evaluation approaches map the causal chain from inputs to outcomes and impact, and test the underlying assumptions to answer the ‘why’-question of impact. A realist evaluation seeks to improve understanding of how and why interventions work or do not work in particular contexts, and why different outcomes are achieved in different contexts. This approach is specifically based on the assumption that there is no one-size-fits-all solution; context strongly influences program outcomes. This implies that understanding context is an important part of understanding how and why programs work or not, so that informed decisions can be made about which programs or policies to use and how to adapt them to local contexts. A realist evaluation thus looks at what works for whom, to what extent, under what circumstances and over what duration (Westhorp, 2014). It is therefore particularly appropriate for evaluating pilot programs that are being scaled out. Realist evaluation tries to explain causation through observable and non-observable processes; in essence, the program activities (observable) influence the reasoning, norms, capacity and collective beliefs of the participants or stakeholders (non-observable) which determine decisions and choices of the same actors that result in program outcomes (observable). This underlying causal process may function differently in one context compared to another (Westhorp, 2014; Pawson and Tilley, 1997). A realist evaluation thus tests how these processes cause desired outcomes in a given context (White and Phillips, 2012). The starting point of a realist evaluation is the ToC of the program or project, which needs to be able to answer the following questions: 1. For whom will this program theory work and not work, and why? 2. In what contexts will this program theory work and not work, and why? 3. What are the main mechanisms by which we expect this program theory to work? 4. If this program theory works, what outcomes will we see? For the purpose of the research in the CRP Livestock priority country projects, we have operationalised realist evaluation through a participatory process developed by Douthwaite et al. (2008). This process helps to make outcomes more explicit in a ToC, by reflecting on groups of ‘next users’, i.e. those actors that are using the research outputs. In particular this looks at the changes in practice of the different actors that are envisioned, and what changes in the set of knowledge, attitudes and skills are required to achieve that change of practice, as well as the assumptions that need to hold for this to happen. 2.2 Contribution analysis To assess whether the ToCs of the CRP Livestock priority country projects have held good and planned outcomes have been achieved, a contribution analysis approach has been applied. Contribution analysis compares an intervention’s ToC against the evidence, and is mostly used in complex systems where changes in outcomes are the result of a several factors in addition to the intervention(s) (Koleros and Mayne, 2019). A contribution story is constructed by building up evidence that demonstrates the contribution of an intervention while also establishing the relative importance of other influences on observed outcomes (White and Phillips, 2012). In contribution analysis, two concepts are important to build evidence of attribution: - Necessity: the intervention actually caused the changed; i.e. nothing would have changed in the absence of the intervention. The intervention was thus necessary. - Sufficiency; the intervention was the only cause of the change; i.e. nothing else was needed to bring about the change. The intervention was sufficient. If evidence can be found to confirm that the intervention was both necessary and sufficient, the achieved change can be attributed to the intervention. The evidence base can be built by quantitative methods, but also by eliminating plausible alternative explanations for the change at outcome level (e.g. other donor interventions, new policies, market forces). Note that causation is often directly experienced or observed by project staff, participants and stakeholders. Collecting views from different key informants is thus essential qualitative data to be used in contribution analysis (Makaurau 2010). Different levels of contribution are distinguished, based on Ton and Glover (2019): - No contribution: there is no evidence that the causal process (change pathway) took place. - Weak contribution: there is some evidence that the intervention contributed to the causal process (change pathway). - Fair contribution: there is evidence that the intervention was a causal factor contributing to the acceleration or scaling of the causal process (change pathway). - Strong contribution: there is evidence that the intervention was a necessary (non-redundant) causal factor of starting the causal process (change pathway). Table 1 categorises the different levels of contribution based on observed changes. Table 1. Distinctive levels of contribution 2.3 Implementation of the methodology To implement the realist evaluation and contribution analysis methodology, KIT developed a process consisting of four main steps: 1) Validation of the projects’ ToCs, 2) Reflections on the validated ToCs with project staff and partners; 3) Assessment of knowledge, attitudes and practices outcomes among next users; 4) Validation workshop to reflect on the results and jointly develop the contribution analysis. Figure 1 provides a timeline for how this process was implemented. The steps are further described below. Figure 1. Timeline of the contribution analysis process Step 1: Validating the initial Theory of Change This consisted of: • Finalisation of ToC diagrams • Mapping of assumptions against scaling ingredients • Identification of anticipated changes among next users (early, intermediate and ultimate outcomes). The country teams identified the change pathways and underlying assumptions during the inception phase of the project. In the first step this has been collated in a ToC, which was validated by the country teams. The output of this step was a revised ToC diagram for each country, depicting the change pathways and underlying assumptions at the start of the country projects, including the anticipated changes (in knowledge, attitude, skills or practices) among next users at outcome level, and a mapping of assumptions against the scaling ingredients (based on Dror and Wu (2020)). Step 2: Reflections on Theory of Change This consisted of: • Reflection on key strategies, activities, outputs and outcomes. Is the project on track to achieve them? • Review of whether changes need to happen in the project to ensure they will be achieved. • Assessment if the assumptions hold. • Reflection on the extent to which the ToC holds. Virtual reflection workshops were held for each country in the first or second quarter of 2021 (depending on the country), to reflect on the ToC, using those developed in step 1. In Vietnam the first reflection meeting took place on the 25th of February 2021. The focus was on revisiting the change pathways and the underlying assumptions, and assessing progress towards achieving outputs and outcomes as identified in the ToC. A second and final reflection was held in each country during the country stakeholder meetings in the third quarter of 2021, to again assess the progress made and what the contributions of the priority country project was to this progress. For Vietnam, the second meeting took place on the 13th of October 2021. Again, these were online, or in the case of Ethiopia and Uganda hybrid meetings. At the start of the project it was anticipated that these meetings would happen in in-country workshops, however due to the Covid-19 pandemic these were moved online, using MURAL. The participants for both sessions were project implementers, from CGIAR and partner organisations. Workshop participants were separated in smaller sub-groups to discuss and reflect on one next user Box 1: Guiding questions for reflection sessions group or impact pathway of the ToC. Part 1: outcomes - What early outcomes have you seen emerge? What other Guided by a set of questions (see Box 1) early outcomes are likely to emerge soon? each group discussed for that particular - How is this different from what was intended? - What are the reasons for this difference, and what has user-group the (early) intended and contributed? (think about: contextual factors, project unintended outcomes that were delays, unexpected issues, assumptions that did not hold emerging and any differences observed etc. compared to the ToC, as well as any - What more is needed to achieve these outcomes? (e.g. adaptations in activities) potential changes required in the - How have/ are partners supporting or hindering these project strategy to achieve planned early outcomes? outcomes. In addition, assumptions - Any other (unintended) outcomes that emerged? were reviewed to assess if they held Part 2: Assumptions and if they influenced the achievement - Did assumptions hold? of outputs and outcomes, and identify - If not, are they a threat to achieving the early outcomes? any new assumptions and/or - Are any assumptions missing? (refer to the scaling ingredients) preconditions needed to achieve outcomes. Step 3: KAP outcome harvesting This consisted of collection and analysis of outcome level data from the project’s ‘next users’ on their perceptions of changes in knowledge, attitudes, and practices, and the likelihood that these will be continued beyond the project. Outcome harvesting was implemented through an adapted Knowledge, Attitudes Practices (KAP) survey to assess changes among next users, at the expected outcome level in the ToC, including perceptions on the contribution of the project to those changes. The anticipated outcomes in the ToC are then assessed (i) whether they occurred, (ii) to what extent, and (iii) why they can be attributed to the country projects. In Vietnam, a tablet-based questionnaire was developed for this study and used for data collection in November 2021. The questionnaire was conducted by the field coordinator of the project. The sample for this survey included: - 60 farmers, 10 from each village, randomly selected from the project participant list, evenly distributed between men and women where possible, - All vets and extension workers involved in the project activities - All district level policy makers involved in the project activities Table 2. Sample for KAP survey Vietnam Next user group Respondents Livestock producers 63 Vets and extension 12 Policy makers 3 Total 78 The questionnaire was based on the project’s ToC and the knowledge attitudes and practices that were expected to change among the next user groups as a result of the project (for more information see Section 3.2). The questionnaire took about 20-25 minutes to complete. The questionnaire was developed in English and translated to Vietnamese by the enumerator, and where needed further translated into other local languages. Questions on knowledge aimed to assess what respondents identified as the most important learnings related to crop-livestock systems and whether they perceived these learnings as applicable to their situation. These questions were either open-ended, or were Likert-type questions with five levels of (dis)agreement. Questions on attitudes were about the agreement with statements about key areas of relevance to the ToC and the assumptions part of the ToC. They also used the agreement scale and had open-ended follow-up questions to understand the motivations behind respondents’ answers. Finally, questions on practices, were aimed at assessing which practices respondents have implemented, which were open-ended and Likert-scale questions with a frequency scale. The survey questions can be found in Annex 1. Step 4: Validation and contribution analysis This consisted of a presentation of and reflection on the analysis of all data collected throughout the process and a joint contribution analysis. A joint validation workshop for all countries was organised on the 15th of December 2022. The Vietnam team was unable to join this meeting and they therefore had a separate meeting on the 22nd of December 2022. During this workshop, the results were presented of the KAP survey and combined with the results of the ToC reflection workshops to jointly develop the contribution analysis. 3 The priority country project in Vietnam 3.1 Background In North-western Vietnam, in the district of Mi Son, in Son La province (Figure 2), CRP Livestock is implementing the project ‘Li-chăn’. This project, promotes research-based interventions, to support smallholder livestock farmers in achieving equitable livelihoods, and reducing environmental degradation. The project works across three farming systems, with different socio-economic and ecological characteristics: 1) intensive system with good access to markets and relatively better Figure 2. Location of Mai Son capacity for innovation; 2) mixed crop-livestock systems; and, 3) district in Son La province (source: https://hdl.handle.net/10568/107 remote extensive systems with low access to markets. These farming systems correspond with their geographical location, i.e. lowlands, mid-lands, uplands (Figure 3). Figure 3. Farming systems types in Son La province (Blanchard et al., 2018, in the project proposal). The project site was chosen because of its specific characteristics; it is mountainous, with steep slopes, resulting in a challenging environment for the smallholder crop-livestock farmers in the area, more than 80% of the population is from ethnic minority groups, mainly Thai, H’mong and Yao, and there is a poverty incidence of 70%, almost threefold that of the rest of the country. A unique feature of the project in Vietnam, compared to the other priority country projects, was that in Vietnam a new site was chosen, and a farming system approach was adopted, shifting away from the strict focus on the pig value chain in the earlier phases of the CRP Livestock and Fish. Therefore, the project started in 2019, with several scoping assessments, including a feeds and fodder assessment (using a tool called G-FEAST), an assessment of breeds and breeding practices, a participatory epidemiology and gender survey, a value chain assessment, an assessment of livestock and the environment, a forage seed social network analysis, and a partnership assessment. Many of the tools used were developed or refined during earlier stages of the CRP Livestock (and CRP Livestock and Fish). 3.2 Theory of change and assumptions Based on these scoping assessments, the project proposal and an initial ToC were developed. The ToC was further refined and validated during the first stage of the process described in the methodology section (the ToC can be found in Figure 4). The project’s ultimate goal was that farmers in Mai Son in the three systems (lowlands, mid-lands, and uplands), in particular ethnic minorities, men and women, have: • sustainably intensified, equitable crop-livestock smallholder systems, • commercialised livestock smallholder production, and • an enabling policy environment for sustainable livestock intensification. The project intended to achieve this through different bundles of interventions, targeted to the three types of farming systems. The outcomes differed for the three systems (Table 3). Farmers were able to choose from different kinds of interventions provided by the CRP Livestock’s four thematic areas, covering the topics of animal health, genetics and breeding, feeds and forages, marketing and business skills, and the environment. Given the restricted timeframe of the remainder of the CRP Livestock (2.5 years), and the fact that the project started in a new site with a farming systems focus, the expectation from the start was that the project would not be able to reach beyond the early outcome level of the ToC by end of 2021. Table 3. Characterisation of the three farming systems, and the corresponding project vision Lowlands Midlands Highlands Farming system Intensive crop–livestock Mixed crop–livestock (beef Extensive grazing at tops with and pork) cropping in valleys Market access Good access to Hanoi and Less reliable access; more Remote with poor road access other major cities distant to cities Main ethnic Kinh and Thai Thai H’mong background Shared vision Branded, healthy and Stronger crop–tree–livestock Better productivity from sustainable green products interactions creating more improved breeds, animal delivered to markets income husbandry and disease (e.g. dried beef and local pork) resilience Source: project documents. Four key groups of stakeholders, or ‘next users’ were identified and targeted with interventions; livestock farmers, vets, extension workers, and policy makers. Table 4 provides and overview of those next users and the changes in knowledge, attitudes and practices that were expected to be observed among these next users. Table 4. KAP outcomes Li-chăn project, Vietnam Actors who are Change in practice required to Change in KAS required to Project strategies to bring about expected to change achieve the project’s vision support change in practice these changes in KAS and practice Farmers Adoption of integrated package Change in knowledge, - Training on relevant themes Improved collective action attitude, and skills for basket - Demonstration farms with among farmers for market of of customised interventions forage trials, and biosecurity livestock (products) measures - Common Interest Groups (CIGs) Vets Local vets diagnose animal Improved skills for disease Training on diseases diagnostics diseases correctly diagnostics and rational and antibiotic use antibiotic use Extension workers Extension workers support Improved skills in crop- Training in crop-livestock farmers in integrated crop- livestock management, management and nutrient flows livestock nutrient flows Policy makers Policy makers have improved Awareness raised among Dialogue with policy makers on dialogue for environmental decision makers about livestock and environment management livestock and environment Source: Theory of Change Li-chăn project. Note: KAS: Knowledge, attitudes and skills. Source: Stylized version of output of validation session with project staff, December 2020. Figure 4. Theory of Change of the Li-chăn project, Vietnam 3.3 Project activities The project implemented a variety of trainings, community engagement and other activities, (summarised in Table 5). This included among others farmer trainings on livestock breeds and breeding, animal health and biosecurity, and feeding and silage making, capacity development of vets and extension workers on artificial insemination (AI), disease management, vaccines and medicines, and nutrient management /drug use), the establishment of common interest groups in which groups of farmers could work together, and general awareness raising on the environment. Table 5. Overview of project activities, Li-chăn project Theme Project activities Genetics 4 trainings provided to farmers on pig and cattle breeding. 4 trainings provided to vets and extension on pig and cattle breeding. Animal health 2 trainings for farmers: 110 people trained on biosecurity, 106 people trained on farm management. 15 demonstration farms (farm house renovation, management, free vaccines, coaching, livestock diary) established. 2 trainings for animal health professionals: 20 + 7 vets trained on diseases, biosecurity, use of vaccines & antibiotics, outbreak investigation and control measures etc. Feeds & forages 180 households trained on feed techniques and practical demonstrations on feed classification and processing methods, silage making for cattle/ buffalo, feed mixing for Ban pigs. Forage trials with 155 households (who could test 4 grasses and 3 legumes). Environment Trainings on erosion prevention, soil fertility and biomass recycling, and composting for farmers CLEANED modelling: Environmental footprint established of different farming systems Nutrient balances modelled: Nitrogen flows and recycling quantified. Livelihoods Formation of 5 Common Interest Groups (CIGs) in 6 targeted villages, almost 60 households involved; Series of trainings provided to the CIGs Community/ public PhotoVoice: participatory method using photography to engage community members and for engagement monitoring and evaluation. Media coverage Art exhibition Partnerships Continuous engagement with local policy makers and other partners. Source: Author compilation from project documents and presentations. 4 Results This section presents the results of the reflections on the Theory of Change, those of the KAP survey, and the contribution analysis conducted during the validation workshop (i.e. the results of Step 4 in the process explained in Section 2.3 that describes the methodology). 4.1 Reflections on the Theory of Change The reflections on the Theory of Change are presented by pathway, which in this case corresponds with the next user group. The reflections on the livestock producer pathway of the ToC in the first and second reflection session are summarised in Table 6. During the first reflection it became clear that certain originally planned activities, outputs and outcomes, were not feasible, in particular the activity regarding the policies for product standards, and this was dropped accordingly. Other activities had not yet started during the first reflection. As there were only 8 months between the first and second reflection it is clear that the main activities of the project were concentrated in quite a restricted timeframe. 13 Table 6. Reflections on the livestock producers pathway of ToC Early outcomes Reflection 1 (February 2021) Reflection 2 (October 2021) Improved knowledge and skills - Genetics No: Training in cattle / pig genetics complete Yes. All training was in animal husbandry practices (farmers, cattle AI service providers, pig AI, boar conducted. AI, biosecurity, among men and women semen). Unclear if this has translated into an outcome. feed and forage and farmers (all) - breeding, animal - Health No: Training underway, no visible outcomes; environment, breeding health, feeds and forages - Feed No: training not yet taken Increased awareness of No: training on environment planned, in combination Yes, completed together with environmental degradation and with feeds and forages training (end Mar) feed and forage training. soils erosion among men and Content on contouring, women farmers (all) composting, animal manure. Improved business skills among No: farmer group formation to improve business skills of Yes. Farmer groups have been smallholder farmers (all) smallholder farmers planned, but not yet undertaken. formed and trained. There is (expectation is increased scale and better market an expectation of more linkages); training starting Q2 2021. productivity over longer term. Improved business skills among No: The only plan is to link value-addition co-operatives No. Decided to stop for now. private sector (value chain) (branded dried meat - existing group but not in the production is mostly actors (all) project site) to the farmer groups; plus some back-stop subsistence oriented and co- support to AI service providers (NIAS) though no specific op has limited capacity for business training planned; training Q2-2021 branding. Policies for product standards No: not sure if this will still happen Stopped, linked to outcome reviewed and improved (A) above. Improved collective action None: group formation (common interest groups) 5 common interest groups among smallholders and starting in Q2 2021 formed. private sector actors (m/w) for marketing of livestock (products) (all) Source: Summary of discussion of reflection workshops. The reflections on the vets and extension next user group (Table 7) show that at the time of the first reflection workshop several activities had already taken place with them, and that expectations around their potential outcomes were positive. However, during the second reflection workshop, it was observed to be difficult to draw conclusions about the ultimate effect of these activities on behaviours of vets and extension as service providers, and of farmers as recipients of their services. Table 7. Reflections on the vets and extension pathway of ToC Early outcomes Reflection 1 (February 2021) Reflection 2 (October 2021) Improved skills and capacity among Yes - Training has enhanced prior basic Difficult to evaluate how they local vets and extension workers for knowledge in biosecurity, animal health, AI and have translated knowledge into disease diagnostics and rational pre-selection (breeding) - better understanding skills in practice - some changes antibiotic use, breeding, AI, and feeds of concepts and importance. Extension are are observable; attitudes have (all) linking farmers to forage seeds certainly changed Improved skills among vets and Yes - Skills & knowledge improved in all areas, Yes. Integrated training helped extension workers in crop-livestock including biosecurity to address different needs of management, nutrient flows, disease farmers. diagnostics and rational antibiotic use, breeding, AI, marketing, feeds (all) New cattle AI service providers have Cattle AI service providers and farmers trained New AI workers have increased the skills, and knowledge to provide AI in pig-AI (some of whom may become a pig AI But unsure if there are more AIs services to farmers service provider); pig semen providers have being carried out and if the been trained (to produce pig semen as a input coverage is higher. into AI). Issues found with fertility of cattle in the area, which limits cattle AI in the short run. Source: Summary of discussion of reflection workshops. 14 As there had not been any activities with policy makers at the time of the first reflection workshop, this pathway was only covered during the second workshop (Table 8). The reflections in particular indicate the high importance of the role of partners and local level decision-makers in the project during a time when due to COVID-19 associated restrictions made it impossible for project staff from Hanoi to travel to the project site. The local partners, with a strong role of the project field coordinator, managed to keep activities going. This may have also led to a higher level of ownership among partners than would have been the case without the pandemic. Table 8. Reflections on the partners and policymakers pathways of ToC Early outcomes Reflections (October 2021 only) Partners are aware of the project and its Yes, partners are actively involved and played a very important role during activities and interested to collaborate on the COVID-19 pandemic to keep activities going when CGIAR project staff project activities was unable to travel from Hanoi. The role of the project coordinator who was in the field, was critical in informing and engaging the partners. Local authorities approve project activities Yes. They were actively involved in a number of activities and consulted easily when needed about the outputs to see what they can continue to support the local farmers with in the future. They also put this in their own action plans. Partners collaborate on activities Yes. They also supported communication to the local media. They are interested in specific interventions as it fits with the plans and would like to upscale. Awareness raised among decision makers The changes are being made from the commune level up, so from commune about livestock and the environment (all) to district to province. The province is now putting this in their plans. (DARD Son La made the push). Policy briefs were not yet developed, at national level this will take a bit more time. Decision-makers at district and provincial Yes, Artificial Insemination, animal health, biosecurity (through demo farms) level understand the needs for livestock development, including vaccination programs and antibiotic use Decision-makers from livestock and No, not yet. This was to improve dialogue between different sectors. At this environment departments at district and time there is no contact with other departments besides livestock. The provincial levels have improved dialogue for department of Environment was invited at the first workshop but there was environmental management no close contact. Source: Summary of discussion of reflection workshops. 4.2 Results of the KAP survey This section summarizes the results of the KAP survey, carried out in November 2022. These results are also presented by next user group. Livestock producers The livestock producer survey started with an open-ended question with regard to the most important learnings livestock producers had gained in the past 18 months (i.e. March 2020, the time when some activities had stated in the field)1. The most frequently mentioned learning was related to producing improved forage (mentioned by 41% of livestock producers), followed by the use of disinfection, making improved silage, and feed mixing and animal nutrition (see Table 9). All farmers 1 What is the most important thing you have learned related to animal husbandry practices, breeding practices, feeds and forages, business management or environmental degradation in the past 18 months? 15 attributed all their learning to the Li-chăn project; only one producer, in addition, attributed this knowledge to friends outside the project site. Table 9. Most important learnings of livestock producers (open-ended question) Most important learnings Research theme Frequency of Share of producers that answer mentioned this learning Production of improved forage Feeds & forage 26 41% Disinfection of stables/ pens Animal health 23 37% Making improved silage Feeds & forage 15 24% Feed (mixing) and animal nutrition Feeds & forage 14 22% Farm management practices Animal health 6 10% Making compost Environment 6 10% Artificial insemination for pigs / cattle Genetics & breeding 6 10% Breeding (with use of ban pigs) Genetics & breeding 5 8% Vaccination / medicine Animal health 4 6% Disease management Animal health 3 5% Use of mineral stone Animal health 2 3% Environmental degradation Environment 1 2% Source: KAP survey, 2021. After asking about their learnings, several follow up questions were asked regarding the producers’ knowledge and attitudes towards the technologies and practicespromoted by the project. (Table 10). These questions were Likert-scale questions, with different answer options depending on the questions (see notes below table for the options). Average scores were calculated (not applicable answers were excluded), and these are colour-coded depending on the degree of agreement/ adoption, where the lowest scores correspond with red, middle with yellow and highest with green. The results show that farmers feel they have improved their knowledge and have learned things they can apply. Likert-scale questions were complemented with open-ended questions, from which additional information about reasoning behind scoring is derived. Most average scores are in the highest category, with the exception of trust in extension workers, the capacity of women of being cattle keepers, and the ability of farmers to protect the environment through their own practices. This indicates that while knowledge can spread relatively quickly, changing attitudes may involve a longer process. A selection of the attitudes is discussed in more detail below: • Use of the project’s technologies and practices: The reasons why farmers were interested in using the technologies and practices promoted by the project included a desire to enhance productivity, reduce incidence of disease, reduce costs, reduce time and labour use, increase income and reduce erosion. Some farmers also indicated that the practices are simple to apply. One farmer was not fully convinced (yet), and indicated their traditional practices were better (e.g. drying straw in the field). A few others felt they had not yet mastered the innovations sufficiently so could not apply them yet. • Receiving more training from the project: Almost all farmers (with the exception of 2) indicated that they would like to receive more training. The two that did not wish to receive more training indicated they are too busy, or already know about the practices. There were many producers 16 who expressed that they has been unable to absorb all knowledge in the timeframe of the trainings. • Commercialisation of livestock: Reasons for wanting to commercialize livestock production included more income, for household assets, including livestock, furniture and savings, and children’s education (9) and for improved household wellbeing in general. Farmers generally trusted the vets but extension services less so (for livestock), mainly because they were yet to receive any advice on animal husbandry from them. • Trust in extension: The trust in extension workers was higher in one village than in the other five. The answers to the open-ended question about this topic, suggest that this is mainly because in the other villages the extension workers focused only on crop production and did not give any advice on livestock keeping,. The producers therefore had no experience with the advice of extension workers for livestock. Some producers indicated they had never received any kind of advice from the extension workers, or had never even seen them in the village. • Women’s capacity for cattle keeping: The attitudes with regard to women’s capacity for cattle keeping (of both men and women) were more negative in two out of six villages, and were also slightly more negative for men than women. Overall, respondents were of the opinion that it was women’s job to take care of pigs, and domestic duties, while men were considered the cattle keepers. While the project actively targeted women in some of the trainings, it did not have a component around gender transformative approaches, and it is therefore unsurprising that this gender attitude did not change. • Protecting the environment: For the question on the ability to protect the environment through their own practices, a disaggregation of scores by village showed that the lower scores were almost entirely from one village. In this village people were of the opinion that their lands were too steep and soils too poor, and that they had no control over this. In the other five villages people shared their learnings around how to address environmental degradation through measures such as planting forage and legumes and tree and grass contours to prevent soil erosion, and using compost and manure to protect soil fertility and humidity. The last set of questions regarded the practices that farmers put in place. For this set of questions the scores are lower; only the practice of asking vets and extension workers for advice has scores in the highest category, while artificial insemination has not become a regular practice yet. It is most likely too early in the process for farmers to already have fully adopted these practices. Reasoning behind the scores is provided below: • Artificial insemination for pigs is not adopted because of a lack of knowledge, a lack of access to a service provider, lack of AI equipment. The availability of a boar for natural mating, beliefs about the effect of AI on the sow, risk of losing money when their pig does not become pregnant, and only having fattening pigs were other reasons provided for not using AI for pigs. • Artificial insemination for cattle was not adopted because many respondents do not have cattle, only buffalo, and all reasons provided for pig AI. Although the scores on practices indicate that certain practices have not yet permeated, an open- ended question showed that producers did make changes in their livestock management practices that directly correspond with the most important learnings they listed (such as on forage, disinfection, stable improvements, composting, and silage making). They also indicated that they were planning to maintain all practices in the future. 17 Table 10. KAP survey scores livestock producers Question Average Min, N Type (range) score max Knowledge 4. This learning is something that I can use to improve my animal husbandry, 61 4.52 1, 5 Agreement (1-5) breeding, and feeding practices 5. This learning is something that I can use to improve/ add value to livestock 11 4.73 4, 5 Agreement (1-5) business management 6. As a result of the project, I have improved my knowledge about animal 62 4.34 1, 5 Agreement (1-5) husbandry practices, animal breeding and feed production 7. As a result of the project, I have improved my knowledge about livestock 9 4.56 4, 5 Agreement (1-5) business management 8. As a result of the project, I have improved my knowledge about 59 4.24 3, 5 Agreement (1-5) environmental degradation Attitudes 9. I am interested in using the technologies and practices the project has 62 4.23 1, 5 Agreement (1-5) promoted 10. I am interested in more training on animal husbandry and breeding 63 4.57 2, 5 Agreement (1-5) practices, feed production or business management 11. I want to commercialise livestock production 63 3.90 1, 5 Agreement (1-5) 12. I trust my local vet to treat my animals 63 4.46 1, 5 Agreement (1-5) 13. I trust the extension worker to support me in my livestock production 63 2.68 1, 5 Agreement (1-5) 14. Women are as capable of being cattle keepers as men 63 2.97 1, 5 Agreement (1-5) 15. Women are as capable of being pig keepers as men 63 4.19 1, 4 Agreement (1-5) 16. There is much I can do in my animal husbandry and feeding practices to 62 3.63 1, 4 Agreement (1-5) prevent environmental damage or soil degradation 17. I am interested in working together with other farmers to market my 63 4.14 3, 5 Agreement (1-5) livestock products, adopt technologies, share knowledge, do quality control or buy inputs together Practices 21. Are you a member of a livestock production organisation/ collective? 63 0.37 0, 1 Yes/No (0-1) 22. In the past 12 months, when doing PIG breeding, have you used artificial 62 0.50 0, 3 Frequency (0-3) insemination services? 23. In the past 12 months, when doing CATTLE breeding, have you used 61 0.00 0, 0 Frequency (0-3) artificial insemination services? 24. When you have a problem with your animals or feed do you ask a vet or 63 2.02 1, 3 Frequency (0-3) extension worker for advice? Source: KAP survey, 2021. Note: Colours indicate the degree to which a particular change in knowledge attitude or practice has taken place. The scores and color-coding are as follows. Agreement (1-5): 1-Strongly disagree, 2-Disagree, 3-Neither agree nor disagree, 4-Agree, 5-Strongly agree; red 1-2.33, yellow 2.34-3.67, green 3.68-5; Yes/No (0-1): 0-No, 1-Yes; red 0- 0.33, yellow 0.34-0.67, green: 0.68-1. Frequency: 0-No, never, 1-Yes, occasionally, 2-Yes, most of the time, 3-Yes, always; red 0-1, yellow 1-2, green 2-3. As outcomes of the new practices, producers listed among others improved productivity/ faster growth, reduction of disease incidence or reduced mortality, and to a lesser extent time and cost savings (Table 11). Only 3 producers indicated they had not seen any changes. 18 Table 11. Outcomes of the project mentioned by livestock producers Outcomes Frequency Share of producers that mentioned this outcome Improved productivity / faster growth 40 63% Lower disease incidence / reduced mortality 28 44% No change happened 3 5% Time savings 3 5% Cost savings 1 2% More confidence to invest 1 2% Source: KAP survey, 2021. Vets and extension workers The survey for vets and extension was structured in a similar way as the one for livestock producers, and started with the most important learnings of vets and extension workers. Their most important learnings regarded especially artificial insemination for cattle, pigs and breeds, and nutrition, feeds and improved forage (Table 12). Out of 12 respondents, 11 attributed this learning to the project, of which 9 mentioned the project as the only source. One farmer attributed the learning to the Agriculture Service Center of Mai Son alone and did not mention the project. Table 12. Most important learnings of vets and extension workers Most important learnings (open-ended question) Frequency of answer Share of vets / extension workers that mentioned this learning Artificial insemination cattle, pigs & breeds 7 58% Nutrition, feed (mix), improved forage 6 50% Farm management 3 25% Biosecurity & disinfection 3 25% Disease diagnostics & vaccination 3 25% Business plan 1 8% “Training of Trainers” training 1 8% Source: KAP survey, 2021. Similar to the producers, the vets and extension workers were asked a number of Likert-scale follow up question, of which the scores are presented in Table 13. All average scores are in the highest category, indicating a positive effect on knowledge and attitudes of vets and extension workers. The scoring questions that pertained to attitudes were followed up with open-ended questions, to gain an understanding of the reasoning behind these scores. Vets and extension wished to continue to engage with the project, because they were keen to learn more, such as about artificial insemination, disease diagnostics, and silage making. One respondent also indicated that a reason was that the project was effective in bringing positive impacts for vets and famers. The reasons provided for wanting more training were therefore quite similar to the question about continuing to engage with the project. Some respondents indicated that the training received so far, was still insufficient to master all key topics. Vets and extension workers were keen to apply their new skills 19 and knowledge to support farmers to improve their productivity, reduce disease, and improve income and wellbeing. According to vets and extension workers smallholders including women farmers, are interested in the practices and technologies promoted by the project. The reasons provided for this (perceived) interest, mostly boiled down to the desire of farmers to learn to increase their productivity, and with positive results like improved productivity, disease prevention and reduced mortality already emerging some respondents indicated they could tell because farmers were asking them about the project’s interventions. According to the vest and extension workers, women are especially interested as they are often in charge of the livestock, and are interested in labour saving measures. However, two respondents also raised issues around barriers for women due to limited understanding of the Vietnamese language. Table 13. KAP survey scores vets and extension workers Question N Average Min, Type score max Knowledge 3. This learning is something that I can apply in my work 12 4.42 2,5 Agreement (1-5) 4. I have improved skills for disease diagnostics 8 4.50 4,5 Agreement (1-5) 5. I have improved knowledge around antibiotic use 8 4.13 3,5 Agreement (1-5) 6. I have improved skills and knowledge about animal breeding and artificial 9 4.44 3,5 Agreement (1-5) insemination 7. I have improved skills in crop-livestock management 1 4.00 4,4 Agreement (1-5) 8. I have improved knowledge in nutrient flows 2 4.00 4,4 Agreement (1-5) 9. This learning was a result of participating in trainings offered by the Li-chăn 12 4.58 3,5 Agreement (1-5) project Attitudes 10. I am willing to (continue) engaging with the project 12 4.92 4,5 Agreement (1-5) 11. I am interested in receiving (more) training 12 4.92 4,5 Agreement (1-5) 12. (Smallholder) farmers are interested in the practices and technologies 12 4.17 1,5 Agreement (1-5) promoted by the project 13. Women farmers are interested in the practices and technologies 12 4.58 1,3 Agreement (1-5) promoted by the project 14. I am interested in applying my new skills and knowledge to support 12 5.00 5,5 Agreement (1-5) livestock producers 15. I feel supported by my employer to apply my new skills and knowledge 6 4.50 4,5 Agreement (1-5) Source: KAP survey, 2021.Note: Colours indicate the degree to which a particular change in knowledge attitude or practice has taken place. The scores and color-coding are as follows. Agreement (1-5): 1-Strongly disagree, 2-Disagree, 3-Neither agree nor disagree, 4-Agree, 5-Strongly agree; red 1-2.33, yellow 2.34-3.67, green 3.68-5. In the past 6 months, 9 out of 12 vets and extension workers implemented changes such as the type of advice and training they provide to farmers (e.g. about breeding, confinement of animals, disinfection, biosecurity, and treatment of lumpy skin disease), the practising and awareness raising around artificial insemination and all respondents except one attributed these changes to the project. The other three respondents indicated they did not make any changes. 20 The vets and extension workers were also asked about the perceived outcomes for farmers, and mainly associated those with increased productivity (9), reduced mortality and lower disease incidence (7), and increased investment in livestock and reduced risk (4). Other perceived outcomes for farmers were improved farm management practices (2), better sales (1), reduced time consumption (1) and improved feeds and forages (1). They had several ideas about the actions that still need to happen to improve adoption of practices among livestock producers (Table 14). These related in particular to artificial insemination, and associated constraints such as access to equipment and good quality semen to conduct AI, changes in attitudes among farmers about AI and its effects on the animal’s reproductive system, and more training for service providers. A range of other barriers were provided, and among those improved access to credit to afford was a frequently mentioned necessity for farmers to invest in practices. Table 14. Actions still needed to improve adoption according to vets and extension workers What is still needed to adopt / maintain interventions? Frequency Share of vets and extension workers that mentioned this issue Challenges related to artificial insemination (AI): Access to AI equipment 7 58% Access to good quality pig/ cattle semen 5 42% Change in attitudes among farmers about AI (and the effect on 4 33% reproductive system) More training for service providers 2 17% After-care after insemination (follow up on whether or not 1 8% animal is pregnant) Other / general challenges Access to credit 7 58% Access to electricity 1 8% Improved road access to some villages 1 8% Strengthening of CIGs management skills 1 8% Access to market information 1 8% Adult learning skills 1 8% Overcoming language barriers 1 8% Source: KAP survey, 2021. Policy makers Finally, the policy makers (which was only a sample of 3), had learnings about options of cross- breeding with local pigs, and technologies in breeding, about the benefits of planting forage, the advantages of biomass recycling to reduce input costs, and the importance of projects being demand-driven. They attributed this knowledge to the project, and to television, and other trainings from the department of Livestock. The scores on the Likert-scale questions indicate that they feel this is knowledge they can use in their work, and their attitudes towards the project and evidence for policy making are also positive. They would like to see that the approach of the project is scaled up and out to other areas. With 21 regard to women’s role, they indicate that language barriers and a lack of self-confidence, and in some cases their lack of decision-making power and control over resources, are limiting women from participating more in trainings. Comparing these answers to the open-ended question related to this issue with the Likert scores seems to suggest that the scores are lower than they should be, as these issues clearly do relate to restricting gender norms. It appears that it is too early in the process to see much application of the new knowledge in policy making, and also the linkages between livestock and the environment are not obvious yet. This aligns with the findings from reflection workshops, that those linkages were yet to be established. Table 15. KAP survey scores policy makers Question N Average Min, Type score max Knowledge 3. This insight is something that I can apply in policy dialogue 3 4.67 4,5 Agreement (1-5) 4. As a result of the project, I have improved my understanding of the needs of 3 4.67 4,5 Agreement (1-5) livestock producers Attitudes 5. I am willing to (continue) engaging with the project 3 4.67 4,5 Agreement (1-5) 6. I am interested in (more) evidence to support policy dialogue on livestock 3 4.33 1,2 Agreement (1-5) production and the environment 7. It would be useful for me to engage in regional level policy dialogue to 3 4.33 4,5 Agreement (1-5) ensure that social economic development plans respond to local livestock producers needs 8. I trust the evidence related to livestock production 3 4.67 4,5 Agreement (1-5) 9. I believe in the value of credible evidence for policy dialogue and reform 3 4.67 4,5 Agreement (1-5) 10. Existing socio-cultural norms need to be changed to ensure equitable 3 3.67 3,4 Agreement (1-5) access (especially for women) to extension services, input suppliers and service providers Practices 13. Do you participate in events that enable you to obtain more evidence of 3 1.67 1,2 Frequency2 (0-2) the value of improved animal husbandry practices? 14. Do you participate in events that enable you to obtain more information 3 1.33 1,2 Frequency2 (0-2) about the link between animal husbandry practices and the environment? 15. Do you make use of the evidence that you have been informed about in 3 1.67 1,2 Use (0-3) policy dialogues and related forums? Source: KAP survey, 2021. Note: Colours indicate the degree to which a particular change in knowledge attitude or practice has taken place. The scores and color-coding are as follows. Agreement (1-5): 1-Strongly disagree, 2-Disagree, 3-Neither agree nor disagree, 4-Agree, 5-Strongly agree; red 1-2.33, yellow 2.34-3.67, green 3.68-5; Yes/No (0-1): 0-No, 1-Yes; red 0- 0.33, yellow 0.34-0.67, green: 0.68-1. Frequency2 (0-2): 0-No, never, 1-Yes, occasionally, 2-Yes, often; red 0-0.67, yellow 0.68-1.33, green 1.34-2; Use (0-3): 0-No, I don’t use any of it, 1-Yes, I use some of it, 2-Yes, I use most of it, 3-Yes, I use all of it; red 0-1, yellow 1-2, green 2-3. The policy makers were also asked about the three most important policy changes they were planning to propose with regard to livestock production and/or processing. There was one topic all three respondents mentioned independently, which was increasing the area of forage. Other topics included increasing the number of cattle (and reducing buffalo) mentioned by two, giving agency to 22 farmers, making interventions more demand-driven (1), paying more attention to silage (1) and supporting innovations to reduce environmental pressure and costs (1). Finally, they were also asked about their perceptions of changes observed in vets and extension workers. These centred around their increased knowledge and skills in biosecurity, farm management, feed preservation, nutrition, artificial insemination and disease diagnostics and the resulting ability to provide better advice to farmers and better understand their demands. 4.3 Contribution analysis In this section we bring the results of the two previous sections together and present the contribution analysis (Table 16). As described in Section 2.1 on realist evaluations, we aimed to explain causation through observable and non-observable processes. The Vietnam priority country project implemented (observable) activities, with a strong focus on trainings and other forms of capacity development. These activities influenced the attitudes, beliefs and capacities of the participants (in essence non-observable) which determined their decisions and choices in terms of livestock keeping and farming practices (observable), which in turn resulted in program outcomes such as increased productivity and incomes (observable). As most interventions of the project only took place in the last year of the project, and for livestock producers even the last six months, it was too early in the process to have objective and quantitative data on the ultimate outcomes. We therefore attempted to make the ‘unobservable’ knowledge, attitudes and skills observable through the KAP survey, and gathered project participants’ perceptions on changes they made and benefits they obtained as a result. These early and intermediate outcomes served as an indicator to assess whether the project was on track to achieve the ultimate outcomes as foreseen in the ToC. The results show that even with a short timeframe, farmers have started experiencing some intermediate outcomes, such as reduced disease incidence and enhanced productivity. The fact that producers saw some results in a short timeframe also encouraged them to continue using the new practices, and may also encourage others to start adopting them. This would require that local partners are now able to scale capacity development activities up to other farmers in and outside the project area. At the same time, it is also clear that certain attitudes and beliefs, such as those around artificial insemination require more time to change. Adoption of artificial insemination and some other practices, were also limited by other factors, such as availability of the necessary inputs and equipment and capital to invest, and these are therefore important factors to consider for scaling such interventions. Artificial insemination for cattle was also hampered by the fact that during AI trainings, the team discovered that much cattle in the communities was infertile due to a combination of factors such as inappropriate feeding, health issues, lack of heat detection, which would require a holistic and more long-term approach to address. There were also issues with regard to semen preservation, which led to poor quality semen. For access to capital for AI, the Vietnamese government offers financial support that farmers can request, however, the application process takes more than a year. For service providers also equipment can be financially supported up to 70%. This is particularly of relevance for cattle semen, which needs liquid nitrogen storage, which requires specialised equipment and access to the liquid nitrogen. For further scaling of the intervention package, these are issues that would need to be addressed. 23 Table 16. Contribution analysis Li-chăn project, Vietnam Early or intermediate outcome ToC reflection KAP survey Additionality Necessity Farmers Change in knowledge, attitude, and skills Yes Yes to some degree, not all Yes. Li-chăn is only Yes for basket of customised interventions for AI, some feel they need intervention among men and women farmers (all) more training Increased awareness of environmental Yes Some, not all relate this to Yes. Li-chăn is only Yes degradation and soils erosion among their own practices intervention men and women farmers (all) Improved collective action among Improved collective 37% member of a collective smallholders and private sector actors action but not (m/w) for marketing of livestock necessarily for (products) (all) marketing Improved business skills among private Stopped - sector (value chain) actors (all) Smallholders (m/w) adopt flexible Not discussed Some Yes. Li-chăn is only Yes combinations of customised intervention interventions Smallholders adopt improved marketing Not a major focus - practices Vets and extension workers Improved skills and capacity among local Yes some changes Yes Yes, training content vets for disease diagnostics and rational observable and approach were antibiotic use, breeding, AI (all) unique to Li-chăn Improved skills among extension workers Yes Yes in crop-livestock management, nutrient flows, disease diagnostics and rational antibiotic use, breeding, AI, marketing (all) New cattle AI service providers have the More AI workers, but Skills yes, but not the skills, and knowledge to provide AI has coverage increased? equipment, good quality services to farmers semen, credit Extension workers support farmers Not discussed Limited trust of farmers in integrated crop-livestock through the extension workers bundle of customised interventions Local vets diagnose animal diseases Not discussed Yes Yes for AI. For correctly more frequently and apply diagnostics it may more rational antibiotic use, apply AI, only have sped up proper breeding practices (all) the process. Cattle AI service providers provide AI Not discussed No, pigs a little services to farmers Policymakers Awareness raised among decision- Slowly, going from Yes Yes, policymakers Yes makers about livestock and the commune up. Also exposed by project. environment (all) unintended awareness on participatory approach. Decision-makers at district and provincial Yes Yes level understand the needs for livestock development, including vaccination programs and antibiotic use Decision-makers from livestock and Not yet - environment deprt.at district and provincial levels have improved dialogue for environmental management 24 The activities related to marketing of livestock products were significantly adapted compared to how these were planned at the start of the project. This culminated in efforts around the establishment of the Common Interest Groups (CIGs). It was too early in the process to register any outcomes as a result of the CIGs, although the KAP survey registered a positive attitude towards collective action; an important first step in the process of making the CIGs operational. The vets and extension workers also had early outcomes in particular with regard to their changes in knowledge and skills around disease diagnostics, vaccines, breeds, animal nutrition, improved forage, farm management, biosecurity and artificial insemination. However, the artificial insemination service providers face a number of constraints to deliver these services, already discussed in relation to the farmers. We observed limited trust among the extension workers farmers in with regard to advice about livestock. For a large part this was due to the fact that they had yet to be exposed to such advice (while in most villages they had received advice on crop production from the extension workers). With regard to the policy makers, the reflection workshops revealed that the level of engagement of local level decision-makers was considered beyond expectations by the project staff. This was in part due to the unexpected occurrence of the COVID-19 pandemic, and the associated measures that restricted staff from travelling from Hanoi to the project site, which provided space for local level actors to engage. The project focussed on interactions with policy makers directly relevant to the project activities (and already known to the project staff) from the department of Livestock. New contacts with the department of Environment were not considered a priority at this stage. The activities of linking government departments, with the aim to improve alignment between departments for better environmental management, were therefore not conducted in the way foreseen. Farmers, vets and extension workers, and policy makers all attributed many or most of the learnings they gained to the Li-chăn project. Farmers in particular, mentioned few other sources of information. The ToC reflections of project staff confirm this and indicate that there are limited other sources for farmers and vets and extension to learn about these themes, apart from government programs that seem to have somewhat limited reach into communities, in particular those that are in remote locations. 4.4 Triangulating with Photovoice Apart from the data collection described in this report, the Li-chăn project carried out other types of data collection, including a baseline survey and a COVID survey. However, an endline survey had yet to be conducted at the time of developing this report, because ultimate impacts were not yet expected in the project’ timeframe. The project also used Photovoice as a method for participatory monitoring and evaluation (see Mai Tu, 2022). Photovoice is a participatory method which uses photography (by project participants or community members) as a means to enhance dialogue and discussion. To this end, community members are given a camera to take photos of situations related to a particular topic of interest. The photos that participants take, are used as a medium to generate discussion about this particular topic. In the Vietnam priority country project, 18 farmers and 2 vets (12 men, 8 women), across the 6 villages in which the project was active, were given a camera, together with training on how to use them, and on interview techniques to collect additional 25 information about the stories the photographs tell. The instructions were to take photographs of livestock-related issues and changes made as a result of the project. This process resulted in 4800 photos and 170 stories. The stories were categorized according to the research themes of the project. In order of occurrence these were feeds and forage (71 stories), livelihoods (29 stories), animal health (28 stories), breeding and genetics (21 stories), livestock and environment (18 stories), however usually stories covered more than one theme. In addition, there were 9 stories about cultural practices (not a research theme of the project). This aligns with the findings of the KAP survey, in which the top 5 topics mentioned by farmers as learnings were related to feeds and forage (3) and animal health (2). These two areas are also the ones that include a number of practices that are easier to implement in the short term, and have some quick wins, and in many cases do not require major investments. The Photovoice stories generally confirm the findings of the KAP survey (Table 17). The themes with most prominence in the KAP survey (i.e. feeds and forages and animal health), are also the ones that feature most in the Photovoice stories. The livelihood stories of Photovoice regard general cultural practices, not changes in practices as a result of the project, with the exception of the CIGs, which did not feature much in the KAP survey. Table 17. Triangulation of KAP survey findings for livestock producers and Photovoice stories Themes KAP survey Photovoice Feeds and Most common theme of learnings and practice changes. Also most popular theme of stories. Themes forages Most frequently mentioned learnings and changes: of stories overlap with learnings and (71 stories) production of improved forage, making improved silage, practices shared in KAP survey. feed (mixing) and animal nutrition. Livelihoods CIGs are not mentioned in the open-ended questions about The stories shared around livelihoods (29 stories) key learnings and changes made, but there is interest to through Photovoice are diverse and often work together with other farmers. Livelihood outcomes regard cultural practices unrelated to the reported include improved productivity and income, time project activities. The CIGs, which are a savings, cost savings, more confidence to invest. project activity, do feature in the stories (yet frequency is not clear from the report). Animal health Learnings and changes made with regard to biosecurity, Themes of stories overlap with learnings and (28 stories) farm management, disinfection, use of mineral stone, use of practices shared in KAP survey. In addition vaccines, disease diagnostics and farm management and stories were shared on keeping dairy cattle. also the trust in local vets. Genetics and Learnings related to breeds and cross-breeding with ban Both the learnings and the obstacles are breeding pigs and AI for pigs and cattle. confirmed by the Photovoice stories. (21 stories) Remaining obstacles for adoption of AI due to negative attitudes and beliefs about the method. Limitations for adoption of AI due to lack of access to equipment and inputs and / or capital to purchase them. Livestock and The frequency of learnings mentioned around the All of the practices listed by farmers as environment environment were low for the initial question on most potential measures to support the (18 stories) important learnings. However when directly asked about environment also featured in the Photovoice what actions could be taken to support the environment, stories. The fact that these stories are fewer the majority of farmers could list several actions. It seems than the other themes also aligns with the these learnings are not top of mind in order of importance, limited importance given to the theme in the but have been internalised nonetheless. KAP survey. Source: Authors’ compilation of KAP survey and Photovoice results. 26 5 Conclusions The CRP Livestock priority country project in Vietnam had a different history than the other three priority country projects. It started fresh in a new site, and deviated from the single value chain approach, by choosing to focus on farm systems of farm households keeping livestock. This meant that the project could not build on years of previous work in the same way that the other country projects could, but that it could benefit from the earlier research in the CRP Livestock (and Fish), including the work in the pig value chain in Vietnam. This meant that assessment tools, such as G- FEAST and CLEANED, that were piloted and refined earlier, could be applied in relatively short timeframes, and that knowledge generated by the different research themes could be utilised in the interventions. As such, the Vietnam priority country project functioned as a pilot test of how other (non-CGIAR) livestock related programs could implement the tools and interventions generated by the research of CRP Livestock. This also meant that in the case of the Vietnam priority country project, there was limited time for intermediate outcomes to emerge. The nature of the interventions, consisting of a set of trainings and other capacity development activities, did lend itself well for a realist evaluation approach, even if intermediate outcomes were yet to be measured through quantitative methods. These particular activities allowed us to examine the logic of the ToC and assess whether it is likely that the changes in capacities and attitudes are likely to lead to the planned intermediate and ultimate outcomes. The results of the KAP survey and the contribution analysis, showed that the logic of the ToC seemed to hold, although there were still a number of factors constraining full adoption of all suggested interventions. This included for example constraining attitudes and lack of access to capital that limit the adoption of artificial insemination. While arguably some of these issues could have potentially been foreseen at the start and included in the original design of the project, it also highlights the importance of an iterative process, in which the barriers and opportunities to reaching a project’s goals in a ToC are continuously assessed and new measures are taken to address such constraints or take advantage of opportunities. 27 References Blanchard M, Van Moere C, Đuc Do Van, Hàn Anh Tuấn , Le Thi Thanh Huyen (2018). Trade-off and synergies of integrating intensive Livestock production with Agroecology in Mountainous regions. Agroecology Futures Regional Forum, Siem reap (Cambodge) 6-8 th November 2018. Douthwaite B, Alvarez S, Thiele G, Mackay R. 2008. Participatory Impact Pathways Analysis: a practical method for project planning and evaluation. ILAC Brief 17. CGIAR Dror, I. and Wu, N. 2020. Scaling better together: The International Livestock Research Institute’s framework for scaling. Nairobi, Kenya: ILRI. Hall A, Sulaiman VR, Clark N, Yoganand B. 2003. From measuring impact to learning institutional lessons: an innovation systems perspective on improving the management of international agricultural research. Agricultural Systems 78: 213-241 Koleros A, and Mayne J. 2019. Using actor-based Theories of Change to conduct robust evaluation in complex settings. Canadian Journal of Program Evaluation 33(3) Kruijssen, F., Dhamankar, M., van Schagen, B. and Posthumus, H. 2021. Applying a Theory of Change based approach to Livestock Research for Development (LR4D): Learnings from the CGIAR Research Program on Livestock agri-food systems. Nairobi, Kenya: ILRI, Amsterdam, Netherlands: KIT. Mai Tu. 2022. Li-chan stories of change: evidence from photovoice. Hanoi, Vietnam: Alliance of Bioversity International and CIAT. Makaurau T. 2010. Outcomes, impacts and causal attribution; affordable, practical, feasible options for the real world. ANZEA regional symposium; May 21st 2010; Auckland, New Zealand Maru Y, Sparrow A, Stirzaker R, Davies J. 2016. Integrated agricultural research for development (IAR4D) from a theory of change perspective. Agricultural Systems 165: 310-320 Mayne J, Johnson N. 2015. Using theories of change in the CGIAR Research Program on Agriculture for Nutrition and Health. Evaluation 21(4): 407-428 Ton G, Glover D. 2019. Improving knowledge, inputs and markets for legume expansion: a contribution analysis of N2Africa in Ghana and Ethiopia. IDS Practice Paper 10. Institute for Development Studies Westhorp G. 2014.Realist impact evaluation; an introduction. Methods Lab publication. ODI / DFAT / Better Evaluation. Overseas Development Institute, London White H, Phillips D. 2012. Addressing attribution of cause and effect in small n impact evaluations: towards an integrated framework. 3IE Working Paper 15. International Initiative for Impact Evaluation. 28 Annexes Annex 1: Survey questions KAP survey Li-chăn project Question Type LIVESTOCK PRODUCERS 1. Which project activities have you or others in your household participated in? select_multiple activities 2. What is the most important thing you have learned related to animal husbandry practices, text breeding practices, feeds and forages, business management or environmental degradation in the past 18 months? 3. From who or where did you learn this? text 4. This learning is something that I can use to improve my animal husbandry, breeding, and select_one agreement feeding practices 5. This learning is something that I can use to improve/ add value to livestock business select_one agreement management 6. As a result of the project, I have improved my knowledge about animal husbandry practices, select_one agreement animal breeding and feed production 7. As a result of the project, I have improved my knowledge about livestock business select_one agreement management 8. As a result of the project, I have improved my knowledge about environmental degradation select_one agreement 9. I am NOT interested in using the technologies and practices the project has promoted select_one agreement 9b. Why or why not? text 10. I am interested in more training on animal husbandry and breeding practices, feed select_one agreement production or business management 10b. Why or why not? text 11. I do NOT want to commercialise livestock production select_one agreement 11b. Why or why not? text 12. I trust my local vet to treat my animals select_one agreement 12b. Why or why not? text 13. I do NOT trust the extension worker to support me in my livestock production select_one agreement 13b. Why or why not? text 14. Women are NOT as capable of being cattle keepers as men select_one agreement 14b. Why or why not? text 15. Women are NOT as capable of being pig keepers as men select_one agreement 15b. Why or why not? text 16. There is NOT much I can do in my animal husbandry and feeding practices to prevent select_one agreement environmental damage or soil degradation 16b. Why or why not? text 17. I am interested in working together with other farmers to market my livestock products, select_one agreement adopt technologies, share knowledge, do quality control or buy inputs together 17b. Why or why not? Please elaborate what you would be most interested in doing together text 18. What are the 3 most important changes you have made in the last 18 months with regard to text animal husbandry, animal breeding, feed/fodder production and/or business management? 19. Which of the above changes are as a result of your involvement in the Li-chăn project? select_multiple change (more than one answer possible) 29 20. Which of the above changes are you planning to maintain in the future (beyond 2021)? select_multiple change (more than one answer possible) 20b. Why not any? text 21. Are you a member of a livestock production organisation/ collective? select_one yes_no_na 22. In the past 12 months, when doing PIG breeding, have you used artificial insemination select_one frequency services? 22b. Why not (more)? text 23. In the past 12 months, when doing CATTLE breeding, have you used artificial insemination select_one frequency services? 22b. Why not (more)? text 24. When you have a problem with your animals or feed do you ask a vet or extension worker select_one frequency for advice? 24b. Why not (or not more often)? text 25. What is the most important change you’ve seen in livestock production in your household text after using the practices and technologies and services provided by the project? VETS AND EXTENSION 1. What is the most important thing you have learned related to providing veterinary / advisory text services to livestock producers in the past 18 months? 2. From who or where did you learn this? text 3. This learning is something that I can apply in my work select_one agreement 4. I have improved skills for disease diagnostics select_one agreement 5. I have improved knowledge around antibiotic use select_one agreement 6. I have improved skills and knowledge about animal breeding and artificial insemination select_one agreement 7. I have improved skills in crop-livestock management select_one agreement 8. I have improved knowledge in nutrient flows select_one agreement 9. This learning was a result of participating in trainings offered by the Li-chăn project select_one agreement 9b. Which parts were not? text 10. I am willing to (continue) engaging with the project select_one agreement 10b. Why or why not? text 11. I am interested in receiving (more) training select_one agreement 11b. Why or why not? text 12. Very few (smallholder) farmers are interested in the practices and technologies promoted by select_one agreement the project 12b. Why or why not? text 13. Very few women farmers are interested in the practices and technologies promoted by the select_one agreement project 13b. Why or why not? text 14. I am interested in applying my new skills and knowledge to support livestock producers select_one agreement 14b. Why or why not? text 15. I feel supported by my employer to apply my new skills and knowledge select_one agreement 15b. Why or why not? text 16. What are the 3 most important changes you have made in the last 12 months in the way you text provide veterinary/ extension advisory services to livestock producers? 17. Which of the above changes are as a result of your involvement in the Li-chăn project? select_multiple change (more than one answer possible) 30 18. What still needs to happen or change for you to get more/ new livestock producers to adopt text the animal husbandry and marketing practices? 19. What is the most important change you’ve seen in livestock producers with respect to their text production and/or marketing practices? POLICY MAKERS 1. What is the most important insight you have gained related to livestock production and the text environment in the past 18 months? 2. From who or where did you get this insight? text 3. This insight is something that I can apply in policy dialogue select_one agreement 4. As a result of the project, I have improved my understanding of the needs of livestock select_one agreement producers 5. I am willing to (continue) engaging with the project select_one agreement 5b. Why or why not? text 6. I am not interested in (more) evidence to support policy dialogue on livestock production and select_one agreement the environment 6b. Why or why not? text 7. It would be useful for me to engage in regional level policy dialogue to ensure that social select_one agreement economic development plans respond to local livetsock producers needs 7b. Why or why not? text 8. I trust the evidence related to livestock production select_one agreement 8b. Why or why not? text 9. I believe in the value of credible evidence for policy dialogue and reform select_one agreement 9b. Why or why not? text 10. Existing socio-cultural norms need to be changed to ensure equitable access (esp.for select_one agreement women) to extension services, input suppliers and service providers 10b. Why or why not? text 11. What are the 3 most important policy changes you plan to propose/ have proposed in the TEXT last 18 months with regard to livestock production and/or processing? 12. Which of the above policy changes are you proposing as a result of your association with the select_multiple change Li-chăn project? 13. Do you participate in events that enable you to obtain more evidence of the value of select_one frequency2 improved animal husbandry practices? 14. Do you participate in events that enable you to obtain more information about the link select_one frequency2 between animal husbandry practices and the environment? 15. Do you make use of the evidence that you have been informed about in policy dialogues and select_one use related forums? 15b. If you don’t use most or all, why not? text 16. What is the most important recent change you’ve seen in vets and extension workers in text Chieng Chung and Chieng Luong commuines related to livestock services? 31 Annex 2: Output of MURAL of ToC reflections Li-chăn, February and October 2021 Farmers Early (signs of) Have you seen the early Was anything If different, what was What else is How did partners Are there any Are there any lessons outcomes outcomes emerge? Yes / different from what reason for the difference? needed to support or hinder the unintended learned? No you intended? If yes, achieve the outcomes? outcomes (positive how? outcome (if or negative) that any)? emerge? Improved Genetics No: Training Farmers have Not yet seen adoption of Could show Local partners also interest by For later outcome on knowledge and in cattle / pig genetics improved knowledge other trainings; expect some adoption involved in all the women to become adoption: If we want skills in animal complete (farmers, but not much this to happen later in in specific areas trainings so far, AI providers (need adoption (health) we need a husbandry practices cattle AI service practices / adoption time; needs more as earlier providing local context to be confirmed; participatory process on among men and providers, pig AI, boar observed, except AI training and practicing outcome - Partners involved in also included both which interventions to women farmers (all) semen). Unclear if (considered as big (e.g. biosecurity) would be worth genetics training gender in training). adopt - breeding, animal this has translated success as in baseline Cattle AI is more complex monitoring; (NIAS, .....); animal Trained trainers Could have done training in health, feeds and into an outcome. many farmers didn't / difficult (though 3 who Time; follow-up health (NIVR); feeds will be an more participatory way, forages Health No: Training know about AI / tried want to be support (NIAS) Genetics: additional with more visuals and in underway, no visible thought it negative). professional) Training Demo trials for certified of AI service outcome as a local languages (e.g. animal outcomes; Except for cattle AI and other activities F&F (new providers Will be result of working health training) to improve Feed No: training not for which adoption is delayed due to Covid-19 legume / monitoring the trial with the local engagement of participants. yet taken still slow pandemic, and therefore forage) still to also they work with partners Some trainers were not outputs/outcomes also be done farmers (agric qualified. For F&F training extension) afterwards will do ToT first (including ethnic minority / gender sensitivities) Yes. All training was farmer-led participatory approach is conducted. AI, forage needed. The first training biosecurity, feed and trials were set was not adapted for forage and up Vietnam. Approaches environment, breeding require local contextualisation. 32 Increased No: training on awareness of environment planned, environmental in combination with degradation and feeds and forages soils erosion among training (end Mar) men and women Yes, completed Project partners Involve local people more. farmers (all) together with feed and supported Training needs to be more forage training. Content greatly during COVID applied - learning by doing on contouring, as the tech team in rather than theoretical. composting, animal Hanoi could not manure. travel. Improved business No: farmer group skills among formation to improve smallholder farmers business skills of (all) smallholder farmers planned, but not yet undertaken. (expectation is increased scale and better market linkages); training starting Q2 2021 Yes. Farmer groups Higher productivity is have been formed and not not yet seen due trained. There is a to AI challenges and expectation of more still to early in the productivity over longer production cycle. term Improved business No: The only plan is to skills among private link value-addition co- sector (value chain) operatives (branded actors (all) dried meat - existing group but not in the 33 project site) to the farmer groups; plus some back-stop support to AI service providers (NIAS) through no specific business training planned; training Q2- 2021 No. Decided to stop for Community AI Two years is too short for now. production is cancelled due to lack establishing a brand - mostly subsistence of funds. Cooperative needed strong partner oriented and co-op has was supposed to be (cooperative) but was limited capacity for supported by local not there. Coop was branding authority but this did outside site so not not eventuate. accessible due to COVID Policies for product Not sure if anything has Co-operative (Amon); standards reviewed happened and what the later planning on and improved (A)? plans are involving farmers and women’s union, amongst others Stopped , linked to outcome above Improved collective none- group formation action among (common interest smallholders and groups) starting Q2 private sector 2021 actors (m/w) for 5 common interest work together for 6 month production plan marketing of groups CIG formed collective input - activities, input, livestock (products) purchase, labour marketing plan. Linking (all) sharing. to market actors. 34 Assumptions Did the assumption hold? If not, is it a threat to achieving the outcomes? How? Relevant issues have been identified Yes Interventions can be identified and implemented in project timeframe Yes that are able to tackle the relevant issues The appropriate partners can be identified Yes Project meets the interests of local authorities and partners in the long Yes term to ensure sustainability Authorities allow meetings of groups in covid-19 times No Depended very much on COVID situation. Travel to Son La has been impeded. Some activities not done - genetics, AI, LLAFS trainings could not be done. They are postponed only, not cancelled. Or moved to virtual where possible. Consistent bundle of interventions can be identified (linked to issues in Not sure Integration of cross-thematic activities started later in project. 'Consistent' is the value chain) wrong term? Market study identifies options that relate to collective action Yes - CIGs formed COVID does not limit markets to operate and trainings to take place NO Market access was constrained by COVID. Trainings postponed. Not a threat to project (yet) Farmers are interested in the trainings and interventions YES - farmer demands were assessed by theme Concept of ‘core’ farmers includes women YES Trainings are accessible to farmers, especially women YES/NO - also for ethnic minorities. Not all trainings accessible. Activities are planned taking into account rainy season for accessibility NO Activities and training are planned to accommodate participation of Yes (mostly) women, and ethnic minorities Local authorities support the activities Yes Missing assumptions virtual platforms available for training local and partner trainers are available and capable 35 Vets and extension workers Early (signs of) Have you seen the early Was anything If different, What else is needed to achieve How did partners Are there any Are there any lessons outcomes outcomes emerge? Yes / different from what was the outcome (if any)? support or hinder unintended learned? No what you reason for the the outcomes? outcomes (positive or intended? If yes, difference? negative) that how? emerge? Improved skills and Yes - Training has As expected None Training needed to reinforce Sub Dept of Interest by women capacity among local enhanced prior basic partners' skills in participatory Husbandry of Son to become AI vets and extension knowledge in biosecurity, consultation and diagnostics La and Agricultural providers (need to workers for disease animal health, AI and pre- (happening in March/August) service center of be confirmed; also diagnostics and rational selection (breeding) - Mai Son provided included both antibiotic use, breeding, better understanding of local expertise and gender in training). AI, and feeds (all) concepts and importance. context Extension are linking farmers to forage seeds difficult to evaluate how need to include more local govt could farmers need to buy need to focus on the they have translated demonstration/practical provide free vaccines (burden) village vet rather than knowledge into skills in training rather classroom vaccine extension workers - practice - some changes sessions alone -( we only because they are close are observable; attitudes showed pictures need hands- to farmers+ the ones have certainly changed on) the farmers contact first; their skills and knowledge is low Improved skills among Yes - Skills & knowledge As expected (2 None Sub Dept of vets and extension improved in all areas, extension Husbandry of Son workers in crop- including biosecurity workers La and Agricultural livestock management, involved- as service center of nutrient flows, disease planned) Mai Son provided diagnostics and rational local expertise and antibiotic use, breeding, context AI, marketing, feeds (all) integrated training helped farmers given diary to record involve local Para-vets allowed to to address different needs all events/ consultations with professionals-need use antibiotics (no of farmers; para-vets, use vaccines, to be paid 36 antibiotics - project teams links made with regulation) - could check the diary every 2 weeks district/ provincial be overused? level to confirm Training helped to diagnosis control LSD outbreak (Mai Son) New cattle AI service Cattle AI service providers Improved Sub Dept of Two commune vets providers have the and farmers trained in knowledge and Husbandry of Son want to start-up as skills, and knowledge to pig-AI (some of whom skills on breed La and Agricultural AI service providers provide AI services to may become a pig AI selection, AI service center of farmers service provider); pig Mai Son provided semen providers have local expertise and been trained (to produce context pig semen as a input into AI) new AI workers have Communal vets need arrangement for increased number of AI? provide animal monitoring (to more coverage? health services prevent malpractices) along with AI (they think and plan better- more integrated service provision) Awareness raised Too soon to see impact Trainings taken to be clarified to be clarified by environment Training needed to among decision makers place - see blog by - Hung reinforce partners' about livestock and the post and country environment - skills in participatory environment (all) brief Hung consultation and diagnostics (happening in March/August) Not sure 37 Assumptions Did the assumption hold? If not, is it a threat to achieving the outcomes? How? Relevant issues have been identified yes Interventions can be identified and implemented in project yes-but COVID19 was a constraint to deliver timeframe that are able to tackle the relevant issues all interventions in time The appropriate partners can be identified yes -NIVR (national)+ Sub DAH in Son La (local partners), Agri services (Mai Son) Project meets the interests of local authorities and partners in yes Sustainability + scaling might be an issue - not addressed the long term to ensure sustainability Authorities allow meetings of groups in covid-19 times no some trainings for vets were not able to take place Vets and extension workers are interested in improving their yes extension workers did not attend many trainings - because of their workload skills and trainings are relevant to their work and distance (mountainous -not a threat but affected participation Partner organisations support their staff to be involved in yes - they allowed them to attend trainings field coordinators have an important role in maintaining relations with trainings" farmers Environmental and livestock department at district and mainly working with Livestock- did not engage has not affected AH/ genetics interventions (maybe feeds and fodder?) provincial level are open to dialogue with Environmental department Missing assumptions online training can/did replace f2f training vets can learn with training material and posters (online/ offline) - refresher training might be needed; learn how to carry out a fee-based service what more support will help to sustain the outcomes? 38 Partnerships and policy makers (only Oct 2021 reflections) Early (signs of) outcomes Have you seen the early Was anything different from If different, what was What else is needed How did Are there any Are there any outcomes emerge? Yes / what you intended? If yes, reason for the to achieve the partners unintended lessons learned? No how? difference? outcome (if any)? support or outcomes hinder the (positive or outcomes? negative) that emerge? Partners are aware of the Yes The level of engagement was The means of project and its activities more than expected given the engagement, directly and interested to short time of the project with local partners in collaborate on project Son la. Tu critical in activities her role to engage and inform the partners Local authorities approve Yes See above, actively involved in Checked initially if the project activities easily a number of activities and project plans were when needed consult about the outputs to appropriate and in line see what they can continue to with local authority support the local farmers with plans in the future. They put this in their action plans. Partners collaborate on Yes They supported Important to activities communication to the local maintain the Awareness raised among media. relationships at decision makers about the 3 levels: livestock and the commune, district, environment (all) province, Awareness raised among The changes are being They are interested in specific In the stakeholder Farmers and decision makers about made from the commune interventions as it fits with the meeting. The district different authority livestock and the level up, so from plans and would like to level authority levels were linked environment (all) commune to district upscale. indicated he wants to up by the project to province. The province scale to other villages to make this is now putting this in their in other the same happen commune. 39 plans. (DARD Son La made the push) Decision-makers at district Yes, AI, AH, biosecurity Policy briefs were not yet and provincial level (through demo farms) developed, at national level understand the needs for this will take a bit more time livestock development, including vaccination programs and antibiotic use Decision-makers from No, not yet. This was to Need to get more contact with There were more A lot of attention on This should have a livestock and environment improve dialogue other departments, the dept existing contacts with the project exhibition focus for if the departments at district between different of env was invited at the first the livestock from the general project continues. and provincial levels have sectors. workshop but there was no department so there public which really (missing in improved dialogue for close contact. were more helps to raise the SAPLING) environmental relationships. awareness of management livestock. Assumptions Did the assumption hold? If not, is it a threat to achieving the outcomes? How? Relevant issues have been identified Yes, short term interventions were identified, they were realistic. With exception of the connection with the environmental department. Interventions can be identified and Yes. implemented in project timeframe that are able to tackle the relevant issues The appropriate partners can be identified Some interventions were developed later based on the needs on the ground, but could work with existing partners but more deep involvement of farmers and local vets. Project meets the interests of local yes (see left) authorities and partners in the long term to ensure sustainability 40 Assumptions Did the assumption hold? If not, is it a threat to achieving the outcomes? How? Authorities allow meetings of groups in The main issue was the inability to travel of the team in There would have been more exchanges and field visits so worked with phone calls covid-19 times Hanoi (no travel last 5 months). But managed with local and photos so it is not the same. But still a lot achieved by Tu and NOMAFSI team. staff and able to run activities. Without a field coordinator with links to the technical team, it would not have happened. Partner organisations support their staff to Strong support from local partners (e.g. vet department Not so much. The grassroot level staff has limited fluctuation so this promotes be involved in trainings sent vets to help deliver training courses and participate as sustainability of the practices. trainees). Missing assumptions Strong presence on the ground of coordinator and local team to keep the linkage. 41