Impact Assessment Series IA paper 07 Managing water and land at the interface between fresh and saline environments An impact evaluation Bronwen McDonald About the CPWF Impact Assessment Series (IAS) The Impact Assessment Series (IAS) provides an outlet for the publication of articles on all aspects of monitoring, evaluation and impact assessment in the CPWF. The audiences for the Impact Assessment Series include CPWF donors who wish to see peer-reviewed impact assessments; CPWF staff who wish to learn from previous experience; and people working for other research for development programs who wish to learn from the CPWF’s approaches, both from what is working and what is not. Articles can be on the following topics: - Designing and setting up M+E systems in CPWF basins - Ex-ante impact assessment of CPWF projects and BDC programs-of-work - Early evaluation of outcomes - Lesson-learning and guidance to CPWF evaluation strategy - Ex-post impact assessment of Phase I projects Copyright © 2011, CGIAR Challenge Program on Water and Food Unless otherwise noted, you are free to may copy, duplicate or reproduce, and distribute, display, or transmit any part of this paper or portions thereof without permission, and to make translations, adaptations or other derivative works under the following conditions: ATTRIBUTION. The work must be attributed, but not in any way that suggests endorsement by CPWF or the author(s). NON-COMMERCIAL. This work may not be used for commercial purposes. SHARE ALIKE. If this work is altered, transformed, or built upon, the resulting work must be distributed only under the same or similar license to this one. Any views expressed in this publication are those of the authors. They do not necessarily represent the views of CPWF, the authors’ institutions or the financial sponsors of this publication. Citation McDonald, B. 2011. Managing water and land at the interface between fresh and saline environments - an impact evaluation. Colombo, Sri Lanka: CGIAR Challenge Program for Water and Food (CPWF). 56p. (CPWF Impact Assessment Series 07). Editing by Kingsley Kurukulasuriya Design and layout by Thomas Meadley Cover photo: © Bronwen McDonald, Gathering shrimp amongst the sea grass, Viet Nam. 2011.04.22.CPWF WP-IAS-08.draftv3 Impact Assessment Series 07 Managing water and land at the interface between fresh and saline environments An impact evaluation Managing water and land resources for sustainable livelihoods in Viet Nam and Bangladesh – evaluation of the Viet Nam component Bronwen McDonald 2011.04.22.CPWF WP-IAS-08.draftv3 Acknowledgements I wish to acknowledge the generosity and preparedness of all those who participated in this evaluation, especially the project team, the Bac Lieu People’s Committee, the Bac Lieu Department of Agriculture and Rural Development, and the producers who welcomed me to their properties. Without their input this evaluation could not have proceeded. Also, very much appreciated were all the efforts of Ms. Hong who made my visit run so smoothly. In particular, I would like to thank Drs. Tuong, Hoanh and Ni who put together a comprehensive program and along with Dr. Can provided translation assistance. Finally, I would like to thank the Challenge Program for Food and Water that provided me with the opportunity to undertake this interesting project and to Dr. Boru Douthwaite who provided guidance and support. Bronwen McDonald Warragul Victoria Australia bronwenmcdonald@hotmail.com This paper is an Outcome Evaluation of the CPWF Project: Sustaining Collective Action Linking Economic and Ecological Scales in Upper Watersheds (SCALES/PN20). CPWF DONORS Enabling poor rural people to overcome poverty Table of contents Acknowledgements ii Abbreviations and acronyms v Executive summary viii Background 1 Description of the project 2 The impact evaluation 4 Findings 6 Post-project impact pathways Model PN10 6 Progress to achieving the project goal 8 Three Impact Stories 10 1. On-farm change 11 2. Improved water management 16 3. A supportive policy environment 17 Less direct impacts 21 A summary of findings against the evaluation questions 22 Lessons for future work 26 People interviewed 28 References 29 Appendix 1. Terms of Reference 31 Annex 1. Significant Change Stories 32 Annex 2. Evaluation Proposal - Process 36 Appendix 2. Project 10: Original Impact Pathway Model 37 Appendix 3. Summary graphs on income change 38 Appendix 4. Limitations of the evaluation 40 Figures Figure 1. Administrative map of Viet Nam showing the location of Bac Lieu Province vii Figure 2. Mekong Delta from space x Figure 3. Gathering shrimp amongst the sea grass 2 Figure 4. The post-project impact pathway, Model PN10 7 Figure 5. The relationship of the three impact stories to the project goal 10 Figure 6. The causal package for producers 14 Figure 7. Mrs. Hoa’s story 15 Figure 8. The causal package for improved water management 18 Figure 9. The Bac Lieu land use map: evidence of use of project data 19 Figure 10. The causal package for the policy environment 20 Figure 11. The story of sea grass 21 CPWF Working Paper - Impact Assessment Series No. 07 iii 2011.07.30.CPWF WP-IAS-07.draftv1 Figure A3.1. Illustration of household income and income sources over time 38 as perceived by farmers in the Minh Dieu Village Figure A3.2. Illustration of household income and income sources over time 39 as perceived by farmers in the Phong Thanh Village Figure A3.3. Illustration of household income and income sources over time 39 as perceived by farmers in the Vinh Loc Village Tables Table 1. Project outputs 8 Table 2. Comparison of total profits between demonstration farms and the controls 12 iv Managing water and land at the interface between fresh and saline environments Abbreviations and acronyms BLPC Bąc Liêu People’s Committee BFRI Bangladesh Fisheries Research Institute BRRI Bangladesh Rice Research Institute CA Comprehensive Assessment of Water Management in Agriculture of CGIAR CGIAR Consultative Group on International Agricultural Research CMOCs Context-mechanism-outcome configurations CPWF Challenge Program on Water and Food CTU Can Tho University DARD (Bąc Liêu) Department of Agriculture and Rural Development DFID UK Department for International Development GIS Geographic Information System IRMC Integrated Resource Mapping Center IRRI International Rice Research Institute IWMI International Water Management Institute LGED Local Government Engineering Department (Bangladesh) RIA2 Research Institute for Aquaculture No. 2 SIWRP Southern Institute for Water Resources Planning UAF University of Agriculture and Forestry, Ho Chi Minh City VRSAP Vietnamese River Systems and Plains model WorldFish World Fish Center CPWF Working Paper - Impact Assessment Series No. 07 v 2011.07.30.CPWF WP-IAS-07.draftv1 Aerial view of land use in Viet Nam vi Managing water and land at the interface between fresh and saline environments Source: Wikipedia C H I N A 1. PHU THO Legend 2. VINH PHUC 3. THAI NGUYEN !̂ Capital City Ha Giang 4. BAC GIANG ! Cities ! Lao Chi HA CAO BANG 5. HANOI Province boundary ! GIANG LAI CHAU LAO BAC 6. BAC NINH TUYEN Rivers CAI KAN QUANG 7. HAI DUONG DIEN YEN LANG 3 BIEN BAI Thai Nguyen SON 8. HAI PHONG 1 ! Viet Tri 2 9. HUNG YEN ! 4 QUANG NINH MYANMAR SON LA HANOI5 Cam Pha 10. THAI BINH Hoa Binh !̂ 6 ! ! 11 9 7 ! HOA 8 11. HA TAY Haiphong BINH 12 10 12. HA NAM !Nam Dinh L A O P E O P L E ' S 14 13 ! D E M O C R A T I C THANH Ninh Binh 13. NAM DINH HOA Thanh Hoa 14. NINH BINH R E P U B L I C ! 15. DA NANG V I E T N A M 16. NINH THUAN NGHE AN 17. BINH DUONG Vinh ! 18. HO CHI MINH Ha Tinh 19. BA RIA VUNG VIENTIANE HA TINH ! TAU !̂ QUANG BINH S o u t h 20. TIEN GIANG C h i n a 21. DONG THAP S e a 22. VINH LONG ot Quang Tri 23. CAN THO ! QUANG TRI Hue 24. HAU GIANG ! T H A I L A N D THUA THIEN-HUE Da Nang 15 ! QUANG NAM Quang Ngai ! QUANG NGAI KON TUM BINH Play Ku DINH BANGKOK ! Qui Nhon !̂ GIA LAI ! C A M B O D I A PHU DAK LAK YEN Buon Me Thuot ! KHANH HOA DAK ! Nha Trang NONG Da Lat BINH ! ! PHNOM PENH PHUOC Cam Ranh !̂ LAM DONG 16 TAY NINH 17 DONG NAI BINH THUAN 18 Ho Chi Minh City VIET NAM LONG AN ! AN GIANG 21 20 19 Reference Map ! ! My Tho Vung Tau Rach Gia 23 ! 22 ! BEN TRE KIEN GIANG 24 TRA VINH SOC TRANG ! CAMAU BAC LIEU BA RIA - VUNG TAU 0 100 200 Map data source(s): UN Cartographic Section, Natural Earth http://ochaonline.un.org/roap Kilometers Figure 1. Administrative map of Viet Nam showing the location of Bac Lieu Province CPWF Working Paper - Impact Assessment Series No. 07 vii 2011.07.30.CPWF WP-IAS-07.draftv1 Executive summary With inputs including (i) approximately US$679,000 over 3 years, 86% from the Challenge Program on Water and Food (CPWF), (ii) support from CPWF in training in impact pathway analysis and for face- to-face meetings between the Bangladesh and Viet Nam components of this project to share experiences and lessons, (iii) an existing water management model The Bac Lieu Province in the Mekong Delta is part ready for further development, (iv) the scientific of the Cà Mau Peninsula and is an important food- knowledge of many local and international experts, growing area in Viet Nam. It has a population of (v) the experience of a wide range of partners in water 830,000 with approximately 116,000 farming families management and production systems, (vi) the active living on small parcels of land producing a range of participation of the provincial and local governments, commodities for food security and the export market. (vii) the agreement of producers to provide their These farmers and aquaculturalists1 (together called businesses as laboratories, (viii) the agreements of producers in this report) are highly dependent on other producers to provide their farms as control accessing the right quality water, fresh or saline or farms with a likely opportunity cost, and (ix) the both, at the right time to grow their crops or raise participation of producer groups who shared their their shrimp, crabs or fish. Water is delivered through experiences and insights, this project: an extensive network of canals and the intrusion of saline water into the area can be controlled on the • produced an improved Vietnamese River southeastern side through the operation of sluice gates, Systems and Plains (VRSAP) model that now a major investment in infrastructure undertaken by contributes to improved sluice gate operations the Central Government of Viet Nam. to better meet producers’ water needs; In the early 2000s, there were conflicts over water • used the model and other data to contribute use as shrimp aquaculturalists in particular began to to the development of the Bac Lieu People’s see their supply of saline water being compromised. Committee’s Land Use Policy, which Also, many producers were living in poverty and in recognizes the benefits of diversification and the some areas inappropriate land use was leading to role of saline water in farming; and unsustainable futures. This project completed in 2007 • developed and evaluated a successful and built on the work of two preceding projects2 participatory extension approach that assists is helping change that situation. What’s more, this producers select appropriate technologies (and impact will continue to grow as the outputs are more reject others) based on on-farm demonstration widely applied. and experimentation. 1 The term ‘aquaculturalist’ is used to differentiate people who On average, the demonstration site farms involved ‘farm’ fish from those who catch fish in the wild. 2 The two projects are Accelerating Poverty Elimination in the project made approximately US$250/ha/ through Sustainable Resource Management funded by DFID year more than the controls (extrapolated from Ni and Increasing Water Productivity by Managing Land-water Interface: Effective Water Control for Solving Conflicts et al. 2007). The producers interviewed believed the among Agriculture-Fisheries-Aquaculture in Coastal Zones funded by CGIAR. financial gain made a significant difference to their viii Managing water and land at the interface between fresh and saline environments lives but they were keen to improve further. The To support the producers the Bac Lieu People’s extension approach used in this project is now being Committee is encouraging banks to provide credit to applied by the Bac Lieu Department of Agriculture producers who lack collateral but who have adopted and Rural Development (DARD) and the number the practices promoted in this project, especially the of sites has gone from eight during the project to 80 application of appropriate technologies. new sites per year for the last 2 years. A DARD survey suggests that of the 10,000 producers who have visited It is not known how much on-farm change has the sites and participate in the discussions 50-70% happened as a result of technology diffusion processes adopt improved technologies. This approach is based stemming from the project. While the DARD can on group processes and the Vietnaamese saying “to see track adoption of technologies it is the process of how once is better then to hear 100 times.” the technologies were selected or rejected that is just as important. Also, the improvement in sluice gate As a consequence of these three main results: operations would have impacted on all producers, not just those on the demonstration sites. Can et • A contribution is being made to poverty al. (2010) and Khiem and Hossain (2007) suggest reduction in the area, which has been incomes have increased as a result of sluice gate demonstrated in two socioeconomic studies operations while the Bac Lieu People’s Committee (Can et al. 2010; Khiem and Hossain 2007) stated that incomes in Bac Lieu are increasing faster and confirmed in interviews with producers. than in neighboring provinces. Incomes in Bac Lieu • More sustainable farming systems are beginning have reached the average for the whole Mekong Delta to emerge as evidenced by the annual survey Region rather than lagging behind as before. conducted by DARD. There is still work to be done including the need • There has been a reduction in the number for more research at the saline-soil interface. A big of conflicts over water resources as reported extension effort is needed to reach more producers. by government officials interviewed in this The potential of the BayFish model, a tool that uses evaluation. VRSAP model outputs in optimizing production • The Southern Institute for Water Resources under different sluice gate operations, still needs to be Planning (SIWRP) has used the improved realized. High on the agenda for the Bac Lieu People’s VRSAP model developed in this project for the Committee is the need to strengthen its alliances whole of the Cà Mau Peninsular. with neighboring provinces regarding water use and it • Neighboring provinces are beginning to use the would also like to see a similar project being applied to on-farm technologies as evidenced from the its coastal non-project area outside the influence of the queries that the project partners have received. sluice gates. • New agribusinesses are beginning to emerge as The success of this project, however, can be either a direct result of the project or possibly as attributed to its positive interaction with policy a flow-on effect. development, integrating good science with the tacit knowledge of producers, having the right players at the CPWF Working Paper - Impact Assessment Series No. 07 ix 2011.07.30.CPWF WP-IAS-07.draftv1 table at the right time and having built-in evaluative the state of the Vietnamese economy in general, the processes so partners could monitor progress. policies of the Central Government, climate change and rising sea levels, and the actions of Bac Lieu’s External factors will continue to influence the neighbors and the upstream users of the waters of the ultimate impact and these include commodity prices, Mekong River. Figure 2. Mekong Delta from space x Managing water and land at the interface between fresh and saline environments Source: NASA Background The Bac Lieu Province in the Mekong Delta is being compromised. This reached a flash point in part of the Cà Mau Peninsula and is an important 2001 when a group of shrimp aquaculturalists broke food-growing area in Viet Nam. It has a population a dam in order to access their water requirements. At of 830,000 with approximately 116,000 farming the time the price of rice was low while shrimp was families living on small parcels of land producing highly profitable, although the latter was vulnerable a range of commodities for food security and for to disease. supplying an export market. These farms are highly dependent on being able to access the right quality of The Bac Lieu People’s Committee and a project water, either fresh or saline or both at the right time. team who were working in Bac Lieu Province The water is delivered through a complex network under a DFID-funded project (2000-2003)31 of canals connecting the Mekong River to the China met to determine how to address these issues Sea and the Gulf of Thailand. While rice and shrimp under that project and to do so on a scientific are the main commodities others include fish (both basis. Specifically, project activities were adjusted freshwater and saline species), crabs and upland crops to address (i) improved water management with such as vegetables, maize and fruit trees. the assumption that saline water should now be considered a resource, not a problem, (ii) creation of In 1994, the Vietnamese Central Government the policies to support diversification of agriculture embarked on a project for building sluice gates to and aquaculture through land use zoning, and (iii) protect large parts of the Delta from intrusion of development of an extension package. The DFID saline water and to convert it into land fit for rice project concluded there were benefits to be gained intensification. An area of approximately 90,000 from implementing a flexible approach to water ha, in the Bac Lieu Province was to be converted resources use allowing diversification in agriculture to freshwater systems even though two-thirds of and aquaculture. This created the basis for the CPWF this land contained acid sulphate soils, a factor that project, the subject of this evaluation. Moreover, to impeded agricultural production. This infrastructure ensure a smooth transition between the two projects project was not completed and only the sluices and help maintain the support of partners, a one in the southeast were built but these impacted year project42was funded by the Comprehensive significantly on the Bac Lieu Province. The area Assessment Program (CA) of Consultative Group in converted to freshwater production gradually Agricultural Research (CGIAR) from mid-2003 to increased until 2000 as did rice production, although mid-2004. the latter resulted in environmental degradation and 31 livelihood deterioration for poorer people (IRRI Accelerating Poverty Elimination through Sustainable Resource Management. 2004). Conflict emerged, especially for the shrimp 42 Increasing Water Productivity by Managing Land-water Interface: aquaculturalists who saw their supply of saline water Effective Water Control for Solving Conflicts among Agriculture- Fisheries-Aquaculture in Coastal Zones. CPWF Working Paper - Impact Assessment Series No. 07 1 2011.07.30.CPWF WP-IAS-07.draftv1 Under the revised land use zoning, there are three Description of the main water resources zones in Bac Lieu. The east is predominantly freshwater and supports rice and project other freshwater commodities. The west is saline and supports shrimp and other saline commodities. The middle area is basically a freshwater zone in the wet season and saline in the dry. The espoused high-level goal for this 3-year project was to increase land and water productivity for improved food security and livelihoods in a manner that was environmentally sustainable and socially acceptable to various resource users (IRRI 2004). It was designed around five key objectives that were described in the project proposal as follows: 1. To enhance understanding of livelihood changes resulting from regional resource management and farm-level technological interventions. 2. To assess the impacts of agricultural and aquacultural land and water uses on water quality, aquatic biodiversity, and inland fisheries. 3. To develop ecologically friendly and socially acceptable techniques for rice and rice-aquaculture production systems for domains with different soil and water quality characteristics. 4. To develop decision-making tools and an institutional framework for integrated multipurpose management of a dual fresh- and brackish-water regime to meet the needs of diverse water users, without an adverse impact on users and the environment outside. 5. To enhance human resources capacity and develop recommendations for resources Figure 3. Gathering shrimp amongst the sea grass management at the farm and regional level. 2 Managing water and land at the interface between fresh and saline environments Source: Bronwen McDonald A project team was formed that included people, (ix) the participation of producer groups who or at least their organizations, who were expected shared their experiences and insights. to be the primary users of the project outputs. This team then designed in-built strategies for ensuring the This Viet Nam project was part of a larger project, uptake of the outputs. namely the Managing Water and Land Resources for Sustainable Livelihoods at the Interface Between Fresh Inputs included: and Saline Water Environments in Viet Nam and Bangladesh (CP WF#10) (IRRI 2004) project that (i) approximately US$679,000 over 3 years, embraced the Gangetic Delta as well. This facilitated 86% from the CPWF, an exchange of knowledge and experiences in (ii) support from CPWF in training in impact managing land and water between the two deltas. pathway analysis and for face-to-face meetings between the Bangladesh and Viet In retrospect, there appear to be several implicit Nam components of this project to share and explicit principles which the project followed that experiences and lessons, contributed significantly to its results in Viet Nam. (iii) an existing water management model ready First, all the necessary stakeholders were at the for further development, table including provincial and local governments, (iv) the scientific knowledge of many local and universities and research institutes, extension officers international experts, and producers. They shared a vision of what had to (v) the experience of a wide range of partners in be done. Second, there was a commitment to strong water management and production systems, scientific evidence-based decision making with activities including evaluative approaches. Third, (vi) the active participation of the provincial and when working with producers, science and technology local government, needed to be blended with producers’ local knowledge (vii) the agreements of producers to provide and wisdom and to do so in a way that results could their farms as laboratories, be seen – not just reported from afar. Last, principles (viii) the agreement of other producers to provide of participatory processes were followed at all levels to their farms as control farms with a likely promote ownership and eventual adoption. opportunity cost, and CPWF Working Paper - Impact Assessment Series No. 07 3 2011.07.30.CPWF WP-IAS-07.draftv1 The impact evaluation Further these questions were to be addressed with reference to the existing hypothesized impact pathway and the stories of the most significant change51that had been gathered for discussion at an international CPWF forum. These documents provided a base point from which to think about the achievements of the program and were used in revising the impact Overall, this evaluation was intended to identify what pathway. The stories, in particular, provided some contribution the project had made to reducing poverty sense-making of the global statements used in the and food insecurity in the Mekong Delta as a result impact pathway model. of an investment by the CPWF. It was guided by the following evaluation questions, focusing on adoption In summary, the evaluation was required to focus and impact. on adoption and impact. 1. What are the changes? A typical definition of impact refers to the long- 2. What is the scope and extent of these changes term consequences62of an intervention but in this on the ground? instance it was too early for these consequences to 3. What were the mechanisms that bought about have fully manifested themselves. There are also many these changes? What contribution did the other contributing factors outside of the control project make to them? Specifically, what role of this project that will impact on the final results, did research play? for example, commodity prices, the state of the Vietnamese economy in general, the policies of the 4. What is it about the context that is driving the Central Government, climate change and rising sea changes? Where could the changes eventually levels, and the actions of Bac Lieu’s neighbors and spread? upstream users of the Mekong River water. 5. What are the benefits and costs of the changes (quantify them as far as possible) both now, and For these reasons it was decided that identifying potentially in the future? What did the project the actual impact pathway would become the invest? basis of this evaluation. It draws on some of the 6. What was the contribution of CPWF in principles described by Mayne (2008) in his work on achieving the changes (to be assessed by contribution analysis for identifying likely impact projecting what would have happened without where attribution is problematic. CPWF funding and comparing it with what did happen)? 51 The stories are included in Appendix 1. 62 7. What are the international public goods that According to the Development Assistance Committee of the Organisation for Economic Cooperation and Development, the project has generated that are related to the (Development Assistance Committee 2006), Impact refers to the positive and negative, primary and secondary long-term changes? effects produced by a development intervention, directly or indirectly, intended or unintended. These effects can 8. What were the differences between the original be economic, socio-cultural, institutional, environmental, and final impact pathway? technological or of other types. These are the effects that any impact evaluation endeavors to identify and analyze. This definition is being applied to this evaluation. 4 Managing water and land at the interface between fresh and saline environments When discussing the terms of reference with the (ORID) model was used (Stanfield 1997) as a commissioner of the evaluation, it was decided that means of refocusing people on that work: this evaluation, wherever possible, should take the a. Objective Level – Gaining an overview of form of a “realistic evaluation” as defined by Pawson the project that was presented by the Project and Tilley (1997). By agreeing to test how realistic Leader. evaluation could be applied in this evaluation, which b. Reflective Level - Obtaining an overall differentiated it from the other CPWF-funded impression of impact using photolanguage evaluations of other deltas, the type of data sought (Catholic Education Office 1986) in a from respondents tended to hinge around the key “before and after” scenario. realistic evaluation question of what worked where for c. Interpretative Level – Revisiting the whom, why and why not. Vision of Success statement to assess what actually happened compared to expected The overall approach to implementing this results. evaluation, therefore, was to: d. Decisional Level - Comparing progress against the original impact pathway model. 1. Base it around the impact pathway drawing on The original is given in Appendix 2. the lessons from the contribution analysis. 3. Interviews with project team members and key 2. Undertake a “realistic evaluation” stakeholders. interpretation of the data. 4. Discussion with a group of producers at three 3. Report in ways that support the above demonstration sites: two points, the main audiences being the a. One farm with a freshwater system. commissioner of the evaluation, the project b. One farm with a fresh and saline water team and the key stakeholders. system. c. One farm with a saline water system. The Terms of Reference for this evaluation are 5. A second workshop with the project team to given in Appendix 1. The constraints of available time discuss any remaining issues and finalize an agreed and resources imposed on this evaluation through impact pathway model based on what actually necessity meant that the data collection had to rely happened. on existing documents and the explicit and tacit knowledge of the project team and the project’s 6. Report preparation, including a draft report that stakeholders. As a result, the following steps were went to both the commissioner of the evaluation undertaken: and the project leader for comment before finalization. The main audience for the report is 1. Review of key documents. the funders, the project team itself and other key stakeholders. 2. Workshop with the project team to assess what outcomes and impacts had occurred Note on comparisons: Although this project was against what was originally planned. Given implemented in two deltas (Gangetic and Mekong) this impact evaluation did not commission an evaluation of the that the project had been finished for a year an Gangetic Delta and, therefore, no comparison between the Objective-Reflective-Interpretive-Decisional two delta regions was possible. Comments on the limitations of the evaluation are given in Appendix 4. CPWF Working Paper - Impact Assessment Series No. 07 5 2011.07.30.CPWF WP-IAS-07.draftv1 Findings 3. Development of a supporting policy environment (shown in orange in Figure 1) where the main actors are the policymakers and leaders of the Bac Lieu People’s Committee ensuring institutional frameworks support and encourage the change. Post-project impact pathways Model PN10 There are some major diagrammatic differences between the new version and the original. First, the It is too early for the full impact of this project to post-project diagram describes who was a partner and be realized – there is still a big extension job ahead. who was an initial user of project outputs. In some In order to assess progress towards that impact instances their names appears twice. In the original this evaluation elected to take an approach based all stakeholders were grouped together because of the on revisiting the hypothesized impact pathway integrated nature of the project, but when it came to formulated for this project,71a copy of which is given assessing impact, it helped to disaggregate this bundle. in Appendix 2. This pathway connects outputs and Final beneficiaries were also identified. stakeholder actions with a series of intermediary outcomes that contribute to the final impact. For this While the picture makes a clear-cut delineation evaluation, it was used as a framework for interpreting between who uses what output, discussions with the evidence and was, therefore, modified in the process. project team revealed that other people were using the outputs as well to inform and build their own The revised impact pathway, developed with the knowledge base or support their work. For example, project team, confirmed three strands to impact or DARD has the main responsibility for future “three impact stories.” These are interdependent but extension efforts in Bac Lieu and therefore it became have emerged as a result of different people picking up the primary user of that output. Members of the Can different outputs. The three impact stories are about Tho University (CTU) staff who participated in and the following: drove the research on extension, however, now use the information gained as part of their teaching program. 1. On-farm change (shown in green in Figure 1) where the main actors are the producers of Bac A second feature is that the post-project diagram Lieu, the ultimate beneficiaries of this project includes evidence of less-direct impacts shown as gray and not just those directly associated with the boxes in Figure 1. In these instances there was less- project demonstration sites. direct project intervention. For example, producers 2. Improvement in water management (shown outside of Bac Lieu were beginning to be aware of in blue in Figure 1) where the main actors are the opportunities existing in Bac Lieu and taking the SIWRP and DARD Water Operations appropriate action. The purpose of this differentiation providing a critical resource to producers. is to illustrate where the main planned impacts are occurring and where impact was spreading further. 71 The impact pathway was developed in February 2006 by two project implementers – Drs. Chu Thai Hoanh and Nguyen Duy Can – at a CPWF Mekong Impact Pathway and Most Significant Change workshop The pathway was then refined by the team of the CPWF Finally, inputs were included in the revised version BFP Impact Assessment Project to complete the picture. 6 Managing water and land at the interface between fresh and saline environments Post-Project Impact Pathways Model PN10 INCREASED LAND & WATER  PRODUCTIVITY FOR IMPROVED FOOD  SECURITY AND LIVELIHOOD  FARMERS IN  NEIGHBOURING  FARMERS IN  REDUCTION IN RISK OF  REDUCED CONFLICTS IN  PROVINCES  BAC LIEU  ENVIRONMENTAL DAMAGE &  WATER REQUIREMENTS  BENEFIT  BECOME RICHER  UNSUSTAINABLE PRODUCTION  BECAUSE OF  IN ACID SOIL AREAS  ADOPTION BY  DIFFUSION  EMERGENCE OF  PEOPLE PRODUCE RICE  NEW  IN THE EAST AND  BUSINESSES IN  PRODUCTS OF SALINE  THE AREA THAT  WATER IN THE WEST  VALUE ADD TO  THROUGHOUT IN THE  & SUPPORT  YEAR  FARMING  ACCESS TO  CAPITAL  THROUGH  INFLUENCE OF THE  FARMERS /AQUACULTURALISTS IMPROVE  BAC LIEU PEOPLE’S  THEIR PRODUCTION SYSTEMS &  COMMITTEE  CULTIVATION TECHNIQUES MOVING AWAY  FROM MONCULTURE  DARD IMPLEMENTATION OF A  LAND USE POLCIY THAT DEFINES  WATER ALLIANCE  BROADER EXTENSION  ZONES FOR FARMING BASED ON  BETWEEN BAC LIEU, CA  PROGRAM BASED ON THE  WATER‐SOIL CHARACTERISTICS BUT  MAU & SOC TRANG  LESSONS FROM THE PROJECT  ALLOWS SOME FLEXIBILITY IN  FORMED TO DEVELOP A  COMMODITY CHOICE  REGIONAL APPROACH  CONTRIBUTION  TO THE INTERNATIONAL  DARD EXTENSION SERVICE  DARD WATER OPERATIONS  BAC LIEU PEOPLE’S COMMITTEE  BODY OF SCIENTIFIC  SIWRM  KNOWLEDGE  OUTPUT 5 (DISSEMINATION STRATEGIES)  PARTNERS, EVALUATORS, &  PARICIPATORY EXTENSION AND EXTENSION MATERIALS   EXTERNAL (TO BAC LIEU)  INFORMATION FOR LAND USE DECISIONS,   SCIENTISTS & POLICY MAKERS  RECOMMENDATIONS FOR REGIONAL LEVEL WATER MANAGEMENT MATERIALS,    2 PHDS, DELTA CONFERENCES, PAPERS, BOOK  OUTPUT 3:  ECOLOGICALLY FRIEDNLY  OUTPUT 4:  OUTPUT 2:  AND SOCIALLY ACCEPTABLE  IMPROVED VRSAP MODEL  INFORMATION ON WATER  OUTPUT 1:  BIODIVERSITY  SOCIOECONOMIC  FARMING SYSTEMS /  AND OTHER SUPPORTING  QUALITY, AQUATIC  TECHNIQUES  INFORMATION & INLAND FISHERS  STUDY  WATER QUALITY MONITORING PROJECT ACTIVITIES &   PROJECT TEAM  PROJECT COORDINATION TO  LEARNING CYCLE   PRODUCE 5 OUTPUTS  PARTNERS: IRRI, WorldFish, IWMI,  BLPC, CTU, UAF, SIWRM, RIA2,  IRMC, FARMERS AT  DEMONSTRATION SITES INPUTS:  $679,000 ON VIETNAMESE OUTPUTS, EXISTING VRSAP MODEL, SCIENTIFIC KNOWLEDGE, EXPERIENCE OF  PARTNERS, FARMERS WHO PROVIDED DEMONSTRATION FARMS, FARMERS WHO PROVIDED CONTROL FARMS,  PRODUCER GROUPS, OUTPUTS FROM PRECEDING PROJECTS LOGISTIC SUPPORT Figure 1. The post‐project impact pathway. 18    Figure 4. The post-project impact pathway, Model PN10 CPWF Working Paper - Impact Assessment Series No. 07 7 2011.07.30.CPWF WP-IAS-07.draftv1 A number of lessons emerged from using the Table 1. Project outputs impact pathway as a framework: Number Output 1. Building an impact pathway model needs to be Output 1 Credible information and a team sport played during project design and knowledge explaining the impact of regional- and farm-level resources revisited regularly. Not many people on the management on the livelihood of project team were familiar enough with it to use producers in the study areas. it during the life of the project. “If only we had Output 2 GIS-based data that show used this at critical stages of the project it could spatiotemporal variation of canal water quality, aquatic biodiversity, have helped us” was one comment overheard. and fisheries, and their dependence 2. People struggled with what was the original on soil type and agricultural and aquacultural resources use. intent behind some of the statements both Output 3 Ecologically friendly and socially in the Vision of Success statement and acceptable techniques for each the original impact model. Some of the of the three farming systems meanings had been lost and statements had (rice-based, aquaculture and rice- aquaculture). become vague or ambiguous. The advice Output 4 Decision-support tools and an given was that this sort of work requires institutional framework developed that any summarized statements should for integrated multipurpose be accompanied by a few sense-making management of a dual freshwater and brackish-water regime to meet paragraphs describing what is meant. the needs of diverse water users 3. The power of the model is in focusing people and the environment. on what needs to be done and seeing the bigger Output 5 Training materials and recommendations for resource picture and how things fit together. This came management at different through, and typical comments were: levels (producers, all levels of government, researchers). • “Now I can see where this (a specific component) fits in!” • “I see there is still a big job to do in order to Progress to achieving the project goal realize the potential of the project.” The project goal defined in the project proposal (IRRI The post-project diagram follows where the 2004) was to increase land and water productivity for names given to the outputs have been abbreviated improved food security and livelihoods in a manner for the convenience of drawing. Full descriptions of that was environmentally sustainable and socially the outputs are given in the project proposal (IRRI acceptable to various resource users at two coastal sites, 2004) and accessible via the web. In summary these one in the Mekong River Delta (Viet Nam) and other are outlined in Table 1: in the Gangetic Delta (Bangladesh). In turn, it was hypothesized that this would contribute to the greater These outputs were designed so that outputs 1-4 goal of the CPWF of reduced poverty and insecurity feed into output 5. in tidal regions (Douthwaite and Alvarez 2006). 8 Managing water and land at the interface between fresh and saline environments The Bac Lieu People’s Committee reported a When Can et al. (2010) where in the process significant decline in poverty from 40 to 8-9% over of collecting income data farmers were asked recent years and now the average income in the to identify what contributed to their improved Bac Lieu Province is the same as the average for livelihood. In the freshwater area the factors the region and not lagging behind as before. The mentioned were suitable technologies, increased People’s Committee was adamant that the project irrigation capacity and easy access to loans. More had contributed significantly to this. The members of specifically farmers mentioned the improvement of the project team, though, were more modest in their availability of freshwater and being able to include claims indicating there was still a big extension and an extra rice crop per year. DARD and this project technology transfer job ahead. were mentioned as being influential. A more detailed picture of change in the Bac Producers in the freshwater-saline zone and the Lieu Province over the last 10 years up to the end saline-only zone specified having alternative land uses, of the project was provided by Can et al. (2010) suitable technologies in aquaculture and again ease who undertook a socioeconomic study continuing of obtaining a loan as important factors in improving on the work of a previous DFID project. Can et al. their income As with the first group, the role of (2010) produced three sets of graphs, one set from water management and the DARD extension service each of three general production areas (rice, rice and along with this project were mentioned as important aquaculture, aquaculture) showing household income contributors to the outcome. Improved access to in 2006 as a percentage of household income in 1996. credit was also mentioned, which was being addressed In each set there was one graph for the “better-off and by the Bac Lieu People’s Committee outside of the medium” group and another for the “poor” group. context of this project but which was part of the supportive environment that worked symbiotically The income increase pattern for both groups in with the project. the freshwater zone was approximately the same as it was for people in the predominantly aquaculture area. In terms of environmental and economic But this is not the case for the “recent intervention sustainability much of the evidence is descriptive but zone” where large areas have acid soils, and endeavors well-founded in science. to convert it to a rice production area have resulted in poor yields. Here the poor suffered a decline in relative • One of the important findings in this project is income but now it is beginning to increase again. A that the levels of acidic pollution have dropped copy of these graphs is given in Appendix 3. over the life of the project (Tuong pers. comm.) and that, in turn, would be contributing to Khiem and Hossain (2007) also noted this improved productivity. gradual rise in income since 2000. Neither Can et • The project team and the extension service al. (2010) nor Khiem and Hossain (2010) make any promoted a theme of “three reductions claim as to how much the project contributed in the (volume of seed used, pesticides, fertilizers)” last few years of their respective studies, but these for three increases (yield, quality, income) studies do confirm the trend referred to by the Bac for rice production. Producers were quick to Lieu People’s Committee. say they believed this approach improved the CPWF Working Paper - Impact Assessment Series No. 07 9 2011.07.30.CPWF WP-IAS-07.draftv1 environment. The adoption of this approach is The first story is about voluntary behavioral being monitored by DARD. change, the second is about changes in infrastructural • The lesson of less-intensive shrimp breeding in management and the third is an enabler. The latter two order to promote sustainability was picked up are inputs into the first. from the Bangladesh component of the project where high-quality shrimp has been bred for The following diagram is a simplified version of decades with fewer instances of disease. how these three fit together. • Further reduction in the incidence of disease resulted from converting to only using one PROJECT GOAL intake of water for a shrimp pond compared to Reduction in poverty and increased the practices of using multiple water changes. food security in the Mekong Delta The risk of using contaminated water was reduced. • Growing sea grass in shrimp ponds has resulted in improvement in the environmental quality of ponds and their sustainability. • Moving from monoculture to polyculture in Impact story 1 both agriculture and aquaculture has improved On-farm change in economic sustainability. technologies As mentioned previously, in order to have a more robust assessment of environmental and economic Impact Story 3 sustainability, as Gowing et al, (2006) has suggested, a Policy longer-term impact evaluation is required. enabling environment Three impact stories Impact Story 2 The final impact pathway model showed three Improved water supply to interdependent impact stories, as referred to farms previously. These were about the following: 1. On-farm change, the change that producers have or are still to make to improve their Figure 5. The relationship of the three impact livelihoods. stories to the project goal 2. Improved water supply to producers to provide water resources to support on-farm change. What follows is a description and extent of these 3. A policy environment that supports the changes, and the contribution of each of these to preceding two impact stories. project outcomes. 10 Managing water and land at the interface between fresh and saline environments 1. On-farm change • Farms in the freshwater areas growing rice, possibly up to three crops per year, upland This impact story is about producers changing crops and freshwater fish increased their profit the way they go about their business to improve per hectare per year ranging from 30 to US$172 their livelihoods. In summary, this project selected higher than in the control farms. a demonstration site extension method that • Farms that had freshwater in the wet was eventually tested at eight sites representing season and saline water in the dry season, different ecological conditions. The features of this and therefore had the capacity to grow method included: both shrimp (and other saline aquaculture products) and rice had profit changes ranging • The identification of a producer who was from -274 to US$804 per hectare per year willing to let his or her farm be used as a when compared to the control farms. At demonstration farm to experiment with new or one site where negative returns were made revised technologies. compared to the controls the technological • Five producers at each demonstration site who changes were discussed, and reasons why these volunteered to lend their farms as controls so had not worked were explained. that comparisons could be made. • Farms in the saline water system growing • Group decision making about which only saline aquaculture products experienced technologies to trial that intuitively took changes in returns ranging from -91 to account of local knowledge. US$821 when compared to the controls. Again the site that experienced a negative • Group assessment of the results. return worked out why. Essentially, each site was a self-contained The following Table 2 provides more details on the experiment that blended science with local knowledge. findings on income at the demonstration sites. Based Support and technical advice were provided by the on this work, Ni et al. (2010) concluded the following: scientists from CTU and the extension service of DARD. In order to facilitate broader adoption, these • Farms in the shrimp-raising area are better-off if sites were visited by producers outside the hamlets in they diversify and introduce other species into the study. the system. • Farms need to identify a “keystone” crop, one In order to make a judgment about the overall that gives some certainty of return rather than merit and worth of this extension approach a self- focusing on more profitable but riskier crops. evaluation was undertaken by the project team that tracked the progress of the demonstration farms, what • Some further refinement of technologies for specific sites is still needed. they did and how they compared to their control farms. This evaluation, currently being published (Ni • Improved irrigation systems were needed in et al. 2007), showed the following: some areas. CPWF Working Paper - Impact Assessment Series No. 07 11 2011.07.30.CPWF WP-IAS-07.draftv1 Table 2. Comparison of total profits between demonstration farms and the controls (Ni et al. 2007) Farms Cropping system in (i) and (ii) A=Total profit of B=Total profit of Difference of shows the rotation of crops on the demonstration the controls A-B farms farms (US$/ha/year) (US$/ha/year) (US$/ha/year) 1 (i) Rice and upland crops 829 799 30 (ii) Rice only 2 (i) Rice 1,026 854 172 (ii) Rice and fish 4 (i) Shrimp 2,136 1,332 804 (ii) Rice and fish 6 (i) Shrimp -7 267 -274 (ii) Rice and fish 5 (i) Shrimp 1,314 887 427 (ii) Rice and fish 8 (i) Shrimp and crab 1,352 531 821 (ii) Fish 9 (i) Shrimp and crab 1,075 1,166 -91 (ii) Fish 10 (i) Shrimp and crab 416 326 90 (ii) Fish • The selected techniques at each demonstration The extension methodology worked and had been site can be up-scaled in the whole Land substantiated. As a result, since the project ceased Use Zone unit with appropriate suggested in 2007, the DARD has extended the number of technologies, provided other factors at a demonstration sites in Bac Lieu (80 per year for 2007 regional level such as transport, storage and and 2008). An annual survey undertaken by DARD processing facilities, and market price control suggests that 50-70% of the 10,000 producers per year are improved accordingly. who are involved in some capacity, either through visiting the pilot sites or participating in discussions, The producer stories related to the author during a now adopt at least one improved practice. field trip to Bac Lieu confirmed the above. There were several recurrent themes including one that, seeing the One question raised by some of the interviewees results of a technology change on-farm, is much more powerful than attending training. The Vietnamese was whether or not the producers in the original pilot saying “to see once is better than to hear 100 times” were special in some way and that the future sites seemed to be the catch cry. Second, all the producer may not have the benefit of these “leaders.” Second, groups expressed some form of empowerment from a significant legacy of this work may be in the groups this process - it made decisions easier, better and less that are formed, and some wondered whether these stressful. Third, there seemed to be agreement that the were strong enough to stand the test of time. Should group processes lead to greater cohesion in hamlets there be some resource dedicated to promoting and offered the potential for collaborative action. and supporting group work beyond the life of a 12 Managing water and land at the interface between fresh and saline environments demonstration site? DARD with its annual survey and “community of support” component was developed monitoring adoption has the capacity to track progress by the Bac Lieu People’s Committee acting in a and make decisions accordingly. number of different ways – by developing a land use policy, by improving water management, by providing DARD acknowledged there is still much to do extension services including promotion through the in order to realize the potential of the project given mass media, by encouraging banks to provide credit, by there are 116,000 farming families in Bac Lieu. The encouraging the private sector to both provide inputs unknown factor is how far the lessons from the and buy farm products for processing and export demonstration sites are spreading by technology outside the province, and so on. diffusion processes, especially the lessons about how to select the most appropriate technologies or varieties This summary provides an explanation at one level for specific situations. but it almost assumes an average experience. The work of realist evaluators like Pawson and Tilley requires One of the requirements of this evaluation is to a deeper analysis of what works for whom, in what try to identify mechanisms that triggered change in circumstance, why and why not. accordance with the principles of realistic evaluation. In order to address this, the next few paragraphs Their approach to establishing causal packages endeavor to analyze what the underlying implicit provides explanation when random assignment or theory of change is and then interpret this following regression analysis is not feasible, as is the case here. the model proposed by Pawson and Tilley (1997) who developed the concept of context-mechanisms- At the basis of their thinking is the concept that outcome-configurations (CMOCs). causality results from mechanisms triggered by an intervention that interacts with contextual factors to Many programs reliant on behavioral change in produce a variety of outcomes. order to achieve a goal develop a specific theory of change. One model proposed by the Cancer Council Figure 6 provides a brief outline of the causal in Victoria, Australia, hypothesized that if you are package for on-farm change that reflects the analysis of encouraging people to voluntarily change you need to data in a realistic evaluation framework. provide an educational opportunity, a peer-support opportunity, one-to-one support and a community To get a sense of what it feels like for a producer supportive of change (Boland pers. comm.). This one must read a translation of Mrs. Hoa’s story type of model has been used in agricultural extension given in Figure 7. She is a shrimp/rice producer who (Boomsma et al. 1996) to develop or explain project participated at the interview with the farmer group led outputs and it seems to have some resonance here. by Mr. Le Van Dang. It is printed with her permission and illustrates how context-mechanisms-outcome In this project educational opportunities were configurations are tightly intertwined. provided through extension, peer-support was provided through the group processes, while one- Given that this is a translation, it has not been to-one advice would have been provided answering edited thoroughly so that it might retain its flavor of a producers’ questions during site visits. The story in the first person. CPWF Working Paper - Impact Assessment Series No. 07 13 2011.07.30.CPWF WP-IAS-07.draftv1 Contexts are evidenced by the descriptions of the existing situations articulated in many papers (IRRI 2004; Can et al. 2010; Khiem and Hossain 2010; Ni et al. 2010) and from the stories told by producers. These included the following: • Poverty or low incomes and a desire to improve living standards. • Conflict over water use. • Policy environment that supported rice monoculture. • Advisers from DARD and CTU who could “help” bring new ideas to the farms. Activities at the farm-level as evidenced by project reports and interviews with the project team included: • Development of a participatory extension process based around hamlets where new or renovated technologies were experimented and evaluated by the local producers. • Provision of water resources more suited to local land use requirements. • Training at the demonstration sites on assessing water quality and development of other skills. Mechanisms as evidenced in the stories from producers interviewed in this evaluation. While there is considerable debate around what constitutes a mechanism this paper is taking the position that mechanism are the ways in which the activities and resources of the project influence reasoning that subsequently results in practice change. and included: • “To see one time is better than hear 100 times” (Vietnamese saying), the “seeing is believing” mechanism. • Sharing experiences makes it easier for me to make decisions, the “if we all agree then we must be onto something” mechanism. • “I trust the advice of specialists, especially when I can participate in the evaluation,” the “having input from scientists and experts gives me confidence” mechanism. Outcomes as evidenced from the interviews and the socioeconomic data include: • Improved livelihoods. • More harmony. • Community cohesion. • Improvement in the environment. Characteristics of the project that contributed to success • Allowing producers to make decisions for themselves and not being didactic. • Sound application of participatory processes that allowed for injection of new knowledge. • Personal passion and vision of the team working with the demonstration groups. (Michael Patton (Pers. Comm.), an eminent American evaluator, often talks about the influence of individuals on project performance, which is difficult to quantify but is often critical to success. This evaluation wishes to acknowledge the dedication and passion of the project team.) • DARD extension staff, who would have responsibility for delivery once the project is completed, participating in the on-farm research and development of the extension process. Challenges for the future • Part of the success of the demonstration sites was the people who stepped up to be leaders. Can these leaders be identified, mobilized and supported in a scaling-up scenario? • What is the future of the groups at the demonstration sites? Can they survive without further intervention? • How well will scaling up the extension methodology to a broader base work? • What will be the impact of emerging contextual factors yet to be defined on these groups and subsequent practice change? Figure 6. The causal package for producers 14 Managing water and land at the interface between fresh and saline environments Mrs. Hoa’s Story We are lucky to be a demonstration site. It brings us new technology, including shrimp and fish. Now local people can observe and talk about the technologies then adapt . Incomes have increased and compared to another area we are quite lucky. Technicians advise us to buy good shrimp varieties and can help solve problems through training. For example, pre-program some farmers tried to raise shrimp by buying post-larva then release them immediately. They didn’t know the survival rate. The project introduced them to make a small area to raise shrimp in a nursery. When these are a little bit big they can be released to farm. It makes shrimp yield go up, and now all farmers follow that technique. Before project, farmerselect rice variety themselves. Didn’t know one good for this area. Ask other areas but these don’t work here. By project and demonstration see many rice varieties at the same time and therefore can select for this area. Now people succeed with new variety and (other) people come to see. Now we have a meeting, talk together and all 27 of us decide to adopt the same variety. Before a technician would come to train but we didn’t see in the real. Sometime, technician say new variety good but when we try not good. Now with demonstration site can see one that does good. Mr. Dang (lead producer at the demonstration site) plays a critical role in this community and can connect with technicians to get new varieties. I suggest every year the extension center send a set of new varieties to Mr. Dang to plant and try and in the end we can select the best for our area. If we do that it means farmers can save time and can save a lot of money. Need to be sure that the variety is good for the farm. And because in small communities there is a lot of interaction between families, they inform each other about the new variety, how to resist salinity, about insects, how to adapt soil fertility and fertilizer, so people come to see. Before the demonstration, when husband want to use new variety, wife may not accept and there can be a strong argument. With model here people get to see, it is easy to decide and there is family harmony. Now if the crop fails the wife doesn’t complain. There is shared responsibility. Mr. Dang then added to Mrs. Hoa’s story that after three years of improved water quality (access to freshwater) he can plant coconuts, plum, rubber, mango, and papaya. Some farmers plant vegetables. Where these won’t grow we can plant eucalyptus and acacias to help the soil and provide wood. Figure 7. Mrs. Hoa’s story CPWF Working Paper - Impact Assessment Series No. 07 15 2011.07.30.CPWF WP-IAS-07.draftv1 2. Improved water management strategies and can then begin to plan their farming business for the following year. The impact story here is about applied science providing the information to assist the delivery of There is some capacity for producers and local appropriate water resources to producers. If their authorities to build temporary dams to further livelihoods are to improve then being sure of water modify the flow of water in their specific area and, in supply is essential. this instance, some form of community agreement is important. Financial support to build these dams The main output produced by this project in regard is provided by the Bac Lieu People’s Committee but to water management was an improved version of the since the water operations at the provincial level have VRSAP model. This model predicts the consequences become more attuned to what is required, the need for of sluice gate operations and provides scenarios to temporary dams has diminished. An additional benefit inform the decisions on when to open or close the of fewer temporary dams, aside from reducing cost gates. This project was built on an already existing and effort, is that more channels within the system model (Hoanh et al. 2006) to include more complex remain open for boats – the main transportation situations to reflect reality such as two-way water mode in many areas. There has been some work done flows. It also added a module for acidity and improved on developing mobile dams made of plastic, which are linkages to GIS facilities for ease of inputting and cheaper and easier to use and therefore provide more presentation of outputs. flexibility but there are still problems with leakage. In order to develop the model, an extensive data Consistent anecdotal evidence suggests that the collection system was developed, expanding on the level of conflicts have reduced. The interviews with a one already in place. All the important parameters, range of stakeholders indicated that the following: such as pH, EC, salinity and so on, were monitored. • At the Bac Lieu level, despite some criticism Experiments were also carried out to understand the that the process is not participatory, producers movement of acidic water in acid-affected areas. This are able to plan better because of certainty of work had two benefits. The first was that it informed water resource. the model and the second was that it provided background information on which technologies might • At the community level, there may be still some be recommended or discouraged. conflict between producers who do not share a common vision about what to produce and The VRSAP model is now run by SIWRP who when. provides scenarios to the DARD (Water Operations) • Bac Lieu Province is part of a much wider who, in turn, use this information along with catchment system. What it decides affects its consideration of other factors, such as rainfall and in- neighbors and vice versa and there has been flows, to manage the flow of water. The model predicts some tension at this level. As a result, the what will happen under various conditions. Once the Bac Lieu People’s Committee has formed an dates of sluice gate operations are determined, early in alliance with two of its neighboring provinces, the rainy season when the sluices are open, producers Cà Mau (downstream) and Soc Trang are informed through a range of communication (upstream) to take a more regional approach. 16 Managing water and land at the interface between fresh and saline environments The formation of this “regional water alliance” example, further research is required on the movement was strongly influenced by running the model of acidic water in the acid sulphate soils areas. The for the whole Cà Mau Peninsula demonstrating analysis of data following the principles of realistic impacts on water quality of various scenarios. evaluation is given in Figure 8. • The improved VRSAP is now used by SIWRP for water resources planning for the whole Cà 3. A supportive policy environment Mau Peninsula and the entire Mekong Delta. The third impact story is about the Bac Lieu A second piece of work was also undertaken to People’s Committee providing the policy environment and leadership to promote change. Much of the understand the aquatic biodiversity and fisheries in information in this section was gathered during the water systems (Baran, Chheng et al. 2010). This interviews with senior people, past and present, from information, along with other items of information, the Committee. In summary the People’s Committee contributed to the development of the BayFish-Bac felt that while the income of people is increasing and Lieu model, again another predictive tool but one that the project contributed to this significantly, many that is more broadly based than the VRSAP model. people are still poor. They see their role as making It considers optimal water regimes and trade-offs and and managing policy that focuses on the happiness is based on integrating local databases, the VRSAP of people through poverty reduction and sustainable model outputs and stakeholder consultations in livelihoods. a Bayesian probabilistic model. To quote Baran, Janatunen et al. (2010) this model allows detailing Specifically, the Bac Lieu People’s Committee has of 1) annual production probabilities in the case of used the information and outputs generated by the a baseline scenario, 2) outcomes for four different project to do the following: sluice gate operation modes, and 3) trade-offs between household income, food security and environmental outcomes for each scenario. It could, for example, answer • Develop a Land Use Plan to 2010, and it did so questions on the impact of improving shrimp farming before the project ended using the information against rice production. as soon as it was available. This land use map (see Figure 9) outlines the different land use zones and, therefore, the identification of types Researchers at CTU have been trained in using of suitable farming options. It reflects a change the model and it has been introduced to the Bac in attitude, considering treating saline water Lieu People’s Committee. According to the project’s as a resource rather than a problem, and it senior hydrologist, Dr. Hoanh, further development recognizes that producers need the flexibility and application of this model will depend on how to respond to market commodity prices and problems in water management in Bac Lieu are other contextual factors. That is, it supports handled and on future collaboration between CTU diversification. and the Bac Lieu People’s Committee in finding alternatives and analyzing trade-offs. • Predict what would happen for different land use zones under various scenarios of sluice The interviews revealed there is still work to do gate operations. It is sharing these results with in understanding parts of Bac Lieu ecosystems. For surrounding provinces and this is promoting CPWF Working Paper - Impact Assessment Series No. 07 17 2011.07.30.CPWF WP-IAS-07.draftv1 Contexts • Conflict over water use creating no-win situations. • A water model for salinity that did not include complex combinations of operating many sluice gates with different flow regimes and propagation of acidic water in the canal network. • Scientists who understood the problem and had the capabilities to develop the model further. Activities • Applied science research including data collection and model redevelopment. Mechanism • Here is a technical problem, use technology to solve it. • Urgent level of demand driven by the government – the “we really need this” mechanism. Outcomes • Improved information for the Bac Lieu DARD Water Operations where people used to inform their decisions for operating the sluices. • Improved information on water management to the whole of the Cà Mau Peninsula. Characteristics of the project • Good science. • Long-term and trusted relationships. Challenges for the future • Completion of the BayFish-Bac Lieu model. • Further development of the VRSAP model depending on future water management strategies and the collaboration of CTU and the People’s Committee. • Further research on the movement of acid in the system. Figure 8. The causal package for improved water management the idea of a strategy for the broader Mekong those without collateral provided they could River Delta. demonstrate they were applying suitable • Consider how the principles and information technologies, especially those promoted by the generated by this project might be applied to project. the area of Bac Lieu nearer the coast and outside the influence of the sluices (and the project Although it is hard to quantify these changes at the area). Poverty and uncertainty are issues here. policy level it represents a significant investment on the part of the Bac Lieu People’s Committee, and • Support the work of the project by helping local the land use map provides visual evidence of one of people understand when, where and what crops the changes. This success has been attributed to i) can be grown. Now people have more choices the quality of the partnership the project has had and can manage their resources better. Form with the Committee and ii) the role of the project in a water alliance with Cà Mau and Soc Trang responding to urgent issues. provinces to develop a regional approach to water management. Another benefit that the Bac Lieu People’s Committee • Encourage the banks to provide credit to acknowledged was the building of local capacities in 18 Managing water and land at the interface between fresh and saline environments Figure 9. The Bac Lieu land use map: evidence of use of project data research and scientific knowledge as a result of sharing This project was launched at the right policy moment. expertise at events, such as the International Delta 05 It addressed a critical policy need to take action to Conference, which was held in Bac Lieu. The People’s reduce both conflict over water use and poverty. It did Committee believes this type of interchange has the so in the face of an expressed Central Government potential to benefit many (who live in tropical deltaic Policy of converting the delta to increased rice areas) and (in the future) they would like to see more production but it is hoped that should the Central international efforts bringing world experts together so Government continue its plan to complete the sluices that the model can help them adapt to climate change that the science, which has backed the Bac Lieu People’ and rising sea levels. Committee decisions, will influence central decision making. To that extent DARD has put considerable One of the major lessons for the Committee effort into strengthening its communication with coming from the project has been the value of the Central Government. The causal package is given using the community to evaluate its work and that below in Figure 10. this is the best way to gain insight into how to provide benefit to the local people. CPWF Working Paper - Impact Assessment Series No. 07 19 2011.07.30.CPWF WP-IAS-07.draftv1 Context • Conflict over water use, especially with shrimp aquaculturalists seeing their supply of saline water compromised. • Poverty levels worse than in other parts of the region. • International scientists, who understood the issues, already had a long-term relationship with the government. • Rice production on acid soils (low yields) and emphasis on a monoculture. Activities • Supply of scientific information to provide policymakers with an evidence base to develop a policy of diversified land use and zoning. • Development of a new land use policy that moved the province away from a trend of more rice production to one that recognized the value of saline aquaculture and took into account the relationship between soil types and forms of agriculture. It supported diversification. • Provided supporting services. • Improved water management. • Encouraged banks to provide credit, especially to those who lacked collateral provided they were using the right on-farm technologies (although this was not part of the project). Mechanism • There is a problem whose solution needs scientific evidence. • The “we need to support change through all appropriate policy and service delivery channels” mechanism. Outcomes • The Bac Lieu People’s Committee used project results to modify their land use plan that was approved by the Central Government in the revised plan Adjusted Land Use Plan for the Cà Mau Peninsula. • Diversification of production that is addressing rural poverty in Bac Lieu. • Less conflicts over water. Characteristics of the project • Use of long-standing and respected relationships. • Clear and urgent demand to address the issue of poverty. Challenges for the future • The Bac Lieu People’s Committee would like to see a similar intervention for the coastal areas that are outside the project areas. • Strengthening alliances with neighbors who are also reliant on the waters of the Mekong Delta. Figure 10. The causal package for the policy environment 20 Managing water and land at the interface between fresh and saline environments Less direct impacts iii) Changes in the surrounding provinces Three main areas of less direct impacts were identified One consequence of the project also beginning to be in the impact pathway analysis. The categorization of seen in areas outside Bac Lieu is that producers in the these as less-direct was because the output users were surrounding provinces, through processes of diffusion, not as well defined, nor were there specific activities have noticed the changes emerging in the Bac Lieu undertaken by the project to influence them directly, Province and are beginning to make enquiries about although these may have been referred to generically. the technologies. For example, the Sub-Institute for Fisheries Research in a neighboring province is i) Changes in access to credit receiving requests to assist it with new species because of the Bac Lieu story. This evaluation has not been As mentioned in the previous section the Bac Lieu able to explore what change has occurred as a result of People People’s Committee now encourages banks to technological diffusion but this is obviously a positive provide credit to those producers without collateral consequence of the project. but who are adopting the technologies appropriate for the land use zone. While the original impact pathways A sea grass industry is emerging based model showed changes in capital (and market) support on growing sea grass (Scirpus littoralis). as important to achieving the ultimate project goal, When grown in shrimp ponds it provides no specific outputs of activities were undertaken a range of benefits. Ni (undated) identified to address this component. The project team had these as: difficulty describing what had been intended here. • Maintenance of low temperature in Despite this, some changes have occurred, possibly the ponds. triggered by the planned impacts. Although obtaining • Filtering of pollutants. credit from banks is still difficult for producers, some • Provision of food to crabs and fish. • Provision of income to the landowners improvements have emerged. (sell raw materials), especially providing income to the poor (landless) through ii) Emergence of new or more agribusinesses handicraft processing (see samples). This idea came about through group Increased farm productivity requires more agribusiness discussions between the scientists and to support it. Some new businesses have emerged as the aquaculturalists with the latter a result of the activity of the project (see the example reflecting that this used to happen years of sea grass in Figure 9). Others have emerged ago. The practice was stopped because without direct contact with the project but maybe it was thought it compromised shrimp through some flow-on effect. Examples to emerge productivity. In order to support this have been rice seed testing as Bac Lieu has some industry the project introduced the buyers natural advantages over its neighbors, outlets for fresh to the farmer aquaculturalists who, in turn, aquacultural products, and other food-processing provided training to the locals in handicraft enterprises. production.. Figure 11. The story of sea grass CPWF Working Paper - Impact Assessment Series No. 07 21 2011.07.30.CPWF WP-IAS-07.draftv1 A summary of findings against the evaluation questions The following is a summary of findings against the evaluation questions. Most of the answers have already been discussed in the previous section but some of the data are presented here for the first time. Question Finding 1. What is the • Producers’ incomes are increasing in Bac Lieu through the application of change(s) appropriate technologies (and rejection of others) but the increase in income is variable across the province. • A land use policy in Bac Lieu now maps out appropriate land uses given soil and water conditions that encourage diversity. It underpins other policies and service delivery and thereby provides the blueprint for producers to plan their business. • There is an improvement in water supply (fresh and/or saline) that matches the appropriate land use zone. This is resulting in less conflict, especially at the provincial level, and providing producers with a critical resource when they need it. Underpinning this is an increase in understanding of the ecosystems in the Mekong Delta. 2. What is the scope and • Average income in Bac Lieu is now the same as for the rest of the Delta and it is the extent of these changes view of all stakeholders that the project would have contributed to this. on the ground? • Producers on the demonstration farms during the life of the project, on average, received an extra US$250 per/ha/year. • DARD has adopted the extension approach developed in this project. The number of demonstration sites has gone from eight during the life of the project to 80 new sites per year for the last 2 years involving 10,000 producers each year. The proportion of those who are now adopting a new or renovated technology is 50-70%. • There are approximately 116,000 farming families in Bac Lieu but the extent of adoption of optimal techniques across the Delta is not known. • The Bac Lieu People’s Committee has used information generated in this project either directly or indirectly to influence its policies and service provision and has indicated that its interest in this use is not over. Climate change and rising sea levels are issues that will need future input from research. • New knowledge on ecosystems in the Mekong Delta, expressed in terms of new scientific knowledge, new decision support tools and new processes for working in this area, has been, and will continue to be, disseminated throughout the wider scientific community. 3a. What were the The main mechanism that triggered change for producers participating in the mechanisms that demonstration sites was “to see one time is better than hear 100 times“ (Vietnamese brought about these saying). Other mechanisms included “sharing experiences makes it easier for me to changes? make decisions” and “I trust the advice of specialists, especially when I can participate in the evaluation.“ For the Bac Lieu People’s Committee the main mechanism was a belief in evidence- based policy development. 22 Managing water and land at the interface between fresh and saline environments Question Finding 3b. What contribution The project made four major contributions: did the project make? 1. It produced an improved VRSAP model for water management that now generates information in the form of scenarios and maps for improved sluice gate operations to better meet the needs of producers. It provided the model and other sources of information to inform the development of the Bac Lieu People’s Committee’s Land Use Policy, which recognizes the benefits of diversification and the role of saline water in farming systems. 2. It developed and evaluated a successful participatory extension approach that assists producers select appropriate technologies (and reject others) for their situation, based on on-farm demonstration and experimentation. This approach is now being applied more extensively by DARD. 3. It make a contribution to the body of knowledge on natural resources management in tropical deltas. 3c. Specifically, what Research played three roles in this project: role did research play? 1. Applied research was behind the many technical papers that resulted in the development and refinement of two models (VRSAP and the BayFish - Bac Lieu) and understanding of the ecosystems in the canal systems. It increased science capability through the development of new knowledge that is still being reviewed through the publication processes. It has already been disseminated through international conferences and as some of the project teams operate in the international arena this knowledge is likely to be passed on integrated into other projects. 2. Farming systems research was used by the producer groups, researchers and extension staff at the demonstration site to validate the merit or worth of new or renovated technologies in particular farming situations. 3. The provision of scientific information to policymakers to provide a robust evidence base for their policy decisions. 4a. What is it about the The contextual factors for the project were: context that is driving • The fact that it was demand-driven – there were problems in the Bac Lieu Province the change? that needed urgent attention at the policy, scientific and farm levels. • The long-standing relationships between the players who acknowledged one another’s potential contributions. • The scientific knowledge and experience of the partners. • The project team had a belief in participatory processes with the project management and when working with producers thereby facilitating the uptake of outputs. For producers, it was the desire to improve their livelihoods to meet their personal goals such as improving their living conditions or a better life for their children. 4b. Where could the • Application of the model and the companion on-farm technologies are beginning changes eventually to be adopted in the provinces neighboring Bac Lieu. spread? CPWF Working Paper - Impact Assessment Series No. 07 23 2011.07.30.CPWF WP-IAS-07.draftv1 Question Finding 4b. (continued) • The Bac Lieu People’s Committee is keen to apply the lessons learnt from this project to areas of Bac Lieu outside of the study area, that is, in the coastal areas. • Some of the experiences of the Vietnamese project were reported in the interviews to have influenced the Bangladesh project but this evaluation could not gather those data. • It is feasible that the information and lessons learnt will permeate to other tropical deltaic areas of the world through mechanisms such as the international conferences being organized by IRRI. • The new knowledge generated is already being applied in the CPWF project CP 25. 5a. What are the COSTS: benefits and costs of the • US$679,000 on Vietnamese outputs, 86% from CPFW over 3 years. changes (quantify them as far as possible) both • Project management and overhead costs. now, and potentially in • Existing VRSAP model. the future? • Existing agricultural and aquacultural technologies and scientific information. • Eight producers who provided farms as demonstration farms. • 40 producers who volunteered to provide farms as control sites with opportunity cost of approximately US$250/farm = US$10,000 • Local knowledge of producers. • Tacit knowledge of partners . • Logistical support . • CPWF provision of opportunities for between-project discussions between the Bangladesh and Viet Nam projects. • CPWF providing training in impact pathway analysis. • In-kind support to Bac Lieu People’s Committee. • Outputs from the previous project (funded by DFID and CGIAR). BeNefITS: • Increased farm incomes. • Increased diversification of farm activities, probably leading to more sustainable futures. • Fewer water conflicts. • Less need for temporary dams. • New knowledge from applied science and farming systems research. Future benefit costs will depend on the following: • Future action of the Vietnamese government. • Commodity prices and the future well-being of the Vietnamese economy. • Climate change and rising sea levels. • The future development of the whole catchment of the Mekong River across five countries. 24 Managing water and land at the interface between fresh and saline environments Question Finding 5a. (continued) One enduring legacy of this project may well be the lessons producers have learnt about how to make the best resource decisions for their farms. 5b. What did the (SEE THE “COSTS” IN QUESTION 5a) project invest? 6. What was the When asked this question most people agreed that, without the project, progress contribution of the could have been made but not as fast or necessarily in the same direction. The former CPWF in achieving the Vice-Chair stated that more progress has been made in the last 5 years (from the end change (to be assessed of the DFID-funded project to the CA and this CPWF project to now) than over by projecting what several decades before. Without the science and processes provided by this project and would happen without its predecessors, trial-and-error would have been the order of the day. CPWF funding and comparing it with what did happen)? 7. What are the This project was a good example of how to resolve resource issues in a tropical deltaic international public environment to reduce poverty and improve food security. goods that the project has generated that are related to the change? 8. What were the The major differences were as follows: differences between the • When it came to an impact evaluation it became necessary to track who used what original and the final output, how and with what result. This meant unbundling the stakeholder mix impact pathway? that was presented in the original diagram. • There was uncertainty about the meaning of some of the terms in the original impact pathway, probably as a result of most of the project team members not participating in the original design, and even for those that did the meaning had become diffuse. The meanings, therefore, had to be reconstructed when developing the post-project model. • Capital and market support was included in the original but the influence of the project was serendipitous rather than planned. People questioned whether or not this should have been in the logic model at all and if it was then what did it mean, and what actions should be taken. • With the knowledge of hindsight less direct impacts were included in the final version. CPWF Working Paper - Impact Assessment Series No. 07 25 2011.07.30.CPWF WP-IAS-07.draftv1 Lessons for future work The project team’s story about partnership This project has been successful and is likely to The Vital Roles of Local Governments and continue to make an impact. So what are the Development Agencies in a Research Project on features that need to be specifically identified for Coastal Resources Management future reference? The success of this project can be attributed to responding to demand at the policy level, Date when the change occurred: 2002 till 2006 integrating good science with the tacit knowledge of producers, having the right players at the table at the Place where the change occurred: Bac Lieu Province right time including the anticipated output users, and of Viet Nam having built-in evaluative processes so partners could monitor progress. The Story: Second, the integrity and passion of the project Usually, research is carried out by research institutes team in knitting all the threads together and and universities and is often divided by sectors, e.g. conducting quality applied science and farming agriculture, aquaculture, etc. The uptake of research systems research have been a major contributing results is often considered a separate step, to be feature. A special note needs to be made of the carried out by development agencies. Interventions by commitment of the team and the DARD extension different development agencies are often also carried staff to an extension process that provided solutions, out independently. built on the local knowledge of producers and left them with decision-making capabilities. Sustainable natural resources management in the coastal zones, where the freshwater and saline water The final word, however, is from the project team interface must take into account diverse stakeholder who believed in enduring and respectful partnerships. interests (e.g., agriculture, aquaculture, capture fishery) The account in the following story was submitted to a and complex multi-scale interactions among different meeting on the CPWF by the project team as a “Most resources (e.g., water, soil, land use). Among PN10 Significant Change” story. It is a shortened version partners are institutions in charge of various resources of that presentation, only referring to the Viet Nam important to coastal management (water, land use, component of the larger project agriculture, aquaculture, and fishery). 26 Managing water and land at the interface between fresh and saline environments What were the critical factors that led to the change? Built on partnership developed from previous projects. Clear understanding of roles and responsibilities of different organizations A good ex-ante analysis of uptake and impact pathways of the project findings They include research and development institutions, Why is the story significant? national, provincial and local governments and farmers. • The involvement of local government ensures The project creates a forum for these institutions to demand-driven research and the relevance of discuss their diverse views on the possible impact of research outputs. They also facilitate rapid uptake proposed resource use interventions. Of particular of technologies. importance are the collaborations/dialogues between • In Viet Nam, the research findings were institutions in charge of research in agriculture incorporated in the provincial land use and water (CTU), research in aquaculture (RIA2), land resources management schemes. management (IMRC), and water management • National agencies are sharing secondary data institutions (SIWRP). for developing resource management domains The inclusion of local governments and extension (RMD), hydraulic and salinity modelling. services (at district and provincial levels) in Viet Nam is pivotal for the speedy dissemination of on-farm What were the constraints? technologies of the project. They participated in on- • Infrastructural development requires time and farm research, and organized field visits and farmer resources; there is a considerable time lag between training, using existing farmer networks. planning and implementation. • Adoption of new technologies by farmers takes The planning and development institutions a few years and, therefore, impacts of improved (SIWRP, IRMC, DARD) are both research partners production systems may not be seen during the and clients of the project. They supplied the project project period. with important secondary data; gave feedback on research findings, and incorporated the appropriate What are the future implications for actions research findings in their development activities/ (e.g., future research), if any? work plans. • Improving coordination among the Top management employees of these national government organizations, and research and development and research agencies were involved development agencies. in the national advisory committee (NAC) of the • Implementing an impact monitoring program project. and impact assessment when the project ends. CPWF Working Paper - Impact Assessment Series No. 07 27 2011.07.30.CPWF WP-IAS-07.draftv1 People interviewed Project Management • Mr. Luong Phong Dong, Head of Agri-Aqua • Dr. T.P Tuong, Project Leader, IRRI. Office and Representative of the People’s • Dr. C.T Hoanh, IWMI. Committee. • Mr. Vo Van Lang, Deputy Head of Agri-Aqua Bac Lieu People’s Committee Office. • Mr. Nguyen Truong Giang, former Vice-Chair of the Bac Lieu People’s Committee. Project Management Unit • Mr. Nguyen Thanh Be, Vice-Chair of the Bac • Mr. Phan Hong Thai,* Secretary of the Project Lieu’s People’s Committee. Management Unit, Synthesis of data and monitoring the project on behalf of DARD. DARD (In this section * indicates those who participated in one or both of the workshops.) Can Tho University • Dr. Duong Van Ni,* Agronomist/ Agriculture Environmentalist. • Mr. Diep Chan Ben,* Deputy Director of DARD • Project Coordinator for the Viet Nam Case Study • Mr. Duong Huyen Vu,* Extension Manager. • Dr. Nguyen Duy Can,* Farming Systems Analyst. • Mr. Le Canh Dung,* Economist. Aquaculture • Mr. Tran Duy Phat. • Mr. Ta Minh Phu, Deputy Director of DARD, Fishing and Aquaculture. SIWRP • Mr. Pham Hoang Giang, Fishery and Aquaculture, • Mr. Nguyen Van Ngoc,* Hydraulic modeller. Extension Officer. RIA2 Water Management • Mr. Thieu Lu,* Head of the Sub-Institute of RIA2 • Mr. Luong Ngoc Lan, Director of DARD, in in Ca Mau. charge of the water management plan for the • Lead Producers and Producer Groups. province. • Mr. Nguyen Van Phong, (Brackish water system) • Mr. Lai Thanh An, Head of Water Resources and 11 aquaculturalists. Department. • Mr. Le Van Dang, (Fresh-brackish water system) • Mr. Nguyen Van Minh, Head of Hydraulic Works and 10 local farmers/ aquaculturalists. Management. • Mr. Tran Cong Lap (Freshwater system) and 15 farmers. Phuoc Long District • Mr. Tran Van An, Head of Agric-Aqua Office, Hong Dan District. 28 Managing water and land at the interface between fresh and saline environments References Baran, E., P. Chheng, F. Warry, T.T. Vo, P.H. Ha, and C.T. Hoanh. 2007. Aquatic resources and environmental variability in Bac Lieu Province (southern Viet Nam). In: Tropical deltas and coastal zones: food production, communities and environment at the land-water interface, eds. C.T. Hoanh, B.W. Szuster, S-P. Kam, A.M. Ismail and A.D. Noble; Comprehensive assessment of water management in agriculture series Vol. 9 Wallingford, UK: CAB International. 13-32. Baran, E., T. Jantunen, P. Chheng and C.T. Hoanh. 2007. Integrated management of aquatic resources: A Bayesian approach to water control and trade-offs in southern Viet Nam). In: Tropical deltas and coastal zones: food production, communities and environment at the land-water interface, eds. C.T. Hoanh, B.W. Szuster, S-P. Kam, A.M. Ismail and A.D. Noble; Comprehensive assessment of water management in agriculture series Vol. 9 Wallingford, UK: CAB International. 133-143. Boomsma, J., B. McDonald and M. Walton. 1996. Target 10: The first three years. Melbourne, Australia: Agriculture Victoria. Catholic Education Office. 1986. Photolanguage Australia. Sydney, Australia. CPWF (Challenge Program on Water and Food). 2007. Annual report (managing water and land resources for sustainable livelihoods at the interface between fresh and saline water environments in Viet Nam and Bangladesh). CPWF. 2006. Annual report (managing water and land resources for sustainable livelihoods at the interface between fresh and saline water environments in Viet Nam and Bangladesh). Development Assistance Committee. 2006. Impact evaluation: An overview and some issues for discussion, organisation for economic and cooperative development. http://www.oecd.org/secure/pdfDocument/0,283 4,en_21571361_34047972_36239874_1_1_1_1,00.pdf Douthwaite, B. and B.S. Alvarez. 2006. Next step towards the PN10 impact narrative: Impact Pathways Model Development, BFP Impact Assessment Project, CIAT. Gowing, J.W., T.P. Tuong, C.T. Hoanh and N.T. Khiem. 2006. Social and environmental impact of rapid change in the coastal zone of Viet Nam: An assessment of sustainability issues. In: Environment and livelihood in tropical coastal zones: Managing agriculture-fishery-aquaculture conflicts, eds. C.T. Hoanh, T.P. Tuong, J.W. Gowing and B. Hardy; Comprehensive assessment of water management in agriculture series Vol. 2 Wallingford, UK: CAB International. 48-60. Hoanh, C.T., N.D. Phong, T.P. Tuong, N.D. Dat, N.V. Ngoc and N.X. Hien 2006. Improvement of hydraulic and water quality modelling for Cà Mau Peninsula, Mekong Delta, Viet Nam. Paper presented at the 2nd Annual Meeting of Project CPWF#10 CPWF Working Paper - Impact Assessment Series No. 07 29 2011.07.30.CPWF WP-IAS-07.draftv1 IRRI (International Rice Research Institute). 2004. Managing water and land resources at the interface between fresh and saline water environments in Viet Nam and Bangladesh – Revised Project Proposal. Challenge Program on Water and Food. http://www.waterandfood.org/fileadmin/CPWF_Documents/Documents/First_call_ projects/PN_10.pdf Kam, S.P., N.V. Nhan, T.P. Tuong, C.T. Hoanh, V.T. Be Nam and A. Maunahan. 2007. Applying the Resource Management Domain (RMD) concept to land and water use and management in the coastal zone: Case study of Bac Lieu Province, Viet Nam. In: Environment and livelihood in tropical coastal zones: Managing agriculture-fishery-aquaculture conflicts, eds. C.T. Hoanh, T.P. Tuong, J.W. Gowing and B. Hardy; Comprehensive assessment of water management in agriculture series Vol. 2 Wallingford, UK: CAB International. 193-205. Khiem, N.T and M. Hossain, 2010. Dynamics and resource use strategies in different ecosystems of the coastal zones of Bac Lieu Province, Viet Nam. In: Tropical deltas and coastal zones: food production, communities and environment at the land-water interface, eds. C.T. Hoanh, B.W. Szuster, S-P. Kam, A.M. Ismail and A.D. Noble; Comprehensive assessment of water management in agriculture series Vol. 9 Wallingford, UK: CAB International. 436-447. Mayne, J. 2008. Contribution analysis: An approach to exploring cause and effect. ILAC Brief No. 16, http://www.cgiar- ilac.org Ni, D.V., T.D. Phat, T. Lu, P.B.V. Tung, D.C. Ben, D.H. Vu, P.H. Thai, P.H. Giang and T.P. Tuong. 2010. Diversified cropping systems in a coastal province of the Mekong Delta, Viet Nam: from testing to outscaling. In: Tropical deltas and coastal zones: food production, communities and environment at the land-water interface, eds. C.T. Hoanh, B.W. Szuster, S-P. Kam, A.M. Ismail and A.D. Noble; Comprehensive assessment of water management in agriculture series Vol. 9 Wallingford, UK: CAB International. 223-238. Pawson, R. and R.Tilley. 1997. Realistic evaluation. London: Sage publications. Stanfield, R.B., ed. 1997. The art of focused conversation. Ontario, Canada: The Canadian Institute of Cultural Affairs. 30 Managing water and land at the interface between fresh and saline environments Appendix 1. Terms of Reference Terms of Reference for Bron McDonald in the CPWF Adoption and Cost-Benefit Analysis Project The International Center for Tropical Agriculture (CIAT) requires the services of an impact evaluation consultant. Bron McDonald, Evaluator, has been identified to carry out the following evaluation that forms part of the outputs of the Adoption and Cost-Benefit Analysis Project. evaluation Design and Implementation The evaluation will be of PN10: Coastal Resource Management for Improving Livelihoods (CRESMIL). Orientation for the evaluation is technical and significant partnership change stories will be written by the project (see Annex 1). The evaluation will follow the proposal in Annex 2. The evaluation should address the following questions: 1. What are the changes? 2. What is the scope and extent of these changes on the ground? 3. How did the changes come about? What contribution did the project make to them? 4. What is driving the changes? Where could the changes eventually spread? 5. What are the benefits and costs of the changes (quantify them as far as possible) both now, and potentially in the future? What did the project invest in? 6. What was the contribution of the CPWF in achieving the changes (to be assessed by projecting what would have happened without CPWF funding and comparing it with what happened)? 7. What was the role of research in achieving the changes? 8. What are the international public goods that the project has generated that are related to the changes? 9. Did the project follow the expected impact pathways (as described in an earlier impact pathways workshop to define them)? The method for carrying out the evaluation is up to the evaluator with the following provisos: • The evaluation should focus on adoption and impact and not be an overall evaluation of the project. • The Project Leader and Boru Douthwaite should agree to the evaluation design. • As far as possible changes should be quantified and costed. • A draft final report of about 30 pages (plus Annexes) is to be produced no later than 15 October, 2008 structured around the research questions (unless another structure makes more sense). The report should include: • A detailed impact pathway (i.e., logic model) for the changes evaluated, documented and compared and contrasted with those originally projected in 1) impact pathways workshops and 2) by the project leader before the beginning the evaluation. • Descriptions of strategies and tools for measuring change. • Descriptions of strategies and tools for determining attribution. CPWF Working Paper - Impact Assessment Series No. 07 31 2011.07.30.CPWF WP-IAS-07.draftv1 ANNeX 1. Significant Change Stories Story 1. The main focus of the evaluation The following story was selected as a favorite because the project has demonstrated its capacity to trigger various changes at a wide range of scales: • At farm level: improved rice and livelihoods of shrimp farmers (especially increased income). • At local management level: increased awareness of the value of brackish water for food production (in this case, shrimp) and thus the need to include brackish water in the management of water resources in coastal areas; strong partnership involving local institutions (especially NGOs): contribution to the production and dissemination of research outputs. • At national planning level: understanding the need to shift from a rice-based production system to a diversified production system in order to enhance the productivity and the ecological and social sustainability of the coastal area; reciprocal benefits of project partnership especially integration of research findings in national planning. • At scientific level: potential for producing International Public Goods on the management of the freshwater and saline water interface for the production of rice and shrimp in coastal areas. INRM research supports livelihood at the interface of freshwater and saline water environments Classification: Technical Name of Person Reporting: TP Tuong on behalf of PN10 members in Viet Nam. Project/Theme/Basin: PN10/Themes 1 and 3/Mekong and Indo-Gangetic basins. Date when the change occurred: 2001 Place where the change occurred: Bac Lieu Province, Mekong Delta, Viet Nam. The Story: Prior to 2000, with the aim of boosting rice production for export, the Vietnamese government invested in water management infrastructure (embankments and sluices) to protect Bac Lieu Province from salinity intrusion. The intervention adversely affected the livelihood of people in the west of the protected area who needed brackish water to raise shrimp. In 2001, demand of aqua products for export increased significantly, and conflicts between shrimp culture and rice culture became serious due to different water quality requirements: saline water for shrimp and freshwater for rice. A DfID-funded project analyzed the pros and cons of the salinity control measures and the land use policy that favored rice intensification. The project proposed a land zoning scheme and the associated sluice operation procedures that would accommodate both rice intensification in the eastern part and shrimp culture in the western part of the area and the shrimp (dry season) and rice (rainy season) systems in the transitional area. Changes in water quality due to sluice operation predicted by the hydraulic and salinity model were analyzed to identify the most suitable option. From 2002 to 2003, the local government adapted the land use zoning in the revised land use plan. Procedures for sluice operation were adopted and a water-quality monitoring network was established. Producers adjusted their production systems according to the zoning. PN10 work, which started in 2004 involved refining the hydraulic and salinity models which they used to compare different water development scenarios (e.g., excavating new canals and dredging old ones) proposed by the local government and to find the impact of sluice operations of the surrounding province on Bac Lieu and vice versa. It also improved production systems in each of the “land use zones” by implementing agricultural and aquaculture experiments with producers, which have very much stabilized due to the preliminary land zoning. 32 Managing water and land at the interface between fresh and saline environments Now, the local water management offices have the capacity to manage the water-quality network and to refine the sluice operations through data monitoring to ensure suitable water quality for different zones. Producers have adapted newly improved production systems and farming technologies to reduce production risks and increase income (e.g., maintaining Scirpus littoralis Schrab in the shrimp fields to regulate pond temperature, reducing shrimp diseases and getting extra income, adopting multiculture with shrimp and crab instead of shrimp monoculture, planting upland crops after two rice crops instead of three in freshwater zones, and using new rice varieties, etc,). Why is the story significant? • National planners accepted the diversification in production systems instead of monoculture with rice as the most dominant crop. • The local government accepted the concept that brackish water is a resource instead of always labeling it a “constraint to production.” • The project helped boost farm income and improve farmers’ livelihoods. • The hydraulic model helped Bac Lieu and surrounding provinces to understand the interaction among water management systems in their own territory. Based on the suggestion from the project, MARD has established the “river basin organization” to coordinate water management in neighboring provinces. • The impacts of research were multi-scale: from regional (land use and water management) to field level (farming technologies). • The concept and methodology can be applicable in other coastal zones. What were the critical factors that led to the change? • Built on the success of previous projects. • A multi-scale approach to resource management and quantification of upstream-downstream interactions among different zones. • Participatory research with farmers and on-farm tests facilitated the dissemination of technologies. • Good communication with local government and development institutions. What were the constraints? • A lot of data were required for hydraulic and salinity modeling; in many cases, governments do not invest systematically nor do they sustain data acquisition. • Local governments have limited human resource capacity for technology transfer, e.g., hydraulic and salinity model, and GIS applications. • Limited resources and time for testing the improved production systems with farmers given that research projects are often time-bound and do not include “extension and development“ costs . What are the future implications for actions (e.g., future research), if any? • Continue the tests of improved production systems with farmers. • Include other water-quality components (e.g., DO acidity) in the water model and use it as decision support tools for production planning. • Expand the study on land use zoning and water management to the surrounding provinces that are sharing the same water control system with Bac Lieu. CPWF Working Paper - Impact Assessment Series No. 07 33 2011.07.30.CPWF WP-IAS-07.draftv1 Story 2 The next story was chosen as a favorite partnership story because it is an excellent example of the impact that can be achieved by identifying and truly involving the key stakeholders and end users – farmers, fishers, research and development (R&D) organizations including top management, district and provincial extension, NGO, local government, government resource managers, government planning and development institutions. The vital roles of NGOs, local governments and development agencies in a research project on Coastal Resources Management Classification: Partnership Name of Person Reporting: Dr. TP Tuong on behalf of PN10 team members. Project/Theme/Basin: PN10, Themes 1 and 3/Mekong and Indo-Gangetic basins. Date when the change occurred: 2002 Place where the change occurred: Bac Lieu Province of Viet Nam and the southwest coastal sub districts (Batiaghata, Dumuria and Paikgacha) of Bangladesh. The Story: Usually research is carried out by research institutes and universities and is often divided by sectors, e.g., agriculture, aquaculture, etc. The uptake of research results is often considered a separate step, to be carried out by development agencies. Interventions by different development agencies are often also carried out independently. Sustainable natural resources management in the coastal zones, where freshwater and saline water interface, must take into account diverse stakeholder interests (e.g., agriculture, aquaculture, capture fishery) and complex multi- scale interactions among different resources (e.g., water, soil, land use). Among PN10 partners are institutions in charge of various resources important to coastal management (water, land use, agriculture, aquaculture, fishery). They include R&D institutions, national, provincial and local governments, NGOs and farmers. The project creates a forum for these institutions to discuss their diverse views on the possible impact of proposed resource use interventions. Of particular importance is the collaboration/dialogues between institutions in charge of research in agriculture (BRRI in Bangladesh, CTU in Viet Nam), research in aquaculture (BFRI in Bangladesh and RIA2 in Viet Nam), land resource management (LGED in Bangladesh, IMRC in Viet Nam), and water management institutions (BWDB in Bangladesh, SIWRP in Viet Nam). The inclusion of an NGO Health, Education and Economic Development (HEED) in Bangladesh and local governments and extension services (at district and provincial levels) in Viet Nam are pivotal in the speedy dissemination of on-farm technologies of the project. They participated in on-farm research, and organized field visits and farmer training, using existing farmer networks. The planning and development institutions (BWDB, LGED in Bangladesh; SIWRP, IRMC, DARD) are both research partners and clients of the project. They supplied the project with important secondary data; gave feedback on research findings, and incorporated the appropriate research findings in their development activities/work plans. Top management employees of these national R&D agencies were involved in the national advisory committee (NAC) of the project. Why is the story significant? • The involvement of local government and NGOs ensures demand-driven research and the relevance of research outputs. They also facilitate rapid uptake of technologies. 34 Managing water and land at the interface between fresh and saline environments • In Bangladesh, research results encouraged BWDB to implement a project on Integrated Planning for Sustainable Water Management (IPSWAM) involving farmer groups in polder 30 at Batiaghata, Khulna. LGED extended its support to determine elevation of water table in Tala upazila of the Satkhira District. • Realizing the importance of, and farmers’ interest in, HYV, the Department of Agricultural Extension (DAE) and BWDB have taken up development interventions to reduce the flood depth in polders 22, 29 and 30 (in Khulna District) so that HYV can be grown in the wet (commonly known as aman) season. • In Viet Nam, the research findings were incorporated in the provincial land use and water management schemes. • National agencies are sharing secondary data for developing resource management domains (RMD), and hydraulic and salinity modeling. What were the critical factors that led to the change? • Built on partnership developed from previous projects. • A clear understanding of roles and responsibilities of different organizations in each country and how they work/interact. • A good ex ante analysis of uptake and impact pathway of the project findings. What were the constraints? • Infrastructural development requires time and resources; there is a considerable time lag between planning and implementation. • Adoption of new technologies by farmers takes a few years; therefore, impacts of improved production systems may not be seen during the project period. What are the future implications for actions (e.g., future research), if any? • Improving coordination among the government organizations, NGOs, and R&D agencies. • Implementing an impact monitoring program and impact assessment when the project ends. CPWF Working Paper - Impact Assessment Series No. 07 35 2011.07.30.CPWF WP-IAS-07.draftv1 ANNeX 2. evaluation Proposal - Process This impact evaluation will employ participatory processes and, for evidence, will rely on a combination of existing data and group wisdom. Essentially, it is an approach designed to facilitate learning. The process will involve a comparison of the initial impact pathway designed in the ex ante phase of the project to what has actually happened. This comparison, if possible, will be done from three different perspectives. First, from the project team, second, from the key stakeholders and last, from the beneficiaries of the project. The Process A six step process is proposed for this evaluation that relies on the participation of the project team, its key stakeholders and where or if practical, some of the project’s beneficiaries. The steps are as follows: 1. A desk study prior to a visit to the Mekong Delta of key existing documents including any evaluations or monitoring data to date. 2. Work with the project team in Viet Nam to: i. Reconstruct the logic model based on what actually happened. ii. Identify evidence that supports the “actual” model. iii. Compare the original logic model with the actual model and seek explanation of any differences between the two. iv. Work with key stakeholders, collaborators and partners in Viet Nam. 3. Repeat the above but with key stakeholders (funders, partners, collaborators). 4. If possible, interview a small group of beneficiaries in Viet Nam. This requires discussion with the project leader on how this might happen and if this is feasible. 5. Preparation of a report (in Australia) that will provide information on the following: i. In summary form, a brief background to the project and its rationale. ii. A description of its evidence-based achievements using the headings of the logic model. iii. Comment on the differences between the original and the actual logic models. iv. Comment on the differing perceptions of the project from the three different viewpoints. v. Lessons learnt about the project. vi. Reflection on the impact evaluation process undertaken here and the lessons learnt, especially those relating to potential improvements while acknowledging tight timeline and limited resources. 6. Presentation of findings to the team in Viet Nam (if possible) and inclusion of their comments into the final report to ensure that the principles of participatory evaluation are adhered to. An electronic version of the final report will be provided by the consultant. 36 Managing water and land at the interface between fresh and saline environments Appendix 2. Project 10: Original Impact Pathway Model CPWF Working Paper - Impact Assessment Series No. 07 37 Main project Outputs Changes in Project Goals Technology stakeholders' and Vision 2011transferred to farmers People can produce attitudes and and fishers also rice and shrimp perceptions through participatory throughout the year, Techniques for rice Improved Adoption of experiments and reduce the risk of and rice-aquaculture knowledge of Iterations of technologies demos, also extension production systems for stakeholders learning and changes services such as environmental at pilot sites ferent cycle in practice leaf-lefts, damages, and no Farmers in the domains with dif broadcasting conflicts in water east and fishers soil and programs, etc. requirements for in the west water quality Stakeholders CPWF GOAL4 production became richer characteristics. modify and innovate 6 8 13 1 Co-development of People canDecision-support tools duce rice in and an institutional project outputs in pro pilot sites Planning and the east and framework for Farmers and fishers management of shrimp in theintegrated multipurpose improve their resources for oughoutmanagement of a dual Scaling up west thr production systems multi-purposes, also the yearfresh- and more participatory in 14 brackish-water and cultivation techniques managementregime Reduced Recommendations 7 poverty and 2 for water 9 insecurity in management tidal regions framework shared with policy makers No conflicts in Model results and and local Capital and market water recommendations for authorities support to farmers requirements establishing the regional to improve the and fishers for production in 17 water management resource 10 the water framework management interface zones 5 15 3 Managers at different levels change policy Reduction in the in resource risk of management for environmental multiple use damages that key 11 cause unsustainable from from from production in from Timeline Outputs-Vision ProblemTree Project Proposal the acid soil areas 16 Farmers and fishers improve their production systems and cultivation techniques 12 Scaling Out 2011.07.30.CPWF WP-IAS-07.draftv1 Appendix 3. Summary graphs on income change The following graphs, which were presented in a socioeconomic assessment undertaken as part of this project, show that changes in incomes have been occurring over the last 10 years (Can et al. 2010) and that all three examples presented have experienced some income growth. As this study was completed at the end of the project, before its full impact has been realized, it is difficult to draw conclusions from the income data since that time. In the process of collecting these data, though, producers were asked what contributed to the increase in income and the technologies. Ideally, a follow-up study in 2 or 3 years’ time matched with a measure of on-change farm would produce a stronger picture. The following three graph s are from Can et al. 2010 showing specific income changes at three sites across Bac Lieu up to the end of the project. The following graphs, which were presented in a socioeconomic assessment undertaken as Each of the visits to the depmarot nosft trhaitsi opnro sjietcets, csohonwdu tchtaet dc haasn pgaerst ionf itnhciosm eevsa lhuaavteio bne ewnh oecrceu prrriondgu ocveerrs trheec olausnt t1e0d ytehaerisr personal experiences confi(rCmaend e tth ael p. i2c0tu10re) tahnadt wthaast raelclo trhdreede ienx tahmep sloesc iopreecsoenotemd ihc asvtue deyx. pAelrli efneclte dth esoym wee rien cboemttee r- off (although not rich yet!g)r oanwdth t. hAes b tehnise fisttusd oyf wthaes cpormojpelcett ewde arte tmheo erne dt hoafn th jeu sptr mojeocnt,e by.e Thfore siet sp feuollp ilme vpalcut ehda st hbeiern new-found ability to underretaalkizee edv, ailtu iast idoinffsi couf lnt etow dspraewci ecso anncdlu osino-nfsa rfmro mte cthne oilnocgoimese. Fdoarta e xsainmcep lteh, atth eti mgreo. uIpn inth e the saline area expressed thpero acdevssa notfa gceosl ltehcetiyn hg adth egsaein deadt af,r otmho ubgehin, gp arobdleu ctoer as ssweessr et haes kqeuda liwtyh aotf cthonet wribauteterd m toor et he accurately, and therefore minackreea bseet tiner idneccoimsieo nasn din t rheeg atercdh tnoo ltohgeiiers .f aIrdmea ollpy,e raa tfioolnlosw. O-utph estrus dmy einnt i2o noer d3 t hyee airnsc’ rteiamsee d capacity they gained fromm waotcrhkeind gw aisth a a g mroeuapsu. re of on-change farm would produce a stronger picture. The following three graphs are from Can et al. 2010 showing specific income changes at three sites across Bac Lieu up to the end of the project. Figure A3.1. Illustration o f household income and income sources over time as perceived by farmers in the Minh Dieu VillageF i(gEuarrel y1 I. nItlelursvteranttiioonn oZf ohnoeu s[echhoalndg iendc ofrmoem a an dsa ilninceo meceo slooguryc teos ao vfreers htiwmaet ears epceorlcoegivye pdr iboyr to 1998 and is fcahramrearcst einri ztheed Mbyin ahl lDuvieiaul Vsoilillasg])e. (ThEaerl y1 9In9t6e rhvoenutsieohno Zldon ien c[cohmaneg weda sf ruosmed a a ssa tlihnee beacsoel o(g1y0 0to% ) (n = 48) a freshwater ecology prior to 1998 and is characterized by alluvial soils]). The 1996 household income was used as the base (100%) (n = 48). 38 Managing water and land at the interface between fresh and saline environments 57    Figure 2. Illustration of household income and income sources overtime as perceived by Figure A3.2. Illustration Figfa o urrm f h e e2r o .s u Ii snell h uth o ste l r d aP tih in oonn co go m fT h e haaon n uh d s eV in hoil c llda o g m ien e c( Rso omecuee r n c at e n sI dn o t v ienr e cv r oe ti mn mteio e n aso sZ puoe rcn r ee c s – ei o W ved veartt ie b mr y ea f n a ad r s m th e pere sre cn in ev ivi trheodn e m P bye h n otn g Thanh Vfailrclmahgaeenr sg( eRinde tchteoen Pta hI nofntreegrs Tvhewhnaatntiehor n V eZicllooalngoeeg y(–R Webceeatwntete rIe nannt edr1v 9teh9n8et-i eo2nn0 v0Zi0ro,o nneam c–id eW nsta uctlehprah anantgede dtsh oteio les an) .vf irrTeoshnhemw e1an9tte9 r6 ecology cbhehatnowgueesednh o t1old9 9ian8 c-fo2rme0s0eh 0ww, aatsce iurd s eesduc olaplsoh gtahyte e b bsaeostiewl s(e)1e. 0nTh0 %1e9) 1 9(98n9- =260 3 0h90o),.u saechido ldsu ilnpchoamte e swoialss )u. seTdh ea s 1th99e 6b ase (100%) h(no u =se 3h9ol)d income was used as the base (100%) (n = 39). Figure 3. Illustration of household income and income sources over time as perc eived by Figure A3.3. IllustratiFoingfau orrmfe he3ro.s u Iislnle uhtshoterla dVt iioinnnhc o oLmf oheco aVunsidellh aiognlecd o (Mimncaeor sgmoineua ralc neInds t oeinrvvceeorn mttiiemo nse o Zausor ncpeees r– co envioevtre dstii gmbnyei f fiaacsram npteelyrrcs aeifinvf eetcdht eebd Vy b iyn h the closure of the sluices). The 1996 household income was used as the base (100%) (n = 43). Loc Villfaagrem (eMrs airng itnhea lV Ininthe rLvoenc tVioilnla Zgeo n(Me –ar gnionta ls iIgnnteirfivceanntitolyn aZffoencet e–d n boyt tshigen cifliocsaunrtely o af ftfhecet esldu ibcye s). The 1996 hotuhsee chloolsdu rien ocof mthee swluaisc uess)e.d T ahse t1h9e9 b6 ahsoeu (s1e0h0ol%d )i n(c58n o   =m e4 w3)a s used as the base (100%) (n = 43).   58    CPWF Working Paper - Impact Assessment Series No. 07 39 2011.07.30.CPWF WP-IAS-07.draftv1 Appendix 4. Limitations of the evaluation The methods used in the design of this evaluation were predominantly qualitative and included the following: 1. A review of existing papers including interim project reports, papers presented at conferences, expert reviewed published papers and those being prepared for publication. 2. Implementation of two facilitated workshops based on an ORID (Stanfield 1997) that takes people through a data-gathering phase, a reaction phase, an identification of the issues and a decisional phase. One was at the beginning of the visit to Viet Nam and the second at the end to review the findings of the interviews. As with any facilitation process the limitations are (i) the extent of espoused and tacit knowledge the participants have and (ii) the facilitator’s skills. Was any critical piece of information missed? Was the emphasis correct? 3. Revision of the existing impact pathway. The limitations of models that hypothesize a theory of action, such as the hypothetical impact pathway, are that these often do not include unintended impacts, do not consider contextual issues sufficiently and often only deal with the average experience. The evaluator used the semi-structured interviews to address these weaknesses. 4. Semi-structured interviews. 5. Group interviews based on producers recounting their stories of the project. The three groups that were interviewed had had a positive experience of the project. Time and logistics prevented interviewing more. Qualitative versus quantitative: The design may be considered less robust than if a randomised control treatment experiment had been untaken although control-treatment experiments were undertaken within each of the demonstration sites as an evaluative activity. An evaluation based on an extensive set of quantitative data was not feasible. Where possible this evaluation relied on what quantitative data there were in papers that were either published or in the process of being peer-/expert-reviewed. In order to overcome this limitation the principles of contribution analysis were applied. Qualitative data and stories of change: On the plus side, it is from the qualitative data and stories that projects and their commissioners can distil lessons for the future, because, often this information provides more insights into why success or failure occurs. The stories of significant change provided were in the positive but in the future they maybe negative stories and could be considered as well to avoid the criticism of being too appreciative and provide a more balanced view. It is often from the lessons from negative results that the most learning occurs. Who participates: One of the main limitations of combining the above methods is that the findings reflect the combined views of only those present. The project leader and the evaluator ensured that many people were included, either as participants in the two workshops and amongst those interviewed, but time was a limitation. Realistic evaluation: The main limitation experienced in using realistic evaluation in this project was that the necessary data collection strategies were not set up at the beginning of the project. As a result there was little to work with. Despite this, it is up to the commissioner to determine if this approach might be useful in the future and therefore planning the approach from the beginning. The application of realistic evaluation principles was essentially a “taste test” in this instance. 40 Managing water and land at the interface between fresh and saline environments Overall, the main strategy taken to reduce the limitations was triangulation, looking for common themes coming from multiple methods. Throughout the whole process the emergent findings from other methods were checked both in subsequent methods and in the final workshop where much of the evidence was reflected upon and validated. Outside of the methods there were other limitations in this evaluation, which included the following: 1. Not being able to compare the findings from the Mekong Delta with those from the Bangladesh component. 2. The unknown slippages in information and interpretation that may have occurred through translation from Vietnamese and English and vice versa, although the author would like to acknowledge the care and effort demonstrated by the project team to minimize this limitation. Their skills in written and verbal English need to be acknowledged and were greatly appreciated. 3. The limitation of time and resources, and the endeavor to do the best with what was available. 4. The methodological skills of the evaluator. CPWF Working Paper - Impact Assessment Series No. 07 41 2011.07.30.CPWF WP-IAS-07.draftv1 42 Managing water and land at the interface between fresh and saline environments IMPACT ASSESSMENT SERIES Impact Assessment Series (IA) IA 01 Stories from the field: a most significant change synthesis. Larry Harrington 2009 IA 02 Most significant change stories from the Challenge Program on Water and Food (CPWF). Cristina de Leon, Boru Douthwaite and Sophie Alvarez 2009 IA 03 Geographical extrapolation domain analysis: scaling up watershed management research projects, a toolkit to guide implementation. Jorge Rubiano and Victor Soto 2009 IA 04 Improving knowledge for targeting interventions: willingness of individuals to participate and calcula- tion of institutional environment indices. Jorge Rubiano and James Garcia International Center for Tropical Agriculture Cali, Colombia 2009 IA 05 Aerobic Rice - responding to water scarcity An impact assessment of the ‘Developing a System of Temperate and Tropical Aerobic Rice (STAR) in Asia’ project. Deborah Templeton and Ruvicyn Bayot 2011 IA 06 Citizen participation in managing water Do Conversatorios Generate Collective Action? An Outcome Evaluation of the CPWF Project: Sustaining Collective Action Linking Economic and Ecological Scales in Upper Watersheds (SCALES/PN20). Diana Marcela Córdoba and Douglas White 2011 IA 07 Managing water and land at the interface between fresh and saline environments - an impact evaluation. Bronwen Mcdonald 2011 IA 08 Strengthening CPWF project evaluations: Assessing research-for-development impact. John Mayne 2011 IA 09 Strengthening CPWF project evaluations: Review of the CPWF small grants initiative. Jonathan Woolley 2011 IA 10 Resilience of small farm households through research partnership: A review of evidence from CPWF projects. Jonathan Woolley and Boru Douthwaite 2011 About CPWF The Challenge Program on Water and Food was launched in 2002 as a reform initiative of the CGIAR, the Consultative Group on International Agricultural Research. CPWF aims to increase the resilience of social and ecological systems through better water management for food production (crops, fisheries and livestock). CPWF does this through an innovative research and development approach that brings together a broad range of scientists, development specialists, policy makers and communities to address the challenges of food security, poverty and water scarcity. CPWF is currently working in six river basins globally: Andes, Ganges, Limpopo, Mekong, Nile and Volta. About this Impact Assessment The Bac Lieu Province in the Mekong Delta is part of the Cà Mau Peninsula and is an important food- growing area in Viet Nam. It has a population of 830,000 with approximately 116,000 farming families living on small parcels of land producing a range of commodities for food security and the export market. These farmers and aquaculturalists (together called producers in this report) are highly dependent on accessing the right quality water, fresh or saline or both, at the right time to grow their crops or raise their shrimp, crabs or fish. This evaluation was intended to identify what contribution the project had made to reducing poverty and food insecurity in the Mekong Delta as a result of an investment by the CPWF. The success of this project, however, can be attributed to its positive interaction with policy development, integrating good science with the tacit knowledge of producers, having the right players at the right time and having built-in evaluative processes so partners could monitor progress. Mailing address: CGIAR Challenge Program on Water and Food P.O. Box 2075 127 Sunil Mawatha, Pelawatta, Battaramulla, Sri Lanka Tel +94 11 288 0143 Fax +94 11 278 4083 Email: cpwfsecretariat@cgiar.org www.waterandfood.org CGIAR