IFPRI Discussion Paper 01022 August 2010 Do Differences in the Scale of Irrigation Projects Generate Different Impacts on Poverty and Production? Andrew Dillon Poverty, Health, and Nutrition Division INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE The International Food Policy Research Institute (IFPRI) was established in 1975. IFPRI is one of 15 agricultural research centers that receive principal funding from governments, private foundations, and international and regional organizations, most of which are members of the Consultative Group on International Agricultural Research (CGIAR). PARTNERS AND CONTRIBUTORS IFPRI gratefully acknowledges the generous unrestricted funding from Australia, Canada, China, Denmark, Finland, France, Germany, India, Ireland, Italy, Japan, the Netherlands, Norway, the Philippines, South Africa, Sweden, Switzerland, the United Kingdom, the United States, and the World Bank. AUTHOR Andrew Dillon, International Food Policy Research Institute Postdoctoral Research Fellow, Poverty, Health, and Nutrition Division Notices 1 Effective January 2007, the Discussion Paper series within each division and the Director General’s Office of IFPRI were merged into one IFPRI–wide Discussion Paper series. The new series begins with number 00689, reflecting the prior publication of 688 discussion papers within the dispersed series. The earlier series are available on IFPRI’s website at http://www.ifpri.org/publications/results/taxonomy%3A468. 2 IFPRI Discussion Papers contain preliminary material and research results. They have been peer reviewed, but have not been subject to a formal external review via IFPRI’s Publications Review Committee. They are circulated in order to stimulate discussion and critical comment; any opinions expressed are those of the author(s) and do not necessarily reflect the policies or opinions of IFPRI. Copyright 2010 International Food Policy Research Institute. All rights reserved. Sections of this material may be reproduced for personal and not-for-profit use without the express written permission of but with acknowledgment to IFPRI. To reproduce the material contained herein for profit or commercial use requires express written permission. To obtain permission, contact the Communications Division at ifpri-copyright@cgiar.org. http://www.ifpri.org/publications/results/taxonomy%3A468� iii Contents Abstract v Acknowledgments vi 1. Introduction 1 2. Survey Area and Data Description 3 3. Estimating Treatment Effects Using Nonexperimental Data 6 4. Empirical Results 9 5. Conclusion 13 References 15 iv List of Tables 1. Distribution of irrigation among agricultural households in Niafunke Cercle, 2006 4 2. Household descriptive statistics, by irrigation status in Niafunke Cercle, 2006 5 3. Household and village characteristics, by irrigation status 5 4. Determinants of irrigation access 9 5. Average treatment effects on the treated—All irrigation types 10 6. Rosenbaum bounds of kernel impact estimates—All irrigation types 10 7. Average treatment effects on the treated, by small and large irrigation types 12 8. Rosenbaum bounds of kernel impact estimates, by small and large irrigation types 12 v ABSTRACT This paper investigates differences in household production and consumption among small- and large- scale irrigators to assess whether the scale of an irrigation project increases household welfare in Mali. Much of the evidence of the impact of irrigation does not use counterfactual analysis to estimate such impact or distinguish between the scale of the irrigation projects to be evaluated. In the dataset collected by the author, both a large-scale irrigation project and small-scale projects are used to construct counterfactual groups. Propensity score matching is used to estimate the average treatment effect on the treated for small and large irrigators relative to nonirrigators on agricultural production, agricultural income, and consumption per capita. Small-scale irrigation has a larger effect on agricultural production and agricultural income than large-scale irrigation, but large-scale irrigation has a larger effect on consumption per capita. This suggests that market integration and nonfarm externalities are important in realizing gains in agricultural surplus from irrigation. Keywords: irrigation, program evaluation, Mali vi ACKNOWLEDGMENTS Financial support for the 2006 data collection was provided by the Projet du Développement de la Zone Lacustre–Phase II” of the International Fund for Agricultural Development (IFAD), the Programme Mali- Nord (GTZ/KfW), the Strategies and Analysis for Growth and Access program (Cornell University and Clark Atlanta University), the Einaudi Center at Cornell University, and the International Food Policy Research Institute (IFPRI). Funding for data collection in 1997/98 rounds was provided by IFAD and U.S. Agency for International Development/Mali. These data could not have been collected without the substantial assistance of Karin Dillon (2006), Sidi Guindo (1997/98), Abdourhamane Maiga (1997/98, 2006), Nouhou Idrissa Maiga, Mamadou Nadio, a dedicated team of survey enumerators, and the helpful cooperation of the residents of the Zone Lacustre. The author gratefully acknowledges the valuable comments and suggestions from Christopher Barrett, Kaushik Basu, Daniel Gilligan, John Hoddinott, David Sahn, and Xiaobo Zhang on earlier versions of this paper during his dissertation as well as several anonymous reviewers. Rachel Gordon provided excellent research assistance. All errors remain the responsibility of the author. 1 1. INTRODUCTION Four percent of total cultivated area is under irrigation in Sub-Saharan Africa (World Bank 2007), which suggests the potential effect that increased irrigation investment may have on agricultural productivity and poverty by expanding the number of total irrigated hectares. However, irrigation projects are not all uniformly successful over the long term and differ along critical dimensions. One focus of debate is whether the scale of the project is correlated with higher productivity and poverty reduction. Lankford (2005) estimates that 44 percent of African irrigated systems comprise less than 100 hectares and 56 percent take up more than 100 hectares, with increasing rates of investment since the 1980s being placed on increasing the number of small-scale irrigation projects. Lipton, Litchfield, and Faurès (2003) identify several mechanisms through which scale differences may affect farmers’ production and consumption. One such mechanism is increased market integration in both labor and input markets caused by economies of scale in larger projects relative to smaller projects. Another mechanism may be the increased spillover of agricultural knowledge or technology when a larger conglomerate of farmers forms in a particular location. Criticisms of large-scale projects as inefficient and wasteful have been countered by some evidence that despite inefficiency, large-scale projects increase food security (Attwood 2005). Smaller-scale irrigation projects may also afford benefits, especially with respect to efficiency. One of the largest costs of larger-scale interventions is the increased coordination costs of operation and maintenance, which pose governance and local capacity to be critical ingredients to success or failure. Farmers working in small irrigation systems may benefit from lower costs of participation as well as more influence over water management. Rosegrant and Perez (1997) argue in their review that farmer participation in the irrigation scheme’s design and management is a critical determinant of success, regardless of the scale of the project. This paper tries to quantify the differences in welfare between farmers that have access to large- scale irrigation, access to small-scale irrigation, or no access to irrigation. We define “small scale” in our sample of village irrigation schemes as those that are 50 hectares or less, and we include a single large- scale project of more than 300 hectares in the sample.1 Two primary identification problems exist for which the econometric strategy attempts to control in measuring the impact of irrigation. The first is the endogeneity of access to irrigation due to nonrandom program placement. This bias occurs when intentional or implicit targeting rules are used to allocate projects to villages. These village-level characteristics are likely to correlate with the explanatory variables if the programs are allocated either to highly productive areas to ensure program success or to less productive areas to target the poor. In either case, estimates of the project impact derived from outcome indicators will contain upward or downward bias, respectively, due to the group’s pretreatment characteristics. Most studies of the impact of irrigation on poverty do not rely on matched treatment and comparison groups or randomized allocation of access to irrigation (see Hussain 2007b for a review), despite the likely correlation of irrigation project placement and farmer selection into programs with observable village and household characteristics, such as existing village infrastructure, market access, household education, and assets. Unobservable characteristics such as political influence or farmer ability may also be correlated with access and outcome variables. A second source of bias in program impact estimates is the selection bias due to nonmandatory program participation. Access to irrigation is likely to be correlated with household characteristics such as education, which may influence the likelihood of technology adoption or the ability of a farmer to lobby on behalf of his or her community for the program intervention and access to water, which is a necessary condition for motorized irrigation. Program placement is not random. Unless the allocation of program interventions was intentionally randomized or can be viewed as a natural experiment, the distribution of observable and unobservable village and household characteristics between treatment (with irrigation) and comparison groups (without irrigation) may not be statistically equivalent. 1 This irrigation scheme is one of the largest in Mali after the Office du Niger and a large scheme proximate to Tombouctou. 2 To quantify the impact of irrigation on household production and consumption, we use propensity-score-matching techniques to construct treatment and comparison groups because the data were not generated as a random experiment, drawing on a growing theoretical and empirical literature (Heckman, Ichimura, and Todd 1997; Dehejia and Wahba 2002; Jalan and Ravallion 2003; Bertrand, Duflo, and Mullainathan 2004; Smith and Todd 2005; Dehejia 2005; Gilligan and Hoddinott 2007). As a robustness check for the propensity score matching, four different estimators (nearest-neighbor matching, matching with 10 closest neighbors, kernel Epanechnikov, and local linear matching) are employed to produce point estimates. The paper cannot completely rule out whether unobservable characteristics are significant sources of bias. However, a rich set of observable household and village characteristics facilitates the construction of legitimate treatment and comparison groups and minimizes concerns with respect to selection on unobservables in the context of this study. Further, we estimate Rosenbaum bounds, which provide sensitivity analysis of our results to increased levels of unobservables on the statistical significance and confidence intervals associated with the key empirical results. We find that the impacts on agricultural production of the two types of irrigation systems are quite similar, but the effects of small-scale irrigation are larger on both production per hectare and agricultural incomes. The effects on consumption per capita are less clear. Pooled estimates for both types of irrigators yield statistically significant effects of irrigation on consumption per capita. However, estimates of the effects of large-scale irrigation are much larger than those for small-scale irrigation, despite the larger production effects of small-scale irrigators. This discrepancy could be explained by the effect of greater population densities at the large-scale irrigation scheme that generate positive externalities through increased nonfarm employment opportunities from irrigators’ investment of agricultural surplus or the potentially reduced costs of agricultural production from integrated input markets, which would increase farmers’ agricultural profits. The analysis in this paper is based on field research, which uses the author’s multi-topic household survey in northern Mali in 2006. Northern Mali is an arid area that possesses few of the preconditions for agricultural growth, such as good soil quality, frequent and adequate rainfall, moderate temperatures, and sufficient infrastructure. Most international attention has been paid to the Office du Niger irrigation scheme2 The second section of the paper describes the data and survey area, as well as describing differences in household and village characteristics for irrigators and nonirrigators. The third section presents the econometric strategy to estimate the treatment effects using nonexperimental data. Section 4 presents the estimates of the effects of irrigation on agricultural production and household consumption among households with access to large-scale irrigation, farmers with access to small-scale irrigation, and farmers with no access to irrigation. The last section concludes. in the Segou region of Mali, located in central Mali. However, small-scale village-level projects (30–40 hectares) and larger-scale projects (300 hectares or larger) dot the Inner Niger Delta and areas north of Tombouctou into the Saharan Desert, which is one of the poorest regions of Mali and an area hardest hit by the Sahel droughts. The impacts of these two types of irrigation projects are compared in this paper. These irrigation projects are not investments by farmers in boreholes or wells for irrigating their personal fields; rather they are community-level investments that result from household, village, and international organization partnerships. In northern Mali, borehole or well investment by households is constrained not only by farmers’ credit constraints but primarily by the arid climate, which results in a low water table and increased difficulty in constructing wells. The types of irrigation projects considered in this paper are those that use motorized pumps to redistribute water from the Niger River throughout a small-scale, canal irrigation system installed as part of international donor– financed projects. 2 The Office du Niger was originally constructed in 1932 as a gravity irrigation scheme during French colonialism. Although the scheme was widely regarded as a failure into the early 1970s, a restructuring of the Office du Niger from 1979 to 1996 has improved the technical efficiency of the institution and increased grain yields for farmers (Couture, Lavigne Delville, and Spinat 2002). 3 2. SURVEY AREA AND DATA DESCRIPTION The data used to conduct the analysis in this paper were collected as part of the Poverty and Food Security Household Survey in Northern Mali 2006 (Etude sur la Pauvreté et la Sécurité Alimentaire au Nord Mali 2006) conducted by the author. This multi-topic household survey was implemented to study household consumption and agricultural production decisions in northern Mali in the cercles (states) of Niafunke, Goundam, Dire, Tombouctou, Rharous, and Bourem. The dataset contains a cross-section of 2,100 households.3 Agroecological conditions throughout most areas of northern Mali would not be favorable for agricultural production without the Niger River, which is the dominant ecological resource in the region. The Inner Niger Delta is a potentially productive agricultural area in which flooding from the Niger augments water levels in temporary and permanent lakes and ponds, as well as smaller streams and tributaries. Soil quality deteriorates rapidly as the distance from the river increases. Farmers harness water resources through motorized pump irrigation and the use of water-recession agriculture around the lakes and streams flooded by the Niger River itself, as its water levels decrease seasonally. Rainfed agriculture is also common, but subject to considerable climatic risk. The Saharan zone receives less than 150 millimeters of rainfall per year. This varies starkly with the Sahelian zone, which receives 200–600 millimeters per year, and the south of Mali, which can receive as much as 600–1,200 millimeters per year (Christiaensen 1998). This paper uses exclusively a subsample of the cross-section of 651 agricultural households in the cercle of Niafunke, where both small- and large-scale projects are found. By restricting the analysis to this cercle, we are able to control for variation across cercles that may affect output and welfare levels of households. Further details related to the documentation of the survey design and implementation can be found in Dillon (2005). Table 1 illustrates the distribution of irrigation among agricultural households in Niafunke Cercle.4 In the survey area, access to irrigation is facilitated by an international organization that provides motorized pumps to villages at no charge or significantly discounted prices; those costs are then reimbursed over an extended period of time and with the condition that villages contribute labor for the construction of the irrigation infrastructure, undertake pump maintenance, and provide fuel for the pump’s operation. Irrigation is primarily used for rice cultivation, rather than sorghum or millet, two traditionally rainfed crops. Rice production that relies on irrigation is cropped once per agricultural cycle. Internationally, increased agricultural production from a dual cropping system is commonly cited as a benefit of irrigation. However, the arid context in northern Mali does not allow a second cropping season after rice cultivation. This is because temperatures in the dry season are high (in excess of 40º Celsius) and receding river levels make it impossible to provide the minimum water required for rice plants or Fifty-two percent of the agricultural households in the subsample had access to irrigation. Small- scale irrigation was the dominant type of irrigation, representing 43.5 percent of farmers; only 8.6 percent had access to large-scale irrigation. The reason for this difference is that small-scale irrigation interventions are much less expensive to build. There is one large-scale irrigation scheme in Niafunke Cercle, whereas many smaller-scale schemes have been installed in rural villages. Both large- and small- scale systems are community-based with farmers playing an integral role on management committees, although governance in the large-scale scheme is heavily influenced by paid employees who manage day- to-day maintenance and water distribution. Both large- and small-scale schemes also provide technical assistance and extension advice to farmers on the timing of planting and the proper application of fertilizers, among other subjects. 3 Dillon (2008) investigates the effects of small-scale irrigation on a subsample of the households from this cross-section, which were tracked from 1997. 4 Households that responded that they did not plant any crops in the previous agricultural season are excluded from the analysis to ensure the quality of matches in creating treatment and comparison groups. Nonagricultural households engage in a variety of activities, including raising livestock, conducting commerce, and working in the public sector. 4 other cash crops. The benefits of irrigation in northern Mali are accrued through increased water supply and control during the primary agricultural season. Table 1. Distribution of irrigation among agricultural households in Niafunke Cercle, 2006 Access to irrigation N Percentage None 312 47.9 Any irrigation 52.1 Small 283 43.5 Large 56 8.6 Number of households 651 Data Source: Author’s calculations from the Poverty and Food Security Household Survey in Northern Mali, 2006. Note: Irrigation projects are classified as small when the total hectares per project are 50 hectares or less. The large irrigation scheme comprises more than 300 hectares in total size. Table 2 contains agricultural production, per capita household consumption, and agricultural income from crop production descriptive statistics. Agricultural households without irrigation cultivate 2.4 hectares, on average, with production of 2,042 kilograms per hectare. Households without irrigation have annual consumption per capita5 of 170,109 FCFA.6 Table 3 investigates the household characteristics among nonirrigating and irrigating agricultural households. Household demographics, including age of the head of household, education levels, and household size and composition, are similar across nonirrigators, small-scale irrigators, and large-scale irrigators. Nonirrigators have lower levels of farm capital (25,346 FCFA) and durables (115,636 FCFA) than irrigators. Small-scale irrigators have slightly higher levels of farm capital (30,302 FCFA) relative to large-scale irrigators (27,640 FCFA), but much lower levels of durables (207,093 FCFA versus 257,224 FCFA). Village characteristics of irrigators and nonirrigators, including distance to a road, illustrate differences across villages. Significant differences exist between irrigators and nonirrigators in road access. Noting those differences, we restrict matches to households in similar villages and with similar household characteristics, which improves the quality of the matches and increases the precision of impact estimates. In the next section, we describe how treatment and comparison groups are constructed by propensity score matching using these observable household and village characteristics. In comparison, households employing small- scale irrigation cultivate 2.85 hectares of which 1.04 hectares are rice crops, whereas large-scale irrigators cultivate rice almost exclusively on 0.63 hectares of land. Small-scale irrigators cultivate more hectares than large-scale irrigators, but their agricultural production is diversified among rainfed and irrigated systems. Large-scale irrigators farm entirely in the irrigated system. Because large-scale irrigators are located in more densely populated areas, additional nonfarm activities are available to them whose returns may be larger than additional cultivation in rainfed agriculture. Despite differences in land cultivated, production per hectare varies between small irrigators (4,496 kilograms/hectare) and large irrigators (3,819 kilograms/hectare), but it is much higher than nonirrigators’ production per hectare. Consumption per capita is also larger among irrigators than nonirrigators. However, higher production per hectare among small-scale irrigators has not increased their per capita consumption (201,930 FCFA) relative to large-scale irrigators (322,869 FCFA). Agricultural income is also included as an outcome variable as differences in the weight of crops produced across farms may bias estimates. When agricultural income is calculated, nonirrigating households have uniformly lower income levels (189,437 FCFA) than irrigating households. Compared with large-scale irrigators, small-scale irrigators report higher income levels with differences of approximately 60,000 FCFA, but the standard deviation of income for small irrigators is also greater than that of large-scale irrigators. 5 Total household consumption is the annualized household consumption aggregate calculated from men’s and women’s assets and nonfood and food expenditures following Deaton and Zaidi (2002) US$1 = 450 FCFA. 6 FCFA is the unit of currency in Mali, the franc of the Communauté financière d’Afrique (CFA). 5 Table 2. Household descriptive statistics, by irrigation status in Niafunke Cercle, 2006 None Small Large Total household hectares 2.4 2.85 0.63 (4.1) (9.79) (.46) Total hectares of rice 1.04 0.63 (1.59) (.46) Production (kilograms) per hectare 2,042 4,496 3,819 (3,208) (3,210) (1,751) Consumption per capita (FCFA) 170,109 201,930 322,869 (102,109) (124,470) (144,201) Agricultural income (all crops) 189,437 355,082 295,311 (223,587) (239,382) (186,942) Data source: Author’s calculations from the Poverty and Food Security Household Survey in Northern Mali, 2006. Notes: Standard deviations are in parentheses. Irrigation projects are classified as small when the total hectares per project are 50 hectares or less. The large irrigation scheme comprises more than 300 hectares in total size. Table 3. Household and village characteristics, by irrigation status None Small Large Household size 6.8 6.9 6.8 (3.8) (4.1) (2.7) Number of males > 17 in household 2.0 2.0 1.8 (1.2) (1.5) (.88) Number of females > 17 in household 1.9 2.2 1.9 (1.1) (1.8) (1.3) Number of children < 17 in household 3.0 2.7 3.1 (2.5) (2.0) (1.9) Age of household head 50.7 49.7 48.9 (38.5) (14.1) (14.7) Ethnicity (1 = Peulh) 0.23 0.11 0 (0.42) (0.31) Head’s education (1 = any education) .02 .05 0 (.12) (.22) Spouse’s education (1 = any education) .01 .02 0 (.10) (.13) Farm capital (FCFA) 25,346 30,302 27,640 (27,996) (31,502) (23,036) Durables 115,636 207,093 257,224 (128,330) (225,229) (220,737) Road access Less than 1 kilometer 21.0 17.1 100 Between 1 and 10 kilometers 50.8 30.0 0 Between 11 and 20 kilometers 16.0 44.2 0 Greater than 20 kilometers 12.2 8.7 0 Data source: Author’s calculations from the Poverty and Food Security Household Survey in Northern Mali, 2006 and IFPRI (1998). Notes: Standard deviations are in parentheses. Irrigation projects are classified as small when the total hectares per project are 50 hectares or less. The large irrigation scheme comprises more than 300 hectares in total size. 6 3. ESTIMATING TREATMENT EFFECTS USING NONEXPERIMENTAL DATA The Evaluation Problem To estimate the impact of irrigation, recognizing the problem of selection bias between treatment and comparison groups, one ideally wants to estimate Δ = Yt 1 − Yt 0, which is the difference of the outcome variable of interest at time t between a state where the household receives treatment and a state where the household does not receive treatment, denoted by the superscripts 1 and 0. The average treatment effect (ATE) is estimated such that ATE = E(Yt 1 – Yt 0). However, it is impossible to estimate Δ in this way because a household exists in one of two mutually exclusive states, either with access to irrigation or without irrigation. The evaluation problem is one of missing data, due to the impossibility of assigning households to both treatment and control groups (Heckman, Ichimura, and Todd 1997; Dehejia and Wahba 2002; Smith and Todd 2005; Dehejia 2005). Randomized experiments enable the construction of the counterfactual E(Yt 0|X,T = 1) by randomly allocating treatment ensuring that E(Yt 0|X,T = 1) = E(Yt 0|X,T = 0). However, randomized experiments are not always possible or plausibly implemented to ensure the absence of selection bias. Hence, we estimate the average treatment effect on the treated households (ATT), given a vector household characteristic X: ATT = E(Δ|X,T = 1) = E(Yt 1 – Yt 0|X,T = 1) = E(Yt 1|X,T = 1) – E(Yt 0|X,T = 1), (1) where, because E(Yt 0|X,T = 1) is unobservable, it is assumed that E(Yt 0|X,T = 1) = E(Yt 0|X,T = 0). An important body of literature has tested the assumptions of nonexperimental estimators against experimental benchmarks and against each other to verify whether identification assumptions can be plausibly asserted (see, for example, Heckman, Ichimura, and Todd 1997; Dehejia and Wahba 2002; Smith and Todd 2005; Dehejia 2005; Diaz and Handa 2006). This literature has found that nonexperimental estimators can perform well if the set of observable characteristics is rich enough to create valid treatment and comparison groups. Critical to the assessment of impact assessment results for nonexperimental estimators is a series of robustness checks that we will describe below. First, we use different estimators to ensure that results are robust to the method of calculating the ATT. Second, we calculate Rosenbaum bounds, which provide some evidence regarding the assumptions implicit in equation (1) and the sensitivity of the results to unobservables. Propensity Score Matching To estimate the effects of irrigation, propensity scores are used to match households with similar observable characteristics, varying only the treatment, which in this case is access to irrigation. The propensity score is simply the probability that a household has access to the treatment, P(T = 1|X,V). Propensity scores are estimated using a probit model in which a vector of household characteristics X and village characteristics V is regressed on P, a household’s access to irrigation, to obtain predictions of household propensity scores, in which observations for all variables are denoted as households (h) within villages (v). P* v,h = βXv,h + γVv,h + εv,h. (2) To estimate equation (2), household variables are used as controls, including household size, household assets such as household durables, age of the household head, an education indicator for the household head and his or her spouse, and farm capital. Village characteristics include indicators for distance to the nearest road in the commune of the observed household. The household and village variables used in the specification to generate the propensity scores are selected based on the inclusion of the largest set of variables that satisfies the balancing property, following Dehejia and Wahba (2002). That is, the treatment and comparison observations are tested to 7 ensure equality of observables across different propensity score groupings, so that there is an appropriate distribution of characteristics in each grouping of propensity scores. Estimates of impact are calculated over the common support of the distribution of propensity scores for irrigators and nonirrigators. The assumption that 0 < P(T = 1|X,V) < 1 is also satisfied in our sample, and the top and bottom 5 percent of the sample have been trimmed, following Smith and Todd (2005). Robustness Checks Four matching estimators are employed as robustness checks. The estimators are a single nearest- neighbor matching estimator with replacement, a nearest-neighbor estimator using the 10 nearest neighbors with replacement, an Epanechnikov kernel matching estimator, and a local linear matching estimator. Both nearest-neighbor matching estimators are constructed with replacement of observations after they are matched. Replacement increases the quality of the matches by using more information to construct the counterfactual, but it increases the variance of the estimator by reducing the number of nonparticipant observations used in the comparison group. A nearest-neighbor estimator using 10 nearest neighbors with replacement increases the quality of matches but with the trade-off that the variance of the estimator is increased. The third estimator employed is the Epanechnikov kernel matching estimator for the average treatment effect on the treated, which is constructed such that 0 1 ( ) ( ) ( ) 1 [ ]( ) ( )( ) j i j j C n t i T k i k C n P X P X Y K aATT Y P X P Xn K a ∈ ∈ ∈ − ∑ = −∑ − ∑ , (3) where T is the treatment group, K is the kernel function, C is the comparison group, and an is the bandwidth parameter proposed in Heckman, Ichimura, and Todd (1997) and Heckman et al. (1998) for treatment observation i and control observation j. The advantage of this estimator is that it gives relatively higher weight to “closer” matches and lower weight to matches that are less close in the calculation of the average treatment effect on the treated. The last estimator is the local linear matching estimator, which is a generalized version of the kernel estimator. The advantage of the local linear matching estimator is that it is generally more robust to data design densities and has a faster rate of convergence at the boundary points (Smith and Todd 2005). The robustness checks ensure that results are insensitive to the estimator chosen, but it is possible, despite balancing propensity scores and ensuring that estimates are made on the common support of the distribution between treatment and comparison observations, that unobservables could bias impact estimates. Either individual unobservable heterogeneity such as farming ability or political influence could bias estimates based on selection of farmers into the irrigation scheme. Likewise, village unobservable heterogeneity such as land quality or organizational capacity to manage the irrigation scheme could bias impact estimates. Rosenbaum bounds (2002) were proposed to simulate the effect of unobserved heterogeneity on impact estimates. If the probability of participation is P(xi) = P(Ti = 1|xi) = F(βxi + γui), the likelihood of participation is determined by the individual’s observed characteristics xi and unobserved characteristics ui. If unobservables have no effect on the probability of participation in irrigation, then γ = 0. If unobservables do affect the probability of participation (γ ≠ 0), then individuals i and j with the same observable characteristics have different probabilities of selecting into program participation. Using a logistic distributional assumption, we can derive the odds ratio that an individual participates in the irrigation scheme (Rosenbaum 2002): 8 ( ) ( )1 ( ) exp( )1 ( ) exp[ ( )]( ) ( )1 ( ) exp( ) 1 ( ) i i j j ji i j j j i i i j P x P x P x x uP x u uP x P x P x x u P x β γ γ β γ − +− = = = − − + − . (4) If matching is properly implemented, differences in observed characteristics are removed from the equation and there is no effect of unobservables on impact estimates due to selection bias. If there are differences in unobserved variables (ui - uj between individuals or if unobserved variables have a nonzero probability (γ ≠ 0) of influencing selection, then estimates are likely to be biased. By using sensitivity analysis, we can estimate the effect of variation in γ on the impact estimates, providing some evidence about the validity of the assumptions on which the nonexperimental estimates are based. 9 4. EMPIRICAL RESULTS Before presenting the main results of the paper, the results from the estimation of propensity scores from the probit regression that estimates the likelihood of a household adopting irrigation are presented in Table 4. A rich set of covariates—including individual, household, and village variables—is used to estimate this relationship, which satisfies the balancing property and which has sufficient overlap of the common support between the distribution of propensity scores for irrigators and nonirrigators. Using these covariate estimates, propensity scores are constructed to ensure matching on these observable characteristics of households in both the treatment and comparison groups. Table 4. Determinants of irrigation access Ir r igation (1 = yes) Number of females > 17 in household -0.053 (0.083) Number of males > 17 in household 0.127* (0.072) Number of children < 17 in household -0.001 (0.068) Ln farm assets in FCFA 0.112* (0.057) Ln total hectares cultivated -0.096** (0.049) Head’s education (1 = yes) 0.809* (0.424) Spouse’s education (1 = yes) 0.437 (0.678) Ln household size -0.165 (0.449) Ln household durable assets in FCFA 0.174*** (0.055) Ln age of household head 0.098 (0.255) Peulh ethnicity (1 = yes) -0.703*** (0.186) Road access within village -0.583*** (0.187) Road access within 1–10 kilometers of village -0.947*** (0.164) Constant -2.724** (1.268) Number of observations 428 Data source: Author’s calculations from the Poverty and Food Security Household Survey in Northern Mali, 2006 and IFPRI (1998). Notes: Standard errors are in parentheses. *** p < 0.01. ** p < 0.05. * p < 0.1. The results of the ATT estimates on production per hectare, consumption per capita, and agricultural income are presented first for irrigators relative to nonirrigating agricultural households in Table 5, whereas Table 7 presents the results disaggregated by large- and small-scale irrigators. Consistent across both sets of results is that irrigation has a significant effect on household production per 10 hectare. In Table 5, ATT estimates across nearest-neighbor, kernel, and local linear matching estimators yield a statistically significant estimate between 1.96 and 2.1 metric tons per hectare for irrigated households. Measuring production using agricultural income produces estimates that vary between 125,600 and 132,469 FCFA. Irrigated households also have higher statistically significant consumption per capita with estimates between 40,113 and 49,877 FCFA. Despite employing different estimators, point estimates of the impact of irrigation do not vary substantially between estimators. Table 5. Average treatment effects on the treated—All irrigation types Any ir r igation Nearest neighbor Nearest 10 neighbors Kernel Local linear matching Production (kilograms) per hectare 2,014 2,101 1,972 1,964 (590)*** (470)*** (468)*** (505)*** Annualized per capita consumption (FCFA) 49,877 42,525 40,113 44,403 (16,219)*** (12,577)*** (12,514)*** (13,877)*** Agricultural income (all crops) 125,600 131,406 127,991 132,469 (43,102)*** (31,584)*** (31,301)*** (34,579)*** N 425 425 425 425 Data source: Author’s calculations from the Poverty and Food Security Household Survey in Northern Mali, 2006 and IFPRI (1998). Notes: The number of observations to estimate the average treatment effect on the treated households (ATT) will not be the same as the number of observations in the dataset, as we trim 5 percent of the observations on both tails of the distribution and impose that matches are made along the common support of the distribution of irrigators’ and nonirrigators’ propensity scores. All standard errors were bootstrapped with 1,000 replications. *** p < 0.01. ** p < 0.05. * p < 0.1. To explore the sensitivity of these estimates to unobservables, Rosenbaum bounds are estimated for the kernel estimates. Table 6 shows the results. By increasing the size of gamma, we can estimate the upper bound significance level of the point estimate and the confidence interval if we scale the influence of the unobservables on the point estimates. Both point estimates of the measures of production (production per hectare and agricultural income) are insensitive to doubling the effect of unobservables on the point estimates at the 10-percent level of statistical significance. However, the consumption per capita estimates are sensitive to increases in gamma. This suggests that irrigation has a strong effect on production, but its implications for consumption per capita may be driven by alternative unobservables, such as market integration, individual ability, or the management of irrigation schemes that could affect the costs to the household. Unobservable factors such as market integration or increased irrigation costs would be reflected in lower per capita consumption, but not necessarily in agricultural production. Table 6. Rosenbaum bounds of kernel impact estimates—All irrigation types Agr icultural production Consumption per capita Agr icultural income (FCFA) Gamma Upper bound significance level Confidence interval Upper bound significance level Confidence interval Upper bound significance level Confidence interval 1 0.00 714 1,441 0.00 7,838 40,647 0.00 71,983 139,209 2 0.01 56 2,628 0.91 -22,779 88,212 0.01 7,653 229,337 3 0.47 -288 3,675 1 -38,767 118,289 0.37 -25,169 285,299 4 0.94 -514 4,537 1 -48,684 139,213 0.87 -46,672 325,034 5 1.00 -682 5,239 1 -55,626 155,602 0.99 -62,129 352,917 Data source: Author’s calculations from the Poverty and Food Security Household Survey in Northern Mali, 2006 and IFPRI (1998). 11 In Table 7, differences in ATT estimates between large- and small-scale irrigators relative to nonirrigating agricultural households are assessed. Small-scale irrigation yields a statistically significant impact estimate of 2.1–2.4 metric tons per hectare, whereas large-scale irrigation has a point estimate of 941 kilograms to 1.1 metric tons per hectare. With respect to agricultural income, estimates of the effect of small-scale irrigation were also generally higher, with estimates between 133,283 FCFA and 142,527 FCFA. Estimates of the effect of large-scale irrigation ranged from 112,774 FCFA to 153,811 FCFA and were more variable than those of small-scale irrigation. The upper bound of the large-scale irrigation point estimate on agricultural income is derived from the single nearest-neighbor estimator. Point estimates estimated with the other three estimators including the 10 nearest neighbors, kernel, and local linear matching may be more precise due to the differences in estimators noted earlier. In Table 8, we investigate the sensitivity of these results for both subsamples of data using the Rosenbaum bounds. Both large- and small-scale production per hectare and agricultural income estimates are robust to doubling the level of unobservables at the 10-percent level of statistical significance. Estimates of the effect of small-scale irrigation on consumption per capita are less precise than in the pooled estimates including all irrigators. No statistically significant effects on consumption per capita were estimated. These consumption estimates are also not robust to increases in the level of unobservables (Table 8). However, we find significant effects of large-scale irrigation on consumption per capita. Point estimates of the effect of large-scale irrigation on consumption per capita range between 135,483 FCFA and 154,840 FCFA. These estimates of impact on consumption per capita are larger than the overall contribution of household income due to agricultural production. This suggests that one of the spillover effects of access to large-scale irrigation may be the investment of agricultural surplus in nonfarm activities and the development of labor markets for agricultural and nonagricultural employment that results from larger population concentrations. Although the point estimates for small-scale irrigators suggest no effect of irrigation on consumption, this set of estimates differs from point estimates of the effect of small-scale irrigation on consumption found in Dillon (2008), who finds similar effects on agricultural production, but statistically significant effects on consumption. Dillon (2008) uses a difference-in-differences estimator with a small panel of small-scale irrigators over a nine-year period in the same area of Mali. Due to those results, the effects on consumption found here should be interpreted with caution, for two reasons. First, the panel data are likely to produce a more precise point estimate than the cross-section of data because panel data estimate control for household fixed effects. Second, consumption variables are likely influenced by different types of unobservables that are more difficult to account for in cross-sections, namely the effect of market integration and labor market opportunities that may differ across villages where large- and small-scale projects are installed. For example, more remote villages with fewer market opportunities may not have nonfarm investment opportunities to reinvest agricultural surpluses. 12 Table 7. Average treatment effects on the treated, by small and large irrigation types Small ir r igators Large ir r igator s Nearest neighbor Nearest 10 neighbors Kernel Local linear matching Nearest neighbor Nearest 10 neighbors Kernel Local linear matching Production (kilograms) per hectare 2,442 2,096 2,126 2,248 1,097 1,080 941 971 (770)*** (614)*** (591)*** (638)*** (580)* (403)*** (373)** (408)** Annualized per capita consumption in FCFA 5,826 8,059 10,545 12,216 154,253 150,459 135,483 154,840 (18,750) (14,217) (13,857) (15,163) (31,879)*** (23,530)*** (23,455)*** (25,864)*** Agricultural income (all crops) 140,247 133,283 142,148 142,527 153,811 122,720 125,507 112,774 (48,243)*** (37,200)*** (37,717)*** (41,874)*** (62,968)** (40,060)*** (40,962)*** (47,131)** N 378 378 378 378 244 244 244 244 Data source: Author’s calculations from the Poverty and Food Security Household Survey in Northern Mali, 2006 and IFPRI (1998). Notes: The number of observations to estimate the average treatment effect on the treated households (ATT) will not be the same as the number of observations in the dataset, as we trim 5 percent of the observations on both tails of the distribution and impose that matches are made along the common support of the distribution of irrigators’ and nonirrigators’ propensity scores. All standard errors were bootstrapped with 1,000 replications. *** p < 0.01. ** p < 0.05. * p < 0.1. Table 8. Rosenbaum bounds of kernel impact estimates, by small and large irrigation types Agr icultural production Consumption per capita Agr icultural income (FCFA) Type Gamma Upper bound significance level Confidence interval Upper bound significance level Confidence interval Upper bound significance level Confidence interval Small irrigators 1 0.00 810 1,777 0.77 -19,253 9,958 0.00 78,651 166,001 2 0.03 -29 3,163 1.00 -45,158 47,848 0.01 9,919 264,164 3 0.57 -438 4,142 1.00 -58,426 76,039 0.27 -25,986 319,872 4 0.95 -702 4,939 1.00 -67,656 98,301 0.74 -51,302 355,991 5 1.00 -926 5,646 1.00 -75,017 114,827 0.95 -70,769 386,659 Large irrigators 1 0.00 231 1,153 0.00 86,223 173,572 0.00 50,640 167,476 2 0.09 -136 1,971 0.00 42,648 226,890 0.02 3,307 257,542 3 0.37 -303 2,550 0.01 15,263 262,887 0.13 -25,041 307,596 4 0.66 -435 3,105 0.03 -4,701 292,241 0.31 -42,792 343,019 5 0.84 -595 3,852 0.07 -21,136 315,660 0.50 -57,314 376,679 Data source: Author’s calculations from the Poverty and Food Security Household Survey in Northern Mali, 2006 and IFPRI (1998). 13 5. CONCLUSION This paper provides evidence about the impact of irrigation projects, which differs according to the scale of the project. Whereas trends in investment have tended toward smaller-scale investment due to coordination problems in larger-scale irrigation schemes, the direct impacts on household production, consumption, and income differentiated by the scale of the project are not well established. Lipton, Litchfield, and Faurès (2003) identify several pathways through which scale may potentially lead to differential impacts on households, including gains in efficiency, higher degrees of market integration in both output and input markets, and spillovers of agricultural knowledge among farmers. The propensity score matching used to estimate impact controls for observable village and household characteristics between households in different irrigation schemes to increase the quality of matches among agricultural households with similar initial characteristics. After controlling for household and village characteristics that may influence outcomes for households, the analysis shows substantial gains in agricultural production and agricultural income for both large- and small-scale irrigation, with substantially larger benefits accruing to small-scale irrigators. However, our results on the effects of irrigation on consumption per capita are mixed. Whereas our results show an overall positive effect of consumption per capita on irrigators in comparison with nonirrigators, the effects of large-scale irrigation on consumption per capita are larger than those estimated with the subsample of small-scale irrigators. This suggests that the dynamics of adoption between the two types of irrigation schemes require more attention in future research. Ravallion and Chen (2005) estimate a large income effect of projects without corresponding increases in consumption per capita for development projects in China. They attribute this to the time horizon of the project as villagers may view certain development projects as a transitory income shock rather than a shift in permanent income. This may also be the case among irrigators if smaller-scale village irrigation projects are perceived as less durable over time. However, the differences in income and consumption effects on the two types of irrigation projects may also be a result of less market integration in small-scale irrigators’ communities in comparison with markets in large-scale irrigators’ communities. Agricultural surpluses may affect household welfare more slowly among small- scale irrigators if asset, input, and food markets are less integrated. It cannot be ruled out that these results may also be biased by household or individual unobservable characteristics, although the agricultural production and income impact estimates are robust to doubling the level of unobservables in the calculated Rosenbaum bounds. Evaluation methodologies, such as randomized experiments, instrumental variable methods (for example, Duflo and Pande 2007), regression discontinuity (Lee and Lemieux 2010), or structural estimation (Heckman 2010) provide interesting alternatives to propensity score matching. A priori, these methods have their own noted econometric advantages and drawbacks, which continue to generate considerable debate,7 but methods must also be matched to data and the development intervention to be evaluated. In the case of evaluating differences between large-scale irrigation and small-scale irrigation projects, it was difficult to identify a sufficient set of instruments that would identify both types of projects, as allocation of projects at the community level was predicated on geographic proximity to water as well as donor funding and priorities. Data limitations and program design precluded the use of structural estimation or a regression discontinuity approach.8 Notwithstanding these significant methodological debates in the evaluation literature, the debate on the appropriate scale for irrigation investments has important policy implications as well as significant unanswered questions for future research. On the research front, there is greater need for panel datasets that observe both small- and large-scale irrigators over time to better understand the dynamics of Further, it is unclear whether these methods would remove concerns about the role of unobservable characteristics on impact estimates. 7 See, for example, Deaton (2010), who critiques a reliance on randomized evaluations, the debate between Heckman (1997, 2010) and Angrist and Imbens (1999), and Imbens (2010) on the use of IV methods, 8 Pretreatment characteristics would also potentially improve the quality of propensity score matches in the paper’s analysis, but the programs included in this analysis did not field baseline surveys. 14 irrigation and provide the opportunity to employ econometric techniques to control for household or individual fixed effects. Our results provide some evidence that small-scale irrigators have higher agricultural production and income. However, the debate with respect to scale may be overly simplistic. The results in this paper do not give an unequivocal response in the case of northern Mali, nor are the projects in northern Mali likely to be identical to those in other countries. More evidence in other countries is necessary to build the evidence base. In many countries, both types of investments are necessary, depending on land and water resource availability, crop types, community organization, and targeting for poverty reduction. Small-scale and large-scale irrigation schemes have large effects on production, but the organization of the scheme will affect the level and distribution of household agricultural surpluses. This coincides with previous findings by Rosegrant and Perez (1997). Further, the costs of investing in irrigation projects differ by scale. Inocencio et al. (2007) review costs of large- and medium-scale projects and find that construction costs were almost twice as large in Sub-Saharan African schemes when compared with other irrigation schemes in other parts of the world. A better understanding of the mechanisms through which the scale of irrigation affects benefits and costs in future research can increase its efficiency and impact on household welfare. 15 REFERENCES Attwood, D. W. 2005. Big is ugly? How large-scale institutions prevent famines in western India. World Development 33 (12): 2067–2083. Angrist, J., and G. Imbens. 1999. Comment on James J. Heckman, “Instrumental variables: A study of implicit behavioral assumptions used in making program evaluations.” Journal of Human Resources 34 (4): 823-827. Bertrand, M., E. Duflo, and S. Mullainathan. 2004. How much should we trust differences-in-differences estimates? Quarterly Journal of Economics 119 (1): 249–275. Christiaensen, L. 1998. Mali case study: Methodology and descriptives. International Food Policy Research Institute, Washington, D.C. Photocopy. Couture, J.-L., P. Lavigne Delville, and J.-B. Spinat. 2002. Institutional innovations and water management in Office du Niger (1910–1999): The long failure and new success of a big irrigation scheme. Working Paper 29. Paris: Groupe de Recherche et d’Echanges Technologiques. Deaton, A. 2010. Instruments, randomization, and learning about development. Journal of Economic Literature 48 (2): 424–455. Deaton, A., and S. Zaidi. 2002. Guidelines for constructing consumption aggregates for welfare analysis. LSMS Working Paper 135. Washington, D.C.: World Bank. Dehejia, R. 2005. Practical propensity score matching: A reply to Smith and Todd. Journal of Econometrics 125 (1- 2): 355–364. Dehejia, R., and S. Wahba. 2002. Propensity score matching methods for nonexperimental causal studies. Review of Economics and Statistics 84 (1): 151–161. Diaz, J. J., and S. Handa. 2006. An assessment of propensity score matching as a nonexperimental impact estimator: Evidence from Mexico’s PROGRESA program. Journal of Human Resources 41 (2): 319–345. Dillon, A. 2005. The extension of the Zone Lacustre, Mali data set: Feasibility study. Report to the International Food Policy Research Institute and the Einaudi Center, Washington, D.C. ________. 2008. Access to irrigation and the escape from poverty: Evidence from northern Mali. IFPRI Discussion Paper 782. Washington, D.C.: International Food Policy Research Institute. Duflo, E., and R. Pande. 2007. Dams. Quarterly Journal of Economics 122 (2): 601-646. Gilligan, D. O., and J. Hoddinott. 2007. Is there persistence in the impact of emergency food aid? Evidence on consumption, food security, and assets in rural Ethiopia. American Journal of Agricultural Economics 89 (2): 225–242. Heckman, J. J. 1997. Instrumental variables: A study of implicit behavioral assumptions used in making program evaluations. Journal of Human Resources 32 (3): 441-462. ________. 2010. Building bridges between structural and program evaluation approaches to evaluating policy. Journal of Economic Literature 48 (2): 356–398. Heckman, J. J., H. Ichimura, and P. Todd. 1997. Matching as an econometric evaluation estimator: Evidence from evaluating a job training program. Review of Economic Studies 64: 605–654. Heckman, J. J., H. Ichimura, J. Smith, and P. Todd. 1998. Characterizing selection bias using experimental data. Econometrica 66 (5): 1017–1098. Hussain, I. 2007a. Direct and indirect benefits and potential disbenefits of irrigation: Evidence and lessons. Irrigation and Drainage 56: 179–194. ________. 2007b. Poverty reducing impacts of irrigation: Evidence and lessons. Irrigation and Drainage 56: 147– 164. 16 Imbens, G. 2010. Better LATE than nothing: Some comments on Deaton (2009) and Heckman and Urzua (2009). Journal of Economic Literature 48 (2): 399–423. Inocencio, A., M. Kikuchi, M. Tonosaki, A. Maruyama, D. Merrey, H. Sally, and I. de Jong. 2007. Costs and performance of irrigation projects: A comparison of Sub-Saharan Africa and other developing regions. IWMI Research Report 109. Colombo, Sri Lanka: International Water Management Institute. Jalan, J., and M. Ravallion. 2003. Estimating the benefit incidence of an antipoverty program by propensity score matching. Journal of Business and Economic Statistics 21 (1): 19–30. Lankford, B. A. 2005. Rural infrastructure to contribute to African agricultural development: The case of irrigation. Report for the Commission for Africa, Overseas Development Group, University of East Anglia, Norwich. Lee, S., and T. Lemieux. 2010. Regression discontinuity designs in economics. Journal of Economic Literature 48 (2): 281–355. Lipton, M., J. Litchfield, and J. M. Faurès. 2003. The effects of irrigation on poverty: A framework for analysis. Water Policy 5: 413–427. Ravallion, M., and S. Chen. 2005. Hidden impact? Household saving in response to a poor-area development project. Journal of Public Economics 89: 2183–2204. Rosegrant, M. W., and N. D. Perez. 1997. Water resources development in Africa: A review and synthesis of issues, potentials, and strategies for the future. Environment and Production Technology Division Discussion Paper 28. Washington, D.C.: International Food Policy Research Institute. Rosenbaum, P. R. 2002. Observational studies. 2nd ed. New York: Springer. Smith, J., and P. Todd. 2005. Does matching overcome LaLonde’s critique of nonexperimental estimators? Journal of Econometrics 125 (1–2): 303–353. World Bank. 2007. World development report 2008: Agriculture for development. . Accessed May 27, 2009. RECENT IFPRI DISCUSSION PAPERS For earlier discussion papers, please go to http://www.ifpri.org/publications/results/taxonomy%3A468. All discussion papers can be downloaded free of charge. 1021. Crop price indemnified loans for farmers: A pilot experiment in rural Ghana. Dean Karlan, Ed Kutsoati, Margaret McMillan, and Chris Udry, 2010. 1020. Institutional and public expenditure review of Ghana’s Ministry of Food and Agriculture. Shashidhara Kolavalli, Regina Birner, Samuel Benin, Leah Horowitz, Suresh Babu, Kwadwo Asenso-Okyere, Nii Moi Thompson, and John Poku, 2010. 1019. Building capacities for evidence and outcome-based food policy planning and implementation: The example of the Comprehensive Africa Agriculture Development Programme. Ousmane Badiane, Sunday P. Odjo, and John M. Ulimwengu, 2010. 1018. Modeling the global trade and environmental impacts of biofuel policies. Antoine Bouët, Betina V. Dimaranan, and Hugo Valin, 2010. 1017. Analyzing nutritional impacts of policies: An empirical study for Malawi. Olivier Ecker and Matin Qaim, 2010. 1016. Optimal rainfall insurance contracts for maize producers in Ghana’s northern region: A mathematical programming approach. Francis M. Muamba and John M. Ulimwengu, 2010. 1015. Impacts of climate change on agriculture and policy options for adaptation: The case of Vietnam. Bingxin Yu, Tingju Zhu, Clemens Breisinger, and Nguyen Manh Hai, 2010. 1014. Spatial price transmission and market integration in Senegal’s groundnut market. Ousmane Badiane, John M. Ulimwengu, and Fleur Wouterse, 2010. 1013. Strategies for adapting to climate change in rural Sub-Saharan Africa: A review of data sources, poverty reduction strategy programs (PRSPs) and national adaptation plans for agriculture (NAPAs) in ASARECA member countries. Jonathan Makau Nzuma, Michael Waithaka, Richard Mbithi Mulwa, Miriam Kyotalimye, and Gerald Nelson, 2010. 1012. Do health investments improve agricultural productivity?: Lessons from agricultural household and health research. Paul E. McNamara, John M. Ulimwengu, and Kenneth L. Leonard, 2010. 1011. Investigating the role of poultry in livelihoods and the impact of avian flu on livelihoods outcomes in Africa: Evidence from Ethiopia, Ghana, Kenya, and Nigeria. Ekin Birol, Dorene Asare-Marfo, Gezahegn Ayele, Akwasi Mensa-Bonsu, Lydia Ndirangu, Benjamin Okpukpara, Devesh Roy, and Yorbol Yakhshilikov, 2010. 1010. Constraints to fertilizer use in Nigeria: Insights from agricultural extension service. Afua B. Banful, Ephraim Nkonya, and Victor Oboh, 2010. 1009. Do household definitions matter in survey design? Results from a randomized survey experiment in Mali. Lori Beaman and Andrew Dillon, 2010. 1008. Livestock development planning in Uganda: Identification of areas of opportunity and challenge. Todd Benson and Samuel Mugarura. 2010. 1007. Migratory responses to agricultural risk in northern Nigeria. Andrew Dillion, Valerie Mueller, and Sheu Salau. 2010. 1006. Do comprehensive Africa agriculture development program (CAADP) processes make a difference to country commitments to develop agriculture?: The case of Ghana. Shashidhara Kolavalli, Kathleen Flaherty, Ramatu Al-Hassan, and Kwaku Owusu Baah, 2010. 1005. The new Nicaraguan water law in context: Institutions and challenges for water management and governance. Paula Novo and Alberto Garrido, 2010. 1004. Potential of carbon markets for small farmers: A literature review. Alessandro De Pinto, Marilia Magalhaes, and Claudia Ringler, 2010. 1003. Understanding gender differences in agricultural productivity in Uganda and Nigeria. Amber Peterman, Agnes Quisumbing, Julia Behrman, and Ephraim Nkonya, 2010. 1002. Old problems in the new solutions? Politically motivated allocation of program benefits and the “new” fertilizer subsidies. Afua Branoah Banful, 2010. http://www.ifpri.org/publications/results/taxonomy%3A468� INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE www.ifpri.org IFPRI HEADQUARTERS 2033 K Street, NW Washington, DC 20006-1002 USA Tel.: +1-202-862-5600 Fax: +1-202-467-4439 Email: ifpri@cgiar.org IFPRI ADDIS ABABA P. O. Box 5689 Addis Ababa, Ethiopia Tel.: +251 11 6463215 Fax: +251 11 6462927 Email: ifpri-addisababa@cgiar.org IFPRI NEW DELHI CG Block, NASC Complex, PUSA New Delhi 110-012 India Tel.: 91 11 2584-6565 Fax: 91 11 2584-8008 / 2584-6572 Email: ifpri-newdelhi@cgiar.org mailto:ifpri-addisababa@cgiar.org� Abstract ACKNOWLEDGMENTS 1. Introduction 2. survey area and data description 3. estimating treatment effects using nonexperimental data The Evaluation Problem Propensity Score Matching Robustness Checks 4. empirical results 5. conclusion REFERENCES RECENT IFPRI DISCUSSION PAPERS