Impacts assessments without true baselines: assessing the relative effects of training on the performance of water user associations in southern Tajikistan
Review statusPeer Review
MetadataShow full item record
Balasubramanya, Soumya; Price, Joseph P. G.; Horbulyk, Theodore M. 2018. Impacts assessments without true baselines: assessing the relative effects of training on the performance of water user associations in southern Tajikistan. 28p. (Online first) doi: 10.1142/S2382624X18500078
Permanent link to cite or share this item: http://hdl.handle.net/10568/92141
External link to download this item: https://www.worldscientific.com/doi/pdf/10.1142/S2382624X18500078
Conducting rigorous evaluations of whether the process of creating new institutions affects their performance of mandated duties presents several challenges. Not only is assignment to process often not random, but when the process of creating new institutions starts, outcomes and other performance-influencing covariates are not measurable because the yet-to-be created institutions are not functioning at baseline. This paper compares the performance of 74 ‘treated’ water user associations (WUAs) in Tajikistan that were created using a longer training process with 67 ‘control’ WUAs that were created using shorter training, to assess the impact of training on WUA performance of mandated duties. First, propensity scores were constructed to estimate the probabilities of being ‘treated’ by treatment status. These results guided the application of the difference-in-difference technique with right-hand side covariates in a context where field measures of outcomes and other performance-influencing covariates were made after the new institutions were created and functioning. The first measures were taken within 12–18 months of the new institutions being functional and the second measures were taken 24 months after the first. This choice of methods introduces a bias due to measurement error causing an underestimate of the treatment effects, while controlling for biases due to time-invariant and time-varying unobservables. An alternative method that only compared the differences in outcomes at a single point in time after the new institutions were created would have provided an inaccurate estimate of the effects of the intervention. This is a context in which methods such as synthetic controls are impossible to employ due to the nature of the intervention, other macroeconomic structural changes, and severe data restrictions. The methodology employed here generates evidence that, while biased toward generating an underestimate of effect, can still be useful and informative for policy and management purposes, and for evaluating the impact of process on the functioning of new institutions in transition settings.
Showing items related by title, author, creator and subject.
Title:Overview of water-related programs in Uzbekistan. [Project report of the Sustainable Management of Water Resources in Rural Areas in Uzbekistan. Component 1: National policy framework for water governance and integrated water resources management and supply part] Date:2017Type:ReportStatus:Open Access
Title:Impact of water prices and volumetric water allocation on water productivity: comparative analysis of well owners, water buyers and shareholders Authors:Kumar, M. DineshDate:2005Type:NewsletterStatus:Open Access
Title:Managing the business: potential and pitfalls of water rights and water tariffs in allocating and managing water in water stressed basins: the case of Rufiji Basin in Tanzania Date:2005Type:Conference PaperStatus:Open Access