A mother with her child in a tribal district of Odisha, India. © 2014 Nikhil Utture, Courtesy of Photoshare APPI/SPREAD Collective Action for Nutrition Social Audit Programme Odisha, India Final Evaluation Report Jessica Gordon, Jean-Pierre Tranchant, Laura Casu, Becky Mitchell & Nicholas Nisbett March 2019 1 Citation: Gordon, J.; Tranchant, J-P.; Casu, L.; Mitchell, B. and Nisbett, N. (2019) APPI/SPREAD Collective Action for Nutrition Social Audit Programme Odisha, India: Final Evaluation Report, Brighton: IDS Authors: Jessica Gordon, Jean-Pierre Tranchant, Laura Casu, Becky Mitchell and Nicholas Nisbett Published: March 2019 Disclaimers: The opinions expressed are those of the authors and do not necessarily reflect the views or policies of IDS, IFPRI or APPI. Copyright: © Azim Premji Philanthropic Initatives, Institute of Development Studies, International Food Policy Research Institute, 2019. ISBN: 978-1-78118-530-8 This is an Open Access report distributed under the terms of the Creative Commons Attribution Non Commercial 4.0 International licence (CC BY-NC), which permits use, distribution and reproduction in any medium, provided the original authors and source are credited, any modifications or adaptations are indicated, and the work is not used for commercial purposes. http://creativecommons.org/licenses/by-nc/4.0/legalcode Institute of Development Studies, Brighton BN1 9RE UK www.ids.ac.uk http://creativecommons.org/licenses/by-nc/4.0/legalcode http://www.ids.ac.uk/ 2 Acknowledgements This report was prepared by the Institute of Development Studies (IDS) for Azim Premji Philanthropic Initiatives (APPI). Analysis and report writing was led by Jessica Gordon, Jean- Pierre Tranchant, and Nicholas Nisbett with contributions from Becky Mitchell and Laura Casu. The project was led by Jessica Gordon and Nicholas Nisbett, with the quantitative survey and analysis led by Jean-Pierre Tranchant, and the qualitative and process components were led by Jessica Gordon with support from Nicholas Nisbett and Becky Mitchell. Additional contributions to the analysis and report were also provided by Panos Deoudes, Manmeet Kaur, and Deeksha Pande, for which we are very grateful. We would also like to extend our thanks to the Development Corner Consulting (DCOR) team led by Satyanarayan Mohanty who led the quantitative and qualitative data collection in Odisha, contributed to the qualitative data analysis, and produced field reports: all of which informed the content of this report. We are particularly grateful to Satya and to Gopal Krushna Bhoi for their warmth and hospitality in the field. We are equally grateful to the Society for Promoting Education and Rural Development (SPREAD) programme team, particularly to Bidyut Mohanty, Rajkishor Mishra, Jitendra Rath, and Vipul Kumar for their input and support towards the evaluation alongside implementation of the programme. We would also like to thank APPI for funding the evaluation and for their continued valuable input, support, and guidance throughout the process. In particular, we are grateful to Mansi Shah, Devjit Mittra, Hrishikesh Parthasarathy, and Avishek Bose. We would also like to acknowledge financial support from the International Food Policy Research Institute (IFPRI) who co-funded this project through the CGIAR Research Program on Agriculture for Nutrition and Health (A4NH). Finally, we are very grateful to the people, the communities, and the officials of the districts of Balangir, Kalahandi, Koraput, Malkangiri, Nabarangpur, and Nuapada for their participation in this evaluation. 3 Executive Summary Introduction This report summarises the background, design and methods, and key findings from an independent evaluation of the Collective Action for Nutrition (CAN) Social Audit programme designed and implemented by the Odisha-based non-governmental organisation (NGO) Society for Promoting Education and Rural Development (SPREAD) and supported by Azim Premji Philanthropic Initiatives (APPI). The evaluation was led by the Institute of Development Studies (IDS), UK in partnership with Development Corner Consulting (DCOR), India. Findings are intended to provide evidence on the effectiveness of the social audit model and feed into future plans for replication and scale-up. Background The first two sections of the report present the background to the intervention and the evaluation design. Whilst India has seen steady improvements in health and nutrition services over the past decade, as reflected in some improvements in nutrition outcomes (e.g. infant mortality rates, child stunting rates), significant quality and coverage issues remain at community level, meaning that wide disparities in nutrition outcomes still exist based on inequities of socioeconomic status, caste, ethnicity, gender, and geography. Social accountability approaches, including social audits, offer a demand-driven alternative to improving delivery of state nutrition-related services. India has a long history of innovation in social accountability approaches with social audits now institutionalised in a number of national policies and programmes including the Mahatma Gandhi National Rural Employment Guarantee Act (MGNREGA) and the National Food Security Act (NFSA 2013), which is the focus of the CAN programme and this evaluation. The NFSA aims to improve food and nutrition security to citizens by bringing together four existing programmes: the Targeted Public Distribution System (TPDS); the Midday Meals (MDM) programme; Integrated Child Development Services (ICDS), and the Maternity Entitlement Scheme (MAMATA). The Odisha-based NGO SPREAD works to empower marginalised communities to access their rights. Their geographic focus is six districts (Balangir, Nuapada, Kalahandi, Malkangiri, Koraput, and Nabarangpur) of the ‘KBK+’ districts of Odisha.1 The three-year CAN programme’s overall goal is ‘Reducing malnutrition among children and women by facilitating efficient implementation of food and nutrition programmes, ensuring transparency, downward accountability, and community participation.’ The programme is running between August 2016 and July 2019. The main intervention focus is a social audit process designed to sensitise communities to their rights and entitlements under the four primary schemes covered by the NFSA. Interim outcomes expected include increased knowledge and uptake of NFSA services, improved participation in community-level governance activities (especially by women), and improved institutional delivery of nutrition services and entitlements. The longer-term outcome anticipated is reduced malnutrition amongst target communities, particularly for women and children, as a result of increased uptake and improved institutional delivery. 1 A number of more marginalised districts which were bounded originally by older boundaries of Koraput, Balangir, and Kalahandi. 4 The SPREAD programme design divides the intervention into three main phases: the pre- social audit, social audit, and post-social audit phase. All three phases involve interaction with a wider range of community and state actors, including NFSA rights holders in the community, government officials, community leaders, and service providers. During the pre- social audit phase, the social audit calendar is developed, community volunteers (Samikshya Sathis – SSs) are recruited, and social audit teams are set up. In this phase, intensive campaigns are also conducted as build-up activities for the social audit. During the intensive seven-day social audit process in each intervention Gram Panchayat (GP), a series of meetings and activities take place, including initial village meetings, focus group discussions (FGDs) with local council members (the Panchayati Raj Institution – PRI), field surveys, data collection with NFSA rights holders and verification of registers and documentation, followed by data compilation, and report writing. The process culminates in the organisation of a public hearing taking the form of a formal village council (PRI) meeting (a Gram Sabha – GS) to share findings and grievances raised as part of the data collection. During the post-social audit phase, the social audit team are expected to follow up on NFSA grievance redressal with the relevant government officials being accountable. In the post-audit social ‘follow-up’ stage, the CAN programme also began work on nutrition behavioural change via a further participatory learning and action (PLA) model, but this part of the intervention is not covered or evaluated here as it fell outside of the period and scope of the evaluation. Evaluation design To assess the impact and operation of this model, the evaluation was tasked with the following two overarching objectives: 1. To determine the short-term impact of the SPREAD model of social audits on improving the delivery of nutrition services and entitlements, as well as the uptake of these services by target groups; 2. To understand how the social audit process leads to: a. Changes in knowledge, behaviour, and practice at the household level b. Community-level changes and outcomes. The design consists of three interwoven methodological components designed to complement each other as part of a mixed methods theory-based design which draws on the programme’s theory of change (Figure 1). These include: 1. A quantitative component, based on a rigorous experimental survey design which measures the impact of the social audit on a range of outcomes; 2. A process evaluation component, which documents critical processes, mechanisms, and outputs to assess quality of the implementation of the social audit programme; 3. A qualitative community-based study, which explores community-level perceptions and contextual factors related to the outcomes of the social audit. The quantitative baseline survey was conducted in December 2017–January 2018, followed by an endline survey in April–May 2018. This was shortly followed by the community-based qualitative and process fieldwork carried out in July–August 2018. Programme data used for the process evaluation were collected and analysed throughout the evaluation. Final mixed methods data analysis, synthesis, and triangulation was undertaken between September– November 2018. The quantitative evaluation is based on an experimental design in which the timing of the social audits was randomly varied across the local administrative units (GPs), to enable a 5 causal estimation of the impact of the social audits. Specifically, the impact estimates are based on exogenous variations in the length of time between the social audit and the endline data collection. Treatment assignment was done at the GP level, with GPs randomly assigned into an ‘Early’ group (E) and ‘Late’ group (L). In the Early group, the gap between the social audit and endline data collection is about three months, whereas in the Late group, it is one month. As such, the focus of the design is on the more immediate outcomes of the project, rather than longer-term outcomes. Quantitative findings draw on two sources of comparisons: (i) the experimental comparison between Early and Late GPs (based on differences in or on endline levels of the outcomes), and (ii) the before–after comparison in all GPs. Combining both sources of comparisons allows us to ascertain if there had been any impact directly as a result of the intervention; as well as whether the impact was immediate and short-lived (changes significantly higher in ‘Late GPs’), or cumulative (changes significantly higher in ‘Early GPs’). This model had limitations, however, as firstly, such a short duration between baseline and endline means that the evaluation focuses on the short-term impacts to communities participating in the social audit, even though we know that such processes of accountability and empowerment require a long time to filter through to improvements in service delivery (if at all). Secondly, because if desired results improved, in both Early and Late GPs, we would be unable to tell whether such improvements were the result of the communities’ participation in the programme; or simply wider societal or service delivery changes that were picked up during this period of time. Despite these limitations, this research design still enables us to rigorously assess the effects of social audits, despite the constraint of having all GPs receiving the intervention in phase 1 of the programming. Outcomes were measured through pre- and post-social audit questionnaires administered to primary caregivers (PCs) of a child below 24 months, pregnant women, men (typically the husbands of caregivers or pregnant women), adolescent girls, Anganwadi workers (AWWs), Accredited Social Health Activist (ASHA) workers, and Sarpanchs. Specific outcomes measured include knowledge of households of NFSA entitlements; access to and uptake of health and nutrition services; the perceived quality and responsiveness of health and nutrition services; food security and Infant and Young Child Feeding (IYCF) knowledge and; participation in community-level governance and civic attitudes. For the quantitative sample, a total of 116 GPs (58 in each treatment arm) were selected. Given 20 observations (households) per GP and an intra-cluster correlation coefficient of 10 per cent, 116 GPs allowed for detection of an effect size of 0.22 standard deviations with a statistical power of 80 per cent. The number of households was later increased to 24 per GP, thus raising the statistical power to 8 per cent. For GP-level outcomes, for which there is only one observation per GP, we are able to detect an effect size of 0.5 standard deviation with a statistical power of 80 per cent. The process evaluation component drew on a range of primary and secondary qualitative and quantitative data sources, including programme documents, and Management Information System (MIS) data, community-level in-depth interviews (IDIs), and FGDs with community members, SPREAD/SS (volunteer) staff, and others and the evaluation’s quantitative data. The qualitative evaluation component methods were based on community- level IDIs and FGDs with community members, government officials, SPREAD/SS/social audit team members and NFSA service providers. Community-level data collection for both qualitative and process components was incorporated into the same tools and fieldwork process. A total of 11 villages were selected as case studies from across three districts (Balangir, Kalahandi, Koraput) based on a stratified random sample from the quantitative evaluation sample. A total of 121 IDIs and 80 FGDs were conducted. 6 Figure 1 Theory of change developed for use by the evaluation Source: Authors’ own. 7 Findings Section 3 of the report contains our findings on the CAN programme’s design, implementation, and processes. These present a largely positive picture, albeit with some challenges faced and areas for further improvement. The programme model was built on existing experience and best practice, relied on the use of comprehensive guidelines (such as the Social Audit Manual), was modified through an extensive piloting process and suitably adapted to the local context to meet the programme’s objectives. In terms of staffing, SPREAD recruited over 700 people at all organisational levels (state, block, district, GP) with only a few minor delays to recruitment and some staff turnover which didn’t appear to affect implementation. Extensive staff training and staff development activities were conducted at all levels and covered a range of approaches including participatory, group, and practice exercises supported by adequate induction materials and resources. Specific detailed training was conducted for the Samakhya Sathis (SS – the two volunteers recruited locally in each GP). Training was also undertaken with Gaon Kalyan Samiti and PRI members, and involved other relevant organisations. This helped develop trust and community/local governance relations. The selection of areas to work on based on nutrition vulnerability appear sensible, as was the decision to move to blanket coverage of Anganwadi Centres (AWCs) in implementation areas. However, given the model’s intensity of personnel and resource requirements, a more streamlined, leaner model may be required for further scale-up. For MIS, SPREAD relied on a new mobile-based system to collect real-/near-time data on all aspects of the social audit process. This included AWC and Fair Price Shop verifications, along with checks on midday meals in schools; as well as interviews with 20 per cent of ICDS beneficiaries from each Anganwadi centre. This data were compiled and shared with the community and relevant stakeholders at the end of the social audit in each GP. Some delays and challenges were faced in setting up and implementing the MIS approach but overall, it was deemed successful in enabling the team to better streamline data entry, cleaning, and analysis, and enable real-time monitoring, decision-making, and sharing with the community during public hearings. Some gaps in the effective monitoring of longer-term outcomes were identified. Overall, many community members reported that their participation in the social audit process was a positive experience and was a useful tool to voice concerns, increase access to services, and involve communities directly in resolving grievance issues. Most government officials and NFSA service providers considered it to be a positive, useful, and well-managed process, although some NFSA service providers felt that issues with the TPDS couldn’t be solved by an NGO and could only be addressed at a ‘higher level’. Over half (59 per cent) of primary caregivers surveyed said that they knew what a social audit was and a similar proportion (57 per cent) said that they were aware that a social audit had taken place in their GP, though there was substantial variation in awareness levels across districts (ranging from 40–80 per cent). Some village-level (Palli Sabha – PS) and wider Gram Sabha (the formal administrative unit often incorporating more than one village) meeting attendance issues were highlighted. The SPREAD team made efforts to address these issues through early and continued positive engagement with communities and officials. Difficulties were faced by the social audit team in improving immediate access to nutrition entitlements and ensuring that grievances remained focused on nutrition issues as opposed to other issues (including housing, roads, etc.). In terms of participation rates, around one in five men (18 per cent) and women (23 per cent) from the quantitative sample attended the Palli Sabha. For women – but not for men – participation in the Gram Sabhas was much lower (10 per cent) than in the Palli Sabhas. The main reasons for lack of attendance at the GS were slightly different between men and 8 women: both highlighted that they were busy and lacked time (62 per cent for women and 44 per cent for men), but men were more likely to be absent from the village at the time of the meeting (38 per cent). Qualitative data suggest other possible reasons for non-participation included women’s status in the household and because someone else in the household participated. Section 4 of the report considered the evidence we have gathered on the outcomes mapped in the theory of change (Figure 1) relating primarily to awareness and knowledge of services, entitlements, and grievance redressal avenues; access to and satisfaction with services; use of grievance redressal and broader measures of community participation and empowerment. We also considered the broader impacts on food security and IYCF knowledge. In terms of knowledge of NFSA services, entitlements, and grievance mechanisms, we see some significant changes, varying by service. Knowledge about the ICDS service of Take- Home Ration (THR) improved amongst caregivers and pregnant women, the latter seeing some very high changes (+25/+21 pp early/late). With no significant differences between Early and Late GPs we cannot be sure that these changes are cumulative or indeed connected to the social audit process. However, in the absence of other supply- or demand- side activities happening with regard to ICDS during this period, it is hard to rule out their connection to the CAN programme. Awareness and knowledge of the MAMATA scheme and entitlements did significantly improve for the target group of pregnant women between baseline and follow-up. Such changes tend to be significantly more marked in Early GPs, indicating a cumulative positive impact of social audits. Knowledge about the TPDS improved substantially or very substantially amongst all groups sampled – caregivers, pregnant women, and men. These included variables measured such as overall knowledge of entitlements and knowledge about PDS committees. The changes for men were the most marked and were also more pronounced in Early than Late GPs. This suggests that these impacts were due to cumulative exposure to the social audit process and the community-level activities it set into motion. More detailed views of those who participated in the community IDIs and FGDs lend further credence to the view that such gains in knowledge were related to attendance at social audit events and processes. In terms of access to, uptake of, and satisfaction with NFSA services and entitlements, we see a more varied pattern but which includes improvement in some areas across all services. For ICDS, we saw an increase in the likelihood that a child was provided with special food following growth monitoring, and a small but statistically significant increase in referral to rehab centres after monitoring. Neither of these changes differed significantly between Early and Late GPs, however. For MAMATA, we see significant increases (+22pp) in enrolment amongst pregnant women in the Late GPs (increases in Early GPs were not statistically significant) and a significant increase (+31 pp) in pregnant women being satisfied by MAMATA. The latter change was significantly different between early and late groups, suggesting that the change in satisfaction was based cumulatively on length of exposure to the social audit and the processes it set in motion. For TPDS, we see improvements for all groups sampled, but particularly for men, including decreases (-7pp) in men reporting having to pay extra money at the ration shop in Early GPs, and a higher likelihood of having been assigned an Aadhar number (+5pp). Both these changes are significantly different between early and late groups, suggesting a cumulative effect of the social audit process. Other changes where we cannot detect a significant difference between Early and Late GPs include, amongst primary caregivers, improvements and likelihood to have an Aadhar number, likelihood that ration shops deliver food on a 9 monthly basis, satisfaction with the TPDS, and the quantity of the food ration. Although these changes can’t be directly linked to the social audit process, the magnitude of the changes in some cases (e.g. 32pp increases in ration shops delivering food monthly in Early GPs and 46pp in Late GPs) and the absence of other major supply- or demand-side changes observed during this period are highly suggestive that these changes are related to the CAN programme. This view is supported by the information garnered from some of the individuals who participated in the community IDIs and FGDs, who reported increases in access and availability of all the services that were the target of the social audit. However, it is also the case that other participants in IDIs and FGDs complained that there have been no changes since the social audit, or discussed the fact that expected improvements in services they thought would result from their village’s participation in the CAN programme has not happened. Others still, stressed that they had difficulty accessing the services (or actively participating in services) due to their marginalisation in village life, as a result of poverty, caste, or tribal status. Some of the most surprising results in the outcomes measured relate to use of and confidence to use grievance redressal processes, particularly in light of the results on the positive effects of social audits, on knowledge of these processes, and on community participation. Confidence in an ability to raise issues regarding ICDS, MAMATA, and TPDS went down, with the difference between early and late groups statistically significant. We have been unable to explain these changes as the data we gathered in community IDIs and FGDs suggest that – in some cases at least – people’s confidence about the use of these processes had improved. The fact that the decrease in confidence is less marked in Early GPs than in Late GPs lends some support to the hypothesis that after having learnt more about service entitlements, peoples’ expectations increased beyond a level that they felt could be adequately met by simply complaining to the service providers. This hypothesis is also somewhat supported by the IDIs and FGDs we undertook with the service providers themselves, who reported that complaints had gone up but that they were frustrated that a lot of what was being complained about was not within their power to address (as these had more to do with systemic supply-side constraints). Our findings on community empowerment and voice in local nutrition-related decision-making strongly support the view that social audits led to very large positive changes in the civic life- related attitudes of caregivers and pregnant women. Both groups of women became more positive about citizens being engaged in civic life and about states being responsive to citizens’ demands. Social audits also strongly enhanced the community participation of pregnant women and the political participation of caregivers (both effects are seen in Early GPs only, which is evidence of a cumulative impact of social audits in this regard). Qualitative data provide a number of further illustrations of people’s growing sense of awareness of the forms of redressal and of being able to voice their problems. Qualitative data also suggested that people became increasingly motivated to take part in community life via their membership of committees as a means to solve future problems. But our qualitative research also emphasised how entrenched power structures are in villages, and how the realities of daily life may limit the participation of others. Again, surprisingly, the changes we see in IYCF/nutrition-related knowledge and practices are not positive. In fact, nutrition knowledge scores amongst primary caregivers declined in Early GPs – with the only positive result being amongst those in Late GPs and relating to ‘other nutrition knowledge’. Declines in knowledge are not present in Late GPs, suggesting that social audits exert a cumulative effect of knowledge. This could be due to the delivery of nutrition knowledge being different in Late and Early GPs (this is an area that deserves further exploration by SPREAD and which they report is being actively addressed now in follow-up activities), and/or with service providers being diverted away from nutrition counselling towards other tasks as a result of the social audits. 10 Our findings on food security and dietary diversity show very significant improvements in dietary diversity scores for women and children. However, as nutritional knowledge amongst samples had not improved during that time and in some cases had deteriorated (with nutrition knowledge actually decreasing in Early GPs), it is hard to link this change to the social audits (although neither can we formally rule out the possibility that social audits had an immediate impact). Some other plausible explanations are that income changed significantly during this period as a result of greater livelihood opportunities, or that these are seasonal variations in food availability. Some mothers interviewed as part of the community IDIs and FGDs, however, do link the social audit to changes in their children’s diets. Further work will be needed to understand changes in this area. Similar to above, findings on occurrence, and awareness of village health and nutrition days showed significant improvements in both Early and Late GPs. The quantitative data made it difficult to attribute this to the social audits, although it is plausible, particularly given the limited positive evidence from the qualitative data that the audits exerted an immediate effect, given the scale of changes seen. Conclusions Section 5 of the report summarises earlier sections and provides broader conclusions on the suitability of the model for the purposes of the programme. In terms of implementation, SPREAD have clearly shown the ability to put in place a large and complex operation – including the recruitment and management of several hundred staff – and implement the social audit process in a way that was mostly welcomed by those in the communities we surveyed or interviewed; most service providers and government officials. Operating at this scale and with the political acumen to do so in a way that was well integrated into local democratic institutions and government services (the PRIs, the Palli Sabhas, the Gram Sabhas, as well as securing the participation of local officials of the concerned services) is an immense first hurdle that many organisations might fail at without proper thought and experience. This is very positive, albeit a high bar for replicability elsewhere, including at larger or less resource-intensive scales. Even bearing in mind these positive findings, participation rates were not uniformly high. Whether participation rates (at around 23 per cent for the Palli Sabha amongst women and 10 per cent for the Gram Sabha) could have been higher depends a little on one’s expectations of poor communities’ capacity to spend at least several hours participating in such an exercise. This may be a question of design assumptions as much as a question of implementation. We observed positive changes in nearly all the areas in which we measured outcomes. In some areas, we cannot relate changes directly and rigorously to communities’ participation in the CAN programme for reasons of the limitations explained in terms of the methodology. However, this includes some large changes in both Early and Late GPs referred to above, for which we cannot as yet offer alternative explanations. Changes in dietary diversity for both women and children, however, though substantial, similarly did not differ between early and late groups and because these changes could plausibly be explained by other factors of increased income and food availability, we are more cautious in linking them to the presence of the social audit. Outcomes that we measured and which showed significant difference between Early and Late GPs (results were all higher in Early as compared to Late GPs, suggesting a cumulative impact as the result of exposure) included awareness and knowledge of, and satisfaction with, the MAMATA scheme amongst pregnant women, knowledge of entitlements and PDS committees, decreases in having to pay extra money at a ration shop, and increases in Aadhar registration amongst men; enhanced community participation amongst pregnant women; and political participation amongst primary caregivers. 11 Finally, there are two areas in which we see outcomes changing in the opposite way to which we would have hoped – the confidence of those surveyed to raise issues with service providers and in women’s IYCF knowledge scores. Qualitative data provide some support to the hypothesis that the former is due to the immediate rise in expectations not being met. We also note that the social audit process did not prioritise IYCF knowledge and many of the service improvements were concerned with delivery of hard outputs (THR, PDS, etc.) rather than improving the ‘softer’ skills and delivery tasks of frontline workers such as AWWs, who have responsibility for nutrition counselling. Wider evidence from India and elsewhere shows that without a concerted, sustained, high-intensity, and well-supported effort to change IYCF knowledge, such changes in knowledge – and certainly practice – will not happen. As this was not a core part of the CAN programme, such changes are therefore not so difficult to understand. In summary and in terms of the original objectives of the evaluation, we do see sufficient evidence that the social audit model has the potential to improve delivery and uptake of NFSA services. These service improvements and uptake were not uniformly positive for all target groups, but in some areas, the changes were significant and directly linked to communities’ participation in the social audit processes. The social audit did also lead to changes in knowledge, behaviour, and practice at a household level in terms of participation in the social audit itself and increased desire to participate in aspects of civic life such as committees related to NFSA delivery. Nutrition and IYCF knowledge, however, did not change. Given that this was not an objective of the evaluation and would not have been possible given our design and methods, we cannot offer a definitive answer to the question of whether a social audit design can result in changes in maternal and child nutritional status within the context of the delivery of NFSA services in Odisha. However, some of the changes we see are of a magnitude rarely witnessed in terms of the Indian services covered by the NFSA, or local democracy strengthening, whether in terms of knowledge/awareness, participation/empowerment, and service delivery/satisfaction. There is good evidence that SPREAD is well placed to continue to implement this model effectively. As the programme is continuing into its next phase, we make some further recommendations for its ongoing implementation in the final section. Implications for CAN programme Phase 2 design  IYCF knowledge more central; AWW (and others’) roles in nutrition behaviour change communication (BCC) more central;  Further operational research and attention to differences in social audit (SA) participation rates, particularly amongst women and more marginalised groups – and to take into account other calls on time (work/household chores, migration);  Further work to understand why confidence to raise issues declines immediately after SA;  Further work to build on positive results in terms of knowledge, access, and satisfaction across the services – and willingness to participate in community life/committees. Longer-term implementation, scalability questions/research  Different intensities of length of exposure/rate of exposure (how many times/year)  Different organisational structures – less resource intensive?  Increased level of policy engagement with state-level actors through strategic partnerships /networks to address service bottlenecks at district and state-level. 12 13 Table of Contents Acknowledgements ............................................................................................................. 2 Executive Summary ............................................................................................................ 3 List of Abbreviations ......................................................................................................... 18 1 Introduction and background ............................................................................... 20 1.1 Context of report ........................................................................................ 20 1.2 Undernutrition in Odisha ............................................................................ 20 1.3 Social accountability and social audits .................................................... 21 1.4 National Food Security Act (NFSA) ........................................................... 22 1.4.1 Targeted Public Distribution System (TPDS) ....................................... 22 1.4.2 Midday Meal (MDM) ............................................................................ 23 1.4.3 Integrated Child Development Services (ICDS) ................................... 23 1.4.4 MAMATA ............................................................................................. 24 1.5 SPREAD CAN programme ......................................................................... 25 1.5.1 Programme background, aims, and overall approach .......................... 25 1.5.2 Overall design ...................................................................................... 26 1.5.3 SPREAD CAN Implementation plan ..................................................... 29 1.6 Baseline characteristics of target groups ................................................. 29 1.6.1 Household characteristics .................................................................... 30 1.6.2 Primary caregivers ............................................................................... 32 1.6.3 Pregnant women .................................................................................. 33 1.6.4 Children below two years of age .......................................................... 34 2 Evaluation design and methods ........................................................................... 35 2.1 Evaluation aims and objectives............................................................... 35 2.2 Programme Impact Pathway for mixed methods approach .................. 35 2.3 Quantitative component design and approach ...................................... 38 2.3.1 Quantitative evaluation design ........................................................... 38 2.3.2 Estimations of treatment effect .......................................................... 43 2.3.3 Quantitative data collection process .................................................. 44 2.4 Process component design and approach ............................................. 45 2.5 Qualitative component design and approach ........................................ 46 2.6 Sampling strategy and process for community-level qualitative and process data collection and analysis ............................ 47 2.7 Ethical considerations ............................................................................. 52 3 Findings at implementation level (design, inputs, outputs) ................................ 53 14 3.1 Design ....................................................................................................... 53 3.2 Staff /volunteer recruitment, training, and characteristics .................... 54 3.3 Targeting and selection of regions and participants ............................. 55 3.3.1 Selection of target areas (state, block, GP, village level) ................... 55 3.3.2 Targeting of participant groups .......................................................... 57 3.4 Programme management and monitoring .............................................. 59 3.4.1 SPREAD management approach and structures ............................... 59 3.4.2 Monitoring frameworks and tools ....................................................... 61 3.5 Social audit implementation .................................................................... 63 3.6 Community knowledge of and participation in the social audit process ................................................................................. 69 3.6.1 Knowledge of the social audit process ............................................... 69 3.6.2 Community participation in the social audit process ........................... 72 3.7 Summary ................................................................................................... 76 4 Findings at outcome level ..................................................................................... 79 4.1 Knowledge of NFSA services, entitlements, and grievance mechanisms ............................................................................ 79 4.2 Access to, uptake of, and satisfaction with NFSA services and entitlements .............................................................................................. 85 4.3 Use of and confidence to use grievance redressal processes ............. 89 4.4 Community empowerment and voice in local nutrition-related decision-making ..................................................................................... 103 4.5 IYCF/nutrition-related knowledge .......................................................... 107 4.6 Village Health and Nutrition Days (VHNDs) .......................................... 110 4.7 Food security and dietary diversity ...................................................... 111 4.8 Summary ................................................................................................. 113 5 Conclusions and recommendations ................................................................... 116 5.1 Institutional accountability for NFSA services delivery improvements 116 5.2 Implications for CAN programme Phase 2 design ............................... 118 5.3 Longer-term implementation, scalability questions/research ............. 118 Annex 1 Definitions of variables for quantitative data ...................................... 119 Annex 2 Mixed Methods Analysis Approach ..................................................... 127 References ........................................................................................................... 129 15 Boxes Box 1.1 SPREAD definition of the social audit process ........................................... 25 Box 1.2 The 11 steps of the social audit .................................................................. 27 Box 2.1 The three evaluation components .............................................................. 35 Box 4.1 Basic description of the community and political engagement indices ...... 103 Figures Figure 1 Theory of change developed for use by the evaluation ................................. 6 Figure 2.1 Key stages of evaluation mixed methods process ...................................... 36 Figure 2.2 Programme Impact Pathway ...................................................................... 37 Figure 2.3 Flowchart of the quantitative evaluation ...................................................... 39 Figure 2.4 Number of days between social audit and endline data collection among Early and Late GPs .................................................................................... 40 Figure 2.5 Mean of vulnerability index across six target districts ................................. 48 Figure 3.1 Areas covered by SPREAD CAN programme ............................................ 56 Figure 3.2 SPREAD management structure ................................................................ 60 Tables Table 1.1 Model calendar for social audit process ............................................................... 28 Table 1.2 Baseline characteristics in selected districts – existing data and evaluation data . 31 Table 2.1 Number of interviews per target group and district ............................................... 43 Table 2.2 Attrition rates for quantitative sample ................................................................... 44 Table 2.3 IDIs conducted per district and target group......................................................... 50 Table 2.4 FGDs conducted by district and target group ....................................................... 51 Table 3.1 SPREAD MIS data on PLA meeting participation ................................................ 67 Table 3.2 Determinants of women’s participation in Palli and Gram Sabha – logistic regression estimates .......................................................................................................... 74 Table 3.3 Perceptions on impact of social audit ................................................................... 75 Table 4.1 DID Estimates of the impact of social audits on THR and GMP functioning – awareness/knowledge (primary caregivers sample) ............................................................ 79 Table 4.2 DID Estimates of the impact of social audits on THR functioning – awareness/knowledge (pregnant women sample) ............................................................... 79 Table 4.3 DID Estimates of the impact of social audits on AWC functioning – awareness (adolescent girls sample) ..................................................................................................... 80 Table 4.4 DID Estimates of the impact of social audits on MAMATA functioning – awareness/knowledge (pregnant women sample) ............................................................... 80 16 Table 4.5 DID Estimates of the impact of social audits on MAMATA functioning – awareness/knowledge (primary caregivers sample) ............................................................ 81 Table 4.6 DID Estimates of the impact of social audits on TPDS functioning – knowledge/awareness (primary caregivers sample) ............................................................ 82 Table 4.7 DID Estimates of the impact of social audits on TPDS functioning knowledge/awareness (pregnant women sample) ............................................................... 82 Table 4.8 DID Estimates of the impact of social audits on TPDS functioning – awareness/knowledge (male respondents sample) ............................................................. 82 Table 4.9 DID Estimates of the impact of social audits on AWC functioning – access/satisfaction (primary caregivers sample) .................................................................. 86 Table 4.10 DID Estimates of the impact of social audits on AWC functioning – access/satisfaction (pregnant women sample) .................................................................... 86 Table 4.11 DID Estimates of the impact of social audits on AWC functioning – access (adolescent girls sample) ..................................................................................................... 86 Table 4.12 Single Difference Estimates of the impact of social audits on AWC functioning (adolescent girls sample) ..................................................................................................... 87 Table 4.13 DID Estimates of the impact of social audits on THR and GMP functioning – access/uptake/satisfaction (primary caregivers sample) ...................................................... 87 Table 4.14 DID Estimates of the impact of social audits on THR functioning – access/satisfaction (pregnant women sample) .................................................................... 88 Table 4.15 DID Estimates of the impact of social audits on AWC functioning (Angawandi Workers sample) ................................................................................................................. 89 Table 4.16 DID Estimates of the impact of social audits on GMP functioning (Anganwadi Workers sample) ................................................................................................................. 90 Table 4.17 DID Estimates of the impact of social audits on MAMATA functioning – access/satisfaction (pregnant women sample) .................................................................... 91 Table 4.18 DID Estimates of the impact of social audits on MAMATA functioning – access/satisfaction (primary caregivers sample) .................................................................. 91 Table 4.19 DID Estimates of the impact of social audits on TPDS functioning – access/uptake/satisfaction (primary caregivers sample) ...................................................... 93 Table 4.20 DID Estimates of the impact of social audits on TPDS functioning – access/uptake/satisfaction (pregnant women sample) ......................................................... 94 Table 4.21 DID Estimates of the impact of social audits on TPDS functioning – access/uptake/satisfaction (male respondents sample) ....................................................... 95 Table 4.22 DID Estimates of the impact of social audits on AWC functioning – confidence (primary caregivers sample) .............................................................................................. 100 17 Table 4.23 DID Estimates of the impact of social audits on AWC functioning – confidence (pregnant women sample) ................................................................................................. 100 Table 4.24 DID Estimates of the impact of social audits on THR and GMP functioning – confidence (primary caregivers sample) ............................................................................ 100 Table 4.25 DID Estimates of the impact of social audits on MAMATA functioning – confidence (primary caregivers sample) ............................................................................ 101 Table 4.26 DID Estimates of the impact of social audits on MAMATA functioning –confidence (pregnant women sample) ................................................................................................. 101 Table 4.27 DID Estimates of the impact of social audits on TPDS functioning – confidence (primary caregivers sample) .............................................................................................. 101 Table 4.28 DID Estimates of the impact of social audits on TPDS functioning – confidence (pregnant women sample) ................................................................................................. 101 Table 4.29 DID Estimates of the impact of social audits on TPDS functioning – confidence (male respondents sample) ............................................................................................... 102 Table 4.30 DID Estimates of the impact of social audits on civic life (primary caregivers sample) ............................................................................................................................. 104 Table 4.31 DID Estimates of the impact of social audits on civic life (pregnant women sample) ............................................................................................................................. 105 Table 4.32 DID Estimates of the impact of social audits on civic life (male respondents sample) ............................................................................................................................. 106 Table 4.33 DID Estimates of the impact of social audits on nutrition knowledge (primary caregivers sample) ............................................................................................................ 109 Table 4.34 DID Estimates of the impact of social audits on nutrition knowledge (pregnant women sample) ................................................................................................................. 109 Table 4.35 DID Estimates of the impact of social audits on VHND (primary caregivers sample) ............................................................................................................................. 110 Table 4.36 DID Estimates of the impact of social audits on VHND (pregnant women sample) ............................................................................................................................. 110 Table 4.37 DID Estimates of the impact of social audits on VHND awareness (adolescent girls sample) ...................................................................................................................... 110 Table 4.38 DID Estimates of the impact of social audits on food security and dietary diversity (primary caregivers sample) .............................................................................................. 112 18 List of Abbreviations A4NH Agriculture for Nutrition and Health AAY Antyodaya Anna Yojna ANC antenatal care APL above poverty line APPI Azim Premji Philanthropic Initiatives ASHA Accredited Social Health Activist AWC Anganwadi Centre AWW Anganwadi Worker BC Block Coordinator BCC behaviour change communication BDO Block Development Officer BPL below poverty line CAN Collective Action for Nutrition CAPI Computer Assisted Personal Interviewing CBO community-based organisation CSO Civil Society Organisation CDPO Child Development Project Officer DCOR Development Corner Consulting DID Difference-in-Difference DO District Officer DPM District Programme Manager FAO Food and Agriculture Organization FIES Food Insecurity Experience Scale FGD focus group discussion FPS Fair Price Shop FRA Forest Rights Act GKS Gaon Kalyan Samitis GMP Growth Monitoring Programme GoO Government of Odisha GP Gram Panchayat GPC Gram Panchayat Coordinator GS Gram Sabha ICC intra-cluster correlation coefficient ICDS Integrated Child Delivery Services IDI in-depth interview IDS Institute of Development Studies IEC Institutional Ethics Committee IFA iron folic acid IFPRI International Food Policy Research Institute IYCF Infant and Young Child Feeding JC Jaanch Committee LPG liquid petroleum gas KBK Koraput, Bolangir, and Kalahandi MAMATA Maternity Entitlement Scheme MC Mothers’ Committee MDM Midday Meal MGNREGA Mahatma Gandhi National Rural Employment Guarantee Act MI Market Inspector MIS Management Information System MOU Memorandum of Understanding MSP minimum support price MUAC mid-upper arm circumference 19 NFSA National Food Security Act NGO non-governmental organisation NHM National Health Mission NRC Nutrition Rehabilitation Centre OBC Other Backward Caste ODK Open Data Kit OSSAAT Odisha Society for Social Audit Accountability and Transparency PC Primary Caregiver PIP Programme Impact Pathway PDS Public Distribution System PHH Priority Household PLA participatory learning and action PP percentage point PPS probability proportional to size PRI Panchayati Raj Institutions PS Palli Sabha PVTG Particularly Vulnerable Tribal Group Rs. rupee RTE Right to Education RTF right to food RTI Right to Information SA social audit SAM severe acute malnutrition SC Scheduled Caste SCN Standing Committee on Nutrition SD standard deviation SEBC Socially and Economically Backward Communities SHG self-help group SMC School Management Committee SPREAD Society for Promoting Education and Rural Development SS Samikshya Sathi ST Scheduled Tribe THR Take-Home Ration TPDS Targeted Public Distribution System TT tetanus toxoid UW underweight VHG Village Health Guide VHND Village Health and Nutrition Day VHSC Village Health and Sanitation Committee WDDS Women’s Dietary Diversity Score WHO World Health Organization 20 1 Introduction and background 1.1 Context of report Azim Premji Philanthropic Initiatives (APPI) has identified fighting malnutrition in India as one of their strategic focal areas of work. In 2015, they signed a Memorandum of Understanding (MOU) with the Government of Odisha to work collaboratively to reduce malnutrition in the state, with a particular focus on stunting reduction amongst women, adolescent girls, and children under five years. In order to achieve this goal, APPI has been working closely with a number of existing public service delivery platforms over the past three years including the Integrated Child Delivery Services (ICDS) as well as various networks of civil society organisations, private sector stakeholders, and community actors. Sixteen key areas of intervention focus were identified which included governance accountability. As part of this strategic effort, in August 2016, APPI entered into a new partnership with the Odisha-based NGO Society for Promoting Rural Education and Development (SPREAD) to design and roll out a three-year community accountability social audit programme entitled ‘Collective Action for Nutrition’ (CAN). In December 2017, APPI commissioned the Institute of Development Studies (IDS) to act as the lead technical partner in the design and implementation of an evaluation of the CAN programme to assess its effectiveness. This final report summarises the background, evaluation design and methods, main findings, and recommendations from the evaluation, incorporating primary and secondary qualitative and quantitative data collection. The main target users of the report are APPI, SPREAD, and the Government of Odisha, as well as others wanting to learn from the social accountability model. It is hoped that findings will feed into ongoing plans and preparations for the next phase of the social audit programme due to commence in December 2018. 1.2 Undernutrition in Odisha Over the past decade, India’s coverage of basic health and nutrition services has steadily improved, partly as a result of universalisation of the government’s Integrated Child Development Services (ICDS), which provides health, nutrition, and education services to young children. Despite substantial investments in roll-out, however, significant coverage and quality issues remain at community-level (Avula et al. 2015; Das Gupta et al. 2005). Less than 55 per cent of mothers and children receive any essential health and nutrition inputs (Avula et al. 2015), 38.4 per cent of children under five are stunted and under 10 per cent of children aged 6–23 months receive an adequate diet (Ministry of Health and Family Welfare (2016). Odisha has an estimated poverty rate of 32.6 per cent which masks the social and spatial variation of poverty and inequity across the state (Thomas et al. 2015). For example, rural poverty rates in the KBK+ districts are 68 per cent compared to the state rural average of 46.9 per cent. There have been some notable overall improvements in health and nutrition outcomes in recent years; for example, infant mortality rates have declined from 65 per cent in 2005/6 to 40 per cent in 2015/16, and rates of stunting in children under five years (height for age) have fallen from 45 per cent to 34.1 per cent in this same period (less than the national average of 38.4 per cent). However, the incidence of child wasting (weight for height) has increased from 19.6 per cent to 20.4 per cent; 34.4 per cent of children under five are underweight; and rates of women with anaemia have risen from 51 per cent to 61.1 per cent (Ministry of Health and Family Welfare 2016). In addition to this, major inequalities of nutrition outcomes still exist based on socioeconomic status, caste, ethnicity, gender, and 21 geography. Therefore, fighting malnutrition, particularly for children and women, remains a major concern and priority. 1.3 Social accountability and social audits Much of the global focus on addressing the high burden of malnutrition has been on strengthening supply-side institutions and policies (Waldman, Reed and Hrynick 2017; Gillespie and Margetts 2013). However, recent accountability literature suggests that this approach is too technocratic and ‘top down’, treating individuals as passive users rather than active citizens that engage with state services in multiple ways (Fox 2015). ‘Social accountability’ approaches offer an alternative model that emphasises demand-side citizen participation in state processes to help shape service improvements. A plethora of initiatives have emerged under this umbrella, including social audits, community scorecards, and citizen report cards (Gaventa and McGee 2013). Whilst approaches vary, most share an interest in the power of information and assume that by publicly exposing weaknesses in services, government actors will be forced to respond and improve service delivery (Houtzager and Joshi 2008). Several recent reviews focusing on social accountability (e.g. Fox 2015; McGee and Gaventa 2011; Joshi 2013) point to the growing but mixed evidence of their impact, which they attribute in part to a lack of clear theories of change or lack of rigorous study design. A few studies have reported significant impacts, including for example, in service delivery in India (Ravindra 2004) and Uganda (Björkman and Svennson 2009). Nonetheless, there is a need for more theory-based research and evidence to understand the particular impact pathways and contextual assumptions that might explain the reasons for success and failure of this type of approach. Findings from this evaluation will be used to contribute to this evidence gap. India has been at the forefront of innovation in social accountability initiatives across many sectors (Pande 2007) and the use of social audits to expose and address state corruption in particular has gained widespread recognition (Fox 2015; Maiorano 2014). In their Social Audit Manual of the Mahatma Gandhi National Rural Employment Guarantee Act (MGNREGA), India’s Ministry of Rural Development define a social audit as ‘verification of implementation of a programme/scheme and its result by the community with active involvement of the primary stake holders’ (GoI 2005). Emerging from informal citizen-led action alongside the Right to Food Movement, social audits have since been formally institutionalised in national policies and programmes. This includes India’s National Food Security Act (NFSA) (GoI 2013) (the focus of the SPREAD CAN programme and this evaluation) which aims to ensure ‘food and nutritional security by ensuring access to adequate quantity of quality food at affordable prices’ (GoI 2013). The Act covers community nutrition schemes, school feeding, maternity entitlements, and food distribution (discussed more in Section 1.4). Other legal frameworks in India that refer to social audits include the Supreme Court’s Direction (2002); Right to Information Act (2005); and Right to Public Services Act (passed in 2012 in Odisha). There have been few independent studies of social audits within the context of the NFSA to date. Existing evidence on the institutionalisation of social audits in India has focused on the Government of Andhra Pradesh’s decision to mandate social audits as part of the National Rural Employment Guarantee Act. An evaluation of this approach reported significant positive impacts on employment and exposure of corruption compared to other states (Afridi and Iversen 2014), but questions have been raised about the model’s sustainability given the capacity and resources required (Aiyar and Samji 2009). Other questions on the nature of citizen–state interactions (Fox 2015; Joshi 2013) highlight differences between confrontational social audit approaches which are designed to expose government failures and pressurise public officials into acting, versus more collaborative community scorecard processes. The Government of Odisha has shown a strong interest in the use of social audits as a tool to improve service delivery and have recently set up the Odisha Society for Social 22 Audit Accountability and Transparency (OSSAAT) under the Panchayati Raj Department and housed in the State Institute of Rural Development, Bhubaneswar. The first official Governing Body meeting was held in June 2018. The main objective of OSSAAT, said Principal Secretary Sri Singh, ‘is to create enabling environment for the Gram Sabha and Palli Sabha to conduct impartial and effective social audits for the development schemes and programmes in its territory.’ Whether or not their existing governance structures can effectively and sustainably manage the sensitive processes social audits entail is a key consideration alongside continued delivery of NFSA services. 1.4 National Food Security Act (NFSA) The NFSA, passed in 2013, aims to provide food and nutrition security to citizens by ensuring access to adequate quality and quantity of food at affordable prices through bringing together four core existing programmes under one Act:  The Targeted Public Distribution System (TPDS) – a food and essential items distribution scheme which targets food-insecure households;  The Midday Meals (MDM) programme – a school-based feeding programme targeting children ages 6–14 years;  Integrated Child Development Services (ICDS) – a supplementary feeding programme targeting children between six months and six years, and pregnant and lactating women;  Maternity entitlements – the Odisha state programme is referred to as MAMATA. The Act has a strong focus on nutrition support to women and children and has also mandated social audits as a process through which citizens can collectively monitor implementation of these food and nutrition entitlements to ensure adequate access to these entitlements. 1.4.1 Targeted Public Distribution System (TPDS) The Targeted Public Distribution System (TPDS) is the largest social protection programme in India, in terms of both government expenditures and number of beneficiary households. The NFSA entitles 50 per cent of the urban population and 75 per cent of the rural population to receive food benefits under the TPDS. The previously universal Public Distribution System (PDS) was converted into a targeted system in 1997. Following this shift, beneficiaries were identified and divided into two categories, as either below the poverty line (BPL) or above the poverty line (APL). People in each category were entitled to a set of food grains at differing quantities and prices. In 2000, an additional classification of Antyodaya Anna Yojna (AAY, poorest of the poor) was introduced to provide the abject poor with dedicated food grain allocations at highly subsidised prices. In 2013, the scope and mandate of the TPDS expanded significantly through its incorporation into the National Food Security Act (NFSA). Before 2013, most households had one of the three types of ration cards previously described (BPL, APL, AAY). Now there are only two categories, namely Priority Household (PHH) and Antyodaya Anna Yojna (AAY). The distribution of subsidised cereals through the TPDS is a major instrument for ensuring the availability of affordable food grain to the public, especially the poor. The quantity and price of food grain available to households depends on the type of ration card assigned to them. Persons belonging to PHH are entitled to receive 5kg of food grains (rice, wheat, and 23 coarse grains) per person per month at subsidised prices of Rs.3/2/1 per kilogramme. AAY households are entitled to 35kg per month at a rate of Rs.1 per kilogramme. The TPDS provides subsidised food grain through a large network of government-licensed fair price shops (FPSs). The government purchases food grain from farmers based on a predetermined price floor, at a minimum support price (MSP). Then it provides highly subsidised food grain to low-income and vulnerable households. State governments are responsible for identifying beneficiaries and selecting fair price shops. The district food office provides beneficiary households with a ration card that serves as identification for accessing the TPDS. Once a household receives its ration card, its members can visit their FPS (all households are assigned to a specific FPS) to buy their food grain every month. Information on eligibility and purchases made from the TPDS are recorded on the card. Recent technological changes have been introduced in order to improve delivery of the TPDS. Beneficiaries are enrolled biometrically by the Unique Identification Authority of India, which assigns each resident a unique Aadhaar identification number. The use of Aadhaar for the distribution of TPDS rations has been controversial and critics have pointed out the large number of access and technological failures and identification errors, amongst broader issues to do with security and privacy. Both exclusion and inclusion errors were recorded, i.e. people who should be included in the TPDS as eligible cardholders but who are excluded in reality, and people who are not eligible to be covered by the TPDS but who are included by mistake or through inefficiency in implementation. 1.4.2 Midday Meal (MDM) The Midday Meal programme was first established in 1995 with a view to improve school enrolment, retention, and attendance, and simultaneously improve nutritional levels among children. Under this school-based feeding programme, children attending government-run schools, aged up to 14 years, are entitled to one free hot cooked nutritious meal per day at school (excluding holidays), served according to prescribed age-appropriate nutritional standards. The scheme is either provided via a decentralised model where meals are cooked on-site by local cooks, helpers, or self-help groups (this is the most common model, particularly for rural areas) or a centralised model where meals are prepared in external kitchens through public–private partnerships and delivered to schools (more common in urban areas with high density of schools). In case of non-supply of entitled meals, the rights holders are eligible to receive a food security allowance. 1.4.3 Integrated Child Development Services (ICDS) The ICDS scheme provides a package of six services to children below six years of age, pregnant women, and nursing mothers. These are supplementary nutrition, immunisation, health check-up, growth monitoring and referral services, non-formal pre-school education, and nutrition and health education. The ICDS scheme provides specific guidance on the weighing and growth monitoring of children, and counselling for mothers/caregivers by AWWs. Each child in the age group of 0– 3 years must be weighed at least once every month and assessed against the World Health Organization (WHO) New Growth Standards register. If the growth trajectory is deemed to be normal, the AWW advises the mother/caregiver to maintain the feeding and hygiene practices that she has been following. If the growth trajectory is below expected parameters, mothers/caregivers are advised on age-appropriate feeding, food preparation, and hygiene practices. Severely malnourished children are referred for further medical check-ups. Supplementary nutrition includes Take-Home Ration (THR) and Hot Cooked Meal. THR or chhatua (a wheat-based preparation) and two eggs per week are given to pregnant and 24 lactating mothers, and children from six months to three years. A hot cooked meal is provided daily to children between three and six years of age who attend the centre. Those children between three and six years who are found to be severely malnourished are also given THR, as well as the hot cooked meal. THR packets are colour-coded (yellow/sky- blue/red2), to ensure that the nutritional supplements are consumed by the intended beneficiary rather than by other family members. 1.4.4 MAMATA The MAMATA conditional cash transfer scheme was launched by the state of Odisha to alleviate maternal and infant undernutrition. It provides monetary support to pregnant and lactating women for the first two live births, to enable them to seek improved nutrition and promote health-seeking behaviour. The intermediate objectives identified by the Odisha government are as follows:  To provide partial wage compensation for pregnant and nursing mothers so that they are able to rest adequately during their pregnancy and after delivery;  To increase utilisation of maternal and child health services, especially antenatal care (ANC), postnatal care, and immunisation;  To improve mother and child care practices, especially exclusive breastfeeding and complementary feeding of infants. The specific target group for this scheme are pregnant and lactating women of 19 years of age and above, except those who already avail of the maternity benefit (i.e. who are themselves employees or wives of employees of state government/central government). The payable amount is transferred directly to the account of the rights holder. This was originally allocated in four instalments. From April 2017 onwards, the money is transferred in two instalments, dependent on their meeting the desired conditions, as specified below. Allocation rules prior to April 2017 set the total monetary support at Rs.5000/- to be allocated as follows:  First instalment: at the end of the second trimester of pregnancy, Rs.1,500/-  Second instalment: after completion of three months after delivery, Rs.1,500  Third instalment: after the infant completes six months of age, Rs.1,000/-  Fourth instalment: after the infant completes nine months of age, Rs.1,000/- Allocation rules after April 2017 set the total monetary support at Rs.5000/- to be allocated as follows:  First instalment: at the end of six months of pregnancy, Rs.3000/  Second instalment: after the infant completes nine months of age, Rs.2000/- Conditionalities for the first instalments include the registration of pregnancy within six months, the receipt of at least one antenatal check-up, iron folic acid (IFA) tablets, one tetanus (TT) vaccination, and at least one counselling session. Conditionalities for the second instalment include the registration of childbirth, child vaccination requirements, weighing of the child at least two times after birth, and at least two IYCF counselling sessions. 2 THR packets are yellow for pregnant and lactating mothers, sky-blue for children aged six months to three years, and red for severely malnourished children. 25 1.5 SPREAD CAN programme 1.5.1 Programme background, aims, and overall approach Founded in 1989, SPREAD is an Odisha-based NGO that works to empower marginalised communities to access their rights in a number of areas including land, food, and education. The organisation works primarily in the Koraput district of Odisha and neighbouring KBK districts which have a high proportion of tribal populations and people living below the poverty line relative to the rest of Odisha. As well as addressing the immediate needs of the communities they work with (such as provision of medical emergencies, health checks, educational supplies, etc.), SPREAD also facilitate participatory development processes that build the capacities of community-based organisations (e.g. self-help groups, village development committees) to develop long-term sustainable development with the community. This often starts with identifying the immediate needs of families and progresses to wider issues faced at the village, GP, and block level. The goal of the three-year ‘Collective Action for Nutrition’(CAN) programme (August 2016– July 2019) supported by APPI is ‘Reducing malnutrition among children and women by facilitating efficient implementation of food and nutrition programmes, ensuring transparency, downward accountability, and community participation’. SPREAD describes the following key intervention areas as the focus of their programme (SPREAD 2017a, b, c, d, e):  Build knowledge of community on the National Food Security Act (NFSA) to improve accessibility;  Promote participation of community and community-based organisations (CBOs) in decision-making on access and entitlements to food and nutrition programmes;  Effective implementation of NFSA and health services (including village health and nutrition days, nutrition rehabilitation centres, village health and sanitation committees (VHSCs), etc.);  Build capacities and strengthen community-based institutions;  Working towards Panchayati Raj Institutions (PRIs) as nutrition champions;  Institutionalisation of the social audit process and grievance redressal mechanisms. The main intervention focus is a social audit process designed to sensitise communities to their rights and entitlements under the four primary schemes covered by the government’s National Food Security Act (NFSA) 2013 (see Section 1.4). By raising awareness of rights and grievances through a transparent participatory process of engagement involving local community members, village leaders, government officials, and others involved in the delivery of NFSA services, culminating in a public hearing to share testimonies and findings from the process, it is hoped that NFSA services will improve. Box 1.1 SPREAD definition of the social audit process Social Audit is a process of Deepening Democracy. It is a process, where the rights holders obtain information on all such schemes, programmes, systems impacting their lives; validate their truthfulness and work towards bringing a positive and reformative change. It is a participatory process which empowers citizens. Social Audit adds value to the whole idea of decentralisation and establishes the Community’s capacity for Planning, Monitoring, and Course Correction. Source: SPREAD Social Audit Manual (2017b). 26 The overall expected outcome for the programme is reduced malnutrition among the target communities (specifically children under five years) through improved delivery of nutrition services and entitlements and increased downward accountability of the ICDS through the social audit model. However, given the nature of the intervention and relatively short time frame of the APPI grant, both organisations acknowledged that they may not see immediate changes in nutritional outcomes for children. Other interim outcomes expected include: Household-specific project outcomes:  Increased knowledge of households (especially eligible women) on NFSA entitlements (especially TPDS), and on the MAMATA scheme;  Improved uptake of nutritional entitlements from ICDS for target groups (pregnant and lactating women; women of children under the age of three years; and adolescent girls);  Improved uptake of antenatal and postnatal services, and knowledge of IYCF practices by eligible women (with a specific focus on pregnant and lactating women; and mothers of children under the age of five years);  Behavioural change in adolescent girls (with respect to IFA supplementation);  Increased immunisation coverage of children under the age of three years;  Improved participation of households (and especially women) in community-level governance activities (with specific reference to Gram Sabhas, Palli Sabhas, and social audits);  Improvement in intra-household decision-making for women on issues related to food security and nutrition. Community-specific project outcomes:  Emergence of empowered communities with more accountable ICDS centres;  Improved institutional delivery of nutrition services and entitlements in Gram Panchayats;  Increased community-level demand for improved nutrition services and entitlements; and improved community participation in decision-making related to food and nutrition programmes;  Establishment of (mal)nutrition as a critical agenda for the Gram Sabha and making ‘malnutrition-free villages’ a goal for PRIs (Panchayati Raj institutions). 1.5.2 Overall design There are nine objectives of the social audit, as outlined in the August 2018 version of the SPREAD Social Audit Manual (p10–11): 1. Promote transparency and accountability in the implementation of a programme. 2. Inform and educate and mobilise community and rights holders about their rights and entitlements of the programme or scheme during the course of social audit. 3. Provide a collective platform, which is inclusive and participatory, to different stakeholders to express their needs and grievances. 4. Improve capacity of the local stakeholders participating the during social audit process. 5. Democratise decision-making by providing a platform for implementing authorities to be accountable to rights holders. 6. Strengthen the scheme by deterring corruption and irregularities and improve the delivery of the programme. 7. Influencing policies with grass-roots realities and evidences. 8. Identifying systemic gaps. 27 9. Establishing and exploring linkages with PRIs and community-based organisations. According to the Social Audit Manual, a social audit is split into three phases: 1. the pre- social audit (seven days’ duration), 2. the social audit (seven days’ duration), and 3. the post- social audit (undefined duration – a continuing process). The whole process is split into eleven steps: Box 1.2 The 11 steps of the social audit Step 1 Meet the official and Panchayat functionaries as an entry meeting Step 2 Collect the official documents required for field verification, review the registers and documents Step 3 Verification physically of the storage centres in schools, Anganwadi Centres, and FPS, door- to-door verification Step 4 Conducting village meeting and FGDs with rights holders Step 5 Filling the verification formats, seeing information gaps Step 6 Consolidation of records that are required for social audit Step 7 Ensuring government official participation in the Gram Sabha on the specified date as well as participation of all rights holders Step 8 Report preparation Step 9 Presenting the social audit report with evidence in the Gram Sabha Step 10 Noting the minutes carefully Step 11 Follow-up of the social audit Gram Sabha with action taken report from the concerned department Pre-social audit phase design The first part of the process is to develop a calendar that includes details of important dates within the areas the social audit will be carried out. This is so that the audit can be conducted at an appropriate time for these other events, or ideally in conjunction with an existing planned Gram Sabha meeting. Next, volunteers who will conduct the social audit are identified and recruited, giving priority to local people who are from marginalised groups. Thirdly, the Social Audit Committee should be set up, and formed at least seven days in advance. The committee should comprise 10–15 people including PRI members, SHG members, Gaon Kalyan Samitis (GKS) members, disabled people, and respected people from the GP. The social audit team should also interact with the Community Committees formed for the NFSA programmes. Social audit phase design The social audit itself is expected to take seven days in total as outlined in the calendar presented in Table 1.1. 28 Table 1.1 Model calendar for social audit process Days Activity Discussion with Points of discussion Day 1 Village meeting, Focus Group Discussion (FGD) and meeting with PRI members Community and PRI members Village profile, schemes and services, exclusion and inclusion errors, hunger and malnutrition, roles and responsibilities of PRI members Day 2 Field survey and data collection Beneficiaries, AWW, ASHA, teachers, jogan sahayak, members of SMC, JC, MC, PDS advisory committee Demographic profile of the village, infrastructure of AWC, school, number of beneficiaries, register verification, cross-check of secondary data with primary data Day 3 Field survey and data collection Beneficiaries, AWW, ASHA, teachers, jogan sahayak, members of SMC, JC, MC, PDS advisory committee Demographic profile of the village, infrastructure of AWC, school, number of beneficiaries, register verification, cross-check of secondary data with primary data Day 4 Field survey and data collection Beneficiaries, AWW, ASHA, teachers, jogan sahayak, members of SMC, JC, MC, PDS advisory committee Demographic profile of the village, infrastructure of AWC, school, number of beneficiaries, register verification, cross-check of secondary data with primary data Day 5 Meeting with service providers AWW, ASHA, teachers, jogan sahayak, members of SMC, JC, MC, PDS advisory committee Based on the data collection from the field, the findings will be shared with service providers Day 6 Data compilation and report writing Team members Day 7 Public hearing and presentations Note: jogan sahayak = supply assistant; SMC = School Management Committee; JC = Jaanch Committee. Source: Authors’ own based on SPREAD Social Audit Manual. The Social Audit Manual lists the principles that should be adhered to during the implementation of social audits (p10):  Social auditors responsible for facilitating social audit need to have access to the complete information prior to social audit, to assimilate and verify the information provided to them by the administration.  A social audit Gram Sabha and public hearing should mandatorily have officials and PRI members to be present, and answer the queries raised there.  Outcomes of a social audit must have legal sanction and the state governments should enact specific rules for this.  Social audit must be conducted in every Gram Panchayat once in every six months.  Social audit Gram Sabha must be presided by an authority other than from the implementing agency.  During the social audit process make sure that a person, related directly, or indirectly to any of the stakeholders of the schemes or services, should not be a part of it, as there is chances of getting false/biased information.  The quorum of a Gram Sabha as defined in the Odisha Panchayati Raj Act will apply to the social audit Gram Sabha too. 29  The social audit Gram Sabha and social audit public hearing should be open for all the members of the public to participate. This includes people from the press, civil society organisations and members, SHG members, etc.  The social audit team should present all recorded information to the Gram Sabha orally.  Social audit must include the exercise of the officials (independent observers) taking and announcing a decision on each deviation presented which is also recorded in the social audit public forum resolution.  A social audit is a joint exercise of the government and citizens. It is the responsibility of the Gram Sabha to conduct a social audit with the help of the social audit facilitators. But acting on the grievances identified during a social audit within a fixed time period is the sole responsibility of the state government.  Civil society organisations should be an important part of the social audit. They should participate in the social audit Gram Sabha and social audit public hearings. Post-social audit phase design After the social audit has finished, the social audit team is expected to complete a number of tasks, including writing up and sharing the report from the public hearing (in the local language), ensuring the grievance redressal process is followed for every issue raised about the NFSA, and following up grievances, within a specific time frame. The team is expected to hold the relevant officials accountable to resolve grievances. 1.5.3 SPREAD CAN implementation plan According to the SPREAD implementation plan (see SPREAD 2016a, Implementation PLAN Final 4.8.16) a total of 240 social audits were planned to take place under the CAN programme. This included 24 pilots within the first year of the project, and the remaining 216 social audits in a corresponding number of GPs (within 24 blocks, 6 districts, and including 2,923 villages) were planned for year 2. The whole programme was planned to take three years, starting August 2016 and ending July 2019. Year 1 August 2016–July 2017 The 24 pilot social audits were planned to take place between February 2017 and July 2017. Year 2 August 2017–July 2018 The remaining 216 social audits were planned to take place during the second year of the programme, 108 between August 2017 and January 2018, and the final 108 between February 2018 and July 2018. Year 3 August 2018–July 2019 Further social audits carried out in the same locations by the SS volunteers that were trained by SPREAD. 1.6 Baseline characteristics of target groups In this section, we summarise the demographic characteristics of households and particular target groups included in the baseline quantitative sample of 116 GPs. The survey is representative of households in the SPREAD programme area but the SPREAD programme area itself is not necessarily representative of the population living in the KBK districts. In fact, SPREAD targets the most food-insecure regions within the KBK area for its CAN programme so that we should expect some systematic differences between the characteristics of study participants and those of the general population. Table 1.2 summarises the mean of some key socioeconomic and food security indicators by districts based on our household survey and contrasts them with the district-level means provided by 30 the POSHAN programme, which collates data from the NFHS4. The following subsection will refer to this table when describing survey respondents. 1.6.1 Household characteristics Religion and caste: 97.7 per cent of respondents belong to Hindu households (defined as the religion of the head), 2.2 per cent to Christian households and 0.1 per cent to Muslim households. This religion breakdown very closely mirrors the figures from the census in Odisha in 2011 for each single district in the sample. 51 per cent of respondents are Scheduled Tribes (STs), 25 per cent belong to Other Backward Castes (OBCs), 22 per cent are Scheduled Castes (SCs), and just 3 per cent are in the ‘general’ category. This caste composition is quite different from that of the census for most districts. Our survey tends to 31 Table 1.2 Baseline characteristics in selected districts – existing data and evaluation data Source: Authors’ own and POSHAN (2018). Characteristic/district Balangir (%) Kalahandi (%) Koraput (%) Malkangiri (%) Nabarangpur (%) Nuapada (%) Source POSH AN (NFHS 4) SPREAD evaluation survey POSHAN (NFHS4) SPREAD evaluation survey POSHAN (NFHS4) SPREAD evaluation survey POSHAN (NFHS4) SPREAD evaluatio n survey POSHAN (NFHS4) SPREAD evaluation survey POSHAN (NFHS4) SPREAD evaluation survey Exclusive breastfeeding 53.4 82.5 67.4 81.4 70.2 79.4 66.2 80.3 71.7 68.4 49.2 80.4 Adequate diet 7.3 32.5 4.5 11.2 3.4 10.3 16.8 21.9 11.5 17.7 2.6 18.8 Mother and Child Protection Card 100 96 96.7 88 98.2 89 95.4 92 92 86 99.1 85 Women who are literate 61.9 61 46.1 38 39.7 27 34.8 32 41.8 23 49.9 40 Women with ≥10 years of education 21.6 21 18.7 11 14.5 7 11.8 6 10.7 3 19.9 10 HH has an improved water source 94 96 93.6 93 84.7 90 89.3 93 98.4 99 95 94 HH has an improved sanitation facility 14.1 21 14.9 19 18.2 11 16.7 12 16.1 9 20.2 27 Open defecation 83.6 79 87.5 80 81.3 87 89.3 87 89 90 84.6 73 32 over-sample the SC (except in Malkangiri where it is the opposite) and ST categories of the population. This is to be excepted as the social audits are implemented in the most remote parts of each district, where the SC/ST share of the population is the highest. Access to drinking water: 77 per cent of households obtain their drinking water from tube wells, 11 per cent from a public tap/sandpipe, 2.5 per cent from an unprotected well. 2.2 per cent of respondents receive their water directly through a pipe to their house/yard. Overall, 94 per cent of respondents get their drinking water from an improved source, a figure in line with POSHAN’s NFHS4 statistics. Access to toilet facilities: 84 per cent of respondents do not have any access to a toilet and defecate in the open. This is in line with information provided by POSHAN (2018) for the sample districts (see Table 2.1). The remaining 15 per cent of respondents use some variations of pit latrines (almost 10 per cent of respondents use a flush-to-pit latrine, 3.2 per cent have access to a pit latrine with slab and 2.2 per cent use a flush to sceptic tank). The rate of open defecation ranges from 73 per cent in Nuapada to 90 per cent in Nabarangpur. Household assets: The average respondent reports that their household owns 2.7 assets out of a list of 9 assets. There is a wide geographic disparity in the assets index as its mean in the sample ranges from two in Koraput to four in Balangir District. 82 per cent of respondents have access to electricity and 12 per cent have access to solar electricity. The most commonly owned assets are: mobile telephone (66 per cent), electric fan (38 per cent), television (33 per cent), and wardrobe (26 per cent). Generators, air conditioners, laptops/computers, and DVD players are all owned by less (and sometimes considerably less) than 5 per cent of respondents. Cooking facilities: Just over half of respondents have a separate room for cooking. 93 per cent of respondents use wood as cooking fuel and 6 per cent use LPG/natural gas. Close to three-quarters (72 per cent) of respondents have an earth floor and the remaining quarter have a cement floor. Construction of home dwelling: Materials for the roof and external walls are relatively varied in the sample. Almost 50 per cent have tiles, 29 per cent have asbestos, 11 per cent have thatch/palm leaf, and 8 per cent have cement for the roof. External walls are made of dirt for 46 per cent of respondents, of cement for 26 per cent of respondents, of bricks for 20 per cent of respondents, of bamboo, or stone with mud for 5.5 per cent of respondents, and of stone for 2.2 per cent of respondents. Food security: to measure food security, we implemented the Food Insecurity Experience Scale (FIES) proposed by Ballard et al. (2013) for the FAO. The scale is made up of eight questions with dichotomous yes/no responses. We have used a 12-month reference period and we applied the scale at the level of the respondent (individual scale). The questions are not meant to be analysed separately. The mean FIES score is 3.02 (out of a maximum of 8). The FIES does not vary much across districts as it ranges from 2.7 in Kalahandi to 3.1 in Koraput and Malkangiri. However, Nabarangpur stands out as the most food-insecure district as the mean FIES there is 3.6. The extent of food insecurity is higher for the SC and ST population (3.2) than for OBCs (2.6). 1.6.2 Primary caregivers The baseline survey included answers from 1,884 primary caregivers. The number of interviewed caregivers per GP ranged from 11 to 23, with a mean of 16.6. Age, gender, and household size: Primary caregivers are 25 years old on average (three- quarters of caregivers are below 29, and the oldest respondent is 60), and are 33 overwhelmingly female (only 1.3 per cent of primary caregivers are male). 80 per cent of primary caregivers are the spouse of the household head, 16 per cent are the daughter/daughter-in-law of the household head, and less than 3 per cent (2.6 per cent) are household heads themselves. 98 per cent of primary caregivers have just one child below 24 months. Households have 3.8 members on average, with 25 per cent of them having more than five members. Literacy rates: 37 per cent of the caregivers are literate, which is far below the Odisha-wide literacy rate for women which is 64 per cent. Rates of female literacy are noticeably lower in the SPREAD sample than in the whole district for Kalahandi (38 per cent against 46 per cent), Koraput (27 per cent against 40 per cent), Nabarangpur (23 per cent against 42 per cent) and Nuapada (40 per cent against 50 per cent). Overall, these figures suggest that the SPREAD programme targets the most disadvantaged districts, and within these districts, that it targets the most disadvantaged areas. Employment: 84 per cent of PC respondents are housewives, 8 per cent are self-employed, 5 per cent are employees, and just under 3 per cent work without pay (as an apprentice or in a family business). 86 per cent of PC respondents who reported a work activity did so in agriculture (57 per cent in their own farm and 35 per cent as an agricultural labourer). Women’s dietary diversity: we also implemented the Women’s Dietary Diversity Score (WDDS) proposed by Arimond et al. (2010). The WDDS is simply the sum of all the food groups consumed by the respondent on the day prior to the survey (if this was a typical day; otherwise, we used the last typical day). The WDDS is based on 21 food groups, using the most disaggregated food groups’ indicators (the base version of WDDS uses nine food groups). The mean score of WDDS is 6.7 (out of a maximum of 21). The mean WDDS is around 7.2 in Nabarangpur, 7.3 in Nuapada, and 7.6 in Balangir. Districts with lower values of WDDS are Koraput (5.9), Kalahandi (6.2), and Malkangiri (6.3). The mean WDDS is lower among SC/ST populations (around 6.5) than among OBCs (7.2). Nutrition knowledge: we implemented a small questionnaire to test nutrition knowledge as in Hoddinott et al. (2016). This entails four questions on breastfeeding, three questions on supplementary feeding, and seven questions on health and nutrition. Respondents on average correctly answered 8.8 questions out of 14. In detail, they correctly answered 2.2 questions on breastfeeding (out of 4), 1.9 questions on supplementary feeding (out of 3), and 4.7 questions on health and nutrition (out of 7). Committee membership: we asked about the existence of 14 possible groups or committees at the local level. Overall, respondents know about the existence of 4.3 groups/committees, with the mean ranging from 3.8 in Nuapada to 4.6 in Nabarangpur. Caregivers are members of 0.6 groups/committees on average, while 34 per cent of caregivers belong to at least one group/committee. 1.6.3 Pregnant women The baseline survey included answers from 510 pregnant women (who were not also primary caregivers). The number of interviewed caregivers per GP ranged from 1 to 11, with a mean of 5.6. Age, gender, and household size: Pregnant women are 24.5 years old on average (three- quarters of pregnant women are below 27, and the oldest respondent is 42). 77 per cent of pregnant women are the spouse of the household head, 20 per cent are the daughter/daughter-in-law of the household head and less than 2 per cent (1.8 per cent) are household heads themselves. 34 Food security: The mean FIES score among pregnant women is 2.6 (out of a maximum of 8), which is noticeably lower than among primary caregivers (3.2). Similarly, the mean WDDS score is 7 for pregnant women against 6.7 for primary caregivers. Both results indicate a lower extent of food insecurity among pregnant women than among primary caregivers. Nutrition knowledge: res