Innovations in Impact Evaluation: What Have We Learned? Insights from the Gender-Sensitive Risks and Options Assessment for Decision Making (ROAD) to Support Work-in-Freedom Phase 2 Claudia Ringler & Nazmun Ratna (with Muzna Alvi, Farha Sufian, Sawsan Abdulrahim, May Adra, Zahid Choudhury) February 22, CEDIL conference (virtual) https://www.ifpri.org/project/gender-sensitive-risks-and-options-assessment-decision-making-road-support-work-freedom Intervention and Evaluation • WiF-2 project: ILO reduces vulnerability to trafficking and forced labour of women and girls from South Asia across migration pathways leading to the care and garment sectors in Arab States • ROAD migration evaluation: How are female migrants’ options shaped by gender and what is the potential of empowerment intervention to reduce trafficking and forced labor? • Geographies: Countries of origin: Bangladesh & Nepal; Destination countries: Jordan & Lebanon Innovative methods in the ROAD migration evaluation 1. ROAD process application to migration: Migration actor networks (using Net- Map analysis) 2. ROAD causal risk diagrams: risk events, drivers, triggers, impacts, controls and mitigants for a broader understanding of entry points for vulnerability reduction 3. Development of the Women’s Empowerment in Migration Index (WEMI) to support program interventions focused on areas of disempowerment 4. Training of women migrant domestic workers as researchers who implement qualitative fieldwork & fieldwork with elderly employers The ROAD Process 1) ROAD (Risks and Options Assessment for Decision- making) process: Net-Map Net-Map 1. Eight core connecting actors across 4 countries: o Ministries of Labor (country of origin and destination) 2. Destination countries o Ministries of Foreign Affairs o Ministries of Interior o Recruitment agencies o Embassies o Human rights commissions o Migrants 2) ROAD (Risks and Options Assessment for Decision- making) process A threat, trend or other risk source causing a DRIVER trigger to occur An action that modifies the likelihood of a driver causing Control the policy intervention outcomes being inconsistent with policy vision TRIGGER An event that is the immediate cause of a risk event Control An action that modifies the likelihood of a trigger causing the risk An event with uncertain consequences: forced labour RISK EVENT and trafficking or other adverse events experienced by female workers across the entire migration chain Mitigant An action that modifies the likelihood of the policy intervention causing the consequence CONSEQUENCE Outcome of a risk event affecting objectives C limate change: National policy of origin Unfair Recruitment Patriarchy and social floods, droughts etc country1 Practices stigma2 Asymmetric information about migration process C8 C5 C7 C5 Escape from Economic Sexual Becoming free Habituation to Influence of debt vulnerability harassment from gender - the lifestyle of intermediaries based violence development (middlemen) in migration Lack of decent jobs in country of origin Shortcut to build wealth quickly C1 C3 C1 C3 C3 C6 C5 Risk event Trafficking and / or forced labour of low skilled female migrants from Bangladesh to the Middle East C3 C4 M3 C2 M2 M2 M4 M1 M1, M3 Lack of decent jobs in host country Labour Physical and Slavery type Reduced income and Irregular migrants in exploitations psychological torments practices remittances harmful situations Causal Risk Diagram Bangladesh Breakout Room Group 1 Triggers and Drivers: Bangladesh & Nepal (in red) Economic Social Triggers • Shortcut to build wealth • Freedom from Gender-Based Violence • Poverty/ Economic vulnerability/ Escape from • Violence Against Women debt/Lack of decent jobs • Influence of intermediaries • Materialistic enticements/ Allurement of better • Social pressure: Social stigma for lifestyle single/divorced women • Extreme workload/ Family responsibilities for single • Good experience of previous women undocumented migrants (friends and • Huge demand from the Middle-Eastern countries/ relatives) Sudden increase in employment opportunities • No access to get information about safe • Sudden death of family members/male earning migration members Drivers • Climate change & 2015 earthquake • Patriarchy & Gender-Based Violence/ Lack of recognition of women’s work • Caste-based discrimination • Lack of participatory governance • Female headed household and lack of social capital • Demand and supply in the global market Triggers and Drivers: Lebanon & Jordan (in red) Economic Social Triggers • Declining capacity of employers to • Social norms afford a domestic workers • Lack of awareness • Financial crisis • Cultural legacies • Failing state & Corruption • Escape from Interpersonal violence (including domestic • Aspirations for better life / Economic violence)/ Escape forced marriage opportunities • Hearing about previous experiences of female workers • Supporting family/poverty reduction / • Discrimination / Racial and religious prosecution Indebtedness • Contact with recruiters in countries of origin • Children’s education • Reputation issues (can work in factory in other country but not home country) Drivers • Kafala System • Weak role of the Ministry of Labour in proposing legislation • Absence of translated contracts • Patriarchal abuses in South Asia/ Stereotyping of women who migrate • War & conflict / Seeking refuge • Economic situation, security & stability in Jordan/ Strong currency in Jordan • Good legislative environment in Jordan/Perception of decent work Controls and Mitigants Bangladesh Lebanon Controls • Agreement with destination locations beyond contracts to • Dismantle kafala system ensure better working conditions • Add migrants into labour laws • Skill development of women including soft and life skills • Adoption of a unified labour contract in line • Collaborative effort of GO and NGO for ethical / informed with international standards migration • Strengthening and activating the role of the • Improved wage structure at home and source country Ministry of Labour in legislation and oversight • Social policies & protection • Follow up with destination country authority and create opportunities (better active collaboration and create spaces) Mitigants • Hotline and resource centres in home country • Legalizing freelance work • Certification of skills gained in host country • Strengthening the community of migrant • Ensure savings through local bank accounts for migrants workers • Reintegration at home with family: Psycho-social • Dismantle kafala system integration, economic integration (micro-business • Inclusion of domestic work in the labour law collaboration with Government and NGOs, availability of loans and information) • Regular monitoring of human rights situation at destination • Media interventions and media role in portraying migrant women 3) WEMI (Women’s Empowerment in Migration Index) Why calculate empowerment • Study how migration affect women’s empowerment: “Does migration make women more empowered”? • Women’s Empowerment in Migration Index (WEMI) is a standardized measure to assess the empowerment of women who migrate • Based on the Pilot study conducted in Bangladesh in 2022 for returnee migrants , we identify: • sources of disempowerment • how migration specific interventions targeted to reduce forced labour and or trafficking of women affect their empowerment Adapt Women’s How to Calculate Pro-WEAI Empowerment in WEMI Empowerment Migration Index Contributions of WEMI • How and to what extent access to information (A2I), created through training and social networks influence agency? • Access to training • Bonding, Bridging and Linking Social Capital • Agency in the work • Attitude towards violence from employer • Respect between employee and supervisor • Defining psychological wellbeing and life satisfaction as a dimension of achievement Index Construction Each indicator receives assigned proportion of the overall weight WEMI WEIGHTED AVERAGE WEMI SCORE (3DE) (NO GPI) EMPOWERED if adequate in 75% indicators Results Table 1. WEMI Results (using 75% cut off-score) Indicator Women Number of observations 1018 3DE score 0.60 Disempowerment score (1 – 3DE) 0.40 % achieving empowerment 0.14 WEMI=3DE % not achieving empowerment 0.86 Mean 3DE score for not yet empowered 0.53 Mean disempowerment score (1 – 3DE) 0.47 WEMI score (3DE score) 0.60 *Adequacy is said to be achieved when the individual WEMI score is above 0.75 out of a total 1 (following the Pro-WEAI methodology). PRerecesntuaglet sco:n tSribouutiorncs eofs in doicfa tdoriss toe Dmisepmopoweermrmentent Group membership 19% Feeedom of mobility 18% The largest Ownership of land and other assets 15% sources are lack of group membership Respect among team members 9% (19%), restricted Access to training 8% mobility in the destination country Access to and decisions on credit 7% (18%) and lack of asset ownership Respect among household members 7% (15%). Attitudes towards domestic violence 6% Attitudes towards violence from employer 5% 0% 2% 4% 6% 8% 10% 12% 14% 16% 18% 20% Robustness & Limitations • Validation analysis- WEMI positively associated with objective welfare & life satisfaction, psychological health, attitudes towards migration. • Sensitivity analysis different weights & cut-offs don’t make a difference • Empowerment is context specific (Destination vs Home country); Generalized index will not be able to capture country specific nuances • Retrospective answers for pilot study (recall bias); (Respondents: Returnee vs Current migrants) • However, possibility of Selection Bias if current migrants are interviewed at the destination country (more empowered women available for interviews) 4) Women migrants implementing research (Lebanon) Implementation • Participatory action research (PAR) • 9 WMDWs (Bangladesh, Ethiopia, Kenya, Nepal, Philippines, Sri Lanka) as co-researchers • How do WMDWs maintain social networks and organize collectively to reduce their risk to exploitation and poor labor conditions? → How do WMDWs try to improve their situation, whether through work alone or working together, to help others or themselves in their communities or different nationalities? Helping Theme • Individual or through • Types of help community networks • Instrumental (money, shelter) • Through embassy or • Advice to demand rights from advocacy/ social service employer organizations • Social support “building each • Giving and receving help other” across national boundaries (preferred in some cases) • Community activists who are the go-to for others when they need help Conclusions: What can these methods tell us that existing methods could not? • ROAD process facilitates the identification of a broader set of triggers/drivers and options (controls/mitigants) than traditional approaches, thus suitable for systemic risk assessment • Migration actor networks elevate focus on poor policies and policy processes • Quantification of Women’s Empowerment through WEMI not only yields important complementary insights to qualitative research, but also allows to identify sources of disempowerment • Engaging with migrants as researchers helps elevate their own perspectives Conclusions: What difference could these methods make to evidence being relevant to decision- makers? • Decision-makers and other key actors (donors/embassy staff / migrants) are in the room, knowledge exchange through engaging with other actors / with researchers • Evidence to ILO on broader factors affecting vulnerability to migration • WEMI provides quantified evidence whose changes can be monitored over time, or following interventions Conclusions: How could we promote the uptake of these methods by researchers? • Providing summaries of methodologies in easily-accessible ways • Outreach on methods through journal articles, policy briefs, blog pieces and conferences like the CEDIL event • Engagement of younger researchers, including PhD students and recent graduates in the study team