1 Independent Impact Assessment of the World Bank-funded Nigeria Avian Influenza Control and Human Pandemic Preparedness and Response Project (NAICP) 21 April 2011 Submitted to Project Management Unit, NAICP by the International Livestock Research Institute (ILRI) Contents 1 Acknowledgements ................................................................................................. 1 2 Abbreviations and acronyms .................................................................................. 2 3 Executive summary ................................................................................................. 4 4 Introduction .......................................................................................................... 18 5 Evaluation process ................................................................................................ 21 5.1 Purpose and scope of the evaluation ................................................................... 21 5.2 Evaluation team .................................................................................................... 21 5.3 Evaluation programme and timetable .................................................................. 21 5.4 Evaluation methodology ....................................................................................... 22 5.4.1 Project description, components and outputs ............................................................. 22 5.4.2 Evaluation framework ................................................................................................... 24 5.4.3 Sources of data.............................................................................................................. 27 6 Evaluation findings ................................................................................................ 29 6.1 Timelines of HPAI outbreaks ................................................................................. 29 6.2 Key findings by outcome pillars ............................................................................ 30 6.2.1 Pillar 1. HPAI Control and Pandemic Preparedness ...................................................... 30 6.2.2 Pillar 2. HPAI Surveillance ............................................................................................. 33 6.2.3 Pillar 3. HPAI Diagnosis, Differential Diagnosis and Pathogen Characterisation .......... 39 6.2.4 Pillar 4. HPAI Outbreak Containment and Disease Control .......................................... 43 6.2.5 Pillar 5. Epidemiological Capacity for Strategy Development and Intervention Targeting ............................................................................................................... 46 6.2.6 Pillar 6. HPAI Prevention ............................................................................................... 48 6.2.7 Pillar 7. Poultry Sector Recovery ................................................................................... 52 6.2.8 Pillar 8. Public Health Pandemic Preparedness ............................................................ 57 6.2.9 Pillar 9. Human Influenza Surveillance .......................................................................... 59 6.2.10 Pillar 10. Public Health Response and Containment ..................................................... 63 6.3 Communications ................................................................................................... 66 7 Relevance, effectiveness, efficiency, impact and sustainability ........................... 70 7.1 Relevance .............................................................................................................. 70 7.2 Effectiveness ......................................................................................................... 71 7.3 Efficiency ............................................................................................................... 78 7.4 Impacts .................................................................................................................. 85 7.4.1 The control of HPAI and the minimizing of human health implications ....................... 85 7.4.2 Assigning strength and authority to national institutions ............................................ 85 7.4.3 Credibility of the Federal Government of Nigeria ........................................................ 85 7.4.4 A template for inter-ministerial collaboration in the health arena .............................. 86 7.4.5 Strengthened national capacity in surveillance and diagnosis .................................... 86 7.4.6 Impacts on the poultry sector value chains .................................................................. 86 7.5 Sustainability ......................................................................................................... 87 7.5.1 Inter-ministerial collaboration. ..................................................................................... 87 7.5.2 Disease response mechanisms ..................................................................................... 87 7.5.3 Biosecurity in the poultry value chain........................................................................... 87 7.5.4 How much should emergency responses stray into broader and longer term capacity development? ....................................................................................................... 87 8 Recommendations ................................................................................................ 89 8.1 Clarity of project document formulation .............................................................. 89 8.2 Monitoring and evaluation ................................................................................... 89 8.3 Procurement ......................................................................................................... 89 8.4 Epidemiology capacity .......................................................................................... 90 8.5 The greater consideration of sustainability issues in WB emergency initiatives .. 90 8.6 Inter-sectoral collaboration .................................................................................. 90 8.7 Surveillance systems ............................................................................................. 90 8.8 Laboratory support ............................................................................................... 90 8.9 Inbuilt flexibility in responding to lessons learnt during a project ....................... 90 ANNEX 1. EVALUATION TEAM ............................................................................................... 91 Annex 2: ANIMAL HEALTH COMPONENT ............................................................................ 93 ANNEX 3 HUMAN HEALTH REPORT .................................................................................... 147 ANNEX 4 LABORATORY ANALYSIS ...................................................................................... 185 ANNEX 5 SECTORAL ANALYSIS............................................................................................ 250 ANNEX 6 LIVE BIRD MARKET IMPACT ASSESSMENT .......................................................... 253 1 1 Acknowledgements The evaluation team1 gratefully acknowledges the warm hospitality, openness and constructive engagement of the Government of Nigeria during the period of this evaluation. In particular we thank the Chief Veterinary Officer, Dr Joseph Nyager and the staff of the NAICP who made themselves available to team members. The team is particularly grateful to the NAICP component coordinators, Drs Sa?idu Mohammed, Shu?aibu Belgore and Olubanji Akeredolu, who went out of their way to assist the team, and to participate in a discussion of the evaluation findings in Ibadan on 24 March 2011. The team is most grateful to the Director General of the International Institute for Tropical Agriculture (IITA) for hosting the team during the planning workshop, during the data analysis phase and for the synthesis workshop. The evaluation team thanks most sincerely Dr Acho Okike of ILRI, Nigeria, for acting as a resource person for this evaluation, and facilitating logistics and communications for the evaluation team. 1 See Annex 1 for team composition 2 2 Abbreviations and acronyms ARIS Animal Resources Information System of AU-IBAR ASF African swine fever AU-IBAR African Union Inter-African Bureau for Animal Resources BSL Biosafety Level CBPP contagious bovine pleuropneumonia CDC Centers for Disease Control and Prevention C-PCR competitive polymerase chain reaction CVO Chief Veterinary Officer DFID UK Government?s Department for International Development EC European Commission ECTAD Emergency Center for Transboundary Animal Diseases ELISA enzyme-linked immunosorbent assay EU European Union FAO Food and Agriculture Organization of the United Nations FCT Federal Capital Territory FELTP Field Epidemiology and Laboratory Training Program FGN Federal Government of Nigeria FLD Federal Livestock Department FMoARD Federal Ministry of Agriculture and Rural Development FMoH Federal Ministry of Health FMoINO Federal Ministry of Information and National Orientation GAINS Global Avian Influenza Network Strategy GPAI Global Program for Avian Influenza and Human Pandemic Preparedness and Response HI Haemagglutination inhibition HP highly pathogenic HPAI Highly Pathogenic Avian Influenza ICR Implementation Completion and Results Report IDA International Development Association IDSR Integrated Disease Surveillance and Response IEG Independent Evaluation Group of the World Bank IITA International Institute of Tropical Agriculture ILI influenza-like illness ILRI International Livestock Research Institute INAPIRP Integrated National Avian and Pandemic Influenza Response Plan IRR internal rate of return IZSVe Istituto Zooprofilattico Sperimentale delle Venezie KAP Knowledge, Attitudes and Practices LBM live bird market LGA Local Government Areas M&E Monitoring and Evaluation MoU memorandum of understanding NADIS National Animal Disease Information and Surveillance system NAICP Nigeria Avian Influenza Control Project ND Newcastle disease NCDC Nigerian Centre of Disease Control NGO Non Governmental Organizations NIRL National Influenza Reference Laboratory NISS National Influenza Sentinel Surveillance 3 NPV net present value NRL AI National Reference Laboratory for AI NURTW National Union of Road Transport Workers NVRI National Veterinary Research Institute, Vom OECD Organisation for Economic Co-operation and Development OIE World Organization for Animal Health PACE Pan African Programme for the Control of Epizootics PAD Project Appraisal Document PCR polymerase chain reaction PDO project development objectives PDS Participatory Disease Surveillance PE participatory epidemiology PMU Project Management Unit PPE personal protective equipment PVS Performance of Veterinary Services R0 basic reproduction ratio RT-PCR Reverse Transcriptase Polymerase Chain Reaction Real-time RT-PCR Real-time Reverse Transcriptase Polymerase Chain Reaction SOPs standard operating procedures SPINAP Support Programme to Integrated National Action Plans for Avian and Human Influenza TADs Trans-boundary Animal Diseases TB tuberculosis UNICEF United Nations Children's Fund USAID United States Agency for International Development US$A United States Department of Agriculture VTH Veterinary Teaching Hospital WB World Bank WHO World Health Organization 4 3 Executive summary INTRODUCTION Nigeria was the first country in Africa to be affected by the H5N1 virus of highly pathogenic avian influenza (HPAI), with outbreaks initially reported in Kaduna State on 8 February 2006. The disease spread rapidly to 97 Local Government Areas (LGAs) in a total of 25 states and the Federal Capital Territory, and some 440,000 birds were culled in the first two months. Nigeria subsequently suffered waves of HPAI outbreaks that peaked twice, in February 2006 and in February 2007. The outbreaks affected 3037 farms/farmers resulting in the culling of 1.3 million of the country?s estimated 160 million birds. The disease was later reported on two farms in July 2008 in Kano and Katsina states, and two positive duck samples were diagnosed in live bird markets (LBM) in Gombe and Kebbi states; the disease was quickly brought under control. Following a request for assistance by the Federal Government of Nigeria (FGN), World Bank (WB) financing of US$ 50 million-equivalent was approved in April 2006. The Nigerian Avian Influenza Control and Human Pandemic Preparedness and Response Project (NAICP) which resulted was then initiated using approximately US$ 10 million reallocated from two other WB projects, and formally commenced in July 2006. It is scheduled for completion by 31st May 2011. The Project Appraisal Document (PAD) of the NAICP contained three Project Development Objectives (PDOs). These were: (i) To support the efforts of FGN to minimize the threats posed by H5N1 to humans and the poultry industry (ii) To prepare the necessary control measures to respond to a possible influenza pandemic (iii) To prevent further spread of the disease to other parts of Nigeria. In November 2010, a WB-funded Implementation Status and Results (ISR) report was prepared by the project management. This report evaluated the degree to which the various project activities had achieved their target indicators. In addition, an internal Implementation Completion and Results report (ICR) was recently prepared by the Project Management Unit (PMU) to assess the extent to which the PDOs and the development outcomes of the project had been achieved. The two ICR reports developed to date both refer to the lack of an end-of-project impact assessment by the PMU, to provide an impartial assessment of the success or failure in meeting the project outcomes. The Borrower?s Implementation Completion and Results Report (6 November 2010), under the section Lessons Learned (pp. 19 ? 20) stated that ?a final impact assessment is imperative near the end of the project to assess whether the project achieved its objectives?. To remedy this constraint, the PMU invited the International Livestock Research Institute (ILRI) to undertake such an impact assessment. The ILRI submitted a proposal to the NAICP outlining the proposed study, which was accepted. THE EVALUATION PROCESS The independent evaluation of the NAICP aimed to improve the relevance and performance of emergency interventions to respond to pandemic disease threats, provide accountability for the many activities undertaken, and derive lessons for better formulation and implementation of future projects undertaken with support from the WB. The evaluation aimed to build on the project assessments carried out by the WB and the FGN which reported on achievements and outcomes, under the general guidelines of the monitoring and evaluation system set up for the project. 5 The evaluation has drawn on accepted methodologies, including those of the OECD and the Independent Evaluation Group (IEG) of the WB, to provide the FGN, WB and other stakeholders with a systematic and objective assessment of the relevance, efficiency, effectiveness, impacts and sustainability of the programmes and activities undertaken. The evaluation placed emphasis on gathering evidence of the effects of project activities on a series of outcome pillars of best practice in HPAI preparedness, response and control. The independent evaluation team was assembled by ILRI, comprised selected ILRI staff, Nigerian and international consultants, and led by an independent consultant as Team Leader. The NAICP had a total of four components (Animal Health, Human Health, Social Mobilisation and Strategic Communication, and Implementation Support/ Monitoring and Evaluation). Within these are 27 sub-components, comprising six in animal health (budget US$ 29.2 million), three in human health (US$ 18.25 million), three in social mobilisation and strategic communication (US$ 4 million) and 15 in implementation support/ monitoring and evaluation (US$ 6.8 million, divided between animal and human health). The evaluation team developed an evaluation framework with which to assess the performance of the NAICP. It comprises a set of 10 outcome pillars on which the project has been evaluated. The outcome pillars were developed by the evaluation team to depict the benchmark ?gold standard? of best practices against which to evaluate NAICP. For the animal health elements, the outcome pillars have drawn on (and further developed) the pillars used during the Second Real Time Independent Evaluation of FAO?s Global Programmes on HPAI. For each pillar a set of generic objectives is presented, followed by a listing of candidate components/activities which would likely fall under the objectives, and finally a set of desired outcomes. The task of the evaluation team was to determine, in as empirical fashion as possible, the degree to which the project outputs have contributed to the target outcome pillars. The team evaluated both direct and indirect impacts, and intended and unintended impacts. The evaluation team assembled secondary data from various sources, including from the NAICP, and designed and undertook selected studies to collect primary data on certain aspects of the interventions undertaken by NAICP. EVALUATION FINDINGS The evaluation findings are reported below under the categories of relevance, effectiveness, efficiency, impact and sustainability. RELEVANCE At the time of the HPAI outbreak in 2006, the FGN was acutely aware of its vulnerability to the possible effects on the poultry industry and of a possible human pandemic. It is sometimes easy to forget the fear created globally by the outbreaks of HPAI in Asia, following on the earlier legacy of SARS, all compounded by the frenetic media coverage. Beyond this, the international community, including the WB, were keen to deliver on the commitments made at the Beijing donor conference in January 2005, and Nigeria was seen to be of strategic significance due to its large population, its role as a regional hub, and the underlying poverty and food security impacts. As far as the structure and objectives of the project are concerned, the aims of containing the disease, reducing risk and preparing for a possible pandemic were broadly fit for purpose, and 6 importantly, they evolved as the project matured and as the FGN and its partners learnt more about the behaviour of the disease, and what they felt could and could not be achieved. The evaluation team used an evaluation framework of 10 benchmark pillars to assess the performance of the NAICP. While not definitive, they have are based on an accepted and peer- reviewed international framework for evaluating infectious disease control at the national level. Two key issues emerge with regard to the relevance of the NAICP design and this framework. First is that while some sub-components and activities might have been placed incongruously under different headings, possibly a reflection of the haste of document preparation, all the framework pillars are indeed represented in the NAICP design. However, certain important subcomponents and activities seem to have received inadequate attention. These include inadequate engagement by FGN with the rapidly growing and highly independent private poultry sector for effective disease control, and the inadequately recognised importance attached to analytical epidemiology as a leading pillar of disease control and disease information communication. EFFECTIVENESS This evaluation has assessed effectiveness at various levels (both from a theoretical and empirical perspective). First, overall effectiveness is considered with regard to the three PDOs. Next, effectiveness is considered in relation to the 10 outcome pillars that have been taken as benchmarks of performance in HPAI control and pandemic preparedness at a national level. Under each outcome pillar, the generic pillar objectives, candidate components and outcomes are listed, followed by an assessment of how effective NAICP has been in meeting these. Effectiveness at the level of PDOs In considering the broad effectiveness of the NAICP, the evaluation team considered the three PDOs, which were: (i) To support the efforts of FGN to minimize the threats posed by H5N1 to humans and the poultry industry (ii) To prepare the necessary control measures to respond to a possible influenza pandemic (iii) To prevent further spread of the disease to other parts of Nigeria. Furthermore, the NAICP had two overall development objectives, which was: ? To minimize the threat posed to the poultry industry and humans by HPAI infection and other zoonoses, and ? To prepare for, control and respond to influenza pandemics and other infectious disease emergencies in humans. In the PAD it is stated that key Indicators of Project Outcome will include evidence of improved effectiveness of government in responding to the risk of an HPAI outbreak and/or pandemic, and contained and diminishing pattern of HPAI infection in poultry. Assessment (i) To support the efforts of FGN to minimize the threats posed by H5N1 to humans and the poultry industry ? There is substantial evidence presented in this report that the NCAIP did indeed support the efforts of FGN to minimize the threats posed by H5N1 to humans and to poultry, through public awareness and capacity-building within the human and veterinary health systems. 7 ? There is reasonable evidence that the NAICP contributed to minimizing the threats posed by H5N1 to humans in the longer term by strengthening preparedness, surveillance and response mechanisms. ? There is very little direct evidence that the NAICP contributed to minimizing the threats posed by H5N1 to humans during the 2006, 2007 and 2008 outbreaks; for reasons that have not been established, there have been 143 confirmed cases of HPAI H5N1 in humans in Africa, and all but two of these have occurred in Egypt. There have been no other confirmed human cases in West Africa. ? There is reasonable evidence that the NAICP contributed to minimizing the threats posed by H5N1 to the poultry industry, including through an effective compensation scheme and delivering appropriate messages about the safety of consuming well-cooked poultry, but there is insufficient evidence that it was responsible for the eradication of the disease. (ii) To prepare the necessary control measures to respond to a possible human influenza pandemic ? There is substantial evidence that the NAICP prepared necessary measures to respond to an influenza pandemic. (iii) To prevent further spread of the disease to other parts of Nigeria ? There is limited evidence that the NAICP prevented the spread of HPAI H5N1 to other parts of Nigeria. The disease occurred in 16 states in 2006, which rose to 22 states in 2007. The NAICP was clearly written in extreme haste, with the intention of presenting a structured plan for the emergency response to HPAI in Nigeria in a very short time. It is easy to criticise the structure of the document with hindsight, but it did cover most of the bases necessary, as reported above. Nevertheless, apart from the cumbersome and unclear presentation of the PDOs, the objectives themselves are inadequately explicit to be used for objective, evidence-based evaluation. One of the recommendations presented below addresses this, suggesting a more structured approach to project design for national zoonotic disease control responses in the future. Effectiveness at the level of the 10 outcome pillars PILLAR 1. HPAI CONTROL AND PANDEMIC PREPAREDNESS Objective2: The development of a clear and technically sound policy for HPAI preparedness and control, effectively communicated to all stakeholders. Candidate components: Legal framework, national policy, national strategy, contingency planning, benefit: cost considerations, research prioritisation, industry development, poverty reduction interface, identification, engagement and communication with all stakeholders. Outcomes3: Sound HPAI policy in place; all stakeholders involved and informed. Overall, the evaluation team consider that the pillar outcome was achieved in a moderately satisfactory manner; there was a sound HPAI policy put in place and considerable effort was put into ensuring that key stakeholders were involved and informed. Specifically: ? the evaluation team considers that overall planning and preparedness was satisfactory 2 These are evaluation-derived objectives considered necessary to achieve the designated outcomes in each pillar; they are not NAICP objectives 3 These are benchmark ?gold standard? outcomes against which the NAICP has been evaluated; they are not necessarily NAICP outcomes 8 ? the presidential-level support was extremely helpful at the outset ? strong leadership was provided by several NAICP champions, including the Chief Veterinary Officer (CVO) of the country and the director of the National Veterinary Research Institute (NVRI), Vom ? some activities were constrained by delays in disbursement of funds ? certain key personalities played a strong role. This might make it difficult to recreate the success of planning and coordination with the different set of leaders now in place ? the inter-ministerial cooperation brought about by the project would have been even more effective if key actors from the ministries of health and agriculture were located in same building ? the existence of federal and state systems was a complicating factor which impacted on efficacy ? the devolution of diagnostic responsibilities to state teaching hospitals was a sound idea but was in practice problematic: only NVRI was able to carry out the improved molecular techniques Without preparedness and control policies in place it is not possible to effectively control animal diseases. The project provided a framework for future control efforts, which will have positive (although difficult to quantify) impacts on poultry-dependent livelihoods, the poultry sector, and consumers. PILLAR 2. HPAI SURVEILLANCE Objectives: Establishment and revision of effective, sustainable and affordable surveillance systems for HPAI in target domestic and wild bird populations. Candidate components: Passive surveillance, active surveillance, wild bird surveillance. Outcomes: HPAI infection status effectively determined and internationally recognised. This pillar outcome was not fully met. The surveillance system took time to expand beyond the 170 control posts and 259 surveillance agents established by the National Animal Disease Information and Surveillance system (NADIS) under the Pan African Programme for the Control of Epizootics (PACE) programme. Efforts to devolve responsibilities to the states at Veterinary Teaching Hospitals (VTHs) were hampered by capacity development and there were problems getting new equipment functioning properly. The active surveillance based on risk was very limited, and wild bird surveillance was not included in NAICP. The turnaround time to provide definitive results for field samples was progressively improved, mostly through improving sample transportation and efficiency of laboratory processes. Specifically: ? the human and animal health surveillance teams worked well together in the field ? innovative partnerships, such as that with the National Union of Road Transport Workers (NURTW), increased effectiveness of sample transportation ? devolution of responsibility to state level was hampered by infrastructure development delays ? live bird market surveillance through NAICP was carried out on a limited scale, it was mainly carried out by FAO ? there were improvements in surveillance, but evidence suggests both human and animal surveillance systems pick up a very small proportion of cases that meet the definition of suspect cases 9 PILLAR 3. HPAI DIAGNOSIS, DIFFERENTIAL DIAGNOSIS & PATHOGEN CHARACTERIZATION Objectives: Establishment and maintenance of internationally recognised laboratory capacity to confirm and where appropriate characterise HPAI infections. Candidate components: Sample collection and shipment; cold chain viability; laboratory facility development; laboratory equipment and reagents; laboratory network and interface; capacity building of laboratory staff. Outcomes: Optimal sensitivity and specificity of diagnostic tools established and results in international public domain. Overall, diagnostic capacities at reference laboratory levels (both in animal and human health) were improved: influenza viruses can now be detected in biological samples with high sensitivity and specificity, and sub-typing is carried out to international standards. Distribution of influenza- positive samples to International Reference Laboratories (CDC-Atlanta, FAO/OIE Reference Laboratory in Padua, Italy) has contributed to international transparency, corroborated by joint global scientific collaboration. The capacity for differential diagnosis (i.e. including consideration for Newcastle disease and other poultry diseases) has received inadequate attention. A lack of appropriate reagents and further training (particularly in pathology) has limited the capacity to provide substantial diagnostic services for poultry diseases other than HPAI. The upgrading and devolution to the states of diagnostic facilities to use molecular techniques and to improve biosafety was limited. However this may not have affected diagnostic performance, as innovative arrangements for transporting samples to laboratories and better systems for processing samples on arrival led to decreased turnaround time. Overall, much of the credit for enhanced diagnostic capacity goes to initiatives by NVRI independent of NAICP, and on the human health side to support received from CDC and WHO. Although the late implementation has meant most improvements in laboratory capacity did not contribute during the outbreak period, these now represent enhanced capacity for future outbreaks. PILLAR 4. HPAI OUTBREAK CONTAINMENT & DISEASE CONTROL Objectives: Plan and implement technically sound, effective affordable, sustainable and socioeconomically acceptable intervention measure to control or eradicate HPAI . Candidate components: Intervention measures (depopulation, decontamination, disposal, movement control, awareness raising, vaccination, poultry restocking, etc); compensation. Outcomes: Disease outbreaks effectively contained and status recognised internationally. HPAI has not been detected in Nigeria since 2008. The two key questions, however, are whether the disease was unsustainable in the Nigerian environment and therefore burnt itself out, and whether the NAICP was responsible for its eradication. The evaluation team is unable to answer the first question, save to comment that the disease did disappear from all other affected West Africa countries as well, some of which had much less effective control programmes in operation than Nigeria. On the second question, it appears that many of the measures put in place by NAICP came too late to have an impact on the outbreaks, although the NAICP structure was used to implement/ distribute in-kind donations from other donors from the onset. The FGN funded many activities under this pillar which were supplemented 10 by substantial in-kind donations, such as disinfectants and PPEs, by other agencies. It is possible that the knowledge of impending WB funds gave FGN the confidence to spend in the early stage, when it was perhaps most needed, safe in the knowledge that funds were coming. The WB-funded compensation mechanism, once revised to reflect market prices, was an effective measure which probably contributed to the speed and effectiveness of the poultry culling on infected premises. This may have slowed down the spread of HPAI. PILLAR 5. EPIDEMIOLOGICAL CAPACITY FOR STRATEGY DEVELOPMENT & INTERVENTION TARGETING Objectives: Collect, synthesise and analyse data on the dynamics and impacts of HPAI, and use the outputs to inform policy and strategy for HPAI control. Candidate components: Epidemiological data handling, processing and analysis; data flow and communication ; data reporting, use and presentation; poultry population demography; outbreak investigation; value chain studies; risk assessment (based on critical control points); socioeconomic impact. Outcomes: Quality data received and disease control strategy regularly updated through a sound evidence base. The effectiveness of this pillar is somewhat difficult to assess. Unquestionably the skills existing in the FMoARD epidemiology unit enabled prompt synthesis of emerging surveillance data, and good communication was established (in the same building) with the office of the CVO. There was an epidemiology team on ground due to the legacy of the PACE programme, a system for surveillance, and an information system (NADIS). Summary reports on HPAI status were produced, the information provided assisted NAICP in its strategy and in targeting interventions, and the disease disappeared. However, on scrutiny of the data made available to the evaluation team by the state desk officers, certain inadequacies in data handling were detected: for example, inadequate standardisation of reporting units (whether household, village, LGA) and inadequate denominator4 data on poultry. Furthermore, it is felt that epidemiological capacity would have been more effective with greater use of risk-based analysis and value chain approaches to increase the understanding of critical control points in HPAI control. PILLAR 6. HPAI PREVENTION Objective: Put in place technically sound affordable and socially viable measures to minimise the risk of HPAI spread, and reduce the risk of human infection. Candidate components: Biosecurity, communication, human protection, vaccination, farm/unit registration, market and slaughter practices; industry support and restructuring. Outcome: Progressive reduction in disease incidence which is independently verifiable. No new human cases. This pillar was seen to be satisfactorily effective in some areas, and less effective in others. The enhanced biosecurity through infrastructural improvements to selected LBMs was undoubtedly effective, and independent evidence of this was generated in the evaluation. However, the number of LBMs upgraded was insignificant in relation to the total number of LBMs in Nigeria; nevertheless this provides a very constructive and positive case study which can be built on in the future. This component introduced during the project has clearly strayed from the original emergency nature of 4 Data on poultry populations, particularly by production system type. 11 the loan, since it will only be able to affect disease risk significantly if a more extensive improvement of LBMs were to be implemented across the country. The training in improved biosecurity for poultry traders did not reduce risky practices undertaken, and improvements in biosecurity for farmers were considered by the evaluation team to be insufficient to prevent HPAI spread. It was felt by many of the key informants consulted that the NAICP made an appropriate decision not to vaccinate poultry, despite intense pressure from international sources such as FAO, given the realistic assessment of the immense logistical and cost complications it would have incurred, and the challenges of developing an exit strategy, a complication that has been faced by Viet Nam, for example. Nevertheless, the evaluation team learnt of reported high levels of unofficial vaccination among commercial poultry enterprises. PILLAR 7. POULTRY SECTOR RECOVERY Objective: To minimise the impact of HPAI on the credibility and economic viability of the poultry industry. Candidate components: Public relations of risk avoidance with poultry and poultry products; viable compensation scheme; dialogue with and support to all sections of poultry industry. Outcome: Affected poultry farms back in business, compliance with FGN policies for HPAI control and prevention is enhanced. Overall, this pillar was seen by the evaluation team as satisfactorily achieving the desired outcome. The compensation scheme was widely seen as being successful, striking the right balance between under-compensating, resulting in low levels of disease reporting and a likelihood of greater spread, and over-compensating; the amounts paid were appropriate after the transition to market-based rates had been made. It was also recognised by most people interviewed for this study as being a transparent process. The messaging produced by the NAICP and others restored the confidence of the general public to consume poultry, and most elements of the poultry sector did indeed recover. PILLAR 8. PUBLIC HEALTH PANDEMIC PREPAREDNESS Objectives: The development of a clear and technically sound policy for HPAI preparedness and control, effectively communicated to all stakeholders. Candidate components: Legal framework, national policy, national strategy, contingency planning, benefit: cost considerations, research prioritisation; identification, engagement and communication with all stakeholders. Outcomes: Sound policy & strategy in place and effectively communicated; all key human and animal health stakeholders involved. This pillar was considered by the evaluation team to have been moderately satisfactorily achieved, but was not put to the test regarding HPAI becoming a pandemic; it was, however, tested to a degree with the advent in 2009 of H1N1/2009 Pandemic influenza. There was general satisfaction with the planning and preparedness activities conducted by the NAICP amongst policy makers, experts and state officials. 12 PILLAR 9. INFLUENZA SURVEILLANCE Objective: Infection status in human populations known and internationally recognised. Candidate components: Laboratory capacity development; surveillance networks established Outcomes: Infection status in humans known and internationally recognised. This pillar is considered by the evaluation team to have been reasonably effective, although some of the results of the analysis raise substantial questions about surveillance in general. HPAI is now a notifiable disease. An influenza surveillance system is now in place, but there appears to be massive under-reporting of cases that apparently match the HPAI case definition; but under- reporting is widespread for all notifiable diseases, not just the influenzas. The strengthening of the surveillance systems has had an apparent impact on the reporting of malaria and H1N1/2009 Pandemic influenza (swine flu). The public health sector is very proud of HPAI surveillance capacity right down to LGA level. The human health laboratories are able to carry out molecular techniques for the detection of Influenza A/B and subtyping of AI H5 and H1/2009 Pandemic influenza both according to CDC-protocols. PILLAR 10. PUBLIC HEALTH RESPONSE AND CONTAINMENT Objective: To ensure the human population at large and the poultry associated households are prepared and protected from the threat of HPAI. Candidate components: Public health communications; risk awareness; laboratory capacity development; sound poultry handling and marketing practices; communication on human behavioural practices. Outcomes: Health facilities regularly stocked with target quantities of anti-viral medication; tested preventive measures communicated to and adopted by at risk health workers and the general public This pillar is seen by the evaluation team to have been reasonably effective, with the caveat that no pandemic occurred, so the response and containment systems were not put to the test. Public health messages on awareness of HPAI, and subsequently on the safety of poultry products, appeared to be successfully delivered and well understood, although with messages emanating from many sources, attribution to NAICP is impossible. Public health messaging by NAICP targeted at children was effective and led to demonstrable beneficial behaviour change, e.g. in hand-washing in the targeted children. Several NAICP materials (such as anti-viral drugs) supplied for HPAI were used for the Pandemic H1N1/2009. Effectiveness in communication In terms of communications targeted at the general public, it is impossible to isolate the contribution made by NAICP?s communication initiatives from those made by a wide range of other actors: it is likely that most people in Nigeria were exposed to a variety of messages emanating from a variety of sources. Under the NAICP-funded communication component a diverse range of media and formats were utilised to deliver messages to the general public as well as more specialised audiences, such as journalists. Results from the KAP surveys undertaken as part of this evaluation suggest that these approaches were relatively successful: there was high penetration of campaign messages; knowledge of campaign messages was good; and appropriate channels for AI information dissemination chosen with 92% of respondents reporting that they obtained health information 13 from the mass media. Messages delivered via radio jingles and other types of broadcast are estimated to have reached up to 50 million people. EFFICIENCY This evaluation has assumed that the audit of project and the accountability of expenditures has been a responsibility of the WB and FGN, and the evaluation has therefore focussed much of its efforts on the effectiveness and the attainment of key outcomes. Nevertheless, a few general comments on efficiency are made here. On the positive side, the designation of responsibility and allocation of substantial funding to the NAICP presented an opportunity for the efficient use of resources, and an efficiently run programme. And this factor undoubtedly did contribute to the strong and effective leadership taken by the FGN. However, on the negative side, the NAICP was designed and implemented as an emergency response to what was perceived to be a very serious situation, but different administrative processes required by the WB and the FGN were most certainly not of an emergency nature; they were slow and laborious. This mismatch inevitably caused problems. The evaluation concluded in several cases that the NAICP delivered (training courses, equipment, infrastructure, etc) after the main disease outbreaks had occurred. As part of the evaluation of NAICP efficiency, a standard benefit: cost analysis was applied to assess the economic performance of the project. Prior to the WB credit, a wide range of assumptions had been made by several different analysts; HPAI was expected to cause major losses, perhaps reflecting the spirit of the precautionary principle that it is reasonable to assume a worst-case scenario in the absence of evidence. In the event, the outbreaks affected less than 1% and 0.3% of the poultry population in the 2006 and 2007 episodes, respectively, before one last, very small outbreak in 2008. It is still uncertain to what degree the lower-than-anticipated losses should be attributed to control efforts by NAICP (and others) versus simply representing the natural course of the disease in the Nigerian context. The analysis of the economic impact of NAICP was based on a comparison between the incremental costs incurred to strengthen HPAI control being undertaken by the FGN, and the incremental benefits (economic losses avoided) due to that strengthened control. The incremental costs are represented by the disbursements of the WB credit over the project period from 2006 to 2010. The incremental benefits are more difficult to estimate. This uncertainty is addressed by considering a range of assumptions and scenarios to represent what might have happened had the WB credit not been granted. A factual scenario was compared with three counterfactual scenarios (a best-case, assuming the disease burned out on its own; a medium-case, assuming the responses by FGN would have limited the initial outbreak as occurred in 2007, but outbreaks similar to those in 2006-2007 would continue; a worst-case scenario, assuming that HPAI would have become endemic despite the response by FGN, and a doomsday scenario, in which the benefit of avoiding a human pandemic is introduced into the analysis, in which the expected global benefit can be expected to be so large as to dwarf the investment and to make the analysis moot. No attempt, therefore, was made to quantify this scenario. The evaluation concludes that the WB emergency credit performed poorly as a financial investment, registering a BCR of less than one and negative NPVs under a range of scenarios, assuming the risk of a human pandemic is not factored in. However, such indicators are of questionable relevance when application of the precautionary principle is deemed appropriate. Previous projections had all assumed much higher impacts in terms of the size of the expected outbreaks, and had reflected the 14 poor understanding of the disease at the time and can be considered valid in the spirit of the precautionary principle. Fortunately, the reality has been that the impact of HPAI was not as great as widely feared; although it certainly caused substantial losses (1% of the national flock), it did not cause the 30% losses originally assumed to justify the WB credit. A benefit cost analysis is not necessarily an appropriate measure for this type of loan. First, the loan was made as an emergency measure in a context in which little was known about the nature of HPAI, especially in Africa. The loan should therefore be evaluated on the basis of the precautionary principle, rather than (with the benefit of perfect hindsight) as a financial investment required to generate a certain level of return. Finally, this type of analysis does not take into account potential longer-term benefits of the NAICP-managed investments which were beyond the immediate HPAI focus of the loan; improved laboratory diagnostic capacity, human capacity developed through training, and the institutional experience with joint veterinary-public health response management which will likely serve to improve control of future outbreaks of emerging diseases than otherwise would have been the case. IMPACTS The evaluation identified the following key impacts of the NAICP. The control of HPAI and the minimizing of human health implications The NAICP undoubtedly made a contribution to HPAI control and to minimizing human health implications, but a cause and effect relationship has not been demonstrated. Project activities also provided a degree of protection during a period of perceived high vulnerability to HPAI introduction. Assigning strength and authority to national institutions The WB credit placed the opportunity to perform squarely in the hands of the FGN, so strengthening Nigerian national institutions on an issue of global public health concern. Credibility of the Federal Government of Nigeria The WB credit provided the opportunity for the FGN to perform, and the evaluation concludes that the FGN indeed picked up that challenge. Although not universally endorsed, the FGN rose to the occasion, and has emerged with international credibility for its handling of the crisis. This does not mean that all activities were faultless and that the FGN is now able to unilaterally respond to new animal and human health emergencies, but it has demonstrated the capacity for leadership. A template for inter-ministerial collaboration in the health arena The NAICP is widely credited with bringing about closer collaboration between human and animal health authorities in Nigeria. It provided a case study on demand-driven inter-ministerial collaboration to respond to a major public health threat, which has undoubtedly helped to set the scene for ?One Health? initiatives in the future. Strengthened national capacity in surveillance and diagnosis The NAICP has made substantial contributions to national disease surveillance and diagnosis, in particular with respect to the influenzas. On the animal health side, this was complemented by independent initiatives taken by NVRI, Vom, and on the human health side, by substantial support from CDC, Atlanta and WHO. The evaluation team concluded that the overall impact of the quarantine station improvement was minimal. 15 Impacts on the poultry sector value chains The NAICP appeared to have had unquantified impacts in maintaining public confidence in the poultry industry, considered as loss-averted benefits rather than gains. SUSTAINABILITY The sustainability of products, outcomes and partnerships resulting from the NAICP is far from straight forward, and much of it is questionable. In the first case, this project was designed as an emergency response to tackle a major national (and potentially regional or global) public health scare, and so in general was not designed to have long-term sustainability features. A few such features emerged as the project progressed, such as the pilot LBM construction, but the major thrust was ?fire-fighting?. Inter-ministerial collaboration This is an area in which sustainability should be promoted by the FGN, as it has great potential for the future management of zoonotic public health emergencies and longer term development opportunities under a ?One Health? umbrella. However, it will likely require special effort to sustain following the closure of the project. This is for two main reasons: firstly that the relationship was never fully established with the FMoH and FMoARD operating from separate buildings, and secondly, that much of the success was built on personalities, which will inevitably change. Disease response mechanisms Sustainability of the improvements in field surveillance and laboratory diagnosis capacity, as well as the performance and continued availability of rapid response teams has not been secured. There is a high turnover of staff which means that many trained staff are no longer at their posts and budgets for continuous training are limited. Funding to ensure continued access to the expensive reagents needed for the improved molecular diagnostic techniques and facilities for correct storing of reagents in the face of frequent power cuts are also uncertain. Plans for how desk officers will be utilised when project funding ceases are still being considered in many states. Biosecurity in the poultry value chain The biosecurity measures necessary for effective HPAI containment and prevention at household, farm, market and consumer level have much wider efficacy implications for broader public health and food safety. However, given that they were mostly very poorly adopted, they are highly unlikely to be fostered without the threat of major disease epidemics driving the communications necessary to influence behaviour and good practices. The major drivers are therefore likely to be improved opportunities in market access, and consumer demand for safer and healthier livestock products. The pilot LBM initiative indicates that there is indeed consumer demand for infrastructures which support more hygienic marketing conditions, and this merits follow up. The interface between emergency responses and longer term capacity development The evaluation team lauds the focus given within the NAICP to address the HPAI crisis as it appeared in 2006. However, as we know with hindsight from this crisis, the realities were different and as far as Nigeria is concerned have been considerably less severe than anticipated. It is therefore important to ensure that resources and programming are not only adequate to resolve the crisis, but also that they achieve an appropriate balance with broader development targets. 16 RECOMMENDATIONS Based on the findings of this independent impact assessment, the evaluation team makes the following general recommendations. As part of the independent assessment of the animal and human health laboratories, a specific set of recommendations has been made, and this is found in Annex 4. Clarity of project document formulation The evaluation team recognises the haste with which the project proposal was prepared in March 2006, and the team has acknowledged that the key ingredients appeared to be present in the proposal submitted. However, the team recommends that future animal disease control projects developed at a national level would benefit from the following: ? In the PAD, PDOs should be made much more explicit in the proposal; in the NAICP they were poorly articulated and inadequately substantiated. ? Better structuring and separation of the key essential components of a national response (for example based on the pillars used in the evaluation framework, identifying policy, planning and coordination, laboratory capacity, surveillance, response, prevention, industry support and/or recovery). ? Within each component, a clearer presentation of objectives, outputs, outcomes and indicators, facilitating provision for a much more rigorous assessment of progress, achievements and impacts. ? An impact pathway, showing how outcomes are designed to lead to impacts can be a valuable planning tool. Monitoring and evaluation For a project of this size and complexity it is essential that a strong M&E system is in place. The evaluation team recommends: ? Much greater consideration should be given by WB to the type of capacity, technical skills and data requirements for M&E of large and complex projects in which multiple agencies of government are involved at federal and state levels. ? An emergency programme by definition operates in a context of uncertainty, so in addition to conventional M&E, capacity and flexibility is needed to learn and to adapt as experience is gained and new information becomes available. In a disease outbreak situation, such a project requires more direct access to, and involvement of, experts (see point further below about epidemiology expertise in particular) and partners who continuously review the relevance and appropriateness of the response strategy. Procurement This project brought together two complex bureaucracies ? the WB and FGN. The incompatibility of these and the overload of demand on the FGN and WB caused several delays and undoubtedly affected project deliverables and outcomes. The evaluation team recommends: ? Much better management and coordination of all procurement procedures should be built into projects from the outset, incorporating relevant staff, training and communications facilities. 17 Epidemiology capacity The project missed opportunities to develop greater capacity and skills sets in the field of veterinary epidemiology. The evaluation team recommends: ? For both the WB in future animal health projects, and the FGN in animal health programmes, greater attention should be given to the standardisation of epidemiological data handling and management procedures, the development of risk-based approaches that can feed into surveillance and intervention strategy development, and the development of host population demography and characterisation data that form key denominators for analysis. The greater consideration of sustainability issues in WB emergency initiatives The evaluation team, while understanding the focus introduced by emergency response funding for specific public health crises, recommends: ? Greater consideration should be given to broader and longer term sustainability issues when developing emergency funding responses. This is particularly relevant to broader generic capacity to address other animal and human health constraints of priority to the borrower, and also longer term capacity needed to ensure sustainability of focussed measures supported. Inter-sectoral collaboration The inter-sectoral collaboration model developed is commended and it is recommended that consideration be given to adopting this as a platform for addressing other problems at the intersection of animal and human health, such as food-borne diseases and zoonoses. Disease surveillance systems The animal and human health surveillance systems, while demonstrating notable improvements, both seem to be only detecting a fraction of notifiable diseases. It is recommended that these be reviewed with the consideration of improving their performance. Laboratory support to diagnosis and surveillance Laboratory improvements have been substantial. However, without continued attention, these benefits are likely to be short lived, and the evaluation team recommends that FGN seeks continued support to ensure their sustainable functioning. Inbuilt flexibility in responding to lessons learnt during a project The NAICP gained in both effectiveness and efficiency by making certain adjustments during the implementation of the project, ensuring that they were agreed by both parties. The evaluation recommends that such flexibility, within well articulated boundaries agreed prior to the initiation of projects, should be standard in such emergency support initiatives undertaken by WB and/or FGN. 18 4 Introduction The Federal Government of Nigeria (FGN) received World Bank (WB) financing of US$50 million- equivalent5 in response to a request for assistance by the FGN, following an avian influenza outbreak detected in the country in early 2006. The Nigerian Avian Influenza Control and Human Pandemic Preparedness and Response Project (NAICP) which resulted was approved in April 2006 and was initiated using the approximately US$10 million reallocated from other WB projects. The NAICP formally commenced in July 2006 and is scheduled for completion by 31st May 2011. The project has been subject to two extensions of 18 months, from its original closing date of June 30, 2009 to December 22, 2010, and with another 5 month extension, from December 2010 to 31st May 2011. Nigeria was the first country in Africa to be affected by the H5N1 virus of highly pathogenic avian influenza (HPAI), with outbreaks initially reported in Kaduna State and confirmed by the Minister of Agriculture and Rural Development (FMoARD; now called the Ministry of Agriculture and Water Resources, but referred to as FMoARD in this report) on 8 February 2006. The disease spread rapidly to 97 Local Government Areas (LGA) in a total of 25 States and the Federal Capital Territory6, and some 440,000 birds were culled in the first two months7. Nigeria subsequently suffered waves of HPAI outbreaks that peaked twice, in February 2006 and in February 2007. The outbreaks affected 3037 farms/farmers resulting in the culling of 1.3 million of the country?s estimated 160 million birds; US$ 5.4 million was paid in compensation by the FGN (FLD 2008). The last outbreak of the first wave of disease was recorded in Anambra State in October 2007. The disease was later reported on two farms in July 2008 in Kano and Katsina States, and two positive duck samples were diagnosed in live bird markets (LBM) in Gombe and Kebbi States; the disease was quickly brought under control. Figure 1: Occurrence of HPAI outbreaks in Nigeria 2006-2008, by month (Source: NADIS) The NAICP contained three Project Development Objectives (PDOs). These were: (i) to support the efforts of FGN to minimize the threats posed by H5N1 to humans and the poultry industry (ii) to prepare the necessary control measures to respond to a possible influenza pandemic (iii) to prevent further spread of the disease to other parts of Nigeria. 5 The WB financing utilized IDA funds from three World Bank financed Projects as follows: (a) a reallocation of US$ 7.2 million from the on-going Fadama II; (b) a reallocation of US$ 5 million from the on-going HSDP II; and (c) the utilization of US$ 50.0 million from the proposed emergency operation credit 6 NAICP Project website, 2008 7 A. Riviere-Cinnamond, Compensation Strategy Nigeria, April 2006 0 10 20 30 40 50 60 70 N u m b er o f O u b reak s 2006 2007 2008 Initiation of NAICP 19 To achieve these objectives, NAICP supported three sets of interventions. These were: (a) response and containment (b) control and prevention (c) preparedness and planning. The project has four components: Animal Health, Human Health, Social Mobilization and Strategic Communication, and Project Management and Coordination, each of which has sub-components and activities. The overall development objective was: ? To minimize the threat posed to the poultry industry and humans by HPAI infection and other zoonoses, and ? To prepare for, control and respond to influenza pandemics and other infectious disease emergencies in humans. In the NAICP it is stated that key Indicators of Project Outcome will include evidence of improved effectiveness of Government in responding to the risk of an HPAI outbreak and/or pandemic, and contained and diminishing pattern of HPAI infection in poultry. In November 2010, a World Bank-funded Implementation Status and Results (ISR) Report was prepared by the project management8. This report evaluated the degree to which the various project activities had achieved their target indicators. In addition, an internal Implementation Completion and Results Report (ICR) was recently prepared by the Project Management Unit (PMU) as part of an Intensive Learning exercise to assess the extent to which the PDOs and the development outcomes of the project had been achieved. The two ICR reports developed to date refer to the lack of an end-of-project impact assessment by the Project Management Unit (PMU), to provide an impartial assessment of the success or failure in meeting the project outcomes. To remedy this constraint, the PMU subsequently invited the International Livestock Research Institute (ILRI) to undertake such an impact assessment (ACIP/AH/CONST/IMP.ASS/2011 ? Consultancy Service for Project Impact Assessment Study). The ILRI submitted a proposal to the NAICP outlining the proposed study, which was accepted. This impact assessment evaluation has a challenging context: ? Nigeria was ranked 134th out of 178 countries in Transparency International?s 2010 corruption index, with a corruption perception score that was worse than it had been in 2008 and 2009. Corruption in Nigeria is a well publicised and widely discussed subject, nationally and internationally. For this reason, the FGN is understandably keen to provide evidence of good practice in implementing this highly publicised national programme. ? The control of HPAI in Nigeria is regarded by many in Nigeria, particularly those in the FGN, as having been a great success; this appears to be judged by the internationally accepted evidence that the threat of HPAI has receded in the country (the last two outbreaks were seen in 2008), the occurrence of only one confirmed human fatality, the documented contributions made to the preparedness and response to the Pandemic H1N1/2009 (swine flu) outbreaks which occurred in 2009 and 2010, and the apparently effective, transparent 8 http://www wds.worldbank.org/external/default/main?pagePK=64193027&piPK=64187937&theSitePK=523679&menuPK= 64187510&searchMenuPK=64187283&siteName=WDS&entityID=0000A8056_2010111514390307 20 and well documented compensation scheme put into operation in 2006. Are these outcomes supported by sound evidence? ? Nigeria made a decision not to vaccinate poultry, despite intense pressure from international sources such as FAO, complicated by reports of unofficial vaccination in some commercial poultry farms. ? While the WB credit was used to establish the NAICP, what role did the project play in the overall efforts to control HPAI and build pandemic preparedness, in particular with respect to the timing of project-funded interventions, and the roles of other players (donors and development agencies)? ? Evaluations of the NAICP to date have been undertaken internally by the monitoring and evaluation team of the project, and externally by mid-term and end of project WB consultancies. The Borrower?s Implementation Completion and Results Report (6 November 2010), under the section Lessons Learned (pp. 19 ? 20) stated that ?a final impact assessment is imperative near the end of the project to assess whether the project achieved its objectives?. This evaluation team has endeavoured to include an understanding of these overarching issues into this independent impact assessment. 21 5 Evaluation process 5.1 Purpose and scope of the evaluation The independent evaluation of the NAICP aimed to improve the relevance and performance of emergency interventions to respond to pandemic disease threats, provide accountability for the many activities undertaken, and derive lessons for better formulation and implementation of future projects undertaken with support from the WB. The evaluation also aimed to build on the project assessments carried out by the WB and the FGN which reported on achievements and outcomes, under the general guidelines of the monitoring and evaluation system set up for the project. The target audiences are both the FGN and the WB. The evaluation has drawn on accepted methodologies, including those of the OECD9 and the Independent Evaluation Group (IEG) of the WB10, to provide the FGN, WB and other stakeholders with a systematic and objective assessment of the relevance, efficiency, effectiveness, impacts and sustainability of the programmes and activities undertaken. The evaluation placed emphasis on gathering evidence of the effects of project activities on a series of outcome pillars (see section 5.4.2 for a fuller description of these) of HPAI preparedness, response and control. 5.2 Evaluation team The independent evaluation team was assembled by ILRI, comprised selected ILRI staff, Nigerian consultants and external international consultants, and led by an independent consultant as Team Leader. The team was: Brian Perry, Independent Team Leader; Tom Randolph, Evaluation Manager; Bernard Bett, veterinary epidemiologist, responsible for the animal health component; Delia Grace, veterinary epidemiologist, responsible for human health component; Anja Globig, veterinary virologist responsible for laboratory assessments; Mohamadou Fadiga, agricultural economist, responsible of the economic impact components; Keith Sones, editorial consultant; Joerg Henning, veterinary epidemiologist, animal health; Pamela Pali, impact assessment scientist, participatory disease surveillance (PDS); Jane Poole, biometrician; Heather Hannah, veterinary epidemiologist, PDS; Ekanem Ekanem, public health epidemiologist, human health; Paul Abdu, veterinary epidemiologist, animal health; Christopher Molokwu, development economist. Further details of the team are presented in Annex 1. 5.3 Evaluation programme and timetable The evaluation used a range of tools and methods, including synthesis and analysis of secondary data, wide stakeholder consultations, key informant interviews, focus group interviews, farmer surveys, desk studies and interactive visits to field sites in which the project has operated. The team adopted a consultative approach, seeking and sharing opinions with stakeholders. The triangulation of information across stakeholders was a key tool for gathering and validation of evidence. The evaluation was undertaken in three phases, which are presented below in Table 1. 9 http://www.oecd.org/document/49/0,3746,en_2649_34435_45978545_1_1_1_1,00.html 10 http://web.worldbank.org/external/default/main?theSitePK=1324361&pagePK=64253958&contentMDK=219 42248&menuPK=5039271&piPK=64252979 22 Table1: Phases of the evaluation process Phase Activities Timing 1. Preparatory phase 1. Background project document assembly 2. Proposal development and submission to WB 3. Clearance (no objection) from WB 4. Briefing with NAICP team in Abuja 5. Team planning meeting in Ibadan, evaluation framework developed 6. Study design, and development of study protocols and instruments clustered in the areas of animal health (and communications), human health (and communications) and impact assessment 7. Inception report submission 1. December 2010 2. 23rd December 2010 3. 4th January 2011 4. 18 ? 21st January 2011 5. 23 ? 31st January 2011 6. 23 ? 31st January 2011 7. 7th February 2011 2. Assembly of secondary data and undertaking of field studies 1. Central data entry facilities established at Ibadan and data entry staff training 2. Field studies in animal health, human health and impact assessment initiated (secondary data gathering procedures, questionnaire surveys, case- control studies and case studies) 1. 1 ? 6th February 2011 2. 2nd February ? 4th March 2011 3. Data synthesis, review and report preparation 1. Data analysis 2. Data synthesis 3. Inter-team presentation and discussion 4. Presentation of draft finding and conclusions to FGN stakeholders; NAICP Component coordinators Drs. Sa?idu FMoHammed, Shu?aibu Belgore, Banji Akeredolu 5. Draft report submitted to WB, PMU and peer review 6. Comments from WB, PMU and peer review received 7. Final report submitted 1. 7 ? 14th March 2011 2. 14 ? 20th March 2011 3. 21 ? 23rd March 2011 4. 24th March 2011 5. 31st March 2011 6. 8th April 2011 7. 21st April 2011 5.4 Evaluation methodology 5.4.1 Project description, components and outputs The NAICP had a total of four components (Animal Health, Human Health, Social Mobilisation and Strategic Communication, and Implementation Support/ Monitoring and Evaluation). Within these are 27 sub-components, comprising six in animal health (budget US$ 29.2 million), three in human health (US$ 18.25 million), three in social mobilisation and strategic communication (US$ 4 million) and 15 in implementation support/ monitoring and evaluation (US$ 6.8 million, divided between animal and human health). A summary of the components and sub-components is shown below in Table 2. Within many of the sub-components are further sub-divisions (activities), some of which are major elements of the project and some relatively minor. 23 Table 2: Summary of NAICP components and sub-components Project Components and sub-components Investment costs (US$) ANIMAL HEALTH a. Strengthening HPAI control programs and outbreak containment plans 6,590,000 b. Strengthening disease surveillance, diagnostic capacity and applied research 4,950,000 c. Strengthening of veterinary quarantine services 5,140,000 d. Enhancing legal and regulatory frameworks for transboundary disease prevention and preparedness capability 1,720,000 e. Improving biosecurity in poultry production and trade 980,000 f. Compensation and economic recovery 9,820,000 HUMAN HEALTH a. Enhancing public health program planning, delivery and coordination 1,480,000 b. Strengthening of national public health surveillance systems 7,930,000 c. Strengthening health system response capacity 8,840,000 SOCIAL MOBILIZATION & STRATEGIC COMMUNICATION Information and communication services 4,080,000 IMPLEMENTATION SUPPORT AND MONITORING & EVALUATION Animal and human health 6,890,000 The recent ISR and ICR reports, mentioned above, present the project outputs. The ISR has 35 indicators in the document published by the WB on 1 November 2010. In Annex 2 of the ICR of 6 November 2010, 61 outputs are listed. There are clear inconsistencies both in the nomenclature used and the number of outputs, and also in the relationship between the outputs in each document and sub-components of the project. Furthermore there are some sub-components for which outputs are not presented in either document. A tabulation of the different summary project outputs for each component in these two reports is presented below in Table 3. As part of the evaluation process, the evaluation team has attempted to rationalise these discrepancies through the use of an independent evaluation framework using a set of outcome pillars designated as bench mark ?gold standard? on which to judge attainment the attainment of key outcomes. 24 Table 3: NAICP components, sub-components and reports outputs (from the recent ISR and ICR reports) NAICP components, subcomponents and reported outputs Animal Health Components Outputs 1.1 HPAI control programmes and outbreak containment plans strengthened ? HPAI Containment (compensation paid, plans) ? Biosafety (emergency equipment provided) ? HPAI Surveillance (Desk officers, training, PDS, infection prevalence studies) ? Diagnostic capacity (laboratory development, training) ? Surveillance system (farm registration) ? Applied research (training, cost sharing studies) ? Biosecurity (upgraded infrastructure, training, film, legislation) ? Biosafety (emergency equipment provided) 1.2 Disease surveillance, diagnostic capacity and applied research strengthened 1.3 Veterinary Quarantine services strengthened 1.4 Legal and regulatory frameworks for trans-boundary animal diseases prevention and preparedness capability enhanced 1.5 Biosecurity in poultry production and trade improved 1.6 Compensation and economic recovery measures implemented Human Health 2.1 Public Health program planning enhanced ? Coordination (Steering committees) ? Legislation (Medical Waste Mgmt Plan, Pandemic Preparedness) ? Implementation (Desk officers, H1N1 structures) ? Strengthen IDSR to include HPAI (protocols & SOPs, training, computers) ? Surveillance surveys (epidemiological & serological studies) ? Strengthen labs (lab development, training) ? Mortality studies ? Community-based surveillance (training) ? Anti-viral stocks ? Medical services (guidelines, training, legislation) 2.2 National Public Health surveillance strengthened 2.3 Health system response capacity strengthened Communication 3.1 Public awareness created and disease risk perception improved ? Communication preparedness (plans, desk officers, materials) ? Stakeholder collaboration (materials, training, resource mobilization) ? Communication modules (training, public awareness campaigns) 5.4.2 Evaluation framework The evaluation team developed an evaluation framework with which to evaluate the performance of the NAICP. This is presented in Table 4, showing a set of 10 outcome pillars on which the project was assessed. The outcome pillars are not presented in the project documentation, but were developed by the evaluation team to depict the achievements necessary within each of the generic pillars of HPAI control, a benchmark ?gold standard? against which to evaluate NAICP. For the animal health elements, the outcome pillars have drawn on (and further developed) the pillars used during the 25 Second Real Time Independent Evaluation of FAO?s Global Programmes on Highly Pathogenic Avian Influenza11. For each pillar a set of generic objectives is presented, a listing of candidate components/activities which would likely fall under the objectives, and finally a set of desired outcomes. The task of the evaluation team was to determine, in as empirical fashion as possible, the degree to which the project outputs have contributed to the target outcome pillars, and what the range of impacts have been. The team evaluated both direct and indirect impacts, and intended and unintended impacts. An overview of the design of specific studies to be undertaken is described in the sections that follow. Table 4. Evaluation framework used in the impact assessment, depicting 10 benchmark pillars (seven in animal health and three in human health) Benchmark pillar Objective(s)12 Candidate components Outcomes Pillar 1. HPAI CONTROL & PANDEMIC PREPAREDNESS The development of a clear and technically sound policy for HPAI preparedness and control, effectively communicated to all stakeholders Legal framework, national policy, national strategy, contingency planning, benefit: cost considerations, research prioritisation, industry development, poverty reduction interface, identification, engagement and communication with all stakeholders -Sound policy & strategy in place -Effectively communicated -All key human and animal health stakeholders involved Pillar 2. HPAI SURVEILLANCE Establishment and revision of effective, sustainable and affordable surveillance systems for HPAI in target domestic and wild bird populations Passive surveillance, Active surveillance, Wild bird surveillance -Infection status in domestic and wild bird populations known and internationally recognised Pillar 3. HPAI DIAGNOSIS, DIFFERENTIAL DIAGNOSIS & PATHOGEN CHARACTERIZATION Establishment and maintenance of internationally recognised laboratory capacity to confirm and where appropriate characterise HPAI infections Sample collection & shipment; cold chain viability; laboratory facility development; laboratory equipment & reagents; diagnostic quality control; laboratory network & interface; capacity building of laboratory staff -Diagnostic capacity conducive to effective geographical and sample load established -Adequate sensitivity & specificity achieved and internationally recognised Pillar 4. HPAI OUTBREAK CONTAINMENT & DISEASE CONTROL Plan and implement technically sound, effective affordable, sustainable and socioeconomically acceptable intervention measure to control or Intervention measures (depopulation, decontamination, disposal, movement control, awareness raising, vaccination, poultry restocking, etc); Compensation -HPAI effectively contained and incidence in at-risk poultry populations eliminated -Disease status internationally 11 http://www.fao.org/pbe/pbee/common/ecg/391/en/Final_RTE2_report.pdf 12 These are evaluation derived objectives considered necessary to achieve the designated outcomes in each pillar; they are not NAICP objectives. 26 eradicate HPAI recognised Pillar 5. EPIDEMIOLOGICAL CAPACITY FOR STRATEGY DEVELOPMENT & INTERVENTION TARGETING Collect, synthesise and analyse data on the dynamics and impacts of HPAI, and use the outputs to inform policy and strategy for HPAI control Epidemiological data handling, processing and analysis; data flow and communication; data reporting, use and presentation; poultry population demography; outbreak investigation; value chain studies; risk assessment (based on critical control points); socioeconomic impact -HPAI infection dynamics understood and internationally recognised -Sound evidence base to HPAI intervention strategies developed Pillar 6. HPAI PREVENTION Put in place technically sound affordable and socially viable measures to minimise the risk of HPAI spread, and reduce the risk of human infection Biosecurity; communication; human protection; vaccination; farm/unit registration; market and slaughter practices; Industry support and restructuring -Biosecurity measures among poultry producers, traders and marketers demonstrably improved -Innovative approaches to safe marketing of poultry investigated & established Pillar 7. POULTRY SECTOR RECOVERY To minimise the impact of HPAI on the credibility and economic viability of the poultry industry Public relations of risk avoidance with poultry and poultry products; viable compensation scheme; dialogue with and support to all sections of poultry industry -Affected poultry farms back in business -Compliance with FGN policies for HPAI is enhanced Pillar 8. PUBLIC HEALTH PANDEMIC PREPAREDNESS The development of a clear and technically sound policy for HPAI preparedness and control, effectively communicated to all stakeholders Legal framework; national policy; national strategy; contingency planning; research prioritisation; identification, engagement and communication with all stakeholders; capacity- building for physicians; provision of diagnostic and treatment facilities; provision of PPEs, drugs and other supplies for AI control -Sound policy & strategy in place and effectively communicated -All key human and animal health stakeholders involved Pillar 9. INFLUENZA SURVEILLANCE Infection status in human populations known and internationally recognised Laboratory capacity development; surveillance networks established; routine surveillance of influenza-like illness established; active and risk-targeted surveillance -Infection status in humans known and internationally recognised Pillar 10. PUBLIC HEALTH RESPONSE AND CONTAINMENT To ensure the human population at large and the poultry associated households are prepared and protected from the threat of HPAI Public health communications; risk awareness; laboratory capacity development; sound poultry handling and marketing practices; communication on human behavioural practices -Health facilities regularly stocked with target quantities of anti-viral medication -Tested preventive measures communicated to and adopted by at risk health workers and the general public 27 5.4.3 Sources of data13 The evaluation team assembled secondary data from various sources, including from the NAICP, and designed and undertook selected studies to collect primary data on certain aspects of the interventions undertaken by NAICP. Animal health (and associated communication) (i) Analysis of secondary data provided by the project and state desk officers ? Epidemic curves and spatial distribution of outbreaks ? Timeline of activities ? Time difference between events ? Measures of mortality and depopulation ? Compensation rates and timeliness of compensation payments ? Quantification of transmission rate of outbreak occurrence (ii) Field surveys involving poultry farmers, fowl sellers, poultry market managers, surveillance and response teams, quarantine officers and key informant interviews ? Farmer interviews ? Participatory surveys to assess the level of underreporting ? Evaluating biosecurity at LBM ? Evaluating biosecurity used by fowl sellers ? Surveillance, response and diagnostic team officers and their supervisors ? Quarantine stations and the knowledge of quarantine officers ? Key informant interviews (iii) Microbiological study comparing traditional and improved live bird markets A. Kaduna (Salmonella spp. and Escherichia coli O157) B. Lagos(Salmonella spp. and Escherichia coli O157) Human health (and associated communication) (i) Analysis of secondary data from the project ? Review of National Influenza Sentinel Surveillance ? Review of swine influenza (H1N1) surveillance and control ? Review of State IDSR database (ii) Key informant interviews ? Policy-makers and experts ? State officers (epidemiology & surveillance ? Project desk officers ? State officials (iii) Surveys of Knowledge, Attitudes and Practices ? General public ? School children ? Physicians 13 A full description of data sources and methodologies is provided in the detailed reports, Annexes 2 ? 7. 28 Laboratory assessments The evaluation of the animal and human health laboratories included: (i) Visits to selected Animal and Human Health Laboratories ? on-site inspections of the facilities ? interviews with key laboratory staff (ii) Analysis of secondary data provided by the laboratories Broader economic impact assessments As part of the independent evaluation, secondary and limited primary data were assembled for and analysis of three main areas; the broad economic impact of the NAICP, the impact of the HPAI and its control on the poultry sector, and the impact of establishing live bird markets. (i) Analysis of secondary data for economic impact analysis ? Social welfare analysis ? Benefit cost ratio analysis ? Breakeven analysis (ii) Primary data for sectoral impact analysis ? Interviews and focus group discussions with poultry value chain operatives ? Collect enterprise data on live bird market traders, broiler producers, layer producers, processors, traders, transporters, feed millers, toll millers, breeders/hatcheries, LBM processors ? Performance analysis of value chain operatives enterprise (iii) Primary and secondary data for the financial analysis of live bird market 29 6 Evaluation findings 6.1 Timelines of HPAI outbreaks For a full understanding of the impacts of the various interventions that occurred at different levels, it was important to dissect the timelines of the disease (both internationally and in Nigeria), of the responses to it and of the engagement of the NAICP. Certain of these are depicted below in Figure 2, below. Figure 2: Timeline of key events in Nigeria To put these timelines in context, Figure 4, below, shows H5N1 outbreaks/cases in poultry by continent, by month, since December 2003. Figure 3: Timeline of HPAI outbreaks globally 2005 2006 2007 2008 2009 2010 2011 December : Technical Committee of Experts established February : National Inter- Ministerial Steering Committee established June: World Bank loan approved September: Funds disbursed Jan: human fatality H1N1 outbreaks H1N1 fatalities 30 At a global level, the disease has progressively declined since 2006, and the only African focus remaining since 2008 has been in Egypt. The apparent seasonal incidence has continued in endemic countries. This inevitably raises the question as to whether its eventual disappearance from Nigeria was influenced by various host and production system factors which affect the dynamics and potential for endemicity of HPAI. These include the relative absence of ducks, which are considered as maintenance hosts of HPAI virus infection, in most ecosystems of Nigeria in which poultry are kept. The re-emergence of HPAI in July 2008 after a 9-month absence from Nigeria coincided with the isolation of virus belonging to sub-lineage III, a distinct genotype of HPAI not previously recorded in Africa, from tracheal swabs collected from apparently healthy ducks (Fusaro et al., 200914). 6.2 Key findings by outcome pillars The key findings and results under each of the NAICP sub-components and activities are presented below in the context of each of the 10 outcome pillars presented in the evaluation framework (Table 3). The pillars are taken as the series of gold standard outcomes; the sub-components and activities of NAICP which appear to fit under each are then identified. For these, the report summarises in bullet point format the key findings emerging from the NAICP under each pillar, and then details the results behind the summarised findings. 6.2.1 Pillar 1. HPAI Control and Pandemic Preparedness Pillar objective15: The development of a clear and technically sound policy for HPAI preparedness and control, effectively communicated to all stakeholders. Candidate components: Legal framework, national policy, national strategy, contingency planning, benefit: cost considerations, research prioritisation, industry development, poverty reduction interface, identification, engagement and communication with all stakeholders. Outcomes: Sound HPAI policy in place; all stakeholders involved and informed. Under this outcome pillar, the following sub components and activities were listed in the NAICP document: ? Rapid assessment of national veterinary services (1e-i) ? Updating the National Emergency Contingency Plan for HPAI (1e-ii) ? Strengthening of veterinary services (1b-i) ? Communication preparedness (3a-i) ? Developing capacity building modules and rapid social and communication assessment (3a- iii) Key findings ? The emergency nature of the circumstances affected the usual project planning approach and demanded decisive and rapid action. ? A national plan was put in place. Although the final approval of the Integrated National Avian and Pandemic Influenza Response Plan (INAPIRP) was delayed until 14 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2681125/ 15 As mentioned in the evaluation framework presented in Table 3, these are evaluation derived objectives considered necessary to achieve the designated outcomes in each pillar, not NAICP objectives. 31 May 2007, the draft contingency plan developed earlier was fit for purpose and enabled emergency response activities to proceed in the interim. ? The INAPRIP was generally considered to be useful by federal, state and LGA officials. ? Key informants in the NAICP considered that the bringing together of partners in a multi-sectoral, multi-disciplinary programme was a major output of the project. This was reportedly the first time this has occurred in Nigeria and was considered to be an approach that could be exploited in the future for other zoonotic diseases. Results This pillar represents the planning, policy formulation, strategy development and communication hub of a national disease control response mechanism. The FGN appeared to have heeded the international call for forward planning before HPAI had been detected in Africa: a Technical Committee of Experts on the prevention and control of HPAI met in Abuja in December 2005, before any cases had been detected in Nigeria, and came out with a set of recommendations to prevent the introduction of HPAI into Nigeria. From these, a national HPAI contingency plan was developed, drawing on local and international expertise and in conformity with the Food and Agricultural Organisation (FAO) of the United Nations format for preparing National Animal Disease Emergency Preparedness and Contingency Plans for major Transboundary Animal Diseases (TADs) and also the Australian Veterinary plan (AUSVETPLAN). With the outbreak of HPAI in February 2006, an Inter-Ministerial Committee on HPAI, comprising the Federal Ministries of Agriculture and Rural Development, Health and Information and National Orientation, together with representatives of the international (UNICEF and UNDP) and donor community (WHO, WB, FAO, EU, DFID and USAID) and the private sector as represented by the Poultry Association of Nigeria (PAN), was set up to ensure proper coordination of information and activities on the prevention, management and eradication of the disease in the country. A Technical Steering Committee on Avian Influenza was also established which reported to the Inter-Ministerial Committee. The Inter-Ministerial Committee was also charged of the supervision of a newly created AI Crisis Management Centre. The structure of the avian influenza coordination bodies is illustrated in Figure 4, below (source Obi et al., 200916). 16 Uzochukwu-Obi, T., A. Olubukola, and G. A. Maina. 2008. Pro-poor HPAI risk reduction strategies in Nigeria ?Background Paper, Africa/Indonesia Team Working Paper No. 5, IFPRI. http://www.ifpri.org/sites/default/files/publications/hpairr05_nigeria.pdf 32 Figure 4: Structure of HPAI coordination committees and groups in Nigeria The national contingency plan for HPAI (Comprehensive Emergency Preparedness and Differentiated Action Plan for the Surveillance and Control of Highly Pathogenic Avian Influenza in Nigeria, September 2006, Federal Ministry of Agriculture and Rural Development) went through a series of reviews17, and was finally published as an internal working document in September 2006. When the first HPAI case in poultry was confirmed in Nigeria in February 2006, work began on the development of an Integrated National Avian and Pandemic Influenza Response Plan (INAPIRP). A draft was circulated to the Technical Steering Committee for comments in October 200618. In May 2007 the Technical Steering Committee requested that the integrated plan was officially approved before the imminent end of the then current presidency: subsequently the Federal Executive Council prepared a memo for its approval at the end of May. The Integrated National Avian and Pandemic Influenza Response Plan covered the period 2007-2009. Senior NAICP officials stressed to the evaluation team that in the face of rapid spread of HPAI throughout much of Nigeria in 2006, there was an urgent need to take action, even in the absence of the completed integrated disease control plan. The emergency nature of the outbreak precluded the usual project planning approach and demanded decisive action on the ground. A key strategy for the planning of the NAICP project was to use existing structures and institutions, using project funds to build on and strengthen these. Certain innovations were supported by the project, such as the recruitment of desk officers for animal health and human health to ensure effective coordination between the animal and human health sciences. Although rapid appraisal of national veterinary services was one of the planned activities under the NAICP (activity 1 e (i)), this was dropped as an internal review of the national veterinary services was 17 The national HPAI prevention and contingency plan was prepared in December 2005; the FMoARD commissioned a group of experts to do this (Olawoye, J., 2010. Report of Institutional Analysis of Public and Private Disease Response Capacity in Nigeria. Pro-poor Strategies for Controlling HPAI in Nigeria, Ibadan). The INAPIRP had to wait until May 2007 when a memo was prepared for its approval. 18 Minutes of Technical Steering Committee meeting 33 already being done at this time and, through collaboration between the FGN and the World Organisation for Animal Health (OIE), the OIE?s Performance of Veterinary Services (PVS) tool was utilised to assess the national veterinary services and a gap analysis was also conducted to identify any deficiencies. In terms of national planning and coordination, due to the personal interest taken by the then President of Nigeria (who is also a poultry farmer), strong national leadership was provided in the response process. As a result the then CVO and the Director of Special Duties in the Federal Ministry of Health (FMoH) both had unhindered access to their respective ministers, who in turn also easily communicated with the President19. It is also widely observed that in response to the HPAI outbreak, there was good coordination and communication between key senior players, including the Director of NVRI, public health officials in the FMoH, Federal Ministry of Agriculture and Rural Development (FMoARD), Federal Ministry of Information and National Orientation (FMoINO) and international organisations, including OIE, FAO and the WB. This was in contrast to the situation before the outbreak when, in the words of a senior NAICP official, the FMoH and FMoARD ?couldn?t even sit at same table?. The achievement by the project to bring together partners in multi-sectoral, multi- disciplinary programme is viewed by those directly involved as a significant achievement ? the first time this has occurred in Nigeria. It is envisaged that this foundation can be built on in the future for campaigns against other zoonoses, such as rabies and tuberculosis. To operationalise coordination at state and LGA levels, animal health and human health Desk Officers were recruited for all 37 states and 773 LGAs. At state level these were recruited through a competitive process which was open only to government civil servants. The state Desk Officers usually combined duties associated with the avian flu response with other responsibilities. At the LGA level, existing surveillance and control officers were simply designated as project Desk Officers. At the outset considerable opposition was reported to the creation of what some viewed as a ?parallel system?; it was argued that the existing state epidemiologists should have assumed these roles, but senior NAICP officials considered that appointing the Desk Officers provided much clearer reporting lines to the project and their roles and responsibilities were also clear: in some cases the state epidemiologists also served as the Desk Officers. The state and LGA Desk Officers reportedly worked together very effectively, and were considered by NAICP officials to be ?the backbone of the surveillance system?. WB-funded projects require the development of a national medical waste management plan. This was done by EnvironQuest, a specialist environmental and waste management company, and covered both safe disposal of infected poultry carcasses as well as safe medical waste management at health care facilities and waste dump sites. The plan was published in January 2007. Fortunately with just one human case reported, the envisaged massive increase in potentially infectious and dangerous medical waste did not materialise, and most of the dead and culled poultry were disposed of by burning or burial. 6.2.2 Pillar 2. HPAI Surveillance Pillar objectives: Establishment and revision of effective, sustainable and affordable surveillance systems for HPAI in target domestic and wild bird populations. Candidate components : Passive surveillance, active surveillance, wild bird surveillance. 19 Personal communication, NAICP Component Coordinators, Animal and Human Health 34 Outcomes: HPAI infection status effectively determined and internationally recognised. Under this outcome pillar, the following sub-component appears in the NAICP: ? Strengthening Animal Disease Surveillance (1b-ii) Key findings ? No outbreaks of HPAI have been reported in Nigeria since July 2008. ? During the project the diagnostic turnaround time improved. Sophisticated laboratory facilities were, however, not established until after the last outbreaks suggesting that speed of response and the reduction in turnaround time were due to improved transport of suspect samples from the field and streamlining of diagnostic laboratory procedures at NVRI. ? Surveillance teams underwent training on topics such as disease tracing, reporting and sample collection. However, the turnover rate of staff is high; NAICP efforts to build capacity in surveillance may not be sustained unless periodic training is offered. ? Surveillance teams also participated in monitoring other diseases, mainly African swine fever and Pandemic H1N1/2009 (swine flu). ? Participatory disease surveillance (PDS)-derived data is not being used effectively. PDS data exists in a database separate from the conventional NADIS network and is not analyzed. There is little evidence for the data leading to actions, decisions or changes. ? While successfully engaging farmers, PDS did not improve surveillance, nor result in increased reporting of any disease despite the endemic prevalence of at least two notifiable diseases (ND and CBPP). The emphasis of PDS for surveillance was limited to case finding for HPAI. ? PDS was useful for appraisal or as a survey of animal health concerns. Village visit discussions yielded important community input which informed vaccination campaigns, actions which would not have happened in the absence of the village visits from the PDS project. ? The basic reproductive rate20 (R0) for HPAI in Nigeria was calculated to be between 1.32 and 1.46 (with village as unit of analysis), just above the threshold (R0 = 1) required for the disease to spread, but probably not high enough for the establishment of endemicity. ? Wild bird surveillance was not undertaken under the auspices of the NAICP. Results Disease surveillance Animal disease surveillance has reportedly been a longstanding feature of veterinary services in Nigeria. Nigeria developed a system of surveillance under the PACE programme for the control of rinderpest and other diseases, and used this as a basis for the development of HPAI surveillance. This included a centrally-based epidemiology unit, and an information system, originally developed under the ARIS system (which is no longer functional) under AU-IBAR. Nigeria has since developed its own animal disease information system, designated the National Animal Disease Information and Surveillance system (NADIS). The evaluation team conducted analyses of secondary data derived from NAICP to assess the temporal and spatial dynamics of HPAI during 2006 ? 2008. These are summarized below, and are detailed in Annex 2. 20 Anderson, R.M., May, R.M. (1991). Infectious Diseases of Humans; Dynamics and Control. Oxford University Press, New York, 757 pp. 35 Two epidemic curves were generated to study the temporal pattern of the outbreaks at a village level and evaluate the timing of the various interventions implemented by the project. The first considered dates when farmers first noticed the disease on their farms as the incidence dates (Figure 5) while the second used dates when farmers formally reported the outbreaks to the veterinary authorities (Figure 6). The numbers of outbreaks used for each curve were 222 and 263, respectively. Figure 5: Temporal pattern of the HPAI epidemic considering dates when farmers noted outbreaks on their farms as the incidence dates. While the two curves are similar, the first suggests that the epidemic might have started a few months earlier than was officially reported. Figure 6: Temporal pattern of the HPAI epidemic using dates when farmers reported outbreaks on their farms as incidence dates. A total of 16 of the 37 states were affected by the outbreak in the first year (2006). This increased to 22 in 2007 and declined to 2 in 2008. Using states as the unit of analysis, the incidence risk was estimated to be 55.2% in 2006 and 56.3% in 2007. Importantly, these measures estimate the risk of a state becoming infected and do not represent HPAI risk at village, farm or bird levels. The number of reported outbreaks by state is illustrated in Figure 7. States with a high number of reported outbreaks included, in decreasing order, Kano, Kaduna, Plateau, Ogun, Bauchi and Lagos. Figure 8 shows the spatial distribution of the local government areas that were affected. 0 5 10 15 20 25 30 35 40 De c Fe b Ap r Ju n Au g Oc t De c Fe b Ap r Ju n Au g Oc t De c Fe b Ap r Ju n Au g Oc t De c N u m b e r of o u tbr e ak s Month/Year 2006 2007 2008 0 5 10 15 20 25 30 35 40 45 Ja n Ma r Ma y Ju l Se p N o v Ja n Ma r Ma y Ju l Se p N o v Ja n Ma r Ma y Ju l Se p N o v N of o u tbreak s Month/Year 2006 2007 2008 36 Figure 7: The number of reported HPAI outbreaks by state between 2006 and 2008. Figure 8: The frequency of HPAI outbreaks in affected local government areas in Nigeria between 2006 and 2008 (n=263 outbreaks). 0 10 20 30 40 50 60 70 Abi a A d am aw a A kw a Ib o m An ambr a B au ch i B ay e ls a B e n u e B or n o Cross R iv er De lt a Ebony i Ed o Ek it i Enu gu FC T Gom b e Im o Jigaw a Kad u n a Kan o Katsin a Ke b b i Kog i Kwa ra La go s Na saraw a Ni ge r O gu n Ond o Osu n Oy o P latea u R iv er s Soko to Tarab a Yob e Zamf ar a N of o u tbreak s States 2008 2007 2006 2006 2007 2008 37 The transmission rate of HPAI outbreaks between villages was estimated by the evaluation team using the outbreak data set. The number of susceptible villages was not available but a working figure of 90,000 was used21. Ward et al. (1997) has indicated that transmission parameters and reproductive numbers are insensitive to the number of susceptible villages used, because at any one time the number of infectious villages is much smaller than the number of susceptible villages. Table 5 gives the results of the analysis, which show that the rate of transmission dynamic of the disease was similar between the two phases of the outbreak. Table 5: Estimates of the transmission coefficients (?) and basic reproductive rate (Ro) derived by the evaluation team for each phase of the outbreak Phase of the epidemic No. of days No. of cases Mean ? Estimated R0 Phase 1a 195 98 0.22 1.32b Phase 2c 381 154 0.21 1.46d a January ? June 2006 b Assuming that the median number of days between noting the outbreak and depopulation is 6 days c November 2006 ? November 2007 d Assuming that the median number of days between noting the outbreak and depopulation is 7 days Surveillance activities clearly comprise a significant component of both preparedness and response to HPAI. Surveillance mechanisms are central to good intelligence on disease occurrence, to responsible international reporting of disease presence and to a strong evidence-base to disease control strategies and policies. Traditionally national surveillance systems for livestock diseases in Nigeria are built on reporting by veterinary services, which are contingent on the capacity of veterinary services to gain access to relevant livestock production systems at appropriate intervals, and to have the necessary awareness and diagnostic skills, supported as appropriate by laboratory capacity. Generally in Nigeria these passive surveillance systems are seen to be weak, but appear to have improved to varying degrees as a result of the NAICP and other capacity building initiatives provided to respond to HPAI. This improvement has been in various elements, notably enhanced training of field staff, the training of ancillary field staff, and improvements in communication between the field and the NVRI. However, improvements are considered by the evaluation team to be quite modest in meeting the broader demands of an effective and sustainable national system of animal disease surveillance for effective HPAI control and beyond. Diagnostic turnaround The evaluation team observed that during the life of the NAICP the diagnostic turnaround improved. But as the more sophisticated laboratory facilities were not established until after the last outbreaks, the speed of response and the reduction in turnaround time were due largely to the improved transport of suspect samples from the field and to the streamlining of diagnostic laboratory procedures within NVRI: the time between despatching samples and receiving laboratory confirmation was shortened in the second half of 2006, but real-time PCR was only established in 2009 (conventional PCR was available in 2006, with an approximately 4 hour longer processing time than real-time PCR; see section 6.2.3). The decentralisation of surveillance teams to operate under the State level authorities may also have contributed to the improved diagnostic turnaround. 21 http://www.cdc.gov/mmwr/preview/mmwrhtml/00001960.htm 38 Given the fading priority attached to HPAI in Nigeria, which appears to be justified given the recent absence of the disease, there is a strong argument that greater advantage should have been taken of the NAICP to broaden passive surveillance to address other national priorities, to justify the considerable financial outlay and even the very survival of institutions and capacities newly established. This is especially important in the poultry sector which is critical to smallholder and emergent farmer livelihoods and national food security, with Newcastle disease (ND) and Gumboro disease among priority diseases acting as serious constraints to rural and commercial poultry rearing. It is unclear how well the HPAI surveillance system has adapted to accommodate these needs, but with the implementation of PDS (see below), examples of a reorientation of priorities was observed. Generally, there was a significant decrease in passive surveillance samples sent to the laboratory for investigation between 2009 and 2010, in the absence of any further reported cases of HPAI. This shift is perceived to represent a rational readjustment of priorities in diagnostic surveillance and resource use. Wild bird surveillance Wild bird surveillance was not carried out by the project. The Wildlife Group of FAO, and Wetlands International carried out collaborative projects with NVRI. Participatory disease surveillance (PDS) The evaluation team evaluated the outcomes of capacity building in participator epidemiology (PE) and PDS on national infectious animal disease surveillance, based on indicators adapted from established surveillance evaluation criteria22, and developed an evidence base for the appropriate application of participatory methods for surveillance systems in Nigeria. In total, data was collected from interviews, questionnaires and a workshop with 36 individuals: 25 PE trained representing the public (72%), private (16%) and academic (12%) sectors, and 11 key informants from NADIS, NVRI, veterinary schools and international organizations. The most significant change technique was used during the workshop to generate information on experiences of PDS implementation. USAID funded the first two introductory PE courses in October 2008 and February 2009 as well as a refresher course in June 2009, training 20 Nigerian veterinarians. The WB funded training for a further 120 veterinarians in October 2010 with a shortened introductory PE course. In groups of up to four, each of the PE trainees from the first two courses conducted 60 village visits, while PE trainees from the last course each conducted 30 village visits. The evaluation?s major concerns were: 1) the over-reliance on qualitative data and self reports, with few concrete examples of verifiable events; 2) the lack of data robustness, as most claims could not be classified into objectively measurable indicators and were too ambiguous to validate; and 3) differential use of words and terms. The evaluation attempted to distinguish between ?actualized? applications and ?suggested? potential applications in order to assess the evidence for and clarify the role of participation in animal health. The understanding of PDS in Nigeria is not restricted to the use of participatory methods, approaches and tools for disease surveillance; rather, the term PDS is used to describe all types of participatory actions related to animal health. The difference between 22 Perry et al. for FAO, 2009 & 2010; Coker R et al, 2008, 2010 & in press; Rushton & Rushton, 2009; CARE?s community based surveillance models, 2009; CDC surveillance evaluation guidelines, 2001 & 2004; Dufour B, 2009; Hadorn & Stark, 2008; Stark et al., 2006; Meynard et al., 2008; Sosin, 2003; US$A/APHIS Veterinary Services, 2006 & others. 39 surveys (or appraisals) and surveillance was equally vague; a 2009 evaluation by a Field Epidemiology and Laboratory Training Programme (FELTP) fellow highlighted that the PDS initiative was essentially a series of village surveys and hence did not meet the definition of surveillance system. Three overarching findings emerged: 1) PDS data is not effectively used. PDS data exists in a database separate from the conventional NADIS network and is not analyzed. There is little evidence for the data leading to actions, decisions or changes. 2) PDS did not improve HPAI surveillance. Furthermore, PDS did not result in increased reporting of any disease, despite endemic prevalence estimates for at least two notifiable diseases (ND and CBPP). The emphasis of PDS for surveillance was limited to case finding for HPAI. This is consistent with the best potential value of participatory surveillance as suggested by PE practitioners, namely case finding during outbreaks. Community involvement in case finding during outbreaks would not alter the extractive nature of surveillance. 3) PDS successfully engaged various veterinary sectors. PDS was useful for appraisal or survey of broader animal health concerns. Village visit discussions yielded important community input which informed vaccination campaigns (for diseases other than HPAI), actions which would not have happened in the absence of the village visits from the PDS project. 6.2.3 Pillar 3. HPAI Diagnosis, Differential Diagnosis and Pathogen Characterisation Objectives: Establishment and maintenance of internationally recognised laboratory capacity to confirm and where appropriate characterise HPAI infections. Candidate components: Sample collection & shipment; cold chain viability; laboratory facility development; laboratory equipment & reagents; diagnostic quality control; laboratory network & interface; capacity building of laboratory staff. Outcomes: Optimal sensitivity and specificity of diagnostic tools established and results in international public domain. Under this outcome pillar, the following sub-component appears in the NAICP: ? Upgrading diagnostic capacity (1b-iii) Key findings ? Diagnostic capabilities were improved over the project lifespan, but corresponding infrastructure development and reagent supply has not been completed. ? During the outbreak period (2006-2008) there is no clear evidence that the NAICP-related actions have had a direct impact on the efficient and cost-effective detection of avian influenza viruses or other disease viruses. ? WB procurement procedures complicated the rapid response needed in an emergency operation. This has contributed to serious delays in the procurement of construction, equipment and consumables. As a consequence, NVRI has gone ahead to procure many of the requested items on its own. ? The National Avian Influenza laboratory at NVRI now meets OIE standards for molecular 40 highly-sensitive detection and specific subtyping of avian influenza viruses H5, N1, H7 and H9. ? Personnel and infrastructural resources are adequate to detect and diagnose AI H5N1 at NVRI. For the determination of pathogenicity (HP H5N1) samples still need to be sent to an International Reference Laboratory until a sequencer is in place. ? Molecular techniques are still to be established at the Veterinary Teaching Hospitals; even when established their sustainability is questioned by the evaluation team. ? Joint actions of FMoA, FMoH, NAIC and other development partners contributed to interdisciplinary and international laboratory networking, sharing of resources and global transparency. Results Laboratory capacity for HPAI diagnosis in Nigeria is centred at the National Avian Influenza Laboratory housed at NVRI, Vom, near Jos in Plateau State. In addition there are five laboratories in Veterinary Teaching Hospitals (VTHs) of State veterinary faculties: Ibadan, Maiduguri, Nsukka, Sokoto and Zaria (see map, Figure 9, below). Broader laboratory diagnostic networking includes laboratories in the West African sub-region and the FAO/OIE reference laboratory in Padua, Italy. The NVRI has the national mandate for the diagnosis and investigations into animal and poultry diseases and vaccines, and research into various aspects of animal and poultry diseases epidemiology and control. Since 2007 the NVRI has been designated by the FAO coordinated Laboratory Network (RESOLAB) as a regional laboratory for the diagnosis of HPAI and other TADs for West and Central Africa. Collaboration, both nationally and internationally and linkage to the satellite labs, the VTH, universities and research centres in the country, in the region (22 other labs from West and Central Africa through regular meetings as Regional laboratory) and internationally/overseas has been improved over the years. The strengthening of diagnostic facilities capable of supporting HPAI preparedness and response has been a prominent component of the NAICP. This has involved supporting the purchase and installation of equipment, provision of reagents, training of laboratory staff, facilitation of proficiency testing networks for PCR and hemagglutinin inhibition (HI) testing, interactions between laboratory staff, training in sample collection and shipment, international sharing of virus isolates and the raising of scientific and risk awareness on influenza viruses. It appears that most of the activities have had a relatively narrow focus on procuring equipment and building technical skills and competencies in Vom, and even this has experienced delays in establishing more sophisticated molecular techniques. There was some duplication of investment by other donors, and the evaluation team heard criticisms of the inadequacy or lack of budgets for maintenance and for replacing and safely storing essential reagents. There is no clear evidence that any of the NAICP related actions have had a direct impact on the efficient or cost-effective laboratory diagnosis of avian influenza viruses during the outbreak period (2006-2008) or other disease viruses. The provision of equipment and civil works for new laboratories by the NAICP all happened after the outbreak period. Construction was finalized by 2010/11, equipment was provided at the earliest in 2009 (for the Human Health laboratory site), and 41 the supply of reagents is still pending; there is therefore no clear evidence that the NAICP contributed to improvements in HPAI H5N1 detection. The consideration of the need for broader laboratory diagnostic capacity to cover other diseases played a secondary role, given that the NAICP was an emergency response project targeted very specifically at HPAI. In future responses of this nature, it is considered important that greater advantage be taken of emergency funding opportunities to broaden and deepen the diagnostic capacity, but acknowledges the added complication involved of long-term supply of (expensive) reagents with a limited shelf life. Once upgrading has been completed, it is expected that the VTHs will take on more of the routine diagnostic load from NVRI by screening incoming field samples for positive reactors by the use of PCR and forwarding these to NVRI for confirmation. However, in practice most samples have been sent directly to the national laboratory at NVRI through the NAICP Desk Officers in each state of the federation and the Federal Capital Territory (FCT). Figure 9: Location of Veterinary Teaching Hospitals In early 2006 it was planned to construct a BSL-3 laboratory on the campus of NVRI. However, a contract has yet to be signed and no equipment has been supplied. The reasons for delays were reported to be: ? Limited availability of commercial expertise in the construction of BSL-3 facilities within Africa. ? Procurement processes of WB did not allow for sole sourcing, and it was almost impossible to find multiple sources of constructors ? Late procurement of equipment on AH site, because it was thought to first finish the design and construction before equipment was procured As time was running out, it was agreed to install modular BSL-3 units, which is expected for contract signature in March/April 2011, with construction finalized in May. It is planned to install the equipment upon finalization of the BSL-3 module. Infrastructure, available technologies and biosafety have constantly been improving since 2006, initiated by the NVRI and without the input of WB funds. With financing from the NVRI budget, and 42 supported by other development partners (among them AU-IBAR, FAO, EU, US$A and the FAO/OIE Reference Laboratory for Avian Influenza at the IZSVe in Padua, Italy), it was possible to upgrade and equip the laboratory and improve the overall operational status to ensure biosafe and bio-secure work with infectious pathogens. Workflow is arranged in a way that cross-contamination as well as reagent and sample damage is reduced to a minimum (inverter for freezers where critical reagents/samples are stored and liquid nitrogen available). The floors are equipped with -80?C and - 20?C freezers with daily temperature checks. Furthermore, liquid nitrogen ensures proper storage of reagents, samples and associated materials. Since 2008/2009 the NRL for AI has been able to perform a range of diagnostic techniques for AI diagnosis, including real-time RT-PCR , for Influenza A, H5, N1, H7, H9. The laboratory is still waiting for reagents from NAICP and had to cease investigations by real-time PCR on samples for active surveillance as the expensive and rare reagents that have been self-procured need to be stored for suspicious samples. In 2008, NVRI asked for Rotagene real-time PCR cycler (Corbett Research) to be provided by the WB funds as this machine is not dependant on further consumables, as is the case with other real-time PCR machines, and therefore would in the long run be cheaper. The WB, however, proposed another (ABI) machine as a consequence of the bidding process. Consequently, the Rotagene cycler was provided by SPINAP and has been in use since October 2009. WB procurement procedures complicated the rapid response needed in an emergency operation. This has contributed to serious delays in the procurement of construction, equipment and consumables. As a consequence, NVRI has gone ahead to procure many of the requested items on its own. Inter-institutional and international collaboration has resulted in at least 27 scientific papers published in research journals (some high ranking), of which 14 provide evidence of collaboration with international partner institutes, e.g. OVI in South Africa, CIRAD in France, IZSVe in Italy and others. As a general statement, the diagnostic capacity for HPAI in Nigeria has progressively improved over the period 2006 to present, particularly at NVRI. However the process is not over, certain infrastructures remain incomplete and the reagents supplied will expire within one year. The contract for establishment of a BSL-3 module type at NVRI (Vom) is being signed possibly in April. During the HPAI outbreak periods, NVRI has relied on the traditional diagnostic procedures of egg inoculation, haemagglutination inhibition and AGID tests; conventional PCR was available at the Central Diagnostic laboratory at NVRI since 2006 and was also used for confirmation; conventional PCR and real-time PCR procedures (which were provided by sources other than the NAICP) were established at the NRL AI, Vom only after the last reported outbreaks in 2008 (see timeline, Figure 10, below). Importantly, it is not possible to attribute all the improvements in diagnostic capacity directly to the NAICP. There have been many players engaged in the building of laboratory capacity. 43 Figure 10: Timeline for HPAI diagnostic capacity Even with the delays in upgrading the NVRI facility in Vom and the VTH laboratories, the key (national reference) influenza laboratory (NRL AI at NVRI) has improved diagnostic procedures and these are now deemed to meet international OIE standards. When fully upgraded (and supplied with reagents correctly stored), the laboratories of Vom and the VTHs should be able to service future outbreaks of influenzas and run PCR for up to 1000 samples23 (but only for a period of one year without new resources). The shortage of reagents has affected the ability to run some active surveillance samples; it is understood that a substantial number of them remain in cold storage, as NVRI does not want to use them for non-suspicious samples and then be faced with a new outbreak to which they cannot respond effectively and rapidly. The constructions of new labs and the installation of equipment are scheduled to be finalized in 2011. Once the new laboratories are functional, biosafety and work flow will be improved and are envisaged to be applicable to other emerging or zoonotic diseases. Apart from using rapid tests and serological assays carried out between 2006-2008, the VTHs have not screened samples for HPAI. Without further financial inputs, laboratory activities, especially rapid nucleic acid detections, will likely cease due to lack of (expensive) reagents. A wide range of training modules for staff at NVRI Vom and VTH related to the laboratory diagnosis of HPAI were carried out and some were also funded by WB; most of these occurred in 2006-2008. Based on interviews with staff in different laboratories, the evaluation team concluded that collaboration between the animal and human laboratory diagnostic networks has been enhanced, and that this had been mostly effected through strong personal contacts rather than closer institutional relationships. 6.2.4 Pillar 4. HPAI Outbreak Containment and Disease Control Objectives: Plan and implement technically sound, effective affordable, sustainable and socioeconomically acceptable intervention measure to control or eradicate HPAI . 23 NVRI: Real-time and Conventional; VTH: only conventional RT-PCR Test capacity: Egg inoculation, HA/HI, AGID ELISA PCR, RtPCR (from other sources) BSL3 lab, Sequencer? ? 2005 2006 2007 2008 2009 2010 2011 44 Candidate components: Intervention measures (depopulation, decontamination, disposal, movement control, awareness raising, vaccination, poultry restocking, etc); compensation. Outcomes: Disease outbreaks effectively contained and status recognised internationally. Under this outcome pillar, the following sub-components appear in the NAICP: ? Targeting virus elimination at the source (1a-i) ? Vaccination campaigns (1a-ii) ? Human capacity building (1e-iii) Key findings ? There is no clear evidence that the NAICP activities had an impact on HPAI incidence in 2006 and 2007. It might have had an impact in 2008, but it is difficult to ascertain whether the rapid decline in 2008 was due to project activities or natural outbreak decay. ? Many NAICP activities took effect only after the HPAI outbreaks of 2006-2008, including much of the training and supply of equipment and materials. Most of the equipment received by the states during the outbreak period came from other donors such as USAID. ? After the NAICP became active, there was an increase in the proportion of farms on which depopulation was carried out based on confirmed diagnostic tests rather than on disease suspicion. ? The depopulation rate, i.e. the proportion of birds culled before dying, increased over time, indicating a more timely response to outbreaks: this, perhaps, resulted in less spread of HPAI. Results With regard to targeting virus elimination at the source, i.e. responding to outbreaks, the activities proposed in the NAICP under this heading included (verbatim): stamping out of infected and at-risk poultry; compensation to farmers and producing companies at a reasonable market-oriented price; disposal of carcasses and potentially infective materials in a bio-secure and environmentally acceptable manner; enhanced bio-security at poultry farms and associated premises, through bio- containment and bio-exclusion; and control of movement of birds and products that may be infected, including controls at the interface of infected/non-infected areas and border controls. The main finding of this assessment is that there is no clear evidence that the project activities had an impact on HPAI incidence in 2006 and 2007. It is also uncertain as to whether the rapid decline in 2008 was due to project activities or natural outbreak decay. Evidence to support this includes data compiled by this study on the risk of a LGA being infected: in 2006 this was 6% (95% CI: 4 - 8%) which was not significantly different from that observed in 2007, 4% (3?6%). Similarly, the transmission rates of the disease between villages were not significantly different at 24% per day and 23% per day in 2006 and 2007, respectively. Also, some WB-funded activities support took effect only after the outbreaks, for example the supply of disinfectants. The supply of PPE, sampling materials and disinfectants derived from NAICP only occurred after the outbreaks. However the FGN, States and other donors (FAO, USAID, EU, South Korea, Israel and China) provided funds and in-kind donations which adequately fulfilled this need. NAICP officials 45 reported that it was not until 2010 that WB funds were used to purchase disinfectant: prior to that sufficient disinfectant from other sources was available. Based on the prevention and emergency plan against HPAI, the actions taken during outbreak responses in Nigeria involved the LGA veterinary officers and three different teams, the HPAI specialist diagnostic team, the infected premises (or response) team and the disease surveillance team. The infected HPAI premises teams were established in 2006 and usually consisted of a supervisor and around five officers. Their duties included: ? depopulation ? sample collection ? quarantine of suspected premises ? disease reporting The response team members who were surveyed for this study reported that they had all attended at least one training session on surveillance and response, with 6 out of 10 respondents having been trained by NAICP. Respondents assessed the training received as being applicable for their HPAI field work. The response teams reported that stamping out was done in the areas surrounding an outbreak during 2006 and 2007; this approach was, however, reported to have been later discontinued. The study found that the self-assessed performance of the response teams, who were responsible for culling infected poultry, disinfecting premises and safe disposal of carcases, improved over the period of the project, although this apparent improvement cannot be attributed solely to the project. In 2006, the teams reported that they lacked guidelines and that case definitions were used as the basis of disease diagnosis. By 2007 the situation had improved due to better laboratory performance, staff being more experienced, disinfectants being available and motorised sprayers being available to facilitate disinfection. Apart from HPAI, it was reported that these response teams also responded to other poultry diseases, including ND. The teams did not consider availability of PPEs to be a constraint to their performance; the main constraints teams reported were cooperation by farmers during field work and maintaining experience within the team. The observation that there has been a high turnover of staff is potentially risky as trained and experienced workers are then lost. There was a moderate increase in reporting of HPAI outbreaks by poultry farmers in 2007, probably due to an improvement in the compensation rates introduced in January 2007. The time between noticing outbreaks and reporting shortened from a median of 6.5 days (in 2006) to 1 day (in 2007). Of the 81 farmers interviewed, 34 indicated that they had HPAI outbreaks/suspicions on their farms: 74% claimed to have reported these cases to the authorities, including to state veterinarians (34%, n = 8), NAICP desk officers (30.4%, n = 7) and private vets (26%, n = 6), and also the NVRI and the teaching hospital in Nsukka. All the cases reported were said to have been followed up: premises were reported by farmers to have been disinfected in 80% of cases and the response teams were reported to have worn PPE in more than 86% of cases. Biosecurity measures that farmers reported they were asked to implement by the response teams following an outbreak included: burying carcasses, restricting entry of visitors, cleaning and disinfecting poultry pens and equipment, washing hands with disinfectants after touching poultry, and delaying restocking. The proportion of farms on which depopulation was carried out based on confirmed diagnostic tests rather than suspicion improved after the project took over, reflecting a decrease in turn-around times of samples submitted to the labs. This in turn is ascribed to more efficient systems for 46 transporting samples and more efficient logistics in laboratories to prepare samples for diagnosis rather than the introduction of faster diagnostic tests, which only occurred later. The proportion of outbreaks for which depopulation was carried out without prior laboratory diagnosis was 27% in first half 2006, 25% in the second half of 2006, 2% in the first half of 2007 and 7% in the second half of 2007. The depopulation rate, i.e. the proportion of birds culled before they died, also increased over time, indicating a more timely response to outbreaks: this, perhaps, resulted in less spread of HPAI. All farms with more than 200 birds were required to register with the FMoARD. The number of farms registered increased slightly in 2007 as compared to 2006 but this declined in 2008-924. Overall, the study found that 68% of eligible poultry keepers registered, and that there was no difference in registration proportions between those who had received training on biosecurity measures and those that had not. A wide range of activities were undertaken under the project to strengthen biosecurity measures by different actors: these are dealt with under the section 6.2.6 (HPAI prevention). Activities undertaken to strengthen quarantine capacity at state and international borders is also dealt with under section 6.2.6 (HPAI prevention). Compensation is dealt with in detail under the section 6.2.7 (poultry sector recovery). Vaccination of poultry was initially included in the list of activities to be supported by NAICP. Project officials reported that there was intense pressure on the FGN from certain quarters, including the FAO, to introduce a policy of vaccination of poultry, and the EU provided US$ 5 million to fund this activity. After considering the issue for more than a year, the FGN decided not to vaccinate, partly because it was realised that Nigeria lacked the capacity to implement a large-scale vaccination programme, especially in rural areas, and partly because of the lack of an effective exit strategy. Key informant interviews with private producers (Poultry Association of Nigeria) revealed that fewer than 30% of commercial farms vaccinated their flocks with illegally imported vaccines. This is unlikely to have influenced the epidemiology of the disease given the uncoordinated manner in which this was done. 6.2.5 Pillar 5. Epidemiological Capacity for Strategy Development and Intervention Targeting Objectives: Collect, synthesise and analyse data on the dynamics and impacts of HPAI, and use the outputs to inform policy and strategy for HPAI control. Candidate components: Epidemiological data handling, processing and analysis; data flow and communication ; data reporting, use and presentation; poultry population demography; outbreak investigation; value chain studies; risk assessment (based on critical control points); socioeconomic impact. Outcomes: Quality data received and disease control strategy regularly updated through a sound evidence base. Under this outcome pillar, the following sub-component appears in the NAICP: ? Strengthening epidemiological analysis (1b-ii) 24 In the study carried out during the evaluation, reasons given for failure to register by farmers included: lack of confidence in the government?s plan to support farmers; small number of birds kept; and lack of assistance during the registration process. 47 Key findings ? The mode of introduction of HPAI to Nigeria has not yet been established. ? Epidemiological capacity was in place at the FGN level before the outbreak of HPAI, resulting from the PACE programme set up for rinderpest control and eradication. The NAICP built on this capacity and the pre-existing NADIS was used for HPAI. ? The relative absence of data on poultry populations at risk and the dynamics of poultry enterprises constrained some of the epidemiological analyses emerging. ? The multiple units for disease reporting on a spatial, temporal and production system basis negatively impacted the quality of emerging epidemiological syntheses. ? Insufficient exploitation of risk-based approaches, including the greater use of value chain descriptive and analytical techniques, constrained the feedback of evidence into national HPAI strategy development. Results Under the earlier PACE programme for the control of rinderpest and other epidemic diseases, Nigeria had developed a system for animal disease surveillance which it used as a base for the development of HPAI surveillance. It was originally developed using the (now defunct) ARIS system of the AU-IBAR. A dialogue was established with the FAO?s TADinfo system, but Nigeria eventually decided to develop its own information system. The National Animal Disease Information & Surveillance (NADIS) network was put in place, with 170 surveillance points initially established and manned by trained surveillance agents who had to identify specific diseases, collect samples and take control measures (this number has been increased to 295, with plans to expand to 600). Through two EC-funded projects in the biennium 2006-2007, FAO contributed to this network by providing technical and capacity development assistance. In addition, the PACE programme conducted workshops and training activities targeted at national veterinarians and livestock farmers. During the Second Real Time Evaluation of FAO?s global programmes on HPAI25, a case study of Nigeria was carried out (in late 2009). The report commented: ?Surprisingly there is little refinement and use of epidemiological data to provide a greater understanding of risk of infection, and the use of such analysis to feed into risk-based surveillance and risk-based strategic response mechanisms, given the limited resources available. There is still no official statement on the source of introduction of HPAI into Nigeria. Part of the dissection of risk is the understanding of market value chains. There has been some strategic studies of value chains in Nigeria26, but little use appears to have been made of this very broad level consultancy study in building up a risk framework?. This evaluation concludes that while there is considerable epidemiological capacity in NADIS, and it has been a valuable synthesiser of data on HPAI, analytical capacity compatible with the needs of Nigeria still falls short and this has not been fully addressed by the project. The evaluation team synthesised secondary data derived from the project and state Desk Officers27, and experienced data quality and standardisation issues which might be reflective of broader 25 http://www.fao.org/pbe/pbee/common/ecg/388/en/HPAI.zip 26 Pagani, P., Abimiku, J.E.Y., Emeka-Okoli, W. 2008. Assessment of Nigerian Poultry Market Chain to Improve Biosecurity. FAO, Rome, 58 pp. 27 There was no HPAI outbreak database in the project that could be readily accessible for an epidemiological outbreak analysis. However, through the NAICP project the secondary data on outbreak occurrence was requested by the evaluation team from the state Desk Officers. 48 inadequacies in NADIS. These relate to: a) standardisation of the unit of reporting (whether household, village, LGA); and b) lack of good denominator data on poultry (or indeed other livestock species) populations. These inadequacies are not exclusive to Nigeria, but do offer an opportunity for greater harmonisation of data units for the future. In addition to denominator data relating to poultry producers, many other actors were affected by HPAI, and indeed had the potential to be involved in disease transmission. It is therefore of value to consider HPAI dynamics at a broader value chain level, but it is uncertain to what degree the epidemiology team in NADIS used value chain understanding in risk assessment and/or active surveillance. In Nigeria a detailed value chain study was undertaken under the auspices of the Pro- Poor Risk Reduction in Africa and Asia project, funded by DFID (Akinwumi et al., 200928). The study concluded that disease transmission pathways are linked to economic incentives faced by chain actors, risks of disease transmission are strongly related to commercial practice, and that consumer sovereignty is insufficient to influence governance and commercial practice in Nigeria. Finally, it concluded that chain actors face economic incentives to conceal information that is essential for effective HPAI control. The evaluation team considers that this study is most valuable, both from the results obtained but also as a methodological approach with broader application. It is uncertain if the results are being used effectively by the NAICP. PDS has been a more recent tool targeted at securing information from grass root sources on livestock and disease dynamics. It appears to have been successful in improving understanding between veterinarians and communities and in better directing resources. However, from an epidemiology capacity point of view, PDS data is not effectively integrated into disease reporting in Nigeria; PDS data exists in a database separate from the conventional NADIS network and is not analyzed. There is little evidence for the data leading to actions, decisions or changes. It is concluded that there opportunities to increase the usefulness of PDS exist: a) institutionalizing analysis of information by and with communities; b) developing an interface through which PDS- derived data can feed into NADIS. The NAICP project sponsored two MSc students29: one on ASF, one on HPAI. These are now both back in the system, with one working in NADIS. 6.2.6 Pillar 6. HPAI Prevention Objective: Put in place technically sound affordable and socially viable measures to minimise the risk of HPAI spread, and reduce the risk of human infection. Candidate components: Biosecurity, communication, human protection, vaccination, farm/unit registration, market and slaughter practices; industry support and restructuring. Outcome: Progressive reduction in disease incidence which is independently verifiable. No new human cases. 28 Akinwumi, J., Okike, Iheanacho, Bett, B., Randolph, T., Rich, K.M. 2009. Analyses of the poultry value chain and its linkages and interactions with HPAI risk factors in Nigeria. Africa Indonesia Team Working Paper, in press. 29 Vakuru Columbia Teru. Thesis title: Situation-based survey of avian influenza viruses in possible bridge species of wild and domestic birds in Nigeria. Master of Science in Tropical Animal Health, Institute of Tropical Medicine (ITM), Antwerp, Belgium September 2009 ? July 2010. Ezenwa Nwakobi. Thesis title: Development of risk scoring tools for African swine fever (ASF) in Nigeria. Master of Science in Veterinary Epidemiology from the Royal Veterinary College, London, September 2009 ? September 2010. 49 Under this evaluation pillar, the following sub-components and sub-sub components appear in the NAICP: ? Vaccination campaigns (1a-ii) ? Biosafety for at-risk stakeholders (1a-iii) ? Strengthening of Veterinary Quarantine Services (1c-I and 1c-ii) ? Improving bio-security in poultry production and trade (1f) Key findings ? Despite intense external pressure, FGN decided not to vaccinate poultry as part of the control programme. This was due in part to a perceived lack of capacity to implement a large-scale vaccination programmes, especially in rural areas, or to develop an appropriate exit strategy. ? Most training of farmers and poultry traders was undertaken by the project after the HPAI outbreaks. ? Training of farmers marginally improved bio-security practices of poultry keeping, but was not considered sufficient to reduce the risk of HPAI spread ? Investments made in the pilot improvement of a small number of LBMs had positive food safety implications. However inevitably the small of LBMs improved had no effect on biosecurity at the national level, but it has been a valuable demonstration project. ? Improvements in biosecurity at live birds markets was more likely to have been due to infrastructure improvements than to training: risky behaviours, such as selling sick birds, were not influenced by training. ? Despite improvements in equipment, infrastructure and staff capacity in pilot, strategically- selected quarantine stations through project support, LGAs still use these facilities primarily for revenue generation rather than for disease control. ? The financial analysis of the LBM investment indicated that except in year 1, during which heavy investment was made, the revenue from the LBM was consistently higher than the total cost, hence a positive net incremental benefit. Results Vaccination of poultry was initially included in the list of activities to be supported by the WB-funded project at the recommendation of FAO/AGAH and the EU. NAICP officials reported that there was intense pressure on the FGN from FAO and others to introduce a policy of vaccination of poultry, and the EU provided US$ 5 million to fund this activity. After considering the issue for more than a year, however, the FGN decided not to vaccinate, partly at least because it was realised that Nigeria lacked the capacity to implement a large-scale vaccination programme, especially in rural areas. Biosafety training was targeted at poultry farmers and traders. Of the farmers surveyed for this study, 62% were men and they kept a median of 1150 birds: 83% kept layers only, 14% kept broilers only, and just 2% kept both. Three-quarters considered that poultry farming was their main income source. Thirteen percent of the sample reported that they had used HPAI vaccines, which is contrary to the official policy in Nigeria. Sixty percent had attended at least one biosecurity training session: 36% were able to identify NAICP as the source of at least one training session. The most commonly recalled lessons learned were the need for fencing, construction of footbaths and the need to 50 restrict visitors to their farms. However, most training of farmers and traders30 on improved biosecurity, and quarantine station officers on inspection and certification of agricultural products undertaken by the project occurred long after the last outbreak occurred. In addition to formal training, farmers also reported they obtained information on HPAI from a range of media: radio, television and newspapers, in that order, were the main sources of such information. The messages that the farmers listed matched those that NAICP and other agencies such as UNICEF disseminated through the mass media; this indicates that the messages were properly assimilated. The study found that although training did marginally improve bio-security practices of poultry keeping, this was not considered sufficient to reduce the risk of HPAI spread. Training was also targeted at poultry traders. For this study 52 fowl sellers from 4 states were interviewed: 70% were male, most were 31-40 years of age and they all reported that poultry trade was their main source of income. The traders were based in both traditional markets and those that had been upgraded under the project. Half of the respondents had attended at least one biosecurity training event and most of these identified NAICP as the trainer. Eighty six per cent of those who had been trained had done so in 2008 or later. Key lessons learned included: disinfection of cages, washing of hands, avoiding mixing poultry, fumigation of live bird markets, separating sick and healthy birds, use of plastic cages and clinical signs of disease. However, fowl sellers reported they were unable to adopt improved plastic or metal cages as they were too expensive, and hand washing was also difficult. More than 80% of both trained and untrained fowl sellers indicated that they had had sick poultry in their consignments; 70% of trained and 90% of untrained fowl sellers experienced some mortalities, but only 19% of trained and 33% of untrained fowl sellers reported these to the relevant authorities. Up to half of the traders reported that they slaughtered sick and/or cooked dead birds. Although it was not originally planned as a project activity, and was introduced during the project midterm review, as project staff started to work with live bird market operators they realised there was an opportunity to engage with this group. Previously the only contact they had had with government agencies was for tax collection. A proposal was therefore made to the WB for an LBM upgrading programme based on a 50% matching grant. NAICP officers regarded this as a very positive development. The biosecurity measures practiced at the improved LBMs was evaluated for this study through questionnaires and inspection. This showed that the improved markets were better than traditional markets with regard to a range of practices including: record keeping, segregation of new consignments and keeping different poultry species separately, use of metal cages, washing and disinfection and veterinary inspection. Biosecurity practices associated with slaughter and transport of poultry was also better in the improved markets. Live bird market investment The enhanced biosecurity through infrastructural improvements to selected pilot LBMs was undoubtedly effective, and independent evidence of this was generated in the evaluation. However the upgrading came after the HPAI outbreaks (see the Figure 11 below), and the number of LBMs 30 Some traders indicated that they were trained by the project in 2007 and 2008 51 upgraded was inevitably insignificant in relation to the total number of LBMs in Nigeria. Nevertheless this does provide a very constructive and positive case study which can be built on in the future. Live bird markets: the economics of the investment The NAICP has piloted 4 upgraded live-bird markets (LBM) as a means to improve biosecurity and reduce the risk of HPAI transmission. The intervention involves improving physical infrastructure and operation of the market, encouraging better practices and changing the business structure for how the market is managed. The project introduced the use of disinfectants and application bio-security by LBM operatives and enforced their use through the LBM elected management by incorporating application of disinfectants and bio-security measures into the by-laws of the LBM management. The major objective of this case study is to analyze the financial sustainability of the pilot improved live- bird market. The data employed in the analysis were cross sectional data obtained from a set of single-visit interviews of LBM traders conducted by ILRI-recruited enumerators for this purpose using cluster sample rapid appraisal survey technique. The Kaduna LBM was selected as a typical pilot improved LBM to study. The prices were constant 2010 prices and the data on live-bird sales, incremental investment, and operating costs were collected following the double difference (?before and after? and ?with and without?) project intervention method. Incremental costs and benefits for the live- bird traders were estimated on annual basis and projected for 20 years. All of the usual caveats apply to the quality of the data given the difficulties for market agents to maintain financial records or recall historical (pre-project) data and the natural tendency of operators to be protective of or unclear about their financial performance. There is also a natural tendency to affirmation bias with participants being optimistic in their expectations about future financial performance, and wanting to feed back positive results to continue attracting external support and subsidies to their business operations. A full account of this assessment is presented in Annex 7. The estimated internal rate of return (IRR) represents the rate of return on the capital invested in the project for the LBM. It is estimated at 34%. The estimated Net present Value (NPV) at an opportunity cost of capital of 12% was US$ 241,417 and the estimated annual incremental benefit was US$ 73,876. These figures indicate that after all costs have been met the LBM sub-project generates a total incremental return of US$ 241,417 given an opportunity cost of capital of 12 percent. A positive NPV makes a project attractive for implementation and the larger the NPV the Figure 11. Timing of live-bird market upgrades, in the contexts of the HPAI outbreaks 52 better. Conventionally, a project is qualified for implementation if the yield rate is equal or greater than the prevailing lending rate or opportunity cost of capital (considered 12%). A sensitivity analysis was conducted and it indicates that the estimated economic ratios are stable under a variety of scenarios in sales revenue and costs. If revenue falls by 10 percent, the IRR will decrease to 18 percent and the NPV will fall by 79%. The switching value for revenue is a fall in revenue of 13 percent. If total cost increases by 10 percent, the IRR will decrease to 20% and the NPV will fall by 69%. The switching value for total costs is an increase in costs by 12%. The financial analysis of the LBM investment indicated that except in year 1, during which heavy investment was made, the revenue from the LBM was consistently higher than the total cost, hence a positive net incremental benefit. That combined with an IRR greater than the opportunity cost of capital indicate that the improved LBM is a profitable business. Quarantine stations The National Agricultural Quarantine Service is tasked with managing movement of animal and their products between states and neighbouring countries. This is widely acknowledged to be a very difficult job: borders are porous, movement of people with livestock the norm and compliance with and enforcement of rules and regulations is weak. Under the project training was provided to quarantine officers on inspection and certification of animals and their products, and investment was made in upgrading facilities. NAICP staff reported that previously quarantine staff had received no training in animal diseases, and training was identified as a key need in a study carried out by the project. Although all surveyed quarantine officers had received training, training carried out by NAICP was not carried out until mid 2009 and late 2010, i.e. after the last disease outbreak. Key informants from the NAICP considered that the primary purpose of quarantine posts was to collect revenue on behalf of local government authorities and state governments: the original purpose, of disease control, has largely been lost. Although there has been an improvement in equipment, infrastructure and staff capacity in the quarantine stations, local government councils were using these facilities for revenue generation instead of using them for disease control due to lack of cooperation from the livestock traders. A similar observation had been made much earlier by Ogundipe (2002)31 who rated the efficiency of the quarantine stations as being high for revenue collection, medium for animal movement monitoring and low for disease monitoring. The number of upgraded quarantine stations is minute compared to the total number of quarantine stations in Nigeria. The overall impact of the quarantine station improvement is considered to be minimal. 6.2.7 Pillar 7. Poultry Sector Recovery Objective: To minimise the impact of HPAI on the credibility and economic viability of the poultry industry. Candidate components: Public relations of risk avoidance with poultry and poultry products; viable compensation scheme; dialogue with and support to all sections of poultry industry. Outcome: Affected poultry farms back in business, compliance with FGN policies for HPAI control and prevention is enhanced. 31 Ogundipe, G. A. T., 2002. Nigerian Veterinary Journal, 23(1); 1 ? 15. 53 The following three activities were planned under this heading for implementation through the NAICP: ? Establishment of a national compensation policy and national compensation fund (1g-i) ? Support to economically vulnerable groups (1g-ii) ? Alternative livelihoods for seriously affected stakeholders (1g-iii) Key findings ? NAICP had no capacity to implement the planned project sub-component to support vulnerable groups based on disbursement and recovery of loans, so these were not implemented. ? The transition to fairer market rates for compensation payments is considered by NAICP to be an important factor in improving the time to reporting outbreaks by farmers. ? The time interval between depopulation and payment of compensation improved during the course of the project but was still very significantly longer than the target: NAICP officials considered the target of 21 days to be wholly unrealistic. ? There is no evidence that there has been a long-term decline in chicken consumption associated with HPAI: this suggests that the message that consumption of well-cooked chicken was safe had been well received by the target audiences. ? There is no evidence of a reduction in poultry keeping associated with HPAI. Most HPAI affected poultry farms were able to resume business. ? Except for breeders and toll millers, all the value chain operators reported suffering severe losses in terms of substantially reduced gross margins during the HPAI outbreaks (2006-2008), especially among those businesses that did not participate directly in the NAICP actions ? Value chain operators report that to date, many have not been able to recover to their pre-HPAI performance, especially among layer producers, processors and traders. Contradicting this perception, however, actors in the value chain also report seeing growth in the numbers in each actor category and in business volumes. Results Only the first of the three planned activities was implemented under the NAICP, that of establishing a national compensation policy and compensation fund. To support economically vulnerable groups, it was planned to provide support to low-income stakeholders, including smallholder poultry farmers. Planned activities were designed to increase awareness amongst this group, improve animal health services at the community level, provide grants for additional compensation and enable livelihood diversification for vulnerable poultry stakeholders who had been severely affected by HPAI. To support adoption of alternative livelihoods it had been planned to enable smallholder poultry farmers who had been affected by HPAI to either resume poultry farming or take up alternative livelihood options. For the former, it was planned to provide advice on restocking and for the latter it was planned to develop a credit mechanism, through which eligible stakeholders could be supported to take up alternative livelihood enterprises, based on their proposals which had been approved by the Compensation Fund. However, the NAICP rapidly came to the conclusion that it had no capacity to disburse or recover the loans that were central to these plans. It also failed to find a suitable partner who could undertake these activities on its behalf: the main Nigerian co-operative 54 organisation ? the Nigerian Agricultural and Cooperative Bank (NACB) - was undergoing a reform process at the time and was, therefore, not available to take on this role. This pillar, therefore, focussed entirely on the compensation scheme. The compensation scheme Immediately after the first case of H5N1 had been confirmed on 8 February 2006, the FGN announced a compensation scheme. Government regulations subsequently required that farms32 needed to be registered to be eligible to receive compensation. Registration efforts by farmers were often poor for various reasons (see farmer bio-security survey results); the registration requirement was not always enforced, and some farmers reportedly received compensation although they were not registered. Compensation was initially based on fixed rates for different categories of bird (see Table 6, below) and there was no consultation with poultry farmers prior to the scheme being launched. The HPAI compensation plan was reportedly the first country-wide compensation scheme to be implemented in Nigeria. Table 6: Initial and revised rates of compensation Species Initial compensation (NGN) *Range of Revised Compensation (NGN) Chickens (commercial) 250 350 to 1,500 Eggs (commercial) 15 Chickens (free-ranging, rural) 250 100 to 750 Guinea fowl 250 100 to 500 Pigeons (fully grown) 250 250 Ducks and geese 1,000 100 to 700 Turkeys (local) 2,500 300 to 1,600 Emus 10,000 Ostriches 20,000 15,000 to 100,000 Ostrich eggs 4,000 *Rates dependent on rate of growth status of the bird. Source: NAICP, 2007 In July 2006 the FMoARD submitted its compensation manual to the WB and invited experts to refine the fixed rate scheme to better reflect market rates. In August 2006 the FGN reported its compensation funds had been exhausted and that a backlog of claims was building up: transition to a WB-funded compensation scheme was proposed33. In January 2007 a no objection to use WB compensation funds was obtained. The first priority of the WB-funded scheme was to clear the back- 32 Farms with 200 birds or more were eligible for registration. 33 Minutes of Technical Steering Committee meeting of 23rd November 2006 55 log of claims that had accumulated under the fixed rate scheme before the new rates were introduced. This study obtained secondary data on the compensation scheme, including the timing of payments and the amount paid, and also had access to the compensation manual. Information about farmers? experiences of compensation was included in a retrospective farm level questionnaire, but the number of respondents who had been affected by HPAI outbreaks was too small to provide useful information. Additional perceptions about the compensation scheme were obtained through key informant interviews and focal group discussions. The median amount paid for compensation was higher in 2007 than in 2006, which probably reflects the transition from the fixed rate scheme to one better reflecting market rates: median compensation paid per bird (all species) in 2006 was NGN 250 compared to NGN 362 in 2007. The transition to market rates is considered by NAICP to be a factor in improving the time to reporting outbreaks by farmers: this time dropped from a median value of 4 days for the last 6 months of 2006, to 1 day for the first 6 months of 2007. The decrease in reporting time by farmers actually began before the market rates were introduced: this is ascribed by NAICP to the earlier publication (June 2006) of the project?s compensation manual which included the intention to make this change. The target time (between destocking and payment) specified in the compensation manual was set at 21 days. The actual lag in payment was much greater; this was partly due to the fact that compensation payments were not conducted after each individual farm outbreak, but within ?blocks? of a few weeks at several times per year. In 2006 the median delay was 212 days. This increased in January 2007 to 242 days, and then reduced to around 150 days for the period February to December 2007. In January 2007 the compensation scheme switched from FGN to WB funding: the spike in the interval between destocking and payment in January can probably be explained by the initial priority at this time of clearing the backlog of old claims that accumulated from when the FFGN funds became exhausted in August 2006. From the outset it was reported that the international community was very sceptical about Nigeria?s capacity to implement a transparent compensation programme. To counter this negative perception, efforts were made to increase transparency: for example, details of pending compensation payments were posted on the NAICP website prior to payment and appeared in some daily newspapers to enable any interested parties to examine these and report any potential irregularities. Some details of all compensation payments made are still available on the NAICP website, including gender of the beneficiary. Limited focal group discussions with poultry industry representatives revealed mixed perceptions of the compensation scheme. While some respondents considered that the compensation scheme had alleviated the impact of HPAI on poultry farmers and had contributed to the revival of the sector, others considered that the amounts paid were too low to enable restocking. Representatives of the broiler industry reported that many farmers, especially in Oyo State, did not receive any compensation. One NAICP official observed: ?no matter how well you do it, perception will remain low - even if you are as clear as water.? 56 In March 2011, NAICP reported that of the US$9.62 million made available via the WB for compensation payments, just US$ 5.4 had been required to compensate 3037 affected farms/farmers representing 1.26 million culled birds. Public confidence in consuming poultry Early public health messages issued by the FMoH suggested that the general public should not eat chicken or eggs. Not surprisingly, the poultry industry was unhappy with this message and protested, including to the President. This resulted in the messages being reconsidered to provide a more balanced perspective of risk and appropriate mitigating measures. Data indicates that there has been no long-term decline in chicken consumption associated with HPAI, which suggests that the revised messages, that consumption of well-cooked chicken was safe, had been well received. Also, there is no evidence of a long-term reduction in poultry keeping associated with HPAI. Performance study of the poultry value chain A sectoral study was conducted by the evaluation team to assess the extent to which market agents have been impacted by the intervention (for full report see Annex 6). It follows a value chain framework to understand how the project has impacted the poultry value chain in terms of biosecurity adoption, governance, cost structure, and performance. The findings indicate that, except for the breeders and toll millers, all the value chain operatives reported suffering severe profit losses during the HPAI outbreak period (2006-8) and in some cases led to untimely sale of assets including land and buildings to pay bank loans, indebtedness, reduced patronage, reduction in the prices of day-old-chicks and loss of birds due to overpopulation, lack of market, and drastic reduction in income. Recovery adjustment after the AI period was not easy for the value chain operatives. Respondents reported that to date, not all the value chain operators have been able to attain their pre-HPAI performance, particularly among layer operators, processors and traders. On the other hand, breeders, broiler producers, transporters and toll millers report better profitability than pre-HPAI, and the perception among breeders/hatcheries, broiler and layer operators, processors and feed and toll millers is that their numbers and business volume has been increasing since the outbreak of HPAI in 2006. Biosecurity practices has intensified in the last five years among the more commercially-oriented actors, which value chain operatives, by and large, attribute to the effort to control HPAI. Although most of the operatives expressed their willingness to implement improved biosecurity despite their burden on operating cost, the added cost can be expected to hamper widespread adoption. No significant change in business structure or practices due to the HPAI control efforts was identified, other than the small number of improved LBM that are being piloted. Contracting arrangements, for example, have not appeared in response to perceived demand for biosecurity. Nor has the experience with HPAI and its control appear to have spawned new types of collective action to coordinate biosecurity, encourage compliance, or offer cost sharing, training and exchange of ideas. Government regulations have played a significant role in improved cleanliness and hygiene in improved markets, but less so elsewhere. Group cohesion and management among the value chain operatives has changed little, except among traders operating in the improved LBM. While some of 57 the other operatives continue to belong to larger organizations such as the national poultry producers of Nigeria, collective action at the local level was not widespread. 6.2.8 Pillar 8. Public Health Pandemic Preparedness Objectives: The development of a clear and technically sound policy for HPAI preparedness and control, effectively communicated to all stakeholders. Candidate components: Legal framework, national policy, national strategy, contingency planning, research prioritisation; identification, engagement and communication with all stakeholders; capacity-building for physicians; provision of diagnostic and treatment facilities; provision of PPEs, drugs and other supplies for AI control. Outcomes: Sound policy & strategy in place and effectively communicated; All key human and animal health stakeholders involved. Under this outcome pillar, the following sub-components appear in the NAICP: ? Enhancing Public Health Program Planning, Delivery and Coordination (2a) ? Strengthening of National Public Health Surveillance Systems (2b) Key findings ? The Integrated National Avian and Pandemic Influenza Response Plan developed (for the 2007 ? 2009) was delayed in its completion but the contingency plan was effectively used in the interim ? There was general satisfaction with the planning and preparedness activities conducted by the NAICP amongst policy makers, experts and state officials. However, since an avian influenza pandemic did not occur and zoonotic H5N1 remained very rare, NAICP did not have an opportunity to demonstrate impact on control of HPAI; HPAI is not now perceived to be a critical health problem in Nigeria. ? The training and provision of equipment and supplies is reported to have had positive impacts in the control of H1N1 ? Physicians who had received training tended to have better knowledge of avian influenza symptoms, case management and notification than non-trainees and are significantly more likely than non-trainees to be willing to undertake management of human AI cases ? Avian influenza surveillance and control capabilities are currently active in the majority of states. Results A detailed account of how the INAPIRP was developed is provided in pillar 1, HPAI Control and Pandemic Preparedness, together with details of the medical waste management plan. Policy makers and experts Policymakers and experts surveyed assigned an overall mean score of 82% (range 67-92%) for importance/relevance and/or appropriateness of project activities. There was general satisfaction with the project but a perception that, as the avian influenza pandemic did not occur, HPAI was not a critical health problem in Nigeria. A minority of respondents felt that the appointment of Human Health Desk Officers amounted to creating a parallel structure within the FMoH and this resulted in 58 conflicts with state epidemiologists. The evaluation team, however, considers that the mechanism was appropriate given the state of knowledge and concern at the time of project inception. The evaluation team questioned human health officials as to how well the INAPIRP document was distributed, and to what extent it was used. In a survey undertaken as part of this study, 77% of officials surveyed at federal level had a copy; at the state level, 35% of officials had copies; while 94% of desk officers had copies. Eighty nine per cent of respondents at the state level considered that the plan was useful. State officers (epidemiology & surveillance) Avian influenza surveillance and control capabilities are currently present in the majority of states. This study showed that there is general improvement in surveillance consistent with the positive impact of project training and inputs as witnessed by increased reporting of notifiable disease; an improvement in the submission of reports from LGAs; and a trend towards an increase in diseases investigated by states. Project desk officers Most officers are able to demonstrate that they carry out AI surveillance/control related activities. Officers reported that they benefited from project-supported training and reported having cascaded training to other personnel: all the desk officers consulted for this study had attended at least one NAICP-supported training on case management, topics including disease surveillance, pandemic preparedness and response, outbreak investigations, emergency and risk communications, strategic communication and planning, and medical waste management. All the desk officers consulted rated the various trainings as very useful. Desk officers reported they were able to carry out training at the LGA level. Other impacts reported included improved collaboration with the animal health and communications component coordinators as a result of training. Most officers find that the trainings are still useful in their current designation and offices. They are applying the training they received to communications, response to other disease outbreaks in the state, proper identification and management of influenza-like illness, propagation of government policies and programmes, implementation of Integrated Disease Surveillance and Response (IDSR), etc. Desk officers reported that NAICP-provided materials had been used for influenza control. For example, 11 (68.75%) of the desk officers in 7 of the 8 states indicated that NAICP provided them with equipments and supplies including computer and accessories, PPEs, communication materials (t-shirts, posters, pamphlets), Tamiflu?, photocopying machines and refrigerators. Most of the equipments and supplies provided by the project were still being used at the time of this study (first quarter 2011). Tamiflu? was used as a prophylactic during outbreaks and PPEs were used to protect field officers during outbreak investigations and used in the hospital to protect health workers. After the HPAI outbreaks, much of the equipment continues to be of use as NAICP-provided materials are reported to be used for swine flu control. PPEs provided by the project were used by States in H1N1 (swine flu) investigations: for example, Lagos State Rapid Response team used the PPEs and Tamiflu? provided by the project used during the swine flu investigations. Structures for avian flu surveillance and diagnostics were used for detection and surveillance of swine flu. 59 Table 7. Amount of Tamiflu? available in the states surveyed State Tamiflu? Given to States Quantity Left at State Estimated Doses used* Enugu 1000 1000 0 Jigawa 1000 300 700 Lagos 1000 300 700 Nasarawa 1000 750 250 Oyo 1000 750 250 Plateau 1000 300 700 State officials State officials were generally satisfied with the Project, although HPAI is no longer regarded as a key public health issue. 77% of respondents indicated that the ministry had a copy of the Integrated National Avian and Pandemic Influenza Response Plan but only 35.5% indicated that the State still has a copy. Overall a mean rating score of 78% suggests that the plan was perceived to be useful to the states in their AI response, especially for surveillance, outbreak response and containment, and social mobilization. The appointment of desk officers was considered to be a very relevant strategy. 6.2.9 Pillar 9. Human Influenza Surveillance Objective: Infection status in human populations known and internationally recognised. Candidate components: Laboratory capacity development; surveillance networks established; routine surveillance of influenza-like illness established; active and risk-targeted surveillance. Outcomes: Infection status in humans known and internationally recognised. Under this evaluation pillar, the following sub-components appear in the NAICP: ? Strengthening of National Public Health Surveillance Systems (2b) Key findings ? Highly pathogenic avian influenza has been made a notifiable disease in Nigeria. ? One in ten physicians reported having encountered a suspect AI case; only 37% of physicians reported that they had ever reported a notifiable disease; this suggests further improvements to surveillance and notification are needed ? Many cases of fever and respiratory signs combined with a history of contact with poultry occur; most are not regarded as suspicious for avian influenza. This suggests that further improvements are needed for influenza surveillance ? However, given the high numbers of people with influenza-like illness and contact with sick poultry, only a very small proportion of potentially suspect cases are covered by surveillance and this may not be sufficient to detect AI circulating at low levels ? There is now reportedly good collaboration between the human and animal surveillance teams34. Laboratory detection / confirmation of the first human H5-case in a deceased person in 2007 was carried out through teamwork by both animal and human health laboratory experts 34 The sustainability is understandably a challenge. A desk officer from one state said that their human health counterparts started missing meetings as from 2008 when no more active cases were being reported. 60 ? NAICP inputs contributed to the diagnosis of the first case of Pandemic H1N1/2009 influenza in Nigeria in 2009 ? Since 2009 the Human Health laboratories have been capable of effective and simultaneous diagnosis of Influenza A/B, H5 and H1N1/2009 Pandemic (swine flu), as well as the seasonal human H1, H3 strains by molecular techniques. A sequencer is in use at the WHO-Influenza Reference laboratory in UCH Ibadan for HIV-strain characterisation and will be augmented to phylogenetic characterisation of influenza viruses once primers are available. ? The level of expertise among staff is high; most of the training and support was provided by CDC and WHO. ? NIRL in Abuja conducts real-time RT-PCR (since 2009; machine provided by NAICP) solely for the discrimination of Influenza A/B, and detection of subtypes H5, Pandemic/H1/2009, seasonal H1, H3. All positive samples are sent to CDC Atlanta for confirmation. All reagents for real-time RT- PCR are obtained through CDC, Atlanta. ? NIRL in Abuja receives approximately 20-40 samples per working day that are immediately processed. The number of staff has increased as a consequence of enforced surveillance activities for H1N1/Pandemic/2009. ? The UCH Ibadan houses a WHO ? Influenza Reference laboratory (since 1974). Although it has the mandate to provide testing for Influenza A for Nigeria and the West African region, and staff is of a high level of expertise, not many activities were carried out due to delays in receipt of equipment, finalization of a new building, lack of reagents and inadequate sample submission. ? Moreover, surveillance seems not to be risk-based and may not be able to detect or quickly respond to changes in risk. Results The overall strategy was to strengthen public health surveillance systems in ways that were sustainable and that would contribute towards improved surveillance of other notifiable diseases besides HPAI. To enhance sustainability laboratory facilities were established in teaching hospitals. The main activities undertaken under the project were upgrading a network of diagnostic labs, introducing improved diagnostic methods, training personnel, construction of new laboratory buildings and the provision of equipment and materials, including vehicles and PPEs. There is a network of Human Health laboratories. One laboratory has been newly established in Abuja and designated to act as the National Influenza Reference Laboratory (NIRL). Since 2008/9 most samples originate from the National Influenza Sentinel Surveillance (NISS). Eight other laboratories act as satellite labs of which four have been upgraded and new laboratory buildings constructed. The UCH Ibadan, University of Port Harcourt Teaching Hospital (UPTH), University of Maiduguri (UMTH) and NIRL Abuja have been approved for constructions of new laboratories. All eight laboratories work in a network for sentinel sites (ILI- symptoms). 61 Figure 12: Distribution of Satellite Laboratories of the Human Health Site. Blue: NIRL, Abuja; Yellow: National WHO Influenza Reference Centre, Ibadan; Red: Teaching Hospitals in Port Harcourt, Lagos, Maiduguri, Kano, Enugu and Sokoto. In 2008 the building of the current NIRL has been upgraded by financial support of CDC and FMoH and was commissioned in 2009 by the US ambassador and the FMoH. Equipment has been installed and reagents provided by CDC for molecular biological diagnosis of Influenza A or B and subtyping H5, Pandemic/H1/2009 and seasonal H1/H3 exclusively by real-time RT-PCR. A separate building planned to eventually serve as a BSL 3 facility has been constructed, but it is not yet equipped (waiting for equipment to arrive as a contribution by NAICP). This new building is planned for serology, conventional PCR, virus isolation (cell-culture), and expected to be functional earliest at the end of 2011. However, the NIRL is planning to move to the Nigerian Centre of Disease Control (NCDC) by end of 2011. It is unclear what will happen to the new building. The NCDC is outside of a residential area, houses a BSL-3 lab and is involved in diagnosis of yellow fever, malaria, Lassa fever and other zoonotic diseases. It is planned that staff and equipment are going to move there in the near future. Joint actions/trainings of NAICP, FMoH and other development partners (such as CDC and WHO) have improved the technical and organisational abilities related to detect H5, Pandemic H1/2009 and seasonal human influenza strains and have contributed to interdisciplinary and (inter-)national laboratory lab-networking including sharing of resources which also resulted in global transparency. Some equipment funded by the NAICP has been installed and is in use. The following procedures are undertaken by the NIRL Abuja: By use of real-time PCR (two cyclers provided by NAICP and CDC respectively) and provision of reagents by CDC molecular diagnosis is carried out for the discrimination of Influenza A/B, and if positive for Influenza A subsequently for Pandemic H1/2009 or, in negative case for H5, seasonal H1 or H3 (CDC-protocols). Detection of N1 is not possible. Supported by CDC the lab is in process of accreditation (principles applied), and a Biosafety and Quality Manager is supervising all lab activities including regular validation of newly acquired primers/probes or PCR kits. Quality assurance includes Proficiency Testing twice annually. This is also supported by CDC and conducted twice annually. Until now the lab has always reached 100% TH NIRL National WHO- Influenza Centre 62 accordance. Maintenance and certification of equipment is carried out by South African AIR Filter Maintenance service (same company as for NVRI). Since 2009 the sample flow has been constantly increasing as a consequence of the global Pandemic H1N1/2009 crisis. Currently, 40-90 samples submitted to the lab on a daily basis, 80% reportedly derive from surveillance, 20% from suspicions. Training is carried out on regular basis, and with the increasing cases of Pandemic H1N1/2009 (swine flu) detections in humans and sample flow to the laboratory, the number of staff has been increased. Staff receives regular training (hands-on training and overseas training) supported by CDC and with a strong linkage also to CDC in Kenya. A regularly published pamphlet provides information on the activities with regard to Pandemic/H1N1/2009 and H5. There is an active collaboration with NVRI Vom which supports activities and provides training courses at NIRL, Abuja. Construction at the Teaching Hospitals is not yet finalized and the new labs are not all operational. Reportedly, some construction was carried out without the consultation of the laboratory experts and thus resulting in a suboptimal lab design which will need further time-consuming and costly modification to make it functional. However, some of the equipment provided by the NAICP (real- time PCR cyclers and freezers) is already in use. Reagents are supplied by other development partners (WHO or CDC). In its role as a WHO reference centre for influenza and arboviruses, the UCH Ibadan has the official mandate to provide confirmative diagnosis for the country and for the West African region. However, support comes mainly from WHO and CDC to fulfil this mandate. Due to limitations of reagent supplies this mandate is not followed to the degree intended. Two real-time PCR cyclers have been provided by the NAICP in 2009 and are in use, but reagents (from other sources) are limited and not yet provided through NAICP. Other equipment has been provided but has not been installed as the finalization and subsequent modification of the new laboratory is pending. The UCH Ibadan also houses a sequencer which is in regular use for HIV and Hepatitis virus characterisation, and there are plans to augment its use to phylogenetic characterisation of influenza viruses once primers have been obtained. There is a joint research activity with NVRI focussing on characterisation of influenza viruses in pigs carried out at the UCH Ibadan (using its resources) by a PhD candidate from NVRI. To summarize, the UCH Ibadan has a long established experience of influenza virus detection and is one of the very rare laboratories in the country that already accommodates high-level expertise and both modern equipment (including a sequencer) plus facilities for virus isolation in cell lines. However, apparently, it is not supported as much as it might be, and thus not involved in sample investigation and confirmative diagnosis as would normally be expected of a WHO Reference laboratory. Some key achievements are: ? Avian influenza was made a notifiable disease. ? There was a significant reduction in turn-around time (interval between receipt of specimen and availability of results) from around 200 days to 30 days or less. 63 ? There has been improved efficiency of influenza surveillance through the support of CDC, the NAICP and other partners. Nearly 1000 samples from influenza-like cases are tested annually. ? There is good collaboration between the human and animal surveillance teams who operate in the field side by side: this is a significant improvement from the situation at the outset when, in the words of a senior member of the NAICP ?we couldn?t even sit at same table.? ? At laboratory level the interdisciplinary approach contributes to a benefit of networking of both Human and Animal Health laboratories. ? More than 2000 people have been trained disease surveillance and notification, although due to a high staff turnover rate, only 60% of trained staff are still in post. Continuous training is desirable but is restricted by funds. ? Distribution of PPEs was considered to have been effective, although the quantity of PPEs available in 2006 was insufficient. ? All 36 states and FCT were provided with vehicles; 774 LGAs received motorcycles as did the veterinary teaching hospitals. In addition 37 boats were provided for use in riverine areas: MoUs with LGAs ensure continued supply of fuel, maintenance and boat operators, recognising that the boats will be used for many other purposes. With regard to Pandemic H1N1/2009 surveillance and control: Key messages: ? Project inputs contributed to the preparedness for the first case of Pandemic H1N1/2009 in Nigeria. ? The surveillance system set up for avian influenza (NISS) was also used for Pandemic H1N1/2009. ? Human Health Laboratories are able to perform Pandemic/H1N1/2009 (?swine flu?) diagnosis (by molecular means) and differential diagnosis to H5, and seasonal H1, H3 Influenza strains. 6.2.10 Pillar 10. Public Health Response and Containment Objective: To ensure the human population at large and the poultry associated households are prepared and protected from the threat of HPAI. Candidate components: Public health communications; risk awareness; laboratory capacity development; sound poultry handling and marketing practices; communication on human behavioural practices. Outcomes: Health facilities regularly stocked with target quantities of anti-viral medication; Tested preventive measures communicated to and adopted by at risk health workers and the general public. Under this evaluation pillar, the following sub-components appear in the NAICP: ? Strengthening health system response capacity (2c) ? Social distancing measures (this was not relevant as an epidemic did not occur) 64 ? Vaccination (of high risk workers & priority groups; cold chain; this was not relevant as an epidemic did not occur) ? Procurement and distribution of drugs and other supplies ? Medical services (medical care; specialised units in hospitals) ? Communication preparedness (3a-i) ? Developing capacity building modules and rapid social and communication assessment (3a- iii) Key findings ? Anti-virals (Tamiflu?) were not needed for HPAI but were probably used for treatment of swine flu cases. Project inputs on risk communications were used in developing messages for other diseases, including H1N1 (swine flu). ? Penetration of campaign messages to the general public was high and the great majority of Nigerians living in urban and non-remote rural areas had good knowledge of campaign messages. This knowledge would have reduced their risk of exposure to AI from poultry. ? Although poultry consumption declined during the outbreak there was no evidence of long-term decline compatible with successful communication efforts to safeguard the poultry industry ? Most Nigerians believe that FGN made great efforts to control AI. ? Around one tenth of secondary school children in Nigeria benefited from information campaigns that were effective in changing knowledge and attitude. Children from schools participating in the campaign were significantly more likely to report that they had washed their hands that day and significantly more likely to report that they washed their hands after handling poultry. ? However, availability of hygiene infrastructure was a stronger predictor of hand washing and self-reported diarrhoea than participation in the campaign, suggesting without investments in provision of hygiene infrastructure, campaigns on hygiene are likely to improve attitudes, but may not improve hygiene or health-related outcomes such as diarrhoea. Results With regard to strengthening health system response capacity, because a pandemic did not occur, the social distancing measures were not required. Vaccines were not available and so the vaccination of high risk groups was also not undertaken; again given the non-occurrence of a pandemic this was not a cause of concern. Procurement and distribution of drugs included supplying PPEs and anti-virals (Tamiflu?): although these were not needed for HPAI, more than 40% of the drugs provided in the states surveyed for this study had been used, most probably for treatment of swine flu cases. Initially the intention had been to establish isolation wards in all states but it quickly became apparent to project staff that this could not be achieved with the resources available. Instead, respiratory ICUs were established which could be achieved within the available resources. In the absence of a pandemic, a major focus of this pillar was developing and disseminating key messages targeted at specific groups, especially the general public, physicians and school children. The effectiveness of this messaging was evaluated through KAP surveys carried out for this study. The key findings of these surveys are: 65 General Public KAP Penetration of campaign messages was high and knowledge of campaign messages was good: 90% of respondents have heard about AI and two-thirds know the key messages. Also, appropriate channels for AI information dissemination appear to have been chosen: 92% of people report getting health information from the mass media. An association between the communication campaign and knowledge amongst the general public was observed: those who heard more messages have better knowledge; knowledge of AI facts covered in the campaign is better than knowledge of AI facts not covered in the campaign; and the channels respondents reported receiving messages on AI matches well with reported use of channels for other purposes. Most respondents (70%) believe (or strongly believe) that FGN made great efforts to control AI and most (75%) believe that people were informed about AI at the time of the outbreak. In terms of behavioural change, good hand-washing practices (self-reported) were significantly associated with better knowledge of NAICP messages. No sustained decline in poultry eating or poultry keeping was reported by respondents, which is compatible with successful efforts to prevent adverse effects on the national poultry industry. Around half respondents believe that the current risk of AI in Nigeria is low. However, many cases of fever and respiratory signs combined with a history of contact with poultry occur and most are not regarded as suspicious for avian influenza. Physician KAP Most physicians who received training under the Project worked in departments where members of the public with suspect cases of avian influenza were likely to present (assuming no selection bias in the sample taken by the evaluation team). Among these trainees, people from one of the eight states surveyed, and men, were heavily over-represented. Trainees tended to have better knowledge of avian influenza symptoms, case management and notification than non-trainees, but differences were not significant (although neither design nor sample size allowed investigation of differences between trainees and non-trainees). Trainees are significantly more likely than non-trainees to be willing to undertake management of human AI cases. One in ten physicians reported having encountered a suspect AI case; however, guidelines on notification and isolation are not being followed by trainees or non trainees. Only 37% of physicians reported that they had ever reported a notifiable disease; given, that notifiable diseases include AIDS, food poisoning and malaria, it seems there is under-reporting of notifiable diseases. School children KAP Children attending schools participating in the campaign were more likely to report presence of hygienic provisions within the control of the school (e.g. basins for washing, soap, hand-drying facilities). Also, children from schools participating in the campaign were significantly more likely to report that they had washed their hands that day and significantly more likely to report that they washed their hands after handling poultry. However, there was no significant difference in the observed hand cleanliness of children who attended schools participating in the campaign and those who did not. Nor was there any significant difference in the reported incidence of diarrhoea in the last month. 66 The strongest predictors of observed clean hands were adequate water supplies, presence of toilets (with separate facilities for girls), and provision of soap, hand-washing and hand-drying facilities. The strongest predictors of diarrhoea were spatial (the state the school is located in), livestock- keeping or contact with livestock (increases risk), and inadequate water provision and lack of soap (increases risk). The (unexpected) importance of animals and diarrhoea suggests further research is needed to investigate possible zoonotic causes of diarrhoea in children. After adjusting for confounding and systematic differences, we found that in states which had experienced an outbreak of avian influenza the campaign improved six-fold the odds of having observed clean hands among school-children; this was significant. In states without an outbreak there was no significant association with hand cleanliness and participation in the campaign. Without investments in provision of hygiene infrastructure, however, campaigns on hygiene are likely to improve attitudes but may not improve hygiene or health-related outcomes such as diarrhoea. The strong association between household livestock-keeping and increased self-reported diarrhoea in children warrants further investigation. 6.3 Communications Under the WB-funded NAICP, communications was one of the four distinct components. The evaluation presented here has used the 10 outcome pillars which cover animal and human health elements of HPAI response and pandemic preparedness. Within these, communications have been dealt with under several pillars, especially HPAI control and pandemic preparedness, public health response and containment, and poultry sector recovery. Here some of the key findings and results specific to the communications elements of NAICP are presented and discussed. Communications are intended to have multiple outcomes, including raising awareness, promoting multidisciplinary participation in planning, fostering greater engagement and commitment by various stakeholders and sectors of society, and promoting behavioural change in some areas. To be effective, these require careful identification of target audiences, and careful testing and tailoring of messages depending on the desired outcome. In some NAICP communications these appear to have succeeded, in some they have been less effective, and in most it has been difficult to disaggregate the attribution to NAICP from that of other organisations. In terms of communications targeted at the general public, it is impossible to isolate the contribution made by NAICP?s communication initiatives from those made by a wide range of other actors: it is likely that most people in Nigeria were exposed to a variety of messages emanating from a variety of sources. Under the NAICP-funded communication component a diverse range of media and formats were utilised to deliver messages to the general public as well as more specialised audiences, such as journalists (see Table 8, below). These included television slots and radio jingles, radio dramas, billboards, videos, printed materials, give-ways (such as t-shirts and computer mouse mats), live events and the project website www.aicpnigeria.org (see table below). Radio and television media slots appeared to be carefully targeted and timed: messages broadcast on television during the World Cup reached audiences estimated at up to 20 million; radio jingles were produced and broadcast to coincide with key Muslim and Christian festivals; billboards were produced in the four main languages used across the country. Communication desk offices in HPAI-affected LGAs received additional funding. In addition a number of communications training and coordination activities 67 were undertaken. These included a consensus building workshop for civil society groups, an exhibition held at the National Council on Information and Communication which showcased the communication achievements of the NAICP, in collaboration with UNICEF, media training workshops were held for 500 journalists, and two component staff members benefited from specialist training in London. The KAP surveys, undertaken as part of this evaluation, suggest that these approaches were relatively successful: there was high penetration of campaign messages; knowledge of campaign messages was good; and appropriate channels for AI information dissemination chosen with 92% of respondents reporting that they obtained health information from the mass media. Messages delivered via radio jingles and other types of broadcast are estimated to have reached up to 50 million people. Two important areas of communication have had more dubious effects. The first is in the multidisciplinary communication required for project management. While on the one hand the three components coordinators have enjoyed a productive professional collaboration and communication among them, their task has not been facilitated by the animal and human health coordinators operating from separate facilities, so limiting the kind of close communication and synergy demanded of this challenging project. The second area is in communications that influence behavioural change in poultry marketing and handling. This was tackled in two ways, one by infrastructural change (the improved live bird markets in a proportionally minute fraction of LBMs in Nigeria) and communications on poultry handling and storage (such as separation of species, use of metal cages, etc). From this evaluation and that of FAO?s programmes in November 2009, it appears that the behavioural changes induced had no effect on the course of HPAI in Nigeria, and indeed still raise broader public health and food safety concerns. 68 Table 8: Summary of communications activities under NAICP Messages Quantity Target group Dates Where messages were sent Audience Information sheets ? Bird flu ? Swine flu Other written media: 1. 2. AI Branded folders ? Posters ? 450, 000 ? FAQs ? 280, 000 ? Leaflets ? 750, 000 ? Large Logo ? 10, 000 ? Small logo ? 70, 000 ? Msg Stickers ? 100, 000 ? Mouse Pads ? 5, 000 ? Folders 5, 000 General public, schools & poultry farmers, live bird markets 2007 ?10 Nationwide ~20,000,000 (assumes seen by 10 people) Banners/billboards ? Metal boards ? Billboards ? Roll up banners ? 58 metal boards ? 37 billboards ? 100 banners General Public 2008-10 ? Geo-political zones (6) ? States (12) ? Nationwide ~ 20 million ? TV documentaries ? TV slots ? 22 episodes ? 12 ? General Public ? Football fans ? 2007 ? World Cup ? NTA network ? National ? 30 million ? 20 million ? Radio Jingles ? Radio documentaries ? Radio Drama (Hausa) ? 168 ? 26 episodes ? 13 episodes ? General Public ? General public ? Hausa listeners ? 2007 ? 11 ? 2008-10 ? 2009 ? Nationwide, FRCN, RayPower network, HOT FM, Vision FM ? Nationwide ? 19 northern states radio stations ? 50 million ? 50 million ? 10 million Journalist training 6 workshops Media chiefs & reporters 2007 -10 Lagos, Kaduna, Jos, Enugu, Abuja, Kano & Owerri 500 journalists Clothing ? Caps ? Tee-shirts ? Aprons ? Branded Overalls ? A I branded bags ? A I branded ? 8, 000 ? 15, 000 ? 3, 000 ? 3, 000 ? 2, 000 ? 1, 500 General Public, LBM & Poultry farms 2007 ?10 Nationwide ~3,000,000 (assumes seen by 100 people) 69 towels Bird flu watch magazine 5 editions @ 10, 000 = 50, 000 General Public 2009 ? 10 Nigeria and outside Nigeria ~500,000 Project website 1 website General public 2007 to date worldwide ~ 2 million Sensitization rallies in motor parks, poultry markets and grazing reserves 7 rallies Mobile and migrant population 2008 & 2009 FCT, Lagos, Kano, Awka & Maiduguri ~7,000 Integrated response plan 5, 500 Ministries, States, Libraries etc 2008 Nationwide 50, 000 70 7 Relevance, effectiveness, efficiency, impact and sustainability 7.1 Relevance How relevant was an approach for assistance by FGN to the WB, and how relevant was the WB response? At the time, the FGN was acutely aware of its vulnerability to the possible effects of HPAI and a possible human pandemic, and aware of further damage to its international reputation should action not be taken. The WB presented a readily accessible option, given its earlier international responses in other regions of the world, and the good contacts between FGN and WB in Abuja. Furthermore, the principle of considering and acting on a request by the FGN for a credit to respond to the threat of HPAI is considered highly relevant. It is sometimes easy to forget the fear created globally by the outbreaks of HPAI in Asia, following on the earlier legacy of the SARS outbreak, all compounded by the frenetic media coverage, aided by the global explosion in a widely diverse set of media outlets, feeding an appetite for fearsome messages. Beyond this, the international community, including the WB, were keen to deliver on the commitments made at the Beijing donor conference in January 2005, and Nigeria was seen to be of strategic significance due to its large population, its role as a regional hub, and its underlying poverty and food security impacts. As far as the structure and objectives of the project are concerned, the aims of containing the disease, reducing risk and preparing for a possible pandemic were broadly fit for purpose, and importantly, they evolved as the project matured and as the FGN and its partners learnt more about the behaviour of the disease, and what they felt could and could not be achieved. An example of the willingness of FGN and WB to adapt was the dropping of the sub-component of Strengthening Applied Veterinary Research, and the activities designed to support economically vulnerable groups, and provide alternative livelihoods for affected stakeholders, when it was seen that these either distracted from the focus necessary, or were not attainable within the resource package assigned. Was the mix of project components relevant to the task at hand? In broad terms the answer is yes, although undoubtedly with hindsight the project composition would likely alter. One candidate area for reconsideration would be the investment made in the strategically selected quarantine stations. The thinking behind this was undoubtedly logical at the time, with the huge traffic in livestock across certain borders, and the inherent risks for disease transmission, but in this case the disease entry (which is still undetermined) was more likely to have originated from longer distance trading in poultry by air or from wild birds; furthermore the resulting changes in quarantine stations has had little effect on disease control, as the FGN continues to struggle with the establishment of good practices at these posts. The evaluation team used an evaluation framework of 10 bench mark pillars to assess the performance of the NAICP. While not definitive, they have provided an accepted and peer-reviewed international framework for evaluating infectious disease control at the national level. Two key issues emerge with regard to the relevance of the NAICP design and this framework. Firstly is that while some sub-components and activities might have been tucked incongruously under different headings, possibly a reflection of the haste of document preparation, all the framework pillars are indeed represented in the NAICP design. However, certain important subcomponents and activities seem to have received inadequate attention. These include the high level of engagement by FGN required for effective disease control with the rapidly growing and highly independent private poultry sector, and the inadequately recognised importance attached to analytical epidemiology as a leading pillar of disease control and disease information communication. The private poultry sector 71 plays a substantial role in Nigeria, and certain elements of it were engaged by FGN. A stark lesson was learnt when the initial compensation scheme was launched, as prices offered under-valued poultry enterprises and resulted in a confrontation with some poultry producers whose livelihoods were threatened. Importantly, the ensuing dialogue resulted in a complete revision of compensation values, and in an emerging scheme which is lauded for its transparency and efficacy in many corners of the world. The key message from this was the importance of engaging directly with the private sector as a partner. As far as the inadequate attention to epidemiology is concerned, it is recognised that the FGN already had an established unit and an information management system (NADIS), and therefore did not wish to allocate project funds to something that was up and running. However, this meant that the epidemiology capacity, so important in informing policy and strategy, stood still rather that leaping forward as it was confronted with what might have become its finest hour. A key illustrator of this is the poor quality of data encountered by the evaluation team when attempting to analyse NAICP records. Not all the secondary data made were available by the NAICP project, and some of the data had to be requested from the state desk officers. Data files were often received in a variety of formats (e.g. MS Excel, MS Word, jpg files, paper records) and had to be converted and complied into spreadsheet and databases. The time-to-event analysis was an important part of the evaluation, requiring specific dates of outbreaks noted by famers, dates reported by famers, etc. However, almost one third of the dates was in incorrect formats and had to be manually modified to allow calculations. Furthermore, the unit of outbreak recording (i.e. farm or village) was not clearly or consistently specified by the state desk officers. These comments relate both to an inadequacy in recognising the relevance of good analytical epidemiology, and to a lack of effectiveness in characterising the epidemiology of HPAI in Nigeria. 7.2 Effectiveness This evaluation has assessed effectiveness at various levels (both, from a theoretical and empirical perspective). First, effectiveness is considered in relation to the 10 outcome pillars that have been taken as benchmarks of performance in HPAI control and pandemic preparedness at a national level. Next, overall effectiveness is considered with regard to the PDOs. Under each outcome pillar, the generic pillar objectives, candidate components and outcomes are listed, followed by an assessment of how effective NAICP has been in meeting these. PILLAR 1. HPAI CONTROL AND PANDEMIC PREPAREDNESS: Objective: The development of a clear and technically sound policy for HPAI preparedness and control, effectively communicated to all stakeholders. Candidate components: Legal framework, national policy, national strategy, contingency planning, benefit: cost considerations, research prioritisation, industry development, poverty reduction interface, identification, engagement and communication with all stakeholders. Outcomes: Sound HPAI policy in place; all stakeholders involved and informed. Overall, the evaluation team consider that the pillar outcome was achieved in a satisfactory manner; there was a sound HPAI policy put in place and considerable effort was put into ensuring that key stakeholders were involved and informed. Specifically: ? the evaluation team considers that overall planning and preparedness was satisfactory ? the presidential-level support was extremely helpful at the outset 72 ? strong leadership was provided by several NAICP champions, including the Chief Veterinary Officer (CVO) of the country and the director of the National Veterinary Research Institute (NVRI), Vom ? some activities were constrained by delays in disbursement of funds ? certain key personalities played a strong role. This might make it difficult to recreate the success of planning and coordination with the different set of leaders now in place ? the inter-ministerial cooperation brought about by the project would have been even more effective if key actors from the ministries of health and agriculture were located in same building ? the existence of federal and state systems was a complicating factor which impacted on efficacy ? the devolution of diagnostic responsibilities to state teaching hospitals was a sound idea but was in practice problematic: only NVRI was able to carry out the improved molecular techniques Without preparedness and control policies in place it is not possible to effectively control animal diseases. The project provided a framework for future control efforts, which will have positive (although difficult to quantify) impacts on poultry-dependent livelihoods, the poultry sector, and consumers. PILLAR 2. HPAI SURVEILLANCE: Objectives: Establishment and revision of effective, sustainable and affordable surveillance systems for HPAI in targeting domestic and wild bird populations. Candidate components: Passive surveillance, active surveillance, wild bird surveillance. Outcomes: HPAI infection status effectively determined and internationally recognised. This pillar outcome was not fully met. The surveillance system took time to expand beyond the 170 control posts and 259 surveillance agents established by the National Animal Disease Information and Surveillance system (NADIS) under the Pan African Programme for the Control of Epizootics (PACE) programme. Efforts to devolve responsibilities to the States at Veterinary Teaching Hospitals (VTHs) were hampered by capacity development and there were problems getting new equipment functioning properly. The active surveillance based on risk was very limited, and wild bird surveillance was not included in NAICP. The turnaround time to provide definitive results for field samples was progressively improved, mostly through improving sample transportation and efficiency of laboratory processes. Specifically: ? the human and animal health surveillance teams worked well together in the field ? innovative partnerships, such as that with the National Union of Road Transport Workers (NURTW), increased effectiveness of sample transportation ? devolution of responsibility to state level was hampered by infrastructure development delays ? live bird market surveillance through NAICP was limited, it was mainly carried out by FAO ? improvements in surveillance, but evidence suggests both human and animal surveillance systems pick up a very small proportion of cases that meet the definition of suspect 73 PILLAR 3. HPAI DIAGNOSIS, DIFFERENTIAL DIAGNOSIS & PATHOGEN CHARACTERIZATION: Objectives: Establishment and maintenance of internationally recognised laboratory capacity to confirm and where appropriate characterise HPAI infections. Candidate components: Sample collection and shipment; cold chain viability; laboratory facility development; laboratory equipment and reagents; laboratory network and interface; capacity building of laboratory staff. Outcomes: Optimal sensitivity and specificity of diagnostic tools established and results in international public domain. Overall, diagnostic capacities at reference laboratory levels (both in animal and human health) were improved: influenza viruses can now be detected in biological samples with high sensitivity and specificity, and sub-typing is carried out to international standards. Distribution of influenza- positive samples to International Reference Laboratories (CDC-Atlanta, FAO/OIE Reference Laboratory in Padua, Italy) has contributed to international transparency, corroborated by joint global scientific collaboration. The capacity for differential diagnosis (i.e. including consideration for ND and other poultry diseases) has received inadequate attention. A lack of appropriate reagents and further training (particularly in pathology) has limited the capacity to provide substantial diagnostic services for poultry diseases other than AI. The upgrading and devolution to the States of diagnostic facilities to use molecular techniques and to improve biosafety was limited. However this may not have affected diagnostic performance, as innovative arrangements for transporting samples to laboratories and better systems for processing samples on arrival led to decreased turnaround time. Overall, much of the credit for enhanced diagnostic capacity goes to initiatives by NVRI independent of NAICP, and on the human health side to support received from CDC and WHO. Although the late implementation has meant most improvements in laboratory capacity did not contribute during the outbreak period; these now represent enhanced capacity for future outbreaks. PILLAR 4. HPAI OUTBREAK CONTAINMENT & DISEASE CONTROL Objectives: Plan and implement technically sound, effective affordable, sustainable and socioeconomically acceptable intervention measure to control or eradicate HPAI . Candidate components: Intervention measures (depopulation, decontamination, disposal, movement control, awareness raising, vaccination, poultry restocking, etc); compensation. Outcomes: Disease outbreaks effectively contained and status recognised internationally. HPAI has not been detected in Nigeria since July 2008. The two key questions, however, are whether the disease was unsustainable in the Nigerian environment and therefore burnt itself out, and whether the NAICP was responsible for its eradication. The evaluation team is unable to answer the first question, save to comment that the disease did disappear from all other affected West Africa countries as well, some of which had much less effective control programmes in operation than Nigeria. On the second question, it appears that many of the measures put in place by NAICP came too late to have an impact on the outbreaks, although the NAICP structure was used to implement/ distribute the in-kind donations from the onset. The FGN funded many activities under this pillar which were supplemented by substantial in-kind donations, such as disinfectants and PPEs, by other 74 agencies. It is possible that the knowledge of impending WB funds gave FGN the confidence to spend in the early stage, when it was perhaps most needed, safe in the knowledge that funds were coming. The compensation mechanism, once revised to reflect market prices, was an effective measure which probably contributed to the speed and effectiveness of the poultry culling on infected premises. This may have slowed down the spread of HPAI. PILLAR 5. EPIDEMIOLOGICAL CAPACITY FOR STRATEGY DEVELOPMENT & INTERVENTION TARGETING Objectives: Collect, synthesise and analyse data on the dynamics and impacts of HPAI, and use the outputs to inform policy and strategy for HPAI control. Candidate components: Epidemiological data handling, processing and analysis; data flow and communication ; data reporting, use and presentation; poultry population demography; outbreak investigation; value chain studies; risk assessment (based on critical control points); Socioeconomic impact. Outcomes: Quality data received and disease control strategy regularly updated through a sound evidence base. The effectiveness of this pillar is somewhat difficult to assess. Unquestionably the skills existing in the epidemiology unit enabled prompt synthesis of emerging surveillance data, and good communication was established (in the same building) with the office of the CVO. There was an epidemiology team on ground due to the spill-over from the PACE programme, a system for surveillance, and an information system (NADIS). Summary reports on HPAI status were produced, the information provided assisted NAICP in its strategy and in targeting interventions, and the disease disappeared. However, on scrutiny of the data made available to the evaluation team by the State desk officers, certain inadequacies in data handling were detected: for example, inadequate standardisation of reporting units (whether household, village, LGA) and inadequate denominator35 data on poultry. Furthermore, it is felt that epidemiological capacity would have been more effective with greater use of risk-based analysis and value chain approaches to increase the understanding of critical control points in HPAI control. PILLAR 6. HPAI PREVENTION Objective: Put in place technically sound affordable and socially viable measures to minimise the risk of HPAI spread, and reduce the risk of human infection. Candidate components: Biosecurity, communication, human protection, vaccination, farm/unit registration, market and slaughter practices; industry support and restructuring. Outcome: Progressive reduction in disease incidence which is independently verifiable. No new human cases. This pillar was seen to be satisfactorily effective in some areas, and less effective in others. The enhanced biosecurity through infrastructural improvements to selected LBMs was undoubtedly effective, and independent evidence of this was generated in the evaluation. However, as a pilot initiative, the number of LBMs upgraded was insignificant in relation to the total number of LBMs in Nigeria; nevertheless this provides a very constructive and positive case study which can be built on 35 Data on poultry populations, particularly by production system type. 75 in the future. This component introduced during the project has clearly strayed from the original emergency nature of the loan, since it will only be able to affect disease risk significantly if a more extensive improvement of LBMs were to be implemented across the country. The training in improved biosecurity for poultry traders did not reduce risky practices undertaken, and improvements in biosecurity for farmers were considered by the evaluation team to be insufficient to prevent HPAI spread. It was felt by many of the key informants consulted that the NAICP made an appropriate decision not to vaccinate poultry, despite intense pressure from international sources such as FAO, given the realistic assessment of the immense logistical and cost complications it would have incurred, and the challenges of developing an exit strategy, a complication that has been faced by Viet Nam. Nevertheless, the evaluation team learnt of reported high levels of unofficial vaccination among commercial poultry enterprises. PILLAR 7. POULTRY SECTOR RECOVERY Objective: To minimise the impact of HPAI on the credibility and economic viability of the poultry industry. Candidate components: Public relations of risk avoidance with poultry and poultry products; viable compensation scheme; dialogue with and support to all sections of poultry industry. Outcome: Affected poultry farms back in business, compliance with FGN policies for HPAI control and prevention is enhanced. Overall, this pillar was seen by the evaluation team as satisfactorily achieving the desired outcome. The compensation scheme was widely seen as being successful, striking the right balance between under-compensating, resulting in low levels of disease reporting and a likelihood of greater spread, and over-compensating; the amounts paid were appropriate after the transition to market-based rates had been made. It was also recognised by most people interviewed for this study as being a transparent process. The messaging produced by the NAICP and others restored the confidence of the general public to consume poultry, and most elements of the poultry sector did indeed recover. PILLAR 8. PUBLIC HEALTH PANDEMIC PREPAREDNESS Objectives: The development of a clear and technically sound policy for HPAI preparedness and control, effectively communicated to all stakeholders. Candidate components: Legal framework, national policy, national strategy, contingency planning, benefit: cost considerations, research prioritisation; identification, engagement and communication with all stakeholders. Outcomes: Sound policy & strategy in place and effectively communicated; all key human and animal health stakeholders involved. This pillar was considered by the evaluation team to have been moderately satisfactorily achieved, but was not put to the test regarding HPAI becoming a pandemic; it was tested to a degree with the advent in 2009 of H1N1/2009 Pandemic influenza. There was general satisfaction with the planning and preparedness activities conducted by the NAICP amongst policy makers, experts and state officials. 76 PILLAR 9. INFLUENZA SURVEILLANCE Objective: Infection status in human populations known and internationally recognised. Candidate components: Laboratory capacity development; surveillance networks established. Outcomes: Infection status in humans known and internationally recognised. This pillar is considered by the evaluation team to have been reasonably effective, although some of the results of the analysis raise substantial questions about surveillance in general. HPAI is now a notifiable disease. An influenza surveillance system is now in place, but there appears to be massive under-reporting of cases that apparently match the HPAI case definition; importantly under-reporting widespread is for all notifiable diseases, not just the influenzas. The strengthening of the surveillance systems has had an apparent impact on the reporting of malaria and H1N1/2009 Pandemic influenza (swine flu). The public health sector is very proud of HPAI surveillance capacity right down to LGA level. The human health laboratories are able to carry out molecular techniques for the detection of Influenza A/B and sub-typing of AI H5 and H1/2009 Pandemic influenza both according to CDC- protocols. PILLAR 10. PUBLIC HEALTH RESPONSE AND CONTAINMENT Objective: To ensure the human population at large and the poultry associated households are prepared and protected from the threat of HPAI. Candidate components: Public health communications; risk awareness; laboratory capacity development; sound poultry handling and marketing practices; communication on human behavioural practices. Outcomes: Health facilities regularly stocked with target quantities of anti-viral medication; tested preventive measures communicated to and adopted by at risk health workers and the general public. This pillar is seen by the evaluation team to have been reasonably effective, with the caveat that no pandemic occurred, so the response and containment systems were not put to the test. Public health messages on awareness of HPAI, and subsequently on the safety of poultry products, appeared to be successfully delivered and well understood, although with messages emanating from many sources, attribution to NAICP is impossible. Public health messaging by NAICP targeted at children was effective and led to demonstrable beneficial behaviour change, e.g. in hand-washing in the targeted children. Several NAICP materials (such as anti-viral drugs) supplied for HPAI were used for the Pandemic H1N1/2009. PROJECT DEVELOPMENT OBJECTIVES When moving to the broader effectiveness of the NAICP, the evaluation team considered the three Project Development Objectives (PDOs), which were: (i) To support the efforts of FGN to minimize the threats posed by H5N1 to humans and the poultry industry (ii) To prepare the necessary control measures to respond to a possible influenza pandemic (iii) To prevent further spread of the disease to other parts of Nigeria. 77 Furthermore, the NAICP had an overall development objective, which was: ? To minimize the threat posed to the poultry industry and humans by HPAI infection and other zoonoses, and ? To prepare for, control and respond to influenza pandemics and other infectious disease emergencies in humans. In the NAICP it is stated that key Indicators of Project Outcome will include evidence of improved effectiveness of Government in responding to the risk of an HPAI outbreak and/or pandemic, and contained and diminishing pattern of HPAI infection in poultry. Assessment (i) To support the efforts of FGN to minimize the threats posed by H5N1 to humans and the poultry industry: ? There is substantial evidence presented in this report that the NCAIP did indeed support the efforts of FGN to minimize the threats posed by H5N1 to humans and to poultry, through public awareness and capacity-building within the human and veterinary health systems. ? There is reasonable evidence that the NAICP contributed to minimizing the threats posed by H5N1 to humans in the longer term by strengthening preparedness, surveillance and response mechanisms. ? There is very little direct evidence that the NAICP contributed to minimizing the threats posed by H5N1 to humans during the 2006, 2007 and 2008 outbreaks; for reasons that have not been established, there have been 143 confirmed cases of HPAI H5N1 in humans in Africa, and all but two of these have occurred in Egypt. There have been no other confirmed cases in West Africa. ? There is reasonable evidence that the NAICP contributed to minimizing the threats posed by H5N1 to the poultry industry, including through an effective compensation scheme and delivering appropriate messages about the safety of consuming well-cooked poultry, but insufficient evidence that it was responsible for the eradication of the disease. (ii) To prepare the necessary control measures to respond to a possible influenza pandemic: ? There is substantial evidence that the NAICP prepared necessary measures to respond to an influenza pandemic. (iii) To prevent further spread of the disease to other parts of Nigeria: ? There is limited evidence that the NAICP prevented the spread of HPAI H5N1 to other parts of Nigeria. The disease occurred in 16 States in 2006, which rose to virtually 22 States in 2007 The NAICP was clearly written in extreme haste, with the intention of presenting a structured plan for the response to HPAI in Nigeria in a very short time. It is easy to criticise the structure of the document with hindsight, but it did cover most of the bases necessary, as reported above. Nevertheless, apart from the cumbersome and unclear presentation of the PDOs, the objectives themselves are inadequately explicit to be used for objective, evidence-based evaluation. One of the recommendations presented below addresses this, suggesting a more structured approach to project design for national zoonotic disease control responses in the future. 78 7.3 Efficiency Efficiency is generally seen as being a function of timeliness, adequacy of resources, cost effectiveness, and the use of partnerships. This evaluation has assumed that the audit of project and the accountability of expenditures has been a responsibility of the WB, and the evaluation has therefore focussed much of its efforts on the effectiveness and the attainment of key outcomes. Nevertheless, a few general comments on efficiency are made here. On the positive side, the designation of responsibility and allocation of substantial funding to one government body presented, in theory, an opportunity for the efficient use of resources, and an efficiently run programme. And this factor undoubtedly did contribute to the strong and effective leadership taken by the FGN. However, this was understandably not absolute, and issues pertaining to the straddling of the project across different ministries have been raised. Furthermore, the evaluation has concluded in several cases that the NAICP delivered (training courses, equipment, infrastructure, etc) after the main disease outbreaks had occurred. Were any of these delays avoidable? Among the many issues resulting in less than ideal efficiency was the complexity introduced by the combined bureaucracies of the WB and the FGN, illustrated in the box below. ?NAICP was an emergency project; the administrative processes supporting it were not? The NAICP was designed and implemented as an emergency response to what was perceived to be a very serious situation. However, it was observed by senior members of the NAICP that although this was clearly an emergency project ? the original project proposal had been written in just three weeks ? the subsequent administrative processes required by the WB and the FGN were most certainly not of an emergency nature; they were slow and laborious. This mismatch inevitably caused problems. The project needed to procure large quantities of goods; construct complex facilities; and facilitate less tangible items such as training and the payment of compensation. The WB procurement unit in Nigeria was reported by NAICP officials to have been ?overwhelmed? by the workload this generated: as a result some procurement processing responsibility was transferred to Washington. In addition to stringent requirements from the WB, clearance also had to be obtained from FGN. Although in the case of the MoARD this (as reported by NAICP officials) did not present significant problems, at the outset the requirements imposed by the FMoH were described as being ?overbearing?. The situation was exacerbated by the fact that over a 5-year period the FMoH had 10 different ministers, six permanent secretaries and four directors, all of whom brought with them their own personal styles and idiosyncrasies which had to be accommodated by the project. The situation was reported to have improved over the last two years but initially this heavy, double layer of bureaucracy was cumbersome and constrained project efficiency. Anecdotal evidence illustrates the type of problems encountered: In one case, documents relating to an international competitive bid (ICB) were submitted the WB but no response was received for six months. Since ICBs take a minimum of six months to process at best, in this example the procurement took one year. In another, having completed the procurement process for a batch of antibiotics, the WB unilaterally cancelled the contract at the final stage. The procurement process for a BSL-3 laboratory, which represented an investment of more than US$1 million and was regarded as a key resource for safe and effective influenza diagnosis, was particularly difficult: three years later (April 2011) the contract has still not been awarded and it appears unlikely that the facility can be installed and commissioned within the lifetime of the project. The aide memoire for the Seventh Implementation Support Supervision Mission (November 22?December 4, 2010) apportions blame to both sides: ?The main reasons for the slower than expected disbursement involved delays in procurement resulting from delays in receiving WB No Objections. On the project side, the quality of documentation submitted to WB should be improved, while on the WB side response time has often been well beyond the 7-day business standard.? 79 Economic efficiency: the economic impact of strengthening HPAI control A common indicator of the effectiveness and efficiency of a public investment, such as that made by the WB and the FGN to strengthen HPAI control in Nigeria, is whether the investment proves justified in terms of the returns generated as economic benefits. The evaluation applied standard benefit: cost analysis to evaluate the economic performance of NAICP. Framing the question If HPAI control had not been strengthened, more HPAI outbreaks would have been expected to have occurred in Nigeria, incurring additional poultry mortality and economic losses in the poultry sector and in the wider economy. Furthermore, the possibility that allowing HPAI to circulate unchecked in the poultry population would have eventually led to a human pandemic cannot be dismissed. The challenge, then, is to estimate the losses that were potentially avoided due to the investment in more aggressive control. The epidemiology of HPAI remains poorly understood, particularly in the African context. When the first outbreaks in sub-Saharan Africa occurred in 2006-2007, the common perception was that the disease would be difficult to control and would spread, cause large losses and potentially become endemic. The proposal for the WB emergency credit, for example, assumed that 30% of the poultry population in Nigeria would be affected by HPAI.36 An OIE-commissioned assessment in September 200737 reported that while initial losses in Nigeria from January-October 2006 were estimated to total US$ 8.4 million, continued direct losses would be much larger; these were estimated to be of the order of US$ 113 million annually, an increase of over 13-fold. An IFPRI study in 2006 projected direct losses of 4% of the Nigerian poultry population, with substantial economic impacts as poultry production declined by 21% and chicken prices by 12% (You and Diao, 200638). Later analyses by IFPRI considered a range of losses, from a continued 10% reduction in poultry production (Diao et al. 200939) to 75-100% losses of the flock (Okpukpara et al. 200940). A study by Fasina et al. (200841) estimated losses based on a ?mild? scenario of outbreaks affecting 10% of the commercial flock, and a ?severe? scenario in which 70% of the commercial flock is affected. As the wide range of assumptions made by several different analysts demonstrates, HPAI was expected to cause major losses, perhaps reflecting the spirit of the precautionary principle that it is reasonable to assume a worst-case scenario in the absence of evidence. 36 ?For purposes of risk analysis, protection rates varying from 33 percent to 80 percent for both poultry and humans have been assumed. The likely impact of an HPAI pandemic on unprotected humans was derived using WHO estimates of human-to-human transmission: (a) 30 percent of unprotected humans are likely to be affected, (b) one tenth of them will require hospitalization; and (c) one and a half percent of all human infection cases will be fatal. The same parameters were assumed for poultry protection, although all birds exposed to the risk were assumed to perish (either from the infection or through stamping out).? (page 31, Technical annex on a proposed credit in the amount of SDR 34.9 million (US$50 million equivalent) to the Federal Government of Nigeria for an avian influenza control and human pandemic preparedness and response project under the Global Program for Avian Influenza and Human Pandemic Preparedness and Response (GPAI) for eligible countries under the Horizontal APL, March 14, 2006. 37 OIE (World Organisation for Animal Health). (2007). Prevention and control of animal diseases worldwide. Economic analysis ?Prevention versus outbreak costs. Final Report, Part I. Civic Consulting - Agra CEAS Consulting, Kent. 38 You L, Diao X. (2006). Assessing potential impact of Avian Influenza on Poultry in West-Africa - A spatial equilibrium model analysis. DSGD Discussion Paper No 40. IFPRI, Washington, D.C. 39 Diao X, Alpuerto V, Nwafor M. (2009). Economy-wide impact of avian flu in Nigeria ? A dynamic CGE model analysis. HPAI Research Brief No. 15. IFPRI, Washington D.C. 40 Okpukpara B, Asare-Marfo D, Birol E, Roy D. (2009). Investigating the potential impact of HPAI on livelihoods in Nigeria. HPAI Research Brief No. 10. IFPRI, Washington D.C. 41 Fasina FO, Sirdar MM, Bisschop SPR. (2008). The financial cost implications of the highly pathogenic notifiable avian influenza H5N1 in Nigeria. Onderstepoort Journal of Veterinary Research 75:39-46. 80 In the event, the outbreaks affected less than 1% and 0.3% of the poultry population in the 2006 and 2007 episodes respectively, before one last, very small outbreak in 2008. With the benefit of hindsight, we now know that the actual losses were much lower than projected. As noted in other components of this evaluation, it is still uncertain to what degree the lower-than-anticipated losses should be attributed to control efforts by NAICP (and others) versus simply representing the natural course of the disease in the Nigerian context. Given this background, the analysis of the economic impact of NAICP is based on a comparison between the incremental costs incurred to strengthen HPAI control and the incremental benefits (in terms of economic losses avoided) due to that strengthened control. The incremental costs are clear and are represented by the disbursements of the WB credit over the project period from 2006 to 2010. The incremental benefits are much more difficult to estimate, considering the uncertainties surrounding the epidemiology of HPAI and the contribution made by intervention activities. How much was avoided as a result of the strengthened control measures compared to losses avoided from controls underway without the WB credit? This uncertainty is addressed by considering a range of assumptions and scenarios to represent what might have happened had the WB credit not been granted. The Factual The initial outbreak precipitated various control interventions and disease surveillance initiatives by the FGN and private actors in the poultry industry. Both FGN and private actors responded to the outbreaks by taking actions within their resource constraints to mitigate the risk of the disease. In the case of private actors, their actions were intended to reduce the risk of the disease to their livelihoods, so in some cases may have included behaviour such as selling their flocks which may have helped the spread of HPAI. The WB credit was intended to strengthen the initial responses, including improving the compliance with FGN strategy. Activities directly attributable to the WB credit were evident particularly from 2007 when outbreaks flared a second time before declining to zero in November, with the final small outbreak in mid-2008. The Counterfactual scenarios A. Best-case scenario This scenario assumes the introduction and spread of HPAI in 2006-2008 was a unique event, and that given the ecological and production systems context in Nigeria, the disease could not sustain itself: it essentially burned out on its own. This also assumes that subsequent to the initial outbreaks, it no longer presented a threat of re-introduction in sub-Saharan Africa, or so low a risk as to be negligible. Under this scenario, the WB credit yielded no incremental benefits to actions already underway by FGN and private actors (Benefit: Cost Ratio = 0), although it is likely to yield certain benefits in the control of future outbreaks of emerging infectious diseases. B. Medium-case scenario Under this scenario, the epidemiology of the disease and/or the responses by FGN and private actors would have limited the initial outbreak as occurred in 2007, but occasional outbreaks similar to those in 2006-2007 would have continued to occur. The risk of re-introduction and spread is estimated based on historical data. C. Worst-case scenario This scenario assumes that HPAI would have become endemic despite the response by FGN and private actors with their own resources (i.e. without support from the WB). In this case, outbreaks 81 would have been expected to follow a normal epidemic curve, initially flaring as it did in 2006 and then continuing at a lower level similar to the pattern of 2007. Doomsday scenario The preceding scenarios consider only losses avoided from reduced poultry mortality. If, however, the benefit of avoiding a potential human pandemic is introduced into the analysis, the expected global benefit can be expected to be so large as to dwarf the investment and to make the analysis moot (Congressional Budget Office, 200542; Cooper, 200643). No attempt, therefore, is made to quantify this scenario. The analysis To conduct the benefit: cost analysis (BCA) of the WB credit, streams of incremental costs and benefits are estimated over the project period (2006-2010), appropriately discounted to factor in the time value of money, to derive the standard financial indicators of Benefit: Cost Ratio (BCR) and Net Present Value (NPV), which are explained below in presenting the results. A discount factor of 12% is used. The incremental costs are based on the reported disbursement of the WB credit over the project period (2006-2010). The incremental benefits include direct and indirect costs, all related to impacts on the supply side. A key assumption is that activities funded by the WB credit only began to significantly improve HPAI control as of 2007, through initiatives such as revamping the compensation formula. Based on this assumption, no incremental benefits are assigned to mitigating the consumer scare and its negative impact on prices and sales volumes because the consumer scare occurred in early 2006 and the market had essentially recovered by the time the effects of the WB-funded interventions could be felt. It is assumed, based on the experience in other affected countries such as Indonesia and Vietnam, that consumer scares do not reoccur. Direct costs: ? The market value of incremental chicken losses. Chicken mortality, which includes both those that die directly from the disease and those that are culled in an attempt to limit further spread of the disease, is estimated based on the different risk scenarios adopted. The ratio of birds killed by disease to those culled is assumed to remain constant at 54.6:45.4 based on 2006-2007 data. Chicken losses are valued at a representative 2006 market price of US$ 6.45/bird (broiler [US$ 7.32] and spent layer [US$ 4.99] retail prices weighted by their shares of the commercial flock [broiler: 62.5%; layers: 37.5%]; as reported in Akinwumi et al44. [forthcoming]). ? The market value of incremental egg losses. Based on the national flock structure, 85% of chicken losses are assumed to occur among the layer population. Egg losses include the value of existing egg stocks associated with these layers that are destroyed when the bird is 42 Congressional Budget Office. (2005). A potential influenza pandemic: possible macroeconomic effects and policy issues. Congress of the United States, Washington D.C. 43 Cooper S. (2006). The avian flu crisis: an economic update. BMO Nesbitt Burns. 44 Akinwumi J, Okike I, Rich KM. (forthcoming). Analyses of the poultry value chain and its linkages and interactions with HPAI risk factors in Nigeria. Africa/Indonesia Team Working Paper, Pro-poor HPAI Risk Reduction Project, Nairobi. 82 lost, plus the value of eggs foregone while the layer operation is rested and before layer production can be restarted. Eggs destroyed are valued at the representative 2006 market price of US$ 2.90/kg (ibid.) and assuming an egg is 47 g. Foregone egg production is valued in terms of the gross margin foregone, assumed to be 10% of the market price (Amos 2006); this recognizes that the other inputs, especially feed, will be diverted to other uses during the resting period and so will not be lost. Numbers of eggs foregone are estimated based on: time to point-of-lay is 5 months; the laying cycle is 11 months during which a layer produces 280 eggs, increasing from 10 to 30 during the first 3 months and falling off from 30 to 10 during the final 4 months; farms are rested for 3 months after an outbreak before resuming production. ? Public veterinary services are assumed to continue culling birds at a cost of US$1/bird, which is applied to the number of incremental chickens culled. ? Compensation is not factored into the counterfactual scenarios because it is assumed discontinued after 2006 due to public budget constraints. Also importantly, it is a transfer rather than a loss. Indirect costs: A multiplier of 1.24 based on results reported by Diao et al. (2009) is applied to the sum value of direct costs to estimate associated indirect costs, which would include production and consumption losses in related sectors across the economy. Production losses in other sectors result from reduced demand for the principal inputs used in poultry production, including feed, pharmaceuticals and transport services. Reduced returns to factors used for poultry production also translate into lower investment, income and employment, which again reduce demand for consumer and investment goods more generally. Diao et al. (2009) found the consumption linkages to be stronger than those on the production side for the Nigerian poultry sector. Incremental mortality: Epidemiological parameters were estimated based on analysis of outbreak data from Nigeria and the historical experience in sub-Saharan Africa (see annex). These were used to estimate expected average annual mortality under each counterfactual scenario. Table 9. Risk parameters by scenario Scenario Risk of introduction Risk of spread Risk of mortality Medium case 0.05 0.27 0.02 Worst case 1.00 0.27 0.02 Table 10. Chicken mortality under the various scenarios (1,000 birds) 2006 2007 2008 2009 2010 Actual Died due to disease 552 112 2 0 0 Culled 440 276 3 0 0 Total 992 388 5 0 0 Medium Case Died due to disease 552 112 19 19 19 Culled 440 276 16 16 16 Total 992 388 34 34 34 Incremental losses Died due to disease 0 0 17 19 19 Culled 0 0 13 16 16 83 Total 0 0 30 34 34 Worst Case Died due to disease 552 376 376 376 376 Culled 440 372 372 372 372 Total 992 688 688 688 688 Incremental losses Died due to disease 0 264 374 376 376 Culled 0 96 369 372 372 Total 0 300 683 688 688 Results Tables 11 and 12 present the results of the benefit-cost analysis. Table 11. Flows of discounted incremental costs and benefits, by scenario 2006 2007 2008 2009 2010 TOTAL Incremental Costs 20.0 6.0 9.3 4.8 0.8 41.0 Incremental Benefits Best case scenario: disease burnt out 0.0 0.0 0.0 0.0 0.0 0.0 Medium case scenario: risk of epidemics 0.0 0.0 0.4 0.4 0.4 1.2 Worst case scenario: endemicity 0.0 2.6 9.2 8.3 7.4 27.4 Table 12. Cost-benefit indicators by scenario Scenario Benefit: Cost Ratio Net Present Value Best case: disease burnt out 0.0 -41.0 Medium case: risk of epidemics 0.0 -39.8 Worst case: endemicity 0.7 -13.6 The BCR is a unit-less measure of the returns expected per dollar of investment. An investment is considered attractive or efficient if it offers a BCR greater than or equal to one, i.e. if it at least pays back the investment. The NPV measures the net return on the investment in 2006 dollars. A viable investment would exhibit a NPV greater than or equal to zero. Under all scenarios, these measures indicate that the WB investment in strengthening HPAI control in Nigeria did not generate sufficient benefits to justify the cost. To achieve at least a break-even level of benefits would have required a much higher incidence or risk of HPAI than could be reasonably assumed to have occurred in the absence of the WB-funded activities. The results are insensitive to the discount rate applied: reducing the discount rate from 12% to 5% only marginally increases the BCR for the worst-case scenario from 0.7 to 0.8. Discussion The results are considered robust. The low rate of return estimated by the analysis suggests that the overall result would be relatively insensitive to changes in the technical assumptions underlying the calculations. Moreover, simulated annual mortality and losses incurred under the different scenarios appear to be reasonable when compared to the levels experienced in 2006 and 2007. What then explains the apparent discrepancy between these results and the projections cited in advance of the 84 investment? As noted in that earlier discussion, previous projections all assumed much higher impacts in terms of the size of the expected outbreaks. Such projections reflected the poor understanding of the disease at the time and can be considered valid in the spirit of the precautionary principle. Fortunately, the reality has been that HPAI was not as dangerous as feared; although it certainly caused substantial losses (1% of the national flock), it did not cause the massive losses that had been predicted, including the 30% losses originally assumed to justify the WB credit. Is it valid to attribute the poor economic performance of the investment to the design of the analysis? Should a counterfactual scenario consider no HPAI control at all, under which HPAI would have spread out of control, rather than considering only the marginal strengthening of HPAI control efforts directly attributable to the WB credit? This is considered unreasonable given that the FGN was already aggressively pursuing control measures, and would have inevitably continued to do so, albeit with limited resources. Also, importantly, private actors within the value chains also react to the presence or threat of disease, and their independent changes in behaviour inevitably dampened the spread of HPAI. However, it may have been that the aggressive control pursued by the FGN from the beginning of the outbreak was made possible because of the early WB commitment made in March 2006. There were suggestions that the availability of advance funding from other existing WB projects even before the first disbursements from the NAICP account strengthened the initial response significantly, although no evidence of this link was found in the NAICP M&E documentation. If this is true, there would be a case for considering yet another counterfactual epidemiological scenario that assumes a much larger initial outbreak under a much weaker FGN response, and which would have yielded more attractive financial performance results. For the purposes of the analysis we have limited the underlying assumptions to those clearly supported by evidence, choosing to err on the conservative side rather than inflate the results. In the end, however, while a better result in terms of investment performance would have been desirable, it is not necessarily an appropriate measure for this type of loan. First, as already alluded to, the loan was made as an emergency measure in a context in which little was known about the nature of HPAI, especially in Africa. The loan should therefore be evaluated on the basis of the precautionary principle, rather than (with the benefit of perfect hindsight) as a financial investment required to generate a certain level of return. Indeed, when some non-negligible risk of a human pandemic enters the equation, the potential returns are very likely to justify levels of investment such as the emergency loan to Nigeria. Finally, this type of analysis does not take into account potential longer-term benefits of the NAICP-managed investments which were beyond the immediate HPAI focus of the loan; improved laboratory diagnostic capacity, human capacity developed through training, and the institutional experience with joint veterinary-public health response management will likely serve to improve control of future outbreaks of emerging diseases than otherwise would have been the case. The evaluation concludes that the WB emergency credit performed poorly as a financial investment, registering a BCR of less than one and negative NPVs under a range of scenarios, assuming the risk of a human pandemic is not factored in. However, such indicators are of questionable relevance when application of the precautionary principle is deemed appropriate. 85 7.4 Impacts In the sections above on effectiveness, many of the areas of impact have been highlighted. Here the key overall impacts are summarized and discussed. 7.4.1 The control of HPAI and the minimizing of human health implications A major finding is that there is no direct evidence that project activities were responsible for bringing the outbreak under control: indeed many project activities occurred after the outbreaks, including training and provision of equipment, materials and facilities. But these findings raise an issue that has constantly confronted the evaluation team, which is when did the clock start in terms of the NAICP and WB funding? On paper it commenced with the advance of US$ 10 million from other projects in June 2006 (just towards the end of the first wave of HPAI outbreaks), and was fully operational in September 2006 when the project?s own funds began to be dispersed. But clearly the country was not holding its breath for this money, and interventions were going on at various levels. There has also been confusion about what the project funds paid for ? the evaluation has focused on the activities reported in the indicator monitoring, but it is unclear if project funds were also paying already for the surveillance, culling and public awareness, training, etc. Did the loan fund core activities early on, as well as the fuller set of activities later on? So the NAICP undoubtedly made a contribution to HPAI control and to minimizing human health implications, but a cause and effect relationship has not been demonstrated. Project activities also provided a degree of protection during a period of perceived high vulnerability to HPAI introduction. 7.4.2 Assigning strength and authority to national institutions The WB credit provided quite a different landscape to HPAI control than occurred in many other countries affected by HPAI. It placed the opportunity to perform squarely in the hands of the FGN, so strengthening Nigerian national institutions on an issue of global public health concern. Many countries struggled to coordinate the stampede of international assistance offered by a plethora of aid and development agencies; the WB credit kept the FGN squarely in the driving seat. 7.4.3 Credibility of the Federal Government of Nigeria The WB credit provided the opportunity for the FGN to perform, and the evaluation concludes that the FGN indeed picked up that challenge. Although not universally endorsed, the FGN rose to the occasion, and has emerged with international credibility for its handling of the crisis. This does not mean that all activities were faultless and that the FGN is now able to unilaterally respond to new animal and human health emergencies, but it has demonstrated the capacity for leadership. This has been particularly reflected in the following activities: ? The establishment of a poultry compensation scheme. The development and implementation of the first nation-wide compensation scheme in support of a stamping-out programme was widely considered to be transparent and effective. Transition to payment of market rates is credited with improving willingness and speed of farmers to report suspect cases. ? The active decision not to engage in a national poultry vaccination programme. ? The NAICP, together with others, succeeded in developing and delivering messages to key audiences which improved knowledge and led to some beneficial changes in behaviour. One important outcome of this was the recovery of the poultry sector due to restoration of public confidence in consumption of poultry. ? The international transparency in HPAI reporting and the placing of virus typing results in the public domain. 86 ? The elimination of HPAI outbreaks (whether attributed to the NAICP or not). 7.4.4 A template for inter-ministerial collaboration in the health arena The NAICP is widely credited with bringing about closer collaboration between human and animal health authorities in Nigeria. It provided a case study on demand-driven inter-ministerial collaboration to respond to a major public health threat, which has undoubtedly helped to set the scene for ?One Health? initiatives in the future. The establishment of desk officers at state and local government levels was a successful and effective strategy. This was manifest in the effectiveness and speed of the response to the Pandemic H1N1/2009 (swine flu) outbreak, which clearly benefited from the project?s investments and ongoing support. 7.4.5 Strengthened national capacity in surveillance and diagnosis The NAICP has made substantial contributions to national disease surveillance and diagnosis, in particular with respect to the influenzas. On the animal health side, this was complemented by independent initiatives taken by NVRI, Vom, and on the human health side, by substantial support from CDC, Atlanta and WHO. The evaluation team concluded that the overall impact of the quarantine station improvement was minimal. 7.4.6 Impacts on the poultry sector value chains The NAICP appeared to have non-quantitative impacts in maintaining public confidence in the poultry industry, considered as loss-averted benefits rather than gains. Investment in infrastructure and some training have played valuable roles in improved cleanliness and hygiene in improved markets, but less so elsewhere. Group cohesion and management among the value chain operatives has changed little, except among traders operating in the improved LBM. While some of the other operatives continue to belong to larger organizations such as the national poultry producers of Nigeria, collective action at the local level was not widespread. No significant change in business structure or practices due to the HPAI control efforts was identified, other than the small number of improved LBM that are being piloted. Contracting arrangements, for example, have not appeared in response to perceived demand for biosecurity. Nor has the experience with HPAI and its control appear to have spawned new types of collective action to coordinate biosecurity, encourage compliance, or offer cost sharing, training and exchange of ideas. 87 7.5 Sustainability The sustainability of products, outcomes and partnerships resulting from the NAICP is far from straight forward, and much of it is questionable. In the first case, this project was designed as an emergency response to tackle a major national (and potentially regional or global) public health scare, and so in general was not designed to have long-term sustainability features. A few such features emerged as the project progressed, such as the pilot LBM construction, but the major thrust was ?fire-fighting?. So how sustainable will the ?fire-fighting? capacity established be? 7.5.1 Inter-ministerial collaboration. In theory this should be an area in which sustainability should be promoted by the FGN, as it has great potential for the future management of public health emergencies and longer term development opportunities under a ?One Health? umbrella. However, it will likely require special effort to sustain following the closure of the project. This is for two main reasons: firstly that the relationship was never fully established with the FMoH and FMoARD operating from separate buildings, and secondly that much of the success was built on personalities, which will inevitably change. 7.5.2 Disease response mechanisms Sustainability of the improvements in field surveillance and laboratory diagnosis capacity, as well as the performance and continued availability of rapid response teams has not been secured. There is a high turnover of staff which means that many trained staff are no longer at their posts and budgets for continuous training are limited. Funding to ensure continued access to the expensive reagents needed for the improved molecular diagnostic techniques and facilities for correct storing of reagents in the face of frequent power cuts are also uncertain. Plans for how desk officers will be utilised when project funding ceases are still being considered in many States. 7.5.3 Biosecurity in the poultry value chain The biosecurity measures necessary for effective HPAI containment and prevention at household, farm, market and consumer level have much wider efficacy implications for broader public health and food safety. However, given that they were mostly very poorly adopted, they are highly unlikely to be fostered without the threat of major disease epidemics driving the communications necessary to influence behaviour and good practices. The major drivers are therefore likely to be improved opportunities in market access, and consumer demand for safer and healthier livestock products. The pilot LBM initiative indicates that there is indeed a market for infrastructures which support more hygienic marketing conditions, and this deserves follow up. 7.5.4 How much should emergency responses stray into broader and longer term capacity development? The evaluation team lauds the focus given within the NAICP to address the HPAI crisis as it appeared in 2006. However, as we know with hindsight from this crisis, and also with hindsight from the subsequent global H1N1 emergence, the realities were different and as far as Nigeria is concerned have been considerably less severe than anticipated. It is therefore important to ensure that resources and programming are indeed adequate to resolve the crisis, but that they achieve an appropriate balance with broader development targets. This is easy to say with hindsight. Projects of 88 this magnitude do not come along often45; indeed they are rare. With this project there was a string of associated development areas which have not been influenced by this project. On the livestock disease diagnosis and control front this includes ND, for which the capacity to diagnose, understand and control has barely changed, despite it being the major cause of losses in the poultry enterprises of the country. On the human health side this includes diagnosis and surveillance of malaria, which while registering some benefits from the NAICP, still remains under reported and inadequately controlled in Nigeria. In retrospect, it seems that an opportunity to give greater attention to broader generic capacity building in the health surveillance and response mechanisms of the country was missed; arguably now is the chance to build on the strengths of the NAICP. 45 The NAICP, at US$50 million, was by far the largest investment from the GPAI programme. The next largest, at US$10 million, was Vietnam 89 8 Recommendations Based on the findings of this independent impact assessment, the evaluation team makes the following general recommendations. As part of the independent assessment of the animal and human health laboratories, a specific set of recommendations has been made, and this is found in Annex 4. 8.1 Clarity of project document formulation The evaluation team recognises the haste with which the project proposal was prepared in March 2006, and the team has acknowledged that the key ingredients appeared to be present in the proposal submitted. However, the team recommends that future animal disease control projects developed at a national level would benefit from the following: ? In the PAD, PDOs should be made much more explicit in the proposal; in the NAICP they were poorly articulated and inadequately substantiated. ? Better structuring and separation of the key essential components of a national response (for example based on the pillars used in the evaluation framework, identifying policy, planning and coordination, laboratory capacity, surveillance, response, prevention, industry support and/or recovery). ? Within each component, a clearer presentation of objectives, outputs, outcomes and indicators, facilitating provision for a much more rigorous assessment of progress, achievements and impacts. ? An impact pathway, showing how outcomes are designed to lead to impacts can be a valuable planning tool. 8.2 Monitoring and evaluation For a project of this size and complexity it is essential that a strong M&E system is in place. The evaluation team recommends: ? Much greater consideration be given by WB to the type of capacity, technical skills and data requirements for M&E of large and complex projects in which multiple agencies of government are involved at Federal and State levels. ? An emergency programme by definition operates in a context of uncertainty, so in addition to conventional M&E, capacity and flexibility is needed to learn and to adapt as experience is gained and new information becomes available. In a disease outbreak situation, such a project requires more direct access to, and involvement of, experts (see point further below about epidemiology expertise in particular) and partners who continuously review the relevance and appropriateness of the response strategy. 8.3 Procurement This project brought together two complex bureaucracies ? the WB and FGN. The incompatibility of these and the overload of demand on the FGN caused several delays and undoubtedly affected project deliverables and outcomes. The evaluation team recommends: ? Much better management and coordination of all procurement procedures be built in to projects from the outset, incorporating relevant staff, training and communications facilities. 90 8.4 Epidemiology capacity The project missed opportunities in the development of greater capacity and skills sets in the field of veterinary epidemiology. The evaluation team recommends: ? For both the WB in future animal health projects, and the FGN in animal health programmes, greater attention should be given to the standardisation of epidemiological data handling and management procedures, the development of risk-based approaches that can feed into surveillance and intervention strategy development, and the development of host population demography and characterisation data that form key denominators for analysis. 8.5 The greater consideration of sustainability issues in WB emergency initiatives The evaluation team, while understanding the focus introduced by emergency response funding for specific public health crises, recommends: ? Greater consideration should be given to broader and longer term sustainability issues when developing emergency funding responses. This is particularly relevant to broader generic capacity to address other animal and human health constraints of priority to the borrower, and also longer term capacity needed to ensure sustainability of focussed measures supported. 8.6 Inter-sectoral collaboration The inter-sectoral collaboration model developed is commended and it is recommended that consideration be given to adopting this as a platform for addressing other problems at the intersection of animal and human health, such as food-borne disease and zoonoses. 8.7 Surveillance systems The animal and human health surveillance systems, while demonstrating notable improvements, both seem to be only detecting a fraction of notifiable diseases. It is recommended that these be reviewed with the consideration of improving their performance. 8.8 Laboratory support Laboratory improvements have been substantial. However, without continued attention, these benefits are likely to be short lived, and the evaluation team recommends that FGN seeks continued support to ensure their sustainable functioning. 8.9 Inbuilt flexibility in responding to lessons learnt during a project The NAICP gained in both effectiveness and efficiency by making certain adjustments during the implementation of the project, ensuring that they were agreed by both parties. The evaluation recommends that such flexibility, within well articulated boundaries agreed prior to the initiation of projects, should be standard in such emergency support initiatives undertaken by WB and/or FGN. 91 ANNEX 1. EVALUATION TEAM Brian Perry, Team Leader: Professor Perry is a international consultant with a speciality in the roles of livestock in processes of pro-poor economic growth and the constraints imposed on these by poor animal health, and in leading independent evaluations. He is a visiting Professor in the Department of Medicine at the University of Oxford, an Honorary Professor at the University of Edinburgh, and an Extraordinary (Honorary) Professor in the University of Pretoria, South Africa. He has recently led a series of independent evaluations of UN programmes; during 2009 he led the evaluation of FAO?s global avian influenza programmes in nine countries of Africa, the Middle East and Asia, and in 2010 he led an evaluation of all the UN?s agricultural programmes in Ethiopia over the last five years. He lives in the Rift Valley of Kenya. Tom Randolph, Evaluation Manager: Dr. Randolph, with a PhD in Agricultural Economics from Cornell University (USA), has extensive experience in the economics of animal health and has been managing ILRI?s research efforts related to HPAI. His areas of expertise include policy analysis and impact assessment. Bernard Bett: Dr. Bett has a PhD in Veterinary Epidemiology from the University of Nairobi (Kenya) and has undertaken research on vector-borne diseases such as trypanosomosis and Rift Valley Fever, and on HPAI. He contributed to a major vaccination trial for HPAI conducted in Indonesia. He has also been involved in the development of surveillance strategies employing participatory epidemiology techniques. Delia Grace: Dr. Grace has a PhD in Veterinary Epidemiology from the Free University of Berlin (Germany) and leads research at ILRI on agriculture-associated diseases of livestock, especially with respect to zoonoses, emerging diseases and food safety. She has particular expertise in developing participatory risk analysis to improve food safety in local markets in less developed countries. Keith Sones: Dr. Sones has a PhD in veterinary parasitology and a background in the veterinary pharmaceutical industry. He is a freelance livestock consultant based in Kenya since 1982 and works for a broad spectrum of public and private sector clients undertaking a wide range of writing assignments. Joerg Henning: Dr. Henning has a Diploma in Tropical Veterinary Medicine (Germany) and PhD in Veterinary Epidemiology from Massey University (New Zealand). He has designed, implemented and analysed various epidemiology studies in Africa, South East Asia and Central Asia and published extensively in international journals. He was, until recently, a Senior Research Fellow in Veterinary Epidemiology at the University of Queensland (Australia). Anja Globig: Dr. Globig is a certified veterinary virologist. Since 2002 she is employed at the Federal Research Institute for Animal Health, the Friedrich-Loeffler-Institut (FLI), in Germany. she has been involved in laboratory diagnosis at the OIE Reference Laboratory for AI and ND for more than 7 years. She is currently a member of the International Animal Health Team and is active in capacity building of laboratories in developing countries. Pamela Pali: Dr. Pali is part of the Poverty, Gender and Impact team at ILRI and is taking the lead on developing tools and instruments for improving assessment of research-for-development impacts. 92 She is currently implementing an impact study of USAID investments in improved surveillance for HPAI, including those related to strengthening participatory epidemiology capacity. Jane Poole: Ms Poole has an MSc in Biometry (Medical, Agricultural and Biological Statistics) from Reading University (UK). She has worked for over 13 years in international agricultural research with the World Agroforestry Centre (ICRAF), CAB International?Africa Regional Centre, and ILRI, where she leads the joint ILRI/ICRAF Research Methods Group. Her main activities include statistical data analysis, reporting of statistical analyses, including contribution to and reviewing research publication, support to research quality systems, project and study design, data management, training and capacity building work areas. Mahomadou Fadiga: Dr. Mohamadou Fadiga is an agricultural economist based in ILRI, Addis Ababa. His research areas are on demand and sector modelling and economic impact of livestock diseases. He holds a PhD in Agricultural and Applied Economics from Texas Tech University. His work has been published extensively in various peer-reviewed journals. Heather Hannah: Dr. Hannah is a veterinarian with an MSc in Epidemiology and Environmental Health from the Colorado State University (USA). She has several years of experience as a field epidemiologist for the Canadian public health service, and more recently as a consultant in East Africa has been contributing to capacity building efforts for participatory epidemiology. Ekanem Ekanem: Prof. Ekanem is a Professor of Epidemiology at the College of Medicine, University of Lagos. He had his training in Public Health from the University of Texas, School of Public Health where he earned the MPH and PhD degrees (Epidemiology). His further training as a WHO Fellow was at the Universit? Libre de Bruxelles, Ecole de Sant? Publique, Belgium, where he earned a Masters degree equivalent in Statistical Methods. He has expertise in epidemiology, biostatistics, population, HIV/AIDS and influenza with specialist knowledge on Botswana, Nigeria and Somalia. . Paul Abdu: Dr. Paul Abdu is veterinarian and professor based in Ahmadu Bello University Zaria, Nigeria. His research areas are in the epidemiology of infectious diseases of poultry and ethnoveterinary medicine. He holds a PhD in Veterinary Medicine from Ahmadu Bello University Zaria, Nigeria. His work has been published in various peer-reviewed journals. Christopher Chukwuka Molokwu: Dr. Molokwu has a PhD in Development Economics from the University of Benin (Nigeria). He has an extensive experience in monitoring and evaluation of donor assisted agricultural development projects. He has been an economic analyst for project preparation, mid-term reviews and project completion reports of several donors (World Bank, AfDB, FAO), and has assisted agricultural development projects in North, Central and West Africa. 93 Annex 2: ANIMAL HEALTH COMPONENT Summary This study was implemented between January and April 2011 and it evaluated the impact of response measures that were implemented under the Animal Health component of the Nigeria Avian Influenza Control Project (NAICP) against highly pathogenic avian influenza (H5N1 virus). Where possible, the study focussed on the measures that were financed through the World Bank emergency funds. A list of these activities and the level of their implementation, as described in the implementation status overview report No. ISR1317, include: ? Preparation of the national preparedness plan involving all the 36 states ? Up to 250 radio and 12 television documentaries were relayed in Ibo, Hausa, Yoruba and other 28 Nigerian languages ? Four selected strategic international quarantine stations (Jebba, Makurdi, Seme and Ikom) were renovated and quarantine staff, importers, and private veterinarians were trained on HPAI detection ? A total of 15 live bird markets were also improved and unspecified number of farmers and fowl sellers was trained on biosecurity practices ? 70% of surveillance and livestock development officers were trained in awareness raising, monitoring, investigation, sampling, safety and test procedures. ? The project targeted reducing the response time from 168 to 48 hours. A large number of poultry were culled as part of the response measures. Compensation was provided initially at a fixed rate of N250 per bird but market rate compensation was provided later when the WB funds emergency funds became available. The assessment of the laboratory renovations and trainings offered as part of the animal health interventions are presented in a separate report. The study and the evaluation criteria and the study framework were developed following a consultative meeting with key project staff in January 2011. Secondary data showing the number of cases observed with time and the interventions that the project implemented were obtained from the project office in Abuja and desk officers at the state level. Primary data were collected through questionnaire surveys in 8 states (2 from each of the 4 HPAI hubs). The main observations made under each sub-component are: Strengthening control and eradication There is no clear evidence that the project activities had an impact on HPAI incidence in 2006 and 2007. It is not clear whether the rapid decline in the incidence of the disease in 2008 was due to the project activities or natural outbreak decay. The risk of an LGA becoming infected in 2006 (6%, 95% CI: 4 - 8%) was not significantly different from that observed in 2007 (4%, 95% CI: 3 ? 6%). Similarly, the transmission rates of the disease between villages were similar between the two periods (24%/day and 23%/day in 2006 and 2007, respectively). Disease surveillance and diagnosis The supervisors of the disease surveillance staff interviewed indicated that surveillance teams have had numerous courses on very relevant topics such as disease tracing, reporting and sample collection. Surveillance teams had also participated in monitoring other diseases, mainly African swine fever and swine flu. However, the rate at which these staff get replaced (2-4/year) is quite 94 high and it is feared that efforts that the project has made to build capacity on surveillance may not be sustained unless plans are made to offer such trainings periodically. PDS data is not effectively used. PDS data exists in a database separate from the conventional NADIS network and is not analyzed. There is little evidence for the data leading to actions, decisions or changes. PDS did not improve surveillance. PDS successfully engaged various veterinary sectors. However, PDS did not result in increased reporting of any disease despite endemic prevalence estimates for at least two notifiable diseases (ND and CBPP). The emphasis of PDS for surveillance was limited to case finding for HPAI. This is consistent with the best potential value of participatory surveillance as suggested by PE practitioners, namely case finding during outbreaks. Community involvement in case finding during outbreaks would not alter the extractive nature of surveillance. PDS was useful for appraisal or as a survey of animal health concerns. Village visit discussions yielded important community input which informed vaccination campaigns, actions which would not have happened in the absence of the village visits from the PDS project. Strengthening veterinary quarantine services No data on the volume of trade of live birds is collected in the two upgraded quarantine stations. Therefore the impact of movement control could not be assessed. The facilities of the quarantine stations had been upgraded but these facilities seem to decay rapidly. No active disease surveillance was conducted at the time of the assessment in the two upgraded quarantine stations and revenue collection by local government councils has prevented quarantine staff from effectively discharging their functions due to lack of cooperation by livestock traders. The HPAI epidemics and the NAICP interventions had some influence on the creation of the National quarantine services (NAQs?). However, the improvement of quarantine stations was completed between October 2008 and June 2009, therefore they had no impact on the HPAI risk. Enhancing HPAI prevention and preparedness capacity The self-assessed performance of trained infected premises response teams seems to have marginally improved over the project period but this can?t be solely related to the project efforts as training was also provide through other organisations. No similar trend was observed with surveillance or special diagnostic teams. Apart from HPAI, other diseases that these teams responded to include African swine fever and Newcastle disease. Improving biosecurity in poultry production and trade Training of farmers on bio-security practices was delayed (started in late 2007). In addition, trained farmers were better than untrained farmers on only 2 out of 13 biosecurity measures assessed. The changes observed, even if they had been implemented at the right time, would not have been sufficient to reduce the risk of the disease. Improved markets had better biosecurity measures than traditional markets. The improvement of the live bird markets was only completed between mid-2009 and mid 2010 when the outbreak had stopped. With the exception of fowl sellers in improved LBMs, training of fowl sellers did not improve bio- security practices of poultry handling. Bio-security of poultry handling was largely driven by the type of poultry market (traditional markets have riskier poultry handling practices while improved 95 markets conduct practices of lower risk in regards to HPAI spread). High-risk trading practices (selling sick poultry or slaughtering sick poultry) were conducted by both, trained and untrained fowl sellers. Compensation and economic recovery The compensation payments were taken over by project on the January 2007 and this resulted in higher compensation amounts paid per bird. Time between noticing outbreaks by farmers and reporting outbreaks to veterinary authorities shortened from a median of 6.5 days (with a range of 0 ? 180) in 2006) to 1 (range 0 ? 264). Improved compensation rates might have given farmers more incentives to report but it is unclear whether the observed reduction in time to reporting was enough to influence the transmission dynamics of the disease. Depopulation rates increased in 2007 and depopulation efforts based on diagnostic test results increased after the project took over. However, this can?t be solely contributed to project efforts; increased experience of field response teams could have been another influencing factor. Methodology Two key meetings were held in the course of development of the evaluation framework (Figure X) that guided the development of the animal health impact assessment methodology. The first was a preparatory meeting held between the evaluation team and NAICP project staff at Abuja on 19th to 21st January 2011 and the second was a planning workshop held by the evaluation team at IITA, Ibadan on 22nd to 31st January 2011. The first meeting allowed the team to identify various interventions implemented by the project that could be evaluated. In the second meeting, the team identified and designed a number of studies including steps that could be taken to assess the interventions described by the NAICP staff in the first meeting. These included: (iii) Analysis of secondary data provided by the project (iv) Field surveys involving poultry farmers, fowl sellers, surveillance and response teams, participatory disease surveillance teams and quarantine officers Analysis of the secondary data Two follow up visits were made to the NAICP headquarters, Abuja to collect related project documents and secondary data. The information and materials provided by the project included: ? List of farmers, fowl sellers and transporters trained on biosecurity practices. The training venue and dates were also indicated ? List of quarantine stations that were upgraded, their locations, date of the upgrade and completion date for each upgrade ? Minutes of steering committee and technical committee meetings. However, the minutes provided covered a short period between end of June 2006 and May 2007, ? List of live bird markets upgraded Through the NAICP project the following data was requested from the state desk officers and provided to the evaluation team: ? Records showing quantities and types of sampling materials and disinfectants supplied to each state, dates when each item was supplied and current status of the materials received, ? Data on surveillance activities conducted since 2006 in each state indicating dates for each surveillance activity, locations visited, whether samples were collected, date on which the samples were shipped to the lab, when results were obtained by the states and the diagnosis made, 96 ? Individual HPAI outbreak datasets on state level indicating the local government area affected, GPS coordinates of the outbreak sites, dates when (a) an outbreak was noted by farmers, (b) an outbreak was reported to an official, (c) samples were send to the laboratory, (d) laboratory results were received by the reporting official, (e) the affected premise was quarantined (f) depopulation was conducted. Furthermore individual datasets contained information on the confirmed diagnostic test results, the flock size of affected premises, the number of birds dying and the number of birds destroyed. ? Compensation manuals and a dataset on compensation giving similar information as the HPAI outbreak dataset but giving more information at the farm-level including amount of compensation paid out. These data were complied into single datasets, cleaned, corrected and analysed using simple descriptive statistical techniques. In addition, some of the records, particularly those identifying farmers and fowl sellers who were trained by the project on biosecurity, were used to build sampling frames for field surveys. Key outputs obtained from the analyses are described below. Epidemic curve and spatial distribution of the disease Only outbreaks with confirmed HPAI diagnostic test results were used to describe the temporal and spatial outbreak patterns. Most of the records in the compiled outbreak dataset contained outbreak information on a village level. Two epidemic curves were constructed using the HPAI outbreak dataset. The first curve used the date when an outbreak was noted by the farmer as the outbreak date while the other used the date when an outbreak was officially reported. The two curves were used to evaluate visually the timeliness of activities conducted by the NAICP project in relation to the outbreak pattern. The number of reported outbreaks per local government area that occurred between 2006 and 2008 were plotted using ArcGIS. This allowed a quantification of outbreak occurrence on both, a temporal and spatial dimension. The shape file of the local government boundaries was obtained from the IITA GIS unit. (i) Timeline of activities Minutes of Steering Committee meetings were reviewed and summarised in a table indicating the date when meetings was held and key discussions or decisions made at each meeting. Although these meetings focussed broadly on HPAI control in the country, this review helped to illustrate the timing of the various World Bank funded activities such as the introduction of market-based compensation scheme. A similar table showing the timing of the various training activities conducted by the project was also made. Graphs showing dates and quantities of disinfectants and sampling materials received by the states were also superimposed on the epidemic curve to assess whether these supplies were made at the time when they were required for the disease control. (ii) Time difference between events The dates defined in the HPAI outbreak dataset were used to determine the number of days between: o noticing outbreaks by farmers and reporting these outbreaks to veterinary authorities, o noticing outbreaks by farmers and sending of samples for laboratory diagnosis, o reporting outbreaks to veterinary authorities and sending of samples for laboratory diagnosis, 97 o sending samples and receiving laboratory results, o reporting outbreaks to veterinary authorities and beginning of quarantine, o quarantine and depopulation, o reporting of outbreaks to veterinary authorities and depopulation, o noticing of outbreaks and depopulation Also the proportion of depopulations conducted with and without confirmed diagnostic test result was calculated. The distribution of each variable describing the days in-between events was shown in histograms and the mean, median, minimum and maximum, and standard deviation of these variables were calculated using SPSS statistical software. The analysis was stratified by year and half year periods mainly to determine whether there was an improvement in the efficiency of response with time. In addition the median number of days in-between events was plotted per month and year to describe the efficiency of activities over time on a more detailed temporal basis. Description of maximum likelihood estimation methods to determine mean number of days will be done here if we get time to do them (iii) Measures of mortality and depopulation Descriptive statistics on the mortality rate (number of birds dead/number of available on the affected farm or village), the depopulation rate (number of birds culled/number of birds available on the affected farm or village) and the depopulation-mortality ratio (number of birds culled/ number of birds dead) were calculated. These measures were evaluated over time to identify changes in the depopulation efforts. (iv) Compensation The dataset on compensation contained records on farm level. The number of farms reporting outbreaks, being depopulated and receiving compensation was shown graphically per month and year to identify patterns of events occurring on a farm level. Descriptive statistics on time between compensation and noticing outbreaks by farmers, reporting of outbreaks to officials and depopulation were calculated per year, half-year and per month. This allowed identifying delays in the payment of compensation over time. Furthermore, descriptive statistics on the compensation amount per bird and farm over time was calculated to identify changes in the compensation values per bird received. (v) Quantification of transmission rate of the disease The rate of transmission of HPAI between local government areas for each epidemic wave was estimated as described by Ward et al. (2009) and Stegeman et al. (1999) using the HPAI outbreak dataset. Most of the cases in this dataset are summarized on a village level (although few cases might have occurred on farm level and need to be summarized on village level, when village level information of each farm becomes available. So the unit of analysis has been assumed to be a local government area (LGA). At each epidemic day, villages were classified as either being susceptible (S), infected as a new case, Ct or running an unresolved case, hence infectious, It. At the beginning of the outbreak, all LGAs were assumed to have poultry. It was also assumed that most of the cases were reported and were resolved through depopulation and that the rate at which new cases occurred 98 depended on the ratio between susceptible (S/N) and infectious units (I/N). The epidemic curve constructed from these data showed that the outbreak could be classified into three phases; a) between January and May 2006, b) between November 2006 and November 2007 and c) in July 2008. A transmission coefficient was estimated for the first two phases of the epidemic -- the last one was ignored because it did not pick up. Given that some of the key World Bank funded interventions, such as the provision of market-based compensation rates, were implemented after the first phase of the epidemic, it might be possible to assess the impact of these interventions by comparing the magnitudes of the transmission coefficients between local government areas using the first one as the reference. This approach was used by Stegeman et al. (2009) to assess the impact of control interventions against an outbreak of classical swine fever in The Netherlands in 1997 - 1998. In their case, however, no specific interventions had been made in the first phase of the epidemic. Field surveys Field surveys (implemented between 14th and 28th February 2011 with enumerators trained between 1st and 7th February) mainly assessed the impact of training activities that the project implemented. Additional information on the number of HPAI outbreaks observed, response activities, information, education and communication messages received were also collected. These surveys involved: (a) sector 2-3 farmers and fowl sellers who had been trained on biosecurity practices, (b) surveillance, diagnostic, response teams who had received training on surveillance and diagnostics, and (c) quarantine station officers who had had trainings on surveillance and movement control. Live bird markets and quarantine stations that had been improved by the project were also assessed. A list of surveys conducted, target sample size, the number of responses received and the size of the population from which these samples were drawn from is outlined in Table 1. 99 Table 1. A list of surveys conducted, target sample size, the number of responses received and the size of the population targeted in the assessment Field survey Sampling design proposed at the start of the assessment Target sample size Responses received 1 Farmer interviews to assess the impact of training on biosecurity practices 8 states (outlined in Table 1). Within each state, 12 sector 2 -3 farmers were to be selected following the criteria specified in the text 96 82 2 Participatory epidemiological surveys in the villages to determine the level of underreporting of HPAI Within each of the 8 states identified above, 6 villages where farms selected for Survey 1 are located were to be selected for PRA interviews 48 30 3 Assessment of the biosecurity measures in improved live bird markets 5 improved live bird markets were to be identified and biosecurity measures used characterised. 5 neighbouring traditional markets were to be used as controls. Sites for the survey would be dictated by the location of the markets 10 6 4 Fowl seller interviews to assess the impact of training on biosecurity practices Within each market described above, 3 trained and 3 untrained traders were to be selected and interview on biosecurity measures. Sites for the survey would be dictated by the location of the markets 60 52 5 Survey of improved quarantine stations This would involve the assessment of improved quarantine stations and interviews of the officers based there. Two stations proposed were Jebba and Makurdi 2 stations 50 officers 2 Quarantine stations 19 officers 6 Interviews of surveillance teams, diagnostic teams, response teams and infected premises teams to assess level of knowledge and types of activities conducted This survey targeted all the 37 states. Questionnaires were sent on email. 37 states 11 Surveillance teams 11 Diagnostic teams 10 Infected premises teams 16 Desk officers 100 7 Key informant interviews This targeted key stakeholders in the poultry industry to assess their perceptions on the impact of the project activities 5 101 The specific activities conducted in the field are described below. (i) Farmer interviews These interviews were conducted in 8 states distributed across the four HPAI hubs as outlined in Table 2. States were stratified into two categories (high and low risk) based on the number of outbreaks that occurred in each state. Table 2. States used for surveys on farm biosecurity practices Hub High risk Low risk Hub A Ogun/Lagos Oyo Hub B Anambra Enugu Hub C Plateau Nassarawa Hub D Kano Jigawa The survey planned to use a total of 96 farmers, 12 from each state. Half of them needed to have been trained by NAICP on biosecurity measures while the other half would act as controls. It was also expected that at least half (n = 3) of the farmers from each group (of trained and controls) would have had HPAI infection in their farms. The sampling design that was, therefore, proposed for farmer selection in each state had the following categories: Trained ? Infected farms ? 3 Trained ? Uninfected farms ? 3 Untrained ? Infected farms ? 3 Untrained ? Uninfected farms - 3 As stated above, a sampling frame giving a list of trained farmers was developed using records provided by the project. However, most of these records had names of farmers, traders, private veterinarians and transporters mixed together without any characteristics that could be used to group the trainees by occupation. A random sample of 10 names was, therefore, drawn and listed serially for each state so that the enumerators could work through the list in a systematic pattern from the first name onwards until a list of six farmers had been identified. Nearest farms that had similar characteristics as the selected farm were recruited as controls. The questionnaire used collected information on: - Respondent and farm characteristics - Disease control practices including the use of vaccines - Biosecurity practices and characteristics of the poultry houses - Trainings attended and whether there are lessons learnt in these trainings that could not be implemented on the farm and reasons for not implementing them - Information, educations and communication experiences - For farms that had outbreaks, additional information was obtained on: o Outbreak information such as the nature of outbreak, number of birds affected, sold, culled, o Performance of the surveillance teams o Experiences with compensation and restocking 102 o Public health implications, i.e., whether a member of the family was exposed to the disease. (ii) Evaluating biosecurity at LBM Four upgraded live bird markets located in Kano (n = 1), Kaduna (n = 2) and Lagos (n = 1) were assessed using a checklist. The checklist had three sections. The first section required face to face interviews with the manager to describe the market characteristics, biosecurity practices including cleaning and disinfection, slaughter practices, transportation and equipment design. The second section involved a general inspection of the market to verify the information given by the manager and the last section involved interviewing traders to collect perceptions on the level of biosecurity measures that had been implemented in the market. Neighbouring (traditional) live bird markets were used as controls. (iii) Evaluating biosecurity used by fowl sellers Five NAICP project-improved live bird markets and 5 traditional bird markets in the same town (to be used as controls) were planned to be visited. Three fowl sellers who had received training from the project and three untrained fowl sellers were planned to be interviewed in each of the 10 markets. This activity therefore planned to interview 60 traders. The questionnaires used sought information on the respondent characteristics, business characteristics, biosecurity measures, courses attended and lessons learned that could be or could not be implemented in the business. (iv) Surveillance, response and diagnostic officers and their supervisors This survey targeted surveillance, response and diagnostic teams in all the states. Questionnaires were sent to them by email via their desk officers. They were requested to send them back to ILRI, Ibadan by email after filling them out. Follow up telephone calls to the desk officers were made to ensure that they forwarded the questionnaires to the target teams as soon as they received them. Topics covered by the questionnaires included respondent characteristics, trainings attended, team structure and performance, experiences with HPAI outbreaks or suspicions, difficulties and challenges encountered and whether their activities were sustainable. (v) Quarantine stations and the knowledge of quarantine officers Two quarantine stations -- Jebba and Makurdi ? were assessed. In each of these stations, one group interview involving quarantine staff was done. (vi) Participatory disease surveillance (PDS) and knowledge of PDS teams The objective of the current assessment was to evaluate the outcomes of capacity building in PE/PDS on national infectious animal disease surveillance, based on indicators adapted from established surveillance evaluation criteria46, and to develop an evidence base for the appropriate and application of participatory methods for surveillance systems. In total, data was collected from interviews, questionnaires and a workshop from 36 individuals: 25 PE trained representing the public (72%), private (16%) and academic (12%) sectors and 11 key informants from National Animal 46 Perry et al. for FAO, 2009 & 2010; Coker R et al, 2008, 2010 & in press; Rushton & Rushton, 2009; CARE?s community based surveillance models, 2009; CDC surveillance evaluation guidelines, 2001 & 2004; Dufour B, 2009; Hadorn & Stark, 2008; Stark et al., 2006; Meynard et al., 2008; Sosin, 2003; USDA/APHIS Veterinary Services, 2006 & others. 103 Disease Information System National (NADIS), the National Veterinary Research Institute (NVRI), veterinary schools and international organizations. The most significant change technique was used during the workshop to generate experiences of PDS implementation. USAID funded the first two introductory PE courses in October 2008 and February 2009 as well as a refresher course in June 2009 training 20 Nigerian veterinarians. The World Bank funded training for a further 120 veterinarians in October 2010 with a shortened introductory PE course. In groups of up to four, each of the PE trainees from the first two courses conducted 60 village visits while PE trainees from the last course each conducted 30 village visits. (vii) Key informant interviews Five key informant interviews were done to gather perceptions on the impact of the project. Those interviewed included (a) the Secretary General, Poultry Farmers Association, (b) DVS and Desk Officer, Ogun state, (c) President, Fowl Sellers Association, (d) DVS, Lagos state and (c) Chief Veterinary Officer of Nigeria and the Director Federal Ministry of Agriculture and Rural Development. Results Data quality Difficulties in data sourcing are always occurring when outbreaks patterns have to be described or outbreak interventions have to be evaluated many years after the actual outbreak occurrence. Primary data on these issues are very prone to large recall bias; therefore secondary data were an important source of information for our evaluation. As not all secondary data was instantly available by the NAICP project, some of the data had to be requested from the state desk officers. Secondary data files were often received in various formats (e.g. MS Excel, MS Word, jpg files, paper records) and had to be converted and complied into spreadsheet and databases. Time to event analysis was an important part of our evaluation requiring specific dates of outbreaks noted by famers, dates reported by famers etc. However, almost one third of the dates was in incorrect formats and had to be manually modified to allow calculations. Unfortunately the level of outbreak recording (i.e. farm or village outbreaks) was not clearly specified by the state desk officers, but we tried to ensure that outbreaks are compiled on a village level for our analyses. As in any secondary dataset, faulty records due to incorrect data entry had to be excluded. Although the data cleaning was a timely process, we are confident that the final records were accurate to allow adequate epidemiological descriptive statistics and meaningful statistical comparisons and parameter estimations. The data quality of the primary data from surveys that were conducted by trained enumerators was very good (e.g. farmer, fowl seller and market manager interviews). The questionnaire design was discussed with these enumerates during workshops and therefore an adequate data quality was expected and meaningful epidemiological descriptive statistics and comparisons could be conducted. However, primary data requested per email from HPAI diagnostics and surveillance teams were poor in both the actual response rate and the completion quality of the survey questions. This might be due to the fact that no personalised interviews were conducted or due to other reasons that might influence responses given by state employees. Therefore the sample size achieved in these surveys was poor as well as the completion rate for some questionnaire variables. 104 Although descriptive statistics were still possible to conduct, meaningful comparisons supporting strong statistical evidence were very limited for the data from the email surveys. Secondary data analysis The results of the secondary data analysis are presented under the following sub-headings: - Epidemiology of HPAI ? to cover epidemic curves, spatial distribution of outbreaks and estimation of transmission parameters, - Response measures ? including surveillance, distribution of disinfectants and PPE, depopulation, training, - Timeliness of responses ?for example time to reporting, quarantine, depopulation, etc., - Other information including postgraduate training, steering committee meetings. Epidemic curves Two epidemic curves were generated to study the temporal pattern of the outbreak and evaluate the timing of the various interventions implemented by the project. The first one considered dates when farmers first noticed the disease on their farms as the incidence dates (Figure 1) while second used dates when farmers formally reported the outbreaks to the response unit (Figure 2). The numbers of outbreaks used for each curve were 222 and 263, respectively. Figure 2. Temporal pattern of the HPAI epidemics in Nigeria using dates when farmers noted outbreaks on their farms as the incidence months. 0 5 10 15 20 25 30 35 40 De c Fe b A p r Ju n A u g O ct De c Fe b A p r Ju n A u g O ct De c Fe b A p r Ju n A u g O ct De c N u m b er o f o u tb re ak s Month/Year 2006 2007 2008 105 Figure 3. Temporal pattern of the HPAI epidemics in Nigeria using dates when farmers reported outbreaks on their farms to veterinary authorities as incidence months. The two curves are similar though the first suggests that the epidemic could have started a few months earlier than was officially reported. Also a delay between noting outbreaks by farmers and reporting HPAI outbreaks to veterinary authorities is noticeable. Proportion of states affected by HPAI outbreaks A total of 16 states representing a proportion of 43.2% (Exact 95% CI: 27.1 ? 60.5%) were affected by the outbreak in the first year (2006). This increased to 22 in 2007 (59.5%, Exact 95% CI: 42.1 ? 75.3%) and declined to 2 in 2008 (5.4%, Exact 95% CI: 0.1 ? 18.2%). Using a state as a unit of analysis, the incidence risk was estimated to be 55.2% (Exact 95% CI: 35.7 ? 73.6%) in 2006 and 56.3% (Exact 95% CI: 29.9 ? 80.3%) in 2007. These measures estimate the risk of a state getting infected and do not represent HPAI risk at village, farm or bird levels. The number of reported outbreaks by state is illustrated in 0 5 10 15 20 25 30 35 40 45 Ja n Ma r Ma y Ju l Se p N o v Ja n Ma r Ma y Ju l Se p N o v Ja n Ma r Ma y Ju l Se p N o v N of o u tbreak s Month/Year 2006 2007 2008 106 Figure . States that had high number of reported outbreaks, in a decreasing order, included Kano, Kaduna, Plateau, Ogun, Bauchi and Lagos. Figure 4 shows spatially the states that were affected by HPAI outbreaks between 2006 and 2008. Proportion of local government areas affected by HPAI outbreaks Using the local government area as a unit of analysis, the incidence risk was estimated to be 6.3% (Exact 95% CI: 4.6 ? 8.3%) in 2006 and 4.4% (3.0 ? 6.1%) in 2007. These measures estimate the risk of a local government area getting infected and do not represent HPAI risk at village, farm or bird levels. Figure 5 shows the number of outbreaks in local government areas that were affected between 2006 and 2008. Figure 4. Number of reported HPAI outbreaks by state in Nigeria between 2006 and 2008. 0 10 20 30 40 50 60 70 Abi a Ad amaw a Akwa Ib o m An ambr a B auch i B ayels a B e n u e Born o Cross R iv er De lt a Ebony i Ed o Ek it i Enu gu FC T Gom b e Im o Jigaw a Kad u n a Kan o Katsin a Ke b b i Kog i Kwa ra La go s Na saraw a Nige r Ogu n Ond o Osu n Oy o P latea u R iv er s Soko to Tarab a Yob e Zamf ar a N of o u tbreak s States 2008 2007 2006 107 Figure 5. States of Nigeria that were affected by HPAI outbreaks between 2006 and 2008 (n=263 outbreaks). 2006 2007 2008 108 Figure 6. Frequency of HPAI outbreaks in local government areas that were affected in Nigeria between 2006 and 2008 (n=263 outbreaks). 2006 2007 2008 109 Transmission rate of HPAI between villages Transmission coefficient of HPAI between villages was estimated using the outbreak data set with 252 outbreaks. Estimates for the first phase of the outbreak (January ? August 2006) was estimated differently from that of the second phase (November 2006 ? November 2007). The second phase represented the period when NAICP was being supported fully by the World Bank. The effects of the changes in the intensification of control measures associated with the World Bank funding, including the introduction of the market-based compensation regime were expected to reduce the transmission coefficient in the second rather than the first phase of the outbreak. The number of susceptible villages was not immediately available but a working figure of 90000 was used. Ward et al. (1997) has indicated that transmission parameter and reproductive number are insensitive to the number of susceptible villages used because at any one time, the number of infectious villages is much smaller than the susceptible ones. Table 3 gives number of newly infected villages, susceptible villages and the daily transmission parameter, ?. An overall transmission coefficient was obtained by averaging the daily ? values. Table 4 gives the results of the analysis. These results show that the rate of transmission of the disease was similar between the 2 phases of the outbreak. Table 3. Number of the newly infected villages (C), infectious villages (I), susceptible villages (S) and the infection parameter (?) for each day of the epidemic N Outbreak date Susceptible New Cases Infectious cases SI/N Beta, ? 90,000 15/11/2006 89999 1 0 0.998711 - 16/11/2006 89999 0 1 0 0 17/11/2006 89999 0 1 0 0 18/11/2006 89999 0 0 0 - 19/11/2006 89999 0 0 0 - 20/11/2006 89999 0 0 0 - 21/11/2006 89999 1 0 0.997423 - 22/11/2006 89998 0 1 0 0 23/11/2006 89998 0 1 0 0 24/11/2006 89998 0 1 0 0 25/11/2006 89998 0 1 0 0 26/11/2006 89998 0 1 0 0 27/11/2006 89998 0 1 0 0 28/11/2006 89998 0 1 0 0 29/11/2006 89997 1 1 0.996134 1.003881 30/11/2006 89996 1 2 0.994845 0.502591 01/12/2006 89995 1 1 0.993557 1.006485 02/12/2006 89995 0 1 0 0 . . . . . . . . . . . . 30/11/2007 89846 0 2 0 0 110 Table 4. Estimates of the transmission rate parameters and Ro for each phase of the outbreak Phase of the epidemic No. of days No. of cases Mean ? Estimated R0 Phase 1a 195 98 0.22 1.32b Phase 2c 381 154 0.21 1.46d a January ? August 2006 b Assuming that the median number of days between noting the outbreak and depopulation is 6 days c November 2006 ? November 2007 d Assuming that the median number of days between noting the outbreak and depopulation is 7 days Response measures Surveillance Figure 6 gives the distribution of surveillance visits made between 2006 and 2010 in 15 states that provided the data. These states included Anambra, Bauchi, Bayelsa, Borno, Rivers, Enugu, Gombe, Kaduna, Katsina, Lagos, Nasarawa, Ondo, Oyo, Taraba, Yobe and Ogun. Kaduna had the largest number of visits (data not shown). The figure shows that surveillance visits in 2009 were unevenly distributed. There was minimal activity between January ? May while visits clustered between September ? December. This uneven distribution might have been induced by delays in mobilization of funds or other logistical constraints. Fourteen percent of the records did not have laboratory results even though samples had been collected for laboratory analysis. There were also cases (n = 15) that had been described as ?self- depopulated?, implying that farmers had culled their birds before response teams arrived on the farm. Figure 7. Number of surveillance visits made between 2006 and 2010 in 15 out of 37 states. Supply of disinfectants and sampling materials Two sets of data giving quantities of disinfectants, PPE and sampling materials distributed to the states were used in this analysis. The first one was provided by the project and it contained records of supplies made in 2006 ? 2007 to all the 37 states ? these were summarised in Figures 7 - 9. The second set was obtained from the desk officers from the 15 states identified above. This dataset contained supplies made in 2008 ? 2009. These datasets were analysed separately because of the differences in the number of states used in these datasets. The dataset from the desk officers was 0 10 20 30 40 50 60 70 80 90 100 Ja n A p r Ju l O ct Ja n A p r Ju l O ct Ja n A p r Ju l O ct Ja n A p r Ju l O ct Ja n A p r Ju l O ct total visits HPAI +ve HPAI -ve 2006 2007 2008 2009 2010 Month Numbe r of visit s 111 also more detailed compared with that provided by the project because it identified different types of disinfectants received. These graphs showed that disinfectants and PPE were distributed to the states since 2006; the latest supplies were given out in November 2010. It is difficult to attribute these supplies to the World Bank funding because the sources of these materials are not identified. Figure 8. Number of manual and motor sprayers supplied to all the states and the timing of the supplies relative to the HPAI outbreak period. Figure 9. Number of PPE units supplied to all the states and the timing of the supplies relative to the HPAI outbreak period. Figure 10. Amount of disinfectants supplied to all the 37 states from 2006 to 2007. 0 10 20 30 40 50 0 50 100 150 200 Ja n Ma r Ma y Ju l Se p N o v Ja n Ma r Ma y Ju l Se p N o v Manual sprayers Motor sprayers Outbreaks N u m b er o f outbreak s N u m b er o f spr ayer s Month (Jan 2006 - December 2007) 0 10 20 30 40 50 0 1000 2000 3000 4000 5000 6000 7000 Ja n Ma r Ma y Ju l Se p N o v Ja n Ma r Ma y Ju l Se p N o v PPE N u m b er o f outbreak s Numbe r of P P E u nits Month (Jan 2006 - December 2007) 0 5 10 15 20 25 30 35 40 45 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 Ja n Ma r Ma y Ju l Se p N o v Ja n Ma r Ma y Ju l Se p N o v Disintfectant Outbreaks N u m b er o f outbreak s A m ount of d isin tfe ctan ts ( lit res) Month (Jan 2006 - December 2007) 112 Depopulation and compensation The proportion of poultry depopulated was estimated using records of poultry population, mortality and depopulation numbers given in the outbreak dataset (on village level). In these datasets, the number of birds that died from the disease and those that were culled did not always equal the population at risk. Because of this, mortality and depopulation proportions were estimated in two steps: (i) using all the records in the dataset (containing 243 outbreaks), and (ii) using only those records where deaths (disease-related mortalities and the number of birds culled) balanced the population at risk (with 218 outbreaks). These analyses gave similar median mortality and depopulation proportions of 0.30 and 0.70, respectively. Figure 10 shows how these proportions (based on 243 outbreaks) varied over time. In general, an increase in depopulation rates as expected led to a decline in mortality rates. The depopulation proportion was higher in 2007 than 2006; this suggests that fewer birds were allowed to die from the disease during the outbreaks in 2007. This intervention might have contributed to a reduction in the rate of disease transmission. Figure 11. Mortality and depopulation proportions following HPAI outbreaks between 2006 and 2008 in Nigeria. On farm-level, farms where depopulation was carried out were classified into two depending on whether confirmatory laboratory results had been received before depopulation was done (Table 5). These results indicate that the proportion of farms that had received laboratory results before depopulation was higher in the second year compared to the first (2006). Table 5. Proportions of farm outbreaks in Nigeria on which depopulation was conducted with or without laboratory confirmation of suspected cases of HPAI Year Period Proportion of farms with depopulation HPAI negative HPAI positive No sample collected 2006 Jan-Jul 12 (5%) 149 (68%) 58 (27%) Aug-Dec 0 15 (75%) 5 (25%) 2007 Jan-Jul 5 (1%) 655 (97%) 12 (2%) 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Ja n Fe b Ma r A p r Ma y Ju n Ju l A u g Se p O ct N o v De c Ja n Fe b Ma r A p r Ma y Ju n Ju l A u g Se p O ct N o v De c Ja n Fe b Ma r A p r Ma y Ju n Ju l A u g Se p O ct N o v De c 2006 2007 2008 Mortal ity an d depo p ulation proportion s Month/Year Mortality rate Depopulation rate 113 Aug-Dec 2 (7%) 26 (86 %) 2 (7%) The amount of compensation paid per bird, as expected, increased as from January 2007, when compensation payments were conducted through NNAICP. Compensation rates derived from the farm-level compensation dataset are given in Table 6. Table 6. Compensation rates per bird (in Naira) derived from the farm-level compensation dataset Year Period Median Minimum Maximum 2006 Jan-Jul 250.0 106.6 3045.0 Aug-Dec 250.0 233.3 1500.0 2007 Jan-Jul 350.0 100.0 4451.3 Aug-Dec 966.3 201.9 2793.4 The median time period between depopulation and compensation payment was about 242 days (minimum 37 days, maximum 370 days) and did not differ greatly between 2006 and 2007 because compensation payments were conducted in blocks (i.e. within certain months over a period of few weeks) and not continuously over time. Trainings Table 7 gives a list of some trainings implemented through NNAICP between January 2007 and November 2010. Most of the trainings captured here were however given long after the HPAI outbreaks had resolved. Table 7. Trainings conducted by the project Date/Month Training January, 2007 and February, 2007 Two meetings held to sensitize Media chief executives on avian influenza reporting. The first one (in January) was held in Lagos (for southern Nigeria) and the second one was held in Kaduna (in February) for the northern Nigeria. 15th ? 20th March 2009 Training workshop on disease surveillance and reporting for south western zone held at Ibadan to strengthen national animal disease surveillance network April ? September 2009 Training of a total of 929 live bird market operators from Plateau, Kaduna, Lagos, Kano and Kwara states. 16 ? 20th June 2009 A training workshop involving quarantine officers, importers of livestock and livestock products and private veterinarians held at Benue hotel, Makurdi September 2010 Completion of MSc courses by two students sponsored by the project. One was enrolled in the Royal Veterinary College, London while the other was enrolled in Institute of Tropical Medicine Antwerp, Belgium. They both did coursework in Epidemiology and thesis work. The titles for their theses are development of risk scoring tools for African swine fever (ASF) in Nigeria and situation-based survey of avian influenza viruses in possible bridge species of wild and domestic birds in Nigeria, respectively. 114 15 ? 16th September 2009 Training of farmers from Kano, Jigawa, Nasarawa and Plateau states on biosecurity practices 28th October 2009 Training of live bird market operators and poultry farmers from Bayelsa, Delta, Edo and River states on level 3 biosecurity measures 1-6th November 2010 A training workshop involving Nigeria Agricultural Quarantine Service on basic principles and practice of border and control post inspection and certification of agricultural products especially animals, fish and their products held at Kaduna Time difference between events Time in days between events (noticing an outbreak, reporting, collection of samples, receipt of laboratory results, quarantine, depopulation, and compensation) was estimated using the outbreak and compensation datasets. Estimates from the two datasets were compared. In the outbreak dataset, cases were mostly recorded at the village level whereas in the culling and compensation dataset, records were mostly given at the farm level. Time intervals estimated from the outbreak dataset (mainly village-level data) included time between: ? noticing outbreaks by farmers and reporting to veterinary authorities, ? noticing outbreaks by farmers and sending of samples for laboratory diagnosis, ? reporting outbreaks to veterinary authorities and sending samples for laboratory diagnosis, ? sending samples and receiving confirmation of laboratory results, ? reporting outbreaks to veterinary authorities and implementation of quarantine, ? noticing an outbreak and depopulation. ? reporting of outbreaks to veterinary authorities and depopulation, Those estimated from the culling and compensation dataset (farm-level data) included time between: ? noticing outbreaks by farmers and reporting to veterinary authorities, ? depopulation and payment of compensation. Results from this analysis are summarised in Table 8. The analysis was stratified by year and period (6 month time intervals between Jan 06 and Dec 08). Some of the general observations that can be made from these findings are: - estimates derived for Aug ? Dec 08 period should be interpreted with caution because only 2 outbreaks were recorded at the time, - quarantine measures and sample collection for laboratory diagnosis were promptly implemented as soon as farmers reported an outbreak, - time interval between noting an outbreak and reporting improved in the second year, particularly between January and July 2007. This however worsened towards the end of 2007. The improvement observed in early 2007 could be associated with the payment of higher compensation rates which provided more incentives for reporting, - time between sending samples for laboratory confirmation and receipt of results greatly shortened towards the end of 2006. This improvement was maintained through 2007. This 115 was associated with improvement in sample transportation to the laboratory or logistics of sample handling within the laboratory. Other analyses Postgraduate students 1. Dr. Ezenwa Nwakonobi is a staff of the Federal Livestock Department deployed to serve as the Logistics Officer of the AICP Animal Health Component, Abuja. He works closely with the state AICP desk officers. Thesis title: Development of risk scoring tools for African swine fever (ASF) in Nigeria. Master of Science in Veterinary Epidemiology from the Royal Veterinary College, London September 2009 ? September 2010. 2. Dr Vakura Columbia Teru is a staff of the Federal Livestock Department deployed to serve as Surveillance Officer with the NADIS the Epidemiology Unit of the Department. Thesis title: Situation-based survey of avian influenza viruses in possible bridge species of wild and domestic birds in Nigeria. Master of Science in Tropical Animal Health, Institute of Tropical Medicine (ITM), Antwerp, Belgium September 2009 ? July 2010. 116 Table 8. Median number of days between activities implemented in response to an outbreak commencing with an outbreak being noticed by farmers to payment of compensation (a) Time intervals estimated from the outbreak dataset Time interval Overall Median Year Period (6-month intervals from Jan 2006 to Dec 08a) 2006 2007 2008 Jan-Jul 06 Aug ? Dec 06 Jan-Jul 07 Aug ? Dec 07 Aug ? Dec 08 Noting an outbreak to reporting 3 3 3 1 3 3.5 2 4 1 Noting an outbreak to sending samples to the lab 3 3 3 2 3 4 3 5 2 Reporting an outbreak and sending samples 1 1 1 0.5 1 0.5 1 0 0.5 Sending samples and receiving lab results 2 5 2 4.5 5 2 1.5 2 4.5 Reporting an outbreak to start of quarantine 0 0 0 0 0 0 0 0 0 Noting an outbreak to depopulation 6 6 7 4 5 6.5 5 9 4 Reporting an outbreak to depopulation 2 1 3 3 1 5 2 5 5 a Jan ? Jul 08 not included because there was no outbreak then (b) Time intervals estimated from culling and compensation dataset 117 Time interval Overall Median Year Period (6-month intervals from Jan 2006 to Dec 08a) 2006 2007 2008 Jan-Jul 06 Aug ? Dec 06 Jan-Jul 07 Aug ? Dec 07 Aug ? Dec 08 Noting an outbreak to reporting 1 6.5 1 - 7 4 1 4.5 - Depopulation to payment of compensation 242 212 242 - 212 186 242 147 - a Jan ? Jul 08 not included because there was no outbreak then 118 Field surveys Results from this section are presented under three sub-sections: - Improving biosecurity in poultry production and trade ?include farmer biosecurity training, fowl seller biosecurity training and live bird market improvement, - Strengthening disease surveillance ? focusses on surveillance teams, diagnostic teams, infected premises teams and participatory disease surveillance teams, - Strengthening veterinary quarantine services Improving biosecurity in poultry production and trade Farmer biosecurity training Background information A total of 82 respondents participated in this survey. Slightly more than half (62%, n = 49) of them were males. Most of them (75%, n = 62) regarded poultry farming as their main source of income. The number of chickens kept by these respondents varied between 15 and 40,000 with a median of 1150. The types of poultry kept included layers only (83%, n = 64), broilers only (14%, n = 11) and a mixture of layers and broilers (2%, n = 2). Most of the respondents (87%, n = 65) had never used HPAI vaccines. Conversely, 94% (n = 72) and 39% (n = 30) had used vaccines to control Gumboro and fowl cholera diseases, respectively. Biosecurity training Sixty percent (n = 48) of the respondents had had at least one training on farm biosecurity; the number of trainings that each respondent had ranged between 4 and 12. When asked to identify the institution that offered each training, only 36% (n = 29) could identify the NAICP for at least one of the trainings. Other institutions mentioned included USAID, Poultry Farmers Association, and women organization, among others. The respondents were asked to list lessons obtained from the trainings that they had managed to implement on their farms. The commonly identified lessons included (i) fencing (29%, n = 15), (ii) construction of a footbath (22%, n = 11), and (iii) restriction of visitors (17%, n = 9). Other lessons cited included provision of protective clothing to staff, movement control, and not sharing equipment with other farms. The respondents were also asked to indicate whether they had trained someone else on biosecurity practices. Responses given on this issue are summarised in Table 9. The number of farmers that had trained other farmers given that they had been trained by NAICP (24) was significantly higher than those who had trained other farmers and had not been trained by NAICP (13) (?2= 15.42; p = 0.00). Table 9. The number of respondents who had trained other farmers on biosecurity practices on poultry production Trained someone else No. of farmers trained on biosecurity Total Trained by NAICP Never or trained by other projects Yes 24 13 37 No 2 4 6 26 17 43 119 In a decreasing order of frequency, some of the biosecurity practices that these respondents trained other farmers on include (i) cleaning and disinfection, (ii) carcass disposal, (iii) disease reporting, and (iv) restriction of visitors from accessing poultry units. Association between training status and implementation of selected biosecurity practices The association between training and use of selected biosecurity measures was analysed using data obtained from self-reported interviews. Only farmers trained by NAICP were included (respondents trained by other projects (n = 19) were excluded from the analysis). The results are given in Table 10. Table 10. Association between training status and use of a given biosecurity practice based on self- reported interviews Biosecurity practice Training status a Total Fisher?s Exact P Trained (NAICP) Untrained A shower and changing rooms are provided for workers 77.8% (21) 70.4 (19) 40 0.68 Distance in meters from the closest poultry farm 190 168 T = -0.31, p =0.76 Other poultry farmers sometimes borrow farm implements from the farm 7.1% (2) 0 2 0.52 Other poultry farmers occasionally visit the farm 57.1% (16) 44.8% (13) 29 0.72 Poultry farm workers visit other farms 21.4% (6) 28.6% (8) 14 0.14 Visitors can easily access poultry premises on the farm 53.8% (14) 53.6% (15) 29 0.23 Presence of a fence and gate around poultry premises 73.1% (19) 82.6% (19) 38 0.07 Visitors can easily access poultry pen on the farm 7.7% (2) 25.0% (7) 9 0.06 Presence of a footbath at the entrance to the farm 42.9% (12) 29.6% (8) 20 0.10 Presence of a footbath at the entrance to each poultry pen 66.7% (18) 65.4% (17) 35 0.27 Level of cleanliness of poultry houses Very clean 21.4% (6) 22.2% (6) 12 0.09 Clean 78.6% (22) 66.7% (18) 40 Dirty 0 11.1% (3) 3 Possibility of poultry coming into direct contact with other animals/birds Very likely 7.4% (2) 7.4% (2) 4 0.14 Likely 40.7% (11) 59.3% (16) 27 Very unlikely 51.9% (14) 33.3% (9) 23 Method of disposing carcasses Buried 70.4% (19) 67.9% (19) 38 0.16 Consumed 3.7% (1) 0 1 Incinerated 0 10.7% (3) 3 Sold 0 7.1% (2) 2 Animal feed 7.4% (2) 3.6% (1) 6 120 Thrown away 11.1% (3) 10.7% (3) 6 Other 7.4% (2) 0 2 a Trained respondents include only those farmers that were trained by NAICP while untrained ones include those farmers that have never had any training before Three practices were significantly associated with training status at 90% confidence. A lower proportion of trained farmers had fences and gates around their poultry premises compared to untrained farmers (73.1% compared to 82.6%). Trained farmers, however, had better measures than untrained one with regards to (i) preventing visitors? access to poultry pens and (ii) putting up a footbath at the entrance to each pen. Disinfectants were replenished on the footbath on daily (9%, n = 41% of the farms), weekly (36%, n = 8) basis or monthly basis. None of the trained farmers? farms were classified as being dirty as compared to 3 farms belonging to untrained farmers. The identified biosecurity improvements were however not adequate to reduce the risk of HPAI transmission. Training and the ability of farmers to identify safe sources of inputs and methods of selling outputs An assessment was also made on the effect of training on the ability of the farmers to identify safe and clean sources on inputs (replacement stock and feed) and secure ways of selling and delivering outputs such as live birds, eggs and dressed broilers. The hypothesis used here was that trained farmers were better in identifying and using outlets that had higher biosecurity standards than the untrained ones. However, there were no differences on all the practices assessed. Both trained and untrained farmers used common sources of feed, had the same considerations for purchasing replacement stock, used the same channels of selling live birds and eggs. Eggs, for instance were sold to traders from the farm. Farm registration Sixty eight percent (n = 48) of the farmers interviewed had been registered. Most of the registrations were done in the year 2007. The respondents indicated that they registered their farms in response to a government regulation that required them to do so. There was no differences in the proportion of farms that registered between trained (19/36) and untrained (16/36) groups (?2=3.79, p =0.44). Reasons given for failure to register by the farmers who had not registered include: - Lack of confidence on the government?s plan to support farmers - Small number of birds kept - Lack of assistance during the registration process Communication messages The respondents indicated that radio (36%, n = 26), television (21%, n = 15), radio and television (13%, 10) and newspapers (4%, n = 3) were the frequently used media for accessing information on HPAI. Table 11 lists key messages received by these respondents between 2004 and 2008. Each respondent had been asked to give a message they heard on HPAI and specify the year when the message was given. These messages matched with those that were developed and disseminated by NAICP and other projects such as UNICEF. 121 Response to outbreaks Of the 81 farmers interviewed, 34 indicated that they had HPAI outbreak/suspicions on their farms. Forty eight percent of these outbreaks/suspicions occurred in 2007 while the rest occurred in 2007. A majority of the farmers (74%, n = 25) reported these cases to the authorities. These included state veterinarians (34%, n = 8), NAICP desk officers (30.4%, n = 7) and private vets (26%, n = 26%). The rest were reported to NVRI and veterinary teaching hospital in Nsukka. All the cases reported were followed up. The respondents gave a list of the type of samples collected as (i) carcasses (48.2%, n = 13), (ii) blood and carcasses (18.5%, n = 5), (iii) swabs and carcasses (7%, n = 2), and (iv) feces and serum. Other assessments that farmers did on the response teams? performance are given in Table 12. 122 Table 11. Information, education and communication messages on HPAI received by farmers in Nigeria between 2004 and 2008 Year Frequency Key messages that the farmers heard 2004 1 (1%) - HPAI is a disease of poultry 2005 4 (6%) - China is experiencing cases of HPAI - Other countries in Asia have been affected as well 2006 48 (69%) - There is a rare disease affecting poultry and humans - Report sick birds to a vet - Clinical signs of HPAI given - Don?t eat poultry carcasses; don?t throw them away either, burn them instead - People wearing masks and gloves are culling chickens because of the disease - Eggs found on trucks moving from north to south destroyed - Sambawa farms have been affected by the disease with massive mortalities - Don?t allow visitors to enter your farm - Take precaution in handling poultry products - Infected birds will not survive more than 24 hours 2007 9 (13%) - Wash hands with a disinfectant after touching poultry or use gloves - Improve sanitary conditions of the farm - Clinical signs of HPAI presented and that it is also zoonotic - Cook chicken meat well before eating 2008 5 (7%) - Report poultry diseases to the local veterinary office - Cook poultry mean well before eating - Wash hands with a disinfectant after touching poultry 2009 1 (1%) - Incinerate dead birds - Call a vet when birds are sick 2010 1 (1%) - Don?t eat sick birds or carcasses Table 12. Assessment done by farmers that had outbreaks on their farms on the performance of the response teams Assessment criterion Yes No Was there a follow up by HPAI field team on the first outbreak 88.2% (n = 30) 11.8% (n = 4) Did field teams do any disinfection during the first outbreak 80.0% (n = 28) 8.6% (n = 3) Did the field teams use PPE during the first outbreak 85.7% (n = 30) -(missing values) Who collected samples in the first outbreak Desk officer 31.0% (n = 9) Response team 31.0% (n = 9) NVRI 4% (13.8%) Other 7 (24.1%) Biosecurity measures that farmers were asked to implement by the response teams following an outbreak were: 123 - Burry carcasses and restrict entry of visitors - Clean and disinfect poultry pens and equipment - Wash hands with disinfectants after touching poultry - Delay restocking (no duration specified) Fowl seller bio-security questionnaire Background information A total of 52 fowl sellers were interviewed in 4 different states (Kaduna=19, Kano=9, Lagos=12, Plateau=12). These respondents were distributed in 11 markets, including 6 traditional and 5 upgraded markets. Trainings Fifty percent (n=26) of the respondents had had at least one training on poultry bio-security whereas 40% (n=21) had not had any training. The rest did not provide information on their training status. Of the 26 trained fowl sellers, 54% (n=14) operated from improved markets and the other 46% (n=12) operated from traditional markets. Similarly, 29% of the untrained respondents were based in the improved markets while the rest 71% (n=15) worked in the traditional markets. When asked to identify institutions that offered the trainings attended, 81% (n=21) of the trained traders identified NAICP while 8% (n=2) identified USAID (training offered in 2006). The other 11% (n=3) of the respondents could not remember the institution that offered the training. This analysis therefore focussed on the 21 fowl sellers that had been trained by NAICP and the 21 respondents that that had not had any training. A high proportion (67%, n=28) of them were male. They identified poultry trade as their main source of income. The proportions of respondents in each age group were as follows: 20-30 years - 24%, 31-40 years - 45%, 41-50 years - 14% and >50 years - 17%. The number of trainings that these respondents had completed ranged between 1 and 5. Fourteen percent (n=2) of them were trained in 2007, 36% (n=5) in 2008, 29% (n=4) in 2009 and 21% (n=3) in 2010. Lessons learnt by the respondents from the NAICP trainings, ranked in a descending order based on frequency, included: - Disinfection of cages - Washing of hands - Avoid mixing poultry - Fumigation of live bird market - Separate sick and healthy birds - Use of plastic cages - Clinical signs of disease Lessons that could not be implemented by fowl sellers included: - Use of plastic or metallic cages - because they are expensive - Procedures associated with LBM improvement in those markets where improvement was not done - Washing of hands 124 Eighty four percent (n=16) of the fowl sellers trained by NAICP had trained other traders. Topics covered in these trainings, in a descending order of frequency, included disinfection of cages, washing hands, clinical signs of disease, buying healthy chickens and not mixing poultry species. Biosecurity practices Most of the respondents traded in live spent layers, live broilers and live indigenous poultry. Dressed broilers were only sold by trained fowl sellers whereas eggs were only sold by untrained fowl sellers. A descriptive analysis of a range of biosecurity practices used by trained and untrained fowl sellers is shown in Table 13. Table 13. A distribution of biosecurity practices between trained and untrained fowl sellers Biosecurity practices Training status Untrained Trained Ways of handling of sick birds Treated 12 (46%) 14 (50%) Isolated 10 (39%) 10 (36%) Slaughtered/consumed 4 (15%) 4 (14%) Ways of disposing carcasses Thrown away 16 (73%) 11 (52%) Buried 5 (23%) 9 (43%) Sale to dog breeders 1 (4%) 1 (5%) Methods of procuring poultry Collecting from farms 13 (62%) 13 (62%) Farmers deliver poultry to trader 1 (5%) 2 (9%) Obtaining from other seller 3 (14%) 1 (5%) Obtaining from other markets 4 (19%) 5 (24%) Ways of transporting poultry Public transport 19 (95%) 18 (86%) Vehicle 1 (5%) 1 (5%) Motor bike 0 1 (9%) Methods of keeping poultry before sale Basket cages 1 (5%) 0 Other cages 15 (71%) 18 (82%) Closed room 2 (10%) 2 (9%) Other method 3 (14%) 2 (9%) Location of poultry kept before selling Stall in the market 19 (90%) 17 (90%) At a shop 2 (10%) 1 (5%) Other 0 1 (5%) Methods of checking the health status of the birds Check birds health themselves 20 (95%) 19 (90%) Trust the supplier that birds are healthy 1 (5%) 2 (10%) Confinement of unsold birds Other cages 16 (80%) 18 (82%) Closed room 2 (10%) 2 (9%) Other method 2 (10%) 2 (9%) Type of cages used Wooden cages 16 (67%) 10 (45%) Metallic cages 7 (29%) 11 (50%) Other cages 1 (4%) 1 (5%) Cleaning of cages Never 0 1 (5%) Daily 12 (57%) 13 (62%) Weekly 8 (38%) 7 (33%) Every 2 weeks 1 (5%) 0 125 Ways of cleaning cages Removal of manure only 6 (43%) 7 (41%) Use of fumigation after removal of manure 0 1 (6%) Use of disinfectant after removal of manure 7 (50%) 8 (47%) Rinsing with water after removal of manure 1 (7%) 1 (6%) Frequency of washing hands Usually 14 (67%) 16 (76%) after handling poultry Sometimes 7 (33%) 4 (19%) Never 0 1 (5%) Although the distribution of bio-security measures conducted were not significantly different (at p<0.05) between trained and untrained fowl sellers, the distribution of biosecurity measures given in Table 13 indicates that trained fowl sellers were more likely to use metal cages than untrained ones. However, metal cages were more common in improved markets than in traditional markets where wooden cages were dominant. The type of market might have influenced the type of cages to use, and hence the type of biosecurity measures implemented, A total of 20 trained and untrained fowl sellers provided answers on the keeping of records showing the source of consignments. A total of 11 untrained fowl sellers (55%) kept records compared to only 3 trained fowl sellers (15%). Reasons given for the inability of some traders not to keep records are given in Table 14. Table 14. Reasons given by some traders for their inability to keep records Reasons for no records Untrained Trained Source always known 1 (12%) 0 Don't know how to do it 2 (26%) 2 (13%) Not important 4 (50%) 6 (37%) No time or too tedious or too difficult 1 (12%) 8 (50%) Sick birds Both trained (81%, n=17) and untrained fowl (90%, n=19) sellers indicated that they had had sick poultry in their consignments. Only 10% of them (independent of the training status) reported these cases to the relevant authorities. In addition, 71% (n=15) of trained and 90% (n=19) of untrained traders had experienced some mortalities in their flocks with 33% (n=7) and 19% (n=4) of trained and untrained sellers reported these mortalities. Concerning is the fact that 52% (n=11) and 48% (n=10) of trained and untrained sellers slaughtered sick birds and sold them. Live bird markets Background information Bio-security measures were evaluated on 3 improved and 4 traditional poultry markets in 4 different states ? Kano, Lagos, Plateau, and Kaduna. In each state except Kaduna (where only one traditional market was visited), 2 markets (one improved and one traditional market) were assessed. All the markets were improved by the NAICP project except the LBM in Plateau that was improved by the Nigerian government. This market (in Plateau) was therefore excluded from the analysis in order to limit the focus on those markets refurbished by the project. 126 All traditional and improved markets operated daily. The improved markets had a median number of 650 (minimum 500, maximum 800) birds per day and the traditional market had a median capacity of 2000 (minimum 700, maximum 3000) birds per day. All improved live markets had poultry and poultry products only while 50% (n=2) of the traditional markets had other food stuffs as well. Improved markets and 75% (n=3) of the traditional markets were managed by the fowl seller association but one traditional market was ran as private businesses. Biosecurity practices Biosecurity practices used in the markets were assessed based on (i) data collected from questionnaire interviews involving the managers of the markets and (ii) visual inspections of the market infrastructure and operations using a pre-determined checklist of questions. Based on the data collected from the questionnaire interviews, we observe that: - None of the traditional markets kept records. Only one of the improved markets kept some records although these had not been updated since in April 2009, - New consignments were kept separately in all the improved markets as well as in 2 out of 3 traditional markets, - Different species of poultry were kept separately in improved markets as well as in 1 out of the 4 traditional markets, - Sick birds were either isolated and treated or brought to the attention of the veterinarian on the improved markets, while isolation and treatment of sick was only practiced in 50%of the traditional markets; other procedures conducted with sick birds on traditional markets included the isolation without treatment, the slaughtering or the sale of live sick birds - Metal cages were only used in improved markets, - Washing and disinfection of the cages, the market and facilities was conducted more frequently in improved markets, - Periodical screening of birds was only conducted in improved markets. Faecal, cloacal and blood samples were being collected either biannually or as part of surveillance programs in these markets. Sampling had been done once in one of the traditional markets, - Improved markets had biosecurity teams but only 1 out of the 4 traditional markets had such a team, - Both improved markets had poultry seller teams as compared to only 1 out of the 4 traditional markets that had similar team, - Both improved markets had routine veterinary inspections compared to only 2 of the 4 traditional markets that had the same service. All inspections were conducted by government veterinarians. Slaughtering facilities - Both improved markets had poultry slaughtering teams working on the markets as compared to only 2 of the 4 traditional markets that had similar slaughter teams, - Improved markets routinely cleaned and disinfected slaughtering tools while 3 out of 4 traditional markets only cleaned these tools with running water without disinfection. The other traditional market used dirty water for cleaning the tools, - Processors disinfected their hands after slaughter in improved markets and in only 1 traditional market, 127 - Waste material was deposited into garbage cans and emptied several times daily into waste containers outside the market in both improved market and in only 1 traditional markets, - Slaughtering countertops on both improved markets, but only on 2 of the 3 traditional markets with slaughtering countertops were washed regularly after processing a batch of poultry , one traditional market did not have a slaughtering countertop - Improved markets and 2 out of 4 traditional markets properly maintained defeathering tanks, including replenishing the water regularly. One traditional market did not have a defeathering tank. Transportation - Improved markets had poultry delivery teams based in the markets but only 1 of the 4 traditional markets had such a team. Delivery teams received poultry, and on improved markets in addition also washed crates and assessed transportation standards, - In all markets vehicles passed through a disinfectant dip or had their tyres cleaned with a pressure sprayer, - Vehicles were cleaned after delivery in both improved markets and only in 1 of the traditional markets, - None of the transporters were wearing PPE equipment on either improved or traditional markets, - Transporters were not allowed into areas other than the delivery and storage space in the improved markets but were allowed in these areas in 3 out of the 4 traditional markets Equipment and building design - Operations such as delivery, storage, selling and slaughtering were clearly compartmentalised in improved markets and not so in traditional markets, - No quarantine station or room was located in either improved or traditional markets, - Only one improved market had a water pump, the rest of the markets did not, - None of the markets had a deep freezer - Both improved markets and one traditional market had generators. Visual inspection of the markets Table 15 summarises the results obtained by analysing the data obtained from visual inspection of the markets. Improved markets were better than traditional markets in (i) being not located in residential areas, (ii) having a perimeter fence around them, (iii) having a gate at their entrances, (iv) having no stray dogs and cats at the market, (v) having a clean supply of water and (vi) having a slaughtering slab. However, improved markets were not different from traditional markets in a number of areas for example, (i) traders or slaughter men did not wear PPE, (ii) did not have vehicle disinfection dip. Perceptions of the traders on biosecurity practices All traders operating from the improved markets were satisfied with the bio-security practices used in those markets. They indicated that cleanliness, regular inspections of markets by the government veterinarians and the introduction of processor boots were the main improvements done. Traders operating from traditional markets were, however, not satisfied with biosecurity measures in those 128 markets. In improved markets and 2 of the traditional markets, penalties were meted out to individuals who violated biosecurity measures. Based on a qualitative assessment, bio-security measures were almost always better on improved markets compared to the traditional markets. 129 Table 15. A comparison of biosecurity measures in improved and unimproved markets based on data collected from visual inspection Improved Traditional Yes No Comment Yes No Comment 1. Market located in residential area? 0 2 (100%) 2 (50%) 2 (50%) 2. Stagnant pools of water near the market? 0 2 (100%) 1 (25%) 3 (75%) Yes-50m between water and market 3. Wild/migratory birds in vicinity of market? 1 (50%) 1 (50%) 0 4 (100%) 4. Separate sections for different poultry species? 1 (50%) 1 (50%) Yes-Use of sick bay for duck cages (outside the main poultry area) 1 (25%) 3 (75%) 5. Separate slaughtering areas for different poultry species? 0 2 (100%) 0 3 (100%) 1 record missing 6. Waterfowl sold at the market? 1 (50%) 1 (50%) 1 (33%) 2 (67%) 1 record missing 7. Ornamental and cage birds sold at market? 1 (50%) 1 (50%) Yes- pigeons 0 4 (100%) 8. Stray dogs and cats at market? 0 2 (100%) 2 (50%) 2 (50%) Yes-in slaughtering areas and with live birds 9. Rodents and flies at market? 2 (100%) 0 4 (100%) 0 10. Other animals sold at market? 0 2 (100%) 0 4 (100%) 130 11. Traders wearing PPE? 0 2 (100%) 0 4 (100%) 12. Slaughtering slab material - Ceramic - Timber - Other - No available 2 (100%) 0 0 0 1 (33%) 1 (33%) 1 (33%) 0 13. Slaughtering wall material - Ceramic - Concrete - Timber - Metal sheet - No available 1 (50%) 1 (50%) 2 (50%) 1 (25%) 1 (25%) 14. Customers allowed in slaughtering areas 1 (100%) 1 record missing 3 (75%) 1 (25%) 15. Hot water drums for scalding 2 (100%) 3 (75%) 1 (25%) 16. Slaughter people wear PPE 2 (100%) 4 (100%) 17. Tyre bath that vehicles have to pass through 2 (100%) 4 (100%) 18. Market designed to allow one-way traffic 2 (100%) 4 (100%) 131 19. Cement flooring in the entire market 2 (100%) 1 (25%) 3 (75%) 20. Adequate drainage (including sink pits and gutters) 2 (100%) 1 (25%) 3 (75%) 21. Roof over entire market to provide shade 2 (100%) 2 (50%) 2 (50%) 22. Perimeter fence around the market 2 (100%) 4 (100%) 23. Gate that can be closed for security checks 2 (100%) 4 (100%) 24. Market regulations displayed at entry 1 (50%) 1 (50%) 4 (100%) 25. Clean and reliable water supply 2 (100%) 4 (100%) 26. Electrical supply existing 2 (100%) 1 (25%) 2 (75%) 132 Strengthening disease surveillance Surveillance teams Background information Only 11 responses received from surveillance teams? supervisors; these came from Ogun (n=6), Nasarawa (n = 2), Rivers (n=1), Lagos (n=1) and Kano (n=1) states. A majority (72%, n = 8) of the respondents were male and had a median age of 45.5 with a minimum and maximum of 30 and 52 years, respectively. Five respondents (one from each state) stated that they started working as surveillance officers since 2006. The respondents listed their main tasks as being: - Design and implementation of surveillance (29.6%, n = 8) - Data and sample collection (25.9%, n = 7) - Sensitizing farmers and traders about disease outbreaks (3.7%, n = 1) - Training on surveillance (3.7%, n = 1) - Disease reporting (14.8%, n= 4) - Disease tracing (7.4%, n = 2) - Implementation of biosecurity measures (7.4%, n = 2) Training All the respondents had undertaken a number of trainings on surveillance but only 4 had been trained at least once by the NAICP. The other trainings were offered by FAO, USAID, NADIS, SPINAP and State HPAI surveillance unit. Topics covered in most of the trainings included: - Communication skills - Sample collection and diagnosis - Review of status of HPAI outbreaks, prevention and control - Disease tracing - Surveillance and biosecurity All the respondents indicated that they had trained other surveillance officers on topics such as: - Disease reporting - Sample collection, diagnosis and prevention - Disease tracing - Planning surveillance Surveillance activities The respondents had conducted active surveillance activities ranging between 2 to 14. Nine of the 10 respondents were able to identify HPAI cases in some of their visits. Most of the respondents (6/9) indicated that their surveillance activities were funded by NAICP. The samples they collected were always submitted to NVRI laboratory in Vom. When asked to specify the policies that the teams were using for their surveillance, the following guidelines were identified: - Guidelines provided by Federal Department of Livestock, - SOPs obtained from various trainings, 133 - Standard operating procedures on sample collection. Field team structure The average number of officers that formed a surveillance team ranged between 4 - 6 in 2006, 5 ? 20 in 2007, 4- 20 in 2008, and 4 ? 40 in 2010. The respondents indicated that between 2-4 staff got replaced each year. When asked to indicate what they used surveillance data for, the supervisors said that results were entered into a database in preparation for analysis by the epidemiology team. They also pointed out that laboratory results were not relayed back to them except when there was a disease outbreak. Slightly over half (55%, n = 5) of the respondent said that they had an electronic data base for storing data. Constraints Table 16 shows how the respondents ranked a list of predetermined constraints that surveillance teams were expected to face. Each response that was marked as being excellent got a score of 1, good got a score of 2 and poor got a score of 3. Table 16. Ranking a list of predetermined constraints faced by the surveillance teams Constraint Excellent Good Poor Average Experience of team members - 16 6 7.3 Quality of knowledge of team members 1 14 6 7.0 Availability of hardware and software 1 6 6 4.3 Availability of training opportunities 2 8 12 7.3 Coordination with other HPAI surveillance teams 3 12 3 6.0 Cooperation with farmers for quality information 3 14 5.7 Quality of collaboration with HPAI special diagnostic team 3 8 3 4.7 Quality of collaboration with infected premises team 3 10 3 5.3 The respondents identified availability of hardware and software and quality of collaboration with HPAI special diagnostic teams as being minor constraints but availability of training opportunities, experience of team members and quality of knowledge of surveillance officers were identified as being major constraints. Areas that they identified for support included: - Funding and training - Transport - Equipment e.g. knapsack sprayer - Laboratory support Other diseases targeted Other animal diseases investigated by the teams were African swine fever and swine flu. 134 Diagnostic teams Background information Only 11 responses received from Plateau (n = 8), Kano (1), Lagos (1) and Ogun (1) were used in this analysis. Most of the respondents were male (81.8%, n = 9) and their median age was 45 years with a minimum and maximum of 34 ? 50 years. The respondents started working as members of diagnostic teams since 2005 (2 respondents), 2006 (n = 3), 2007 (n = 3), and 2008 (n = 1). Their main tasks included: - Sample processing (33.3%, n = 9) - Disease investigation (22.2%, n = 6) - Diagnosis (22.2%, n = 6) - Reporting (7.4%, n = 2) - Logistics and cataloguing (7.4%, n = 2) - Training (3.7%, n = 1) - Risk factor analysis (3.7%, n = 1) Training Eight out of 10 respondents had attended training on HPAI diagnosis. The trainings were done in 2006 (by 4 respondents), 2007 (n = 3) and 2008 (n = 1). None of the trainings were offered or sponsored by the NAICP. The institutions that gave these trainings include APHIS/USDA, USAID, CIRAD, FAO/OIE through Padua reference lab and Egyptian government. The trainings were either done within the country (in Abuja, Jos and Vom), USA (Ames, Iowa, Atlanta), Egypt (Cairo) or in Kenya. The respondents were trained on: - emergency preparedness and response, - classical diagnosis, - virus isolation and molecular identification including the use of real time PCR, - Principles of veterinary epidemiology, including building a veterinary epidemiological network. All the officers trained had trained someone else specifically on sample collection and diagnosis using PCR techniques, virus isolation and serology and on planning surveillance and importance of biosecurity. Diagnostic and surveillance activities Respondents were asked to specify reference materials or policies that they used for their routine diagnostic work. The following materials were cited: - Comprehensive emergency preparedness and differential action plan for HPAI surveillance - OIE and WHO manual of diagnostic tests - Standard operating procedures on sampling for avian flu prepared in February 2006 135 Team size Forty percent of the responses obtained indicated that the average size of a diagnostic team was 30. The rest indicated that team sizes ranged between 5 and 8 and that on average, 2-5 staff got replaced each year. When asked to state the optimal size of a diagnostic team, the respondents indicated that the team needed to be made up of 5 -6 members. Constraints faced by the diagnostic teams The respondents were asked to rank a list of pre-determined constraints faced by a diagnostic team. The results of this analysis are given in Table 17. Table 17. Ranking of the constraints faced by diagnostic teams Constraint/challenges Excellent Good Poor Average Cooperation of farmers during field work - 18 6 8.0 Working with the infected premises team 2 18 - 6.7 Coordination with other HPAI specialist diagnostic team 4 12 3 6.3 Working with disease surveillance team 4 14 - 6.0 Availability of PPE equipment 3 14 3 6.7 Training opportunities - 12 12 8.0 Knowledge of team members 3 16 - 6.3 These results indicate that diagnostic teams perceived working with other diagnostic and surveillance team members was not a big challenge but securing cooperation from famers and finding training opportunities were their main constraints. Areas they identified for support included: - financial support for reagents and consumables - vehicles, sampling and PPE equipment - Training opportunities Other diseases Other diseases that teams responded to include: - African swine fever - New castle disease Infected premises teams (response team) Background information All infected HPAI premises team that were trained by NNAICP (n=6) were established in 2006. Fifty percent of the respondents (n = 3) started working as infected HPAI premises team supervisors in 2006, 33% (n = 2) in 2007, and 17% (n = 1) in 2008. The important tasks of these officers included: - Depopulation - Sample collection 136 - Quarantine of suspected premises - Disease reporting - Treatment and vaccination Training All the respondents (n= 10) had had at least one training on surveillance and response. Only 6 of these respondents were trained by NAICP, the 4 respondents not trained by NAICP were excluded. The NAICP trainings were done in 2006 (for 1 respondent), 2007 (n = 2), 2009 (n = 2) and 2010 (n = 1). Topics covered in the trainings included use of PPE, disease surveillance, sampling techniques and the handling of suspected and confirmed HPAI cases. The training was assessed by all supervisors as being applicable for their HPAI field work. Only 3 (50%) of the respondents trained by the project had trained someone else using the knowledge obtained from the workshops. This horizontal training focussed on sample collection, personal protection and disease control. HPAI outbreaks or suspicions managed by the teams Only 4 teams provided information on HPAI outbreak responses that they responded to. The number of outbreaks attended to varied greatly by team. They indicated that stamping out was done in surrounding areas in 2006; this approach was however discontinued. Similarly, infected premises teams indicated to have culled flocks based on suspicious/clinical case definition only in 2006. Between 2007 and 2010, response teams focused more on sampling for laboratory confirmation and disinfection. Team structure Only 4 of the 5 supervisors provided information on the field team structure. Teams in different states remained largely of the same size between 2006 and 2009, but there were large variations in the team sizes between states. Apart from the supervisor, teams consisted of 1-10 officers (median: 5 members). The median number of team members in 2006 was 5 (Min=1, Max=10), in 2007 was 5 (Min=1, Max=7), in 2008 was 5 (Min=1, Max=7) and in 2009 was 5 (Min=1, Max=7) and in 2010 was 4 (Min=1, Max=7). The respondents indicated that the median number of a response team should be 5 people with a maximum of 6. Performance The response team performance was rated by the supervisors (1=excellent, 2=good, 3=poor). Overall the response team performance was lower in 2006 (mean score=1.67), compared to 2007 (mean rank score=1.37) and 2008 (mean rank score=1.42). These differences were however not statistically significant. The individual mean rank scores for different activities conducted by the response teams are listed in Table 18 137 Table 18: Mean scores for the performance of activities conducted by response teams between 2006 and 2008 Procedures 2006 2007 2008 Sampling 1.8 1.4 1.7 Stamping out infected farm 1.8 1.2 1.0 Stamping out surrounding area 1.3 1.5 Disinfection 1.8 1.4 1.5 Reasons given for the poorer performance on 2006 included: - No guidelines were available - Only case definitions were used for disease diagnosis Teams improved their performance in 2007 probably because: - Laboratory performance had improved - Staff became more experienced - Disinfectants became available - Motorised sprayers became available that made disinfection easier The poorer performance score for sampling in 2008 is probably due to the fact that very few samples were collected in that year. To summarize, based on a qualitative self-assessment by response team field supervisors, the overall performance of response teams improved between 2006 and 2007. Constraints Constraints faced by infected premises response team were rated by the supervisors (1=excellent, 2=good, 3=poor). The results of this analysis are given in Table 19. Table 19. Ranking the constraints faced by infected premises response team Areas of work Mean rank score Cooperation of farmers during field work 2.0 Working with the HPAI diagnostic team 1.6 Coordination with other response teams 1.6 Working with disease surveillance team 1.6 Availability of PPE equipment 1.6 Training opportunities 1.8 Knowledge of team members 1.8 Experience of team members 2.0 These results indicate that the main constraints of infected premises response teams were cooperating with farmers during field work and building or maintaining experience within the team. They however did not consider working with other two teams, the surveillance or 138 diagnostic teams, as being a constraint. They also did not perceive the availability of PPE as being a challenge. Sustainability of infected HPAI premises teams Supervisors were asked if infected HPAI premises teams should further exist and supported although there are no HPAI outbreaks occurring anymore in Nigeria. A total of 3 (60%) supervisors indicated that these teams should further exist, and 2 supervisors indicated that these teams are not longer necessary. Other diseases Newcastle disease was monitored by all three teams, Gumboro by two teams, Fowl Pox and Fowl Cholera by only one team. The mean number of response team visits targeted at Newcastle disease were 17.5 (2006), 27.7 (2007), 31.7 (2008), 27.0 (2009) and 34.7 (2010). PDS Context & Potential Bias The evaluation?s major concerns were: 1) the reliance on qualitative data and self reports of varying levels of credibility with few concrete examples of verifiable events; 2) data robustness as most claims could not be classified into objectively measurable indicators and were too ambiguous to validate, and 3) differential use of words and terms. The evaluation attempted to distinguish between ?actualized? applications and ?suggested? potential applications in order to assess the evidence for and clarify the role of participation in animal health. The understanding of PDS in Nigeria is not restricted to the use of participatory methods, approaches and tools for disease surveillance; rather, the term PDS is used to describe all participation for animal health. The difference between surveys (or appraisals) and surveillance was equally unclear; a 2009 evaluation by a FELTP fellow highlighted that the initiative was a series of village surveys and did not meet the definition of surveillance system Relevance of PDS The evaluation identified two major priorities for HPAI surveillance: early detection and reporting of HPAI and estimating HPAI prevalence. There was consensus that PDS played no role in early detection or reporting of HPAI. However, some suggested that having not identified any cases of HPAI by PDS was consistent with, and potentially supportive of, active country-wide sero-surveys which determined absence of disease. Whilst PDS results were consistent with sero-surveys, representativeness was weak and lacked uniformity: the 800 visits conducted in the year since the last reported cases of HPAI were conducted only in 6 state, attendees were 90% backyard farmers and no standard or random sampling protocol was followed. No inferences on incidence or prevalence were possible from PDS data for any notifiable disease with respect to the country as a whole, the poultry industry or all elements of the poultry production value chain. 139 Whilst not addressing surveillance priorities, village visits provided information which lead to mass vaccination campaigns for rural livestock, specifically for ND in Plateau state and for CBPP in Kebbi state. Key informants agreed that attention and resources for building surveillance capacity should not disproportionately focus on PDS. In preparing surveillance officers for another HPAI-type of event, key informants with the most outbreak response experience prioritized training in field epidemiology and outbreak investigation well above PDS. The initiative was not equally relevant to all stakeholders as village visit reports were not uniformly shared. In cases such as the vaccination campaigns, the data was shared among the decision makers, namely the state level actors. In most cases, however, reports were sent directly to the PDS focal point in the national office, bypassing several groups, including Local Government Area coordinators. Effectiveness of PDS for surveillance As the last reported case of HPAI occurred prior to the first PE training course, no cases of HPAI were identified by PDS. The evaluation identified that in the approximately 800 PDS village visits, no specimens were collected for analysis and no cases of any notifiable disease were detected and reported. Without evidence, quantifiable epidemiological measures of effectiveness, such as sensitivity and positive predictive value, could not be assessed for PDS for any notifiable disease. The evaluation workshop highlighted an important surveillance inconsistency. The estimated prevalence of ND and CBPP in Nigeria suggests that 800 village visits should have yielded several positive events. In fact, all 21 PE practitioners claimed to have ?identified? cases of notifiable diseases during village visits, and there was most certainly discussion of historical outbreaks. However, no specimens were collected and no reports were submitted, making validation impossible. This example underscores the challenges of systemic under-reporting: popular modalities such as PDS, which is claimed to motivate the veterinary sector and improve relationships with farmers, does not in fact, result in improved surveillance. Effectiveness of the intervention was also compromised by data incompatibility. PDS data is complied in a standalone dataset. A PDS village visit summary is not equivalent to a confirmed or suspected case, which is the absolute requirement for entry into NADIS, the conventional national surveillance database. Consequently, PDS data is not integrated or analyzed with NADIS data. In addition, the evaluation identified that PDS data is not analyzed or otherwise communicated even though capacity building in this aspect was conducted. Efficiency of PDS for outbreaks Using timeliness as a measure of efficiency, none of the PDS initiatives in Nigeria were efficient. The first PDS course and related village visits began in October 2008, fully 3 months after the last reported outbreak of HPAI (Figure 11). (The four peaks in PDS visits correspond to: October 2008 140 (the first introductory PE course); February 2009 (the second introductory PE course); June 2009 (refresher training), and October 2010 (World Bank introductory PE course). Figure 12. HPAI outbreaks reported by farmers (blue) and EDRAIA PDS visits (pink) and World Bank PDS visits (violet). The major actions resulting from PDS were two state-wide vaccination campaigns (not for HPAI), the first in mid to late 2010. The decisions to vaccinate were not direct responses to stem ongoing outbreaks, rather, they resulted from state level veterinarians making village visits and gaining a better understanding of the challenges faced by farmers. In these two activities, there was no need for quarantine, movement controls or other urgent measures, including reporting and confirmation. As a survey instrument, timeliness of reporting village visits is not critical. Whilst reporting village visit data could be done in a more timely manner, monthly reporting as performed is adequate. Efficiency is also measured by the ability to survey for multiple diseases at once. The evaluation identified that for surveillance, namely case finding, PDS efforts were limited to HPAI as evidenced by no reports or specimens for any other notifiable diseases. Other notifiable diseases were discussed and for purely survey or appraisal purposes, the approach can be considered efficient. Impact of PDS initiative The impact of the initial pilot was the expansion of the number of states represented from 5 states and the FCT to all 36 states. Specifically, World Bank funds were applied replicating the USAID funded EDRSAIA Introductory PE course, with minor modifications, namely increasing trainee numbers from 20 to 120 and reducing course duration from 10 to 6 days. When additional funding was made available, emphasis was laid on disease detection rather than disease reporting. Greater evidence of ownership would have been demonstrated by adapting 141 the pilot to meet alternative needs and priorities, including for instance, activities identified by the Epidemiology Unit itself, such as database modifications for integration of PDS data, activities suggested in the FELTP evaluation from mid 2009, such as developing a platform which involves local partners at the grassroots level in surveillance and data entry to reduce reporting delays, or activities identified during the current evaluation such as deploying PDS teams to investigate ongoing outbreaks. During the evaluation, two real and concrete examples of the positive impacts of PDS as a rural survey or appraisal tool emerged. They were the state-wide mass vaccination campaigns in rural areas of Plateau and Kebbi states for ND and CBPP respectively, both demonstrating evidence that visits included discussion of multiple diseases. The real benefits to surveillance of PDS were more difficult to estimate but would appear to be few. Results from approximately 800 EDRSAIA PDS visits, of which an estimated 90% of participants were backyard farmers from 6 states were consistent with the negative findings from the HPAI sero-surveys from various conventional active surveillance initiatives, despite not being representative of the nation or of all poultry industry sectors. The greatest theoretical benefit of PDS as a surveillance tool, as identified by PE practitioners, was potentially for case finding. PDS for HPAI did not result in reports for any other notifiable disease, and the expectation is that participatory case finding would also be single disease focused in a time-limited manner. Overwhelmingly, participants and key informants were positive and enthusiastic about PDS. A hard-to-objectively-define ?feel good? factor emerged which included among others, mention of interaction, improved relationships, recognition of communities as knowledge partners and unverifiable subjective statements of improved reporting. However, the lack of reports of any notifiable disease from roughly 800 village visits, reinforces under-reporting as a significant challenge, despite improved communication, visibility, mutual understanding between vet services and farmers and ?feeling good?. The evidence suggests that PDS is not the right modality for improving surveillance or compliance to surveillance requirements, even for civil servants. Unfortunately, identifying the ?right? way to improve reporting and surveillance was not within the scope of the current evaluation. Sustainability The evaluation identified that all sectors from NADIS to the academic and private vets were motivated, engaged and interested in pursuing more PDS initiatives. Several practitioner and key informants identified that PDS effectiveness and long-term sustainability required greater 142 involvement of actors at the frontline, specifically private sector veterinarians as well as non- professionals including paravets and community animal surveillance representatives. A major challenge identified was the lack of any standardized guidelines for PDS and no protocol for PDS data analysis. This is also not a common understanding of the entry point for PDS identified cases into the national surveillance algorithm. When asked to identify the point of entry for a case of HPAI (or any other notifiable disease) detected by PDS into the NAICP?s Emergency Action Plan information flow diagram, fewer than 30% of key informants and PE practitioners thought it was the same point as an index case identified by passive surveillance, namely through the reporting channels to the Local Government veterinary officer. Half identified the NADIS Zonal Coordinator and/or the State Director of Veterinary services. Interestingly, there was a common understanding of the reporting line for PDS village reports (all trainees from EDRSAIA and most from World Bank (they have not all completed as of writing) submitted reports once monthly to the PDS focal point). For sustainability, clarifying this misunderstanding about the point of entry into the system for cases and outbreaks detected by PDS is critical. The current evaluation agrees with the earlier FELTP evaluation that PDS is nearly 100% reliant on donor funding (USAID, World Bank, SPINAP) and not sustainable unless externally funded. Village visits from the first two cohorts were discontinued when EDRSAIA funding ended and the few ongoing visits are the final elements of the requirements of the final cohort of a minimum of 30 visits/state. More than one informant identified that evidence of demonstrated results was essential for but did not guarantee sustainability. For surveillance, PDS would have to prove itself as producing actionable information, for example by reducing the number or severity of outbreaks or informing policy at the central level. The evaluation found little evidence of action from PDS as a surveillance tool, though the theoretical possibly exists for case detection in outbreaks. The vaccination campaigns for ND and CBPP are excellent evidence that village visits are useful as survey or needs assessment tools (no actual cases of either ND or CBPP were reported by PDS, it is likely that enough villages identified these as important animal health concerns to the right individuals). Evaluation outcomes Three overarching themes emerged during the evaluation and are presented below. 4) PDS data is not effectively used. PDS data exists in a database separate from the conventional NADIS network and is not analyzed. There is little evidence for the data leading to actions, decisions or changes. 5) PDS did not improve surveillance. PDS successfully engaged various veterinary sectors. However, PDS did not result in increased reporting of any disease despite endemic 143 prevalence estimates for at least two notifiable diseases (ND and CBPP). The emphasis of PDS for surveillance was limited to case finding for HPAI. This is consistent with the best potential value of participatory surveillance as suggested by PE practitioners, namely case finding during outbreaks. Community involvement in case finding during outbreaks would not alter the extractive nature of surveillance. 6) PDS was useful for appraisal or as a survey of animal health concerns. Village visit discussions yielded important community input which informed vaccination campaigns, actions which would not have happened in the absence of the village visits from the PDS project. Strengthening veterinary quarantine stations Quarantine stations Quarantine stations were upgraded to enhance disease surveillance and control in the country. Makurdi is one of the stations that were improved because it is strategically located as it links North and South-East; large volumes of animals therefore pass through the post. The stations selected also had a potential to serve other countries in the West African region. When fully upgraded, the stations were expected to provide facilities for spraying and dipping of confined animals as well as for other clinical procedures that may be required. In this assessment 2 stations were visited and assessed. These are Makurdi and Jebba. The level of renovation/supply of facilities and equipment in each of these stations was assessed and classified into the following three classes: i. Facilities and equipment were supplied or renovated very sufficiently: These included computers, camp bed, handy recharge, rain boots, rain coat, ceiling fan and offices ii. Facilities/equipment supplied or renovated sufficiently: This included office renovation, feed and watering troughs, dog cages and kernels, borehole facility, stables, scanning machine, main gate, fencing, stables, wire mesh, perimeter fence, borehole facility, drainage system, fencing, loading and off loading ramp, inspection gallery, spray races iii. Facilities/equipment supplied or renovated insufficiently Microscope, road, labour line More than half of the equipment supplied or facilities were sufficiently done or provided. Training of the quarantine station officers 144 Sixteen out of 17 officers interviewed (94%) had attended at least one technical training since 1991; a majority (76%, n = 16) of these trainings were offered between 2006 and 2010. NNAICP sponsored 13 out of the 37 (35%) trainings offered after 2006 where 85% (n = 29) of the quarantine staff were trained. Topics covered in these trainings included: emerging animal diseases and zoonotic diseases (3), biosecurity (3), use of computer (1), disease diagnosis (4), disease reporting (1), control of HPAI (11), role of quarantine in control of transboundary animal diseases (3), role of surveillance of HPAI and other transboundary diseases (1), sampling and sample handling (3) and others (12). Quarantine officers interviewed stated that the stations were meant to contribute to the implementation of movement control, disease surveillance and diagnosis (10/13-77%) and revenue collection (2/13-15%). Most animals e.g. cattle, dogs, goats, sheep, caged birds, poultry and wildlife had to be transported through the stations. Secondary data collected indicate that more goats and sheep were transported through these stations compared to the other animals. These records show that 54,521 cattle, 13 dogs, 224,791 goats and sheep, 50,001 poultry and 2 wildlife were transported through these stations per month. The quarantine station officers collaborate with the police, customs, army and staff of the state government where the station was located. The respondents indicated that the level of cooperation between quarantine station officers and traders and other stakeholders was moderate (6/10-60%). Owners were expected to meet the costs of keeping animals in the stations while the government was expected to maintain the quarantine stations, regardless of whether it had been upgraded or not. Concluding remarks Although there has been an improvement in equipment, infrastructure and staff capacity, revenue collection by local government councils has prevented quarantine staff from effectively discharging their functions due to lack of cooperation by livestock traders. The number of upgraded quarantine stations very minute compared to the total number of quarantine stations in Nigeria. The overall impact of the quarantine station improvement is therefore minimal. Conclusions From the various analyses conducted, it appears that the project activities were not implemented at the right time to have an impact on the disease. Some of the interventions, however, such as skills acquired by surveillance and response teams from the various training course they received could still be used to address other disease problems. Apart from culling 145 and compensation measure that was fully supported through the World Bank funds, it was always difficult to separate activities that were implemented by the project from those of the other projects such as FAO, USAID, etc. The secondary data obtained from the project and the desk officers were very valuable in showing processes and interventions implemented at the time of the outbreak. These data were however poorly recorded and there might be a need to address data collection and recording standards within the epidemiology units at state and federal levels. References 1. CARE. Promising practices for community-based surveillance. Experiences from CARE?s work with community-based models of avian and human influenza surveillance in five countries. 2009. 2. CDC Working Group. Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group. MMWR Recomm Rep. 2001. 50(RR- 13):1-35. 3. CDC. Framework for evaluating public health surveillance systems for early detection of outbreaks, recommendations from the CDC Working Group. MMWR. 2004, 53(No. RR-5) 4. Coker R, Rushton J,Mounier-Jack S,Karimuribo E, Lutumba P, Kambarage D,Pfeiffer DU, St?rk E, Rweyemam Mu. Towards a conceptual framework to support one-health research for policy. Lancet, In press. 5. Dufour B. Technical and economic evaluation method for use in improving infectious animal disease surveillance networks. Veterinary Research, 2009, 30: 27-37. 6. FAO Expert Meeting on Surveillance and Diagnosis of Avian Influenza in Asia. Guiding Principles for Highly Pathogenic Avian Influenza Surveillance and Diagnostic Networks in Asia. 2004. 7. Hadorn DC. St?rk KDC. Evaluation and optimization of surveillance systems for rare and emerging infectious diseases. Veterinary Research, 2008, 39:57-78. 8. Krumkamp R, Mounier-Jack S, Ahmad A, Reintjes R, Coker R. Evaluating health systems' preparedness for emerging infectious diseases: a novel conceptual and analytic framework. Health Policy. 2010, 98(2-3):91-7. 9. Meynard JB, Chaudet H, Green AD, Jefferson HL, Texier G, Webber D, Dupuy B, Boutin JP. Proposal of a framework for evaluating military surveillance systems for early detection of outbreaks on duty areas. BMC Public Health. 2008, 8:146. 10. Mounier-Jack S, de Sa J, Krumkamp R, Coker R. A toolkit for rapid assessment of health systems and pandemic influenza preparedness and response: Systematic Rapid Assessment Toolkit (SYSRA). LSHTM. 2008. 11. Perry B. (Team leader) et al. Second Real Time Evaluation of FAO?s work on the HPAI: Evaluation Report. 2010 12. Perry B. Kamarudin MI, Tarazona C. Final Report : Independent Evaluation of FAO?s Participatory Disease Surveillance and Response Programme in Indonesia, 2009 13. Rushton J, Rushton R. Evaluation of the process of PE/PDS introduction and impact of the PDS methodology on the nationa suvillance system in Egypt: Final Report. January 2009. 14. Salman MD. The role of veterinary epidemiology in combating infectious animal diseases on a global scale: the impact of training and outreach programs. Prev Vet Med. 2009. 92(4):284-7 15. Sosin DM. Draft framework for evaluating syndromic surveillance systems. Journal of Urban Health: Bulletin of the New York Academy of Medicine. 2003, 80:2 Supplement 1, i8-i19. 146 16. St?rk KDC, Regula G. Hernandez J, Knopf L, Klemens F, Morris RS, Davies P. Concepts for risk- based surveillance in the field of veterinary medicine and veterinary public health: Review of current approaches. BMC Health Services Research. 2006, 6:20-28. 17. Stegeman, A., Elbers, A.R.W., Smak, J., de Jong, M.C.M., 1999. Quantification of the transmission of classical swine fever virus between herds during the 1997-1998 epidemic in the Netherlands. Preventive Veterinary Medicine. 42, 219 ? 234. 18. USDA/APHIS Veterinary Services. USDA Surveillance and Data Standards. 2006. Chapter 1: Standards for key components of a surveillance system. 19. Ward, M.P., Maftei, D., Apostu, C., Suru, A., 2009. Estimation of the basic reproductive number (Ro) for epidemic highly pathogenic avian influenza subtype H5N1 spread. Epidemiology and Infection. 137, 219 ? 226. 147 ANNEX 3 HUMAN HEALTH REPORT Summary of studies Study Major issues Level n 1. National policy-makers and experts survey ? Relevance of project National 10 2. Sate officers (epidemiology & surveillance) survey ? Current capacity for AI control ? Improvements in general disease surveillance related to project 8 states 6 3.Project desk officers- survey ? Project impact ? Current capacity for AI control 16 4.State officials survey ? Current capacity for AI control ? Other impacts and benefits attributable to project 8 states 31 5.Review of National Influenza Sentinel Surveillance ? Time between receipt of sample and results National Secondary data 6.Review of Swine Influenza surveillance and control ? Project inputs used for swine flu surveillance and control National Secondary data 7.General public survey ? Knowledge, attitude and practice relevant to avian influenza 8 states 152 8.Physician survey ? Knowledge, attitude and practice relevant to avian influenza 8 states 90 9.Schoolchildren survey ? Effect of project on reported hand- washing, hand cleanliness and reported diarrhoea 8 states 966 children in 32 schools Key messages 1. Policy makers and experts ? There was general satisfaction with the project but a perception that as the avian influenza pandemic did not occur, this was not a critical health problem in Nigeria 148 2. State officers (epidemiology & surveillance) ? Avian influenza surveillance and control capabilities currently present in majority of states ? General improvement in surveillance consistent with positive impact of project training and inputs 3. Project desk officers ? Most officers are able to demonstrate carrying out AI surveillance/control related activities ? Officers report having benefited from project supported training and report having cascaded training to other personnel ? Project provided materials are reported to be used for AI control ? Project provided materials are reported to be for swine flu control 4. State officials ? State officials were generally satisfied with the project. However, AI is no longer regarded as a key public health issue 5. Review of National Influenza Sentinel Surveillance ? Avian Influenza was made a notifiable disease ? There was a significant reduction in turn-around time (interval between receipt of specimen and availability of results) from around 200 days to 30 days or less ? There has been improved efficiency of Influenza surveillance through the support of the NAICP and other partners. Nearly 1000 samples from Influenza-like cases are tested annually 6. Review of Swine Influenza surveillance and control ? Project inputs contributed to the diagnosis of the first case of swine flu in Nigeria ? The surveillance system set up for avian influenza (NISS) was also used for swine flu ? Tamiflu? provided by the project was used for the treatment of other influenza ? Designated influenza sentinel sites were not used by the public at a higher level than comparable non-designated facilities 7. General Public KAP ? Penetration of campaign messages was high: 90% have heard about AI ? Knowledge of campaign messages was good: Two thirds know the key messages ? Appropriate channels for AI information dissemination chosen: 92% of people report getting health information from the mass media ? Association between the campaign and knowledge seen: o Those who heard more messages have better knowledge o Knowledge of AI facts covered in campaign is better than knowledge of AI facts not covered in campaign o The channels reported for receiving messages on AI matches well with reported use of channels for other purposes 149 ? Most respondents (70%) believe or strongly believe that government made great efforts to control AI ? Most respondents (75%) believe that people were informed about AI at the time of the outbreak ? Good hand-washing practices (self-reported) were significantly associated with better knowledge of AICP messages ? No sustained decline in poultry eating or poultry keeping is reported which is compatible with successful efforts to prevent adverse effects on the national poultry industry ? Around half respondents believe that the current risk of AI is low ? Many cases of fever and respiratory signs combined with a history of contact with poultry occur and most are not regarded as suspicious for avian influenza. Methods are needed to better survey this group 8. Physician KAP ? Most trainees worked in departments where members of the public with suspect cases of avian influenza were likely to present (assuming no selection bias in our sample). Among trainees, people from Oyo state and men were heavily over-represented. ? Trainees tended to have better knowledge of avian influenza symptoms, case management and notification than non-trainees, but differences were not significant (although neither design nor sample size allowed investigation of differences between trainees and non- trainees). ? Trainees are significantly more likely than non-trainees to be willing to undertake management of human AI cases. ? One in ten physicians reported having encountered a suspect AI case; but guidelines on notification and isolation are not being followed by trainees or non trainees. Only 37% of physicians reported that they had ever reported a notifiable disease; given, that notifiable diseases include AIDS, food poisoning and malaria, it seems there is under-reporting of notifiable diseases 9. School children KAP ? Evidence strong ? Children attending schools participating in the campaign were more likely to report presence of hygienic provisions within the control of the school (e.g. basins for washing, soap, hand- drying facilities). ? Children from schools participating in the campaign were significantly more likely to report that they had washed their hands that day and significantly more likely to report that they washed their hands after handling poultry. ? However, there was no significant difference in the observed hand cleanliness of children who attended schools participating in the campaign and those who did not. Nor was there any significant difference in the reported incidence of diarrhoea in the last month. ? The strongest predictors of observed clean hands were adequate water supplies, presence of toilets (with separate facilities for girls), and provision of soap, hand-washing and hand- drying facilities. ? The strongest predictors of diarrhoea were spatial (the state the school is located in), livestock-keeping or contact with livestock (increases risk), and inadequate water provision and lack of soap (increases risk). 150 ? The (unexpected) importance of animals and diarrhoea suggests further research is needed to investigate possible zoonotic causes of diarrhoea in children. ? After adjusting for confounding and systematic differences, we found that in states which had experienced an outbreak of avian influenza the campaign improved six-fold the odds of having observed clean hands among school-children; this was significant. In states without an outbreak there was no significant association with hand cleanliness and participation in the campaign. ? Without investments in provision of hygiene infrastructure, campaigns on hygiene are likely to improve attitudes, but may not improve hygiene or health-related outcomes such as diarrhoea. The strong association between household livestock-keeping and increased self- reported diarrhoea in children warrants further investigation. Study 1 Policy-makers and experts: Objective: To assess the perception of project impact by key stakeholders Methodology: We purposively selected 10 national-level policy makers and experts known to be knowledgeable about avian influenza control and were able to provide an external perspective. The interviewees were: ? Director of Special Project ? Chief Consultant Epidemiologist ? Head of Public Health Laboratory ? Scientific Officer (Surveillance) ? Deputy Director (Ministry of Information) ? WHO Director of Disease Control DDC ( focal person on Avian Influenza) ? Local experts ----Academicians (4) Only two persons had some involvement in the project (scientific officer: focal person on Influenza surveillance and the Deputy Director, Communications: closely involved in message development and other communication interventions). No-one contacted refused to be interviewed. They were asked to rate fifteen aspects of the project according to importance/relevance and/or appropriateness, with 1 corresponding to not important and 5 to maximum importance. These scores were standardised to 100. After the rating, open-ended questions were used to elicit further opinions of the project. Comparisons between different groups of experts was carried out using the ttest. Results: Relevance and appropriateness of project Policymakers and experts assigned an overall mean score of 82 (range 67-92%) to different aspects of the project indicating a high level of satisfaction. Satisfaction was highest for training, communication and provision of infrastructure. Use of the Integrated Pandemic Preparedness and Response Plan and the importance of the problem of AI ranked lowest. This probably 151 reflects the very low mortality and morbidity attributable to HPAI in Nigeria. Respondents felt AI was not as important as malaria, tuberculosis or HIV/AIDS and a relatively low rating was given to the importance of AI as a health problem in Nigeria (mean score 60). However, a higher rating was given to the project addressing a genuine threat (mean score 66): this may reflect that while AI did not turn out to have any substantial human health impacts, best evidence at the time of the outbreak was AI had the potential to be a serious human health problem in Nigeria and warranted substantial input. Academicians generally rated the NAICP project significantly less relevant than did policy makers (55 versus 65; p =0.001). Open questions added some detail to the ratings provided. For example, one concern was that the appointment of Human Health Desk Officers amounted to creating a parallel structure within the Ministry of Health and this resulted in conflicts with state epidemiologists. However, no major criticisms were made of project activities, outputs, outcomes or impacts. Rating of NAICP human health strategies/activities by national policy-makers and experts QUESTI ON NO. Relevance and/or importance of different aspects of NAICP project SCORE Max=100 15 Messages encouraging the general public to seek help from health facilities in case of suspected Avian Flu. 98 10 Nation-wide training of doctors on AI case management 98 12 Communication strategies and activities 96 8 Supply of computers and software to the 36 states 96* 9 Training of laboratory personnel on PCR virological diagnosis 96* 13 Messages for school children to wash their hands before and after meals 92 14 Messages for the general public to eat well cooked chicken 92 3 Upgrading of and equipping influenza diagnostic laboratories 90 5 Integrated Plans in the response 90* 7 Medical Waste Management Plan 78 6 Appointment of Desk Officers 76 11 Stockpiling of antivirals 74 4 Use of Integrated Pandemic Preparedness Plan 70** 2 AI a genuine public health threat/problem 66 1 Importance of Avian Influenza as a public health problem 60 * One respondent gave no opinion ** two respondents gave no opinion Key messages: ? There was general satisfaction with the project but a perception that as the avian influenza pandemic did not occur, this was not a critical health problem in Nigeria 152 Study 2 State epidemiologists or disease surveillance officers (state level experts) Objective: To assess current capabilities of surveillance and response to avian influenza and improvements in human disease surveillance attributable to the project Methodology: The survey took place in the eight states selected for surveys ? four considered high risk for AI and four similar but considered low risk for avian influenza. Questionnaires were developed and interviews conducted with the State Epidemiologist in six states and the State Disease Surveillance Officer in two states ? these officers were not employed by the project but were familiar with it. Primary data was obtained on Rapid Response teams and other aspects of disease surveillance and control capability including reporting of other notifiable diseases (malaria). ANOVA analysis was used to compare the mean number of reported malaria cases per year on the null assumption that the means would be the same. Results: Aspects of the project was considered relevant and appropriate by state officials The state level experts considered the appointment of Desk officers to be highly relevant and the Integrated plan to be useful. As for national experts, they gave a lower rating to the importance of AI as a human health problem Relevance and/or importance of project activities SCORE Appointment of Desk Officers 92 Usefulness of Integrated Plan in the response to the outbreak in this State 84 Extent the Integrated Plan was applied 78 AI a genuine public health threat/problem 62 Avian influenza surveillance and control capabilities are currently present in majority of states In five of the eight states, state officials reported that there was an AI work plan in use. Five states also have functional Rapid Response teams, as defined by evidence of regular meetings (meeting monthly in four states and quarterly in one. In four of the eight states (50%), state officials reported that copies of the Integrated National Avian and Pandemic Influenza Response Plan were available. In six of the eight states (75%) there is an electronic surveillance database. Analysis is done manually in one state while five do so electronically. 153 General improvement in surveillance consistent with positive impact of project training and inputs Only two states (Kano and Lagos) reported suspected cases of Avian Influenza. Kano reported a case in 2008 and 2009 while Lagos reported in 2008. This probably reflects the failure of AI to become established in bird populations and hence little risk to humans. However, as discussed later there are many cases in the community that meet the definition of AI suspect cases but are escaping surveillance for various reasons. Secondary data showed an increase in reporting of notifiable disease, consistent with a positive impact of project training and material inputs (see example from Anambra in Figure x)However, given the high prevalence of many notifiable diseases in Nigeria (including diarrhoea, malaria, dysentery, tuberculosis, pneumonia, HIV-AIDS) and the number of notifications, it is likely that the majority of suspect notifiable diseases are not being reported. For example, malaria is endemic throughout the country with more than 90% of the total population at risk of stable endemic malaria). At least 50% of the population suffers from at least one episode of malaria each year (FMOH 2001). Comparing this proportion with the number of cases reported, shows considerable under-reporting (Table x)> Table x shows the tendency for increased reporting of malaria cases by year: however, this was not significant (p=0.24). 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec No . of c ase s Month Reported cases of Notifiable diseases in Anambra State, 2008-2010 2008 2009 2010 154 Cases of malaria reported per year in the eight states surveyed YEAR NASARAWA JIGAWA ANAMBRA PLATEAU KANO LAGOS 2008 621,49 31,196 143,001 168,386 --- 46,625 2009 541,60 24,826 121,497 171,240 665,276 --- 2010 139,428 371,594 152,074 114,275 654,016 ** Population 1,800,000 4,300,000 4,100,000 3,100,000 9,300,000 9,000,000 There was also a tendency to improvement in the submission of reports from LGAs: on average 97% of the LGAs submitted their monthly surveillance data in 2008 and 100% in 2010; this was significantly higher: (p=0.006) Percentage of LGA IDSR results submitted to States YEAR ANAMBRA JIGAWA NASARAWA PLATEAU KANO LAGOS 2008 97.2 98.5 93.6 100 No data 95.8 2009 100 85.8 91.7 99.5 100 100 2010 100 100 100 No data 100 97* ? Based on five months of report There was also a trend towards an increase in diseases investigated by states. Between 2007 and 2010, a total of 117 outbreaks were reported to have occurred in the 8 states of which 62 (53%) had been investigated. The percentage of outbreaks investigated increased significantly from 44.8% in 2007 to 58% in 2009, but declined to 55% in 2010. The general trend to improvement on three different measures of disease reporting (notifiable diseases other than AI; percentage of LGAs submitting reports; disease outbreak investigation) is encouraging. Moreover, state officials considered project inputs contributed to this. However, many other factors could affect reporting and given the absence of a control group or detailed baseline data, it is not possible to attribute improvements directly to the project. Key messages ? Avian influenza surveillance and control capabilities currently present in majority of states ? General improvement in surveillance consistent with positive impact of project training and inputs Study 3 Project desk officers Objective: To assess current capabilities of surveillance and response for avian influenza and other impacts and benefits of the project perceived by project desk officers 155 Methodology: Interviews were conducted with sixteen Desk Officers (eight Human Health Desk Officers and eight Communications Desk Officers) from the eight states visited (four at high risk for AI and four at low risk). Results: Perceived impacts of project by desk officers The desk officers reported that the project had positive impacts on control of AI. They considered that the major impact of the project was in creating awareness on AI, supporting and strengthening disease surveillance and fostering inter-sectoral collaboration. Establishment of Rapid Response Teams, Inter-ministerial committees and strengthening disease surveillance were considered the two most important helpful aspects of the project to the states. Most officers are able to demonstrate carrying out AI surveillance/control related activities Fifteen of the 16 Desk Officers (94%) reported having a copy of the Integrated National Avian Influenza Response plan and 12 of them (75%) could show a copy. Of the three who did not have a copy, one was able to indicate a source where he/she could get a copy. Al 16 Desk Officers or their predecessors had prepared AI work plans for utilizing funds from the NAICP and 13 could show a copy of the plan. The major elements of the work plan included securing and furnishing office accommodation, production of communication materials and strengthening surveillance activities. More than two-thirds of the proposed activities in the workplan were reported as completed. Fourteen of the 16 desk officers submitted their activity reports to the relevant component coordinators. Human Health and Communications Desk officers reported good collaboration with their Animal Health counterparts. The team in Anambra reported that they had been able to respond to and contain an outbreak of AI in poultry at in collaboration with animal health colleagues. The Kano team responded to AI outbreak in Kano without needing to wait for external assistance or guidance. Desk officers reported better active surveillance for AI in many of the states. Officers reported having benefited from project supported training and report having cascaded training to other personnel All the desk officers had attended one or more NAICP-supported training. Nine had attended training on case management, 10 on Disease surveillance and 13 on pandemic preparedness and response. Other trainings attended include outbreak investigations, emergency and risk communications, strategic communication and planning and medical waste management. All the desk officers rated the various trainings as very useful Desk officers report they were able to carry out trainings at the LGA level including AI case identification and management, interpersonal communication skills, outbreak investigations and disease surveillance, role of communication in attitude change, waste management, etc. The 156 trainings were organized for both small and large numbers of participants ranging from 13 to 132. Fifteen of the 16 officers still find the trainings useful in their current designation and offices. They are applying training to communications, response to other disease outbreaks in the state, proper identification and management of Influenza-like illness, propagation of government policies and programmes, implementation of Integrated Disease Surveillance and Response (IDSR), etc. Project provided materials are reported to be used for AI control Eleven (68.75%) of the desk officers in 7 of the 8 states indicated that NAICP provided them with equipments and supplies for AI-related activities. The equipments and supplies provided included computer and accessories, personal protective equipment, communication materials (t- shirts, posters, pamphlets), tamiflu, photocopying machines, refrigerators, etc. Most of the equipments and supplies were currently still being used. These equipments helped the states in various ways. Computers were used for management of surveillance database, and writing of reports (5 of 8 states). The photocopying machine assisted in mass production of training materials and other documents. Tamiflu was used as prophylactic basis during outbreaks and PPEs were used to protect field officers during outbreak investigations and used in the hospital to protect health workers. After the HPAI outbreaks, equipment continues to be of use to the states. However PPEs, tamiflu were not reported to be in current use. Project provided materials are reported to be for swine flu control PPEs provided by the Project was used by States in swine flu investigations. The Lagos State Rapid Response team in particular used the PPEs and tamiflu provided by the project used during the swine flu investigations. Structures for avian flu surveillance and diagnostics were used for detection and surveillance of swine flu Key messages: ? Most officers are able to demonstrate carrying out AI surveillance/control related activities ? Officers report having benefited from project supported training and report having cascaded training to other personnel ? Project provided materials are reported to be used for AI control ? Project provided materials are reported to be for swine flu control 157 Study 4 State Officials Objective: To assess current capabilities of surveillance and response for avian influenza and other impacts and benefits of the project perceived by state officials outside the project Methodology: Interviews were conducted with Permanent Secretaries, Directors of Disease Control/Primary Health Care, State Epidemiologists, State Disease Surveillance Officers and State M&E Officers in the 8 target states. A total of 31 respondents served as the key informants. Data were collected using interviewer administered semi-structured interview schedule. Results: Avian influenza was not regarded as a key public health issue. However, most (71%) report the project addressed an important threat: only 16% believed it addressed a very important threat. Response plans were not widely available at state level (although they were widely available at Desk Officer level). Twenty-four respondents (77%) indicated that the Ministry had a copy of the Integrated National Avian and Pandemic Influenza Response Plan and only 11 (35.5%) indicated that the State still has a copy. Five on the respondents (16.1%) did not know whether or not a copy was available. Only 46% felt that the integrated response plan was useful to the AI response in the state and 11% felt that the plan was not useful. Overall a mean rating score of 78 suggests that generally the plan was perceived to be useful to the State in their AI response. Respondents considered the most useful aspects of the plan were Surveillance, Outbreak Response and Containment and Social Mobilization. The aspects that they found to be least useful were Monitoring and Evaluation and Logistics. It was agreed that that appointment of desk officers was a very relevant strategy. The perceived roles of HH desk officers were listed as coordination of AI-related activities in the State while the Communication desk officers were expected to sensitize and educate the public. 0 5 10 15 20 25 Avian Influenza Tuberculosis Measles HIV/AIDS No. of responses Reported diseases of public health importance in the last five years 158 The respondents reported that the equipment supplied by the NAICP was largely used during the AI outbreak. But at the time of this survey only 33% reported frequent use of these equipment. Key messages State officials were generally satisfied with the project. However, AI is no longer regarded as a key public health issue Study 5 Review of NISS sample turnaround as a result of project inputs Methodology: Expert opinion on establishment of National Influenza Sentinel Surveillance (NISS) and Review of NISS database to assess evidence of improved efficiency. Results: There was a significant reduction in turn-around time (interval between receipt of specimen and availability of results) from around 200 days to 30 days or less. Currently the NIRL has the ability to produce results within 24 hours of receipt for samples especially for samples from suspected influenza outbreaks. From the database it is evident that results from routine surveillance can be available in less than a week. There has been improved efficiency of Influenza surveillance through the support of the NAICP and other partners. Nearly 1000 samples from Influenza-like cases are tested annually. Over 3000 samples have been tested to date. The NAICP provided and supported maintenance of communication infrastructure, vehicle (pick- up van) and other logistics that facilitated influenza surveillance activities. 159 Study 6 Review of evidence of Project supporting Management of Swine flu (2009 Pandemic H1N1) The influenza diagnostic system for Avian Flu was also very useful for the testing of 2009 pandemic H1N1 samples. In particular, it enabled to country to be able to detect the first few as well as subsequent cases of swine flu in the country. The evidence for this is summarized below: ? A case of a lady that had just travelled in from America (and who had previous contact with a confirmed case of Influenza A (H1N1) in America) had symptoms of fever and cough. The LSMOH surveillance team (RRT) was deployed to investigate the suspected case. Nasopharyngeal and Oropharyngeal samples were taken and sent to the National laboratory Asokoro. The Asokoro Influenza lab confirmed that the sample was positive for pandemic influenza A swine H1. Aliquotes sent to CDC Atlanta tested positive for H1N1. Following outbreak of pandemic H1N1, the surveillance system, which was originally set up for H5N1, was expanded to include surveillance for pH1N1 Each state was given a seed stock of 1000 doses of Tamiflu?. We determined usage as the difference between quantity provided by the project and the quantity left. The table below shows the doses of Tamiflu? left in the various states as reported by State Officials. Amount of Tamiflu?available at the States surveyed State Tamiflu Given to States Quantity Left at State Estimated Doses used* Enugu 1000 1000 0 Jigawa 1000 300 700 Lagos 1000 300 700 Nasarawa 1000 750 250 Oyo 1000 750 250 Plateau 1000 300 700 Available data do not suggest that the change in outpatient attendance occurring at sentinel sites was significantly different from observed change in non-sentinel sites. For example data from Enugu and Anambra States (in the same geopolitical zone) were compared. The percentage change in base ?year (2007) attendance in Nnewi (sentinel sites) was compared with the change observed at UNTH Enugu (non-sentinel). The difference was not statistically significant (0.54) 160 Key messages ? Project inputs contributed to the diagnosis of the first case of swine flu in Nigeria ? The surveillance system set up for avian influenza (NISS) was also used for swine flu ? Tamiflu? provided by the project was used for the treatment of other influenza ? Designated influenza sentinel sites were not used by the public at a higher level than comparable non-designated facilities -50 0 50 100 150 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Percentage change in outpatient attendance (2007-2008) in Nnewi and UNTH Nnewi, Anambra UNTH, Enugu 161 Communications activities The project distributed a range of communications materials and estimates that these reached millions of Nigerians as shown in the matrix below Compiled from interviews with project members Messages Quantity Target group Dates Where messages were sent Audience Information sheets ? Bird flu ? Swine flu Other written media: 3. 4. AI Branded folders ? Posters ? 450, 000 ? FAQs ? 280, 000 ? Leaflets ? 750, 000 ? Large Logo ? 10, 000 ? Small logo ? 70, 000 ? Msg Stickers ? 100, 000 ? Mouse Pads ? 5, 000 ? Folders 5, 000 General public, schools & poultry farmers, live bird markets 2007 ?10 Nationwide ~20,000,000 (assumes seen by 10 people) Banners/billboards ? Metal boards ? Billboards ? Roll up banners ? 58 metal boards ? 37 billboards ? 100 banners General Public 2008-10 ? Nationwide ? Geo-political zones (6) ? States (12) ~ 20 million ? TV documentaries ? TV slots ? 22 episodes ? ? 12 slots ? General Public ? Football fans ? 2007 ? World Cup ? NTA network ? National ? 30 million ? 20 million ? Radio Jingles ? Radio documentaries ? Radio Drama (Hausa) ? 168 jingles ? 26 episodes ? 13 episodes ? General Public ? General public ? Hausa listeners ? 2007 ? 11 ? 2008-10 ? 2009 ? Nationwide, FRCN, RayPower network, HOT FM, Vision FM ? Nationwide ? 19 northern states radio stations ? 50 million ? 50 million ? 10 million Journalist training 6 workshops Media chiefs & reporters 2007 -10 Lagos, Kaduna, Jos, Enugu, Abuja, Kano & Owerri 500 journalists Clothing ? Caps ? Tee-shirts ? Aprons ? Branded Overalls ? A I branded bags ? A I branded towels ? 8, 000 ? 15, 000 ? 3, 000 ? 3, 000 ? 2, 000 ? 1, 500 General Public, LBM & Poultry farms 2007 ?10 Nationwide ~3,000,000 (Assumes each item seen by 100 people) Bird flu watch magazine 5 editions @ 10, 000 = 50, 000 General Public 2009 ? 10 Nigeria and outside Nigeria ~500,000 Project website 1 website General public 2007 to date Worldwide ~ 2 million Sensitization rallies in motor parks, poultry markets and grazing reserves 7 rallies Mobile and migrant population 2008 & 2009 FCT, Lagos, Kano, Awka & Maiduguri ~7,000 Integrated response plan 5, 500 Ministries, States, Libraries etc 2008 Nationwide 50, 000 162 The communication channels chosen by the project seemed well suited to the channels available in Nigeria (see box). Media channels available in Nigeria In Nigeria, the public sector has dominated television and radio broadcasting (probably because both private and public stations compete for the same pool of advertising revenue while the state can subsidise high operating costs through public funding). The largest broadcasting companies are the government-owned Federal Radio Corporation of Nigeria (FRCN) and the Nigerian Television Authority (NTA). The NTA operates about 97 stations across the country, creating the largest television network on the continent. Each state also has a broadcasting company that broadcasts one or two locally operated terrestrial stations. This means that there are 39 radio stations and 37 television stations owned by the different state government across the country. Most operate partially independent of the state governments. According to the National Bureau of Statistics, 88 per cent of households have access to radio (ownership: 72.9 per cent) and 51 per cent to television (ownership: 36.6%) Apart from the Lagos and North West regions, state radio is still dominant in the country. FRCN claims to be ?reaching an estimated 120 million listeners, broadcasting in English and 15 local languages? RayPower FM is a private station listed in Abuja, Plateau and Rivers and is considered one of the most popular stations. Hot FM and Vision FM are also private stations listed in Abuja. There are more than 150 national and local newspapers and publications most of which are privately owned. A study commissioned by the Advertisers Association of Nigeria (ADVAN), the Media Independent Practitioners Association of Nigeria (MIPAN) and the Association of Advertising Agencies of Nigeria (AAAN) in 2009 estimated the newspaper market in Nigeria at 1.9 million papers sold daily with six papers constituting 92% of sales. The World Bank estimated newspaper circulation at 25 per 1000 people. 7. Survey of general public knowledge, attitude and practice Methodology Population of interest: General adult public in Nigeria ? around 80 million Population reached by project: AICP intended to reach: all Time of interventions: from 2006-2011, more targeted messages appeared later Sampled: 152 people Study design: cross-sectional survey Sample selection: eight states were selected, four that were considered high risk for avian influenza outbreaks and four considered low risks (n=74) 163 Data collection and analysis: A structured interview was used to collect data from the general population. Eligible respondents were men and women aged 15 years and above. In each state, two localities were identified. One of the localities was the State capital while the other was a rural town close to the State capital. In the selected communities, a busy market and/or a motor-park were identified. Ten adults were interviewed in each location (urban and rural). Data were entered into an excel spreadsheet and statistical comparisons used Stata. Data was analysed without taking into account clustering as sample sizes were insufficient to allow multi- level modelling. Questions: We tested knowledge of bird flu and ascertained attitudes towards the disease and control campaign. We evaluated perception of receiving messages about AI and the channels used for this. We also assessed reported declines in poultry-keeping and consumption as a result of AI. We also looked at occurrence and health seeking behaviour for symptoms compatible with AI Biases and caution with interpreting results: Although results were internally consistent and compatible with the literature in change on knowledge, attitude and practice, the sample size was relatively small making multi-level modelling impractical and the tests used may over-estimate the significance of difference. Because of the sampling strategy, remote rural areas were not included and this may introduce systematic bias as these areas are most difficult to reach with extension messages. The results obtained were self-reports and hence may be inaccurate- in particular behaviour on hand- washing and health seeking behaviour are often incorrectly reported because of normative biases. Some of the question referred to the time of the outbreak, which was five years ago opening the possibility of recall bias. During the AI outbreak there were many different providers of information ? both public, civil society, and media. The general public would not be expected to recall who provided this information and hence it is not possible to definitively attribute changes in knowledge, attitude and practice to the project communications. Demographic The sample was 51% female, and 43% urban, which corresponds well with the population of Nigeria (51% male and 50% urban). Our respondents had a higher than average level of literacy (77% of women and 86% of men versus 68% and 76% national). This probably reflects the bias towards selection from non-remote rural areas. Age yrs 15-19 20-24 24-29 30-34 35-39 40-49 50+ Female % 1 12 12 16 16 26 15 Male% 0 4 10 17 20 37 11 164 All 1 8 11 17 18 31 13 Knowledge of AI People had a good knowledge of avian influenza; around two thirds of people asked knew about risk and transmission of avian influenza. Only 18% of the respondents did not get any questions correct. Correct % Incorrect Don?t know n 1. People can get bird flu 69 15 17 150 2. Contact with sick birds a risk 76 8 15 99 3. Contact with sick people a risk 49 29 22 95 4. Eating under-cooked poultry a risk 72 6 21 99 5. Eating well-cooked poultry not a risk 67 4 23 96 6. Insect bites not a risk 36 18 45 96 Awareness was higher for those messages that were part of the communication campaign (messages 1, 2, 4, and 5). This suggests that the communication campaign had a role in disseminating information. There was wide variation in knowledge across states and this was not related to state having experience of AI. Neither was variation associated with differences in human development between states High AI risk states Low AI risk states Plateau Kano Anambra Lagos Nasarawa Jigawa Oyo Enugu Know people can get AI (%) 70 57 50 19 100 95 80 75 Know contact with sick birds is a risk for AI (%) 80 57 44 75 100 90 86 64 165 Know contact with sick people is a risk for AI (%) 47 36 38 75 75 70 40 7 Know raw poultry is a risk for AI (%) 82 43 56 33 100 95 86 64 Know cooked poultry safe (%) 80 100 94 67 88 100 100 100 Know insects don't spread AI (%) 73 93 81 67 59 85 100 93 Mean number NAICP messages known 2.6 2.6 2.1 0.5 3.5 3.8 2.6 2.8 Mean number AI facts known 3.5 3.9 3.1 0.7 4.6 5.3 3.3 3.6 Human development index 0.392 0.436 0.427 0.607 0.488 0.362 0.478 0.502 Human poverty 36.5 43 22.8 14.5 38.5 48.4 21.9 28.6 There was no difference between rural and urban respondents in mean number of messages known or among literate and illiterate people, but men knew significantly more messages than women (2.7 versus 2.3; p=0.04). This may reflect the higher literacy of men. Reported awareness of messages about AI People had heard or seen an average of 3.4 channels providing messages on avian influenza (range 0-9). The most frequently reported sources were radio programs. Only 11% (n=153) of the respondents reported not hearing a message from any source. Source % Radio programs 65 Radio jingles 59 Television adverts 54 Television documentaries 42 Newspaper 34 Pamphlets 22 Billboards 22 Mobile phone 16 Internet 10 Radio programs and jingles were the media channel most reported as a source of AI messages and internet least. 166 The message most often remembered was to report sick or dead birds followed by cooking chickens well. Separating birds according to type (e.g. keeping ducks and chickens apart) being the message least likely to be spontaneously recalled. Knowledge of AI significantly related to number of messages reported received People who knew that humans could get bird flu reported significantly more channels providing messages than people who did not know humans were susceptible (3.9 versus 2.5, p= 0.002); the same was the case for people who correctly knew that raw birds were a risk (4.0 messages versus 1.8 p=0.0001). However, for messages which were not part of the campaign (e.g. the role of insects in transmission) there was no significant difference in the number of messages recalled (3.3 versus 3.6, p=0.63). This suggests that messages heard had a role in increasing knowledge (but possibility confounding). Choice of channel for dissemination of AI messages The mass media was the most frequently mentioned source of health information with 92% of respondents getting information by this route. This justifies the extensive use of radio and television by the campaign and the training of journalists 0 10 20 30 40 50 60 70 80 90 Separate birds according to type Wash your hands with soap and water Keep birds from the house Cook your chicken very well Report sick or dead birds 167 Media sources used The media source accessed most frequently by the respondents was the radio. Daily Weekly Monthly Rarely Never Radio jingles 51 16 11 11 5 Radio programs 50 17 10 12 6 TV ads 42 14 6 18 14 Mobile phone 34 11 3 11 29 TV documentaries 26 16 9 27 13 Billboards 21 8 8 20 29 Newspaper 16 13 10 29 24 Internet 7 5 6 12 58 Pamphlets 6 8 8 39 30 There was a good match between media reported to be used daily or weekly and reports of hearing messages about AI on this media. The exception was mobile phones which were not used in the campaign 0 10 20 30 40 50 60 70 80 90 100 168 Attitude towards AI Most people considered there was no or little risk of AI happening in their locality. Similarly, only 21% of people disagreed or disagreed strongly with the statement that ?The danger from bird flu is now passed?. Not surprisingly, there was a high correlation (0.76) in perceived risk in wild birds and in poultry AI in wild birds here (%) AI in poultry here (%) No risk 44 34 Little risk 25 22 Medium risk 14 23 High risk 11 16 No response 6 4 However, more than half the respondents agreed with the statement that ?Poultry can spread other serious diseases? and only 13% disagreed with the statement. This indicates no general complacency about animal diseases. Positive attitude to government control campaign but media considered alarmist Most respondents were positive towards the campaign, 70% of those who responded agreeing or agreeing strongly with the statement that ?Government made great efforts to control bird flu?. However, most people thought messages had been alarming, 67% agreeing with the statement ?There were too many frightening-stories in the media about bird flu?. 0 10 20 30 40 50 60 70 80 Got AI message Use daily/weekly 169 75% agreed or strongly agreed that ?Most people knew about bird flu at the time of the outbreak? Strongly agree Moderat e agree Neutral Mod disagree Strong disagree No answer n I reduced eating chicken during the time of bird flu 49 14 11 5 19 1 150 The danger from bird flu is now passed 28 26 25 9 11 1 149 Most people knew about bird flu at the time of the outbreak 54 21 12 7 3 2 149 Poultry can spread other serious diseases 32 24 21 9 11 2 149 Government made great efforts to control bird flu 47 23 19 6 5 1 150 There were too many frightening stories in the media about bird flu 49 18 16 10 5 1 148 Reported practice No sustained decrease in poultry consumption linked to AI outbreak One of the objectives of the campaign was to protect the poultry industry from unwarranted declines in consumption. This seems to have been reasonably successful. People report a decline in consumption during the outbreak 63% of people agreed or strongly agreed with the statement that ?I reduced eating chicken during the time of bird flu?. In another question on household consumption the decline in poultry consumption during the outbreak was less evident: reported daily or weekly poultry consumption dropped by 22% (significant at p=0.02, paired ttest). However, there was no significant difference in the proportion of those consuming chicken daily or weekly currently and those consuming chicken daily or weekly before the outbreak (p=0.39, paired ttest). Before AI outbreak % During AI outbreak % Current % Eat chicken daily 3 3 2 Weekly 33 25 30 Monthly 33 29 41 Rarely 29 32 11 170 Never 1 8 14 Daily or weekly 36 29 32 High level of poultry keeping and no evidence for decline after the AI outbreak The majority (64%) of the respondents currently keep chickens or other poultry, while 75% keep some animals on the premises and 68% keep domestic livestock. Comparing the proportions of those who have kept poultry in the past but now, with those who kept other animals in the past but have stopped now, there is no evidence that people are dropping out of poultry keeping at a higher rate. Keep now Stopped keeping % % Chickens 63 8 Other poultry 21 7 Farm animals other than poultry 29 9 Dogs/cats 31 9 More than two thirds of people (68%) report that they come close enough to touch chicken or other poultry daily or weekly. Among those who keep poultry, 81% report close contact while among those who don?t keep poultry only 43% report close contact (p=0.000). However, there was no significant difference in contact reported between those who eat chicken daily/weekly and those who eat chicken less regularly. Good hand-washing practices were significantly associated with better knowledge of AICP messages Around 83% of people reported satisfactory hand cleaning (use of soap or another cleaning agent). Freq. Percent Wash with water only 25 17 Wash with water and soap 101 67 171 Wash with water and some other cleaner 22 15 Other way of washing 2 1 Good hand-washing was associated with having received messages on AI by multiple channels but this was significant only at p=0.055. Group Obs Number of channels giving messages on AI In adequate hand washing 28 2.6 Adequate hand washing 123 3.6 Many cases of fever and respiratory signs combined with a history of contact with poultry occur and most are not regarded as suspicious for avian influenza Our survey like others finds a high level of self-reported illness in the community and a low level of consultation at health facility or pharmacies. % reporting illness % of these getting treatment outside Facility % of these getting treatment Health Facility Sore throat in last month 16 85 15 Fever last month 36 75 25 Cough last month 28 83 17 Given the very high level of poultry keeping (64%) and of frequent contact with poultry (68%), and the high mortality of poultry in Nigeria, the implication is clear than many cases which could fit the case definition occur in the community and that no action is taken about these. In our survey 17% of respondents reported fever and either cough or sore throat or both in the last month and 62% of these reported daily or weekly contact with birds. Poultry in Nigeria have a high mortality rate. Only 32% of these sought treatment at a pharmacy or health facility. If we extrapolate to the population of Nigeria it is clear that every month there are millions of cases which could fit the definition of suspicious for avian influenza. Obviously only a small proportion of these are treated as actually suspicious. (From NISS records (>2000), 6% of people with flu like symptoms sampled (quota sample from 4 hospitals) reported contact with sick or dead poultry and 0.6% reported eating raw or under-cooked poultry.) There are two implications: a) the possibility of avian influenza circulating at a low level; b) the need for surveys to better identify and target those at risk for zoonoses. 172 8. Physicians KAP Most trainees worked in departments where members of the public with suspect cases of avian influenza were likely to present (assuming no selection bias in our sample). Among trainees, people from Oyo state and men were heavily over-represented. Private sector not included but 80% of health care takes place here Trainees tended to have better knowledge of avian influenza symptoms, case management and notification than non-trainees, but differences were not significant (although neither design nor sample size allowed investigation of differences between trainees and non- trainees). Trainees are significantly more likely than non-trainees to be willing to undertake management of human AI cases. One in ten physicians reported having encountered a suspect AI case; but guidelines on notification and isolation are not being followed by trainees or non trainees. Only 37% of physicians reported that they had ever reported a notifiable disease; given, that notifiable diseases include AIDS, food poisoning and malaria, it seems there is a problem with reporting of notifiable diseases. Methodology Population of interest: There are around 40,000 physicians in Nigeria. Of these perhaps 1,000- 10,000 are in the ~100 higher level facilities. ((These include 19 specialist and teaching, 24 federal medical, and 59 tertiary facilities (Federal Ministry of Health, 2005)). Assuming around one 10- 50% of physicians in higher level facilities are in departments where they may first point of contact with suspect AI cases then the target population is from 100-5,000 doctors. NAICP intended to reach: 2,500 people trained in AI diagnosis and treatment with a focus on tertiary level. The project trained 10 trainers from each state (360) including doctors, nurses and pharmacists. These were supposed to conduct further trainings at state level in both private and public sector using the budgets devolved to state level. We could not obtain data on the number trained. Time: 2006-2010 Surveyed: 90 physicians Study design: cross-sectional survey Sample selection: 8 states were selected ? four were considered high risk for AI and four considered low risk. Teaching universities or federal medical centres were targeted, as most training was concentrated here and human avian influenza being a severe disease might be 173 more likely to be encountered at tertiary facilities47. Enumerators visited Department heads of Medicine, Paediatrics, and Family Medicine from and asked them to nominate respondents, without any specification regarding training. Questions: We assessed knowledge on treatment of AI and practices related to the detection, reporting and isolation of suspect cases of human avian influenza Data collection: Biases and caution with interpreting results: The sample selection was by quota sampling and so may not be generalisable to the population of interest. Moreover the sample size was very small (only 11 trainees) making differences difficult to detect. Demography The study comprised 90 doctors from 8 states; 67% were male; 44% were specialists (consultant, registrar, resident or epidemiologist) and 53% were of senior grades (the above plus senior medical officers). Of those interviewed, 89% were in positions where potentially members of the public (relevant discipline). 64% were generalists (family practice, general practice, general outpatients, medicine, community practice); 24% were women/children specialists (paediatrics and obstetrics/gynaecology); and 11% were other (internal medicine, pathology, preventative dentistry). Findings Trained physicians were found in around half the medical facilities visited; however, training was unevenly distributed and women are under-represented Eleven (12.2%) of the respondents reported having received training on AI. This suggests that project training reached a substantial minority of physicians. Moreover, all of those trained were working in a relevant discipline (that is one where they were likely to be consulted by patients with influenza like illness. Of those trained, eight were from Oyo, one each from Nasarawa, Anambra and Jigawa and none from Lagos, Plateau, Enugu or Kano. If this is representative of physicians trained it would suggest that Oyo was much over-represented in training. Another possible bias is in gender representation: no women were trained and 20% of male doctors were trained (p=0.013; Fishers exact). However there were no significant differences in seniority or status between those who received training and those who did not ( 64% of trainees were senior and 45% were specialists ) 47 Some studies from Nigeria report >90% of patients attending tertiary facilities do so without being referred (Akande et al., 2004) 174 No significant difference in knowledge between people who had received training and those who had not, but there was a tendency for trainees to have better knowledge We asked a series of questions around ability to diagnose avian influenza. This included some signs that appear in the case definition of avian influenza for Nigeria (fever, cough and sore throat); some signs that are not specifically named in the case definition but are influenza like and commonly reported in human cases of avian influenza (head ache, muscle aches, nasal discharge); some signs which have been reported for avian influenza but are not mentioned specifically or indirectly in the case definition (diarrhoea, vomiting) and one sign which is neither mentioned in the case definition nor associated with avian influenza (itching). On none of these parameters was there any significant difference between those who had been trained and who had not. On the whole, people who were trained tended to recognise true signs and possible signs which are not in the case definition. We constructed a knowledge score whereby 2 points were given for signs mentioned in the case definition, one for other signs, and two marks deducted for the sign not associated. Trained people scored higher but the result was only significant at p=0.052 (ttest). Untrained Trained Sign mentioned in case definition Fever is a sign 92 100 Sore throat a sign 66 91 Coughing a sign 82 91 Possible sign not mentioned in case definition Nasal discharge a sign 86 91 Muscle aches a sign 49 55 Head ache a sign 37 45 Diarrhea a sign 16 36 Vomiting a sign 14 9 Not a sign Itching a sign 8 0 At least one false sign 27 45 Overall All true signs and no false signs 44 45 Knowledge score 61 73 Among trainees, 100% correctly Oseltamivir (Tamiflu?) as drug of first choice for treatment compared to 75% of non-trainees; however, this difference was not significant. Likewise 100% of trainees correctly said that suspect patients should be isolated compared to 92% of non-trainees; again this difference was not significant. CDC: Close to all respondents (91% or more) correctly identified each of the following signs/symptoms Fever, sore throat, cough, nasal discharge, body ache, headache A large minority (28%) incorrectly indicated rash as a sign/symptom of infection 175 A large percentage responded ?Don?t know? for sore eyes (22%), vomiting (18%) and diarrhea (28% Attitude: Trainees have more positive attitude towards managing suspect AI cases Respondents were asked if a patient presented with symptoms suggestive of avian influenza would they be willing to manage such a case. All trainees indicated they would be willing but only 66% of non-trainees: the difference was significant (p=0.03, Fishers? exact test). Reasons given for reluctance to manage (n=28) are given in table x. Lack of facilities was the most commonly given reason and around a third of respondents gave more than one reason. Only one respondent cited lack of training. Reason not to treat no facilities Contagious nature of disease Not my specialty no drugs to treat other more than 1 reason not to treat % 52 42 22 15 21 36 Practice: Management of AI cases Two out of ten of the trained doctors and 8 out of 77 untrained doctors reported having encountered a suspect human AI case in practice (or 11.5%. of all physicians). Extrapolated over 40,000 physicians this implies at least 40,000 suspect cases of avian influenza. Guidelines on notification and isolation are not being followed by trainees or non trainees National Standard Operating Procedures for the Management of Suspected Human Influenza A/H5N1 Cases in Nigeria state that suspect cases should be immediately notified and immediately isolated. Neither of the trained participants referred patients or mentioned notify under ?other actions?. Neither of the trainees indicated that they isolated the patient Trained Untrained N = 2 N= 8 referred 0 2 treated 2 3 isolated 0 3 other 1 0 Notified disease in past 2 3 176 When asked if they had ever notified a disease, only 37% reported that they had notified disease to local authorities in the past. Given that the list of 23 notifiable diseases includes AIDS, food poisoning, diarrhoea, malaria and pneumonia it is highly implausible that the majority of physicians have never encountered a notifiable disease. This indicates a problem with incentives or mechanisms for reporting The respondents were also asked if they would report avian influenza to the authorities and 97% agreed they would (including all the trainees). There was no statistical difference between trained and untrained in reported willingness to notify authorities. (For comparison, a study of physicians in a tertiary hospital in 2008 found physicians knew the following diseases to be reportable were: avian flu (68.6%), tuberculosis (50.3%), HIV/AIDS (48.4%), pneumonia (12.5%), and malaria (16.3%) (Adefue et al). In a more recent study by CDC in Nigeria: ? 100% of respondents indicated that epidemic diseases should be reported ? 99% of respondents indicated that they would report suspected bird flu in a patient ? 67% responded that they had ever reported a notifiable disease and 33% responded that they had not. 177 9. School Children KAP Children attending schools participating in the campaign were more likely to report presence of hygienic provisions within the control of the school (e.g. basins for washing, soap, hand-drying facilities). Children from schools participating in the campaign were significantly more likely to report that they had washed their hands that day and significantly more likely to report that they washed their hands after handling poultry. However, there was no significant difference in the observed hand cleanliness of children who attended schools participating in the campaign and those who did not. Nor was there any significant difference in the reported incidence of diarrhoea in the last month. The strongest predictors of observed clean hands were adequate water supplies, presence of toilets, and provision of soap, hand-washing and hand-drying facilities. The strongest predictors of diarrhoea were spatial (the state the school is located in), livestock-keeping or contact with livestock (increases risk), and inadequate water provision and lack of soap (increases risk). After adjusting for confounding and systematic differences, we found that in states which had experienced an outbreak of avian influenza the campaign improved six-fold the odds of having clean hands; this was significant. In states without an outbreak there was no significant association with hand cleanliness and participation in the campaign. Without investments in provision of hygiene infrastructure, campaigns on hygiene are likely to improve attitudes but may not improve hand hygiene or health-related outcomes such as diarrhoea. Methodology Population of interest: Secondary school children in Nigeria: 6 million (enrolment is around 30% of potential secondary school going population) NAICP intended to reach: 570,000 Time: 2008, all states targeted and asked to visit at least ? of LGAs. Within LGAs asked to cover both rural and urban Surveyed: 966 children in 32 schools Study design: cross-sectional survey Sample selection: 8 states were selected ? 4 which experienced an avian influenza outbreak and 4 which didn?t. In every State two localities, the State capital and a nearby rural community were selected. Schools in these two localities were classified as ?exposed? if communication interventions were directly implemented during the years of the HPAI outbreak, and non- 178 exposed? if such interventions were not specifically implemented. In each locality, one exposed and one unexposed school were randomly selected. (The list of such schools was provided by the Communications Desk Officer of the State. In States where communication interventions were not specifically implemented in schools, two schools in a state capital and two in a rural were randomly selected. A group approach was adopted in administering the questionnaire. All selected students were assembled in one hall. Data collectors explained each question in detail item by item before responses were solicited Questions: We directly observed the cleanliness of children hands and we also asked questions about their hand-washing behaviour. Proper hand-washing has been shown to reduce diarrhoea, so we asked question on diarrhoea frequency as another check for correct and frequent hand washing. We also asked about other factors that could influence washing and diarrhoea (water provision and sanitation at school and home, contact with animals.) Data collection: In the selected schools class registers were used to systematically select 30 SS2 (Senior Secondary Class 2) students who completed the questionnaire. Biases and caution with interpreting data. The sampling strategy meant remote rural areas were not included. The states where AI outbreaks did not take place may have differed systematically from those where AI outbreaks did take place. The states where campaign interventions did not take place may have differed systematically from those where interventions did take place. Responses on hygienic behaviour are known to be subject to strong normative biases and self-reports are of little reliability. Visual observation of hands for cleanliness is specific but not very sensitive; hands can appear clean and yet be contaminated with faecal and other bacteria. School children demographics Of the children surveyed 54% were male and 46% were female; 50% were urban and the remainder rural ? hence our sample was reasonably representative of the secondary school population. The average age was 17 years with a range from 11 to 26 years. Children reported an average of 10 people in the household, of whom 5 were adults. They reported 66% of households kept chickens and 75% kept some farm animals, while 82% kept farm or companion (dog/cat) animals. Hygienic facilities at schools The schools visited were assessed for provision of hygienic facilities (pupil responses). There were considerable infrastructure problems: nearly a fifth of schools had no toilets and poor water supply (either no running water or use of streams). 179 Pupils at schools involved in the campaign were more likely to report problems with infrastructure which were not within the control of the school (presence of toilets, water infrastructure, rural location) but more likely to report presence of hygienic provision within the control of the school (wash facilities present at the toilets and canteen, soap present at wash facilities). This was significant for two out of six indicators; however the relatively small number of schools (32 schools) meant that test lacked power to detect significant differences even if present. Location and demography All Campaign n=352 No campaign n=614 State with AI outbreak 50% 51% 49% Urban 50% 46 52 School population 1201 1258 1158 Mixed school (boys & girls) 35% 42 32 Infrastructure and hygiene aspects outside of control of school All Campaign No campaign p Toilets present 81% 74 85 0.482 Inadequate water supply 21% 25 19 0.740 Canteen present 97% 91 100 0.482 Infrastructure and hygiene aspects within control of school All Campaign No campaign Wash facilities at toilets 38% 58 26 0.059* Number of wash facilities at toilets 2 2 3 0.571 Hand drying facilities 6% 5 6 0.706 Separate toilets by sexes 26% 16 31 0.399 Wash facilities at canteen 34% 40 31 0.501 180 Soap at canteen 23% 42 11 0.036** Univariate association between reported hand washing, hand cleanliness, and hygiene-related health outcomes and attending a school which participated in the campaign Children from schools participating in the campaign were significantly more likely to report that they had washed their hands that day. (Although, given the high level of self-reported hand washing this only amounted to a 6% increase). Children from schools participating in the campaign were significantly more likely to report that they washed their hands after handling poultry. This increase was substantial. Other differences were not significant Campaign No campaign Risk ratio p Report hand washing today 95.74 91.86 1.05 0.008 Report washing after handling poultry 36.36 23.13 1.36 0.001 Report washing before eating 80.68 75.08 1.02 0.493 Report washing after eating 63.07 59.61 1.01 0.900 Report washing after using toilet 56.82 57.98 0.98 0.611 However, there was no significant differences in the cleanliness of hands of children who attended schools participating in the campaign. Nor was there any significant difference in the reported incidence of diarrhoea in the last month Campaign No campaign Risk ratio p visibly clean hands 83.52 88.76 0.98 0.517 visibly clean nails 67.9 75.9 0.94 0.099 visibly short nails 51.42 55.7 0.92 0.157 Had diarrhoea in past moth 21 22 0.93 0.558 Important determinants of hand cleanliness and hygiene-related health outcomes Unsurprisingly inadequate water, use of borehole water, absence of soap are associated with having visibly dirty hands. 181 One of the strongest and highly significant predictors of clean hands was having separate toilet facilities for girls. Having toilets present at the school, washing facilities at the toilet and canteen and hand drying facilities present at the toilet also reported the odds of having clean hands. An urban location and access to tap water were also important. Two findings were unexpected: having soap present at the canteen as associated with dirty hands. This may be the result of confounding as schools with soap at the canteens were more likely to be urban and in states with outbreaks of AI (factors associated with less clean hands). The associate of dirty hands with reported washing after toilet use probably reflects the unreliability of self-reporting around measures of hygiene. Factors associated with having dirty hands % Case % Control Ratio p inadequate water 18.47 46.36 0.26 0.000 state had outbreak of AI 46.36 70 0.37 0.000 use bore hole water 35.15 58.82 0.38 0.000 report washing after use toilet 67.45 77.27 0.57 0.039 soap present at canteen 23.6 32.67 0.64 0.046 no soap at school 80.68 88.46 0.56 0.069 no soap at home 23.47 32.39 0.64 0.098 Factors associated with having clean hands % Case % Control Ratio p separate toilets for girls 29.08 4.55 8.61 0.000 visibly clean nails 78.07 45.45 4.27 0.000 wash facilities at toilet 40.05 11.82 4.98 0.000 toilets present 81.88 70 1.94 0.003 short nails 56.55 43.52 1.69 0.011 hand drying facilities 6.67 0.91 7.8 0.017 182 wash facilities at canteen 36.23 26.36 1.59 0.041 tap water 33.46 23.53 1.63 0.044 urban school 50.18 40 1.51 0.045 Two major areas of determinants can be associated with self-reported diarrhoea: location (states without AI); factors around hygiene and factors around livestock. These were largely consistent. Not touching chickens was protective whilst having chickens at home, other poultry (not chickens) at home, and other livestock (not poultry) at home, and touching chickens every day were all risk increasing. Washing facilities at the canteen and soap at home reduced self-reported diarrhoea whereas no soap at home and inadequate water facilities at school both increased risk. Two findings were unexpected: drying facilities at the canteen and self-reported washing after eating were associated with dirty hands. In the first instance, this may be the result of confounding, or because drying facilities are contaminated; in the second incidence the unreliability of self-reporting around measures of hygiene must be considered. Factors decreasing likelihood of reporting diarrhoea % Case % Control Ratio p state had outbreak of AI 38.16 53.22 0.54 0.000 Never touch chickens 4.26 12.7 0.31 0.001 Mixed school 26.57 38.07 0.59 0.002 Washing facilities canteen 26.57 36.19 0.64 0.01 Soap at home 85.56 90.86 0.59 0.028 Factors increasing likelihood of reporting diarrhoea % Case % Control Ratio p Chickens at home 76.7 63.71 1.88 0.000 Livestock (not poultry) at home 50.75 36.25 1.81 0.000 Touched chickens today 60.64 45.2 1.87 0.000 183 Report washing after eating 82.66 68.95 2.15 0.000 Inadequate water 30.43 18.77 1.89 0.000 Any livestock at home 84.06 72.52 2.00 0.001 No soap at home 33.86 22.48 1.77 0.008 Drying facilities canteen 9.18 4.96 1.94 0.022 Any animals at home 87.92 81.23 1.68 0.024 Other poultry at home 24.37 18.83 1.39 0.085 Further exploring associations between exposure to the campaign and changes in attitude, practice and health outcomes Descriptive statistics suggested a problem with confounding- there seemed to be systematic differences between states which reported a flu outbreak and those that didn?t and between hygiene infrastructure in schools with and without participation in the campaign. To adjust for this we carried out two logistic regressions We used robust standard error to take into account clustering by school. We included parameters predictive of hand cleanliness in univariate analysis and which literature or logic suggested where causally linked to hand cleanliness. Provision of non-permanent hygiene material in schools (e.g. wash facilities near toilets, soap, hand-drying material) were not included as there provision would be expected to be an outcome of the campaign. Other indicators of hand cleanliness (short nails, clean nails were not included because of collinaearity, and self-reported parameters were not included because of known biases. Because separate toilets for girls was perfectly predictive of the dependent outcome in states having an experienced an outbreak of avian influenza, we used the related data on whether the school was mixed or not. States which have experiences an avian influenza outbreak In states which had experiences an outbreak of avian influenza the campaign improved six-fold the odds of having clean hands; this was significant. Robust r=11.5 Odds Ratio Std. Err. P>z [95% Conf. Interval] Campaign 6.01 4.64 0.02 1.32 27.26 184 Mixed school 5.32 3.58 0.01 1.42 19.91 Urban school 1.02 0.59 0.98 0.33 3.17 Inadequate water 8.44 7.86 0.02 1.36 52.33 Toilets at school 28.45 26.24 0.00 4.66 173.49 No soap at home 0.21 0.13 0.01 0.06 0.69 In states which had not experienced an outbreak of avian influenza there was not significant improvement. (But a tendency for the campaign to be associated with reduced odds of hand cleanliness.) There are two possible explanations: a) the presence of an avian influenza outbreak increased motivation to deliver the campaign and/or change behaviour in response to the campaign; b) states with AI outbreaks differ in some other aspects which made them more responsive to the campaign. r2=20.34 Odds Ratio Std. Err. P>z [95% Conf. Interval] Campaign 0.28 0.21 0.09 0.06 1.24 Mixed school 1.61 1.14 0.50 0.40 6.47 Urban school 0.77 0.34 0.56 0.33 1.82 Inadequate water 0.07 0.07 0.01 0.01 0.53 Toilets at school 1.07 0.44 0.86 0.48 2.39 No soap at home 1.12 0.75 0.87 0.30 4.20 For both groups of states, inadequate water supplies were a strong and significant predictor of poor observed hand cleanliness. (This is a post-hoc analysis and so must be treated with caution). Hub High risk Low risk Hub A Ogun/Lagos Oyo Hub B Anambra Enugu Hub C Plateau Nassarawa Hub D Kano Jigawa 185 ANNEX 4 LABORATORY ANALYSIS ASSESSMENT OF LABORATORIES SUPPORTED BY THE NIGERIAN AVIAN INFLUENZA CONTROL AND HUMAN PANDEMIC PREPAREDNESS AND RESPONSE PROJECT Nigeria, 7-24 March 2011 A report submitted to the Team Leader, Independent Evaluation Team By Anja Globig, DVM, PhD FRIEDRICH-LOEFFLER-INSTITUTE OF GERMANY 186 Draft submission: 30 March 2011 Revised submission for review: 08 April 2011 Final: 18 April 2011 Abbreviations Ab Antibody AC Air Conditioner Ag Antigen AH Animal Health AI(V) Avian Influenza (Virus) Appr. Approximately AU-IBAR African Union ? Interafrican Bureau for Animal Resources (N)AICP (Nigerian) Avian Influenza Control Project BSC Biosafety Cabinet BSL Bio-Safety Level CBPP Contagious Bovine Pleuro-Pneumonia CDC Centre of Disease Control CIRAD Centre de coop?ration internationale en recherche agronomique pour le d?veloppement (French Research Centre) CMC Crisis Management Centre CSF Classical Swine Fever CVO Chief Veterinary Officer DLS Department of Livestock Services DNA Desoxyribose Nucleic Acid ECE Embryonated Chicken eggs EID Emerging Infectious Diseases EU European Union FAO Food and Agriculture Organization of the United Nations 187 FCT Federal Capital Territory FMOH Federal Ministry of Health FLI Friedrich-Loeffler-Institut (Germany) FMD Foot-and-Mouth Disease GLP Good Laboratory Practice HH Human Health HIV Human Immunodeficiency Virus HPAI(V) Highly Pathogenic Avian Influenza (Virus) IBD Infectious Bursal Disease ILT Infectious Laryngo Tracheitis IZSVe Insituto Zooprofilattico Sperimentale delle Venezie (Italy) MD Medical Doctor MoA Ministry of Agriculture MoH Ministry of Health NADIS National Animal Disease Information System National NAICP Nigeria Avian Influenza Control and Pandemic Preparedness and Human Response Project NISS National Influenza Sentinel Surveillance NVRI National Veterinary Research Institute ND (V) Newcastle Disease (Virus) NIRL National Influenza Reference Laboratory (Abuja) NP Nucleoprotein (of influenza A virus) NRL National Reference Laboratory OFFLU OIE/FAO Network of Expertise on Animal Influenza OIE Office International des Epizooties (World Organization for Animal Health) OVI Onderstepoort Vet Institute (Pretoria, South Africa) PCR Polymerase Chain Reaction PBS Phosphate Buffered Saline PM Post mortem PPE Personal Protective Equipment 188 PPR Peste des Petits Ruminants PT Proficiency Test QA Quality Assurance QC Quality Control QMS Quality Management System RNA Ribonucleic Acid RT-PCR Reverse Transcriptase ? Polymerase Chain Reaction, NOT: real-time PCR RT-qPCR Reverse Transcriptase ? quantitative Polymerase Chain Reaction (?real-time RT- PCR?) SMOH State Ministry of Health SOP Standard Operating Procedures SPINAP(-HI) Support Programme to Integrated National Action Plans for Avian (and Human Influenza) Sr. Senior TAD Transboundary animal diseases TB Tuberculosis UK United Kingdom UCH University College Hospital USA United States of America USAID United States Agency for International Development USDA United States Department of Agriculture VLA Veterinary Laboratories Agency (Weybridge, UK) VTM Virus Transport Medium VTH Veterinary Teaching Hospital WB World Bank WHO World Health Organization WHO-NIC World Health Organization National Influenza Center Executive Summary When highly pathogenic avian influenza (HPAI) of the subtype H5N1 hit Nigeria as the first African country in early 2006 the Federal Government of Nigeria (FGoN) requested a World Bank 189 (WB) International Development Association (IDA) credit of US$50 million-equivalent, provided under the Global Program for Avian Influenza and Human Pandemic Preparedness and Response (GPAI) to fight ongoing outbreaks. As establishment of a reliable laboratory by enhancing laboratory capacity is essential for generating reliable and accurate data in order to develop a strong veterinary and public health response and to play a critical role in surveillance, detection & diagnosis of animal diseases as well as in monitoring of genetic changes of pathogens, it was an outstanding opportunity to use the credit for improving infrastructure and technical skills and for developing new and strong collaborative links among groups and institutions to share knowledge. Five years after implementation, the NAICP requested an independent evaluation team to carry out an impact assessment to provide an impartial assessment of the success or failure in meeting the project outcomes. This report adds to the component for Animal Health, subcomponent ?Strengthening disease surveillance, diagnostic capacity and applied research? and the component of Human Health, subcomponent ?Strengthening of national public health surveillance systems?. ??? Relevance and Status quo: Animal Health (AH) component: Personnel and infrastructural conditions are adequate to detect and diagnose AI H5N1 for the NRL AI at NVRI, Vom. For the determination of pathogenicity (HP H5N1) samples need to be sent to an International Reference Laboratory. Evidence: high, attribution to the project: low (for NVRI) Thus far infrastructural conditions are not yet set for operation at any of the VTH (still awaiting finalization of new laboratories and installation of equipment). After completion of upgrade and provided functionality (electricity, water, equipment) the VTH will need further support in order to achieve the ultimate objective of being involved in screening samples for the detection of nucleic acid of Influenza viruses. Human Health (HH) component: Since 2009 the Human Health laboratories have been capable to effectively and efficiently diagnose H5 and Pandemic H1N1/2009 as well as the seasonal human strains H1, H3 by molecular techniques. A sequencer is in use at UCH Ibadan for HIV characterisation and will be augmented to phylogenetic characterisation of influenza viruses once primers are available. Level of expertise among staff is high whereby most of the trainings and financial support was given through CDC (USAID) and WHO. ??? Effectivity: 190 Upgrade and Functionality: Infrastructure, Equipment, Biosafety level, Reagents AH+HH component: ? Upgrades of Animal and Human Health laboratories are regarded as very essential as the majority of the laboratories currently lack appropriate biosafety levels to work with zoonotic agents. Provision of equipment is needed to guarantee and maintain (i) appropriate biosafety level during laboratory operation [molecular detection versus classical (virus isolation=generation of high viral loads)], (ii) high quality (sensitivity and specificity) of investigations, (iii) high quality of reagents and samples ? Delays in constructions of new lab facilities, provision of equipment and supply of reagents to the laboratories (evidence: high, attribution to the project: high): civil works have just finalized or are still pending o not all reagents / equipment have arrived. o Installation of most of the equipment will take place after finalization of constructions. o Some provided equipment (in 2009) is in use, e.g. generator, freezers, copy machines and real-time PCR machines (at Human Health laboratories). ? Even without timely finalization of fully functional laboratories provided by the NAICP, the NVRI/Vom, the NIRL, Abuja and UCH Ibadan were able to upgrade their laboratories and improve diagnosis in terms of effectivity and efficiency matching international standards [OIE (Animal Health) WHO/CDC (Human Health] with the support of other development partners in joint action with the FGoN,. (Evidence: high, attribution to the project: low) Quality assurance: AH + HH component: applicable for the NRL AI at NVRI, Vom, and for the NIRL, Abuja, and UCH Ibadan: ? In process of laboratory accreditation (ISO 17035), principles applied (Evidence: medium, attribution to the project: low) ? Internal and External Quality programme applied, participation in annual proficiency tests (molecular, serological) since 2008 (Evidence: medium, attribution to the project: low) ? Quality officer/manager assigned + quality manual applied (Evidence: low, attribution to the project: low) ? SOPs for all methodology prepared (Evidence: high, attribution to the project: low) ? OIE (Animal Health) and WHO/CDC (Human Health) guidelines and manuals are followed (Evidence: high, attribution to the project: low) 191 Performance and capacity building: AH component: ? The National Reference Laboratory for Avian Influenza (NRL AI) at NVRI Vom has augmented the range of diagnostic techniques now including real-time PCR which is in use for testing original samples since October 2009. Since 2006 conventional PCR has been used to confirm positive isolation results, which has been used as the technique of choice, but in a different laboratory (Central Diagnostic lab). Since 2008 a conventional PCR is routinely used also at the NRL AI. All positive samples were sent to OIE/FAO Reference Lab at IZSVe in Padua, Italy, for confirmation, and only two out of 80 samples had a disparate result (negative in Padua, positive in Vom). A sequencer will be provided by NAICP (to be installed after the BSL-3 lab is constructed, not before May 2011) and will be used for further clarifications (pathogenicity test) according to the OIE manual and for phylogeny studies. Strong and continuous support is given by the IZSVe. (Evidence: high, attribution to the project: medium) ? Sample flow has been constantly decreasing since 2008. Surveillance activities and samples submitted to the lab have been constantly slowing down (Evidence: medium, attribution to the project: low) ? Until date the Veterinary Teaching Hospitals have not been able to contribute to screening of samples for AI apart from using rapid test and serological assays (2006- 2008) for the detection of antibodies against AIV. All suspicious samples sent to the VTH are further distributed to the NRL AI at NVRI Vom. HH component: ? Since 2009 the NIRL in Abuja is using real-time PCR (machine provided by NAICP) solely for the discrimination of Influenza A/B, and detection of subtypes H5, Pandemic H1/2009, seasonal H1, H3. In a first step, the real-time PCR is used to discriminate Influenza A or B. If positive for Influenza A, further testing targets H1/Pandemic, or, in case this is negative, subtypes H5, H1&3 seasonal. All positive samples are sent to CDC Atlanta for confirmation. All reagents for real-time PCR are obtained through CDC, Atlanta. (Evidence: high, attribution to the project: medium) ? The NIRL in Abuja is receiving appr. 40-80 samples per working day. Samples originate from National Influenza Sentinel Surveillance (NISS) (a project supported by CDC) and are immediately processed for detection of Influenza A/B and H1N1/Pandemic/2009. Number of staff has increased during the last two years and as a consequence of enforced surveillance activities for Pandemic/H1N1/2009. (Evidence: high, attribution to the project: low) ? Since 1974 the UCH Ibadan houses a WHO ? Influenza Reference laboratory. Although it has the mandate to provide testing for Influenza A not only nationally, but also for the West African region, not many activities were carried out due to delays in reception of equipment and finalization of new building, but also and mainly due to lack of reagents and sample submission. Reagents for Pandemic H1/2009 have been obtained in January 2011 through CDC. Number of samples received for Influenza detection is low. Holding the mandate as a Regional and WHO- Reference Center for Influenza (besides other viral 192 diseases) the UCH Ibadan would require stronger support and recognition by the government to fulfil its mandate. Staff skills/expertise and lab networking: AH + HH component: ? Joint actions/trainings of NAICP/FMoH/FMoA and other development partners (OIE Reference laboratories, FAO, NAMRU, other international institutes, CDC, WHO) improved the knowledge of lab performances related to Avian and Pandemic H1/2009 Influenza and contributed to interdisciplinary and international laboratory linkage/collaboration/sharing of resources and global transparency. (Evidence: high, attribution to the project: low) ? Specific skills in molecular biological diagnosis of influenza viruses have been improved since 2008, and there is sufficient staff in each of the laboratories able to perform nucleic acid detection according to algorithms of international guidelines. ??? Efficiency Timeliness AH + HH component: ? With the installation and use of PCR, in particular real-time PCR, laboratory based investigation until result communication is reduced from 24-120 hours to < 12 hours. (Evidence: high, attribution to the project: low-medium) ? Time between sending off samples and receiving confirmation was shortened in the second half of 2006, but real-time PCR was only used since 2009 (conv. PCR in 2006 was mainly used at NVRI for confirmation of virus isolates) ? Improvement of time between sending off samples and receiving confirmation might not be due to the improvement of diagnostic procedures conducted, but the logistics associated with the sample processing in the lab or the transport conditions / logistics of sending samples to the lab. (Evidence: low, attribution to the project: low) ? Inter-laboratory communication and networking as well as logistics of sample transport and processing within the Reference laboratories have been improved since 2007, so that rapid response to suspicious samples can be expected. (Evidence: medium, attribution to the project: low) ? All labs have implemented real-time PCR after the last outbreak in 2008, therefore efficiency of real-time PCR application still needs to be seen. (Evidence: medium, attribution to the project: low) Cost-effectiveness AH + HH component: 193 ? While molecular biological technology contributes decisively to biosafety, speed (<12 hours) and precision, it is very expensive as opposed to classical techniques (virus isolation). Latter provides a result after minimum 24 hours, or with negative samples >120 hours. ? Provided there are plans about funding of expensive reagents (in particular enzymes and extraction kits) for PCR, this technique should be given first priority regardless of costs, because it (i) matches current international standards (new gold standard), (ii) is safe and (iii) adds to the speed of result generation and thus outbreak response. ? Only few of the equipment could be used until the end of the project and there is no direct and timely correlation with HPAIV H5N1 detection + outbreak control and the provision of equipment / reagents. ??? Impact ? Once the new buildings are equipped and fully functional (and provided that reagents had been stored properly), the laboratories of VTH (n=5), UCH (n=7) and Reference labs (n=3) will be well prepared for future outbreaks and each of the labs will be able to run PCR for approx. 500-1000 samples and for the duration of one year (expiry dates) provided the reagents have been properly stored. It will further improve biosafety and work flows applicable also to working with other emerging or zoonotic diseases. (Evidence: low, attribution to the project: high) ? There is no clear evidence that any of the NAICP related actions had an impact on the detection of other animal or human diseases. The real-time PCR has yet only been used for Influenza detection and not for nucleic acid detection of any other animal disease. Exception is at UCH Ibadan, where the Virology Division houses a WHO Reference Centre for Polio and HIV and where real-time PCR is routinely in use for predominantly HIV-diagnosis among others since 2008. (Evidence: high, attribution to the project: high) ? Human Health laboratories (Influenza Reference labs) are well capable to simultaneously test for H5 and Pandemic H1/2009 in addition to the common human influenza strains. This is a result of efforts supported by other projects (CDC, WHO) and not clearly and directly related to the NAICP. (Evidence: high, attribution to the project: low) ??? Sustainability: ? Without further (donor, but preferably governmental) financial input + increase of targeted surveillance activities on annual basis, laboratory activities (esp. referring to rapid nucleic acid detections) are going to cease eventually due to lack of reagents which are expensive. ? By keeping on track with interdisciplinary collaboration (One-Health approach) and thereby sharing resources might result in cost-effective genetic characterisation of viral strains, improvement in research of emerging/zoonotic agents and, in a more practical sense, the cost-effective maintenance, calibration and service of critical equipment. 194 ? The WHO-NIC virology lab at UCH Ibadan is committed to surveillance, diagnosis, monitoring and control of influenza in Nigeria while theoretically serving as Reference Centre for arbo- and influenza viruses in Nigeria and in the African region. It holds a strong mandate and takes its mission seriously. The role of the Department of Virology at UCH Ibadan should be stronger recognized by the relevant authorities and subsequently strengthened to become more integrated into the activities of Influenza and zoonotic virus detection. The strong interdisciplinary approach contributes to a benefit of networking of both Human and Animal Health laboratories. Aggregate messages summarizing the outcomes of the lab assessment: ? Diagnostic capabilities were improved over the time of the project, but infrastructure and sufficient supply with reagents is not yet completed. ? There is no clear evidence that any of the NAICP related actions have had an impact on the efficient or cost-effective laboratory diagnosis of avian influenza viruses during the outbreak period (2006-2008) or other disease viruses. ? Whether the upgrade of the laboratories will have a positive impact for better detection and more efficient control of future outbreaks remains to be seen. ??? Improving laboratory capacity requires further investment in both time and money. It is essential that long term goals are set, as opposed to short term fixes, and that the finances are made available to enable and sustain continuous quality improvements. Undoubtedly, all subject-related parties must feel committed to further improve the situation and collaborate with stakeholders and the international scientific community to maintain a high level of laboratory diagnosis, preferably augmented to other emerging or important diseases. Acknowledgments The consultant would like to sincerely acknowledge the support given by Professor Brian Perry, and by ILRI, particularly Thomas Randolph and Acho Okike, who decisively contributed to the unhindered flow of this mission, and Joerg Henning for constant support, briefing about the project and discussions as well as Delia Grace, Bernard Bett, Heather Hannah, Pamela Pali for productive discussions. She is particularly grateful for the tremendous support given by Delia Grace during her acute affliction. Discussions with Prof. Ekanem Ekanem and Prof. Paul Abdu were very fruitful and are much acknowledged. The author wishes to cordially thank the accompanying persons for the lab visits in Vom, Zaria and Ibadan, Kunle Adesoye (Ministry of Animal Health, Evaluation and Monitoring) and Francis Olanwale (Ministry of Health, Desk Officer) for all the back-up, advice, time spent, support and trust given. 195 Her particular and cordial acknowledgement refers to all lab directors and laboratory officers met, in particular Dr. Adedeji Adebayo, Dr. Paul Shilo, Amina Garba S. Abu, Adama Ibrahim Musa, Abugail Mwakamma, Pamela Amaka Onyiah (NIRL Abuja), Dr. David Shamaki, Dr.Tony Joannis (NVRI Vom), Prof. L. Sai?du, Yunusa Yahahya Pala, Dr. Musa I Wazini, Dr. Alihn M Wakawa (VTH Zaria), Prof. Temitope O. Alonge, Prof. David Olaleye, Dr. Georgina Odaibo, Dr. Clement Meseko, Dr. Bakarey Adeleye Solomon (UCH Ibadan), and many more who spent a lot of their precious time in answering questions and for warm-hearted welcome and great discussions. Finally, the author addresses her acknowledgements to her home institute, the Friedrich- Loeffler-Institut in Germany, particularly to Prof. T. Mettenleiter, Wolfgang B?hle, Sandra Blome, Elke Starick, Klaus Depner and Detlef H?reth-B?ntgen for back-up and for immediate support. 1 Introduction Following the first outbreaks of HPAI H5N1 in Nigeria in early 2006, the Federal Government of Nigeria (FGoN) requested assistance from the World Bank. A US $50 million project, entitled the Nigeria Avian Influenza Control and Pandemic Preparedness and Human Response Project (NAICP) was activated as an emergency operation in June 2006. The project addresses both the animal and human sides of avian influenza and has four components: Animal Health, Human Health, Communication & Public Awareness, and Project Management. The WB project was approved in April and was initiated using US $ 10 million advanced from other projects; it formally commenced in July 2006 and is scheduled for completion by 31st of May 2011.The project development objectives were designed to support the efforts of the FGoN to minimize the threats posed by H5N1 to humans and the poultry industry, to prepare the necessary control measures to respond to a possible influenza pandemic, and to prevent further spread of the disease to other parts of Nigeria. To achieve these objectives, the Nigeria Avian Influenza Control Project (NAICP) supported three sets of interventions: (i) response and containment; (ii) control and prevention; and (iii) preparedness and planning. The project has four components: Animal Health, Human Health, Social Mobilization and Strategic Communication, and Project Management and Coordination, each of which has sub- components. Five years after, the NAICP requested an independent evaluation team to carry out an impact assessment to provide an impartial assessment of the success or failure in meeting the project outcomes. This report adds to the component for Animal Health, subcomponent ?Strengthening disease surveillance, diagnostic capacity and applied research? and the component of Human Health, subcomponent ?Strengthening of national public health surveillance systems? and focuses on the laboratory upgrade during the project period. The veterinary laboratory network in Nigeria comprises the National Reference Laboratory (NRL) for Avian Influenza (AI) at the National Veterinary Research Institute (NVRI) in Vom and 5 veterinary faculty diagnostic laboratories. The existing laboratories that have been earmarked to carry out diagnosis of HPAI include the central NVRI and five State Veterinary teaching Hospitals (VTHs) at the veterinary faculties of Ibadan, Maiduguri, Nsukka, Sokoto, and Zaria. Laboratory diagnostic networking includes laboratories in the West African sub-region and the FAO/OIE 196 reference laboratory at IZSVe in Padova, Italy. The NVRI, Vom has the national mandate for the diagnosis and investigations into animal/poultry diseases, animal/poultry disease vaccines and research into various aspects of animal/poultry diseases epidemiology and control. Following the upgrading at the VTHs, they are expected to take routine diagnostic load off NVRI by screening incoming field samples for positive reactions and forwarding these to NVRI for confirmation. However, most samples are usually sent directly to the national laboratory at NVRI through the AICP Desk Officers in each state of the federation and the Federal Capital Territory (FCT). 197 Figure 1: The laboratory network of Animal Health laboratories that were included in the upgrade plans of the project. There also is a network of Human Health laboratories. One laboratory has been newly established in Abuja and designated to act as the National Influenza Reference Laboratory (NIRL) in Abuja. Since 2008/9 it receives samples from National Influenza Sentinel Surveillance (NISS). Eight other laboratories act as satellite labs of which four have been upgraded and new constructions established. The UCH Ibadan, University of Port Harcourt Teaching Hospital (UPTH), University of Maiduguri (UMTH) and NIRL Abuja have been approved for constructions of new laboratories. All eight laboratories work in a network for sentinel sites (ILI-symptoms). Since 1974 the Department of Virology at UCH Ibadan is a designated WHO- and Regional Reference Center for Arbo- and Influenzaviruses. TH NIRL National WHO- Influenza Centre 198 Figure 2: Distribution of Satellite Laboratories of the Human Health Site. Blue: NIRL, Abuja; Yellow: National WHO Influenza Reference Centre (WHO-NIC), Ibadan; Red: Teaching Hospitals in Port Harcourt, Lagos, Maiduguri, Kano, Enugu and Sokoto. The Friedrich-Loeffler-Institut as Federal Research Institute for Animal Health and independent higher federal authority is affiliated with the German Federal Ministry of Food, Agriculture and Consumer Protection. The FLI is active in the fields of virology, bacteriology, parasitology, epidemiology, prion research and related sciences. A major focus is notifiable animal diseases. The FLI houses National Reference Laboratories (NRL) for more than 40 (notifiable) animal diseases reference laboratories of the Office International des Epizooties (OIE) for AI, ND, CSF, bovine herpesvirus-1 infections, brucellosis, bovine leucosis and rabies. Further, the FLI acts as Collaborating Centre for Zoonoses in Europe of the OIE and houses a Collaborating Centre of the World Health Organisation (WHO) for rabies. The FLI as Lending Employer was contracted by ILRI to provide a consultant to assist under the evaluation project with the following scope: 1. Preparation of a draft framework and detailed protocol for the laboratory assessment component, including any survey instruments required, where applicable. 2. Make site visits to a sample of veterinary and medical laboratories that benefited from NAICP investments in infrastructure and training to: a. Evaluate the current operating status of the infrastructure and equipment installed, and their ability to detect HPAI and/or other influenzas, and other diseases more generally b. Evaluate the current capacity and performance of laboratory staff to detect or confirm HPAI and/or other influenzas, and other diseases more generally c. Document and evaluate the capacity building undertaken at different levels and by speciality d. Collect data needed to describe the evolution of the performance of the laboratory in detecting HPAI or influenza since the beginning of the project e. Assess the sustainability of the improved capacity, both in terms of budgetary commitments for maintenance of critical infrastructure, purchase of supplies, and staff, and in terms of maintaining the level of staff training. 3. Analyze the information and data collected and prepare a draft technical evaluation report for submission to the Team Leader by 18th March 2011, revising as needed based on feedback. The entire ToR is presented in Annex 1. 199 2 Methodology Country and Laboratory Visits Schedule for lab visits Immediately after arriving the country in the late evening of 7 March 2011, a tight schedule was set to visit two Animal Health laboratories in Vom and Zaria, and two Human Health laboratories in Abuja, and Ibadan (table 1) starting in the morning of 8 March 2011. Laboratory visits were followed by days in Ibadan for data entry, analysis, preparation of report and presentation (18- 20/3) as well as attending the inter-team workshop (21-22/3). A detailed itinerary is found in Annex 4. Table 1: Dates of laboratory visits, duration of stay in the laboratory and activities performed. Abbr.: NIRL: National Influenza Reference Laboratory; NVRI: National Veterinary Reference Institute; VTH: Veterinary Teaching Hospital; UCH: University College Hospital Laboratory Date Duration Activities NIRL, Abuja 8-9/3 1,5 days Laboratory assessment NVRI, Vom (Jos) 10-11/3 1,5 days Laboratory assessment accompanied by Kunle Adesoye VTH, Zaria 14-15/3 1,5 days Laboratory assessment accompanied by Kunle Adesoye UCH, Ibadan 17-18/3 2 days Laboratory assessment accompanied by Francis Olanrewaju Laboratory assessment Interviews were conducted with the directors, lab managers and technical staff (scientists and/or technicians) of each of the labs. On-site inspections were carried out. Primary data collection: Two types of questionnaires were designed: (i) Questionnaire about training otained through the NAICP or other donors that was handed over to each of the staff persons to be filled (ii) Questionnaire for data on equipment provided and its usage, reagent stock and expiry dates, sample accessions and turn around time that was filled during interview A total of 22 questionnaires were filled (N=9 (NVRI, Vom), n=3 (VTH, Zaria), n=4 (NIRL, Abuja), n=6 (UCH, Ibadan). Secondary data collection: During the visits the collection of secondary data was attempted, but sometimes unsuccessful. The following documents were collected: 200 ? Equipment lists, reagents lists (NAICP-provided) for all labs, list of existing and used equipment from NVRI Vom ? Annual reports (NVRI, Vom) ? Data provided by use of Epi-Info (NIRL Abuja) ? Publication list (NVRI, Vom) Answers of the questionnaires, available secondary data and results of visual inspections of the lab set-up were incorporated into an excel file, and separated for the years 2006/7, 2008/9 and 2010/11 for each of the lab visited to document the evolution of the lab capacities in detecting influenza (and other diseases) since the beginning of the project. Annex 1 presents the data (in extracts). Data entry and analysis During the field trips a detailed excel file was developed as an instrument for the lab assessment covering most important indicators for different evaluation criteria (table 2). Excel file contains data space for more than 350 entries for each of the lab visited and for each of the years 2006/7, 2008/9, 2010/11. The assessment is descriptive, there was not enough data made available, also from other laboratories that were involved in the upgrade activities, to conduct an evidence-based analysis. Table 2: Evaluation criteria and their indicators for operational status, staff performance and capacity building 201 Evaluation Critera Indicator Improvement of infrastructure, equipment, biosafety level, reagent supply Improvement of staff skills and performances and availability Improvement in capacity and diagnostic capability Geographic location / accessibility Equipment Communication means Biosafety Available technology/tests Availability Reagents (AI) Overall staff Accessions / # samples Sample submission Lab Activity Quality assurance Staff performance Training Lab diagnosis International Training > 3 weeks Training QA issues Application of training inputs Linkage with satellite labs International donor organizations Geographic location / accessibility Communication means / connectivity Equipment / PCR Availability Reagents (AI) / PCR Available technology/tests / PCR Accessions / Timeliness Lab Activity / PCR Staff performance/molecular techniques Accessions / Reporting Overall status of lab Building Administrative Data Organization basic supply Infrastructure constraints Staff performance Training Lab diagnosis/other diseases Available technology/other diseases Quality assurance Biosafety Biosecurity constraints Laboratory Networking constraints Budget Equipment maintenance Reagent supply Transferation of training inputs Biosecurity Quality assurance Progress of Capacity building Laboratory collaboration Relevance Construction, Infrastructure, equipment, reagents, operational status Staff skills and availability Sustainability Effectivity Construction, Infrastructure, equipment, reagents operational status Staff skills and availability Construction, Infrastructure, equipment, reagents, operational status Construction, Infrastructure, equipment, reagents, operational status Staff skills and availability Progress of Capacity building Staff skills and availability Efficiency Impact Progress of Capacity building 202 3 Results Relevance of Avian Influenza laboratories and evolution during 2006-2011 When Nigeria was hit by HPAIV H5N1 as the first African country in 2006 with an onset of multiple outbreaks affecting a total of 25 states followed by one detected human lethal case in 2007, HPAIV H5N1 was regarded as a major threat not only to the public health but also the economy of Nigeria. Consequently, for the laboratory component all efforts were tried to: ? improve diagnostic capacities in terms of upgrade of lab equipment, reagents and constructions ? improve logistics of sample transport ? reduce the time of sample reception in the laboratory until response ? enhance staff diagnostic performance and skills Five million USD of the WB grant were planned to be spent on strengthening disease surveillance, diagnostic capacity and applied research for the Animal Health component of the NAICP already in 2006, and a further millions of USD for a similar approach at the Human Health component. Major priorities for enhancing the laboratory capacities to detect HPAI/Influenza A and other emerging or zoonotic disease agents were targeted at: ? sound and biosafe construction and infrastructure of a well-contained laboratory building ? robust, reliable equipment for modern, rapid and sensitive diagnosis according to international standards (OIE, WHO) ? high-quality reagents for sensitive and specific diagnosis ? technical expertise for reliable and timely diagnostic performance or confirmative diagnosis (training) ? laboratory biosafety and biosecurity when handling zoonotic disease agents ? maintenance of equipment, reagent supply, technical skills Despite the grant available as early as 2006 from WB, most of the obvious improvements of Animal and Human Health laboratories for rapid diagnosis of Avian Influenza since 2006 derived from their own efforts in complementation to the support of other development partners (WHO, FAO, USAID, CDC, USDA, International Reference laboratories?) in upgrading, receiving equipment and reagents (not funded by the NAICP) for molecular diagnosis, as well as in capacity building in terms of national and/or international training (Animal Health site). Those efforts have improved the diagnostic capacities and reduced the turn around time for sample investigation of passive surveillance since 2008/2009. 203 Processes for new constructions of lab buildings, procurement of equipment and reagents by the NAICP were laborious and resulted in serious delays, so that civil works were finalized in early 2011 or later. Some of the equipment provided by the project, however, have been used since 2009 among which were generators, freezers, copy machines and real-time PCR cycler (ABI 7300, 7500), the latter at the Human Health labs only. Despite AI there is a range of diseases that are regarded as most important for Nigeria (table 3). While AI remains a priority disease, there are also other diseases that appear to be neglected by national or international recognized control-programs, e.g. Newcastle Disease. It has been reported that an unknown, but high number of poultry dies as a consequence of unspecified poultry disease, possibly ND. Samples are rarely submitted to the laboratories giving evidence that there is a strong need for enhanced surveillance for poultry diseases and a national campaign to reduce poultry mortalities not solely focussing on AI but also ND and other poultry diseases. Zoonotic diseases like Salmonellosis, Brucellosis and rabies certainly receive less attention and need to be addressed for Public Health reasons. Table 3: Diseases reported by the laboratories visited and regarded as priority diseases, separated for Animal and Human Health ANIMAL HEALTH HUMAN HEALTH TAD Zoonotic Zoonotic Human Disease 1. ND 1. HP H5N1 1. Salmonellosis 1. Malaria 2. CBPP 2. Rabies 2. HP H5N1 2. HIV 3. ASF 3. Brucellosis 3. Rabies 3. TB 4. PPR 4. TB 4. Lassa 4. GE 5. FMD 5. Salmonellosis 5. Yellow Fever 5. Resp. Disease 6. Anthrax 6. Trypanosomiasis Effectiveness of the WB project Animal Health Component Reportedly, as a consequence of bureaucratic processes within the WB procedures the constructions of new laboratories for the NVRI Vom and the 5 VTH have just finalized, whereas installation of procured equipment is still pending apparently for all of the laboratories and certainly for the visited labs of the Animal Health component. 204 The personnel and infrastructural conditions of the NRL AI at NVRI, Vom, are adequate to detect and diagnose HPAIV H5N1, but are not as yet for the 5 VTH. Since 2008 surveillance activities have been constantly decreasing, reagents been used up and there is a concern, that laboratory activities will eventually cease if not continued on a constant basis. National Veterinary Research Institute ? National Avian Influenza Laboratory (Vom) The Institute was established as a Veterinary Laboratory in 1924 to combat livestock diseases following the outbreaks of Rinderpest in the West African sub-region. The institute established two colleges in 1941 and 1956 to provide training for Middle Level Manpower for the livestock industry and laboratories. The mission of the NVRI is ?to be the foremost veterinary research institute in Africa, producing international quality vaccines and offering services for the identification, control, and eradication of economically important livestock diseases; through best practices, research excellence, and applying modern technology; with highly trained, experienced and motivated personnel.? The NVRI?s mandate is: ? To conduct research into all aspects of animal diseases, their treatment and control ? To develop and produce animal vaccines, sera and biological to meet the national demand ? To provide surveillance and diagnosis of animal disease ? To introduce exotic stock for improved egg, meat and milk production ? To train intermediate manpower in Veterinary laboratory technology and animal health and production technology ? ??? Construction, Infrastructure, equipment, reagents, operational status of the National Reference Laboratory (NRL) for AI at NVRI, Vom Following the outbreaks of HPAI H5N1 in early 2006 it was planned to construct a BSL-3 laboratory on the campus of the NVRI. However, until date a contract has not been signed, and no equipment has been supplied. Reasons for delay of WB / AICP carried out for the Animal Health Lab component were reported to be: ? Availability of a company that constructs a BSL-3 within Africa. ? procurement process of WB does not allow for a sole source, and it was almost impossible to find other sources of constructors 205 ? late procurement of equipment on AH site, because it was thought to first finish the constructions before equipment is procured Finally, it came to an agreement to establish a modular BSL-3 lab, and a contract will be signed in March/April 2011, and constructions might be finalized in May. It is planned to install the equipment after finalization of the BSL-3 container. With finances of the NVRI own budget (FGoN), supported by Chinese government, SPINAP AU- IBAR, FAO, EU, USDA and the FAO/OIE Reference laboratory for Avian Influenza at the IZSVe in Padua, Italy, but in addition certainly through personal commitment and dedication, it was possible to upgrade and equip the laboratory and improve the overall operational status to ensure biosafe and bio-secure work with infectious pathogens. Workflow is arranged in a way that cross-contamination and reagent & sample damage is reduced to a minimum (inverter for freezers where critical reagents/samples are stored, liquid nitrogen). The floors are equipped with -80 and -20?C freezers with temperature check and liquid nitrogen ensure proper storage of reagents, samples and associated material. Since 2008/2009 the NRL for AI has been able to perform a broad range of diagnostic technique for AI, including real-time PCR targeting Influenza A, H5, N1, H7, H9 (table 4). The lab is still waiting for reagents from NAICP and had to cease investigations by real-time PCR on samples for active surveillance as the expensive and rare reagents that have been self- procured or provided by SPINAP (AU-IBAR) need to be stored for suspicious samples. In 2008 NVRI asked for Rotagene real-time PCR cycler (Corbett) to be provided by the WB funds as this machine is not dependant on further consumables like it is the case with other real-time PCR machines, and therefore would in the long run be cheaper, but WB proposed another (ABI) machine as a consequence of the bidding process. Consequently, considering sustainability, the Rotagene cycler was provided by SPINAP (AU-IBAR) in 2009 and is in sparse (because of expensive reagents) use since October /2009. Infrastructure, available technologies, biosafety have constantly been improving since 2006 without the input of NAICP and based on the efforts of the NVRI with support of the FGoN and other donors (see above). Table 4: Overview on the range of laboratory diagnosis at the National AI-Lab in Vom Methods since Comments Detection of Ag /Ab HA/HI <2006 Reagents provided through IZSVe, Padua AGID <2006 Reagents provided through IZSVe, Padua ELISA Through supplier (own budget) Conventional PCR M, H5 2006 A cycler was available and used at the Central Diagnostic Laboratory to confirm virus isolates Conventional PCR H5, H7, N1, H9 2008 A new conventional PCR cycler provided by the FGoN in 2008 and set up in the NRL AI Primers and protocols IZSVe, Padua 206 Real-time PCR H5, H7, N1 2009 Primers, probes and protocols IZSVe, Padua, Real-time cycler (Rotagene) through SPINAP Sequencing Proposed date: 2011 In the process of procurement (WB), 2 persons are able to run a sequencer Rapid antigen test Out of stock Virus isolation Egg inoculation <2006 ECE derive from SAN-free chicken Cell culture <2006 Theoretically possible, but constraints with cells, currently not performed, sharing resources with UCH Ibadan ??? Performance, staff skills and availability A sufficient number of personnel (n=13) of whom five are veterinarians and five are technicians and the rest supportive staff is employed at the NRL for AI. A proportion of 5/10 lab people have acquired good knowledge on HPAI diagnosis through international training (table 5). The staff is well-trained (10/10), either due to direct training inputs or through Training of Trainers activities. They are sufficiently involved in their respective activities for at least 35 hours a week. However, training has been mainly carried out through other donors (Evidence: medium, questionnaires, attribution: low). The lab is progressing towards ISO 17025/2005, and external audits are going to take place shortly (reportedly, no evidence). Investigations follow the international standards as given in OIE guidelines and manuals of IZSVe. Besides approaching accreditation, the labs annually participate in Proficiency Tests (PT) (molecular and serological) since 2008 supervised by the FAO/OIE Reference Laboratory at IZSVe in Padua, Italy. Due to constraints with sample shipment in 2008 only 70% was achieved (RNA degradation through long shipment), but in 2009 90% and now 100% (evidence: strong, attribution: 0%). During the outbreak time in 2006-2008 a total of 80 samples were sent to the FAO/OIE Reference Laboratory in Padua, Italy, all results were in consensus apart from two samples sent in 2007 that were positive at NVRI, but negative at the IZSVe. (Confirmation and evidence awaited) Despite improved conditions compared to 2006 and despite the fact that the mortality rate of poultry is still high in Nigeria, the sample flow to the lab, particularly deriving from passive surveillance, has been constantly decreasing (table 10). A possible explanation might be a general fatigue in reporting poultry deaths by the farmers. Therefore, the trend of laboratory activity has moved from routine diagnosis towards applied research between 2008 and 2011. ??? 207 Progress of Capacity building and laboratory networking Staff has received regular training, both internationally (a total of 15 months was spent by the head of the lab at the IZSVe Padua in 2006, 2008 and 2009/2010) and nationally, but mostly reportedly funded by other sources than WB (USAID, EU, FAO / IAEA, other institutes, table 5). The lab provides/offers regular trainings to students and staff of VTH mostly funded by the own budget. All have been trained either by outside or in-house training in performing real-time PCR. Training content could be applied and further distributed (Evidence: medium, attribution: low). There is evidence that lab staff of VTH and also Human Health Laboratories (esp. NIRL Abuja) recently received training through staff of NVRI about molecular techniques in laboratory diagnosis of Influenza viruses. Table 5: Trainings provided for NRL-AI - staff (n=20); Evidence: low (questionnaire), attribution: reportedly low; no evidence that any main training came from the NAICP. Year 06/07 08/09 10/11 Funded by Diagnosis of AI (ND) including nucleic acid detection and/or sequencing 10 8 1 FAO, IAEA, WHO, IZSVe Padua, USAID, NVRI (Vom), EU, OVI, NVSL (Ames, Iowa) Biosafety biosecurity 1 USAID Since 2007 the NVRI has been designated by the FAO coordinated Laboratory Network (RESOLAB) as a regional laboratory for the diagnosis of HPAI and other TADs for West and Central Africa. Collaboration, both nationally and internationally and linkage to the satellite labs, the VTH, universities and research centres in the country, in the region (22 other labs from West and Central Africa through regular meetings as Regional laboratory) and internationally/overseas has been improved over the years. International collaboration has resulted in 14 publications with international partner institutes, e.g. OVI, CIRAD, IZSVe et al. (Evidence: strong, attribution: zero-low) On national level the NVRI shares resources within network of institutes of the FMoA and FMoH, esp. with the UCH Ibadan (sharing of capacities, e.g. H1-pig surveillance undertaken by PhD student from Vom at UCH Ibadan). (Evidence: strong, attribution: 0) However, interdisciplinary communication could still be improved, e.g. when the Human Health site finds suspicions through Sentinel Surveillance activities associated to mortality in poultry. International institutional collaboration: Shipment of samples (list pending), publication list, evidence: strong, attribution to the project: 10%) Main constraints associated with the effectivity of NAICP for laboratories: 208 ? Provision of equipment is reportedly strictly dependant to the bureaucratic procedures of WB, so that preferred equipment often could not be provided. For example NVRI asked for Rotagene real-time PCR cycler as this is not dependant on further consumables like it is the case with other real-time PCR machines, and therefore would in the long run be cheaper, but WB proposed another (ABI) machine. Considering sustainability by reducing side-costs, the Rotagene cycler was procured through SPINAP (AU-IBAR) and not through WB. (Not cost-effective by the project) ? Obviously, WB procurement procedures did not consider the rapid response needed in an emergency operation. This has contributed to serious delays in the procurement of construction, equipment and consumables. As a consequence, NVRI itself has gone ahead to procure many of the requested items (table 4). ? Training activities did mainly come from other sources and not from the project ? Train the Trainer activities are currently ceasing because of lack of funds Veterinary Teaching Hospital, Ahmadu Bello University, Zaria The mission of the ABU VTH Zaria is to provide excellent learning environment for veterinary students, interns, residents and other professionals through training and continuing education; to provide excellent veterinary services to individual livestock owners, producers, veterinarians and other institutions; to contribute to the discovery of new knowledge through bio- technological research and clinical trails for the enhancement of animals and public health. The vision of the ABU VTH Zaria is to be a nationally and internationally recognized centre of excellence for effective and efficient veterinary clinical teaching, research and services to veterinarians and the public. The first outbreak of HPAI H5N1 was initially suspected from pathological lesions in poultry at VTH Zaria and later confirmed by NVRI, Vom. Construction, Infrastructure, equipment, reagents, operational status Civil works of an isolated well-designed building purposed for running a BSL-2 laboratory including a molecular lab have just been finished, however, the new laboratory is neither yet functional nor equipped although reagents procured by NAICP have just arrived. Currently, the building does neither have electricity nor running water and is yet not combined to the generator. A concern is that reagents cannot be stored properly due to irregularity of electricity and thus fluctuation of freezer-temperatures. This may result in damage to the reagents, esp. enzymes before they can be used. As the PCR reagents have an expiry date within one year, it will be essential to as quickly as possible install equipment, connect to electricity and generator, connect to water in the new building. Some equipment has been already supplied by the NAICP (table 6). In general, quality of water and frequent worsening power supply are of rising concern to establish a high quality of laboratory diagnosis. 209 Table 6: Equipment provided by the NAICP Type of equipment/ Reagent Date when installed and usable Used for HPAI diagnosis in the past (yes/no) Currently used Comments (current condition of equipment, reasons why not in use etc.) Generator 2009 No No Exposed to outside uncovered, not yet plugged to building, functionality is unknown Civil works 2008-2009 No No Waiting for equipment and reagents Borhole + Water tank 2008 No No Manual switch on for pump to pump into water tank Car 2006 Yes Yes Due to a serious lack of reagents the lab is not able to conduct any test for AI and hardly any test for any other disease at the moment. Until 2008 samples collected during post mortem investigation were tested with rapid test (Synbiotics) and until 2009 it was possible to investigate serum sample in HI and AGID. Yet, the NAICP did not contribute to improvement in diagnostic capacities. However, the provision of a new lab and equipment will decisively impact the laboratory functionality in future (see chapter ?impact?). ??? Staff performance and availability During the project period, the number of staff has increased from six in 2006 to twelve in 2010. Salaries are paid by the University. All lab staff is well trained in clinical diagnosis, post-mortem investigations and serological investigations. One biotechnologist has been trained in India and elsewhere in molecular biological investigations (not by the project) and will be the person in charge (and with high level of expertise) once the newly equipped and upgraded laboratory is functional. Publications written by the lab staff in the periodic scientific bulletin of ABU VTH Zaria give evidence for the staff?s motivation and generally high level of education which is in striking contrast to the current poor diagnostic infrastructural possibilities. ??? Progress of Capacity building Lab staff has been trained through the NAICP in sample shipment, serological techniques of AI diagnosis, general features about AI, case definitions of HPAI and diagnosis, outbreak investigation, surveillance, reporting and actions as well as classical and molecular diagnosis techniques (NVRI, Vom) and by SPINAP AU-IBAR between in 2006-2008, but training activities slowed down since 2009 and seem to cease eventually. However, a part of the VTH mandate is to transfer training to students and other lab staff, mainly in serological test performances. The VTH Zaria produces a regular bulletin (Veterinary Clinical Practice) where recent discoveries and laboratory procedures/investigations are published. For example, the volume 2, No. 1, 2009 presents a review about the application of PCR in laboratory diagnosis of various animal 210 diseases. This shows not only high interest in establishing this method but also required expertise. Several publications in during 2006-2008 published by the VTH staff and supervised by the Head of the NRL AI at NVRI Vom states collaboration of VTH and NVRI (e.g. Wakawa et al., 2008; Adene et al., 2006). A review (Avian Influenza: A Review) already published in 2005 in the Nigerian Veterinary Journal shows that VTZ Zaria was prepared for Avian Influenza before HPAI H5N1 entered the country. Human Health National Influenza Reference Laboratory, Abuja Construction, Infrastructure, equipment, reagents, operational status In 2008 the building of the current NIRL has been upgraded by financial support of CDC and FMoH and was commissioned in 2009 by the US ambassador and the FMoH. Equipment has been installed for molecular biological diagnosis of Influenza A and subtyping H5, Pandemic H1/2009 and seasonal H1/H3 exclusively by real-time PCR. A separate building has been constructed (BSL-3) in close proximity to the molecular lab, but is not yet equipped, waiting for >50 equipment to arrive. This new building is planned for serology, conventional PCR, virus isolation (cell-culturing), functional earliest end of 2011. However, the NIRL is planning to move to the Nigerian Center of Disease Control (NCDC) by end of 2011. It is unclear what will happen to the new building. The NCDC is outside of a residential area, houses a BSL-3 lab and is involved in diagnosis of Yellow Fever, Malaria, Lassa Fever and other zoonotic diseases. It is planned that staff and equipment are going to move there in the near future. Some equipment funded by the NAICP has been installed and is in use, other equipment is awaited (for more details refer to Annex 7.1, table 7) ? copy machine ? freezer, fridges ? PCR hood ? ABI-real-time Cycler ? UPS ? Laptops, Computers ? Centrifuge ? Pipets ? Table and chairs ? Tips ? Microscope ??? Performance, staff skills and availability 211 ? Molecular diagnosis (real-time PCR) of Influenza A/B, Pandemic H1/2009, H5, H1 and H3 seasonal (CDC-protocols), not N1 ? Salary of some staff is provided by CDC ? In process of accreditation (principles applied), a Biosafety and Quality Manager is supervising all lab activities including regular validation of newly acquired primers/probes or PCR kits. Accreditation is supported by CDC. ? Proficiency Testing (CDC) twice annually, 100% each time ? All reagents are provided by CDC upon request ? Maintenance and Certification of equipment is carried out by South African AIR Filter Maintenance service (same company as for Animal Health Laboratory) ? Sample flow: Every day 40-90 samples submitted to the lab, 80% derive from surveillance, 20% from suspicions. ??? Progress of Capacity building and laboratory networking ? Training is carried out on regular basis, and with the increasing cases of Pandemic H1N1/2009 detections in humans and sample flow to the lab, the number of staff has been increased. ? All technicians and staff are able to perform real-time PCR, and everyone has a specific task and role. Two persons are managing the data by use of EPI-Info. ? Training of regional hospital staff is carried out every four months ? Staff receives regular training (hands-on training and abroad training) supported by CDC. ? There is a strong linkage also to CDC in Kenya. ? A regularly published pamphlet informs about the activities with regard to Pandemic H1N1/2009 and HPAI H5N1. ? There is an active collaboration with NVRI Vom which supports activities and is active in training at NIRL Abuja. Main Constraints ? No own lab budget / funds ? Irregularities in electricity, no inverter for sensitive reagents ? Timely supply of PCR kits or primers/probes ? Collaboration within the network of Teaching Hospitals is not as good as supposed to be, there is a lack of understanding each others role and a need for a MoU for coordination of various activities and better linkage/collaboration University College Hospital, Ibadan 212 The Department of Virology was established in 1963 with a grant from the Rockefeller Foundation to the University of Ibadan. In 1982, the laboratory became a fully fledged Department of the University for the Postgraduate training in virology. The activities of the Department of Virology were initially directed towards the surveillance and study of viruses transmitted to man and animals by insects vectors. Some viruses were newly discovered and hitherto not described in Nigeria or elsewhere in the world. The Department played a major role in the investigation and confirmation of major epidemics of yellow fever which occurred in 1969, 1973/74 and 1986/87 as well as epidemics of the Lassa fever in Nigeria. The Department was actively involved in the investigation of the 1974 pandemic of influenza, and was accordingly recognized by WHO as one of WHO National Influenza Centres. In addition, investigations target poliomyelitis, African swine fever, measles, HIV/AIDS, rubella, avian influenza, dengue and other arboviral fevers. The Department of Virology has a very strong director, who is a veterinarian and actively involved in world-wide networks for virology with focus on HIV, Polio and Hepatitis. The mission of the Department is to be globally acknowledged and become a strong Regional WHO laboratory. The WHO-NIC virology lab is committed to surveillance, diagnosis, monitoring and control of influenza in Nigeria while serving as Reference Center for other laboratories. The Department of Virology is designated as WHO reference centre for arboviruses and influenza, the National Polio Laboratory, HIV Reference Laboratory and recently Avian Influenza Reference laboratory in Nigeria. ??? Functionality ? Construction of new laboratory building (BSL-2) for AI-diagnosis finalized in 2011 and funded by NAICP ? Construction carried out without consultation of lab experts, designed in a suboptimal way (work flow), so that further modification are needed. The architect came from the Northern States and he hardly managed to visit the site and never met the director for consultation. Finally, the building looks more like a living house. ? Equipment has arrived in May 2010, but not yet installed because constructions have just finished (installation planned within the next 3 months) ? Equipment provided by NAICP and in use (more details in table 7, Annex 1): o 2 Real-time PCR cycler (1 ABI 7300, 1 ABI 7500), provided in 2009 o Freezers, provided 2010 213 o Other equipment provided in 2010, but waiting for new constructions to be finished for installation ??? Performance ? Since 1974 National WHO Influenza Center ? Supported by WHO, CDC ? The UCH Ibadan is one of the rare centres (if not the only one) which can grow influenzan viruses in cell culture (MDCK, ECE) ? While conventional PCR is in use since 2007 for subtyping of H5 and H1/H3, real- time PCR (Pandemic H1/2009, H1,H3 seasonal, Flu A/B) has been used since 2009. Protocols of CDC/WHO are followed, reagents provided by CDC and WHO, but not yet from the NAICP ? In the basement of the building of the Department of Virology at UCH Ibadan there is a big division for molecular diagnostic work that is in use strictly supervised by a Molecular lab manager. The molecular division houses four real-time PCR cyclers (of which two were provided by the NAICP in 2009), several conventional PCR cyclers among an extraction robot, a sequencer and separate rooms for appropriate work flow. Most of the activities target HIV and Hepatitis research, while Influenza forms the minority of work. However, capacity (both equipment and skills) is strong for molecular research and should be applied for influenza research as well. ? Sequencing already for HIV, Hepatitis, in process for Influenza (primers awaited) ? Supply of reagents through WHO-AFRO funded by CDC/USAID, not from NAICP ? In process of accreditation (principles applied), accredited labs for Polio and HIV ? Annual maintenance/certification of equipment by a specifically trained engineer ? Proficiency Testing (WHO Hong Kong) twice annually since 2007, always 100% ??? Capacity building and lab networking: ? Multiple trainings for staff for molecular diagnosis carried out mainly by joined WHO/CDC/FMoH ? Increase of staff performing AI lab diagnosis, currently, 5 persons able to conduct cell culture, and 2 persons able to run PCR ? On teaching: The Department of Virology has produced 24 PhD and 150 Masters graduates in Virology ? Interdisciplinery link: Strong collaboration Animal and Human Health (NVRI Vom/UCH Ibadan): A PhD from NVRI Vom is currently performing a screening of pig samples deriving from various farms of Nigeria for Pandemic H1/2009. He is using real-time PCR cycler and reagents of the UCH Ibadan. ? Sharing resources might be possible (sequencer, maintenance of equipment) 214 Efficiency of the project for the laboratory component Timeliness & Cost-effectiveness of molecular biological technology in general While molecular biological technology contributes decisively to biosafety, speed (<12 hours) and precision, it is very expensive as opposed to classical techniques (virus isolation). Latter provides a result after minimum 24 hours, or with negative samples >120 hours. Although the classical pathway is much cheaper, it should not be given first priority as it is (i) not biosafe and requires high biosafety levels which are even after upgrading not given at the VTH and should only be conducted at the reference laboratories housing BSL-3 conditions, (ii) not sufficiently sensitive as only live virus can be grown and (iii) time consuming and laborious. Table 7: Advantages, disadvantages and costs associated with different laboratory testing procedures Technique Advantage Disadvantage Costs/sample (pool) Rapid Antigen detection test Rapid, within 45 minutes Lacks sensitivity, only usable in suspicious cases ~ 15-20 USD/sample (pool) Virus isolation Cheap, live virus for further studies (phylogeny, IVPI) Laborious, not biosafe as generation of high viral loads, time consuming ~ 1 USD /sample (pool) PCR Rapid, within 12 hours Sensitive, specific (esp. real-time PCR) Safe (lysis of virus) Risk of contamination Expensive Sensitive enzymes (needs constant storage) ~ 15 USD/sample (pool) None of the NAICP activities had an impact in cost-effectiveness. The reagents provided recently or in process of provision will last for appr. 1 year and then expire by end of 2011. There are no project plans for further distribution of reagents and there is a need for finding future funding sources. Animal Health In the past and until today, the NRL AI at NVRI Vom has been the centre for sample processing for antigen or nucleic acid detection as the VTH until now have been just involved marginally in AI diagnosis (serum investigation and usage of the rapid test between 2006-2008). Therefore, the VTH served mainly as an intermediate station for samples to be submitted to the NRL AI at NVRI Vom. However, they were active in receiving dead poultry from the farmers and conducting necropsies, sample collection and shipment of samples to Vom. 215 At the NRL AI Vom the following changes contributed to improved speed for investigation and reducing the time between sample reception and communication of a result: ? Improved logisitics: Samples are sent through plane (Jos airport) or through vehicle to the motor park (station) in Jos, where samples are collected through the logistics officer at any time and within one hour ? Improved work flow at the NRL AI in Vom since 2007/8: Whereas, until 2007/8 all samples were sent first to the central diagnostic laboratory to conduct necropsies and preliminary tests, workflow was changed in 2008, so that suspicious samples are directly sent to the AI-lab, where, under a BSC-II the bird is carefully opened, and samples taken and directly screened by conventional PCR since 2008 or by real-time PCR since 2009. For active and passive surveillance without an indication for a strong suspicion, samples are still first sent to the Central diagnostic lab, and all samples taken from poultry subsequently sent to the AI-lab for further investigation. ? Improved technology at the NRL AI in Vom: A real-time PCR (provided by SPINAP AU- IBAR) is in use since October 2009 which contributes to the speed of investigation (<10 hrs after sample reception). For suspicious samples a real-time PCR is carried out targeting the M-gene and subsequently H5, N1 genes. All positive samples are sent to the reference laboratory in Padua. Between 2006-2008 some reagents were provided by the NAICP but have been used up until now. ? Improved skills: While the number of staff conducting classical diagnostic techniques for AI detection (Virus isolation, HI, AGID) decreased between 2006 and 2010, the number of staff being able to conduct molecular techniques have increased (table 8) Table 8: Number of staff present to conduct a specific test in 2006 and 2010 ? Improved communication: Anecdotally, all positive results are reported promptly by mobile phone to the sample sender, the desk officers, VTH and to NADIS. However, results from negative samples deriving from active surveillances which were sent off to the lab were not communicated back to the desk officers, some of the active surveillance samples were not processed in the lab, because of lack of reagents (pers. Comm.) Type of test Number of staff present to conduct the test 2006 2010 Virus isolation in embryonated chicken eggs 11 6 HI Test / AGID 11 5 Conventional PCR 5 15 Real-time PCR 0 3 ELISA (Serology) 4 5 216 In general, the number of surveillance samples submitted varied over time ? submission peaks and reduction in number of sample submissions based on lab data (2010: Reduction of sample submission for passive surveillance by 50%) and surveillance officer data. The samples deriving from passive surveillance that were received were investigated promptly in real-time PCR (pers. comm.). Eighty-one samples from active surveillance are still pending for investigation by real- time PCR due to a lack of reagents. However, all have been tested negative in conventional PCR. Table 9: Constant decrease of samples submitted to the lab. All samples have been investigated by Virus isolation followed by conventional PCR 2006 2007 2008 2009 2010 620 563 437 333 164 Human Health With the emergence of the pandemic strain H1N1 in 2009, activities in the human health laboratories switched from H5 surveillance to Pandemic H1/2009 screening. As H5 detection is integrated in the work-flow, the global pandemic crisis had an impact on the improvement for H5 surveillance activities in humans besides H1N1. Samples mainly come from NISS and therefore from hospitals where people are found to have ILI-symptoms. Roughly, the NIRL Abuja receives samples from North-Eastern states of Nigeria, whereas UCH Ibadan theoretically is responsible for samples from South-Western states. Each sample is accompanied by a sample submission form asking for age, symptoms and also poultry die-off that the patient had potentially been in contact with. However, most of the activities were initiated by CDC or WHO rather than through the NAICP. NIRL, Abuja With the beginning of using real-time PCR in 2009 for sample investigation the lab has been very efficient in sample investigation. The time for generation of results decreased to <24 hours. However, when reagents are not available, sample investigations are pending for longer times. Reagents are shipped through CDC from the USA and not yet provided by the NAICP. Data are stored and analysed by use of EPI-Info for which most of the staff has been trained through national trainings (NAICP, CDC). While in 2008/9 only two of the staff members were able to carry out the real-time PCR, in 2010 the number has increased to four. A benefit from the NAICP is the improvement in logistics. Samples from other regions are sent by DHL and investigations are conducted promptly. 217 Sample flow is increasing since 2008, and a proportion of app. 3-4% are positive for Pandemic H1N1/2009 Table 10: Number of samples submitted and investigated by real-time PCR at NIRL Abuja 2008 2009 2010 Investigated Positive/H1 Investigated Positive/H1 Investigated Positive/H1 45 0 1104 13 2505 93 UCH, Ibadan Surveillance activities from the Southern states are reported to be too low, only 20 samples received from almost entirely Lagos TH, other samples taken on own initiative. Table 11: Number of samples submitted and investigated by real-time PCR at UCH Ibadan 2009 2010 Investigated Positive/H1 Investigated Positive/H1 172 2 313 15 16 virus isolates could be grown in MDCK-cells. Main Constraints associated with the effectivity of NAICP for Animal Health laboratories: ? Provision of equipment is strictly dependant to the bureaucratic procedures of WB, so that the preferred equipment (also regarding the sustainability) often could not be provided. ? No lab budget at the VTH to allow autonomous procurement of kits/reagents ? Power supply, water have worsened over the past 4 years (anecdotally) ? Lack of equipment, reagents for the VTH until reception of procured material ? No vaccination of lab staff (H1/3 seasonal, Pandemic H1/2009) Constraints Human Health Laboratories: ? There is a clear lack of funding for o reagents (PCR reagents very expensive) o Research and Training o Surveillance activities None of the Human Health laboratories has an own lab budget and all totally rely on external funding. Salaries are paid by the University, or in case of NIRL Abuja for some staff by CDC. 218 Timeliness of reagent supply->impact on diagnostic speed (turn-around time) Electrical irregularities became worse-> Impact on sensitive reagents (PCR), there is a need for installation of inverters covering at least 8 hours of electricity in emergency situation for freezers containing sensitive reagents. Lab networking between satellite labs of Human Health needs improvement -> surveillance, samples submission Surveillance activities are decreasing The UCH Ibadan holds a strong mandate, but needs more recognition and involvement in activities. Impact of the project for the laboratory component In joint action with the respective Departments of Agriculture and Health in, the NAICP, amongst other stakeholders and international institutes, have been involved in achieving the prerequisites for improving laboratory diagnostic of Avian Influenza Virus (AIV). International training and cooperation activities have improved disease response and management skills through capacity building and transfer of technology. This has also led to viral strains and biological materials being made available globally for applied research (phylogeny data) and ? through transparency- improved understanding of the epidemiological situation concerning AI. Low budget of the laboratories: With the current level of national funding for activities carried out in the laboratories, it is difficult to sustain the progress in the introduction of new activities for disease detection. Animal Health Once the the BSL-3 is constructed, equipped and fully functional at NVRI Vom, not only work flow and biosafety but also research capability will decisively improve (provided further funding!). Augmentation of the diagnostic/scientific operating range to phylogenetic studies will on the one hand better link with the scientific community (Genebank accessions, publications) and on the other hand increase educational background and training amplitude (sequencing) for veterinary and medical trainees. (Evidence: low, attribution to the project: high) Once the VTH will be upgraded it will: ? Significantly improve biosafety and work flow ? Improve diagnostic speed provided full functionality ? Improve diagnostic range from serology to nucleic acid detection by conventional PCR ? Improve content of curriculum for students / trainees (molecular technology, theoretically also applicable to other viral, bacterial diseases) ? Increase motivation and dedication of lab staff 219 NVRI Vom will serve as a source of protocols and guidelines If, finally, laboratory capacity is improved at the VTH, and provided further (financial) support, it will significantly optimize responses to an H5N1-epidemic or an outbreak of another pathogen, and will support national and international agendas to reduce the impact of emerging pathogen threats. (Evidence: low, attribution to the project: high) However, real-time PCR has not been used for the purpose of other diseases apart from AI because primers are missing, and compared to alternative methods, is too expensive. But there is ongoing effort to include real-time PCR as a back-up or confirmative diagnostic tool for TAD. Within the central diagnostic lab there is a molecular biology division which uses conventional PCR for the most common animal diseases. Human Health The impacts of past activities supported by the NAICP at the Human Health site are the following: ? significant improvement of biosafety, containment, quality assurance and work flow ? New building will also be used for diagnosis of other diseases and, provided an increased biosafety level, might enable to isolate viruses of other (zoonotic) diseases (e.g. Lassa). ? Towards One-World-One-Health concept: o Interdisciplinary approach has been strengthened over the past 4 years o International collaboration (sequencing projects) Preparation for the possible emergence of a Human H5 pandemic strain had a positive impact on the detection of Pandemic H1/2009 through enhanced collaborations, some equipment in place and better logistics and expertise. UCH, Ibadan In the basement of the building of the Department of Virology at UCH Ibadan there is a big division for molecular diagnostic work that is in use strictly supervised by a Molecular lab manager. The molecular division houses four real-time PCR cyclers, several conventional PCR cyclers among an extraction robot, a sequencer and separate rooms for appropriate work flow. Most of the activities target HIV and Hepatitis research, while Influenza forms the minority of work. However, capacity (both equipment and skills) is strong for molecular research and should be applied for national influenza research as well. Sustainability Animal Health 220 Without ongoing funding the laboratories will not be able to maintain BSL-2 or BSL-3 laboratories and will not be able to sustain nucleic acid detections as the reagents are expensive. The provided reagents will last for approximately 1 year until they expire. Currently, there are no targeted surveillance programs on Animal Health site neither for Influenza nor for any other animal disease planned or implemented on annual basis. The sample size arriving at the labs is constantly decreasing although there are reports about frequent poultry die-offs that would need clarification. NVRI Vom The NRL AI at NVRI Vom has shown evidence that it has improved and currently has full capability to diagnose AI, even without the input of external funds. This must be seen as a result of an individual?s and his team?s dedication and motivation to fulfil the mission of a Reference Laboratory. Maintenance, certification, calibration of equipment (69.000 USD) will be soon carried out by company from South Africa. The NRL AI at NVRI Vom still stores reagents for nucleic acid detection under perfect conditions with an inverter attached to the freezer containing the critical enzymes. However, by end of this year, most of the reagents will be expired or used up and might fail to produce qualitative valid results. (Evidence: high, attribution to the project: low) NRL AI at NVRI Vom has found a supplier in Lagos (Turmer Wright LTD, 15 Adenekan Salako Close OGGA Lagos) who easily and within a few days can ship any sort of reagent to the lab upon request. As a benefit of the good and sustained linkage with the OIE/FAO Reference Lab at IZSVe in Padua, reference material is shipped easily and at low costs from there to Vom as proven in the past. (Evidence: medium, attribution to the project: low). Sporadic or hardly any sample flow to the lab: A continuous sample flow is crucial to sustain the diagnostic process and keep all parties involved. In contrast to all other laboratories that do not have any own lab budget, the NVRI Vom is actively supported by the FGoN and receives a lab budget that also might cover purchase of new kits and reagents in case of emergency. But lab budget is not sufficient to cover investigation of samples from active surveillance (screening of healthy poultry flocks). Also, other research activities (augmentation of the scale of research activity) rely on external funds. In relation with fading funds (expired projects) for particular disease detection (e.g. CSF, FMD), also laboratory capacity might have to step backwards to very basic investigations. However, there is ongoing effort to include real-time PCR as a back-up or confirmative diagnostic tool for TAD. The NVRI produces hyperimmunsera and other diagnostic material, like RBC, ECE, VTM and is able to switch to alternate diagnostic techniques in case finances do not allow nucleic acid detections. 221 VTH Zaria Unlike the NVRI, the laboratories at the VTH do not have an own lab budget and are 100% dependant on external funds which makes a sustainable long-term maintenance difficult. After the new laboratories will have made fully functional it will be still a long way to fulfil the mandate of screening samples by molecular technology which needs strong input and support by a ?supervising? institute. At the VTH Zaria reagent for PCR have just been received, but most likely will be stored under suboptimal conditions as there are irregularities in electricity and freezers will have fluctuating temperatures. This will have a bad impact on the functionality of enzymes and they might damage very soon. Therefore, there is a chance that, once the equipment is fully functional and staff prepared to start nucleic acid detection, the reagents might be spoiled by then if not stored properly and in a different place, preferably at NVRI Vom. The NVRI will most likely have to play an important role in supervising and should function as a permanent mentor providing training and expertise, but also as a supplier of reagents upon request. Protocols should be harmonized among the laboratories which might be still a long way to go and will certainly need a lot of input and support. If VTH are fully functional and equipment installed, there should be an agreement that servicing and maintenance of critical equipment (BSC-II, PCR cycler?) will be provided as well in future. Human Health All laboratories visited still store reagents (under suboptimal conditions in the labs as an inverter is not available and fluctuations in temperatures of freezers do occur) for nucleic acid detection. However, by end of this year, most of the reagents will be expired and might fail to produce qualitative high results. (Evidence: high, attribution to the project: low) None of the Human Health laboratories have an own lab budget for autonomous purchase of diagnostic material. Upon request CDC or WHO provide reagents for testing for no costs, but sometimes shipment takes some time. (Evidence: medium, attribution to the project: low) Certification and Maintenance of sophisticated laboratory equipment is carried out in an efficient and effective way by a specifically trained engineer at UCH Ibadan. CDC funds accreditation and maintenance/servicing of equipment at NIRL Abuja. There is a risk that activities (particularly surveillance) for Influenza will slow down eventually. Recommendations Laboratory visits and collection of sensitive data 222 ? The data/analysis from the laboratory assessment is an extremely delicate matter. Any kind of publication or presentation of the data where the laboratory is named should be first approved by the respective authorities of that country and the laboratory directorsof the respective institutes. ? A laboratory assessment is always subjective and might be biased. Data deriving from such assessment should be confirmed either by review of available reports/publications or consultation with other persons known to the region and the lab, or by execution of a second visit. ? Two independent consultants, ideally one national and one international consultant of possibly different disciplines (e.g. one vet, one epidemiologist), undertaking the interviews together would significantly improve quality and number of data obtained from laboratory assessments within shorter time. Laboratory functionality, staff, capacity building ? When designing a new lab, it would be crucial to consult the lab staff and listen to them prior the start of constructions, otherwise the constructions might result in a way that does neither favor the work flow nor appropriate biosafety rules for decent laboratory diagnosis. ? When purchasing equipment it would be necessary to have a preliminary critical and careful assessment whether the laboratory will be able to use and maintain the equipment. Worst what could is the allocation of modern and tremendously expensive equipment that afterwards cannot be used due to lack of reagents/consumables or lack of maintenance possibilities. ? All laboratories should have inverters installed to provide at least 8 hours of electricity in emergency situation (power cuts) for freezers containing sensitive reagents (enzyme) to make sure there is no damage of the expensive reagents. ? International accreditation (ISO 17025) is a pledge for sustained good laboratory practice and should be targeted. However, in most cases it is prohibitive and thus not sustainable in resource-constrained settings. It is worthwhile to explore alternate accreditation models in such settings, which can be used as a stepwise approach to full international accreditation. Alternatively, funding plans for accreditation might be considered. ? Improving laboratory capacity by long-term commitment to staff training, quality control, and biosafety will be absolutely essential for the (Veterinary) Teaching Hospitals which will have a newly introduced lab and lab set up: o NRL AI at NVRI Vom could act as a supervisor based on MoU in all activities and capacity building for VTH o Likewise, WHO Influenza Lab and NIRL Abuja could act as joint supervisors based on MoU in all activities and capacity building for TH (Human Health) o Support and guidance through the supervisor in serving as a repository for reagents and consumables to be distributed to the (V)TH upon request o Support and guidance in self-sufficient equipment maintenance and laboratory management including Quality Assurance, biosafety and biosecurity, also given by the supervisors 223 o Standardized approach by introducing and using internal quality control (IQC) and external quality control (EQC) under a QA system with a harmonized national guideline ? Long-term in-house training, ideally twinning and mutual exchanges and collaboration with Reference Labs with a broad range of basic methods and technology for animal disease detection ? Institute directors and laboratory leadership should provide stronger commitment also to staff health. Already existent funding for vaccines (WHO) should be used to ensure that all lab staff is vaccinated when handling with the respective zoonotic agents where vaccination is feasible. International organization (WHO, FAO, OIE) might consider drafting recommendations for an affordable and easy-applicable system to assure staff health to be implemented in the national governmental system. Surveillance plans ? To estimate whether H5N1 is still circulating undetectably in the country, human labs might consider investigations of targeted human serum samples (LBM, slaughter houses, farmers, ILI+dead poultry) for H5 antibody reaction in consultation with international influenza laboratory. ? National Annual Plans for targeted surveillance (e.g. compensation of travel costs when samples are collected in the fields), ideally PDS teams take samples when confronted with increased poultry mortalities in one epidemiological unit Further capacity building of laboratories and improving (interdisciplinary) lab networking in Nigeria In order to further strengthen laboratory capacity and improve lab networking there is a need for: ? Improvement and maintenance of a high standard of laboratory operation requires financial commitment and support from the government and a time commitment from laboratory leadership and staff. All need to be convinced that maintaining the level achieved and augmenting the diagnostic range is essential for delivering good quality animal disease diagnosis and research: o Government and the respective investigation centre / institution might have to (re-)consider the importance of laboratory diagnosis within control and response strategies of animal diseases and strengthen its role in annual plans. o Promotion of self-sufficient disease control programs. ? International support and assistance in the preparation and submission of funding proposals for continuation of existing programs (competition, rewarding system for the best laboratories?) ? a National Plan for Sustainability, e.g. reagents and consumables supply for nucleic acid detection (PCR) required ? regular meetings of state officers to find best practices 224 MoUs might be useful to clearly coordinate mandates and activities of laboratories as well as sharing resources (also interdisciplinary) about: ? trainings activities among the different levels of laboratories, preferably combined with an annual in-country Proficiency testing system (molecular) ? regular meetings of the lab directors (also interdisciplinary as far as zoonotic diseases are concerned) and lab managers ? In order to launch a continuous sample flow to the laboratory, it is recommended that field veterinarians and laboratory and epidemiology experts, regional laboratories, governmental decision makers etc. team up and plan together regularly (meetings) in a way that routine sampling activities in the field and routine testing in labs are effectively implemented (in annual plans) and coordinated. This should be based on MoU. ? Better communication between Animal and Human Health side: Reporting of sick/dead birds occurring in connection with NISS (persons with ILI-symptoms) should be immediately reported to the Animal Health Desk officers who could send PDS teams to that site to take samples from the dead / sick poultry ? Sharing resources: o Sequencer at UCH Ibadan: Supported by the CDC, the Department of Virology at UCH Ibadan will soon be able to conduct sequencing of Influenza strains and could serve as a source of information for other Phylogenetic data on influenza strains sent in from the network of laboratories within Nigeria. There already exists a strong interdisciplinary collaboration so that joint actions could be a fruitful outcome of such linkage. o Maintenance and Calibration of critical equipment: For the NIRL Abuja and NRL AI at NVRI Vom a company has been hired (funded by FGoN) for servicing, changing Hepa filters in near future. The costs are high (60.000 USD). In long- term future, an engineer that has been specifically trained in the maintenance and calibration of BSC-II and cyclers and currently does the service for the UCH Ibadan could be shared among the different labs which would significantly decrease costs. o Reagents that have been sent to the VTH prior to its functionality (new building and infracstructure) of diagnostic operation should be submitted to NVRI Vom for proper storage. o Introduction of a maintenance budget for each laboratory, also at the Teaching Hospitals. Governments, donors and the laboratories should standardize where possible on equipment and establish or work with regional or local companies to establish long term maintenance arrangements that are affordable within the budget. In their functions as Regional Service Laboratories the NVRI Vom as well as the WHO-Influenza Center at UCH Ibadan should receive all samples tested positive for Influenza at the any of the Veterinary or Human Teaching hospitals for confirmation and further research. Both should have the mandate to act as superior Reference Centers. 225 International disease control programs should focus on upgrading and promoting an advance laboratory, particularly the laboratories that have already been designated as Regional (Veterinary/WHO) Service Laboratory (e.g. NVRI Vom and WHO-Center at UCH Ibadan) to better coordinate assistance in capacity building of the national labs on country level and within a the Western African region. A strong linkage of the Regional Veterinary/WHO Service Laboratory with International Reference Laboratories as well as International Organizations and the allocation of funds for related activities would help to fulfill the following services (provided the availability of sufficient funds): ? A collection of harmonized, standardized protocols, SOP, guidelines, recommendations ? Procurement of reagents for building a stock from which material can be allocated to the labs in the region upon their request and stored properly ? Support other laboratories in sample submission to Reference laboratories ? Training work-shops and long-term assistance in disease control, diagnosis and application of technology ? Promote train the trainer workshops (quality assurance, biosafety/biosecurity and laboratory management) ? Annual technical meetings with the authorities and scientific lab staff of national laboratories of the region to plan for common strategies, address the most urgent needs, coordinate activities with national labs for animal disease control, and scientific exchange ? Twinning activities with the labs in the region ? Organization and execution of Proficiency Testing in the Region Lessons learnt during country/laboratory visits ? All persons met and interviewed in the laboratories were very open to address the strengths and weaknesses of their labs, and were very supportive in answering the questions and sharing information. ? The consultancy would have been more effective, efficient and less stressful if there were at least 2 persons, preferably, a lab person combined with an epidemiologist, working in a team for the lab evaluation. ? It would have been more efficient if questionnaires were submitted to the labs prior to the lab visits in order to fill data about equipment and sample submissions and results. The filled questionnaires could have then been used as a basis for discussions and verification while trying to find evidence. Problems encountered ? The required time for visiting labs and asking questions was underestimated. ? The consultant often was confronted with corruptions ? At NIRL Abuja, director was only available for discussions for short time due to their own busy schedules, so they could not be asked directly, and the lab staff often was not sure about specific questions. 226 ? At NVRI Vom a virus in the computer hindered taking a look into all soft-copy data sets, information were sent by email with some delay as the person in charge was on duty travel. 227 Appendices Tables ToR Calendar and Itinerary Persons met Excel-file with data (evaluation criteria) 228 TABLES Table 1: Geographic location, linkage with satellite labs, overall status of lab, administrative data, overall facility and basic supply for the Animal Health Labs 1+2 (NVRI, Vom, VTH Zaria) and Human Health Labs NIRL Abuja and UCH Ibadan, separated for the years 2006/07, 2008/09 and 2010/11 229 Table: Improvement of infrastructure at NVRI, ABU VTH Zaria, NIRL Abuja and UCH Ibadan during the years 2006-2011. N/A= Not applicable 230 Table 3: Evolution of communication means, allocation of lab activities and number of staff available and competent at NVRI, ABU VTH Zaria, NIRL Abuja and UCH Ibadan during the years 2006-2011. 231 Table 4: Staff skills related to Avian Influenza for NVRI Vom, ABU VTH Zaria, NIRL Abuja and UCH Ibadan. Abbr.: # staff= number of staff. Most improvements did not relate to the NAICP Table 5: Technologies available at the different labs. 232 233 Table 6 : Training / Education provided to lab staff (not exhaustive); most training was not funded by NAICP, but many other donors (see text). Abbr. : # staff = number of staff 234 235 Table 7 : Equipment available at the different labs separated for the years 2006/07, 2008/9, 2010/11. Note that the numbers given for 2008/9 and 2010/11 is the total of equipment (sum) and not the number of equipment provided each year. Colour coding: no colour: in use and provided by other donor; green: provided by NAICP and in use; red: provided by other source and not in use; yellow: NAICP equipment still under procurement or not yet installed. 236 237 Table 8 : Reagents available at the NVRI, ABU VTH Zaria, NIRL Abuja and UCH Ibadan at different years. Numbers indicate the number of reactions. Colour coding: no colour: in use and provided by other donor; green: provided by NAICP and in use; red: provided by other source and not in use; yellow: NAICP reagents still under procurement or not yet delivered. Funding source and constraints of reagents supply. 238 239 Table 9 : Quality assurance and Biosafety/Biosecurity applied at the different laboratories and improvement during 2006-2011 240 241 Table 10 : Accessions (only AI) to the laboratories for the different years, turn-around-time for sample submission to reporting and shipment of samples to international laboratories. 242 Table 11 : Most important donor organization as reported by the laboratories related to AI and the years 2006-2011 Table 12 : Laboratory collaboration, incomplete for NVRI Vom. 243 244 ToR Terms of Reference Laboratory Expert Impact Assessment Study of the Nigeria Avian Influenza Control and Human Pandemic Preparedness and Response Project (NAICP) ILRI has been invited by the World Bank to undertake an impact assessment of the World Bank?s emergency loan to support the Government of Nigeria?s response to the outbreak of highly pathogenic avian influenza (HPAI) from 2006 - 2008. The consultant will be a member of a team of ILRI and other independent scientists implementing the study as outlined in the proposal submitted by ILRI to the World Bank in December 2010, and will be responsible for undertaking an assessment of laboratory capacity, performance and quality. This component is expected to cover issues primarily related to the impact of NAICP interventions on improving the capacity of laboratories in Nigeria to diagnose and characterize HPAI, and to establish capacity to detect human influenzas. Specific responsibilities of the Laboratory Expert will be: 4. Based on the study proposal, inception report, and these terms of reference, prepare a draft framework and detailed protocol for the laboratory assessment component, including any survey instruments required, where applicable. This will be submitted to the Team Leader for review before undertaking the mission to Nigeria. 5. Make site visits to a sample of veterinary and medical laboratories that benefited from NAICP investments in infrastructure and training to: a. Evaluate the current operating status of the infrastructure and equipment installed, and their ability to detect HPAI and/or other influenzas, and other diseases more generally b. Evaluate the current capacity and performance of laboratory staff to detect or confirm HPAI and/or other influenzas, and other diseases more generally c. Document and evaluate the capacity building undertaken at different levels and by speciality d. Collect data needed to describe the evolution of the performance of the laboratory in detecting HPAI or influenza since the beginning of the project e. Assess the sustainability of the improved capacity, both in terms of budgetary commitments for maintenance of critical infrastructure, purchase of supplies, and staff, and in terms of maintaining the level of staff training. f. Analyze the information and data collected and prepare a draft technical evaluation report for submission to the Team Leader by 18th March 2011, revising as needed based on feedback. The consultancy is expected to require approximately 12 days. Travel expenses will be paid separately. Calendar and Itinerary Thursday, 3/3/2011, 6-19: Trip to Berlin to Nigerian Embassy and back to obtain visa. 245 Sunday, 6/3/11, 7 am-4:40 pm: Travel from Stralsund to Cologne, stay over night at relatives Monday 7/3/11, 7 am-8 pm: Travel from Cologne to Frankfurt + Flight to Lagos, discussions and document sharing with Joerg Henning; arrival at International Guesthouse IITA, Lagos; arrangements, emails until 23:30 Tuesday 8/3/11; 6:30 am: Travel from Guesthouse to airport; flight from Lagos to Abuja with Nigerian Airline, due to lack of communication: not been picked up, 11 am: finally picked up by driver, drive to Abuja City Center (> 1hour); check-in Hotel, 12:30: immediately to AICP-office (meeting Ismaila M Shinkafi and Mrs Agbai), 1-4 pm: NIRL in Abuja, inspection, discussions, interviews 18-19 pm: meeting with Kunle Adesoye about travel arrangements from Thursday onwards 20-0:00: Work on instrument for lab assessment (excel file, questionnaires) Wednesday 9/3/11; 9am-4pm: NIRL Abuja, collecting data and interviews; not accompanied by anyone; Dr. Adedeji Adebayo (Assistant director) not available as appointment with minister. The only ones assisting eagerly are the female laboratorians (scientists). 12:30-1pm: Brief and only discussions with director; 2-3:30 pm no person available in lab for interview. 3:30-4:30pm interview and data collection of lab scientists, inventory, equipment, samples investigated and positive etc. 6pm-23:00: Work on data, instrument, emails, questionnaires Thursday 10/3/11; 8-9: Data entry, instrument, preliminary summary NIRL Abuja 9-12:15: Travel to Vom + lunch, meeting Tony Joannis 1-2: Briefing with Director Dr. David Shamaki 2-6: very good discussions with Kunle and Tony Joannis; hand over of questionnaire to be filled In guesthouse, no internet 7:30-10 pm: Working on questionnaire, instrument, excel file, summary of information Friday 11/3/11: 9-15:30: Lab assessment NVRI Vom, questionnaire; lab staff not available for filling the questionnaire, virus on computer that made it impossible to look into data or to hand over data. Dr. Joannis promised to send data by email within the next days 15:45-18:30: Travel to Abuja -> Rockview hotel 20:00-22:30: Work on questionnaires, documentation. Internet not working Saturday, 12/3/11: 8-14:35 non-stop: Work on Evolutionary Tool for Animal + Human Health Component (raw-data incorporation) + Protocol + modification of questionnaires 15:30-19:00: Emails + answers + work on tool (excel-file) 21:00-22:30: Organisation, screen of documents on ILRI-webpage, copy and download (self-briefing) 246 Sunday, 13 March 2011: 8:30-12: finalizing questionnaires, emails, documents, filling data in excel sheet; writing on findings for NVRI Vom 12:30 : Check out 12:30-17: Drive to Zaria with K. Adesoye; night in Teejay Palace Hotel 20:30-23:00: Document screen and data entry (NVRI Vom) Monday, 14 March 2011: 7:30-9:00: preparation, work on summary-report for NVRI Vom 9:30 Departure from hotel to VTH ABU Zaria 10-10:30: Visit of new lab building 10:30-14:30: Interview with Director, Lab Manager, Lab biotechnologist 14:30-15:00: Lunch break 15:00-16:30 Visit of labs currently performing diagnostic tests 17:00: Back at hotel 20-23: working on documentation; night in Teejay Palace Hotel (8000 N) Tuesday, 15 March 2011: 8-13:00: Drive to Abuja from Zaria, car broke down 150 km away from Abuja, not repairable; Continue drive by Taxi to Abuja 13:00 Office AICP 13:30-14:20: Drive to airport (Domestic) 16:30-18:00: Flight to Ibadan 18-19:30: Drive to IITA guesthouse 20:30-22:00: Discussions Joerg, Acho, Bernard 22-23: Emails (information from Team Leader to submit preliminary report on 17 March) Wednesday, 16 March 2011: 8:00-8:15: Talks with Heather, Joerg, Bernard 8:30-8:50: Talks with Acho at IRLI ? office 8:50-9:00: Talks with Joerg 9:00-9:25: Talks with Thomas Randolph 9:30-13:00 : Work on Excel file, data incorporation 13:00-14:00: Lunch 14:00-15:30: Emails+Discussions Heather Hannah 15:30-16:30: Meeting Dr. Olarewaju, HH Desk Officer, discussions including Acho 16:30-18:15: Preparation questionnaires for Human Health and print-out; Documentation, Data-entry in excel file Later: All efforts to concentrate failed, too exhausted!! Thursday, 17 March 2011: 6-7: Data entry 8:30-9:45: In office, print out, collection of Material 9:00-9:45: Drive to UCH Ibadan 9:45-14:00: Discussions with Professor David Olaleye, Director of the Virology Division, and Dr. Georgina ? 14:00-16:00: Discussions with Director of UCH Ibadan, Professor and tour-around the hospitals; 16-16:30: Discussions, and looking to the analysis of H1-subtyping by real-time PCR (QA-QC) 17-18:30: Emails in Office 20:30-23:00: Data entry, draft preliminary skeleton of report; submission to teamleader 247 Friday, 18 March 2011: 8:30: Drive to UCH 9:15-: Discussions, with Director Prof Olaleye; other activities, electricity problems; waiting for lab staffs availability; person with key disappeared 10-11:30: Visit of the new lab 11:30-13:30: Visit of currently used lab with Dr. Clement Meseko and Dr. Bakarey Deleye Solomon 13:30-17:00: Filling questionnaire with CD and BDS, compiling data 17:45-18:30: Back to the office, data entry Saturday, 19 March 2011 9:30- 22:30: Writing on report, data entry + preliminary analysis Sunday, 20 March 2011: 8:00 Visit of Dr. Georgina, final discussions and hand-over of some documents 9:30-10:30: Internal meeting with Team and Team leader (Prof. Perry) 10:30-18:20: Work on presentation, report and analysis 20:30-0:00: Work on presentation, data entry Monday, 21 March 2011: 9:00-18:00: Inter-team workshop on results of impact assessment 19:00-21:00: Work on key messages, timeline, efforts to obtain documents from Dr. Joannis Tuesday, 22 March 2011: 8:00-10:00: Work on key-messages, contradictions and matches, efforts to obtain more documents for evidence 10:00-13:00: Work shop and presentation of findings 14:00-18:45: Writing report and submission of preliminary version to Brian Perry by Email 19:00-0:00: Fever (>40?C), clinic, sub-consciousness, overnight at clinic Wednesday, 23 March 2011: Diagnosis of clinic: Malaria, Feverish and diarrhea, no work possible Thursday, 24 March 2011: 8-15:00: Still not well, resting 15:00: Departure to Lagos 22:50 Flight to Frankfurt Monday ? Wednesday, 28-30 March 2011: Finalization of draft report and submission to Brian Perry Sunday ? Monday, 3-4 April 2011: Review of the WB-draft report circulated by the team leader Wednesday-Friday, 6-9 April 2011: Finalization of lab-assessment report to be submitted to the Team leader 248 Persons met Name Position Email Mobile E V. - T E A M Prof. Brian Perry Thomas Randolph Acho Okike Kemi Delia Grace Heather Hannah Pamela Pali Joerg Henning N A I C P Ismaila M. Shinkafi Planning Officer/AICP danalhajis@yahoo.com 08089939955 Mrs C. V. Agbai Subcomponent Coordinator (Surveillance)/AICP Chivicagabai@yahoo.co m 08035624428 N I R L A B U J A Dr. Adedeji Adebayo BSC MSC PhD Assistant Director (Human Health Lab, NIRI) adedejiseyedeola@yah oo.com 08037203712 Dr. Paul Shilo Lab manager NIRI Amina Garba S. Abu Biotechnologist Ammee.garba@yahoo. com 07036006608 Adama Ibrahim Musa Biotechnologist Daccu2003@yahoo.co m 08038599445 Abugail Mwakamma Technician abiinkwe@yahoo.com 08050565377 Pamela Amaka Onyiah Biotechnologist pamelaonyiah@yahoo. co.uk 08034954861 Abdul Driver 080319231348 Kunle Adesoye NAICP, Monitoring and Evaluation kadesoy@yahoo.com N Dr. Tony Joannis Head of AI-lab tmjoannis@yahoo.com +234-80 37024280 249 V R I Dr. David Shamaki Director Research dshamaki@nvri.gov.ng +234-8037000449 V T H Z A R I A Dr. L. Sai?du Director/Senior Consultant lasawabo@yahoo.com 08034504097 Yunusa Yahahya Pala Technician? yypalas@yahoo.com Dr. Alihn M Wakawa DVM drsaniwakawa@yahoo. com 08037044713 Dr. Musa I Wazini DVM ibwazkah@yahoo.co.uk 08036659789 I B A D A N H H Francis Olanrewaju Desk officer AICP Francis_olawale2001@ yahoo.co.uk 07032327155 08051049937 Prof. Temitope O. Alonge Chief Medical Director, UCH Prof. David Olaleye Director Virology, UCH ibvirology@yahoo.com Dr. Georgina Odaibo Deputy of Director for Virology Dr. Clement Meseko, DVM PhD / Vom/UCH Ibadan cameseko@yahoo.com +234-80-39183988 Dr. Bakarey Adeleye Solomon, DVM Molecular lab manager, UCH drbakarey@yahoo.com +2348025187407 250 ANNEX 5 SECTORAL ANALYSIS Introduction The poultry sub-sector is an important segment of Nigeria agricultural economy. The total population is estimated at 160 million birds comprised of indigenous poultry in backyard extensive systems and exotic breeds in commercial intensive systems. Exotic breeds account for 24 million layers and 40 million broilers (Akinwumi et al., forthcoming). By some accounts, it is the most commercialized of all of Nigeria?s agricultural sub-sectors, contributing up to 36% of total protein intake in Nigeria, attracting significant inflow of investment and, thus, generating a net worth of USD 1.7 billion a year (Federal Republic of Nigeria, 2007). Based on FAO classification, the Nigeria poultry sector can be divided into sector 1 commercial (more than 10,000 birds), sector 2 medium scale- commercial (between 2,500 and 10,000), sector 3a small-scale commercial (between 500 and 2500 birds), sector 3b backyard (between a few and 1500 birds), and sector 4 rural (between a few and 1500 birds (Pagani, 2007). A sectoral study focusing on the poultry and egg value chains in Nigeria was conducted to assess the extent to which market agents in these value chains were impacted by the intervention. The principal categories of agents considered included breeder/hatchery farmers, broiler famers, layer producers, traders, live-bird market (LBM) traders, processors, transporters, small-scale feed (?toll?) millers, and branded feed millers. This study sought to answer three fundamental questions: (1) to what degree HPAI impacts different agents differently, (2) how the intervention has mitigated the impact of the disease for the affected agents, and (3) did the intervention create additional costs and institutional restructuring for various groups of actors. The study addressed these issues within a value chain framework to understand how the project has impacted the poultry value chain in terms of biosecurity adoption, governance, cost structure, and performance. Methods A rapid qualitative appraisal was conducted by cluster sampling of representatives of the various categories of poultry value chain operatives in Nigeria?s four geographic clusters where the NAICP was active. These are North West Cluster with Jigawa, Kano, Katsina and Kaduna States; North Central Cluster with Nassarawa and Plateau States; South East Cluster with Enugu and Anambra States; and South West Cluster with Oyo, Lagos and Ogun States. There were no feed millers, toll millers, breeder/hatchery operators in Nasarawa and Jigawa States and no breeder/hatchery operator in Kano State. Numbers of operators participating were: live-bird market case study (7), broiler producers (17), layer producers (16), processors (11), traders (11), transporters (13), feed millers (11), toll millers (12), breeders/hatcheries (8), key informants interview (3), market survey (8), LBM processors (6). Overall, the planned number was 150 and the achieved sample size was 123. The selected participants met in focus groups of 8-12 people, during which a checklist of questions was discussed to solicit and understand the participants? perceptions. Independent of the focus group discussions, the participants were also administered individually a questionnaire by enumerators to collect more detailed information. A market survey for current prices of inputs and products was also conducted in each of the selected States Information collected included participant perceptions about changes in compliance with respect biosecurity measures and the impact of compliance on their poultry-related livelihood and general welfare. Data were also collected for estimating representative enterprise budgets to gauge the business performance of the various actors along the chain before, during and after the outbreak. 251 Findings The use of convenience sampling, the small numbers of respondents by category, and the reliance on perceptions and self-reported practices create the potential for significant bias -- particularly affirmation bias -- in the information collected. Moreover, trends in profitability were difficult to analyze due to the impact of other factors independent of HPAI, such as the sharp increases in food and feed prices over this period. The findings should therefore be considered as indicative, but not conclusive. Key points emerging from the information collected are summarized below. Differential HPAI impacts ? With the exceptions of the breeders/hatcheries and toll millers, all of the various value chain actor categories suffered severe losses during the HPAI outbreaks with substantial reductions in their gross margins, leading to reported instances of untimely sale of assets, including land and buildings to pay bank loans. The impacts of HPAI identified were indebtedness, reduced patronage, reduction in the prices of DOC and loss of birds due to overpopulation, lack of market, and drastic reduction in income. Impact of the intervention on disease risk ? Most operatives acknowledged the role government intervention played in rescuing the poultry industry albeit some perceived lack of fairness in the manner in which compensation was allocated. ? Transporters judged compensation to have greatly enhanced compliance, but that the public awareness efforts were even more effective. ? Breeder and hatcheries were already practicing good biosecurity before the outbreak, so they did not perceive any change in the risk they faced. They perceive the need, however, for even stricter biosecurity measures. ? Broiler and layer operators, processors and transporters?whether they benefited from training or not?reported practicing biosecurity before the HPAI outbreaks, and subsequently intensifying their biosecurity voluntarily despite the extra cost incurred. Transporters felt that the extra cost was not high and that their implementation of better biosecurity helped to stop the outbreak. ? Over half of traders are thought to comply fully with the recommended biosecurity measures (although this appears to be at odds with certain results from the biosecurity adoption component in the animal health section of this report). ? Traders in LBM noted that regulations are enforced with respect to market cleanliness, improved hygiene, construction of pits for waste disposal and not allowing live birds to leave the market once they have entered it. Biosecurity is also enhanced by the improved LBM infrastructure and use of better cages. ? Transporters noted that some regulations affecting them had changed after the outbreak began, but that public awareness efforts had not reached many transporters. ? Biosecurity practices, which include cleaning of premises, using selected raw materials, and disposing waste properly, remain voluntary and are used variably among millers. Some toll millers continue to recycled used bags either as is or after washing, but this adds to cost. Impact of the intervention on the poultry business ? The recovery is ongoing at different paces through cost savings and/or improved revenues. ? Layer operators, processors and traders have not been able to recover to pre-HPAI levels. Layer producers and processors indicated that total operating cost has significantly 252 increased, while revenue has grown at a slower rate. The cost structure for traders, on the other hand, has become slightly more competitive while revenue has declined slightly relative to pre-HPAI, leading to a similar result. ? On the hand, breeders, broiler producers, transporters, and toll millers report better profitability than pre-HPAI. ? Traders participating in the few improved LBMs have fully recovered from the HPAI episode, as opposed to traders more generally. ? Layer operators noted more government regulation as part of the control efforts, though weakly enforced. This includes farm registration, biosecurity measures and disease reporting. ? The perception among breeders/hatcheries, broiler and layer operators, processors and feed and toll millers is that their numbers and business volume has been increasing since the outbreak of HPAI in 2006. ? Existing collective action mechanisms such as trade associations among breeders, hatcheries, producers, and traders facilitated their participation in the Interventions such as training were facilitated by Conclusion Biosecurity is not new to the more commercially-oriented value chain operatives but its application has intensified in the last five years. More advanced measures are also being introduced in Nigeria, particularly in layer and broiler operations, the processing business and live-bird markets. The value chain operatives, by and large, attribute these improvements to the effort to control HPAI. Although most of the operatives expressed their willingness to implement these measures despite their increasing effect on operating cost, the added cost can be expected to hamper adoption of improved biosecurity. No significant change in business structure or practices due to the HPAI control efforts was identified, other than the small number of improved LBM that are being piloted. Contracting arrangements, for example, have not appeared in response to perceived demand for biosecurity. Nor has the experience with HPAI and its control appear to have spawned new types of collective action to coordinate biosecurity, encourage compliance, or offer cost sharing, training and exchange of ideas Government regulations have played a significant role in improved cleanliness and hygiene in improved markets, but less so elsewhere. Group cohesion and management among the value chain operatives has changed little, except among traders operating in the improved LBM. While some of the other operatives continue to belong to larger organizations such as the national poultry producers of Nigeria, collective action at the local level was not widespread. References Akinwumi J, Okike I, Bett B, Randolph TF, Rich K. (2009). Analyses of the poultry value chain and its linkages and interactions with HPAI risk factors in Nigeria. HPAI Research Brief 16. Pro-poor HPAI Risk Reduction Project. Washington, D.C. Federal Republic of Nigeria. Avian Influenza Control and Human Pandemic Preparedness and Response Project National Baseline Survey Final Report December 2007 Pagani, P., J.E. Abimiku W. Emeka-Okolie. Assessment of the Nigerian poultry market chainto improve biosecurity, November 2008 253 ANNEX 6 LIVE BIRD MARKET IMPACT ASSESSMENT Introduction The NAICP has piloted 5 upgraded live-bird markets (LBM) as a means to improve biosecurity and reduce the risk of HPAI transmission. The intervention involves improving physical infrastructure and operation of the market, encouraging better practices and changing the business structure for how the market is managed. The project introduced the use of disinfectants and application bio-security by LBM operatives and enforced their use through the LBM elected management by incorporating application of disinfectants and bio-security measures into the by-laws of the LBM management. The major objective of this case study is to analyze the financial sustainability of the pilot improved LBM. The framework for the financial analysis is based on the discounted measures of project worth approach in investment analysis. The annual costs and benefits were projected for 20 years and the applicable discount rate set at 12%. It is important to point out that although the general design, theory and implementation are similar across the pilot LBMs, their operating scales and quantum of investment differ greatly. Given the heterogeneity of these variables, their arithmetic average may not be a good representative of their central tendency. It was therefore deemed sufficient to examine a single case study to study their financial sustainability. Methods and procedures The data employed in the analysis were cross sectional data obtained from a set of single-visit interviews of LBM traders conducted by ILRI recruited enumerators for this purpose using cluster sample rapid appraisal survey technique. Kaduna LBM was selected as a typical pilot improved LBM to study. The prices were constant 2010 prices and the data on live-bird sales, incremental investment, and operating costs were collected following the double difference (?before and after? and ?with and without?) project intervention method. Incremental costs and benefits for the live- bird traders were estimated on annual basis and projected for 20 years. All of the usual caveats apply to the quality of the data given the difficulties for market agents to maintain financial records or recall historical (pre-project) data and the natural tendency of operators to be protective of or unclear about their financial performance. There is also a natural tendency to affirmation bias with participants being optimistic in their expectations about future financial performance, and wanting to feed back positive results to continue attracting external support and subsidies to their business operations. This report is a very short summary of five detailed excel work sheets comprising: The operating inputs analysis worksheet, the input and output prices analysis worksheet, the variable and investment cost analysis worksheet, the operating pattern analysis worksheet that quantified the throughput in detail, the cash flow analysis worksheet. They present a full account of the financial spreadsheet. These enabled the estimation of Internal Rate of Return (IRR), Benefit Cost Ratio (BCR), Net Present Value (NPV) and Annual Incremental Benefit (AIB) for the improved LBM. Assumptions The improved LBM was established in 2009. We assume a 3-year adjustment period as the volume of business for the improved LBM increases before reaching full capacity. Costs and Benefits Investment in the LBM consists of the building and the installed accessories, which amounted to USD 110,500. Operating cost comprises annual costs of product stocks of local chicken, broilers, layers and crates of eggs purchased for sale from the farm gate; and annual costs for water, rent, transportation, feed, disinfectants, medication and labour. 254 The tangible monetary benefits of the upgraded LBM are derived from the mark-up on the sales of local chicken, broilers, layers and crates of eggs. Intangible benefits are derived from increased sales and processing of disease-free birds in a hygienic environment. Prior to the upgrading of the LBM, traders were not using disinfectants and applied medication and biosecurity sparingly. The use of disinfectants, medication and biosecurity has gained more popularity among the traders since the upgrading. The pilot LBM has introduced the use of plastic cages, which are more conducive and hygienic for transporting live birds. This has greatly minimized losses of birds while in transit. Other non-quantified benefits include better institutional arrangements for sales, inputs and services supply, community-driven market organization and waste disposal arrangements. Findings The estimation of the NPV and IRR is based on the incremental net benefit stream, which requires that the profit level prior to the intervention (without and before the investment) be netted out on an annual basis from the benefit stream with the intervention. The steady state without the intervention (traditional LBM) is presented as project year zero, which explains the zero net benefit for that year while the net benefit for other years may be negative or positive. The net benefit is negative in year 1 because of the high magnitude of investment (negative net benefit).The NPV is positive if and only if the returns to the intervention are greater than the situation without intervention (traditional LBM). The estimated internal rate of return (IRR) represents the rate of return on the capital invested in the project for the LBM. It is estimated at 34%. The estimated net present value (NPV) for the improved LBM at an opportunity cost of capital of 12% was USD 241,417 and the estimated annual incremental benefit was USD 73,876. A positive NPV makes a project attractive for implementation and the larger the NPV the better. Conventionally, a project is qualified for implementation if the yield rate is equal or greater than the prevailing lending rate or opportunity cost of capital (considered 12% in this case). A sensitivity analysis was conducted and it indicates that the estimated financial ratios are stable under a limited range of scenarios in sales revenue and costs. If revenue falls by 10 percent, the IRR decreases to 18% and the NPV falls by 79%. The switching value for revenue is a fall in revenue of 13%. If total cost increases by 10%, the IRR will decrease to 20% and the NPV will fall by 69%. The switching value for total costs is an increase in costs by 12%. The details are in table 2. Conclusion The piloting of upgraded LBM was adopted by NAICP as a longer-term strategy for improving biosecurity in the poultry sector to reduce future risk of HPAI and other emerging diseases. A financial analysis of this institutional intervention was conducted and the results indicated that except in the first year when the initial heavy investment was made, the revenue from the LBM was consistently higher than the total cost generating a positive net incremental benefit. That combined with an IRR greater than the opportunity cost of capital indicate that the improved LBM is a profitable venture. The analysis suggests that it will remain profitable and therefore sustainable as a business for the market agents who participate in it. The nature of the institutional arrangements as community-driven with by-laws and an elected management body offer additional reasons for believing it will be sustained. 255 Table 1: The pilot LBM cash flow analysis results (in USD 1,000) Project Year Revenue Investment Operating Cost Total Cost Cash Flow Net Benefits 0 312 0 278 278 34 0 1 5 110 5 116 -113 -147 2 509 0 465 465 44 11 3 662 0 604 604 58 24 4 to 19 662 0 604 604 58 24 20 662 0 604 604 58 24 Total 12,747 110 11,628 11,739 1,005 293 Note: The presented figures are rounded Table 2. Sensitivity analysis results (IRR in %, NPV and AINB in Naira 1,000, change in %) Base Value Sales fall Sales switching Value fall by Total Costs Increases by Costs Switching Value increases by -10% Change -13% Change 10% Change 14.5% change IRR 34 18 -46 12 -64 20 -41 12 -65 NPV 36,212 7,462 -79 274 -99 11,083 -69 -224 -101 AINB 11,081 5,077 -54 3,576 -68 6,185 -44 3,982 -64