Consultancy for Final Evaluation of Health Programme, Nkhotakota, Malawi

Terms of Reference: Final Evaluation of the Improved Utilisation and Opportunities for Reproductive Health for Women and Youth Project

Nkhotakota District, Malawi

Background / Context

Malawi has one of the highest maternal mortality rates in the world, where one out of every 36 women dies due to pregnancy-related complications. Despite having made significant progress over the last 25 years, the 2015 maternal mortality ratio is 510 / 100,000 live births; significantly higher than the Millennium Development Goal target of 280. While evidence-based, reproductive health interventions to address maternal mortality are being implemented in Malawi, the quality, availability, and accessibility of these services – especially for poor and vulnerable populations – remains a challenge. Adolescent pregnancy is of particular concern in Malawi. By the age of 18, 50% of girls have married, and one quarter has had their first child. Adolescents are estimated to account 20% of maternal mortality in Malawi.

Introduction

With funding from the Scottish Government, Merck for Mothers, and the Elizabeth Taylor AIDS Foundation, Concern Worldwide is implementing a maternal and reproductive health project in Nkhotakota district, Malawi from November 2012 – March 2016. The purpose of the project is to reduce maternal morbidity and mortality through the following strategic objectives:

1. Improve utilization of high quality maternal and reproductive health services through increased availability and accessibility of services
2. Improve utilization of high quality community-based family planning services, including youth-friendly reproductive health, through increased availability and accessibility of services
3. Improve capacity of government and traditional leadership structures to plan, manage, support, and monitor key maternal and reproductive health activities

A total of 33,515 women of reproductive age are targeted to benefit from a full package of facility and community-based interventions in three of seven Traditional Areas (TAs) in the district: Malengachanzi, Mwadzama, and Mwansambo. An additional 34,378 women will benefit from improved health services throughout all seven TAs in district.

As detailed in the attached logical framework (Annex 1), the project has implemented a range of activities at the district, facility, and community levels; both through direct implementation and partnership. The principal partner is the Ministry of Health, specifically the Nkhotakota District Health Office. Other partners include VillageReach, Grassroot Soccer, Interchurch Organization for Development Cooperation (ICCO), and Banja La Mtsogolo (local affiliate of Marie Stopes International).

Purpose

As the project nears its completion, Concern seeks to hire an independent consultant to conduct an external evaluation of the project. The final evaluation will take stock of project achievements against targets and identify key successes and challenges associated with the implementation of the project from the perspectives of stakeholders at all levels including beneficiaries, volunteers, community leaders, health workers, district officials, project staff and partners.

The final evaluation report will be used as a key learning document to inform the design of Concern’s future maternal, reproductive, and adolescent health initiatives in Malawi and beyond.

Evaluation Questions

The overall evaluation questions are organized by the DAC criteria, as follows:

DAC Criteria

Evaluation Question

Effectiveness

1. To what extent were the three strategic objectives achieved?
2. Was the programme logic well thought through and how did the project interventions contribute to the achievement of the three strategic objectives?
3. What were the key successes and challenges that lead to the achievement or non-achievement of the objectives?
4. What steps were taken to address issues of inequality and ensure the interests of the most marginalised were taken on board during programme planning, implementation and monitoring? How effective was this?

Efficiency

1. How did the implementation of key activities through partners affect project implementation and achievement of project objectives?
2. How did collaboration with partners and other district-level stakeholders contribute to opportunities for further scale-up of project approaches within a broader context?
3. Were financial resources well used? Could things have been done differently and how?
4. Was the programme M&E system fit for purpose? What evidence is there of effective Results Based Management in the programme?

Relevance

1. Were the project activities designed to the unique needs of target population based on appropriate contextual analysis or formative research?
2. To what extent were the strategic objectives and associated activities relevant to and in support of district and national-level Ministry of Health strategies and priorities?

Sustainability

1. Is there evidence that project outputs and outcomes will contribute to lasting benefits beyond the life of the project?
2. What are the key factors which may positively or negatively influence the likelihood of the project’s overall sustainability?

Impact

1. Overall, what real difference has the project made to its intended beneficiaries (women of reproductive age)?
2. What has changed as a direct result of the project?

While the evaluation will consider the overall project implementation and achievements, emphasis will be placed on assessing key elements of the project for which Concern has particular interest. These include the adolescent health interventions under Objective 2, as well as the social and behaviour change (SBC) activities implemented under Objective 1 (see log frame) and the overall impact of those specific interventions on the target population.

Conversely, several project approaches have already been evaluated. The Health Institution Capacity Assessment Process (HICAP) tool – a key methodology under Objective 3 - was evaluated in August, 2015. The hotline service (Chipatala Cha Pa Foni) being scaled-up under this project was extensively evaluated in 2013. The consultant is therefore expected to review these documents and incorporate the relevant findings of these previous evaluations into the overall report.

Methodology

The consultant will lead a participatory evaluation. This means that project staff, district stakeholders, and community representatives will be involved in the implementation of final evaluation activities and make meaningful contributions to the analysis of findings. The evaluation will utilize a mixed-methods approach incorporating both quantitative and qualitative data, as well as the review of secondary data sources.

The consultant will furnish a written methodology for the final evaluation (including the proposed number of key informant interviews, focus groups discussions, observations, and locations) to Concern for comment and planning purposes prior to the evaluation.

Secondary Data: The consultant will review project documents (proposals, donor reports, mid-term review) to assess progress of project activities in relation to the project design and targets set. The consultant will also review key tools and curricula designed and utilized by the project, including SKILLZ Malawi and SKILLZ Girl curricula, the HICAP tool and HICAP evaluation, and key social and behaviour change tools developed.

Quantitative Data: All quantitative data will be collected and analysed before the final evaluation begins. The consultant will review the project’s completed results framework, which will provide baseline and endline data against targets for the key project indicators[1]. This should form the basis from which the effectiveness questions are answered.

Qualitative Data: As part of the final evaluation field work, in-depth interviews or focus group discussions will be conducted with project staff and partners, district-level stakeholders, health facility staff, community volunteers, and project beneficiaries.

Outputs

• Lead an in-country debriefing meeting with Concern project staff, senior management, and other stakeholders prior to departure from Malawi
• Evaluation report of no more than 25 pages to be submitted at the end of the assignment. Report should summarize findings against each of the evaluation questions and provide specific recommendations for future reproductive and maternal health programmes
• Stand-alone 2-3 page executive summary of evaluation report
  Timeframe

The consultancy must take place between February and March 2016. All deliverables must be received and invoices must be paid no later than 31st March 2016.

Lines of Communication

Briefing and de-briefing with the Country Director and Programmes Director; daily coordination with the Health and Nutrition Coordinator.

Budget

The total budget for the evaluation is in the range of 10,000-12,000 EURO as a maximum. This includes the consultant’s daily rate, international travel, visa / immunization expenses, lodging, and per diem based on Concern’s standard policy.

Person Specifications

Essential:

• Expertise in community-based reproductive and maternal health in sub-Saharan Africa
• Demonstrated experience conducting mixed-method evaluations of community-based health programmes
• Strong analytical abilities; capable of synthesizing and making sense of information across a range of sources
• Ability to lead and facilitate learning among an evaluation team consisting of project staff and district stakeholders
• Excellent writing skills in English

Desirable:

• Expertise in adolescent health and youth programming
• Expertise in social and behaviour change
• Familiarity with the reproductive, maternal, and adolescent health context in Malawi

[1] Baseline data is mainly from the 2010 DHS (district level DHS data was used where possible). A midterm survey was conducted in May 2015 to provide updated, project-specific data against key indicators as well.

HOW TO APPLY:

Please submit an Expression of Interest by January 8th 2015 to Emily.bradley@concern.net outlining exact availability in line with the approximate timeline. The expression of interest should contain: (a) a technical offer and (b) a financial offer, comprising:

1. Technical offer:
2. Up to date CV of the consultant
3. Technical proposition detailing proposed methodology and resources needed (max 3 pages).
4. An example of a report from similar work which demonstrates evidence of the skills and experience required.
5. Financial offer:
6. A list of all expenses expected to be incurred by the consultant including a daily rate. (Please note that any per diems for time in the field will be paid in line with Concern’s policies and that accommodation and transport arrangements will be managed by Concern).