Terms of Reference for Final Project Evaluation: SHAPE Project
1. General Background
Concern Worldwide has been operational in Sierra Leone since 1996. Since 2002 the emphasis has moved away from emergency interventions to rehabilitation and now long term development, transitioning from a short term project approach to a longer term, more sustainable, programmatic approach. The main programme focus is on health (maternal and child, and WASH), education and food, income and markets (FIM), while (gender) equality, social protection, DRR and HIV and AIDS are mainstreamed. In line with Concern’s mandate to respond to emergencies, a large-scale response to the Ebola public health emergency commenced in June 2014 and is ongoing to date.
Concern has been operational in Tonkolili District since 2002. Health interventions in the district seek to support primary health care system strengthening through implementation of the Sustainable Health Actions through People’s Empowerment (SHAPE) project. Funded by the EU and Irish Aid, SHAPE is part of the broader Tonkolili Integrated Poverty Reduction Programme, which aims to take a multi-sectoral holistic approach to improving the lives of the most poor and vulnerable households and communities in the district.
The overall objective of the SHAPE project is to ‘improve the health outcomes of Tonkolili District in line with the Poverty Reduction Strategic Plan-II for Sierra Leone and as measured by MDGs 4 and 5’. The specific objective is ‘*to improve access to and coverage of health services through strengthening referral systems, empowering communities to monitor quality of services and capacity development of local authorities in Tonkolili District,** with a special focus on six Chiefdoms: Kholifa Rowalla, Kunike Barina, Kunike, Kholifa Mabang, Malal Mara and Yoni’.*
The SHAPE project timeframe is from January 2011 to 16th April 2015 (includes a no-cost extension period approved by the donor). The project is being implemented by Concern in partnership with the Tonkolili District Council (TDC) and the District Health Management Team (DHMT).
This project has four expected results to be achieved through implementation of a number of key activities:
Result 1:Enhanced planning, coordination and management capacity of local authorities in provision and distribution of health services for the entire population of Tonkolili District
Result 2:Improved service delivery and community participation in community health actions through strengthened community planning and monitoring of health services of 60 health facilities
Result 3: Improved access to and utilisation of maternal and child health services through expanded BEmONC services at four targeted health facilities and enhanced referral system across entire district
Result 4:Strengthened advocacy, transparency, accountability, quality of programme and effectiveness of aid through alignment and harmonization with EU and non-EU supported development programmes and mainstreaming child rights, gender based violence, HIV and AIDS.
The worst Ebola outbreak in the country occurred during the last phase of the project which necessitated a no-cost extension approved by the donor. The Ebola crisis affected the project implementation as well as the targeted population’s health seeking behaviours. As at 11th February 2015 confirmed cumulative cases for Tonkolili were 451 while confirmed deaths were 157. The SHAPE project responded to the Ebola crisis by conducting social mobilization activities to create awareness in target communities on transmission and prevention of Ebola. This included sensitising mothers to seek health care for non-Ebola related illnesses (especially for expectant mothers and under 5 children) despite fears of contracting Ebola at clinics, and support to infection prevention and control to reduce Ebola infections among health workers.
2. Objectives of the final evaluation
- To critically review the achievement of the project in terms of attaining its objectives using standard evaluation criteria: relevance, effectiveness, efficiency, impact and sustainability.
- To reflect on the successes, challenges, best practices and key lessons learned during the project period.
- Suggest key practical targeted recommendations and learning to be considered by Concern Worldwide for future projects and programmes.
3. Methodology
The final evaluation will use both qualitative and quantitative approaches to address the key evaluation questions.
Quantitative Knowledge, Attitude and Practice (KAP) survey and rapid health facility assessment (RHFA) data collection will be carried out by Concern Worldwide. The data sets from the KAP survey and the RHFA will be given to the consultant in an excel data set for review and analysis.
In addition, the consultant will undertake a capacity assessment with Tonkolili District Council (TDC) and District Health Management Team (DHMT) including the health management committee (HMC) and Ward Development Council (WDC). This will include a review of the existing coordination mechanisms between the TDC and the DHMT and PHUs and a review of technical capacity of M&E staff. The consultant will also be required to collect qualitative data from community members (men and women groups) living in target areas to explore their views and perceptions of existing health services, current care-seeking behaviours for antenatal care and delivery for pregnant women and malaria, pneumonia and diarrhoea in children under five and factors influencing these care-seeking behaviours. The capacity assessment with DHMT and TDC and qualitative data collection will be the responsibility of the consultant.
The main purpose of this consultancy therefore is to provide appropriate and disaggregated end-line indicator values that will enable Concern to measure progress and performance by comparing the end-line values against baseline values for each of the project objectives and results. The consultant will use this information to evaluate the project against the specific criteria outlined above and present the findings in a final report.
3.1 Key Outputs Expected from the Consultancy
- Inception report consisting of interpretation of the TOR, proposed methodology, and consolidated workplan. The workplan should show realistic timelines, roles and responsibilities and cost estimates of the evaluation
- Reviewed data collection tools and methodology and qualitative analysis
- Review of quantitative data collection tools and processes and analysis of data with key findings
- Presentation of first draft of the report to ACDP, National Health Coordinator, Area coordinator and relevant program staff and project partners before departure
- A final report incorporating comments on the draft report and presenting final key findings and recommendations, within two weeks of receiving field feedback
4. Summary of processes to be followed by Consultant and estimated time frame
Pre-fieldwork / Pre-travel
Week 1:
· Review all of project documents (Proposal, log frame, baseline and midterm review reports, M&E database in excel format, monitoring feedback reports).
· Review and input into qualitative data collection tools and methods.
Fieldwork / In-country
Week 2: Conduct in-depth interviews with DHMT, TDC, HMCs and women’s and men’s focus groups
Week 3: Carry out data analysis (including appropriate statistical tests to compare baseline, midline and endline data on project indicators)
Week 4: Prepare a draft report which includes detailed conclusions and recommendations as per the objectives of the Evaluation and is comprised of the following elements:
- A stand alone Executive Summary
- Introduction
- Methodology
- Answered questions/ Findings:
o Problems and needs (Relevance)
o Achievement of purpose (Effectiveness)
o Sound management and value for money (Efficiency)
o Achievement of wider effects (Impact)
o Likely continuation of achieved results (Sustainability)
5.Overall assessment
- Conclusions and Targeted Recommendations
- Appendices of (a) terms of reference, (b) summary of activities, meetings, and discussions, (c) summary results of any field work, (d) brief description of any workshops/training provided, (e) qualitative data collection tools used and (f) bibliography.
· Present findings to Concern and partner staff in a briefing session.
· Consultant receives feedback on the first draft from Concern and partner staff
Post field / Post country
Week 5: Prepare the final evaluation report, not exceeding 40 pages excluding annexes incorporating the feedback both from the document review and briefing session and submit final report to Concern.
5. Lines of communication
The Consultant will primarily communicate with the Programmes Director, National Health Coordinator, the Project Manager and Programme Quality and Monitoring Coordinator.
6. Duration
The overall consultancy work as outlined above is expected to take between 4-5 weeks in the months of April-May 2015 and final report presented to Concern no later than 31st May 2015.
7: Composition, skills and experience of the review/evaluation team.
Education:The consultant or consultancy company should have post-graduate degree in Development Studies or Public Health.
Experience:Practical and relevant experience in conducing evaluations of multidisciplinary indicators and particularly in health (including KAP surveys and Health Facility Assessments) with particular emphasis on data analysis, and writing up of evaluation reports.
FOR FULL CONSULTANCY DETAILS PLEASE SEE ATTACHED ToR DOCUMENT
HOW TO APPLY:
Those interested in the consultancy must include in their application a detailed technical and financial proposal with the following components:
· Technical
o Understanding and interpretation of the TOR
o Methodology to be used in undertaking the assignment
o Work plan for the assignment
· Financial Proposal
o Consultant’s daily rate in either Euro or Leones
o Other costs e.g. accommodation, travel, and printing.
· Organizational and Personnel Capacity Statement
o Relevant experience related to the assignment
o Contacts of organizations previously worked for
o Curriculum vitae of key personnel
o Any publications
o A sample of previous work
Full consultancy proposal documents should be submitted to andrea.solomon@concern.net by the date of Friday 3rd April 2015.
