MID TERM EVALUATION CONSULTANT

Terms of Reference for Evaluation

Evaluation Summary

Program/Project: Building Health Systems: Promoting Eye Health for Children in East Africa

Partner Organization Sightsavers, FHF and COECSA

Project start and end dates January 2013 – December 2016

Phase of project Phase 1

Evaluation Purpose To assess the extent to which the project objectives, outputs and activities have been met over the past two years

Evaluation Type Mid-term

Commissioning organization CBM

Contact person Sam Ogolla

Primary Methodology Participatory methodology that must include project site visits

Proposed Evaluation Start and End Dates May 29 to June 29, 2015

Anticipated Evaluation Report Release Date August 1, 2015

Recipient of Final Evaluation Report The SiB/ CBM Consortium

Evaluation Objective, Scope and Intended Use

Objective

The main purpose of this internal evaluation is to assess the extent to which the project objectives, outputs and activities have been met over the past two years; January 2013-December 2015.The Project is now in its third year hence it is imperative to evaluate the progress made, successes achieved and identify any challenges and lessons to inform the remaining period of the project.

The evaluation will therefore seek to answer to the following questions:

• What progress has been made towards achieving the Joint Consortia objectives?
• How has the project contributed to the following so far?
• The strengthening of referral systems
• How beneficial were the up skilling courses conducted for the eye health workers? Is there evidence of improved quality of clinical care?
• Are the eye units sufficiently resourced to deliver the necessary services?
• Clinical and preventative service provision support, is it adequate?
• How has the delivery on the “Joint Commitments” worked? Has collaboration been to the level envisaged at the planning phase? What has been the level of its success? Is there anything else that needs to be done to improve this collaboration?
• What progress has been made by each consortium in delivering its planned outputs? Is each consortium likely to meet its targets? What challenges are being encountered? What needs to be done differently for each consortium to reach its targets, if applicable?
• One of the key issues identified is the poor uptake of referrals. Only 15% of the children referred from the schools have reported to the health facilities. What are the barriers-why is 85% of the children not reporting to the health facilities? How can the barriers be addressed? Similarly how can uptake of referral from one level health facility to the next be improved?
• Local ownership; to what extent (breadth and depth) is local ownership evident? Assess the partnership levels and relationship between the project management (consortia) and the implementing partners, mainly the Ministries of Health and Education. How is consultation and decision making process done? Any challenges and how they can be addressed?
• For purposes of future similar projects, are there any other stakeholders that were left out that might make meaningful contribution to the success of this project?
• Are there any potential challenges and risks that may affect the success of the project during the remainder of its lifetime? How can they be mitigated?
• Assess the sustainability potential of the project in the following two areas;

i. Ability of the relevant Ministries and health facilities to continue offering the services with the same level of quality after Dec 2016

ii. Potential for replication or scaling up the CEH model by either the Ministries or any other stakeholders

Scope of the evaluation

The physical scope of this work is East Africa, where the project is being implemented. In Kenya, this will cover all the eight regions. In Tanzania, it will cover 12 health regions while in Uganda it will cover 10 health regions. Details on the exact locations in each country will be found in the project document.

The scope of content will cover the performance against key parameters including the project’s relevance, effectiveness, efficiency, sustainability, timelines of activity implementation, and its strengths and weaknesses, best practices and lessons learnt so far, and recommendations.

Evaluation Criteria

In order to generate the information needed to achieve the goal, the consultant(s) will be guided by 5 criteria as explained below. The consultant(s) will develop specific questions that answer the following broad questions under each of the criteria

• Relevance; To what extent is the objectives and design of the project fitting with the current global/ regional/national policies and laws including challenges and concerns, the needs, policies and priorities of intended beneficiary, the specific objectives, role and comparative advantages of the organization?
• Efficiency To what extent did the project convert its resources and inputs (such as funds, expertise and time) economically into results in order to achieve the maximum possible outputs, outcomes, and impacts with the minimum possible inputs?
• Effectiveness; To what extent did the project achieve its outputs and outcomes? Importantly, to what extent are unplanned outputs and outcomes been achieved? And what factors influenced achievement or non‐achievement of the planned and unplanned outputs and outcomes?
• Impact;To what extent did the project have positive or negative, intended or unintended, primary or secondary effects at the macro (sector) or micro (household or individual) level? These may include: Positive spill‐over’s, e.g. adoption of any good practices, concepts, models, approaches, policies or methods by those outside the immediate SiB partners, institutional changes in partners, both positive and negative, effects on health, effects on livelihoods, effects on gender relations or other social relations. Are any external factors likely to jeopardize the project’s direct impact?
• Sustainability and replication; To what extent is there continuation of some or all benefits from project after completion? These may be political, financial, institutional, economic, social and/or environmental. To what extent is the project or aspects of the project scalable or replicable? Specifically, what would be the probable implications of scaling the action [up or down (if relevant)] in terms of costs, cost‐effectiveness, or efficiency?
• Gender; The evaluation must consider gender aspects of the project and report on this. Data collected must be disaggregated according to gender.
• Child Safeguarding; First and foremost, please keep in mind to adhere to child safeguarding ethics during the entire evaluation process if children are involved. The evaluation will also review how child safeguarding has been adhered to by the project and include the findings in the report. Methodology

The evaluation will be carried out in conformity with the ethical principles, standards and practices of any evaluation. Given the geographic area to be covered, the evaluation work will be conducted by a team of two consultants. The consult we are interested in shall take lead of this assignment, coordinate all the evaluation activities, and be responsible for the deliverables of this assignment. The lead consultant shall evaluate the project activities in Uganda and Kenya, and work closely with the co-consultant, already identified, who will evaluate the project activities in Tanzania.

The consultant(s) shall jointly identify comprehensive participatory methodology for undertaking this evaluation. The methodology must include among others literature reviews, interviewing field/project site visits among others. The consultants will jointly prepare all the evaluation tools that will operationalize and direct this evaluation. They will define an appropriate sample size in a way that will avoid selection bias, and ensure that evaluation meets the principles of participation involving both male and female beneficiaries. The evaluators are expected to conduct a participatory evaluation providing for meaningful involvement by project partners, beneficiaries and other interested parties in the data collection and analysis, in the formulation of recommendations, and sharing of feedback. The field visits and contact sessions with beneficiaries must attempt to reach/meet 25% of the target. This project is in partnership with the several Ministries and other stakeholders therefore stakeholder participation is fundamental to this evaluation.

Attachments

1. BHVI and CBM Project proposals, log frame and work plans
2. BHVI and CBM bi-annual reports to SIB
3. Minutes of all joint consortia meetings
4. SiB Letter of variation, reflecting the revised target outputs

Expected results and deliverables

Evaluation Reports

The evaluators will submit a draft evaluation report for review by the Consortia leads (BHVI and CBM) as follows:

Inception Report: 14 days from signing the contract

Draft Report: 7 days after completing the field work

Finalized Report: 7 days after receiving feedback on the draft report

Reporting Format

All the evaluation reports will be presented in the following format:

• Abstract/executive summary
• Evaluation methodology
• Informants/stakeholders of the evaluation
• Findings
• Recommendations
• Attachments of the evaluation tools

Evaluation Team and Management Responsibilities

Commissioning responsibility

This evaluation has been commissioned jointly by secretariat of the SiB Consortium, composed of the CBM Consortium and the BHVI Consortium. The two secretariat will hold the overall responsibility for the evaluation exercise, will coordinate the activities, and approve the deliverables of the assignment. Management of the evaluation and logistics

Explain who is responsible for getting what done - explain different offices’ (or people’s) scope of responsibility (e.g. Member Association, Regional Office, Country Office). Who is responsible for organizing transport, accommodation, booking interviews and organizing venues, flights? Who is responsible for ensuring disability inclusive venues/ translators etc.?ADMINISTRATIVE / LOGISTICAL SUPPORT

Budget

The consultant(s) shall submit to SiB a comprehensive technical and financial proposal detailing how they intend to undertake this assignment, and indicating their daily rates for the assignment. SiB will negotiate with the wining consultant(s) the final fees in line with the budget available for this evaluation. The daily fees proposed should exclude cost of economy class flights (where applicable), in-country accommodation (bed, breakfast, evening meals), and local transport, which will be covered by the implementing partners in each country.

Logistics

SiB will cover the following costs:

• Economy class airfares and VISAs. (where applicable)

• In-country transportation

• Hotel accommodation (bed, breakfast and evening meals)

• Meeting venue hire and associated equipment

• Provide the required project documents

The consultant(s) is expected to provide or cover all other costs and materials not mentioned above related to this exercise (e.g. laptops)

Payment Schedule and Mode

The following payment schedule will be followed

• On signing the contract: 20%

• Submission of draft report: 30%

• On acceptance of final report: 50%

• Payment shall be made upon the submission of each deliverable as shall be agreed upon, accompanied by a signed invoice from the consultant.Required Qualifications

Given the expansive geographic coverage, interested consultants or consultancy firms are expected to propose a team leader, and assemble two teams to undertake this work concurrently. They shall have a demonstrated competence in having undertaken similar work before, including experience in program design and management, planning, monitoring and evaluation. The consultant(s) must demonstrate knowledge and skills in the following areas:

Essential qualifications include:

• Experience with programming and evaluations in public health;

• Strong analytical, writing and presentation skills;

• Experience in working and/or evaluating regional cross-border projects.

• Knowledge of the regional dynamics in the health sector in East Africa.

Desirable qualifications include:

• A Masters/Postgraduate degree in Public Health, Development or other relevant Social Sciences

• At least 5 years of experience working with Ministries of Health in East Africa or elsewhere in Africa;

• Working knowledge of the eye care sector in East Africa;

• Understanding of the health systems approach in health sector development;

HOW TO APPLY:

APPLICATION PROCEDURES

Interested applicants should submit a technical and financial proposal indicating their daily rates for the assignment. The expression of interests will be assessed according to standardized quality assessment criteria, as well as on the basis of their competitiveness and value for money in line with the budget available for this evaluation.

Application documents should be sent to reach CBM Human Resources not later than the 22nd of May 2015 to: recruit@cbmi-nbo.org

Given the time constraints for this assignment, applications will be reviewed on an on-going basis and a contract shall be awarded as soon as a qualified applicant has been identified.