****TERMS OF REFERENCE****
****Baseline, Midterm, and Final Evaluation of the Beoog Biiga II Program****
****Purpose****
The purpose of this Terms of Reference is to describe the tasks and responsibilities of a consultant to conduct the baseline, midterm and final evaluations of the Global Food for Education Program (titled “Beoog Biiga II”), implemented by Catholic Relief Services (CRS) in Burkina Faso.
This ToR has been shortened for advertising, and a full version is available upon request of interested applicants.
****Background****
Through its program in Burkina Faso, Catholic Relief Services (CRS) will implement a four-year United States Department of Agriculture (USDA) McGovern-Dole International Food for Education and Child Nutrition Program (MGD) award, Beoog Biiga II. In this project, CRS aims to improve the literacy and health and dietary practices of school age children in Burkina Faso. Beoog Biiga II will intervene in all primary schools and community-led preschools (bissongos) in the provinces of Bam and Sanmatenga in the Central North region of Burkina Faso.
****Beoog Biiga II Evaluation Plan****
In compliance with the terms of CRS’ agreement with USDA, CRS has developed an evaluation plan that details the purpose, scope, methods and approximate timeline for all of Beoog Biiga II’s evaluation activities. The evaluation plan a) explains how the project will be evaluated including the gathering of baseline data and special studies that will be conducted; b) gives information on the project’s midterm and final evaluations; and c) describes how the evaluation activities will be managed. It also includes the project’s theory of change, evaluation methods and the timing of their activities, plans for the sharing/dissemination of evaluation findings, the project’s target beneficiaries, the project results indicators and their targets.
The evaluation plan will guide all actions taken by CRS, and external consultants in regards to the project’s baseline assessment, midterm and final evaluations. The evaluation plan is currently being finalized. The following ToR is based on the current evaluation plan, which is subject to change. The consultant will be provided with the final version of the evaluation plan once approved by USDA, and is expected to follow all elements such as the methodology, sampling design, key questions, etc as stated therein. Changes to the evaluation plan may be made with written approval of USDA.
Evaluation Design
The evaluation design will be a hybrid performance evaluation. To assess nearly all the expected results, a longitudinal design will be followed. To investigate in greater detail the effectiveness of the mentoring program however, a rigorous evaluation component will accompany the performance evaluation design. The outcomes of interest are literacy achievements and attendance rates in of students in schools with mentoring programs compared to students in schools without.
The mentoring program will only take place in 200 out of the 800 schools. Among these 200, 50 schools are already using the mentoring program and will therefore be excluded from the sample of treatment schools. This will allow a randomized assignment of the mentoring program to 150 schools.
The mentoring program will be phased in over the first 3 years of the project, 50 schools each year. To simplify the measurement, only the first group of 50 schools will be followed from baseline to final, thereby eliminating the need for multiple baseline data collection in years 2 and 3.
****Beoog Biiga II Project Description****
Beoog Biiga II will focus on improving student learning outcomes. CRS will work with government and non-governmental partners to improve literacy instruction and student achievement through teacher and administrator trainings, increasing access to reading opportunities outside of regular class time, improving teacher and school accountability, as well as improving school supplies and materials. Community libraries and additional literacy support to students through mentoring, reading groups and summer reading camps will motivate and give students the skills to achieve higher literacy.
Beoog Biiga II will reach approximately 409,041 direct beneficiaries over the life of the project. CRS will work in all primary and preschools of Bam and Sanmatenga Provinces. The distribution of beneficiaries is as follows:
- 291,983 primary and preschool students (between 170,000 – 200,000/year)
- 3,629 teachers
- 90 preschool caregivers
- 9,933 PTA and community management committee (COGES), and cooks
- 51,120 SILC group members
- 2,250 community mentors
- 20,000 community members (using libraries)
- 36 local government officials
- 30,000 parents
- 81 school administrators (district level inspectors and pedagogic advisors)
The project will carry out the following activities: Provide School Meals to all students, and take home rations to girls; distribute deworming, Vitamins and Minerals to students; train teachers, PTA members and cooks in good health and nutrition practices; distribute school supplies and materials; train teachers and school administrators in literacy instruction; build the capacity of PTAs and local government to support schools; provide mentoring for girl students; establish community libraries and reading groups; establish a standardized reading assessment tool; form Savings and Lending Groups; raise awareness of the importance of education.
****Baseline Assessment****
The baseline assessment is the first, major component of the project’s evaluation plan. The purpose is to establish indicator baseline information and targets for which the project will regularly measure performance. To do so, the project will conduct a repeat (or follow-up baseline) as part of the midterm and final evaluations. The baseline survey will be conducted within the first six months of the project, and before the project activities begin. The baseline survey will generate data against which to measure changes based on the inputs, outputs, results and outcomes of the project. The data collection techniques will include questionnaires, direct observations, student reading assessments, key informant interviews, and focus group interviews.
Purpose and Scope
The purpose of the baseline survey is a) to establish the baseline value of USDA’s SO1: Improved literacy of school-age children and SO2: Increased use of health and dietary practices and b) to establish baseline values of the main outcome/output indicators for both SOs.
For SO1, the survey will provide baseline values for the project’s three intermediate results:
- Current reading ability of students
- Current degree/level of student attentiveness
- Current level of student attendance
For SO2, the survey will provide the baseline values for the project’s five intermediate results:
- Current knowledge of health and hygiene practices
- Current knowledge of safe food prep and storage practices
- Current knowledge of nutrition
- Current access to preventative health interventions
- Current access to requisite food prep and storage tools and equipment
****Methodology****
The beneficiary-focused baseline survey is the first component of a longitudinal performance evaluation methodology, which will include quantitative and qualitative methods. All project indicators will be measured through the performance evaluation methodology, except for the girls mentoring component that will be assessed using a rigorous evaluation method.
All data must be disaggregated and analyzed by respondent group (parent, student, etc.) including male and female. The surveys will be complimented by a participatory, beneficiary-based qualitative exercise to get a more detailed picture of constraints to education (specifically literacy), opportunities, etc. The qualitative exercise may also try to determine which of the project activities contributed more or less to the student literacy outcomes.
****Sample Methods and Size****
The sample methods will be made available to the selected consultant.
****Sample size:****
Target GroupPopulation Sample Size Students 291,983 782 Girl Mentees 16,875 493 Teachers 3,629 34 Administrators 81 13 PTA and community management committee (COGES) presidents 7,726 34 Food preparers 2,207 34
****Schedule of Baseline Survey Activities:****January 26th, 2015 through March 20th, 2015.
****Midterm Evaluation****
The consultant will conduct a formative process-focused, participatory mid-term evaluation, using USDA guidelines. This mid-term performance evaluation will assess the progress of the program’s implementation of project activities and their results towards intended outcomes. It will include a repeat, or follow-up baseline. It will also continue to rigorously evaluate the mentoring program.
The findings will enable the project team to assess progress in implementation; assess relevance of interventions; and adjust strategies, management structures and/or activities for improved project effectiveness, efficiency, impact and sustainability. The midterm evaluation will also document lessons learned, and recommend changes to activities or implementation strategies as needed.
Purpose and Scope
The purpose of the midterm evaluation is to critically assess whether the project is on track to meeting its objectives. This will be done by examining the extent to which the project activities are being implemented as planned, the current levels of key indicators (ie. student’s reading ability), as well as documenting lessons learned. The evaluation will be both retrospective and prospective, assessing at the mid-point the following principles developed by the Organization for Economic Cooperation and Development (OECD)[1]: 1) the relevance of all interventions, 2) the effectiveness of implementation strategies and activities, 3) the efficiency of the project and, 4) the impact, 5) the likely sustainability of initiated actions.
Methodology
The midterm performance evaluation will compile and examine quantitative data on progress towards project indicators using data collected through the established project monitoring system. As with the baseline, the midterm will continue a rigorous evaluation of the girls’ mentoring activity.
****Schedule of Midterm Evaluation Activities:****March- May, 2016.
****Final Evaluation****
The consultant will conduct a comprehensive, final evaluation in the last year of the project to measure change and overall success of the project by comparing indicator values end data against targets and baseline values. The final evaluation will also seek to provide lessons learned and recommendations for USDA, program participants, CRS, and other key stakeholders for future programs.
Purpose and Scope
The purpose of the final evaluation is to assess the extent to which the project achieved expected results, as well as appropriateness of project design elements, implementation, management structure, and replicability. It will assess constraints, lessons learned and good practices, opportunities as well as successes in implementation, and determine the relevance and effectiveness of the strategies and approach utilized in the implementation of the program.
Methodology
The final evaluation methodology will follow that of the baseline and midterm. More information is available upon request.
Schedule of Final Evaluation Activities:March-May, 2018.
****Work to be Accomplished****
The completion of the baseline, midterm and final evaluations will be predicated on achievement of the following activities:
- Study Beoog Biiga II program objectives, Evaluation Plan, Performance Monitoring Plan, activities, and USDA M&E policy
- Develop data collection tools in consultation with the CRS Burkina country program team and Regional MEAL advisor
- Train data collectors on how to administer the data collection tools, field test the tools with the data collectors and make changes as needed
- Oversee implementation of data collection in schools (daily quality control of questionnaires, etc.) and collect all other data needed (e.g. conduct key informant interviews/focus group discussions)
- Analyze data collected and identify questions raised by data that might need further study
- Prepare draft and final report for review by CRS staff, and address comments
- Conduct a feedback workshop with project stakeholders
****Deliverables****
The key deliverables for the Beoog Biiga baseline, midterm and final evaluations are:
- Submit the evaluation workplans, data collection tools, and finalize all tools with CRS Burkina country program and Regional MEAL advisor
- Submit draft and final evaluation reports with data collection tools, raw data files, and data quality report as annexes
- Conduct feedback workshop with project stakeholders (logistics organized by CRS) highlighting key lessons learned
****Profile of Consultant/Firm****
The consultant will have the following qualifications:
- Demonstrated knowledge and experience conducting performance and impact evaluations of development projects, especially involving education and nutrition
- Uses acceptable analytical frameworks and proposes strong methodology for addressing gender dimensions
- Demonstrated expertise in evaluation design, statistical analysis and sampling, investigation skills, development of data collection tools, questionnaire design, data collection, conducting interviews, data analysis, and report writing.
- Knowledge of education, health and commodities sectors (experience evaluating USDA funded projects ideal)
- Work experience in Burkina Faso or West African countries preferred
- Good verbal and written communication skills in English and French
- Experience using local consultants and data collectors, and involving project stakeholders in the evaluation
- Experience in conducting quantitative surveys; experience with ICT tools a plus
- Experience in data processing and data analysis using statistics software
Applications and Contact Information
Interested candidates should apply by submitting the following documents no later than Feb 9th, 2015.
- Organizational capacity statement outlining relevant experience and capability to deliver on activities as outlined above
- Resume of each proposed consultant and their role in this particular evaluation
- Financial offer that includes totals by major cost category (e.g. labor, travel). Note that CRS will select the consultant based on best overall value, not exclusively on price.
[1] Organisation for Economic Co-operation and Development. DAC criteria for evaluating development assistance.Retrieved November 2013 from: http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentas...
HOW TO APPLY:
The primary contact in Burkina Faso will be Carey McIntosh, the Head of Programs for CRS Burkina Faso (carey.mcintosh@crs.org).
