1. Introduction
Save the Children in Mozambique is seeking the services of consultants to conduct an endline evaluation for the project PAGEM (Programme for the Advancement of Girls Education in Mozambique), funded by DFID.
The project strives to promote the development of education for marginalised girls in ten districts in the provinces of Gaza, Manica and Tete in Mozambique, enabling them to access education, to learn effectively, and to be supported to remain in school.
In order to identify and assess the impact of the project on the project’s target group it is necessary to track and measure the changes experienced by those who are (treatment group), and who are not (control group), exposed to the project’s activities in order to compare and contrast these effects at midterm and endline.
The baseline was conducted in 2013 and midline in 2015. The endline process is planned from August 2016 to January 2017. (see section 6)
2. Scope of Work
The main aim of this assignment is to generate strong evidence on the interventions that work for improving girls’ school attendance and learning outcomes.
This Independent Evaluation is a donor requirement, and must meet the project’s needs as well as the donor compliance regulations. For this purpose, the project is contracting an Independent Evaluator to conduct an Endline study, composed of i) Planning phase (ii) Database preparation ii) tool re-design phase; (iii) training of staff and field workers (iv) field work and data collection; (v) data analysis; and (vi) report writing. Quality controls must be guaranteed throughout.
The sample of the endline study is 1454 households, including the testing of one eligible girl in each household.
The independent, external evaluator selected and contracted, will:
- Work with the project and donor to improve the midline dataset
- Agree with the project and donors the preferred approach to the analysis of data, creating a data analysis plan for approval by Save the Children
- Help to review and re-design tools (including girls’ learning test and interview tools) in electronic version
- Select and provide training of the enumerators and supervisors, ensuring the right linguistic considerations are made and elaborating content for training programme and field guides
- Ensure that the tests are loaded onto the set of tablets (provided by the project) well in advance of the data collection process beginning and that consistent IDs are used for both HHS and learning tests
- Pre-test tools in the field and report back on proposed revisions
- Ensure a robust structure and process of monitoring and supervision for all phases of data collection
- Organize all logistics necessary to collect the data in the field, including detailed data collection and transport plans, and all transport to field sites
- Collect data for learning (reading and maths test) in households and attendance (spot checks) in schools
- Conduct a Household Survey
- Conduct qualitative research (such as focus group discussions, case studies, key informant interviews, class observation tools etc)
- Review data as it is collected, at periodic intervals, and take corrective measures immediately
- Take part in Skype/conference calls regularly, as well as producing weekly field reports and a final data collection report within 3 days of finishing data collection
- Incorporate results from output indicators (in logframe) into their analysis
- Be responsible for data quality checks at the end of data collection process
- Verify that the data collected is reflected in the final database and is cleaned, with any issues rectified as swiftly as possible.
- Ensure that qualitative data has been fully coded and analysed so that it is integrated into the findings and analysis of the report, according to the key outputs and outcomes of the project and in relation to the evaluation questions.
- Merge endline and midline datasets
- Conduct presentational statistics and inferential statistical analysis on the quantitative analysis
- Triangulate qualitative and quantitative data in analysis
- Follow the report structure stipulated by the donor in producing a final report and remain available for revisions according to the donor's feedback
The endline will be conducted in the same enumeration areas where the baseline and midline were conducted, in 10 districts across Gaza, Manica and Tete province.
The data will be collected from the same households that were surveyed during the midline. All Households must be followed up and included in the survey and explanations given for any households that are not possible to locate after which households will be substituted.
3 Quantitative and Qualitative Research
The evaluator will be required to use the quantitative research methodologies outlined in the M&E Framework, that enable an assessment of the attributable impacts of the project particularly around educational outcomes, described below:
Household Survey: A comprehensive population based survey will focus on an in-depth interview with the head of the household or primary caregiver and the children. The Household Survey takes about 2 hours and has already been formulated.
Learning Outcomes Assessment: in order to test the level of children in mathematics and Portuguese. The tests have already been devised.
Attendance data for cohort study: The project will also collect the data of the final spot check (verifying of attendance within the classroom) that will be used to analyse the attendance during the endline.
The Independent Evaluator will use qualitative research methodologies to explore and explain the nature of causality and resulting changes that are observed. The qualitative research methodology seeks to examine the theory of change and intervention logic of the project including the linkages between marginalisation (risk, barriers), the project’s intervention outcomes and impacts. Qualitative data will include: Focus group discussions with girls, with boys and groups of adults. In all we expect 5 focus groups to take place in each of 10 sites/communities. In depth interviews with in-school and out-of-school girls, government authorities, parents, community leaders, women’s group leaders, members of school committees, teachers, and community literacy promoters.
4. Data analysis:
Quantitative data analysis: the Independent Evaluator will provide a clean and merged dataset in Stata file formats, a cleaned dataset with errors and inconsistencies rectified. They will provide all raw data files and the estimation dataset, which contains the various merged raw data files. They will also provide Excel exported version.
Qualitative data analysis: Qualitative data collection will be recorded in the field and transcribed. For the analysis of qualitative data and report writing, qualitative data will be coded and appropriate software applied for managing and content analysis.
Quantitative and qualitative data will be triangulated. Analysis will be disaggregated by gender and by other relevant variables (e.g. age, language).
5. Areas of expertise for Independent Evaluator Team
It is expected that the mininum requirement in terms of composition of the team would be:
Evaluation team leader (based in Mozambique or elsewhere)
Fieldwork manager (based in or with extensive experience in Mozambique, and fluent in Portuguese and English)
Highly qualified statistical analyst and quantitative research specialist
Qualitative research specialist
Electronic database technician experienced in use of tablets for data collection
Gender specialist, Education specialist, EGRA/EGMA methodology specialist (some of these possibly met within the same consultant)
A supervision structure of staff to oversee data collection in 3 provinces (not possible to travel by road between the three provinces)
Translation and interpreting services
In any case, the Independent Evaluator should ensure that their proposed team includes the technical expertise and practical experience below:
o Extensive experience in collecting qualitative and quantitative data in rural contexts
o High level skills in statistical analysis (proficient user of STATA or equivalent) and in triangulation of qualitative and quantitative data
o Skills and experience evaluating projects concerning education and gender
o Capacity to integrate and cross reference data and findings gathered by the project (qualitative and quantitative)
o Experience gathering data in Mozambique
o Experience conducting research, including learning tests, with children
o Fluent in English (senior researchers writing report)
o Fluent in Portuguese and English (data collection manager).
o Experience in large sample sizes, household surveys and information management
o Experience recruiting and training enumerators, particularly in Mozambique
o Experience using tablets for data collection and creation of database
o Experience of Girls Education Challenge evaluation
The data collection manager should have experience managing large data collection teams, working in Mozambique, be fluent in English and Portuguese, and have strong experience of evaluations particularly in education sector and particularly in Mozambique.
Experience in evaluations of Girls Education Challenge is an advantage.
Samples of work may be requested.
6. Timeline
It is expected that the following timeline be held (to be confirmed in contract):
Contract process completed: 27 July
Initial meetings: 28 July
Training and desk review completed: 10 September
Data collection: Sept and October (estimated 35 days, provincial teams working simultaneously)
Data cleaning completed: 30 November
Analysis and report drafting completed: 10 January
Final report due: 30 January
Period of feedback and revisions: 1 Feb - 30 March
7. Application process
Institutions, consultant teams and other bodies are invited to submit application for this work in the format of an extended expression of interest.
Max 2 pages: Team composition: explain how, through proposed team members, you fulfil the skills and experience required for the work. Specify any key individuals yet to be identified. Please also specify the daily rates applicable for all members proposed.
Max 3 pages: Explain how your proposed team structure (including all staff to be hired) and processes of recruitment, training and data collection, will be able to fulfil the requirements of the work, and how they will be deployed through the different phases of the work: Design of tools, recruitment and training of staff, data collection, data analysis, final report writing. Please specify the fees or fee range (daily) you would normally consider for the different elements of the team to be recruited.
Max 1 page: Outlines the major challenges you see for the work, and the ways in which you will specifically mitigate those challenges, as well as the specific methods you will employ to ensure quality control throughout the process.
All applications should be received by Monday 18th.
Applications should be submitted electronically to:
All applications will be acknowledged by email.
Only preferred candidates will be invited, after being sent extended Terms of Reference, to submit a full technical proposal, financial proposal and work plan.
HOW TO APPLY:
Application process
Institutions, consultant teams and other bodies are invited to submit proposal for this work in the format of an extended expression of interest.
Max 2 pages: Team composition: explain how, through proposed team members, you fulfil the skills and experience required for the work. Specify any key individuals yet to be identified. Please also specify the daily rates applicable for all members proposed.
Max 3 pages: Explain how your proposed team structure (including all staff to be hired) and processes of recruitment, training and data collection, will be able to fulfil the requirements of the work, and how they will be deployed through the different phases of the work: Design of tools, recruitment and training of staff, data collection, data analysis, final report writing. Please specify the fees or fee range (daily) you would normally consider for the different elements of the team to be recruited.
Max 1 page: Outlines the major challenges you see for the work, and the ways in which you will specifically mitigate those challenges, as well as the specific methods you will employ to ensure quality control throughout the process.
All applications should be received by Monday 18th.
Applications should be submitted electronically to:
All applications will be acknowledged by email.
Only preferred candidates will be invited, after receiving an extended Terms of Reference, to submit a full technical proposal, financial proposal and work plan.
