Terms of Reference
Consultancy for an Impact Evaluation
of Swiss Church Aid’s Bangladesh Country Programme 2017-2021
1 Purpose of the consultancy
Swiss Church Aid (HEKS/EPER) seeks to procure the services of a researcher/consultant or a team of researchers/consultants to design, plan and conduct an impact evaluation of its Bangladesh country programme (2017-2021).
The desk research/inception phase is expected to take place during January/February 2021, while the field research/data collection phase is expected to take place between March and May 2021. The data analysis and report writing are expected to take place during June 2021. Due to the ongoing Covid-19 pandemic, the set-up and scheduling of the impact evaluation need to remain flexible.
It is expected that the researchers/consultants will have extensive knowledge of and experience in designing, planning, and conducting impact evaluations using various designs.
2 Purpose and scope of the evaluation
The planned evaluation is an impact evaluation which builds upon a data collection conducted in 2016. The purpose of the impact evaluation is threefold. Firstly, conducting an external end-of-phase evaluation is a PCM requirement and serves the purpose of being accountable to the People of our Concern (PooC), our partners, and donors. Secondly, the results of the impact evaluation will be utilized for the purpose of steering, i.e. for adapting the next country programme phase (2022-2025). Lastly, the entire impact evaluation process will also support an institutional learning process: which impact evaluation designs, approaches, and methods are suitable for a programme focusing on social integration.
The impact assessment is expected to deliver key findings on the validity of the programme’s theory of change. Its findings and recommendations will inform the planning of the subsequent phase (2022-2025). It is anticipated that the country programme Bangladesh will operate beyond 2025, particularly because social integration, the key component of the country programme, is a complex, long-term process. Moreover, for the programme to maximize its effectiveness it is important to increase its influence at policy making level. Accordingly, it is envisioned to gradually scale up the programme in terms of the number of people reached, as well as in terms of geographical coverage, provided the theory of change is accurate and depending on the availability of funds.
The target audience of the impact evaluation are Swiss Church Aid staff of the Bangladesh country programme, as well as the partner organisations implementing the activities of the programme and its projects, and the Desk Officer for the Bangladesh country programme. They will utilize the results to adapt their programming, and to transfer lessons learnt to other interventions. The results are also targeted at the MEL unit of Swiss Church Aid in Switzerland which will utilize the results for reporting purposes, and to improve Swiss Church Aid’s impact assessment policy and practice.
The data collection for the impact evaluation will focus on a set of indicators selected by the HEKS/EPER Office and HQ and outlined in the programme’s intervention logic and M&E plan. The data collected on these indicators will serve as measurement for the 2017-2021 programme’s progress and will also be utilized as baseline values for the 2022-2025 programme.
3 About Swiss Church Aid
Swiss Church Aid is the aid organization of the Protestant churches in Switzerland. It is engaged in development cooperation, humanitarian aid, and church cooperation. In 2019, Swiss Church Aid worked in 248 projects worldwide with a total budget of nearly 63 million Swiss Francs.
4 Description of the intervention
The overall goal of the HEKS/EPER country programme Bangladesh is to achieve the social inclusion of Dalit and Adibashi in Northwest Bangladesh.
In order to achieve the social inclusion of Dalit and Adibashi in the Northwest it is fundamental that the (O1) PooC communities are enabled to claim their rights and fulfil their obligations, so that they can be active participants of the society; (O2) enjoy equal rights and have access to services, whereby (O3) the secured access to land is particularly important; and are given equal economic opportunities in order to (O4) improve their economic status, to benefit from the country’s economic development and to contribute to a more inclusive growth. Working towards social inclusion requires addressing grievances and conflicts between the minority and majority groups in society. In this regards, HEKS/EPER aims at (O5) an improved relationship with mainstream society, which is inherently linked to (O6) an improved security of PooC communities.
The discrimination of Dalit and Adibashi is manifold and has lasted for many generations. This requires a holistic perspective to understand the complexity of social exclusion. And it requires systemic working approaches to address the variety of barriers and constraints preventing these communities from being citizens with equal rights and equal opportunities. The overarching approach of the country programme is the human rights-based approach (HRBA), while an inclusive market development approach is applied in creating economic opportunities and the diapraxis approach contributes to improving relationships between Dalit and Adibashi groups on the one hand, and between them and mainstream society on the other hand. The cross-cutting issues are gender, resilience building and conflict sensitivity.
The Bangladesh country programme concentrates its working area on the Northwest part of Bangladesh, namely the six districts Bogra, Naogaon, Joypurhat, Dinajpur, Nilphamary and Thakurgaon in the Rangpur and Rajshahi Divisions. The programme focuses on Dalit and Adibashi living in rural and urban zones of the plain land areas of these districts. The country programme aims at reaching out to 40,443 direct PooC and an estimated number of 63’496 indirect PooC through the direct interventions of the partner organisations ESDO, GBK and ARCO. The regional level advocacy forums of the NNMC spread over 10 districts of the Northwest region of the country. However, this advocacy work is not part of the impact evaluation.
5 Evaluation Criteria and Questions
The impact evaluation will primarily address the OECD-DAC criterion of effectiveness and will further assess to what extent the Bangladesh country programme contributed to the observed changes.
- Economic status: How has the economic status of Dalit and Adibashi in NW Bangladesh changed during the past five years? Has there been a diversification of income for Dalit and Adibashi households in NW Bangladesh during the past five years? How have the changes in income affected household dynamics? How do the different effects on women and men affect the legitimacy and implementation of the programme?
- Access to services and rights: How has the implementation of equal rights and the access to services changed for Dalit and Adibashi in NW Bangladesh during the past 5 years? How and why do Dalit and Adibashi have improved access to services? How and why do Dalit and Adibashi have greater enjoyment of equal rights? Do PooC have secured access to land?
- Social inclusion: How have relations with mainstream society changed for Dalit and Adibashi in NW Bangladesh during the past five years? Which aspects of value chain development (if any) have led to social inclusion? Has the interaction of Dalit and Adibashi with members of the mainstream society (because of integrative value chains) influenced the attitude of mainstream society towards them? How are the positive and negative effects of social inclusion perceived by Dalit and Adibashi? Has the focus on Dalit and Adibashi as PooC resulted in greater resentment towards them from mainstream society? What (if any) are the negative effects of social inclusion into mainstream society?
- Security: How and why has the security of the PooC improved? How and why has violence between PooC and mainstream neighbours decreased? How and why has violence within the PooC communities decreased?
- Covid-19: How has Covid-19 affected PooC communities, and how do these communities cope with the current situation?
6 Impact Assessment Design
Swiss Church Aid understands “impact” as the changes produced by an intervention. Consequently, this impact evaluation should not only measure or describe the changes that have occurred but should also identify the role of the Bangladesh country programme in producing these changes (causal attribution, causal contribution, or causal inference). The data collection conducted in 2016 serves as a point of comparison. If possible, the impact evaluation should include a difference-in-difference design.
Swiss Church Aid acknowledges that there are various causal approaches and impact evaluation designs, only some of which rely on a counterfactual. For this endline assessment, it is anticipated that the data collection conducted in 2016 (target group and comparison group) will serve as a point of reference, and that a difference-in-difference design may be possible. The quantitative data collection (to establish effectiveness) should be combined with a qualitative design, such as contribution analysis (to establish the “why” and the “how”). Alternative impact evaluation designs may be suggested as part of the application or during the inception phase.
The researcher / consultant is expected to comment on the proposed design in the application (maximum 1 page). The selected researcher / consultant will be able to finalise the design and methodology, and suggest adaptations, in the inception report.
The 2017-2021 country programme includes a Covid-19 response project which was not part of the 2016 data collection. Hence, the impact evaluation design for this project will have to be different from the overall design. Also, the target area of the 2017-2021 country programme differs slightly from the target area of the previous programme phase, due to the “graduation model”. Based on the “graduation model”, communities phase out of the country programme depending on their progress. This has to be taken into account for a potential difference-in-difference analysis and may be further explored during the inception phase.
- As part of the application, the researcher / consultant is expected to comment on the proposed design and methodology; comments on the ToR are also invited
- Inception report, finalising the design and methodology for the impact evaluation, including a draft evaluation matrix, a draft sampling strategy, and a draft sample size
- Quality assurance plan, setting out the systems and processes for assuring the quality of the research process and the deliverables
- Data collection tools (draft, final); these tools may include research instruments used during the 2016 data collection, any revisions to these tools, and/or the development of new tools
Impact evaluation phase:
- Training conducted for co-consultant(s) (if any) and possibly for enumerators on the impact evaluation design, sampling framework, research instruments, and research ethics. Duration and content of training to be determined by the researcher / consultant
- Data collection, together with co- consultant(s) (if any) and possibly enumerators
- Fully “cleaned-up” dataset in Excel, if applicable
- Documentation of qualitative data (notes or transcripts, as appropriate and feasible)
- Impact evaluation report (draft, final)
- Presentation of impact evaluation findings to the evaluation steering committee, the programme team, and key stakeholders
- Regular progress reports submitted to Swiss Church Aid’s MEL Officer in Zurich during the consultancy period, detailing 1) activities / tasks completed to date, 2) any challenges faced, 3) any adjustments made in response to the challenges, 4) any deviations from the timeline and explanations for the deviations, and 5) additional human resources and/or logistical support needed.
- Support for the development of target-specific products, such as a policy brief
# of working days
January / February 2021
March to May 2021
Impact Evaluation phase
Applicants are expected to submit their daily rate, including VAT, and their estimated travel costs to and from Dhaka as part of the application. Accommodation and transportation in country will be arranged directly by the Swiss Church Aid Country Office and / or the partner organisations and do not need to be included in the budget. Enumerators will either be field staff or will be hired locally – if required – and do not need to be included in the budget.
7 Management Roles and Responsibilities
The researcher/consultant will be responsible for administering and supervising the impact evaluation, including the methodological preparation, the data collection and analysis, and writing the impact evaluation report.
The Steering Committee consists of the Country Director of the Swiss Church Aid Country Programme in Bangladesh, the Desk Officer of the Country Programme based at Swiss Church Aid HQ in Zurich, and the MEL Officer of Swiss Church Aid, also based in Zurich.
The Steering Committee will oversee the administration and overall coordination of the entire impact evaluation process, including monitoring progress. The main functions of the Steering Committee are:
- Establishing the Terms of Reference
- Reviewing, commenting on and endorsing the Inception Report
- Engaging periodically with the substance of the evaluation and providing preliminary feedback to the researcher
- Managing the commenting process on the draft impact evaluation report
- Reviewing, commenting on and endorsing the final report
- Establishing a dissemination and utilization strategy
The MEL Officer (HQ) will lead the Steering Committee. During its work, the Steering Committee will strive towards reaching consensus on any issues that are discussed. In case of disagreement among the Committee members, the MEL Officer (HQ) will decide. In case of disagreement regarding comments on the draft endline assessment and/or impact evaluation report, minority views will be reflected in an annex to the final report.
The Swiss Church Aid Country Office in Bangladesh will assist the researcher/consultant by providing all necessary documents (2016 data collection report, proposals, logframes, workplans, budgets, reports) and by facilitating access to key stakeholders and specific information or expertise needed to complete the evaluation. Together with the partner organisations, the Swiss Church Aid Country Office will coordinate the field research, including meetings, transportation and logistics, as well as organising the debriefing in country.
The researcher/consultant will report directly to the MEL Officer (HQ) who will manage the consultancy contract(s).
8 Follow up of the Evaluation
- Assessment of the reports: The impact evaluation report will be assessed by the MEL Officer (HQ) against Swiss Church Aid’s Quality of Evidence
- Management response: The Country Director of the Bangladesh Country Programme will write a management response, providing the Country Team’s perspective on the results and recommendations. The Country Director will also develop an Action Plan to ensure that findings and actionable recommendations are disseminated, and appropriate action is taken.
- Swiss Church Aid will publish a summary of the impact evaluation report on its website. Further external dissemination will be discussed by the Steering Committee and other relevant stakeholders, as applicable.
9 List of Documents
Documents for the desk review will include:
- Bangladesh Country Programme 2017-2021 proposal
- Annual reports of the country programme
- Reports of the Partner Organisations
- 2016 data collection report
10 Evaluation Team / Qualifications
HEKS/EPER welcomes applications from individual researchers/consultants, as well as teams who meet the required qualifications.
The researcher/consultant (or the team) is expected to meet the following qualifications:
- Master’s degree in social sciences (e.g. sociology, anthropology, development studies)
- At least 5 years of research and other relevant professional experience, including experience in complex contexts
- Sound understanding of various approaches to assess impact
- Sound expertise in quantitative and qualitative data collection and analysis
- Thematic expertise (value chain approach, human rights-based approach, social inclusion, social networking)
- Understanding of the discrimination and marginalization of Dalit and Adibashi communities
- Strong analytical skills and proficiency in writing in English
- Ability to deliver quality reports/analysis and results in line with established deadlines
- Excellent facilitation and coordination skills
- Ability to work in a team, communication skills, flexibility, and organisational skills
Interested and eligible candidates should submit the following:
- A cover letter, indicating how the candidate meets the required qualifications (maximum 2 pages)
- A CV (maximum 3 pages)
- Comments on the proposed design (maximum 1 page)
- Two references
If selected for an interview, the candidate will be requested to submit one example of previous, similar work.
The following criteria will be used when evaluating the proposals:
- The experience and competence of the researcher / consultant in relation to the Terms of Reference
- The ability to perform the assignment in the appropriate time, and
- The budget for the assignment
Submit completed applications (preferably in one file) with the e-mail title “Application Impact Evaluation Bangladesh” to Annika.Klotz@heks.ch by November 4th, 2020. Please indicate in your application on which website or through which mailing list you came across the call for applications.
For questions about your application, please contact Ms. Annika Klotz, MEL Officer at Swiss Church Aid in Zurich (Annika.Klotz@heks.ch).
1. PooC = People of our Concern
2. Diapraxis = A strategy towards improving relationships. Diapraxis means dialogue through practice. Diapraxis is a methodology designed to contribute to nonviolent transformation and prevention of conflict. The HEKS/EPER diapraxis approach refers to interlinkages between “classical” development work, such as livelihoods or rural development activities, and conflict transformation endeavours.
3. In 2016, the corresponding questions were: How and why have interactions between Dalit and Adibashi on the one hand and mainstream society on the other hand increased? How and why has the attitude of neighbouring Muslim-Bengali communities changed towards Dalit and Adibashi?