Terms of Reference
Title: Evaluation of DVV International Project in Mozambique
Project number: 8147
DVV International (DVV I) is the Institute for International Cooperation of the Deutscher Volkshochschul-Verband e.V. (DVV), the German Adult Education Association. DVV represents the interests of approximately 900 adult education centres (Volkshochschulen) and their state associations, the largest further education providers in Germany. As the leading professional organisation in the field of adult education and development cooperation, DVV International has committed itself to supporting lifelong learning for more than 45 years. DVV International provides worldwide support for the establishment and development of sustainable structures for Youth and Adult Education. DVV International is financed by the German Federal Ministry for Economic Cooperation and Development (BMZ). The current funding period is from 2017 to 2019.
In order to continually improve our programmes, and to ensure that they are responding to the needs of the countries in which we work, DVV International regularly evaluates its programmes. Against this background, DVV International is seeking applications from candidates to carry out an evaluation of our programme in Mozambique, in line with the orientations and guidelines below.
2. Background of the project
DVV International has been supporting adult education activities in Mozambique since 2008, and opened a country office in Maputo in 2010. Early activities focused on supporting the Ministry of Education to implement the recommendations of a situational analysis carried out in 2008, and of the Belem Framework for Action that emerged from CONFINTEA VI. These key documents informed the initial programme development of DVV International in Mozambique. Early initiatives focused on reducing the national illiteracy rate in response to the identified priorities of the Mozambican government. Therefore, innovative approaches to adult literacy, such as family literacy programmes were supported in Nampula Province, while support to university programmes sought to contribute to the professionalisation the adult education sector.
After some years of focusing mainly on literacy programmes and REFLECT (Regenerated Freirean Literacy through Empowering Community Techniques), an Integrated Literacy Programme (combining literacy with various other skills acquisition) was piloted, evaluated and continues to be implemented. This approach has been well-received, and the Ministry of Education and Human Development have recently taken an interest in scaling up the approach.
In parallel with these micro/meso initiatives, substantial work was carried out on the macro level, including the development of a national adult education strategy, an overhaul of the human resource structure for adult educators, and the development of a dedicated national curriculum for the training of adult educators. However, much of this macro level work has been set back following restructuring within the education sector, and budget cutbacks in the wake of the economic downturn in the past few years.
In the most recent funding period (2017-2019), the project in Mozambique has introduced a new element of non-formal vocational education, which is still at a very early stage in its implementation and development.
3. Objective of the programme
The overall objective for the programme of DVV International in Mozambique is:
Adult education (AE) contributes to poverty reduction and sustainable development.
The planned outcomes are:
- Provision of AE is improved and expanded;
- Partner and education networks In Mozambique are strengthened in their institutional capacities;
AE gained increased consideration in policies, poverty reduction and development strategies, and in governmental budgets at local, provincial and national level.
4. Activities of the programme
With the objective to sustainably develop social structures for adult education, DVV International’s programmes aim to make an impact on three levels (micro, meso and macro) in accordance with the above-mentioned objectives.
On the macro level:
DVV International is engaged in lobbying and strategy development for adult education and lifelong learning and plays an active role in the field of literacy and non-formal education as member of the national working group on adult education, in cooperation with the Ministry of Education and Human development. DVV International also works with the national education coalition MEPT, and the Movement for Mobilisation of Resources and Advocacy for Adult Education (MASMA). Curriculum development for the adult education sector (andragogy) is another key area of cooperation with government.
On the meso level:
DVV International supports the organizational development of its partners through targeted trainings on areas such as gender mainstreaming, monitoring and evaluation, financial management, and reporting. Other trainings deal with areas such as the integration of literacy contents with skills acquisition, small business, agriculture, advocacy and lobbying.
On the micro level:
Activities focus on two key approaches to promoting literacy and skills development at the community level. Firstly, the Integrated Literacy Programme, which combines elements of traditional literacy/numeracy acquisition with participatory approaches linked to livelihoods and community development. The second approach is non-formal vocational education focused on the acquisition of practical skills for youths that will enable them to access formal employment or engage in self-sustaining livelihood activities.
5. Purpose of the evaluation
As the above-mentioned objectives of the programme are quite broad, the following guiding questions are intended to provide a more specific focus for the evaluation:
Evaluation questions on the Macro Level
- DVV International’s activities on the macro level have decreased in recent years; which of the activities implemented in the last three years on the macro level have been effective.
- What recommendations can be made to expand and strengthen interventions on the macro level? What should the focus areas of future work be?
Evaluation questions on the Meso Level
- Has DVV International been successful in developing the organisational capacities of its partners?
- What kinds of (or which) activities have proven to be effective, and which have not yielded the expected results?
Evaluation questions on the Micro Level
- Non-formal vocational education (NFVE) is a relatively new area for DVV International in Mozambique; given that DVV International in Mozambique has a relatively small programme and limited budget, how can impacts be maximised in this area?
- What recommendations can be made regarding DVV International’s involvement in NFVE moving forward? Is the current approach viable to carry into the next 3-year funding phase? What, if anything, should be changed?
Overarching evaluation question
- Do the activities on each of the three levels mutually support and reinforce one another?
6. Guiding questions of the evaluation
The evaluation will follow the five DAC (Development Assistance Committee) Principles for Evaluation of Development Assistance, and the guiding questions listed below.
The extent to which the aid activity is suited to the priorities and policies of the target group, recipient and donor. In evaluating the relevance of the project, it is useful to consider the following questions:
- Macro Level:
- Have the activities carried out on the macro level been relevant; have they responded to the needs of the sector?
- Which topics on the macro level should DVV International focus on in the future?
- What activities would be recommended moving forward for DVV International to contribute to the development and strengthening of AE-related policies and strategies?
- Meso Level:
- Which activities have been particularly relevant and impactful to the partner organisations to further develop their organisational capacities? What were/are the strength and weaknesses of DVV International in this area?
- In which areas do the partner organisations show a need for further organisational development?
- What other kinds of support should DVV International provide to its partners in future?
- Micro Level
- How can DVV International offer relevant non-formal vocational education to youths, given the available funding and the large number of youths in need of training?
- Should DVV International continue to work in the field of non-formal vocational education? What alternative approaches could be used to reach the same target group?
- Which approaches (e.g. REFLECT, Integrated Programme, NFVE) should DVV I continue to support in the future? Why?
A measure of the extent to which DVV International’s project in Mozambique is attaining its objectives. In evaluating the effectiveness of the project, it is useful to consider the following questions:
- Macro Level:
- Has DVV International been able to contribute to the development of national policies and strategies in Mozambique?
- Which of the activities on the macro level have been effective and why?
- Which of the activities on the macro level have not been successful and why?
- On which activities should DVV I focus its activities in the next three years (and why)?
- What opportunities and unique selling points (USPs) exist for DVV I moving forward on this level?
- Meso Level
- Have the activities implemented by DVV International been successful?
- Have the organisational capacities of the partner institutions been improved?
- Which activities have been particularly successful and which were not so successful?
- What factors contributed to success on this level, and what more can be done to improve, and to mitigate challenges?
- Micro Level
- Has the Integrated Programme been effective in responding to the needs of the target groups? What factors have contributed to its success, or otherwise?
- Is the non-formal vocational education appropriate for the needs of the target group?
- What could be done to improve the non-formal vocational education approach?
- Could the NFVE approach be combined/aligned with the integrated programme approach? How? What would be the advantages/disadvantages of this?
- Which approaches of DVV I (e.g. REFLECT, NFVE, Integrated Programme) best respond to the needs of the target groups.
To what extent do the outputs (qualitative and quantitative) of the project represent an efficient use of the inputs (resources). Have resources been used in a cost-effective manner to achieve the desired results? Or, could alternative approaches have delivered the same results in a more cost-effective manner? In evaluating the efficiency of the project, it is useful to consider the following questions:
- Have the resources invested (on the macro, meso and micro level) been reasonable and adequate compared with the results? Have the resources been used effectively in the achievement of the results?
What positive and negative changes has the project produced, directly or indirectly, intended or unintended. What are the main impacts and effects that have resulted from the project on the social, economic, environmental levels, and in relation to other development indicators? The evaluation should also explore the positive and negative impact of external factors on the project activities and goals, such as economic, political, environmental or social changes. When evaluating the impact of the project, it is useful to consider the following questions:
- Macro level:
- Have the policies, strategies and curricula developed on the macro level been implemented and do they show a visible impact?
- What kind of impact have newly adopted policies on adult education in Mozambique had on the sector?
- Has the situation of adult education improved as a result of new policies?
- Meso Level
- Can it be proven that the organisational work of the partner institutions has improved? In which areas?
- If there is evidence for the improvement of the organisational capacities of the partners, what kind of impact does that have on the service delivery to the target groups?
- If there is no evidence for any improvement in the area of organisational capacities in some partner institutions, what are the reasons? Is DVV International implementing the right activities?
- Do some partner institutions show better results than others? Why?
- Micro Level
- What are the key impacts on the lives of participants of DVV International’s work on the micro level?
- Which of the activities show the greatest impact? Why?
- What are the success factors on the three levels contributing to successful outcomes? What are the major risks impacting on success, and what can be done to overcome these risks?
- Have any of DVV International’s activities led to any kind of unintended (positive or negative) results?
Are the benefits of the project activities likely to continue (sustain) after donor funding has been withdrawn. Is the projects environmentally, as well as financially sustainable. When evaluating the sustainability of the project, it is useful to consider the following questions:
- Macro Level
o What can be done to achieve a sustainable impact even if relevant political stakeholders and personnel change?
o Will the consideration of AE issues on the macro level continue even if DVV International ceases its engagement on the macro level?
o How can DVV I best carry its past work on this level forward in the restructured education ministry/government (i.e. curricula, training manuals, etc.)?
- Meso Level
o Are the improvements the partner organisations have achieved sustainable?
o What needs to be done to contribute to a long-term and sustainable improvement of partner institutions’ capacities?
- Micro Level
o Are the impacts of DVV International’s activities on the micro level sustainable? What more could be done to improve sustainability?
o Is the non-formal vocational education approach sustainable in its current form?
- Which are the other key international organisations working in the field of Adult Education?
- What coordination mechanisms exist among the different AE stakeholders?
- Are the various programmes in the field of Adult Education complementary? How could DVV International improve coordination with other stakeholders?
- Would it be possible to join forces in certain fields and forge synergies with other institutions? How, and with which ones?
Furthermore, the evaluation should examine the project in relation to the various indicators, examining previous funding phases and the current funding phase, while making recommendations for the next 3-year funding phase, and the longer-term perspective over the next 9 years.
7. Target group of the evaluation results
The target group of the evaluation will be the project management team of DVV International country and regional office, as well as the head office in Bonn, Germany. The results of the evaluation will also be sent to the funding institution in Germany (BMZ). The results of the evaluation will also be presented to the partner institutions, and will be used as a key source to critically reflect on the activities and measures of the project and inform future planning and implementation.
8. Expected Input / Financial Proposal:
The indicative number of working days is 25. Applicants are expected to submit their own proposals for the number of working days, phases of the work, and the distribution of working days per phase:
1 day total of briefing and debriefing;
3 days of document review;
1 day preparation of Inception Report;
2 days of mission preparation;
10 days of field mission;
3 days of data analysis;
3 days of report drafting;
2 days of report finalization.
Please take the indicative distribution of working days into account in your suggestions regarding the evaluation methodology.
Financial proposals for the evaluation should be as detailed as possible and include all honoraria and eventual interpretation/translation costs, as well as all envisaged travel costs and any other costs (including taxes).
9. Methodological approach
A combination of primary data collection and secondary data review is expected during the evaluation. Secondary data sources will be made available to the evaluator. The following secondary documents will be provided:
- Programme proposals
- Annual programme reports
- Final programme reports
- Mid-term and other kinds of (self)evaluations relevant to the programme
- Partner documents of project implementation (as far as available)
- National policies and strategies on adult education
- Regional policies and strategies on adult education
To collect information from primary data sources, interviews with key stakeholders are recommended, including:
- Information sources from within DVV International:
- Management (including financial management) of regional and country office of DVV International
- Senior management in DVV International head office
- Information sources on Macro Level
- Representatives from national/regional governmental institutions dealing with adult education
- Representatives from national non-governmental organisations dealing with AE
- Representatives from other international organisations dealing with AE in Mozambique
- Information sources on Meso Level – stakeholders from partner institutions
- Trainers and teachers from relevant partner institutions
- Beneficiaries from partner institutions
- Information sources on micro level
- Teachers/facilitators of various programmes
- Participants of various non-formal vocational education programmes of DVV I
- Possibly, also participants of non-formal vocational education programmes provided by other institutions
- Families of learners/participants (also from previous years)
Inasmuch as data is available, both qualitative and quantitative data should be collected and triangulated.
10. Indicative Timetable
The indicative timetable for the evaluation is as follows:
- Selection and contracting of evaluators: May to 1st week of June.
- Briefing: 1st week of June
- Submission and acceptance of inception report: 1st two weeks of June.
- Evaluation activities and field work: June-July.
- Debriefing with DVV International country office: Last week of July.
- First draft of evaluation: 1st week of August.
- Feedback on first draft: 2nd week of August.
- Second draft: 2nd week of August.
- Feedback on second draft: 3rd week of August.
- Final report and debriefing: Last week of August.
A detailed proposal for the calendar of activities is include at the end of this document.
11. Tasks of the evaluator/evaluation team
The evaluation team will be expected to fulfil the following tasks:
- Methodological design and planning of the evaluation in communication with DVV International;
- Desk Study of relevant project documentation (application and log-frame, annual plans and reports, other monitoring documents and project reports, teaching and PR material);
- Design of data collection instruments for the field phase;
- Preparation of an Inception Report;
- Conducting of the field phase with participation and involvement of all relevant stakeholders (see above methodological approach);
- Compilation of the preliminary evaluation report, integration of comments, and finalization of the report.
The following deliverables are expected:
- Inception report: The expert/team of experts will present a report providing detailed information about methods and evaluation questions. This report will be presented to DVV International, discussed and, if need be, amended.
- Debriefing in the country: By arrangement with the DVV International country office, the (team of) expert(s) will present the preliminary results of the evaluation, responding to questions and providing any necessary clarifications, documenting the discussion, which will then form part of the final report.
- Draft evaluation report: DVV International will receive a first draft of the final report, which will be examined on the basis of the DAC Evaluation Quality Standards. On this basis, comments will be made on the first draft before it is returned to the expert(s). A second draft will then be submitted for further comments and feedback.
- Final evaluation report: The final report will take into account all comments on the draft report, provide detailed information on entire evaluation process, and detail all results and recommendations.
The evaluation report should be produced in English and submitted to DVV International electronically. A respective template for the final report will be provided in the inception phase to the contractor. An executive summary / introductory section should also be made available in Portuguese.
13. Responsible persons at DVV International and logistics
DVV International will be the contracting authority for this evaluation. The main contact persons will be:
- The Senior Manager for Monitoring and Evaluation in DVV International head office in Bonn, Germany
- The Senior Desk Officer for Mozambique of DVV International in Bonn, Germany
- The DVV International Country Director in Mozambique
- The Regional Director of DVV International in Southern Africa
The DVV International country office will provide all relevant documents and support to the evaluator/ team of evaluators, and assist in arranging interviews and field visits etc., with stakeholders and partners.
Acceptance of the final report will be by the Senior Manager for Monitoring and Evaluation and the Senior Desk Officer for Mozambique of DVV International in Bonn, Germany.
14. Qualifications and profile the evaluator/evaluation team
The consultant(s) should have the following qualifications and experience:
- Have a proven track record of conducting evaluations of development projects (please, provide proof of this experience);
- Be able to demonstrate sound knowledge of evaluation methods and instruments;
- Have good knowledge of the sector of youth and adult learning and education, non-formal education and lifelong learning;
- Have experience of stakeholder participation and transparent knowledge management;
- Be familiar with the national context of Mozambique and the regional context of Southern Africa,
- Have excellent analytical and writing skills;
- Fluency in Portuguese and English.
15. Quality standards
The candidates should propose a suitable evaluation design and methodology, well adapted to the project objectives and TOR for the evaluation, in order to meet the evaluation’s goal and objectives.
The evaluation must comply with the following standards and guidelines:
- DeGEval-Standards http://www.alt.degeval.de/ calimero/tools/proxy.php?id=19084
- OECD-DAC Principles for Evaluation of Development Assistance
All interested candidates should submit the following documents:
- CV(s) of person(s) to be involved in the assignment;
- Technical proposal covering the following aspects (maximum of 8 pages);
- Narrative overview displaying an understanding of the task
- Detailed methodology
- Work Plan, including detailed proposed steps of implementation
- Consideration of any specific requirements envisaged for the evaluation
- Internal Quality Management
- A detailed financial proposal with all expected costs.
All applications must be sent by 17:00 hours (German time) on May 28th to the following e-mail addresses:
- Dr. Thomas Lichtenberg (Senior Manager on Monitoring and Evaluation at DVV International Headquarters in Bonn): firstname.lastname@example.org
- Ms. Julia Steffen (Senior Desk Officer for Mozambique at DVV International Headquarters in Bonn): email@example.com
- Mr. David Harrington (Regional Director of DVV International Southern Africa): firstname.lastname@example.org
- Mr. Augusto Macicame (Country Director of DVV International Mozambique): email@example.com
Late submissions will not be accepted.
17. Evaluation criteria of proposals
The proposals will be assessed according to the following criteria:
- Qualifications of the evaluation team (counts one third) (based on criteria mentioned above);
- Technical proposal (counts one third) (based on criteria mentioned above);
- Financial proposal (counts one third) (best cost-service relation).
18. Request for clarification
If you should have queries concerning this application process, please send your questions to the contact persons in Section 16 above. The deadline for submitting queries is May 24th at 17:00 hours (German time) Answers to all questions will be published on our website (https://www.dvv-international.de/dvv-international/stellenangebote/) for all interested parties the following day.
Tentative Timeline for the Evaluation
Selection of consultant
Signing of contract
Delivery of inception report
Feedback on inception report
Evaluation activities / Field work
Presentation of preliminary results
Delivery of first draft
Feedback on first draft
Delivery of second draft
Feedback on second draft
Delivery of final report
 Situational Analysis of Literacy, Adult Education and Non-Formal education in Mozambique (UNESCO, 2008)
 Belem Framework for Action (UIL, 2010)