End of Project Evaluation for the EU-funded Project in Tanzania. To certification and beyond: Market access for sustainable coffee, horticulture, and
Project Consortium
Tanzania Coffee Research Institute (TACRI): A Government research body with a mandate to initiate, implement, promote and carry out research on coffee production, processing, quality, farming systems, and husbandry of other crops associated with coffee. TaCRI has implemented three EuropeanCommission programs in the past and currently works with Solidaridad ECA to implement her CoffeeResilience Program in Northern Tanzania.
Recommended:
Tanzania Tea Research Institute (TRIT): A Government research body that supports the development of both small and large scale tea producers through appropriate, cost-effective, high-quality researchand technology transfer, to ensure the sustainability of the Tea industry. TRIT is well versed in practicaltechnological innovation for the tea sector.
Tanzania Country Office: Carry out Country-level implementation roles for the PACE Project. The organization has been implementing programs in coffee, horticulture, sugarcane, cotton,dairy, gold, and landscape management in Tanzania. We have a rich experience in promoting best practices and sustainable production, trade, and market development for producers. Our programs inTanzania focus on value chain development, food security, gender inclusivity, ICT, climate innovations, and investment in viable businesses for impact creation while working together with governments, multilateral agencies, private companies as well as commodity producers.
Project Background
Coffee, horticulture, and tea products are among the most important export agricultural commodities in Tanzania. Tanzania’s main export markets are India, Japan, China, Netherlands, Germany, UnitedKingdom, Kenya, Pakistan, and the United Arab Emirates. Coffee is Tanzania’s third-largest export crop contributing 14.1% of total export value. The production in 2016 totaled 59,502MT, worth US$ 173.6 million (55% Arabica and 45% Robusta). The main Arabica growing areas include North Kilimanjaro,Ruvuma, and Mbeya while those for Robusta include the Bukoba area of the Kagera region. Thehorticulture industry in Tanzania is the fastest growing sub-sector within the agricultural sector withan annual average growth of about 9 to 12% per annum. This record of growth is more than double the overall annual growth rate of the agricultural sector. In 2015, horticulture contributed 38% of the foreign income earned from the agriculture sector. The exports in 2015 reached US$ 545 million,compared to US$ 64 million in 2005. The Horticulture sub-sector employs about 2.5 million people, which makes the industry a major player in the country’s economic growth and employment sector. Tea contributes more than US$30 million to Tanzania’s export earnings, making it the fifth-largest export crop after cashews, coffee, cotton, and tobacco. The tea industry provides employment to50,000 families and directly or indirectly benefits more than 2 million Tanzanians.
Global brands and retailers focus on sustainability as an integral part of their corporate socialresponsibility policies designed to ensure environmental, social, and economic welfare. In many ways, the rise of Voluntary Sustainability Standards (VSS) is a response to wider industrial and social transformation, and it has a growing impact on international trade. However, complying with VSS isnot simple, especially for small and medium-sized farmers, as it takes great effort and investment in people, infrastructure, and institutions. Furthermore, the infrastructure and the public finance needed to support the compliance process do not exist. Often, this stops the majority of farmers from exporting to profitable markets in developed countries. Without a clear strategy and proactive support from governments and the international community, VSS runs the risk of reinforcing of thewalls that already shut out small-scale producers.
Project Objectives
The overall objective of this action is to address issues surrounding the effective use of certification schemes and the use of voluntary sustainability standards (VSS) to improve and drive market opportunities for Tanzanian “branded” sustainable tea, coffee, and horticulture products. Solidaridad proposes to mobilize the value chain actors, including the government, research, private sector, and the VSS service providers to sensitize and create awareness among smallholder farmers on the sustainable production of high-quality agricultural commodities, food safety, benefits of VSS mechanisms, and eventual integration of sustainability practices in their business-as-usual model.
Project area/regions
The project is implemented in Songwe, Mbeya and Ruvuma regions (Coffee), Njombe and Mbeya regions (Tea and Horticulture) and Zanzibar (Pemba and Unguja – horticulture)
The Specific objectives includes:
- improving domestic, regional, and international market access for Tanzanian “branded” sustainable tea, coffee, and horticulture products; and
- Supporting theadoption of identified voluntary sustainability standards.
Purpose of the Evaluation
The objectives of the End of Project Evaluation are to:
- Assess performance towards the program’s impact, outcome and output indicators so far and identify possible gaps towards achieving full results (accountability).
- Document the expected and unexpected, positive and negative, outcomes and lessons learned, provide information on the project approaches identifying what has worked and what has not worked for learning and as a guide for future planning and potentialup scaling
- Validate the assumptions in the ToC (will be shared with successful applicant) that can be used for designing future strategies.
Read Also:
Scope of Work and Key Tasks
The End of Project Evaluation will cover the implementation period from the start of the program. This evaluation is expected to assess the effectiveness of the implementation strategy of the project in a consortium setup. This will include the implementation modalities and participation by the other consortium partners.
The evaluation will include a review of the project design and assumptions made at the beginning of the program development process. It will assess whether results achieved, the implementation strategy has been optimum, and recommends areas for improvement and learning. The evaluationwill also assess whether project resources were efficiently utilized to produce planned results and whether results realized within the planned period. Lastly, the sustainability of the project willbe looked at to ensure a lasting impact.
Indicative Evaluation questions:
- Relevance of the program strategy (design and focus of the program)
- To what extent are the program objectives meeting the needs of small-scale coffee, tea, and horticulture farmers and their families?
- Does the program address the key bottlenecks in accessing markets for the farmers?
- Are the underlying assumptions and context of the program still correct?
- Are the program results framework, indicators, and targets met?
- Effectiveness (project progress)
- Are the chosen program interventions and activities delivered the desiredoutcomes?
- To what extent was the program implementation effective (Management arrangements, workplanning, finance and co-finance, project-level monitoring and evaluation systems, Stakeholder Engagement, reporting, communications)
- What factors have contributed to or hindered the achievement of intended outputs and outcomes? If any
- To what extent have partners in the consortium played their expected roles toward the achievement of the program goal?
- Has the consortium partnership been appropriate and effective?
- Outcomes and Impact
How does the program contribute towards lasting change to the lives of beneficiaries with regard to:
- Increased sustainable production of coffee, tea, and horticulture; if there would beno sustainable agricultural practices and production systems, what would the localproduction situation look like?
- Adoption of VSS and accessibility of markets; if there would be no adoption of VSSand market linkages, what would the situation look like?
- Has project impact beneficial development effects(income generation, gender equality, and women’s empowerment, improved governance,etc.?
- Efficiency:
- Are the inputs identified were realistic, appropriate and adequate for the achievement of theresults?
- Do the actual results (outputs and outcomes) justify the costs incurred?
- Are resources effectively utilized?
- What are the factors contributing to implementation efficiency?
- Sustainability:
- What is the likelihood of continuation and sustainability of program outcomes and benefits after completion?
- What are the best practices and key lessons learned based on the experiences of project implementation?
The Consultants may choose to categorize the questions differently, as long as these questions are addressed.
Methodology
This evaluation requires a mixed-method approach that allows for methodological triangulation toincrease the validity and credibility of data. Participatory methods shall be used to collect bothqualitative and quantitative data. The key methods shall include the following but are not limited to:
- A household survey using a survey questionnaire (a representative sample of the total beneficiary population can be used)
- Focus Group Discussion (FGDs) with lead farmers and AMCOSs/farmer groups boardmembers
- Literature review of existing documents and review of context and poverty trends including the project proposal, MTE report and other documents, annual and quarterly reports,monitoring, and evaluation reports
- Field observation of the targeted project areas
- Key informant interviews (KII) with AMCOSs, farmer groups board members, Leadfarmers, Extension officers, DAICOs, project partners (TACRI, TRIT), and project management team reflection and feedback sessions with the project management team.
The final methodology to be adopted shall be discussed and agreed upon with consultants/firm duringthe inception meeting.
The End of Project evaluation team will first conduct a document review of project documents (i.e. Initiation Plan, ProjectDocument, Project Inception Report, Baselines, MTE report, Tracking Tools, Project Steering Committee meetingminutes, Financial and Administration guidelines used by Project Team, etc.) provided by the ProjectTeam. Outcome and output indicators from the Logical Framework and reported upon in the baseline, MTE and annual reports will be used to the extent possible. In case these indicators are not sufficientlyreported upon, data collection will need to be complemented.
The consultants will participate in an inception workshop to clarify their understanding of theobjectives and methods of the end of project evaluation, producing the end of project evaluation inception report thereafter. The end of project evaluation mission will then consist of interviews and site visits to selected areas in the project areas.
Expected Outputs & Deliverables
Solidaridad and the consultant will work together to identify the study team/enumerators. However, Solidaridad recommends that consultants identify one Team Leader (TL) to coordinate/conduct the comprehensive study and liaise with Solidaridad.
To collect information from the field, an adequate number of Field Enumerators (FEs) will be deployed in the facilitation of Solidaridad’s PMEL Officers.Supervision of the fieldwork and the quality (reliability and validity) of the data/information collectedfrom the field is the primary responsibility of the TL. The TL will work closely with Solidaridad PMELOfficers and Project Team. In each step and process, consultation with the Program Manager are PMELOfficer is vital.
The consultancy/consultant team will be primarily responsible to:
- An expression of interest (EOI) for the study, outlining the proposed methodology andprocess of data collection and analysis that specifies:
- Proposed methodology for implementation of the study, including sample sizes.
- A detailed timeline for the study
- Inception report based on the work plan and EOI:
- Detailed methodology and evaluation matrix
- Draft data collection tools for all indicators to be measured in the study, standardindicators definitions agreed upon by the project should be used to develop data collection tools.
- Presentation of methodology to the Solidaridad team in an inception meeting and incorporated suggestions provided.
- Training of enumerators (Solidaridad PMs and PMEL Officers will be part of it)
- Draft a report of the study for the feedback and comments of Solidaridad and the partners involved.
- Presentation of the main findings of the study at field-level validation involving project teams, local authorities, and other stakeholders as agreed with the project team.
- A final report in English following the Guidelines under Annex 1.
- Datasets: The consultant shall submit all datasets, cleaned & raw, transcription or audio filesduring focus group discussions, KI, scripts for data analysis, and any form of data captured during the implementation of this assignment
- The evaluator shall also submit a pager info graphic summary of the impact achieved by the project.
Read Also:
Coordination and cooperation with Solidaridad
The Consultant undertaking the study will work under the coordination of PMEL Officer, who willsupport the mobilization of respondents and offer technical backstopping. The Project team will coordinate consortium partners in ensuring that the sampled survey respondents (beneficiaries andKey informants) are sensitized and mobilized for the exercise while availing a workable itinerary andtechnical backstopping.
Profile of the Evaluators
The end of project evaluation team will ideally consist of independent consultants that will conduct the end of project evaluation – one team leader (with experience and exposure to projects and evaluations in Tanzania) and local team experts.
The team must have strong combined expertise in the sectors of Certifications and VoluntarySustainability Standards, agricultural economics, extension and education, gender, and Natural Resource Management. The successful applicant will propose a team that can provide adequate experience to conduct the assessment and that meets the following requirements:
- Advanced degree (masters) in the area of agriculture, economics, agri-business development, social sciences, or other related disciplines;
- At least five years of proven experience in leading and conducting similar exercises in Africa
- Demonstrated experience in both quantitative and qualitative methods of data collection and analysis; including survey design, implementation of surveys and statistical data analysis, and the use of participatory appraisal techniques in data collection and analysis.
- Ability to interpret and analyze complex qualitative and quantitative data and to present findings and recommendations in a clear and concise way;
- Strong expertise in value chains (coffee, other food crops, horticulture)
- Knowledge and sensitivity to the political and social context of Tanzania context is an asset;
Application and Timeframe
Each application package should include the following:
- An application letter addressing the selection criteria including how the firm’s/group’sprevious experience matches the consultancy objectives as well as the interest for theconsultancy opportunity. It should also indicate the availability of the assessment team. The letter should be no longer than two pages.
- A brief proposal for the study with a proposed methodology and work plan (not more than 8 pages). The proposal should have a budget with detailed breakdowns of different costs involved. A budget with aggregated figures will not be accepted. The budget must be in Euro (EUR). Any other currency will not be considered.
- A sample of recently written reports for a similar assignment in the VSS thematic sector.
- Updated CVs for at least two lead consultants proposed to conduct the assignment including relevant work experience and qualifications.
- Proposed work plan for the assignment.
The overall time frame of the assessment will be 30 working days, which will include inception meetings, finalizing survey methodology, training of enumerators, data collection exercise, processingof data analysis, report writing, and presenting the findings of the survey. The team undertaking thestudy will be fully responsible to administer the study including management of data collection.
Details will be worked out jointly with the consultant team and project staff. However, the work is expected to commence in June 2024 for an estimated 30 days’ period.
Note that the evaluation will be carried out in all project areas that include Mbeya region (Rungwe, Busokelo, Mbeya DC districts), Njombe (Lupembe, Njombe DC, Njombe TC and Wanging’ombe), Ruvuma region (Mbinga DC, Mbinga TC and Nyasa districts), Songwe region, Zanzibar (Pemba and Unguja).
How to apply
The deadline for submission of the EOI is 27th May 2024.
Submissions are by email to procurement.eca@solidaridadnetwork.org while addressed to:
Country Manager – SOLIDARIDAD TANZANIA
Uzunguni street, Sekou Toure road, opposite Kibo Palace hotel.
Arusha, Tanzania
Note: Canvassing will lead to automatic disqualification and the only successful candidates will be contacted.
NB: In the e-mail, the subject indicates: End of Project Evaluation-To Certification and Beyond Project
Guidelines on Suggested Contents for the End of Project Evaluation Report
- Basic Report Information (for the opening page or title page)
- Title of project
- Project Number
- End of Project Evaluation period and report period
- Region and countries included in the project
- Implementing Partner and other project partners
- End of Project Evaluation team members
- Acknowledgements
- Table of Contents
- Acronyms and Abbreviations
Main Report Information
- Executive Summary (3-5 pages)
- Project Information Table
- Project Description (brief)
- Project Progress Summary (between 200-500 words)
- Concise summary of conclusions
- Recommendation Summary Table
- Introduction (2-3 pages)
- Purpose of the End of Project Evaluation and objectives
- Scope & Methodology: principles of design and execution of the End of Project Evaluation, approach anddata collection methods, limitations to the End of Project Evaluation
- Structure of the End of Project Evaluation report
- Project Description and Background Context (3-5 pages)
- Development context: environmental, socio-economic, institutional, and policy factors relevant to the project objective and scope
- Problems that the project sought to address: threats and barriers targeted
- Project Description and Strategy: objective, outcomes and expected results, description of field sites (if any)
- Project Implementation Arrangements: short description of the Project Board, key implementing partner arrangements, etc.
- Project timing and milestones
- Main stakeholders: summary list
- Findings (12-14 pages)
- Project Strategy
- Results achieved for all project indicators
- Project Implementation and Adaptive Management
- Sustainability
- Conclusions and Recommendations (4-6 pages)
Conclusions: Comprehensive and balanced statements (that are evidence-based and connected to theEnd of Project Evaluation findings) which highlight the strengths, weaknesses and results of the project
Recommendations
- Corrective actions for the design, implementation, monitoring and evaluation of the project
- Actions to follow up or reinforce initial benefits from the project
- Proposals for future directions underlining main objectives
- Annexes
Read Also:
Annexes to be included in the final report:
- End of Project Evaluation ToR
- End of Project Evaluation matrix (evaluation criteria with key questions, indicators, sources of data, and methodology)
- Questionnaire or Interview Guide used for data collection
- End of Project Evaluation mission itinerary
- List of persons interviewed
- List of documents reviewed
- Data set
