ECHO Action Final Evaluation - Consultant

SCOPE OF WORK

Organizational and project background

The International Rescue Committee (IRC) responds to the world’s worst humanitarian crises and helps people whose lives and livelihoods are shattered by conflict and disaster to survive, recover, and regain control of their future. The IRC Yemen began its programs in October 2012 and currently operates in the South and North of the country through direct programming and local partnerships.

The country program operates in Aden, Abyan, Lahj, Al Dhale’e and Shabwa, Hodeidah, and Sana'a governorates and continues to assist the affected population to achieve the following Outcomes:

• People meet their basic needs and protect livelihoods during emergencies.
• Adolescents and adults are physically and mentally healthy.
• Children are well nourished and protected from all forms of undernutrition.
• Women and girls use timely and effective Reproductive Health (RH) services.
• Women and girls are protected from and treated for the consequences of gender-based violence.
• People are safe in their communities and receive support when they experience harm.

The International Rescue Committee (IRC) with funding from DG ECHO is implementing life-saving health, and nutrition interventions in Al Dhale’e and Lahj governorates and supporting health and GBV response activities in Shabwah governorate. The coverage of emergency response activities was also expanded to include Aden, Abyan, Lahj, Al Dhale’e, Taizz, Hodeidah, Marib, Shabwah, Al Bayda, Al Jawf, Hajjah and Sa’ada. The Action began on 1 April 2021 to 31 March 2025, addressing the needs through five results:

• Result 1 (R1- Health): Communities are protected from and treated for communicable and non-communicable diseases, including mental health conditions; and women and adolescent girls are protected from and treated for the complications of pregnancy and childbirth.
• Result 2 (R2- Nutrition): Children under the age of five (CU5) and pregnant and lactating women (PLW) are protected from and treated for acute malnutrition.
• Result 3 (R3- Emergency): Strengthened emergency and preparedness response mechanisms (EPRM) delivered in a flexible and timely manner that addresses the needs of affected populations including vulnerable groups such as internally displaced persons (IDPs) and Muhamasheen.
• Result 4 (R4 – Protection/GBV): Ensure that adolescent girls and women who suffer harm receive GBV response services and access age-appropriate and culturally acceptable PSS activities.
• Result 5 (R5 - Advocacy): Evidence-based advocacy towards duty-bearers and decision-makers to strengthen policies and systems to address barriers to accessing quality basic services and build resilience.

Rationale and Purpose

The primary purpose of the evaluation is to assess the completeness and effectiveness of the project

deliverables and achievements realized and take stock of the positive and negative changes brought about as a result of the project. The evaluation's information and recommendations will inform the IRC and DG ECHO of the existing gaps and possibly better project design and implementation. As such, the Consultant is expected to provide information about success factors and recommendations for improvement areas.

Specific Objectives of the Evaluation

• To assess the program’s impact since its inception in April 2021 (determine the extent the project generated significant positive or negative, intended or unintended effects in the target population)
• To identify lessons learned and good practices and provide recommendations.
• To determine the extent the project achieved its objectives and outcomes as described in the project log frame.

Evaluation Criteria and Questions

The evaluation will employ the five specific evaluation criteria for humanitarian action from OECD- DAC. This evaluation will focus on aspects of relevance, effectiveness, efficiency, impact and sustainability. The following questions, among others, should be addressed:

Relevance:

• Were the activities in line with the strategic priorities of DG ECHO?
• To what extent was the project aligned to the immediate needs and priorities of the target programme participants (including vulnerable groups) as well as the priorities of the donor?
• Did this programme effectively reach the most vulnerable households?
• Was the programme intervention and outputs appropriate to the context and in line with the communities and programme participants needs and priorities?
• Were the programme activities addressing the challenges/problems that were identified at programme design stage?

Effectiveness:

• To what extent were the intended results achieved, and what internal and external factors influenced the achievement or non-achievement of those outcomes?
• To what extent did the project meet its targets and deliver outputs and outcomes? / What was the performance against the stated indicators?
• To what extent were communicable and non-communicable diseases prevented and treated within the target communities?
• What measures were implemented to protect and treat mental health conditions?
• To what extent has the project protected and treated Women and adolescent girls from complications of pregnancy and childbirth? How were women and adolescent girls specifically targeted for protection and treatment during pregnancy and childbirth complications?
• How successful were the efforts in protecting children under five and pregnant/lactating women from acute malnutrition?
• How effective were the GBV response services in providing support to adolescent girls and women who experienced harm?
• Were age-appropriate and culturally acceptable psychosocial support (PSS) activities implemented? If so, what were they?
• What evidence-based advocacy strategies were employed to engage duty-bearers and decision-makers?
• To what extent were policies and systems strengthened as a result of the advocacy efforts?
• How well were emergency and preparedness response mechanisms delivered in a timely and flexible manner?
• What strategies were employed to strengthen the overall emergency response capacity?
• How effective were the program's accountability and feedback mechanisms in ensuring beneficiary participation, promoting inclusivity in implementation, and improving program quality at all stages, including considerations for age, gender, and disability? What adaptations are necessary to further engage communities effectively throughout the project cycle?
• What key challenges (internal and external) to implementation were encountered and how effectively did the project respond/manage these? What can the project learn from these challenges?

Efficiency:

• What measures had been put in place to ensure efficient utilisation of resources during implementation?
• Were adequate human and financial resources applied to delivering project outcomes? How efficient was the communication and joint planning/implementation between IRC, MOH, MOSAL and other relevant line Ministries, as well as external coordination with other stakeholders through Technical Working Groups?
• Were outputs delivered in a timely manner?
• The type of delivery mechanism used during the EPRM responses – was this the most efficient use of resources and preferable/accessible for the targeted community members?
• How satisfied were the beneficiaries with the EPRM process (time taken to respond, selection criteria, and the quality of assistance)?
• To what extent did the EPRM coordinate with other emergency mechanisms (Rapid Response Mechanism (RRM) and Cash Consortium of Yemen (CCY)?

Impact:

• What real difference has the project made to the programme participants , taking into account the gender dimension?
• Which other factors contributed to the changes that were generated, and to what extent can the changes be attributed to the project activities?
• Are there any ill effects or unplanned impacts because of this project? (for example, protection-related issues)

Sustainability:

• What is the likelihood of the continuation of benefits of the program components after the program has been completed?
• To what extent did the programme utilize established institutions/mechanisms to ensure sustainability at the end of the project?
• To what extent were relevant partnerships/capacity developed to ensure sustainability?
• Was an exit strategy developed to ensure sustainability?

Scope of consultancy

The Consultant will design an appropriate evaluation methodology based on their understanding of the expectations of the terms of reference. After consolidating the evaluation findings, the Consultant is also expected to conduct a learning meeting with IRC staff. The evaluation should be utilization-focused as IRC intends to use the findings to inform current and future programming. The technical proposal should provide a detailed methodology for this assignment. Key activities should include, but are not limited to:

• Drafting data collection tools
• Developing the data collection methodology
• Developing the data analysis plan
• Conducting a learning meeting with IRC staff
• Presenting evaluation findings to IRC staff
• Production of the evaluation report

Reporting Arrangements

The Consultant will report to the MEAL Coordinator in close collaboration with the Technical Coordinators and Managers.

Duration of assignment

The consultancy must not exceed 40 working days and should take place starting from mid-January 2025.

DELIVERABLES

1. The Consultant will produce and submit an inception report detailing the evaluation design and methodology, tools, and instruments to be used in this evaluation within five days of signing the contract/agreement. The inception report will be submitted to IRC for review and presented virtually for feedback. IRC will approve the final inception report after submission before continuing any activities.
2. The Consultant will produce and submit a draft evaluation report for review and virtually present it to IRC for comments and validation. The outline for this delivery could be as agreed in the inception report and should meet the evaluation objectives as indicated in this TOR.
3. The Consultant will conduct a virtual learning meeting with IRC staff.
4. The Consultant will develop a final report, incorporating feedback from IRC. The Consultant is accountable for maintaining the requirements for the final report's content, format, and length, overall quality, and agreed timelines. The final report should incorporate all comments from IRC.

Intended Users

The intended users for the evaluation will be:

• All interested parties in IRC, including the Senior Management Team, MEAL Department, Grants Management, Technical Coordinators, field staff and Advisors.
• EPRM Partners.
• Representatives of DG ECHO as the donor that has funded the project.

Evaluation Roles and Responsibilities

IRC will:

• Facilitate engagement with the IRC staff and other key stakeholders.
• Provide all necessary project documents and contacts of relevant stakeholders.
• Review the Consultant's proposal, tools, and evaluation report.

The Consultant will:

• Be responsible for all aspects of the entire evaluation process, including evaluation preparation, data collection, analysis, and report writing.
• Be responsible for organizing a learning meeting with the IRC staff.
• Be responsible for paying any tax or other fees related to this assignment.
• Be responsible for their working tools such as computer and data analysis software.
• Ensure the highest research ethical standards and transparency are upheld during the evaluation.
• Ensure that the time scheduled is adhered to.
• Ensure that the consultancy fee covers ALL the consultancy-related costs, including internet costs, communication, etc.
• Abide by IRC Yemen working days, i.e., Sunday, Monday, Tuesday, Wednesday & Thursday, and ensure all communication is done during those weekdays.