Consultant, Final Evaluation of the EU4Health Project

Organizational Context

Purpose: This final evaluation will assess all aspects of performance of the Provision of Quality and Timely Psychological First Aid to People Affected by Ukraine Crisis in Impacted Countries (hereinafter EU4Health) project against its intended objectives and outcomes. The evaluation aims to provide findings and recommendations to inform future programming of the IFRC, Red Cross and Red Crescent National Societies, and other stakeholders.

Commissioner: IFRC Regional Office for Europe

Audience: IFRC, RCRC National Societies, IFRC Reference Center for Psychosocial Support, European Commission Directorate-General for Health and Food Safety.

Consultant or Consulting Organisation: The evaluation will be conducted by an independent consultant or consulting organization, consisting of a lead evaluator and up to two team members. They should not be directly involved in the project or have any potential conflict of interest.

Duration: Up to 60 days (including fieldwork and report drafting).

Period of Engagement: from March 2025 to September 2025

Deadline for Submission of Final Evaluation Report: Within a month after finalization of data collection. Date to be confirmed.

Evaluation Location: Primarily remote, with potential field visits to selected countries involved in the project (up to three field visits will be carried out to the sample countries and the rest will be engaged remotely).

Background

The EU4Health project, funded by the European Commission Directorate-General for Health and Food Safety (hereinafter DG SANTE), was initiated on 8 June 2022 and will run until 31 October 2025, to ensure that sufficient psychological first aid, mental health and psychosocial support were provided to displaced people from Ukraine, via helplines and other service platforms. The project has a budget of EUR 31,190,000. It aspires to achieve four expected results: (1) Increased capacities are in place through the training of RCRC staff and volunteers, frontline responders, and other professionals to provide Psychological First Aid (PFA) and Mental Health and Psychosocial Support (MHPSS) to affected people; (2) PFA helplines and other service platforms are available for all people displaced from Ukraine in need as well as for host communities; (3) The well-being of RCRC staff, volunteers, and frontline responders involved in the response activities is supported, and (4) cooperation on Psychological First Aid activities between all relevant partners is strengthened.

The project activities are being implemented by the Red Cross Societies in 28 countries: Belgium, Bulgaria, Croatia, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Lithuania, Luxembourg, Moldova, Montenegro, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, and Ukraine, in collaboration with the IFRC Regional Office for Europe and IFRC Reference Center for Psychosocial Support.

Out of the 28 national societies, 26 are actively implementing and reporting while 2 (Moldova, Norway) are participating in the project experience exchange, networking and coordination activities, with no obligation to report.

Job Purpose

Evaluation Purpose and Scope

1. Primary Objectives:

• Evaluate the impact of mental health and psychosocial support activities including the psychosocial first aid (as a set of skills and intervention) and es provided to people displaced from Ukraine through helplines and other service platforms established or scaled up within the project.
• Assess the extent to which the MHPSS capacity strengthening initiatives such as trainings and technical support within the project have increased the ion and implementation, and institutional capacities of National Societies' to provide quality MHPSS services.
• Assess the enhanced formal institutional frameworks, including strengthened policies and newly developed strategies, e.g. MHPSS strategy, caring for staff and volunteers policies, established to improve organizational effectiveness.
• Evaluate the impact of the MHPSS initiatives implemented within the project to support the mental health and well-being of targeted first responders, including Red Cross staff and volunteers.
• Assess the effectiveness of coordination mechanisms between National Societies, IFRC, and other MHPSS actors, including the public authorities, civil society actors, and others.
• Identify the key internal and external challenges and barriers encountered during project implementation, as well as the strategies and approaches employed to mitigate or address them.
• Provide recommendations for similar future initiatives, including lessons learned on operational aspects such as programme management, coordination, reporting, monitoring and evaluation, and implementation strategies for the effective delivery of MHPSS services to the affected populations.

2. Secondary Objectives:

• Assess the extent to which the project implementation and the technical support provided by the IFRC, including the IFRC Psychosocial Centre, to the implementing Red Cross National Societies has contributed to the improved institutional capabilities of the National Societies, in the areas of program management, program quality assurance, data collection and management, communications, and financial management.
• Document best practices and lessons learned for wider dissemination among IFRC and external partner organisations.

Evaluation Methods and Process

The evaluation will be conducted in adherence to IFRC evaluation criteria, largely adopted from the OECD/DAC criteria, and involve both qualitative and quantitative data collection.

Methods:

• Desk Review: Analysis of project documentation, Monitoring and Evaluation plan, progress reports, assessments, Caring for Staff and Volunteers baseline surveys, Organizational Capacity Assessment data, and other relevant documents.
• Key Informant Interviews: Interviews with IFRC staff, National Society teams, health authorities, and other stakeholders.
• Focus Group Discussions: Group interviews with volunteers, frontline responders, and displaced communities.
• Field Visits: Visits to selected project countries to interact directly with the project recipients and the implementing National Societies. Preliminary criteria for the selection of NS, to be refined during the evaluation's inception phase, include the following: The NS's level of engagement in MHPSS programming prior to the project; the duration and timing of the NS's involvement in the project; the geographical scope and reach of the NS's activities within the project.
• Quantitative Surveys: Structured surveys to collect data from participants, recipients, and stakeholders to quantify key outcomes and impacts of the project.

Job Duties and Responsibilities

The consultant or consulting organization will develop an inception report with a detailed methodology and data collection plan, to be reviewed and approved by the IFRC Evaluation Management Team (EMT). The consultant will be responsible for all data collection and analysis activities. The IFRC will provide necessary support, including mapping key informants and facilitating scheduling the interviews and focus group discussions as needed.

An EMT will be set up to manage and support the evaluation. The EMT will oversee and support the implementation of the evaluation process, including the quality assurance and approval of deliverables by the commissioner and recruitment of the evaluator(s).

Evaluation Criteria and Questions:

1. Relevance:

• Was the project designed to meet the needs of the affected population and the operational contexts?
• How did the project ensure meeting the evolving needs of the target populations as the crisis progressed?

• To what extent did the project complement other interventions (including by public authorities, other humanitarian organizations, and IFRC partners)?
• To what extent did the project facilitate the integration of MHPSS into other interventions implemented by the NSs targeting the same population, specifically individuals displaced from Ukraine? Were there synergies and consistencies among the different components of the project?

• To what extent were the intended project goal, outcomes and outputs of the project achieved?
• What factors contributed to or hindered the achievement of the project's objectives?

• To what extent were the resources (funds, available expertise, time) used in an  efficient manner to achieve the project's objectives?
• What challenges impacted the efficiency of the project’s implementation?
• To what extent could the same results have been achieved with fewer resources?
• To what extent the initial project design impact the efficiency of implementation?

• Was the target population reached, especially the most vulnerable groups, such as children, older adults, individuals with disabilities, single-parent families, and those with pre-existing mental health conditions?? Were there any gaps in coverage?

• What were the significant changes (positive and negative) brought about by the project?
• To what extent has the project contributed to the perceived well-being of the targeted populations (both intended and unintended impact)? (Where applicable the data collected at the baseline assessment phase will be used for comparison)

• To what extent are the observed benefits and outcomes of the project, as identified through the evaluation, likely to be sustained beyond its completion? What sustainable strategies have been implemented and what plans are in place for the phased transition or conclusion of project activities?
• What are the National Society’s intentions and plans for sustaining MHPSS programming beyond the project?

• Were mechanisms in place to ensure accountability to recipients and other stakeholders? Which mechanisms?
• To what extent were affected populations consulted in the planning, implementation, and monitoring of the project? And how , i.e. what methods were used?
• To what extent were the activities and outputs of the project consistent with the overall goals and objectives?

Job Duties and Responsibilities (continued)

Evaluation Deliverables

• Inception Report: A scoping document outlining proposed methods, data collection tools, and timelines.
• Debriefing and Validation of Preliminary Findings: Presentation of preliminary findings to IFRC and National Societies.
• Draft Report: A draft evaluation report detailing key findings, recommendations, and lessons learned.
• Final Report: The final report will contain a short executive summary (no more than 1,000 words) and a main body of the report (expected to be no more than 10,000 words) covering the background of the intervention evaluated, a description of the evaluation methods and limitations, findings, conclusions, lessons learned, and recommendations. Recommendations should be specific, feasible and evidence based. It will contain appropriate appendices, including a copy of the ToR, cited resources or bibliography, a list of respondents, and any other relevant materials. The report will also be made available to RC/RC stakeholders and external partners involved in the response via the IFRC Evaluation Databank, in a format agreed by all stakeholders involved.

The final report will be submitted one week after the receipt of comments on the draft report and will have to be validated by the Commissioner. Once validated, the final report and the Management Response will be shared with the RC National Societies, IFRC senior management, wider IFRC stakeholders and following approvals, the report be posted on the IFRC Evaluation Databank.

Evaluation Timeline

Evaluation Quality and Ethical Standards

The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of the people and communities involved and shall abide by the “do no harm” principle. The review should be technically accurate and reliable, conducted in a transparent and impartial manner, and contribute to organizational learning and accountability. Therefore, the evaluation team should adhere to the evaluation standards and applicable practices outlined in the IFRC Evaluation Framework. The IFRC Evaluation Standards are:

Education

• Utility: Evaluations must be useful and used.
• Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost-effective manner.
• Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with particular regard to the welfare of those involved in and affected by the evaluation.
• Impartiality & Independence: Evaluations should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders.
• Transparency: Evaluation activities should reflect an attitude of openness and transparency.
• Accuracy: Evaluations should be technically accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.
• Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate. Evaluation process should incorporate and amplify the voices of affected populations, highlighting their perspectives to assess the impact of MHPSS interventions and identify improvements and future implementation strategies.
• Collaboration: Collaboration between key operating partners in the evaluation process improves the legitimacy and utility of the evaluation.

It is also expected that the review will respect the seven Fundamental Principles of the Red Cross and Red Crescent Movement: 1) Humanity, 2) Impartiality, 3) Neutrality, 4) Independence, 5) Voluntary service, 6) Unity, and 7) Universality. Further information can be obtained about these Principles at https://www.ifrc.org/who-we-are/international-red-cross-and-red-crescent-movement/fundamental-principles.

Roles of Evaluation Stakeholders

1. IFRC (Project Commissioner)

The IFRC serves as the commissioner of the evaluation, responsible for overseeing its overall planning, management, and implementation. The specific roles include:

• Defining the scope and objectives of the evaluation in alignment with project goals.
• Allocating resources and ensuring the timely provision of logistical and financial support for the evaluation process.
• Selecting and contracting the external evaluators or evaluation team.
• Ensuring adherence to ethical standards and principles in conducting the evaluation.
• Facilitating communication and coordination among key stakeholders involved in the evaluation.
• Reviewing and validating evaluation deliverables, such as the inception report, methodology, and final report.
• Disseminating the evaluation findings to relevant stakeholders and integrating lessons learned into future programming and decision-making processes.

2. IFRC Reference Center for Psychosocial Support

The IFRC Reference Center for Psychosocial Support will play a key role in providing technical expertise related to MHPSS throughout the evaluation process. Responsibilities include:

• Contributing to the design of the evaluation methodology to ensure it adequately captures the impact of MHPSS interventions.
• Providing input into the development and refinement of data collection instruments, ensuring their relevance and alignment with MHPSS best practices.
• Supporting the interpretation of evaluation findings related to MHPSS interventions and ensuring technical accuracy in reporting.
• Advising on ethical considerations specific to MHPSS evaluations, particularly in engaging with vulnerable populations.

Experience

3. Implementing National Societies

The implementing National Societies will play a critical facilitative role in ensuring access to essential data and key respondents. Their responsibilities include:

• Facilitating access to project recipients, staff, volunteers and other key respondents for data collection.
• Providing secondary data and relevant documentation to support the evaluation process.
• Coordinating logistics for field visits and interviews within their operational contexts.
• Supporting the evaluation team in navigating cultural and contextual dynamics to enhance the accuracy and relevance of data collection.
• Offering insights and feedback based on their experiences in project implementation to enrich the evaluation's findings and recommendations.

Knowledge, Skills and Languages

Consultant’s Qualifications and Experience

The lead evaluator should have:

• 7-10 years of experience in evaluations, preferably in public health or mental health and psychosocial support.
• Master’s degree in international development, social sciences, humanitarian assistance or other related fields.
• Basic knowledge and training in monitoring and evaluation of MHPSS interventions.
• Strong knowledge of Red Cross Red Crescent Movement and the ability to provide strategic recommendations.
• Good knowledge of strategic and operational management of humanitarian operations and an ability to provide strategic recommendations to key stakeholders.
• Excellent writing and communication skills in English (reference of sample work required).
• Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions and to prepare well‐written reports in a timely manner.
• Experience in managing teams remotely and comfortable working with online platforms

Competencies, Values and Comments

The consultant will provide an independent, objective, and critical perspective, and will be the primary author of the evaluation report. They will be hired through a transparent recruitment process, based on professional experience, competence, ethics and integrity for this evaluation.

The consultant should take all reasonable steps to ensure that the piece of work is designed and conducted to respect and protect the rights and welfare of the people involved and to ensure that the evaluation is technically accurate and reliable, is conducted in a transparent and impartial manner, and contributes to better programming, organizational learning and accountability.

Fees and Payment Scheme

Payment shall be made upon satisfactory completion of work in accordance with the ToR. The following shall be the payment scheme to the Consultant and condition for the liquidated damages:

• 20% of the total amount after submission and approval of the inception report.
• 80% of the total amount after submission and approval of the final report and the delivery of final presentations of the review findings.

Note: The consultant is solely responsible for the payment of taxes and other lawful fees that may be incurred from this contract.

Application Procedures

Interested evaluation teams should submit their application materials to hr.europe@ifrc.org with ‘EU4Health Final Evaluation’ written in the Subject Line. The application include:

• Curriculum vitae of the lead evaluator and team members
• A one-page expression of interest
• A technical proposal, including the detailed budget and timeline
• Contact details for three professional referees
• Two samples of similar work previously completed.

For any questions regarding this evaluation, please contact hr.europe@ifrc.org