Cash for Shelter Response Evaluation ‎

Organizational Context

The war in Ukraine continued to escalate throughout 2023, with relentless strikes and ground fighting destroying lives, livelihoods and part of the country’s civilian infrastructure, increasing humanitarian needs. Although attacks on energy infrastructure that had driven needs during the winter months decreased in February 2023, homes, schools, water systems and hospitals continued to be attacked in the following months. From March to June 2023, fighting intensified in front-line communities making humanitarian assistance increasingly difficult to provide in an area where needs are acute. In the south, attacks on port and grain infrastructure since the termination of the Black Sea Initiative in July 2023 created additional challenges for agricultural exports from Ukraine. (UNOCHA sitrep). It is estimated that by the end of 2023 the conflict had affected 24 million people in Ukraine, while the number of people in need of humanitarian assistance increased from 2.9 million before 24 February 2022 to 17.6 million in January 2023. Humanitarian organizations in Ukraine have dramatically scaled up their operations, reaching over 15.8 million people through the end of 2023, including 1 million in areas under the temporary military control of the Russian Federation, where access was a challenge. Furthermore, an estimated 8.3 million people, including IDPs, non-displaced conflict-affected people and returnees needed Shelter and Non-Food Items (SNFI) support in 2023. The majority were in the central, southern and eastern oblasts, including in areas near active hostilities and in areas under the temporary military control of the Russian Federation. While the impact of the crisis crosses all sectors, the key challenge for winter preparedness is the damage to housing and the energy sector. In the summer of 2022, URCS, IFRC and MCTD signed an agreement to provide cash assistance to households hosting IDPs throughout Ukraine, scaling up the programme and becoming the only partner of the Ministry for its implementation covering 23 Oblasts1 from September 2022 till August 2023 (the cash for shelter project) and providing UAH 450 per month for each IDP hosted, (approximately CHF 12). On September 2022, the parties signed an amendment to increase the cash transfer amount to UAH 900 (approximately CHF 24) for the winter months effective from October 2022 to March 2023 to ensure that affected populations have access to warm, safe, and dignified living conditions, with access to essential social services.

Job Purpose

The project evaluation aims to provide organizational learning and improvements that can inform future cash interventions in Ukraine and identify key lessons, recommendations, and highlight potential areas for strengthening response capacities of implementing partner national societies and local communities. While the exact evaluation criteria will be determined later, as a minimum it will cover the effectiveness, impact and sustainability of the project.

Job Duties and Responsibilities

The cash for shelter project evaluation process aims to assess the implemented activities’ effectiveness, impact, and sustainability. It will analyse the program's effectiveness in addressing the shelter needs of internally displaced persons (IDPs) by supporting households financially to be able to host IDPs in Ukraine with the aim to cover increased utility costs. Additionally, the evaluation will focus on reviewing the program's efficiency in terms of administrative processes and cost-effectiveness, while ensuring transparency and accountability in fund utilization. The assessment will measure the program's ability to provide short-term support to the affected population and identify key learnings applicable for future cash for shelter initiatives. By identifying any gaps or challenges in the program's implementation, the evaluation will provide actionable recommendations and learnings that can inform future programs, emphasizing improvement to financial management practices and adherence to international Core Humanitarian Standards (CHS). Communities and people affected by crisis: • Receive assistance appropriate and relevant to their needs. • Have access to the humanitarian assistance they need at the right time. • Are not negatively affected and are more prepared, resilient and less at-risk as a result of humanitarian action. • Know their rights and entitlements, have access to information and participate in decisions that affect them. • Have access to safe and responsive mechanisms to handle complaints. • Receive coordinated, complementary assistance. • Can expect delivery of improved assistance as organisations learn from experience and reflection. • Receive the assistance they require from competent and well-managed staff and volunteers. • Can expect that the organisations assisting them are managing resources effectively, efficiently and ethically. To achieve these objectives, the team will coordinate with technical staff to refine strategies for collecting evidence on: a. Relevance/appropriateness - Effectiveness in identifying vulnerable households - Fairness of beneficiary selection - Quality assurance mechanisms - Appropriateness of cash for shelter - Alignment with URCS guidelines b. Coverage - Reach across conflict-affected areas - Potential improvements to distribution methods c. Efficiency/effectiveness/accountability - Coverage of utility costs - Transparency mechanisms - Feedback systems and community concerns d. Impact - Evidence of reduced burden on host communities. e. Connectedness/Sustainability - Enhanced URCS capacity for cash programs - Effectiveness of Movement partners' support f. Unintended Consequences/Lessons Learned - Expected and unexpected impacts - Program design and implementation lessons NOTE: The consultant will refine these questions with the management team. Limitations Significant time has passed between the cash distributions of the project and the proposed evaluation. Consequently, it is highly unlikely that cash recipients will remember or be able to distinguish between cash received as part of the program (March 2023 to August 2023) and payments received either before or after this time. Therefore, it will not be necessary to conduct interviews with direct beneficiaries. Rather, the evaluation will rely on analysing existing data, post-distribution monitoring survey with hosts, focus group discussions reports with IDPs, and key informant interviews with people involved into the programme.

Job Duties and Responsibilities (continued)

Methodology The methodology will follow IFRC Evaluation Framework standards for planning, managing, conducting, and utilizing evaluations. A Review Board with members from URCS and Partner National Societies experienced in Ukraine's Crisis Response will guide the process and endorse findings. The evaluation consultant will use mixed methods including: - Desktop review of operation documents, organizational background, policies, SOPs, IFRC reports, and secondary data (PDM, learning reviews, FGDs, surveys) - Key informant interviews with institutional stakeholders - Focus group discussions with key participants The inception report must detail a proposed methodology including a consultant-recommended sampling method. The evaluation consultant is expected to deliver the following outputs: • Inception Report: The inception report will be a scoping exercise for the review/evaluation and will include the proposed methodologies, data collection and reporting plans with draft data collection tools such as interview guides, the allocation of roles and responsibilities within the team, a timeframe with firm dates for deliverables, and the travel and logistical arrangements for the team. • Debriefings / feedback to management at all levels: The team will report its preliminary findings to the EMT prior to leaving the country. This will include a presentation to key stakeholders as determined by the EMT, a detailed PowerPoint presentation and sufficient time for discussion and suggested edits. • Draft report: A draft report identifying key findings based on facts and including recommendations and lessons for the current and future operation, will be submitted by the evaluation consultant within a reasonable time (one week) of presenting the initial findings. • Final report: The final report will contain a short executive summary (no more than 1,000 words) and a main body of the report (no more than 10,000 words) covering the background of the intervention evaluated, a description of the evaluation methods and limitations, findings, conclusions, lessons learned and clear recommendations. Recommendations should be specific and feasible. The report should also contain appropriate appendices, including a copy of the ToR, cited resources or bibliography, a list of those interviewed and any other relevant materials. The final report will be submitted two days after receipt of the consolidated feedback from EMT. Details of the final report are outlined in the table below. The final evaluation report will be made public and will be published on the DEC and other websites, such as ALNAP and ReliefWeb. Review Board A review board with maximum three Movement partners and URCS DDG (Programmes) experienced in Ukraine crisis cash for shelter support will review, guide and endorse the report. Evaluation Management Team A six-member team will manage the evaluation, including URCS Head of QCA Department, BRC PMEAL and CEA delegate, BRC Program Manager, URCS Program Support Lead, and Head of PMER Unit. Evaluation Quality and Ethical Standards The consultant must design and conduct the evaluation to protect rights and welfare of participants while ensuring technical accuracy, reliability, and legitimacy. The IFRC Evaluation standards to be followed are: - Utility: Ensure evaluations are useful and used - Feasibility: Conduct realistic, diplomatic, cost-effective evaluations - Ethics & Legality: Maintain ethical standards and participant welfare - Impartiality & Independence: Provide comprehensive, unbiased assessment - Transparency: Maintain openness throughout the process - Accuracy: Ensure technical accuracy and sound methodology - Participation: Involve stakeholders meaningfully - Collaboration: Partner with key operators to improve legitimacy The evaluation must respect the seven Fundamental Principles of the Red Cross and Red Crescent.

Education

• University degree or equivalent • Post-graduate degree in public health, development, or other major post-graduate qualification in the relevant field

Knowledge, Skills and Languages

• Excellent written and spoken English and Ukrainian or Russian skills required, as well as excellent analytical and presentation skills

Competencies, Values and Comments

Consultant and Qualifications Consultant(s), An independent consultant with knowledge of local context will be hired to conduct the evaluation and serve as primary report author. The consultant must have no prior involvement or vested interest in the URCS operation being evaluated and will be selected through a transparent recruitment process based on professional experience, competence, ethics, and integrity. The consultant will report progress and challenges to the evaluation management team: • Demonstrable experience in leading evaluations of humanitarian programmes responding to refugee and displacement crises and preferably previous experience of conducting evaluation in Ukraine or impacted countries. • Knowledge of activities generally conducted by humanitarian organizations in the shelter response. • Field experience in the evaluation of humanitarian or development programmes, with prior experience of evaluating RCRC programmes desirable. • Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner (examples of previous work). • Experience in qualitative and quantitative data collection and data analysis techniques, especially in emergency and recovery operations. • Demonstrated capacity to work both independently and as part of a multi-discipline, multi-national team. • Ability to work within tight deadlines and manage with available resources. • Availability for the time period indicated. Additionally, the following overall skills and expertise are considered central to the successful delivery of the evaluation: • Process Management: Effective process management skills are crucial to ensure that the evaluation is conducted efficiently. • Evaluation Methodologies: A strong understanding of evaluation methodologies, including both quantitative and qualitative approaches, is essential. This includes skills in designing surveys, conducting interviews, and collecting and analysing data. • Change Management: Red Cross programs often involve organizational change. A consultant should understand change management principles and be able to facilitate the transition to new processes and systems effectively. • Stakeholder Engagement: Effective communication and interpersonal skills are essential for working with various stakeholders, including Red Cross leadership, branch managers, volunteers, and community members to gather input and assess project accountability. • Report Writing: The ability to create clear, concise, well-structured and evidence-based reports is crucial for conveying findings and recommendations to Red Cross leadership. A consultant should be able to present complex information in an accessible manner. • Accountability and Transparency: Knowledge of accountability and transparency principles in humanitarian and development work, ensuring that the evaluation holds URCS and partners accountable to the communities and donors. • Cultural Sensitivity: Awareness of and respect for cultural differences and the ability to navigate diverse cultural contexts when interacting with community members. • Legal and Ethical Considerations: Understanding the legal and ethical responsibilities of a consultant, especially when dealing with sensitive volunteer and community information, is critical. Available Data and Documentation The following documents will be made available to the consultant(s) to form the basis of the secondary data review and inform the Inception Report. 1. Final DEC phase 2 narrative report and output tracker. 2. Monthly relevant reports (reached dashboards). 3. FGD reports and PDM main findings dashboards. 4. Steering Committee - presentations, MoMs. 5. URCS Prykhystok introduction. 6. URCS Prykhystok webpage. 7. URCS OnePlan 2023 – 2025. 8. URCS Prykhystok on social media.