TERMS OF REFERENCE FOR:

Final evaluation of the DEC Ukraine Humanitarian Appeal Phase 2b project

Country(ies) covered: Romania

Project dates: 1 September 2023 – 31 August 2025

Overall objective of evaluation: Determine the success of the Phase 2b project with the level of achievement of project objectives and generate learning to improve Red Cross` preparedness for response to potential population movement crises in Romania

1. BACKGROUND AND RATIONALE

Following the escalation of the international armed conflict in Ukraine on 24 February 2022, it is estimated that around 8 million people, including third-country nationals, have crossed the border into Romania from Ukraine and Moldova by 31 December 2024. As of January 2025, UNHCR has recorded 179,822 Ukrainian refugees in Romania with 192,560 people registered for temporary protection or similar national protection schemes (UNHCR, Operational Data Portal, Ukraine Refugee Situation, Romania)

Due to insufficient government support and the difficulty for displaced Ukrainians to find jobs and affordable housing in Romania, many are running out of savings. This has made their needs more urgent and severe.

The DEC Phase 2 b project aiming to address the effects of the population movement crisis in Romania was designed by RRC, IFRC and BRC and was launched on 1 September 2023 with the end date of 31 August 2025.

Project target population: Displaced people from Ukraine and host communities.

OBJECTIVES AND FOCUS OF THE EVALUATION

The evaluation focus is informed by the IFRC Evaluation Framework, BRC Evaluation Policy, and the evaluation topics prioritised by DEC for the Ukraine response: localisation of humanitarian action, quality of partnerships, accountability to affected populations (AAP).

In consultation with the implementation stakeholders as main users of the evaluation results BRC has defined the following objectives of the DEC Phase 2b project final evaluation:

Objective 1: To make a judgement about the project`s overall performance and the level of achievement of its objectives in compliance with the Core Humanitarian Standard (CHS).

Objective 2: To assess the extent to which the community has been engaged and how the project has been accountable to the affected population, generating learnings for the future.

Objective 3: To determine the extent to what the response was led by the Romania Red Cross branches and to develop evidence-based recommendations on enhancing a locally led action.

To meet the objectives of the final Phase 2b project evaluation, BRC will collaborate with the evaluation consultant and stakeholders to collect and analyse evidence and answer the evaluation questions related to complementary evaluation criteria adopted by BRC for comprehensive evaluation of humanitarian work.

While the evaluation criteria and questions will be finalised later in collaboration with the evaluation consultant, BRC organizationally adopts the ten criteria listed below and puts the emphasis on evaluation of achieving project objectives in adherence to the Core Humanitarian Standard (CHS) commitments:

Communities and people affected by crisis:

• Can exercise their rights and participate in actions and decisions that affect them.
• Access timely and effective support in accordance with their specific needs and priorities.
• Are better prepared and more resilient to potential crises.
• Access support that does not cause harm to people or the environment.
• Can safely report concerns and complaints and get them addressed.
• Access coordinated and complementary support.
• Access support that is continually adapted and improved based on feedback and learning.
• Interact with staff and volunteers that are respectful, competent and well-managed.
• Can expect that resources are managed ethically and responsibly.

1. Relevance & Appropriateness:

To what extent did the targeted needs of the project correspond to actual and evolving priority needs of target population?

To what extent has the assistance to Ukrainian refugees and host population been delivered in a timely manner?

To what extent has the community engagement approach (communication, participation and feedback loops) been relevant to Ukrainian refugees and local host communities and appropriate to the context of population movement from Ukraine to Romania?

To what extent has the complaints mechanism been relevant and appropriate to the context of Ukraine crisis and population movement to Romania?

2. Coherence:

To what extent do the policies and strategies of the RRC, BRC, and IFRC align with the broader humanitarian response frameworks, such as those set by the Government of Romania and UNHCR, and how has this alignment (or lack) impacted the programme?

Were RRC, BRC and IFRC policies and strategies integrated with other regional strategies for refugee support?

Was the project aligned with the country-level coordination architecture to ensure complementarity with the response of other actors to population movement in Romania?

3. Connectedness:

Were short-term emergency activities being implemented in a way that considered longer-term and interconnected factors such as localisation and community resilience?

To what extent have the Romania Red Cross branches had ownership of resources, analysis, and decision-making within the project?

To what extent has the management of human resources and support to local Romania Red Cross branch staff been appropriate?

4. Coverage:

To what extent were different refugee and host, rural and urban, population groups included in or excluded from the project intervention?

Were vulnerable groups of the host community and refugee newcomers able to access programs and services provided under the project?

To what extent has the community engagement actions (communication, participation and feedback loops) been implemented and effective with both Ukraine refugee and host populations?

5. Effectiveness:

To what extent has the intervention achieved its objectives/intended outcomes stated in the Phase 2b project plan document?

To what extent has the project addressed identified needs of Ukrainian refugees and host communities?

To what extent can the achievement of results be verified by systematically collected data?

To what extent can changes be attributed to the project?

To what extent have changes been captured using participatory methods?

What unintended outcomes (positive and negative) were produced at community, Romania Red Cross Headquarters and branch levels?

6. Efficiency:

How well have project resources been used?

To what extent has the use of resources been accounted for?

To what extent have local staff and volunteers` experience and competencies been adequate to implement the project?

How effective has communication between BRC, IFRC, Romania Red Cross HQ and local branches been?

Were the roles and responsibilities of BRC, IFRC, Romania Red Cross HQ and branches clearly defined and understood?

To what extent has the response improved over time based on feedback and learning?

What were the challenges to action community feedback and learning?

7. Impact:

How has the project contributed to social cohesion among refugee and host community groups in Romania?

How has the project contributed to community resilience among refugee and host communities in Romania?

To what extent has the project contributed to a system level change for Ukrainian refugees, host communities, Romania Red Cross branches?

To what extent have refugees and host community been empowered to take ownership of project activities and outcomes?

To what extent have branch capacities been strengthened by the project?

Can any negative effects on branches observed be attributed to the project? Which ones and to what degree?

8. Sustainability:

Will the benefits to Ukrainian refugees, host populations and Romania Red Cross branches last in time?

Which Romania Red Cross branch capacities are likely to be sustained beyond the Phase 2b project?

How effective and realistic is the exit strategy of the project?

9. Equity:

To what extent have project activities promoted inclusive participation and representation of the diverse host community and Ukrainian refugees?

To what extent has the distribution of resources and benefits responded to and reduced the disparities as well as addressed inequalities in both refugee and host communities?

Have disaggregated data been collected and used for analysis by project implementers?

10. Protection and safety:

How adequately did the project safeguard the wellbeing, rights, and physical or emotional safety of individuals and refugee and host communities concerned?

To what extent has the complaints mechanism been implemented and effective for Ukraine refugee and host community program participants?

How effective has the signposting or referral system been for host and refugee program participants?

EVALUATION PROCESS AND METHODOLOGY

The evaluation process will be organized around several phases with regular debriefings:

Inception call to introduce structure and composition of both evaluation management and evaluation implementation teams and clarify roles and responsibilities and the timeframe according to this Terms of Reference (ToR).

Secondary data analysis phase– to provide the evaluation team access to available project documentation and allow them to formulate their proposal for the evaluation approach.

Inception report phase – to collaboratively finalise the methodology details and data collection tools, evaluation deliverables and the workplan.

Primary data collection and analysis phase – to gather and analyse the evidence per each evaluation question during field visits and remotely.

Preliminary findings presentation – to discuss preliminary results and recommendations.

Draft evaluation report phase – to systematize and organize evaluation narrative and annexes.

Final evaluation report with annexes – to work on feedback and finalise the evaluation narrative.

Final evaluation results presentation – to communicate final evaluation conclusions and recommendations to stakeholders

The evaluation methodology will be finalised by the evaluation consultant/team and BRC evaluation management team in close collaboration. The mixed methods approach with qualitative and quantitative data collection and analysis will be followed:

1. Desk research to review secondary data: project documentation such as Phase 2b plan and interim reports, output table, budget, risk register, and other available sources of information such as project implementation guidelines and monitoring tools, project agreements with branches, complaints procedure, needs assessment, field visits, PDM reports, feedback analysis reports, case studies, learning events reports, meeting notes, informational materials, etc.

2. Survey with project participants:representative phone survey with end users of humanitarian assistance to complement available project data.

3. Key informant interviews: with staff of BRC, IFRC, RRC Headquarters and branches, and government service providers as relevant.

4. Focus group discussions with Red Cross volunteers and community representatives*:* to generate insights and learning on the particular areas of focus*.*

5. Field visits: to make real-time observations about organization of activity spaces, information provision on the site, and services delivery.

TIMEFRAME AND DELIVERABLES

The evaluation will take place in the period from June to August 2025 with key deliverables and indicative due dates as follows:

1. Draft inception report (5 days) – June 16 2025

2. Inception report (1 day) – June 24 2025

3. Preliminary findings presentation (10 days) – July 21 2025

4. Draft evaluation report (En) (6 days) – August 4 2025

5. Final evaluation report (En, Romanian) (3 days) – August 18 2025

6. Presentation of the final evaluation conclusions and recommendations (1 day) – August 19 2025

   Total working days: 26

All products arising from this evaluation will be owned by the RRC, IFRC and BRC. The evaluator(s) will not be allowed, without prior authorization in writing, to present any of the analytical results as his/her own work or to make use of the evaluation results for private publication purposes.

SAFEGUARDING AND ETHICAL CONSIDERATIONS

The following eight evaluation standards summarize key principles that guide how evaluation is conducted by the British Red Cross:

Utility – Evaluations must be useful and used.

Feasibility – Evaluations must be realistic, respectful and managed in a sensible, cost-effective manner.

Ethics & Legality – Evaluations must be conducted in an ethical and legal manner, with particular regard for the welfare of those involved in and affected by the evaluation.

Impartiality & Independence – Evaluations should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders. Independence refers to external evaluations, for which evaluator/s should not be involved or have a vested interest in the intervention being evaluated.

Transparency – Evaluations should be conducted in an open and transparent manner.

Accuracy – Evaluations should be technically accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.

Participation – Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate.

Accountability – Evaluations should be conducted upholding accountability standards by adequately documenting the evaluation process and products, aligning evaluation practice with an equity approach, and with the development of recommendations that are detailed and actionable.

It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality.

CONSULTANT PROFILE

Evaluator(s), with knowledge of local context who will provide an independent, objective perspective as well as technical experience on evaluations, will be hired to conduct the evaluation. The evaluation team lead will be the primary author of the evaluation report. S/he will not have been involved or have a vested interest in the IFRC/RRC operation or context being evaluated, and will be hired through a transparent recruitment process, based on professional experience, competence and ethics and integrity for this evaluation. The evaluation consultant will report on progress or challenges to the evaluation management team. The evaluator should have the following characteristics:

• Demonstrable experience in leading evaluations of humanitarian programmes responding to refugee or displacement crises and preferably previous experience of conducting an evaluation in Ukraine neighbouring countries.
• Knowledge of humanitarian assistance in cash, relief, health, MHPSS, and livelihoods sectors.
• Proficiency in English, Romanian and/or Ukrainian languages.
• Field experience in the evaluation of humanitarian or development programmes, with prior experience of evaluating RCRC programmes desirable.
• Strong analytical skills and ability to clearly synthesize and present findings, draw practical conclusions, make recommendations and to prepare well-written reports in a timely manner (examples of previous work).
• Experience in quantitative and qualitative data collection and data analysis techniques, especially in migration and displacement response operations.
• Demonstrated capacity to work both independently and as part of a multi-discipline, multi-national team.
• Ability to work within tight deadlines and manage with available resources.
• Availability for the time period indicated.

Additionally, the following overall skills and expertise are considered central to the successful delivery of the evaluation:

• Process Management: Effective process management skills are crucial to ensure that the evaluation is conducted efficiently.
• Evaluation Methodologies: A strong understanding of evaluation methodologies, including both quantitative and qualitative approaches, is essential. This includes skills in designing surveys, conducting interviews, and collecting and analysing data.
• Understanding of Localisation of humanitarian aid: British Red Cross wants its partners to be stronger, more sustainable and empowered to lead upon the delivery of effective and relevant services and action, thereby improving the support they provide for vulnerable communities.
• Stakeholder Engagement: Effective communication and interpersonal skills are essential for working with various stakeholders, including Red Cross leadership, branch managers, volunteers, and community members to gather input and assess project accountability.
• Report Writing: The ability to create clear, concise, well-structured and evidence-based reports is crucial for conveying findings and recommendations to Red Cross leadership. A consultant should be able to present complex information in an accessible manner.
• Accountability and Transparency: Knowledge of accountability and transparency principles in humanitarian and development work, ensuring that the evaluation holds Romania Red Cross and partners accountable to the communities and donors.
• Cultural Sensitivity: Awareness of and respect for cultural differences and the ability to navigate diverse cultural contexts when interacting with community members.
• Legal and Ethical Considerations: Understanding the legal and ethical responsibilities of a consultant, especially when dealing with sensitive volunteer and community information, is critical.

How to Apply:

Interested applicants should submit their expression of interest to the following emails: JoannaReid@redcross.org.uk and IrynaKysliakovska@redcross.org.uk by 25 May 2025. In the subject line, please state the following details (DEC2b Project Final Evaluation Consultancy – Last Name, First Name).

Application materials should be submitted in English which will include:

1. Curriculum Vitae (CVs) of the consultant and all team members, if relevant.
2. Cover letter clearly summarizing your experience as it pertains to this evaluation.
3. 2-3-page approach paper explaining proposed evaluation questions, evaluation methodology with proposed number of KIIs/FGD/surveys/visits
4. Two samples of previous written by consultant final humanitarian project evaluation reports, preferably for Ukraine neighbouring countries/Europe
5. Two professional references, most recent clients/employers
6. Financial proposal: A detailed financial proposal outlining daily fees for each team member and all expected reimbursable costs

Application material is non-returnable, and we thank you in advance for understanding that only short-listed candidates will be contacted for the next step in the application process. Please take note that incomplete applications will be rejected.

In the British Red Cross we will not tolerate any form of misconduct, including sexual harassment, exploitation and abuse. We have a code of conduct in place and are committed to creating a culture of integrity in the organisation where misconduct is not tolerated, situations of abuse are quickly investigated and perpetrators are dealt with effectively.