Purpose:

The Norwegian Red Cross (NoRC) together with the Ukraine Red Cross Society (URCS) seek a consultancy firm to evaluate the impact and effectiveness of the Mobile Health Units (MHUs) and identify key lessons and recommendations to improve ongoing and inform future health operations.

Audience

The main audience of the evaluation is URCS, Norwegian RC, German RC, French RC, Swedish RC and other Partner National Societies (PNSs) as well as the Norwegian Agency for Development Cooperation (Norad) as the donor. The results of the evaluation will constitute an important component in NoRC’s annual report to Norad.

Commissioners

This evaluation is being commissioned by Norwegian Red Cross in collaboration with the Ukraine Red Cross Society (URCS).

Duration

60 working days

Timeframe

June 2025 – October 2025

Methodology summary

Field visits (including Key Informant Interviews, Focus Group Discussions)

Review of primary and secondary data

Location

Ukraine: Dnipropetrovsk, Ternopil, Khmelnytskyi, Odesa, Sumy and Kharkiv oblasts

Application requirements

Proven expertise in public health.

Language requirements:

Ukrainian and English

The evaluation team should be composed of enumerators who are fluent in Ukrainian for the interviews. Review of secondary data requires being able to read Ukrainian. The report is to be written in English.

Background:

The ongoing armed conflict in Ukraine has had a profound impact on the country’s health and social care system, significantly reducing access to essential health and social services for many communities, especially in conflict-affected regions and regions with high number of internally displaced persons (IDPs) and people living in remote areas. Vulnerable populations, including elderly individuals, people with disabilities, those in rural or isolated areas, and IDPs, often face substantial barriers in accessing healthcare due to infrastructure damage, limited mobility, and resource constraints. Amidst these challenges, the Ukrainian Red Cross Society (URCS), with support from the Norwegian Red Cross (NoRC), has implemented a critical healthcare initiative through Mobile Health Units (MHUs) in Dnipropetrovsk, Khmelnytskyi, Ternopil and Odesa (through German Red Cross) regions. NoRC increased its support to URCS (and established a bilateral partnership in March 2022[1]) since the escalation of the conflict in February 2022 with funding from the Ministry of Foreign Affairs/ Norwegian Agency for Development Cooperation (Norad) and its own funds.

The following output on MHUs has been identified within the Outcome 1: Vulnerable and conflict affected people have improved access to quality health care[2]

Output 1.1: Improved access to primary health care services.

NoRC is only one of the many partners of URCS and is focusing on specific geographical areas whilst other partners support the same activities in other areas. NoRC is signing yearly agreement with URCS.

NoRC allocated the following funds to the URCS MHU program:

• 2022: 39,000,000 NOK.
• 2023: 40,000,000 NOK.
• 2024: 110,000,000 NOK.

Mobile Health Unit (MHU) Project

The MHU Project is designed to provide mobile healthcare services to underserved communities where fixed healthcare facilities may be lacking or overwhelmed. The MHUs are equipped with essential medical supplies and staffed by healthcare professionals trained to offer a range of services, including primary healthcare, maternal and child health services, health screenings, and health education. This mobile approach enables flexible, rapid response to health needs in shifting and hard-to-reach areas, especially critical in regions like Dnipropetrovsk, Ternopil, and Khmelnytskyi, where conflict and displacement have disrupted access to healthcare. The MHU Project also plays a significant role in health promotion, disease prevention, and mental health support, which are crucial for enhancing community resilience in times of crisis. More than 239,000 people received services from the 31 MHUs supported by NoRC (10 in Dnipropetrovsk, 9 in Khmelnytskyi, 7 in Ternopil, 5 in Odesa through the German Red Cross) as of Q3 2024. The MHUs use DHIS2 as a health management information system.

This evaluation will target the MHUs supported by NoRC as well as the MHUs in Kharkiv supported by French RC and those in Sumy supported by Swedish RC.

Mobile Health Units function as dynamic healthcare providers, delivering essential services to communities in need. Each MHU typically consists of a small, multidisciplinary team, including one doctor, two nurses, and often a psychologist or gynecologist based on the assessed needs of the target population. The selection of regions and specific areas within the oblasts to work is guided by needs assessments, prioritizing underserved, remote, and conflict-affected communities with limited access to healthcare facilities. The MHUs operate on a rotational schedule, ensuring consistent coverage across multiple locations within a designated area. They provide primary healthcare services such as medical consultations, treatment for common illnesses, health education, and follow-up care for chronic conditions. MHUs are equipped to dispense essential medications, offer psychosocial support, and address reproductive health needs through gynecological services. By maintaining close collaboration with local health authorities and tailoring their services to community-specific challenges, the MHUs play a critical role in bridging healthcare gaps and enhancing the overall well-being of the populations they serve.

MHUs are directly linked to Primary Healthcare Centers (PHCCs) and operate under the direct supervision of the PHCC director, ensuring alignment with local health strategies and priorities. The PHCC director plays a key role in coordinating the MHU schedule, determining the rotation plan, and specifying the priority areas and services based on the community’s health needs. This collaboration enables the MHUs to function as an extension of the PHCC, enhancing access to care in underserved areas while maintaining continuity with the broader healthcare system. The PHCC director also oversees the allocation of resources, supports the integration of MHU services with existing health programs, and ensures that critical cases requiring specialized care are referred back to the PHCC or other appropriate facilities. This structured relationship fosters synergy between mobile and fixed health services, maximizing the impact of healthcare delivery in vulnerable communities.

Evaluation purpose and scope

The purpose is to evaluate the performance and results of the Mobile Health Units (MHUs) program by focusing on the relevance, coherence, effectiveness, efficiency, impact, and sustainability criteria. In addition to identifying key lessons and recommendations based on progress to date, the exercise aims to assess the evolving needs of the targeted communities, providing insights to inform decision-making and the next phase of program design.

Given the complex environment in which these projects operate, an independent evaluation is essential to provide insights into the operational challenges and successes of delivering healthcare in conflict settings, assess how effectively and efficiently the projects meet the health needs of target populations, and identify areas for improvement. The aim is to determine whether the desired results on outcomes/goal level as defined in the project package have been achieved and whether these results will be sustained in the future through fostering long-term ownership and promoting cost-effective solutions. More precisely, it will examine the sustainability and scalability of the MHU models, with the ultimate goal of informing the strategic direction of URCS and NoRC as they continue to deliver essential health care services in Ukraine. Another important aspect is to determine how the project contributes to strengthening the national health system. Additionally, this evaluation seeks to understand the link between the Home-Based Care (HBC) project and the MHUs when they operate in the same geographical area. Evaluating the current accountability framework, with a focus on the functionality and outcomes of the feedback mechanism should also be a focus of this evaluation.

URCS and NoRC are committed to enhancing healthcare and resilience within conflict-affected and remote communities in Ukraine. Through this evaluation, URCS and NoRC seek to ensure that their interventions are both responsive to immediate health needs and aligned with long-term public health goals. Furthermore, the evaluation will support evidence-based decision-making and contribute to knowledge sharing on best practices for health care delivery in Ukraine.

NoRC started to support this project in 2023, but no baseline data has been gathered at the time. The project team has however organized satisfaction surveys in 2023 and in 2024 and collected answers from people using the services of the Mobile Health Units.

The evaluation will also aim at quantifying baseline values of the relevant outcome indicators for the interventions commencing in 2025, while assessing contextual changes and the evolving needs of the targeted population to inform the future design phase.

Finally, the evaluation should also suggest monitoring tools to effectively track progress, assess performance and ensure accountability.

Regarding the scope of the evaluation:

NoRC is supporting 31 MHUs out of approximately 120 MHUs run by URCS across the country in Dnipro, Ternopil, Khmelnytskyi and Odesa[3]. The evaluation will target those in addition to the MHUs in Kharkiv supported by French RC and the MHUs in Sumy supported by Swedish RC. The data should be presented by each geographical location, per the 4 oblasts supported by Norwegian Red Cross and per total.

Evaluation criteria, objectives, questions

The evaluation criteria to be assessed are (but not limited to):

1) Relevance and appropriateness

• The identification of humanitarian needs both at the start-up of the program and at the time of the review. The assessment should focus on health, protection and basic needs.
• The extent to which the programs contributed to strengthening the national health system and to increase the health coverage in the targeted locations. Including, to what extend are the MHUs connected to a primary health care facility.
• The ability of the programs to utilize the appropriate tools to identify vulnerable groups/vulnerable individuals.
• The extent to which the programs were able to target the vulnerable population and meet their health needs. Special focus shall be given to the needs of disabled persons, women, children, elderly people, and other people with special needs. The availability of PHCC services for IDPs compared to the local population should also be assessed.
• The extent to which the communities have been included in the project cycle and their contribution to decision making.
• Is there a different in relevance for the services provided by the MHUs according to geographical locations including different oblast.
• The ability of the programs to identify SGBV cases and refer them to health and protection services.
• The ability to diagnose, provide immediate care and refer patients in needs of psychosocial support. Assess the added value of having a psychologist integrated in certain MHUs (versus referral).

2) Impact

• The intended and non-intended positive impacts of the programs on the targeted communities.
• The possible negative consequences of the programs’ interventions on the targeted and non-targeted population.
• The change in the access levels of the targeted population to primary health services as a result of the program.
• The satisfaction of the communities and the MHU patients with the health services provided.
• The percentage of people receiving services stating that what has been provided is relevant to their needs.

3) Coherence

• To what extend do MHU and Home-Based Care projects ensure continuum of care within primary health care (including referrals when they operate in the same area).
• The level of contribution to the strategic and programmatic directions of the URCS.
• The synergy with actions undertaken by other Partner National Societies (PNSs) and other local stakeholders and its alignment with public sector interventions.

4) Effectiveness

• The degree to which the programs were able to reach the desired objectives and targets.
• The challenges that were faced during the design, implementation and monitoring phases and the ability of the implementing teams to timely resolve the arising complications.
• The operational successes that were achieved during the execution phase.
• The appropriateness and relevance of the employed monitoring systems to provide timely updates to the implementing teams.
• The quality and comprehensiveness of the data provided in the DHIS and relevant dashboards.
• The quality of services provided and the extent to which health technical standards were applied.
• Where MHU and Home-Based Care collaborate together in the same geographical area, does it lead to improved continuum of care and adherence of treatment?

5) Efficiency

• An analysis of the cost per patient.
• The efficiency of the services provided compared to alternative approaches.
• The efficiency of the coordination mechanisms employed to optimize results and reduce duplications.
• The capacity of the implementing teams to achieve the intended results in terms of number and qualifications of the staff.
• The procedural bottlenecks and required adjustments to the existing workflows to promote efficiency.
• How do the MHU and Home-Based Care team coordinate to ensure continuum of care (where they operate in the same geographical area).

6) Sustainability

• To what extent the programs build on and contribute to developing existing local capacities.
• Whether the achieved results of the projects can be considered sustainable in a medium and long-term.
• To what extent are the Regional Department of Health satisfied with the MHU services, how do they foresee future needs, development/transformation of the model, the minimum required number of MHUs, foreseen changes in health statistics and its impact on MHU work.
• To what extend are PHCCs willing to continue the project with changes in provision (without providing medicine, remuneration to the involved specialists etc.), how do they foresee development/transformation.
• Identifying the major factors influencing the achievement or nonachievement of sustainability of the programs.
• Identify the URCS Regional Offices satisfaction with the MHU services, future demands, development/transformation, possibility for future support.
• The level of scalability of the MHU model.
• The level of sustainability of integrated approach MHU and HBC.

Evaluation methodology

The technical proposal (see below) should include the outline of a suitable methodology and appropriate data collection methods adapted to the purpose of the evaluation. The evaluation team should rely on a mixed methodology, combining quantitative and qualitative tools to meet the requirements of the ToR.

The evaluators must adopt a consultative and participative methodological approach. The approach should strive to include the following elements:

• Desk review of all project documentations (listed below).
• Key informant interviews and/or Focus Group Discussions focusing on collecting the feedback of the targeted population.
• Key informant interviews/Focus group discussions with facility staff, program staff/volunteers and representatives of URCS, NoRC, partner national societies and local authorities and actors.
• Field visits to assess the quality of the services provided.
• Comparative analysis across the different oblasts.

The evaluation team is free to suggest any additional methodological approaches and data collection methods and questions that are not mentioned in the ToR.

The on-site data collection should be conducted in the following project locations:

• Dnipropetrosk, Ternopil, Khmylnytskyi, Odesa, Kharkiv and Sumy.

Alternatives for remote data collection in hard to reach areas can be discussed.

Relevant written documents will be provided by Norwegian Red Cross as well as URCS. Key sources of written information include:

• Project documents (assessments, project plans and logical frameworks, budgets, indicator tracking tables, satisfaction surveys conducted in 2023 and 2024, annual reports)
• Norwegian Red Cross results framework and PoA
• Annual internal Norwegian Red Cross results reports
• Annual report from Norwegian Red Cross to Norad
• Relevant background and strategy papers (from Norwegian Red Cross and partners)
• Grant agreement between Norad and Norwegian Red Cross
• URCS policies
• DHIS2 data
• Data from the Ministry of Health, WHO, OCHA, other humanitarian actors

Deliverables

• Inception report. An inception report with a detailed description of methodology to be used, a data collection plan, draft data collection tools, overview of information sources, a timeline for deliverables and solutions to identified challenges. Noting that data collection tools should be developed in collaboration with NoRC and URCS. The Inception report (5-10 pages maximum) should demonstrate a clear understanding and realistic plan of work for the review, checking that review plan agrees with Term of Reference as well as the expectation of NoRC and URCS. It should also include a travel and logistical arrangements for the evaluation that will be discussed and agreed upon. Note that the car travels to the MHU locations will be arranged by the Red Cross (whilst flights and train tickets will be at the charge of the evaluators).
• Draft Evaluation report by mid-September. Submitted for review, including findings and recommendations. NoRC/URCS will be given one week to review the draft evaluation document and provide feedback. The report needs to be written in English.
• Suggested monitoring tools for the projects. In English and in Ukrainian.
• Feedback Workshop with NoRC, URCS and other PNSs to present the initial findings, conclusions, and recommendations from the draft report before revision and final approval of the final report. Recommended monitoring tools should also be presented during this workshop.
• Final report by beginning of October. A comprehensive report addressing feedback on the draft, with specific sections on findings, conclusions, and actionable recommendations. The report should include an executive summary, background of the intervention evaluated, a description of the evaluation methodology and limitations, findings, conclusions, lessons learned, recommendations, and appropriate appendixes, including data collection instruments, and an overview over men/women interviewed as well as key RC/RC and informants and other stakeholders interviewed. The analysis should be sex and age disaggregated as far as possible. The analysis needs to be disaggregated per oblast and per the 4 oblasts supported by NoRC. The final report also needs to include an annex with recommended monitoring tools to be used by URCS.
• Presentation of Findings: Presentation of key findings and recommendations to URCS and NoRC. Two workshops should be organized.

The deadline for submitting a proposal is 27.05.2024 with estimated start-up 30.06.2025. Tentative dates for key deliverables are:

• Inception report: 14.07.2025
• Data collection: July-August
• First draft report: 19.09.2025
• Final report: 10.10.2025

Proposed timeline

Time schedule

Activities and deliverables

Week 1-2

Activity

• Initial meeting with NoRC and URCS to finalize the evaluation scope and methodology.
• Desktop study: review of project documents, and related primary/secondary resources for the evaluation
• Development of detailed inception report: data collection/analysis plan and schedule, draft methodology, and data collection tools

Deliverables

• Inception report, data collection/analysis plan and schedule, draft methodology, and data collection tools.

Week 3-7

Activity

• Key informant interviews with project staff and authorities
• Data collection in target communities in Dnipropetrosk, Ternopil, Khmylnytskyi, Odesa, Kharkiv and Sumy according to data collection schedule.

Deliverables

• Interviews completed according to data collection plan

Week 8-9

Activity

• Prepare draft evaluation report
• Organize feedback Workshop of initial findings, conclusions, and recommendations before revision

Deliverables

• Draft version of evaluation report.
• Monitoring tools
• Feedback workshop

Week 10

Activity

• Address feedback with revisions in report where appropriate
• Revise and submit final evaluation report

Deliverables

• Final draft of evaluation report
• Final version of suggested monitoring tools for the projects

Week 11

Activity

• Presentation of findings

Deliverables

• Presentation of findings to URCS, NoRC and other PNSs

NB: This is only a suggested timeline which is subject to revision and which should be agreed on with the evaluation team.

Evaluation quality and ethical standards

The evaluators should take all reasonable steps to ensure that the evaluation is designed and conducted to respect and protect the rights and welfare of people and the communities of which they are members, and to ensure that the evaluation is technically accurate, reliable, and legitimate, conducted in a transparent and impartial manner, and contributes to organizational learning and accountability. Therefore, the evaluation team should adhere to the evaluation standards and specific, applicable process outlined in the IFRC Framework for Evaluation. The IFRC Evaluation Standards are:

1. Utility: Evaluations must be useful and used.
2. Feasibility: Evaluations must be realistic, diplomatic, and managed in a sensible, cost-effective manner.
3. Ethics & Legality: Evaluations must be conducted in an ethical and legal manner, with particular regard for the welfare of those involved in and affected by the evaluation.
4. Impartiality & Independence; Evaluations should be impartial, providing a comprehensive and unbiased assessment that takes into account the views of all stakeholders.
5. Transparency: Evaluation activities should reflect an attitude of openness and transparency.
6. Accuracy: Evaluations should be technical accurate, providing sufficient information about the data collection, analysis, and interpretation methods so that its worth or merit can be determined.
7. Participation: Stakeholders should be consulted and meaningfully involved in the evaluation process when feasible and appropriate.
8. Accountability: Evaluations should be conducted ensuring the accountability by adequately documenting the evaluation process and products, aligning evaluation practice with an equity approach, and implementing the recommendations into actions

It is also expected that the evaluation will respect the seven Fundamental Principles of the Red Cross and Red Crescent: 1) humanity, 2) impartiality, 3) neutrality, 4) independence, 5) voluntary service, 6) unity, and 7) universality. Further information can be obtained about these principles at: https://www.ifrc.org/who-we-are/international-red-cross-and-red-crescent-movement/fundamental-principles

Evaluator(s) qualifications

The evaluation team proposed should have the following skills, experience and qualifications to be selected for this evaluation:

• Expertise in Public Health: Advanced degree in Public Health or a related field for the lead evaluator(s).
• Expertise in Project Evaluations: Demonstrated experience in evaluating health projects, particularly in conflict and emergency settings. Proven track record of conducting qualitative and quantitative research is required. Minimum of 8 years of monitoring and evaluation experience required in large scale and complex humanitarian operations.
• Language requirements: Ukrainian and English. The evaluation team should be composed of enumerators who are fluent in Ukrainian and/or Russian for the interviews. Review of secondary data requires being able to read Ukrainian. The report is to be written in English.
• Knowledge and experience working with the Red Cross Red Crescent Movement is highly valued.
• Experience working in Ukraine and knowledge of the Ukrainian health system is preferred.

NB: The evaluation team needs to be composed of female enumerators to conduct interviews with female MHU patients.

[1] Before that support was channeled through the ICRC (Access to Education, Financial Development).

[2] As defined in NoRC international strategy and operational framework.

[3] Odessa is financially supported by NoRC by managed by German RC.

How to Apply:

To get the ToR in PDF – Go to: Norwegian Red Cross ToR Health evaluation MHU.pdf

Proposals will be ranked according to how they fulfil the three following requirements:

• Team qualifications (50%)
• Methodological approach (25%)
• Price (25%)

Interested candidates should submit their application material by 27.05.2024 to:

• Maika Skjønsberg, Deputy Regional Representative – Europe region, Norwegian Red Cross, maika.skjonsberg@redcross.no

The subject of the email should be “Application – Health Evaluation Ukraine”.

Application materials should include:

1. Curricula Vitae (CV) for all members of the team applying for consideration.
2. Cover letter clearly summarizing your experience as it pertains to this assignment, your daily rate, and three professional references.
3. A brief description of your firm or institution (for applicants other than individual contractors).
4. Technical proposal not exceeding five pages expressing an understanding and interpretation of the TOR, the proposed methodology, and a time and activity schedule.
5. Financial proposal itemizing estimated costs for services rendered (daily consultancy fees), accommodation and living costs, transport costs[1], and any other related supplies or services required for the evaluation.
6. Two examples of an evaluation report most similar to that described in this ToR.

Application material is non-returnable, and we thank you in advance for understanding that only short-listed candidates will be contacted for the next step in the application process.

According to Norwegian tax law, Norwegian Red Cross can only enter consultancy contracts with companies registered as legal entities with their respective tax authorities. For individual consultants, temporary employment contracts are signed.

Contracted consultants shall sign the Norwegian Red Cross “Standard Procurement Terms and Conditions” and the “Code of Conduct for Corporate Social Responsibility for Suppliers to the Norwegian Red Cross”.

Consultants are responsible for buying their own travel and health insurance. NoRC security regulations must be followed, including signature of proof of life and acknowledgement of risk. A security briefing will be provided prior to departure and upon arrival in country. Field Security Rules will be shared prior to travelling.

The consultancy firm needs to ensure that the individuals who will travel to Ukraine have the necessary visas to enter the country.

[1] Your transport costs should include flights and train tickets but exclude transport to the field locations from the main cities: this will be covered by the Red Cross in Red Cross vehicles.