Gender Equality and Social Inclusion (GESI) Pilot Project Evaluation

TERMS OF REFERENCE (TOR)

Gender Equality and Social Inclusion (GESI) Pilot Project Evaluation

1. Background of the Organization

MSI ETHIOPIA Reproductive Choices, the former Marie Stopes International Ethiopia (MSIE) is a not-forprofit partner of the UK based international organization Marie Stopes International, engaged in the provision of a range of quality Family Planning, Reproductive Health, and Maternal and Child Health services in Ethiopia. Since its establishment in 1990, MSIE-RCs mission has been to enable Ethiopian women to have Children by Choice, not Chance. MSIE-RC, with an objective of decreasing maternal mortality and morbidity through expanding choices for women and couples - has served clients through its 20 clinics, over 200 social franchising clinics, 6 mobile teams and support of 91 public facilities.

The scope of this project will mainly focus on the public sector and mobile teams channels of service delivery.

2. Background of the GESI Pilot project:

To address some of the challenges faced by women and girls in realizing their sexual and reproductive health and rights (SRHR) in rural areas, MSIE-RC has designed an innovative pilot to engage men and better support women and girls to overcome barriers related to gender and social norms affecting their SRH access. The pilot ran for five months from October 2023 to April 2024 in the Geze Gofa Woreda and consists of three main elements:

(1) Training of healthcare workers (HCW) and health extension workers (HEW) via value clarification and transformation (VCAT) workshops.

(2) Engagement with men and community leaders.

(3) Monthly community dialogues with mens and womens development armies as well as leaders.

By generating evidence and evaluating this pilot, MSIE aims to tailor its activities to better engage men and support women and girls to access SRH services, and share insights with the wider sector.

3. Proposed Methodology

The mixed-methods study will include a cross-sectional post-pilot survey of those participating in the pilot activities (mens and womens development armies) and qualitative interviews with those involved in the pilot:

3.1 Sex- disaggregated post pilot follow-up survey

Repeat of sex disaggregated survey with womens and mens development armies, and community dialogue participants as well as with the HEWs.

A standardized post-pilot questionnaire will be administered to pilot participants after the end of the pilot activities. The survey will cover participants involvement with the pilot, demographic information, perception of various gender and social norms, discussion of family planning services, uptake of family planning services and feedback on the pilot activities. Data will be entered electronically, using a secure KoboToolbox (ODK Software) account and online form. SPSS will be used for qualitative data analysis.

A simple random sample of all of each study population will be identified using the contact information for pilot participants held by the leaders of the womens and mens development armies:

• Each kebeles women development armies (total 16 kebele) have set-up groups of approximately 20 participants therefore, approximately 320 women have participated in the pilot activities. Therefore, we will have target a sample of 320 women community dialogue participants for the post-pilot survey.
• Each kebeles men development armies (total 16 kebeles) have set-up groups of approximately 20 participants therefore 320 men have participant in the pilot activities of community dialogue. Therefore, we will target a sample of 320 for post pilot survey.
• 31 HEWS, 48 WDAs and 42 MDs will take part in this survey.
• Sample sizes have been calculated using Cochran's equation, assuming 50% population proportion, confidence level of 95% and margin of error 5%, together with a population correction.

3.2 Qualitative interviews

Will use semi-structured interview and discussion guides to explore participants experience of the pilot and their perceptions of key gender and social norms. In-depth interviews (IDIs) will be used to explore individual pilot participant and key informant experiences while focus group discussions (FGDs) will be used to explore group consensus on what has worked and what was more challenging among those facilitating the activities. Topics will cover various aspects of gender norms, such as joint-decision making, household tasks, gender-based violence, community expectations and acceptance of contraception use. Both interviews and discussions will take place face to face, in a private location. They will be recorded and transcribed. Transcriptions will be uploaded to qualitative analysis software (Dedoose, MAX QDA, Atlas ti) for analysis.

Sample size for qualitative study will include:

Data collection will occur at the end of the pilot in the second half of April 2024. Data collection should occur at the end of the pilot (mid-April 2024 to mid-May 2024) for a duration of two to three weeks. Data collectors are hired and are trained for a minimum of two days in approaching clients, appropriate informed consent procedures, and the aims of the survey and qualitative research, before beginning data collection. Purposive sampling will be applied for qualitative data collection.

4. SCOPE OF WORK:

The MSIE team, with the support of the MSI Global Support Office (GSO), will supply the topic guides for the questionnaires, in-depth interviews, and focus group discussions.

The consultant will lead on the data collection and analysis, result interpretation and report write-up for the evaluation. This includes:

1. Study and implement the pilot evaluation protocol detailing the sampling strategy and approaches to be used in data collection to cover the study objectives.

2. Adapt and translate all required tools in line with global guidelines (questionnaire, consent form, refusal logs). The topic guides should also be back translated and submit summary of translated documents to MSIE-RC.

3. Pre-test and pilot all data collection tools.

4. Train research assistants in selected regions for the purpose of conducting data collection as required. At a minimum, the following topics should be covered: an overview of the MSIE-RC program (service delivery and channels), research ethics (including informed consent), fieldwork logistics, and data collection techniques (including role play).

5. Manage the data collection and supervision process with close coordination and oversight from the MSIE-RC Research, Monitoring & Evaluation (RM&E) team.

6. Oversee data quality in-real time through the development of data quality assurance processes / workflows (e.g., high frequency checks).

7. Employ the development of electronic data collection approach, that is, Computer Assisted Personal Interview (CAPI) using kobo collect (or other compatible data collection platform) specifically for electronic data collection.

8. Submit raw data (or make available data in OKT platform) and when applicable recordings/transcripts to MSIE-RC for further Analysis.

9. Analysis of both qualitative and quantitative data

10. Submission of first draft report

11. Submission of final report with summary of major findings.

5. EXPECTED DELIVERABLES/OUTPUTS:

• Inception report
• Draft and final copies of the topic guides (questionnaires, IDIs and FGDs), refusal log, consent forms
• Sampling strategy plan for enumerators as well field monitoring and supervision plan
• Submission of hard copies of the signed consent forms and completed refusal logs.
• Any .do files, SPSS files used to check data quality during the data collection,
• A field report of the data collection exercise including how data quality issues and gaps, if any, were addressed.
• A clean SPSS data set of all survey data, which is organized using the standard variables and response options found on the standard questionnaire and which is submitted to MSIE-RC no later than May 15^th, 2024.
• Transcripts and recording of in-depth interviews and transcripts.
• Any additional deliverables that are part of the standard MSI exit interview package,
• Draft report of findings
• Final report of findings with summary of major findings.

6. ROLES & RESPONSABILITIES

7. TIME FRAME:

• This exercise is expected to be executed beginning from 15th April 2024.

• Fieldwork activities should be completed by 10st May 2022.
• Following completion of field activities, data entry and cleaning task should be completed by 15^th May 2024.
• The final data set should be submitted to MSIE by 15^th May 2024.
• The final report should be submitted before the end of May 2024.

The agency should also be available to attend any queries on the data set.

8. LOGISTICS AND PROCEDURES:

Upon signing the contract with the consultant, MSIE-RC will provide pilot package to the consultant and will provide any other support such as space for data collectors training. MSIE-RC will also provide a list of participant details, sites, and locations for the data collection. The consultancy will be responsible for organizing fieldwork travel arrangements for research assistants and for procurement any printed survey materials or mobile data collection hardware.

• 9. MINIMUM REQUIREMENTS FOR APPLICANTS:

• All project members must demonstrate pro-choice values,
• At least five years of experience in managing research [including protocol development, tool preparation, fieldwork coordination, established client confidentiality practices]
• Experience with using Survey Kobo, or ODK
• Experience with pre-testing surveys and qualitative guides.

• Experience managing logistics including materials personnel,
• Agency has technical specialists in public health and family planning with proven track record of prior quantitative research,
• Field supervisors experienced on research activity or data collection with similar research and have public health/nursing or health/social science background,
• Enumerators experienced in conducting research activities.
• Experience on analysis of both qualitative and quantitative data.
• Experience on repot write-up for mixed research method with appropriate triangulation.
• Experience on summary report write-up from mixed method research.

CODE OF CONDUCT:

Client confidentiality

It will be the responsibility of both the consultant and MSIE-RC to ensure that study documents and collected information is kept confidential, safe, and secure. The consultancy will be required to adhere to the confidentiality and privacy guidelines stipulated by MSIE-RC.

Adult & child safeguarding policy

The consultancy will be required to adhere to the adult and child safeguarding policies followed by MSIE. Our commitment is to:

1. Safeguard clients (including children): by actively identifying clients who may be vulnerable and at risk of significant harm, by preventing intentional or unintentional harm, abuse, and exploitation within MSI operations, and promptly responding, if harm or abuse occurs.

2. Safeguard MSI People: by promoting a safe working environment and by avoiding situations and behaviours which may be misinterpreted and potentially lead to false allegations against team members.

3. Safeguard MSI: by demonstrating a genuine commitment to Safeguarding all our clients, our team members, and preventing cases of abuse happening which may tarnish MSIs reputation and affect its ability to continue work and receive funding.

Incident reporting

All researchers have a duty to report any witnessed or suspected cases of a team member, volunteer or research partner committing abuse or exploitation, or any breach of the safeguarding policies, or code of conduct.

CONTRACT PAYMENT TERMS:

• Proposals are invited based on these Terms of Reference outline in this document. Bidders are invited to note that, if awarded the contract, payment terms will be subjected to negotiation.
• The payments will be made upon successful completion of the deliverables and submission of invoices in instalments.
• The payment for the work shall be as follows:
• 30% of the budget will be paid upon both parties signing the contract.
• 30% of the budget will be paid upon delivery of cleaned dataset and draft report.
• 40% of the budget will be paid upon the submission of the final report with summary.
• MSIE-RC reserves the right to withhold all or a portion of payment if performance is unsatisfactory or if work/outputs are not achieved.

SUBMISSION OF PROPOSAL:

The consultant shall submit a signed, sealed, and complete technical and financial proposal in a separate envelope. The submission can only be made in person. Along this submit a soft copy of the technical proposal with CD. Bidders are also required to submit legal documents (renewed business license, tax identification, value added tax registration certificate). The submission package should include:

• Agency profile and background resources
• At least one sample of similar past fieldwork projects executed,
• Proposed project work plan and budget broken down by each activity,
• Curriculum vitae including contact details of key project members,

All selected candidates will be required to sign a self-declaration of compliance statement and the MSI code of conduct. Background checks will also be conducted during the recruitment process.

The evaluation of the bids will be based on the following criteria.