Key learnings from an outcome and embedded process evaluation of a direct to beneficiary mobile health intervention among marginalised women in rural Bihar, India

Introduction Mobile Vaani was implemented as a pilot programme across six blocks of Nalanda district in Bihar state, India to increase knowledge of rural women who were members of self-help groups on proper nutrition for pregnant or lactating mothers and infants, family planning and diarrhoea management. Conveners of self-help group meetings, community mobilisers, introduced women to the intervention by giving them access to interactive voice response informational and motivational content. A mixed methods outcome and embedded process evaluation was commissioned to assess the reach and impact of Mobile Vaani. Methods The outcome evaluation, conducted from January 2017 to November 2018, used a quasi-experimental pre-post design with a sample of 4800 married women aged 15-49 from self-help group households, who had a live birth in the past 24 months. Surveys with community mobilisers followed by meeting observations (n=116), in-depth interviews (n=180) with self-help group members and secondary analyses of system generated data were conducted to assess exposure and perceptions of the intervention. Results From the outcome evaluation, 23% of women interviewed had heard about Mobile Vaani. Women in the intervention arm had significantly higher knowledge than women in the comparison arm for two of seven focus outcomes: knowledge of how to make child's food nutrient and energy dense (treatment-on-treated: 18.8% (95% CI 0.4% to 37.2%, p<0.045)) and awareness of at least two modern spacing family planning methods (treatment-on-treated: 17.6% (95% CI 4.7% to 30.5%, p<0.008)). Women with any awareness of Mobile Vaani were happy with the programme and appreciated the ability to call in and listen to the content. Conclusion Low population awareness and programme exposure are underpinned by broader population level barriers to mobile phone access and use among women and missed opportunities by the programme to improve targeting and programme promotion. Further research is needed to assess programmatic linkages with changes in health practices.