78242 Finance & PSD Impact APRIL 2013 The Lessons from DECFP Impact Evaluations ISSUE 23 Our latest impact note provides one of the still rare examples of where an encouragement design was successful in getting sufficient take-up to measure the effects of a program, and shows how an aggregate policy can be evaluated. Harnessing Emotional Connections to Improve Financial Decisions: Evaluating the Impact of Financial Education through Mainstream Media Gunhild Berg and Bilal Zia Financial education is important, yet way out. We study the effectiveness of these there is a considerable knowledge gap in messages through three quantitative surveys how best to deliver it. The literature on and three qualitative focus groups, with the careful evaluations of financial literacy is former providing insight on the mechanism small but growing, and has moved away of measured impacts. from classroom based interventions to more innovative delivery mechanisms such as The measurement challenge videos and DVDs. Yet, the scope and reach The methodological challenge of even the best produced DVDs is limited associated with such an evaluation is non- on the supply side, and attracting viewership trivial. Critically, it is impossible to can be significantly challenging on the externally restrict viewership since the soap demand side. opera is nationally televised with no Entertainment media offers not only variation in TV signal strength, at least in broader outreach since nearly every urban areas where this study focuses. household nowadays has a TV, but also a We employ a symmetric captive audience. Furthermore, as emotional encouragement where a randomly selected connections are established between a show treatment group was encouraged through and its audience, the program provides a financial incentives to view Scandal, and potentially powerful platform for another set of individuals – the control communicating messages and influencing group – was similarly encouraged through behavior. There is considerable evidence, identical financial incentives to view a especially in the health and education fields, comparable soap opera (“Muvhango�) that on the success of media campaigns in overlaps with Scandal in television improving social behavior. primetime. Hence, we were able to create an We evaluate financial education artificial separation between treatment and through a popular television soap opera in control groups, and our analysis shows that South Africa, “Scandal!� The intervention adherence to these allocations was very high entailed a 2 month long storyline featuring – more than 95%. one of the main characters of the soap What do we find? borrowing excessively through hire-  The analysis finds individuals assigned purchase, gambling, and falling into a debt to watch Scandal had significantly trap; and eventually seeking help to find her higher financial knowledge on issues Do you have a project you want evaluated? DECRG-FP researchers are always looking for opportunities to work with colleagues in the Bank and IFC. If you would like to ask our experts for advice or to collaborate on an evaluation, contact us care of the Impact editor, David McKenzie (dmckenzie@worldbank.org) highlighted in the soap opera storyline, only a few women seemed to remember, and in particular messages delivered by the they associated their memory more with the leading character – a 4.5 percentage fact that the NDMA character was female rather than her representation of the NDMA. point increase over the control group. In contrast, all participants readily recalled  We find an increase in the likelihood of the main soap character and aspects of the borrowing through formal channels – 22 financial storyline delivered by her. Upon percent compared to 13 percent in the further inquiry, they attributed better recall control group, and a greater likelihood to to the fact that the main character was a borrow for productive purposes such as mainstay in the soap opera, and was popular investments in household durable goods and likeable. and vehicles. Direct measures of hire- Upon reflection, it is clear that the NDMA purchase use also show a significant advisor was external to the main soap and decline – 15 percent compared to 19 only appeared in 2-3 episodes, never to percent in the control group. Further, we reappear again. Hence, the audience did not find a 5 percentage point lower get the opportunity to establish or maintain a propensity to gamble among the connection or following with this character. treatment group. Implications  One aspect of the soap storyline that did 1. Our findings show an important role for not sustain traction beyond immediate entertainment media as an accessible and effects was a public call to action for the important tool for policymakers to deliver South African National Debt Mediation carefully designed educational messages that Association (NDMA). The soap featured resonate with the audience and can influence an NDMA counselor who helped the financial knowledge and behavior. main character build a financial plan to 2. Our analysis suggest that emotional connections and familiarity with media enable her to save enough and repay her personalities certainly play a role in debts, yet the longer term analysis shows motivating knowledge and behavior change no recall of the NDMA or even memory among viewers, and that harnessing such that a debt counselor appeared in the potential can be an important channel for soap storyline. achieving development impact. 3. Finally, our methodology illustrates the Mechanism of impact benefits of complementing quantitative In order to understand the mechanism of analysis with qualitative work to better impacts, we conducted three focus group discussions three months after our final understand mechanisms behind measured quantitative survey. In these meetings, the impacts. focus group leaders were unable to elicit recall of NDMA from the participants and For further reading see: Berg, Gunhild, and Bilal Zia. 2013. “Harnessing Emotional Connections to Improve Financial Decisions – Evaluating the Impact of Financial Education through Mainstream Media,� World Bank Policy Research Working Paper no. 6407, April 2013. Recent impact notes are available on our website: http://econ.worldbank.org/programs/finance/impact