Assessing the Impact of Information Campaigns on Raising Parental Awareness of the BOS Program1 Policy Brief June 2012 71425 KEMENTERIAN PENDIDIKAN DAN KEBUDAYAAN Photo: World Bank supported the purchase of goods and services meant to enhance the quality of education (e.g. texts and library books, teaching- learning materials, teacher professional development activities, and remedial learning) and have reduced the need for schools to seek funds from parents to cover these expenditures. The BOS program provides schools with greater autonomy and flexibility in managing their resources. Schools in turn are expected to use these resources in a transparent and accountable way. The program has strengthened school planning and budgeting processes and has begun to open these up to parental and community oversight. Given the traditionally passive role which parents play in school activities and management, compounded by a lack of information about school affairs, this clearly represents a fundamental reform of education in Indonesia. II. The BOS Information Campaign: Why and What BOS lightens the load – A campaign commercial explaining the purpose of BOS While parental knowledge is crucial for the success of BOS, concerns were raised at the Ministry of Education and Culture about the level of parental awareness of the program. These I. School Operational Assistance Program - concerns were confirmed by a nationally representative survey 1 conducted in 2009. Parents were generally aware of the overall objectives of the program but were less knowledgeable about Indonesia has shown an increasing commitment to education the detailed workings of BOS – knowledge necessary to actively over the last decade. Since 2001, spending has doubled in real participate in school affairs and hold schools accountable for the terms largely as a result of a constitutional requirement obligating use of BOS funds (see Figure 1). the government to spend a fifth of its budget on education. A cornerstone of this increased investment has been a school Figure 1: Many parents knew about BOS objectives, but far grants program aimed at providing financial support to all public, fewer had any detailed knowledge of the program Percentage of parents who gave correct answers to questions about the BOS private and religious primary and junior secondary schools. program, 2009 The School Operational Assistance program (BOS), initiated by 80 the Ministry of Education and Culture in 2005, provides basic 70 education schools with block grants. The program currently 60 provides grants to 228,000 schools benefiting an estimated 43 50 % of parents million students. In 2012, 8.1 percent or Rp 23.5 trillion of the total government education budget was spent on BOS. 40 30 The aim of the school grants program, allocated on a per-student 20 basis, is three-fold: to improve access to and raise the quality of basic education, to reduce the financial burden for students 10 and to support school based management reforms. Grants have 0 1 This policy brief is based on a more detailed forthcoming report - World Bank, 2012. ‘Assessing the impact of information campaigns on raising parental Note: Parents were randomly selected from a nationally representative sample of awareness of the Indonesian school grants program’ 702 primary and junior secondary schools. Photo: World Bank In light of this limited awareness, the Ministry of Education and Culture, with the support of the World Bank, developed a social marketing and information campaign to raise awareness of the program. The campaign had two main components:  A national television and print media campaign targeting the general public across the whole of Indonesia. The aim of the campaign was to provide basic BOS information and advocate for greater transparency using public service announcements on television and print advertorials. The national campaign was complemented by a range of activities designed to support journalists covering the BOS program.  A trial of alternative campaigns in four districts and selected schools. The main aim of these trials was to identify the most effective way of reaching parents with information about the program. Methods used to reach parents with information on BOS included the organising of school meetings, sending letters and SMS to parents and providing information either As part of their media workshop, journalists discuss the benefits of BOS with a poor household in through new and more colorful school notice boards or giving Tulungagung, East Java small pocket books, containing information on BOS, to parents. free schooling to all. After 2010, a number of new advertisements Each element of the campaign had to explain what BOS was and were designed with support from the World Bank. They included: how it could be used, in clear and simple messages, and outline the  A second advert featuring the famous personality, Cut Mini, responsibilities of each level of the system to make BOS succeed. which corrected the earlier misconception that BOS would A private firm was engaged to design and implement each cover all individual student costs (e.g., transport, workbooks). component of the campaign.  An advert featuring MoEC officials highlighting the important To explore the impact of different campaign approaches, a role districts needed to play in supporting the BOS program. nationally representative survey of parents of primary and junior  An advert featuring colourful balloons lifting children off the secondary school students was carried out before (2009) and ground. This was intended to show that BOS would “lighten� after (2011) the campaign. For the national television and print the cost of education, with the balloon becoming a striking and campaign, this allowed for a simple comparison of levels of memorable icon for the campaign. knowledge on BOS between parents who saw different aspects of the campaign and those who did not. In the four participating  A set of animated adverts explaining the purpose of BOS and districts, different campaign methods (e.g. SMS, school meetings, the need for parents’ participation in discussing, planning, and pocket books, etc.) were randomly assigned to different parents monitoring the use of BOS funds. Print advertorials with similar and schools. This allowed for a rigorous evaluation of the change in messages were placed in several national newspapers. parental knowledge associated with each adopted approach. The BOS television campaign was successful at reaching a large III. The Results of the National Television and Press number of parents - 60 percent of parents surveyed had seen at least one of the BOS commercials. The national press adverts were Campaign frequently complemented with advertorials in the local media sponsored by district education offices. However, only 7 percent At the national level, the campaign focused on television of parents remembered seeing an advertorial in either the local announcements and advertorials placed in the national print media. or national press. There was also a significant difference in recall Over the last five years, opinions in the Ministry of Education and among the various television approaches. The “balloon� ads and Culture have differed on the best approach and this has resulted the 2009 ads featuring famous personalities (Dik Doank and Cut in a number of different campaigns. Prior to 2010, two television Mini) were remembered by approximately a third of parents. Far advertisements featuring famous personalities were aired (Cut fewer parents recalled the advert with MoEC officials (13 percent), Mini and Dik Doank). One of these advertisements was wrongly and the recent animated adverts were remembered by less than 10 interpreted by the public as suggesting that BOS would provide percent of respondents. 2 Parents who remembered seeing the national television campaign Prior to the campaign the majority of reporting on BOS focussed were more aware of BOS objectives but no more likely to have on disbursement delays and local corruption cases. Relatively little detailed knowledge of the program. A simple comparison showed reporting examined the benefit of BOS to schools and children. that the proportion of parents who had seen a BOS advert were 48 percentage points more likely to know about BOS objectives than Although there is less information on the effect of activities directed parents who had not (Figure 2). However, there was no marked towards journalists, there was a considerable change in the tone difference in parents’ grasp of detailed BOS knowledge (e.g. of national media reporting. Before the interventions, only 34 awareness of eligible uses of BOS funds) between the two groups. percent of national media reports were positive (Figure 3). After the interventions, the percentage of positive reports had increased to These results are only suggestive as it was not possible to control 45 percent, and the overall number of reports on BOS had increased for differences between parents who had seen the BOS adverts four-fold (Figure 3). While it is difficult to attribute all these changes and those who had not. It is possible that the better knowledge of to campaign activities, they do appear to have been related to better BOS amongst parents who had seen the campaign was the result coverage of the BOS program in the national media. not of messages contained in the television commercial but rather of other factors. For example, parents seeing the adverts may Figure 3: Activities directed at journalists were associated with an be wealthier than parents who did not see the advert. Wealthier increase in the number of articles on BOS and an improvement in parents may have a better understanding of BOS, regardless of their tone whether they saw the BOS campaign, because they have greater Total number of press reports on the BOS program by tone and period access to information on BOS from other sources. It is possible that parental wealth may be driving the higher levels of BOS knowledge after media interventions February 2011 amongst the group who saw the BOS campaign rather than the campaign itself. during media interventions August 2010 - January 2011 Figure 2: Parents seeing BOS television commercials were more likely to know BOS objectives before media interventions Percentage of parents who knew the objectives of BOS by exposure to the January -J uly 2010 national television campaign 100 % of parents w ho knew BOS objectives average monthly media reports on BOS 90 Negative Neutral Positive 80 Notes: 30 national and 10 local media (e.g. newspapers and websites) were covered 70 in the monitoring exercise. Positive stories included stories that highlighted the % of parents 60 importance of BOS for learning while negative stories frequently covered cases of misappropriation of BOS funds. Approximately 80 journalists were involved in local 50 media workshops on BOS. 40 30 20 IV. Experimenting with Approaches at the School Level 10 A number of different approaches to reaching parents were trialled 0 in four districts (Ambon, Malang, Sumbawa and Tulungagung). In Malang, half of the parents included in the evaluation were randomly selected to receive a set of SMS messages providing information on BOS. In Sumbawa, half of the schools included in the evaluation were To complement the national television campaign, several activities randomly selected, and, with the help of facilitators, school meetings were designed to support journalists covering BOS. These included were organized to convey BOS information. In Tulungagung, one press conferences at national and district levels to launch the BOS approach sent letters on the BOS program home with children while program, workshops to provide a broader perspective on BOS and another gave parents free BOS pocket books. Schools included in its emphasis on transparency and accountability, media visits to the evaluation were randomly selected into one of three groups districts and schools demonstrating good practice and writing depending on whether parents in the school received a letter, a competitions. The purpose of these activities was to attempt pocket book or neither (the control group). Randomly selecting to make BOS reporting more accurate as well as more balanced. schools into different groups allows the true impact of the different 3 Photo: World Bank campaign approaches on parental knowledge and participation to be identified. School meetings convened by the principal or a community leader improved parental knowledge and participation significantly. The proportion of parents with knowledge of BOS objectives increased by 16 percentage points solely as a result of schools holding meetings of this kind (Figure 4). The meetings were also successful at increasing the more detailed knowledge necessary to actively participate in school affairs. The number of parents who knew the eligible uses for BOS grants more than doubled in schools that held meetings compared to those that did not (comparison schools). A key component of the success of this approach can be attributed to the high attendance of parents in school meetings. In Sumbawa, approximately three-quarters of parents attended the school meeting. The impact of text messages sent to parents was smaller than the impact associated with school meetings. The proportion of parents who knew the per-student amount of BOS increased by seven New colorful BOS notice boards provided to schools participating in the evaluation percentage points for those who were meant to receive the text message (Figure 5). However, these messages did not appear to third of individuals selected to receive the text message actually improve parental knowledge of the objectives of BOS. received it. This was largely because telephone numbers were incorrect or text messages were sent to other family members The more limited effect of text messaging was associated with or were deleted before parents could read them. Some of these problems in sending text messages to the correct parent. Only a problems are clearly risks associated with using an approach of this Figure 4: School meetings were successful in raising awareness as well as conveying detailed knowledge of the BOS program Impact of school meetings on parents’ knowledge of BOS objectives and eligible uses 60 60 % of parents that knew % of parents that knew BOS objectives eligible uses of BOS 50 50 40 40 % of parents % of parents 30 30 20 20 10 10 0 0 Note: Schools were randomly assigned into two groups – a control group which did not receive the project organised school meeting and a treatment group that did receive the meeting. The difference between the proportion of parents in each group that knew about BOS is the average treatment effect. The difference is statistically significant at the 1 percent level. 4 kind. However, some of the mis-targeting (e.g. incorrect telephone Photo: World Bank numbers) could potentially be improved and this would raise considerably the overall impact of this approach. Figure 5: Text messages had a more modest impact on awareness of BOS Impact of text messages on parents’ knowledge of BOS per-student spending uses 60 % of parents that knew BOS per student 50 allocation 40 % of parents 30 20 A text message used to deliver BOS information directly to parents in Malang, East Java 10 0 schools have available to them and the reduction this implies for their own contribution to the school is also crucial. Indonesia, no doubt partly because of its broad geographical range and Note: Parents were randomly assigned into two groups – a control group which did diverse, often scattered population, understood that a significant not receive the text message and a treatment group that did receive a text message. information campaign about BOS was essential. Just as important The difference between the proportion of parents in each group that knew about was the decision made to use the campaign to test a variety of BOS is the average treatment effect. The difference is statistically significant at the 5 percent level. dissemination techniques. Sending letters and giving away pocket books to parents had no The national television campaign appears to have been discernible impact on knowledge of the BOS program. While most remembered by a large proportion of parents in Indonesia. parents received the letters and pocket books, it appears that the There is some evidence to suggest that knowledge of overall letters were not read or the messages were unclear and that pocket BOS objectives was higher for parents who were exposed to the books were either given away or the messages about BOS ignored. campaign. Adverts which contained popular personalities or were centred on a logo or icon (e.g., the balloons “lightening� the burden In two of the trial districts, new colorful BOS notice boards were on children) tended to be most memorable. However, national provided to schools participating in the evaluation. It was not television campaigns are expensive, and their effectiveness needs possible as part of the evaluation to assess their impact on parental to be considered against the cheaper and more effective school- knowledge because all schools, rather than a randomly selected based interventions trialled in the campaign. sample, were provided with the notice boards. However, many parents reported seeing the new notice boards, and it is likely that Print advertorials did not seem to be an effective way of the messages and information on the BOS program they provided communicating with parents but working with journalists improved overall awareness levels. to improve reporting had pay-offs. The media interventions conducted encouraged a more accurate and balanced reporting of BOS and have been effective at both raising the amount of BOS V. Conclusions and Lessons Learned reporting in the press and also its tone. Continuing to work with journalists is important to ensure that BOS is reported fairly and Empowering parents and the community with knowledge about that it remains a topic that receives wide attention in the popular BOS is essential if the program is to be transparent, accountable press. and open to parental participation. It is also important for broader school-based management reforms. Ensuring that Direct school-parent interactions offer an effective way of informing parents, particularly poor parents, understand the resources that parents about the BOS program. In particular: 5  School meetings appear to be a very effective way of Photo: World Bank communicating with parents. They are inexpensive to arrange and, given the large impact they have on parental knowledge, relatively cost-effective.  Text messaging also appeared to be a promising approach, but difficulties in implementing this approach limited its impact. If difficulties in reaching the correct parents could be overcome, this could provide an alternative to school meetings.  Notice boards appear to have the potential to improve parental knowledge of BOS and are a useful complement to the other approaches trialled. The different communication strategies also improved different areas of BOS knowledge. For example, school meetings increased knowledge of BOS objectives and eligible uses whereas text messaging improved the knowledge of the per-student allocation. While further study is clearly required to understand the reasons underlying these findings, they tentatively suggest that a combination of approaches may be most effective in raising A school meeting held as part of the information campaign in SDN Kukin, Sumbawa, West Nusa Tenggara awareness and parental participation in school affairs. Information campaigns need to be conducted regularly to reinforce Finding the resources necessary to conduct information earlier information and also to communicate program changes. campaigns on a regular basis is difficult. But the costs associated Since the evaluation was conducted, the amount of BOS grant with campaigns of this kind must be weighed against the potential per-student has increased, and it is vital to communicate these benefits that come from stronger parental accountability at the changes if parents are to continue to hold schools accountable school level. The findings from this study suggest that the costs of for BOS funds. Furthermore, recent research from India suggests information campaigns do not have to be excessive and are a small that regular information campaigns are much more effective than price to pay for encouraging parents to take a more active role in single events.2 their children’s school. 2 Pandey, P., Goyal, S. and V. Sundararaman. (2011). ‘Does information improve school accountability? Results from a large randomized trial’. World Bank South Asia Discussion paper series, No. 49 printed on recycled paper Preparation of this document received partial funding from the Government of the Netherlands under the supervision of the World Bank. The findings, interpretations, and conclusions expressed in this paper do not necessarily reflect the views of the Government of Indonesia or the Government of the Netherlands. For more information, please contact Mae Chu Chang, mchang@worldbank.org, or Samer Al-Samarrai, salsamarrai@worldbank.org Human Development Sector World Bank Office Jakarta Indonesia Stock Exchange Building, Tower 2, 12th Floor Jl. Jenderal Sudirman Kav. 52-53 Phone: (021) 5299 3000 Fax: (021) 5299 3111 www.worldbank.org/id/education