76976 The World Bank PREMnotes 2013 NUMBER 24 Special Series on Building Monitoring and Evaluation Capacity in the Republic of Yemen Ingrid Ivins and Helena Hwang This note disseminates key lessons learned from a World Bank–financed project in the Republic of Yemen, “Monitoring and Evaluation of the Poverty Reduction Strategy Paper and Reform Programs,� which established and operationalized a Poverty Reduction Strategy Monitoring Unit. The approaches used, such as a study tour to Uganda, focused international training sessions, and the successful work on improving and minimizing the number of indicators, provide some lessons learned for other countries seeking to build M&E capacity. Policy makers learned from Uganda’s good practices of connecting accountability with resource allocation and of the importance of a strong and independent statistics office. The Republic of Yemen’s experience also illustrates the value of having a powerful M&E champion to support such a significant initiative. Finally, the inclusion of civil society organizations in the planning process and in M&E outputs, especially on the central level and on policy matters, increased popular support and was an important factor in building M&E capacity. Country Context civil service (a legacy of the 1990 reunification The Republic of Yemen’s history as a fragmented when existing civil services were merged without country dates back to northern Yemen’s roots reform), and the persisting need to clarify insti- within the Ottoman Empire, and southern Ye- tutional mandates and structures. The World men’s rule under the British. After an intense, Bank has noted that the Republic of Yemen will two-decade long civil war, the north and south not meet its Millennium Development Goals were reunified on May 22, 1990. Conflict, how- for 2015. The country will have to overcome ever, continues among warring tribes and factions. serious challenges to achieve its strategic vision, Tension has increased as a result of the current such as the impending rapid decline in oil rev- wave of protests across the Middle East. enues, the weak capacity of governance institu- tions, the pressures of high population growth, the worsening scarcity of fresh water, and the Political Economy present challenges resulting from protests and The Republic of Yemen is the poorest country regime changes. in the Middle East and North Africa region. The Following the World Bank’s re-engagement fragility of the political and security environ- with the Republic of Yemen in January 2012, an ment and the social unrest linked to persisting Interim Strategy Note (ISN)1, covering the period poverty are among key country risks. Most FY2013–14, was presented to the World Bank’s institutions are relatively weak, due to capacity Board of Directors in November 2012. The time issues, corruption, over- or understaffing in the frame of the ISN was planned to match that of the FROM THE POVERTY REDUCTION AND ECONOMIC MANAGEMENT NETWORK current transition government in the Republic of Yemen, of the PRSP and help in identifying and gathering informa- expected to continue through 2014. tion to construct the monitoring indicators, the M&E Unit The ISN framework is oriented around three main itself was not established. As a result, there was no central- principles: (i) intensifying participation and inclusion, ized unit within the government to establish a statistical especially among women and young people; (ii) strength- baseline, and no agreement on indicators or on which ening institutional capacity, governance, transparency, and agency would lead the data collection, resulting in tensions accountability; and (iii) enhancing the operational flexibil- between MOPIC and the National Statistics Office (NSO).2 ity of the World Bank–supported program. Strengthening At the time, the priority was not focused on data gathering. institutional capacity is essential to the development/ Rather, the focus was on establishing the poverty reduction redevelopment of the Republic of Yemen’s M&E program. program as outlined in the interim World Bank–IMF PRSP The transitional government has prepared a develop- monitoring review. ment program for the next two years, called the Transitional Program for Stabilization and Development. The program Phase II: Developing M&E includes several short-term emergency priorities, such as Capacity (2006–7) security stabilization, peaceful power transfer, and humani- tarian needs. Medium-term priorities include economic The Republic of Yemen’s “Strategic Vision 2025� is the growth sectors, basic infrastructure, social protection, country’s long-term development plan, first outlined in and good governance/state building. A joint donor group 2006, and oriented toward lifting its international ranking is working with the authorities on this program to assist from a least-developed country to one of “medium human during the fragile transition stage. development� by 2025. The plan was translated into a series of successive national plans, including the Third Socio-Economic Development Plan for Poverty Reduction, Phase 1: Post-Reunification (1995–2005) which is packaged together with the second PRS as a single For the purposes of this note, three phases of M&E develop- program under the DPPR. The government developed an ment have been identified. Following reunification, efforts M&E strategy for the DPPR that included specific measures toward promoting good governance and pursuing compre- to elaborate a poverty monitoring system covering data hensive reforms began in 1995 with the implementation generation, analysis, storage, and dissemination of informa- of the Economic, Financial, and Administrative Reform tion required to track trends in poverty and to help assess Program. Efforts continued with the development of the progress toward achieving PRSP goals. Second Five Year Economic Plan and Poverty Reduction The German Agency for International Cooperation Strategy (PRS). The government completed the Interim made significant contributions to building capacity Poverty Reduction Strategy Paper in 2000, which was fol- within MOPIC by placing an employee in-house for two lowed by the Republic of Yemen’s first Poverty Reduction years to assist with the development of the M&E master Strategy Paper (2003–5). However, the PRS-related plan- plan, as well as the M&E strategy for the Third Socio- ning was not reflected in budget lines because the country Economic Development Plan for Poverty Reduction. The did not have the required capacity given the weakness of government of Yemen developed the National Monitoring its financial systems. and Evaluation Strategy as an M&E system for the DPPR. In 2003, the Republic of Yemen benefited from a PRSP MOPIC led the creation of this system, which involved a trust fund grant; the purpose of the grant was to support participatory process with line ministries and other key the government of Yemen’s implementation of its first stakeholders to draft the goals and activities all the way PRSP, as well as to strengthen partnership, outreach, and through to the completed M&E framework. MOPIC’s dissemination activities with parliamentarians and civil so- M&E Unit carried out multiple rounds of workshops ciety, all of which were identified in the Joint International with line ministries and other stakeholders to whittle Monetary Fund (IMF)–World Bank Staff Assessment of down the number of monitoring indicators. The DPPR 2002. One of the four components housed in the grant stressed the need to reduce the budget deficit, strengthen included the institutional setup of a PRSP Monitoring Unit national accounts, and improve the country’s statistical in the Ministry of Planning and International Cooperation capacity. Also highlighted in the DPPR was the need for (MOPIC). Implementation of the program was delayed, sectoral reforms to increase efficiency and better manage however, because the training and awareness campaigns on scarce resources as well as expand the still insufficient the PRSP and PRSP monitoring were not initiated during budgeting systems. the first two years of the program. Although 12 workshops Training workshops covered a variety of M&E func- were held between 2003 and 2005 to improve awareness tions, including alignment of outcomes with related indi- 2 PREMNOTE MARCH 2013 cators and preparation and implementation of the M&E goals were extracted from the DPPR, including sector work programs consistent with the overall M&E strategy. strategies. Next, a list of implementation measures and/ or programs was constructed and plotted against goals Phase III: Operationalizing in a matrix format. Strong and robust indicators—con- sistent among agencies—were established, and a simple an M&E Unit (2007–10) analysis of monitoring indicators conducted. The project As mentioned earlier under phase I, a PRSP monitoring reduced the number of existing DPPR indicators from unit did not yet exist in MOPIC, though some degree of 450 to below 200.3 MOPIC held a significant number progress had been made with the training and workshops of workshops with individual sector representatives to that took place during phase II. The World Bank project drill down on what information was collected, requiring that began in October 2007 was created in response to the many negotiations to reach agreement on which minis- recommendation to establish and operationalize the PRSP try/agency would provide data for specific indicators. monitoring unit as stipulated in the M&E strategy. The Each line ministry or relevant agency now has a person(s) project supported capacity building for operationalizing devoted to M&E, as do the offices in the governorates. an M&E system in MOPIC to enhance the institutional Not every goal was met by the end of the project, as and individual M&E competencies of staff in MOPIC an indicator database developed in DevInfo4 was not and across the line ministries to help operationalize and complete at that time. There were also issues with data maintain an M&E system focused on poverty reduction. collection, as the same data were sometimes reported The process of building M&E capacity in the Republic from multiple agencies. of Yemen entailed a large number of (often overlapping) ac- Activity 2: Stakeholder consultation on the program tivities, which made progress challenging to follow at times. interventions. With the assistance of international and The MOPIC project coordinator mitigated this challenge national experts, the M&E team from MOPIC finalized with regular communication and document sharing with the outputs from Activity 1 for stakeholder consultations. the World Bank for terms of references and overall project The consultations were open to the government, the donor management, as well as regular coordination with the World community, and civil society. Bank Yemen Country Office on issues related to procure- Activity 3: Study tour to Uganda and training activi- ment and financial management. The project accomplished ties in the United States and Jordan (MOPIC 2009). As many of the original goals and made significant progress in part of the project, a delegation from the Republic of building M&E capacity. Yemen conducted a study tour to Uganda in November This initiative consisted of five main activities (see 2009 to learn from the country’s M&E practices and figure 1). experiences. Seven staff members from the M&E Unit at Activity 1: Refinement of outcomes and indica- MOPIC and from the line ministries participated, along tors. First, the relevant outcomes and development with three representatives from civil society organiza- tions. The group discussed many useful topics Figure 1. Activities to Build M&E Capacity in the with their Ugandan counterparts, including the Republic of Yemen (2007-10) inclusion of civil society organizations in the planning process and in M&E outputs; the strong Refinement of outcomes and indicators sponsorship from the prime minister’s office of the M&E coordination process to provide the leverage and power needed to push through the necessary Stakeholder consultation on the program interventions changes; and the importance of establishing a permanent secretariat in each ministry. The main lessons from the Uganda study tour are outlined Study tour to Uganda (November 1–7, 2009) and training in box 1. To learn from other experiences, Yemeni in the United States and Jordan government members also participated in training activities at the World Bank in Washington, DC, Planning and strengthening the M&E and in Amman, Jordan. implementation process Activity 4: Planning and strengthening the M&E implementation process. Technical assis- Enhancing capacity for planning M&E tance was provided to distill the M&E strategy implementation mechanisms document into working plans within the overall framework of agreed outcomes, indicators, and Source: Authors’ illustration. MARCH 2013 PREMNOTE 3 Box 1. Uganda Study Tour Yemen government participants: Visits to Ugandan government included: • Ministry of Planning and International Coordination • Office of the Prime Minister • Head, Monitoring and Evaluation Unit • Ministry of Public Service • General Manager for Studies —Office of Budgeting at the Ministry of Finance, Planning, and • General Manager, Sectoral Policy Economic Development • Head, Gulf Cooperation Council (GCC)-funded projects —Ministry of Education monitoring unit —Ministry of Water, Lands, and Environment • Three National Statistics Office coordinators —Ministry of Local Government Yemen civil society participants: —Planning Unit of the General Authority for Development of • Three representatives North Western Areas —National Planning Authority (NPA) Main lessons learned from the study tour: • Sponsorship from the Ugandan Prime Minister’s Office for the M&E coordination process at the central level provides the necessary leverage and political collateral to activate different commissions and structures across government ministries and institutions. This spon- sorship also contributes to ensuring timely access to data and information for regular reporting by the ministries, as well as coordination of government performance control functions at the central, ministry, sector, and local levels. • Connecting accountability with resource allocation provides incentives to activities across sectors. In Uganda, the Ministry of Finance, Planning, and Economic Development produces a biannual report on budgeting performance, with a focus on accountability, resource allocation to achieving certain results, and establishment of a budget control and accountability unit within the ministry to supervise activi- ties in the following five sectors: education, health, water, transport and public works, and agriculture. • A strong and independent (or quasi-independent) statistics office is the key to a successful M&E program. The development of a strate- gic plan to improve statistical activities with the aim of disseminating final results of censuses and surveys in no more than six months enhanced usage of such data in Uganda’s planning and budgeting. Key products of Uganda’s Statistics Bureau included: population and social surveys, data on the private sector and industry, information technology assessments, department surveys and capacity building, and statistical coordination services. • M&E efforts are greatly supported by establishing a permanent secretariat in each ministry to act as a technical bureau responsible for coordinating special tasks for the ministry. This supported the preservation of accumulated technical expertise and its concentration and ensured business sustainability in each individual ministry. Source: Authors’ compilation. proposed program interventions. The training and work- While the World Bank supported the project, it met shops provided the relevant teams with the skills required most of the objectives and made significant progress. How- to eventually implement the M&E master plan. ever, though the M&E Unit was established in MOPIC, Activity 5: Enhancing capacity for planning M&E most of the staff was actually pulled from other depart- implementation mechanisms. This activity developed a ments within the ministry (such that new hires were not draft plan, defining and establishing the monitoring struc- made). The database in DevInfo is not complete, and tures, and advancing the work into actual implementation. sometimes receives the same data from different sources, Developing the master plan formed the basis for implement- for example, the NSO and the Ministry of Health. Data are ing a coherent M&E system for DPPR-PRSP programs linked often delayed and have to be requested numerous times. to sector M&E systems. MOPIC has developed an M&E The project had a slow start—it was approved in March master plan and put in place an integrated M&E system 2007, but did not become effective until October—however, to implement the plan, linking it to sector M&E systems eventually a fully staffed M&E Unit was up and running, and aligning it to agreed outcomes, as well as adhering to supported strongly by the minister, raising its status and institutional, structural, and technical proposals built into strengthening its mandate. Nonetheless, there were some the master plan. MOPIC is responsible for coordinating challenges regarding low project implementation capacity the DPPR-PRSP monitoring process and therefore report- and political delays because key reports had to be approved ing structures to and from other sectors have been aligned by the Yemeni Cabinet before the project could move on to accordingly. subsequent new activities. 4 PREMNOTE MARCH 2013 M&E Champions M&E-related policies and activities had not yet taken place. The champions instrumental in moving forward the process Figures 2 and 3 illustrate the main M&E activities, as well of building M&E capacity included the former Deputy as some of the roles and responsibilities for M&E activities Prime Minister and former Minister of MOPIC (and cur- outlined in this note. rent Managing Director of the Social Fund for Develop- During the process of building M&E capacity, it ment) and the Vice-Minister of the DPPR. became clear that more cooperation with the NSO was The minister of MOPIC fully and openly supported needed. Line ministries occasionally complained of data the project. This moved transactions and activities along issues between themselves and the NSO, including a lack when delays cropped up and raised the level of respect of access to NSO data, or, duplicative and inconsistent re- of the M&E Unit among the other ministries and agen- sults. This led to some delays in data gathering and report- cies involved. MOPIC led the process of adopting a more ing. To address this issue, a Cabinet resolution was passed systematic approach to M&E and enhancing the capacity in 2010 (Cabinet Resolution No. 314) to reinforce and to assess the impact of the Republic of Yemen’s poverty reduction efforts and progress in achieving its strategic Figure 2. Fourth DPPR M&E Activities vision. The Social Fund for Development (SFD) is a local Yemeni agency that is known and respected for its own Raise M&E processes and was involved in several of the activi- Awareness Revise Laws ties. At the end of the project, a large degree of progress Build and was made, and the objectives remain important for the Capacity Regulations country, as highlighted in the Fourth DPPR, published in 2011. Drawing on lessons from the third DPPR and previous PRSPs, this fourth DPPR is the first plan to have Produce Analyze its own M&E framework and mechanism, with a separate M&E Policy Data and annex solely devoted to M&E. Information TASKS and Data Conclusions and Next Steps for Ensure Strengthening Results-Based Implemen- Assess Effects M&E in Yemen tation Include The objectives of the World Bank–funded project were, Citizen and still are, very relevant to the issues facing the Republic Voices of Yemen. Although the topic of M&E had been raised for several years before grant approval, implementation of Source: MOPIC 2011 (annex 1). Figure 3. Examples of Fourth DPPR Roles and Responsibilities • Receives and revises reports • Ensure availability of the data • Propose objectives and general Line Ministries Governorates and Districts MOPIC and data from central and lo- and information related to eco- policies for sector development. cal ministries and government nomic, social, and cultural de- • Set programs to implement bodies. velopment projects. DPPR policies and programs at • Organizes data, information, • Follow up on annual develop- the sectoral level. and indicators to improve sector ment plan implementation and • Prepare, assess, propose, and and governorate performance. derive statistical indicators for distribute development projects • Submits regular reports on the progress level. in the sector plan. progress, obstacles, and solu- • Prepare monitoring and prog- • Submit proposals related to tions to the Cabinet to support ress reports for DPPR projects amendments to be made to DPPR objectives. and submit to relevent bodies. the sector plan in light of any • Prepares final report of develop- developments. ment plans and achievements at national level. Source: MOPIC 2011 (annex 1). MARCH 2013 PREMNOTE 5 frame the involvement of the NSO in the development istrator for the Trust Fund for Statistical Capacity Building of the M&E process between MOPIC, the line ministries, (TFSCB). Helena Hwang, a Consultant for the Poverty and other agencies. Implementation of this resolution is in Reduction and Equity Department (PRMPR) of the World progress, though arguably at a slow pace. The government Bank, has worked at the Bank for six years, contributing does not yet have a centralized online system that links the to technical assistance and training for governments on statistics office and line ministries to the M&E Unit to al- building M&E capacity. She previously worked in manage- low for regular updating of all indicators. The Republic of ment consulting. Yemen also has a statistical master plan to be implemented by the NSO, which brings some coordination challenges. Notes As the official state statistical agency, the NSO is key to 1. An Interim Strategy Note is prepared for countries that many, if not most, of the monitoring indicators. are in transition from conflict or political crisis, countries Although the infrastructure for a monitoring system going through an unusually uncertain period, and countries has been established within MOPIC, the practical mea- where the Bank is reengaging after a prolonged hiatus. sures of identifying indicators and implementing a system 2. In the Republic of Yemen, the organization is referred for systematic M&E have not advanced further. After the to in English as the Central Statistics Office, or CSO. This grant closed, the next step was to be implementation of note uses the translation National Statistics Office, or NSO, the M&E master plan. However, shortly after grant closing, to prevent confusion with the common acronym for civil social unrest began, stalling further progress, which serves society organizations. to illustrate the additional challenges faced in fragile and 3. A list of 450 indicators was reduced to the current 194, conflict-affected country settings. as per the Fourth DPPR, released in April 2011. It was estimated that only 30 percent of the original indicators Acknowledgments even had baseline data to use for comparison. 4. DevInfo is the online database that was created as part The authors thank Wael Zakout (Yemen Country Man- of the World Bank project to operationalize the M&E Unit ager), Wilfried Engelke (Senior Economist, MENA Pov- in MOPIC. It is maintained and updated by the M&E Unit, erty and Gender Group), Ali Dahhaq (M&E Unit Head, and as the unit collects information from different depart- MOPIC), Christina Malmberg Calvo (Sector Manager, ments, it then enters it in to DevInfo, which can be accessed Poverty Reduction and Equity Department), Keith Mackay at: http://www.devinfoyemen.org/di6web/home.aspx. Most (Consultant, Poverty Reduction and Equity Department), of the Web site is in Arabic, except for the “Gallery� section, Gladys Lopez-Acevedo (Senior Economist, Poverty Reduc- which has some indicators in English. tion and Equity Department), and Anna Reva (Operations Officer, Poverty Reduction and Equity Department) for References their very helpful comments. The views expressed in this MOPIC (Ministry of Planning and International Cooperation). note are those of the authors. To access other notes in this 2008. “Monitoring and Evaluation Strategy for the Socioeco- series, visit www.worldbank.org/poverty/nutsandbolts. nomic Development Plan for Poverty Reduction (2006–10).� Monitoring and Evaluation Unit, Sana’a, Yemen. About the Authors ———. 2009. Back to Office Report: Exploratory Tour on Monitoring and Evaluation. M&E Unit, November 1–7, Kampala, Uganda. Ingrid Ivins, a Statistician in the Development Economics ———. 2011. “The Fourth DPPR M&E Framework, Towards a Data Group (DECDG), managed the World Bank proj- Results-Based M&E.� Sana’a, Yemen. ect profiled in this case study. Ingrid has worked at the World Bank. 2009. “Country Assistance Strategy for The Repub- World Bank for 20 years, with experience in West African lic of Yemen for the Period FY 2010–13.� Yemen Country countries and the Middle East and North Africa (MENA) Management Unit, Washington, DC. ———. 2012. “Interim Strategy Note for The Republic of Yemen region, and has led multiple projects to build statistical for the Period FY 2013–14.� Yemen Country Management capacity in the MENA region. She is currently the admin- Unit, Washington, DC. This note series is intended to summarize good practices and key policy findings on PREM-related topics. The views expressed in the notes are those of the authors and do not necessarily reflect those of the World Bank. PREMnotes are widely distributed to Bank staff and are also available on the PREM Web site (http://www.worldbank.org/ prem). If you are interested in writing a PREMnote, email your idea to Madjiguene Seck at mseck@worldbank.org. For additional copies of this PREMnote please contact the PREM Advisory Service at x87736. This series is for both external and internal dissemination 6 PREMNOTE MARCH 2013