67662 The World Bank Human Development Network - Education System Assessment and Benchmarking for Education Results (SABER) GRENADA Education Management Information System (EMIS) COUNTRY REPORT Emilio Porta, Jennifer Klein, Gustavo Arcia and Harriet Nannyonjo February 2012 Acknowledgements This report was prepared by a team led by Emilio Porta, Senior Education Specialist at the Human Development Network/Education at the World Bank; and consisting of Gustavo Arcia, Consultant to the Human Development Network/Education of the World Bank and Senior Economist at Analítica LLC in Miami, Florida; Jennifer Klein, Consultant to the Human Development Network/Education at the World Bank, and Harriet Nannyonjo, Senior Education Specialist, LCSHE, World Bank. The report was prepared under the guidance of Elizabeth King, Robin Horn and Chingboon Lee. The views expressed here are those of the authors and should not be attributed to the World Bank Group. All data contained in this report is the result of collaboration between the authors, the Organization of Eastern Caribbean States, and participants in the benchmarking exercise. All errors are our own. This benchmarking study arose from an active partnership between the Education Reform Unit of the Organization of Eastern Caribbean States (OECS) and the World Bank. The benchmarking exercise was done during an OECS workshop conducted in Castries, St. Lucia, from January 23 to January 28, 2011, with the participation of government officials from Antigua & Barbuda, the Commonwealth of Dominica, Grenada, St. Kitts & Nevis, St. Lucia, and St. Vincent & the Grenadines. A delegate from Montserrat also attended as an observer. The workshop and benchmarking exercise were done under the invaluable leadership of Marcellus Albertin, Head of the Education Reform Unit (OERU) at the OECS. His unflagging support, enthusiasm, and institutional supervision were fundamental for the cooperation of all participants and for the success of the workshop. To him we owe a great deal of gratitude. We would like to thank the OERU staff that helped us with workshop logistics, especially Emma Mc Farlane- Jouavel and Beverly Pierre. We would also like to thank the workshop participants: Doristeen Etinoff, Priscilla Nicholas, and Patricia George from Antigua & Barbuda; Ted Serrant, Robert Guiste, and Weeferly Jules from Dominica; Pauleen Finlay, Michelle Peters, and Imi Chitterman from Grenada; Gregory Julius from Monserrat; Quinton Morton, Ian Gregory, and Laurence Richards from St. Kitts & Nevis; Kendall Khodra, Nathalie Elliott, Sisera Simon, Evariste John, and Valerie Leon from St. Lucia; Dixton Findlay, Keith Thomas, and Junior Jack from St. Vicent & Grenadines; Darrel Montrope, Jacqueline Massiah, Sean Mathurin, and Loverly Anthony- Charles from the OECS. Abbreviations EMIS Education Management Information System MOE Ministry of Education OECD Organization for Economic Cooperation and Development OECS Organization of Eastern Caribbean States SABER System Assessment and Benchmarking for Education Results SEAT SABER EMIS Assessment Tool UIS UNESCO Institute for Statistics UNESCO United Nations Educational, Scientific, Cultural Organization GRENADA ESTABLISHED  Aspect of Data Quality Benchmark Prerequisites of Quality Established ¤¤¤¢ Assurances of Integrity Established ¤¤¤¢ Methodological Soundness Established ¤¤¤¢ Accuracy and Reliability Emerging ¤¤¢¢ Serviceability Emerging ¤¤¢¢ Accessibility Established ¤¤¤¢ 1 BACKGROUND Education Data in Grenada The current Education Management Information System (EMIS) in Grenada relies on a paper data collection, but an electronic system is under development. The development of the electronic EMIS was delayed following Hurricane Ivan in 2004 when funding shifted to school reconstruction over EMIS development. As of January 2011, the electronic system had been piloted in one secondary school and 15 primary schools. EMIS STAFF. EMIS staff includes one Planning Officer, one Project Planner, one Statistician, one Statistical Clerk, two Information Technology (IT) Officers, one Education Officer for Information and Communications Technologies (ICT) in schools, and three Information Officers, two of which have been specifically trained in EMIS. These staff members are located in the Ministry of Education (MOE) Information Systems Department and communicate with the seven district education officers and with schools. FACILITIES AND EQUIPMENT. The electronic system will rely on the broadband connection that is now available in 95% of schools. All primary and secondary schools have one computer allocated for administrative use. EMIS DATA. Grenada’s EMIS collects the following types of data: Ü Student data – age, sex, grade, monthly attendance records. Ü Teachers’ monthly attendance records and qualification status. Ü School and classroom infrastructure data. Ü School revenue and expenditure. DATA COLLECTION AND PROCESSING. In the current system, schools collect data and send the completed paper forms to the district officers. District officers forward the forms to the Ministry of Education, which does all verification and processing of the data prior to the production of the reports. Any issues with a form are communicated directly with the principal of the school. PUBLICATIONS. Data are distributed to stakeholders including schools, other ministries, and national and international organizations. 2 The EMIS in Grenada ESTABLISHED:  In January 2011, Grenada’s EMIS was assessed using the SABER-EMIS Assessment Tool (SEAT) and overall the EMIS was categorized as ESTABLISHED (0.62). Among the six Organisation of Eastern Caribbean States (OECS) countries, Grenada had the fourth highest overall score behind Dominica, St. Kitts, and St. Lucia. Grenada outperformed the OECS Figure 1. SABER EMIS Scores in the OECS average on three of the SEAT’s Aspects of Quality: Prerequisites of Quality, Assurances of Integrity, and Accessibility (Table 1). Grenada’s score on Accessibility (0.69) was far above the OECS average (0.53). Grenada was just below the OECS average on Methodological Soundness and Accuracy and Reliability. Serviceability, Grenada’s lowest score (0.50), was the score furthest below the OECS average due to a lack of established processes for data revisions. The next sections of this country report will analyze Grenada’s performance on the sub-components of each Aspect of Quality in order to present a detailed portrait of the strengths and weaknesses of the EMIS and many concrete actions that the country can take to improve education data quality. Table 1. SABER EMIS Scores in the OECS Countries (2011) OECS Dominica Antigua Grenada St. Kitts St. Vincent St. Lucia Average Pre-Requisites 0.70 0.52 0.68 0.66 0.45 0.64 0.61 of Quality Assurances of 0.58 0.53 0.61 0.44 0.50 0.64 0.55 Integrity Methodological 0.83 0.50 0.67 0.67 0.83 0.67 0.69 Soundness Accuracy and 0.70 0.48 0.58 0.75 0.53 0.58 0.60 Reliability Serviceability 0.61 0.29 0.50 0.79 0.43 0.68 0.55 Accessibility 0.47 0.47 0.69 0.61 0.36 0.56 0.53 Overall 0.65 0.46 0.62 0.65 0.52 0.63 0.59 Latent Emerging Established Mature 0 – 0.3 0.31 - 0.59 0.6 - 0.79 0.8 - 1 3 PREREQUISITES OF QUALITY ESTABLISHED:  Grenada has ESTABLISHED (0.68) the Prerequisites of Figure 2. Prerequisites of Quality Quality (Figure 2) necessary to support an EMIS and was ESTABLISHED above the OECS average score (0.61), but still has a few areas    in need of improvement. General laws are in place specifying responsibilities for collecting and disseminating education data, but they need to be more specific (Table 2, 0.1). There is excellent data sharing between public schools and the MOE, but less private school, university, and finance data is shared with the MOE (0.2). Institutions have a legal obligation to report data, but there are no established penalties for not reporting (0.4). Currently, only 40 percent of schools report data on time. Current EMIS staffing and training levels are insufficient and hardware needs to be updated (0.5). The management of existing human and physical resources is inefficient and there is little monitoring of resource use (0.6). Quality awareness (0.8) was the Grenada’ lowest score: Management seems to promote ad hoc quality improvement measures, but processes are not in place to focus on quality. Formal internal reviews occur annually to monitor the data quality and user feedback is welcomed (0.9). The annual review mechanism adequately addresses new and emerging data requirements with users and stakeholders (0.11). OECS Table 2. Prerequisites of Quality: Subcomponents Grenada Benchmark Average Responsibility for collecting and disseminating education data is Established 0.1 0.75 0.75 clearly specified  Established 0.2 Data sharing and coordination among different agencies are adequate 0.75 0.50  Individual/personal data are kept confidential and used for statistical Established 0.3 0.75 0.79 purposes only  Statistical reporting is ensured through legal mandate and/or Established 0.4 0.75 0.58 measures to encourage response  Staff, facilities, computing resources, and financing are Emerging 0.5 0.50 0.63 commensurate with the activities  Processes and procedures are in place to ensure that resources are Emerging 0.6 0.50 0.63 used efficiently  Education statistics meet user needs and those needs are monitored Established 0.7 0.75 0.75 continuously  Emerging 0.8 Processes are in place to focus on quality 0.25 0.63  Established 0.9 Processes are in place to monitor the quality of data processes 0.75 0.33  Processes are in place to deal with quality considerations in planning Established 0.10 0.75 0.58 the stat program  Mechanisms exist for addressing new and emerging data Mature 0.11 1.00 0.54 requirements  4 ASSURANCES OF INTEGRITY ESTABLISHED:  Grenada and St. Lucia were the only two OECS Figure 3. Assurances of Integrity in the OECS countries to have ESTABLISHED Assurances of Integrity. Grenada had two MATURE sub-components: 1) Choices of sources, statistical techniques and ESTABLISHED decisions on dissemination are sound (Table 3, 1.3) and    2) Advance notice is given immediately for major changes in methodology, source data, and statistical techniques (1.8). Despite MATURE performance on some sub- components, others were not as strong. For example, only informal mechanisms protect the professional independence of the data producing institution (1.1). This is a key area for improvement since a law of this type can ensure that statistics are produced on an impartial basis. The statistical department only comments on technical errors but not misinterpretations (1.4). Also, the terms and conditions under which statistics are collected, processed, and disseminated have not been written in detail, but a general version of this information is available to the public (1.5). The process of employing EMIS staff is transparent and credentials considered for recruitment and promotion (1.2). Also, staff guidelines are in place but the communication of these guidelines could be improved (1.9). OECS Table 3. Assurances of Integrity: Subcomponents Grenada Benchmark Average Emerging 1.1 Statistics are produced on an impartial basis 0.25 0.38  Emerging 1.2 Professionalism of staff is actively promoted 0.50 0.42  Choices of data sources and statistical techniques are made solely by Mature 1.3 1.00 0.83 statistical considerations  Agency is entitled to comment on erroneous interpretation and misuse Emerging 1.4 0.25 0.58 of statistics  Emerging 1.5 Terms and conditions are available to the public 0.50 0.33  Public is aware of internal governmental access to statistics prior to Established 1.6 0.75 0.38 their release  Established 1.7 Products of education statics agency are clearly identified 0.75 0.50  Advanced notice is given of major changes in methodology, source Mature 1.8 1.00 0.71 data, and statistical techniques  Guidelines for staff behavior are in place and are well known to the Emerging 1.9 0.50 0.83 staff  5 METHODOLOGICAL SOUNDNESS ESTABLISHED:  In terms of Methodological Soundness, Figure 4. Methodological Soundness in the Grenada’s EMIS is ESTABLISHED (0.67). OECS countries Grenada scored just below the OECS average (0.69) and had the same score as both St. Kitts and St. Lucia (Figure 4). Grenada earned one MATURE score due to its ESTABLISHED    use of internationally accepted standards and guidelines for structure, concepts and definitions (Table 4, 2.1) established by the UNESCO Institute for Statistics (UIS) and the OECS Education Reform Unit (OERU). Grenada also follows the International Standard Classification of Education (ISCED) in all education sector data except financial data (2.3). Expanding the use of ISCED to financial data would ensure complete consistency with ISCED and improve Grenada’s score on this subcomponent. Currently, Grenada’s EMIS produces around 65 percent of UIS indicators annually, which results in a EMERGING benchmark on the scope of statistics sub-component (2.2). Expanding the scope of statistics produced to 100 percent of UIS and OECD indicators is ideal and can enable additional domestic, regional, and international education policy analysis. OECS Table 4. Methodological Soundness: Subcomponents Grenada Benchmark Average Overall structure, concepts and definitions follow regionally and Mature 2.1 internationally accepted standards, guidelines, and good 1.00 0.83  practices Scope is in accordance with international standards, guidelines, Emerging 2.2 0.25 0.42 or good practices  Classification systems are consistent with international Established 2.3 0.75 0.83 standards, guidelines, or good practices  6 ACCURACY AND RELIABILITY EMERGING:  The Accuracy and Reliability of Grenada’s EMIS data Figure 5. Accuracy and Reliability is EMERGING (0.58) and Grenada’s score was just below the OECS average (0.60) (Figure 5). Grenada earned a MATURE score on three sub-components in Table 5: Ü Other data sources are routinely assessed (3.4) EMERGING Ü Intermediate results are validated against other    information (3.7) and Ü Statistical discrepancies and problems in statistical outputs are investigated (3.8). Source data in Grenada are well defined and classified, and they have a time stamp (3.2). Also, statistical discrepancies, other potential indicators, and problems in statistical outputs are investigated, but results of investigations are not made public (3.9). Grenada had 2 LATENT sub-components due to the lack of timeliness of data (3.3) and studies of revisions to inform the process (3.10). Grenada could improve by working on timeliness, conducting studies of revisions, expanding the comprehensiveness of data collected (3.1), and also by improving the statistical techniques used for data compilation (3.5), editing, transformation, and analysis (3.6). OECS Table 5. Accuracy and Reliability: Subcomponents Grenada Benchmark Average Source data are obtained from comprehensive data collection that Emerging 3.1 0.50 0.58 takes into account country-specific conditions  Data are reasonably confined to the definitions, scope, classifications, Established 3.2 0.75 0.50 and time of recording required  Latent 3.3 Source data are timely (6 months after event) 0.00 0.46  Other data sources, such as censuses, surveys, and administrative Mature 3.4 1.00 0.42 records, are routinely assessed  Data compilation employs sound statistical techniques to deal with Emerging 3.5 0.50 0.79 data sources  Other statistical procedures (data editing, transformations, and Emerging 3.6 0.25 0.63 analysis) employ sound statistical techniques  Intermediate results are validated against other information where Mature 3.7 1.00 0.67 applicable  Statistical discrepancies in intermediate data are assessed and Mature 3.8 1.00 0.92 investigated  Statistical discrepancies and other potential indicators or problems in Established 3.9 0.75 0.71 statistical outputs are investigated  Studies and analyses of revisions are carried out routinely and used Latent 3.10 0.00 0.33 internally to inform the processes  7 SERVICEABILITY EMERGING:  The Serviceability of Grenada’s EMIS data is Figure 6. Serviceability in the OECS EMERGING (0.50) and the average score is below the OECS average (0.55) but higher than both Antigua and St. Vincent. Grenada’s sub-component scores vary from LATENT to MATURE. Grenada’s only MATURE score was a EMERGING result of regular cross-checking and consistency    checking for all data to ensure that statistics are consistent within the dataset (Table 6, 4.3). Grenada conducts an annual census of enrollments but financial data is not currently collected (4.1). Administrative school census data are released 3 months or less after the initiation of the school year (4.2). Time series data are available for more than five to ten years, but no established procedures exist for revisions (4.4/4.6) and no preliminary data are produced (4.7). In general, Grenada could improve the Serviceability of its EMIS by annually collecting financial data, releasing administrative data within two months of the start of the school year, and establishing revision processes. OECS Table 6. Serviceability: Subcomponents Grenada Benchmark Average Established 4.1 Periodicity follows dissemination standards 0.75 0.96  Established 4.2 Timeliness follows international dissemination standards 0.75 0.63  Mature 4.3 Statistics are consistent within the dataset 1.00 0.71  Statistics are consistent or reconcilable over a reasonable Emerging 4.4 0.50 0.54 period of time  Statistics are consistent or reconcilable with those obtained Emerging 4.5 0.50 0.33 through other data sources and/or statistical frameworks  Latent 4.6 Revisions follow a regular and transparent schedule 0.00 0.21  Latent 4.7 Preliminary and/or revised data are clearly identified 0.00 0.46  8 ACCESSIBILITY ESTABLISHED:  Grenada has ESTABLISHED (0.69) EMIS Accessibility Figure 7. Accessibility in the OECS ESTABLISHED and had the highest score of the OECS countries.    Grenada earned three MATURE scores: Ü EMIS statistics are clearly presented with disaggregation and underlying data for charts (Table 7, 5.1); Ü data are released to all users at the same time (5.4) Ü all statistical releases identify a contact person in case of required assistance (5.8). Grenada has established a pre-announced schedule for annual data releases, but the data is typically released more than six months late (5.3). There are procedures in place for releasing non-published data and non- confidential data to a restricted group (5.5), and metadata are also available upon request (5.6). No catalogs of publications and other services are available (5.9). Accessibility is one of the key missions of an EMIS because it creates and maintains the public image of the EMIS and enables greater accountability. Grenada could develop a more accessible EMIS by improving the timeliness of data releases, making metadata and data catalogs publically available, and creating a catalog of publications and other services. OECS Table 7. Accessibility: Subcomponents Grenada Benchmark Average Statistics are presented to facilitate proper interpretation and Mature 5.1 1.00 0.96 comparisons (layout, clarity of texts, tables, and charts)  Emerging 5.2 Dissemination media and format are adequate 0.50 0.54  Emerging 5.3 Statistics are released on a pre-announced schedule 0.50 0.38  Mature 5.4 Statistics are made available to all users at the same time 1.00 0.79  Established 5.5 Statistics not routinely disseminated are made available upon request 0.75 0.75  Documentation on concepts, scope, classifications, basis of recording, data sources, and statistical techniques is available, and differences Established 5.6 0.75 0.58 from internationally accepted standards, guidelines, or good practices  are annotated Established 5.7 Levels of detail are adapted to the needs of the intended users 0.75 0.38  Mature 5.8 Contact points for each subject field are publicized 1.00 0.38  Catalogs of publications and other services, including information on Latent 5.9 0.00 0.00 any charges, are widely available  9