35676 TRANSPORT NOTES ROADS AND RURAL TRANSPORT THEMATIC GROUP THE WORLD BANK, WASHINGTON, DC Transport Note No. TRN-29 February 2006 Success Factors for Road Management Systems Kevin McPherson and Christopher R. Bennett _________________________________________________________________________________________________________ Over the last 20 years most road agencies have implemented some form of computerized road management system (RMS). The purpose of these systems is to assist the road agency in the planning and prioritization of road investments. While some systems have been successful, there have also been many which have `failed' in one or more areas. This is in spite of large investments of time and money. While it is often easy to identify the symptoms of failure, the causes are often complex and multi-layered. However, for every failed system, there is a more successful implementation somewhere in the world, in an agency that often suffers from similar problems and constraints. This Note describes the different factors associated with `successful' RMS implementations. It is a summary of the report `Success Factors for Road Management Systems'. The full report is available for download from www.road- management.info. The Note has been produced with the financial assistance of a grant from TRISP: a partnership between the United Kingdom Department for International Development and the World Bank for learning and sharing of knowledge in the fields of transport and rural infrastructure services. condition, traffic and related data, for road planning and 1 What is a Road programming. Associated with the RMS are appropriate Management System? business processes to use the RMS to execute the business needs of the roads agency. The major functions of the road management process can be categorized as: Processes, People and Planning; Technology Programming; Preparation; and As shown in Figure 1, like any system, an RMS relies on three fundamental components: Processes, People, and Operations. Technology. There must also be a commitment to adequate funding. If any of these are lacking, the system A road management system (RMS) is concerned with road monitoring, planning and programming. Major activities include: Processes People Funding Funding Needs Assessment; Strategic Planning, including budgeting for development and asset preservation; Technology Development, under budget constraints, of Successful multi-year works expenditure programs; and projects properly address all three Collection of Data. All of the above activities factors need data. Major data items include road Funding inventory, condition, traffic, and economic data. will not be successful. An RMS is defined here as any system that is used to Figure 1: Processes, People, Technology and store and process road and/or bridge inventory, Funding Page 2 Transport Note No. TRN-29 February 2006 The best technology in the world will ultimately fail if Key Performance Indicators (KPI): implemented in an environment where there are no Indicators by which the agency assesses its people to run it, or where the processes are not in place performance, for example the average condition to utilize it. For an RMS to be successful, the importance of the network, number of km of pavements of each component must be clear. maintained, etc. Regular data collection provides the basis for many of the KPIs. Executives and managers need to be demonstrably committed to the system, both in their relations with Five-Year Goals: The goals reflect the overall external stakeholders and internally in their agency objectives of the agency, and should be through good management principles. Policies should consistent with their long-term Strategic Plan. explicitly state the goals and objectives of the Annual Asset Management Plan: The annual organization in regard to road asset management, and asset management plan describes the specific procedures should detail exactly how the RMS will be activities that are required to achieve the used to achieve these goals. agency's goals, including detailed annual work programs. This will include asset management Ideally, a separate organizational unit would have explicit plans for pavements and structures as well as responsibility for the system and data, and would be identifying areas where new roads and/or staffed with well-qualified and trained personnel who are capacity improvements are required. pro-active in developing and expanding the system. Financial Plan: The financial plan will describe Apart from the key process of asset management, a the current and future sources of financing road number of other subsidiary processes would contribute, maintenance (Road Maintenance Fund, including budgeting, financial management, human government funding, international donor resource management, and IT management. These assistance etc.) and disbursement schedules. subsidiary processes would be essential in ensuring sustainability of the system in terms of data collection, The RMS should be capable of supporting all of the above staff training and development, hardware and software areas. maintenance etc. Needs Assessment The data collection equipment and IT hardware and software should be fit for purpose, actively used, properly One of the key objectives in implementing an RMS is to maintained, and covered by some sort of maintenance provide justification for budget, and to help direct limited agreement and replacement strategy. funds towards those areas where the return on investment will be greatest. This note analyzes each of the key areas above, and identifies the factors that contribute to successful A `Needs Assessment' is an unconstrained analysis of the implementation of Road Management Systems. needs of the road network. It is performed in order to calculate the real costs of maintaining or improving the value of the asset, or to bring it up to some agreed level of service if there is an existing maintenance backlog. It Processes is unconstrained by budget, therefore it helps to develop a strategy for the agency. It may also be used by the The RMS must play an active role in the agency's agency as a justification for budget requests, which business. This requires appropriate functionality, and generally arise from a constrained analysis. ensuring that it fits into the organization's business processes. Asset Value `Asset Value' is an important concept in monitoring The Annual Report/Business Plan infrastructure. Agencies should have clearly defined goals and objectives to maintain or to increase asset value over Most agencies have a statutory requirement to prepare time. Asset value also provides a mechanism to compare an annual report or business plan. If the RMS contributes the value of investments in different types of to this report, it will help ensure its sustainability. infrastructure either within an agency or within a country. The Annual Report sets out the existing performance and Asset value can be measured relatively easily using data also forecasts future investments. Elements typically that is normally readily available within a road agency. include: Page 3 Transport Note No. TRN-29 February 2006 This data includes inventory, condition, and cost of new However, if the RMS is an integral part of the planning construction, rehabilitation and repair. Schliessler and process, and is giving sensible predictions, then a Bull (2004) present a simple technique for estimating substantial portion of the predicted program would asset value. The literature contains a number of normally be implemented. examples of more detailed approaches. All RMS would benefit from having a mechanism for Accuracy of RMS Forward Work monitoring the rate of program implementation. Programs Most RMS are used to prepare annual forward works programs. These predict the future maintenance needs People for the network, usually on section-by-section basis. A key issue to be considered in the RMS is how realistic the Institutionalization predicted maintenance program reflects the actual maintenance requirements. In other words, is the system Institutionalization means "to make part of a structured producing the correct results? and well-established system". Aspects considered to be important in institutionalization of an RMS are those that are similar for any management system. They include: Correctness may be defined in terms of: The type of maintenance treatment being Establishment of an organizational unit with recommended; specific responsibility for the system; The extent and location of that treatment; and Establishment of a budget for the operation of the entire system, including all staffing, The recommended year for implementation. equipment, data collection (contract or in- house), field travel, etc.; Prior to any agency implementing an RMS for planning Presence of appropriately qualified personnel, purposes it needs to ensure that the predictions are with good management skills, with access to and sensible in the local context. This is done using a `hit- control over their budget; rate' (ie the number of correct predictions) analysis. Specific and detailed job responsibilities for all It is important to appreciate that there will never be a aspects of the system; 100% agreement between predictive models and A program for continual quality improvement; assessed maintenance needs. Often, predictive models take into account factors such as economic evaluation, Clear management reporting; and budget limitations, etc. which are usually omitted from A regular audit of all elements and the taking of engineering assessments. There are also often problems corrective actions where necessary. with data, the system may not take into account all issues (eg traffic safety considerations) and there are the Training basic limitations with any statistical model. As a minimum, the training portion of an implementation However, by instituting a robust feedback mechanism project for of an RMS should cover: which identifies areas where the model's predictions are significantly different to assessed needs, the overall Principles of network referencing; accuracy and relevance of the predictions is improved. Experience has shown that improvements to data quality Roles and responsibilities; and regional calibration can increase the hit-rate Data collection policies and procedures; accuracy to over 80%. However this requires a program Network-level versus project-level data; of continual quality improvement in the agency. Accuracy and tolerance on data; Implementation of RMS Programs Data quality assurance; System operation; An indication of the role of the RMS in an agency is the degree to which the RMS programs are actually Reporting; and implemented. Due to budget, technical and logistical Auditing. factors even if the predicted forward works program was 100% accurate, it would not be 100% implemented. Page 4 Transport Note No. TRN-29 February 2006 It is also important that basic contract management skills use of IT across the organization, and a piecemeal are available in order to manage data collection approach to IT implementation. All of these can lead to procurement. Many countries outsource data collection, loss of sustainability on the IT side. for a variety of reasons as discussed later. The ability to prepare a Terms of Reference, evaluate proposals, Key responsibilities for IT Divisions should include: negotiate contracts, administer and quality assure a data collection contract requires strong management abilities, IT Budgeting; skills and experience that are often taken for granted in IT Procurement; an RMS implementation. Training in these areas is also seen as a necessary part of the total picture. Network Administration; Systems Administration (including data In addition, an agency should also have a set of training backups); materials with which to train its own staff. Organizations IT Security; often claim that staff retention is a problem, also staff are often moved into different job areas or promoted. It Development of IT Architectures; is therefore important that training materials are User Awareness and Training; and available, and that the client's staff are trained to deliver User Support. new training if and when it becomes necessary. Many clients opt for a `train-the-trainer' approach in implementation of systems, often to cut expensive IT Budgeting consulting time from the implementations, but without IT Budgets should cover procurement of all new development and handover of training materials, this hardware and software, operation of the network cannot happen successfully. (including costs of leased lines etc.), hardware and software replacement, warranty and maintenance Continual Quality Improvement agreements, support etc. Quality management is vital to the success of any IT Architectures and Standards business enterprise. The continual improvement of the organization's performance should be a permanent A `Technology Architecture' is a series of principles, objective of the organization (ISO, 2000). Quality is an guidelines or rules used by an organization to direct the institutional responsibility, rather than the responsibility process of acquiring, building, and maintaining its IT of any one particular individual or office. resources. It acts as a framework within which the IT infrastructure can be established, and should support the Road agencies that have successfully employed RMS for a applications and data which are required by the number of years have, during the course of their organization to support its business needs. implementations, initiated programs of continual quality improvement. This has been apparent in many of the The benefits of establishing a technology architecture areas covered in this note--data collection, training, road are: network modeling, and hit-rate analysis. Control of diversity, and establishment of No system, and no organization, is static. Continual effort standards within the organization: Diversity is required to improve it at all times. This requires drive increases technology and support costs, and can and dedication from the agency, and particularly from the obstruct interoperability, information sharing individual staff involved. and system integration. Easier procurement process: An architecture eases and speeds up the procurement process Information Technology because only appropriate products will be considered. Economies of scale can also be IT Management applied, both to major expenditures and also to supplies. Any sizeable organization implementing any form of Clarifies long-term goals and provides a management system, such as an RMS, should have a building-block to respond to environmental separate IT Division. If there is no Division with overall changes: An architecture increases the order responsibility for IT, then it is likely that there will be a lack of IT policy, lack of a strategy for development and Page 5 Transport Note No. TRN-29 February 2006 and predictability of future technology upgrades supplier responds to other client requests for and expansions. enhancements; and Increases the stability and reliability of Exchange of Ideas: There are often user network services. conferences held by the supplier at which ideas and experiences can be shared with other users. For example, if an agency maintains three separate database systems, then it requires that database and There are also potential disadvantages including: systems administrators have advanced skills in three separate packages. It usually also means that licensing Requirements: The functionality may not be fees on the whole are significantly higher than if a site- exactly what is required, so some workarounds license or enterprise license was procured. Also, three may be needed; separate support and maintenance agreements are required. Upgrades to a newer version of the database Customization: The time to develop new ideas management system (DBMS) are also much easier to may take longer, since the software supplier has plan and implement. Any in-house developers of a responsibility to existing clients; and applications can also concentrate on one database Cost: Many agencies have difficulties in funding environment instead of multiple. support and maintenance agreements1. Without a formal technology architecture for the A well-chosen COTS package from a good software organization, and without proper control of all supplier is always preferable to bespoke development. procurements, it is likely that there will be piecemeal IT There are many such packages on the market in the implementation within the organization. This will increase areas of road and asset management as well as road costs, and decrease efficiencies thereby influencing the planning. Some key features of these systems that likely success of any IT project, including an RMS. should be included in any requirement or specification for RMS are given in Box 1. Use of Commercial Off-the-Shelf (COTS) Software Careful review and assessment of COTS packages and the suppliers is essential. This assessment needs to In industry, most large organizations have a policy of consider the financial stability of the company, the using commercial off-the-shelf software (COTS) if at all technical capabilities of the system, and the product's possible, rather than developing bespoke software, direction. Of particular importance is prior to (either in-house of through consultancy projects). The procurement conducting a `Gap Analysis' which potential advantages of COTS over bespoke software are: establishes the system's functionality and features relative to the agency's operating procedures. There may Cost: They are usually much cheaper to buy be differences which necessitates either changes to the than to develop from scratch; procedures or refinement of the software. Independence: The client is not tied to one For agencies that have existing developed systems, it is particular consultant since many may offer often quoted that they do not wish to change from their implementation support for the application; bespoke system because they would lose the past Timeframe: It can be implemented more investment. However, that investment would not quickly in the organization (ie it does not have necessarily be lost by any future move to a COTS to be developed first); package. Normally with an RMS, the data represents 70 ­ Experience: It has usually been implemented in 80% of the cost of implementing the system. If the data a number (sometimes many hundreds) of other can be brought across to the new system (which should client organizations for a number of years, and be a requirement of any implementation project), then has therefore been subject to rigorous user much of the investment that has gone into the original testing in addition to the normal in-house system is retained. testing of the software supplier; Functionality: It often provides more useful functionality than the client originally considered; 1 Support and maintenance agreements are usually in the Ongoing Development: There is usually order of 12 ­ 20% per annum of the original cost of the continual upgrades of the software as the software. However, that cost must be compared with the cost of consultants developing enhancements. Page 6 Transport Note No. TRN-29 February 2006 Functional Requirements for an RMS complete replacement of systems and re-training of staff every 4 ­ 5 years. Box 1 summarizes some of the key functional requirements for any RMS. A full list is given in Box 1 Some Key Functional Requirements for an RMS McPherson and Bennett (2005). It should be noted that this is provided as a guide only. It is not intended to be en exhaustive list of functionality required for every RMS, Terminology and Local Language: All Screen Labels, Menu Items, and Reports should and in fact some agencies may decide that they may not be configurable to the local language and terms. wish to have all the features listed. Road Network Referencing: Different network referencing schemes should be However, as their experience with an RMS grows, supported. agencies tend to find that they need more and more in- Network Editing: Should permit splitting built functionality and features. Most reputable COTS and joining of road sections, also modification of systems support more than 90% of the functions in Box 1 road section lengths, while preserving integrity to some degree. The exact way in which these functions of all data stored against the affected sections. are implemented may differ, so it is always worth having User-Defined Items and Attributes: detailed discussions with suppliers and other users to Should allow the user to define the types of data to be stored, and to define what attributes are gain a better understanding of the working of the to be stored against each type of inventory. systems prior to procurement. It is also necessary to There should be no restriction on the number combine functional requirements with technical and type of items or their attributes. requirements, to match the architecture of the agency ­ Multi-Media Storage and Display: Should ie operating systems, RDBMS, GIS, and other allow storage and display of multi-media objects applications. (eg photographs, video clips etc.) as attributes of inventory items. Finally, many COTS packages manage both roads and Security: Allow security setup so that different users may have access to different bridges, as well as other infrastructure assets. There are application modules and data activities. many benefits to be gained from maintaining all asset Integration with GIS: The RMS should inventory and condition data in a single system, not the integrate with GIS to allow display of inventory least of which is that it enforces the use of a common and condition data against maps of the road network referencing system. network. Reporting: Reporting should be flexible, and System Acceptance Testing the interface must enable the user to define his own reports from the GUI without reprogramming of the application. A set of acceptance tests should be agreed with the Automatic Sectioning: An automatic supplier/consultant at the start of the project, and these sectioning function to collate and summarize should be gone through formally when the software or data for analytical purposes. The user should be implementation is completed. Few road agencies actually able to define the sectioning criteria using any perform proper and thorough acceptance testing, often of the key inventory or condition data. because their IT divisions are not fully involved in the Data Transformations: Sectioned data need implementation (see above), and also because the task to be transformed to the automatically generated sections using different criteria. of acceptance is delegated to junior staff. Since the agency will be using the RMS for managing their business Trend Analysis: Should allow production of reports/graphs showing trends any attribute of it is essential that it be properly tested. Formal any database item over time. acceptance tests reduce the possibility of bugs in the Audit Trail: All data changes should be system, and can minimize any ease of use issues. audited, including time of change, username responsible for making the change, and value of Hardware and Software Support and previous data item. Maintenance Other Asset Inventory: Should permit storage of, or cross-references to, other major assets such as bridges and other structures. Best-practice IT policy dictates the use of hardware and software maintenance agreements. By not having maintenance agreements, organizations run the risk of Hardware agreements provide a guarantee of service and having obsolete systems within a very short time period, replacement of spare parts within agreed timeframes also of having to maintain staff skills in old versions of dependent upon business needs, and removes the need software. Maintenance agreements are cheaper than to have funds earmarked for purchase of spare parts Page 7 Transport Note No. TRN-29 February 2006 (hard disks, new monitors etc.) whenever a breakdown (such as inventory and condition) for use by a second occurs. application. Software agreements are necessary to obtain continued More and more agencies are moving towards integrated software support after warranty and upgrades without systems. This makes it even more critical when planning having to purchase new software licenses. They enable an RMS implementation to choose the correct software organizations to keep up-to-date with technology package in terms of its functional and technical changes in an incremental fashion, as well as to avail of requirements. The ability of a system to be able to store security patches which are becoming of growing concern information on any asset, as well as exchange data across all software platforms and systems. between other systems, is becoming more important. This should also be built into Technical and Functional Put simply, in the long term, support and maintenance Requirements for systems. agreements usually reduce the total costs of ownerships of systems. Data Collection Systems Integration Introduction Large agencies that have been using IT for a number of years eventually find it necessary to integrate their Data (ie inventory, condition, traffic, environmental, and systems, otherwise they end up with many different cost data) is vital to the success of any RMS. Without databases containing the same information, or references data, it is not possible to conduct proper analyses and to information held in other databases. After a while, monitoring of the road network. Problems with data is manual procedures can no longer cope. one of the main causes cited in failure of an RMS. For road agencies, often the road database is separate Data are also expensive. Each data item requires time, from the bridge database, which is separate from the effort, and money to collect, store, retrieve, and use. traffic database, which is separate from the routine The first rule of data collection is that data should never maintenance management system database, which is be collected because `it would be nice to have the data', separate from the GIS etc. All of these systems should or because `it might be useful someday'. There have been ideally use a common referencing system (ie the road several papers advising on the necessity to collect only network). Any changes to this common referencing what is needed, and to collect it at the required system will involve the same changes to multiple `Information Quality Level' (IQL). Bennett and Paterson, databases, and when changes are made to one database (2000) describe the IQL concept in detail. but not another, then problems start to occur with data integrity. Recent literature has shown that 70% of all However failure of the RMS due to `data collection' is not software development effort is now focused on systems seen as a failure of the data collection itself, rather a integration. failure to properly institutionalize data collection. Specifically: Some major software suppliers have recognized this, and many RMS have the ability to store information on a There are often no explicit data collection number of different types of asset (roads, bridges, signs, policies; traffic data etc.). This has several potential advantages, Budgets are not made available for data including savings on database licensing costs, a larger collection; user base with which to exchange knowledge in the agency, but most importantly enforces data integrity Staff are not properly trained or monitored; between all these systems because they are forced to use There are no quality assurance procedures; the same referencing system. Ideally too, all of the above There is no auditing; and systems would also use the same GIS system and the same GIS data without having to share it manually There is no replacement strategy for specialist across different divisions of the same organization. equipment or vehicles. Many of the major RMS also provide APIs (application All of these areas need to be specifically addressed in programming interfaces) to allow other applications to implementation of an RMS. integrate with them. Common functions include the ability to reference data to the road network (even if held Location Referencing in another database system), and to retrieve information Page 8 Transport Note No. TRN-29 February 2006 The ability to manage data on road networks depends on reference markers (eg kilometer posts), and an accurate and viable Location Referencing system. perhaps also structures. These are usually Most countries have in place some sort of linear incorporated into a GIS. referencing system, whereby roads (or sections) are Policies for collection and update of these data should be uniquely identified, and where data are referenced to explicit, and should guide management in ensuring that distance from the start of the section in a defined positive there is sufficient budgeting and staff made available for direction. Normally too, there is some system of the activities required to implement these policies. More kilometer posts or other distance markers which are fixed data, collected at a higher level of quality or detail, markers at known points on the section, which enables requires more budget and more people. Any referencing of other items (eg traffic signs) as a relative implementation project for an RMS should explicitly distance from these markers. The ability to collect and quantify these relationships in terms of costs, and get report on data relative to those fixed markers is up-front agreement with executives and managers that particularly important in helping field staff locate items of these levels of budget and staff are sustainable in the interest. Collection of data against those markers can medium to long term. Equipment also needs to be also significantly reduce the cumulative error that can be considered. built up if measurements are taken from the start of the section. Data Collection Process Data Collection Policies In terms of the process by which data are collected, there Policies for data collection can be quite simple. They are essentially two choices an agency can make: to should describe at a high level the type of data that will collect the data in-house or to contract out the collection. be collected, its frequency, and its level of detail (or, The choice depends very much on the individual agency, IQL). They should also describe the process by which and its ability to purchase, operate, and maintain any data will be collected (ie in-house, or by contract). specialist equipment required. Policies such as these give the organization clear Specialist equipment are used in surveys to collect data guidance on their duties and responsibilities, and also such as roughness, GPS, deflections, skid resistance and serve to communicate with other agencies when it comes pavement structure. Operating this equipment places to discussing sharing of data. demands on an agency that may be difficult to meet, such as: The basic types of data collected by road agencies are: Road inventory data: These are typically The capital cost of equipment is high so funds collected in a once-off exercise. They are then may be diverted from other, equally important, updated when changes are made to the road. It activities to fund the equipment; is common to verify/update the data every five The agency may have difficulties in funding, years or so. This may include video. importing/acquiring and installing spare parts or Pavement condition data: These may be to service the equipment (especially if the collected at different frequencies, depending on equipment's host vehicle is imported from the road class. Main roads and major highways overseas); may be monitored at more frequent intervals, The equipment requires specialist staff with a often 1--2 years, while minor roads may be high degree of skills and training, which in a monitored at 2--5 year intervals. The frequency road agency may be used irregularly and needs to be sufficient to identify major changes therefore staff, skills and training tend to be lost which will influence road maintenance decisions. over time; and/or Traffic data: Traffic volume data are usually The requirements for calibration and checking collected through a set of permanent traffic which may not always be performed well by staff count stations around the country, without full training and documented quality supplemented by short term counts (typically assurance procedures. seven days for traffic volumes) at other locations. Axle load data are usually collected at Many agencies can manage certain types of special a relatively small number of representative survey equipment by themselves, and maintain the static locations around the road network. equipment and staff skills to do so. Often, these countries Location data: Coordinate data on the road use a specialist project with consultants or contractors to network itself and/or point locations for location start the data collection, and during this time the Page 9 Transport Note No. TRN-29 February 2006 agency's staff are trained in the use and operation of the New Zealand, for example, has recently introduced equipment. After the project the staff continue the data stricter quality assurance procedures, including issuing collection. multi-year data collection contracts with the same contractor to get continuity on the survey teams, as Other agencies make a policy decision to completely out- previous analyses had shown variations between source specialist surveys. Consultants or contractors consultants. who work full-time with the equipment, use highly skilled personnel, who strive to acquire or even develop latest It should be noted that QA applies to the entire data technology, and who follow approved calibration and collection and data reduction process. Road data are validation procedures, are often in a much better position usually voluminous, and data needs to be controlled and to collect good data. managed carefully once it has been submitted to the client's office. Corrections or changes to data on the One reason identified for out-sourcing data collection was client side should be properly documented, if possible in to reduce the ability of agency staff to manipulate data the computerized RMS through automatic auditing for their own purposes. If this approach is used it is vital capabilities. Also, regular audits should be carried out to that the agency have in place their own data collection ensure that the quality procedures are actually being procedures, and the ability to claim liquidated damages followed. The results of audits should be documented, for the submission of late or inaccurate data. and actions followed up. Only by following such The recommendations are clear. Agencies that have procedures can users (both internal and external) have difficulties in calibrating, operating, maintaining, and/or confidence in the data, and hence have confidence in any obtaining funding for warranties and maintenance analysis that goes with it. agreements for specialist data collection equipment, should adopt and implement policies to contract out the Data Collection Contract Management data collection services. Although it is recommended for some agencies to This decision, of course, has ramifications in terms of consider adopting policies to contract out data collection institutionalization and training for management of data see 6.4 above), if such policies are adopted there needs collection contracts, as well as data quality assurance, to be careful thought given to institutional capacity for which are discussed below. the management of the data collection contracts. Procurement and management of a nation-wide data Data Quality Assurance collection contract on 10,000+ km of roads is a challenge Agencies must have in place Quality Assurance (QA) for any agency. Some basic procurement principles, that are sometimes neglected, are given in Box 2. procedures for data. The QA procedures must be consistent with the data collection policies of the agency, especially with regard to the Information Quality Level It must be recognized that the effort that has to go into (including and accuracy and frequency of data collection). management of a data collection contract is immense. Specialist skills are often necessary to understand the QA on the client side is necessary irrespective of whether data and to be able to validate it. Spot-checks are often the data are collected in-house or not. worthwhile as part of a quality assurance program. Dedication and resources can often be stretched to the It is not safe to assume that the consultant or data limit as data starts arriving and has to be quality assured collection contractor will provide good quality data. There before payment is made. are many opportunities for error in the overall data The following client staffing estimates (for contract collection process, even when dealing with automated supervision, contract management and quality assurance data collection equipment and with experienced of data) are made for typical data collection contracts of contracting firms. Equipment is being developed or 10,000 km of road network: upgraded all the time, and with each new development comes a new set of problems. Also new or inexperienced contractor staff in the field may be unaware of, or forget Roughness Data: 1 ­ 2 full-time client staff for to follow, equipment calibration procedures. the duration of the contract (which may be up to nine months, depending on circumstances and logistics). Even agencies that have been contracting out data Inventory Data: Depending on the number of collection for many years, using experienced contractors, inventory items and their number of attributes, are continually looking at ways to improve data quality. 2 ­ 3 client staff full-time for the duration of the Page 10 Transport Note No. TRN-29 February 2006 contract. If video data are being collected under A combined data collection contract with all three major the same contract, and where there is the ability types of data being collected at once, can obviously pose to cross-check video with inventory, then estimate 3 ­ 4 client staff full-time. severe organizational difficulties for an agency. Any RMS implementation project that includes data collection GPS Data: One client staff member full-time for the duration of the contract. Requires good GPS should look to train appropriate numbers of client staff in and GIS skills. the art and science of data quality assurance. Much of the data quality assurance is best done in spreadsheets Box 2 prior to uploading to the RMS, therefore good Basic Principles For Data Collection Contract spreadsheet and database skills are often required, along Management with good analytical skills. Require the contractor to survey a validation In contrast to the above approach, some RMS projects network (minimum 100 km) prior to the full have the Consultant collect and load data into the newly- survey. This will help the contractor to sort out all implemented RMS, with little, if any, involvement of the sorts of logistical and technical issues early on and client. This misses an opportunity for client quality before the full survey commences. This validation assurance procedures to be developed and implemented survey data should be completely processed and for data collection. It is unlikely that after hand over of imported to the RMS where it is verified as the system that the client's staff will have the skills to suitable. perform this process independent of the Consultant. Require every data collection team of the It is often the case too that some data are very complex contractor to perform the validation survey. If and difficult to interpret (eg FWD data requires specialist there are different teams, different vehicles, and knowledge). It is strongly recommended that if the different equipment, then all should be tested. agency does not have the skills to quality assure this Require the contractor to produce his own data itself, then it should hire an independent contractor Quality Assurance Plan prior to the start of the or consultant to do it on their behalf. contract. This should be approved by the client. It may also be useful to ask for the GIS Data Collection Contractor's Quality Assurance Plan as part of the proposal, and include evaluation of the Quality GIS data are highlighted separately here, since there are Assurance Plan in the technical evaluation. some special issues that apply to it because of the potential for sharing with other agencies. Require documentary evidence of calibration prior to, and during, the surveys. Even for road network data (which is often the only Require data to be submitted within a short spatial dataset that a road organization is responsible time period after collection (less than 2 weeks if for), it is often difficult for a road agency to keep it possible, and certainly not more than 1 month). current. It is not uncommon for a road network to change by up to 10% per annum, when new road construction, Pay only for data approved, not for time, and not for data submitted. It will be necessary to road realignments, road widening, and road transfers agree upon the time-frame for approving data (between agencies) are taken into account. There must (usually 30 days or less) and to ensure that the therefore be a policy relating to this data in the same client's staff are allocated sufficient time to check way as applies to any other data (ie how often is it to be the data. updated, and to what accuracy will it be collected). Have a liquidated damages clause in the When implementing GIS in a road agency, the factors in contract which can be used in the event of late Box 3 should be addressed regarding GIS data collection. submission or continued submission of poor quality data. With experienced contractors, and a track record in the agency for data collection by contract, the staffing levels may be reduced, but not significantly. Some agencies procure a consultant to do the data collection contract management and quality assurance on their behalf. Page 11 Transport Note No. TRN-29 February 2006 Box 3 Key Points for Implementation of a GIS Project specifications and Terms of Reference should be more explicit, and the proposals should require Agree to the policies, standards and Consultants to detail how they will address accuracies with internal stakeholders and with institutionalization in their project plan. Proposals should external stakeholders from other relevant be scored on how well this is addressed, and it should agencies. Mapping data are much more likely to attract a much higher weighting than the technology, be shared, and taken out of context, than most especially since the technology is usually well other road data. established. Instead, at present technology dominates. In particular, agree policies for updating the Thus, for RMS implementations to be successful, future geographic representation of the road network, projects must reflect the findings of this project in these taking into account whether the agency has the three key areas. ability to collect its own GPS data, or whether it needs to hire GPS contractors. There is also the Processes possibility of getting road construction contractors to provide as-built drawings and/or GPS The introduction of an RMS by itself is not a guarantee coordinates of new roads, although this does not that it will be used, or that it will be successful. The help in the case of road conversions. agency must also follow basic asset management Metadata standards should be agreed upon principles. Strong involvement of executives and and implemented. managers prior to and during the implementation of the system is absolutely necessary. If it is not seen as having Data quality standards should include data cleanup procedures, snapping of lines, closing of value by the agency's higher management, they will not polygons etc. as well as domains of values for provide the necessary support and funding to maintain the system. It is therefore important to have: attribute data. Consideration should be given to versioning of Business Plans, using `Asset Value' and other data to allow historical spatial analysis. Key Performance Indicators derived from the RMS. This is an executive and managerial responsibility. It also helps put focus on the RMS itself, and improves the chances that budget and funds are available to run the system. Conclusions Institutional support consisting of high ranking decision-makers fully-committed to the asset What makes an RMS successful? In addition to funding, management/asset preservation `philosophy'. there are three key factors: Processes, People and Regular briefings given to ministers and other Technology. If any of these are weak or fail then the RMS high government officials on the importance of will be compromised. asset preservation, and what is being done to make sure that the preservation of the road Some agencies successfully consider all three factors, but infrastructure is dealt with satisfactorily. many do not. The focus of too many projects is on the technology element, with insufficient attention given to Have specific and realistic key performance the institutionalization of the system and the necessary indicators and targets to measure their asset support systems. value and to preserve/enhance that value. Monitor those targets, and assess at the end of The major success factor in the implementation is each year whether they have achieved them or therefore institutionalization (processes and people) not, and take appropriate action. By publishing rather than technology, although the latter is also this information in Annual Reports, the agency is important. It is essential to ensure that there are specific accountable for it. items in a Terms of Reference to deal with Have annual budgets in place for data collection institutionalization, at the very highest level. Instead, the and operation of the RMS. Even if this initially focus has usually been on technology. Too many Terms requires donor funding support, there should be of Reference call for consultants to implement a system, a phased increase in local budgeting to ensure and then give `on-the-job training' and a high-level that the RMS is self-funding within a given presentation to management at the end. This is timeframe. completely inadequate. Page 12 Transport Note No. TRN-29 February 2006 Have policies and procedures in place for data o Data Collection ­ planning, collection, and for quality assurance of that management, supervision and data. coordination. Technical (internal and/or external) auditing o Data Quality Assurance ­ verification must be carried out on data and systems, and and checking of all data. the recommendations acted on. o Management Reporting ­ reporting and A program of Continual Quality Improvement is presentation to management. also critical. No system is static. All systems can Strong contract management skills are be improved. necessary especially for agencies that contract out portions of data collection. People The agency should follow good basic An RMS (including all computer systems, data, policies management principles, covering procedures, and procedures) should be driven by a dedicated group records, auditing etc. within the agency, probably in the planning division or There should be a commitment to Continual equivalent. This dedicated group should actively seek to Quality Improvement. promote the system within the agency, including to higher level management; raise awareness of the Technology system; manage data collection; constantly look for ways of improving data collection procedures and data quality RMS are demanding with regard to their IT requirements. assurance; research off-the-shelf packages and systems It is important that the RMS implementation should fit on the market; create and maintain technical and within the overall IT strategy of the agency, and should functional requirements for planning and programming be properly supported from an IT perspective. systems; and coordinate all efforts related to the RMS in terms of other applications. Terms of Reference should explicitly reflect the IT support in the agency, they should not To ensure that an appropriate staff environment: implement a system in isolation from the IT strategy of the agency. If necessary, assistance There should be an organizational unit must be provided to define an IT strategy and to established with specific responsibility for the implement it. RMS. Road agencies should consider outsourcing or There should be a budget for the operation of external hosting of their systems where the system, including all staffing, equipment, possible, but this depends on the technical data collection (contracted or in-house), field capabilities of the private sector and the road travel, quality assurance etc. agency's organizational policies. There should be clear job descriptions for the Any sizeable organization procuring IT should various activities, and a career path for those in have a Technology Architecture, or explicit the unit. technology standards and directions. This is There should be a continual training and important to avoid a profusion of different development program (and budget) for staff to infrastructure software (operating systems, deal with staff turnover and re-training where databases, GIS etc.) with all the attendant necessary. This should potentially include support issues; it is also important in helping to Master's or other post-graduate degrees which define a replacement / upgrade strategy for will increase the attractiveness of working in this hardware and software. There are also distinct area. economies of scale that can be achieved through centralized procurement of hardware and Jobs should be filled with appropriately qualified system software. personnel, with good management skills, and with access to, and control over, their budget. All IT implementations should use commercial off-the-shelf (COTS) products wherever Job responsibilities should explicitly include: possible. o Management of the Road Network For any future implementation of an RMS, a set Referencing System ­ control, of functional and technical requirements should verification, education and be drawn up. Functional requirements should dissemination to other stakeholders. include the functions that the software should Page 13 Transport Note No. TRN-29 February 2006 perform. From the wealth of experience explicitly stated in Terms of Reference. The available, it is relatively easy to determine client also needs to make sure that their IT generic functional requirements of an RMS to infrastructure (including hardware, systems suit a road agency of a given size. Key functions software, databases and GIS) is able to support that should be in any system are given in what they wish to do with a web-enabled McPherson and Bennett (2005). Technical system. Requirements should describe the technology environment within which the RMS will fit (ie hardware, operating systems, databases, GIS, and other applications). This should relate to the References agency's Technology Architecture. Bennett, C.R. and Paterson, W.D.O. (2000). A Guide to Agencies should develop and adhere to a long- Calibration. HDM-4 Reference Series ­ Volume 5. term IT budget strategy that includes costs of PIARC, Paris. Available for download from hardware and software maintenance agreements www.lpcb.org. (in addition to hardware replacement strategies). ISO (2000). Quality Management Systems ­ Fundamentals and Vocabulary. Available for Terms of Reference requiring `integration' other download from www.iso.org. applications, such as HDM-4, with an RMS should be more precise, to raise client McPherson, K. and Bennett, C.R. (2005). Success Factors for Road Management Systems. Report to the World awareness of the issues, and will enable the Bank. Available for download from www.road- consultant to get a clearer understanding of the management.info. client's needs prior to bidding. Schliessler, A. and Bull, A. (2004). Road Network The real requirements for web-enabling of Management. UN-ECLAC/GTZ Report. Available for systems should be more carefully assessed, and download from www.gtz.de/roads.