Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) Report Number : ICRR0021554 1. Project Data Project ID Project Name P112872 AF: Customs Reform & Trade Facilitation Country Practice Area(Lead) Additional Financing Finance, Competitiveness and P155443 Afghanistan Innovation L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-D0760,IDA-H5680 30-Jun-2014 69,436,539.86 Bank Approval Date Closing Date (Actual) 25-May-2010 30-Jun-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 50,480,000.00 0.00 Revised Commitment 71,963,142.72 0.00 Actual 69,547,490.72 0.00 Prepared by Reviewed by ICR Review Coordinator Group Antonio M. Ollero Fernando Manibog Christopher David Nelson IEGFP (Unit 3) 2. Project Objectives and Components a. Objectives The project development objective of the Afghanistan Second Customs Reform and Trade Facilitation Project was “to improve the release of legitimate goods [by customs] in a fair and efficient manner”. b. Were the project objectives/key associated outcome targets revised during implementation? No Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components The project had five components: Countrywide Computerization of Customs Clearance Operations (US$16.39 million estimated at appraisal; US$23.52 million after the provision of additional financing of US$7.13 million; US$21.29 million actual expenditure) supported the continuation of the national rollout of the Automated Systems for Customs Data (ASYCUDA), with the: (a) upgrade of the system from ASYCUDA++ to ASYCUDA World and the continuation of the rollout to new locations (the additional financing required the rollout to cover all remaining office locations); (b) introduction of additional system functions to reduce revenue leakage and minimize human interaction; (c) design of software solutions to accommodate cargo tracking in the computerized customs clearance system (the additional financing required further technical systems integration including the configuration of bar code readers, deployment of cameras, and use of biometrics); (d) development of an ASYCUDA Unit within the Afghanistan Customs Department (ACD) (the additional financing supported the development of a sustainability strategy for this ASYCUDA Unit); (e) improvement of human resource management in ACD; and, (f) the provision of communications infrastructure through a fiber optic/very small aperture terminal (VSAT) connectivity (the additional financing provided support for the fiber optic/VSAT connectivity). Installation of Executive Information Systems for Customs, Allowing Real Time Monitoring of Operations (US$2.19 million estimated at appraisal; US$0.05 million actual expenditure) supported the introduction of new information technology (IT) functions including the preparation of statistical reports, alerts to reduce revenue leakages, and summary reports to management. The component covered the: (a) provision of business solution analytics and reporting software to the ACD and the Ministry of Finance (MOF); (b) installation of a monitoring and supervision system covering the operations of customs offices, individuals, and transactions; (c) facilitation of investigations and optimization verification and supervision mechanisms; and, (d) tracking of non-compliant trade transactions in real time using alert notices. Development of Cross-Border Customs-to-Customs (C2C) Cooperation ($0.55 million estimated at appraisal; US$0.80 million after the provision of additional financing of US$0.25 million; US$0.07 million actual expenditure) aimed to review the available options to improve C2C cooperation between Afghanistan and bordering countries for better border management and trade facilitation, in collaboration with other donors. The component encouraged the ACD and customs offices in neighboring countries to: (a) sign cooperation agreements (the additional financing supported the conclusion of a memorandum of understanding with Iran); and, (b) develop C2C data exchange capabilities (the additional financing supported real-time data exchange with Pakistan and Tajikistan). Provision of Adequate Customs Infrastructure to Enable Modernized Operations (US$25.19 million estimated at appraisal; US$31.15 million after the provision of additional financing of US$5.96 million; US$32.67 million actual expenditure) would finance the infrastructure required to make selected customs facilities fully operational, including through the provision of: (a) goods and equipment; (b) technical Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) assistance (TA) for engineering planning, design and implementation; (c) operational and management planning; (d) support for incremental operating costs. The customs offices to be improved and rehabilitated included: the Jalalabad Inland Clearance Depot (ICD); the Kabul ICD; the Khost ICD (new construction); the Nimroz ICD; the Torkham Border Post; the Andkhoi ICD; and the Aqina Border Post (the additional financing included the Abu Nasr Farahi ICD, the Aqina ICD, the roads at Kandahar and Weesh, and the temporary offices at Ghazni, Paktika and Aqina to the original list). Technical assistance was to be provided for a feasibility study for a multi-modal container freight station near the Jalalabad Industrial Park. Technical Assistance to Support the Development of an Adequate Regulatory, Administrative, and Institutional Framework for Customs (US$6.18 million estimated at appraisal; US$14.33 million after the provision of additional financing of US$8.15 million; US$15.43 million actual expenditure) would provide technical assistance to address major cross-cutting issues identified in the Governance Accountability Action Plan (GAAP) for the Afghan Customs, including: (a) cumbersome and opaque procedures; (b) lack of adequate cross-border data exchange; (c) high-level discretionary powers of customs officers; (d) limited operational-level ownership of the reform process; (e) the unclear mandate and poor status of the customs office; (f) interference of other agencies at the border and, in general, with customs operations; and, (g) poor human resource practices at customs. (The additional financing added studies for a National Single Window, Business Process Reengineering, and Enterprise Resource Planning to the technical assistance package). e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project cost was estimated at appraisal at US$50.5 million. After the provision of US$21.5 million in additional financing, the project cost was estimated at US$72 million. The actual disbursement was US$69.5 million. Financing: The project was financed with an original grant from the International Development Association to the Islamic Republic of Afghanistan of SDR 33.33 million (US$50.5 million equivalent) and an additional grant of SDR 15.3 million (US$21.5 million equivalent) to scale-up the project activities, for a total grant amount of SDR 48.63 million (US$72 million equivalent). Borrower Contribution: No financial contribution was expected from the borrower. Dates: The project was approved on May 25, 2010 and became effective on December 28, 2010. The first restructuring of May 27, 2014, with US$43.6 million disbursed, extended the project closing date by a year. The second restructuring of June 11, 2015, with US$51.2 million disbursed, provided additional financing to the project of US$21.5 million and extended the project closing date by a year and a half. The additional financing was a response to a request by the new Afghan government for a third phase to the Customs Reform Program, to which the Bank decided that the provision of additional financing to this project would allow the new government "the time to decide on its priorities and future course of action" with customs reform (ICR, page 17). The additional financing was approved on June 11, 2015 and became effective on June 14, 2016. The additional financing took one year to become effective because Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) it took time to conclude an agreement with the National Procurement Commission on implementation arrangements. The third restructuring of June 26, 2015, with US$51.2 million disbursed, revised aspects of the disbursement arrangements, the financial management system, the procurement, and the implementation schedule of the project following a decision by the new government to implement the additional financing activities through an internal unit; this restructuring extended the project closing date for the third time and by a year. The ICR does not explain why the second and third restructurings were not packaged as one, coming within two weeks of each other. The fourth restructuring of December 18, 2017, with US$65.9 million disbursed, changed the project closing date for the fourth time, by a half year. None of the restructurings changed the project development objective. The project closed on June 30, 2018, four years after the original closing date of June 30, 2014. 3. Relevance of Objectives Rationale The project was relevant to the development priorities in Afghanistan at the time of project appraisal in 2010. Afghanistan had been undergoing a decade-long economic, political, and social transformation since the Taliban fell in 2011 and economic growth had averaged 8 percent in 2007-09. But the state remained fragile, and the country, poor. Afghanistan required massive reconstruction, but was beset by security challenges, the threat of illegal opium activity, and high dependence on foreign aid rather than domestic revenues. Customs receipts comprised a sizable part of the government’s fiscal revenue, but the Afghanistan Customs Department remained structurally weak. The ACD had previously received considerable support from donors for infrastructure improvement and operational technical assistance and had subsequently raised collections from US$50 million in 2003/04 to US$733 million in 2008/09, but much work remained to be done. The project would comprise the second phase of the customs reform program and help the ACD gain greater border control over goods, vehicles, and people and thereby further improve revenue collection. The project would be supportive of the Afghan National Development Strategy of 2009, which emphasized: (a) increasing domestic revenue mobilization; (b) improving transport services, customs administration, and logistics management to facilitate trade; (c) removing trade impediments and lowering trade barriers to enhance the flow of goods, services, and investments in the economy; and (d) improving border management and strengthening customs cooperation at the regional level to increase security and help fight drug smuggling and other cross-border crimes. The project was designed to build on the initial gains made by the predecessor Emergency Customs Modernization and Trade Facilitation Project, which aimed to help Afghanistan to start to establish a more efficient customs and transit regime. The outcome of this emergency assistance operation, which was approved by the Bank in December 2003 and which closed in December 2010, was rated satisfactory, having increased customs revenues and declarations and having reduced trade transaction costs. The project would have been followed by a third operation requested by new Afghan government in 2015, but to which the Bank responded with the provision of additional financing to this project instead. The project would Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) later be related to the Fiscal Performance Improvement Project, which the Bank approved in 2017, and which aims to improve domestic revenue mobilization and public expenditure management in the country. The project remains relevant to the development priorities of Afghanistan at the time of project closing in 2018. The “Afghanistan National Peace and Development Framework” of 2016 documented rising fiscal constraints faced by the country as security expenditures continued to increase while donor financing declined, and domestic resource mobilization remained limited, squeezing funding for economic and social development. The project would be responsive to these challenges if it facilitated trade and raised customs receipts, thereby helping boost economic activity and increase fiscal revenue. The project remains aligned with the assistance strategy of the World Bank Group in Afghanistan at the time of project closing. The “Country Partnership Framework (CPF) for the Islamic Republic of Afghanistan for FY2017 to FY2020” committed the Bank Group’s support to three pillars of the partnership strategy with Afghanistan: building strong and accountable institutions; supporting inclusive growth; and promoting social inclusion. The first pillar includes the objective “to improve public financial management and fiscal self- reliance”, and the second pillar, the objective “to improve domestic and regional integration through transport, trade, and information and communication technology connectivity”. The project to reform customs and facilitate trade would be aligned with these objectives. Rating Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To improve the release of legitimate goods. Rationale This ICR Review will not undertake a split rating of the efficacy of the project objectives, as was done in the ICR, for three reasons. First, the project objective did not change with any of the four project restructuring episodes (Section 2.e). Second, the provision of additional financing to the project in June 2015 only introduced additional activities to the five components of the project and did not alter the thrust or content of the components (Section 2.d). And third, modifications in the project outcome indicators served to improve the measurement of the project's performance and did not detract from the project's underlying theory of change (Section 9.a). The degree of achievement of the objective “to improve the release of legitimate goods” is rated as substantial. The average time for clearance by customs of goods inland was reduced to 67 minutes by 2018, from a Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) baseline of 148 minutes, exceeding the target of 105 minutes. The clearance times inland were measured at three ICDs, namely, Jalalabad, Herat and Heiratan. The ACD also carried out a physical verification of the reported ASYCUDA data through a survey of clearance times at Kabul and Jalalabad. The automation of the clearance process evidently made an impact on the average clearance time, compared to that of the manual process. The average time for clearance by customs of goods at the border was reduced to 20 minutes by 2018, from a baseline of 36 minutes, exceeding the target of 27 minutes. The clearance times at the border were measured at the three main border crossings in the country, namely, Torkham, Islamqala, and Torghundi. As with the inland data, the ACD conducted a physical verification of the ASYCUDA border data the through a survey of clearance times at Torkham. The number of non-compliances detected was 3.1 percent by 2018, short of the target of 7 percent. The number of non-compliance detected through the ASYCUDA Risk Management System had been 5.5 percent in the baseline when the risk management system was introduced in August 2013. The system is currently operational in 19 locations. According to the ICR (page 23), the commodity-based criteria used by the system directed a high proportion of trucks to the red rather than to the green and yellow channels for physical inspection. However, the disproportionately large volume of trucks in the red channel could not be properly examined and inspected, keeping the detections low. A subsequent shift to a multi-criteria risk assessment system, implemented at four main customs offices, reduced the number of trucks directed to the red channel and raised the number non-compliance detections. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective To improve the release of legitimate goods in a fair manner. Rationale The degree of achievement of the objective to “improve the release of legitimate goods in a fair manner” is rated as substantial. The average rating in subjective user satisfaction surveys was 7.4 out of 10 by 2018, exceeding the target rating of 7 out of 10. Three independent consulting firms conducted the Customs User Perception Survey separately in 2012, 2015, and 2017, interviewing over 400 system users, and covering over four regions of the country. Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) The percentage of electronic versus manual declarations --- the electronic transfer of public services --- reached 96 percent by 2018, meeting the target. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 3 Objective To improve the release of legitimate goods in an efficient manner. Rationale The degree of achievement of the objective “to improve the release of legitimate goods in an efficient manner” is rated as substantial. The cost savings to business due to customs reform was placed at US$150,000 per day as calculated in 2016, and projected through 2018, meeting the target. The total implied cost saving to the private sector was estimated at US$13.05 million from 2003 to 2016 using the Trade Logistics Impact Reform Model developed by the Bank. Rating Substantial PHREVDELTBL PHOVRLEFFRATTBL Rationale All three objectives were substantially achieved and thus the overall rating is Substantial. Overall Efficacy Rating Substantial PHREVISEDTBL 5. Efficiency Economic Efficiency: The project envisaged that a more efficient customs service would reduce multiple cargo handling, truck waiting times, cargo delay times, transit and clearing time variability, and overall trade costs. The ERR of the project was estimated at appraisal at 146 percent. During project implementation, an Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) economic analysis done by M/s Geopolicity in 2014, before the project activities were scaled up and additional financing of US$21.5 million was provided, calculated an ERR in the range of 120 to 170 percent. Two ex-post economic analyses, both prepared in 2018, produced even higher economic rates of return, when increased customs revenues were added as a benefit of the project. The first, using the actual trade volumes for 2010-18 and updated forecasts (break trend method) for 2019-30, reported an ERR of 233 percent. The second, which, in addition, valued time as a tariff (traders were willing to pay, on average, between 0.6 and 2.1 percent ad valorem for each day of time saved), yielded an ERR of 223 percent. Operational Efficiency: The project effectiveness date was postponed by three months when the project implementing consultant, the UNOPS, insisted on a UN audit (under the UN’s single audit principle), which required a waiver by Bank management to the audit clause in the Bank’s standard form of agreement. The conflict between the UN and Bank’s audit practices was not foreseen. The effectiveness of the additional financing was delayed by a year, after the new government decided to implement the activities covered by the additional financing through an internal project implementation unit (PIU). The National Procurement Commission did not renew the contract of the UNOPS, while the ACD took a year to set up the IPU. The delay in the effectiveness of the project, coupled with a delay in the first disbursement to the UNOPS as well as the need to complete infrastructure contracts at Abu Nasr Farahi, factored into the first extension of the project by one year to June 30, 2015. The project would be extended three more times. The project was extended by one and half year to December 31, 2016 to accommodate the scaling up of project activities and the provision of additional financing, following the restructuring of June 2015. The project was delayed by another year to June 31, 2017 following the restructuring of June 2016, under which the government dropped the services of the UNOPS and chose to complete the project through a PIU. Finally, delays with the Nimroz ICD truck parking, the Aqina boundary wall, the National Single Window feasibility study, the ASYCUDA data center IT equipment procurement, and the e-recruitment system necessitated a final extension of the project by six months to June 31, 2018. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  146.00 Not Applicable 100.00 ICR Estimate  223.00 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) 6. Outcome The outcome of the project is rated as moderately satisfactory. The project development objective was substantially relevant to the development priorities in Afghanistan and to the Bank Group's country partnership framework for the country. The degree of achievement of the objective “to improve the release of legitimate goods [by customs] in a fair and efficient manner” was substantial, in all aspects of the objective --- the improvement in the release of goods, the efficiency in the release of goods, and the fairness in the release of goods. The efficiency of the project was modest, with substantial achievement for economic efficiency, but operational efficiency was modest. The modification of the PDO outcome indicators (without any change in the PDO) during the additional financing stage of June 2015 led to a more focused performance measurement and did not detract from the project's underlying theory of change. Consequently, a split rating does not apply. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Overall, the risk to the sustainability of the development outcome of the project is rated as high. Political Risk: There are multiple sources of political risk to the sustainability of the project outcomes, including: possible changes in the political leadership in Afghanistan following elections, which may impinge on the degree of political commitment to the project objectives; possible changes in the management of the MOF and the ACD (the Director General and the Deputy Minister of Customs changed six times between 2010 and 2018); any interference by politicians and even by traders in the recruitment, staffing, and posting of ACD staff; and, any rift between Afghanistan and its neighbors on border issues, including on cross-border data exchange. Economic Risk: The vulnerability of the Afghan economy to macroeconomic shocks poses risks to the gains so far made by the project with customs reforms and trade facilitation, including: the transparency of trade transactions; simplified customs procedures; standardized customs documents; the ability to make customs declarations online; better and more accurate customs and foreign trade statistics; less corruption; increased customs revenues; and a more predictable trade environment. Afghanistan must ensure macroeconomic stability to secure these gains moving forward. Financial Risk: With the government expected to face deficits well into the future, budgetary resources are not assured to: maintain and further develop ASYCUDA; retain the ASYCUDA National Project Team; pay the ASYCUDA staff on a UN pay scale; and, upgrade the ICT infrastructure. Some financial support for customs reform may be forthcoming from the Bank-financed Fiscal Performance Improvement Support Project for 2017-22 (the project was approved in December 2017) which aims to help improve domestic resource mobilization and Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) public expenditure management in Afghanistan, including by encouraging a performance-oriented management culture in the Ministry of Finance. Eventually, the government needs to sustain the increase in customs revenue in the medium- to long-term to help maintain a system of financial incentives to the ASYCUDA staff. Technical Risk: The lack of an out-of-country disaster recovery center (the ASYCUDA is backed up only by an in- country data recovery center) poses a major technical risk to the sustainability of the project outcome, considering the security problems in the country. The other principal technical risk pertains to the changing ICT ecosystem, which would require: upgrading the ICT infrastructure; ensuring technical compatibility between newer and legacy parts; upgrading the ASYCUDA system and its applications; and, ensuring the capacity of national technical experts to operate the newer technologies. 8. Assessment of Bank Performance a. Quality-at-Entry The project was aligned with the Governance Accountability Action Plan (GAAP) for the Afghan Customs Department, which had been drafted as part of an overall effort by the international donor community to improve customs infrastructure, including the border transit system, to reform customs governance, and to facilitate trade in Afghanistan. The project was developed to build on the gains made by the Bank- financed Emergency Customs Modernization and Trade Facilitation Project. The Project Paper was comprehensive and considered the technical, procurement, financial and institutional aspects of the operation. Risk factors faced by, and lessons learned, from the preceding project were highlighted. In addition, the Bank conducted two separate studies during project appraisal to prepare for project implementation. The Bank also developed a Governance and Accountability toolkit, assessed conflict and fragility in the country, and reflected the findings and recommendations in project design. Because Afghan government agencies had inadequate capacity for project management, the Bank and the government agreed: (a) to make the UNOPS an implementing partner of the ACD in the execution of the project, and (b) to engage the UNCTAD to provide technical assistance to the government in developing ASYCUDA and facilitating trade. Quality-at-Entry Rating Satisfactory b. Quality of supervision The Bank fielded two implementation support missions a year during the eight-year life of the project. The work of the supervisions missions was augmented by consultations, telephone conversations and video conferences between the Bank, including Bank customs specialists, and the project implementing agency and the implementing partner. Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) The supervision missions filed 16 Implementation Status and Results Reports (ISRs), the final report of which rated both the Development Outcome and the Implementation Progress of the project as moderately satisfactory. According to the ICR (page 42), the ISRs were clear, detailed, and candid, focusing on important implementation events and issues. The Bank also produced Aide Memoires intermittently. The task team was generally adequately staffed, including by technical experts, both staff and consultants, financial management specialists, and procurement specialists, although there were problems with staffing the procurement team at the start of the project. Because the task team members were based either in Kabul or elsewhere in the region, they were able to respond to reasonable requests by the government in a timely and efficient manner. The task team was headed by a task team leader and a co-leader, who acted in those capacities from project identification to project closing, ensuring continuity in the supervision of the operation. According to the ICR (page 42), the Bank’s technical and fiduciary teams were focused on the project’s development impact, which led to adjustments in the project, including through two level-2 restructurings (which required Bank board approval) and four closing date extensions. The project was implemented in a challenging environment. Most of the project activities, including the ASYCUDA rollout and infrastructure construction, were carried out in security-challenged locations, including Khost, Farah, Abu Nasr Farahi, and Kunduz. Until 2013, the task team had been able to visit locations cleared by the Bank and UN security. From 2013 onward, the ability of the task team to supervise the project closely was constrained by the fact that most project locations were now in highly un-secure areas. To ensure adequate supervision of the project, the Bank relied on an independent M&E unit within UNOPS, and reporting directly to UNOPS management, to visit the field locations periodically, to inspect the project works, to meet contractors and project engineering staff, and to prepare comprehensive monitoring reports. According to the ICR (page 42), UNOPS shared these reports with the Bank and reported on remedial action taken on the findings cited in these reports. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) The original Project Paper of April 2010 defined six outcome indicators to measure the achievement of the project objective: two each to evidence the effectiveness, the efficiency, and the fairness in the release of goods. At the June 2015 restructuring, the following changes were made to the PDO outcome indicators: Effective release of goods • Indicator 1: Time for customs clearance -- retained. • Indicator 2: Rate of detection of non-compliance -- retained. Efficient release of goods • Indicator 3: Effective rate of physical productivity -- dropped. • Indicator 4: Economic cost per declaration -- dropped. • Replaced by new indicator: Cost savings to business due to the reformed customs system. This was considered a superior measure of reform progress as it was tied to the gains made by key project beneficiaries. Moreover, the methodology to measure these savings, developed by the Bank, had been successfully employed elsewhere to measure the gains from Bank trade facilitation projects. Fairness in release of goods • Indicator 5: Customs indicator from Logistics Performance Index (LPI) -- dropped. Country LPI rankings were influenced by other countries’ performance and were measured with a lag, and hence LPI customs indicator was an unsuitable project performance measure. • Indicator 6: User satisfaction from subjective surveys -- retained. b. M&E Implementation The ACD was the implementing agency of the project, but used the UNOPS to execute the project activities, at least until before the provision of additional financing to the project. As implementing partner to the ACD, the UNOPS performed all project functions, including M&E. According to the ICR (page 39), the UNOPS executed its M&E duties rigorously. The project’s long-term plan was divided into quarterly plans, which were prepared at the start of every quarter, complete with milestones, deliverables, human resource requirements, and procurement schedules. The UNOPS submitted these plans to its Afghan country office, which scrutinized the plans, monitored milestones, and prepared, in turn, Project Assurance Reviews. In addition, the UNOPS reported deviations from plan in Issue Reports, which cited the reasons for any delay in project activities and projected the expected dates when delayed activities were to be completed. Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) For its part, the Bank task team collected project data, updated progress on the performance indicators against baseline values, and highlighted issues requiring the attention of Bank management. Progress with the project milestones and guidance to the implementing agency were recorded by Bank supervision missions in ISRs and Aide Memoires. c. M&E Utilization The findings of M&E reports were used for strategic and operational decision-making. According to the ICR (page 39), data gathered on the number of non-compliances detected by the ASYCUDA risk management system influenced the decision to shift from a commodity-based to a multi-criteria risk assessment system. Data collected on customs efficiency from the user perceptions survey were reportedly “internalized” by the ACD, and subsequently used by the ACD in reporting on customs operations. M&E Quality Rating Substantial 10. Other Issues a. Safeguards Environmental Safeguards: The environmental assessment at appraisal triggered environmental safeguards policy OP/BP 4.01 (Environmental Assessment). The project was assigned an environmental category “B” at appraisal. The category was maintained at the time of the provision of additional financing to the project. The project’s physical investment activities were small in scale and involved the improvement, upgrading, and rehabilitation of ACD facilities. An Environmental and Social Safeguards Management Framework (ESSMF) was prepared for the project to guide the assessment and mitigation of the potential environmental impact of each physical investment activity. Most potential environmental impacts were expected to be related to project construction activities, the possible presence of landmines in project locations, and the decisions on where to locate certain facilities, including power sub-stations. During project implementation, potential adverse environmental impacts were addressed through the application of the environmental codes of practice detailed in the project ESSMF, including those relating to safety procedures and landmine risk. Some instances of non-compliance were observed, although compliance improved as project implementation progressed. Overall, the project did not pose any significant or irreversible environmental impact during implementation. In its last ISR, the Bank supervision team rated compliance with overall environmental safeguards as moderately satisfactory. Social Safeguards: The Bank appraisal team flagged land ownership and land acquisition as potential social issues. However, the team did not expect any land acquisition for the ACD facilities and considered involuntary resettlement as a low risk. Nonetheless, the Bank required documentation that any land involved Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) in the project was free of encroachment, squatters, and other encumbrances by ensuring that land ownership had been transferred to the relevant authorities. Project stakeholders also raised local employment as a potential social issue. Consequently, the Government identified mechanisms to ensure that local populations benefitted from project's construction work. The Government also made a commitment that archaeological, historical, and burial sites, if any were affected by the project, would be protected. b. Fiduciary Compliance Financial Management: The UN financial management system --- covering funds flow, accounting, and audit --- was used when the UNOPS was the implementing partner of the ACD (with the UNOPS also providing technical support to the ACD on financial management matters), while the Afghan country financial management system was used when the additional financing was implemented by the ACD through the Project Implementation Unit. In general, financial management capacity and staffing were adequate with the UNOPS, but weak with the ACD. However, it helped that the ACD retained the project’s key financial management staff. For activities under the UNOPS management, a project-specific float account was used into which advances were paid. The UNOPS maintained segregated accounting records using the ATLAS software. Private auditors hired by the UNOPS conducted annual audits and the audit reports were received on time. For activities under ACD management, a designated account managed by the ACD was used. The ACD maintained basic subsidiary books of record in Excel and submitted consolidated interim financial reports. The Supreme Audit Office of Afghanistan conducted annual audits of ACD transactions. Except for the years 2012, 2013 and 2017, the SAO audit reports were received within their due dates. Audit observations were also addressed in a satisfactory and timely manner. Overall, there were no major financial management issues with the project and financial management performance was rated at various times as either moderately satisfactory or satisfactory. Procurement: Procurement activities followed rules and procedures stipulated in the Bank’s Procurement Guidelines and the operation’s Project Paper, Financing Agreement, Project Implementation Manual, and Procurement Plan, which was revised as necessary in consultation with the ACD and always subject to the Bank’s approval. For example, urgent requisitions filed by the ACD and fulfilled piece-meal were always included in the Procurement Plan. Procurement had a slow start because of an initial insufficient staffing of the procurement function, the departure of international procurement specialists, lack of in-country capacity (absence in the country of authorized dealers for equipment manufacturers), poor responses to solicitations, conflicting procurement procedures and approval requirements between UNOPS and the Bank in the case of direct procurement, and a deteriorating security situation in the country. But, there were no cases of mis- procurement or unresolved complaints from bidders. In its last ISR, the Bank supervision team rated procurement as moderately satisfactory. Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory --- Satisfactory Bank Performance Satisfactory Satisfactory --- Quality of M&E Substantial Substantial --- Quality of ICR Substantial --- 12. Lessons Five lessons are drawn from the ICR (pages 44-46), with some adaptation. Projects in Afghanistan must be flexible with project design, the work program, the implementation arrangements, and even the M&E plan. The political, economic, and security situation changes abruptly and the FCV environment is particularly challenging in Afghanistan. The new government requested a third phase to the customs reform program, to which the Bank responded with the provision of additional financing to the ongoing project while the new government considered its future priorities. The government requested dispensing with the services of the UNOPS, to which the Bank authorized the ACD to implement the activities associated with the additional financing using the ACD's own internal implementation unit. The Bank's flexibility allowed the project to progress toward its objectives. The use of the UNOPS as implementing partner of the ACD was appropriate given the low technical and institutional capacity of the ACD, particularly with infrastructure development. It also helped that the UNOPS was capable with logistics, communications, and security. These were important contributions by the UNOPS to the project in the context of the challenging FCV environment in Afghanistan. It was important, however, that the UNOPS supported the ACD to build capacity in many aspects of project implementation and management. Eventually, the ACD implemented the activities supported by the Bank's additional financing using its own implementation unit. The employment of key ASYCUDA staff under UNCTAD contracts helped reduce staff turnover, particularly of highly-skilled information technology (IT) staff. Staff turnover had been a major concern since the predecessor Emergency Customs Modernization and Trade Facilitation Project was started in 2004. UNCTAD employment contracts were valuable because they offered higher pay and better security arrangements. However, Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) UNCTAD employment can only serve as an interim solution, as the ACD will eventually have to design a pay structure that offered better salaries and job security. Continuity in the composition and staffing of the project unit was a critical success factor with this project, according to the ICR. Many of the project staff had previously served with the predecessor project, the Emergency Customs Modernization and Trade Facilitation Project, which helped with staff familiarity with the project objectives and project operations. Moreover, the UNOPS Team Leader, the UNCTAD Field Commander, and many in the middle management of the ACD were involved with the project, from project start to project closing. The continuity in project staffing and management ensured a consistent attention to project implementation. Standard Bank procurement guidelines which, among others, require procurement only from authorized dealers can be problematic in some countries, including Afghanistan. Many equipment manufacturers do not have authorized dealers in Afghanistan. Many bids received for supply contracts were from un-authorized dealers, derailing the bidding process. Eventually, the project had to resort to using UN Long-Term Agreement (LTA) suppliers. Future Bank projects in Afghanistan and elsewhere where this situation applies should carefully consider this procurement risk. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provides a comprehensive record of the project. The ICR adequately documents the context of the project (pages 10-11), the results framework (pages 11-13), the project design (page 14-15), and changes to the project activities, cost and financing, implementation arrangements, performance indicators, and duration (pages 16-20) over four restructuring episodes, including the scaling up of project activities and the provision of additional financing to the project in 2015. The assessment of the project's results is evidence-based. The ICR provides a useful summary of the operational performance, set within the results framework for the project (pages 21-22), both before and after the provision of additional financing to the project. The ICR also offers a detailed narrative supporting the efficacy ratings of the project (pages 11-15), similarly before and after the provision of additional financing, and detailed accounting of the results and activities of the five project components (pages 25- 29). In addition, the ICR provides an informative summary of the efficiency rating of the project (pages 29- 31), complete with an annex explaining the details of the economic efficiency calculations (pages 66-72). The analysis of the project outputs and outcomes is candid. The ICR elaborates on both the positive (pages 36-37) and the negative (pages 37-38) factors that affected project implementation and the project outcomes. Many of these are reflected in the lessons learned from the project (pages 44-46). It also helps Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AF: Customs Reform & Trade Facilitation (P112872) the analysis that both the UNOPS’s and the ACD’s project completion reports are included as annexes. The minor deficiency of the ICR is its length, 46 pages excluding the annexes. Although outside the ICR preparation guidelines, it provides a complete record of the project. The ICR's rationale for conducting a split rating is unclear, but it was helpful for the purpose of comparing the performance of the UNOPS and the ACD before and after the additional financing in 2015. a. Quality of ICR Rating Substantial Page 17 of 17