69169 Final Program Evaluation of the Second Year of The Leadership and Communication Capacity for National Renewal Program (LCCNR) in Timor- Leste December 11, 2009 Marc Sommers Program Evaluation Expert CMPartners marc.sommers@tufts.edu LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 Table of Contents Page I. Executive Summary 2 II. Introduction 5 III. Reviewing LCCNR’s Goals, Objectives and Hypotheses 6 IV. Methods 8 V. Findings on Contextual Issues 9 VI. Findings on LCCNR’s Training Activities 10 LCCNR’s Program Approach 11 Training for the NP Sector 12 Sectoral Training in Year #2 of LCCNR 13 VII. Conclusions 23 VIII. Recommendations 29 General Recommendations 29 Specific Recommendations for Addressing the NP Process 30 Appendices: Appendix I: Detailed Findings on Sectoral Context, Particularly National Priorities 32 Internally Displaced Persons 32 Youth 34 The National Priorities Process 35 Appendix II: Discussion Paper about the National Priorities Process 40 Boxes: Box 1: Participation Statistics: LCCNR 2008-2009 (Program Year #2) 14 Box 2: An International Official on the National Priorities Process 36 Box 3: A Timorese Official on the National Priorities Process 36 1 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 I. Executive Summary The second year of the Leadership and Communication Capacity for National Renewal Program (LCCNR) in Timor- Leste took place from October 2008 through October 2009. It featured a considerable effort to train and encourage trainees to advance their expertise and confidence as communicators and negotiators, which program officials consider to be central components of leadership. The main program activities were an initial three-day training, 2-day follow-up workshops, coaching sessions, and a final gathering for participants in all of the training activities. There was also a SIMOSOC simulation, which provided insight into the dynamics of social and political conflict and illuminates a number of principles of leadership, communication, conflict resolution and joint decision- making. Field methods for this final evaluation featured extended interviews with open- ended, qualitative questions about LCCNR’s program and the context of their training work. The interviews were carried out in August and September 2009. Particular attention was paid to LCCNR’s experience training Timorese officials involved in the government-led National Priorities process, which was found to be troubled while yielding useful short-term results. The LCCNR program trained 107 Timorese fairly extensively in its second program year. They invested a high degree of expense, effort and time to enhance the leadership, communication and negotiation skills of their trainees. On balance, emphasizing quality for a small number of trainees, given the prodigious contextual and educational constraints confronting most trainees, proved to be the right choice. Overall, again given the challenges that LCCNR faced in training Timorese, the program proved very successful in its training work. Its adaptability to changing contexts, its decision to produce and distribute a training manual, and its success at enhancing the capacities of Timorese officials with LCCNR, were all noteworthy as well as direct contributors to its overall success. LCCNR’s ability to attain its goals, objectives and hypotheses were assessed as follows: Goals • Did LCCNR improve the effectiveness of East Timorese leaders? (Goal #1) Definitely, but for a finite number of leaders. A second, related result is that the experience of being an LCCNR trainee cultivated confidence in young Timorese that they might, one day, become leaders themselves. • Did LCCNR increase prospects for effective policy outcomes? (Goal #2) It is difficult to determine, except for the achievements that it recorded among groups of trainees working, for example, in the Ministry of Social Services and in one (and perhaps more) of the Suco administrations. But whether the work of these trainees will result in improved policy outcomes is fairly impossible to assess. Objectives • Did LCCNR enhance the responsiveness of state institutions? (Objective #1) The answer would be yes, but to a limited degree. Certainly in the Suco in Dili where several LCCNR trainees use the program’s skills and tools effectively. And certainly among the eight trainees working in the Dialogue Group at the Ministry 2 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 of Social Services. Hopefully, LCCNR’s work with NP officials will yield a similar result. • Did LCCNR facilitate the improvement of leader relations? (Objective #2) In a limited number of cases, the answer would be yes. Again, the results at MSS and in a highly volatile Suco in Dili prove that even training a small number of people can yield a narrow amount of demonstrable and positive impact. Hypotheses • Did LCCNR increase the frequency and quality of communication by Timorese leaders, thus improving the quality of leadership in Timor-Leste? (Hypothesis #1) This is probably the case, given that the traditional model of leadership in Timor- Leste does not necessarily include effective communication. LCCNR is contributing to a challenge of this model through its emphasis on effective, inclusive communication and information-sharing. • Did targeting trainings at specific sectors, where participants are working on the same issues, enhance LCCNR’s ability to effect leadership positively in Timor- Leste? (Hypothesis #2) The sector approach yielded very limited results. The assumption that a program can positively impact a sector by training a mere 35 people is not supported. However, smaller units of these (poorly defined) sectors, again such as the Dialogue Group at MSS and at least one Suco in Dili, yielded impressive results. • Did LCCNR’s emphasis on improving the communication skills of leaders in targeted sectors help Timorese leaders build effective, enduring relationships with their constituents and across class, gender, ethnic, tribal, political, regional and other lines? (Hypothesis #3) The best that can be said about this broad and difficult-to-measure hypothesis is that some trainees were very likely working in the directions of inclusive decision-making and building trust and good communication across many lines. That said, it will take a truly concerted effort to begin to eradicate discrimination against and the disempowered status of most Timorese women. General Recommendations 1. Revise and strengthen the current sectoral approach. 2. Transition LCCNR into a Timorese-led program, where most trainers are Timorese and the curriculum is increasingly Timor-focused. 3. Develop LCCNR as a multi-year program. 4. When relevant (such as with high-level government officials), speak openly about and frankly about the fact that white, internationals are the trainers/experts and Timorese are the trainees. 5. Simplify the training jargon as much as possible. 6. Simplify the syllabus. 7. Address LCCNR’s troubling branding issue. 8. Maintain the current level of program monitoring and evaluation while incorporating information-sharing activities as an ongoing component of this work. 9. Make more use of evaluation findings and reports. 3 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 10. Mainstream gender and class concerns in program materials and messages. Specific Recommendations for Addressing the NP Process 1. Direct LCCNR at two specific problems currently confronting the National Priorities process: (1) International-Timorese relations; and (2) operating within a strictly hierarchical environment in government institutions. 2. Devise a curriculum and overall approach to address these problems. 3. Use LCCNR to help NP Stakeholders develop and implement steps to address the two problems mentioned in Recommendation #1. 4 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 II. Introduction Can a training program aimed at enhancing leadership skills positively impact the leaders of a new post-war nation? If so, will the program’s impact prove to be lasting? These two questions lie as the core underlying purpose of the Leadership and Communication Capacity for National Renewal Program (LCCNR) in Timor- Leste. This report contains the final evaluation of the second year of LCCNR (known as LCCNR 2), which took place from October 2008 through October 2009.1 Mainly based on field research that took place in August and September 2009, it will also draw from findings that were undertaken for the midterm report2 and, secondarily, data gathered for all five evaluation reports that have examined LCCNR’s two program years (2007- 2009).3 This final evaluation will focus on the impact and effectiveness of the sectoral approach that was the most significant revision of LCCNR’s second program year. LCCNR carried out three sets of training activities. The initial set was for leaders of what was called the IDP (Internally Displaced Persons) Sector. The second set of training activities was for a selection of government and non-government youth leaders. The final activity set was for Timorese government officials who are directly involved with the National Priorities (NP) process, which is a process through which the government of Timor-Leste (in coordination with international donor partners) identifies priorities and sets out ways to achieve them. This report is organized in the following way. It will review the program’s goals, objectives and hypotheses before describing the methods that were employed for gathering field data for this report. Next, it will examine contextual issues that directly influence the three sectors that LCCNR has sought to impact by training some leaders in each sector: IDPs, youth and National Priorities. After this consideration of context, the report will detail findings about the impact of LCCNR’s training activities on its trainees. The final sections will provide, respectively, conclusions and recommendations for LCCNR. An extended discussion of context, particularly concerning the National Priorities sector, can be found in Appendix I. This report will draw upon research findings and analysis from two recent reports from CMPartners. The first is LCCNR’s Mid-Term Field Evaluation (May-June 2009).4 The Mid-Term Report looked exclusively at the impact and context of LCCNR’s trainings in the first two sectors within which it trained: the IDP and youth sectors. Field research for 1 As of this writing, a limited extension of this second program year is underway. The extension would close LCCNR in November 2009 instead of October. The extension period will not be included in this evaluation. 2 The midterm report was published July 20, 2009. It was based on fieldwork undertaken in May and June 2009. 3 See, for example, the section in the Conclusion entitled “A Challenging Context,” which directly draws from evaluation research since 2007. 4 The report can be found on CMPartners’ website at: ENTER WEB ADDRESS WITH HYPERLINK. 5 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 this report looked primarily at LCCNR’s work with leaders in the National Priorities sector. Accordingly, while some interviews for this report involved the IDP and youth sectors, the Mid-Term Report will be used as a central resource for considerations of these two sectors. The second is a discussion paper entitled Enhancing Post-War Government Capacity: The National Priorities Process in Timor-Leste. It was written in September 2009 and contains findings and analysis of field interviews with those involved in the National Priorities effort. Publication of the paper is imminent. The paper will be used as a resource for discussions in this paper about the context of the NP. It is also included at the end of this report in its entirety (see Appendix II). III. Reviewing LCCNR’s Goals, Objectives and Hypotheses Gauging the impact of LCCNR’s second year will largely involve an analysis of its achievements with reference to the program’s goals and objectives. It will also assess the validity of the program’s hypotheses, which directly connect to LCCNR’s goals and objectives. In this section, the program’s goals, objectives and hypotheses will be introduced and considered. They will then surface again in the conclusion, when the findings and analysis derived from the evaluation research period will be used to assess program impact. It is useful to keep in mind that LCCNR’s goals and objectives are ambitious while its reach and budget are limited. The unit cost per program trainee is fairly high. Since LCCNR has poured considerable effort and cost into a finite number of trainees, the program’s expectation, one must assume, is that training small numbers of leaders well matters much more than reaching more leaders but having less time to train them. In short, LCCNR’s approach favors quality over quantity to achieve positive impact, and it is from this initial evaluation factor that the program will be partly assessed. LCCNR’s second-year goals are: 1. Reinforce leadership capacities across state and non-state actors in both formal and non-formal institutions within specific sectors. 2. To enhance communication skills and encourage information sharing within and amongst state institutions, civil society and the public in the hopes of improving policy outcomes. A second key evaluation factor concerns the issue of training participants by sector. LCCNR sought, in its second year, to achieve its first goal (above) by training one group of 35 trainees in each of three separate sectors. This proved to be a tall task. There is also a definitional challenge related to this second evaluation factor. Defining members of certain groups as sectors was initially examined in the Mid-Term Field Evaluation of July 2009, which found the definition of ‘sector’ to be difficult to pin down. Fortunately, program officials have devised a general working definition for a sector, which will be used in this report. According to LCCNR, a sector is “A demarcated 6 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 group of people who work on specific issues.” This definition will be referred to in subsequent analysis later in this report. A third key evaluation factor is raised in the second program goal. It concerns how leadership is defined. The program is inferring that communication and information- sharing are core parts of effective leadership, and that, by training leaders to improve their communication and information-sharing skills, policy outcomes will improve. LCCNR’s second-year objectives are: 1. Assist the state in establishing the systems and attitudes necessary to fostering better communications. 2. Support leaders from diverse groups in society as they seek to improve their ability to engage with each other in a spirit of cooperation and mutual understanding. Since LCCNR is a leadership training program, the first objective effectively equates training a finite number of Timorese leaders to become better communicators with helping Timor-Leste’s government become better at communication. However, it is not clear with whom the state is communicating. The frame of reference is vague. The second objective infers that training a small number of Timorese leaders in particular sectors will improve the quality of positive engagement with a broad array of Timorese leaders. It also directly addresses a concern that the authors of a pre-program scoping mission found: that Timorese leaders want to improve their engagement with other leaders; a significant finding, given that relations between Timor-Leste’s political parties is persistently spirited and contentious. The fractious nature of Timorese politics also makes achieving this objective problematic, particularly since LCCNR was able to train a small number of leaders. To facilitate the evaluation of LCCNR’s second year, the program’s two goals and two objectives will be boiled down in the following way. By training approximately 35 leaders in each of three separate sectors, LCCNR hopes to achieve four outcomes: 1. Improve the effectiveness of leaders (Goal #1); 2. Increase prospects for effective policy outcomes (Goal #2); 3. Enhance the responsiveness of state institutions (Objective #1); and 4. Facilitate the improvement of leader relations (Objective #2). LCCNR’s second-year hypotheses are: 1. Increasing the frequency and quality of communication by Timorese leaders will improve the quality of leadership in Timor-Leste. 2. Targeting trainings at specific sectors, where participants are working on the same issues, will enhance LCCNR’s ability to effect leadership positively in Timor- Leste. That is, does it indeed make a difference to train people in the same sector, 7 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 who are all working on the same (or similar issues)? Will they solve their problems more effectively if they collectively use LCCNR’s skills and tools? 3. LCCNR Year #2’s emphasis on improving the communication skills of leaders in targeted sectors will help Timorese leaders build effective, enduring relationships with their constituents and across class, gender, ethnic, tribal, political, regional and other lines. • This result will lead to more effective and responsive Timorese institutions. The first two hypotheses suggest that LCCNR will achieve positive impact by enhancing the communication skills of leaders in specific sectors. The third hypothesis is difficult to evaluate because it is unusually broad and ambitious. Examples of achievement (or not) will be mentioned, should they surface. IV. Methods The field methods for this final evaluation feature extended interviews with open-ended, qualitative questions about LCCNR’s program and the context of their training work. Most of the interviews took place in Timor-Leste between August 24 and September 4, 2009. Follow-up interviews took place via Skype and telephone from September 8 until September 25 of the same year. In all, 42 interviews were conducted with a total of 55 people. All but 17 of those interviewed were Timorese (69%). All interviews with the 17 international officials took place in English. A translator joined me for all interviews with Timorese that were conducted in Tetun or Portugese instead of English. Since the source of all interview data remains confidential, no names will be attached to any particular individuals. Most of the field interview work focused on understanding the context and impact of the third sector of interviews because training work in the National Priorities sector only began after fieldwork for the previous evaluation had ended. Accordingly, half of all of those who were interviewed – 10 of the 16 interviews with trainees (62.5%), 12 of the 17 interviews with international officials (70.5%) and an additional five interviews with Timorese who were not program trainees – were directly involved in the National Priorities process, including officials whom LCCNR trained. A few were interviewed more than once. Five IDP sector trainees were also interviewed, as well as one youth sector trainee, two youth experts and 13 out-of-school youth. The remaining interview sources were a combination of program staff and Timorese with experience in training programs. Other evaluation activities were part of the evaluation research process. I observed three separate LCCNR training events: a NP workshop, the SIMSOC simulation exercise, and the graduation day for youth and IDP sector trainees. In addition, I attended a National Priorities quarterly meeting that was held at the Ministry of Finance on August 25. 8 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 The interviews featured open-ended questions that invited interviewees to comment about the training experience (if they were trainees), the relevance of the training program (if they were familiar with LCCNR), and contextual issues relating to the IDP, youth and, especially, National Priorities sectors. Additional questions were added to explore the ways in which Timorese acquire new information and issues relating to interactions between different groups of people that relate to the program’s prominent interest in improving communication. The different groups included men and women, Timorese and International officials, and Timorese who had been educated within Timor-Leste and Indonesia or were educated outside of the immediate area (Australia in particular, as well as other places, such as Portugal). Three other issues relating to the evaluation research process will be added here. First, while I asked no questions that directly addressed LCCNR’s goals, objectives or hypotheses, the responses were analyzed, in part, with reference to these three crucial sets of program concerns. Second, since the prior evaluation studies focused on the IDP and youth sector concerns, the evaluation findings and analysis for this report will also draw from the field research that took place in May-June 2009. Third, the LCCNR program, which had been planning to close down in October 2009, received the go-ahead to carry out a third workshop in November for National Priorities trainees – the same number of workshops as the IDP and youth sector trainees received. Yet since this workshop took place well after field research for this evaluation ended, it is not a part of the analysis for this report. V. Findings on Contextual Issues Research on the three sectors in which LCCNR became engaged in its second year illustrated the truly considerable challenges that leaders within each sector, and thus LCCNR’s program, faced. Setting LCCNR’s work into context was important in gauging the program’s ability to achieve positive impact within each sector, as well as help determine how the program might address sectoral concerns in future years. Most of the findings on context are contained in Appendix I of this report. Since the context about Internally Displaced Persons (IDPs) and Youth was covered in detail in the previous report, the sections in Appendix I on these two topics will provide short updates. Much more will be said, however, about the high profile and controversial National Priorities process, which was not discussed in earlier reports. The technical capacity needs of Timorese government officials proved to be a persistent theme during interviews about the NP. Given its direct relevance to LCCNR’s goals and objectives, considerable attention was paid to NP issues. A brief review of context in the three sectors will is provided here. In the IDP sector, the neighborhoods to which most of the IDPs returned remain tense and sometimes volatile. Several officials reported that resentment of returnees about government payments they had received upon their return home is thick. Threats against returnees, or the returnees’ neighbors, remain. The bitterness surrounding the return of IDPs also appears to have catalyzed existing tensions, aggression and clashes involving Martial Arts Groups in 9 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 urban neighborhoods. In the youth sector, rampant youth unemployment remains a widespread concern, and indications of substantial if not severe separations between youth by class and education (including suspicions by out-of-school youth about youth who were educated in Australia) are significant. Finally and critically, research on the NP process revealed evidence of short-term success, as well as disturbing signs of inter- relational problems mainly involving relations between international officials and Timorese government officials. A second issue concerned hierarchical relations that stymie communication between Timorese officials at high and lower government levels. The state of affairs between internationals and Timorese in particular has promoted a broadly shared impression that highly paid internationals (together with high level Timorese officials) are dominating central government initiatives while most Timorese government officials hold entirely subordinate roles. Accordingly, capacity-building efforts that are largely built around the expertise and leadership of international trainers (such as LCCNR) run the risk of unintentionally reinforcing the following stereotypes: that most international officials have technical ability and initiative while most Timorese officials do not. The findings provided in the NP section of Appendix I (as well as all of Appendix II) detail a situation of particular concern for those involved in that process. It is also one that challenges LCCNR and, for that matter, any agency attempting to address the pressing capacity needs of Timorese officials. How a future LCCNR program might be used to address the NP’s pressing concerns is part of this report’s recommendations for next steps. VI. Findings on LCCNR’s Training Activities This section will begin with a short review of LCCNR’s program approach.5 It will then provide review some key findings about LCCNR’s NP sector work (that is, LCCNR’s provision of training to some Timorese government officials who are involved in the NP process). This section will conclude with a review of the most prominent findings arising evaluations of LCCNR’s second program year, drawing from training experiences with IDP and youth leaders in addition to NP sector trainees. This discussion is meant to provide summations of key findings: more detail about findings about LCCNR’s work in the first two sectors (IDP and youth leader training) can be found in the Mid-Term Field Evaluation (May-June 2009) report. 5 The description is based on an excerpt from pages 5 and 6 of the Mid-Term Field Evaluation (May-June 2009) report. 10 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 LCCNR’s Program Approach LCCNR’s second program year featured a considerable effort to train and encourage trainees to advance their expertise and confidence as communicators and negotiators, which program officials consider to be among the central components of effective leadership. The main program activities were an initial three-day training retreat, 2-day follow-up workshops, coaching sessions, and a final gathering for participants from all of the training activities. The core of the program’s curriculum lay in its Seven Element Framework for managing negotiations.6 The elements are, in sequence: (1) Understand Interests (2) Develop Options for Joint Gain (3) Use Objective Criteria and Standards (4) Assess the Alternatives to a Negotiated Agreement (5) Develop a working relationship with your counterpart (6) Use effective communication, and (7) Develop operational and mutually understood commitments. The program supplied additional tools (and frameworks) that complemented the Seven Elements and focused on specific communication and relationship skills, such as the Ladder of Reflection and the Active Learning Cycle. The Ladder of Reflection and the Active Learning Cycle were scarcely mentioned during interviews. Instead, the main tool/framework that trainees and program staff alike highlighted, in addition to the Seven Elements, was the Four Quadrant Tool. This tool is represented as a circle cut into quarters. It provides simple questions for addressing each part of the 4-part sequence: (1) identifying a problem (or conflict); (2) analyzing and prioritizing the causes of the problem; (3) exploring possible options for addressing the prioritized causes; and (4) developing specific action steps for the implementation of the prioritized option, developed in Quadrant 3. The underlying pedagogical ideas appear to be two-fold. First, the Four Quadrant Tool begins with the situation as it exists and ends with concrete steps that can be undertaken in reality, while the two intermediate steps feature creative thinking for both understanding the current problem (#2) and identifying possible solutions to resolving it (#3). Second, it provides a structured method for enhancing one’s decision-making skills. The second year of LCCNR’s program targeted leaders in three specific sectors for training: leaders who were directly involved with IDPs and their communities of origin, youth leaders, and Timorese government officials who are directly involved in the NP process. The rationale for this sectoral approach was sound: directing a program with limited reach and resources at leaders who are engaged with shared issues and challenges increases prospects for positive program impact. The program, however, spread itself across three sectors in a single year. This was done at the behest of the World Bank, not LCCNR officials. In fact, LCCNR had made a strong recommendation to the World Bank 6 The Seven Element Framework was developed at the Harvard Negotiation Project and inspired by the book, Getting To Yes: Negotiating Agreement Without Giving In, by Roger Fisher, William Ury, and Bruce Patton (Second Ed., Penguin Books 1991). Additional information can be found at www.pon.harvard.edu. 11 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 to focus exclusively on one sector for its entire second year. The World Bank decided not to take this recommendation on board. The question of whether training one group of sector leaders (approximately 35-40) is sufficient for achieving positive impact in that sector, through intensive training of a finite number of leaders in each sector, will be addressed later in this section. Finally, a word on changing contexts for trainees and the adaptability of LCCNR. The mass return of IDPs to their former communities was an issue that the LCCNR team addressed in three specific ways. First, it addressed this issue during coaching sessions with IDP trainees when large returns were underway in March and April 2009. Second, the issue of IDP returns was addressed in the first follow-up workshop which took place during this period. Finally, at the request of IDP trainees, the LCCNR team wrote a new negotiation case based on real-life Timorese experiences for the final, May workshop for IDP trainees, which allowed trainees to directly address the many challenges of mass IDP returns to their former communities within the context of the training. Training for the NP Sector Tasked by the World Bank with the challenge of providing leadership training to Timorese government officials participating in the NP sector and who, for the most part, have attended related training workshops in the past, LCCNR’s program has found itself involved in the divisive debate over international-Timorese relations. It was an uneasy place to begin training Timorese government officials. Led by World Bank officials, the LCCNR team set out in April and May to gauge the interest of important Timorese officials in the NP process, such as Director Generals, Directors and Focal Points, in attending an LCCNR training workshop. Initially, there was considerable resistance to entering a training program. Among the reported reasons were: • Some of the officials who received a written invitation from the World Bank to attend the LCCNR’s leadership training felt that the letter, in the words of one of those who were invited, “sounded like an order” to attend. “There was no information about who was doing the training, the content of the program.” As a result, some invitees delegated lower level officials to attend the LCCNR workshop. • LCCNR was targeting officials whose work schedules were unusually demanding. Many said they declined to attend because they lacked the time. One Director General explained that he was “too busy to attend the leadership training program and I couldn’t get a person [on my staff] to replace me” at the training. A third official reported that his Minister did not allow any officials to attend the LCCNR workshop because “we were busy preparing for the budget review meeting.” Others said that they were under significant pressure to deliver budgetary and reporting deadlines, and did not consider the training opportunity to be a high priority. 12 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 • The leadership training did not seem especially relevant to some of those who were invited. Many preferred technical training on more specific skills that are, in the view of one official, “relevant to their daily work.” Among those mentioned were learning English, time management skills, writing skills, and strategic planning skills. Foreign languages proved a persistent concern. One official noted that “A big difference between national and international officials is the language difference. I may know certain information better than an international, but I can’t express it smoothly in English or Portugese.” Another official noted that leadership training is potentially frustrating because, in the hierarchical, decision-making system of Timorese bureaucracies, there may be few opportunities to apply the skills. “I can attend a training,” one official explained, “But I cannot forcefully present my positions to the Minister. If he says ‘A,’, it will be A. If he says ‘B,’ it will be B. It’s all about power.” • In the high pressure environment surrounding the NP, LCCNR officials reported that some Timorese officials sought to keep a low profile and avoid situations where they might be embarrassed. As one official noted, some of the officials were “afraid of failing, so they didn’t want to get trained.” • Some officials seemed exasperated by the lack of coordination between different training efforts. One noted that “There are a lot of trainings done by many countries. But they all have different approaches and systems. So Timorese come back from trainings with different approaches and systems. Which one is the best to use in Timor-Leste? Sometimes, all these trainings confuse me.” Given some of these concerns, the LCCNR team chose to postpone the scheduled workshop in June and instead conduct a series of diagnostic interviews with high level Timorese government officials who were directly involved with the NP. The purposes were to better inform participants about LCCNR’s program, stimulate interest in attending the workshops, and learn about the National Priorities process. The effort produced positive results: before the diagnostic, LCCNR officials were having difficulty generating interest in the LCCNR workshop with NP officials. That began to change following the diagnostic, which proved to be an important breakthrough for LCCNR’s ability to work productively with Timorese government officials who worked on the NP. Sectoral Training in Year #2 of LCCNR Participation The participation statistics support the evaluation finding that LCCNR’s program had the strongest and most positive impact among IDP sector trainees. IDP sector work attracted the highest number of people to all of the workshops. The drop-off in Youth sector trainees who attended the second workshop was much greater than for the other two groups. Judging from the interview data, among the reasons behind the drop-off was that 13 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 some trainees questioned the program’s relevance for them, since a clear finding with Youth sector trainees was that most of them do not consider themselves to be leaders. Another reported reason is that some Youth sector trainees declined to attend the second workshop without being paid. (Although LCCNR’s policy was not to pay per diem, the program covered lodging, transport and meals for all workshop participants.) Attendance to the SIMSOC simulation, which is an activity that enables trainees to begin to apply the skills they are learning, again indicated the comparative level of engagement in LCCNR’s program among IDP sector trainees. The NP sector’s participation numbers are much smaller than the first two sector groups. LCCNR officials anticipated this all along, largely because there were far fewer people who could be invited. Box 1: Participation Statistics: LCCNR 2008-2009 (Program Year #2) 7 Sector Groups: Group 1/IDP Group 2/Youth Group 3/NP No. of people invited to the 1st workshop 80 82 29 No. of people attending the 1st workshop 45 42 20 No. of people attending the 2nd workshop 33 27 16 No. of people attending the 3rd workshop 32 22 15 No. of people attending the SIMSOC simulation 32 25 N/A 7 Statistics were graciously provided by LCCNR program staff. The third workshop for Group 3/NP took place after research for this evaluation was concluded and is not analyzed in this report. 14 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 Retention and Application Gauging the degree to which trainees retained the skills and tools that were taught in LCCNR workshops will be divided into two parts. This is because the NP sector trainees received considerably less training experience: they had attended one or two training workshops by the time of field research for this report. They will thus be addressed separately. NP Sector Trainees: A prominent finding arising from interviews with this trainee group was that one to two workshop experiences was insufficient to ensure lasting impact for most trainees.8 Some of the NP sector trainees appeared to be jaded by the sequence of training projects that they had experienced before LCCNR came along. The interviews revealed three additional important findings: • Most trainees believed that the program was relevant, interesting and useful. Indications of specific tools and skills that they retained was much less frequent: the descriptions tended to be general, even vague. Having the workshops in Baucau instead of Dili was also, in general, thought to be a very good idea. As one trainee observed, “It was nice to get outside of town to get refreshed.” • A related finding was that most trainees who were interviewed contended that the training they received was insufficient. Illustrative of this sentiment is the following comment from one trainee: “For me, the training was very important. It’s appropriate for senior level officials. But it wasn’t long enough. One week of training is not enough.” • As with trainees from the other sectors, some trainees registered real interest in the structure that LCCNR’s approach to negotiation supplied. A common way of describing LCCNR’s particular contribution is that it helps trainees be “systematic” when attempting to negotiate or address a conflict. • The particular relevance of LCCNR’s workshop, in the view of many NP trainees who were interviewed, lay in its potential to improve their negotiation skills. “When we have good relations,” one trainee stated, “we can move ahead together.” It can also boost a person’s confidence in themselves as negotiators. As one trainee stated, “The training is very useful because it motivates trainees to negotiate with anyone.” While negotiation is not the only skill that the LCCNR training emphasized, its particular relevance to NP officials is noteworthy and repeatedly surfaced during interviews. “I didn’t expect to find negotiation in the training,” one trainee commented. “What I learned was encouraging. It was really helpful.” Another official recommended that the training be more explicitly directed at the challenge of negotiating with international officials: 8 Since NP trainees were interviewed before LCCNR’s second year was extended to include a third workshop for NP sector trainees in November 2009, it is highly likely that the trainees’ skills and retention improved after that experience. 15 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 We need the training to help us work with international officials [he mentioned donor officials in particular], to learn to communicate and negotiate better with them. I have a problem when internationals say they want this or that. I need more skills to learn how to manage [their demands]. If there’s no negotiation, then the two parties stay apart. Confidence in applying negotiation skills would seem to be necessary as well, since working with international experts can be intimidating. As one Timorese official observed about himself, “In our hearts, we want to work with internationals, but we feel so inferior. Sometimes I just don’t have the confidence.” While skills transfer and retention was weak among NP sector trainees generally, the training experience was not without significant impact for many of those interviewed. A prominent finding was that the training experience gave trainees the confidence to negotiate, communicate more effectively with others (or at least try to do so) and contribute to finding solutions to conflicts – even, just perhaps, with the international advisors who clearly intimidate many of the NP sector trainees. For people unaccustomed to making such efforts, these can be breakthrough experiences. Whether trainees continue to attempt to assert leadership skills (the issue of LCCNR trainings boosting confidence was a significant finding for many youth sector trainees as well) is impossible to determine from this evaluation. However, building the confidence of what was clearly a substantial proportion of trainees proved an indicator of impact. IDP and Youth Sector trainees: In terms of retention and application, the IDP and Youth sector trainees divided into, roughly, four groups. The first group contained those trainees who learned the skills and then confidently adapted and applied what they had learned afterwards. These were mostly IDP trainees, and for good reason. The IDP trainee group contained some veteran leaders, including Suco and Aldeia level officials. These experienced leaders had a level of confidence in acquiring and applying LCCNR’s skills and tools that other trainees simply did not have. They clearly created a kind of group chemistry with many in the IDP trainee group that did not exist with the Youth sector group. Examples of their application of the skills can be found in the Mid-Term report (see, for example, pages 15-16) as well as in the Evidence of Sectoral Impact section (below). While the numbers of this group are difficult to estimate, it was clearly a substantial minority of IDP group trainees and a much smaller number of Youth sector trainees. Members of this group constitute the most substantial and potentially lasting indications of positive impact from LCCNR’s training. Their ability to master LCCNR’s skills and tools, and then confidently adapt them to local contexts and to their own preferences, was thoroughly impressive. That this group was a minority and not a majority of trainees is not surprising. LCCNR’s pedagogy and approach to communication, negotiation and leadership is a challenging, complex and, for Timorese, new approach that most 16 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 Timorese trainees were unable to fully master and confidently apply. But when the training took hold, the results were striking. The second group of trainees was small but worth noting. This would be trainees who grasped what LCCNR was teaching but, afterwards, chose not to apply what they had learned. An IDP sector trainee who was interviewed in June (a Suco official) explained his reason for doing so: his methods work, so there would be no reason to change them. He found the training interesting, but in the end, he saw no point in drawing from anything that LCCNR had taught him. This group may include more than one trainee – only one of those interviewed could be considered a member. The existence of this group, however small, appears to indicate something that many of those who have been interviewed suggested: that the leadership and negotiation skills that Timorese already know and use work. Some may see LCCNR’s techniques as a method for improving their effectiveness. Others do not. The third group tended to memorize LCCNR’s material but were then unable to apply what they had memorized. These trainees could recite aspects of the Seven Elements and the Four Quadrants, often with remarkable ease and precision. But when it came to sharing stories of whether and how they applied what they had learned afterwards, most were stumped. Being conceptually creative with what they had gathered, and adapting their new knowledge to fit everyday situations, proved too difficult for most. For example, many were unable to shift from knowing all Seven Elements to, perhaps, selecting and applying just two or three of the elements to address a conflict. Not surprisingly, members of this group complained about how LCCNR trainers answered their questions. The trainers use questions to ask trainees how they might address a challenge. Most trainees were clearly not used to this, and were frustrated by the apparent refusal of the experts – the LCCNR trainers – to give them specific direction (see further explanation of this issue just below). The members of this group clearly comprised a substantial number of trainees (likely the majority), particularly among Youth sector trainees. Members of this group tended to be fairly young, were educated exclusively in Timor- Leste (or perhaps in Indonesia as well), and either lacked confidence as leaders or didn’t believe that they were leaders. Again, this was particularly the case with the young trainees. One trainee reported a discussion during a break at an LCCNR workshop. He reported that one of his colleagues wondered whether the World Bank Training was appropriate for him since “I realized that I won’t ever become a leader. There’s already a Chef de Suco and Chef de Aldeia where I live. So what’s the reason for me to attend a leadership training workshop?” In general, trainees belonging to this group were mainly educated during the Indonesian occupation, when memorization lay at the center of the pedagogy for all formal education. Such an educational experience prepares people to be obedient and competent. But it does not prepare them to think creatively – precisely the approach that LCCNR pedagogy requires. The learning curve for this set of LCCNR trainees was, in general, 17 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 substantial, and many clearly were unable to master the creative application of LCCNR’s skills and tools with any confidence or broad success. Many members of this group also noted that they viewed the “World Bank Training” (by far the most common name for the LCCNR training) as a course. Many stated that they were motivated, in part, by the fact that completing their training would result in their receiving a certificate, which, as one youth trainee stated, “can help us get a job or a scholarship.” All of this said, it is important to note that this same set of trainees regularly reported that being an LCCNR trainee boosted their confidence, and gave them a sense that, one day perhaps, they would become leaders. This was a common comment during interviews with female youth trainees, for example. They enjoyed being a part of LCCNR, and many said that the attention and positive feedback they received from LCCNR staff and trainers surprised them, as if they did not anticipate that they would be treated so well. The training experience, in short, was entirely new for them. Sure, it was challenging, difficult and seemed far too advanced for them. But it was also thought-provoking. In its way, this is indeed positive impact of a sort: LCCNR may well have played an necessary first step for helping some trainees move from rote to creative learning. The fourth group of trainees did not gain much lasting impact from the LCCNR experience. One can assume that some number of any training contingent would likely contain a handful of trainees of this sort. That was the case here. Cynicism and practicality characterized this group by trainees, which was certainly the minority of trainees. Members of this group could not recall in any detail what they had learned during LCCNR workshops, or described concepts in ways that were incorrect. There are reasons for this. First of all, it was widely stated that some Timorese attend large numbers of trainings, most particularly national government officials. As one commented, “Some people ignore trainings because they’ve gone to so many.” “We’ve attended more than one hundred trainings,” another high level Timorese government official stated. “So they feel the same. We may discover new things, but we just see it as nothing new.” As a result, some trainees hold out little hope of receiving any real impact from attending any training workshop. In addition, some reported that the different pedagogical approaches of different training programs ultimately confused them. That alone could help to undermine a trainee’s motivations. Another factor that can limit a trainee’s initiative is that some were sent to the training by their office. In such cases, a person may be entirely unmotivated to apply themselves during the training. Another reason is that the trainings might be overlong and too difficult to grasp. Trainees shared a list of other reasons for attending the LCCNR workshops, none of which were directly tied to building one’s capacity as leaders. A person might attend a training because of: 18 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 • The promise of eating good food and having, in the words of one trainee, “anything they want to drink, like Coca-Cola.” Another trainee added that “Some people just go to a training to get food: breakfast, snacks and so on.” • The hope of receiving some funding (including some of those attending LCCNR trainings, because they were considered a “World Bank Training”). “A lot of participants,” one leader stated, “they just go, they don’t pay attention. They make drawings and they expect to be paid for attending.” Another remarked, “Some trainees were very committed to learning at the workshops. But others were there to get money for transport. They weren’t happy to be there.” • The hope of receiving a certificate, which may help them get a better job in the future.9 Certainly members of the first three groups may well have enjoyed the food that they ate at the workshops and held out similar hopes for funding and a certificate. What sets the members of this fourth group apart is that they seemed to lack interest in the training experience itself. Language and Learning Challenges This brief section will review what emerged as prominent language and learning challenges during LCCNR workshops. Some have been mentioned earlier in this report, and others were listed in the Mid-Term report. They are collected here in an attempt to provide a cohesive presentation of them. 1. The structure of the training sessions largely featured international trainers with Timorese colleagues and translators. The Timorese staff members do much more than just translate. They also lead workshop discussions and lead presentations. Yet it was clear to the trainees who was in charge of the trainings: the international trainers. The trainers were popular, and in some cases revered. Having internationals strongly in the lead of training sessions was unavoidable and appropriate. However, two significant and largely unavoidable constraints arose from the structure of workshop teaching. First, the presence of internationals as ‘experts’ lent unfortunate strength to the stereotypes of internationals with superior knowledge and skills and Timorese with inferior knowledge and skill sets. LCCNR is merely a two-year program, and the roles of Timorese staff have been expanded dramatically. This by itself is a sign of LCCNR’s positive impact, because capacity building took place within the organization, not just in workshop settings. Second, Timorese staff who served as translators regularly required considerable time to translate even short, declarative statements by the international trainers (who spoke in English). The pronounced difference in the vocabularies of Tetun and English requires translators to illustrate concepts from English with examples. As a result, translations in Tetun during training sessions can be lengthy. Some trainees commented that the 9 These three possibilities were drawn from the Mid-Term report (p. 12). 19 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 translations include far too many examples, which sometimes made it hard to follow the narrative in the training session. Some reported that they periodically gave up trying to keep up if they could not follow the examples. 2. There were frequent expressions of frustration about how LCCNR trainers responded to questions. Repeatedly, trainees would ask trainers for what they considered answers to particular questions. The trainers would sidestep them. One trainee expressed his frustration as follows: “We expect to get some response from trainers according to his perspective. But the trainer provided no solutions for us.” Afterwards, some of the trainees reportedly discussed among themselves their frustration with the tendency of LCCNR trainers who encourage trainees to address problems themselves instead of providing concrete answers for them. According to Timorese culture, however, a trainee explained that “Trainees didn’t express this to the trainers because they don’t want to offend them.” 3. The tendency to expect explicit information to be delivered to trainees by their trainers is an outcome of the way in which most Timorese have been educated. As mentioned above, most were educated in an Indonesian system that privileged mastering material through memorization. Trainees with this sort of educational background were challenged by the way in which LCCNR program trainers taught their material. A veteran trainer in Timor-Leste explained the challenge in the following way: “People here aren’t used to conceptual learning and creative thinking. It’s a small group who can take training ideas and adapt them creatively” Those that were able to adapt LCCNR’s training ideas creatively were members of the first group, described above. They were a prominent but minority group of trainees. Evidence of Sectoral Impact Before sectoral impact can be detailed, the concept of sector, as applied in LCCNR’s program, must be assessed. All three ‘sectors’ are moving targets, particularly the IDP sector. When the second program year started, the challenge of large IDP populations was arguably the most pressing issue in Timor-Leste. LCCNR’s idea was to help some leaders deal with mass returns to their communities. They did, but while the IDPs formed a kind of sector at the outset of LCCNR’s second year, the group’s sectoral unity has mostly evaporated, since most IDPs are back in their former homes. At the time, this was a daring and appropriate move by LCCNR, and the virtual evaporation of the ‘sector’ is not, in any way, a program weakness. LCCNR’s second target group – youth – are less a sector than a population demographic. On the other hand, Timorese government officials working on NP concerns certainly formed a sector in the way that LCCNR had defined it: as a demarcated group of people who work on the same issues. Setting this general framework aside, it is nonetheless possible to detect impact by smaller groups of trainees within each group. It should also be noted that the issue of 20 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 LCCNR’s sectoral approach in its second program year was extensively explored in the Mid-Term report for the IDP and Youth sector groups (see pages 6-9). What follows here is a more succinct description of impact in all three sectors. IDP Sector Trainees: The most significant and promising signs of impact were found among trainees from the IDP sector group. Evidence of generally collegial if not excellent relations among trainees within this group of trainees was notable. Nearly all of the most accomplished trainees (in terms of learning and applying what they learned) belonged to this group as well. In terms of sectoral impact, it can be gauged if the IDP ‘sector’ is divided into smaller groups. In this case, strong evidence of impact could be found in one Suco in Dili that was plagued by violence during the 2006 crisis and among some members of the Dialogue Group in the Ministry of Social Services (MSS). These are substantial accomplishments, and they are noted in addition to the impact that LCCNR’s training efforts had on individuals as well. In the Suco in Dili mentioned above, the fact that LCCNR trained several Suco leaders allowed them to work as a group to solve what appears to be endless numbers of land and other conflicts. These trainees are experienced at negotiation, communication and demonstrating leadership, and have all applied the training concepts that they learned from LCCNR into their work as Suco leaders. Two have held small training sessions about what they learned from the LCCNR program for other leaders. The two also used what they learned to institute a traditional approach to conflict resolution, the Ai Lulik (detailed in Appendix I), in their Suco. As one of the trainees explained, The ‘World Bank Training’ is relevant to our work because during the training, we learned about communication, options and relations. We also learned how to listen to people. So we were able to hear that people were coming together to decide which option to choose for our Suco. And they chose this option: to use the Ai Lulik. LCCNR staff targeted members of the Dialogue Group within MSS for inclusion in their IDP sector training group. As a result, 8 Dialogue Group members attended the LCCNR training and all 8 received certificates. Others refused to attend the training, saying that they didn’t need it. What resulted was a division of members of the Dialogue Group into two teams: one that had been trained by LCCNR to negotiate conflicts involving IDP returnees and residents, and one that was not trained by LCCNR. As one trainee explained, We separated between those who had the World Bank Training and those who did not. We do our mediations separately. There are now two groups of mediators. There are tensions between the two groups, but we try to mediate before we go to the field. One of those whom the LCCNR trained explained the difference that the training experience has made. “Before we were trained by LCCNR,” the official said, “we did a lot of dialogues between IDPs and residents. But there was no systematic approach. It was spontaneous.” After being trained, he said that he and his colleagues had the ability to analyze a problem effectively. “We tried to approach conflicts in a systematic way. We made a plan, an assessment and then we implemented it.” 21 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 The MSS official also made an interesting decision that trainees have to make after they complete LCCNR’s program. They must decide whether to play the role of a leader or a mediator when addressing a dispute. As he explained, If you come [to a dispute] as a leader, you come to speak and to listen, and then you make a decision: you’re the decision-maker. But if you come as a mediator, you listen to the problems of each party and receive their decision: the decision comes from the bottom [that is, agreed upon by the parties in conflict]. In the LCCNR training, you’re trained to be a leader or a mediator, depending on who you are. The MSS official then recommended that the World Bank help MSS trainees become LCCNR trainers, so they can train the community leaders with whom they regularly work. Youth Sector Trainees: Most youth trainees who were interviewed expressed a lack of confidence in themselves as leaders. They were also unsure whether would be seen or accepted as leaders, female youth trainees in particular. At the same time, the utility of the LCCNR training was clear to most. They viewed the training workshops as an opportunity to build their capacity to become leaders in the longer term and help them find a job in the near term, too. As a result, many expressed particular interest in getting a graduation certificate, as it could help certify them as trained leaders. It also just might help them land a job. In addition, since many youth are unemployed, their interest in receiving compensation of some sort in exchange for attending a training workshop was generally strong. As one youth leader explained, When we ask youth to participate in a training, their answers are always the same: is there any subsidy for transportation (that is, will they receive any money to attend the training)? Youth always complain about this. Most of our youth are unemployed, so what’s mostly on their mind is money. So if there’s a training, if there’s an incentive to attend, some money, then we can go and take part. A consistent research finding was that the training experience lent trainees confidence. As a female youth leader observed, “Before being trained by the World Bank, we didn’t have the self confidence to face people and address their conflicts. But after the training, it’s different. We feel stronger and have confidence.” It is a sentiment that many youth trainees stated. Again: boosting the confidence of young Timorese trainees was, given how often their authority is challenged if they try to get involved in leadership activities, a significant and positive program impact. NP Sector Trainees: Because the training experience was so short, and the evaluation research period took place while the second workshop was underway, it was not possible to gauge sector impact. However, there was a substantial minority of trainees from MSS (a total of six), and this was intentional. MSS officials sought to replicate what they viewed as the successful impact of LCCNR training of Dialogue Group members by doing the same thing for MSS officials engaged with NP activities. 22 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 The SIMSOC Simulation Observation of the SIMSOC simulation activity strongly suggested that participants were engaged with and enjoyed it throughout. SIMSOC is a simulation designed by William Gamson, which provides insight into the dynamics of social and political conflict, and illuminates a number of principles of leadership, communication, conflict resolution and joint decision-making. In brief, SIMSOC consists of a single society comprised of four regions with a very unequal distribution of resources. Those who live within SIMSOC must do everything that persons are required to do in the real world: the must subsist; they must secure employment; and they must decide how to allocate whatever resources they possess. The simulation provides trainees with an opportunity to try out the negotiation, communication and leadership skills they are learning from LCCNR. Trainee comments about their participation in SIMSOC were uniformly positive. VII. Conclusions LCCNR’s second program year mainly consisted of conducting a set of workshops for one set of trainees from each of three sectors. For the first two sets of trainees (the IDP and youth sectors), the level of attention and investment in each trainee was considerable: three workshops, a simulation exercise, coaching meetings (consisting of a kind of private tutoring that reviewed what trainees learned in workshops and may have applied afterwards) and a final graduation day. The NP trainees initially received two workshops, although several who attended the first training declined to attend the second, and some who attended the second were not present at the first. A third workshop (in November 2009) was added after field research for this report was completed. In addition, it should be noted that Timor-Leste’s Ministry of Economy and Development organized a session in October 2009, following the second workshop, to give LCCNR trainees from the first two NP workshops a chance to present what they had learned to their colleagues (LCCNR observed the presentation but did not help trainees prepare their presentation). This section will briefly review some final comments about LCCNR’s second year before turning to the program’s goals, objectives and hypotheses. A Challenging Context Timor-Leste presents a number of challenges for any capacity building activity. Drawing on field research over four field visits to the country over the past two years, here are some prominent contextual concerns: • A corps of older, tested and generally confident leaders who dominate a hierarchical system in which direct communication between leaders and subordinates is rare. 23 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 • A traditional approach to resolving conflicts that features meting out harsh penalties to the guilty party. The purpose is to create a kind of ‘demonstration effect’ to scare others from attempting similar offenses. • A cultural tendency to avoid sharing negative comments or critiques directly. In training contexts, concerns or complaints generally only surface afterwards (if at all). • A tendency for many to avoid answering questions directly (a process of “going sideways” with answers). This is done for a variety of reasons, including fear of humiliation, the legacy of harsh interrogations during the era of Indonesian occupation, and, possibly, a lack of motivation: if there’s nothing to be gained by sharing information, then a person may not do so. • Low levels of technical capacity in government bureaucracies and broad skepticism, of not cynicism, about the utility of attending training workshops. • A prodigious number of training programs that, most unfortunately, are uncoordinated and so invite many of the same Timorese, over and again, to successions of workshops. The impact of such overlap promises to undermine motivation to attend workshops and is undoubtedly one of the reasons why cynicism about training programs runs so high among Timorese officials. • Reports of widespread fears in the population: of out-of-school male youth, of being publicly exposed as having the ‘wrong’ answer or lacking information, and of leaders. • A highly alienated and large swath of the population that is made up of unemployed or underemployed, out-of-school youth. • Strikingly serious reports of disempowered women in Timorese society, and widespread domestic violence. • Suspicion, resentment and sometimes sharp divisions separating international advisors and their Timorese government counterparts. • A tradition among many Timorese (particularly those schooled in Timor-Leste during the Indonesian occupation) of memorizing material and lacking experience or confidence as creative thinkers. • A tendency for Timorese to withdraw to maintain risk averse strategies in the face of difficulties and disputes with powerful Timorese or foreigners. 24 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 LCCNR’s Training Program In its second program year, the LCCNR program trained 107 Timorese fairly extensively. They invested a high degree of expense, effort and time to enhance the leadership, communication and negotiation skills of their trainees. As a result, some of trainees came away with skills and tools that they appreciated, understood and successfully adapted to their lives and work. Others tended to memorize the information and then struggle to figure out how to adapt what they had learned. There were also problems of confidence for many, and especially for youth, it was difficult to be regarded as leaders when most adults considered them much too young for such a rare and esteemed position. The challenges were particularly significant for female youth trainees. As is quite common in most training programs, a minority of trainees appeared to, essentially, tune out the training program for significant periods or be almost entirely unable to absorb much information from their training experience. This group derived little lasting impact from their experience with LCCNR. The program emphasized providing quality training to a finite number of trainees. Indeed, the number of follow-up workshops and coaching sessions, in addition to the impressive SIMSOC simulation and the high degree of investment in monitoring and evaluation work, appears to have been well beyond all other training programs in Timor-Leste. On balance, emphasizing quality for a small number of trainees, given the prodigious contextual and educational constraints confronting most trainees, proved to be the right choice. Part of LCCNR’s evident success, across both of its program years, lay in its ability to adapt to changing circumstances. It twice demonstrated this program characteristic in its second year. For the IDP trainees, LCCNR adapted its follow-up workshops, as well as its coaching sessions, to directly address the enormous challenges that the trainees faced up on the mass return of IDPs to their former homes. For work in the National Priorities sector, the diagnostic exercise proved to be a turning point, as it increased the interest and credibility of the program in the eyes of Timorese officials. In addition, LCCNR endeavored to support the expansion of the training and management responsibilities of its Timorese staff in the process of carrying out its program. Finally, and impressively, LCCNR produced and distributed a manual in Tetun for all trainees. The manual responded to trainee requests for one. It contains the different materials and tools that LCCNR taught in its workshop sessions. Overall, given the constraints and challenges that LCCNR faced in training Timorese, the program proved very successful in its training work. Its adaptability to changing contexts, its decision to produce and distribute a training manual, and its success at enhancing the capacities of Timorese officials with LCCNR, were all noteworthy as well as direct contributors to its overall success. As with any training program, some decisions that were made had promising and perhaps unfortunate consequences. There were repeated indications that, for many if not most trainees, the training pedagogy was too detailed and complex. It appears likely that more trainees would have had a better chance of grasping and using LCCNR’s skills and tools if there were fewer of them. Simplifying the syllabus and highlighting a smaller number 25 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 of key points, concepts and tools promises to result in a higher degree of retention and application of what trainees learned. That said, this is a judgment call. Cutting back on content might undercut a breadth in the program that some might deem necessary. Yet the judgment of this evaluator is that, given the context within which LCCNR operates, the syllabus and pedagogical approach should be streamlined and simplified. It became clear which concepts and issues proved particularly important to trainees. They would include: • The idea that there could be two parties with different and equally valid perspectives about the same issue (instead of there being a ‘right’ and ‘wrong’ party in a conflict) was a particularly powerful idea for many trainees. The process of doing this is of particular interest to many trainees. The interpreted this work as meeting separately with each party before bringing the two parties together. • Related to this is an underlying approach that runs to the heart of what LCCNR effectively aimed to do: to change the role of a leader from a dominant decider who works alone to a collaborator who helps parties in a conflict find a resolution together. This idea proved popular with many trainees. • The idea that LCCNR would create a way to approach problems in a way that was “systematic” was a theme that repeatedly arose during interviews. The interest in learning how to conduct analysis of a situation was a related, and also popular, theme. • The Four Quadrant tool. The NP experience in particular pointed to a useful next step for LCCNR, should it continue its work in Timor-Leste: to consider fashioning their training work to address two specific trainee concerns – how to negotiate with powerful (and intimidating) international advisors, and how to problem-solve and negotiate in bureaucratic situations that are hierarchical. While the difficulties involving Timorese government officials and international advisors have been detailed in this report, it is also true that, over and over again, the relevance of the training was raised with reference to the “top down” ways of government bureaucracies. Several of the NP sector trainees said that since their jobs were essentially to carry out orders, it was difficult to figure out how to apply LCCNR’s skills and tools at work. Addressing explicit challenges such as these promises to enhance the utility of the training program (should it continue). Close affiliation with the World Bank yielded both helpful and unhelpful results. The World Bank’s invitation letters certainly garnered a great deal of attention and almost certainly positively influenced some people to attend the workshops. Directing LCCNR to work in three sectors in a year significantly undercut the program’s ability to achieve sectoral impact. However, the World Bank’s decision to include Timorese government officials working on the National Priorities process proved to be a potential asset for the program and, depending on how the program and its findings are applied, for those engaged in the NP process as well. This is because the LCCNR program team, and this evaluator, were able to learn about the pressing challenges and difficulties that currently plague the NP process. Again, research findings gathered by this evaluator were so striking that extensive findings are being shared in Appendix I as well as in the form of a 26 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 discussion draft (Appendix II). They also significantly influenced this report’s recommendations, as will shortly be described. For this alone, the World Bank’s directive style is to be commended. On the other hand, close identification with the World Bank, and maintenance of an office on the World Bank grounds, led LCCNR to be widely known as the “World Bank Training” and generate a very limited identity of its own. Because of this, LCCNR was also thought of as a kind of World Bank affiliate, meaning that trainees felt they could approach LCCNR for help in getting loans from the World Bank. Such results will likely continue, should the program be extended, until literal and programmatic distance is cultivated between the program and the World Bank. Goals Goal #1. Reinforce leadership capacities across state and non-state actors in both formal and non-formal institutions within specific sectors. • Did LCCNR improve the effectiveness of East Timorese leaders? Definitely, but for a finite number of leaders. A second, related result is that the experience of being an LCCNR trainee cultivated confidence in young Timorese that they might, one day, become leaders themselves. Goal #2. To enhance communication skills and encourage information sharing within and amongst state institutions, civil society and the public in the hopes of improving policy outcomes. • Did LCCNR increase prospects for effective policy outcomes? It is difficult to determine, except for the achievements that it recorded among groups of trainees working, for example, in the Ministry of Social Services and in one (and perhaps more) of the Suco administrations. But whether the work of these trainees will result in improved policy outcomes is, again, fairly impossible to assess. Objectives Objective #1. Assist the state in establishing the systems and attitudes necessary to fostering better communications. • Did LCCNR enhance the responsiveness of state institutions? The answer would be yes, but to a limited degree. Certainly in the Suco in Dili where several LCCNR trainees use the program’s skills and tools effectively. And certainly among the 8 trainees working in the Dialogue Group at MSS. Hopefully, their work with NP officials will yield a similar result. 27 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 Objective #2. Support leaders from diverse groups in society as they seek to improve their ability to engage with each other in a spirit of cooperation and mutual understanding. • Did LCCNR facilitate the improvement of leader relations? In a limited number of cases, the answer would be yes. Again, the results at MSS and in a highly volatile Suco in Dili prove that even training a small number of people can yield a narrow amount of demonstrable and positive impact. Hypotheses Hypothesis #1. Increasing the frequency and quality of communication by Timorese leaders will improve the quality of leadership in Timor-Leste. • This is probably the case, given that the traditional model of leadership in Timor- Leste does not necessarily include effective communication. LCCNR is challenging this model through its emphasis on effective, inclusive communication and information-sharing. Hypothesis #2. Targeting trainings at specific sectors, where participants are working on the same issues, will enhance LCCNR’s ability to effect leadership positively in Timor-Leste. • The sector approach yielded very limited results. The assumption that a program can positively impact a sector by training a mere 35 people is not supported. However, smaller units of these (poorly defined) sectors, again such as the Dialogue Group at MSS and at least one Suco in Dili, yielded impressive results. Hypothesis #3. LCCNR Year #2’s emphasis on improving the communication skills of leaders in targeted sectors will help Timorese leaders build effective, enduring relationships with their constituents and across class, gender, ethnic, tribal, political, regional and other lines. • This result will lead to more effective and responsive Timorese institutions. • The best that can be said about this broad and difficult-to-measure hypothesis is that some trainees were very likely working in the directions of inclusive decision-making and building trust and good communication across many lines. That said, it will take a truly concerted effort to begin to eradicate discrimination against and the disempowered status of most Timorese women. 28 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 VIII. Recommendations While it is unclear, as of this writing, whether LCCNR will continue to operate in Timor- Leste, the following recommendations are offered. They are divided into two sections: one for general recommendations for the program. The second is a specific set that this evaluator strongly recommends should be used to address significant problems in the National Priorities process. General Recommendations 1. Revise and strengthen the current sectoral approach. Consider devoting a full program year to one sector, and use evaluation work to gauge sectoral impact. Training three groups from a single sector in one program year would almost certainly increase prospects for achieving sectoral impact – particularly if additional activities were included that linked the trainee groups together. 2. Transition LCCNR into a Timorese-led program, where most trainers are Timorese and the curriculum is increasingly Timor-focused. 3. Develop LCCNR as a multi-year program. 4. When relevant (such as with high level government officials), speak openly about and frankly about the fact that white, internationals are the trainers/experts and Timorese are the trainees. Open up the training format so that issues which are important to trainees can be explored (such as negotiating with internationals and powerful superiors), perhaps through role play. 5. Simplify the training jargon as much as possible, to work to ensure that the ideas can be effectively communicated in Tetun. 6. Simplify the syllabus, so that the most salient concepts and ideas for trainees are emphasized and, hopefully, mastered. 7. Address LCCNR’s troubling branding issue. Whether or not the World Bank decides to continue supporting LCCNR, the program should move its office out of the World Bank premises and develop a separate identity that features the provision of unbiased, technical expertise. The continuing reputation of LCCNR as the “World Bank Training” most certainly provides the program with a level of prominence that can be helpful. Yet the negative connotations of and reactions to LCCNR as a World Bank activity outweigh this benefit. 8. Maintain the current level of program monitoring and evaluation while incorporating information-sharing activities as an ongoing component of this work. 29 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 9. Make more use of evaluation findings and reports. Incorporate the sharing of information with interested government and non-government officials (including interested international government and non-government officials) as a component of LCCNR’s work. The program’s evaluation findings on leadership, training, sectoral work, IDPs, youth and many other concerns are clearly of interest to many Timorese and international officials. Producing regular short reports and briefings (in relevant languages) also provides the program with opportunities for demonstrating many of the core values it promotes, such as communication and sharing information. 10. Mainstream gender and class concerns in program materials and messages. Field evaluation work continues to find persistent signs of significant power differences according to class and gender. They indirectly impact the program because a minimal awareness or denial of their influence promises to hamper the effectiveness of leaders. Specific Recommendations for Addressing the NP Process LCCNR has already demonstrated its capacity to train Timorese effectively to enhance their leadership skills, and recognize and adapt to new situations and address emerging concerns. It is thus strongly recommended that LCCNR be used to address two major problems that plague the NP process (and, indeed, government operations in Timor- Leste). Of course, the details should be worked out with the key institutions involved, particularly government officials of Timor-Leste, in addition to the primary donor institutions who are providing international advisors to Timor-Leste’s national government. What follows, then, is a framework for recommended next steps:10 1. Direct LCCNR at two specific problems currently confronting the National Priorities process: (1) International-Timorese relations; and (2) operating within a strictly hierarchical environment in government institutions. Broad funding for such an operation would be useful, given the many stakeholders involved and the pressing need to promote a nonpartisan environment of sharing and learning. 2. Devise a curriculum and overall approach to address these problems by: (1) Facilitating a series of guided, frank discussions with key party members on how to address the problems (drawing from Recommendation #6 of Enhancing Post-War Government Capacity; Appendix II); (2) Using evaluation material to regularly produce and disseminate information about the problems and how they are being addressed; and (3) Using training 10 Recommendations on how to address significant problems in the NP process can be found at the end of this report’s Appendix II, on page 42. 30 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 sessions to develop customized, specific skills for NP stakeholders to address these problems. Clearly, the skills of negotiation and communication – as highlighted by several NP trainees – need to be highlighted in training workshops. But in addition, the presence of white/international trainers and Timorese trainees should be used as an asset to conduct role plays on how to enhance international-Timorese relations through the effective application of communication and negotiation skills. 3. Use LCCNR to help NP Stakeholders develop and implement steps to address the two problems mentioned in Recommendation #1 (above). Consider using LCCNR to address all seven recommendations at the end of Enhancing Post- War Government Capacity (Appendix II), should those recommendations be addressed. 31 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 Appendix I: Detailed Findings on Sectoral Context, Particularly National Priorities Internally Displaced Persons Nearly all of the IDPs who were housed in camps following the 2006 crisis have returned to their homes. Several hundred remain in transitional housing of some sort. They are mainly families or individuals who, for various reasons, appear to have nowhere to go. They cannot return to their former homes, perhaps because there is no longer any room for them. Another reason is that some couples are comprised of one spouse from the West and one from the East. Before the crisis, such a union presented little or no problem to the couple. But with a surge of people identifying themselves as either Easterners or Westerners in the wake of the clashes in 2006, the identities can now invite trouble. An ‘East-West’ couple may not know where they can safely reside. During the 2006 clashes, many of the neighborhoods to which the IDPs have returned were filled with burning homes and rioting between neighbors, armed groups and groups of threatening, angry youth. Most of those who contributed to the violence in 2006 still roam the streets of those neighborhoods with impunity. These are not comfortable circumstances. Resentments, bitterness and fear reportedly affect many of the residents of neighborhoods where the 2006 crisis took place. As a high-level international agency official noted about the IDP returnee communities, “At water points, in overcrowded schools and in other ways, we certainly don’t see what the government officials are saying: that there are no problems” in communities where the IDPs have returned. In fact, the official is concerned that the Eastern and Western identities that spurred the 2006 riots “have not died and gone away.” On the contrary, the official said that new community conflicts “may be re-expressed as East-West conflicts.” In the current hothouse political environment created by intense competition between political parties, the official’s concern that community conflicts could reemerge as politicized combat between Easterners and Westerners raises an issue of considerable concern. The difficulties involving IDPs and what are known as ‘residents’ (that is, those from the same neighborhoods as IDPs who did not become IDPs) continue to spill over into the Ministry of Social Services (MSS), where several of the IDP sector trainees work. There regularly are ‘dialogues’ involving IDPs and residents over ongoing conflicts and difficulties between members of the two groups. One of the most bitterly divided Sucos in Dili has found that conflict and violence continues to run so high that the Suco leaders resorted to a centuries-old, pre-colonial method for, essentially, outlawing conflict. The leaders held a community-wide ceremony for what is known as the Ai Lulik [Holy Wood]. The Ai Lulik was installed in front of the Suco office building and is, in the words of one of the Suco leaders, A symbol for unity and peace – and no violence. Inside the symbol [of the Ai Lulik], there’s a policy: no violence, no drugs, and no theft. In short, it addresses everything you cannot do. Before the Catholic religion [arrived in East Timor], our ancestors agreed not to have violence, only peace. No conflict. This is the background [for our installing of Ai Lulik]. 32 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 Fines against anyone who commits any sort of theft or communal violence are heavy (the Ai Lulik does not seem to apply to domestic violence). As the official explained, after Suco community members roasted and ate a buffalo together at the Ai Lulik ceremony, “We agreed that if someone disobeys, they have to pay to make the Ai Lulik holy again” by providing another buffalo and pay for an Ai Lulik ceremony themselves. Such a ceremony is expensive. The official considered the decision to bring back the Ai Lulik tradition was essentially a matter of desperation. The rationale for bringing back the Ai Lulik was detailed by one of the Suco leaders who employed it: We have to look to our past experiences because of so many conflicts here. We have tried to use government laws and policies, the reconciliation process, the reintegration process, the community police and UNPOL [United Nations Police]. But sometimes none of these measures are effective. So we use an old tradition [the Ai Lulik] as our last option to finally end the violence. The leaders in this particular Suco, in other words, have essentially found that communal violence following the return of IDPs to their homes continues to be high, if not uncontrollable. Reliance on laws and the courts has not proven effective. As the Suco leader explained, If someone is involved with violence, the courts may put him into prison for two years. But after he gets out, he has the same attitude. He’s still violent, still causing conflict. For some of the urban poor, desperation is so great that entering prison is a goal to achieve, not something to avoid. If someone is unemployed, the leader observed, He may carry out a violent crime so he can go to prison. Once he’s in prison, human rights officials will insist that the prisoners are treated as human beings. Some people even leave prison fat. Conditions in prisons may be much better for a desperate person than in Dili. The leader explained that Some perpetrators in our community have had good experiences while in prison. They say that they don’t have to pay for anything, they get free food and they don’t have to pay to go to the toilet. You don’t need money to sleep. As a result of the benefits that prisons are thought to provide to criminals, they are popularly known in Dili as “Free Hotels.” 33 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 Youth The view that the main perpetrators of crime, and most of those who seek to enter Free Hotels (that is, those who commit crimes in order to go to prison), are poor male youth appears to be widespread in East Timor. An extended interview with 10 unemployed male and 3 female youth in a Dili neighborhood revealed two themes that supported findings from earlier field evaluation research visits: • The youth equated probably the most prominent form of youth leadership with violence. They unanimously considered leaders of Martial Arts Groups (MAGs) as major youth leaders. The reason that was provided, essentially, was that such male youth had significant influence over others: both lower-level members of their MAG, whom they control, as well as most community members, who fear them. As one male youth explained, “We trust [Martial Arts Group leaders]. They are famous, people are afraid of them, and they are powerful. If someone has killed people, other people will look to him as a powerful man. If you kill someone, it shows that you have a strong determination to do something. Most people aren’t brave enough to kill someone.” “Some leaders work hard to become famous,” another male youth explained. “But others kill to become famous.” The other main kind of youth leader that was named was the Suco level youth leader, a political position. • Nearly all youth leaders are male. A female youth stated that “There are only male youth leaders, never female youth leaders.” Another added that “Female youth can’t be leaders because they fear men. They can be beat if they try to become a leader.” While the reporting of such views cannot be said to be broadly representative of out-of- school youth views (due to time constraints on the field evaluation research), the connection between some youth leadership and violence is a theme that has regularly surfaced in interviews with youth during evaluation visits to Timor-Leste. And certainly there has been limited evidence of female youth leaders of any prominence. For those who are leaders, they have significant constraints on what they can do. As a female leader at the Suco level explained, “Men fight at night. When they do, the male youth leader gets involved. Also, I’m not allowed to go out at night. Females cannot go out at night.” An interview with an international official with extensive experience working with out- of-school youth in Timor-Leste elicited two additional themes that resonated with earlier evaluation research work for this program. One was the sense of insecurity and fear that reportedly pervade the lives of many ordinary Timorese. There is a general sense, the official related, that “Instead of being positive” about the future, “many people expect that something bad will happen.” As a result, some adults are afraid of idle male youth who congregate in public spaces or of what might result from political party meetings. 34 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 The National Priorities Process The NP is a process through which the government of Timor-Leste (in coordination with international donor partners) identifies priorities and sets out ways to achieve them. Interviews were conducted with Timorese government officials who were invited to the LCCNR training workshops, those who attended the workshops, and Timorese and international officials who are directly involved in the NP process. While the impact of the LCCNR workshops on trainees will be examined in the next section, analysis of findings concerning the NP process itself will be considered here. The analysis revealed clear evidence of noteworthy short-term success. A widely shared view among those interviewed was that the NP’s short-term achievements are substantial. The NP is operational, it has developed a system of regular reporting, and it provides a highly useful framework for substantive discussions involving donor agency and Timorese government officials on collaboration. At the same time, the August-September field evaluation research revealed alarming problems that could threaten prospects for longer-term success. Emotions ran remarkably high when Timorese and international officials discussed the NP. Everyone displayed some degree of dismay and discouragement about it. Most expressed high levels of frustration about difficulties concerning the NP process. By far the most prominent reported source of problems concerned the nature of relationships and interactions between Timorese government and international agency officials. Most of what officials shared about the NP during interviews can be summed up in following phrase: relations between Timorese and internationals are generally dysfunctional. Reporting findings that are mostly negative (or mostly positive) requires context. The model for enhancing the capacity of post-war governments by placing international technical advisors into the government structure while training national civil servants is widespread. It is not at all unusual for personal frustrations to surface during discussions about a process as challenging as the NP, particularly when it is taking place in a nation as new, and with so many compelling challenges, as Timor-Leste. Timorese and international officials alike shared the view that the development of Timor-Leste’s governance capacity remains a work in progress. Nonetheless, the consistency of forceful expressions of frustration during nearly every interview about the NP was striking. Indeed, the gathered concerns were so significant that I submitted a short discussion paper about the current situation soon after the field research period ended (see Appendix II). Drawing from the interview data on the NP for this report, it appears that many Timorese government officials are becoming alienated from the way in which the NP process is evolving while many international officials are already exasperated by it. The noticeable separation between many international agency officials and their Timorese government counterparts is illustrated by descriptions of the problem by, respectively, an international and a Timorese official who are both involved in the NP process. The international official detailed the core of the problem with the NP in the following way: 35 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 Box 2: An International Official on the National Priorities Process Timor-Leste is so ungrateful about the assistance it gets from international advisors. Development partners [in Timor-Leste] are at the stage where they need more buy-in from the Timorese, who cannot hide behind the lack-of-capacity curtain. The Timorese blame things on foreigners later, whether it’s their fault or not. It’s a cultural issue more than anything. The Timorese don’t see us [international officials working on the NP process] as being here to help. There’s a huge issue with the income gap [between the salaries of international and Timorese officials]. But at the same time, the Timorese are not taking advantage of the foreigners’ presence while it’s in Timor- Leste, and extracting as much as they can – in terms of the skills that they have [to improve]. Most foreigners are here to help. As foreigners, we are here for a short time and we have to report on something that we’re assigned to do. So we keep trying to build Timorese capacity, or work together, as much as possible. But you can’t just and wait forever. At the end of the day, certain things are not completed [by Timorese government officials]. You can’t just sit around and wait to decide that, yes, maybe the Timorese should decide to participate more [in the NP process]. The biggest lesson on my mind lately is that you can only help those who want to be helped. The Timorese government official’s perspective pushes back about how international officials generally regard their Timorese counterparts: Box 3: A Timorese Official on the National Priorities Process Sometimes I’m the only Timorese in the Ministry meetings. The work here is often very chaotic. I appreciate the quality of the international advisors, but the missing link is that there’s no transfer of knowledge [to Timorese government officials]. The Ministers ask the international advisors to do Ministry functions. This should be the work of the national staff. Because of high pressures to do certain work, most advisors are tempted to do the Ministry functions themselves. In terms of the outcomes, the results look good. The international advisors are doing the hard [bureaucratic] work, while they’re also doing policy work. But in two to five years, there will be no national staff that do their own jobs because their work is being overtaken by the international advisors. In the long term, it’s not good for the international advisors to do everything and for the Ministers to rely on them. Most Timorese believe that the international advisors should do their work because the international advisors are paid so much more. The general feeling of the international advisors is that Timorese don’t have 36 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 the capacity to do their jobs. But they don’t give the Timorese the strategies and skills to do their work. The Timorese don’t have capacity in all things, but the capacity is there. We don’t have someone like this: someone who can build people’s capacity to help them achieve their targets. International advisors don’t give space to Timorese to build themselves. One clear shared theme in the above statements is that international advisors are doing much of the work of Timorese bureaucrats. Both are resentful about this, if for different reasons. A second shared theme is that international advisors are providing limited capacity building, mentoring or training of Timorese government officials. A third shared theme is that international advisors are under pressure to produce results – even if that means doing the work of their Timorese counterparts. All three of these themes were supported by interview data with other international and Timorese government officials. Additional prominent themes arising from the collected interview data about the NP process are: • No one who was interviewed disputed the fact that Timorese government officials are sorely in need of significant if not dramatic improvements in their technical capacity. At the same time, it was widely believed that few – but not all – international officials were effective at building the technical capacity of Timorese officials. • Many international officials feel pressure regarding which part of their job they should concentrate on: building the capacity of Timorese counterparts or carrying out pressing and extensive government work that is before them. The reported tendency is for most officials to get government work done: work deadlines are often pressing, and capacity building can be a slow, unstructured and frustrating process. They often appear to see no choice but to carry out important government work. If they did not, an international official surmised, “The Ministries would grind to a halt.” International advisors are reportedly selected largely on their technical expertise and background (evaluating “soft skills” are often more difficult to determine during a selection process). While this may not be at all surprising, there were no reports of regular supervision, direction or training provided to international advisors concerning how they can effectively build the technical capacity of Timorese counterparts. • The lack of a sense of ownership of the NP process by Timorese civil servants is pronounced. There were widespread reports that few or no Timorese attend most NP meetings in Ministries and other government offices, and that many if not most Timorese officials tasked with NP responsibilities are reluctant to contribute. The bitterness that many expressed towards international officials was palpable. As one high level Timorese government official stated, “In many meetings, I tell internationals that ‘You fail because you don’t want to transfer skills to the 37 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 Timorese.’ Some internationals don’t want to transfer skills; they prefer to do the work themselves. So when they leave, they don’t leave behind anything, and we lose.” • Most NP meetings reportedly take place in English, the lingua franca of international officials, and not Tetun, the lingua franca of Timorese government officials. Relatively few Timorese government officials speak English, and of them, few reportedly speak English with confidence. As a result, Timorese officials may decline to attend the NP meetings partly to avoid potential embarrassment arising from a limited facility in English. There may be other reasons: not attending may be aimed at communicating displeasure at the perceived dominance of international advisors over important government affairs. • A great many Timorese civil servants have effectively disengaged or withdrawn from the NP process. It appears that most of this resistance is indirect: the officials may perform few or perhaps none of their responsibilities for the NP (in addition to attending few or no NP meetings). Unhappy relationships appear to have regularly arisen that involve international officials pressuring Timorese officials to submit their NP work responsibilities on time. When they do not, some international officials do the work instead. This situation is a regular source of conflict, resentment and exasperation between international and Timorese officials. Contributing to this tense situation was a strong impression that sensitivities about internationals generally runs high. The following comment from an experienced trainer for LCCNR (who is an international, as is this evaluator) is worth noting. The trainer found that there was “extraordinary sensitivity about internationals in Timor-Leste. In my experience, it’s unique.” • There is a colossal disparity in compensation between international and Timorese officials. International officials are paid dramatically more than their Timorese counterparts.11 While the disparity may be deemed unavoidable by many, it does not seem to have been directly addressed, even though it is clearly a cause of deep resentment among many Timorese officials. In one unit, it was expressed as a kind of joke: international advisors who tried to speak Tetun were called “Tetun 7,000” (that is, the advisor earning $7,000 dollars a month was trying to speak their language). The severe disparity in compensation also provides an available rationale for Timorese seeking to disengage from the NP process. • Language and technical challenges are persistent. They negatively impact the NP process in a host of ways. Some of those who were interviewed expressed concern that the frameworks, matrices and other technical tools are foreign imports, understood and controlled by international officials, and may be subject to change (largely depending on whether incoming international officials seek to do so). 11 This is a prominent issue in the texts of both officials highlighted in Boxes #1 and #2 above. 38 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 In addition, the technical tools typically rely on some English words that may not translate easily into Tetun. Some Timorese government officials are reportedly unsure about the differences between concepts such as objectives, goals, outputs and outcomes. “A lot of staff don’t know English,” a Timorese official explained. “Some don’t know the difference between a goal and an objective. We only need one of them: pick one and then let’s get the indicators.” Accordingly, they may be unsure how to fill in, for example, an NP matrix. Asking for help or clarification from an international official in such cases may not occur. • Thrust into a high-pressure environment featuring a foreign language, new concepts and techniques, and tight deadlines, many Timorese civil servants who were interviewed related significant concerns about their confidence and ability to succeed. In contrast, some higher level Timorese officials and some international officials suggested that the bureaucratic work ethic among Timorese civil servants was a significant problem. These conflicting perspectives did not seem to have been regularly or effectively addressed.12 • Extensive training efforts for Timorese government officials to date are widely thought to have been, in general, ineffectual. Issues such as the venue of training courses (abroad is preferred; whether domestic trainings should be held in or outside of Dili is debated), the degree to which the content and pedagogy of training courses are coordinated, and the relevance and timing of training courses, all surfaced in discussions with international and Timorese officials. Despite broad agreement about the need for effective technical training for Timorese government officials, cynicism about the impact of most training programs proved commonplace. The gathered sense from the findings for this field evaluation research effort are that short-term achievements in the development of the National Priorities process have come at a considerable cost. The evidence strongly suggests that the push to produce has yielded a mixture of exasperation among many international officials and resentment, bitterness and even alienation among many Timorese officials. The result has been mutual suspicion, distrust and social separation between influential, higher level Timorese officials and the international advisors with whom they work, and the expanse of Timorese civil servants in the government bureaucracy. Communication and trust between most international and Timorese officials are weak and appear to be almost entirely unexplored as challenges to the NP process. As one Timorese official concluded, “International advisors think that they’re introducing an ideal world” to Timorese government officials. Yet in the official’s view, the new concepts and ideas that the advisors are introducing “are not adaptable to the circumstances here. The advisors are not able to come down to our levels.” 12 Higher level Timorese government officials, such as Ministers, are considered politicians, given that they are politically appointed to their positions. Top civil servants, such as Director Generals, are also seen as higher level officials. The bulk of civil servants, on the other hand, are not. Instead, they are widely considered bureaucratic functionaries with diverse ranges of backgrounds and skill sets. 39 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 Appendix II: Discussion Paper about the National Priorities Process Discussion Paper No. 2* ENHANCING POST-WAR GOVERNMENT CAPACITY The National Priorities Process in Timor-Leste by Dr. Marc Sommers13 Program Evaluation Expert, CMPartners (Cambridge, MA, U.S.A.) & Leadership and Communication Capacity for National Renewal Program (LCCNR) in Timor- Leste September 2009 The model for enhancing the capacity of post-war governments by placing international technical advisors into the government structure while training national civil servants is sensible and, in most cases, probably appropriate. Recent research in Timor-Leste (August-September 2009) strongly suggests that the application of this model there has generated functional as well as startlingly dysfunctional results. The evident social distance and dysfunctions that mark many of the professional relationships between international and national personnel (which largely center on technical advice and decision-making responsibilities) could create significant problems in the medium and longer term if underlying issues are not addressed. Carrying out final evaluation research work to assess the impact of LCCNR’s second program year in Timor-Leste included field interviews with international and Timorese officials who are directly involved in the nation’s National Priorities (NP) process. It incorporated research on the context of LCCNR’s leadership training activities for some Timorese government officials who are part of the NP process. This short discussion paper responds to a request to share findings, analysis and suggestions on the NP. It is based on interviews with 25 Timorese government and international officials who contribute to it. Findings: The NP is a process through which the government of Timor-Leste (in coordination with international donor partners) identifies priorities and sets out ways to achieve them. A widely shared view among those interviewed was that it has produced substantial achievements in the short term. The NP is operational, it has developed a system of regular reporting, and it provides a framework for substantive discussions involving donor agency and Timorese government officials on collaboration. * This paper is meant to stimulate discussion It draws from findings and analysis which the author carried out. It does not, in any way, represent the views of the LCCNR team, the World Bank or CMPartners, LLC. A discussion of the National Priorities process will be incorporated into the author’s forthcoming final evaluation report on LCCNR’s second program year. 13 The author is a Jennings Randolph Senior Fellow at the United States Institute of Peace and an Associate Research Professor of Humanitarian Studies at the Fletcher School, Tufts University. 40 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 The findings also revealed a broad sense among interviewees that the process is troubled and is generating resentment and even alienation among Timorese government officials. Three elements of this dysfunction were widely reported: 1. Interview data with both international and Timorese officials strongly supported the common view that most Timorese officials have significant technical capacity deficits while most international advisors are ineffective at building capacity. 2. In a country where few speak English, it is nonetheless the primary language used in high-level NP meetings. In all reported cases, international advisors (among those referred to as “internationals”) who have been assigned to government ministries or secretariats mainly attend these meetings. Many meetings take place where few or no Timorese officials are present. Timorese officials related that many government officials refuse to attend these meetings to avoid embarrassment due to their limited ability in English and the frequent dominance of discussions by international officials.14 3. The central instruments of the NP process – the NP matrix and monthly and quarterly reports – regularly create situations where international officials lead the process and, directly or indirectly, pressure Timorese to follow through. International advisors were widely reported to be carrying out much of the planning and reporting work that Timorese government officials are expected to perform. This result clearly frustrates international and Timorese officials alike. Many Timorese officials explained the dominance of international officials over the NP process in the following way: international advisors should do more work because they are paid dramatically more than Timorese government officials. While resentments clearly run strong about this issue, a growing sense of alienation from many international officials and the NP process was also evident. There was a pervasive sense among Timorese officials that the NP process is mainly for “big” Timorese politicians and international donor agencies (generally known as “donor partners”). The perception that politicians and foreigners dominate the NP process also appears to have strengthened a view among some Timorese that a hierarchical relationship separates influential international officials and powerful Timorese from nearly all Timorese, even government civil servants of high rank. Analysis: Significant changes in the functioning of the NP process appear to be necessary because communication and trust between most international and Timorese officials are weak and unexplored. Reliance on a foreign language and imported frameworks, instruments and methods, together with strong pressure to deliver on tight deadlines, appears to have accelerated already deep senses of resentment, technical inadequacy and social separation among many Timorese, as well as profound exasperation among internationals. In addition, the willingness of many Timorese to withdraw from activities and ‘let’ internationals perform them runs the risk of further strengthening the truly damaging stereotype that most internationals have ability and initiative while most Timorese do not. This situation promises to impede efforts to build capacity and expand management responsibility to the Timorese. The evaluation research indicates that positive short-term results may be achieved only at the cost of exacerbating distrust and distance between internationals (as well as powerful Timorese) and most Timorese civil servants. In addition, capacity-building efforts featuring internationals may prove counterproductive if they reinforce social differences that already plague the NP process and international-Timorese relations generally: the predominance of highly paid international experts over Timorese with limited technical expertise. 14 In some cases, a Timorese official related, the NP representative for government departments and institutions is determined not by their position but by how well they speak English. 41 LCCNR Training Program Final Program Evaluation, Year #2 – December 2009 Suggested Next Steps: The development of a Timorese government process that seems foreign to most Timorese civil servants calls for frank and open discussions involving international and Timorese officials on existing problems and needed reforms. Responsibility for the difficulties that have arisen must be shared. The roles, influence and limits of international advisors and agencies in Timorese government functions and policies should be examined. Given the depth of frustration and resentment that has arisen, the discussions will no doubt be difficult, uncomfortable and essential. If they are managed productively and fairly (not doing so could make the current situation more difficult), they promise to replace current destructive tendencies and attitudes with far more productive ones. The outcomes of such discussions on improving ways to build Timorese capacity and feature Timorese participation may call for substantial revisions to the NP process. The following ideas draw directly from findings arising from field evaluation interviews and are submitted for consideration: 1. Seek out and highlight the concerns and priorities of Timorese civil servants with responsibility for NP functions. 2. Develop a viable framework for NP timeframes and products that does not feature the contributions of international officials. 3. Feature Tetun (or Tetun and English) in National Priorities meetings and documents. 4. Simplify or remove complex matrices, spreadsheets and frameworks. 5. Highlight concepts and vocabulary that translate into Tetun.15 6. Find out which international advisors are effective at providing advice. Learn from them and, especially, the Timorese officials with whom they work. Find out precisely why Timorese civil servants may decline or resist new work responsibilities and learning about technical skills. Use this information to: • Reform standards and techniques for international advisor ‘soft skills.’ Retrain or, perhaps, replace international advisors who cannot effectively transfer their expertise to Timorese. • Revise priorities and methods for training civil servants. Institute training evaluations that accurately highlight retention and relevance concerns and recommend revisions for enhancing effectiveness. Conclude training efforts that remain ineffective. 7. Improve the alignment of the NP process with other central government functions. 16 15 Some interviewees noted that differences between objectives, goals, outputs, outcomes, etc. are too often unclear to many Timorese officials. They might be replaced by, at most, two categories, such as expected and achieved results. Final decisions on such concepts, however, must be made by Timorese and according to translations into Tetun. 16 Such as budgeting, which was a source of considerable discussion and frustration during some interviews. 42