4Mj- Consultative Group on lqpl International Agricultural Research To Iy lqqq The Impact of the International Agricultural Research Centers: Measurement, Quantification, and Interpretation M. P. Collinson and E. Tollens ISSUES IN AGRICULTURE 6 1M1p 1y 1p 14V Msp-Msk sp "Issues in Agriculture" is an evolving series of booklets on topics connected with agricultural research and devel- opment. The series is published by the Secretariat of the Consultative Group on International Agricultural Research (CGIAR) as a contribution to informed discussion on issues that affect agriculture. The opinions ex- pressed in this series are those of the authors and do not necessarily reflect a consensus of views within the CGIAR system. Published by the Consultative Group on International Agricultural Research, CGIAR Secretariat, 1818 H St., N.W., 3 Washington, D.C., 20433, United States. (1-202) 473-8951; Fax (1-202) 473-8110, July 1994. The Impact of the International Agricultural Research Centers: Measurement, Q3uantification, and Interpretation M. P. Collinson and E. Tollens ISSUES IN AGRICULTURE, NO. 6 ConsultativeGrl... , A.g.r.i t .....R e.......................R.es.er. ,9 *ZS000gg00|0|00 s- B B',.,.'., ............... .. .... . ..............,.SEEg 'B :: ......'" ss osf0 ...... =D . .. public and ivteetordnor......... .. .s p....prts a. neto r GroupS B Svo' ::tb:ihe in 1971 BSB SSSSSSS:2 ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ . ..... . . ...... IkJ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~. g. . . . . . . . . . . - . . . . . . . . . . . . . . . . . . Th Wol Bankg . te Ioo an AgriclueOgn tio ol,e Nfa..cff.Xtin - - -, a. tg -n.te ars, iE (UNDP) gEare cspnor f h CGAR Th ifma fte ru sa seio ofica of the ~ which piovids teOIARsysemigaGB wit 'a' Serearatn.alingo 5C The: OGIA:R is ass::ise byg:g af Tiei & 9- A= iugittee,5 withggggiii,i a_ Secretariatg atF g-A B B ' .=GWeW 'aa ,,DsadCaaaaeth edn .,W 4o~ cBut:: :::el by>a5f.,.,<: seRvral,,5R,5>>.f:A5 7,.gg --Euv , e contis. Dvlig N conymmeso h GA r Ch=-- , ;>ga::R-azl,sg hI4iafaBX'Bg hi4nesa.R MexicoR Nigeri. the Contents Page Acronyms and Abbreviations Used in This Paper ........ iii Introduction ............. ............................... 1 History of Impact Assessment in the CGIAR ................ 1 The Research and Development Process and IARCs .....3 The Development Process ...................................... 4 Implications for CGIAR Impact ................ .............. 6 A Conceptual Framework for Impact Assessment ........7 The Research Cycle and Feedback ............ ............. 7 Institutional Impact ............................................ 10 Scientific Impact ............................................ 10 Impact Independent of Farmer Decision Making.. 11 Impact in Farmers' Fields and Beyond: Levels and Methods of Assessment .............1............. 11 Evaluation on a Farming System Level ........ ........ 12 Evaluation on a Global Level ............................... 14 Economic Models ........................... : 16 Challenges and Strategies for IARCs .......................... 17 Planning .............................. 18 Donor Needs .............................. 19 Conclusion .............................. 21 References .............................. 21 i e.>... .. .::: 1 ... Acronyms and Abbreviations Used in This Paper CGIAR Consultative Group on International Agricultural Research CIAT Centro Internacional de Agricultura Tropical CIIFAD Cornell International Institute for Food, Agriculture, and Development CIMMYT Centro Internacional de Mejoramiento de Maiz y Trigo CIP Centro Internacional de la Papa IARC International Agricultural Research Center ICARDA International Center for Agricultural Research in the Dry Areas ICLARM International Center for Living Aquatic Resources Management ICRISAT International Crops Research Institute for the Semi-Arid Tropics IFPRI International Food Policy Research Institute IIMI International Irrigation Management Institute IITA International Institute of Tropical Agriculture ILRAD International Laboratory for Research on Animal Diseases IRRI International Rice Research Institute ISNAR International Service for National Agricultural Research NARS National Agricultural Research Systems R & D Research and Development TAC Technical Advisory Committee (of the CGIAR) USAID United States Agency for International Development The Impact of the International Agricultural Research Centers: Measurement, Quantification, and Interpretation* M. P. Collinson' and E. Tollens2 Introduction The mission of the Consultative Group on Interna- tional Agricultural Research (CGIAR), with its seven- teen international agricultural research centers, is to improve the welfare of poor people in developing coun- tries in ways that also improve the future productive capacity of their natural resources while protecting the quality of our wider environment. Impact studies help us to understand how technology influences agricul- tural production and the welfare of agricultural produc- ers and consumers. This information, in turn, can be used to improve the efficiency of resource allocation for research at the international agricultural research cen- ters (IARCs). Many practical obstacles must be over- come, however, before such studies can fulfill these roles. This is an abbreviated version of a paper of the same title by the authors which is forthcoming in Experimental Agricul- ture (1994) Vol. 30. Experimental Agriculture is published by Cambridge University Press. We are grateful for permis- sion to reproduce significant parts of that article. CGIAR Secretariat, The World Bank, Washington, D.C. 2 Department of Agricultural Economics, Catholic University of Leuven, Leuven, Belgium. Impact studies have another benefit. Recent re- ductions in funding for international agricultural re- search have created new pressures on the centers to demonstrate the impact of their results. Evidence of success, provided by impact studies, helps donor rep- resentatives defend CGIAR funding in their domestic budget process. In a time of recession, when burgeon- ing environmental problems and the breakup of the Soviet Union bring new demands on aid, donors need stronger and clearer evidence of the value of their investments in the CGIAR. History of Impact Assessment in the CGIAR In 1979, eight years after its founding, the CGIAR sponsored a review of impact assessment methods and results (Scobie 1979). Scobie found that the high-yield varieties introduced in the mid- 1960s benefited mainly low-income consumer groups. He also concluded, however, that they were not an effective means to redistribute incomes among rural groups in which productive assets are not equitably distributed. He further concluded that international investments in agricultural research could be expanded significantly and maintain an attractive economic rate of return. The literature on impact assessments that has accumu- lated since then supports his conclusion (Evenson 1992). Before 1985, impact assessments in the CGIAR system were dominated by studies on the short-strawed rice and wheats developed by the International Rice Research Institute (IRRI) and Centro Internacional de Mejoramiento de Maiz y Trigo (CIMMYT), part of the vast body of literature on the 'green revolution." Using national-level statistics, the United States Agency for International Development (USAID) documented the 2 development and global spread of high-yield varieties of BE^G': .ffi..6.M.^6 .X.. .B..sS.BB .. .a.>...........M .. .... .... ..t . ... .... ....iiE . ........ rice and wheat. Such publications began in 1969 with data from the 1965-66 crop year and continued through 1986 (Dalrymple 1986a, 1986b). These USAID-sup- ported studies were later extended to hybrid maize (Timothy, Harvey, and Dowswell 1988). CIMMYT has recently updated the data for maize and wheat (CIMMYT 1992), and updated figures for rice and wheat show the extensive impact of the short-strawed materials. Today the developing countries produce some 460 million tons of paddy rice annually from more than 140 million hectares, two-thirds (67 percent) of which is planted with seeds based on IRRI's semidwarf materi- als. Asia produces 91 percent of this total, and rice provides between 35 and 60 percent of household calories for its 2.7 billion people. Similarly, the devel- oping countries grow some 226 million tons of wheat from 100 million hectares, the seed for more than 60 percent of which is based on CIMMYT semidwarf mate- rial. The benefits of such advances in research continue to multiply long after the initial breakthrough. During the two decades of the 1970s and 1980s, the rice yield in eleven green revolution countries in Asia increased by 63 percent, from 2.03 tons per hectare at the begin- ning of the 1970s to 3.31 tons per hectare by the end of the 1980s. In the 1980s, wheat yields in developing countries increased by 37 percent, from an average of 1.64 tons per hectare to 2.24 tons per hectare. Four dimensions of the diffusion process for research and technology sustain the flow of benefits: * Farmers using the new technology get higher yields year after year. i Further adaptations continue to raise yield ceilings. * Adaptations extend the technology to farmers growing the crop under different soil and water conditions. 3 ... . . : B. . . . . . . . Adaptations extend the crop to areas previously unsuited to it, creating a new cropping opportu- nity for farmers working there. Studies by IRRI have shown that poor urban and rural consumers have benefited from the reduction in the real price of rice caused by higher production. Benefits have spread beyond the irrigated areas. As a result, labor demand in new areas has brought immi- gration of labor, thereby helping to equalize wage rates across environments (David and Otsuka 1991). During the period 1984-86, the CGIAR donors funded a major study of IARC impact (Anderson, Herdt, and Scobie 1988). It was supported by twenty-six monographs, including case studies (mostly qualita- tive) of IARC impacts on individual countries, three regional studies, and other studies on topics of special importance to the donors. A parallel study on the impact of CGIAR training in developing countries was also published by the CGIAR Technical Advisory Com- mittee (TAC) (TAC 1986). These studies will not be repeated. In the future, the main responsibility for impact assessment will rest with the centers themselves. Most IARCs, however, are not organized for systematic assessment. Self-select- ing successes often become the focus for ad hoc studies; cases of negative returns to research investments are downplayed or obscured (Anderson and Herdt 1990). Some CGIAR centers have revised their impact assess- ment needs, responding to the pressures of five-year external reviews, growing constraints on funding, and the adoption of more formal management processes in a search for greater efficiency. For example, both the International Center for Agricultural Research in the Dry Areas (ICARDA) and International Crops Research Institute for the Semi-Arid Tropics (ICRISAT) recently recruited agricultural economists to study impact as- 4 sessment. Impact assessment continues to pose major re- source allocation dilemmas for individual centers and for the CGIAR system as a whole. Full-blown impact assessment of all its research products, in collabora- tion with its diversity of clients, may require a doubling of the total CGIAR budget. The Research and Development Process and IARCs Three levels of activity characterize agricultural research: * strategic research, which is mission focused, seeking to understand those natural and human processes identified as important to the solution of a specified problem; * applied research, which uses existing knowl- edge to identify approaches and develop proto- type technologies to solve problems of wide- spread importance; and - adaptive research, which aims to articulate farmers' problems, identify appropriate ap- proaches to solution and relevant prototype tech- nologies, and fit these to the particular circum- stances of a specific group of farmers. Another category of research adds to the body of knowledge available to be used, when relevant, in the design of prototype solutions. Basic research, often termed "blue sky" research, has no specific problem focus and is not found in the CGIAR. Figure 1 shows a research paradigm that links these three levels and follows the "farmer back to farmer" configuration. In this model, farmers' problems are identified in the diagnostic stage of the adaptive research cycle, ideally by using methods that involve the farmers themselves. Many of these problems can be 5 Figure 1 An idealized research paradigm (adapted from CoUinson, 1982) Target group farmers in a region (1) Survey diagnosis of ON-FARM (3) Experiment on farmer priorities, apparently resources and environment ADAPTIVE relevant materials problems RESEARCH and technology under and development farmers' conditions opportunities (2) Identification and evaluation of materials and techniques offering potential for problem or opportunities (4) Unsolved technical STATION- (6) Body of knowledge of problems and possible BASED materials and techniques new practices and TECHNICAL suitable for the climate materials relevant RESEARCH and soils of the region to farmers' development opportunities (5) Commodity and disciplinary research, solving priority technical problems and investigating possible new materials and practices (9) Principles, (7) Transitional NTERNATIONA and methods for problems often RESEARCH manipulation of poorly understood researched phenomena (8) Strategic research understanding natural 6 phenomena ,$Du~~~~~~~~~~ ............gD solved within a two- to five-year research cycle by adapting technologies that are available from previous formal research or from the experiences of other groups of farmers. Some problems need new options and prototypes. If they are important to enough people, they find a place in the applied research agenda at a national or regional level: Problems whose underlying processes are poorly understood also find priority in the strategic research agenda-again, if the problems are important to enough people. At the strategic level, many such problems are relevant to a number of countries and are researched most cost effectively at the regional or international level. These cycles of applied and strategic research will often occupy a ten-year period. Perhaps the greatest challenge facing the agricul- tural research community is to build the capacity to operate this research paradigm effectively, in partner- ship with resource-poor farmers in developing coun- tries. In practice, these three levels of research do not follow as neat a sequence as Figure 1 suggests. Existing knowledge often can offer some solution. Yet it is frequently clear to researchers that better solutions are possible. Choices among options, rather than one final answer, are increasingly important. The more options that are available, the greater the chance that one will fit the circumstances of any given farming system, and that farmers will readily adapt it to their needs. Supplying nitrogen to plants is an example. It can be done in many ways: by moving to new land; rotating with green manures; pumping nutrients with legumi- nous trees; adding compost or organic animal manure; adding inorganic nitrogen out of the bag; and, for some types of plants, fixing nitrogen from the atmosphere. If nitrogen fixation ever can be introduced to other plant types, it will be a major research breakthrough. This effort has already spanned decades but is still pursued as another option, perhaps a superior one, for farmers. 7 The Development Process Once appropriate technology is available, other research and development (R & D) sectors need to mobilize its diffusion. It takes time for innovations to spread across target communities, through farmer-to- farmer contact or even an efficient extension service. Farmers themselves will usually experiment on small parts of their fields until convinced of a novel technology's value to them. Making new methods accessible to farmers may sometimes require making credit available so they can afford the purchase. This demands effective enabling institutions and innovative rural banks. Adop- tion by 80 percent of farmers is often assumed as a ceiling, and it may take a decade to achieve this degree of acceptance. Thus, when new knowledge is needed from strategic research, a twenty-year period is not unusual from the initiation of research to develop options to full benefits of results by farmers. Many factors inhibit performance in the other R & D sectors, and IARCs sometimes invest in solutions to such inhibitions when it seems important to their research interests. Where the market niches are too small to attract commercial seed producers, for ex- ample, Centro Internacional de Agricultura Tropical (CIAT) has had success promoting skilled local farmers as bean seed suppliers to their communities. Vegeta- tive propagation at Centro Internacional de la Papa (CIP) has helped promote local production of improved materials with potatoes, and at the International Insti- tute of Tropical Agriculture (IITA), efforts to use cassava seed for propagation also aim to reduce bottlenecks on the spread of improved cassavas. For roots and tubers that are usually propagated vegetatively, low multiplication rates make diffusion of 8 *All dollar amounts are U.S. dollars. new plant material particularly slow. Thai scientists selected a CIAT-developed cassava clone, CM 407-7. After several years of testing, it was released as Rayong 3 in northern Thailand in 1984. Innovative farmers were supplied with 600 stakes each and gave 80 percent of harvested stakes to neighbors. The area for stake multiplication was only 16 hectares in 1986. Even though it is estimated that some 70,000 hectares were planted with the new material in 1990, this is still only 4 to 5 percent of the cassava area in Thailand. Adopting farmers get 10 to 15 percent more revenue based on a 5 percent starch premium and in 1990 earned an estimated $3.8 to $4.6 million30' in extra income. In the long run, Thai cassava will become more competitive in the European market from this kind of innovation (Henry 1991). Extra production from such innovations may cre- ate surpluses where markets are limited. Research can create new market opportunities. Integrated cassava- drying projects in Latin America have been promoted and supported by CIAT and other R & D institutions. From a beginning in 1982 with a single factory for dried cassava chips for animal feed in Colombia, this indus- try grew to 153 factories in five countries by 1990. Product differentiation - adding dried cassava for animal feed to a market exclusively for fresh cassava for human consumption -created a wider cassava market with more stable prices, thereby stimulating the adop- tion of dried cassava processing technology. In Colombia, introduction of improved production technology has been integrated with the cassava-drying plants in the expectation that a more stable market would encourage farmers to increase and intensify their production. Between 1982 and 1990, the proportion of cultivated land devoted to cassava quadrupled among producers in the Cordoba area, and areas of fallow and 9 yams were reduced. The value added to small farmers' incomes from dried cassava production in 1990 was estimated to be $6.6 million, with $1.4 million to local processors: reduced imports of sorghum accumulated savings of $6.0 million of foreign currency by 1990. In Colombia, processing each 1,000 metric tons of dried cassava is estimated to generate 185 person-years of direct labor and 37 person-years of indirect labor (Henry 1991). Although, as these examples show, centers have intervened in the wider sectors of R & D, many bilateral and multilateral development agencies are active in these sectors. It is clear that the CGIAR, with a budget representing 6 to 7 percent of total developing country agricultural research budgets, has a real comparative advantage only in a limited sector of the sequence. Because the centers' global and regional mandates are for strategic and some applied research, many of the results are intermediate products, to be shaped by further applied and adaptive research to the circum- stances of the markets formed by diverse groups of farmers within each country. National agricultural research systems (NARS) have a clear advantage in this applied and adaptive research; at the same time, im- proved contacts between NARS and farmers better inform the research agenda for IARCs. Implications for CGIAR Impact This role in providing many countries with unfin- ished intermediate products has two key implications for the assessment of IARC impact. First, NARS form a geographically widespread and diverse set of clients. CIMMYT, perhaps the most global of the centers, inter- acts with up to 100 countries every year. Second, IARCs are highly dependent on the performance of other 10 institutions for successful impact in farmers' fields: on ii B:. } omB....'...>-gitiiiiEEEE-EB- the extension services, on available infrastructure, on market access for both products and for inputs, on policies and, most immediately, on NARS. The stronger NARS benefit most from CGIAR inter- mediate products (Anderson, Herdt, and Scobie 1988). Weak NARS are unable to adapt IARC products and unable to feed back information to help formulate a relevant international research agenda. The CGIAR acknowledges that practice falls far short of the R & D sequence idealized in Figure 1; its mandate includes building national capacity to organize and operate an effective research process. Some 20 percent of CGIAR resources are invested in capacity building with NARS. This role involves CGIAR scientists in training, develop- ment of methodology, and collaborative research with national scientists, including adaptive research when the aim is a better interaction with farmers. In Africa in particular, IARCs are drawn into applied research and adaptive research, because NARS of some countries have little capacity for these activities. Yet the real comparative advantage of an international effort re- mains in strategic research relevant to the problems of many countries. The fact that IARCs play a role only in a narrow sector of the total R & D sequence is often overlooked by donors eager for evidence of impact in farmers' fields. A Conceptual Framework for Impact Assessment The CGIAR centers have two broad categories of impact. They have direct impact on production, con- sumption, and human welfare. They also have indirect impact on the research capabilities of NARS and univer- sities in developing countries, and on the general un- derstanding of nature, enlarging the scientific stock of knowledge. Although CGIAR centers have a large effect on building scientific capacity in universities, this is an often neglected aspect of their work (Wilson 1989). 11 The framework in Figure 2 illustrates the diversity of products from IARCs and the complexity of the chain of repercussions resulting from farmers' adoption of new technologies. Some indicators listed merely record implementation; others quantify its effects. The Research Cycle and Feedback The conceptual framework shown in Figure 2 is relevant to both planning the allocation of research resources and the subsequent ex-post evaluation of the impact of research products. The current drop in CGIAR funding has increased the need to choose among re- search initiatives and to explain those choices when stakeholders, many NARS, and many donors seek cen- ter expertise to address their priority problems. As a result, IARCs have growing interest in planning re- source allocation. Setting priorities involves comparing the benefits expected from alternative research initiatives with their estimated costs and the probability of success. One way to measure potential benefits is to estimate current losses caused by the various problems under consider- ation. Several IARCs, including CIP, ICRISAT, and the International Center for Living Aquatic Resources Man- agement (ICLARM), have made such calculations in preparing new 1994-98 five-year budgets. CIP has recorded and published its process of priority setting (Collion and Gregory 1993). Solving a research problem is often a continuing process. New knowledge from strategic research allows more options and more effective technologies. The adoption of a succession of improvements gradually reduces the available benefits identified in the initial assessment. Although the greater understanding de- rived from continued strategic research will usually yield additional benefits, at some point the resources involved-the scientists and funds-would bring greater 12 benefits if they were applied to another problem. This Agricultural research in CGIAR Centres Indirect impact Direct impact C Institutional impact Scientific impact Production, consumption and human welfare Impact and effect Building capacity in NARS Advances in science and technology With farmer adoption of Without farmer adoption and universities -fundamental research: increased new technologies -biological control of pests -training knowledge and understanding -improved food policies -networking -strategic research; new and v -collaborative research improved research tools and I -improved research process approaches r Improve quality of manpower Additions to body of knowledge Increased production and/or higher More sustainable production Greater effectiveness of quality production and/or more stable (improved resource management) research institutions production and/or lower cost production -resource conservation -reduction of pesticides 0 -improved environment quality 0 Measurement of indicators M -number of people trained: -number and quality of publications -adoption rates of new technologies -reduced use of chemicals, l -number of research reports, -participation in international -improved quality of agricultural products fertilizers and pesticides journals, etc. scientific conferences -improved farm income -loss of species U2 -number of network meetings; -improved nutritional status and -pollution visits of research workers; -human welfare I amount of germplasm received; number of joint experiments Co. is the issue of the costs and benefits of more informa- tion. A key role for research managers is to identify when to switch resources to new problems. Maintenance research is an exception to this idea of a reducing benefit pile. Continuing efforts are needed to control expected mutations of disease vectors by identifying new sources of resistance. With rice, for example, the fight to control the brown planthopper has been essential to the protection of previous yield gains, particularly in Indonesia. The hopper had always been present as a rice pest, but the denser plant canopy of the semidwarf varieties provided a moist, shady envi- ronment that favored it. IRRI collaborated with govern- ments to introduce resistant varieties, the first of which were quickly overwhelmed by a second biotype of the hopper. A second round of new materials remains resistant but will probably provide only temporary respite. If such maintenance research stops, the risk of a food crisis increases. CIMMYT has estimated that 50 percent of its wheat research has been devoted to keeping ahead of mutat- ing pathogens. Genetic studies at the center have recently identified the basis of a durable form of resis- tance to leaf rust, one of the three major rust diseases of wheat, an achievement recently confirmed after sev- eral years of testing by the U.S. Department of Agricul- ture (USDA) (Ingersoll 1992). Again, a rough assess- ment is helpful in allowing stakeholders to judge CIMMYT's achievement. The yield losses that can be avoided if all new wheat materials incorporate this trait should conservatively total 1 percent of the annual crop value, on the order of $135 million each year to produc- ers and consumers in developing countries. CIMMYT will save on the costs of maintenance research, thereby releasing resources to address other problems. A yield loss assessment helps identify the potential benefits from strategic research programs that bring new understanding as the basis for new options to solve 14 a problem of broad scope. Centers can use such assessment of the numbers of farmers likely to benefit and the estimated level of benefit for each farm to inform, and even excite, their donors, as demonstrated by the following examples. Nonchemical control of the Mexican bean weevil, a pest of the stored bean crop in Latin America and Africa, has been achieved by adding natural resistance to commercial varieties. Over time, CIAT's work to incor- porate the arcelin gene from a wild bean species into varieties grown by farmers will save producers hun- dreds of millions of dollars and will also protect the environment by offering an alternative to pesticides. The International Laboratory for Research on Ani- mal Diseases (ILRAD), a strategic research laboratory that has worked for fifteen years to control theileriosis and trypanosomiasis in cattle, is developing bioeconomic models to value the losses from these diseases. Recent tests of the models have estimated annual losses from trypanosomiasis in Zimbabwe at $6.2 million. Further refined, these models will be used to estimate conti- nent-wide losses from the disease and to assess the economics of alternative control options. As outlined earlier, the full research cycle, from specification of the problem to full farmers' adoption of technical solutions, may occupy a twenty-year period- too long for the redesign or adjustment of ongoing programs. More important to research managers is short-term feedback to identify new options and to improve the shaping of technological solutions to farm- ers' circumstances. NARS have always provided feed- back to IARCs on the value of the materials supplied to them through the international trials networks. IARCs, in their collaboration with NARS, widely promote the need for on-farm research in which farmers assess new materials and practices in their own fields. This re- search is increasingly perceived as the exposure, test marketing, and adaptation of technology options. A recent example from Malawi documents farmers' com- 15 '";L':::. B B .:S.:§ ::.:S.:,': >>>g :::a.:: . ;. S <<. . .....::o: ::::g.- g *;~~~~~~~~~. ....... C. -:.2::>ff0 through national statistics to make estimates of im- pact, even on a global level. Some global and continen- tal impacts of the green revolution's short-strawed rice and wheat varieties have been noted. On the whole, because of the time required for adoption to be reflected in national production statistics, they are of limited use for feedback to adjust ongoing programs. Furthermore, as Table 1 shows, aggregate national data, particularly for crops used heavily for farm family subsistence, can be unreliable. Finally, cause and effect are less easily related at the aggregate level. Production can rise from increases in the area planted as well as from yield improvements. But where national data are reliable (particularly where land use areas and yields are well documented or survey data can be linked to increases in seed or input sales), they are cheap to obtain and analyze and offer strong reinforcement to donors of money well invested in the past. Table i " Comparative Cassava Production Figures for Nigeria, 1979-82-(in thousands of tons) Federal Cental Food aI. ... U.S. Ofice of Bank of AgriCuture Dtent of Year Statistics Nigeria Organization WsrNG-):cultwre 1979 1.621 1,976 105 100 1980 1,492 1,988 11,000 131 1981 872 2,159 11,000 1 1982 943 2,308 11,700 11700 Sour..e-Stifel 1992. 2 4 t - - ; ~__ _ _ _ _ _ _ _ _ _ __ . .. . . ............. . igg.gg-g . ggg g... igggggiiiig ............. -i i gg gig. g ' ........ ................ .... - g gEiiiii2-:.... .. . ..... ... ,: In some cases, centers have collaborated with and supported government statistical departments, supple- menting the data to be captured in routine sample surveys. One recent example is CIAT in collaboration with the Government of Rwanda, where new varieties of climbing beans were reported to be popular among farmers. In 1992, CIAT collaborated with both the Departement des Statistiques Agricoles, which regu- larly documents trends in the 93 percent of Rwandan households that are dependent on agriculture, and the Institut des Sciences Agronomiques du Rwanda to evaluate the spread of climbing beans. They have always been popular in the Gisenye and Ruhengeri areas of the north and west, but surveys as late as 1986 showed that only 5 percent of farmers in the central and southern areas grew climbing beans. The 1992 sample survey, however, reported improved varieties of climb- ing beans on 43 percent of farms, representing 450,000 rural households. Estimates from the survey, carried out in one of Rwanda's two bean seasons, gave an annual value of net benefits of between $4 million and $8 million for Rwandan farmers (Sperling 1993). This tyfpe of documentation has increased during the past two years. Table 2, from a recent publication by the Latin America-based centers, demonstrates the impact of their collaboration in germplasm develop- ment with the national programs in the region (CIMMYT, CIAT, and CIP 1992) . Benefits from the improved materials have been allocated equally between IARCs and NARS. For beans, maize, rice, and wheat, the impact for which is clearly documented, the annual value of in- creased production is $1,050 million, or nearly ten times the total 1990 budgets of the three centers combined. Because international markets for rice and 25 .Table. 2 . . = ~~~~~~~~~~~~~~~~~~~~~~~~~~~~,,,.,.... . :.'5......''. '.:.i'. .......... .......... ... ............... Progress in Genetic Irovet in Beanis Maize, Rice and Wheat-in atow A1erica Unti 1990 Achievement ~ e#nb Msiae Rice eS Number of teleased. varieties, lHnes, or both orsgatingfromCIAR centers 9- 86 69 134 Area grovm with these varieties, 1990 (thousands of hectares) 370 2,002 1,208 4,074 Number of vaieties released with ptatda m, if mcentrs, 8 111 60 272 Area grownw tese vweties, 1990 (thousands of hectares) 60 1,848 800 4,560 'Criop :aea ifetd s mtes er~Is, 1990(%s) : 4.9 16.7 25.1 81.2 Estim atd rprducion ro thesce materas 199 ''''u s i"'''' tons, , ,,,122 1,696 836 3,523 imad cxtra value, 1990 ~US $ milionl ' 60.8 203 209 567 Price saving to consumers, 1990 (%J 5 0 24 0 Costs of centers' researh progrms, l9w (VUS nUons)* 3.3 8.8 4.0 -,4 Intemal rate of return on research programs up to 1990 616 5 69 67 X Includes overhead. Source: CIAT, CIMMYT, and CIP 1992. beans are limited, as production went up, prices went down, making consumers the principal beneficiaries of new technologies for these crops. Thirty-three varieties of potatoes developed at CIP have been released in Latin America, but the impact has not yet been documented. For other commodities, such as pasture and cassava, research started from a much more limited knowledge 26 base, and impact is only now being observed. ...........o Economic Models Aggregate production data can be multiplied by the price of the product to give a simple economic valuation of benefits. Such simple valuations, however, do not reflect the full economic consequences of the increases in production, because the impacts of technological innovation are confounded by the effects of changing market prices and changing policies on the target crop as well as by the effects of similar changes on other crops. In addition, although more efficient production means cheaper consumer prices, it may also mean changed demands in the labor, machinery, and sup- plies markets. Economic models capture these repercussions and offer an aggregate measure of the economic gain created by technological innovation. The internal rate of return, shown in the final line of Table 2, is one such measure. Based on the gain created, it specifies the rate of return to the investment in germplasm research. A detailed description of types of economic models and a list of results from their application can be found in Evenson (1992). Such models have their disadvantages. For non- technical audiences, they offer less transparent results than the less sophisticated aggregated adoption, area, and production data, and they identify cause-and-effect relationships less clearly. As mentioned earlier, diverse agencies contribute to the total R & D process and thus to the CGIAR's dependence on others to mobilize its products. It is often unclear in model applications whether the benefits that stream from new technology are appropriately allocated over the full range of invest- ments that contribute to technology adoption and diffu- sion. As in a recent CIMMYT study of wheat research in Nepal, it is often appropriate to acknowledge that inno- 27 vation would occur without a national research pro- gram-in the Nepal case, as spillover from research in India and Pakistan. In this study, the returns to research are measured by the economic gains accruing to the investments made by the national research pro- gram, net of the contribution of the spillover and of the returns to investments by other sectors in the R & D process (Morris, Dubin, and Pokhrel 1992). Finally, it is important to note that the interna- tional centers have wider criteria for success than economic surplus alone. They are interested in alleviat- ing poverty, reducing farmers' risks, and sustaining the natural resource base. Although economic models can be used to show the distribution of benefits between producers and consumers, they cannot yet capture the other dimensions important to the mission of the CGIAR centers. Challenges and Strategies for IARCs The centers must help donors assure their con- stituencies that investments in research are valuable. They must also meet their own needs for priority setting and program planning. The centers are most efficient in producing intermediate products that are useful to many countries. Such products need further research to tailor them to farmers' circumstances. Beyond that, institutional and policy support are also needed to enable farmers to exploit these products. Historically, researchers in general, and particu- larly international researchers, have depended on oth- ers to identify the technology needs of small farmers and to mobilize their research results to meet those needs. This dependence is the first major challenge for IARCs because researchers feel pressure to justify their work 28 in terms of progress in development, without having any influence over many of the factors that propel develop- ment (Hardie 1988). The difficulty of linking research to the market has long been acknowledged, in industry and in private- sector agricultural research. For each product that succeeds in the market, a typical manufacturing com- pany generates fifty-eight new product ideas. After business analysis, seven of these generally reach the development stage. Of these seven, six are eliminated during development, testing, or commercialization. Almost 75 percent of new product expenses (and thus the work of eight out of ten development scientists and engineers) are devoted to projects that will not be justified in terms of commercial success (Booz, Allen & Hamilton Inc. 1968). The preceding example reflects an average success rate of 14 percent for the manufactur- ing companies surveyed. It demonstrates that research cannot be planned with certainty. Serendipity remains an important element. In agriculture, the vagaries of the weather are an added source of uncertainty. Production, and therefore farmers' incomes, varies from year to year. This uncer- tainty also complicates and often prolongs the research process. DeKalb Seed Company makes some 5,000 crosses to identify one new commercial maize hybrid. Pioneer Seed Company has released an average of twelve commercial conventional hybrids per year dur- ing the past decade on an annual budget of approxi- mately $19 million. Although similar data for CIMMYT cannot be used as a test of comparative performance, they are of interest. National programs in developing countries released more than 300 maize varieties or hybrids containing CIMMYT material between 1981 and 1990 (Table 3), an average of thirty per year on an annual average maize program budget (including over- head) of approximately $12 million (CIMMYT 1992). Improved maize based on CIMMYT materials now covers just less than 10 million hectares-or some 12 29 - :-..~Table 3=: . Malze'varieties and hybrids containiag CIMMYT germplasm released in developing countrites,by region. 1966-90. Releases containng C.MMYT germplaszn Code 1 Code 2 Code 3 Total Sub-Saharan Africa: 1966-70 0 0 3 3 1971-75 4 1 6. 1l 1976-80 6 4 3 13: 1a981-85 - .: : . E 13 14 25 52 ,1986-90 '12 17 20 4'9 -Total ' 5 36 57 128 1960-70 0 0 0 0 19t7175 0- 2 0 2 1976-,,,S,,O -0,- 1 0 1 1981-=8=5:' 1 2 0 3 1986-90 0 .4 0 4 Total 1 --.9 0 10 Asia: 1966-70 0 0. 3 3 1971-7S 0 0 2 2 1976-80 6 1 I- '9 16 1981-85 9 2 22' 33 1986-90 10 7 33 50 Total 25 10 69C 104 Latin America: 1966-70 8 3 9 -20 :1A971-75 4 2 8 14 1976-80 6 -; " = ' H . 11 6 - 0 - 23 1981-85 26 16 33 75 198690 1z2 23 36 71 Tot, 5:. 55 92 203 An 4eveloping countries,: 1966-70' 8A-:: .. ::,- S3 9 26 1971-75 8 3a 8 29 1976-80 18 16, 6 53 1981-85 48 33' 33 163 1986-90 34 47 36 174 Total 116 102 92 445 Code I = Direct use of CIMMT germplasn. Code 2 = Selection from CItMMYT trials. Code 3 = Contains some,CIMMYT germplasm. Sopuree: Lopez, Pivira, M.&. And M.L. Morris. 1994. impacts of Iternational Ma=ze Breedi Research In the Developig World, 1966-. Mexico D.F: eCTMT. 30 _ _ _ _: _ ---.-._= percent of the total maize area of developing countries. These numbers reflect the situation of many dryland crops in agricultural sectors dominated by small farm- ers. The potential for further impact remains huge; achieving it will require governments and the interna- tional community to address the performance of comple- mentary functions essential to the R & D sequence, such as produce marketing, farm input supply, farm credit, and extension services. Research will not always be the priority candidate for investments. Unfortu- nately, the adaptive research function remains weak. There is only very sporadic coverage of small-farm sectors by cadres that understand commrunities and can enroll farmers in partnership by the use of partici- patory techniques. This direct exchange between re- searchers and resource-poor small farmers is a prereq- uisite for better market information and better balance in the demand for and supply of technology. The second major challenge faced by IARCs is their diversity of products with a clientele that is scattered across the developing world. A typical center may interact with thirty national programs and have consid- erably more than 100 research projects. The cost of assessing the impact of these projects on so many clients is prohibitive, especially because weak agricul- tural statistics mean that primary data must be col- lected to measure impact in most partner countries. The key to monitoring, evaluation, and impact assessment is the same as that for better articulation of small farmers' technology needs: an institutionalized cadre of professionals with skills at the grassroots level to whom this work is routine. Continual interactions with farmers and communities, an integral part of the adaptive research function, generates powerful feed- back on the effectiveness of the R & D sequence. From experiences with a variety of farm-level models, a consen- 31 ...................... . . .:. . .z . . .> . . . '' ..... sus is emerging that a participatory approach that mobilizes both the communities and the public service together will make good use of scarce professional resources. A concerted effort to rationalize this grassroots function would widen coverage and generate informa- tion for better decision making at the program and policy levels. Planning Monitoring the research cycle is important to IARCs. Preliminary assessment and intermediate feedback, within the timeframe of the cycle, are particularly important to priority setting and program planning. Evidence of early adoption, or early farmer assess- ments, offer the benefit of timely feedback to modify continuing programs. Special studies of final impact are of little use to program planning because the R & D cycle through diffusion of the technology to final impact assessment is too long. IARC programs normally will have moved on before the results of such assessments are available to feed back into the planning process. Where research for better solutions will continue, the impact of an earlier iteration can measure the residual benefits for the evaluation of further research. At the same time, special studies of final impact do add to our knowledge of the technology adoption and diffusion process. When this kind of research is undertaken, an explicit aim in its planning should be to use the information for as many purposes as possible to offset the heavy overhead costs of primary data collection. Subsequent studies can be integrated with the collection of government agricultural statistics where this is dependable and where there is similar value to NARS partners in dem- onstrating the success of previous investments of pub- 32 lic funds. B.. . . .. .. . . . . .i Wherever primary data collection is planned and the products of more than one IARC have helped to improve local farming, there is scope for intercenter collaboration in sharing fieldwork expenses. Donor Needs Donors that sponsor development are interested in the effectiveness of their funding. The current reces- sion coincides with a burgeoning demand for aid for Eastern Europe, for United Nations activities, and for the environment. The scarcity of funds has sharpened donors' interest in evidence of impact, and IARCs are responding, some more strongly than others. Preliminary assessments of the potential impact of research are valuable to donors, particularly when a research calendar highlights mileposts as intermediate steps toward solution of problems and adoption of the technology by farmers. Although they are not measures of impact, such achievements allow donors to monitor progress and help reconcile them to the longer time horizons of strategic research programs. The expansion of research on natural resource management in the CGIAR will involve longer-term programs and less definable products. In many cases, the product will be an understanding of natural pro- cesses and the avoidance of losses, often measured in terms of topsoil saved, intact groundwater, or other external factors. Clear preliminary planning is needed before investments to realize such products will be accepted. Planning should emphasize intermediate achievements and projections of the expected social benefit of resource management interventions. One dimension of the ecoregional strategy recently adopted by the CGIAR is to show early benefits to the production system in which sites are located; an understanding of the interactions between human decisions and natural resource processes accumulated over the long term. 33 The CIMMYT data in Table 3 illustrate the record- ing and presentation of intermediate achievements. By maintaining a data base of plant material sent to national programs and the pedigrees of materials re- leased by them, CIMMYT is able to highlight the impor- tance of its germplasm. These are milepost achieve- ments that demonstrate the effectiveness of the center's partnership with national programs on three conti- nents. Donors use the documentation of final impact to demonstrate the success of their funding in their do- mestic budget process. Because studies of final impact are expensive and must be selective, the aim is to show examples of high returns on IARC investment. Benefit flows can evaluate returns beyond the costs of the immediate program, for example, against the total in- vestment in research in the commodity and even against the investment in the center as a whole. Although economic models are useful for such studies, for the greatest impact it is important to document farmers' adoption, improvements in house- hold situations, any differential impact on livelihood between men and women, and the aggregate impact on production. Such parameters add value for public relations purposes and for donors. IARCs think that such studies should be done in partnership with na- tional programs, to help convince their Ministries of Finance to improve research funding as an investment in the nation's future. IARCs also bring significant returns to the domes- tic agricultural economy of some of the CGIAR's major donors. Quantifying these returns can be valuable to future center support. To date, only Australia has quantified the benefits it has received from improved wheat materials made available from CIMMYT: "Since 1977, Australia has contributed an average of US$2.8 million per year to core programs of the 34 CGIAR Centers. Of this some 6% has gone to CIMMYT's syB g..... g lB........ .. . . .... . . . .......................::: gv. lll. iz; wheat program . . . Australia has received overall cost reductions averaging some US$75 million per year re- sulting from the improved wheat varieties derived from CIMMYT" (Brennan 1989). The Brennan study attracted wide attention in Australia: "Mr John Kerin, Minister for Primary Industries and Energy, recently announced that the use, in Aus- tralia, from 1974-1990, of wheat germplasm imported from CIMMYT has resulted in additional income for the wheat industry of over two billion dollars-a sum suffi- cient to fund both ACIAR [the Australian Council for International Agricultural Research] and Australia's contribution to the CGIAR at their present levels for the next 100 years!" (Tribe 1991). There is also an important educational task in raising awareness among donors and their constituen- cies on the nature of the research process. Their expectations should be based on an understanding of two characteristics of the process: the uncertainty of research as a business and the time it takes to complete. Conclusion Research initiatives are defined by the problems they seek to solve, not the product they hope to identify. Failure is common. Even with success, the final nature of the product, and therefore its congruence with mar- ket needs, is initially unknown. These uncertainties are heightened by the difficulties of identifying market needs among small farmers to help shape the product as the research process progresses. Better market information will reduce the failure rate of new technologies and enhance the efficiency of 35 . . . : . . . . . the research process. This can be achieved by wide promotion of and support for adaptive research, and by rationalization of the organization of field-level staff working in research, extension, and evaluation. Ratio- nalized field-level organization can easily provide peri- odic impact assessment information for clients ranging from research managers to policy makers and donors. Routine information flows from the field will enhance the relevance of decisions at each of these levels to the needs of developing country rural populations. 36 References Anderson, Jock R., and R. W. Herdt. 1990. "Reflections on Impact Assessment." In Ruben G. Echeverria, ed., Methods for Diagnosing Research System Con- straints and Assessing the Impact of Agricultural Research. Vol. 2. The Hague: ISNAR. Anderson, Jock R., R. W. Herdt, and Grant M. Scobie. 1988. Science and Food: The CGIAR and Its Part- ners. Washington, D.C.: CGIAR and The World Bank. Booz, Allen & Hamilton, Inc. 1968. "A Program for New Product Evolution." Quoted in Robert R. Rothberg, ed., Corporate Strategy and ProductInnouation. 2nd ed. The Free Press, 1981. Brennan, J. P. 1989. 'Spillover Effects of International Agricultural Research: CIMMYT-based Semi-Dwarf Wheats in Australia." Agricultural Economics 3:323- 32. Byerlee, D., and Piedad Moya. 1992. "Ex-post Assess- ment of Research Impact: CIMMYT's Experience." In David R. Lee, Steven Kearl, and Norman Uphoff, eds., Assessing the Impact of International Agricul- tural Research for Sustainable Development. Pro- ceedings from a 1991 CIIFAD Symposium at Cornell University, Ithaca, N.Y. CIAT, CIMMYT, and CIP. 1992. CIAT, CIMMYT & CIP: Their Role in Agricultural Research in Latin America and the Caribbean. Cali, Colombia: CIAT. CIMMYT. 1992. CIMMYTAnnual Report 1991. Improv- ing the Productivity of Maize and Wheat in Develop- ing Countries: An Assessment of Impact. Mexico D.F.: CIMMYT.. 37 ;~~~~~~~~~~~~~~~. . . . . .. .. . .................................. ,.., ,T ..... j . .. T....... .. : :, ,' . , . , Collinson, M. P. 1968. The Evaluation of Innovationsfor Peasant Farming. East African Journal of Rural Development. Vol. 1, No. 2. Kampala, Uganda: Makerere University. Collinson, M. P. 1982. 'Farming Systems Research in East Africa: The Experience of CIMMYT and Some National Agricultural Research Services, 1976-81." International Development Papers No. 3. East Lan- sing: Michigan State University. Collion, M. H., and P. Gregory. 1993. Priority Setting at CIP. Lima: CIP/ISNAR. Dalrymple, Dana G. 1986a. Development and Spread of High Yielding Rice Varieties in Developing Coun- tries. Washington, D.C.: USAID. Dalrymple, Dana G. 1986b. Development and Spread of High Yielding Wheat Varieties in Developing Coun- tries. Washington, D.C.: USAID. David, C. C., and K. Otsuka. 1991. The Green Revolu- tion and Income Distribution Across Production Envi- ronments in Asia. Los Banos, Philippines: IRRI. Domingo, A. A., L. D. Barnachea, S. S. Mina, and D. R. Minnick. 1989. The IRRI Short Term Training Program: An Impact Assessment, Training and Tech- nology Transfer Department, IRRI. Los Banos, Phil- ippines: IRRI. Evenson, R. E. 1992. "Notes on the Measurement of the Economic Consequences of Agricultural Research in Investments." In David R. Lee, Steven Kearl, and Norman Uphoff, eds., Assessing the Impact of Inter- national Agricultural Research for Sustainable De- velopment. (Proceedings from a 1991 CIIFAD Synm- posium at Cornell University,) Ithaca, N.Y. Hardie, John D. M. 1988. "Measuring Development 38 Effects of Agricultural Research in the Third World: ~~~~~~~~~~~~~~~~~~~~~~...a........... Case Study Methodologies." Paper prepared for the Rutgers University/ISNAR Agricultural Technology Workshop, July 1988. Hazell, Peter B . R., and C . Ramasamy. 1991. The Green Revolution Reconsidered. Baltimore: John Hopkins University Press. Henry, G. 1991. "sAdoption of Cassava Technologies: Constraints, Strategies, and Impact." Working Pa- per No. 93. Trends in CIAT Commodities 1991. CIAT, Cali, Colombia. Horton, Douglas E. 1990. "Assessing the Impact of International Research: Concepts and Challenges." In Ruben G. Echeverria, ed., Met hodsforDiagnosing Research System Constraints and Assessing the Impact ofAgricultural Research. Vol. 2. The Hague: ISNAR. Ingersoll, Bruce. 1992. "Breakthrough Promises to End Blight That Afflicts Wheat Crops." Wall Street Journal. October 23. Kiray, M., and J. Hinderink. 1968. "Interdependencies Between Agroeconomic Development and Social Change" Journal of Development and Social Change. 4:4. Morris, Michael L., H. J. Dubin, and Thaneswar Pokhrel. 1992. "Returns to Wheat Research in Nepal." CIMMYT Economics Working Paper Series 92-04. CIMMYT, Mexico D.F. Norgaard, R. B. 1988. "The Biological Control of Cassava Mealy Bug in Africa." American Journal of Agricultural Economics 70(2): pp 366-371. Scobie, Grant. 1979. "Investment in International Agricultural Research: Some Economic Dimen- 39 sions." Staff Working Paper No. 361. World Bank, Washington, D.C. Smale, M., Z. H. W. Kaunda, H. L. Makina, and M. M. M. K. Mkandwire. 1993. Farmers' Evaluation of Newly Released Maize Cultivars in Malawi: A Comparison of Local Maize, Semi-flint and DentHybrids. Lilongwe, Malawi, and Harare, Zimbabwe: CIMMYT. Sperling, Louise. 1993. "The Impact of Improved Climbing Beans in Rwanda: Preliminary Results of Collaboration, DSA/ISAR/CIAT." Paper presented at the seminar on bean crop research, Institut des Sciences Agronomiques du Burundi, Bujumbura, Burundi. Feb. 1993. Cali, Columbia CIAT. Louise Sperling, Loevinsohn, M.E. and B. Ntambovura. 1993. "Rethinking the Farmer's Role in Plant Breed- ing: Local Bean Experts and On Station Selection in Rwanda." ExperimentalAgriculture, Vol 29 No.4. pp 509-519. Stifel, Laurence D. 1992. 'Impact Assessment in Africa: Experiences of IITA and Other IARCs." In David R. Lee, Steven Kearl, and Norman Uphoff, eds., Assessing the Impact of International Agricul- tural Research for Sustainable Development. Pro- ceedings from a 1991 CIIFAD Symposium at Cornell University, Ithaca, N.Y. TAC. 1986. Training in the CGIAR System. Rome: TAC. Timothy, D. H., Paul H. Harvey, and Christopher R. Dowswell. 1988. Development and Spread of Im- proved Maize Varieties and Hybrids in Developing Countries. Washington, D.C.: USAID. Tribe, D. E. 1991. Doing Well by Doing Good. Agricul- tural Research: Feeding and Greening the World. 40 Leichhardt, NSW: Pluto Press Australia Ltd., in .| -s. .::. ::s ..... . - v. . .............. ............................ .s:g .: $gB0:: i ..... iiEf-iig:.....i.iEg> .;.: . :; ...-.-E:-::i::EEE::EE :: -: : :::::::::::-::.:...:: association with the Crawford Foundation for Inter- national Agricultural Research. Wilson, Lawrence A. 1989. The Establishment of Linkages Between Universities and International Agricultural Research Centersfor the Development of Third World Countries. Ibadan, Nigeria: IITA. @ Printed on Recycled Paper 4 1