The Third Tokyo Workshop on ODA Evaluation 12-13 November 2003 Hosted by the Ministry of Foreign Affairs ----- Foreword Donor-partner cooperation is a key for reliable and effective evaluation, which is an important management tool for effective development assistance. Given this idea, the Ministry of Foreign Affairs of Japan has been hosting evaluation workshops to provide an opportunity to consolidate a donor-partner relationship of Asian countries. The first workshop was held in 2001 to facilitate common understanding of ODA evaluation. The highlights of the discussions included the involvement of partner countries in ODA evaluation, the importance of capacity building, as well as strong political, legal and institutional support for effective evaluation. In the second workshop of 2002, the focus was placed on scaling up evaluation objects from project to program and policy levels. In international fora, there is a growing interest of evaluation as a key management tool. In March 2003, DAC Working Party on Aid Evaluation discussed joint evaluation in its focus session. The Initiative for Development of East Asia (IDEA) ministerial meeting, held in 2002, pointed out the importance of donor-partner collaboration in the field of development issues. Following the discussions of the two Tokyo workshops, DAC WP-EV and IDEA ministerial meeting, the Ministry of Foreign Affairs of Japan hosted The Third Tokyo Workshop on ODA Evaluation, on 12 and 13 November 2003 at the Keio Plaza Hotel in Tokyo. The objectives of the workshop were as follows: a) To facilitate common understanding that ODA evaluation is a key tool for more effective development assistance; and b) To discuss and identify challenges and future actions for joint donor-partner evaluations. Government officials of eighteen Asian countries were invited from Bangladesh, Brunei, Cambodia, China, India, Indonesia, the Republic of Korea, Laos, Malaysia, Mongolia, Myanmar, Nepal, Pakistan, the Philippines, Singapore, Sri Lanka, Thailand and Vietnam. Representatives of a DAC member country (France) and multilateral organizations: Asian Development Bank (ADB), World Bank, DAC, United Nations Development Programme (UNDP) and United Nations Children's Fund (UNICEF) also participated in the workshop. Participants engaged in three sessions of vigorous discussions, which culminated in the compilation of a Chairperson's Summary by Chairperson of the Workshop Prof. Ryokichi Hirono, Professor Emeritus of Seikei University. The Chairperson's Summary is provided overleaf. ----- Chairperson's Summary Prof. Ryokichi Hirono, Chairperson of "The Third Tokyo Workshop on ODA Evaluation" Japan hosted "The Third Tokyo Workshop on ODA Evaluation" on 12-13 November 2003 at the Keio Plaza Hotel. This was the third workshop on ODA Evaluation following the last two years. Fifty participants including senior government officials from eighteen Asian countries, evaluation specialists of a donor country and multilateral development agencies, Japanese government officials of ODA-related ministries, staff of the Japan International Cooperation Agency (JICA), Japan Bank for International Cooperation (JBIC) and academic persons also attended this workshop. A number of people attended the workshop in an observer status. The participants recognized the importance of donor-partner cooperation to tackle development challenges and global development issues. To enhance the effectiveness of development and cooperation, it is necessary to set out a long-term comprehensive development strategy and a management system. Through this workshop, the participants expressed their shared recognition of the importance of donor-partner cooperation based on partner countries・ownership. In particular, joint donor-partner monitoring and evaluation was recognized as an effective tool for management of development assistance and overall development activities. Participants from Asian countries, together with bilateral and multi-lateral organizations, have discussed and shared their knowledge and experiences on the methods to manage ODA activities. While recognizing the critical importance of a comprehensive development approach, the participants, based on their experiences, exchanged their views mainly on how to manage ODA more effectively, and on challenges and difficulties in conducting joint evaluation. In each country, the general public is increasingly concerned with transparency, accountability and participation in conducting government activities. In this respect, it was significant that this workshop was open to the public. All participants appreciated Japan's initiative to organize the third workshop for the purpose of deepening understanding of the importance of monitoring, evaluation and collaboration among Asian countries and the donor community. Some highlights of the discussions include: It is necessary to streamline a management cycle of planning, implementation, monitoring and evaluation for maximum development impact in partner countries. The key is how to integrate monitoring and evaluation systems into the whole development process for greater effectiveness of development and cooperation. Recognizing the importance of country ownership and multiplicity of donors, it is important that the scope of evaluation should cover the sector- and country-level in addition to the project level. The transaction costs are generally reduced when donors and partners cooperate within a common framework, in spite of some difficulties. What is necessary to tackle the development challenges through collaboration between donors and partner countries is to agree on common development goals. Stakeholders including government, local bodies, private sector and civil society should be involved, wherever appropriate, in the planning process. Also, we need to consider how donor countries and organizations align with the partner country's priority development goals. Coordination should be made in a flexible and country-specific manner. In order to use the limited resources more efficiently, long-term comprehensive strategic goals are essential. In this connection, achieving MDGs poses a great challenge. In managing development activities, monitoring and evaluation indicators should be set to assess to what extent the objectives are achieved. More reliable quantitative indicators are needed while the importance of qualitative indicators should not be underestimated. Although outcome-based evaluation is important, the assessment of process and secondary impact should not be overlooked. Moreover, the ability to collect and analyze data should be enhanced for regular measurement of the set indicators. The effectiveness of development activities is improved when evaluation results are utilized effectively. It is, therefore, essential to establish feedback mechanisms so that the evaluation results are incorporated into the planning and implementation process. For this purpose, development organizations are encouraged to make active use of evaluation results. The key is capacity building in partner countries. There have been many practices of donorpartner and South-South cooperation for human resource development. The workshop provided an opportunity for exchange of knowledge and experiences among donors and partners in monitoring and evaluation. The workshop recommended that, in addition to the establishment of an evaluation association in each country, evaluation networks should be strengthened among Asian countries. The workshop welcomed the proposal made by Thailand to host a joint evaluation seminar next year and the proposal by Vietnam to conduct a joint monitoring and evaluation exercise, with possible impact on capacity building. In this connection, the participants welcomed the initiative of the Japan Evaluation Society in following up these efforts in cooperation with other similar organizations. The report of this workshop will be published soon. The participants from Asian partner countries are kindly expected to utilize the results of the workshop for further improvement of their monitoring and evaluation systems. ----- 1. Opening Session Mr. Hajime Furuta, Director-General, Economic Cooperation Bureau, MOFA Mr. Furuta made opening remarks, explaining that Japan has hosted the workshop on ODA evaluation since 2001. The first workshop was an opportunity to enumerate challenges, with which ODA evaluation is faced. In the second workshop, the focus was placed on how to scale-up evaluation objectives, given the emergence of sector-wide and countrywide development strategies. Mr. Furuta stated that the third workshop would highlight "joint evaluation," following the discussions in the previous two workshops. It is generally agreed that monitoring and evaluations need to be revised in accordance with sector-wide and countrywide approaches. Joint evaluation could be a possible method in estimating the progress of common goals at sector and country levels. Keynote speech: "National Policy on Project Monitoring and Project Performance Evaluation" as a Management Tool of ODA in Indonesia Mr. Benny Setiawan, Director, Directorate of Development Funding Monitoring and Evaluation, BAPPENAS, Republic of Indonesia Mr. Benny Setiawan delivered the keynote speech on behalf of BAPPENAS of the Republic of Indonesia. The scale of development in Indonesia has increased tremendously over recent years. However, the government has limited options in financing development programs due to limited resources. A good monitoring and reporting system is an important source of information and this is a good tool to scrutinize development projects. The long-standing support provided by Japan in the area of ODA is greatly appreciated. Questions must now be asked however, as to how effective ODA has been in Indonesia. Whereas monitoring is the continuous assessment of project implementation in relation to agreed schedules, evaluation however, is a discontinuous function of assessing the degree to which objectives are being achieved and the validity of approaches. Evaluation entails more in-depth analysis and assessment. Evaluation can be classified into the following: ex-ante evaluation; on-going evaluation; terminal evaluation; and ex-post evaluation. The need for effective monitoring and evaluation is increasingly recognized as indispensable to project implementation and management. The Country Portfolio Performance Review (CPPR) plays a key role in such monitoring and evaluation and has several aims. The first is to improve project management. A further aim is to turn around problems and also to streamline the disbursement process. The necessity for the roll-over of unspent funds also needs to be recognized. Corruption is an issue that also needs to be tackled. Project readiness criteria should be formulated. Future projects should be judged in terms of implementation readiness prior to approval. A part of the problem in implementing ODA projects is inadequate implementation readiness, and this requires the formulation of implementation readiness criteria. Project proposals that lack a concrete framework should be rejected and a system of rewards/penalties could also usefully be implemented. Experience in Indonesia confirms that effectiveness of evaluation is dependent upon capable human resources. The Government of Indonesia's evaluation mechanism, BAPPENAS, periodically conducts three-month meetings to monitor progress, identify problems that have arisen in the execution of projects, and find solutions to these problems. Obstacles that have been identified and reported generally embrace the delay in the procurement process, shortage of counterpart funds, project location and project design revision, and security problems. A number of approaches can be taken to achieve effective ODA monitoring and evaluation. First, an improved understanding of the achievement of government activities, and in particular, development projects, will enhance the ability of departments/line agencies as a whole to allocate and manage resources in a more effective manner. Second, since the type of problems and obstacles faced in the implementation of ODA projects are similar, the Government of Indonesia and major donors have to cooperate to follow up progress that has been made in solving agreed upon actions. Third, experience has shown that part of the problem with the slow disbursement of ODA lies in inadequate implementation readiness. Fourth, good governance has become unavoidable not only for governments but also for the rest of society including corporate institutions. Fifth, evaluation efforts need coordination between cross-sectoral institutions and regional programs. ODA has significantly contributed to Indonesia's development, but it remains a concern about how to improve the mechanisms for planning, monitoring and evaluating ODA loans. ODA loans therefore need to be managed in a more transparent and accountable manner in Indonesia. Discussion This session was chaired by Prof. Ryokichi Hirono, Professor Emeritus of Seikei University. The Chair thanked Mr. Setiawan for his presentation and welcomed the recommendations: need for better understanding of the achievement of government activities; need for coordinated work between beneficiaries and donors; implementation readiness, commitment by stakeholders for good governance; and cross-sectoral cooperation. He opened the floor for comments or questions. Mr. Quesnel of UNICEF thanked Mr. Setiawan for his presentation, pointing out that it showed clearly the discipline that is required in the management and evaluation of ODA projects. Very often key assumptions are forgotten that are key for the readiness and success of the project. Mr. Quesnel stated that the challenge being faced is how to move from the activity/project level to a program level on a sectoral basis, which could then develop to strategic performance. Another, and more difficult area, is the realm of policy and how to evaluate results at the policy level. Mr. Bountheuang Mounlasy of Lao PDR thanked the Indonesian delegation for providing their comprehensive experiences in ODA evaluation and monitoring. He pointed out however that evaluation feedback was not, however, transferred into the project cycle on many occasions. He suggested that discussion should also be engaged concerning the methods of inputting evaluation feedback into the project cycle. Mr. Santos of the Philippines mentioned the problem of counterpart funding problems that had been raised in the keynote speech, explaining that the Philippines experienced the same problem. He asked about the severity of the counterpart funding problems in Indonesia. Mr. Setiawan responded that one of the reasons for the failure to follow the Presidential Decree in Indonesia was the lack of an incentive or penalty scheme. With regard to counterpart funding problems, Mr. Setiawan noted the difficulties in providing the counterpart funds, particularly in one installment. He explained that in the Indonesian experience, counterpart funds are provided in installments, but very often difficulties experienced in counterpart funding could delay ODA projects. The Chair pointed out that the lack of feedback and its input into the project cycle was a perennial problem not only in partner countries, but also in donor countries. Concerning feedback, Mr. Lundgren of the OECD-DAC asked about the role of the CPPR that had been described in the Indonesian presentation. He asked whether the CPPR could provide an opportunity to review ongoing program effectiveness from an evaluative perspective. Mr. Setiawan responded that CPPR was an evaluation mechanism in cooperation with major donors with the aim of reviewing implementation of ODA projects. The CPPR was implemented, he continued, with the World Bank, ADB and JBIC in order to prepare concrete solutions to specific implementation issues. Mr. Walter of the ADB pointed out the importance of the Indonesian CPPR scheme. He stressed the importance of ongoing evaluation in comparison with ex-post evaluation. The challenge, he said, was to implement ongoing evaluation in a joint manner and feed evaluative content into the ODA project cycle. Mr. Matsuzawa of JBIC stated that project relevance to the planned strategy was an issue that faced JBIC as well as the Indonesian government. He emphasized the importance of making strategic changes to projects as and when required. The Chair emphasized the importance of strategic evaluation, as raised by UNICEF and JBIC. He referred to results-based management at the project level by the US government, which was reflected in the budget process, which provides large incentives or disincentives. Mr. Quesnel stated that in terms of monitoring and evaluation, project management has been developed extensively, but this was insufficient when elevating project results to other levels. The development community will have to retool itself, because the whole system of project management is subject to external factors. At the strategic level it is important to act in partnership, for which joint evaluation is essential. Three basic principles exist: (i) ownership of developing countries and projects must be demand driven; (ii) investment is essential for institutional capacity building; and (iii) the need for proper policy. Ms. Chia of Malaysia stated that ODA was increasingly a scarce commodity and it must be made demand-driven so as to reach development priority areas. The donor community should also examine institutional aspects and awareness should be raised concerning the need for strategic and policy-level evaluation. Mr. Kamelgarn of France stated that evaluation had to date been conceived to serve the needs of donors. If evaluation is now to be also utilized by partner countries, their own demands in terms of accountability and learning need to be identified. The Chair highlighted the need for greater transparency and accountability on the part of donors and beneficiaries. He thanked Mr. Setiawan for his keynote speech as a departure point for discussion in other sessions and closed the opening session. ----- 2. First Session Agenda 1: Monitoring & Evaluation in a Management System of International Development Goals-- "Ongoing Work of Monitoring & Evaluation on MDGs" "Perspectives on Monitoring and Evaluation of MDGs" Dr. Fadzai Gwaradzimba, Senior Evaluation Advisor, Evaluation Office, United Nations Development Programme The challenges and opportunities for Millennium Development Goals (MDGs) are varied. What is unique about the MDGs is the time-bound element to them and that they shift thinking away from the input model. The eight goals that have been set encourage donor and partner countries to form compacts and coordination. Mutual accountability between donors and stakeholders is also important. The MDGs also provide an opportunity for the creation of common global development frameworks. The issue of transparency is vital to MDGs as is the formulation of realistic national goals and monitoring targets. MDG Reports set the benchmarks for analysis and provide a public affairs document to serve as a catalyst for galvanizing action and national debate on setting priorities. They also provide a country-level instrument for monitoring. Evaluation questions include value addition, content and quality, country ownership, capacity issues, advocacy and dissemination strategies and alignment and linkages with other national development frameworks. The implications for the UN system for MDG Reports include the need to keep reports brief and visually appealing. A common framework is an important issue for reporting on progress towards poverty eradication and the MDGs and better overlap and coordination is required between the teams preparing the PRSP and MDG Reports. Other challenges include a global commitment to finance MDGs, transparency and mutual accountability, country ownership, harmonization of efforts, alignment of tools and instruments at a country level, development of evaluation capacity, analysis, and partners・collaboration. A vital issue is to come to a decision as to whether the MDGs identify appropriate indicators. "The World Bank's Treatment of the Millennium Development Goals" Mr. Ajay Chhibber, Director, Operations Evaluation Department, World Bank The MDGs provide a unique opportunity to improve development outcomes. The MDGs also pose challenges. First is the challenge of how to localize the MDGs. A further challenge is how to prioritize sectors. Progress in these sectors is vital to achieving MDG outcomes. Then there is the challenge of how strategies focused on outcomes can be effectively monitored. There are data issues, definitional issues, measurement issues, and process issues that all require a response. The Bank's sector strategies place the MDGs alongside other sector goals in a broader developmental framework. Furthermore, the Bank's sector strategies increasingly recognize the multisectoral determinants of outcomes, but they do not yet provide guidance on how to develop multi-sectoral strategies to achieve specific outcomes in various country settings. The importance of cross-sectoral strategies derives from the influence of several sectors on each MDG outcome. Bank-supported global programs broadly support the MDGs but have had limited influence on developed country policies and their management poses challenges for the Bank. The Bank lacks leverage on developed country policies but stronger advocacy would help. Developing countries are, for the most part, participants rather than full partners. The MDGs draw attention to the multi-sectoral determinants of outcomes. The Bank increasingly recognizes these interrelationships but it now needs to take the next step and develop and implement cross-sectoral strategies to help achieve intended outcomes. Developing multisectoral strategies will require effective coordination to design and implement outcome-based, cross-sectoral country strategies. Achieving MDG outcomes by 2015--and sustaining them beyond 2015--will require a break from historical trends in a number of countries. Signing on to global targets without determining the priority to be attached to individual targets in specific circumstances, or developing more feasible localized targets, is risky for donors and developing countries alike. The Bank needs to define the objectives and targets of its country programs with greater specificity based on national targets, set priorities and make the necessary trade-offs. "DevInfo: A User-friendly Software Available for Joint Data Collection" Mr. Quesnel, UNICEF Mr. Quesnel provided an introduction to DevInfo, a software program to manage a database. In DevInfo all data is compiled into one system, which allows for easy retrieval of data. DevInfo is based on user-defined indicators, sectors, themes, etc., and can be easily adapted to local specifications. There are now more than 80 countries where DevInfo is operational. The DevInfo software demonstrates that collaboration already exists at the global level and would be very useful to joint evaluation in the future. Discussion This session was chaired by Prof. Ryokichi Hirono, Professor Emeritus of Seikei University. Mr. Mehrotra of India thanked the presenters for their informative presentations. He welcomed the progress made in compiling a comprehensive reporting framework, but noted that there was a danger of moving away from the explicit points decided at the Monterrey Conference (*1). He mentioned that the larger share of investments to achieve the MDGs were being made by developing countries themselves--as compared to external assistance from multilateral or bilateral agencies. While there was focus to report the progress made by developing countries in achieving the MDGs, there was no concomitant country-level reports for developed countries with regard to the achievement of goals and objectives that they are committed to on issues such as ODA, debt relief and market access. Mr. Mehrotra urged efforts to report effectively and comprehensively on Goal 8 of the MDGs. (*2) (*1) The International Conference on Financing for Development, so-called Monterrey Conference, was held from 18–22 March 2002 in Monterrey, Mexico. The UN-hosted conference adopted the Monterrey Consensus, which declared that efforts would be made to effectively utilize development resources including domestic savings, trade and investment, and ODA. (*2) Goal 8 of the Millennium Development Goals (MDGs) is to "Develop a global partnership for development" with a target of addressing the special needs of the least developed countries. Mr. Sheikh of Pakistan noted the comment of the UNDP presenter that some countries are off track to achieve the MDGs by 2015. He asked what the UNDP was doing to put such countries back on track. Another question raised was why countries could not have ownership of the MDGs and other targets. Ms. Menon of the UNDP stated that the overarching framework for development is the MDGs and the Millennium Declaration. Attention needs to be focused now on each country's own declaration and its own leadership in setting development strategies. Ms. Menon asked how the countries present were showing leadership in their own countries and if any efforts were being made for South-South cooperation. Ms. Gwaradzimba responded that costing was very important to accurately grasp the needs in achieving MDGs. Once needs are quantitatively understood, it would provide more leverage for recipient countries to discuss concrete issues with development partners. The most important thing is for each country to set its own goals and targets. Ownership is seriously compromised if goal-setting and other activities are left to outside organizations such as UNDP and the World Bank. Mr. Chhibber responded that it had been pointed out that Goal 8 was difficult to monitor. There are non-governmental institutions that are also looking at the issue of evaluating Goal 8 of the MDGs. Concerning the question of the PRSP framework, the World Bank is conducting an evaluation of the PRSP with the IMF, which will be completed early in 2004. One of the questions being addressed in the evaluation of the PRSP is how realistic is the link between the targets that have been set and financing frameworks. Mr. Quesnel stated that in international development work, all money flowed from donor countries. Basically, the donors have stated official development policies and the governing body for this is peer review through the OECD. A significant issue is the question of where the money goes, a question that is increasingly being asked by taxpayers in donor countries. Next, the boards of international financial institutions (IFIs) are led by evaluations in making decisions, which prompts major donors to provide funds in large amounts through these IFIs. The next grouping is civil society, which is attracting increasingly large sums of funding from donors. A fourth grouping is the UN system, which in recent years has not been attracting as much funding due to its lack of effective evaluation. The final grouping in the system is the countries themselves, which receive the smallest amount of funding, due to a lack of trust and lack of confidence in their capabilities. This is why monitoring and evaluation need to be focused on the demand side in recipient countries. Mr. Tomimoto of JICA explained that JICA is developing indicators for monitoring donor practices. In October 2003, a seminar on aid effectiveness was held jointly hosted by Japan, the UK and Vietnam. Mr. Lundgren of the OECD referred to the issue of Goal 8, noting that the OECD collects data on ODA flows and debt relief. The OECD also implements peer reviews and its efforts fit in with some of those of the UN system. Mr. Kamelgarn of France referred to the issue of joint ventures on aid effectiveness and urged for a focus on Goal 8 of the MDGs and its evaluation. Mr. Pibulsonggram of Thailand urged the need for enhanced South-South cooperation. Thailand is attempting to be an active development partner and this can be a good means of working with donors and other recipient countries. So far Thailand is actively promoting Goal 8, through collaboration with its neighboring countries. Mr. Bountheuang Mounlasy of Lao PDR explained that Lao PDR was currently working on its own PRSP, although it was being compiled under a different name, which will be submitted in the near future. In order to materialize ownership at the country level, capacity must be built in the governments of the recipient countries. Once ownership is achieved at the country level, efforts could be made to work towards achievement of Goal 8 of the MDGs. He stressed the importance of aid coordination and management. Ms. Chia of Malaysia stated that her country considers the MDGs an excellent framework for a multi-sectoral approach. However, each country must design its own policies for development on the basis of its own unique circumstances. The MDGs do provide a coherent framework with a multi-sectoral approach. Malaysia now offers a technical cooperation program in the spirit of South-South cooperation to provide human resource development in the form of training in Malaysia. Malaysia is therefore seeking to enhance partnership with donor countries and agencies to move forward with South-South cooperation. Mr. Mehrotra of India pointed out that the concern on market access and trade-related issues were much greater than that on ODA issues. Mr. Setiawan stated that strategies to achieve MDGs should be matched to conditions in partner countries. He suggested that strategy for the MDGs should be divided in two parts: the national level and the global level. Mr. Quesnel noted that one of the main motives in providing ODA from the point of view of taxpayers was poverty reduction. He explained that one of the issues attracting press attention was the increasing number of unstable and conflict countries, which reduced confidence in the efficacy of ODA from the perspective of the taxpayer. To avoid losing taxpayer confidence collegial efforts should be made to monitor aid effectiveness. Concerning the quality of data collected through DevInfo, Mr. Quesnel acknowledged that although questions concerning quality did exist, the important point to note was that a database did exist in the form of DevInfo, and it was important to build on the data that already existed. Mr. Chhibber of the World Bank stated that for most countries basic data did exist and it would be very valuable to make that data available in a comprehensive manner. The Chair highlighted that in terms of measuring performance of MDGs, while data should be provided by developing countries themselves, there is an urgent need to improve data quality. He explained that in the past it was not the practice of the Government of Japan to share evaluation reports with development partners. He suggested that the second session discussions should be based on historical awareness of what progress has already been made, and what still remained to be done. Prof. Hirono then adjourned the first session. ----- 3. Second Session Agenda 2A: Joint Evaluation 1 Group 1 "Key Issues of Joint Monitoring and Evaluation in Asia--at Country and Sector Program Level" Prof. Hiromitsu Muta, Professor, Tokyo Institute of Technology The objective of ODA evaluation is to secure accountability and transparency in both donor and recipient countries. ODA evaluation plays a role in assisting performance management. Ongoing evaluation and ex-ante evaluation is essential to improve ODA quality, in addition to ex-post evaluation. ODA evaluation has been conducted mainly by donors; however, it is now important to conduct joint monitoring and evaluation at the country and sector level. Greater emphasis should also be placed on results. Another trend is the one from individual projects to the sector level. It is difficult however, in terms of methodology, to identify which outputs and projects have been effective in achieving goals on the program or sector level. The targets therefore should be made clear on a sectorwide or nationwide level. Effective data collection and a method for providing feedback are also required. Mr. Daniel Kamelgarn, Head of the Development Evaluation Unit, Ministry of the Economy, Finance, and Industry, France Mr. Kamelgarn reported on the results of the 撤artners in Development Evaluation・workshop held in Paris in March 2003. The main topics discussed were: (i) evaluation practices according to community; (ii) current issues and challenges; (iii) accountability; and (iv) enhancing collaboration and evaluation capacity. The workshop in France concluded with a discussion on enhancing collaboration and evaluation capacity, and reciprocity and partnership were two keywords in these discussions. The time has come to move from the evaluation of aid to the evaluation of development. The main lessons learned from the workshop were: (i) the shared desire for dialogue; (ii) the demand for reciprocity and partnership; and (iii) the role of evaluation in the promotion of aid. Discussion This session was co-chaired by Mr. Mitsuhiro Wada, Director, Research and Programming Division, Economic Cooperation Bureau, MOFA and Mr. Hans Lundgren, Head, Evaluation Section, Review and Evaluation Division, Development Co-operation Directorate, OECD. Mr. Amornchewin of Thailand asked a question about impact evaluation at the program level. Prof. Muta acknowledged that currently not much consideration was given to impact at the sector or program level. It is important to establish organized programs and ensure that final targets are considered at the program level. This has an impact on individual project considerations, and it could be the case that current projects are not suitable to be considered at the program level. Mr. Kiyohara of MOFA provided a brief introduction to MOFA's evaluation activities. Mr. Wada added that MOFA's evaluation activities were in a constant state of evolution. Mr. Wada posed a question concerning feedback of joint evaluation and asked if current practices were adequate. Another issue to consider is that of data collection. Evaluation costs are also an important consideration and methods need to be sought to reduce costs. Mr. Wada asked participants to consider the issue of civil society involvement. Mr. Santos of the Philippines stated that more work needed to be done in formulating methodologies for joint evaluation. Existing methodologies have room for improvement. Mr. Mehrotra of India pointed out that while it was perfectly acceptable and useful to have a joint monitoring and evaluation at the project level with an external development assistance provider, the following three aspects, at the country and program level, needed to be considered: (i) the involvement and participation of external development assistance providers; (ii) whether it was appropriate to expect a recipient country to go through a joint evaluation at a program level; and (iii) harmonization issues between development providers. Ms. Chia of Malaysia asked participants to consider if it would be preferable for the developing countries to develop a framework for evaluation that would be acceptable to donor countries and that would avoid the need for multiple evaluations that would overlap. Mr. Quesnel of UNICEF asked to what extent Japan was utilizing external evaluation. He noted that external evaluation tended to take a larger picture than internal evaluation, and that external evaluators could be used to challenge existing government projects. He also asked how external evaluations could become a part of domestic discussions on program effectiveness. Regarding MOFA's practices in evaluation, Mr. Kiyohara explained that MOFA established a "wise men" committee in 2001 and has incorporated their comments into implementation bodies in the Ministry. Prof. Muta addressed the issues of internal vs. external evaluation, noting that for the independent evaluator the problem was that there was little knowledge about the project, whereas the people implementing the project had extensive knowledge. He suggested that it may be ideal to initially undertake internal evaluation in a detailed manner, the results of which could be further evaluated independently. Through the practice of evaluation, joint methodologies can be formulated, and ideally in the future it is hoped that recipient countries will implement joint evaluation themselves. Mr. Lundgren noted that guidance on joint evaluations had been developed, and that there was also a generally accepted terminology with agreed definitions of basic evaluation criteria. However, each sector specific evaluation needs to have its own framework containing the most relevant questions in the particular context. Mr. Kamelgarn stressed the importance of joint evaluation on a sector or program level, implemented by partner countries themselves. Mr. Chhibber raised the question of from where and which point it would be best to start to consider joint evaluation. Frameworks and methodologies for joint evaluation are essential and an important achievement of the workshop would be to agree to further consider in detail how to tackle issues of frameworks, methodologies and feedback. Ms. Menon of the UNDP highlighted the problem of differences in procedures and requirements of evaluation on the country level. She asked whether it would be useful for developing countries to formulate a framework for evaluation, given that the framework of the DAC had not been taken up to a great degree. She suggested that a framework for evaluation should be developed in the spirit of the Rome Declaration based on country leadership. This workshop could agree on preliminary steps to move work on evaluation forward. The purpose of joint monitoring and evaluation should be to contextualize ODA implementation on the country level. An official from JBIC stated that JBIC gets expert opinions in the country of implementation, and was seeking to focus evaluation on the program or sector level. JBIC seeks to be flexible in formulation of methodologies. The importance of flexibility to ensure adaptability and effi- ciency in evaluation activities was stressed by other participants. The need to engage civil society in evaluation was also raised. Mr. Quesnel noted that when development assistance was originally launched in the 1960s it was based on national programs in place since the end of World War Two. Since then assistance has filtered down to ad hoc project level assistance and there is now a need to move back towards the program and country level. He suggested that rather than referring to "ownership" by developing countries, the terminology should refer to "leadership" by developing countries, and these countries should be expected to take the lead in their own national aid programs and methodologies for evaluation. To date, evaluation has been the tool of the supply side and this situation needs to be reversed to give evaluation to the demand side. Mr. Santos stated that the challenge was to find means of ironing out differences in the methodologies evaluation. Donors and recipients should treat each other as equal partners. Ms. Chia stated that based on experience with JBIC, elements of evaluation are being incorporated into programs in an ongoing manner. She mentioned the importance of scaling-up from a project level to a program level. Mr. Pibulsonggram of Thailand stated that it was important come up with a joint evaluation system based on a common framework that could be formulated by recipient countries rather than donor countries. Mr. Wada thanked participants and closed the session. Group 2 "Key Issues of Joint Monitoring and Evaluation in Asia--at Project Level" Mr. Graham M. Walter, Director, Operations Evaluation Division 1, Operations Evaluation Department, ADB Mr. Walter explained the importance of monitoring and evaluating ODA programs. Performance management is the evaluation of results from an ODA project and using performance information makes for better performance-enhancing decisions, leading to a number of benefits. The ADB evaluates projects it funds before, during, and after these projects are complete, with the results from all these, and not just post-project evaluations, being used as feedback for new projects. Performance assessment criteria used in monitoring and evaluation are relevance, effectiveness, efficiency, sustainability, as well as institutional development and other impacts. Issues to consider in respect to monitoring and evaluation include the problems of attribution, unclear specification of expected results, lack of data, and harmonization of systems for joint evaluation. In addition, questions to consider include who should participate in joint evaluations, how the ADB's system differs from others, whether harmonization can be achieved while meeting individual internal needs, and whether transaction costs can be lowered. Mr. Matsuzawa of JBIC explained that in the case of partner country participation for JBIC project evaluation cases, ex-ante evaluations and monitoring take place between the two parties, followed by a compilation of the project completion report. Next, the JBIC and its partner conduct a post evaluation. Feedback is then given through the sharing of evaluation reports and holding a feedback seminar. The partner's role in this phase is to follow up on the recommendations and disseminate the feedback to beneficiaries. Mr. Tomimoto of JICA noted that JICA had recently become an independent administrative institution and explained that this has called for increased transparency and a result-oriented organization. For the last 30 years, JICA has undertaken various types of evaluation, always with the principle of respecting the ownership of partner countries and undertaking joint evaluations with them. JICA provides technical assistance in order to promote capacity building and puts an emphasis on sustainability. JICA has recently been thinking about ways to invite other partners into our joint collaboration. Finally, through the process of re-organizing itself, JICA has adopted the project evaluation approach. Discussion This session was co-chaired by Mr. Ikufumi Tomimoto, Deputy Managing Director, Planning and Evaluation Department, JICA and Mr. Takeo Matsuzawa, Deputy Director General, Project Development Department, JBIC. Mr. Matsuzawa opened the discussion phase of the session, inviting the participants to make comments or ask questions from the perspectives of capacity of partners, partnership and ownership (who is doing what), harmonization, and a results-based approach (project purpose linked to partner goals). Mr. Piter Pangaribuan of Indonesia stated a lack of preparation was the primary cause for Indonesia being unable to attract any externally funded projects for FY2003. In addition, he asked for guidance from the ADB on how it rated the level of satisfaction for a project. Mr. Walter responded that the ADB had also seen project readiness problems in Indonesia. Further, he stated that implementation performance and development objectives were the two areas used to evaluate success, with the latter often being the area mostly judged less successful. Mr. Myint Theong of Myanmar expressed his appreciation to the Government of Japan for its assistance in aid projects in his country. He then asked the ADB for clarification on who was to be involved in project evaluation with respect to civil society. Mr. Bingjian Kan of China also asked Mr. Walter for his opinion on who should be the basic participants in joint evaluations. Mr. Walter said that civil society was a large body, so all relevant civil society stakeholders needed to be included in an evaluation. Mr. Tomimoto responded that more and more civil society groups had been critical of Japanese ODA. As a result JBIC has been re-working its guidelines to allow for more effective participation from stakeholders and had also been encouraging local participation in its projects. Ms. Thelma Manuel of the Philippines pointed out that there was a tendency for one group to dominate an evaluation and encouraged a more equitable method for evaluations. She further stated that some countries had problems coming up with counterpart funds and this would pose a problem also for joint evaluations. Mr. Vum Lun Mang of India endorsed the importance of a results-based approach in development. An internal evaluation of total external assistance received by India had resulted in a decision to restrict the acceptance of bilateral development cooperation to six countries. This will result in improved aid harmonization and greater efficiency of aid. Recognizing the importance of civil society organizations (CSOs) in the development process, the Government of India decided that other bilateral partners could route their funds through CSOs. Since joint evaluation is an evolving process, the involvement of various stake holders provides valuable input into evaluation. Ms. Nasanbuyan Lodoidamba of Mongolia stated that it would be very useful if donors allowed related government authorities to join evaluation groups. Mr. Tomimoto then commented that evaluations must be aligned and harmonized. He stated that there was a trend to simplify the delivery of aid. As capacity is very important, for that reason JICA was providing much training aid. Ms. Nanda Alahakone of Sri Lanka noted that donor participation was much stronger at the country level than at project level. She also stressed that a logical framework be adopted in devising evaluation frameworks. Mr. Nik Adnan Nik Abdullah of Malaysia said it was important to note that there had not been attempts to harmonize various donors・reporting formats. With respect to ownership by partners, Malaysia has been trying to get the involvement of civil society actors in the implementation of projects, but instead this was mostly being done at the technical committee level. Mr. Dai Thang VU of Vietnam stated that since the donor community had returned to Vietnam in the last ten years, monitoring and evaluation had played an integral part in the project cycle, but it was still limited and needed to be expanded. In addition, he pointed out that the Government of Vietnam, the ADB, JBIC and the World Bank had started to initiate harmonization. Dr. Ismail Zabihullah of Bangladesh said there was always a gap between stakeholders in respect to joint evaluations and the challenge was to bridge the gap through better understanding. Further, Dr. Zabihullah stated his support for the themes capacity of partners, partnership and ownership, harmonization, and results, but added that the most important area not discussed was political commitment. In concluding Group 2 session, Mr. Walter reiterated the points made by the various participants. Mr. Tomimoto then stated that this was the third such ODA workshop and noted his expectation that a comprehensive framework be developed in order to achieve concrete developments in the future. Finally, Mr. Matsuzawa outlined the discussion of the participants in relation to their points of view on capacity of partners, partnership and ownership, harmonization, and results. ----- 4. Third Session Agenda 2B: Joint Evaluation 2 "Future Work of Joint Monitoring and Evaluation" Mr. Lundgren of the OECD presented the results of discussions in the first working group. He extracted a number of points from the presentation and the ensuing discussion: ・Participants noted the difficulty in individual attribution from joint efforts; ・There was general agreement on the need to develop a common joint evaluation framework; ・Many noted the importance of moving to higher level frameworks beyond a set of projects; ・PRSPs or other similar overall strategies can provide a useful overall framework for some ・Joint sector evaluation may not be appropriate in all circumstances, notably when the external donors are providing only a limited amount of resources to the sector; ・Leadership of partners in the evaluation process is critical for ownership and more effective follow up action; ・Participants agreed on the need for attention to capacity building issues in evaluation; and ・Some emphasized the importance of addressing "the bigger picture", including the Monterrey Consensus and the MDGs including Goal 8. In respect to Group 2, the co-chairperson Mr. Matsuzawa of JBIC said that the group had heard a report on the ADB's activities in joint monitoring and evaluation, followed by a presentation on this same topic by JBIC. Capacity, harmonization, ownership and partnership, and a results-oriented approach were the main themes for discussion in this group. With respect to capacity, it has developed on an individual level but it is still required on an institutional level. The different procedures of various donors are a burden on recipient countries, thus highlighting the need to pursue harmonization. For ownership and partnership, many countries have already established joint monitoring systems but this is an area that still needs to be expanded. Finally, there is also a need to promote and enhance a results-oriented approach in ODA. It was added that in light of this being the third workshop held on ODA evaluation, it was now time for concrete actions to be taken. Discussion This session was chaired by Prof. Ryokichi Hirono, Professor Emeritus of Seikei University. Mr. Tran of Vietnam stated that the challenge to pursuing greater evaluation was to discover the methodologies for evaluation and mutual understanding. Finally, he stated, while Vietnam welcomed joint evaluation, the country strongly recommends that the donor community provide capacity building assistance and consider it a pre-condition for joint evaluation. Mr. Mounlasy of Laos stated that his country had created a mechanism for joint evaluation, set up following the Monterrey Meeting with the assistance of JICA. He then said that Laos was experiencing few difficulties in its joint evaluation and it was looking forward to moving ahead with joint evaluation. Mr. Pibulsonggram of Thailand stated his support for the co-chair's remarks with respect to now being the time for action. Within the ASEAN framework, Thailand is trying to bridge the gaps among the member countries through the sharing of its expertise in trade, agriculture, infrastructure, tourism, and human resources development. Thailand has now graduated from being an aid receiving country and was now a country who wished to learn from other donor countries・experiences. Finally, Mr. Pibulsonggram announced that Thailand wished to host a seminar the following year along with other donor countries on evaluation capacity building. Mr. Chhieng of Cambodia explained that his country was following the CD mechanism and saw it as a forum for policy dialogue between donors and the developing community. Mr. Chhieng also said there was a need for Cambodia to enhance its capacity building with a more systematic and coherent base, as well as a need for enhancing capacity in an evaluation framework Ms. Than Than Lin of Myanmar stated that her country was receiving aid for six projects, and noted that a technical cooperation program from JICA had been playing a vital role in the socioeconomic development of the country. She further noted that other international bodies were assisting Myanmar through aid programs, such as UNDP's Human Development Initiative Program, which was mostly focused on primary healthcare. She then thanked, on behalf of Myanmar, the international community for the aid programs it had been providing. Mr. Setiawan of Indonesia said that one important aspect in joint evaluation at the country and individual level was a common understanding, which helped to facilitate future projects. He further said that joint monitoring evaluation needed to be started at the project level. Mr. Walter of the ADB raised the possibility of making projects less ad hoc and instead moving cooperating from a project-to-project basis to a more continual level of cooperation. He further said that based on Group 2's discussions, a need was raised to harmonize reporting requirements in country project review missions. Finally, Mr. Walter said he was pleased there might be an initiative for joint evaluation in Thailand, and also that Thailand was willing to host a donors meeting. Mr. Tomimoto of JICA expressed JICA's willingness to support local initiatives. He added that JICA viewed capacity building as its foremost aim in funding projects and hoped to see advancements in this area. Mr. Tomimoto then said that there should be a message from Asia that its experiences could be shared with Africa. Mr. Hirono reiterated the importance of extending Asian experiences to Africa. Ms. Chia of Malaysia stated that her country had a bilateral technical cooperation program, which accepted about 1,000 international participants a year. Ms. Chia then noted her country's support of TICAD and said Malaysia would host the next Asia-Africa Forum. This technical assistance program is premised on "win-win partnerships." Mr. Quesnel of UNICEF proposed that good practices in monitoring and evaluation needed be documented so as to promote their use. He further proposed that South-South visits be encouraged to facilitate exchanges. Mr. Mehrotra of India said that his country's government had decided to restrict bilateral development cooperation at the government level to six countries. This is expected to result in improved aid harmonization and greater efficiency of aid. As to those countries from which India will no longer accept bilateral development assistance, Mr. Mehrotra stated that India was encouraging them to route their assistance through NGOs or multilateral aid agencies. Mr. Pibulsonggram of Thailand told the meeting that Thailand and New Zealand had set up the Mekong Institute (MI). He then stated that he would suggest to the MI to set up a training course on joint evaluation. ----- 5. Closing Session Closing Address Approval of Chairperson's Summary Prof. Hirono announced that based on the discussions, he along with the assistance of others, produced a Chair's Summary outlining the aims and outcomes of the workshop. Prof. Hirono then proceeded to read the summary to the participants. The participants signified their approval by a round of applause. Closing Remarks Mr. Noriki Hirose, Deputy Director-General, Economic Cooperation Bureau, MOFA Mr. Hirose expressed his heartfelt gratitude to the representatives of Asian partner countries, and bilateral and multilateral development institutions for concluding the two-day workshop successfully with the adoption of the Chairperson's Summary. He also commended Prof. Hirono, the chair of this workshop, and co-chairs of sub-groups for their devoted efforts. Mr. Hirose concluded by reiterating the importance of donor-partner cooperation for strengthening management systems. He noted that this year Japan revised its ODA Charter, in which more emphasis was placed on evaluation as a tool to improve aid effectiveness. Japan will continue to work with partners towards an effective aid management system. Mr. Hirose expressed the hope that Asian countries and bilateral and multilateral organizations would share the findings of this workshop to realize greater aid effectiveness. ----- Appendix I: Program 12th November Wednesday 11:00-12:30 Opening Session Chair: Prof. Ryokichi Hirono, Professor Emeritus of Seikei University Opening Remarks Mr. Hajime Furuta, Director-General, Economic Cooperation Bureau, MOFA Keynote speech "National Policy on Project Monitoring and Project Performance Evaluation" as a Management Tool of ODA in Indonesia Presenter: Mr. Benny Setiawan Director, Directorate of Development Funding Monitoring and Evaluation, BAPPENAS, Republic of Indonesia 12:30-14:30 Lunch Break 14:30-17:00 First Session *Coffee Break at around 15:30 Chair: Prof. Ryokichi Hirono, Professor Emeritus of Seikei University Adoption of the Agenda Agenda 1: Monitoring & Evaluation in a Management System of International Development Goals--"Ongoing Work of Monitoring & Evaluation on MDGs" Presenter: Dr. Fadzai Gwaradzimba Senior Evaluation Advisor, Evaluation Office, United Nations Development Programme Mr. Ajay Chhibber Director, Operations Evaluation Department, World Bank 18:00 Reception hosted by Mr. Furuta, Director-General, MOFA 18:00 13th November Thursday 10:00-12:30 Second Session (divided into sub-groups) Agenda 2A: Joint Evaluation 1 *Coffee Break at around 11:00 Group 1 Co-chair: Mr. Mitsuhiro Wada Director, Research and Programming Division, Economic Cooperation Bureau, MOFA Mr. Hans Lundgren Head, Evaluation Section, Review and Evaluation Division, Development Co-operation Directorate, OECD "Key Issues of Joint Monitoring and Evaluation in Asia--at Country and Sector Program Level" Presenter: Prof. Hiromitsu Muta Professor, Tokyo Institute of Technology Mr. Daniel Kamelgarn Head of the Development Evaluation Unit, Ministry of the Economy, Finance, and Industry, France Group 2 Co-chair: Mr. Ikufumi Tomimoto Deputy Managing Director, Planning and Evaluation Department, JICA Mr. Takeo Matsuzawa Deputy Director General, Project Development Department, JBIC "Key Issues of Joint Monitoring and Evaluation in Asia—at Project Level" Presenter: Mr. Graham M. Walter Director, Operations Evaluation Division 1, Operations Evaluation Department, ADB 12:30-14:30 Lunch Break 14:30-16:00 Third Session Chair: Prof. Ryokichi Hirono, Professor Emeritus of Seikei University Agenda 2B: Joint Evaluation 2 "Future Work of Joint Monitoring and Evaluation" Presenter: Chairs of sub-groups at second session 16:00 Coffee Break 16:30-17:30 Closing Session Chair: Prof. Ryokichi Hirono, Professor Emeritus of Seikei University Approval of Chairperson's Summary Closing Remarks Mr. Noriki Hirose, Deputy Director-General, Economic Cooperation Bureau, MOFA 17:30 Closing ----- Appendix II: List of Participants Bangladesh Mr. MD. Ismail Zabihullah Additional Secretary, Economic Relations Division, Ministry of Finance Mr. Delwar HOSSAIN Assistant Chief, Economic Relations Division, Ministry of Finance Brunei Darussalam Mr. Shaikh Fadilah Ahmad Deputy Director, Multilateral Economics Department, Ministry of Foreign Affairs Mr. Mohammad Alias Serbini Minister Counselor, Embassy of Brunei Darussalam Cambodia H.E. Dr. Yanara CHHIENG Deputy Secretary General, Council for the Development of Cambodia Mr. Yutha POR Chief, Bilateral Cooperation Division, External Finance Department, Ministry of Economy and Finance China Mr. Bingjian KANG Deputy-Director, Department of International Trade and Economic Affairs, Ministry of Commerce France Mr. Daniel KAMELGARN Head, Development Activities Evaluation Unit, Treasury Department, Ministry of the Economy, Finances and Industry India Mr. Vivek MEHROTRA Joint Secretary, Department of Economic Affairs, Ministry of Finance Mr. Vum Lun Mang V Deputy Secretary, Department of Economic Affairs, Ministry of Finance Indonesia Mr. Benny SETIAWAN Director, Directorate of Development Funding Monitoring and Evaluation, National Development Planning Agency (BAPPENAS) Mr. Yan Piter Pangaribuan Head, Sub Directorate Monitoring and Evaluation of Multilateral Cooperation, National Development Planning Agency (BAPPENAS) Laos Dr. Bountheuang MOUNLASY Director General, Department of International Cooperation, Ministry of Foreign Affairs Mr. Sithong CHITNHOTHINH Director General, Asia-Pacific and Africa Department, Ministry of Foreign Affairs Malaysia Ms. Patricia CHIAYoon Moi Director, External Assistance Section, Economic Planning Unit Mr. Nik Adnan NIK ABDULLAH Principal Assistant Director (Bilateral), External Assistance Section, Economic Planning Unit Mongolia Ms. Suvd BADARCH Director, Policy Planning, Information and Monitoring Department, Ministry of Foreign Affairs Ms. Nasanbuyan LODOIDAMBA Assistant Director, Department of Economic Cooperation Management and Coordination, Ministry of Finance and Economy Myanmar Ms. Daw Than Than Lin Deputy Director, Foreign Economic Relation Department, Ministry of National Planning and Economic Development Dr. Myint THEONG Deputy Director, Central Medical Store Depot, Department of Health, Ministry of Health Nepal Dr. Madhav Prasad GHIMIRE Joint Secretary, Foreign Aid Coordination Division, Ministry of Finance Mr. Khaga Nath ADHIKARI Under Secretary, North and South East Asia Division, Ministry of Foreign Affairs Pakistan Mr. Muhammad Asif SHEIKH Chief, Public Investment Programming and Development Policies, Ministry of Planning and Development Mr. Muhammad Arshad SULTAN Section Officer, Economic Affairs Division, Ministry of Economic Affairs and Statistics Philippines Mr. Augusto B. SANTOS Deputy Director-General, National Economic and Development Authority (NEDA) Ms. Thelma MANUEL Chief Economic Development Specialist, National Economic Development Authority Republic of Korea Mr. Yong Duck LEE Director, Evaluation Team, Korea International Cooperation Agency Singapore Mr. Chee Wee Steven PANG First Secretary, Singapore Embassy Sri Lanka Mr. Jayamaha Hitihamilage Jayatilaka JAYAMAHA Additional Director General, Department of External Resources, Ministry of Policy Development and Implementation Ms. Nanda ALAHAKONE Director, Performance Evaluation Unit, Monitoring and Progress Review Division, Ministry of Policy Development and Implementation Thailand Mr. Pradap PIBULSONGGRAM Director-General, Department of Technical and Economic Cooperation, Ministry of Foreign Affairs Mr. Banchong AMORNCHEWIN Chief, Japan Sub-Division, External Cooperation Division I, Department of Technical and Economic Cooperation, Ministry of Foreign Affairs Mr. Kawin THANGSUPANICH Director, State Enterprise Project Loan Division, Public Debt Management Office, Ministry of Finance Vietnam Mr. Manh Cuong TRAN Chief, International Financial Institution Division, Foreign Economic Relations Department, Ministry of Planning and Investment Mr. Dai Thang VU Expert, Foreign Economic Relations Department, Ministry of Planning and Investment Asian Development Bank Mr. Graham M. WALTER Director, Operations Evaluation Department, Division 1 World Bank Mr. Ajay CHHIBBER Director, Operations Evaluation Department IMF Ms. Saika KIN Programs Officer, Regional Office for Asia and the Pacific, International Monetary Fund OECD (DAC) Mr. Hans E, LUNDGREN Head, Evaluation Section, Review and Evaluation Division, Development Co-operation Directorate UNDP Dr. Fadzai GWARADZIMBA Senior Evaluation Adviser, Evaluation Office Ms. Saraswathi Menon Director (designate), Evaluation Office UNICEF Mr. Jean S. QUESNEL Director, Evaluation Office ----- Appendix III: Profiles of Senior Representatives Bangladesh Mr. MD. Ismail Zabihullah Additional Secretary, Economic Relations Division, Ministry of Finance Mr. Zabihullah holds a Bachelor of Law degree as well as a Master's degree in English. Before taking up the current position, Mr. Zabihullah served as Deputy Commissioner for the City of Faridpur, Private Secretary to Honorable Communications Minister, among numerous other notable positions. Brunei Mr. Shaikh Fadilah Ahmad Deputy Director, Multilateral Economics Department, Ministry of Foreign Affairs In 1988, Mr. Ahmad received his BA Honors degree in Development Studies from the University of East Anglia in England. He served as Deputy Director in the Multilateral Economics Department for the Ministry of Foreign Affairs, and has held the current position since 2002. Cambodia H.E. Dr. Yanara Chhieng Deputy Secretary General, Council for the Development of Cambodia In 1988, Dr. Yanara received his Ph.D. in Economics. He also holds Master's degrees in Economics as well as Business Management. He became Director General for the Ministry of Planning in 1988. In 1993, Dr. Yanara joined the Council for the Development of Cambodia, where he remains today as Secretary General. China Mr. Bingjian Kang Deputy-Director, Department of International Trade and Economic Affairs, Ministry of Commerce After completing his post-graduate studies at Nankai University's College of Economics in China, Mr. Kang started his career at the Ministry of Commerce in the Department of Human Resource. In 2003, he spent some months at the Executive Bureau of Trade Development, but soon after joined the Department of International Trade and Economic Affairs. India Mr. Vivek Mehrotra Joint Secretary, Department of Economic Affairs, Ministry of Finance Mr. Mehrotra holds two Master's degrees, one in Political Science and the other in Economics. Ever since 1974, Mr. Mehrotra has been a member of the Indian Administrative Service. Before taking up his current position, he served as Secretary to the Government of Haryana, Fisheries Department and a Member of the Sales Tax Tribunal in 2002. Indonesia Mr. Benny Setiawan Director, Directorate of Development, Funding Monitoring and Evaluation, National Development Planning Agency (BAPPENAS) In 1991, Mr. Setiawan graduated from the University of Illinois with a Master痴 degree in Policy Economics. Before taking up his current position of Director of Monitoring and Fund Evaluation for BAPPENAS, Indonesia's National Development Planning Agency, Mr. Setiawan was Director for Politics, Defense, and Security. Laos Dr. Bountheuang Mounlasy Director General, Department of International Cooperation, Ministry of Foreign Affairs Dr. Bountheuang Mounlasy has both a Master's degree and a Ph.D. in Economics. From 1990 to 2003, he was working for the Committee for Investment and Cooperation. Now, he serves as Director General of the Department of International Cooperation for the Ministry of Foreign Affairs. Malaysia Ms. Patricia Chia Yoon Moi Director, External Assistance Section, Economic Planning Unit Ms. Chia graduated from Vanderbilt University in 1985. In 1989, she joined the Economic Planning Unit in the Prime Minister's Department, where she currently holds the position of Director of the External Assistance Section. Mongolia Ms. Suvd Badarch Director, Policy Planning, Information and Monitoring Department, Ministry of Foreign Affairs After completing her studies at the Diplomatic Academy in Moscow, Ms. Suvd joined the diplomatic service in 1972. From 1991 to 1994, she was posted as First Secretary of the Embassy of Mongolia in Moscow. After numerous positions in the Ministry of Foreign Affairs, she took up the current position. Myanmar Ms. Daw Than Than Lin Deputy Director, Foreign Economic Relation Department, Ministry of National Planning and Economic Development Ms. Lin holds a Master's degree in Economics. She joined the ministry in 1983 and became Deputy Director in 2001. Nepal Dr. Madhav Prasad Ghimire Joint Secretary, Foreign Aid Coordination Division, Ministry of Finance Dr. Ghimire earned a Ph.D. in Resource Policy in 1992. From 1997 to 2001, he served as Joint Secretary of the National Planning Commission Secretariat, Ministry of Population and Environment, and Ministry of Home Affairs. In 2001, he joined the Ministry of Finance and has held the current position since 2003. Pakistan Mr. Muhammad Asif Sheikh Chief, Public Investment Programming and Development Policies, Ministry of Planning and Development Mr. Sheikh received a Master's degree in Economics in Pakistan and a Master's degree in Development Economics in the United States. He started his career as Research Officer of the Economic Affairs Division in 1970. Mr. Sheikh also served as Deputy Chief in the Ministry of Commerce from 2001 to 2002. Philippines Mr. Augusto B. Santos Deputy Director-General, National Economic and Development Authority (NEDA) In 1978, Mr. Santos graduated from the University of the Philippines with a Master's degree in Business Administration. He was Chief Economic Development Specialist Assistant at the National Economic and Development Authority in 1988, and working his way up from Director and then Assistant Director-General. In 2000, he attained his current title. Republic of Korea Mr. Yong Duck Lee Director, Evaluation Team, Korea International Cooperation Agency Mr. Lee holds an MBA in International Trade. He joined the Korea International Cooperation Agency in 1991. By 1995, he was Manager of the Research and Survey Division. In 2002, Mr. Lee became Director of the Management Information Service Team, and the following year he was Director of the Training Affairs Division Team II. Singapore Mr. Chee Wee Steven Pang First Secretary, Singapore Embassy In 1999, a year after graduating from Tokyo University, Mr. Pang joined the Ministry of Foreign Affairs. Since 2002, he has been the First Secretary of the Singapore Embassy in Tokyo. Sri Lanka Mr. Jayamaha Hitihamilage Jayatilaka Jayamaha Additional Director General, Department of External Resources, Ministry of Policy Development and Implementation Mr. Jayamaha started his carrier in 1981. He received a M.Phil. in Economic Development in 1986 from the University of Glasgow. In 1997, he became Director of the Japan Division. In 2001, still overseeing the Japan Division, Mr. Jayamaha was appointed Additional Director General in charge of mobilizing ODA from Western countries, the European Union and other multilateral agencies located in Europe. Thailand Mr. Pradap Pibulsonggram Director-General, Department of Technical and Economic Cooperation, Ministry of Foreign Affairs After graduating from Auckland University in New Zealand, Mr. Pibulsonggram joined the Ministry of Foreign Affairs in 1973. His very first post was to Cairo. In 1996, he became Ambassador in Budapest and then Croatia. He also served as Director-General of Department of ASEAN Affairs, Department of Information and Department of Economic Affairs. Vietnam Mr. Manh Cuong Tran Chief, International Financial Institution Division, Foreign Economic Relations Department, Ministry of Planning and Investment Mr. Tran obtained Master's degree in Hydraulic Engineering at the University of Roorkee in India. From 1979 to 1993, he was teaching at Hanoi Water Resources University, and then joined the Ministry in 1993. France Mr. Daniel Kamelgarn Head, Development Activities Evaluation Unit, Treasury Department, Ministry of the Economy, Finances and Industry Mr. Kamelgarn is a graduate of the University of Sorbonne in Paris and of the HEC School of Management. Mr. Kamelgarn has been working for the Agence Francaise de Developpement (AFD) since 1980, and is also Head of the Development Activities Evaluation Unit in the Treasury Department of the French Ministry of Economy, Finance and Industry. ADB Mr. Graham M. Walter Director, Operations Evaluation Department, Division 1 Mr. Walter received his Master's degree in Development Economics from the University of Sussex in the United Kingdom. Mr. Walter joined the Asian Development Bank in 1981, assuming several notable positions. He moved in 1997 to the Operations Evaluation Department, eventually accepting his current position of Director. World Bank Mr. Ajay Chhibber Director, Operations Evaluation Department Mr. Chhibber graduated with a Master's degree from the Delhi School of Economics in 1976 and Ph.D. from Stanford University in 1983. He joined the World Bank and served in a variety of research, policy and operation positions including involvement in the 1986 and 1997 World Development Reports. In 2003, he assumed his present position. IMF Ms. Saika Kin Programs Officer, Regional Office for Asia and the Pacific, International Monetary Fund Ms. Kin graduated from the University of Chiba in 2000. She received her Master's degree in International Development Studies from the National Graduate Institute for Policy Studies in Tokyo. In 2003, she took up her current position in the IMF. OECD Mr. Hans E. Lundgren Head, Evaluation Section, Review and Evaluation Division, Development Co-operation Directorate Mr. Lundgren is Head of the Evaluation Section at the Development Co-operation Directorate and the coordinator of the DAC Network on Development Evaluation. Prior to joining OECD/DAC in 1987, he worked in field operations with the UNDP in West Africa and at the UNESCO headquarters managing the Nordic Trust Fund. He has an MBA from the University of Gothenburg in Sweden, as well as post-graduate degrees in International Relations from the College d'Europe in Belgium, and in International Economics from the Graduate Institute of International Studies in Geneva, Switzerland. UNDP Dr. Fadzai Gwaradzimba Senior Evaluation Adviser, Evaluation Office Ms. Gwaradzimba graduated from University of Oregon with a Master's degree in Public Affairs and International Studies in 1979. She also received a Ph.D. in International Relations in 1992 from the Paul H. Nitze School of Advanced International Studies, Johns Hopkins University. Her first position was Assistant Secretary of the Ministry of Foreign Affairs in Zimbabwe. For the next few years, she worked for various UN agencies such as the United Nations Interim Administration Mission in Kosovo. In 2001, she accepted her current position. UNICEF Mr. Jean S. Quesnel Director, Evaluation Office Mr. Quesnel is currently Director of the Evaluation office of UNICEF and also coordinates the UN Inter-Agency Thematic Group on Evaluation Norms and Procedures. Before joining UNICEF, he was Acting Director General of the Performance Review Branch and held numerous positions during his many years with the Canadian International Development Agency (CIDA). Mr. Quesnel holds degrees in Letters, Philosophy, Business and Public Administration, and Project Management. Japan Professor Ryokichi Hirono Professor Emeritus, Seikei University Chairman of the Workshop Prof. Hirono received his MA in Economics from the University of Chicago in 1959. He began his career as a translator for the Department of State of the United States from 1955 to 1960. He taught at Seikei University from 1961 to 1996 while pursuing his career in universities in foreign countries and international organizations. He served as a consultant in the Asia Development Bank from 1969 to 1970 and from 1972 to 1973, Director of Evaluation Division of UNDP from 1987 to 1989, and as Advisor to the Research Institute of Development in the National University of Australia from 1989 to 1996. He is currently a member of the Policy Planning and Evaluation Division of the UNDP and the ODA Round-Table Conference and Evaluation of Aid Committee of the Ministry of Foreign Affairs. He is also the Director of The Japan Society for International Development and the Global Environmental Forum. Born in 1931. Mr. Hajime Furuta Director-General, Economic Cooperation Bureau, Ministry of Foreign Affairs After receiving a Bachelor's Degree in Law from Tokyo University in 1971, he joined the Ministry of International Trade and Industry. In 1974, he studied at Ecole Nationale d但dministration (ENA) in France. Throughout his career, he served in many important posts including the Executive Director at JETRO's New York office and the Executive Assistant to the Prime Minister. In 1996, he became a Director at the Budget and Accounts Division, Minister痴 Secretariat. Mr. Furuta was responsible for economic cooperation and international trade policy, as a Director-General, between 1998 and 2000. He has held his present position since 2002. Born in 1947. Professor Hiromitsu Muta Professor, Tokyo Institute of Technology Prof. Muta pursued his study of education and received his BA in 1971, MA in 1973 from the University of Tokyo and Ph.D. in 1990 from the Tokyo Institute of Technology. He began his career as a researcher in the Research Institute of the Ministry of Education. Later, he worked as a researcher in various universities, including Stanford University in 1980, and Harvard University in 1981. He has taught at the Tokyo Institute of Technology since 1985. He has served in his current position since 1996 as Professor at the Graduate School of Decision Science and Technology, Tokyo Institute of Technology and also serves as Director of the Center for Research and Development of Educational Technology. He has also been active in various academic societies, including as Vice President of The Japan Evaluation Society, and as a Board Member of The Japan Society for International Development. Born in 1946. Mr. Noriki Hirose Deputy Director-General, Economic Cooperation Bureau, Ministry of Foreign Affairs Mr. Hirose graduated from the Graduate School of Economic, Department of Economics of Doshisha University with an MA in Economics in 1975, joining the Economic Planning Agency in the same year. From 1980 to 1982 he engaged in post-graduate research at the Graduate School of Economics of Princeton University. In 1985, Mr. Hirose was appointed Deputy-Director of the General Affairs Division, Planning and Coordination Bureau of the National Land Agency. From 1986, Mr. Hirose served in various senior positions at the Economic Planning Agency. He has also worked as a Senior Research Fellow at the Institute for International Policy Studies (IIPS) in 1991 and as a Senior Associate Fellow at St. Anthonya's College, Oxford University. Mr. Hirose entered his present position in July 2003. Born in 1948. Mr. Ikufumi Tomimoto Deputy Managing Director, Planning and Evaluation Department, JICA Mr. Ikufumi Tomimoto graduated from the Department of Education of Waseda University, Japan in 1974 with a BA in Social Science. He completed graduate school at the University of Wisconsin-Madison in 1979 with a Master of Public Policy Administration. Mr. Tomimoto has worked at JICA since 1974, when JICA was known as the Overseas Technical Cooperation Agency (OTCA). His work with JICA has taken him to postings in Thailand, the United States and Austria among other locations. Mr. Tomimoto was appointed to his present position in March 2002. Born in 1950. Mr. Takeo Matsuzawa Deputy Director General, Project Development Department, JBIC Mr. Matsuzawa graduated from the Faculty of Economics of the University of Tokyo with a BA in Economics. He joined the Overseas Economic Cooperation Fund (OECF), a forerunner of JBIC, in the same year. From 1979, Mr. Matsuzawa has served the OECF and then JBIC both in Japan and overseas, including Thailand and Bangladesh. From 1988 to 1991 he was seconded from OECF to the Project Finance Division of the Bank of Tokyo. Mr. Matsuzawa has served in his present position since June 2001. Born in 1956. Mr. Mitsuhiro Wada Director, Research and Programming Division, Economic Cooperation Bureau, Ministry of Foreign Affairs Mr. Wada graduated from the Faculty of Law of the University of Tokyo in 1983, entering the Ministry of Foreign Affairs in the same year. He also obtained an MA degree in Asia Studies from the University of California, Berkeley in 1987. Since 1983, Mr. Wada has served in a variety of positions, including at the Management & Coordination Agency (now the Ministry of Public Management, Home Affairs, Post and Telecommunications), and in overseas diplomatic missions in the People's Republic of China, Geneva and Indonesia. Mr. Wada has served in his present position since September 2003. Born in 1960.