Co-chairsf Summary The 11th ODA Evaluation Workshop in Manila on Nov. 26-27, 2012 Co-hosted by Ministry of Foreign Affairs, Japan And National Economic and Development Authority, Republic of the Philippines The 11th ODA Evaluation Workshop was held in Manila, the Philippines, on November 26 and 27, 2012, and was jointly hosted by the Government of Japan and the Government of the Philippines. 1. Opening Session Opening and welcoming remarks were delivered respectively by representatives of the two co-hosts: Mr. Akira Fukushima, Deputy Director-General of the International Cooperation Bureau at the Ministry of Foreign Affairs (MOFA), Japan, and Mr. Rolando Tungpalan, Deputy Director-General of the National Economic and Development Authority (NEDA), the Philippines. This workshop was co-chaired by Mr. Naonobu Minato, Director of the ODA Evaluation Division at MOFA and Mr. Roderick Planta, Director of Project Monitoring Staff at NEDA. Mr. Minato explained the background to this workshop and Mr. Planta outlined the program for the two days of sessions. To open the workshop, Mr. Atsushi Sasaki, Director General of the Evaluation Department at the Japan International Cooperation Agency (JICA) made a presentation describing the outline and features of JICAfs operations evaluation. 2. Session 1: Development of Human Resources for Enhancing Evaluation Capacities In this session, human resource development (HRD) for enhancing evaluation capacities was discussed. First, a presentation was made by Mr. Tara Sapkota, Undersecretary of the Ministry of Finance, Nepal. Mr. Indrasathi Muniandy, Treasurer of the Malaysian Evaluation Society (MES), moderated the discussion that followed the presentation. Discussions by participants and moderation focused around the following themes and points related to human resource development in evaluation: (1) There is a greater need to develop and improve both the conceptual and practical framework for Monitoring & Evaluation (M&E) for the participating countries including Nepal; (2) Efforts should be streamlined and institutionalized to educate, train and coach officers and personnel involved in M&E to evaluate development projects and operate budget programs and activities by providing simple, practical and flexible guidelines such as operational steps, methods and tools; (3) There is a need to enhance skills and knowledge in methodology for data collection and analysis, as well as techniques of preparing evaluation reports in useful manner; (4) The right kind of incentives and rewards need to be identified for those who have excellent track record in M&E in order to sustain interest and motivation, as well as providing right incentives for the organizations to invest in HRD for evaluation capacity development (ECD); (5) Greater efforts to establish and harmonize evaluation standards, ethics, codes of conduct and competencies among the participating countries. Perhaps Evaluation Societies in member countries can play a proactive role in this matter; (6) Recognizing the fact there is a lack of trained personnel in the supply side, while demand for M&E is increasing, participating countries should design comprehensive road maps for training on ECD, as well as carrying out need assessments and gap analysis to design comprehensive training modules in M&E; (7) To further enhance the level of competency in M&E, university programs should be introduced to accredit and certify evaluators according to their level of competency. 3. Session 2: Development of Institutions for Enhancing Evaluation Capacities In the second session, two presentations were made on efforts and challenges for enhancing institutional evaluation capacities: the first was by Ms. Yohandarwati Arifiyatno, Director for Sectoral Development Performance Evaluation, BAPPENA, Indonesia, and the second was by one of the co-chairs, Mr. Roderick Planta of NEDA, the Philippines. Mr. Kabir Hashim, a Member of Parliament of Sri Lanka and member of the Sri Lanka Evaluation Association, as well as a board member of the International Development Evaluation Society (IDEAS), moderated the discussion that followed the presentations. The two presentations evoked a lot of interest and many questions were raised. Some of the main issues raised were: 1) The PDCA (Plan - Do ? Check ? Act) cycle and the role of M&E in Indonesia and the Philippines; 2) Institutional problems including frequent personnel rotation in governments and a lack of communication among different organizations; 3) The evaluation culture and social context in each country; 4) Utilization of the log-frame and results framework; 5) Making effective feedback to policymakers and high levels of governments; 6) Generic and national institutional capacity building as opposed to donor-driven project-based and thematic-based capacity building; 7) The role of academia and evaluation societies. 4. Session 3: The Role of the APEA Network and its Future Possibilities In Session 3, the participants discussed the role and future possibilities of the newly established Asia Pacific Evaluation Association (APEA) as the first evaluation network in the Asia Pacific region. Presenters were two of the leading members of the APEA: Prof. Ryokichi Hirono, Senior Advisor of the Japan Evaluation Society (JES) and Dr. Champak Pokharel, President of the Nepal Evaluation Society (NES). They outlined the background to the establishment of the APEA and provided updates on its development, as well as its expected work plans and the possibility of cooperation with countries in the region. Dr. Romeo Santos, President of the Pilippinas Development Evaluators Association (PHILDEV) and another leading member of the APEA moderated the discussion that followed the presentations. Participants made various comments on points that included: 1) The historical significance of establishing the APEA; 2) The vision of the APEA to promote Asia-Pacific value-added; 3) Expected actions of the APEA for solving evaluation problems in the region and its prioritization of its actions; 4) Sustainability of the APEA including in the financial aspect; 5) The importance of good relations with governments of the countries of the region and international development organizations; 6) The current thrust of the APEA; 7) Cooperation with other international and regional evaluation associations of comparative strength. 5. Session 4: The Emerging Agenda and Challenges for Evaluation In this session, the emerging agenda and challenges for development evaluation in the 21st century were discussed with enthusiastic and active participation from everyone. Ms. Susan D. Tamondong, Vice President of International Development Evaluation Association (IDEAS) opened the session, by relating the discussions of previous day to her presentation on this topic. She provided several references on this topic and posed some enduring evaluation questions at the end of her presentation, for further analysis and research. Mr. John Samy, originally from Fiji and a former Asian Development Bank (ADB) official, moderated the open forum that followed after Ms. Tamondongfs presentation. Questions and comments were made by participants mainly on the following points: 1) New trends in evaluation, such as focusing more on sector wide approaches (SWAps) rather than on projects alone; 2) Harmonization and making evaluation into a learning process rather than police work and making evaluation meaningful and effective rather than it being treated as nuisance to government; 3) Policy coherence, not only among organizations commissioning evaluations but also within and between governmental ministries, and how to achieve it; 4) New evaluation methods to maximize learning and the importance of impact evaluation; 5) Attribution and how to address the difficulty of identifying the causal relationships and impact of development, through evaluation design; 6) And most importantly, ethics and equity, which increasingly play a very important role in evaluation work for the 21st Century. The code of ethics among evaluation associations serves as guide among young and expert evaluators to promote quality and equitable evaluations.