FY2022 MOFA ODA Evaluation Results
Review of Japan’s ODA Evaluations from FY 2015 to FY 2021
| Chief Evaluator | OHNO Izumi Professor, National Graduate Institute for Policy Studies |
| Consultant | International Development Center of Japan Inc. (IDCJ) |
| Period of the Evaluation Study | August 2022 to November 2022 |
Background, Objectives, and Scope of the Evaluation
The purpose of this review is to provide recommendations for future ODA policies, issues to be considered for their implementation, and new perspectives to be included. This review was conducted in line with the main items of the Development Cooperation Charter, with the scope of including the third-party policy-level ODA evaluations conducted by MOFA from FY 2015 to FY 2021, as well as the FY 2014 evaluation reports and JICA ex-post evaluation reports (for projects implemented in FY 2015 and later). In addition, supplementary information from the White Papers on Development Cooperation, Diplomatic Bluebooks, and other sources was reviewed, and opinions were solicited from experts.
Summary of Evaluation Results
(1) Trends and Characteristics of Japan’s Development Cooperation
Trends in Japan’s ODA were analyzed taking 2015 as the baseline year. Japan’s ODA has responded to international situations and global challenges by changing the allocation of the limited financial resources to different sectors and regions, without any significant budget increase. The “partnerships” emphasized in the Development Cooperation Charter (2015) have been strengthened since the revision of the Charter, as evidenced by the trends of the Grant Assistance for Japanese NGO Projects and collaboration with international organizations. Furthermore, considering the increase in the flow of non-ODA funds to developing countries, the partnership between ODA and the private sector continues to be important.
(2) Review of ODA Evaluation Reports Using the Development Cooperation Charter (2015) as the Reference Criteria
A meta-evaluation of the 59 ODA evaluation reports for FYs 2014–2021 was conducted from the perspective of ODA evaluation, according to the items of the Development Cooperation Charter. Overall, a high degree of consistency was found for “Relevance of Policies,” positive effects for “Effectiveness of Results,” and “Appropriateness of Processes” was generally evaluated to have been properly implemented. There was also considerable mention of “diplomatic importance” and “diplomatic impact.” However, when looking at the individual evaluation results for each of the items of the Development Cooperation Charter, not all items were necessarily rated highly.
(3) Classification of the Recommendations and Lessons Learned from the ODA Evaluation Reports
A total of 285 recommendations and lessons learned were extracted from the 59 ODA evaluation reports. They were reviewed on the basis of the items listed in the Development Cooperation Charter and classified into sub-categories used in the previous study.Note These results were compared with the results of the “Review of Japan’s ODA Evaluations from FY 2003 to 2013.” In the review based on the Development Cooperation Charter, the top three items in terms of the number of recommendations and lessons learned were: “A more strategic approach” under “Implementation Principles;” “Principles for effective and efficient development cooperation [other than (a) to (c)]” under “Implementation Principles;” and “‘Quality growth’ and poverty eradication through such growth” under “Priority Issues.” In addition, combining the number of multiple items related to “cooperation with various funds and actors,” the total number of recommendations and lessons learned was equivalent to the second rank overall.
Note: FY 2020 MOFA ODA Evaluation “Review of Past ODA Evaluations (Country Assistance Evaluations) and Study of Country Assistance Evaluation Methodologies.”
(4) Issues and Perspectives to be Considered in the Formulation and Implementation of Future Development Cooperation Policies
The main points to be considered are “strategic” cooperation and “national interest.” It is necessary to consider “strategic” cooperation in two dimensions: “strategic,” in terms of enhancing development effectiveness, and from the perspective of national and diplomatic strategies such as the National Security Strategy and “Free and Open Indo-Pacific” (FOIP). In development cooperation, “national interests” are to be considered in a broader sense, and it is necessary to reach a national consensus on the expansion of the scope of development cooperation as well as the principles of its implementation. If Japan is to use its ODA budget to actively engage in specific fields, public understanding is essential. So, it is important to constantly disseminate public information and monitoring and evaluation results in an easy-to-understand manner.
Recommendations Based on Evaluation Results
(1) Recommendations on the Development Cooperation Charter
(a) Clarification of “Strategic” Cooperation in the Development Cooperation Charter
The relationship between the Charter and national strategies should be clarified, while paying attention to a longer-term, global perspective of the sustainable development of developing countries. In addition, the policy of cooperation for non-military purposes should be adhered to and the acceptable scope of assistance and implementation principles should be clarified, as well as the partnership policy with international organizations, other donors, emerging donors, NGOs, etc. If the needs for assistance that involves military personnel further expands, the creation of an international cooperation mechanism under a non-ODA framework should be considered.
(b) Strengthening Coherence and Focus on the Structure and Content of the Charter
Some issues of the current Development Cooperation Charter are that the relationship between the objectives, principles, and implementation considerations is not always clear, and the areas of assistance listed under the three priority issues are too detailed. The contents and structure should be made clearer and easier to understand, for example, by separately describing the items to be considered in all cooperation and those that should be given greater or lesser weight depending on the project so that the goals of the Charter can be easily reflected in individual cooperation. In addition, the text should be plainer and easier to understand to ensure public participation and consensus.
(c) “Implementation Principles” and “Implementation Arrangements”
Among the items mentioned in “Implementation Principles” and “Implementation Arrangements,” “proactive contribution to international discussions,” “principles for securing the appropriateness of development cooperation,” “strengthening partnerships,” and “strengthening the foundations for implementation” remain important. Regarding cooperation with the involvement of military personnel etc., continuous efforts should be made to confirm compliance with implementation principles in a transparent manner.
(2) Recommendations for Development Cooperation Policies and Their Implementation
(a) Reinforcing the Strategic Aspects of Development Cooperation Implementation
“Strategic” cooperation to enhance development effectiveness remains important. In addition to prioritization, efforts should be made for Japan to become a preferred partner, by reaffirming the nation’s strengths, such as support to self-help efforts with exit strategies, support to policy and institutional aspects and infrastructure operations, and mobilization of human resources who are familiar with Japan and its approach.
(b) Continuing Efforts to Strengthen the Implementation Arrangements
In formulating and implementing individual development cooperation policies, the recommendations and lessons learned with regard to “strengthening collaboration with other actors,” “monitoring and evaluation,” and “public relations” in ODA evaluation reports should be taken into account in all policies. In addition, recommendations and lessons learned on multi-country and regional cooperation, assistance to conflict-affected countries, disaster relief, and individual sectors should be referenced in the relevant policies.
(c) Establishment of Outcome Indicators
Presenting numerical indicators will help clarify strategies and priorities, is one of the means of communicating the achievements of development cooperation to the public in an easy-to-understand manner, and can help build public consensus. Therefore, Japan should set outcome indicators at the implementation level for the targets it is working on and visualize their status of achievement.
(3) Recommendations on ODA Evaluation Methodology
(a) Strengthening the Link between Policy-level ODA Evaluation and the Development Cooperation Charter
It is important to: 1) select evaluation themes that stress their relationship with the Charter; 2) consider the timing of the ODA evaluation review so that the pros and cons of revising the Charter and items to be considered can be fully identified; 3) review the achievements of results in the ODA evaluation, especially for priority policies described in the Charter; and 4) revise the evaluation perspective by linking it with the “Implementation Principles” and “Implementation Arrangements” described in the Charter.
(b) Clarification of the Process Leading to the Evaluation Results
The manner of describing the process and results of the evaluation work varies, and many reports do not have clear descriptions. The transparency of evaluation results would be enhanced if the process and results of the rating, weightage, and overall judgment for each factor to be considered are described in the evaluation report.