What is ODA Evaluation?
ODA evaluation is the task of assessing and analyzing the implementation status and effects of Official Development Assistance (ODA), which the Government of Japan carries out for the purpose of contributing to securing the peace, stability and prosperity of the international community.
ODA evaluation has two objectives. The first is to manage and improve ODA to make it more effective and efficient by feeding back the recommendations and lessons drawn from investigations of the implementation status and effects of ODA, into ODA policy formulation, project formation, implemenation and supervision. The second is to ensure public accountability as well as to promote public understanding and enhance support toward ODA by publishing the evaluation results. With regard to ODA evaluation, the Development Cooperation Charter that was revised in June 2023 states that “in light of the importance of evaluation and improvement not only for maximizing effectiveness and efficiency of cooperation but also for ensuring accountability to the public, ... Japan will appropriately conduct evaluations.”
Japan has steadily conducted ODA evaluations since 1975, prior to the enforcement of the Government Policy Evaluations Act (GPEA) in 2002. It has developed its evaluation methodologies based on not only actual practices but also the evaluation criteria of international institutions engaged in ODA, including the Development Assistance Committee of the Organisation for Economic Co-operation and Development (OECD-DAC).
This Annual Report gives an overview of the ODA evaluations (third-party evaluations) conducted by the ODA Evaluation Division, Minister’s Secretariat of the Ministry of Foreign Affairs (MOFA) of Japan, separately from evaluations based on the GPEA.
Implementation Framework
In Japan, ODA policies are planned and formulated by MOFA, while the implementation of individual ODA projects is primarily the responsibility of the Japan International Cooperation Agency (JICA). ODA evaluation is conducted through mutual collaboration between MOFA and JICA with each performing different roles.
Currently, MOFA primarily evaluates the policy aspects of ODA by commissioning third parties such as external experts and consultants. JICA, on the other hand, mainly evaluates the individual projects that it is responsible for implementing and supervising.
MOFA also assists partner countries in enhancing their evaluation capabilities.
Utilization of ODA Evaluation Results
It is important that the results and recommendations obtained from ODA evaluations be taken seriously by relevant parties such as MOFA, which is responsible for planning and formulating ODA policies, and JICA, which is responsible for implementing individual projects, and that they be utilized for planning and formulating ODA policies as well as implementing and supervising ODA projects.
Accordingly, once evaluations are completed, evaluators report the results and their recommendations directly to MOFA officials. In the next fiscal year, MOFA and JICA work together to formulate specific actions in response to the recommendations. Then, two fiscal years after the evaluation, MOFA checks the implementation status of these response actions and publishes the results in its annual evaluation report.
It is through this process that MOFA improves ODA management and ensures public accountability, which are the objectives of ODA evaluation.
Evaluation Targets
ODA evaluations (third-party evaluations) conducted by MOFA are classified into Country/Regional Assistance Evaluations, which target ODA policies for individual countries or regions, and Thematic/Cooperation Modality Evaluations, which target ODA projects conducted based on specific development themes such as education, health, or the environment, as well as ODA projects conducted through specific cooperation modalities (schemes) such as technical cooperation or grant aid.
Since FY2017, MOFA has also been conducting third-party evaluations of individual grant aid projects in which the maximum amount of aid offered is one billion yen or more. Internal evaluations are conducted for individual projects in which the maximum amount of aid offered is at least 200 million yen but less than one billion yen.
Furthermore, since FY2021, MOFA has been conducting third-party evaluations of individual projects implemented under Grant Assistance for Japanese NGO Projects.
Evaluation Viewpoints/Criteria
MOFA conducts ODA evaluations (third-party evaluations) from two different viewpoints: the extent to which Japan’s ODA contributes to economic and social development in partner countries (Development Viewpoints), and the benefits that the evaluated ODA policies have brought to Japan’s national interests (Diplomatic Viewpoints). MOFA’s ODA Evaluation Criteria are described below.
1 Evaluation from Development Viewpoints
MOFA has set the evaluation criteria described below as suitable for evaluating Japan’s ODA policies based on the six internationally recognized Evaluation Criteria presented by the OECD-DAC (Relevance, Coherence, Effectiveness, Efficiency, Impact, and Sustainability). Under each criterion, specific verification items are provided.
Relevance of Policies
How relevant is the ODA policy to Japan’s high-level policies, the needs of partner countries, and global priority issues? What advantages does Japan’s ODA have compared to that of other donors?
Effectiveness of Results
To what extent are the initial targets and objectives achieved in line with plans? What were the specific outputs, outcomes, and impacts?
Appropriateness of Processes
How appropriate are the process of planning, formulating and implementing the ODA policy, and the implementation structure? Has effective coordination with other donor countries, international organizations, NGOs, and other actors been undertaken?
2 Evaluation from Diplomatic Viewpoints
Diplomatic Importance
In what respect is Japan’s ODA important in resolving global priority issues, strengthening bilateral relations, and promoting Japan’s security and prosperity?
Diplomatic Impact
How has Japan’s ODA contributed to promoting Japan’s presence in the international community, strengthening its bilateral relations, and promoting Japan’s security and prosperity?
ODA Evaluation Guidelines & Handbook
MOFA has published “ODA Evaluation Guidelines,” which set out the basic principles of MOFA’s ODA evaluations, and “ODA Evaluation Handbook,” which describes the specific evaluation methodologies and procedures. These are prepared mainly to be applied in practical ODA evaluation work, but also offer useful information for anyone interested in ODA and its evaluation.
https://www.mofa.go.jp/policy/oda/evaluation/basic_documents/index.html 