(3) Conducting Evaluation
In order to enhance management and ensure the accountability of ODA, the Government of Japan has been working on thorough implementation of the following measures: (i) enhancing the PDCA cycle (project formation (Plan), implementation (Do), evaluation (Check), follow-up activities (Act)), (ii) strengthening the Program Approach, and (iii) reinforcing “visualization.”
With regard to enhancing the PDCA cycle, the ongoing efforts include: (i) formulating Country Development Cooperation Policies for all recipient countries of Japan’s ODA, (ii) convening the Development Project Accountability Committee, (iii) setting indicators for each project, and (iv) strengthening the evaluation mechanism.
In order to implement ODA projects more effectively and efficiently, strengthening of the PDCA cycle not only at the project level but also at the policy level is required. To this end, MOFA conducts policy evaluations of economic cooperation policies based on the “Government Policy Evaluations Act (GPEA) Note 23,” as well as ODA evaluations by third parties who approach the evaluation from a neutral position. The recommendations and lessons obtained from these evaluations are reflected in ODA policy to further improve ODA management.
Third-party evaluations Note 24 are mainly carried out at the policy level (e.g., Country Assistance Evaluations and Thematic/Aid-Modality Evaluations). They evaluate from development viewpoints based on three evaluation criteria, namely, whether the ODA policies are consistent with Japan’s high-level policies, the needs of the aid recipient countries, and international priority issues (Relevance of Policies), how effective Japan’s ODA is in practice (Effectiveness of Results), and whether appropriate processes had been taken to ensure the validity of the policies and effectiveness of the results (Appropriateness of Process).
In addition to development viewpoints, evaluations also incorporate the idea that it is important to verify the diplomatic impact of implemented policies and programs. Basically all of MOFA’s third-party evaluations have been conducted using the diplomatic viewpoint since FY2015. Continued efforts are underway to improve evaluations from the diplomatic viewpoint in order to clarify the diplomatic importance and impact of ODA (how Japan’s ODA has contributed to realizing its national interests). Furthermore, by making all evaluation reports available to the public on the MOFA website (https://www.mofa.go.jp/policy/oda/evaluation/index.html) and other means, MOFA fulfills its accountability to the public on how ODA has been used and what effects it has produced, while increasing the transparency of ODA, promoting public understanding about it, and encouraging public participation in it.
Moreover, JICA also conducts evaluations on respective projects of grant aid, ODA loan, and technical cooperation, as well as thematic evaluations on each project. JICA conducts consistent monitoring and evaluation from the ex-ante stage to the implementation and ex-post stage for each project and has established a coherent evaluation mechanism for these three aid modalities. As for projects that exceed a certain amount, JICA conducts ex-post evaluations by third party evaluators (external evaluations). JICA also strives to enhance impact evaluations Note 25, recognizing the
importance of quantitative examination of the effects of their projects.
These evaluations are conducted primarily based on the Evaluation Criteria of the Development Assistance Committee (DAC) of the Organisation for Economic Co-operation and Development (OECD) Note 26.
As an example of how Japan’s ODA is viewed abroad, according to the Opinion Poll on Japan conducted among ASEAN member countries in March 2020 Note 27, over 80% of the respondents answered, “Japan’s ODA is beneficial to my country.” In addition, according to the Opinion Poll on Japan conducted in three African countries (South Africa, Kenya, and Côte d’Ivoire) in November 2019 Note 28, nearly 80% of the respondents answered, “Japan plays an important role in development cooperation, including Official Development Assistance (ODA), in the international community.”
- Note 23: Other than at the policy level, ex-ante evaluations are conducted on loan aid projects in which the maximum amount of loan offered through an Exchange of Notes (E/N) is 15 billion yen or more and on grant aid projects in which the maximum amount of aid offered through an E/N is 1 billion yen or more. In addition, ex-post evaluations are conducted on “pending projects” and “incomplete projects.” (“Pending projects” are projects for which the loan agreement has not been signed or loan disbursement has not begun after five years have elapsed following the decision to implement the project. “Incomplete projects” are projects for which loan disbursements have not been completed after ten years have elapsed following the decision to implement the project.)
- Note 24: Regarding Grant Aid implemented by MOFA, internal evaluations have been carried out for projects of 200 million yen or more, and third-party evaluations have been carried out for projects of 1 billion yen or more since FY2017. The results of these evaluations have been publicized and utilized to improve the formation of future ODA proposals.
- Note 25: Evaluation method verifying the effects of development projects by using methods from statistics and econometrics.
- Note 26: DAC evaluation criteria: In December 2019, coherence was added to the five criteria of relevance, effectiveness, efficiency, impact, and sustainability that had been in use since 1991, resulting in a total of six evaluation criteria.
- Note 27: MOFA commissioned a polling institute to conduct an opinion poll on Japan, which targeted 300 individuals between the ages of 18 and 59 residing in 10 ASEAN countries (Brunei, Cambodia, Indonesia, Laos, Malaysia, Myanmar, the Philippines, Singapore, Thailand, and Viet Nam) using both the internet and interviews with some selected individuals.
https://www.mofa.go.jp/press/release/press4e_002784.html - Note 28: In the same way as the above-mentioned footnote 27 , MOFA commissioned a polling institute to conduct an online survey targeting 1,500 individuals between the ages of 18 and 59 (500 individuals from each country) residing in South Africa, Kenya, and Côte d’Ivoire.
URL: https://www.mofa.go.jp/press/release/press4e_002784.html