A total of five third-party evaluations were conducted in fiscal year 2021, consisting of three evaluations of Japan’s ODA to Timor-Leste, Peru, and Malawi, one thematic evaluation (Education Cooperation Policy), as well as one evaluation of individual grant aid projects implemented by MOFA, namely, the Evaluation of Japan’s Grant Aid to the Democratic Socialist Republic of Sri Lanka (Economic and Social Development Programme) in FY2017.

Evaluation from Development Viewpoints

In the evaluations of Japan’s ODA to three countries and the thematic evaluation, “highly satisfactory” or “satisfactory” ratings were received across all the evaluation criteria: Relevance of Policies, Effectiveness of Results, and Appropriateness of Processes. The evaluations confirmed that these ODA policies were consistent with Japan’s high-level policies and the needs of the partner countries, that Japan’s cooperation contributed to resolving the development issues faced by the partner countries, and that policy formulation and implementation were carried out through appropriate processes.

In the evaluation of individual grant aid projects implemented by MOFA, theRelevance of Plans was rated “highly satisfactory,” but the Effectiveness of Results was rated “partially satisfactory” as the achievement of the goals of economic and social development of the region and support from Japanese corporations could not be verified in the short-term. In addition, partly because of the impact of the COVID-19 pandemic, some equipment required repairs and some necessary work had not been completed.

Rating standards
Highly Satisfactory: All verification items produced highly satisfactory evaluation results.
Satisfactory: Most verification items produced highly satisfactory evaluation results.
Partially Satisfactory: A number of verification items produced highly satisfactory evaluation results, but there were some issues to be resolved.
Unsatisfactory: Most verification items produced unsatisfactory evaluation results.

(Note 1) For country assistance evaluations and thematic evaluations.

(Note 2) For evaluations of individual grant aid projects implemented by MOFA. Based on the results of the Analysis of Third-Party Evaluation of Bilateral Grant Aid Projects Conducted by MOFA and Proposal of Evaluation Methods carried out in FY2020, evaluations have been conducted using the two criteria “Relevance of Plans” and “Effectiveness of Results” since FY2021 by combining the Development and Diplomatic Viewpoints. As for the criterion “Appropriateness of Processes” that evaluates whether the processes effectively ensured the relevance of plans and effectiveness of results, it was integrated into the evaluation questions for Relevance of Plans and Effectiveness of Results, instead of being an independent evaluation criterion.

Evaluation from Diplomatic Viewpoints

Since 2015, all of MOFA’s ODA evaluations have included evaluation from Diplomatic Viewpoints, which assesses the effects of ODA on Japan’s national interests. The evaluation criteria consist of “Diplomatic Importance” (why the ODA is important to Japan’s national interests) and “Diplomatic Impact” (how the ODA contributed to Japan’s national interests). In the evaluation of grant aid projects implemented by MOFA, as a result of the analysis conducted in FY2020, the “Diplomatic Viewpoint” and “Development Viewpoint” have been combined since FY2021. Also, verification items related to “Diplomatic Importance” are now included among “Relevance of Plans,” and verification items related to “Diplomatic Impact” are included among “Effectiveness of Results.”

In the FY2021 country assistance evaluations, the significance of Diplomatic Importance and Diplomatic Impact was confirmed for all countries, while a certain degree of impact was observed. It was confirmed that ODA has been used effectively as an important diplomatic tool to strengthen bilateral relations, promote understanding toward Japan, and enhance friendly relations. On the other hand, with regard to strengthening economic relations and promoting the expansion and investment of private-sector companies, the evaluations pointed out that there are still areas that have not yet produced results.

The Evaluation on Japan’s Education Cooperation Policy concluded that the implementation of ODA had produced diplomatic impacts including enhancing Japan’s presence and trust in Japan, as well as strengthening bilateral relations.

Recommendations

The five ODA evaluations conducted in FY2021 make recommendations based on their respective individual circumstances (see “Follow-up ODA Evaluation Results” for the respective recommendations and the response actions taken). Of those recommendations, the following are common to multiple evaluations or applicable to other cases.

Recommendations Common to Multiple Evaluations

● Continued and Strengthened Support in the Field of Human Resource Development

In the evaluation of Japan’s ODA to all three countries, recommendations were issued on continuing and strengthening support in the field of human resource development. The evaluations pointed out the importance of human resource development for achieving sustainable economic growth (Timor-Leste), that technical cooperation leading to long-term human resource development should be continued (Peru), and that Japan has comparative advantage in technical cooperation through human resource development at the grassroots level (Malawi).

● Promoting Improvements to the Investment Environment and Cooperation with the Private Sector in Partner Countries

One of the policies put forward by the Government of Japan is to promote public-private cooperation in ODA projects. In all three country assistance evaluations, recommendations were made on initiatives to promote activities by private corporations, and to strengthen cooperation with the private sector. The evaluations touched on providing support for developing legal systems toward improving the investment environment (Timor-Leste, Malawi), as well as the proactive utilization of private-sector cooperation schemes (Peru).

● Strengthening Public Relations on Japan’s Cooperation

The Evaluation of Japan’s ODA to Malawi and the Evaluation on Japan’s Education Cooperation Policy recommended strengthening publicity on Japan’s cooperation toward the people of partner countries and the international community.

Recommendations with Possible Applicability to Other Cases

● Setting of Policy Targets during Policy Formulation

The Evaluation on Japan’s Education Cooperation Policy pointed out that Japan’s “Learning Strategy for Peace and Growth” (formulated in 2015) does not set out an implementation period nor criteria, targets, and indicators for the achievement of goals. Therefore, it is unclear against what the achievement status is measured. It recommended that these should be established and incorporated into the policy during the next policy formulation. This is not limited to education cooperation policy; setting policy targets is important to implement and monitor the policy, conduct objective evaluations, and bring about improvements in the next policy. Therefore, target-setting should be considered in the formulation of other ODA-related policies.