This annual report presents an overview of MOFA's third-party ODA evaluations in FY 2019. The results of the evaluations, which were commissioned to external evaluators, have been summarized from the development and diplomatic viewpoints, respectively.

Evaluations from Development Viewpoints

In terms of Relevance of Policies, most evaluations resulted in highly satisfactory ratings. The target development cooperation policies were consistent with Japan's high-level ODA policies, international priority issues, and the needs of partner countries. Furthermore, the Effectiveness of Results in relation to development effects was also rated highly as it was confirmed that Japan's assistance made certain contributions to the development issues of partner countries. For some evaluations, however, it was pointed out that the expected effects had not been confirmed and that only a limited contribution to development had been observed.

As for Appropriateness of Processes, which is assessed to ensure the effectiveness and efficiency of ODA, the evaluations resulted in generally satisfactory ratings. However, it was pointed out in some evaluations that more effort should have been made in terms of information sharing, efficiency of operations, and public relations/information disclosure.

Evaluations from Diplomatic Viewpoints

The Development Cooperation Charter (2015) defined development policy as one of the most important tools for pursuing diplomacy proactively. Since then, all of MOFA's ODA evaluations have been conducted from diplomatic viewpoints to measure their impact on Japan's national interests.

This evaluation is intended to further fulfill accountability to the public by clarifying the following.
1 Why is the ODA in question important for Japan’s national interests (diplomatic importance)?
2 How has the ODA in question contributed to Japan’s national interests (diplomatic impact)? 

During the planning phase of the evaluation for FY 2019, MOFA's policy makers provided evaluators with briefings on ODA's diplomatic importance and its contributions to national interests. In addition, policy makers also provided the data necessary for evaluators to analyze target policies. MOFA continuously provides an environment that allows evaluators to conduct high-quality ODA evaluations.

Significant diplomatic impacts have been confirmed in several evaluations. The evaluation of ODA to the Philippines found that improved trust in Japan Brand for ODA projects and recognition of Japan's ODA by the public through high-profile media coverage resulted in an enhanced bilateral relationship. The evaluation of ODA for the Promotion of Women's Empowerment found that the training projects for female entrepreneurs resulted in capacity building for not only African women but also Japanese women through interactions and peer-learning.

While international methods for conducting evaluations from diplomatic viewpoints have yet to be established and a process of trial and error continues, interest in how ODA contributes to national interests is growing globally. In December 2019, the OECD/DAC revised the evaluation criteria and added "Coherence" as a new criterion to assess how well target policies are linked to other policies in a country. This new criterion is consistent with Japan's concept of "evaluations from diplomatic viewpoints." MOFA continues to make efforts to further improve its evaluations in order to obtain the public's understanding of ODA.

Recommendation Trends and Response Actions

The recommendations for the five ODA evaluations conducted in FY 2019 were based on the individual evaluation circumstances. While the recommendations cannot be compared because Country Assistance Evaluations and Thematic/Aid Modality Evaluations have different perspectives, they have some common features. The recommendations and response actions can be summarized and categorized into two areas, as described below.

1 Promotion of information disclosure on the content of assistance

It was recommended that Japan should disclose detailed information on the content of the assistance, including its objectives and background, to fulfill its accountability to the public and promote public understanding.

Evaluation of Japan's ODA to the Republic of the Philippines: It was recommended that more detailed information should be made widely available concerning projects for the economic and social development program (Grant Assistance) and grant assistance for grassroots and human security.

Evaluation of Japan's Non-Project Grant Aid for Provision of Japanese Next-Generation Eco-Friendly Vehicles to Peru in FY 2013: It was recommended that Japan should disclose specific details, including the concrete content, its objectives, and the targets to be achieved, for projects related to non-project grant aid (economic and social development program) in order to ensure accountability to the public.

MOFA has so far described the objectives and other related matters concerning each project sufficiently in its press releases and published information on the websites of overseas establishments concerning matters such as handover ceremonies. MOFA will continue to strive to disclose easy-to-understand information so that the public can understand the projects.

2 Establishment of indicators to achieve the target

It was recommended that indicators should be set in advance to measure the degree of achievement of policy goals.

Evaluation of Japan's ODA for Women's Empowerment: It was recommended that Japan should introduce results-based management to achieve its gender ODA policy goals. In order to regularly monitor and measure its progress, Japan should make action plans that include approaches to make sure of gender-mainstreaming in every phase of development cooperation and also set deadlines. It was also recommended that Japan should pledge not only the total ODA volume and the numbers of beneficiaries, but also the share of ODA committed to gender equality.

MOFA will consider the introduction of a results-based management approach while making further efforts to implement assistance and follow ups. Moreover, to address gender-mainstreaming in every phase of development cooperation, MOFA will promote policy-level research and studies and also collect and analyze good practices on women's empowerment.

Evaluation of Japan's Non-Project Grant Aid for Provision of Japanese Next-Generation Eco-Friendly Vehicles to Peru in FY 2013: It was pointed out that the overall structure of grant aid was difficult to understand, and it was recommended that the objectives, targets and other related matters should be explained more clearly and disclosed to the public.

MOFA will make efforts to fully describe the objectives of projects in order to get public understanding.

Examples of Response Actions to Recommendations

Once the Philippines reaches the status of a more developed country, it is expected to cease being a recipient of yen loans that are provided on the condition that they are used to acquire Japanese technologies. However, there are strong calls from the Philippines for assistance that utilizes Japanese technologies, and efforts need to be made to meet these requests.

Evaluation of Japan's ODA to the Republic of the Philippines: It was recommended that greater emphasis should be placed on assistance that utilizes Japanese technologies through co-financing with other donors as well as projects conducted in collaboration with the private sector using overseas investment and loans in infrastructure development.

MOFA will work closely with the Philippines to discuss what forms of cooperation would best meet the country's needs while complying with international rules concerning development cooperation for more developed countries. MOFA will also consider strengthening the partnership with the private sector and other donors.

Evaluation of Grant Assistance for Japanese NGO Projects: Operations of Grant Assistance for Japanese NGO Projects were highly commended in terms of respect for the autonomy of NGOs and their unique values.

However, it was recommended that strategies which clearly stipulate the direction and priority issues in Grant Assistance for Japanese NGO Projects should be documented.

MOFA will endeavor to clarify the overall direction and strategy of Grant Assistance for Japanese NGO Projects while examining the possibility of establishing a system to give due consideration to the organizational base of each NGO.

Evaluation of Japan's ODA for Women's Empowerment: It was recommended that Japan should design a gender flagship project that represents Japan's ODA.

Furthermore, it was recommended that delivering a performance with great impact and results would promote collaboration among Japan and other donors, which may lead to better partnerships with other donors and enhance Japan's ODA in the gender sector.

The Japanese government has so far provided assistance with a gender perspective through public infrastructure projects where Japan has expertise, such as construction of women-only train cars in India and women-only buses in Pakistan. Such assistance for empowering women by providing access to society has been highly recognized.

MOFA aims to spread such good practices and achievements to the public in Japan and abroad and also examine the feasibility of similar projects in other countries.

Evaluation of the SATREPS Program (Science and Technology Research Partnership for Sustainable Development): A recurrent issue was that after research proposals for the SATREPS Program are submitted and the Japanese authorities conduct the screening phase, the implementing agencies in partner countries appeared not to have received specific feedback on why proposals were rejected.

More information that can benefit researchers moving forward should be provided, such as specific information on what criteria led to a proposal being rejected and what improvements could be made to increase the probability of a proposal being selected in the future.

It was recommended that sharing future-oriented information with the relevant organizations in partner countries and thereby foster ownership is critical.

MOFA will coordinate with relevant organizations to explain the reasons why proposals were rejected when informing partner countries of the screening results.

Evaluation of Japan's Non-Project Grant Aid for Provision of Japanese Next-Generation Eco-Friendly Vehicles to Peru in FY 2013: It was recommended that Japan should consider taking measures such as disseminating information to local industry groups and utilizing publicity effects through the Japanese community when implementing similar cooperation projects in the future.

MOFA will consider setting targets for the market development of Japanese products among the public in partner countries and will carry out PR activities.