Highlights of FY 2018 MOFA’s ODA Evaluation
Summary of ODA Evaluation
This annual report presents an overview of MOFA’s third-party ODA evaluations in FY 2018. The results of evaluations commissioned to external evaluators from development and diplomatic viewpoints are summarized, respectively.
Evaluation from Development Viewpoints
In most evaluations, target policies were consistent with Japan’s high-level ODA policies, international priority issues, and recipient countries’ needs, which resulted in high ratings for Relevance of Policies. Furthermore, Effectiveness of Results as to development effects was also highly rated as it was confirmed that Japan’s assistance made certain contributions to the development issues of recipient countries. However, there seemed to be some issues to be resolved in the maintenance of records, the system of information disclosure, and public relation since there were two evaluations accessed as “partially unsatisfactory” for Appropriateness of Processes to ensure effective and efficient ODA.
● Ratings
Since FY 2017, MOFA has been using a revised rating scale using alphabetical ratings (A to D) to evaluate development effect, aiming for clear and comprehensible evaluation reports for all. The alphabetical scale, on the other hand, can be misleading due to its simplicity; therefore, we recommend readers to refer to the summary of each evaluation result.
Note 1 Rating Scale
Highly Satisfactory: All evaluation questions have highly satisfactory results.
Satisfactory: Most evaluation questions have highly satisfactory results.
Partially Unsatisfactory: Some evaluation questions have highly satisfactory results although there are some issues to be resolved.
Unsatisfactory: Most evaluation questions do not have satisfactory results.
Note 2 Concerning the Evaluation of Japan’s ODA to Costa Rica and Nicaragua, the assessment of Nicaragua was conducted through desk study including questionnaire survey instead of field surveys due to its socio-political situation. Although the evaluation results are obtained individually, the ratings of Costa Rica and Nicaragua are merged for convenience since their rating results are the same. Moreover, recommendations are taken as “lessons learned” based on the evaluation results, also deriving common lessons from both countries.
Evaluation from Diplomatic Viewpoints
The Development Cooperation Charter (2015) defined development policy as one of the most important tools for pursuing diplomacy proactively. Since then, all MOFA’s ODA evaluations have been conducted from diplomatic viewpoints to measure the impact on Japan’s national interests.
This evaluation is used for clarifying the following in order to further fulfill accountability to the public: (1) why ODA is important for Japan’s national interests (diplomatic importance) and (2) how ODA has contributed to Japan’s national interests (diplomatic impact).
Since more specific evaluation questions were presented based on external experts’ advice in FY 2018, there were improvements in the evaluation’s quality and volume comparing to the previous year. However, some differences were also seen in each evaluation.
Significant diplomatic impacts have been confirmed in several evaluations: improvements in the business environment derived from collaboration between public and private sectors (Indonesia) and fostering of pro-Japan views and enhancement of friendships through close cooperation with JICA experts and volunteers (Costa Rica).
Some other countries have also started evaluating from diplomatic viewpoints or conducting similar evaluations gradually. Although a process of trial and error will continue because this evaluation method is yet to be established in the international community, MOFA continues to make efforts to further improve evaluations to obtain the public understanding of ODA.
Response Actions to Recommendations
The recommendations for the four ODA evaluations conducted in FY 2018 have some common objectives. The recommendations and response actions can be summarized and categorized into three areas.
Further Expansion of Assistance for Rural Development
When a development plan of a target country aims at reducing regional disparities, Japan should support projects not only in urban areas but also in rural areas and provide assistance in such a way that can bring outcomes in broader areas. Support to raise the standard of living for the entire country should also be considered.
Evaluation of Japan’s ODA to Angola: It was recommended that Japan should cooperate more actively in Angola’s rural areas since one of the items that were strengthened in the new Angola National Development Plan (PDN) was the “Balanced development of various regions.” MOFA will consider formulating projects in rural areas, taking into account their requests and needs, their security situation, and whether Japan’s support is visible.
Evaluation of Japan’s ODA to Costa Rica and Nicaragua: Pertaining to assistance to Costa Rica, it was recommended that Japan should support the systematization of its distribution nationwide rather than limited areas. Specifically, it identified the following as effective: establishing livelihood improvement approach in rural farm areas and expanding technical assistance for promotion of Micro, Small, and Medium Enterprises (MSMEs) f rom urban areas to nationwide. MOFA will continue to support reduction of economic disparities and will consider further assistance for people’s l ivel ihood improvement based on Costa Rica’s efforts. Moreover, MOFA is currently training stakeholders in Costa Rica to improve their knowledge and skills to teach others as part of the MSMEs support projects.
Strengthening of Public Relations (PR)
Enhancement of publicizing Japan’s cooperation in the recipient countries was recommended in order to gain more recognition of Japan’s assistance. It was suggested that Japanese embassies should devise more strategic PR within the limited budget, such as PR plan targeting the young.
Evaluation of Japan’s ODA to Costa Rica and Nicaragua: Pertaining to Costa Rica, it was recommended that PR strategies were necessary to raise greater public recognition of Japan’s ODA. For example, sharing Japan’s PR strategies with the stakeholders in Costa Rica is necessary for cooperation and development of specific PR activities. It also indicated that PR with higher impact should be employed, such as regular use of social media, which makes information more accessible for people of all ages including the young. MOFA will continue to practice various PR activities through Embassies in addition to reaching media individually and will strive for promoting PR activities to reach all generations.
Evaluation of Non-Project Grant Aid to Togo in FY 2013: It was recommended to strengthen consultation and public relations regarding Japan’s assistance in countries where Japan has no Embassy. Specifically, it suggested holding events periodically to share Japan’s all ODA information and allocating training courses for fostering human resources with knowledge of Japan while taking into account countries where Japan has no Embassy. In response to the recommendation, MOFA will continue to have discussions with the recipient’s government and ODA stakeholders and act considering various factors, including their needs.
Strengthening of Japan’s Implementation Structure and Reinforcement of Cooperation with Relevant Organizations
It was suggested to make further efforts to provide ODA that would trigger private investment and enhance synergies through effective collaboration with private sector and NGOs with comparative advantage in specific fields.
Evaluation of Japan’s ODA to Indonesia: Since the private sector was expected to play a major role in development cooperation in Indonesia, it was recommended to state further enhancement of collaboration with the private sector and ODA agencies in Japan’s development cooperation policy. MOFA will state it in the next revision of Country Assistance Policy for Indonesia. It was also recommended to hold periodically comprehensive discussions on Japan’s ODA among ODA Task Force. MOFA will hold ODA Task Force meeting to oversee the overall picture of Indonesia’s development plans, engaging wider stakeholders, such as the Japanese Embassy, the Japan International Cooperation Agency (JICA), the Japan Bank for International Cooperation (JBIC) and the Japan External Trade Organization (JETRO).
Evaluation of Japan’s ODA to Angola: Strengthening ODA implementation structure in Angola was suggested as Japan’s ODA to Angola may increase in future. In response, JICA extended the scale its office in Angola in July 2018 and will examine a possibility to further reinforce Japan’s ODA implementation structure, taking into account the future ODA’s volume.
Evaluation of Japan’s ODA to Costa Rica and Nicaragua: It was confirmed as lessons learned that collaboration with the Inter-American Development Bank (IDB) was more effective than Japan’s bilateral assistance. It is because of not only larger fund scale resulting in more significant development impacts but also smoother project formulation and implementation, and reduced cost and workload. Several co-financing projects with IDB are currently undertaken: two in Costa Rica and one in Nicaragua. MOFA will continue to explore co-financing opportunities proactively.
Examples of Response Actions to Recommendations (Lessons Learned) based on Individual Circumstances
Evaluation of Japan’s ODA to Angola: Since the inauguration of President Lourenço in 2017, Angola has been undergoing significant changes such as a new Angola National Development Plan (PDN). Therefore, it was recommended to incorporate new items proposed in the Recommendations based on evaluation results into Japan’s Country Assistance Policy for Angola and its Rolling Plans. In the event of significant changes in development policy of a recipient country, MOFA will consider reflecting them in Rolling Plans which can more flexibly respond instead of Country Assistance Policy revised every five years. Moreover, Angola’s latest development plans will be reflected in the Country Assistance Policy at next revision.
Evaluation of Japan’s ODA to Costa Rica and Nicaragua: A lesson was learned that strategic combination of various ODA aid modalities could further generate combined effect along with high efficiency. MOFA will continue to actively explore possibilities for effective use of combined aid modality.
Evaluation of Non-Project Grant Aid to Togo in FY 2013: It was recommended to improve monitoring and recording of procured items from delivery to sale. MOFA will make sure to follow up on this matter by determining the recipient country’s intention as precisely as possible.