Special Interview

“Considerations on ODA Evaluations”

Ministry of Foreign Affairs (MOFA) has recently been striving to expand the “evaluation from diplomatic viewpoints” in the ODA evaluation. It has been implemented in all evaluations since FY 2015. We interviewed four experts regarding the following points in ODA evaluations: the significance of the evaluation from diplomatic viewpoints; the differences when considering “national interests” in ODA from the perspectives of both ODA evaluation and development cooperation; expected synergies to improve the transparency and accountability as the evaluation’s goals.

Interviewers

Muraoka: Development Cooperation Charter ment ions “evaluat ion from diplomat ic viewpoints.” When formulating this charter, MOFA conducted a review of Japan’s ODA evaluations for the past ten years. Professor Hirono was the chief evaluator for the review. What are your thoughts on “evaluations from diplomatic viewpoints,” such as how ODA that uses tax revenues contributes to diplomatic policies?

Prof. Hirono:

1. Ideal Japan’s ODA: Promoting "global interests" and "national interests in the narrow sense”

It can be said that every country’s foreign policy protects and promotes a broad meaning of national interests.
　First, the “national interests in the broad sense” includes universal interests which are common to all countries and people in the world, namely “global interests” and “national interests in the narrow sense.” Japan is currently mainstreaming these global interests into its diplomatic policy. If neglecting global interests, no country would be trusted and respected by the world as a responsible member of the international community. “Global interests” means to secure the principles of “freedom from fear, hunger, and ignorance” adopted by the UN Charter. That is, ensuing “human security” through resolution of domestic and international conflict, peacebuilding, poverty eradication, education dissemination, and health promotion. Since then, “freedom from natural and environmental degradations and social discrimination” was added to the definition of global interests. In other words, environment conservation and establishing fair societies have been newly added in response to changes in the international community. Japan considers all of these as global interests and is actively working toward the development and independence of developing countries that are not particularly blessed in economic, social, environmental, and safety aspects. Thus, it is required for Japan’s ODA evaluation to assess from the perspective of promoting global interests.

Japan’s diplomatic and ODA policies also have other essential responsibilities, such as promoting its national interests in the narrow sense. That includes Japan’s security and stability, prosperity, ensuring the safety of its nationals, occupying an honored position in the international society, and maintaining goodneighborly and friendly relations. Furthermore, the development of the global environment is crucial for Japan to lead to achieve them. Pursuing national interests in the narrow sense is common among all ODA donor countries, although their specific details, scope, and degree vary from country to country. Not only will they differ depending on the each country's historical background, economic and social structure, dominant values, religious background, but also vary based on individual and security priorities. Therefore, respecting diversity is indispensable in the formulation and implementation of ODA policies. As social divisions are more and more prominent in both at home and abroad in today’s world, the promotion of national interests in the narrow sense should be employed based on transparent and inclusive international rules. Under such circumstances, it is no overstatement to say that there is an urgent need for assistance to enhance global interests. The same can be said for the evaluation.

2. Make More Use of Japan’s Strengths: A bridge between developed and developing countries

Dissonances are intensifying not only between developed and developing countries, but also among developed countries, among emerging, more developed, and least developed countries, as well as among countries with different political systems, in formulating and implementing international cooperation policies. For example, the America First policy, EU’s refugee policy, China’s Belt and Road Initiative, issues surrounding Russia’s CIS countries, and escalating conflicts in the Middle East are all contributing to the dissonances. Japan is currently a member of the developed countries. However, it has experiences of hardship and pain as a developing country not only under the unequal trade treaty during the modernization of the Meiji period but also as a recipient country during the post-war reconstruction process. Further, despite the recent economic globalization, Japan has a relatively low degree of income disparities compared to other developed countries, and it can be said that there is a minimal tendency for social divisions and conflicts among the people. Japan has a high-level ability to reconcile various interests, which is crucial for promoting smooth international cooperation. It is essential to make full use of this unique strength in this conflicting international community. While encouraging developing countries to accept and observe the international rules from the medium to long-term perspective, Japan should advise Western countries to avoid using urgent and rigid principles. It is essential for Japan to recognize its significant role in bridging between the Western countries and developing countries.

3. Create Japan’s Model for International Cooperation Framework: ODA that reflects a donor country’s ability to contribute

In promoting international cooperation, it is critical for Japan to encourage ODA donor and recipient countries that overemphasize putting their own country first, to follow the agreed international cooperation rules. A policy of “our country first” is not the problem. The real problem is the fact that ignoring the agreed international cooperation framework will lead to destabilization and mutual distrust of the international community. Therefore, Japan should further advocate this to the international community. Besides, severe and more diverse natural disasters are frequently occurring in various parts of the world, affected by climate change, which requires a more extensive, fast, and effective international cooperation system. Currently, all major ODA donor countries have substantial budget deficits and are experiencing economic growth slowdown. Taking that into account, Japan should establish a role-sharing and cooperation system based on the comparative advantage of each stakeholder as ODA’s important partners, such as private companies, civil society, universities, research institutions, and foundations. Private companies, in particular, have enormous financing, technical, and management capabilities. By doing so, Japan should be able to establish both domestic and international environments to promote “Cooperation that reflects a donor country’s ability to contribute.” Providing this Japan’s new model in international cooperation, Japan should encourage other countries to follow suit. Furthermore, it is also essential to demonstrate leadership in building an international cooperation framework that is rich in cultural diversity as one of Japan’s important diplomatic goals since Japan is aiming to be a nation with a cultural export-driven economy. This can be achieved by further disseminating knowledge, skills, experiences, and values of all levels nationals, appreciated by the international community, along with the unique arts and culture of each country.

Muraoka: We incorporated some enhanced evaluation points from diplomatic viewpoints into the Guideline last year. I would like to ask you to tell us what you have noticed in that respect from working as a chief evaluator for Japan’s ODA to Angola.

Prof. Inada: Evaluating from diplomatic viewpoints is quite difficult unless national interests are clearly defined in the first place. When it comes to defining national interests, there are several ways to understand them, including high-level and middle-level perspectives, as well as a project-level view. There are more than one definition of national interests in the decision-making process in ODA; therefore, it is challenging for the third-party to assess them from diplomatic viewpoints.

Muraoka: You mean, it’s challenging to evaluate one by one as an additional task within limited time and resources... .

Prof. Inada: In addition to time constraints, there is also a fundamental issue of what our national interests are. In the case of “evaluation from diplomatic viewpoints,” it is also a national interest in the narrow sense when diplomatic relations between Japan and recipient country are strengthened by providing ODA. I believe that it is a long-term national interest to make it easier for Japanese to take an active part in the international community by contributing to the improvement of the economic society of developing countries with resources in which Japan has comparative advantages.

From this point of view, human resource development is a resource that Japan has comparative advantages and can make a good use of. The point is to train people in developing countries and increase the number of Japanese who are involved locally. It doesn’t matter whether they are experts or volunteers (JOCVs). Japanese experts and JOCVs can feel proud of themselves by getting involved in the human resource development of the recipient countries. Besides, this can also lead to an increase in the number of people who are pro-Japan or well-versed in Japan as well as can provide a place for experts and young to shine. In my opinion, to increase such opportunities is a national interest that will help Japanese people to live a better life in the long run. It takes time to develop human resources, but costs less compared to other projects, so I think it is a fruitful activity.

Muraoka: That can actually lead to winning trust from developing countries... .

Prof. Inada: That’s exactly right. Don’t you think that the most important thing is to increase the number of people thriving on the work which they can feel proud to be part of? I had an opportunity to meet with an official of the Embassy of China when I visited Angola for the evaluation survey last year. He says, “Among Japan’s ODA projects, Japan’s human resource development project is highly valued. Chinese people work on a project for a few years and then quickly move on to another project. On the other hand, Japan’s human resource development project is long-standing and effective. That’s Japan’s strength.” Within Japan's limited ODA budget, the key to the Japanese assistance model is that it is something that the recipient appreciates, the Japanese can thrive on, as well as both Japan and the recipient country can benefit from. I think this could further lead to Japan’s national interests.

Muraoka: When evaluating from diplomatic viewpoints, it is more persuasive in the use of ODA as tax money if we can clearly explain causality between trust we are winning from developing countries through ODA and diplomatic objectives.

Prof. Inada: From the perspective of the effective use of taxes, frankly, I’m afraid that some projects are not generating enough results for the money spent on dispatching Japanese experts and JOCVs. However, if you look at them from the national interest point of view, regardless of their effectiveness, they are still part of Japan’s national interests in the long run. That’s because a contact point is created by Japanese people getting involved locally in the recipient country, and close relationships are built by making more local people familiar with Japan.

Miyamori: I agree. Japan’s ODA, once boasted the most substantial aid in the world, has sent an enormous number of experts and JOCVs all over the world, and we can’t deny the fact that it has contributed much to Japan’s internationalization.

Prof. Inada: You’re correct. ODA is said to be a diplomatic tool, and historically, abundant ODA funding and tools became a kind of leverage and contributed to an increase in diplomatic influence. Recently, the impact of financial aid has declined, but the high ability, which Japan has accumulated in terms of expertise in high-quality manufacturing, infrastructure development technology, and fair procedures in procurement, is still essential for Japan. However, it will not last long unless this expertise is objectively valuable and applicable worldwide. Knowledge of privatization as well as in improving governance promoted by the World Bank are efficient and necessary in the development and continue to be long-lasting, despite being criticized from various directions. I feel that Japan should also have competitive expertise in intangible aspects and strive to engage Japanese people in the international community.

Muraoka: In other words, we should enhance the areas of our comparative advantages... .

Prof. Inada: That’s right. I think it is necessary to strengthen such areas.

Muraoka: In the field of ODA evaluat ion, it is pointed out that we should evaluate not only from development viewpoints but also from diplomatic viewpoints. However, we have been assessing through trial and error since there is no established methodology. Please tell us what you think about national interests in terms of development cooperation, from your current position as the president of the Japan Society for International Development (JASID).

Prof. Yamagata: We, evaluators, are strongly aware that we represent the people of Japan when evaluating since ODA evaluations are conducted by the Japanese government. I had been feeling that it stirred quite a lot of confusion how to define “Japan’s national interests” at ODA project sites since it was clearly stated in the Development Cooperation Charter in 2015. We felt the urge to help resolve that situation. So, we held a special session at the annual conference of JASID in 2018. Then, as a result, Mr.Kiya and I published a book called “Why should we act for international cooperation?” (Masahiko Kiya and Tatsufumi Yamagata, 2019).

I explained in the book that “for” in the phrase “Japan’s ODA should be provided for the Japanese people” had two meanings. One is “on behalf of,” which implies that the Government of Japan will provide assistance on behalf of the people of Japan. The other is “for the sake of,” which means to show who the beneficiaries are, and in this case, it is the aid benefiting people in developing countries. We considered the impression of the Japanese in using ODA for people of developing countries as beneficiaries. According to the analysis by Professor Hulme of the University of Manchester, there are four possible feelings behind the reason why the Japanese use ODA for developing countries: (1) pure humanitarian purposes; (2) moral responsibility for what developed countries have done to developing countries as colonial suzerain states; (3) mutual interests, where benefits of a developing country received by Japan’s aid will eventually benefit Japan; (4) direct self-interest. In the 2000s, idealism with full of hope for the new millennium permeated throughout the world, and even talking about national interests of donor countries was considered unacceptable. For the following ten years, it seemed improper to not mention national interests. It makes me wonder if it’s a reaction to the prior decade. We brought up a question to readers in this book that it is a matter of personal choice, which of those four reasons is more important.

Muraoka: I think the most difficult part for a third-party evaluation team is what they are supposed to assess is so ambiguous when evaluating from diplomatic viewpoints. We will keep discussing with them about this so that we can deal with them in MOFA side.

Prof. Yamagata: Regarding evaluation as a PR tool, it is better to present positive aspects of the evaluation target; however, it is natural to have negative aspects pointed out in the evaluation. In my case, I always remind myself to point out shortcomings while listing all the positives.

Muraoka: We respect objective assessment by chief evaluator and hope that development cooperation staff can utilize evaluation results. Evaluation from diplomatic viewpoints is a very difficult subject. Personnel of MOFA are stakeholders themselves, and they are professionals in diplomacy. Therefore, they may feel that they don't need to be told by others. However, a thirdparty evaluation doesn’t mean to assess their diplomatic practice itself. We are experimenting to see if we can create a compelling story by examining published facts in order to convince them that evaluation is conducted from the perspective of how ODA contributes to diplomatic goals.

Prof. Yamagata: It is challenging to link each development project and its impact on the entire socio-economy of each recipient country.

Also, I feel that the general public is seeking a clear and distinct impact of ODA. For example, when I give a lecture, I’m sometimes asked, “Are partner countries grateful for Japan’s ODA in the first place? If the recipient country is grateful, we should continue. Isn’t it worthless if they don’t even appreciate the aid?” Now, it’s easy for me to answer “They are grateful.” However, we need to explain that being grateful doesn’t make recipient countries vote for Japan in the UN, for example, since there are other donors who aid the same developing countries.

I would also like to emphasize that Japan has been doing its best for a long time with pride in providing high quality professionalism and support as Japan’s feature. However, it can be easily forgotten unless continued, even if a partner country recognizes such perception of Japan. If the partner country thinks “Japan uses ODA for Japanese companies” and such a thought disseminates, we might lose the partner country’s trust we worked hard to win. I strongly feel that we must avoid such a situation.

Muraoka: MOFA is striving to enhance “evaluation from diplomatic viewpoints” by presenting specific evaluation questions. Professor Yamaya served as a chairperson at the expert review meeting on the survey of the trial results towards the enhancement of “evaluation from diplomatic viewpoints” in FY 2017. In response to these results, the ODA Evaluation Division revised the ODA Evaluation Guidelines (11th Ed.). However, we continue to experiment due to issues such as difficulty in proving the cause-and-effect connection between ODA outcomes and diplomacy.

Miyamori: Is there any other country commissioning “evaluation from diplomatic viewpoints” to a third party?

Prof. Yamaya: I don’t think there is. It’s an organizational issue when the ODA Evaluation Division, which is under Minister’s Secretariat, instead of the department taking charge of all foreign policies, must evaluate from the diplomatic perspectives. Nevertheless, you have done it so far. In my opinion, it cannot be more remarkable to have the number of evaluation reports including evaluation from diplomatic viewpoints, increase more and more. It will certainly be quite an advantage for MOFA once these reports get accumulated. Even if personnel change, successors will be able to understand easily, which will contribute to the continuation of diplomacy.

Miyamori: Are there any other Japanese ministries commissioning policy evaluation to a third party?

Prof. Yamaya: No. There is no other. In that sense, third-party ODA evaluation is quite unique, and that is also its strength. Other ministries have to conduct self-evaluation since it is stipulated by the law, and it is difficult to accumulate expertise for evaluation. On the contrary, MOFA’s ODA evaluation uses a considerable amount of external expertise, and also a specialist is appointed as a director of the ODA evaluation division. That contributes too much higher-level, high-quality reports. When you line up all evaluation reports, they sure become an asset.

Muraoka: It is important to be rational when we line them up. When evaluating from diplomatic viewpoints, sources of information are picked through trial and error. We have been putting a little more serious effort for the past few years, and it seems to be that we have stepped up to the next stage from where we started when we were just trying to put together something representable.

Prof. Yamaya: You are indeed entering the next stage. The reports can be used for educating researchers and scholars, as well as fostering consultants. The quality of evaluation will improve if there are many of such consultants. MOFA is developing human resources in that respect.

Muraoka: There are also challenges in rating when evaluating ODA.

Prof. Yamaya: Japanese society is very sensitive toward the word, “evaluation” written in Japanese, and they tend to misunderstand it or get offended by it, although that won't be a problem if written in English. I think everyone has a hard time dealing with that.

Muraoka: We hope to achieve a persuasive evaluation, including ratings from multiple perspectives while adhering to the principle of ensuring the independence of evaluators who make value judgments.