Summary of FY2017 MOFA’s ODA Evaluation
Brief Summary for Evaluation Report on Individual Project under Grant Aid (Brief Summary)
Full text available here.
https://www.mofa.go.jp/policy/oda/evaluation/FY2017/pdfs/grand_aid.pdf 
| Evaluators (Evaluation Team) |
Chief Evaluator | Prof. Juichi INADA Senshu University, Japan |
| Advisor | Prof. Yasushi KATSUMA Graduate School of Waseda University |
|
| Consultants | Global Group 21 Japan, Inc. | |
| Period of the Evaluation Study | August 2017 - February 2018 | |
| Field Survey Country | Jordan and Palestine | |
Objective of Evaluation
Among the completed projects under the Grant Aid by Ministry of Foreign Affairs of Japan (MOFA) with project amount more than one billion Japanese yen, the following projects were evaluated this time;
1. Emergency Grant Aid for the Project to Implement Emergency Measures in Response to the Influx of Syrian Refugees (FY 2013 Jordan)
2. Non-Project Grant Aid (FY 2014 Palestine)
Until now, MOFA has been conducting policy/program level evaluations (third-party evaluations) rather than individual project level. To strengthen the feedback mechanism of its Grant Aid, MOFA decided to implement evaluation of individual projects.
1. Emergency Grand Aid for the Project to Implement Emergency Measures in Response to the Influx of Syrian Refugees (FY 2013 Jordan)
Project outline
A large-scale pro-democracy movement broke out in Syria on March 2011 that developed into the Syrian crisis, and more than 500,000 Syrian refugees inflowed into Jordan through September 2013. In reaction to this, the Japanese government extended Grant Aid in the amount of one billion Japanese yen to Jordan for emergency and humanitarian purposes. Specifically, Compactors, Wheel Loaders, Water Tankers and Vacuum Trucks were procured and delivered to the municipalities (host communities) receiving Syrian refugees scattered all around Jordan.
Summary of Evaluation
1 Evaluation from Development Viewpoints
(1) Relevance of Project (Evaluation result: C. partially unsatisfactory)
The policy decision to extend Emergency Grand Aid in 2013 when Jordan’s economic and social situation was severe and vulnerable, was evaluated as highly satisfactory. However, because the objective of this project was to respond to the emergency and humanitarian needs of the Syrian refugees in Jordan, whether the project’s scope suited that objective was the crucial point in evaluating the project relevance. This project contained a certain aspect of satisfying the development needs of municipalities who had been underserved before the Syrian crisis. Specifically, wheel loaders for construction works, not directly related to Syrian refugees, were procured under this project. Another such case is that a bulk of equipment was delivered to the southern region where the number of Syrian refugees was relatively small. These outcomes did not correlate well with the emergency and humanitarian purposes of the project. Therefore, the project scope should have been examined more carefully to respond to the project purposes properly. As a result, the project relevance was found to be partially unsatisfactory.
(2) Effectiveness of Results (Evaluation result: C. partially unsatisfactory)
Within the limited number of municipalities visited during the on-site survey, it was affirmed that the equipment needed in each host community was delivered, and were operated and maintained in good condition, in general. However, the project included some questionable equipment and coverage, as mentioned above (a), as to whether the project was truly an effective emergency and humanitarian project. Therefore, project effectiveness was judged partially unsatisfactory.
(3) Appropriateness of Processes (Evaluation result: C. partially unsatisfactory)
In light of the emergency nature of this project, it was found that the procedures were rapidly implemented. However, it was also found that there were some areas for further improvement, such as processes to examine and ensure the relevance of project scope, public relations and information disclosure, and with regards to maintaining record of changes in the project scope.
2 Evaluation from Diplomatic Viewpoints
The Japanese government has continuously extended its bilateral assistance to Jordan with a total amount of more than 100 billion Japanese yen, including a Yen ODA loan for the support of Syrian refugees. With this support, the relations between Japan and Jordan have been very good, including top leaders of both sides. For Japan, who relies on the Middle East region for most of its energy resources, supporting the stability of Jordan who has a diplomatic relations with Israel and proactively promoting the Middle East peace process, directly serves the overall stability of the Middle East, and in turn, meets a Japanese national interest as well. In the international society, such as the UN General Assembly, Japan has been appealing for contributions to Syrian refugees, and leading the donor community in Jordan recently.
However, since this Emergency Grand Aid was just a portion of the Japanese government’s support to Jordan, it is difficult to evaluate its importance and direct effect on the impact of the diplomatic relations between Japan and Jordan from this Emergency Grand Aid alone. Therefore, the series of Japan’s aid to Jordan was grasped from diplomatic viewpoints, and this was evaluated as a part of that.
Recommendation
1 Clarification of project scope according to each purpose of emergency / humanitarian assistance and development assistance
Although this project had emergency and humanitarian purposes, it included a strong development-oriented scope which was neither urgent nor humanitarian based. It should be made clear how to prevent such development objectives to enter into the scope of an emergency and humanitarian assistance project.
2 Enhancement of function to ensure the relevance of project scope in the project planning stage
In the project planning stage, not only is it important to respect the project contents requested by the recipient country, it is also important for MOFA to confirm the relevance of project scope swiftly in the beginning stage. This form of initial project assessment should be strengthened.
3 Preservation of discussion record of changes to project scope
When important changes to the project scope were made, the process of consideration should be made clear and the record should be appropriately kept.
4 Implementation of appropriate public relations
The information disclosure is limited for this project and the overall picture of Japanese support for Syrian refugees is not sufficiently communicated to the public. Therefore it is necessary to make more appropriate public relations about such projects to show the presence of Japan in the Middle East as a key region through fulfilling its global responsibility for the Syrian crisis.
5 Report of operation & maintenance from recipient country
Jordan was not required to report the operation & maintenance conditions after the delivery of equipment. In order to ensure project effectiveness, it is recommended to require the Ministry of Municipality Affairs of Jordan (the project executing ministry) to report back to Japan after some time, for example, three to five years after project completion.
2. Non-Project Grant Aid (FY 2014 Palestine)
Project outline
The economy of the Palestinian Authority (PA) continued to be in a difficult situation from the destruction of a wide range of social and economic infrastructure through the series of conflicts after September 2000, followed by drastic decrease in employment opportunities. To enhance the Middle East Peace Process in such conditions, it was necessary to stabilize and develop the livelihood of Palestinians by supporting economic and social development efforts of the PA. Non-Project Grant Aid (NPGA) of one billion Japanese Yen was extended for that purpose, and this NPGA was intended for the purchase of commodities critically needed for daily economic activities in the Palestinian Territories; gasoline and diesel oil. Furthermore, the fund accumulated through selling such commodities (counterpart fund) was to be utilized for economic and social development in the Palestinian Territories.
Summary of the Evaluation
1 Evaluation from Development Viewpoints
(1) Relevance of Project (Evaluation result: A. highly satisfactory)
In 2014, the Gaza conflict broke out further affecting the chronically difficult PA budget situation. This NPGA was extended under such backdrop as a part of international support to PA. Therefore, project relevance is considered to be highly satisfactory.
(2) Effectiveness of Results (Evaluation result: B. satisfactory)
The share of this one billion Yen NPGA against the total assistance to PA, and the share of the imported gasoline/diesel oil against the total such imports to Palestine Territories are small. However, from 2007 to 2014, NPGA of about one billion Yen was extended to PA annually, and as a whole, this was very much appreciated by PA as a stable and predictable source of financing. The counterpart fund under FY2014 NPGA was yet to be deposited and implemented at the time of on-site survey, but considering the past track records, the counterpart fund is expected to be used effectively for priority projects.
(3) Appropriateness of Processes (Evaluation result: A. highly satisfactory)
In light of implementing this NPGA in an emergency, the processes were taken without delay. Furthermore, cooperation with JICA’s technical cooperation was pursued actively, and implemented projects were regularly monitored by the PA and the Representative Office of Japan to the PA (ROJ).
2 Evaluation from Diplomatic Viewpoints
Support to the PA from the Japanese government since 1993 stood cumulatively at USD 1.78 billion (as of July 2017), and since 2007, ten Non-Project Grant Aids to the PA totaling 10.2 billion Yen have been extended. Such steady and continuous support is considered to have helped develop favorable relations between Japan and Palestine.
The Representative Office of Japan to the PA (ROJ) was proactively using development assistance as a means of diplomacy by collaborating with JICA in utilizing the counterpart fund of NPGA. ROJ was also playing an important role for enhancing both development and diplomatic impact through brisk public relations activities including SNS.
However, since this NPGA was just a portion of the Japanese government’s support to the PA, it is difficult to evaluate its importance and direct effect on the impact of the diplomatic relations between Japan and Palestine from this NPGA alone. Therefore, the series of Japan’s assistance to the PA was grasped from diplomatic viewpoints, and this NPGA was evaluated as a part of that.
Recommendations
1 More collaboration between MOFA Headquarters and its Overseas Establishments
Submission of the completion report about NPGA’s counterpart fund projects from recipient government to MOFA is a prerequisite for MOFA to examine the new counterpart fund project requests from the recipient government. Therefore, it is important for both MOFA Headquarters and its Overseas Establishments to collaborate more closely to be able to achieve further diplomatic impact and accountability to the public at the same time.
2 Compilation of counterpart fund project operations as office references
A combination of counterpart fund projects monitored by ROJ and technical cooperation by Japan International Cooperation Agency (JICA) were actively pursued and implemented at the field level. Such examples of collaboration of different schemes should be compiled as office references and shared among concerned parties elsewhere.
3 Implementation of more active domestic public relations activities
The information disclosure of this NPGA was limited, and the publicizing of information on counterpart fund projects was also not sufficient in Japan. Since counterpart fund projects were highly appreciated in Palestine, MOFA should implement more active information disclosure and domestic public relations activities. That in turn will improve the understanding of NPGA scheme among a wider public audience, and further enhance diplomatic impact.