FY2020 MOFA ODA Evaluation Results
Analysis of Third-Party Evaluations of Bilateral Grant Aid Projects Conducted by MOFA and Proposal of Evaluation Methods
| Chief Analyst | SATO Kan Hiroshi Chief Senior Researcher, Research Operations Department, IDE-JETRO |
| Senior Advisor | Prof. INADA Juichi School of Economics, Senshu University |
| Consultant | International Development Center of Japan Incorporated |
Background and Scope of Work
● Background
In evaluating individual bilateral grant aid projects implemented by MOFA over the past three years, the same methodology has been used as when evaluating ODA policies. However, both evaluation teams and project stakeholders have pointed out numerous incongruences and issues with the use of this methodology for the evaluation of individual projects due to the following characteristics.
- Bilateral grant aid conducted by MOFA consists mainly of foreign-currency assistance for the purchase of goods or materials such as Economic and Social Development Programs and is positioned as “aid that must be implemented in close coordination with diplomatic policies and may require flexibility.” The nature of the projects is therefore different from that of those implemented by JICA.
- While bilateral grant aid projects are of great significance in terms of diplomatic strategy, there are limitations to reviewing each project’s diplomatic impact and quantitative effects.
- Some of the verification items contained in the evaluation criteria and contents of analysis are redundant.
● Scope
To propose an evaluation framework and methodology fully in line with the characteristics of individual bilateral grant aid projects conducted by MOFA.
Main Points of Proposed New Evaluation Methods
● Evaluation Criteria
- Projects shall have two evaluation criteria, “Relevance of Plans” and “Effectiveness of Implementation and Results.”
- “Appropriateness of Processes” shall be included within the evaluation questions for “Relevance of Plans” and “Effectiveness of Implementation and Results,” instead of as an independent evaluation criterion. Processes should also be checked for factors such as their transparency.
- “Diplomatic Viewpoints” shall be merged with “Development Viewpoints.” Verification items for “Diplomatic Importance” shall be merged with “Relevance of Plans,” and verification items for “Diplomatic Impact” with “Effectiveness of Implementation and Results.”
● In the Event of an Obligation to Accumulate Counterpart Funds(Note)
- These shall essentially be included among the targets of evaluation. However, while evaluation sub-questions shall be posed separately from “accumulation” and “application/projects,” and the actual “application/ projects” shall be subjects of study, considering that the Government of Japan does not make “immediate use” a requirement, whether to include them among the targets of evaluation and rating shall be deliberated separately for each project.
(Note) Funds accumulated by the government of the recipient country as the proceeds from the sale of the goods or materials procured through the grant aid. These funds can be used for projects, the procurement of goods or materials, etc. that will contribute to economic and social development in the recipient country based on deliberations on their application with the Government of Japan.
● Ratings
- The rating method shall encourage the explicit indication of the basis for “Lessons Learned.”
- Four-level ratings are proposed, as are used for MOFA’s policy-level ODA evaluations and JICA’s grant aid postproject evaluations. However, individual evaluation reports shall not use four-level alphabetical ratings such as “A” through “D.”
● Project Documents and Materials
- The basic set of project documents shall be provided immediately after contracting with third-party evaluators under a strict obligation of confidentiality.