Summary of Other ODA Evaluations
ODA Evaluation based on the Government Policy Evaluations Act (GPEA)
With the entry into force of the GPEA in 2002, each ministry and agency is required to conduct self-evaluations of policies under its jurisdiction. ODA evaluations based on the GPEA have been undertaken since then.
The Ministry of Foreign Affairs (MOFA) conducts ex-post evaluations of overall policy on economic cooperation, ex-post evaluations on pending projects and incomplete projectsNote 1, and exante evaluations on projects exceeding a certain valueNote 2 in accordance with the GPEA and its Order for Enforcement.
● MOFA website:
https://www.mofa.go.jp/mofaj/gaiko/oda/shiryo/index_hyouka05.html 
Note 1: “Pending projects” are projects for which the loan agreement has not been signed or loan disbursement has not begun after five years have elapsed following the decision to implement the project. “Incomplete projects” are projects for which loan disbursements have not been completed after ten years have elapsed following the decision to implement the project.
Note 2: Ex-ante evaluations are conducted on loan aid projects which maximum amount of loan offered through an Exchange of Notes (E/N) is ¥15 billion or more, and on grant aid projects which maximum amount of aid offered through an E/N is ¥1 billion or more.
Other ministries and agencies of the Government of Japan also evaluate ODA-related policy planning, programs, and projects implementation under their jurisdiction based on the GPEA. For details, please refer to the websites of each ministry and agency.
● Financial Services Agency
Training for officials in charge of financial administration/supervision in developing countries in Asia and other areas (Seminar to financial administrators/supervisors)
https://www.fsa.go.jp/seisaku/index.html
(See page 114 of the FY 2018 Policy Evaluation Report)
● Ministry of Internal Affairs and Communications
Promotion of global strategy in the ICT sector
http://www.soumu.go.jp/menu_seisakuhyouka/kekka.html
(See the evaluation report for main policies, implemented in FY 2018) (The programs include non-ODA projects.)
● Ministry of Justice
Promotion of international cooperation in legal affairs
http://www.moj.go.jp/hisho/seisakuhyouka/kanbou_hyouka_hyouka01-03.html
(See page 156-192 of the FY 2017 Ministry of Justice Ex-Post Evaluation Results Report)
● Ministry of Finance
Assistance extended through Multilateral Development Banks
https://www.mof.go.jp/about_mof/policy_evaluation/mof/fy2018/evaluation/30hyouka.pdf
(See page 216 of the FY 2018 Ministry of Finance Evaluation Report)
● Ministry of Education, Culture, Sports, Science and Technology
Promotion of international exchange
http://www.mext.go.jp/a_menu/kouritsu/detail/1405275.htm
(The programs include non-ODA projects.)
● Ministry of Health, Labor and Welfare
Participation in and contribution to the international community
https://www.mhlw.go.jp/wp/seisaku/hyouka/keikaku-kekka.html#hyouka
(See the preliminary analysis chart of the policy evaluation)
● Ministry of Agriculture, Forestry and Fisheries
Formulation of comprehensive food security against various risks
http://www.maff.go.jp/j/assess/hanei/sougo/h30/attach/pdf/h30-1.pdf
(The programs include non-ODA projects.)
● Forestry Agency
International cooperation and contributions for promotion of sustainable forestry management
http://www.maff.go.jp/j/assess/hanei/zisseki/h29/pdf/sheet28_17.pdf
(The programs include non-ODA projects; ODA evaluation can be found in Measure 8 on page 17-4.)
● Ministry of Economy, Trade and Industry
Support to overseas markets development
https://www.meti.go.jp/policy/policy_management/seisaku_hyoka/2018/index.html
(See the ex-post evaluation 4-2 of FY 2018 policy evaluation report) (The programs include non- ODA projects.)
● Ministry of Land, Infrastructure, Transport and Tourism
Promoting International Cooperation and Coordination
http://www.mlit.go.jp/common/001303481.pdf
(The programs include non-ODA projects.)
● Ministry of the Environment
International coordination and cooperation for global environmental conservation
http://www.env.go.jp/guide/seisaku/index.html
(See the ex-post evaluation of FY 2018 policy evaluation report) (The programs include non-ODA projects.)
Furthermore, since 2017, MOFA has been conducting internal evaluation on individual projects under Grant Aid implemented by MOFA.
https://www.mofa.go.jp/mofaj/ms/oda/page24_000056.html
Evaluations by Japan International Cooperation Agency (JICA)
JICA conducts evaluations (external evaluations by external third-party evaluators based on the project cost and internal evaluations by JICA overseas offices, etc.) on individual projects of three development assistance schemes of technical cooperation, ODA loan, and grant aid (implemented by JICA). In addition, JICA implements comprehensive and cross-sectoral evaluations under specific themes, such as by region, sector, or assistance methodology, etc., impact evaluations for evidencebased practice, and process analysis with a focus on the processes through which projects effects are produced.
From the perspective of learning, JICA aims to utilize evaluation results for further improvement of project implementation, and to fulfill its accountability, such as ensuring the objectivity and transparency of evaluations and publicizing evaluation results etc.
● Evaluation on the JICA website
https://www.jica.go.jp/english/our_work/evaluation/index.html
Partner Country-led Evaluation
MOFA implements partner country-led evaluation once a year, aiming to enhance partner country’s evaluation capacity, where a governmental and/or private consulting organization evaluates programs under specific themes such as health, transportation, and disaster risk reduction.
MOFA Partner Country-led Evaluations for FY 2018
Partner Country-led Evaluations Report on Third Country Training Programmes in the Environment Sector 2012-2018 in the Framework of Japan-Mexico Partnership Programme (JMPP) (Brief Summary)
Full text of the summary and report is available here:
https://www.mofa.go.jp/policy/oda/evaluation/FY2018/pdfs/jmpp_summary.pdf
https://www.mofa.go.jp/mofaj/gaiko/oda/files/000496684.pdf
| Country | The United Mexican States |
| Evaluators | Mexican Agency for International Development Cooperation (AMEXCID) |
| Consultant | Eriko Yamashita |
| Evaluation Period | November 13, 2018 to February 28, 2019 |
Monitoring mission of TCTP: advisory activities by IMTA officials in Honduras
Evaluation Description
(1) Background
Japan-Mexico Partnership Programme (JMPP) celebrated its 15th anniversary in 2018, and Mexican Agency for International Development Cooperation (AMEXCID) and Ministry of Foreign Affairs of Japan (MOFA) agreed to conduct this partner country-led evaluation to analyze the outcomes generated by JMPP in its beneficiary countries as well as in Mexico along with analysis on its process.
(2) Evaluation Purpose
Objectives of this evaluation study are as follows: review Japan’s overall policies to JMPP; use lessons learned from this review as a reference in policy planning, as well as in its effective and efficient implementation, of Japan’s future assistance to Mexico; ensure Japan’s accountability by making the evaluation results widely available to the general public.
(3) Evaluation Targets
This evaluation covers four Third Country Training Programmes (TCTP) implemented under JMPP in the environment sector between 2012 and 2018 (equivalent to FY 2012 - 2017 in Japan).
(4) Evaluation Methodology
The evaluation was conducted in line with the following three criteria based on the Guidelines for the Partner Country-led Evaluation (June 2018) by MOFA.
- Relevance of Policies
- Effectiveness of Results
- Appropriateness of Processes
Evaluation results
(1) Relevance of Policies
Japan’s support for regional development in Latin America in the environment sector is highly consistent with the high-level policies of both Japan and Mexico at the time of planning as well as ex-post evaluation. For Mexico, it is consistent with the International Development Cooperation Law (2012), National Development Plan 2013-2018, and other related legal frameworks in the environment sector. Strengthening the capacity of Mexican institutions for international cooperation through the target TCTPs in the environment sector was highly consistent with Japan’s ODA policies. Mexican regulations for AMEXCID and the target TCTP implementing agencies in the environment sector also confirmed the importance of international cooperation through strengthening regional cooperation as well as the promotion of triangular cooperation.
(2) Effectiveness of Results
Most of the specific goals established for each TCTP were achieved, and the effectiveness of the target TCTPs was high at the time of finalizing the four target TCTPs.
It was confirmed that several participating institutions implemented their country’s political instruments and pilot projects as results of their Action Plans developed in the TCTPs. Through this process, the knowledge and techniques gained from the TCTPs were also disseminated, and they contributed to the improvement of the institutional capacities of the beneficiary institutions.
The capacities of AMEXCID and the Mexican implementing agencies of the target TCTPs to conduct international cooperation were strengthened through the target TCTP implementation. AMEXCID, JMPP’s coordinating organization, has demonstrated its increasing capacity to support the related Mexican agencies and actors from both technical and financial perspectives in the target TCTPs.
(3) Appropriateness of Processes
The high-level decision-making process by the JMPP Planning Committee has been recognized as a strength of JMPP. Additionally, TCTP in the JMPP framework was confirmed to be one of the most advanced schemes of international cooperation in Mexico in terms of its established mechanisms and operational tools for implementation, which were developed during this evaluation target period.
The diagnostic activities in the target TCTP planning process used to identify the needs and demands of the beneficiary countries were an integral and critical component that enhanced the comparative advantages of the JMPP approach. Among them, AMEXCID and the Mexican implementing agencies highly appreciated JICA’s important role, especially for JICA’s regional network and technical support.
In terms of monitoring and follow-up mechanisms, the target TCTPs emphasized providing institutional follow-up during the three years of the implementation period. Its effectiveness in generating concrete results was highly regarded by the beneficiary countries as well as the Mexican implementing agencies. However, at the time of this evaluation, the practice of monitoring activities is not systematically structured in the established TCTP process.
Recommendations
(1) Enhance strategic selection process of TCTP beneficiary participants.
(2) Strengthen and integrate monitoring and evaluation mechanisms in the TCTP implementation framework to increase the effectiveness of TCTPs.
(3) Support sustainability of the TCTP achievements to generate further impact.
(4) Develop further triangular cooperation projects, including TCTPs in the environment sector, where JMPP’s comparative advantages are substantially recognized by the beneficiary countries as well as the Mexican stakeholders.
(5) Strengthen and expand the coordination role of AMEXCID further.
(6) Strengthen the visibility of JMPP and increase its publicity activities.