(3) Conducting evaluation
In order to enhance the transparency of ODA projects and to improve its accountability, Japan has been working on the thorough implementation of the following measures: (i) enhancing the PDCA cycle (project formation (Plan), implementation (Do), evaluation (Check), follow-up activities (Act)), (ii) strengthening the Program Approach, and (iii) reinforcing “visualization.”
With regard to enhancing the PDCA cycle, the ongoing efforts include: (i) formulating Country Development Cooperation Policies for all recipient countries of Japan’s ODA, (ii) convening the Development Project Accountability Committee, (iii) setting indicators for individual projects, and (iv) strengthening the evaluation mechanism.
In order to implement ODA projects more effectively and efficiently, strengthening of the PDCA cycle not only at the project level but also at the policy level is required. To this end, MOFA conducts policy evaluations of economic cooperation policies based on the “Government Policy Evaluations Act (GPEA)” (Note 10), as well as ODAevaluations by third parties who approach the evaluation from a neutral position.
Third-party evaluations are mainly carried out at the policy-level (e.g., country assistance evaluations and Thematic/Aid-Modality evaluations). They evaluate from development viewpoints based on three evaluation criteria, namely, whether the policies and programs are consistent with Japan’s high-level ODA policies and the needs of the aid recipient countries (relevance of the policies), whether the initial goals have been achieved (effectiveness of the results), and whether an appropriate process was taken until the implementation of the policies (appropriateness of the process). In addition to evaluations from development viewpoints, evaluation is also conducted based on diplomatic viewpoints, considering the importance of verifying the diplomatic impact of implemented policies and programs. Since FY2015, basically all of MOFA’s third-party evaluations have been conducted using the diplomatic viewpoints, and efforts are now underway to improve evaluations from those viewpoints in order to clarify the diplomatic importance and diplomatic impact of ODA (how Japan’s ODA has contributed to realizing its national interests). As an example of how Japan’s ODA is perceived overseas, in the Opinion Poll on Japan in Ten ASEAN Countries (Note 11), close to 90% of the respondents indicated that Japan’s ODA has been helpful in the development of their countries.
Furthermore, the widespread publication of the evaluation results through the MOFA website and other means plays a role in fulfilling accountability to the public, in the aspects of how ODA is being used and what effects it has had. (Note 12)
JICA also conducts evaluations on respective projects of grant aid, ODA loan and technical cooperation, as well as thematic evaluations on each project. JICA conducts consistent monitoring and evaluation from the pre-implementation stage to the implementation, and post-implementation stages for each project, and has established a coherent evaluation mechanism for these three schemes of assistance. These evaluations are conducted in accordance with the DAC Criteria for Evaluating Development Assistance, and for projects that exceed a certain cost, post-project evaluations by third party evaluators (external evaluations) are carried out. JICA also strives to enhance impact evaluations (Note 13), recognizing the importance of quantitatively assessing the effects of their projects.
Issues regarding Budget Execution Management at JICA
Each year, the Japan International Cooperation Agency (JICA) Operation Expenses Grant is disbursed from the Ministry of Foreign Affairs (MOFA) budget to JICA as funds for the project expenses for technical cooperation and its administrative expenses. During FY2017, it was confirmed that out of this JICA Operation Expenses Grant, the estimated amount of necessary payments within the fiscal year for technical cooperation will greatly exceed the budgeted amount. In light of this situation, JICA urgently reduced expenditures within FY2017 by such means as slowing down the budget executions by rescheduling planned projects to subsequent fiscal years and suspending preparations for the implementation of new projects, in order to keep the expenditures within the budget of FY2017. This process perplexed the parties concerned, including the government officials in the countries where the projects had been scheduled, consultants and technical cooperation experts.
In response to the occurrence of this issue regarding budget execution management, MOFA and JICA immediately implemented steps to strengthen budget execution management for the projects implemented under the JICA Operation Expenses Grant in June 2018. They established the Budget Execution Management Office in JICA as the department with the clear authority and responsibility for the cross-sectoral management and control of the JICA budget in July 2018, and took measures such as strengthening governance through the board of directors and improving systems to bolster budget execution management. Moreover, they convened the “Advisory Committee to Strengthen Budget Execution Management” under the President of JICA to clarify the causes that created the situation, and provide recommendations for further measures to prevent recurrence. The Committee, comprised of external experts in the areas of organization and management, accounting management and incorporated administrative agency auditing, and IT systems, held deliberations over a total of eight sessions and complied its final report in December.
As the background of the issue, the report lists the fact that new projects were formulated enthusiastically and swiftly during the last Medium-term Objectives Period in response to the rising demands for funds, to cover the strong development needs in developing countries, accompanied by the intention to execute the entire budget for the period which ends in FY2016. It also mentions that the expected amount of expenditures in future fiscal years was not managed appropriately, and that JICA executed the budget assuming a down side in execution and additional budget allocations. Based on these considerations, the report provides specific recommendations such as the correction of the relaxation of prior control on the JICA budget, timely and appropriate budget execution management at the individual project level, appropriate estimation and management of future-fiscal-year expenditures, visualization of the state of budget execution, and clarification of the role of the board of directors and its members.
MOFA and JICA accept the fact that this budget execution management issue undermined the trust and expectations of the Japanese public, recipient countries and other related parties, and are going forward with the rectification of internal control through such means as the appropriate implementation of the recommendations of this report. MOFA and JICA will strive to recover trust, so that such an issue will never arise again.
- Note 10: Ex-ante evaluations are conducted on loan aid projects, in which the maximum amount of loan offered through an Exchange of Notes (E/N) is 15 billion yen or more, and on grant aid projects, in which the maximum amount of aid offered through an E/N is 1 billion yen or more. Ex-post evaluations are conducted on “pending projects” and “incomplete projects.” (“Pending projects” are projects for which the loan agreement has not been signed or loan disbursement has not begun after fi ve years have elapsed following the decision to implement the project. “Incomplete projects” are projects for which loan disbursements have not been completed after ten years have elapsed following the decision to implement the project.)
- Note 11: Results of an opinion poll on Japan commissioned by MOFA and conducted by a public opinion research agency, on 10 ASEAN countries (Brunei, Cambodia, Indonesia, Laos, Malaysia, Myanmar, the Philippines, Singapore, Thailand and Viet Nam) in March 2017. The poll targeted 300 respondents aged 18 to 59 in each country, and was conducted through a combination of online surveys and partially through face-to-face interviews.
URL: https://www.mofa.go.jp/press/release/press4e_001780.html - Note 12: ODA evaluation: https://www.mofa.go.jp/policy/oda/evaluation/index.html
- Note 13: Impact evaluation refers to a method for verifying the effects of development projects by using methods from statistics and econometrics.