Japan's Official Development Assistance White Paper 2007
Next PageMain Text > Part II ODA ODA Record for FY2006 > Chapter 2 Details about Japan's Official Development Assistance > Section 6. Formulation and Implementation of Aid Policy > 3. Matters Essential to Effective Implementation > (1)Enhancement of Evaluation
3. Matters Essential to Effective Implementation
(1) Enhancement of Evaluation
In order to implement ODA more effectively and efficiently, it is important to get proper and accurate knowledge of the implementation process and the effects in order to improve them when necessary. It is also important to explain to taxpayers how ODA is used and how effective it is. To this end, relevant ministries and agencies including MOFA as well as implementing agencies such as JICA and JBIC conduct monitoring and evaluation.
ODA evaluation is positioned in the framework of PDCA Cycle (Plan → Do → Check → Act). The evaluation results provide feedback to the departments in charge so as to contribute to improving the formulation and implementation of ODA policy. The results are also conveyed to the parties concerned in recipient countries and released to the general public through websites and other resources to ensure accountability.
Evaluations conducted by MOFA focus particularly on policy-level evaluations (country assistance policies and priority issues) and program-level evaluations (sectors and aid modalities). Furthermore, third-party evaluations have been promoted to ensure objectivity.
< Policy-level and program-level evaluation >
In FY2006, MOFA conducted country assistance evaluations for Viet Nam, Bhutan, Morocco, Zambia, and Madagascar. Japan conducted evaluations with particular emphasis on whether its assistance policies were consistent with the needs of developing countries, whether these policies had been effective, and whether assistance had been provided through appropriate implementation processes.
For example, in the country assistance evaluation for Viet Nam, it was not only recognized that the basic objectives and direction of Viet Nam's economic development plans were fully consistent with Japan's assistance policies, but it was also concluded that Japan's assistance contributed substantially to Viet Nam's poverty reduction efforts. As recommendations for the future, the evaluation pointed out that new social and economic issues would be likely to arise upon Viet Nam's entry into the World Trade Organization (WTO) and that assistance aimed at resolving such issues must be emphasized. The evaluation report also indicated that it was desirable not only to formulate assistance objectives between Japan and Viet Nam but also to share final goals.
Priority issues evaluations focused on issues including agriculture and farm community development, environmental and forest preservation, and regional cooperation (in Central America). In these evaluations relevance of Japan's policy with the undertakings of the international community, the effectiveness of the ODA projects, and the appropriateness of the process were evaluated. As a result, it was confirmed that, in the area of environmental and forest preservation, synergy effects could be realized by providing grant aid, technical cooperation, and loan aid taking advantage of the special characteristics of each instrument and thereby forming linkages between them. It was also shown that the aid instrument that proves effective differs depending on the recipient country's skills and whether or not it has formulated afforestation plans, which means that it is important to apply the scheme that is appropriate for assisting self-help efforts of the recipient country.
In program-level evaluations, in collaboration with NGOs, MOFA conducted sectoral assistance evaluation for Thailand in the health care sector on a one country, one sector basis. In addition, MOFA conducted aid modality evaluation on development research.
< Project Level Evaluations >
Evaluation of individual projects is one of the areas to be enhanced, as it is important for accountable to the public, effective and efficient aid.
For all yen loan projects, JBIC conducts ex-ante evaluations at the preparatory stage of all the projects. Two years after the completion of projects JBIC conducts ex-post evaluations by external evaluators in accordance with international standards for evaluation of relevance, efficiency, effectiveness, impact, and sustainability. In order to develop a more thorough evaluation system, JBIC has conducted "mid-term reviews" since FY2004 to verify the appropriateness and effectiveness of the project plan five years after the conclusion of loan agreements, along with "ex-post monitoring" seven years after the completion of projects to verify effectiveness, impact, and sustainability.
Furthermore, impact assessments that quantitatively analyze the effect of yen loan projects on people are conducted. For example, in addition to the ex-post evaluation of "Social Sector Development Project in Amazon Area/ Social Sector Development Project in Sierra Area" in Peru, thematic evaluation concerning the improvement of the living environment and livelihoods in poor communities was conducted. This evaluation, which employed econometric techniques, analyzed the impact of small-scale infrastructure projects (e.g., water supply, roads, small-scale electrification) on residents financed by the social investment funds established under the Fujimori administration.16 As a result, it was confirmed that water drawing time was reduced and the incidence of diarrhea among children under six declined in beneficiary households in regions in which water supply projects were implemented in comparison with regions in which they were not. It was also confirmed that small-scale electrification projects led to an increase in business start-ups among beneficiary households.
In technical cooperation projects, JICA worked consistently to conduct evaluations at each project stage: prior to commencement, during implementation, at the time of conclusion and following completion. In addition, the recommendations and lessons learned through these evaluations were systematically fed back to inform project planning and implementation. Furthermore, in order to enhance the transparency and objectivity of evaluations, the participation of outside experts in evaluations has been expanded through, for example, secondary evaluations by outside experts of the results of evaluations at the time of conclusion and the inclusion of comments by outside experts in the results of ex-post evaluations.
In the Project for the Development of Human Resources in Health in Senegal, the mid-term evaluation conducted during implementation in 2004 found the multiple implementing agencies had identified differently the initial target of the project, i.e., healthcare personnel working in the primary healthcare system. Their activities were also being conducted in the field according to different priority rankings. In response to these circumstances, the chiefs of healthcare posts, who were involved in all of the various activities, were clearly defined as the target, and it was recommended that a coordinating conference be held periodically among the implementing agencies. As a result of these recommendations, the implementing agencies effectively used the results of the project in the latter half of the project to produce synergies, and in the evaluation conduct at the completion of the project in 2006, it was confirmed that favorable results had been obtained.
With regard to grant aid, MOFA has conducted ex-post evaluations of projects since FY2005 to check how facilities and equipment are used and how effective they are in identifying problems and difficulties of each project. In FY2006, ex-post evaluations of 81 projects were conducted. Among these, MOFA conducted primary evaluations of 69 grant aid projects exceeding ¥1 billion which were completed three to five years ago in 38 countries. In FY2006, MOFA also introduced third-party secondary evaluations to verify the validity and appropriateness of primary evaluations. Furthermore, project-level ex-post evaluations by sector, country and region have been conducted through third-party entrustment since FY2006. The lessons obtained from those ex-post evaluations will be reflected in the development and implementation of new projects.
< Seeking a comprehensive evaluation system suited to the inauguration of the new JICA >
Under the new JICA to be inaugurated in October 2008, study is now being given to establishing a monitoring and evaluation system suitable for all assistance instruments, be it technical cooperation, loan aid or grant aid. With respect to the program-level evaluations performed respectively by MOFA and the new JICA, it has been decided to set up an effective and efficient division of roles that eliminates overlap, with both offices performing coordination at the stage of evaluation plan formulation.