Evaluation Series | DAC Evaluation Quality Standards
Purpose, planning and design
2.1 Rationale and purpose of the evaluation
The rationale, purpose and intended use of the evaluation are stated clearly, addressing: why the evaluation is being undertaken at this particular point in time, why and for whom it is undertaken, and how the evaluation is to be used for learning and/or accountability functions.
For example the evaluation’s overall purpose may be to:
-
contribute to improving a development policy, procedure or technique,
-
consider the continuation or discontinuation of a project or programme,
-
account for public expenditures and development results to stakeholders and tax-payers.
2.2 Specific objectives of the evaluation
The specific objectives of the evaluation clarify what the evaluation aims to find out. For example to:
-
ascertain results (output, outcome, impact) and assess the effectiveness, efficiency, relevance and sustainability of a specific development intervention,
-
provide findings, conclusions and recommendations with respect to a specific development intervention in order to draw lessons for future design and implementation.
2.3 Evaluation object and scope
The development intervention being evaluated (the evaluation object) is clearly defined, including a description of the intervention logic or theory. The evaluation scope defines the time period, funds spent, geographical area, target groups, organisational set-up, implementation arrangements, policy and institutional context and other dimensions to be covered by the evaluation. Discrepancies between the planned and actual implementation of the development intervention are identified.
2.4 Evaluability
The feasibility of an evaluation is assessed. Specifically, it should be determined whether or not the development intervention is adequately defined and its results verifiable, and if evaluation is the best way to answer questions posed by policy makers or stakeholders.
2.5 Stakeholder involvement
Relevant stakeholders are involved early on in the evaluation process and given the opportunity to contribute to evaluation design, including by identifying issues to be addressed and evaluation questions to be answered.
2.6 Systematic consideration of joint evaluation
To contribute to harmonisation, alignment and an efficient division of labour, donor agencies and partner countries systematically consider the option of a joint evaluation, conducted collaboratively by more than one agency and/or partner country.
Joint evaluations address both questions of common interest to all partners and specific questions of interest to individual partners.
2.7 Evaluation questions
The evaluation objectives are translated into relevant and specific evaluation questions. Evaluation questions are decided early on in the process and inform the development of the methodology. The evaluation questions also address cross-cutting issues, such as gender, environment and human rights.
2.8 Selection and application of evaluation criteria
The evaluation applies the agreed DAC criteria for evaluating development assistance: relevance, efficiency, effectiveness, impact and sustainability. The application of these and any additional criteria depends on the evaluation questions and the objectives of the evaluation. If a particular criterion is not applied and/or any additional criteria added, this is explained in the evaluation report. All criteria applied are defined in unambiguous terms.
2.9 Selection of approach and methodology
The purpose, scope and evaluation questions determine the most appropriate approach and methodology for each evaluation. An inception report can be used to inform the selection of an evaluation approach. The methodology is developed in line with the evaluation approach chosen.
The methodology includes specification and justification of the design of the evaluation and the techniques for data collection and analysis. The selected methodology answers the evaluation questions using credible evidence. A clear distinction is made between the different result levels (intervention logic containing an objective-means hierarchy stating input, output, outcome, impact).
Indicators for measuring achievement of the objectives are validated according to generally accepted criteria, such as SMART (Specific, Measurable, Attainable, Realistic and Timely). Disaggregated data should be presented to clarify any differences between sexes and between different groups of poor people, including excluded groups.
2.10 Resources
The resources provided for the evaluation are adequate, in terms of funds, staff and skills, to ensure that the objectives of the evaluation can be fulfilled effectively.
2.11 Governance and management structures
The governance and management structures are designed to fit the evaluation’s context, purpose, scope and objectives.
The evaluation governance structure safeguards credibility, inclusiveness, and transparency. Management organises the evaluation process and is responsible for day-to-day administration. Depending on the size and complexity of the evaluation, these functions may be combined or separate.
2.12 Document defining purpose and expectations
The planning and design phase culminates in the drafting of a clear and complete written document, usually called “Terms of Reference” (TOR), presenting the purpose, scope, and objectives of the evaluation; the methodology to be used; the resources and time allocated; reporting requirements; and any other expectations regarding the evaluation process and products. The document is agreed to by the evaluation manager(s) and those carrying out the evaluation. This document can alternatively be called “scope of work” or “evaluation mandate”.