Evaluation Series | DAC Evaluation Quality Standards
Implementation and reporting
3.1 Evaluation team
A transparent and open procurement procedure is used for selecting the evaluation team.
The members of the evaluation team possess a mix of evaluative skills and thematic knowledge. Gender balance is considered and the team includes professionals from partner countries or regions concerned.
3.2 Independence of evaluators vis-à-vis stakeholders
Evaluators are independent from the development intervention, including its policy, operations and management functions, as well as intended beneficiaries. Possible conflicts of interest are addressed openly and honestly. The evaluation team is able to work freely and without interference. It is assured of co-operation and access to all relevant information.
3.3 Consultation and protection of stakeholders
The full range of stakeholders, including both partners and donors, are consulted during the evaluation process and given the opportunity to contribute. The criteria for identifying and selecting stakeholders are specified.
The rights and welfare of participants in the evaluation are protected. Anonymity and confidentiality of individual informants is protected when requested or as needed.
3.4 Implementation of evaluation within allotted time and budget
The evaluation is conducted and results are made available to commissioners in a timely manner to achieve the objectives of the evaluation. The evaluation is carried out efficiently and within budget. Changes in conditions and circumstances are reported and un-envisaged changes to timeframe and budget are explained, discussed and agreed between the relevant parties.
3.5 Evaluation report
The evaluation report can readily be understood by the intended audience(s) and the form of the report is appropriate given the purpose(s) of the evaluation.
The report covers the following elements and issues:
3.6 Clarity and representativeness of summary
A written evaluation report contains an executive summary. The summary provides an overview of the report, highlighting the main findings, conclusions, recommendations and any overall lessons.
3.7 Context of the development intervention
The evaluation report describes the context of the development intervention, including:
-
policy context, development agency and partner policies, objectives and strategies;
-
development context, including socio-economic, political and cultural factors;
-
institutional context and stakeholder involvement.
The evaluation identifies and assesses the influence of the context on the performance of the development intervention.
3.8 Intervention logic
The evaluation report describes and assesses the intervention logic or theory, including underlying assumptions and factors affecting the success of the intervention.
3.9 Validity and reliability of information sources
The evaluation report describes the sources of information used (documents, respondents, administrative data, literature, etc.) in sufficient detail so that the adequacy of the information can be assessed. The evaluation report explains the selection of case studies or any samples. Limitations regarding the representativeness of the samples are identified.
The evaluation cross-validates the information sources and critically assesses the validity and reliability of the data.
Complete lists of interviewees and other information sources consulted are included in the report, to the extent that this does not conflict with the privacy and confidentiality of participants.
3.10 Explanation of the methodology used
The evaluation report describes and explains the evaluation methodology and its application. In assessing outcomes and impacts, attribution and/or contribution to results are explained. The report acknowledges any constraints encountered and how these have affected the evaluation, including the independence and impartiality of the evaluation. It details the techniques used for data collection and analysis. The choices are justified and limitations and shortcomings are explained.
3.11 Clarity of analysis
The evaluation report presents findings, conclusions, recommendations and lessons separately and with a clear logical distinction between them.
Findings flow logically from the analysis of the data, showing a clear line of evidence to support the conclusions. Conclusions are substantiated by findings and analysis. Recommendations and any lessons follow logically from the conclusions. Any assumptions underlying the analysis are made explicit.
3.12 Evaluation questions answered
The evaluation report answers all the questions detailed in the TOR for the evaluation. Where this is not possible, explanations are provided. The original questions, as well as any revisions to these questions, are documented in the report for readers to be able to assess whether the evaluation team has sufficiently addressed the questions, including those related to cross-cutting issues, and met the evaluation objectives.
3.13 Acknowledgement of changes and limitations of the evaluation
The evaluation report explains any limitations in process, methodology or data, and discusses validity and reliability. It indicates any obstruction of a free and open evaluation process which may have influenced the findings. Any discrepancies between the planned and actual implementation and products of the evaluation are explained.
3.14 Acknowledgement of disagreements within the evaluation team
Evaluation team members have the opportunity to dissociate themselves from particular judgements and recommendations on which they disagree. Any unresolved differences of opinion within the team are acknowledged in the report.
3.15 Incorporation of stakeholders’ comments
Relevant stakeholders are given the opportunity to comment on the draft report. The final evaluation report reflects these comments and acknowledges any substantive disagreements. In disputes about facts that can be verified, the evaluators investigate and change the draft where necessary. In the case of opinion or interpretation, stakeholders’ comments are reproduced verbatim, in an annex or footnote, to the extent that this does not conflict with the rights and welfare of participants.