NZAID Tools Activity Cycle Tools 

Evaluation Series | Dissemination and Use of Evaluation Guideline

The Purpose of Evaluation

The NZAID Evaluation Policy Statement identifies three purposes for evaluation: accountability, learning and improvement.

This guideline outlines:

  • the ways that dissemination and use is linked to these purposes and outlines the principles, and
  • practices that assist in getting the best returns from evaluative activities

Evaluations can be useful to programmes and activities, not just in terms of the findings but also from working through the thinking processes required.

Accountability

Evaluation findings can contribute to meeting formal accountability requirements, for example, reporting to government on the effectiveness of NZAID’s development assistance. As well as evaluations of individual programmes and activities, NZAID produces annual reports that summarise evaluative information generated over the previous year. Review and evaluation reports complement information from other ongoing performance monitoring.

Learning

Learning from evaluations can happen both while the evaluation process is underway and more formally at the end through the reporting and dissemination process. These have been described in the literature as “process use” and “findings use”. This means that evaluations can not only contribute to the use of findings but also assist the people involved in the process to “think evaluatively”.

Reporting

Dissemination of evaluation findings involves actions to inform stakeholders about the information that has been produced by the evaluation. The reporting stage is an important part of any evaluation. This usually involves a written report and is ideally followed by communication with key stakeholders and other interested parties. This communication needs to be tailored to the interests and needs of the various audiences. A well designed dissemination strategy will enhance the use of an evaluation. Such a strategy needs to be worked out in consultation with internal and external stakeholders. This is best done at the planning stage rather than leaving it until the end of the evaluation.

A written report will be one product of an evaluation. Other dissemination methods can also be used. This is particularly important when disseminating findings to communities who have been involved in the evaluation but can also be useful to increase interest of other stakeholder groups. Interactive people to people transmission of the information is often more effective than paper reports. Seminars, workshops and meetings with presentations using different media are often an appropriate means of conveying the evaluative information. The cost of these methods needs to be built into the evaluation budget at the start.

During the planning stage of an evaluation discussions are held with the relevant stakeholders to determine when, how and to whom the evaluation findings will be delivered. A range of options can be considered with the emphasis on interactive approaches. The choice of approach is tailored to the particular needs and interests of the various audiences. In some cases this will require translation into another language.

NZAID evaluation reports are lodged in the Evaluation and Review Reference Collection (ERRC), a summary of each is made available through the NZAID website, and the full report may be requested.

NOTE: To facilitate dissemination via the website, any information that could prevent the release of a report under the Official Information or Privacy Acts, or would breach evaluation ethical standards should be placed in a Confidential Annex. Guidance on the kinds of information that must be placed in a Confidential Annex is available in the NZAID Guideline on the Structure of Review and Evaluation Reports. Guidance on NZAID’s process for screening review and evaluation reports to ensure they can be made available as summaries is available in the NZAID Evaluation and Research Committee Process Guideline.

It is important to give the minister or his office advance notice of the forthcoming release on the website of any summaries of review, evaluation or research reports that have political implications or contain sensitive information.

Use of Evaluative Information

Use of evaluative information occurs when the findings (and recommendations) are applied in some way to the activity or programme or, for findings that can be generalised, applied more widely. The key purpose of applying this type of information is for improvement of policy and programmes. This can happen both during and after implementation of the evaluation. The NZAID Evaluation Committee process is one mechanism that facilitates the use of information from evaluations. It is important to acknowledge that evaluation findings are just one input into decision-making about activities and programmes. Such decisions are affected by other considerations including political factors that can sometimes be contradictory to the evaluation findings. The quality and credibility of an evaluation is a factor that affects how influential it is and whether and how the findings are used. The information generated by evaluations must be both relevant and timely to be of most use.

Wherever possible, the evaluation should be planned so that the findings are available at a time when they can feed into and inform decisions. For a longer term evaluation, short reports can be planned to occur at regular intervals throughout the evaluation process with a complete report at the end. The style of written reports can affect uptake and use of the information, with some evidence that the less technical and formal reports are, the more likely it is that they are understood and used.

Research on evaluation suggests the following points regarding dissemination and use:

  • The more the audiences for an evaluation are involved in the planning, the more the findings will be used
  • If the evaluation addresses questions of importance to the audiences, it increases the likelihood that the findings will be used
  • The more interactive the reporting is, the more likely that findings will be used
  • Evaluation findings are only one information input into a decision making process

It is also worth noting that an evaluation of an activity, programme, strategy, or policy, can also be used to promote NZAID’s effectiveness. It is obviously an advantage if the findings are positive, but negative findings that have been used to improve an activity or programme can also be seen as positive. In the latter case, the organisation will be seen as having the capability to learn from experience and adapt to changing needs and situations.