Understanding the terminolgy
Understanding the terminolgy
Evaluation terms have evolved considerably over recent years and many do not have standard definitions. Within NZAID it is important that there is both consistency in the use of terms, and clarity about their meaning. Several donor agencies describe evaluation as the retrospective assessment of a completed activity. Some donors include prospective assessments and reviews of on-going activities in their definition of evaluation and others include assessments of activities that are still in the planning stages. In 1998 a DAC study concluded that evaluations are most useful when they are linked to improvement of current programmes and planning for future programmes rather than retrospective analysis of completed interventions. In recognition of this NZAID has adopted the DAC definition of evaluation:
An assessment, as systematic and objective as possible, of a planned, on-going or completed development intervention.
This definition describes evaluation in its broadest sense, encompassing all types of evaluative processes or assessments, and emphasises that evaluation is not a discrete event at the end of an activity, but rather the integration of evaluative processes throughout the activity cycle. NZAID also uses the term evaluation in a more restricted sense to refer to assessments of activities that have reached the completion stage. To avoid confusion between these two usages, NZAID refers to the range of evaluative processes occurring over the course of the activity cycle as Evaluative Activities. NZAID uses the term Evaluationto describe assessments of outcomes and/or impact carried out after a project or programme has become fully operational, during the completion stage, or some time after the intervention has been completed.
The following terms are used in a variety of ways to describe evaluative activities:
- Outcome
- Refers to short and medium term effects of a development Activity.
- Impact
- Refers to the wider, deeper and long-term effects of a development Activity.
- Impact evaluation
- Sometimes associated with the establishment of causal attribution between a development Activity and immediate, intermediate or longer-term outcomes or results.
A given evaluative activity may assess both outcomes and impact. The assessment of outcomes and impact requires working with stakeholders to identify the positive, negative, intended and unintended effects that occur during or as a result of a development activity. The assessment of outcomes and impact considers the overall effects of an activity on poverty and sustainability by assessing its social, cultural, gender, economic, financial, political, institutional, environmental and technical aspects.
The establishment of attribution requires the use of methodology such as pre-post intervention comparisons and with/without comparisons. This requires considerable forward planning, and a high level of resources. There are also ethical and technical considerations that constrain the use of these approaches in many development settings. In many cases it is more feasible and cost-effective to assess the contribution of development activities to the achievement of outcomes.
- Monitoring
- The systematic collection of data to provide management and the main stakeholders of an on-going development intervention with indications of how allocated funds are being used and progress towards achievement of expected outcomes. Monitoring may be formal and/or informal.
- Review
- Describes evaluative activity taking place at key points during the lifetime of an Activity to gain a better understanding of what is being achieved and to identify how implementation can be improved. Reviews lie in between monitoring and full evaluations. They check that the overall direction is still relevant and ask whether the development Activity is likely to meet the purpose for which it was planned.
NZAID distinguishes a variety of evaluative activities that occur early in the Activity Cycle, before implementation has begun. These pre-implementation activities include but are not limited to: appraisal; needs assessment; literature review/desk studies; scoping studies; feasibility studies; the development of programme theory/logic; and planning for monitoring, review and evaluation.