Activity Monitoring
Annex 1 – Activity Design Principles of Monitoring
Indicators and their role in monitoring
Indicators help to make the process of assessing performance and outcomes manageable. Indicators are inevitably partial, selective and subjective. They do not have any objective validity in themselves but rather are…
things that serve to give an indication, sign, symptom, hint or suggestion of something else
There is unlikely to be any “correct” set of indicators for assessing the performance, outcomes and impact of a development intervention, but rather a range of possible signs which can be detected and used to help measure these, with varying degrees of certainty. A key task is to make the selection of indicators and their analysis more useful, less arbitrary and more accountable.
Indicators may be qualitative or quantitative. Quantitative indicators are expressed in numerical form, however they vary in precision. They can record precise amounts, (e.g. wages actually paid) or estimate rough quantities (e.g. unrecorded income from informal sector activities). Qualitative indicators are expressed in verbal form. They may assess observable characteristics (e.g. villagers’ perceptions of whether they are poor or not), as well as ideas or attitudes (e.g. whether or not women feel they have more self-confidence). Data from assessments linked to qualitative indicators can also be categorised and ranked like quantitative indicators to varying degrees of precision.
Indicators also vary in their degree of directness or “proximity”. For example, the type of material used in house construction may be a direct indicator of the impact of a housing improvement programme, or an indirect “proxy” indicator for increased wealth in a rapid household poverty assessment.
When considering the selection of indicators it is important to bear in mind that:
- Indicators are partial and selective. Underlying values inevitably influence the selection of any particular set of indicators.
- The relative significance of indicators will vary according to the values, priorities and levels of knowledge of different stakeholders and their ability or willingness to respond.
- When the main aim is programme learning and improvement, a ‘most significant change approach’ (see below) can be a useful complement to other monitoring methods.
- Both quantitative and qualitative aspects should be measured, e.g. it is not enough to know how many people have been trained, we also need to know what they have learned and whether they are successfully applying their new knowledge.
- More information is not necessarily better, and collecting too much information can waste scarce resources.
- Human indicators should usually be disaggregated by gender (e.g. number of boys and number of girls entering tertiary education) and by socio-economic groups (e.g. number of students from outer islands or poor households entering tertiary education).
Different approaches for selecting and assessing indicators
A number of approaches have been developed for selecting indicators.
SMART criteria are oriented towards enhancing the speed and ease of data collection.
Specific to the intended changes
Measurable and unambiguous
Attainable or achievable by the Activity
Relevant to the Activity
Time bound
SPICED criteria give particular emphasis to participatory approaches to development.
Subjective in that informants are uniquely placed to offer insight based on their experience
Participatory, involving these informants
Interpreted or explained to provide an understanding of the local context in which they occur
Cross-checked against other indicators, informants and methods
Empowering to affected groups
Diverse in nature and measured from a variety of informant groups
CREAM criteria focus on managing for development results; and the Most Significant Change approach enables stakeholders to identify changes that may not originally have been envisaged.
Clear: Precise and unambiguous
Relevant: appropriate to the subject at hand
Economic: available at a reasonable cost
Adequate: provide a sufficient basis to assess performance
Monitorable: amenable to independent validation
The most significant change technique
When using the most significant change technique beneficiaries and/or staff are asked to report the most significant changes (positive, negative, planned, unplanned) over the last period and explain why they have identified these as significant. This approach allows different stakeholders to identify those changes that are important to them, and these can then be compared with the original indicators established for the Activity. The most significant change technique should be used to complement one of the other techniques.