NZAID Tools Activity Cycle Tools 

Evaluation Series | Developing Terms of Reference for Reviews and Evaluations

What to include in your TOR

The detailed content of TOR will differ for each review or evaluation. The TOR will be tailored to the situation and particular activity being reviewed or evaluated. There are, however, generic elements of a TOR for reviews and evaluations that need to be considered:

Background information and context

Provide a succinct summary of the:

  • history of the activity being evaluated or reviewed
  • purpose, aims, objectives and intended outcomes of the activity, and how these have changed over time (append the Monitoring and Evaluation (M&E) framework to the TOR if appropriate)
  • organisational, social and political context in which the review or evaluation will occur
  • main stakeholders involved in the activity - including targeted beneficiaries, community groups, civil society organisations, implementing agents, partners and donors.

The rationale and purpose of the review or evaluation

Clearly explain why the review or evaluation is being undertaken and how it will benefit the stakeholders:

  • why the review or evaluation is being carried out (specify for different stakeholders)
  • the rationale for conducting this review or evaluation at this time
  • to whom the results of the review or evaluation will be reported
  • how the findings of the review or evaluation will be used (including for accountability, learning and improvement).

Back to top

The scope of the review or evaluation

The scope describes the boundaries, the scale and/or the limits of the review or evaluation. For example include:

  • activities, or aspects of activities, that are being evaluated or reviewed
  • time period that is covered by the evaluation or review
  • geographical focus
  • target groups
  • issues that are outside of the scope of the evaluation or review
  • which of the five DAC criteria for evaluating development assistance (relevance, effectiveness, efficiency, impact and sustainability) this review or evaluation will address, and why others are not addressed. [Reference: NZAID Evaluation Policy Statement and NZAID Guideline on Evaluation and the Activity Cycle]

Objectives of the review or evaluation

This is a key section of the TOR. Here the main issues (objectives) and questions that the review or evaluation needs to answer/address are laid out. Clarity and focus in this section is critical to the outcome of the review or evaluation, and the way the findings of the review or evaluation will be reported.

Refer to the M&E framework for the activity as a basis for identifying the objectives and questions. If an adequate M&E framework is not available, then this can be developed as the first output of the review or evaluation. 

While developing objectives, think about the purpose of the evaluation or review, and the DAC Evaluation Criteria that you have specified in the TOR section on scope.

For each objective develop evaluation questions [Reference: NZAID Guideline on Evaluation and the Activity Cycle]. Design the questions to be ‘open ended’ to avoid limiting the evaluator or reviewer. Keep questions specific, and focused on the activity being evaluated or reviewed. Prioritise and limit the number of questions to make the objectives achievable.

Check that there is a logical ‘flow’ in the TOR between evaluation questions, objectives and purpose: by answering or addressing the evaluation or review questions, the objectives of the review or evaluation should be achieved. By achieving the objectives, the review or evaluation should fulfil its purpose.

Cost effectiveness/value for money: A key objective of an evaluation (and often a review) should be an assessment of whether the activity provided value for money. An assessment of how much money has been spent on activity(s) can be compared qualitatively with the broad outcomes, impacts or changes brought about by the work.

Generally, the evaluation or review should examine value for money in two ways.

  • If possible, comparisons of value for money should be drawn with experience or norms in other activities (in the same country/region or internationally), where similar outcomes or impacts have been aimed for and/or achieved.
  • The activity’s own cost structures should be analysed to identify cost effectiveness issues, including whether savings could have been made (without disproportionately compromising outcomes) through different methods or management, procurement, prioritisation, design, etc.

Back to top

Evaluation Methodology

 

Avoid prescriptions

Rather than identifying a sequence of tasks, focus on clarifying the principles underpinning the evaluation, and an approach that will ensure an accountable, independent and transparent evaluative process that builds local capacity.

It is the responsibility of the evaluation/review team to design the detailed methodology to the answer or address the evaluation/review questions set out in the TOR, while adhering to the principles and approach specified in the TOR. 

 

The methodology describes the processes by which evaluations and reviews are carried out. An effective methodology is very important as it engenders stakeholder ownership, accountability, independence, transparency, reliability of findings, and helps build capacity.

In considering the methodology of the review or evaluation, refer back to the purpose of the review or evaluation, and NZAID evaluation guiding principles. Use participatory processes in the review or evaluation where ever possible. [References: NZAID Draft Evaluation Policy Statement; NZAID evaluation guidelines on Participatory Evaluation]

In the methodology section of the TOR, provide the evaluators or reviewers with enough guidance so that they understand the approach that is expected, and the principles underlying the review or evaluation. Further, provide enough detail so that evaluators or reviewers will have confidence that the objectives of the TOR can be achieved.

Consider the following key questions when describing the approach and principles of the methodology in the TOR:

  • Which of the NZAID Guiding Principles (Page 5) apply to this review or evaluation?
  • How can the evaluation or review be made into a capacity building and learning process?
  • Who should be involved in the evaluation process and how?
  • How can we understand the perspectives of those whose lives should have been affected by this work?
  • If a participatory approach is appropriate, in what ways should the evaluation or review be participatory: in the design? in carrying out the review or evaluation? In collecting data and information?, and who should be involved in the participatory process?
  • How can we keep this simple, while involving the right groups in the right ways?
  • How can we ensure honesty, openness, transparency and independence in this process?
  • How will crosscutting and mainstreamed issues be considered in the evaluation? [Reference: Screening Guide for Mainstreamed and Other Cross Cutting Issues]
  • How can we ensure evaluation and review key informants and participants are identified and available to participate?
  • Are there other details that the contracted evaluators/reviewers need to consider?

Ensure that the review team receives copies of and understands the NZAID policies on crosscutting and mainstreamed issues, and other relevant policies.

State in the TOR that reviewer(s) or evaluator(s) provide an ‘evaluation plan’ that clearly sets out how the review or evaluation will be carried out inf line with the approach and principles specified in the methodology section of the TOR, and describes in detail the methodology that will be used. Some examples of questions that evaluators/reviewers should answer in their ‘evaluation plan’ are set out in Annex 1 of these guidelines.

Also state in the methodology section:

  • who will ‘approve’ the evaluation plan (this could be a steering committee, or the NZAID staff managing the review or evaluation)
  • the level of detail required in the evaluation plan
  • that the ‘evaluation plan’, questionnaires, checklists of questions and a summary of survey results should be appended to the main written report.

Note: SAEG Evaluation Advisors can assist with the development of the methodology.

Back to top

Governance and management of the review or evaluation

There is a difference between governance and management. The TOR should clarify and differentiate between the governance (decision-making arrangements, often using a steering committee) and the management of the review or evaluation. Include the following points in a TOR:

  • Who is the evaluation or review commissioner/client?
  • Governance - what are the roles and responsibilities of a steering committee (e.g. signing off TOR, approving the evaluation plan, addressing issues that arise)?
  • Who will ultimately ‘sign off’ the evaluation report as final?
  • Management - what are the management arrangements for the review or evaluation? Who is responsible for contracting issues, and other day to day administration and arrangements? Who is responsible for managing feedback from stakeholders and ensuring the feedback is adequately addressed by the evaluation or review team?
  • Who else has responsibility for the evaluation process, and what responsibilities do they hold?
  • What level of evaluative independence is required, and how should this reflect in the governance and management arrangements?

Note: You can discuss these questions with a SAEG Evaluation Advisor.  

Composition of the evaluation or review team

Review or evaluation can be internally or externally led, and be conducted by an individual or a multi-disciplinary team. An experienced evaluator should be included in the team.

Clearly state the roles and responsibilities of the team leader and team members. If more than one person is to conduct the review or evaluation, all should have specific roles and responsibilities, and each should contribute to the report writing.

Consideration should be given to the level of independence required if the evaluation/review team is likely to include an NZAID staff member.

In the TOR state the attributes required of the evaluation team. Consider the following when choosing team members:

  • Given the review or evaluation objectives and likely methodology, what skills, knowledge, experience does the evaluation team need?
  • Does the team need to be internal (from NZAID or a Partner) or external to the programme/activity (independent) or should the team be a mixture of the two?
  • How will the team reflect gender/ethnic concerns in the work?
  • How will partner capacity be developed, and how can team membership assist in developing partner capability?

[Reference: Further information on team composition, skills and experience can be found in the generic NZAID Guideline on Terms of Reference].

Back to top

Outputs and reporting requirements

[Reference: NZAID Guideline on the Structure of Review and Evaluation Reports]

In this section of the TOR note:

  • the deadlines for submission of the draft and/or final versions of the evaluation plan, the findings of the review or evaluation, the written report and other outputs of the review or evaluation, and milestones for payment
  • the format(s) of the evaluation or review findings/feedback (video, oral feedback in-country/region, debriefings or workshops, written reports, etc)
  • the proposed content and approximate length of the written report
  • the content of the appendices to the written report
  • the method of delivery (e.g. by hand, electronic versions)
  • the processes around what will happen to the written report:
    • the final draft report will be ‘peer reviewed’ by NZAID staff, or other appropriate person(s), and feedback from stakeholders requested1. Further work, or revision of the report, may be required if it is considered that the report does not meet the TOR, there are errors of fact or the report is incomplete or not of an acceptable standard
    • the final report will be ‘appraised’ before being considered for public release by NZAID’s Evaluation and Research Committee (ERC).

Note that adequate time should be built in for feedback from stakeholders and peer reviews, and resulting changes to the report

[Reference: NZAID Evaluation and Research Committee Process Guideline]

Follow up of evaluation or review

This section of the TOR should outline how:

  • the evaluation findings will be discussed, who will be involved in discussions and how any outcomes from this process will be followed up and/or included in the final report
  • how the final written report will be followed up.

[Reference: Guideline on Dissemination and Use of Evaluation Findings]

Sources of written information

The TOR should include a list of written material and documents that evaluators or reviewers are required to refer to, but not limited to, as part of the review or evaluation.

Back to top

1 Some questions that may be asked of stakeholders and peer reviewers are: is the information accurate and relevant? is the report clear and logical? is the analysis of findings strong? are there gaps in the report? are views expressed appropriately? back