Sector trends
Fantastic evaluators and where to find them: A guide to commissioning evaluations.
So, you’ve been funded to deliver a fantastic initiative in your community. Congratulations! But, there’s a catch! Your funder is asking you to do something called an evaluation as a condition of your grant. In this article, we will answer some of the burning questions you may have about commissioning evaluations.
Kieran Gulvin
Social Impact Lead (grants)19 February 2025
Commissioning Evaluations
By the end of this impact article, we aim to answer the following questions about commissioning evaluation:
- What is an evaluation?
- Who can do an evaluation?
- How is evaluation different from research?
- How can I prepare to commission an evaluation?
- Where can I find an evaluator?
- How can I pick the right evaluator for the job?
- What does a potential evaluator need to know to know in an invitation to tender?
Before we start: What is evaluation? A quick refresher
Evaluation is a systematic process to determine the merit, worth, value or significance of something (Scriven, 1991). An evaluation may seek to quantify the outcomes and impacts of an initiative and reflect on the processes that produce it. To achieve this, evaluators will seek to collect evidence about your project, analyse this evidence, and produce a report which you will provide to your funder to fulfil the grant condition, and hopefully use for a range of other purposes too!
Okay, great, but who are evaluators?
The field of evaluation is made up of a wide range of professionals from different backgrounds, often with training in the social sciences, but not exclusively. What they all have in common, however, is that they are adept at using their skills in research methods and knowledge of evaluation theory to form overall judgements about the project. They may have worked in consultancy firms, academia, not-for profit organisations and government agencies.
How is evaluation different from research?
There are some pretty big crossovers in research and evaluation, in that they use a lot of the same tools to do the job. However, they differ in a number of crucial ways, which are particularly important to be aware of if you’re contracting for an evaluation professional. These differences are outlined in Table 1.
Table 1: Differences between research and evaluation
|
Difference |
|
Research |
|
Evaluation |
|
What is the overarching aim? |
|
Aims to contribute to the wider body of human knowledge by producing and testing theory. |
|
Aims to make evaluative conclusions about the merit, worth, value or significance of a specific initiative. |
|
Where do the questions come from? |
|
Emergent from scholars in a discipline. |
|
Originate with the key stakeholders and primary users of the evaluation findings. |
|
What are the types of questions most typically asked? |
|
What’s so? (Making conclusions about how the world works). |
|
So what? Now what? What was needed? (Making conclusions about how good a project is, and how well was it done). |
|
How is the quality of it judged? |
|
By peer review in a discipline. |
|
By how useful it is to those who will take action and make decisions. |
Source: Michael Quinn Patton (2014). Evaluation Flash Cards: Embedding Evaluative Thinking in Organizational Culture. St. Paul, MN: Otto Bremer Foundation. Available at: https://ottobremer.org/news_stories/evaluation-flash-cards/
One of the key risks for contracting organisations can come from evaluations that focus on answering research questions rather than evaluation questions. Without clear, relevant evaluation questions aligned with the initiative and its outcomes, stakeholders may lack the necessary information to make effective decisions (Martino & Fry, 2023). Additional risks include:
- Misunderstanding the evaluation’s purpose, leading to flawed design, including poorly framed evaluation questions.
- Failing to apply the findings of the evaluation, resulting in wasted effort—often a consequence of unclear objectives.
- The intended audience not understanding or trusting the evaluation findings, which may stem from poor design or inadequate communication.
Given the risks, finding a great evaluator, if you don’t have one in-house, can be critical to making sure that your organisation gets what it needs from the evaluation.
How do I prepare for finding an evaluator?
We recommend using the handy Lotterywest Community Impact Hub’s Impact Planner tools to develop a Theory of Change, Program Logic and Outcomes Measurement Plan prior to looking at external evaluators. These will help you communicate to your potential evaluators:
- The outcomes you’re hoping to see as a result of your project,
- The logic behind how the outcomes will come about, and
- The types of evidence you might collect to demonstrate the outcomes of your project.
We also recommend completing as much of the Evaluation Plan Template in the Lotterywest Grant Impact Guide (Appendix 4) as possible. This will help to clarify the scope of work, the purpose of your evaluation, your indicators of change, the questions you’re hoping to answer, and other pieces of information that will prove useful when writing your invitation to tender or request for quote.
If there are certain parts of the plan you need some external expertise to complete, that’s okay! You can make completion of the plan part of the scope of work for the external evaluator.
Where can I find an evaluator?
There are a few places you might want to look for somebody to do your evaluation. The Australian Evaluation Society Consultants Directory contains a useful list of members who provide commercial evaluation services. Filtering the Learn from others section of the Community Impact Hub to see evaluation reports will also provide some examples of Lotterywest-funded evaluations, and the organisations that delivered them.
How do I pick the right evaluator for the job?
The Australian Evaluation Society’s Professional Learning Competency Framework can start to point us in the right direction. This document outlines the competencies that providers of evaluation services should have and highlights the complex nature of conducting quality evaluation work. It’s important to note that an evaluator may not necessarily have all of the necessary competencies by themselves but may work within a team that contains all the competencies.
A brief overview of the competencies you might be looking for in an evaluator are as follows:
-
Evaluative attitude and professional practice: These competencies focus on the self-reflection and professional development of the evaluator, which in turn inform all other competencies.
-
Evaluation theory: The evaluator’s knowledge of the theoretical foundations of evaluation, which are distinct from other forms of inquiry.
-
Culture, stakeholders and context: How well the evaluator understands the context in which the evaluation exists, the needs of stakeholders of the evaluation, and the cultural considerations they may need to be aware of.
-
Research methods and systematic inquiry: The knowledge and skills needed to collect valid and reliable data on which evaluative judgements may be based.
-
Project management: The project management skills required for an evaluator to successfully negotiate, scope and manage the evaluation to completion.
-
Interpersonal skills: The interpersonal skills required for evaluators to communicate effectively with clients, consumers and other stakeholders.
-
Evaluation activities: The specific tasks an evaluator may carry out to undertake an evaluation. These include activities such as defining the evaluation purpose, generating evaluation questions, collecting data collection activities and evaluation reporting.
Specialist Skills
Your evaluator may also require certain specialist skills and knowledge in order to successfully deliver the evaluation. These may include, but are not limited to:
Subject matter expertise
The task of evaluation may require a deep understanding of the initiative that is being evaluated. You may need to contract an evaluator that has had experience in evaluating similar projects in the past or possesses knowledge of the field your initiative operates in to be evaluated in a suitable manner.
Experience in participatory approaches
In conducting evaluation on initiatives that aim to help vulnerable or under-represented populations, participatory approaches are considered best practice. Using these approaches requires specialist skills and expertise, and often involves including staff with lived experience of the subject matter in the evaluation team.
Economic analysis skills
Sometimes, stakeholders may require economic questions to be answered as part of the evaluation. These could be questions such as: “Do the benefits outweigh the costs?” and “Is this a more efficient use of our limited resources?”. Answering these kinds of questions may require specialist skills in conducting economic analysis.
Cultural awareness and sensitivity
When working with First Nations or multicultural communities, evaluators must demonstrate cultural competence, including an understanding of historical, social, and cultural contexts. This involves building respectful relationships, engaging in meaningful consultation, and ensuring that evaluation methods are culturally appropriate and inclusive. Collaborating with community members and incorporating their perspectives is essential to delivering a respectful and effective evaluation.
Okay great!
Now what do I actually need to do?
Contacting evaluators to participate in a request for quote or tendering process is the next step. This should be integrated in existing procurement processes and policies that your organisation has in place, with specific information you may need to include in your briefing document listed below:
Information about your organisation, and the project being evaluated
A brief overview of what your organisation does, and how it does it can be useful to include. You will also want to include details on the project that needs to be evaluated. Handy tip: this is where you can include your Theory of Change, Program Logic and Outcomes Measurement Plan!
Evaluation scope
It’s important to outline what falls inside and outside the scope of your evaluation. If your evaluation is being conducted as part of a grant condition, you may specify that the scope of the evaluation is to cover only the activities being delivered using the grant funds.
Evaluation purpose
Apart from meeting the grant condition set by your funder, it’s important to know what you may wish to use your evaluation for in the future. Considering the evaluation purpose may also inform the deliverables that your evaluator produces throughout the course of the evaluation. Evaluation purpose is discussed more thoroughly in the Grant Impact Guide.
Potential evaluation questions and methodology
Having a sense of the evaluation questions you want the evaluator to answer, and the methods you may want them to use to answer them can be useful to help prospective evaluators cost their work appropriately.
Deliverables
A written evaluation report is the most common deliverable that funders ask for as part of requesting an evaluation, but your evaluator may also produce other materials which help you communicate the impact of your project. These may include:
- An accessible, plain English summary of the evaluation findings.
- A final evaluation plan.
- Production of an ongoing monitoring framework for the project.
- Presentation of results to external stakeholders.
- Infographics to be used in communications materials.
Time frames
At a minimum, your evaluator needs to know when the final report is due to your funder. If you have an idea of when you would like certain components of the evaluation completed, such as finalisation of the plan, or data collection, it’s useful to include this information in your brief as milestones.
Budget
If your evaluation has been funded as part of your grant, being up front about the evaluation budget during the contracting process will help your evaluator refine their scope of work to fit the budget. You may also wish to include any support you may provide as part of the evaluation, such as staff resources and provision of existing data.
Conclusion
A good evaluation can stand not only as assurance for your funders that you have achieved your intended outcomes, but also as a fabulous impact narrative that you can share with your beneficiaries and stakeholders to demonstrate the impact of your initiative. You can use key information from your evaluation to develop persuasive future funding proposals, and even to create communications and marketing content. Evaluations can be so much more than documents – they can be a roadmap to kicking future goals!
Explore the links in this article
-
Learn from Others - Reports
Filter to see evaluation reports - read about Lotterywest-funded evaluations and the organisations that delivered them.
-
Lotterywest Grant Impact Guide
Find an Evaluation Plan Template (Appendix 4) and read the Lotterywest guide to support outcomes-based grant making.
-
Evaluation Flash Cards
Embedding Evaluative Thinking in Organizational Culture.
-
Australian Evaluation Society Consultants Directory
Find a list of members who provide commercial evaluation services.
-
Evaluation VS Research
Evaluation VS Research: what's the difference and why does it matter?
-
Indigenous Evaluation Strategy
A whole-of-government framework for Australian Government agencies to use when selecting, planning, conducting and using evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people.
-
Australian Evaluation Society's Learning Competency Framework
Outlines the competencies that providers of evaluation services should have and highlights the complex nature of conducting quality evaluation work.
Kieran Gulvin is an evaluator by training, specialising in the homelessness and mental health sectors. He previously worked as a mental health outreach worker and early intervention behavioural specialist. Kieran combines his experience in service delivery and expertise in outcomes measurement to drive positive change. Kieran is currently the Social Impact Lead (Grants), Lotterywest.
Try these next
Measuring social impact – What’s in it for me, you, and the community?
Read the article
101 | Data, evidence and evaluation – essential ingredients for creating an effective impact narrative
Read how to create an effective impact narrative
WEBINAR - Evaluation Planner tool
Learn how to use the Evaluation Planner tool on the Community Impact Hub
Lotterywest Quick Guide to Grant Impact
Understanding, planning, and measuring grant impact
Learn about wellbeing
Understand how your community is going to help you to better target and plan your project.
Ready to plan your project?
Understand your vision, plan your impact and report on the outcomes of your project with three easy interactive tools in the Community Impact Planner.
Acknowledgement of Country
The Western Australian Community Impact Hub acknowledges and pays respect to the Traditional Owners of the land on which we are based, the Whadjuk people of the Noongar Nation and extends that respect to all the Traditional Owners and Elders of this country. We recognise the significant importance of their cultural heritage, values and beliefs and how these contribute to the positive health and wellbeing of the whole community.