Plan: Research questions and outcome measures

Plan

Evaluators
Practitioners
Senior management

After diagnosing your equality gaps, the next step is to plan your evaluation. The first part of planning is to decide what your research questions and outcome measures are.

Types of evaluation

The OfS categorises evaluation into three types:

  1. Narrative
  2. Empirical enquiry
  3. Causal

These types are explained in Table 1. While this guide is focused predominantly on studies generating causal (type 3) evidence using institutional data, we have provided a flow chart to help you determine what kind of evaluation is most suitable and links to resources to help you achieve it.

Table 1: OfS types of evaluation
Type of evidence Description Evidence Claims you can make
Type 1 - narrative The impact evaluation provides a narrative or a coherent theory of change to motivate its selection of activities in the context of a coherent strategy. Evidence of impact elsewhere and/or in the research literature on access and participation activity effectiveness or from your existing evaluation results. We have a coherent explanation of what we do and why our claims are research-based.
Type 2 - empirical enquiry The impact evaluation collects data on impact and reports evidence that those receiving an intervention have better outcomes, though does not establish any direct causal effect. Quantitative and/or qualitative evidence of a pre/post intervention change or a difference compared to what might otherwise have happened. We can demonstrate that our interventions are associated with beneficial results.
Type 3 - causal The impact evaluation methodology provides evidence of a causal effect of an intervention. Quantitative and/or qualitative evidence of a pre/post treatment change on participants relative to an appropriate control or comparison group who did not take part in the intervention. We believe our intervention causes improvement and can demonstrate the difference using a control or comparison group.

Identifying research questions for causal evaluation

These questions follow the general format:

Did the intervention change a specific outcome among the target group?

For example:

Did peer mentoring reduce stress among nursing students?

Did the online Cognitive Behavioural Therapy programme reduce depression among students?

To help formulate your questions, you should also consider:

  • Who will use the findings and how?
  • What do stakeholders need to learn from the evaluation?
  • What questions will you be able to answer and when?

Identifying outcome measures

Once you have established your research questions, you will need to consider which outcome measures best enable you to answer them and demonstrate success. The measures should link closely with the process, outcomes and impact you have recorded in your theory of change. A simple way to think about which measures to select is:

I’ll know the outcome has been reached when I see this indicator.

I’ll know that student engagement has improved when I see average attendance has reached 70%.

Outcome measures resources

For more information on identifying suitable outcome measures, see:

Back to top