The University of Southampton

Evaluation

There are several different ways to approach evaluating a partnership or specific activity. Not all of these will be applicable in every case. When considering what and how to evaluate it is important to be realistic about what can be achieved and be clear what you would like to measure. For example:

Have the school pupils learnt what you wanted them too?

Do pupils or teachers have a better understanding of what academic research is?

Have you changed pupils’ perception of their own education in the relevant subject?

Have you changed the number of pupils considering coming to university?

Will teachers change their practice as a result of engaging with your activity?

Have you inspired teachers to further their own academic careers?

Does the engagement work you have done deliver impact for the REF?

Ethics: It is important to be aware that evaluation activities that include focus groups or questionnaires, particularly when pupils are involved, may require ethics approval. This could include the need for parental consent and the anonymity of participants.

 

Area of evaluation Method Advantages Disadvantages
Outcome for the pupils/quality of activities Show of hands, thumbs up/thumbs down, bottle tops/buttons in buckets labelled ‘good’, ‘okay’, ‘poor’

Quick and easy.

If use a physical method (e.g. bottle tops) these can be counted and recorded.

Assesses enjoyment rather than learning outcomes. May not be honest unless is anonymous.
Verbal questioning as plenary to assess understanding of subject material Quick and easy. Can be fun e.g. a quiz. Difficult to record or gather data. Unlikely to represent whole group.
Focus groups Decide focus in advance. Possible to gather more in-depth information, opinions and experiences. Smaller numbers possible than questionnaires. Can be logistically difficult. Needs to be soon after engagement for pupils to recall things fully.
Questionnaires (see examples in Appendix 6) Recordable and measurable responses. Allows focus on a particular area of interest. Time for preparation and analysis needed. Questions need careful planning. Can be logistically difficult. Need pre- and post-intervention data to provide evidence of impact. Pupils may not put thought into answers.
Assessed work on return to school Levelled responses. Allows focus on particular area of curriculum. May be attractive to some schools. Preparation required and time to mark. Can be logistically difficult. Might not work for some schools if little spare lesson time.
Tracking e.g. progress at school or applications to university Provides data based on fact rather than opinion. For university applications software already exists (HEAT). Logistically very difficult. Not easy to prove a causal link.
Outcome for the teachers and researchers Focus groups/ interviews Can set questions in advance. Possible to gather more in-depth information, opinions and experiences. Opportunities to follow lines of enquiry as they arise. Constructive feedback from professional to improve activities/deliver what schools need. Can be logistically difficult to arrange.
Questionnaires Recordable and measurable responses. Allows focus on a particular area of interest. Likely to be very few in number so won’t provide a large data set. Preparation required and time to analyse. Need pre- and post-involvement questionnaires to provide evidence of impact.
Tracking e.g. career progression Provides long term impact data based on fact rather than opinion. Logistically very difficult. Not easy to prove a causal link.
Papers or research published Provides evidence of value which can be used to support individual careers and effect culture change in wider area of SUPs. Must be well thought out from the beginning of an intervention if there is to be a valid research outcome directly relating to work done through the partnership.
University recruitment Questionnaires Provides recordable data on opinions directly after engagement. Opinion doesn’t necessarily translate to actual attendance at university. Preparation required and time to analyse. Need pre- and post-involvement questionnaires to provide evidence of impact.
Tracking e.g. progress at school or applications to university Provides data based on fact rather than opinion. For university applications software already exists (HEAT). Logistically very difficult. Not easy to prove a causal link.
Impact in REF See Writing REF Case Studies.

 

Back to Costs and Funding

…On to Reflection.