Evaluating with Kirkpatrick

You’ve decided to start evaluating training but don’t know how? After getting all the excuses out of the way (see my previous blog), we’ll look at a well-known evaluation model for training: The Kirkpatrick Model.

The Kirkpatrick Model has been around since the 1950s and consists of four levels: Reaction, Learning, Behaviour and Results. In 2010, The New World Kirkpatrick Model has been introduced with modest changes, but the four levels still exist.

Although the latest model suggests starting with level 4 (measuring organisational impact), you can still start with level 1. That’s the level you’re probably most familiar with and from there you can start adding more levels to your evaluations.
Let’s find out more about the four levels and my method suggestions for each level.

Level 1 – Reaction

This level asks participants for feedback on how they felt about the training. Was the training engaging and was the content relevant to them?

Methods for level 1
  • Gather feedback through a survey. Include one or two open questions to let participants reflect on their experience for more insights.
  • Ask participants to write one or two paragraphs about their training experience. You can prompt them with a few open questions. This takes more effort to analyse but you often get more detailed responses.
  • Organise a verbal feedback round at the end of the training or the next day.

Level 2 – Learning

At level 2 you’re measuring to what degree participants have gained new knowledge, learnt new skills, or changed their attitude. You’d also like to measure how committed and confident they are to applying what they’ve learnt.
Methods for level 2
  • Include a pre and post assessment. Then you’ll have a baseline and can objectively measure what participants have learnt. Whether the assessments are multiple choice tests, short answer questions, assignments, observations of a task or something else will depend on the topic and your resources.
  • If a preassessment isn’t viable or logical (e.g., for onboarding), only use a post-assessment. It will at least show the knowledge level, skills, or attitude after participating in the training.
  • Ask for verbal or written responses from participants. Examples are self-reflection. questionnaires and interviews.

Level 3 – Behaviour

Level 3 measures the change in behaviour: Are the participants applying the knowledge, skills or change in attitude to their work, resulting in an increased performance? Please note that with required drivers in place, such as coaching, job aids and recognition, the application on the job can increase from 15% to 85% (Brinkerhoff in ‘Telling Training’s Story (2006)). To measure the change in performance you need to allow time (e.g., three months) between the training event and measuring to allow participants to implement what they’ve learnt.

Methods for level 3
  • Observe the participants in their work environment. This could be done by the trainer or their supervisor. Support the observer(s) with a checklist/questionnaire so they know what to look out for. The checklist will also increase the consistency of the evaluations.
  • Give the participant an on-the-job activity in which they can show off what they’ve learnt and assess.
  • Use a self-reflection questionnaire to find out how participants changed their performance in the workplace after completing the training.

Level 4 – Results

At level 4 we’re looking at the impact of the training on the organisation. Does the training have a positive effect on the organisation? The training goals should be aligned with the organisational goals and have measurable indicators. Still, it’s not easy to isolate the impact of training on the organisational performance. Leading indicators can help make the connection between individual efforts and organisational results. Kirkpatrick’s leading indicators are customer satisfaction, sales volume, quality, market share, cost containment and employee engagement. For these results to show you need to look at a longer period after the training event.

Methods for level 4

Collect data on the chosen indicator(s). Besides Kirkpatrick’s, you could also use other indicators. Examples are wastage, the number of complaints, non-compliance, the number of accidents, system downtime, attrition rates, recruitment cost, etc. Then check whether there’s a change in those indicators after participants completed the training, for example over a 3-, 6- and 9-month period. Additionally, you need to find out if there have been other factors that may have influenced those indicators before attributing all the changes to the training.

A few comments

Let me finish those levels with a few comments:
  • A successful evaluation is based on SMART learning objectives, aligned with organisational goals. Therefore, you need to start thinking about evaluation at the start of your project.
  • Participants can be super enthusiastic about a training session (level 1) but that doesn’t mean they’ve learnt a lot (level 2) or will improve their performance (level 3).
  • A positive outcome for the reaction (level 1) and learning (level 2) doesn’t automatically mean an increase in performance (level 3). For example: Participants might not be confident or committing to the application of the new skill, knowledge, or attitude.
  • You’ll only get a positive result for the organisation (level 4) when participants have gained knowledge or skills (level 2) AND changed their behaviour (level 3). So, the results of level 2 and 3 can predict level 4.
  • If you think Kirkpatrick is not the right model for your organisation or project, have a look at the better evaluation website to find a more suitable model.

Summary

Kirkpatrick’s evaluation model is an easy and well-known model to start evaluating training. It consists of four levels: reaction, learning, behaviour, and results. For each level there are different ways to measure the outcomes. Ultimately, you’d like to measure the impact of training on the organisation in level 4.
(Visited 43 times, 1 visits today)