Measuring The Impact Of Your Talent Development Initiatives

There have been a number of books written on this subject and a professional organization - ASTD - which has a ROI Network that supports learning evaluators. Kirkpatrick, Jack Phillips and a number of others have dedicated years of research to finding out how to quantify learning effectiveness. As Lord Kelvin put it "When you can measure what you are speaking about and express it in numbers, you know something about it."

If all of this is so obvious, why don't more people do it? Well for one...it is extremely difficult to isolate the impact of a training variable. Certainly, control groups are useful, and early studies in Behavior Analysis by Neil Rakham helped establish behavior correlation to outcome. But the process is long, tedious and above all, expensive. An organization must have the maturity and the mindset to actually invest in measuring the impact rather than trying to "club it" as an inherent part of the day to day role of the talent development function.

To understand the process let's take a look at Kirkpatrick's model. He has defined four levels of evaluation. Simply put they are as follows:

Level 1: Reaction

? Measured through spot feedback taken from the participants of a learning initiative.
? Easy to administer and is usually focused on a few aspects that include learning content, facilitator skills and infrastructure.
(Level 1 does not significantly aid the learning process apart from a superficial overview on the participant ?feelings')

Level 2: Learning

? Measure of retention of the concepts covered during a learning initiative. Generally slightly better measure of the impact as opposed to ?Level 1'.
? Administered through a Knowledge Test or Role Plays as a part of the learning initiative or soon after.

Level 3: Behavior

? Measures the application of learning and manifests itself in changed behavior at the workplace.
? Execution includes customer and employee satisfaction surveys as well as Behavior Analysis through workplace observation.
(Most organizations stop at this level of measurement. This entails no additional investment other than the standard talent management and development measures which provides comprehensive inputs for feedback and coaching)

Level 4: Impact

There is some amount of back and forth on this between the gurus and while Kirkpatrick states this as the last level of evaluation, Phillips has added a 5th level which is ROI. To compare, Kirkpatrick insists that ROI is an inherent part of the 4th level. In any event, to determine ROI, clear measures must be in place.

? Challenging to measure across an organization not least because of the reliance on line management, and the frequency and scale of changing structures, responsibilities and roles, which complicates the process of attributing clear accountability.
? Clouded by external factors which greatly affect organizational and business performance, obliviating the true cause of poor results.
? Requires the establishment of a research team to measure results; This is a large investment in what appear to be ?proving' or ?disproving' a hypothesis.

Therefore, while assessment is critical, it is recommended that very few organizations actually start the process by investing in a Level 4 frameworks. It must be a gradual process and once the organization reaches a reasonably evolved stage of evolution as far as processes and ?people practices' are concerned, the focus should shift to fine tuning to the greatest extent possible evaluation approach of the organization.

So what do we stand to lose if we don't reach level 4 very quickly? Transitioning from Levels, 1 to 2 and then to 3 is relatively easy. However in order to incorporate the most relevant learning initiatives into the talent development process, the next best alternative is to work with a control group with similar operating conditions and refine the developmental initiatives based upon these outputs rather than go in for a full fledged ?level 4'. While you may not get perfection, it will give you what you need.

To borrow from Jane Austen "The power of doing anything with quickness is always prized much by the possessor, and often without any attention to the imperfection of the performance." Read more at WNS