Shades of the 1980s!

Imagine my surprise when I read a long-established training vendor’s recent press release. Omitting the organization’s name, I’m going to quote from it:

“(Company X) conducted a survey of its clients to determine the effectiveness of online training. Comparing over 100,000 pre- and post-test scores, (Company X) found that clients using (Company X’s) courses significantly increased their trainees’ technical and safety knowledge.

After reviewing the tests results from a large population of trainees, (Company X) can demonstrate a 30% increase in technical skills knowledge when comparing pre-test to post-test scores.”

What an empty claim! Ridiculous! And, in the 1980s, one made by many training vendors regarding the videotapes they then hawked — before the customers figured out the vacuousness of such claims.

Think about the meaninglessness of these results for a minute.

The only true measure of one’s training program depends upon an increase in the trainee’s on-the-job performance — which translates into learning retention and increased skills.

Short-term improvement, as measured by “immediate testing” (pre- and post-test score differential) means next to nothing.

Just remember your own experience in high school and college classrooms. You studied late at night to pass a test and then, within days or weeks, you forgot much of what you had temporarily learned. You experienced short-term retention but impacted your long-term retention knowledge very little.

The truth is that every vendor of training can make the same claim as Company X. All of them can demonstrate a marked increase in test scores between the pre-test and the post-test. No vendor has an advantage in that measurement tool.

But, you should be looking at long-term retention which can only be measured if you re-administer a post-test six months, or more, after the initial training. Combined with the evaluation given by the trainee’s supervisor that is the only way you will be able to measure the effectiveness of your training initiatives.

What’s more, when it comes to correctly designed e-Learning, a student should be able to demonstrate 100% mastery. Well designed e-Learning allows the trainee to repeat all of the practice exercises and problems as many times as he wishes — in order to thoroughly master the material before taking the final exam.

So, forget comparing pre- and post-test scores. They matter little. It’s finding a long-term retention measurement that will prove the worthiness of your training.

More on Thursday – – – – –

— Bill Walton, Founder, ITC Learning (Tuesdays & Thursdays)