Phil La Duke
OSHA requires that workers be provided training and that the results of this training be evaluated. Unfortunately most safety professionals who design training don’t know squat about designing quality evaluative tools.
For reasons I’d rather not get into, I am taking an on-line safety-training course and it is awful. Apart from the six factual errors in the first nine lessons the methods they use to evaluate training are abysmally bad. For starters, the course designers use far too many true and false questions. What do I have against true or false questions? Plenty.
I read somewhere that the odds in favor of correctly guessing the answer of a true or false questions is 63% (don’t quote me on that since I don’t remember the source or the context) but even if we assume that the true or false question is perfectly constructed the probability of guessing correctly is 50% and so few questions are perfectly written that its safe to say that the probability of guessing correctly is much higher.
True or false questions are generally the result of lazy course development. It’s seems easy to right a good true or false question but it is surprisingly difficult to so. Authors of true and false questions tend to provide clues to the answer by using absolutes, like “must”, “always”, or “never”; if you see these clues you can almost always bank on the answer being false, because one only needs to produce a single exception to the absolute rule set out in the questions. Even something like “all giraffes have long necks and spots” is probably false since if one has enough time and energy one could probably find an example of a malformed or mutated giraffe that didn’t have a long neck and the question becomes false.
Beyond the simple-mindedness of true or false questions there’s the uncertainty of just what the true or false question is evaluating. These questions cannot measure anything beyond the memorization of facts. In her book (the best book on designing training I have ever read and I have read scores of them), Design For How People Learn, Julie Dirksen distinguishes between recognition and recall. Recognition questions are the ones that we with which we are most familiar; they test whether or not we can recognize a true statement versus a false one or if we can correctly choose a correct response from a list of possibilities. Recall questions are more open and may contain numerous correct answers—essay questions. Of course essay questions may not be correctly assessing the learner’s ability to synthesize information and or apply complex concepts in the workplace. Plus they are a pain in the ass to grade and all but the most sophisticated eLearning is unable to process a recall question. So what do we do? We take the easy way out. This is fine if we are trying to teach someone trivia, but for crying out loud we are trying to evaluate whether or not someone can safely drive an industrial vehicle or work in a confined space? Forget whether or not this is the BEST way to evaluate learning and consider if it is even a responsible way of testing these skills. When we provide ineffective training—whether it be in core skills or in safety—people are injured, crippled, or die.
The only way we can truly hope to understand whether or not a worker has sufficient training to safely do his or her job isn’t to write better true or false, or multiple choice questions, it is to be on hand to demonstrate the skill and provide a safe opportunity to practice and fail. By providing this kind of training and evaluating this kind of training can we really be sure that the people we train can do the job relatively safely.
So the next time you find yourself taking a quiz, evaluation, knowledge check, or test and you are asked a true or false question, you can hold in the utmost contempt the lazy or inept developer who took the easy way out.
I highly recommend you pick up a copy of Julie Dirksen’s Design For How People Learn; it’s truly a magnificent work that is meticulously researched and cites other great books. In addition to having a lot a great advice for both neophytes and experts it’s an easy and enjoyable read. I found profound applications to safety (as I have been on about so much lately, I truly believe that if there is one element that stands above all others in providing a safer workplace it is training and competency.)
If I can just rant a bit, the only field besides safety that organizations assume any dolt can do it’s training. You got PowerPoint? You got a projector? Well then pull together a deck and train us on that stuff you know. It’s an absurd proposition. I have a degree in training, and three separate certifications in training methodologies, but in the eyes of a lot of business leaders all that means nothing—since apparently the ability to train is imprinted on us at birth like ducklings taking to water.
Never mind that the training combines graphic arts, an understanding of how people learn and retain information, the ability to quickly build a classroom rapport, and other skills too numerous to mention, in the minds of many leaders all anyone needs to be a trainer is a slide deck an audience capable of being bored to death. Things are getting so bad that we know have “webinars” where the first thing the speaker does is mute everyone’s lines so they can pontificate like a bi-polar preacher on acid while people literally work on other things, but don’t worry if you can’t make the meeting the slide deck is available on the k:/drive.
