by Sherry Engel
As most of my colleagues know, evaluation and metrics is a topic I love to discuss and debate. So when chatting with author Karl Kapp about Learning in 3D, our conversation immediately went to how evaluation may differ with Learning in 3D. Let’s take a look at a few of the things we discussed.
Level 1 – “Smiley sheets” provide input on many of the reactions of the learners. I’d like to examine just one aspect….learner’s confidence in applying the skills after participating in the learning event. My initial reaction is that learners should have a higher level of confidence after participating in a 3D learning experience than if they participated in a 2D learning experience. Interesting enough, Karl and I had a great discussion about a study that was done on learners in a 2D environment vs. 3D environment and their confidence level related to applying the skills after completing the learning. Which audience do you think was more confident? Well, to my surprise, it was the learners in the 2D environment. I sat back and began analyzing why this may be the case. As Robin’s previous blog indicated (and quoted from Karl and Tony’s book), when learning in a 3D environment, “The person becomes emotionally involved and behaves and acts as he or she would in the actual situation.” Think back to your past learning experiences. Are you more confident in the classroom or once on the job? Since learning in 3D provides that emotional connection, as it would on the job, learners are more apt to experience the feelings of self-doubt during learning. Is that a bad thing though? Wouldn’t you rather those feelings occur during learning rather than once on the job?
Level 2 - Learning can be measured in so many ways, however, too often we approach Level 2’s with simple “true/false, multiple choice” tests. Is that type of evaluation conducive to a 3D environment? Karl and I had a great discussion on this one as well. He shared a story with me about an individual that compared test scores from students that participated in a 2D learning environment and those that participated in a 3D learning environment. Which do you think were higher? Don’t let this one shock you…but it was the scores from the 2D environment. We began to discuss why that might occur. Well, as typical with most organizations, this organization did their level 2 testing with a simple “true/false, multiple choice” test. What type of learning does that test? In most cases its factual/knowledge-based learning. I’d like to challenge this organization to change up their evaluation to test application of behaviors in the learning environment. I’ll bet you see the students from the 3D environment testing better on that one!
Level 3 – As you know Level 3 is all about measuring application of behaviors on the job. I’m not going to go into great detail about this one, but naturally, the more we can replicate in the learning the actual environment in which the learner will be applying the skills, the higher chance of success for transfer of this behavior on the job.
Level 4 – Well, with level 4, it’s all about alignment to the business goals and objectives. Karl and Tony’s book nicely captures how learning in 3D isn’t just about the technology but how we can meet specific business needs with this type of learning environment. As with any type of learning--classroom, e-learning, simulation, 3D--upfront analysis of the performance that will be impacted and how that aligns to company business goals and objectives is crucial for true business impact to occur.
As most of my colleagues know, evaluation and metrics is a topic I love to discuss and debate. So when chatting with author Karl Kapp about Learning in 3D, our conversation immediately went to how evaluation may differ with Learning in 3D. Let’s take a look at a few of the things we discussed.
Level 1 – “Smiley sheets” provide input on many of the reactions of the learners. I’d like to examine just one aspect….learner’s confidence in applying the skills after participating in the learning event. My initial reaction is that learners should have a higher level of confidence after participating in a 3D learning experience than if they participated in a 2D learning experience. Interesting enough, Karl and I had a great discussion about a study that was done on learners in a 2D environment vs. 3D environment and their confidence level related to applying the skills after completing the learning. Which audience do you think was more confident? Well, to my surprise, it was the learners in the 2D environment. I sat back and began analyzing why this may be the case. As Robin’s previous blog indicated (and quoted from Karl and Tony’s book), when learning in a 3D environment, “The person becomes emotionally involved and behaves and acts as he or she would in the actual situation.” Think back to your past learning experiences. Are you more confident in the classroom or once on the job? Since learning in 3D provides that emotional connection, as it would on the job, learners are more apt to experience the feelings of self-doubt during learning. Is that a bad thing though? Wouldn’t you rather those feelings occur during learning rather than once on the job?
Level 2 - Learning can be measured in so many ways, however, too often we approach Level 2’s with simple “true/false, multiple choice” tests. Is that type of evaluation conducive to a 3D environment? Karl and I had a great discussion on this one as well. He shared a story with me about an individual that compared test scores from students that participated in a 2D learning environment and those that participated in a 3D learning environment. Which do you think were higher? Don’t let this one shock you…but it was the scores from the 2D environment. We began to discuss why that might occur. Well, as typical with most organizations, this organization did their level 2 testing with a simple “true/false, multiple choice” test. What type of learning does that test? In most cases its factual/knowledge-based learning. I’d like to challenge this organization to change up their evaluation to test application of behaviors in the learning environment. I’ll bet you see the students from the 3D environment testing better on that one!
Level 3 – As you know Level 3 is all about measuring application of behaviors on the job. I’m not going to go into great detail about this one, but naturally, the more we can replicate in the learning the actual environment in which the learner will be applying the skills, the higher chance of success for transfer of this behavior on the job.
Level 4 – Well, with level 4, it’s all about alignment to the business goals and objectives. Karl and Tony’s book nicely captures how learning in 3D isn’t just about the technology but how we can meet specific business needs with this type of learning environment. As with any type of learning--classroom, e-learning, simulation, 3D--upfront analysis of the performance that will be impacted and how that aligns to company business goals and objectives is crucial for true business impact to occur.
