One of the questions I hear most often around the use of learning tools such as mobile and social is, “How do we measure?”  There is always reluctance to deploy a new technology initiative if there is no concrete way to measure its success. When people ask me how to measure these new ways of learning, I always answer with a question of my own (even though my father told me not to do that): How are you measuring learning now?

shutterstock_136658924

It may seem an abrasive way to answer the question, but in reality this is where you need to start. There is no reason you cannot measure use of a social learning technology or a mobile deployment the same way you are measuring learning right now — unless, of course, you are doing a terrible job at measuring learning right now. In that case, you have bigger fish to fry.

Look at it this way: let’s simplify a learning process or initiative into the act of using a screwdriver to screw a cover onto a wooden box. That’s the activity in its entirety. How does Wooden Box, Inc. measure the process of screwing the cover onto the box? Chances are they are looking at the following:

  • How long it takes to screw on the cover
  • The tightness of the cover once it is screwed on
  • The alignment of the cover to the box
  • How many covers are screwed on in a day/month/year

One day the production manager comes to the floor and announces that everyone in the Box Cover Attachment division is getting a brand new cordless electric screwdriver. Once the employees start using this new tool, is the company suddenly going to stop measuring the items they had previously measured? Of course not. Those same metrics are precisely what you need to see if the new tool is working well or not. In fact, they will likely add a few metrics, such as:

  • Battery life of the screwdriver
  • Cost to charge the battery
  • Cost to replace the battery
  • Reduction in wrist injuries

Whether your organization uses the Kirkpatrick Model or some other measurement framework, the outcomes you are looking for in traditional learning modalities are essentially the same for new media. Level 1 is huge here, especially early on, to get a sense of whether or not you are delivering in a way that is resonating with your learners. According to the preliminary data from Brandon Hall Group’s Social and Collaborative Learning Survey, learner satisfaction is the number one metric for social learning tools, followed closely by demonstration of knowledge acquisition. Not exactly unheard-of metrics.  But now there are additional metrics specific to the tool. For mobile, it could be user adoption or device preferences.  For social media, you can measure items such as the number of contributions a person makes to discussion threads, or how many questions they answer in forums.

Don’t let your new media initiatives get derailed by naysayers who demand measurement. The measurement is already there. The question is not if there will be anything to measure, but rather selecting just what new metrics you want to include. These new technologies generate more than enough data for you to sift through.

David Wentworth, Senior Learning Analyst
Brandon Hall Group

David Wentworth

David Wentworth has been a senior research analyst in the human capital field since 2005 and joined the Brandon Hall Group in 2013. He has authored reports and articles on various human capital subjects with an emphasis on workforce technology. He has contributed to several reports published by ASTD, including authoring Mobile Learning: Learning in the Palm of Your Hand, The Rise of Social Media: Enhancing Collaboration and Productivity Across Generations, and Instructional Systems Design Today and in the Future. His work has also appeared in Compensation & Benefits Review and T+D Magazine.

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.