Friday, August 24, 2007

How do I measure thee? Let me count the ways!

...eLearning, that is! I prefer the multiple choice test. Its easy. Write a question. Create a few answers. I always add an "E) All of the above" just to make 'em think. Nobody really holds me accountable for the quality of the questions just as long as I can show who has, and who hasn't, passed the course. I don't know what any of these people actually do for the company so we don't ever check to see if our course helped them at all. No...wait...that's not true. I just remembered. Our LMS sends out an automated email survey 6 months after they took the course. So, I'm totally covered on the whole Level 3 thing.

I'm going to go an take The eLearning Guild's survey on Measuring eLearning Success. We SO do it right! Maybe I'll right a book on our process.

[The story you've endured is fictional and the main character in no way reflects the exact historical perspective of this blogger. EXCEPT for the fact that he IS going to complete the Survey and you should too]


Christy Tucker said...

Of course, anyone who does more than multiple choice tests will have problems finding ways to answer the survey. God forbid if you use instructors for your online learning; if that's the case, you clearly don't do anything to measure learning.

At least, that's the impression I got from the survey questions. It's very much geared towards corporate learning and ignores academic and other learning.

Just my 2 cents on what I found to be a very disappointing survey.

Brent Schlenker said...

Hi Christy! Many thanks for the feedback. Perhaps my tongue-in-cheek post wasn't the best way to shed light on the survey. My experience is quite jaded in this space.

With that said, the authors of this report include Will Thalheimer, Allison Rossett, Roy Pollack, Sharon Shrock, Margaret Martinez, Bill Coscarelli. They have crafted the report to gain insight into specific areas of measurement and testing.

There is so much that COULD be asked in our surveys and don't because we are sensitive to the amount of time it takes our members to take them.

Can you give me more specific feedback on what you thought was missing and/or why you were disappointed?
Would you be interested in writing a case study and/or being interviewed? Perhaps there is an angle specific to your industry that we have overlooked and you can help us shed some light.

Thanks again for commenting. It really is great to hear from colleagues in this format

Bob Greenwood said...

I loved your tounge-in-cheek comment Brent. Sadly it is way too close to the truth. Most of our 'measurement' is CYA instead of WCAWYK (we care about what you know:).

In my case many of our courses are government mandates. In most cases the only point of the exercise is to see that 'pass' or 'complete' in the Student's LMS transcript.

Keep us the good fight...

Brian Dusablon said...

Great post - we have problems measuring results as well. We really don't do it - and it's more because we don't know the best way, rather than not wanting to at all.