While writing this I continue to look back at my original statement to understand more about how we got to this point. So let me try and be more clear about the point I was trying to make in the original post.
"In the corporate world we should only really care if the learning is transferred to the job…period! Is the output increased, or of higher quality because of our learning intervention. This has always been a problem for training departments because we lookI still stand by the statement. Just because we put some kid through StoreLayout training and Product Location training and he scores 100% on all of the tests (evaluations) DOES NOT mean he will PERFORM. The measure MUST be how he/she performs ON THE JOB, period. Sure the training product had impact because he passed the test. But maybe he's a jerk and still not very helpful to customers. But as long as someone can pass our test the training product is evaluated as a "successful" implementation. What we tend to do when we look at training as a product is to wash our hands of the problem once we have implemented the "Training Solution" (in-class trng, eCourse, etc.). And THAT "approach is totally wrong" because we stop at level 2 evaluations. Today's learning systems allow for level2 evals at the end of the training to make someone "certified". The click2death elearning that is produced today cannot support any more than that. (Please don't tell me about your level 3 and 4 solutions you have created. It's just not necessary.) If you support your client past the implementation of your training solution as part of your service then I applaud you.
at everything we do as a product, and we “evaluate” if the product had
impact. The approach is totally wrong."
However, I'll bet what you are finding is that your trainees, while good right after the training still refer back to their coworkers and cheatsheets to maintain their knowledge. Nobody will argue that.
Our transition from an eLearning Development community that creates a product (with .4% effectiveness) to a community that provides a service supporting informal learning must come sooner rather than later.