____________________________________________________________________
Levels of Evaluation
- Reaction- (improve,validate,collect data) attitude/opinion
Reaction is the step of collecting information via surveys, people give opinions and feelings rather than hard facts.
- Learning- (monitor in controlled training classroom) performance/assessment
Learning is the training portion where you are following training objectives and you are being tested in a controlled training venue. (from here we dove into the types of assessments)
- Pre/before..."placement or guage your knowledge"
- Pre requisite-required to attend
- Post-after course is complete
- Embedded-during/through course
- Transfer-can you take your skills out to the real world
Transfer is the level where you are not training in a controlled enviroment anymore. You having a on the job assessment or if you are a phone operator you are taking actual calls from real customers.
- Business Impact- real numbers and figures on improvements or progress made
Business impact is the add number value to training. If there is a certain percent increase or decease. Dont get this confused with a montary value.
- ROI- (return on investment) dollars and cents/compares program business to its cost
Return on investment is plain and simply the bottomline. How much money did we save, did, we earn. A no "BS" dollar number.
NOTE: Donald Kirkpatrick is responsible for levels 1-4, Jack Phillips added in level 5
____________________________________________________________________
Evauation Myths
This was an exercise in groups of two. We drew out of a hat one of the 10 evaluation myths. We then discussed in a group if we agreed or disagreed with the myths. I will list them below.
- My boss does not require me evalaution, so why should we do it?
- Measuring progress towards objectives doesn't work for evaluation
- Too many variables affects the behavior change so i can't evaluate impact.
- In my own situation, I can't measure the result of my training efforts.
- I have a proven track record, so I don't need to justify my existence.
- I am not sure where to begin...what information to collect.
- Measurement is only appropriate in production and financial areas.
- If i can't calculate the ROI, then program evaluation is worthless.
- Evaluation is too expensive.
- If we conduct evaluation, we'll get criticized.
We established that each myth is just that.....a myth.
____________________________________________________________________
We also went into the Perfomance Gap Analysis. This was brought up when we discussed the 6th myth. I found a site that has outline almost exactly like the instructor.
http://www.w-win.com/RegionalPlanning/templates/Step6.pdf
____________________________________________________________________
We then got into Assessment Inro and Rubric Intro.
we were handed "test item critique"
http://evalsum09.wikispaces.com/WED463+Handouts
We went online and found websites that had information on test setup or format. We made corrections to the paper and then discussed. From there we were put into groups again and were tasked with creating our own test questions. We discovered it is not an easy task.
Aha: You have to be extremely specific when writing a question, know your desired outcome and have on answer.
____________________________________________________________________
The last tasking we were handed as we left was an ISD model project. Go to the same link above and look at ISD exercise. We were put into groups and given a due date by 10 july @ 1700.
____________________________________________________________________
Recap:
Class was fast pace and if someone would have missed this weekend they would be behind the power curve. If you buckle down and do your work this is a very do-able class.
I agree, this class was very fast pace. I have gotten alittle lost occassionally and I was sitting in the class room. Overall, I think the concept is great and am learning much,
ReplyDelete