Week7

= Evaluation =

In Morrison, Ross, and Kemp, read chapter 10, 11, and 12

**7.1 Background and Rationale** This week you’ll address the fifth and final phase of the ADDIE process, Evaluation, where you’ll create a plan for the formative evaluation of your project and the testing instrument/s for your learners. Then you’ll pull all five of your weekly ADDIE steps together into a single unified document. With this step, your project will be done and ready for presentation to the class next week. We considered evaluation a bit during Week 4, the Design Phase, when we were focusing on objectives. At that point, we wrote objectives with evaluation plans in mind, following the principle of OCPA (Objective, Content, Practice and Assessment) alignment. This week, we’re diving much deeper into evaluation. Remember our metaphor of instructional design as architecture? How does that idea apply to evaluation? First, let’s review: That’s just a bit too neat and tidy, however. In the real world, whether a building or an instructional unit is the topic, no one really waits until the entire project is done to begin to judge its value. Using our metaphor, consider how frequently various stakeholders would view the project and evaluate its progress. Will three windows really give enough light in this space? Should an additional bathroom be installed here? Does the frame appear to be sturdy? The same thing applies to a course. As it’s being designed and developed – that is, "formed" – various stakeholders are also evaluating the emerging structure. Is this too much content to be covered in a four-hour timeframe? Will this particular target audience respond well to a role-play exercise? Should we add another example to clarify that second point? This is called "formative" evaluation. At the completion of an ID project, a summary, or "summative," evaluation is often conducted, and then at intervals after that – in some cases, over the course of years – experts return to check that the course is still performing as intended; this verification is called "confirmative" evaluation. Note that to this point, the focus has remained on evaluating the course itself. Of course, in many contexts, it’s just as important to "evaluate" the learners; for the sake of clarity, we’ll refer to evaluation of learners as "assessment," and of the course itself as "evaluation," although the terms are often used synonymously. ("Measurement" is another term frequently applied to learners.) The higher the stakes for the learner, the more critical it is that the instruments and procedures used to measure their knowledge or skill acquisition are fair and defensible. In some cases, legal issues can turn on the validity and reliability of tests.
 * 1) Analysis is like identifying the need for a building, its occupants, the uses to which the building will be put, the proposed location and characteristics of the site
 * 2) Design is like creating a blueprint, taking into account all the unique components discovered during Analysis. Nothing has been built yet, but it is possible to imagine the finished structure.
 * 3) Development is like construction – the foundation is poured, the walls are built, the structure takes shape and one can literally see the building becoming "real"
 * 4) Implementation is like the move-in phase; the occupants arrive and begin to inhabit the building
 * 5) Evaluation, then, is like the process of judging the success of the building; does it really satisfy the original need? Are the occupants pleased and is the structure effective?

**Guiding Questions** Consider these questions this week as you complete your learning activities.
 * 1) Considering the differences between "evaluation" and "assessment" and how are both used during the "E" step of ADDIE?
 * 2) What are "validity" and "reliability" to an ID? How can you ensure that your assessments are both valid and reliable?
 * 3) What options do you have to measure learner achievement, and what principles should you follow as you construct test instruments?
 * 4) How important is formative evaluation? Summative evaluation? How are both of these conducted?

**Outcomes** By the end of this week’s lesson, you’ll be able to:
 * 1) Complete the evaluation phase for an ID project, including a plan for formative evaluation.
 * 2) For each objective, describe the test item/s or other evaluation method/s intended to assess the level of achievement of the learners who will be experiencing your instructional unit.

**7.2 Course Project** Last week concerned the "I" step in ADDIE, Implementation. This week, we move to the fifth and last step, E for Evaluation. Your course project will then be complete and ready for presentation to the class next week. **Deliverable** Use the ADDIE Template provided in the Resources for this course to document these elements: For example: **Objective =** To calculate the number of sofa-loveseat combinations to be ordered from the manufacturer for the third quarter of the year, using an established formula based on historical sales results from the previous three years. **Assessments =**
 * Document your plan for a formative evaluation during the full development of your instructional unit. Note that you are only being asked to provide a plan, not the results from a completed evaluation.
 * Using the OCPA chart, describe planned test items mapped to each of the instructional (performance) objectives in your completed design document.
 * 1) Multiple-choice. Provide historical sales results for sofa-loveseat combinations for the same quarter of the previous three years.
 * 2) Application: Ask learners to use the standard formula to calculate the correct number of sofa-loveseat combinations to order.

E-mail your completed course project, including all the weekly submissions combined into a single document (the evaluation plan will be the last section) to your facilitator in the time identified by him/her. Use this naming convention for your document: You may turn this in this week for instructor comments for final fine-tuning, or next week if you need more time. File name: LastnameFirstinitial_project (for example: PattersonB_project)

Advance Preparation
Next week is the final week of our course. You will present your project to the class and provide thoughtful feedback on your classmates’ projects as well. Begin thinking about how you intend to introduce and "frame" your project to the class. Expect that the time required to review and compose feedback responses to at least two of your classmates’ projects may be significant.