Create and share a new lesson based on this one.

Additional Resources for you to Explore
You’ll remember from the watch section of this module that we talked about some basic standards and concepts of evaluation. Western Michigan University has several checklists for evaluation that may be useful for you as you consider how and what to evaluate in your courses and in your broader language program (www.wmich.edu/evaluation/checklists). One checklist provides some nice general guidance on your evaluation questions. Your evaluation questions are the “end in mind” - what do you want to know about your course/program? Questions should be framed so they will yield answers that provide determinations of significance or merit. They should be directly relevant to design and outcomes of your program. They should be reasonable in terms of scope and resources needed to answer them. They should further be specific, answerable, and complete. Wingate and Schroeter of Western Michigan University developed a program evaluation checklist and said regarding completeness: 

“A set of evaluation questions is complete when the questions thoroughly address the purpose of the evaluation and evaluation users’ information needs. The question set should be purposefully selected from a broad range of possible topics (e.g., program design, context, process, implementation, products, outputs, outcomes, impacts, efficiency, cost-effectiveness, etc.). A set of evaluation questions does not need to address all of these topics, but there should be a sound rationale for the inclusion or exclusion of potential topics.”

Types of Evaluation & Standards:

There are several types of evaluation that might be valid. While we’re not trying to turn you into professional evaluators, there is some value in knowing the various types of evaluation and some key characteristics. The table below (Stufflebeam, 2001) reflects how well certain evaluation methods align with evaluation standards:
(For full article: https://drive.google.com/file/d/0B-pkUgSK2ouDYWNHWG9YZlBITzg/view)

Regardless of what type of evaluation you pursue (and it very well might be mixed methods), let the basics standards guide your work. Every evaluation should consider these factors:

- Utility – useful, of value, relevant, timely
- Feasibility – practical methods, access to information/resources
- Propriety– ethical, respectful, responsible
- Accuracy – complete, correct, appropriate methods/analysis

The Joint Committee on Standards for Evaluation in Education has published Program Evaluation Standards Statements which can help you further consider each of the items listed above.

Many universities and organizations have established guidelines and checklists for quality online courses and teaching. Check into any of the following to help identify what standards and checklists might be a good fit as you evaluate your program and courses.

Quality Matters Rubric  http://www.moodlerooms.com/sites/default/files/slideshow/slides/kari_walters_qm_rubric.pdf

http://www.drexel.edu/~/media/Files/inspire/pdf/Quality_Matters_Rubric_Standards_2011-2013.ashx

iNACOL - National Standards for Quality Online Teaching, and Quality Online Courses http://www.inacol.org/resources/publications/national-quality-standards/

SREB - Online Teaching Evaluation for State Virtual Schools
http://publications.sreb.org/2006/06T02_Standards_Online_Teaching.pdf

California State University, Chico - Rubric for Online Instruction
http://www.csuchico.edu/roi/the_rubric.shtml

Formative vs. Summative Evaluation:

There can be a temptation to only use summative evaluations, which are often used to determine whether a program should consider, for example. One of the reasons people tend to lean toward summative evaluations is there is usually only one to do.

But there is great value in formative evaluation, as it can drive iterations and improvements in your program. While formative evaluation requires on-going attention and dedication to measuring your program,  they can play a key role in your program’s success.

As we learned back in the ADDIE unit, evaluation is critical to knowing if everything we have been doing is working and why or why not. Formative evaluation makes it easier for us to course correct as we go, because we get useful data along the way. Summative evaluation can be useful when we’re trying to make major program decisions such as justifying continuing or discontinuing a course.

Learning By Doing:

Above all, the key in any program evaluation is to just do it! But don’t just do it for yourself. Remember to involve your stakeholders. Building a culture that values evaluation can take a while, but it fosters shared vision and dedication to continuous program improvement.   

As you pursue evaluation, remember you always have your mentor and this DL SIG community where you can turn for support, ideas, and encouragement.

More resources:

This worksheet is a great example template to get you started.
An example of a google form using the ACTFL can-do statements can help you evaluate student success, which is often used as an indicator of program success.

To learn more about evaluation, read Program Evaluation, Alternative Approaches and Practical Guidelines; Fitzpatrick et al; 2010.
Slides from this lesson