As noted in my prior post, have decided to use the Multiple Methods Evaluation model (triangulation and bracketing), which will give me the flexibility and variety to be able to evaluate the many different aspects and also the diverse needs of the learners (Hogarty, 2008).
The paradigm that seems to fit well with this evaluation model and eLearning, is the Eclectic-Mixed Methods-Pragmatic Paradigm discussed in Reeves' Educational Paradigms article (1996).
The Guidelines that I have chosen to work with in this evaluation project are:
SD3 From Guidelines Wiki
Do students gain knowledge relevant to employment and/or current thinking in their field?What I want to evaluate here, is that my learners (the staff at MIT) have an appropriate computing skill set to be able to carry out their job effectively.
ST1 From GuidelinesWiki
Do you have a way to identify student needs and respond to them?
The Needs Analysis will hopefully confirm that providing online software training for the staff at MIT will improve the opportunities for learning and meet the needs of the learners.
Have been reading Chapter 6 "Conducting a Needs Analysis" from Interactive Learning Systems - Reeves, Thomas & Heldberg (2003). Reeves et al says "The overall purpose of needs assessment is to provide information to guide decisions about aligning an interactive learning system with important needs of specific audiences". This seems to fit well with what I am trying to achieve.
Of particular interest was a "Checklist for Evaluating an Elearning Program or Courseware" which looks at the Content and the Instructional Design of the course. This will be a useful guide when I am looking at developing elearning modules.
Kaufman, Rojas & Mayer (1993) state that "The bottom line of the value of needs assessment is that it should provide the data needed to support the next level of design decisions so that the project moves forward to prototyping and formative evaluation activities". Sounds good to me!
The types of evaluation I intend to use are:
- Focus groups
Hegarty, Bronwyn (2008), Types of Evaluation Models, downloaded from http://wikieducator.org/images/b/b6/Types_of_Evaluation_Models08.pdf
Kaufman, R.A., Rojas, A.M., & Mayer (1993). Needs Assessment: A user's guide. Englewood Cliffs, NJ: Educational Technology Publications.
Reeves, T. & Hedberg, J. (2003). Interactive Learning Systems Evaluation. Educational Technology Publications, Englewood Cliffs, New Jersey