Weeks five and six: Evaluation methods

To recap, as I do not have an online course at present, I will be conducting a Needs Analysis to evaluate the idea of creating online software training for the staff at MIT, ie Microsoft Word, Excel, PowerPoint and other web-based programmes that we use at MIT. I intend using Blackboard, as this is the learning platform used at MIT. I want to ensure that the learning modules that are developed do meet the needs of our learners, and this will be a big part of the Needs Analysis, finding out from the learners, what their needs are - how they best learn. From informal discussions that I have had with staff, it appears that there is definite interest and different users would use this in different ways. So my initial thoughts are to provide for different needs and learning styles by providing a variety of learning strategies. But let's not jump the gun.... hopefully the Needs Analysis will confirm my thinking.



As noted in my prior post, have decided to use the Multiple Methods Evaluation model (triangulation and bracketing), which will give me the flexibility and variety to be able to evaluate the many different aspects and also the diverse needs of the learners (Hogarty, 2008).

The paradigm that seems to fit well with this evaluation model and eLearning, is the Eclectic-Mixed Methods-Pragmatic Paradigm discussed in Reeves' Educational Paradigms article (1996).

The Guidelines that I have chosen to work with in this evaluation project are:

SD3 From Guidelines Wiki
Do students gain knowledge relevant to employment and/or current thinking in their field?What I want to evaluate here, is that my learners (the staff at MIT) have an appropriate computing skill set to be able to carry out their job effectively.

ST1 From GuidelinesWiki
Do you have a way to identify student needs and respond to them?
The Needs Analysis will hopefully confirm that providing online software training for the staff at MIT will improve the opportunities for learning and meet the needs of the learners.

Have been reading Chapter 6 "Conducting a Needs Analysis" from Interactive Learning Systems - Reeves, Thomas & Heldberg (2003). Reeves et al says "The overall purpose of needs assessment is to provide information to guide decisions about aligning an interactive learning system with important needs of specific audiences". This seems to fit well with what I am trying to achieve.

Of particular interest was a "Checklist for Evaluating an Elearning Program or Courseware" which looks at the Content and the Instructional Design of the course. This will be a useful guide when I am looking at developing elearning modules.

Kaufman, Rojas & Mayer (1993) state that "The bottom line of the value of needs assessment is that it should provide the data needed to support the next level of design decisions so that the project moves forward to prototyping and formative evaluation activities". Sounds good to me!

The types of evaluation I intend to use are:

• Interviews


• Questionnaires


• Focus groups


• Checklists

References:

Hegarty, Bronwyn (2008), Types of Evaluation Models, downloaded from http://wikieducator.org/images/b/b6/Types_of_Evaluation_Models08.pdf

Kaufman, R.A., Rojas, A.M., & Mayer (1993). Needs Assessment: A user's guide. Englewood Cliffs, NJ: Educational Technology Publications.

Reeves, T. & Hedberg, J. (2003). Interactive Learning Systems Evaluation. Educational Technology Publications, Englewood Cliffs, New Jersey