11.9. Effective Student Outcomes: A Comparison of Online and Face-to-Face Delivery Modes

Editorial

As you know, I am not much for delivery-method comparisons of the “no significant difference” variety. Students, on the whole, are going to learn, despite good or poor course design and regardless of the delivery mechanisms.

As the author of this month's article points out, many of the “no significant difference” studies are, according to the Institute for Higher Education Policy, seriously flawed—both theoretically and methodologically. The study reported below attempts to specifically address some of these flaws and produce a sound, quantitative study.

It is almost impossible to "hold all else constant" (outside the world of statistical theory) when evaluating courses taught using different delivery methods. The measurements chosen for this study are independent of the delivery method with exactly the same pre- and posttests being administered to all sections and the percentage of gain being the unit of measurement.

This study also relates learning styles to student success and satisfaction. This is a first step towards investigating online learning as a discreet phenomenon, in and of itself. What makes a good online course? How do we best prepare students for online learning? Along what lines can we individualize online learning experiences?

These are some of the pressing questions deserving of our attention.

Mauri Collins
DEOSNEWS Editor

Effective Student Outcomes: A Comparison of Online and Face-to-Face Delivery Modes

Jane M. Carey

Abstract

This study compares outcome measures for identical courses offered in two delivery modes. One delivery mode is Web-based, online, and asynchronous, and the other is face-to-face and synchronous. Outcome measures for both modes include gain scores (difference between pretest and posttest knowledge), grade expressed as a percentage, and student satisfaction. These outcome measures are used to determine whether online students learn as effectively as face-to-face students. Analysis of the data suggests that online students are gaining knowledge comparable to the face-to-face students and that the online students are as satisfied on most dimensions as the face-to-face students. Kolb's Learning Style instrument is utilized to determine if one learning style lends itself better to online learning than other styles.

Introduction

A myriad of studies have concluded that there is no significant difference in the learning outcomes of online students and face-to-face students (Russell 1999; Barry and Runyan 1995; Cheng, Lehman, and Armstrong 1991; Martin and Rainey 1993; Moore et al. 1990).

In April 1999, the Institute for Higher Education Policy released a report to the American Federation of Teachers and the National Education Association regarding the effectiveness of distance learning in higher education (IHEP 1999). This report reviews contemporary research and comes to the following conclusions:

1. Of the hundreds of articles written on distance learning, only a small percentage contain original, quantitative research.
2. The bulk of these quantitative studies show that there is no significant difference in the learning outcomes between online and conventional classes (see Russell 1999).
3. Most of these studies are methodologically flawed in one or more of the following ways:

a. There is no attempt to control for extraneous variables, which in turn limits the ability to demonstrate cause and effect.
b. Subjects are not randomly selected and thus the ability to generalize from these studies is compromised.
c. The validity and reliability of the instruments used to measure student outcomes and attitudes are questionable.
d. “Reactive effects” (feelings of students and faculty) are not controlled for.
e. Reported outcomes are for single courses and not for programs.
f. Differences among students are not taken into account.
g. Dropout rates for online courses are not explained or taken into consideration.
h. Student learning styles relating to technology are not considered.
i. There are no theoretical or conceptual frameworks utilized.

The researcher attempts to address many of these concerns with this current project. The following section includes a review of the emerging theoretical frameworks. Section i (above) indicates that there is no theory in distance learning. This claim is erroneous. Although the development of theory is in its infancy, there is a growing body of literature that posits, tests, and advances the state of theory.

Online- and Distance-Learning Theory

Theory relating to distance learning—and particularly online learning—is in the early stages of development. This is understandable, since online distance learning has been developed only recently—although other forms of distance learning, such as interactive television and correspondence courses, have existed longer. Distance learning formats are recent, compared to face-to-face formats. Internet delivery of courses—and, indeed, the Internet itself—has been in existence for only a very few years.

To understand the state of theory in distance education, it is important to first look at the definition of distance education first posited by Keegan in 1980, and later refined to be essentially as follows:

“Distance education is characterized by a quasi-permanent separation of teacher and learner throughout the learning process, is supported by an educational entity (which may be public or private), uses technical media to carry the course content, provides two-way communication, and a quasi-permanent absence of a learning group” (paraphrased, Keegan 1986).

Several theories from other disciplines have been adapted to distance education. These theories include Instructional Systems Design (Banathy 1968); Keller's (1968) Personalized System of Instruction; Information Processing Model (GagnÈ 1985); Kolb's (1984) Adaptive Learning Style; Field Dependence/Independence (Witkin 1978); Hemisphericity (Torrance 1980); Jungian Approach (Jung 1971); Rogers' Theory of Innovation (1983); Theory of Cognitive Speed (Fulford 1993); Transactional Distance Model (Saba 1988); Miller and Husmann's (1996) Holistic Model of Primary Factors; and Engagement Theory (Kearsley and Shneiderman 1998). Studies have also explored the relationships between distance education and group size, learner persistence, teacher-learner interactions, gender, delivery modes, and organizational support, among others.

In general, a review of the literature on theory in distance learning indicates student outcomes—such as performance, satisfaction, persistence, and intention to enroll in another distance education course—are commensurate with the traditional face-to-face educational formats. Important aspects of distance education that need attention include a well-designed course orientation, frequent and high-quality student/teacher interactions, clear communication of course expectations (including due-dates for assignments), and a mechanism for frequent feedback on course performance. One of the distinct advantages of distance education is that it increases education access in remote rural areas and to busy adult students who have not been served well by traditional learning formats.

The Study

A course in management information systems had been offered at a small, urban, upper-division university in the conventional face-to-face mode for two years. This junior-level course is required by all undergraduate businesses and accounting majors. An institutional decision was made to offer the same course in an online mode while still offering several sections using the conventional delivery mode.

The institution and the professor wanted to ensure that the quality of learning for the online students was comparable to that of the face-to-face students. A pretest/posttest had been in use for two years and was designed to assess students' understanding of generalized knowledge that the course addresses. Use of a pretest/posttest instrument aids in the control of student differences in content knowledge prior to the beginning of the course. Some students enter the course with a deep knowledge of computer concepts, while others have very little computer knowledge. The gain scores show progress. This approach is far superior to using only final grades or some traditional outcome measure. In addition to gain scores, final grades expressed as a percentage and a satisfaction measure were used.

Research Hypotheses

• H1. There will be no significant differences between the learning outcomes of students receiving the online-delivery mode and the face-to-face delivery mode.
  
• H2. There will be no significant differences between the satisfaction levels of students receiving the online-delivery mode and the face-to-face delivery mode.
  
• H3. There may be an interaction effect between the course-delivery mode and Kolb's Adaptive Learning Style (Kolb 1984) measure.
  
• H4. Dropout rates for the online classes will be higher than for those of the face-to-face classes.
  
• H5. A model, which builds on existing theory, is proposed to predict student success in courses delivered in an online mode (see Figure 1).

This model is based on a model developed by Bostrom, Olfman, and Sein (1988) to investigate the end-user learning process for software applications. Learner characteristics consist of motivation to learn, learning style, prior knowledge of the course content, demographic characteristics (age, race, sex, etc.), and intelligence, among others.

The delivery mode is generally categorized as online or face-to-face. But it is understood that there are many variations of each of these modes, and aspects include selected technology, ease of use, technological familiarity, technology stability, and classroom environment variables such as size, configuration of seats, classroom technology, among others.

Course-design elements include pedagogical style (lecture, group, collaborative learning, etc.), student-centeredness, reinforcement of concepts, anchoring of concepts to previous knowledge and course content, and others. The course content not only refers to the specific subject matter (such as information systems), but also conceptual versus skills, applied versus theoretical, complex versus simple, amount of material to be covered, stability of course material over time (highly changing versus static), salience to the learner, and other aspects.

Course outcomes include such traditional measures as final course grade, student evaluations of the course, student satisfaction, and the ability to apply the concepts learned in the course. Also, it may be possible to ascertain whether the student has the ability to use the knowledge gained in one course in a subsequent course that builds on the material learned in that course. The AAHE (American Association of Higher Education) has identified seven principles of good practice that also can be characterized as learning outcomes. These include (a) increased contacts between students and faculty, (b) increased cooperation among students, (c) more-active learning, (d) prompt feedback, (e) emphasis of time-on-task, (f) high expectations, and (g) respect of diverse talents and ways of learning.

It is outside the scope of this study to be able to validate the entire model, but a preliminary confirmation of some of the relationships will be explored.

Methodology

The course that is the focus of this study is designed to provide the undergraduate, global business major at a small, urban, upper-division university with the information-system concepts necessary to be a general manager. It is also designed to accustom the student with the technical jargon associated with information systems. This jargon will be used every day in the work place. There is also a skills component to this class that includes assignments on Internet use, CD-ROM database use, Microsoft PowerPoint, Microsoft Access, Microsoft Excel, creating a Web page, and a student-generated individual application.

Some students take this class in the Web-based, online, asynchronous mode, while other students elect to enroll in the more traditional, synchronous, face-to-face mode. The students answer a set of fifteen questions designed to determine whether they have the hardware, software, level of computer experience, and self-determination to successfully complete the course online. If a student can answer yes to the majority of the questions, he/she is recommended to take the course online. However, there is no enforcement of this condition, and students are allowed to make their own decisions.

Students signing up for the online sections of the course, are required to attend an orientation and to come in to the University (or other site) with a picture ID to take one exam. This requirement helps to mitigate the possibility that someone other than the student is completing the course. We have no fail-proof means for authenticating appropriate users or preventing someone other than the student from using the student's accounts to navigate the course.

Courses taught in both delivery modes use FirstClass software for instructor-student interaction and submission of assignments. FirstClass is conferencing software that also supports regular e-mail. The assignments are identical for both delivery modes and include exploration of the Internet, use of CD-ROM databases (available in the university library), proficiency in use of the software packages in the Microsoft Office Suite—including Word, PowerPoint, Excel, and Access—and developing a Web page using Netscape Composer. The online learning is self-directed, whereas the face-to-face learning is instructor-directed. The face-to-face students not only get support and lectures on how to complete the assignments, but also attend lectures covering the information systems concepts contained in the textbook. The content in the textbook forms the basis for two exams. The online students receive the PowerPoint slides, but do not receive the benefit of the instructor's insights. All students have access to Web-based practice exams that are scored for the students automatically.

All students are encouraged to interact with the professor on a regular basis. Questions may be asked at any time via FirstClass. If a student is having difficulty completing an assignment, he/she may request a face-to-face help session with the professor. A conference will be set up in FirstClass, which will allow other members of the class to see the questions and answers of others, often resulting in a clearer understanding of some of the more difficult issues. This technique is typically referred to as FAQs (frequently asked questions).

The online delivery mode has been available to the students for several semesters. The course is offered in this format for a variety of reasons:

1. The concept of the “virtual university” is the wave of the future; since this course is about information systems (IS) and concepts, it seems to be a natural for testing out the online, computer-based delivery system.
2. The program has only two full-time professors in the area of IS to meet the demands of both undergraduate and graduate students. Online delivery allows each professor to service more students while allowing the students to complete their degree in a timely manner.
3. The level of computer experience and expertise of undergraduate students is widely varied. Some students are very comfortable with computers and do not need the face-to-face instruction. Some students are somewhat inexperienced with computers and have more difficulty learning new skills and concepts. These students are advised to take the face-to-face sections. Having two types of classes will meet the needs of both the experienced and inexperienced students.

Research Design

This research has a two-by-four factorial design. The treatment variables are course-delivery mode and Kolb's (1984) Adaptive Learning Style (a measured but unmanipulated variable). The assignment of subject into delivery mode is not randomized. A concerted effort is made to try to match the student's experience and background to the appropriate delivery mode (see Table 1). This nonrandom assignment causes limitations to the current study.

Table 1. Matching Student Background with Appropriate Course Delivery Mode

Performance Measures (Dependent Variables)

Several outcomes were used to measure performance:

1. gain score (the difference between the pretest and posttest measures);
2. final course grade (expressed as a percentage) (assignments, exams, and the instructor were exactly the same for each delivery mode); and
3. student self-reported satisfaction (assessment instrument).

Results

• H1. There will be no significant difference between the learning outcomes of students receiving the online-delivery mode and the face-to-face delivery mode.

Gain Scores

Identical pretests and posttests were administered to both classes (online and face-to-face) over two semesters. Gain scores were obtained by subtracting the pretest scores from the posttest scores.

The range of the gain scores was 0 to 13. The mean of the gain scores for the online classes was 4.84, whereas mean for face-to-face class was 6.47. The number of subjects in the online mode was 103, and in the face-to-face there were 60. These differences were not significant (p = .271).

Grades (Final Grade Expressed as a percentage)

The grades ranged from .383 = E (or Fail) to .946 = A. T-tests for independent samples were used. The mean grade for the online course was .802 and the mean grade for the face-to-face class was .778. These are not statistically different (p = .803).

• H2. There will be no significant differences between the satisfaction levels of students receiving the online-delivery mode and the face-to-face delivery mode.

Satisfaction

The range of possible satisfaction was from 1 (strongly dissatisfied) to 5 (strongly satisfied). T-tests for independent samples were used. The mean for the online class was 4.0, and for the face-to-face class it was 4.39. These are not statistically different (p = .893).

The results can be summarized in Table 2.

Table 2. Satisfaction Measurements

• H3. There may be an interaction effect between the course-delivery mode and Kolb's Adaptive Learning Style measure.

Kolb's Learning Style and Delivery Mode

Performance = f(mode + learning style)

Analysis of Variance was run on each of the performance variables to determine if mode and/or learning style have any main effects on the performance variables. Only two effects were significant. Mode of delivery had a significant effect on the satisfaction variable (p = .023). Students who took the course in the face-to-face mode were more satisfied with the course. Also, the Active Experimentation style had a slightly significantly effect on the grade variable (p = .053). Students with an Active Experimentation style received better grades in both delivery modes of this course than did students having other learning styles. This was true regardless of mode of delivery. There were no interaction effects between the mode of delivery and the learning styles. Therefore, Hypothesis 3 is rejected.

• H4. Dropout rates for the online classes will be higher than for those of the face-to-face classes.

The analysis for this hypothesis is done at a different level than the others. From the spring of 1996 to the spring of 1999, the same instructor offered seven sections of the course in online delivery mode and seven sections in the face-to-face delivery mode. Table 3 shows the number of dropouts per section and the results of the t-test used to determine the statistical difference. A student was counted as a dropout if he/she dropped either course after the official drop-add period, which was one week into the semester. There were a small number of students who moved from the online-delivery mode course section to one of the face-to-face sections, and vice versa. These students were not counted as dropouts.

Table 3. Dropouts per Session

These dropout rates are statistically significantly different at the .000 level. Dropout rates have various implications. One aspect of dropout rates is that a liberal withdrawal process leads to higher grades. Those students who are anticipating low grades often drop out rather than receive the low grade (even an expected C can elicit such behavior). The online student has less emotional investment and attachment to the class and thus may find it easier to justify withdrawing. Students in face-to-face classes develop a personal relationship with the instructor, who sees them on a regular basis. This relationship and also concern on the part of the instructor leads to fewer drops in the face-to-face class. Consequently, many of those students who are convinced (or convince themselves) to stay in the class, may end up with a poor grade if they do poorly on the final exam or the final project. A separate study should be conducted to determine how the differences in drop rates affect the student outcomes.

• H5. A model, which builds on work by Bostrom, Olfman, and Sein (1988), is proposed to predict student success in courses delivered in an online mode (see Figure 1).

At this juncture, a justification of the proposed model is discussed. The current study is not sufficient to provide validation of this model. Future research will be needed.

Relationship one: Course content and the delivery mode of the course. At this time, little research has been conducted to determine what type of course is best delivered online. From the author's experience and observations of over 600 online students, it appears that concrete, specific skills lend themselves well to online learning, while more abstract concepts do not. The information systems course in this study contains both types of knowledge. The assignments include concrete, specific computer-based skills such as using Microsoft Access, PowerPoint, and Excel, creating a Web page using Netscape Composer, and using online library database systems and the Internet to conduct research. The students are also required to learn various abstract concepts such as strategic use of information resources in business settings, systems theory, various types of information systems such as decision support systems, expert systems, management-information systems, executive-support systems, and transaction systems. They demonstrated their understanding of this more abstract type of content through exams.

A deep understanding of the complex relationship between course content and course-delivery mode could only be confirmed by conducting a meta-analysis using many courses. Trying to categorize the courses would be difficult. Also, many courses include skills and concepts together, such as the target course in this study.

Relationship two: Delivery mode will interact with student characteristics and course-design elements to impact course outcomes. To date, the research shows that delivery mode has no significant impact on course outcomes. However, some research has indicated that individual characteristics, such as learning style, interact with delivery-mode and/or course-design elements to impact course outcomes. It makes sense that the three (course-design elements, course-delivery mode, and student characteristics) will all interact to affect course outcomes.

Future research will be needed to explore these relationships and to confirm this preliminary model.

Discussion

This study addresses many of the concerns of the IHEP report. It does so in the following manner.

Concern 1. There is no attempt to control for extraneous variables, which in turn limits the ability to demonstrate cause and effect.

A concerted effort has been made to ensure that the online course is as nearly identical to the face-to-face course as possible. The instructor is the same person. The assignments for each section are exactly the same, the exams are exactly the same, the materials used for lectures and study guides are identical, the number of points are equal and so on. As much as possible, the only difference is the delivery mode.

Concern 2. Subjects are not randomly selected, and thus the ability to generalize from these studies is compromised.

Unfortunately, a conscious decision has been made not to randomly assign students to delivery modes. Students are allowed to take any of the sections. Therefore, the assignment into classes is not at all random, but rather self-selected. In addition, students answer a series of fifteen questions (see Appendix A in attachment) to help the student decide which mode to sign up for. However, this is an informal check and is not enforced. After the orientation at the beginning of the semester, there is some movement between sections. Some who sign up for the online section decide they are better suited to the face-to-face delivery, and vice versa. Indeed, there is NO random selection, and it is quite likely that the students who choose the online class have more experience and knowledge about information systems than do those who choose the face-to-face mode. Thus, the necessity of the pretest/posttest scores is reinforced. Use of grades alone would certainly not be appropriate. In conclusion, this concern is not addressed by this study. Allowing random assignment to sections was considered, it was determined that student needs and desires were more important than research concerns.

Concern 3. The validity and reliability of the instruments used to measure student outcomes and attitudes are questionable.

The pretest/posttest instrument (see Appendix B in attachment) was refined over a two-year period and is designed to measure general information-systems knowledge. This knowledge is conceptual and not skills-based. The scores have been correlated with other student outcomes, such as GPA and level of computer experience. It is not a nationally accepted or validated instrument. Although it has face-construct validity, it has not been truly validated the way a nationally accepted instrument may have been.

Well-validated assessment instruments in academe are rare. In the case of validating instruments to measure online-learning outcomes, the problem is compounded by the variety of subjects (rather unlimited) that are offered online. We are trying to determine whether content material has been sufficiently learned rather than how the student perceives or feels about the delivery mode.

The satisfaction instrument is multiquestion and a factor analysis is performed. The resultant factors are subjected to a reliability test. This does not ensure validity, but it does help in this regard.

Concern 4. “Reactive effects” (feelings of students and faculty) are not controlled for.

Since a single faculty member is the instructor for all students, the feelings of the faculty member are consistent across all subjects. This would be a problem in a situation where several faculty members hold differing views and feelings about delivery modes.

The students, on the other hand, are bound to have a myriad of “feelings” about both modes of delivery. The use of a multi-item scale rather than a single item measure helps control for these possibilities. Also, students are asked how they “feel” about computers and related technology and how they view themselves in relation to technology. They are also asked whether they were able to get into the delivery mode they prefer, or whether they were forced (due to schedule constraints or availability) to take the mode they did not prefer. Analysis of these questions should help to control for “reactive effects.”

Concern 5. Reported outcomes are for single courses, and not for programs.

Unfortunately, this is a shortcoming of this study and will be a problem for some time until a full program of study is offered online. Many institutions are beginning to offer such programs, but they are a recent development.

Concern 6. Differences among students are not taken into account.

This study uses several means to try to take individual student differences into account. All students are asked to take the Kolb Adaptive Learning Style Inventory. It is hypothesized that learning style will have a main effect on the performance outcomes and an interactive effect with the delivery mode. In other words, one or more learning styles will perform better, depending on the delivery mode that is used. Learning Style is a measured, but unmanipulated, treatment variable. The differences in computer experience levels and knowledge is also taken into consideration by use of pretest/posttest measure. The study does not rely solely on such outcome measures as final grades that can be related to a number of individual differences such as intelligence, study habits, outside commitments, jobs, and family.

Concern 7. Dropout rates for online courses are not explained or taken into consideration.

This study does compare the dropout rates of students enrolled in the two different modes. The main concern is that the number of students who drop out may cause the outcomes to be skewed. The final grade mean for the online section may be artificially higher, since those who dropped out would have most likely earned lower grades. This study also attempts to explain why dropout rates are higher for the online classes than the face-to-face classes.

Concern 8. Student learning styles relating to technology are not considered.

This study does use Kolb's Adaptive Learning style inventory and attempts to determine which (if any) of the learning styles are more likely to help students succeed in the online mode or the face-to-face mode (see concern 6 for more information).

Concern 9. There are no theoretical or conceptual frameworks utilized.

This study presents a preliminary theoretical model. The proposed relationships are not addressed by the current findings. Additional research is needed to confirm this model. A meta-analysis of existing research may help to explain the relationship between course content and delivery mode. To date, instructors may have filtered out some courses as poor candidates for online delivery. It is important to understand the criterion by which courses are selected for online delivery and what online delivery can enhance and what is diminished.

Conclusion

The relationship between course-delivery mode and course outcomes is poorly understood. To this date, most research indicates that there is little difference in the performance of students taking online courses and students taking face-to-face classes. Much of this research is suspect since it is often conducted by the instructors who are teaching the classes themselves. These insturctors have vested interest in showing positive outcomes for the online-delivery mode.

The concerns of the IHEP are well founded and must be addressed in subsequent studies. We are just beginning to understand how to deliver courses online. Persons in the education discipline may have an understanding of educational outcome measures, but those educators in other disciplines don't have the training or the desire to spend much time on these issues.

However, we will never be able to halt the increasing rate at which we are delivering online courses. These courses will be offered more and more, regardless of outcomes. It is imperative that we begin to understand how to measure and improve learning outcomes for online courses. If we don't begin, we may end up with a generation of learners who have failed to grasp and understand the skills and knowledge they need to succeed in their work and, indeed, in their lives.

References

Banathy, B. H. 1968. Instructional systems. Belmont, CA: Fearon Press.

Barry, M., and G. Runyan. 1995. A review of distance learning studies in the U.S. military. The American Journal of Distance Education, 9 (3): 37–47.

Bostrom, R. P., L. Olfman, and M. K. Sein. 1988. End-user computing: A research framework for investigating the training/learning process. In Human factors in management information systems, ed. Jane M. Carey. Norwood, NJ: Ablex Publishing Co.

GagnÈ, E. 1985. The cognitive psychology of school learning. Boston: Little, Brown, & Co.

Cheng, H. C., J. Lehman, and P. Armstrong. 1991. Comparison of performance and attitude in traditional and computer conferencing classes. The American Journal of Distance Education 5 (3): 51–64.

Fulford, C. P. 1993. Can learning be more efficient? Using compressed speech audio tapes to enhance systematically designed text. Educational Technology 3 (2): 51–9.

Institute for Higher Education Policy. 1999. [online]. What's the difference? A review of contemporary research on the effectiveness of distance learning in higher education. Available at http://www.ihep.com/PUB.htm#diff

Jung, C. G. 1971. Psychological types. (H. G. Baynes, Trans., revised by R. F. Hull). Vol. 6 of Bollingen Series XX, Princeton, NJ: Princeton University Press.

Kearsley, G., and B. Shneiderman. 1998. Engagement theory: A framework for technology-based teaching and learning. Journal of Distance Education 38 (5): 20–3.

Keegan, D. 1980. On defining distance education. Distance Education 1 (1): 13–36.

Keegan, D. 1986. The foundations of distance education. Bechkenham, Germany: Croom Helm.

Keller, F. S. 1968. Good-bye teacher. Journal of Applied Behavior Analysis 1: 78–89.

Kolb, D. A. 1984. Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall.

Martin, E. D., and L. Rainey. 1993. Student achievement and attitude in satellite-delivered high school science course. The American Journal of Distance Education 7 (1): 54–61.

Miller, M. T., and D. E. Husmann. 1996. A holistic model for primary factors in the ecology of distance education course offerings. Journal of Distance Education 11 (1): 101–10.

Moore, M. G., M. M. Thompson, A. Quigley, G. C. Clark, and G. G. Goff. 1990. The effects of distance learning: A summary of literature. University Park, PA: The American Center for the Study of Distance Education.

Rogers, E. M. 1983. Diffusion of innovations. New York: The Free Press.

Russell, T. L. 1999. The no significant difference phenomenon. Chapel Hill, NC: Office of Instructional Telecommunications, North Carolina State University.

Saba, F. 1988. Integrated telecommunication systems and instructional transaction. The American Journal of Distance Education 2 (3): 17–24.

Torrance, E. P. 1980. Your style of learning and thinking, forms B and C. Athens, GA: University of Georgia.

Witkin, H. A. 1978. Cognitive styles in personal and cultural adaptation. Heinz Wener Lecture Series, Vol. III. Worcester, MA: Clark University.

Author Information

Jane M. Carey
Associate Professor of Information Systems
Arizona State University West
School of Management
P.O. Box 37100
4701 West Thunderbird Rd.
Phoenix, AZ 85069-7100

Office: 602-543-6216
Fax: 602-543-6221
E-mail: jcarey@asu.edu

Appendix A.

ISM 301 - Online
Advisement Questions
To determine whether a student should be advised to take ISM 301 face-to-face or by guided learning, please ask the following questions.

1. Do you own a computer?

2. Does your computer have a modem?

3. Does your computer have a CD-ROM drive?

4. Do you subscribe to an Internet provider?

5. Do you consider yourself an experienced computer user?

6. Did you complete CIS 105 within the last 2 years?

7. Did you receive an A or a B in CIS 105?

8. Are you familiar with Microsoft Word?

9. Are you familiar with Microsoft PowerPoint?

10. Are you familiar with Microsoft Excel?

11. Are you familiar with Microsoft Access?

For numbers 8–11, the guided-learning, self-paced modules will start with the basics and work up to advanced skills, so you don't have to be an expert user. Familiarity with these packages will be helpful.

12. Are you disciplined enough to spend 6–9 hours per week working on your own on this course?

(We expect face-to-face students to spend 3 hours per week in class and another 3–6 hours per week on assignments and studying concepts for a total of 6–9 hours. Guided learning students should do the same.)

13. Do you prefer working by yourself to working with others?

14. Do you like to explore the computer and its options?

15. Do you believe that the computer is more than just a tool to accomplish a task?

If you answered yes to at least 10 of the 15 questions, you should be capable of handling ISM 301 in an online-mode rather than face-to-face. This is not a guarantee that you will pass it or make a high grade, only that you have the potential to be successful in this mode.

Appendix B.

ISM 301 - Pretest/Posttest
This does not count towards your grade - but you must complete it
Do not use your text

Name ______________________________

Part I. Multiple Choice. Please write the letter of the answer you feel best fits in the blank to the left of the question.

_____ 1. Common computer systems tasks requiring telecommunications include:

1. off-line text retrieval
2. inquiry/response
3. batch transaction processing
4. printing of checks
5. none of the above
   

_____ 2. Which of the following are telecommunications applications?

1. voice mail
2. facsimile
3. EDI
4. inquiry/response
5. all of the above
   

_____ 3. Under the old information architecture, most of the computing was done on:

1. mainframes
2. workstations
3. laptops
4. microcomputers
5. all of the above
   

_____ 4. Unit testing is another name for:

1. conditional statement testing
2. hardware testing
3. program testing
4. logic testing
5. all of the above
   

______ 5. An information systems plan must:

1. support the overall business plan of the organization
2. contain a business strategic plan, listed environmental opportunitities, and a list of critical success factors
3. delineate existing business needs
4. eliminate application development backlogs
5. none of the above
   

______ 6. Prototyping is all of the following except:

1. a method that can reduce the time, cost, and efficiency in specifying requirements
2. a working model of a system
3. more explicitly iterative than the conventional “life cycle”
4. a refined version of the final system
5. no exceptions - all fit
   

_____ 7. Users:

1. often have expectations that lead them to be disappointed even if the system works well
2. have little trouble in defining precise and proper measures of system quality
3. do not participate in systems development projects
4. should be leaders of the systems development projects
5. none of the above
   

_____ 8. An information system can be defined as:

1. A set of interrelated components that collect (or retrieve), process, store, and distribute information
2. a set of procedures for processing and storing data
3. a tool for supporting management
4. all of the above
   

_____ 9. Processing:

1. converts raw material (data) into a more meaningful form
2. is the first of the information systems activities
3. transfers information to the people or activities where it will be used
4. all of the above
5. none of the above
   

_____ 10. Standard Operating Procedures:

1. are used to coordinate a person's work
2. are formal rules
3. have been developed over a long period of time
4. all of the above
5. none of the above
   

_____ 11. The claim of individuals to be left alone, free from surveillence or interference from other individuals or organizations, is known as:

1. privacy
2. freedom of speech
3. the categorical imperative
4. copyright
5. none of the above
   

_____ 12. Secondary storage refers to:

1. relatively long-term storage of data outside of the CPU
2. relatively short-term storage of data inside the CPU
3. massive data storage in the CPU
4. storage of less important data and information
5. storage of alphabetic characters only
   

_____ 13. Optical disk technology is becoming popular particularly for the storage of all the following except:

1. pictures
2. data that is constantly changing
3. sound
4. full motion video
   

_____ 14. The telecommunications medium with the highest transmission speed is:

1. fiber optic
2. microwave
3. conditioned wire
4. voice wire
5. 10BaseT
   

_____ 15. Which of the following is not one of the main conversion strategies?

1. parallel
2. direct cutover
3. unit
4. pilot
5. all of the above
   

Part II

16. ISDN

17. CASE

18. OPP

19. CAD

20. GSS

21. ESS

22. AI

23. MIS

24. LAN

25. WWW