DEOSNEWS Vol. 2  No. 5.
Copyright 1992 DEOS - The Distance Education Online Symposium
   
Editor: Morten Flate Paulsen, MFP101@PSUVM.PSU.EDU
Review: Kenneth W. Borland
   
DEOS was established with a grant from the Annenberg/CPB Project.
   
---------------------------------------------------------------------------
   
ABSTRACT
   
At the post-secondary level, various studies have been conducted to
evaluate the educational effectiveness of different distance learning
technologies. No published study was found that compared the perceived
effectiveness and satisfaction of university faculty teaching in tradi-
tional and distance education courses. This study gathered responses from
46 faculty who taught via the Washington Higher Education Telecommunication
System (WHETS), a video interactive television system, over three semesters
to evaluate their perceptions and satisfaction. In general, faculty were
satisfied with their instruction over WHETS in terms of the effectiveness
of their teaching strategies, lack of need to make major curricular
adaptations, and ability to perform selected activities in the classroom.
Over two-thirds said they were satisfied with teaching over WHETS. The
major factor contributing to loss of satisfaction was quality of student--
teacher interaction. Results are being used to design effective faculty
development programs.
   
   
THE AUTHORS
   
Janet Ross Kendall, Assoc. Director, Extended Academic Programs
Washington State University, 202 Van Doren Hall, Pullman, WA 99164-5220
Bitnet: KENDALLJ@WSUVM1
   
Muriel Oaks is Director of Extended Academic Programs and WHETS at Washing-
ton State University.
   
   
             EVALUATION OF PERCEIVED TEACHING EFFECTIVENESS:
             COURSE DELIVERY VIA INTERACTIVE VIDEO TECHNOLOGY
                   VERSUS TRADITIONAL CLASSROOM METHODS
   
                   By Janet Ross Kendall and Muriel Oaks
   
Washington State University (WSU) has used the Washington Higher Education
Telecommunication System (WHETS) since l985 to deliver courses among
students at WSU's four campuses -- Pullman, the main campus, located in a
rural area near the Idaho and Oregon borders; and Spokane, Tri-Cities, and
Vancouver, WSU's branch campuses, located in urban areas around the state.
(A WHETS site is also available at the University of Washington in Seattle,
and the University of Idaho is linked with an adjoining microwave system.)
WHETS is a two-way video interactive telecommunications system that uses
microwave technology to link sites. Students enrolled at the Pullman campus
tend to be younger, full-time students. Students at the branch campuses
are, for the most part, older (average age approximately 35 years),
employed professionals who take courses on a part-time basis, although the
demographics are beginning to change as more undergraduate programs are
added.
     Courses offered via WHETS are at both the graduate and undergraduate
levels in such diverse areas as engineering, humanities, business adminis-
tration, and human nutrition. The majority originate from the Pullman
campus where most WSU faculty are located. About 30% of classes originate
from one of the branch campuses. Courses taught to students at one campus
may be delivered simultaneously to students at the other campuses. As of
1990, about 35 different courses are taught on WHETS each semester.
     In recent years there has been a great increase in the use of distance
education technologies to deliver courses to students at all educational
levels. A recent report by the U.S. Office of Technology Assessment,
Linking for Learning: A New Course for Education (1989), examines the use
of such technologies to improve the quality of education for students and
training for teachers at the elementary and secondary (K-12) levels. The
report finds, for example, that prior to l988, fewer than 10 states were
significantly involved in distance learning; today virtually every state is
involved in distance education activities.
     At the post-secondary level, a number of studies have been conducted
to evaluate the educational effectiveness of various technologies. Many
have focused on the achievement of students in distance education and
traditional classrooms (e.g., Chu & Schramm 1967, 1975; Ritchie & Newby
1989; Seigel & Davis 1990; Smith 1983; Whittington l986). Almost without
exception such studies have shown that students taking courses via distance
education technologies achieve as well as students taking courses via
traditional methods.
     Others have examined students' satisfaction with various distance
education technologies (e.g., Topper et al. 1975). For example, Barker and
Platten (l988) sent a survey to the 31 students enrolled in a graduate
teacher education course presented through the TI-IN Network. Responses
indicated that 53.8 percent felt that the satellite instruction was as
interesting as that received in a regular classroom; 38.4 said it was less
interesting, while 7.6 percent said it was more interesting. Over two--
thirds (69.2%) said they would enroll in other credit courses offered via
satellite. Grimes, Nielsen and Niss (1988) evaluated the influence of
student-teacher contact on on- and off-campus students' attitude toward the
subject matter. Contrary to their expectations, there was little difference
among groups; they stated, "we can not draw a general conclusion that
greater student-teacher contact will lead to more positive attitude
formation for students in a telecourse situation" (p. 42). Ritchie and
Newby (1989) randomly assigned 26 college undergraduates to three classroom
settings: traditional classroom with an instructor, TV studio with an
instructor, and a "distant" studio classroom with television monitors and
no instructor. In contrast to Grimes et al., their students in the "dis-
tant" classroom rated instruction less enjoyable than those students who
had an instructor with them in the classroom; they also rated their
involvement as significantly lower than the other two groups. However, the
situation was somewhat contrived, and results may not generalize to
students in real "distant" classrooms.
     To our knowledge, no studies have compared the perceived effectiveness
and satisfaction of university faculty teaching in traditional and distance
education courses. Thus, the study reported here was conducted. Faculty
were surveyed about their satisfaction with teaching over WHETS, the
effectiveness of using the system to reach their educational goals, and
their willingness to use the system again. Such information is critical in
developing successful faculty development programs to increase faculty
effectiveness in using the existing as well as the emerging distance
education technologies.
   
   
METHOD
   
     Instruments: Two questionnaires were used to collect the evaluation
data reported here. Construction of the first questionnaire involved two
steps. First, interviews were conducted with five WSU faculty members with
experience teaching via WHETS as well as in traditional classrooms. The
faculty were from five different academic departments and taught different
types of courses (e.g., lecture vs. case study, graduate vs. undergrad-
uate.) Their perceptions were used to construct a structured, self-admi-
nistered questionnaire. The questionnaire consisted of demographic,
open-ended, and forced-choice questions, several of which used a four-point
Likert scale.
     Because many of the forced-choice questions did not allow faculty to
rate WHETS as "equally" effective in comparison to a traditional classroom,
a revised questionnaire was prepared. This second questionnaire was
intended to provide data more useful to personnel working on faculty
development and support services. Personnel recognized the importance of
knowing when WHETS is rated as equally effective so that time is not spent
working on situations where WHETS and traditional classrooms do not differ.
Three faculty who work with WHETS teachers identified 38 questions where an
"equally effective" response would be more meaningful for faculty develop-
ment planning; four questions judged to be unimportant were eliminated, and
for the remaining 34 questions the four-point Likert scale was changed to a
five-point scale (see Table 1 for example). Several additional questions
were also written to collect information not included on the first ques-
tionnaire.
   
Table 1 EXAMPLE OF 4-POINT AND 5-POINT SCALE
---------------------------------------------------------------------------
                              Strongly                Strongly
                              Disagree                 Agree
---------------------------------------------------------------------------
I have had to give up
some favorite teaching           1       2       3       4
techniques because they
don't work on WHETS.             1     2     3     4     5
---------------------------------------------------------------------------
   
     Subjects: All 46 faculty who taught over WHETS during the l989
calendar year received a copy of the first questionnaire. Each of these
faculty members also had several years of experience teaching in tradi-
tional classrooms.
     The second questionnaire was sent to the 12 faculty who taught over
WHETS during summer 1990 and the 33 faculty who taught over WHETS during
fall 1990. All faculty had several years of experience teaching in tradi-
tional classrooms.
     Procedures: To gather faculty perceptions used in constructing the
first questionnaire, faculty were interviewed in a location and at a time
of their choice. Interviews were tape recorded to ensure accuracy of
reporting of responses. Faculty were assured of confidentiality and told
that tapes would be erased when the report was completed.
     The mail survey procedures suggested by Dillman (l978) were followed
for distributing and collecting all sets of questionnaires (i.e., use of a
cover letter, pre-addressed return envelope, follow-up letter thanking
faculty for participation, follow-up letter to faculty who had not yet
returned the survey).
Data Analysis: Frequency distributions and crosstabulation tables were
constructed to examine faculty responses.
   
   
RESULTS AND DISCUSSION
   
     The first questionnaire was returned by 32 of the 46 faculty who
taught during 1989, a 70% response rate. The second questionnaire was
returned by 11 of the 12 faculty (92%) who taught during Summer 1990 and by
25 of the 33 faculty (76%) who taught during Fall 1990. Responses were
combined from the three sets of questionnaires when questions were ident-
ical, providing data from 68 faculty; responses were analyzed separately
for the questions that were different on the 1989 and l990 questionnaires.
     Perceived instructional effectiveness: Faculty were asked to compare
the effectiveness of their teaching strategies over WHETS versus the
traditional classroom. As Table 2 shows, almost all faculty used lecture,
group discussion and question/answer strategies; the other four strategies
were used by a majority of faculty. Lecture, case study and question/answer
strategies were judged to be as effective over WHETS as in the traditional
classroom. Ten percent of faculty indicated lecture was more effective over
WHETS, and nearly one-third felt similarly about lab/demonstration strat-
egies. In contrast, a significant percentage said group discussion,
seminar, socratic, and question/answer strategies were less effective.
   
Table 2 USE OF TEACHING STRATEGIES OVER WHETS*
--------------------------------------------------------------------------
Question 6. Some teaching strategies are believed to be better suited than
others to use when teaching a course over WHETS. Select the statement which
best describes your experience with the effectiveness of each teaching
strategy when teaching on WHETS as compared to the traditional classroom
setting.
                                     Effectiveness compared to
                                        traditional setting
Strategy              Who Uses        Less     Equal     More
                       %  (N)          %         %         %
--------------------------------------------------------------------------
Lecture               100 (68)         35        55        10
Group discussion       96 (65)         72        25         3
Case study             49 (33)         33        61         6
Seminar                54 (37)         62        35         3
Question/answer        96 (65)         48        48         4
Socratic               59 (40)         60        32         8
Lab/demonstration      64 (23)         35        35        30
--------------------------------------------------------------------------
*  Table includes all 1989 and 1990 respondents except
   "lab/demonstration" (1990 only).
   
     Table 3 describes curricular adaptations faculty said they made when
they began teaching on WHETS. Few faculty felt they needed to make changes
to their syllabus, their course organization, assignments or exams. The
majority indicated they had to make adaptations to their delivery, audio--
visual aids, and interaction with students.
   
Table 3 CURRICULAR ADAPTATIONS*
--------------------------------------------------------------------------
Question 5. Thinking now about the most recent course you taught on WHETS,
what adaptations, if any, did you make when you began teaching the course
on WHETS?
                           No or minor        Significant
                             changes        or major changes
Curricular Components         (%)                 (%)
--------------------------------------------------------------------------
Course syllabus                79                  21
Organization of Course         68                  32
Interaction with students      38                  62
Handouts                       54                  46
Audio-visual aids              48                  52
Assignments                    71                  29
Exams                          81                  19
Delivery of content            49                  51
--------------------------------------------------------------------------
*  Table includes all 1989 and 1990 respondents.
   
     Faculty were asked to indicate whether using WHETS has affected their
ability to perform selected activities in the classroom (Table 4). Items to
which the majority of the 68 faculty indicated their ability was no
different whether using WHETS or not included: respect students' opinion
and feelings, stimulate students' critical thinking skills, prepare lesson
plans based on needs of the learner, and encourage expression of different
points of view. Half the faculty said WHETS sharpened their ability to
consciously plan for the instructional event, and over four-fifths said
they could use audio-visual aids as well or better over WHETS. Consistent
with the results reported above, half the faculty selected "cannot do as
well when using WHETS" to describe their ability to give hands-on experi-
ence, encourage participation in class discussions, and actively involve
students.
   
Table 4 PERCEPTIONS OF EFFECTIVE TEACHING USING WHETS*
--------------------------------------------------------------------------
Question 13. The following list contains characteristics of effective
teachers as described by a group of WSU professors. Using the key given,
indicate the number of the response choice which best describes you for
each item on the list.
                           Can't do as             Can do better
                          well on WHETS    Equal    with WHETS
Teaching activities              %           %       %    (N)
--------------------------------------------------------------------------
Actively involve students       49          48       3    (67)
Encourage participation         54          40       6    (67)
Respect student's opinions      15          84       1    (67)
Stimulate critical thinking     26          72       2    (65)
Give hands-on experience        55          45       0    (66)
Consciously plan for the
 instructional event             5          46      49    (65)
Prepare lesson plans based
  on needs of learners          18          65      17    (66)
Encourage the expression of
  differing viewpoints          39          59       2    (66)
Use active, interesting
  audio-visual aids             18          42      40    (67)
--------------------------------------------------------------------------
*  Table includes all 1989 and 1990 respondents.
   
     Faculty were asked to respond to several open-ended questions. To "the
one best thing about WHETS is . . .", 53% of the 68 faculty mentioned
issues related to the value of increased access to education for place--
bound students. One faculty member wrote: "It truly is providing desper-
ately needed education to underserved areas." Time and cost savings for
students and faculty were cited by 15% of respondents, and 12% mentioned
that the quality and motivation of off-campus students was the one best
thing about WHETS. Interestingly, there were a wider variety of comments
made by faculty who taught in the fall 1990 semester than by those who
taught during fall and spring semesters of l989. Fewer faculty from a
narrower range of academic departments had taught over WHETS as of 1989,
and the institution viewed WHETS as a less expensive way to deliver high
quality courses to placebound students; faculty seemed to be reflecting
this view. By fall 1990, more faculty had gained experience teaching over
WHETS, and a number of undergraduate general studies classes were being
taught to two branch campuses. The wider variety of comments probably is
due to the greater number of faculty from more departments who could
discuss their experiences in teaching over WHETS with each other and with
their colleagues, and who were able to see more benefits, as well as
drawbacks, inherent in the system.
     Level of satisfaction: When asked to rate their level of satisfaction
with their teaching careers as a whole, 97% of the 1989 faculty reported
they were "satisfied" or "very satisfied." 1990 faculty responded similar-
ly, with 94% rating their teaching careers the same. Sixty-nine percent of
the 1989 faculty gave these same ratings when asked to describe their level
of satisfaction when teaching on WHETS; 31% chose "not satisfied" or "not
at all satisfied." 1990 faculty gave similar satisfaction ratings to
teaching on WHETS: 58% chose "satisfied" or "very satisfied." However, 31%
chose "equally satisfied", and only 11% indicated they were not satisfied.
Although the percentages indicating satisfaction with teaching on WHETS are
lower than those for their teaching careers as a whole, more than two--
thirds of 1989 faculty are satisfied with teaching via WHETS, and 89% of
1990 faculty are equally as satisfied or better.
     The major factor contributing to differences in satisfaction levels
was a loss in the quality of the student-teacher interaction. Three--
quarters of the faculty said what they missed most in teaching via WHETS
was the lack of personal interaction with students at the other campuses at
which the course was received. This can also be seen in responses to
related forced-choice questions (see Tables 2, 3 and 4). For example, 88%
of 1989 faculty and 83% of 1990 faculty agreed it was very important to
visit students at other campuses. The five-point scale helped to clarify
responses to another related question, "My teaching over WHETS is more
impersonal than in an on-campus class." In 1989, 66% agreed that their
teaching over WHETS was more impersonal than in a traditional class. 1990
responses showed that only 41% agreed, while 18% felt teaching was similar
in either type of class setting.
     Student characteristics: When asked to describe off-campus students in
comparison to on-campus students, 1989 faculty were nearly unanimous that
off-campus students were more motivated and had a higher achievement level
(Table 5). The five-point scale modified faculty members' responses
somewhat: two-thirds agreed that off-campus students were more motivated
and had a higher achievement level, but one-quarter and one-third, respect-
ively, indicated that there was no difference between the two groups of
students. There was less agreement about students as "self-starters"; 62%
of 1989 faculty chose "self-starter" while 38% said off-campus students
need more direction than on-campus students. Thirty-eight percent of 1990
faculty chose "self-starter", 35% chose the neutral response, and 26% said
off-campus students need more direction. This latter finding may be related
to the larger number of undergraduate classes being taught in 1990,
enrolling students less confident about their academic skills. In contrast,
most courses taught in 1989 were graduate courses which, for the most part,
enrolled working professionals.
   
Table 5 STUDENT CHARACTERISTICS (1)
--------------------------------------------------------------------------
Question 4. How do off-campus WHETS students compare to on-campus students
taking the same course?
                         %           %           %
                      selecting   selecting   selecting
                      response    response    response
   
               1989     1,2                     3,4
               1990     1,2           3         4,5
--------------------------------------------------------------------------
More motivated           87                      13  Less
                         64          24          12   motivated
   
More life               100                       0  Less life
 experience              70          21           9   experience
   
Older                    87                      13  Younger
                         56          24          20
   
High achievement         83                      17  Just get by
 level                   62          32           6
   
Self starters            62                      38  Need
                         38          38          26   direction
   
Academically (2)                                     Academically
 confident               18          56          26   insecure
--------------------------------------------------------------------------
(1)  Data reported separately for 1989 and 1990, as %.
(2)  Question not asked in 1989.
   
   
CONCLUSIONS
   
     In considering their teaching strategies, the majority of faculty
perceive lecture, question/answer, and case studies as more or equally
effective over WHETS in comparison to a traditional classroom setting;
group discussion and seminars were rated as less effective by a majority of
faculty. The professors surveyed felt their general instructional effec-
tiveness was not significantly diminished; they identified several instruc-
tional activities they could perform equally well in a WHETS or traditional
classroom. The major concern faculty had was with the loss of the active
involvement with students that they experience in a traditional classroom.
Data collected in 1989 suggested this concern affected their personal
satisfaction with teaching over WHETS; responses to the five-point scale
used in the 1990 questionnaire suggest most faculty (89%) are equally as
satisfied or better with teaching via WHETS. In fact, if asked, 96% would
teach over WHETS again. Personnel working with WHETS faculty will continue
to recommend that faculty consciously implement strategies that enhance
interaction. Improvement of communications between faculty and "receive--
site" students could be enhanced by bringing the two closer together
through large screen television monitors, frequent written feedback from
students, making electronic mail and computer conferencing systems easier
to access, and encouraging personal visits to the "remote" campuses
receiving the course.
     This questionnaire will be completed by faculty teaching via WHETS
each semester. Future analyses will examine effectiveness ratings by
various groupings of faculty (e.g., undergraduate general studies vs.
graduate business administration). Student evaluations of faculty teaching
effectiveness and students' course grades are also being collected and will
be correlated with faculty responses. The ultimate purpose of the data
collection and analyses is to design meaningful and effective faculty
development programs.
   
   
REFERENCES
   
Barker, B.O., & Platten, M.R. 1988. Student perceptions on the
  effectiveness of college credit courses taught via satellite.
  The American Journal of Distance Education 2: 44-50.
   
Chu, G.C., & Schramm, W. 1967. Learning from television: What
  the research says.  Washington, D.C.:  National Association of
  Educational Broadcasters.
   
_____.  1975.  Learning from television:  What the research says.
  Palo Alto:  Stanford University.  ERIC Document Reproduction
  Service, ED 109 985.
   
Dillman, D.A. 1978.  Mail and telephone surveys:  The total
  design method.  New York:  Wiley.
   
Grimes, P.W., Nielsen, J.E., & Niss, J.F.  1988.  The performance
  of nonresident students in the "Economics U$A" telecourse.  The
  American Journal of Distance Education 2: 36-43.
   
Ritchie, H., & Newby, T.J.  1989.  Classroom lecture discussion
  vs. live televised instruction:  A comparison of effects on
  student performance, attitude, and interaction.  The American
  Journal of Distance Education 3: 36-45.
   
Seigel, A.E. & Davis, C.  1990.  Delivering undergraduate
  engineering courses on television:  How do grades compare?
  [Abstract].  Proceedings of the Seventh Canadian Conference on
  Engineering Education (pp. 10-13).  Toronto, Ontario, Canada.
   
Smith, J.  1983.  Evaluation of the telecourse program at
  Saddleback College:  Student retention and academic
  achievement.  Unpublished doctoral dissertation, Nova
  University, Fort Lauderdale.
   
Topper, M.D., Singleton, W.L., Attebury, A., Birdwell, D.B., &
  Schumann, D.  1975.  Telecourses: The "Open University" in
  Dallas.  Southern Methodist University.
   
U.S. Office of Technology Assessment.  1989.  Linking for
  learning:  A new course for education.  Report # OTA-SET-430.
  Washington, D.C.: U.S. Government Printing Office.
   
Whittington, N.  1986.  Instructional television:  A research
  review and status report.  Austin:  Coordinating Board, Texas
  College and University System, Division of Universities and
  Research.
   
----------------- End of DEOSNEWS Vol. 2  No. 5 ------------------------
   
 

Top of Page

The American Center for the Study of Distance Education (ACSDE)
The Pennsylvania State University
College of Education
110 Rackley Building
University Park, PA 16802-3202
Phone (814) 863-3764  FAX (814) 865-5878
ACSDE@psu.edu
www.ed.psu.edu/ACSDE

©2001 The Pennsylvania State University
College of Education