DEOSNEWS

Volume 13 - Issue 2

13.2 Total Quality Management Strategic Plan for Distance Course Development

Editorial

This issue of DEOSNEWS examines one university’s effort to address two challenges in distance learning: retention and student satisfaction. Faced with poor student satisfaction ratings in courses delivered by videoconferencing and high drop-out rates in online classes, Northeastern State University implemented a plan to address these challenges through development and implementation of a plan that ties faculty compensation to production and delivery of high-quality learning experiences in these two environments. Although not a classic TQM project, the initiative described here, with its emphasis on establishing quality indicators and measures for key elements of the online teaching-learning process—course quality and quality of instruction—reflects a growing body of literature relating to quality issues in distance education. This trend is an important one for our field. Not only is it providing strong evidence of both an intent to develop, and success in developing, high-quality distance education courses, but it is also setting the stage for a much-needed examination of quality issues and standards in “traditional,” face-to-face instruction.

Melody Thompson
Director, American Center for the Study of Distance Education
DEOSNEWS editor

Total Quality Management Strategic Plan for Distance Course Development
Donna G. Wood, Melissa Roberts Becker, Rodney L. Osborne, Joyce A. Van Nostrand, Sharon A. Winn, and Chuck Ziehr

Northeastern State University, Tahlequah, Oklahoma

Distance learning courses are the newest product category sweeping the educational markets internationally. Like any other product or service, the key to the success of a distance learning course is quality. Experts assert that advertising and public relations tactics cannot motivate consumers to purchase poor quality products. In the mid-to-late 1960s, the Japanese introduced the concept of product quality through an approach called Total Quality Management or TQM, which was developed by W. Edwards Deming, an American. The Japanese began to focus on major improvements that enhanced and improved product-performance over the long term. Consumers responded to the emphasis on quality and made Japanese products very successful. By simply stressing the concept of product quality, the Japanese wrested market after market from the United States (Schultz & Barnes, 1999).

Taking a cue from Japanese TQM programs, Northeastern State University in Oklahoma developed and implemented a total quality management strategic plan for their distance learning program and courses, which is discussed in this case study. Complete quality-measuring instruments, or rubrics, and application forms were developed, the links to which are included in this case study.

Need For Quality Assurance

While NSU has offered videoconference classes for a number of years, and began offering online courses in the late 1990s, student reaction had not been seriously measured until the spring 2000 semester, when 73% of students taking videoconference classes indicated that technical problems interfered with their learning. During the 2001 academic year only 61% of students taking videoconference classes indicated they learned as much in a videoconference class as an on-site class. In addition, during the 2001 academic year, the dropout rate in some online classes was 20 to 30 percent higher than in on-site classes.

Despite the poor student evaluations for videoconference classes and the high dropout rate for online classes, the demand for distance classes continued. The disparity between distance learning success and the demand for more distance courses led the Distance Learning Committee to risk collegial suicide by changing the requirements for instructor compensation. In order for instructors to receive compensation for distance courses, their courses would now have to be evaluated against standards of best practice through a peer review process. The previous compensation policy granted full compensation for simply teaching a distance course. Course development and quality was not an issue.

Based on a report by the Distance Learning Policy Laboratory Quality Assurance Subcommittee of the Southern Regional Education Board, several higher education institutions rated peer review as the most effective method of ensuring quality (p. 8). The Northeastern administration granted the Distance Learning Committee the responsibility for determining the amount of compensation based on the quality of course development. The criteria for quality and the development of an evaluation instrument were also the responsibility of the committee.

Evaluating the Quality of Distance Courses

The committee developed a rubric in order to evaluate the quality of distance course experiences. It reviewed best practices and standards established by professional organizations and other institutions. The development of the evaluation criteria was based on the North Central Accreditation Best Practices for Electronically Offered Degree and Certificate Programs (Addendum to the Handbook of Accreditation, Second Edition 47 Chapter Reference C). Other sources for the criteria included: ADEC Guiding Principles for Distance Teaching and Learning; Seven Principles of Good Practice in Undergraduate Education by Chickering and Gamson; Seven Principles of Effective Teaching: A Practical Lens for Evaluating Online Courses by Graham, Cagiltay, Lim, Craner, and Duffy; the Maryland State Department of Education Instructional Rubric for Reviewing Web-Based Courses Secondary Level Students, and California State University-Chico Committee for Evaluation of Exemplary Online Courses. The committee discussed issues such as whether to equate point values with the amount of compensation, point structure assigned to the compensation, and reliability among the various evaluators.

Finding “early adopters” in distance course development was the next step. This was handled through an application process for videoconference course development and for online course development.

Compensation

The Distance Learning Committee then moved full-throttle through dangerous waters with a map that changed the course of tradition. Compensating faculty for teaching a distance course was being changed to an expectation of quality in teaching in a non-traditional mode. Compensating faculty for quality course development was the proverbial paradigm shift.

The policy states that faculty can receive $1,800 for online courses meeting the criteria. For videoconference course development the total compensation is $1,000 with an additional $250 up to three times for course improvement.

Videoconference Course Development Standards and Expectations

The videoconference course development evaluation instrument was linked to a certification program. Faculty members were required to complete the program before applying for compensation, and the committee strongly recommended certification before teaching the course. The certification program consisted of four areas: basic equipment operation, instructional materials conversion, instructional strategies, and logistical planning for remote sites.

Basic equipment operation training occurred during a one- to two-hour workshop. Although NSU provides student facilitators to operate the equipment during class time, faculty could better integrate the equipment into the course if they understood the operational procedures. Instructors demonstrated the following skills at the conclusion of the workshop: initiate and terminate a connection between videoconference sites, operate the instructor and student microphones, play a video tape and/or CD/DVD over the videoconference system, demonstrate appropriate voice projection and microphone placement, and demonstrate awareness of camera positions relative to student eye contact.

Course material development for digital transmission comprises the second component with the most important criterion being readability by students at remote sites. Faculty document these criteria by producing printed materials appropriate for use on a document camera; producing a word-processed document appropriate for use on a computer that is functioning through the videoconference system; using a web browser to increase font sizes of web pages for better visibility through the videoconference system; and demonstrating Section 508 compliance of course web sites if appropriate.

Instructional strategies that enhance student engagement comprise the third component. Strategy descriptions include collaborative inquiry, reflective thinking, technology enhancement, or service learning. The NSU Center for Teaching and Learning provides workshops in various teaching strategies for faculty development.

Logistical planning for remote sites is the fourth and final component. Instructors address remote site procedures in their course syllabi, e.g., camera purview for attendance and participation, equipment operation for student presentations, assessment proctoring, and course activities schedule. The course calendar is crucial when sending courses to other institutions with a different academic calendar. The syllabus becomes a part of a Receive Site Agreement with the other institutions hosting the course, and the agreement serves as a contract with all expectations articulated. Faculty members who have not previously taught by videoconference use this component as a planning tool in the event of equipment malfunction.

Online Course Development and Expectations

The online course development evaluation instrument uses the Likert scale: Does Not Meet Criteria, Partially Meets Criteria, or Meets Criteria. This standard parallels the videoconference course development instrument. However, the criteria for online course development are based primarily on course design. The videoconference course development criteria focus more on the physical delivery of the content, i.e., operating equipment, transmitting course content effectively, and planning for instructional activities in different locations. The committee determined the online course criteria after researching best practices literature. They are as follows:

1. The learning experiences have a clear purpose with well-defined objectives and outcomes.
2. Well-designed discussion and cooperative assignments are incorporated to facilitate collaborative learning among students.
3. The learner is actively engaged.
4. The course goes beyond knowledge transmission to knowledge creation and critical thinking.
5. The learning environment makes appropriate use of a variety of media.
6. There are clear mechanisms and guidelines for interaction between the student and the instructor.
7. The course emphasizes time on task.
8. Prompt feedback is provided.
9. Effective procedures are in place for acquiring access to appropriate resources to support the course.
10. The course respects diversity, varied talents, and different learning styles.

Examples for implementation of each criterion are included for instructors using the instrument as a guide in course development.

Rubric Application

Actual application of the rubrics resulted in evaluating courses submitted by seven instructors. Two volunteer members of the Distance Learning Committee evaluated each course. Almost all committee members were conducting videoconference or online courses, or both, at the time of evaluation. Immersion in a course outside the committee member’s discipline was necessary though time-intensive. Evaluators essentially developed perspectives from both the instructor and student roles. Several committee members submitted rubric evaluations that documented how the standards were met or not met, and also contained “suggested areas or activities for consideration”. After forwarding the evaluations and compensation recommendations to the Vice President for Academic Affairs, the rubric evaluation was then shared with each respective applicant. Most applicants welcomed this feedback as a type of objective peer evaluation. All instructors received compensation according to the policy.

The Distance Learning Committee plans further study of the application and evaluation processes, outcomes, and evaluators’ comments during the next academic year. Revisions, such as adding the “suggestions” component to each instrument, may be piloted. Further faculty education concerning “course evaluation” versus “teaching evaluation” may be undertaken. The committee realizes it will have to re-evaluate the instruments often, but the continued use of these rubrics promises to benefit students, faculty, and the entire university community.

Conclusion

While developing the rubrics, the Distance Learning Committee recognized the need to move forward with distance courses to meet both traditional and non-traditional student expectations. To provide a successful learning experience for students while maintaining academic rigor, the guidelines serve as a tool for faculty members who are developing distance courses. The online course development instrument addresses issues that primarily focus on student engagement and clearly-stated course objectives. The videoconference course development instrument addresses the necessity for clearly communicating with partner institutions and planning for the logistics of real-time, remote instructional delivery.

The criteria identified by the Distance Learning Committee will be used to measure the dropout rate in online classes and the low student satisfaction rate in videoconference classes. The effects of implementing the standards will be closely monitored, and the results will be disseminated through the Northeastern State University website.

References
American Distance Education Consortium. Guiding principles for distance teaching and learning. Accessed on September 14, 2002 at http://www.adec.edu/admin/papers/distance-teaching_principles.html

California State University-Chico, Committee for Evaluation of Exemplary Online Courses. Rubric for online instruction. Accessed on October 4, 2002 at www.csuchico.edu/tlp/webct/rubric/rubric_final.pdf

Chickering, A.W., & Gamson, Z.F. (1987). Seven principles of good practice of undergraduate education. AAHE Bulletin, 39 (7), 3-7.

Graham, Cagiltay, Lim, Craner, and Duffy (2001, March/April). Seven principles of effective teaching: A practical lens approach to evaluating online courses. The Technology Source. Accessed on August 21, 2001 at http://ts.mivu.org/default.asp?show=article&id=839.

Higher Learning Commission. (2000). North Central Best Practices for Electronically Offered Degree and Programs, Handbook of Accreditation, Second Edition 47, Chapter Reference C.

Maryland State Department of Education (2001). Instructional rubric for reviewing web-based courses secondary level students. Accessed on October 4, 2002 at http://www.msde.state.md.us

Schultz, D. E. & Barnes, B.E. (1999). Strategic brand communication campaigns. Lincolnwood, Illinois, NTC Business Books, 5th Edition.

Southern Regional Education Board, Distance Learning Policy Laboratory Quality Assurance Subcommittee (2002). The challenges of quality assurance in a distance learning environment. Accessed on October 5, 2002 at http://www.electroniccampus.org/policylab/Reports/Quality%20Assurance%20Draft_LG.pdf

About the Authors
Donna G. Wood, Ed.D., is associate professor in the College of Education at Northeastern State University. She also directs the Center for Teaching and Learning and is Interim Dean of the Graduate College. She began development of distance courses during the fall of 1999 and directs the efforts of distance learning at NSU today. Dr. Wood may be contacted at <wooddg@nsuok.edu>.

Melissa Roberts Becker, Ed.D., is assistant professor at Northeastern State University, Tahlequah, Oklahoma in the Department of Educational Foundations and Leadership. Currently, Dr. Becker teaches online instructional technology courses for education majors and was the first virtual professor at NSU. She earned her doctorate from Baylor University.

Rodney L. Osborne, Ph.D., is currently associate professor in the Mass Communication Department at Northeastern State University. He received his Bachelor's Degree in Advertising from Kansas University, his Master's of Communication Degree from Oklahoma State University, and his Ph.D. in Consumer Behavior from Oklahoma State University. Before joining the academic ranks, Dr. Osborne worked for the Oklahoma State University Public Information Department, was Advertising Director for two separate newspaper groups in Oklahoma and Kansas, and owned a small advertising agency.

Joyce A. Van Nostrand, Ph.D., R.N., is an associate professor and Chair of the Department of Nursing on the Muskogee campus of Northeastern State University, Tahlequah, Oklahoma. With the vision of reaching out to regional and rural registered nurses, via both video conferencing and online, RN to BSN program completion options were successfully developed and implemented, the first for the university. Full approval to continue these distant education delivery modalities was received from the Oklahoma State Regents for Higher Education in spring 2003, based in part on successfully benchmarking with other U.S. undergraduate and graduate online nursing programs. Dr. Van Nostrand continues to teach online and video conferenced courses each semester and is a member of the university’s Distance Education Committee.

Sharon A. Winn, Ph.D., is an Associate Professor of English at NSU who has developed several online courses in literature. Her course in T.S. Eliot was one of the first courses reviewed under the new online rubric. She is NSU's 2004 nominee for the Boyer award for Innovative Excellence in Teaching, Learning and Technology.

Chuck Ziehr, Ph.D., is Professor of Geography at Northeastern State University. He has taught for 14 years at NSU and for 12 years before that at East Carolina University (where he taught geography and directed the Y.H. Kim Social Science Computer Lab). He has a Ph.D. from Indiana University, a M.S. from the University of South Carolina, and a B.A. from the University of Arkansas. Dr. Ziehr is an economic geographer who also specializes in geographic information systems (GIS). All his courses are web-enhanced and can be accessed at http://arapaho.nsuok.edu/~ziehr/. He is active in faculty governance, academic technology innovations, and first-year student success/retention.