11.7. Distance Education by Interactive Videoconferencing in a Family Practice Residency Center

Editorial

All current wisdom to the contrary, there are times when dense blocks of information delivered in a lecture format, or a lecture plus question time, really is the most efficient, effective, culturally sensitive choice. I know that sounds like heresy to contemporary ears in education, and coming from me—constructivist to the core—too! I am, after all, the one who spends her days trying to transition faculty from classroom lectures to online discussions.

When I first read the following article, I really had to stop and think. This lecture stuff went so much against my grain. It wasn't until I broadened my context and looked at the function of this content delivery in the matrix of medical education that it all made sense. My biggest complaint with classroom lectures is that—in so many instances—it is purely knowledge transmission, divorced from the community of practice in which it belongs. In this case, the information transmission was a vital part of a series of practical, applied learning experiences.
The videoconferencing reported below is a great example of lectures being used to their best effect—and the synergy of scattered practitioners able to share a single learning experience. Information could be taken and, with some immediacy, turned into active, working knowledge—a far cry from many lecture experiences.

Mauri Collins
DEOSNEWS Editor

Distance Education by Interactive Videoconferencing in a Family Practice Residency Center

Orlando F. Mills, James F. Bates, Vicki Pendleton, Kathleen Lese, and Michael Tatarko

Abstract

Interactive videoconference technology may be a useful adjunct in family practice residency education by expanding the number of lectures available for residents. In this study, live videoconferences were arranged between the Johnstown (Pennsylvania) family practice residency and other sites. Participants completed standard evaluation forms, and both technical and educational quality were evaluated by quantitative and qualitative analysis. Thirty videoconferences were conducted and 382 evaluation forms were analyzed. Overall, the quality was good. In the best conferences, there was good sound and video, and speakers gave clear handouts in advance and used slides that were not overly wordy or complicated. The worst conferences suffered from poor sound or video transmission.

Introduction

Following graduation from medical school, all physicians enter a residency training program in the specialty of their choice. Residency training programs last from three to five years—and sometimes longer—and consist of both formal classroom learning and hands-on patient care under the supervision of medical faculty or other experienced program graduates, also known as “attending” physicians.

The Residency Review Committee (RRC) for family practice determines the rules and regulations that govern family practice residency programs. The RRC mandates that each family practice residency program provide a minimum of three conferences/seminars per week for family practice residents. The seminars are a mixture of core and subspecialty topics designed for family practice physicians in training. The lectures provide the family practice resident the opportunity to interact with other family practice residents and experienced or “attending” physicians about common and uncommon clinical problems. Planners for each residency program spend a significant amount of time designing and presenting a particular series of lectures.

An alternative to this time-intensive process is to “share” lectures with other residency programs or educational facilities using interactive videoconferencing. Interactive videoconferencing has been used in emergency medicine (Tachakra et al. 1997), urology (Hayes et al. 1998), dermatology (Perednia 2000), radiology (Gray, Somers, and Buckley 1998; Franken, Driscoll, and Berbaum 1998), otorhinolaryngology (Blakeslee et al. 1998), psychiatry (Zarate et al. 1997), family practice residency precepting (Mills et al. 1999) and other applications (Lewis et al. 1998; Reid et al. 1998; Zollo et al. 1999). This technology allows participants inresidency programs to follow lectures from other programs or educational facilities without leaving their site.

Literature Review

There is support in the literature for videoconferencing as a good method for medical education in residency programs. Reid et al. (1998) transmitted twelve video continuing medical education conferences to four sites in Nova Scotia and reported satisfactory scores from 195 attendees. Zollo et al. (1999) described their telemedicine system (interactive videoconferencing) in Iowa that transmits good quality continuing education conferences to sixty affiliated hospitals.

The literature also shows that videoconferencing is sometimes evaluated negatively by learners. Lewis et al. (1998) reported that resident attitudes about videoconferencing became more negative after viewing subspecialty videoconferences. Lewis and his colleagues did not measure the technical quality of the transmission, nor did they comment on the skill or preparation of the speaker. It is quite possible that the audio or video failed at times, or that the speaker was not skilled in presenting via videoconference. Such factors would serve to reduce the value of the conference.

Our family practice residency program started a project to evaluate interactive videoconferencing as a possible learning modality for our resident physicians. We began with the premise that videoconference education could provide a feasible, reliable, and acceptable supplement to the existing lecture curriculum. We designed our study specifically to evaluate both the technical quality and the educational-content quality of videoconference lectures for residency education.

Context

The site for this project was the training program in family medicine in Johnstown, Pennsylvania. Johnstown is a small town with a rural surrounding area. Since access to experts in subspecialty medical fields is limited in the Johnstown area, we felt that videoconference education was warranted.

Methods

Equipment

We used a two-way videoconferencing unit (Tandberg 3000) that communicated with one or more remote units in Pennsylvania over three integrated services digital network (ISDN) lines at a total transmission speed of 384 kilobits per second—approximately twelve times the transmission speed of a home-computer connection. We chose this speed so that high-quality video and audio transmission could be achieved in real time. Johnstown’s family practice residency program connected to the Hershey Medical School, which is our organization’s academic affiliating medical school, located in Hershey, Pennsylvania. We connected to the medical-student classroom at Hershey for twenty-five conferences, and to three other sites in Pennsylvania for five conferences. The additional sites were chosen by a local area health education center as part of a health care communications project. The videoconference equipment at each site had a mounted camera with tilt-and-zoom capabilities to enhance the residents’ feeling of actually being at the main conference site.

Videoconference Communication

The conferences were held once or twice per month, from noon until 1:00 pm or from 8:00 until 9:00 am, in our family practice residency conference room in Johnstown. We held twenty-five noon conferences between a medical student lecture hall at the Hershey Medical School and the Johnstown family practice center conference room. The noon lectures were held between February 13, 1998 and February 11, 2000. In addition, we held five morning conferences in the form of multisite seminars involving physicians and other health care personnel located in the Pennsylvania communities of Coudersport, Honesdale, and Hershey. In the multisite conferences, the four sites were linked by a videoconferencing bridge that allowed simultaneous transmission to more than one site. The five multisite morning seminars occurred in October 1998, January 1999, April 1999, July 1999, and January 2000.

The connection between our family practice conference room and other sites was usually established ten to fifteen minutes before the conference. On occasion, we encountered line difficulties; sometimes it took two or three attempts before we completed the connection. During the conference, problems with the audio, video, and/or audio-visual aids were addressed by telephone communication between the technicians. Our site was in “receive-only” mode for the first sixteen teleconferences from the Hershey Medical School lecture hall, meaning that the distant site could not view our conference room during the seminar. During the other fourteen conferences, the personnel at the remote site could see and speak with our attendees in real time.

Participants

The audience for the twenty-five noon conferences was a combination of second-year medical students on site at Hershey Medical School and our residents, and the sessions were held by videoconference. Our attendees included the videoconference technician; family practice faculty; first-, second-, and third-year family practice residents; medical students; nurses; and others from the residency. The Hershey Medical School chose the lecture topics and speakers. The speakers were aware that we were attending the lecture and usually acknowledged our site during the lecture, occasionally directing questions to us. Some lecturers provided us with a copy of the lecture handout, and others did not.

The audience for the five multisite conferences included our residents and two to four participants at each of the other three sites. The speaker rotated among the four sites. For the multisite conferences, handouts with lecture objectives, topic outlines, and references were provided one week beforehand.

Evaluation

We anticipated that a program could be valuable only if the technical quality of the transmission was good. For this reason, we asked the participants to evaluate two domains of the conference: the technical quality and the educational quality. The participants rated the technical quality by three items—”image clarity,” “voice/audio,” and “interactive video”—on a 1 (lowest) to 5 (highest) scale. The participants evaluated the educational quality of the conference on a scale from 1 (strongly disagree) to 5 (strongly agree) for each of the following statements: “This was a valuable learning experience;” “The topic was interesting to me;” “This was better than watching a video of the same topic;” “I would prefer seeing the lecture in person;” and “The lecturer had difficulty teaching this way.” Participants could include general comments at the bottom of the form. As participants arrived in the conference room, evaluation forms were distributed to each individual. At the end of each Hershey Medical School conference, the connection was terminated and evaluation forms were completed and collected.

Data Analysis

Data were entered into SPSS for Windows (Version 7.5). All summary statistics were calculated using this software package. We excluded from our analysis all items that had values missing. We wanted to have one overall technical-quality measure for each seminar, so we created a composite score. First, the responses on participant evaluation forms were averaged. Next, the mean scores from each participant were averaged for every seminar. The comments provided by participants at the bottom of the evaluation form were aggregated using the content-analysis approach described by Stange et al. (1994).

Results

Quantitative Analysis

Our residents completed a total of 382 evaluations of the thirty videoconferences: twenty-five lectures from the Hershey Medical School and five multisite seminars. Table 1 shows an analysis of each survey item. To assess the validity of the responses, we worded some items in the negative and some in the positive. Cronbach’s alpha for all eight items of the survey was .74. When we excluded the item “interactive video,” Cronbach’s alpha was .69. Cronbach’s alpha indicates that the eight items were internally consistent or that each item consistently measured the same information.

Table 1. Ratings by 382 Participants

*Total number of participants was 382. Percentages shown above are based on completed items.
**Lower than 100% because of rounding.

Session scores varied widely by seminar. Mean session scores started high, dropped midway in the conference series, and later increased. Mean session scores were slightly higher for the multisite seminars (3.88) than for the single-site conferences (3.70).

There was an increase in attendance at the seminars over time. The multisite conferences had nearly twice as many participants (21.2 vs. 10.9) as the Hershey medical school lectures.

Highest-Rated and Lowest-Rated Seminars

We reviewed the seminars rated in the top 10% and the bottom 10% to explore the determinants of success or failure in our conferences. Results from the three highest-rated seminars (labeled H1, H2, and H3) and the lowest-rated seminars (L1, L2, and L3) are shown in Table 2.

Table 2. Mean Ratings by Participants on Best and Worst Conferences

Seminars H1 and H2 had good ratings for audio, video, and interactive video (>4.3). All three high-rated seminars had positive feedback for interest and value (>4.5). Seminar H3 had a lower video quality, but the rest of the ratings were excellent. Seminar H3 was the first session held, and its high rating might be attributed to first-time enthusiasm and the fact that there were only four people in attendance.

In seminar L1, there was poor audio for most of the session, and in seminar L2 there was poor audio for at least one quarter of the seminar. The rest of the scores for these seminars reflected the lack of audio transmission. In seminar L3, there was average-quality audio and video (3.25), low interactivity (2.6), marginal interest (3.6) and value (3.5), and a strong preference for seeing the lecture in person, implying that this seminar was not suited to the videoconference format (see Qualitative Analysis for more).

Qualitative Analysis

We reviewed all comments written on the evaluation forms and clustered the responses into five categories: audio problems, video problems, lecture-preparation or presentation problems, general positive comments (e.g., “good lecture,” “good learning experience”), and general negative comments (e.g., “poor conference,” “speaker fair”). The category with the most comments was general positives, followed by lecture preparation/presentation problems, audio problems, general negatives, and video problems.

We examined the comments for the top and bottom 10% of the conferences. The top three conferences had positives such as “good lecture,” “good topic,” and “great educational tool.” Only one participant commented that the transmitted picture was dark. The participants did not comment about any audio problems.

The worst three conferences had significant audio problems: no sound at all in one seminar, poor sound in another conference, and sound that faded in and out in the third. In one conference, there were also problems with the quality of the handout, slides that were unreadable because of too many words per line and lines per page, and monotonous speakers.

In the best learning experiences, the speaker (1) made a handout available at our site before the conference, (2) attempted to include our learners in the seminar by asking us questions, and (3) had clearly written slides and had organized the talk.

During the question-and-answer phase of each seminar, the learners asked occasional questions at the distant site, but we could not hear the question. The speaker needed to repeat the learners’ questions so that our participants could follow along.

Discussion

We found that the thirty interactive videoconferences were generally well received, with notable exceptions: when the audio or video faltered; or when presenters did not provide lecture handouts in advance, did not create legible slides, or did not reiterate the distant audience’s questions. Our participants were neutral about seeing the lecture in person and believed the lecturer adapted reasonably well to conducting the educational session by interactive videoconference. Our residents responded very positively to multisite conferences in which the lecturer gave them questions in advance and asked for their input during the seminar. After audio-visual technical problems were eliminated, seminars were judged most positively when the speaker was organized, had prepared readable slides, repeated audience questions, and included our residents in the session.

Potential Use in a Family Practice Residency

In this study, we found that interactive videoconferences were technically feasible. We were able to make fairly reliable connections, and the conference quality was acceptable after the academic medical center upgraded its equipment. The convergences that experienced audio and/or video failure offered little or no value to the participants. When the audio and video were satisfactory, the perceived conference quality depended on the skills and preparation of the lecturer and on the learning needs of our participants.

Our study supports the use of videoconference education as a supplemental teaching/learning resource for family practice residency education. The use of this technology allows the residents to participate in conferences with medical schools (or, potentially, other residencies) that otherwise would not be easily accessible. Topics that can be shared among residencies by videoconference include subspecialty pediatrics, research, critical appraisal, law and medicine, practice management, genetics, sports medicine, ethics, and adolescent medicine. Lectures, tutorials, and seminars could also be broadcast from a community residency to one or more medical schools. The use of videoconferencing may provide cost- and time-saving opportunities to deliver current and relevant medical education to family practice residents—particularly since on-site instruction by a physician with expertise in a subspecialty medical field may not be feasible.

Cost Considerations

Since the initiation of our videoconferencing program in 1997, costs of the necessary equipment have decreased dramatically. Today, a basic videoconferencing unit with ISDN capabilities costs about $12,000-$15,000 per unit, compared to $80,000 in 1997. Interactive videoconferencing units require high-speed transfer of information, generally across phone lines or ISDN lines to support a high-quality image and sound. Cost sharing for ISDN lines among hospital departments can reduce costs to any single department. After initial installation and troubleshooting, most problems are usually with the phone lines or connections into the hospital.

Best Practices for the Prospective Videoconferencing Lecturer

Based on the results of this study and the experience of others’ research (e.g., Lewis et al. 1998), we believe that videoconference lecturers should follow specific “best practices.” Two to three days before the conference, the lecturer should provide the receiving site with a handout that covers the main points and key illustrations. The connection should be made and adjustments performed ten to fifteen minutes before the conference. The speaker should test all equipment—including overhead projector, slides, and videos—before the conference. The speaker should coordinate his/her movements with the camera technician prior to the conference for the best possible view. The speaker must stay near the transmitting microphone, and should repeat all questions asked by the audience at his/her site. The speaker should review the photographic/computer-generated slides, and restrict each slide to six words per sentence and six sentences per page. Color balance should also be performed periodically between sites so that slides are transmitted clearly. The presenter should attempt to involve the audience in the presentation by asking questions and using other approaches.

Conclusions

Over the two-year study period, medical videoconferences were technically feasible and acceptable to most participants in our family practice residency program. Initially, the most frequent technical problems were with audio and video transmission, but these improved as the technology of the equipment improved. Other concerns included poor speaker preparation and poor slide quality. When speakers prepared well and involved our residents as active learners in the videoconference, the conference was successful. Videoconference education appears to hold promise for residency education, especially at rural sites where access to content experts in subspecialty medical fields is limited. Prospective videoconference educators will have a successful videoconference when they follow best practices outlined in this article.

Note

This study was supported in part by a grant from the South-central Area Health Education Center of Pennsylvania and by a grant from the Center of Excellence for Remote and Medically Underserved Areas, Loretto, PA.

References

Blakeslee, D. B., W. J. Grist, M. E. Stachura, and B. S. Blakeslee. 1998. Practice of otolaryngology via telemedicine. Laryngoscope (108): 1–7.

Franken, E. A., C. E. Driscoll, K. S. Berbaum. 1989. Teleradiology for a family practice center. JAMA (261) 20: 3014–5.

Gray, W. P., J. Somers, and T. F. Buckley. 1998. Report of a national neurosurgical emergency teleconsulting system. Neurosurgery (42) 1: 103–8.

Hayes W., W. G. Tohme, D. Komo, et al. 1998. A telemedicine consultative service for the evaluation of patients with urolithiasis. Urology (51) 1: 39–43.

Lewis, Y. L., R. P. Bredfeldt, S. W. Strode, and K. W. D’Arezzo. 1998. Changes in residents’ attitudes and achievement after distance learning via two-way interactive video. Family Medicine 30 (7): 497–500.

Mills, O. F., M. Tatarko, J. F. Bates, T. A. Hunsberger, E. Everhart-Yost, and V. Pendleton. 1999. Telemedicine precepting in a family practice center. Family Medicine 31 (4): 239–43.

Perednia, D. A. 2000. Telemedicine in dermatology: 21st century update. Progress in Dermatology (34) 1: 1–10.

Reid, D. S., L. E. Weaver, J. M. Sargeant, M. J. M. Allen, W. F. Mason, P. J. Klotz, and D. B. Langille. 1998. Telemedicine in Nova Scotia: Report of a pilot study. Telemedicine Journal (4) 3: 249–58.

Stange, K. C., W. L. Miller, B. F. Crabtree, P. J. O’Connor, and S. J. Zyzanski. 1994. Multimethod research: approaches for integrating qualitative and quantitative methods. Journal General Internal Medicine 9 (5): 278–82.

Tachakra, S., A. Sivakumar, J. Hayes, and M. Dawood. 1997. A protocol for telemedical consultation. Journal of Telemedicine and Telecare (3) 3: 163–8.

Zarate, C. A., L. Weinstock, P. Cukor, and C. Morabito. 1997. Applicability of telemedicine for assessing patients with schizophrenia: acceptance and reliability. Journal of Clinical Psychiatry (58) 1: 22–5.

Zollo, S. A., M. G. Kienzle, Z. Henshaw, L. G. Crist, and D. S. Wakefield. 1999. Tele-education in a telemedicine environment: Implications for rural health care and academic medical centers. Journal of Medical Systems 23 (2): 107–22.

Author Information

Orlando F. Mills, MD, MPH
Conemaugh Memorial Medical Center
1086 Franklin Street
Johnstown, PA 15905
Phone: 814-534-9364
Fax: 814-534-3136
E-mail: omills@conemaugh.org

James F. Bates, Ph.D., Director
Center of Excellence for Remote and Medically Under-Served Areas
Saint Francis University
Loretto, PA 15940

Vicki Pendleton, RN
Telemedicine Development Specialist
Center of Excellence for Remote and Medically Under-Served Areas
Saint Francis University
Loretto, PA 15940
Phone: 814-472-3146
E-mail: vpendleton@cermusa.org

Kathleen Lese, MA, MLIS, Director
South-central Area Health Education Center of PA
PO Box 509
Carrolltown, PA

Michael Tatarko, MD
Conemaugh Memorial Medical Center
1086 Franklin Street
Johnstown, PA 15905