11.8. Creating Streamed Instruction for the Deaf and Hard-of-Hearing Online Learner

Editorial

Ordinarily I don't show much interest in papers that make reference to computer programs and platforms, as I find this severely limits their “shelf-life.” I believe in this case, however, the value of the information contained in this paper outweighs that consideration. The software packages mentioned are all available from their companies’ respective Web sites, so are neither expensive nor difficult to find, nor is high-end equipment required. When I thought about it, my 82-year-old father owns all the technology mentioned below—except, of course, the video streaming servers!

You wouldn't think that streamed audio and video would be a good instructional choice for deaf and hard-of-hearing students, but James Mallory makes a cogent case for their use. I watched him in a studio delicately balancing all the various instructional elements as he videotaped Visual Basic (VB) programming instruction, talking and signing with choreographed precision.

The most compelling message of this paper is “It ain't that hard, folks!” Technology has gotten to the point where last year's $1,500 iMac sits (sadly underutilized) on my own desk at work. It came equipped with a high-speed transfer cable (FireWire), enough processing speed (500 megahertz), and disk storage space (30 gigabytes)—and free iMovie2 software to allow ME to edit digital video! And no, I'm not Ms. SuperTech—my specialty is unadorned, text-based computer conferencing!

Mauri Collins
DEOSNEWS Editor

Creating Streamed Instruction for the Deaf and Hard-of-Hearing Online Learner

James R. Mallory

Abstract

Two technology paradigm shifts are happening that will radically change the way instruction is delivered to the remote deaf and hard-of-hearing learner. First, due to the increase in digital storage technology and high-speed connections that are available for the remote deaf audience, video streaming and digital video devices (DVDs) will become the delivery method of choice in the near future. Secondly, digital camcorders, screen capture programs, teachable voice recognition software, and other tools allow homemade multimedia presentations to be easily and economically produced. These high-tech tools, in addition to proven instructional delivery and pedagogical concepts, will drastically change the distance learning world for deaf and hard-of-hearing learners.

Introduction

Videotaped instruction has been an effective delivery tool for deaf, online learners for many years, but it is quickly being replaced. Students want to access their instruction anytime, anywhere, and don’t want to have to carry videotapes around with them to home and work. CD-ROMs, although popular for many other uses, are not as feasible for multimedia instruction for the deaf as DVDs; CDs do not store as much information and they are not as simple to use as logging in to a Web site that has multimedia examples and video streamed modules. The growing remote deaf and hard-of-hearing population also want to log in to their instructional information “anyplace, anytime” without having to carry anything with them, even if it is a CD-ROM or DVD.

Some technical challenges still exist, which is why adoption of streaming technology has not been as fast as it will be during the next few years. These challenges include:

• the learners' Internet connection speed;
  
• the size of streamed video files vs. the clarity of the video which is to be streamed;
  
• the readability of sign language if the video also includes a white board or projection from a computer monitor;
  
• the cost and complexity of producing, digitizing, compressing, and posting streamed video;
  
• the size constraints of the server where digitized streaming video files will be stored;
  
• captioning requirements for streamed video; and
  
• the need to interface with other software (such as Flash) to build readable instructional modules.
  

The National Technical Institute for the Deaf (NTID) has successfully piloted different delivery methods for its distance learning courses, including multimedia instruction without video and video streamed modules in some of its online Visual Basic (VB) programming courses designed for remote deaf learners. This article shares some of our experiences in developing this instruction.

Instructor Development Tools without Streaming Video

Although most of this article deals with the production of streaming video, simpler but useful multimedia modules can be developed right on the instructor’s desktop without ever having to use streamed video. This type of multimedia instruction avoids many of the technical issues and costs mentioned below and allows the instructor who must operate on a low budget to develop useful multimedia instructional material.

If computer programming or similar computer-dependent concepts are being taught, executable programs are an excellent tool in explaining concepts to a remote audience. Some other useful tools I have used successfully are screen capture programs such as TechSmith's Camtasia for multimedia capturing of the monitor’s screen events or their Snagit program for single-screen capture in conjunction with voice captioning software such as Dragon Naturally Speaking. This type of multimedia instruction can be produced without having to get into the video and streaming arena. More sophisticated, but more complex, are Flash presentations with captioning. When the teacher becomes efficient with these types of content-authoring tools, then great multimedia examples can be developed.

Evolution from Video Tapes to Multimedia and Streaming Video

“In the mid 1960s a group of pioneering teacher educators at Stanford University in Palo Alto, California began an experimental use of a new technology, video tape recording” (McCurry 1999). Since this time videotaped instruction with sign language, captioning, graphics, and animations has proved to be a viable option in the delivery of remote courses and as a supplement to traditional courses for deaf and hard-of-hearing students.

The National Technical Institute for the Deaf (NTID) has successfully developed and offered a number of both technical and nontechnical courses for both on-campus and remote delivery using videotapes as a primary delivery of instruction. When combined with online group conferencing and delivery software, video taping still works effectively for this type of instruction.

Remote students are starting to demand high-tech alternatives to online instruction. One negative aspect of using videotapes is the requirement of physically keeping the videotapes with the student for accessing lectures and having a VCR to view them. If a remote student wants to access a lecture and has some time available at work or has traveling commitments, he/she must keep the videotaped lesson in hand in order to view the lesson. This situation is not much different from carrying a CD-ROM or DVD, even though the technical medium and delivery vehicle is more high-tech. Most people do not have VCR or DVD players available at their workplace, but they often have high-speed Internet access. Students want to be able to view their instruction “anytime, anyplace.” Another negative associated with videotaped instruction is the inability to index a lesson. The student must fast-forward or reverse his/her VCR to find a particular instructional unit. The alternative to videotapes is using a CD-ROM or DVD type of instruction that would eliminate this problem. CD-ROMs will eventually be replaced with DVDs for this purpose. The instructor in real time also cannot modify media that are carried by the students if he/she wishes to modify the lesson. The instructor could make changes, however, if the media were streamed from the Web. Another challenge is instructional delivery to an international audience. Videotape players in the VHS format are not consistent, even throughout the English-speaking countries. An alternative to these instructional formats is to use streaming video or CD-ROMs.

Video streaming is just becoming a viable option for supplementing instruction for deaf and hard-of-hearing students. More user-friendly editing software, digital camcorders, higher broadband connection speeds for students at home and work, the cost-effectiveness of larger servers, and the changing demographics of our student audience are all contributing to this trend.

Deaf and Hard-of-Hearing Student Audience

There is a remote adult population that needs to be trained or retrained and may not normally be inclined to take courses at NTID, such as geographically remote deaf adults with full-time jobs and families and single working mothers. A survey of approximately 2,000 online students at the Rochester Institute of Technology (RIT), of whom 50 were deaf, gathered a 10% response rate and found that 70% of these students had broadband Internet access at home, work, or school. This shift in the access speed of the student population makes video streaming a more viable option.

Universities in general and, more specifically, educators of the deaf have been slow to adopt the use of video streaming technology due to a variety of concerns. Morensen, Schlieve, and Young (2000) experimented on a limited basis using some streaming media with their hearing students in the Department of Technology at the University of North Texas. One of their comments was, “One interesting question would be to determine the minimum technical expertise of a student enrolled in a streaming media type of course. A student must be able to access the Internet, download the streaming media player, install the player, and perhaps configure his or her machine for optimal performance” (Morensen, Schlieve, and Young 2000).

This learning curve for the installation and use of streaming media players is not expected to be a problem with the NTID computer programming students who will be taking these courses, as it is assumed that these students have needed skills to use this streaming technology. This could be a challenge early on for some faculty and students in nontechnical majors, but the technology will become more user-friendly as it evolves.

Companies and organizations are increasing the use of streaming media as a communication vehicle and offer closed captioning to deaf and hard-of-hearing individuals. “The market for streaming media applications is growing rapidly to now include Web-hosted events, investor relations and digital broadcasts of seminars, conferences and entertainment. The need to caption these proceedings and make them available to everyone is critical” (Solomon 2001).

Non-Deaf Audiences

I have found that instruction developed for the deaf and hard-of-hearing for whom English is a second language is also ideal for foreign students. These populations have similar pedagogical needs.

Persons with nonimpaired hearing whose primary language is English also enjoy this type of instruction. As an example, NTID’s distance learning programming courses were so simplistic and user-friendly that corporations such as Pittsburgh Telephone and PaeTec have requested that the programs be used to train their nontechnical personnel in computer programming languages.

Terminology and Distance Learning at RIT/NTID

In a literature search, the terms “distance learning,” “online learning,” “e-learning,” “anytime, anyplace learning,” and “remote learning” are used interchangeably. There is not yet a standard labeling for this type of learning. The term “online learning” or “distance learning” is used in this article to describe courses that are offered to both on-campus and remote deaf students. NTID delivers four computer programming courses in an online format. (To find out about RIT’s distance learning program, visit <http://www.distancelearning.rit.edu/>. For more information about the four-course certificate for the deaf, visit <http://www.distancelearning.rit.edu/inquire/programs_courses/undergrad_cert/programming.cfm>.)

Two of these courses teach C++ programming. Two teach Visual Basic programming and the lectures are being delivered via video streaming. In the fall 2001, the streamed video is being used for the first time as a primary instructional delivery. In the past, videotapes with graphics and animations were used to deliver the lectures.

Video Streaming Definition

Video streaming can be considered to be a progressive downloading of a video file. A storage space (buffer)—much smaller than the whole video file—is assigned to the file on the computer's hard drive. The video file begins to download into that buffer. In a short time, as soon as the buffer is full, the file begins to display on the computer's screen. The video file continues to stream down from the server, into the buffer and out onto the screen. The video stored on the server can be either live or prerecorded and stored on digital media.

The ultimate goal of video streaming is to develop a clear multi-image, synchronized instructional presentation. There are actually separate “streams” that exist in a streaming video that need to be synchronized—i.e., the audio must be synchronized with the video, the computer display, and the captioning and the graphics. Having separate streams is a tremendous advantage when trying to deliver instruction over the Web, using limited bandwidth. One stream could be the video that includes the instructor signing, one stream could be the audio, another stream could be the captioning, and another could be the computer monitor projecting the programming code or object that the instructor is explaining.

Types of Video Streamed Files

The three most popular video file types are Apple Quicktime, Microsoft Windows Media Player, and RealNetworks RealPlayer. The advantages and disadvantages of each will be discussed below to indicate why RealPlayer was selected for the project described in this article. “Synchronized Accessible Media Interchange (SAMI) and Synchronized Multimedia Integration Language (SMIL) are used in conjunction with Media Player and RealPlayer, respectively, for coordinating multiple clips of the streamed video which is also crucial for captioning on this type of project.

Although Quicktime, a “.mov” type of file, is very popular for standard movie downloads, it is not as proven as Media Player or RealPlayer for streaming video and audio. Quicktime only recently entered the streaming market. Its performance on an IBM-compatible personal computer (PC) depends greatly on the speed of the computer processor and the compression settings of the movie. Certain kinds of QuickTime Scripting work only on an Apple MacIntosh (Mac) platform and not on a PC. Quicktime is, however, easy to embed into HTML documents.

Media Player, an “.avi” type of file, works well in the PC environment but not on a Mac platform. It works only with the Internet Explorer browser. It is easy to embed into HTML code. According to Microsoft, SAMI “is simply a format optimized for authoring captions (CC) and audio descriptions (AD) in a single document. SAMI is based on HTML to provide a familiar, user readable format.” SAMI coordinates the different video streams for this type of media player. SAMI functions for windows technology as SMIL does for RealPlayer.

RealPlayer, an “.rm” type of file, became the streaming software of choice for this project. RealPlayer works on both Macs and PCs. It has a proven track record in the streaming video arena. It is complicated to embed into HTML. The technical guide states that “To combine the two, an image needs to be embedded into the HTML document which is a link to the Real Video, which opens into a floating window and blocks the main page.” When a RealPlayer streaming presentation contains multiple clips, such as sign language videos, computer screen images, and captioning, it needs to use SMIL to coordinate the parts. SMIL is a simple but powerful markup language for specifying how and when clips play. An SMIL file is not required to stream just one clip. When multiple clips exist, such as in this project, SMIL's markup language specifies how and when the clips will play. After writing the SMIL file, the video is then ready to put on the RealServer and link to the Web page.

Creating the Video

The primary equipment needed to produce a streamed video are a video camcorder, a means of exporting the movie into the computer, a computer for delivery of the instruction (if it is computer-based instruction), and a server or device to store the video on so it can be accessed from the Web. Capturing a video and digitizing it is relatively easy. Converting the video-to-video stream into RealPlayer format is the most difficult part of the project.

Recording a video is most easily done with a digital camcorder and a FireWire connection to the computer. This allows the user to directly and quickly put the movie into the computer in a digital format. This movie can then be edited using a product such as Adobe Premier to produce a high-quality video.

An analog camcorder could be used with standard audio/visual (AV) ports and a video capture card on a computer. This method requires more steps because the movie has to be converted from analog to digital by going through the AV port and back. This method would work, but the video quality would be degraded.

Resource Issues

The teacher of the video lessons needs to have adequate equipment to properly deliver the instruction. I used a Windows-based Dell Inspiron 8000 laptop, with 256K RAM and an 800 MHz CPU. Performance was barely adequate when multitasking several programs such as the Hilighter illustrator program, a screen capture program, and the programming code compiler.

Streaming the Video

Streaming is the most difficult part of this process. The goal is to code the streamed video onto a Web page. The developer needs to know the name of the server and where to put the video file. Streaming software needs to be used at this point. RealPlayer worked fine for this project. The basic version is free, but it will allow a maximum of only twenty-five users. RealPlayer can be downloaded from <http://www.real.com>.

Fundamental steps used to develop and post a streaming video are to do the following:

1. Simultaneously do the following:

a. Use a camcorder to record the instructional video(s), the instructor signing and explanations.

b. Use a pointing or editing software package to emphasize key concepts or to write on the screen, such as Altiris Hilighter. (We did not have good luck with smart whiteboards due to the back and forth movement required by the instructor and the shadowing effect of front projection. Rear projection white boards would work better, but they are cost prohibitive.)

c. Use a screen capture software package (such as Snagit or Camtasia) to capture the computer monitor events. (We tried but did not have good luck capturing the monitor via a camera, even with a high-quality, production-ready video camera.)

d. Capture the audio input along with the video. This separate file will be transcribed and captioned later.

2. Transfer the video(s) and audio(s) from the camcorder to the local PC via FireWire or other means.

3. Edit the video(s) on the local PC by doing the following:

a. Separate the audio and video streams.

b. Create a caption text file manually using a text editor or use voice recognition software (such as Dragon Naturally Speaking or IBM's ViaVoice).

c. Create all other streams necessary (e.g., instructor's signing and explanations, graphics, computer monitors).

d. Create an SMIL file to synchronize all streams and create the final layout.

4. Set up the Web server—this serves as the Web page host. (This should be performed by an individual with computer expertise.)

5. Set up the RealPlayer server. (This should be performed by an individual with computer expertise.) This is an application that handles the streaming and synchronization of the audio/video/captioning/monitor image into one package.

6. Transfer the video from the local PC to the RealPlayer server.

7. Create a Web page on the Web server pointing to the final SMIL file on the RealPlayer server.

8. Test all files to ensure that they are downloadable and receivable by the student audience.

Size vs. Clarity of Streaming Video Files

To make the video large enough to read once it is streamed and converted to RealPlayer, the monitor in which the screen was captured had to be set to 640 X 480 pixels. This presented some challenges when creating and displaying graphics or demonstrating something such as Visual Basic, where screen space is precious because the control object tool box, the property tables, the tool bars, and the actual forms must all be displayed. The clearer the movie file is to view, the larger its file size must be.

There is a trade-off between a file size that is adequate to understand sign language and the instruction when it is streamed to the user's desktop and a practical file size to store and stream over a broadband connection. For video alone, fifteen frames per second throughput is normally fine for adequate viewing on the Web. When combining multiple streams, as previously mentioned, however, throughput of thirty frames per second is recommended for the greatest clarity. The signal will tend to degrade after the various streams are combined and posted on the Web. Although there is no limit to how large a file can be in order to be stored and streamed from the Web, I found it practical for editing to restrict the video to seven minutes or less.

Streaming Video Software

There are various PC video editing software programs available, Adobe Premiere runs on either platform and was chosen for this project. Another Mac-based video editing software, iMovie, is very popular.

Streaming Video Servers

A large-capacity server is required for video stream hosting. A good “rule of thumb” is that video typically consumes one megabyte per minute for an average video. The server described in this article used Redundant Array of Independent (or Inexpensive) Disks (RAID) to build in redundancy. RAID collectively acts as a single storage system, which can tolerate the failure of a drive without losing data and which can operate independently.

Although the video is stored on the server, CD-ROMs and DVDs are also used for two purposes. First, the addition of CDs and DVDs gives more options to students who do not have high-speed Internet access. These students can now view the clips while away from faster network connections. A DVD can store up to 1.6 gigabytes (about two hours run time) of video, while a CD can store up to 650 megabytes (about one hour run time) of video. Future technology will increase this capability multifold.

Secondly, having multiple DVDs and CDs incorporated into the server will allow all the video streams to be sorted by particular class and session time/date. By providing multiple media access on the server itself, one will find that seek times for specific files will be much more responsive. For instance, if there are multiple students simultaneously accessing the same files on the server at the same location, the server will be able to address only one request at a time. If these same students are making the same request to multiple CDs or DVDs, the response time will be increased by the dedicated access time each student will be receiving. The goal is to provide the most flexibility allowable to students at the fastest possible throughput, regardless of location—anywhere and anytime.

Technical Expertise for Creating Streaming Video

Beyond the learning curve, minimal technical expertise in using the editing software is required when producing the video. Captioning can be done using a common text editor. The difficult part is actually converting the video to the streamed format and posting this on a Web server. A computer expert would need to help with this part of the process.

Other Instructional Issues for Deaf and Hard-of-Hearing Video Production

Other issues to be considered when developing video for deaf and hard-of-hearing include the following:

• Wear a dark shirt without logos.
  
• Pause when showing monitor images.
  
• Sign clearly and deliberately.
  
• Capture a large enough video image so that the instructor signing does not go off the screen.
  
• Clearly label each video with a number and limit the file names to eight characters or less with no spaces to eliminate potential problems when media is stored on Unix-based servers.
  
• Don’t try to lecture multiple lessons without taking a break in between. (I found no more than three or four video lessons should be developed without a significant break; a short break after every two lessons boosts the quality of the instruction.)
  

Final Remarks

There are a variety of instructional elements an instructor can now develop to make his/her distance learning instruction palatable for deaf and hard-of-hearing online learners. Some of these are easier and more cost-effective to implement than others. Multimedia computer monitor demonstrations can easily and economically be developed and captioned using a standard computer and microphone along with software tools readily available today.

Production of streaming video, however, takes a higher level of technical knowledge and resources. Streaming video with captioning is just becoming a viable instructional tool for the deaf and hard-of-hearing audience. Capturing and streaming a video with a person using voice, sign language, and captioning is not difficult.

When also incorporating instructional media such as computer monitors, white boards, text, and graphics, the streaming process becomes incredibly challenging and complex due to a variety of issues previously discussed. NTID has had some early success with this process and will continue to forge ahead with this new technology for the benefit of deaf and hard-of-hearing individuals across the globe.

Acknowledgments

The computer server and the newly purchased video streaming equipment and software used on this project were made possible by a $52,500 New York State Vocational and Technical Education Association (VATEA) grant received in the spring of 2001. This effort is a continuation of work described in a paper delivered to the Instructional Technology and Education of the Deaf Supporting Learners, available at <http://www.rit.edu/~techsym/>.

I thank Mauri Collins for her editing help, for sharing her publishing expertise, and for her encouragement. I thank Steve Campbell and Richard Rizzo, NTID Department of Technical Services & Operations, for sharing their expertise and for the technical help setting up the server, debugging and the piloting and streaming of the video. I thank Josie Kurz for her help as a student worker in converting the video images. I also thank David Conyer, Robert Brewer, and the rest of the Instructional Television production support crew at NTID for their many hours in the research, development, and production of the video portions mentioned in this article.

References

McCurry, D. S. 1999. Technology for critical pedagogy: Beyond self-reflection with video. Proceedings of the Society for Information Technology in Teacher Education. San Antonio, TX.

Morensen, M.. P. L. Schlieve, and J. I. Young. 2000. Delivering instruction via streaming media. Tech Trends 44 (2).

Solomon, P. 2001. Word Wave and e-Media Partner to Deliver First Live Web Cast With Closed Captioning. USA-L_News@yahoogroups.com business wire.

Author Information

James R. Mallory
Department of Applied Computer Technology
National Technical Institute for the Deaf
Rochester Institute of Technology
Rochester, NY 14623-5604

Phone: 716-475-2865
Fax: 716-475-7101
E-mail: jrmnet@rit.edu