Seeing what Others Hear.
by Lynn Thomasson
It’s hard enough for students to keep pace with a professor’s lecture and still be able to scribble down notes. But what if you’re a deaf student? Forget taking notes; you’ve got to keep your eyes on the interpreter.
This was the challenge for Alex McLin, a hearing-impaired computer science major at Carolina. “Often, hearing-impaired students have to look away from the interpreter to look at the board or what the lecturer is doing. They end up missing information because their eyes aren’t on the interpreter,” McLin says. So he suggested a twist on Facetop technology, a software system that allows users who are in different locations to work together on the same document or graphic. The users can also see and hear each other through live video. (See Endeavors, Spring 2004, “Seeing Eye to Eye.”)
To solve his problem, McLin wanted to point the Facetop camera at his interpreter, and then take notes on a Tablet PC. Facetop developer Dave Stotts, along with Gary Bishop, got to work making changes to the Facetop system. McLin spent a summer tweaking the original Facetop, then took a version of the system to class and jotted down his notes. Computer science researchers at Carolina continued to work on the code and finished the project after McLin graduated.
“It’s Facetop technology turned around so that you can look at your paper on the Tablet PC and your signer at the same time,” Bishop says. “You can draw a little mustache on your signer if you want to.”
With Facetop, a hearing-impaired student can point a camera at his interpreter and send the live video image to his Tablet PC. This allows the student to watch his interpreter and take notes at the same time. Photo: Lynn Thomasson.
Newer generations of Windows and better Tablet PCs allow Facetop video to run smoothly on a bigger screen. If you’re watching people move their hands in sign language, fluidity of motion is key.
“Here is a student who realized that this is a technology that could address a real problem — not just a problem for him but a problem for all hearing-impaired students,” Stotts says.
Facetop researchers also added a recording capability that synchronizes notes written on the Tablet PC with the video and audio of the class. That allows students to pick out the important notes. They can also see and hear just those moments of the lecture — a feature that could make the technology valuable for all students.
Stotts and Bishop are working with Disability Services at Carolina to make Facetop useful for more students. They also plan to team up with researchers and more hearing-impaired students at East Carolina University to test the technology.
Researchers at UNC-Chapel Hill and ECU are brimming with ideas about how to bring Facetop to more people. With the cell phone fast turning into an all-in-one device, Stotts says, someday you might see people on the street communicating in sign language over their phones.
Dave Stotts, associate professor of computer science, and Gary Bishop, professor of computer science, developed this new Facetop technology with graduate students Kyle Gyllstrom, James Culp, and Dorian Miller. Stotts and Jason Smith, a former Carolina graduate student, co-invented the original Facetop.
Lynn Thomasson is a Carolina undergraduate majoring in journalism.
