Flexiflyer with wheels (used with permission)

Long-promised video here!

Here’s an historical post. The key part is the video below which was created in 1972.

The video has been promised to accompany my chapter in the Festschrift for Ursula Bellugi and Ed Klima published in 2000. In that chapter I analyzed Lou Fant’s live interpretation of a spoken reminiscence.  I was the (English) speaker; Lou used American Sign Language (ASL). My story told about trying to ride a Flexiflyer down a steep U-shaped driveway and back up.

Spoiler alert:  I crashed.

Key Findings:  Fant managed to get the description of the relationship of the house, the driveway and the sled’s riders correct, if mirror-image of the actual space, without having seen the place, without any gestures from me (the speaker), and with the English input message being pretty sketchy. And the cool part? He continues still interpreting for at least a full minute, before he commits himself to those spatial relationships.

It takes a village…

Support from the National Institutes of Health and National Science Foundation over the past 40 years fueled the research of Klima and Bellugi, and many of their students and colleagues. I was privileged to be a member of the laboratory research staff from Fall 1970 through Spring 1973, and an irregular visitor thereafter. This video was created in 1972 at the Salk Institute. It was preserved during the early 1980s (on VHS cassette). It was digitized in 2000 with help from Stanford’s Academic Technology Laboratory (a support facility for faculty), and from Treehouse Video. I’m thrilled to report that the .mov (Quicktime format) video still plays, and has now been uploaded to YouTube, and is presented for your viewing pleasure, with gratitude to all those who helped along the way.

Pack rat unveiled

I saved that bit of video from Fant’s first (or perhaps second) visit to Salk Institute on one of my irregular visits back to La Jolla. When copying from helical scan 1/2″ videotape to VHS cassette, I remembered that I had consciously chosen to tell about an event that no one in the room had heard before, one from my childhood, so that it would be a genuine listening and viewing experience for both the hearing and deaf people present (not a retelling of a familiar story). Of course the interpreter hadn’t heard this story before, and he didn’t have much context about me either. I thought I had been quite clear about the physical space – I could picture it even many years after the event – how the house was situated, where the driveway started, turned, and ended at the street again, and what it was like to ride the wheeled sled. On listening again, I realize that the physical space is difficult to imagine if you were depending on the spoken message only.

Interpreter stays vague for a full minute

Our visitor, the exemplary interpreter and sign language educator, Lou Fant agreed to contrast “transliteration” and “interpreting.” I’ll offer a brief definition of these terms, knowing full well that other experts out there can elaborate in greater depth. Transliteration is a more English-influenced rendering into signs; interpreting is provides simultaneous translation into ASL, a different language, with English influence kept to a minimum. The excerpt shown here was the first part of the illustration of “interpreting.” The key surprise for me in reviewing the video was that Fant managed to keep the message vague as he worked out how all the different parts of the space described fit together. The ability to be vague had never been catalogued as a characteristic of the competent interpreter before. When I told him I was planning to look at this bit of video at long last and asked whether he’d like to see what I was finding and writing about him, he gave his blessing to my work without his review. I’m delighted to be able to present his spontaneous interpretation now almost 40 years after it was first produced.

And thanks to an interpreting instructor who uses the chapter from the Festschrift for asking where that video is.  Rachel, it’s here now.

Tuesday, March 9, we got the next update on YouTube’s automated captioning efforts. I heard it on NPR’s “All Things Considered” afternoon program, in which Robert Siegel interviewed Ken Harrenstien of Google with a (female) interpreter providing voice for the Google engineer.

Audio and transcript are available at http://www.npr.org/templates/story/story.php?storyId=124501330.

Harrenstien acknowledges that automated captioning today stumbles on proper names, including trademarks and product names:  ”YouTube” that comes out “You, too!” And automated captioning has difficulty with videos that have music or other sounds in the background. But, he characterizes himself as a technology-optimist, anticipating that in 10 years things will be much improved.

Benefits of captioning

Like “curb cuts” which have become the symbol indicating that solutions for disabled people (here, those in wheelchairs) resolve needs for others (strollers, roll-aboard luggage, shopping carts), captions have benefits that extend beyond hearing impairment.

• Deaf and hearing impaired people can enjoy the huge inventory of videos on YouTube. (The still frame that opens this post is from an announcement by President Obama in response to the Chilean earthquake. Making emergency and other time-sensitive news available to those who cannot hear meets the requirements of laws and regulations in the US. And more importantly, it meets the moral or ethical standards we expect from a civilized society where we include everyone in the polity.)
• If you’re in a noisy environment or located close to others who will be bothered by the audio, you can figure out what the video is saying even without benefit of headphones
• Small companies can afford to provide captions on their webcasts, often the heart of learning about new products
• Non-native speakers of English have a much better chance of understanding speech at ordinary (rapid) rates with the assist of captions
• Captions provide input to machine translation services, so that there soon will be captions in other languages besides English as well; as automated speech-to-text technology improves, we’re going to see other input languages as well
• Captions provide much better input to (current) search technology than speech does, so there’s hope of finding segments of videos that might not appear in written form

Professional captioners need not despair

I read the YouTube blog post of March 4 and the comments following it, and recalled the announcement of the limited trial with selected partners last November.  James expresses concern in his comment about the recent YouTube announcement that people, like him, who earn their living as captioners for post-production houses will lose their jobs as a result of the automated captioning.  My response seconds HowCheap’s comment that professional captioners will continue to find work both as editors of the automated speech-to-text and for organizations prefer doing their own captioning. Organizations that produce professional quality video typically start from a written script, adjust for the few changes that happen in the spoken version, and then set the timing of the text with the video.

The huge number of videos on YouTube are uploaded by individuals or by small organizations who may not be aware of the benefits from captioning, and likely don’t know about the tools available.  According to YouTube’s fact sheet: “Every minute 20 hours of video is uploaded to YouTube.” That’s a volume that is beyond the capacity of professional captioners and the organizations that employ them.

A proposal for improving the quality of captions

How shall we improve the quality of automatically produced captions?

I’d like to see interpreter training programs (ITPs) make editing automated captions a course assignment, a program requirement, or a component of an internship. Engagement with spoken language, not one’s own, is a challenge.  People phrase things in ways you don’t; they use unfamiliar vocabulary and proper names (streets, towns, people, products) that I need to look up.  Both ITPs for training sign language interpreters and those for people learning to interpret between 2 spoken languages may allow entry to students whose skills in listening, writing or spelling may be lacking.  How many caption-editing assignments are enough? Shall we also coordinate quality checks by others in the same or a different program?  Such assignments will guide students toward greater appreciation for the challenges of speech in online settings, with a task that provides an authentic service.

VRS and VRI

In the case of ITPs for sign language interpreters, the improved listening to online speech is great preparation for work settings such as VRS and VRI.  Video Relay Service (VRS) in the US is regulated by the FCC: deaf signers who cannot use the telephone (because their speech is not intelligible and they cannot hear well enough to understand speech over the phone) make use of intermediaries (interpreters) to communicate with hearing non-signers. (Think of simple tasks such as calling the school to notify them that your child will be absent; scheduling a haircut; ordering a pizza for delivery, not to mention more complex transactions involving prescriptions, real estate contract negotiation, billing disputes.)  Video Remote Interpreting (where the deaf and hearing parties are physically together, with the interpreter remote from them) is a service with similar requirements for the interpreter (listening to speech over a phone or data line, and rendering accurate translations in real time).

Broad multi-disciplinary open source content quality

Programs training instructors in English as a Second Language (ESL) could also participate.  Students in speech therapy and audiology would benefit from both the direct engagement with spoken language “in the wild” and with future colleagues in other disciplines. There are advantages to engaging a variety of people who are studying for professions that emphasize expertise in spoken and written English.

Looks like an open source content development effort to me. Yes, it will require a little bit of coordination, but not terrific overhead. How about it, ITP program directors?

“Bob’s House” marked a milestone for PepsiCo, for sign language and Deaf folks, and for the SuperBowl. This commercial aired as part of the pre-game activities in 2008. If you haven’t seen it, check it out by clicking on the image above.

Summary of the key points of the narrative

Two men are driving through a residential neighborhood at night, blaming one another for not knowing which house is Bob’s. The driver leans on the car’s horn, arousing the occupants of the darkened houses who turn on the interior lights, peek outside and even step onto a now lighted porch. The one house that remains dark must be the one belonging to their deaf friend. They park, ring the doorbell, and enter as their host Bob signs and mouths “sorry” to his neighbors. The passenger’s “We’ll miss the kick-off” cues the viewer that it’s Superbowl time; the driver drinking from a Pepsi bottle suffices to identify the sponsor.

Deaf culture

This commercial captures a traditional bit of Deaf folklore, in which creating noise annoys hearing people and simultaneously allows identification of the Deaf target of interest. Deaf (written with a capital D) carries the implication of cultural identification with the community that shares values, behaviors, and sign language. The usage follows the civil rights movement’s embrace of term “Black” from an ethnic slur to an identity of pride. (Little “d” deaf refers to the audiological condition of hearing impairment, where a deaf person cannot perceive the unaided auditory signal to understand speech. The term “hard of hearing” may refer to a person whose hearing impairment is less severe, as measured by a hearing test, but also has a sociological meaning, referring to the person with an audiological impairment who functions within a speech environment more easily, through hearing aids and speechreading.)

Beyond the narrative structure of the commercial, which reproduces a familiar sign language joke in a film treatment, there are additional messages in the commercial that indicate its genuine portrayal of Deaf culture.

• Deaf (and deaf) people can drive
  The National Fraternal Society of the Deaf (founded 1901, closed in 2004) provided life insurance to deaf people who couldn’t qualify with other companies. The Frat functioned as a social service and advocacy group as well.  It meets for the last time in early March 2010 to close the books. As with many charitable institutions which hope their mission will be accomplished, the Frat is closing in part because deaf people no longer suffer discrimination in insurance coverage, but also because there are other mechanisms and organizations which handle the advocacy portions of its mission.
• Turning on the dome light helps visual communication in the dark
  The driver turns on the light in the car to have the argument with the passenger about who was supposed to bring the street address.
• Doorbells become flashing lights
  Perhaps you caught the fact that the guests’ doorbell press triggered flashing lights inside Bob’s house, a visual signal of a typically auditory message. Harris Communications is one of several companies that supplies specialty products, including vibrating alarm clocks and flashing lights hooked to audio signals, for the specialized audience, “Deaf and Hard of Hearing people, people interested in sign language, Deaf Culture, hearing loss and more.” Hmmm, sounds like everyone, doesn’t it?

The companion video presents the employees who perform in the commercial, capturing their genuine surprise and delight about sharing the Deaf world with the Superbowl audience.

Congratulations to PepsiCo for responding to a Deaf employee’s suggestion, and for involving the National Association of the Deaf in turning the suggestion into reality. The performers in this video are not necessarily trained as actors, but the authenticity of their signing is obvious to Deaf and signing audiences.

Here’s the big question:  What products shall we make so that we don’t have to retrofit the environment?

Additional resources:

Carol Padden & Tom Humphries, “Deaf in America: Voices from a culture” explicates of the foundations of Deaf culture through stories, jokes, poetry and the perspective of a Deaf-centered world.

Photograph detail UPI © 1980

I met Ted Kennedy only once.

In an earlier part of my professional life, I worked as a sign language researcher and interpreter based in New York City.   (more…)