Skip Navigationtrace.wisc.edu HelpSearchBottom of Page

Modality Translation on the Grid

Real-Time, On-Demand Access to Information
Across Modalities

The concept of translation on the Grid addresses people's need for getting information in another form than is provided by default. People that benefit from translation services include:

Since the users' needs vary widely depending on the kind of "limitation" they encounter, there are a variety of translation services addressing these needs. Text-to-speech translation addresses people with visual impairments, but also facilitates eyes-free interaction for everybody. Speech-to-text translation addresses people with hearing impairments as well as people who cannot or do not want to use a keyboard for typing under certain circumstances. Speech-to-sign and sign-to-speech addresses deaf and hard-of-hearing people who use sign language. International language translation addresses the limitations of information exchange in a different language. Language/cognitive level translation addresses cognitively impaired people. Finally, image and video description as text or speech addresses people with visual impairments, but also people who cannot or do not want to use their eyes under certain circumstances.

While some of these services can be provided in a fully automated manner today, others might still need human assistance for the next decade. Although more and more automatic services will be implemented with emerging technologies, the early implementations may not be as mature as needed in some cases. In these cases a "Try Harder" feature could be used that would switch from local automatic services to more sophisticated service implementations on the network, and finally could bring in human intervention to assist the automatic translation process as well as provide a backup solution.

The Trace R&D Center is part of the University of Wisconsin in Madison. We are part of the EOT-PACI partnerships and currently investigate options and promote possible solutions for translation services within the Grid, particularly for deaf and hard-of-hearing users.

Modality Translation Graphic Text to Speech Service Speech to Text Service Speech to Sign Service Sign to Speech Service International Language Service Language Level Translation Service Image and Video Description Service D

Text-to-Speech Translation

icon depicts the conversion of text to speech

Today's speech technology produces acceptable quality for various applications. Uses either pre-sampled speech (restricted vocabulary) or synthetic speech.

Applications include:

Speech-to-Text Translation

icon depicts the conversion of speech to text

Today's speech-recognition software only achieves reasonable recognition rates with a restricted vocabulary or with speakers for whom the system was trained for.

There are two technical implementations for human-assisted real-time speech-to-text translation services:

  1. A trained person using a stenographic keyboard.
  2. A dedicated speaker (for whom the system is trained) re-voicing everything that has been said.

Applications include:

Speech-to-Sign Translation

icon depicts the conversion of speech to sign

Sign language interpreters provide instant speech-to-sign translation on site. Given a high-bandwidth wide-area network (Grid), this service can be provided remotely. Today, research is developing automatic real-time speech-to-sign translation using a "Signing Avatar." This multi-disciplinary approach combines several technological areas: speech recognition, linguistics (language representation), machine translation, sign language representation, computer animation.

Applications include:

Sign-to-Speech Translation

icon depicts the conversion of sign language to speech

Sign language interpreters provide instant sign-to-speech translation on site. With a local video camera connected to a high-bandwidth wide-area network (Grid), this service could be provided remotely. Inter-disciplinary research in the areas of computer vision, linguistics and machine translation aims to provide this service in a full-automatic manner.

Applications include:

International Language Translation

icon shows the American flag with an arrow pointing to a German flag

Internet translation services already provide instant text-to-text translation from one language to another. This technology still needs to mature. Ongoing research is also exploring automatic speech-to-speech translation for personal and business communications.

Applications include:

Language / Cognitive Level Translation

icon showing a thick text being translated to a thinner text with fewer words

Language/cognitive level translation is the art of transforming words and sentences to express the same meaning in a higher or lower language (abstraction) level. We are still far away from any kind of automation for this service.

Applications include:

Image / Video Description

icon shows a video camea conversion into both speech and text simultaneously

Today, verbal descriptions of images and videos are created by humans. Once recorded, this description can be replayed by a machine whenever somebody requests it. We are still far away from an automatic image/video description service which would include technologies from the fields of computer vision, pattern recognition, knowledge representation and linguistics.

Applications include:

Gottfried Zimmermann, Gregg Vanderheiden, Al Gilman Trace R&D Center, University of Wisconsin-Madison [zimmer@trace.wisc.edu, gv@trace.wisc.edu, asgilman@iamdigex.net]