Washington, Oct 1 : Researchers at IBM, UK, have developed a software that converts spoken words into sign language and engages an avatar to communicate using gestures.

Its developers say the onscreen translator, named “Say It Sign It”, could work as a pop-up on a television, personal computer, mobile phone or auditorium screen, giving the hearing impaired wider access to television, radio and education.

“It was inspired by a vision that a deaf colleague on my team had of seeing, not words being brought up on his phone, but an animated character signing in British Sign Language,” said Andy Stanford-Clark, master inventor at IBM Hursley in the UK.

Ben Fletcher, then an intern for the Extreme Blue program, and now a permanent member of the team, which also includes researchers from the Universities of Oxford, Durham, Glasgow and East Anglia, was initialled into the project.

Fletcher is deaf and British Sign Language is his first language.

“He was able to give feedback and keep us honest,” said Stanford-Clark.

The program has about three main components. First a voice recognition system takes words spoken into a microphone and converts those into a stream of text. The text is then sent through a translation program that looks for patterns in the words and applies different rules based on those patterns.

For example, if someone says: “My name is Andy,” a rule would transform that to the phrase in British Sign Language, which is “Name me Andy”.

Once the word order is worked out, the program refers to a dictionary of gestures supplied by the Royal National Institute for Deaf People, which are used to direct the computer-animated character.

Tying it all together is a piece of software, which allows different applications and computers to link together without being in the same building or even the same country.

Stanford-Clark said though the system is still in the prototype phase, the team envisions a couple of different scenarios for its use.

The program could be hosted by a service Web site. A speaker would sign in on one end, while the recipient would sign in on the other end. As the person would start speaking, words would be converted on a central server and then animated on the end-user's screen.

The system could also work from a converter box atop a television, or it could be sold individually as a program that speakers and receivers would install on their own computers, he said.

“The model is quite different from the things that I have seen in the past. It's built in a way that, in principle, can be applied to other foreign languages,” said Guido Gybels, director of new technologies at the Royal National Institute for Deaf People in London.

“However, there are significant technical and scientific challenges, in addition to cultural ones, that need to be addressed before we can see this as an off-the-shelf product,” he said.

“For starters, the speech recognition technology is still a long way off from capturing free-flowing unconstrained human dialogue. Furthermore, while the English language (as well as other languages) has been studied in great depth, sign language has not.

“If scientists are going to accurately convert speech into sign language they will need that understanding.

“And lastly, the notion of using virtual humans and animated characters doesn't always fly with people you're asking to use these kind of services. Not all humans are going to accept a replacement, especially one that may not have the natural, subtle motions – the finger, lip and movements – of the real thing,” he added.

According to a Discovery News report, presently “Say It Sign It” translates spoken English into British Sign Language, and the team is working towards translating it between other spoken languages and forms of sign language. (With inputs from ANI)