Smart glove translates sign language into speech in seconds

Scientists have created a glove that translates sign language into speech in a matter of seconds through a smartphone app.

The hi-tech glove sends electronic signals generated from different movements of the fingers to a circuit board, which are then transmitted wirelessly to the phone. 

The app translates letters from movements on one hand, as per the American Sign Language (ASL) standard, into phrases, which are uttered by an automated voice. 

The patented technology aims to reduce the language barrier between people with hearing or speech impediments and those who don’t understand sign language.

 

The device is inexpensive, flexible and highly durable, according to the team of bioengineers at University of California, LA

Components used to make the device are cheap, flexible and highly durable, according to the team of engineers in the US, who plan to commercialise the technology if they can speed up the translation time even further. 

The technology relays spoken words at the rate of about a one word per second, but an even faster system could come close to replicating the speed of everyday verbal conversations, and ultimately overcome the language barrier. 

‘Our hope is that this opens up an easy way for people who use sign language to communicate directly with non-signers without needing someone else to translate for them,’ said Professor Jun Chen at University of California, Los Angeles.

‘In addition, we hope it can help more people learn sign language themselves.’

The system includes a pair of gloves with thin, stretchable sensors that run the length of each of four fingers and thumb.

The sensors, made from electrically-conducting lines of thread, record hand motions and finger placements that stand for individual letters, numbers, words and phrases.

It is capable of reading the individual letters of the ASL alphabet, as well as single hand signals that relate to specific phrases – such as the hand symbol for ‘I love you’, consisting of the thumb, forefinger and little finger raised.

The system includes gloves with thin, stretchable sensors that run the length of each of the five fingers to a coin-sized chip

The system includes gloves with thin, stretchable sensors that run the length of each of the five fingers to a coin-sized chip

ASL differs from the British Sign Language (BSL) system – most notably the former uses one hand instead of two.

The device then turns the finger movements into electrical signals, which are sent to a coin-sized circuit board on the glove, positioned just above the wrist.

The board transmits those signals wirelessly to a smartphone that translates them into spoken words at a rate of about a one word per second.

The UCLA researchers had worked with four deaf ASL users to effectively train the glove and its accompanying app.

The wearers repeated each hand gesture 15 times and a custom machine-learning algorithm turned these gestures into the letters, numbers and words they represented.

In total the system learned to recognise 660 signs, including each letter of the alphabet and numbers 0 to 9.

A digital rendering of the system that helps convert sign language into speech. UCLA has filed for a patent on the technology

A digital rendering of the system that helps convert sign language into speech. UCLA has filed for a patent on the technology

The researchers also stuck adhesive sensors to testers’ faces – one in between their eyebrows and another on one side of their mouths – to capture facial expressions, which are a key part of ASL and altering meanings of hand signs. 

Previous wearable alternatives were limited by bulky and heavy designs or were uncomfortable to wear, Dr Chen said.

But the new device is made from light but long-lasting, stretchable and inexpensive polymers attached to a glove.

UCLA has filed for a patent on the technology, although a commercial model based would require added vocabulary and an even faster translation time.

The team would also have to adjust the algorithm and the app to be able to work for BSL and be used on both hands.

HOW DOES ENGLISH SIGN LANGUAGE VARY GLOBALLY?

There are about 37.5 million deaf or hard-of-hearing people in the US and American Sign Language (ASL) is the third most used language in the country.

ASL is a complete language that uses signs made by moving the hands combined with facial expressions and body postures.

It is the primary language of many North Americans who are deaf and is one of many communications options for people who are deaf or hard-of-hearing.

British Sign Language (BSL) is used in place of ASL in the UK.

BSL users utilize both hands for the alphabet while ASL users only utilize one.

ASL communicators and verbal English speakers rely on the same neural skills to chat, a report says (file photo)

ASL communicators and verbal English speakers rely on the same neural skills to chat, a report says (file photo)

Read more at DailyMail.co.uk