Scientists have created a glove that translates sign language into speech in a matter of seconds through a smartphone app.
The hi-tech glove sends electronic signals generated from different movements of the fingers to a circuit board, which are then transmitted wirelessly to the phone.
The app translates letters from movements on one hand, as per the American Sign Language (ASL) standard, into phrases, which are uttered by an automated voice.
The patented technology aims to reduce the language barrier between people with hearing or speech impediments and those who don’t understand sign language.
The device is inexpensive, flexible and highly durable, according to the team of bioengineers at University of California, LA
Components used to make the device are cheap, flexible and highly durable, according to the team of engineers in the US, who plan to commercialise the technology if they can speed up the translation time even further.
The technology relays spoken words at the rate of about a one word per second, but an even faster system could come close to replicating the speed of everyday verbal conversations, and ultimately overcome the language barrier.
‘Our hope is that this opens up an easy way for people who use sign language to communicate directly with non-signers without needing someone else to translate for them,’ said Professor Jun Chen at University of California, Los Angeles.
‘In addition, we hope it can help more people learn sign language themselves.’
The system includes a pair of gloves with thin, stretchable sensors that run the length of each of four fingers and thumb.
The sensors, made from electrically-conducting lines of thread, record hand motions and finger placements that stand for individual letters, numbers, words and phrases.
It is capable of reading the individual letters of the ASL alphabet, as well as single hand signals that relate to specific phrases – such as the hand symbol for ‘I love you’, consisting of the thumb, forefinger and little finger raised.
The system includes gloves with thin, stretchable sensors that run the length of each of the five fingers to a coin-sized chip
ASL differs from the British Sign Language (BSL) system – most notably the former uses one hand instead of two.
The device then turns the finger movements into electrical signals, which are sent to a coin-sized circuit board on the glove, positioned just above the wrist.
The board transmits those signals wirelessly to a smartphone that translates them into spoken words at a rate of about a one word per second.
The UCLA researchers had worked with four deaf ASL users to effectively train the glove and its accompanying app.
The wearers repeated each hand gesture 15 times and a custom machine-learning algorithm turned these gestures into the letters, numbers and words they represented.
In total the system learned to recognise 660 signs, including each letter of the alphabet and numbers 0 to 9.
A digital rendering of the system that helps convert sign language into speech. UCLA has filed for a patent on the technology
The researchers also stuck adhesive sensors to testers’ faces – one in between their eyebrows and another on one side of their mouths – to capture facial expressions, which are a key part of ASL and altering meanings of hand signs.
Previous wearable alternatives were limited by bulky and heavy designs or were uncomfortable to wear, Dr Chen said.
But the new device is made from light but long-lasting, stretchable and inexpensive polymers attached to a glove.
UCLA has filed for a patent on the technology, although a commercial model based would require added vocabulary and an even faster translation time.
The team would also have to adjust the algorithm and the app to be able to work for BSL and be used on both hands.