Google Glass helps autistic children read facial expressions

Google Glass can help children with autism read facial expressions to assess people’s emotions, a new study has revealed.

Children with the developmental disorder often fail to recognise basic facial emotions, which makes social interactions and developing friendships difficult.

Researchers used the internet-connected eyewear, which includes a small head-mounted display positioned above the wearer’s right-eye to display notifications.

Researchers paired Google Glass with smartphone software designed to read facial expressions captured by the camera on the front of the eyewear and provide social cues to the wearer on the display.

The project, known as Superpower Glass, taught the children to recognise the emotions conveyed in people’s facial expressions.

Google discontinued its Glass eyewear for consumers worldwide in 2015, however, the company is working on a new version of the hardware for enterprise customers.

 

Children with the developmental disorder often fail to recognise basic facial emotions, which make social interactions and developing friendships difficult. Prior to the study Alex, nine, (pictured) could not look people in the eye

The research was conducted by Stanford University School of Medicine.

Researchers programmed the companion smartphone app for Google Glass to use games to provide real-time cues about the facial expressions of those around them. 

Google Glass, which needs to be paired with a smartphone, consists of a glasses-like frame equipped with a camera to record the wearer’s field of view, as well as a small screen and a speaker to give the wearer visual and audio information.

As the child interacts with others, the app identifies and names their emotions through the Google Glass speaker or screen.

After just one to three months of regular use, parents reported children with autism made eye contact more regularly and related better to others.

And their autism got better, with some dropping from severe to moderate.

The pilot project was carried out with 14 families with a child aged between three and 17 with a clinically confirmed autism diagnosis.

The child tested the glasses for an average of 10 weeks.

Prior to the study Alex, nine, could not look people in the eye despite gentle encouragement from his mother Donji Cullenbine.

She explained: ‘I would smile and say things like, “You looked at me three times today”… But it didn’t really move the bar.’

A few weeks into the trial, Alex (pictured) began to realise that people's faces hold clues to their feelings. In the US there is currently an 18 month waiting list to see a specialist

A few weeks into the trial, Alex (pictured) began to realise that people’s faces hold clues to their feelings. In the US there is currently an 18 month waiting list to see a specialist

THE SIGNS AND SYMPTOMS OF AUTISM

According to the Centers for Disease Control and Prevention, people with autism have trouble with social, emotional and communication skills that usually develop before the age of three and last throughout a person’s life. 

Specific signs of autism include: 

  • Reactions to smell, taste, look, feel or sound are unusual
  • Difficulty adapting to changes in routine
  • Unable to repeat or echo what is said to them
  • Difficulty expressing desires using words or motions
  • Unable to discuss their own feelings or other people’s
  • Difficulty with acts of affection like hugging
  • Prefer to be alone and avoid eye contact
  • Difficulty relating to other people
  • Unable to point at objects or look at objects when others point to them

A few weeks into the trial, Alex began to realise that people’s faces hold clues to their feelings.

She said: ‘He told me, “Mommy, I can read minds”. My heart sang. I’d like other parents to have the same experience.’

Associate professor of paediatrics and biomedical data science Dr Dennis Wall said the treatment could help fill a major gap in autism care. 

In the US, there is currently an 18 month waiting list to see a specialist.

Early autism therapy has been shown to be particularly effective, but many children aren’t treated quickly enough to get the maximum benefit.

He said: ‘The only way to break through the problem is to create reliable, home-based treatment systems. It’s a really important unmet need.’

The app uses machine learning to recognise eight core facial expressions: happiness, sadness, anger, disgust, surprise, fear, neutral and contempt.

Typically, children learn to recognise emotions by engaging with people around them.

For children with autism, it’s different. 

‘They don’t pick those things up without focused treatment,’ said Professor Wall.

Families used the therapy for at least three 20-minute sessions per week with parents completing questionnaires at the start and end of the pilot.

In interviews, parents and children also gave feedback about how the programme worked for their families.

The researchers designed three ways to use the face-recognising programme.

In ‘free play’ children wear Google Glass while interacting or playing with their families, and the software provides the wearer with a visual or auditory cue each time it recognises an emotion on the face of someone in the field of view.

In ‘guess my emotion’ game mode a parent acts out a facial expression corresponding to one of the eight core emotions, and the child tries to identify it.

The game helps families and researchers track children’s improvement at identifying emotions.

In ‘capture the smile’ game mode children give another person clues about the emotion they want to elicit, until the other person acts it out, which helps the researchers gauge the children’s understanding of different emotions.

The study found children with more severe autism were more likely to choose the game modes rather than free play.

Further research is planned, but Professor Wall said the initial results were promising.

‘Parents’ comments in interviews such as “a switch has been flipped”, “my child is looking at me” or “suddenly the teacher is telling me that my child is engaging in the classroom” was really heartwarming and super-encouraging for us to hear,’ Professor Wall said.

Around 700,000 people in the UK are on the autism spectrum – more than one in 100 and it affects one in 59 children in the US.

The study was published in the journal Digital Medicine.

WHAT COMPANIES ARE WORKING ON AUGMENTED REALITY GLASSES?

Bose joins a quickly growing list of tech firms that say they’re developing augmented reality eyeglasses. 

The first company to enter the race was Google, which released the Google Glass in 2011. 

Google Glass, now referred to as Glass, has been changed from a consumer-facing product to an enterprise product, used by companies like Boeing. 

Since then, several companies have come out with their own products.

Secretive startup Magic Leap began working on a prototype several years ago, but finally debuted its ‘mixed reality’ smart glasses late last year. 

Magic Leap says its AR glasses will ship in 2018 after a multi-year wait. 

Tech company Vuzix, based in Rochester, New York, is launching its Vuzix Blade glasses later this year for about $1,300. 

Smart glasses that superimpose computer-generated images onto the world around you could start at at £1,000 when they are released this year. Magic Leap, the usually secretive Google-backed company behind the gadget, says it is working on multiple versions of the gadget

Smart glasses that superimpose computer-generated images onto the world around you could start at at £1,000 when they are released this year. Magic Leap, the usually secretive Google-backed company behind the gadget, says it is working on multiple versions of the gadget

They use a tiny projector to show a virtual image in the top right hand corner of their lenses. 

Wearers can connect to WiFi and read emails and other messages via the display, as well as use Alexa, Amazon’s digital assistant, to issue voice commands. 

Amazon is also rumored to be working on its own AR glasses to be released sometime in the future. 

Additionally, Intel released its prototype smart glasses, the Vaunt, earlier this year. 

The glasses use retinal projection to put a tiny display on the wearer’s eyeball. 

 



Read more at DailyMail.co.uk