Memes can now be described to blind people by AI program that recognizes photos
- Researchers have developed a tool to help the visually impaired enjoy memes
- The software breaks memes down into templates to describe the image and text
- The project aims to make social media accessible for 1.3 billion visually impaired
Memes have taken over the internet, but for people with visual impairments they can be difficult to understand.
A new research project has developed a system to translate memes into a format that’s easier for the 1.3 billion people on the planet with some form of visual impairment.
The project analyzes visual memes and breaks them down into three different templates that will make them easier to understand for people with visual impairments.
Researchers can bread down a meme into three different templates, making it easier for people with visual impairments to understand the context and meaning.
The tool, developed by researchers from Carnegie Mellon and Columbia University, takes advantage of a little-known feature of Facebook, Instagram and Twitter that lets users add alternative text to images.
The software compares the new image to those found in its database of stored memes, and then applies one of the three translation templates to it.
One template can break the meme down into audio information, a second uses the alt text feature to describe the image in the meme, and a third separates the text overlaid on the meme from the background image.
The new images are then added to the existing catalog of memes, making it easier to identify future memes.
Text can be broken out of the image to make it available for braille devices or audio services.
According to the team, 80 percent of the 1.7 million images shared over Twitter during the research period were not originals.
Current estimates are that just .1 percent of the images online are accessible to visually impaired people.
The memes are stored in a catalog that can be accessed and updated when new variations on memes appear.
Recently, there have been a number of different attempts to use the internet to bridge the gap between sighted people and the visually impaired.
A small handheld device called Bonocle was announced this spring to help translate text on laptops and tablets into brail.
A medical group in London produced a series of GIFS to simulate the effects of seeing with a number of visual impairments, including cataracts, glaucoma, and diabetic retinopathy.
WHAT IS BRAILLE?
Braille is a system of raised dots that can be read with the fingers by people who are blind or who have low vision.
Braille is not a language but a code by which many languages such as English, Spanish, Arabic, Chinese, and dozens of others may be written and read.
Braille is used by thousands of people all over the world in their native languages.
Braille symbols are formed within braille cells.
A full braille cell consists of six raised dots arranged in two parallel rows each having three dots.
The dot positions are identified by numbers from one through six and sixty-four combinations are possible using one or more of these six dots.
A single cell can be used to represent an alphabet letter, number, punctuation mark, or even a whole word