It turns out it doesn’t take much for hackers to see what’s being displayed on your computer screen.
A team of researchers have discovered that ultrasonic sounds picked up by a webcam microphone can be analyzed using machine learning to determine what’s being shown on a remote computer screen.
The attack could allow anyone savvy enough to spy on someone’s browsing activity with ease.
Researchers found that ultrasonic sounds picked up by a webcam mic can be analyzed using machine learning to determine what’s being shown on a remote computer screen
Scientists from University of Michigan, University of Pennsylvania and Tel Aviv University observed subtle acoustic noises coming from LCD screens – specifically faint, high-pitched sounds that are generated to power the display.
The pitch will fluctuate depending on what users are looking at, as varying levels of power are needed to display pixels on a screen.
Hackers can then capture the audio being recorded by the computer’s microphone, an external webcam’s microphone, a webcam used in a chat service like Google Hangouts or even voice-activated smart speakers like an Amazon Echo, and use machine learning to deduce what someone is looking at.
Researchers could successfully judge a person’s display activity from up to 30 feet away in some cases.
They’re calling this phenomenon ‘Synesthesia,’ or a side-channel attack that can reveal what’s on a screen just by looking at ‘content-dependent acoustic leakage from LCD screens.’
‘The pertinent sounds are so faint and high-pitched that they are well-nigh inaudible to the human ear, and thus (unlike with mechanical peripherals) users have no reason to suspect that these emanations exist and that information about their screen content is being conveyed to anyone who receives the audio stream, or even a retroactive recording,’ according to the study.
Researchers collected numerous of recordings of LCD display sound using an LG V20 phone (pictured) in various positions and fed that information into a machine learning algorithm, which analyzed the data to predict what was on users’ screens
They displayed the frequencies on a spectrogram, or a visual representation of frequencies of sound as they change over time (right). It appeared as a zebra stripe pattern
‘In fact, users often make an effort to place their webcam (and thus, microphone) in close proximity to the screen, in order to maintain eye contact during videoconference, thereby offering high quality measurements to would-be attackers.’
For the study, researchers examined the acoustic leakage from a variety of LCD monitors, spanning different sizes and makers. The monitors were made between 2003 and 2017.
They collected numerous of recordings of LCD display sound using an LG V20 phone in various positions and fed that information into a machine learning algorithm, which analyzed the data to predict what was on users’ screens.
Researchers then displayed the frequencies on a spectrogram, or a visual representation of frequencies of sound as they change over time. They often appeared as a zebra stripe pattern.
For the study, researchers examined the acoustic leakage from a variety of LCD monitors, spanning different sizes and makers. The monitors were made between 2003 and 2017
In some cases, they had between a 90 percent and 100 percent success rate, Wired noted.
The system was even capable of identifying which of the 10 most popular websites was display on a monitor with 96.4 percent accuracy.
Scientists also demonstrated in their research how attackers could infer what someone is typing by analyzing the audio frequencies generated when an on-screen keyboard is used.
While on-screen keyboards are meant to act as a security mechanism for password entry, the researchers demonstrated that even these functions aren’t safe from hackers’ spying eyes and ears.