Smartwatch Motion Tracking Malware Could Steal PIN Codes

A Copenhagen computer engineering student has demonstrated a technique that could allow attackers to steal keypad login credentials by tracking a user’s hand movements using malicious code running on a smartwatch.

The technique, detailed in a master’s thesis by French student Tony Beltramelli at the University of Copenhagen, builds on earlier work by Romit Roy Choudhury, associate professor at the Department oF Electrical and Computer Engineering of the University of Illinois, who last year demonstrated how a Samsung Gear Live smartwatch’s motion sensors could be used to log a user’s keystrokes on a standard computer keyboard.

Keypad spying

Beltramelli’s work goes a step further, using machine learning techniques to try to predict a user’s input in a 12-digit keypad of the kind used in cash machines or smartphone login screens.

“The goal of this work is to raise awareness about the potential risks related to motion sensors built into wearable devices and to demonstrate abuse opportunities leveraged by advanced neural network architectures,” Beltramelli wrote (PDF) in the thesis, titled “Deep-Spying: Spying using Smartwatch and Deep Learning”.

He built a customised application running on a Sony SmartWatch 3 to record accelerometer and gyroscope data, which was transmitted via Bluetooth to a nearby LG Nexus 4 Android device, and then to a server for analysis.

The server-side code used a machine learning algorithm called RNN-LSTM, or Recurrent Neural Network – Long Short-Term Memory, to guess what characters were being entered based on the recorded movements. The algorithm has in the past been used in computer-vision and language processing.

Partial accuracy

He said the architecture is currently able to achieve a maximum accuracy of 73 percent for touch-screen entry and 59 percent for keypad entry. In a video, the system is demonstrated guessing five out of eight characters correctly.

Beltramelli said such a system could be trained to detect keystrokes with a high degree of accuracy from a wide variety of keypads. He said similar systems could be used to crack gesture-based lock screens.

Such an attack would in theory rely on a user being tricked into installing malicious software on a smartwatch. Researchers have, however, demonstrated flaws in the Android mobile operating system that would allow the installation of malicious code with no user interaction. Research published last summer found numerous security flaws in Android-based smartwatches including the Sony SmartWatch and Samsung Gear Live.

“These observations imply that a cyber-criminal would be able, in theory, to eavesdrop on any device operated by the user while wearing a WAD (Wearable Wristband and Armband Device),” Beltramelli wrote.

He suggested users could protect themselves by wearing such devices on the hand they don’t use to enter sensitive keypad data.

Are you a security pro? Try our quiz!