How a smartphone can overhear, spy, and track
An interesting fact recently appeared in the media: when communicating with a Wired reporter, a Facebook employee asked to turn off the smartphone so that it was more difficult for social networks to record the fact of their communication. To do this, Facebook could use satellite navigation or a microphone, although to prevent surveillance it would be enough to turn them off. There is probably something else on the phone that is worth being afraid of. A modern smartphone is packed with sensors: 2-3 cameras, a light sensor, an accelerometer, a gyroscope, GPS and GLONASS, a magnetometer and others. As NASA researchers using sensors for remote spacecraft repairs for other purposes, attackers can learn a lot about the owner of a smartphone by gaining access to one or more sensors. This is not about cases of physical hacking of gadgets by installing chips or adding wires , but by solving sensors such as spying on users, intercepting or getting a pincode.
Illustration for the PlaceRaider application that creates a 3D model of a room from frames made without the user's knowledge
The accelerometer is able to track the movement of the device in three axes. Phone, he, in particular, helps to turn the image, set it vertically or horizontally depending on the position of the device. In 2006, the acceleration sensor first appeared in the phones, then it was the Nokia 5500 model, in which the accelerometer helped to implement "sports" functions - a pedometer. The gyroscope was first added to the Apple smartphone - to the iPhone 4. The sensor allows you to drive a car in racing games without pressing the arrows and ensures that the person is on the displayed map. For these amenities you have to pay security. Malefactors, having got access to data from one or several sensors, are capable to pull out from them a lot of useful. Such access is easy to get: go, for example, to this page from a smartphone, and you will see that JavaScript easily receives data from a gyroscope. The same is possible with HTML5.
Smartphones on iOS and Android are blocked using a digital pin code, graphic or fingerprint. In the first two cases, the phone can be hacked by finding out how it changes position when the user unlocks. In the case of a digital pincode, researchers from the University of Newcastle UK learned how to guess from the first time in 74% of cases using several sensors - an accelerometer, a gyroscope and a magnetometer. On the third attempt, they cracked the code in 94% of cases.
Popular browsers Safari, Chrome, Firefox, Opera, and Dolphin initially have access to sensors, so attackers simply add an appropriate exploit to the site, rather than asking the owner for permission that is required when installing the application from stores.
A graphical pincode requires a quick password entry of four or more points on a 3x3 field. The field has 389,112 possible combinations, but researchers at the University of Pennsylvania are confident that in reality, users use an order of magnitude less circuitry. Some combinations are inconvenient for permanent use. The application running in the background, at the right time starts the accelerometer, then disables it and transmits data to fraudsters. Researchers needed only one sensor for hacking.
A similar method in 2015 was used by scientists from the IT-University of Copenhagen, only in this case, smart watches watched not only entering the code on a smartphone, but also entering a pin from a card at an ATM or store. Data from the gyro sensor in the clock was transmitted to the smartphone, from where it was sent to the server and unloaded into CSV.
Millions of people work on laptops and desktop computers every day. Scammers can find out what a person is typing on the keyboard if the smartphone is close to it. Scientists from the Georgia Institute of Technology in 2011 programmed mobile devices to monitor the text typed on the keyboard: gadgets measured the vibrations of the table surface. According to scientists, the procedure was not easy, but the accuracy of determination at that time was up to 80%.
The iPhone 3GS smartphone didn’t work for this kind of work, but the iPhone 4, the first smartphone with a built-in gyro sensor, performed well. A group of researchers attempted to use a microphone, a more sensitive sensor, for surveillance. In the end, the accelerometer turned out to be the preferred method, since it is traditionally less protected by the system.
The technique developed by scientists looked for successive pairs of keystrokes. The application finds out in which place of the keyboard the keys were pressed - on the left-top and on the right-below, on the right-below and on the right-on-and also determines the distance for each pair of keys. It then compares the results with the preloaded dictionary. The method worked with words of three or more letters.
It is possible not only to steal data from the accelerometer, but also to control the device with its help, forcing the smartphone to carry out the actions necessary for fraudsters. Speaker for $ 5 helped to crack 20 accelerometers from 5 manufacturers using sound waves. A group of researchers from the University of Michigan and the University of South Carolina used a "music virus", as they called their technique in an interview with The New-York Times, to make the Fitbit app believe that the user had completed thousands of steps and control a toy car using a phone. The goal of the researchers was to create software solutions to counter such attacks.
Since the gyroscope picks up sound vibrations, it can be used as a hard disk of a computer for hidden listening. Scientists from Stanford University and specialists from the Israeli defense company Rafael have found a way to turn the gyroscope of an Android smartphone into a constantly on microphone. They developed the “Gyrophone” application: the sensors of many Android devices detect vibrations from the sound frequency from 80 to 250 hertz.
The voice of an adult male has a frequency of 85 to 155 Hz, women - from 165 to 255 Hz. Consequently, the gyro sensor is able to listen to human speech. The iPhone's gyroscope uses a frequency below 100 Hz, so it is not suitable for the same purpose, but, nevertheless, it can help you in individual words to recognize the speaker's gender. The accuracy of the instrument in 2014 was not very high - up to 64%.
Coordinated work of several sensors in the smartphone and machine learning will help to track the movements of the device owner with the satellite navigation turned off. The illustration below shows how exactly the route is determined by the method proposed by a group of researchers from the Institute of Electrical and Electronics Engineers (IEEE). Green indicates the path that the user has traveled by transport, orange indicates the distance traveled, and black indicates the data from the GPS.
The PinMe application maps information from sensors with open data. First, the exploit receives information about the latest IP address of the smartphone and connecting to Wi-Fi to determine the starting point of the route. Then - in the direction, speed and frequency of stopping recognizes the difference between walking, driving a car and public transport, flights on an airplane. PinMe compares the obtained data with information from open sources: navigation data takes from OpenStreetMaps, elevation map in Google Maps, route data from airline schedules and railway lines. To clarify the route, the application used the Weather Channel weather service: accurate information about the temperature and air pressure helps to neutralize the influence of weather conditions on the information collected by the sensors.
In 2010, a similar technique was used by the Japanese telecommunications corporation KDDI: the accelerometer in the smartphone was used to spy on employees. Data from the sensor made it possible to understand whether a person is walking up or down a flat surface, shaking out rubbish from their bins or washing floors. In 2015, specialists from Nanjing University in China used data from the accelerometer to monitor the movement of people in the subway.
To determine the location of the owner of the smartphone can the application that receives data on the battery status. Such information can be obtained by any application, since it does not require additional permissions. Scientists from Stanford and specialists from the defense company Rafael, who have already been mentioned above, have developed the Power Spy technology.
The location of the user is determined with 90% accuracy due to the analysis of the discharge rate of the battery: this is how scientists determined the remoteness of the gadget from the repeaters. But such accuracy is possible only if the user is not the first time along this route.
In 2012, the American military research center in Indiana and scientists from Indiana University developed the PlaceRaider application for Android 2.3 smartphones, which could reconstruct the user's environment in 3D.
The user had to download the application with the ability to take photos and give him permission to use the camera and send them. PlaceRaider, working in the background, turned off the shutter sound, so as not to disturb the user. Then the program randomly took photos, saving information about the time, place and orientation of the smartphone. After filtering photos and removing bad frames made, for example, in the user's pocket, the application sent them to the server, where the 3D model of the room was created.
To test the effectiveness of this idea, scientists gave “infected” phones to twenty volunteers who did not know about the application, and sent them to the office with various simple tasks. At the next stage, two groups of people viewed the results: one - separate photos, the second - 3D models. Both groups were looking for QR codes, checks, documents, as well as calendars, which attackers could use to determine when the victim would not be in a certain place.
The application for the "end user", that is, in the worst case - the criminal, and in ours - the scientists, allowed to bring certain parts of the frame in the best traditions of Hollywood films. In this case, the person who opened the 3D model could click on a certain point, after which the application looked for better quality photos from the base, taken closer to the place of interest. The image below shows the number of the check lying on the table.
The greater the power, the greater the responsibility: it must be remembered by the developers of smartphones and applications to them, which today open up unlimited possibilities for hacking users' wallets, tracking movement and determining interests for more accurate advertising targeting. In real life, of course, most of these researchers are interesting at best to the screenwriters of the Black Mirror.
Real hackers periodically develop cool ways to withdraw money from the public, but they do not implement them very well. For example, in February 2018, they were able to upload a miner to government sites in the UK, USA and Canada, forcing them to earn cryptocurrency for themselves within four hours. Instead of receiving a huge amount of information from these sites and selling it, they connected a miner and earned $ 24. However, after clarifying the circumstances and the money, the mining service did not pay them.
Illustration for the PlaceRaider application that creates a 3D model of a room from frames made without the user's knowledge
The accelerometer is able to track the movement of the device in three axes. Phone, he, in particular, helps to turn the image, set it vertically or horizontally depending on the position of the device. In 2006, the acceleration sensor first appeared in the phones, then it was the Nokia 5500 model, in which the accelerometer helped to implement "sports" functions - a pedometer. The gyroscope was first added to the Apple smartphone - to the iPhone 4. The sensor allows you to drive a car in racing games without pressing the arrows and ensures that the person is on the displayed map. For these amenities you have to pay security. Malefactors, having got access to data from one or several sensors, are capable to pull out from them a lot of useful. Such access is easy to get: go, for example, to this page from a smartphone, and you will see that JavaScript easily receives data from a gyroscope. The same is possible with HTML5.
Smartphones on iOS and Android are blocked using a digital pin code, graphic or fingerprint. In the first two cases, the phone can be hacked by finding out how it changes position when the user unlocks. In the case of a digital pincode, researchers from the University of Newcastle UK learned how to guess from the first time in 74% of cases using several sensors - an accelerometer, a gyroscope and a magnetometer. On the third attempt, they cracked the code in 94% of cases.
Popular browsers Safari, Chrome, Firefox, Opera, and Dolphin initially have access to sensors, so attackers simply add an appropriate exploit to the site, rather than asking the owner for permission that is required when installing the application from stores.
A graphical pincode requires a quick password entry of four or more points on a 3x3 field. The field has 389,112 possible combinations, but researchers at the University of Pennsylvania are confident that in reality, users use an order of magnitude less circuitry. Some combinations are inconvenient for permanent use. The application running in the background, at the right time starts the accelerometer, then disables it and transmits data to fraudsters. Researchers needed only one sensor for hacking.
A similar method in 2015 was used by scientists from the IT-University of Copenhagen, only in this case, smart watches watched not only entering the code on a smartphone, but also entering a pin from a card at an ATM or store. Data from the gyro sensor in the clock was transmitted to the smartphone, from where it was sent to the server and unloaded into CSV.
Millions of people work on laptops and desktop computers every day. Scammers can find out what a person is typing on the keyboard if the smartphone is close to it. Scientists from the Georgia Institute of Technology in 2011 programmed mobile devices to monitor the text typed on the keyboard: gadgets measured the vibrations of the table surface. According to scientists, the procedure was not easy, but the accuracy of determination at that time was up to 80%.
The iPhone 3GS smartphone didn’t work for this kind of work, but the iPhone 4, the first smartphone with a built-in gyro sensor, performed well. A group of researchers attempted to use a microphone, a more sensitive sensor, for surveillance. In the end, the accelerometer turned out to be the preferred method, since it is traditionally less protected by the system.
The technique developed by scientists looked for successive pairs of keystrokes. The application finds out in which place of the keyboard the keys were pressed - on the left-top and on the right-below, on the right-below and on the right-on-and also determines the distance for each pair of keys. It then compares the results with the preloaded dictionary. The method worked with words of three or more letters.
It is possible not only to steal data from the accelerometer, but also to control the device with its help, forcing the smartphone to carry out the actions necessary for fraudsters. Speaker for $ 5 helped to crack 20 accelerometers from 5 manufacturers using sound waves. A group of researchers from the University of Michigan and the University of South Carolina used a "music virus", as they called their technique in an interview with The New-York Times, to make the Fitbit app believe that the user had completed thousands of steps and control a toy car using a phone. The goal of the researchers was to create software solutions to counter such attacks.
Since the gyroscope picks up sound vibrations, it can be used as a hard disk of a computer for hidden listening. Scientists from Stanford University and specialists from the Israeli defense company Rafael have found a way to turn the gyroscope of an Android smartphone into a constantly on microphone. They developed the “Gyrophone” application: the sensors of many Android devices detect vibrations from the sound frequency from 80 to 250 hertz.
The voice of an adult male has a frequency of 85 to 155 Hz, women - from 165 to 255 Hz. Consequently, the gyro sensor is able to listen to human speech. The iPhone's gyroscope uses a frequency below 100 Hz, so it is not suitable for the same purpose, but, nevertheless, it can help you in individual words to recognize the speaker's gender. The accuracy of the instrument in 2014 was not very high - up to 64%.
Coordinated work of several sensors in the smartphone and machine learning will help to track the movements of the device owner with the satellite navigation turned off. The illustration below shows how exactly the route is determined by the method proposed by a group of researchers from the Institute of Electrical and Electronics Engineers (IEEE). Green indicates the path that the user has traveled by transport, orange indicates the distance traveled, and black indicates the data from the GPS.
The PinMe application maps information from sensors with open data. First, the exploit receives information about the latest IP address of the smartphone and connecting to Wi-Fi to determine the starting point of the route. Then - in the direction, speed and frequency of stopping recognizes the difference between walking, driving a car and public transport, flights on an airplane. PinMe compares the obtained data with information from open sources: navigation data takes from OpenStreetMaps, elevation map in Google Maps, route data from airline schedules and railway lines. To clarify the route, the application used the Weather Channel weather service: accurate information about the temperature and air pressure helps to neutralize the influence of weather conditions on the information collected by the sensors.
In 2010, a similar technique was used by the Japanese telecommunications corporation KDDI: the accelerometer in the smartphone was used to spy on employees. Data from the sensor made it possible to understand whether a person is walking up or down a flat surface, shaking out rubbish from their bins or washing floors. In 2015, specialists from Nanjing University in China used data from the accelerometer to monitor the movement of people in the subway.
To determine the location of the owner of the smartphone can the application that receives data on the battery status. Such information can be obtained by any application, since it does not require additional permissions. Scientists from Stanford and specialists from the defense company Rafael, who have already been mentioned above, have developed the Power Spy technology.
The location of the user is determined with 90% accuracy due to the analysis of the discharge rate of the battery: this is how scientists determined the remoteness of the gadget from the repeaters. But such accuracy is possible only if the user is not the first time along this route.
In 2012, the American military research center in Indiana and scientists from Indiana University developed the PlaceRaider application for Android 2.3 smartphones, which could reconstruct the user's environment in 3D.
The user had to download the application with the ability to take photos and give him permission to use the camera and send them. PlaceRaider, working in the background, turned off the shutter sound, so as not to disturb the user. Then the program randomly took photos, saving information about the time, place and orientation of the smartphone. After filtering photos and removing bad frames made, for example, in the user's pocket, the application sent them to the server, where the 3D model of the room was created.
To test the effectiveness of this idea, scientists gave “infected” phones to twenty volunteers who did not know about the application, and sent them to the office with various simple tasks. At the next stage, two groups of people viewed the results: one - separate photos, the second - 3D models. Both groups were looking for QR codes, checks, documents, as well as calendars, which attackers could use to determine when the victim would not be in a certain place.
The application for the "end user", that is, in the worst case - the criminal, and in ours - the scientists, allowed to bring certain parts of the frame in the best traditions of Hollywood films. In this case, the person who opened the 3D model could click on a certain point, after which the application looked for better quality photos from the base, taken closer to the place of interest. The image below shows the number of the check lying on the table.
The greater the power, the greater the responsibility: it must be remembered by the developers of smartphones and applications to them, which today open up unlimited possibilities for hacking users' wallets, tracking movement and determining interests for more accurate advertising targeting. In real life, of course, most of these researchers are interesting at best to the screenwriters of the Black Mirror.
Real hackers periodically develop cool ways to withdraw money from the public, but they do not implement them very well. For example, in February 2018, they were able to upload a miner to government sites in the UK, USA and Canada, forcing them to earn cryptocurrency for themselves within four hours. Instead of receiving a huge amount of information from these sites and selling it, they connected a miner and earned $ 24. However, after clarifying the circumstances and the money, the mining service did not pay them.
Source: https://habr.com/ru/post/410159/