Script Update: Dump iOS Frequent Locations – Now with KML & CSV Output!

Update Details

I have added some output options to the script – CSV and KML.

See a related post here – “Parsing iOS Frequent Locations”

The script can now be called with a ‘-output’ argument with the following options:

  • k – KML Output only
  • c – CSV Output only
  • e – Everything (KML and CSV)

Note: The verbose script output is still available from standard output.

Usage

python dump_freq_locs.py –output e <StateModel#.plist>

Updated script (v1.1) is available in GitHub

Output Examples

Example of the CSV output in Excel


Example of the KML output in Google Earth


Example of the KML output in Google ‘My Maps’

 

 

Continue reading Script Update: Dump iOS Frequent Locations – Now with KML & CSV Output!

Parsing iOS “Frequent Locations”

The Artifact:

The phrase “Location, Location, Location” has special meaning for those looking for real estate but can also mean everything to a forensicator looking for locational data. One of the most useful (or creepy if its yours) artifacts on an iOS device is the “Frequent Locations”. iOS will store, for a lengthy amount of time, locations a user’s device has been. This feature came out in iOS 7 and is used to record a pattern of activity for the user. Most often, a user will notice a message like one in the screenshot below – this is the ‘routined’ process in action on iOS devices. My phone “knew” I was likely headed to Arlington at this specific moment and ‘helped me’ by showing the traffic times. (Way to be creepy, iPhone!)

Users can review and clear their frequent location histories as shown in the next screenshot. 

Settings -> Privacy -> Location Services -> System Services -> Frequent Locations

Got an iPhone? Play along! Take a look at your frequent locations.

Don’t have locations? Location Services must be enabled.  iOS Location Services must be enabled and more specifically the option ‘Frequent Locations’ under ‘System Services’ must be turned on. There are many services that may be configured under ‘Location Services’ this is only one setting. The ‘Frequent Locations’ are enabled by default.

Locations are clustered by general areas. In the example above you can see I live in the Northern Virginia area and visit many cities in and around the area. These clusters also include the number of sub-locations and general timestamps. You can select one of these areas and get a more detailed street-level overview. This level shows that there are a number of ‘visits’ for each location, and again – a general day-based timestamp.

If we keep selecting down, we can get specific visit and timing information. Each location can have multiple visits. (Note the ‘N Nelson St.’ example with 49 visits above.) The example below shows one visit on December 13th at around 6:30 – 6:45pm, however the visit radius (shown in blue) can of course be very specific or more general depending on GPS availability. (I believe this was a visit to Whole Foods.)

Going back to the top – I recently traveled to San Francisco to teach my SANS FOR518 course and have locations from there in the broader ‘United States’ cluster history. Since I traveled outside of my normal Northern Virginia area, iOS made a new “cluster” of locations to include those in the San Francisco Area but also some in the NoVA area as well! I’m sure there is an algorithm somewhere to determine this – but sometimes it just doesn’t make sense how it is organized.

The Script:

Available Here

I wanted to study these artifacts in more detail as they have obvious benefit to the forensics community. These ‘frequent locations’ are stored in two binary plists in the following location:

/private/var/mobile/Library/Caches/com.apple.routined/

  • StateModel1.archive
  • StateModel2.archive

(Note: These files are protected and are only accessible in a physical dump or by physical (jailbroken) access to a device. You will not find these files in an iCloud/iTunes backup or a logical dump by forensic tools.)

I was elated to find them in a common file format, however the binary plist format used is the NSKeyedArchiver format which is less than human analyst friendly. (More on these types of binary plists in a future blog article.)

I needed to write a script to truly begin to understand how and why these entries get created as well as put the plist data into context. Enter, the script.

Big caveat here. I’m the first to tell anyone that I’m not a programmer. In fact – I’ve been avoiding it like the plague for the longest time (I had a not-so-fun experience with it in school – mind you I loved my scripting course.) I have come to realize that as a forensic analyst some times you just have to suck it up and script something to make your life easier – this is one of those times.

I decided to learn and script in Python since there were many forensic-based tools and packages that already use Python – it seems pretty industry standard. I’ve also noticed that I keep saying to myself (and my students) that this would be so much better or easier if there was a tool to do this or to do that! Sometimes you just gotta do it yourself if you want it done! So I will be trying to contribute as needed to the Mac forensics community – I foresee this being the first of many scripts.

The dump_freq_locs.py script takes in the StateModel#.archive files (where # is 1 or 2).

The script also has two dependencies, hexdump.py and ccl_bplist.py. These files can be installed or just simply placed in the same directory you are running the dump_freq_locs.py script from. (Installation on OS X 10.11 systems are limited thanks to SIP.)

The script parses the StateModel files and dumps the information to standard output so you will likely want to redirect to a file for later analysis.

$ python dump_freq_locs.py StateModel1.archive > SM1_parsed.txt

The script output contains three sections some of which I can describe here, but as stated above – I don’t know the significance of some of these items….yet. Parsing is the first step toward understanding.

  • Metadata – Contains metadata about the file (eg: when it was last updated, last location information, as well as other timestamps.)
  • RTVisitMonitor Data – Location information, timestamps, edge detection, outlier, last visit, and LOI (Location of Interest?) information.
  • Location Data – The ‘meat’ of plist file. Each location entry has the following:
    • A timestamp of when it was last updated (not necessarily visited)
    • Hexdump output of the location BLOB data containing reverse geolocation data
    • Hex Output – so you can input it into your favorite hex editor for additional analysis
    • Location Data –  The decimal latitude and longitude as well as ‘confidence’ and ‘uncertainty’ (I’m assuming these have something to do with how big the blue radius is) and the last update timestamp.
    • Visits – Entry and Exit timestamps for each visit.
    • Transition Data – When the device was ‘out of range’ of a location. The ‘Motion Activity’ may have something to do with if the device was in a vehicle or in a persons pocket while walking. Looking at my own data a value of 0 seems to be when I’m walking and a value of 4 seems to be when I’m in a car. Not sure what value 2 is yet.

This script was tested of a variety of different devices including iPhones and iPads on different iOS’s from iOS 7-9. As usual the format of these StateModel files has changed slightly from version to version. If you see I’m missing certain information from the parsed output please let me know! On that same note, if I’m not doing something Pythonically correct or weird – let me know too! I’m new to this Python/scripting thing.

I plan on adding future capabilities to this script, like CSV and KML outputs. If you have specific output format needs – let me know, I’ll take them into consideration!

Enjoy!

Sample script output is available here.

Continue reading Parsing iOS “Frequent Locations”

Parsing iOS “Frequent Locations”

The Artifact:

The phrase “Location, Location, Location” has special meaning for those looking for real estate but can also mean everything to a forensicator looking for locational data. One of the most useful (or creepy if its yours) artifacts on an iOS device is the “Frequent Locations”. iOS will store, for a lengthy amount of time, locations a user’s device has been. This feature came out in iOS 7 and is used to record a pattern of activity for the user. Most often, a user will notice a message like one in the screenshot below – this is the ‘routined’ process in action on iOS devices. My phone “knew” I was likely headed to Arlington at this specific moment and ‘helped me’ by showing the traffic times. (Way to be creepy, iPhone!)

Users can review and clear their frequent location histories as shown in the next screenshot. 

Settings -> Privacy -> Location Services -> System Services -> Frequent Locations

Got an iPhone? Play along! Take a look at your frequent locations.

Don’t have locations? Location Services must be enabled.  iOS Location Services must be enabled and more specifically the option ‘Frequent Locations’ under ‘System Services’ must be turned on. There are many services that may be configured under ‘Location Services’ this is only one setting. The ‘Frequent Locations’ are enabled by default.

Locations are clustered by general areas. In the example above you can see I live in the Northern Virginia area and visit many cities in and around the area. These clusters also include the number of sub-locations and general timestamps. You can select one of these areas and get a more detailed street-level overview. This level shows that there are a number of ‘visits’ for each location, and again – a general day-based timestamp.

If we keep selecting down, we can get specific visit and timing information. Each location can have multiple visits. (Note the ‘N Nelson St.’ example with 49 visits above.) The example below shows one visit on December 13th at around 6:30 – 6:45pm, however the visit radius (shown in blue) can of course be very specific or more general depending on GPS availability. (I believe this was a visit to Whole Foods.)

Going back to the top – I recently traveled to San Francisco to teach my SANS FOR518 course and have locations from there in the broader ‘United States’ cluster history. Since I traveled outside of my normal Northern Virginia area, iOS made a new “cluster” of locations to include those in the San Francisco Area but also some in the NoVA area as well! I’m sure there is an algorithm somewhere to determine this – but sometimes it just doesn’t make sense how it is organized.

The Script:

Available Here

I wanted to study these artifacts in more detail as they have obvious benefit to the forensics community. These ‘frequent locations’ are stored in two binary plists in the following location:

/private/var/mobile/Library/Caches/com.apple.routined/

  • StateModel1.archive
  • StateModel2.archive

(Note: These files are protected and are only accessible in a physical dump or by physical (jailbroken) access to a device. You will not find these files in an iCloud/iTunes backup or a logical dump by forensic tools.)

I was elated to find them in a common file format, however the binary plist format used is the NSKeyedArchiver format which is less than human analyst friendly. (More on these types of binary plists in a future blog article.)

I needed to write a script to truly begin to understand how and why these entries get created as well as put the plist data into context. Enter, the script.

Big caveat here. I’m the first to tell anyone that I’m not a programmer. In fact – I’ve been avoiding it like the plague for the longest time (I had a not-so-fun experience with it in school – mind you I loved my scripting course.) I have come to realize that as a forensic analyst some times you just have to suck it up and script something to make your life easier – this is one of those times.

I decided to learn and script in Python since there were many forensic-based tools and packages that already use Python – it seems pretty industry standard. I’ve also noticed that I keep saying to myself (and my students) that this would be so much better or easier if there was a tool to do this or to do that! Sometimes you just gotta do it yourself if you want it done! So I will be trying to contribute as needed to the Mac forensics community – I foresee this being the first of many scripts.

The dump_freq_locs.py script takes in the StateModel#.archive files (where # is 1 or 2).

The script also has two dependencies, hexdump.py and ccl_bplist.py. These files can be installed or just simply placed in the same directory you are running the dump_freq_locs.py script from. (Installation on OS X 10.11 systems are limited thanks to SIP.)

The script parses the StateModel files and dumps the information to standard output so you will likely want to redirect to a file for later analysis.

$ python dump_freq_locs.py StateModel1.archive > SM1_parsed.txt

The script output contains three sections some of which I can describe here, but as stated above – I don’t know the significance of some of these items….yet. Parsing is the first step toward understanding.

  • Metadata – Contains metadata about the file (eg: when it was last updated, last location information, as well as other timestamps.)
  • RTVisitMonitor Data – Location information, timestamps, edge detection, outlier, last visit, and LOI (Location of Interest?) information.
  • Location Data – The ‘meat’ of plist file. Each location entry has the following:
    • A timestamp of when it was last updated (not necessarily visited)
    • Hexdump output of the location BLOB data containing reverse geolocation data
    • Hex Output – so you can input it into your favorite hex editor for additional analysis
    • Location Data –  The decimal latitude and longitude as well as ‘confidence’ and ‘uncertainty’ (I’m assuming these have something to do with how big the blue radius is) and the last update timestamp.
    • Visits – Entry and Exit timestamps for each visit.
    • Transition Data – When the device was ‘out of range’ of a location. The ‘Motion Activity’ may have something to do with if the device was in a vehicle or in a persons pocket while walking. Looking at my own data a value of 0 seems to be when I’m walking and a value of 4 seems to be when I’m in a car. Not sure what value 2 is yet.

This script was tested of a variety of different devices including iPhones and iPads on different iOS’s from iOS 7-9. As usual the format of these StateModel files has changed slightly from version to version. If you see I’m missing certain information from the parsed output please let me know! On that same note, if I’m not doing something Pythonically correct or weird – let me know too! I’m new to this Python/scripting thing.

I plan on adding future capabilities to this script, like CSV and KML outputs. If you have specific output format needs – let me know, I’ll take them into consideration!

Enjoy!

Sample script output is available here.

Continue reading Parsing iOS “Frequent Locations”