prajitdas / androidanalytics

System call analysis and Google Play Store data and app permission analysis.

Hi there 👋

androidanalytics's Issues

Add logging functionality to syscall analysis

Extract permissions from Play Store

Directly extract the permissions from the Google Play Store instead of using the apk decompilation technique.

HTTP Error 404

It would seem that the data returned by the query
SELECT * FROM appurls WHERE parsed = 0
may contain a url which does not have a corresponding app in the Google Play Store. This is happening, perhaps because the developer pulled the app or it was removed by Google for some reason. In this case we should remove the url from our own database and continue getting data for other apps. The exception Traceback for the call can be seen below. Fix this!
Error is in code for both getting data and urls in the crawlURLs.py

Traceback (most recent call last):
File "./crawlURLs.py", line 311, in
sys.exit(main(sys.argv))
File "./crawlURLs.py", line 306, in main
doTask(sys.argv[1])
File "./crawlURLs.py", line 294, in doTask
getURLsForParsingAppData(dbHandle) # Extract app data
File "./crawlURLs.py", line 283, in getURLsForParsingAppData
extractAppDataAndStore(dbHandle,row[1])
File "./crawlURLs.py", line 171, in extractAppDataAndStore
page = urllib2.urlopen(req).read()
File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 406, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 519, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 444, in error
return self._call_chain(_args)
File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
result = func(_args)
File "/usr/lib/python2.7/urllib2.py", line 527, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 404: Not Found

Use config files for DB

DB config is hardcoded in downloadAPKs.py and crawlURLs.py. Use config files instead.

Complete permission metadata changes in collection folder

Fix the permission extracted bug

The permission extracted column should be updated when the permissions for an app has been extracted. This is not necessarily a bug if we want to update permissions in the future but in order to extract data from the sites we are getting blocked so we need to maybe mark extracted apps so that we can reduce repetitions.

Fix the permission extracted bug in apkDecompile

The bug was partially fixed in #7 and commit 87fb0d8 but the apkDecompile.py code still isn't updating perm_extracted in appurls table

Complete app description extraction and topic generation implementation

Change Request 1: Create developer country information

Add a column to the developer table called developer country. Populate that using a separate script. Use the developer address information whenever possible or get the information from DBpedia or other sources.

The Playdrone data collection for app permissions are blocking due to a MySQL error

In the JSON for com.iqmobile.iqburpingphone.json we found a permission denoted as: "'android.permission.INTERNET'". So, we have to sanitize the data now!!!

Wrapper for the API

A wrapper code for the individual APIs do not exist right now. Such a wrapper would make using this API a lot easier. There are plans to do that eventually. At the moment the individual modules can be used to accomplish the necessary tasks.

Database change for Developer

Change developer country column to developer location. The column doesn't represent the developer country but represents the address if it's there on the Google Play Store website.