fabsx00 / python-joern

A python interface to joern (deprecated).

python-joern

Introduction

A thin python interface for joern and a set of useful utility traversals.

Dependencies:

• py2neo 2.0.7 (http://py2neo.org/2.0/)

Installation

$ sudo pip2 install git+git://github.com/fabsx00/python-joern.git

Example

The following is a simple sample script. It connects to the database and runs a gremlin traversal to retrieve all node with attribute 'functionName' set to 'main'.

from joern.all import JoernSteps

j = JoernSteps()

j.setGraphDbURL('http://localhost:7474/db/data/')

# j.addStepsDir('Use this to inject custom steps')

j.connectToDatabase()

res =  j.runGremlinQuery('g.idx("nodeIndex")[[functionName:"main"]]')

for r in res:
    print r

python-joern's Issues

Using Sideffect Variables via python-joern

I am trying to adopt your Taint-Style Description example for sprintf but am lost in the mix of python, groovy and germlin. Here is what I try to do:

funs = ['sprintf']
for f in funs:
    #find calls of bad function
    print "Calls to evil function %s:" % f
    calls = j.runGremlinQuery("getCallsTo('%s')" % f) 
    for c in calls:

        #params = j.runGremlinQuery("g.v(%d).callToArguments().filter{ it.childNum!='0' }.definitions().filter{it.type=='Parameter'}"%c._id)
        params = j.runGremlinQuery("g.v(%d).callToArguments().filter{ it.childNum!='0' }.sideEffect{ paramName = '.*' }.filter{ it.code.matches(paramName) }.unsanitized{ it.isCheck( paramName ) }.params( paramName )"%c._id)
        if set(params):
            print "call: %s " % c['code']
            print "\tCan be reached from function parameters:"
            for p in params:
                print "\t\t%s" % p

However when I execute this script I get:

Traceback (most recent call last):
  File "my_query.py", line 23, in <module>
    params = j.runGremlinQuery("g.v(%d).callToArguments().filter{ it.childNum!='0' }.sideEffect{ paramName = '.*' }.filter{ it.code.matches(paramName) }.unsanitized{ it.isCheck( paramName ) }.params( paramName )"%c._id)
  File "/usr/local/lib/python2.7/dist-packages/joern-0.1-py2.7.egg/joern/all.py", line 44, in runGremlinQuery
    return self.gremlin.execute(finalQuery)
  File "/usr/local/lib/python2.7/dist-packages/py2neo/ext/gremlin/__init__.py", line 36, in execute
    response = self.resources["execute_script"].post({"script": script})
  File "/usr/local/lib/python2.7/dist-packages/py2neo/core.py", line 288, in post
    raise_from(self.error_class(message, **content), error)
  File "/usr/local/lib/python2.7/dist-packages/py2neo/util.py", line 215, in raise_from
    raise exception
py2neo.error.BadInputException: javax.script.ScriptException: groovy.lang.MissingPropertyException: No such property: paramName for class: Script26

So it seems like groovy tries to do a replacement here looking for paramName, which is obviously wrong.
How do I need to escape this so groovy stays out of my gremlin code?

Can't run scripts

I followed installing instructions but still there's a problem with requests via python (same issue with joern-tools

From sample script

#!/bin/python2

from joern.all import JoernSteps

j = JoernSteps()
j.setGraphDbURL('http://localhost:7474/db/data/')

j.connectToDatabase()

res =  j.runGremlinQuery('getFunctionsByName("main")')

for r in res: print r

Following backtrace:

Traceback (most recent call last):
  File "./launch.py", line 17, in <module>
    res =  j.runGremlinQuery('getFunctionsByName("main")')
  File "/usr/lib/python2.7/site-packages/joern-0.1-py2.7.egg/joern/all.py", line 44, in runGremlinQuery
    return self.gremlin.execute(finalQuery)
  File "/usr/lib/python2.7/site-packages/py2neo/ext/gremlin/__init__.py", line 36, in execute
    response = self.resources["execute_script"].post({"script": script})
  File "/usr/lib/python2.7/site-packages/py2neo/core.py", line 316, in post
    raise_from(self.error_class(message, **content), error)
  File "/usr/lib/python2.7/site-packages/py2neo/util.py", line 235, in raise_from
    raise exception
py2neo.error.NoClassDefFoundError: javax/transaction/SystemException

There's only one related link in google. The possible issue there is non-installed Gremlin-plugin for Neo4j server. But, well, it's there:

$ cat /var/log/neo4j/console.log | grep Gremlin | tail -n 2
2016-06-07 13:10:25.060+0300 INFO  Loaded server plugin "GremlinPlugin"
2016-06-07 13:10:25.061+0300 INFO    GraphDatabaseService.execute_script: execute a Gremlin script with 'g' set to the Neo4j2Graph and 'results' containing the results. Only results of one object type is supported.

Tried both: compiling&installing gremlin from source and installing from snapshot from docs

Additional note: I use Arch Linux and grub neo4j from aur and it installs the server separately (/usr/lib/neo4j, /usr/share/java/neo4j, /usr/share/neo4j, etc) so that can be the problem

can't run traversal "unsanitized"

It confuses me so much. I have installed the latest version of python joern, and I use the following traversal
getArguments('memcpy', '2')
.sideEffect{ paramName = '.len.' }
.filter{ it.code.matches(paramName) }
.unsanitized{ it.isCheck( paramName ) }
.params( paramName )
It is showed on https://joern.readthedocs.io/en/latest/querying.html as an example, but I failed.
I have checked the code over and over, but I have no clue.
Please tell me how to solve it!

C code/graphs aren't present in the repo for unit test cases

I see you have unit tests but I don't see the test cases anywhere. If you checked them in I would love to add tests for my steps as I go.

Need to update joern-lookup for being compatible with py2neo v3

So I tried to use joern-lookup and I had a failure with the dependencies. I was with py2neo 3, so I installed py2neo v2 and it solved the issue.
We need to update python-joern or joern-tools for being compatible with py2neo 3.

LookupError: No plugin named 'GremlinPlugin' found on graph

The Gremlin plug-in has been installed, but this error still occurs when running the project

Missing files

py2neo 2.0.7 no longer available

The url for py2neo 2.0.7 https://py2neo.readthedocs.io/en/latest/2.0/ is no longer available. And it seems that py2neo does not provide the 2.0 version now.

Usage of wildcards to find calls

I was trying to use python-joern to query the graph for dangerous methods. As suggested in the documentation.

However I found the methods getCallsTo and queryNodeIndex do not behave as expected. I found that the only way to get all calls to functions including 'cpy' was to use getNodesWithTypeAndCode.:

from joern.all import JoernSteps

j = JoernSteps()
j.setGraphDbURL("http://localhost:7474/db/data/")
j.connectToDatabase()
#res = j.runGremlinQuery("getCallsTo('strcpy')") # WORKS!
#res = j.runGremlinQuery("getCallsTo('*cpy*')") # DOES NOT
#res = j.runGremlinQuery('getNodesWithTypeAndCode("Callee","strcpy")') #WORKS
res = j.runGremlinQuery('getNodesWithTypeAndCode("Callee","*cpy*")') #WORKS
#res = j.runGremlinQuery('queryNodeIndex("type:Callee AND name:*cpy*")') # DOES NOT
#res = j.runGremlinQuery('queryNodeIndex("type:Callee AND name:strcpy")') # DOES NOT
#res = j.runGremlinQuery('queryNodeIndex("*")') # DOES NOT, causes NPE
print(res)
for r in res:
    print r

Looking into the groovy code I found that you explicitly filter out the wildcard operator in getCallsTo.

Could you elaborate on that?

Check for AssignmentExpr uses

Some AST nodes (AssignmentExpr in particular) have been renamed during the development of the php support for joern (see octopus-platform/joern#86). Before we merge the php-support, we need to ensure that none of the steps rely on old node names.