client9 / libinjection Goto Github PK

got some samples with unicode U+FFFD / utf-8 EFBFDB as a whitespace character

Not sure what database , encoding does this.

or if the input into the form was invalid.

http://www.utf8-chartable.de/unicode-utf8-table.pl?start=65280

perhaps interesting (in japanese)
http://itpro.nikkeibp.co.jp/article/COLUMN/20090525/330611/

if make test_unit has need fails, jenkins needs to complain.

I can manually generate tests from sqlmap.. but I need to automate them.

In a simple implementation, libinjection could be used on all HTML FORM POSTs, which might include very long binary data.

To prevent long scanning and parsing times, we should either

• reject inputs > 65k (or some other configurable limit)
• abort parsing if we go too far

Unfortunately, MySQL allows 1G SQL statement!
http://dev.mysql.com/doc/refman/5.0/en/packet-too-large.html

Looks like default is 32MB.

as mentioned in
http://www.client9.com/2012/09/06/libinjection-new-techniques-in-detecting-sqli-attacks/
http://www.client9.com/2012/07/27/new-techniques-in-sql-obfuscation/

MS SQL Server has a money type, that uses the "$" and can contain commas. Should be easy to add to the parser.

Patch: qerub@836a8fa

select $foo$ blah $bar$ foo $bar$ ending $foo$;
blah foo ending

if a word / string starts with "U&" just stop.

see
4.1.2.3. String Constants with Unicode Escapes

http://www.postgresql.org/docs/9.1/static/sql-syntax-lexical.html

Soon the changes will slow down, and libinjection will need to be managed more with releases and release notes.

One attack against libinjection is cases where you can pad out the start using comma separated values.

1,2,3,4,5,6 UNION ALL ...

comma separated values should be folded into one value

1 UNION ALL ...

Thanks for creating the SQLi detection library!
This post is part question, part evasion report.

We set up a deliberately vulnerable web application along with a simple database*. The following string, when used as an input to a vulnerable form field, triggers a SQL injection. The result of the injection is that the query gets executed, such that the username for [email protected] is updated to hold the site admin's password:

','','default.jpg'); UPDATE [User] SET Name=(SELECT Password FROM [User] WHERE Email='[email protected]') WHERE Email='[email protected]'; --

As evident, the string expects to be embedded in a single quote expression. The first part of the string is used to close off the original expression, the second part acts as the payload, and the final part (the double dash) is used to ensure that the suffix of the original expression is ignored. Oddly, while the string triggers an injection fairly straightforwardly, it is not detected by libinjection. Odder still is that by using a simplified payload query, like below, libinjection does detect the injection.

','','default.jpg'); SELECT Password FROM [User] WHERE Email='[email protected]'; --

Before reporting this as a false negative, we double-checked to make sure that we were feeding the diagnostics page with the correct string and that it contained no unicode quotes. The string is not short and since we did not modify the query string directly, we do not suspect any URL-decoding issues to be at fault.

Could this be an evasion? Could libinjection be having trouble with the compound statement? Or are we doing something wrong? Any thoughts on this behavior would be appreciated. If additional details are needed, please do not hesitate to ask.

Thank you,
Neer Roggel

*Installation details:
Web server: Microsoft IIS v7.5.7600.16385
DB: Microsoft SQL Server Express 10.50.1600.1 (with Advanced Services, 64-bit)

folding code enhanced to [1sn],[1sn] -> [1sn]

need unit tests.

currently the rules for "sos" fingerprints can't distinguish between:

1. foo" + "bar
2. foo "+" bar

They are semantically different. 1 is likely an attack but 2 is not.

Need to fix so these types of attacks/probes are detected by default... may cause slight increase in FP but that's ok.

Is there a recommended way to get the tokens after testing a string? I assume they are in the tokenvec of the sfilter? Would be great if they would get exposed in sqlparse.h

https://libinjection.client9.com/diagnostics?id=1%2C%281%29&type=fingerprints

https://libinjection.client9.com/diagnostics?id=Toronto%2C+ON%2C+Canada+-+%28YYZ%29&type=fingerprints

https://libinjection.client9.com/diagnostics?id=1%2C1--&type=fingerprints

https://libinjection.client9.com/diagnostics?id=%281%29-x&type=fingerprints

https://libinjection.client9.com/diagnostics?id=1+function+%281%29+&type=fingerprints

https://libinjection.client9.com/diagnostics?id=%27%2F1x-&type=fingerprints

https://libinjection.client9.com/diagnostics?id=x|x||1&type=fingerprints

https://libinjection.client9.com/diagnostics?id=%22%2F%22&type=fingerprints

https://libinjection.client9.com/diagnostics?id=select+%27and%27&type=fingerprints

https://libinjection.client9.com/diagnostics?id=x%2Fvoid%281%29&type=fingerprints

https://libinjection.client9.com/diagnostics?id=select+x+from+y+where&type=fingerprints

https://libinjection.client9.com/diagnostics?id=x%2F*&type=fingerprints

https://libinjection.client9.com/diagnostics?id=1x%28%28%28&type=fingerprints

https://libinjection.client9.com/diagnostics?id=1%29%2C%281&type=fingerprints

https://libinjection.client9.com/diagnostics?id=x%2C+%40x%2C+%40x&type=fingerprints

sqlparse_map.py is a data structure... in python

it could be converted to JSON or YAML to make life easier for porting.

"make test_unit" failed on my Ubuntu box.
Python version is 2.6.2

make test_unit
./test_driver.py ../tests/test-.txt
Traceback (most recent call last):
File "./test_driver.py", line 3, in
from subprocess import check_output
ImportError: cannot import name check_output
make: ** [test_unit] Error 1

Which version of Python do you use?
Thanks.

Microsoft's SQL parser does this

SELECT1; --> SELECT 1;
SELECT.1; --> SELECT .1;

so in a sqli like this:

'AND 1.-1LIKE.1 EXEC xp_cmdshell 'dir

libinjection gets confused by the LIKE.1 (thinks it's a single token of 'none' type).

As mentioned in:

http://www.slideshare.net/nickgsuperstar/libinjection-isecpartners
http://www.client9.com/2012/07/27/new-techniques-in-sql-obfuscation/

oracle has special fp types:

binary_[float|double]_[nan|infinity]

treat as numbers

unit tests for the sql parser are good.

Now we need test for the folder.

can you give readme?

ASP removes the "%" from query arguments when it's not a url escape sequence.

i.e.

u%nion --> union

Not clear what the right layer is here (or should handle this).

q'[char]

You can also use the following notation to define your own delimiter characters for the literal. You choose a character that is not present in the string, and then need not escape other single quotation marks inside the literal:

-- q'!...!' notation allows use of single quotes inside literal
string_var := q'!I'm a string, you're a string.!';
You can use delimiters [, {, <, and (, pair them with ], }, >, and ), pass a string literal representing a SQL statement to a subprogram, without doubling the quotation marks around 'INVALID' as follows:

func_call(q'[SELECT index_name FROM user_indexes
WHERE status ='INVALID']');
For NCHAR and NVARCHAR2 literals, use the prefix nq instead of q, as in the following example, where 00E0 represents the character é:

where_clause := nq'#WHERE COL_VALUE LIKE '%\00E9'#';

Some tokens behave different in different databses.

For example "IN" can be a keyword OR a function

'current_user can be a "value" or a function current_user()
select current_user
select current_user()

there are others. Right now I just handle the "IN" case. Need to document the others and to make a consistent system for handling them.

Recently found out I was using strchr in the string parsing code. This failed with strings that had an embedded null in them.

I switched them to memchr + length

Need test cases and valgrind to make sure I don't have off by 1 issues.

Apparently windows uses a C89 style...

mysql special

_CHARSETNAME'string'

single or double quote
_latin1'foo bar'
_latin"foo bar"

these are not valid
_latinthis doesn't work
_latin1'no'

probably just need to stop parsing and say "sqli" if found

Like to cause code changes

pgsql can do this:

select '1'::float8::float8::float8;
1

The "::" operator needs to handled.

• ./c/reader -x data/sqli-arithmetic_blind_sqli.txt data/sqli-arithmetic_variations.txt data/sqli-comparitiveprecomputation.txt data/sqli-edb-17934.txt data/sqli-forums.txt data/sqli-insert_attacks.txt data/sqli-isc_sans.txt data/sqli-misc.txt data/sqli-mysql-implicit.txt data/sqli-phpids.txt data/sqli-spiderlabs-201107.txt data/sqli-spiderlabs-201112.txt data/sqli-spiderlabs-201205.txt data/sqli-sqlmap-20130419.txt data/sqli-sqlmap.txt data/sqli-sqlmap_examples.txt data/sqli-themole.txt data/sqli-wordpress_rbarnett.txt
  SQLI : 84547
  SAFE : 5
  TOTAL : 84552
  [Cppcheck] Starting the cppcheck analysis.
  [Cppcheck] Processing 1 files with the pattern '**/sqli-positive.xml'.
  [Cppcheck] Parsing throws exceptions. javax.xml.bind.UnmarshalException
  • with linked exception:
    [org.xml.sax.SAXParseException; systemId: file:/var/lib/jenkins/workspace/libinjection-samples-positive/sqli-positive.xml; lineNumber: 5; columnNumber: 98; Element type "error" must be followed by either attribute specifications, ">" or "/>".]
    Build step 'Publish Cppcheck results' changed build result to FAILURE
    Build step 'Publish Cppcheck results' marked build as failure
    Finished: FAILURE

clang static analyzer is damn good!

code reads one byte past boundary in this loop.

for (ptr = end; ptr >= start; ptr--) {

(if ptr == start, the ptr-- goes before start of string)

scan-build: Using '/usr/local/bin/clang' for static analysis
/usr/local/bin/ccc-analyzer  -g -O3 -Wall -Wextra -Wcast-align -Wshadow -Wpointer-arith -Wcast-qual -Wstack-protector -D_FORTIFY_SOURCE=2 -O1 -o testdriver libinjection_sqli.c testdriver.c
libinjection_sqli.c:608:13: warning: Out of bound memory access (access exceeds upper limit of memory block)
        if (*ptr != '\\') {
            ^~~~
1 warning generated.
scan-build: 1 bugs found.

Need to track false positives... need to make sure changes don't cause massive FP problems.

'make test_negative'

cc -I. -I/root/libinjection/php/build -DPHP_ATOM_INC -I/root/libinjection/php/build/include -I/root/libinjection/php/build/main -I/root/libinjection/php/build -I/usr/local/include/php -I/usr/local/include/php/main -I/usr/local/include/php/TSRM -I/usr/local/include/php/Zend -I/usr/local/include/php/ext -I/usr/local/include/php/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /root/libinjection/php/build/libinjection_sqli.c -fPIC -DPIC -o .libs/libinjection_sqli.o
/root/libinjection/php/build/libinjection_sqli.c: In function 'flag2delim':
/root/libinjection/php/build/libinjection_sqli.c:82: error: 'FLAG_QUOTE_SINGLE' undeclared (first use in this function)
/root/libinjection/php/build/libinjection_sqli.c:82: error: (Each undeclared identifier is reported only once
/root/libinjection/php/build/libinjection_sqli.c:82: error: for each function it appears in.)
/root/libinjection/php/build/libinjection_sqli.c:84: error: 'FLAG_QUOTE_DOUBLE' undeclared (first use in this function)

full log https://gist.github.com/bizonix/7311865

well that's embarrassing.

Due to a misconfiguration, valgrind was not able to detect read-overruns. And there are read-overuns. Unlikely to cause a security problem, but may cause instability in application (crashes).

You can see the results now here:

http://jenkins.client9.com/job/libinjection-valgrind/57/valgrindResult/?
http://jenkins.client9.com/job/libinjection-valgrind/57/console
http://jenkins.client9.com/job/libinjection-valgrind/57/testReport/?

About 25 of 120 tests are failing.

I suspect it should be easy to fix. This ticket will track process.

http://www.arneswinnen.net/2013/09/automated-sql-injection-detection/

http://www.arneswinnen.net/wp-content/uploads/2013/09/extract-blind-sqlmap-payloads.sh_.zip
http://www.arneswinnen.net/wp-content/uploads/2013/09/Time-based-SQLi-payloads.zip

https://github.com/Kanatoko/libinjection-Java

If there are any problems ( ecpecially about LICENSE/Copying) please notify me.
Thanks!

found a const char ptr = strstr(cs + pos, "/");

in the code.

Maybe have problems with null inside the string.

In the latin1, ISO-8859-1, character 0xA0 is a 'non breaking space'

need to determine how databases deal with that... a token boundary OR just part of a word.

e.g.

is SELECT[[0xA0]]FOO

SELECT FOO (two words)

or

SELECT_FOO (one word with space in middle)

Crafted payload with single token value exceeding 32 bytes (e.g. comments) can lead libinjection into reading data out of stack boundary.

Functions in questions are: st_assign_char() and st_assign() where st->len should be guarded by MSIZE or LIBINJECTION_SQLI_TOKEN_SIZE.

Note: issue was raised to Nick Aug 2013, somehow the fix did not go into the latest release. This is for tracking purposes.

Is CURRENT_DATECURRENT_TIME on https://github.com/client9/libinjection/blob/master/c/sqlparse_map.py#L106 an accident or should it really be there?

In some IronBee rules we check all HTTP headers for SQLi. Typically you will see:

Accept: */*

This accept header value is detected as SQLi pattern "oc".

Both "left join" and "left outer" are defined in multikeywords.
On the other hand, "left" alone is defined in sql_keywords as {"LEFT", 'f'} .

It seems that currently both "left join" and "left outer" is tokenized as "fk", But is it an intended behavior?

I expected that in

../data/sqli-misc.txt 98 False 1o(o1 914 1001 RLIKE(-1)UNION SELECT 1,2,3,4 FROM CREDIT_CARDS

the "-1" would be folding into just "1", making the fingerprint more interesting

1010 RLIKE(1) UNION
1o(1)

which is covered in the fingerprint file.

nickg

Looks like I configured Jenkins incorrectly, and we had a regression:

data/sqli-forums.txt 6800 1c error 24/*
data/sqli-forums.txt 9282 1c error 417652--

Due to typo? strspn vs strcspn in C code.

nickg

End processing if we encounter a nested C-style comment.

/* foo /* bar */ crap */

Should say it's a sqli False positive rates should be very low.

select 1 where 2=1/* /*/ */ */ or 1=1--;

is valid on PostgreSQL(8.3.5) and MS SQL Server(2012).

Currently '1/* /*/ */ */ or 1=1--' is not treated as SQLi.
How should we do?

Very small SQLi hard to detect without false positives.

In revision 1.2.0 and earlier, the following SQLi is not detected:

1*1--
1*1/*

This is due to some checks to avoid having false positives.

Fix for #2 coming shortly. #1 is more problematic.

There is a array buffer overflow vulnerability in function parse_word of libinjection_sqli.c,if one keyword is more than 32,the sf->current->val[i] will be overflowed.Below is one new patch function.

static size_t parse_word(struct libinjection_sqli_state * sf)
{
char ch;
char delim;
size_t i;
const char cs = sf->s;
size_t pos = sf->pos;
size_t wlen = strlencspn(cs + pos, sf->slen - pos,
" {}<>:?=@!#~+-/&|^%(),';\t\n\v\f\r"\000");
size_t kwlen = wlen < LIBINJECTION_SQLI_TOKEN_SIZE ? wlen : (LIBINJECTION_SQLI_TOKEN_SIZE - 1);
st_assign(sf->current, TYPE_BAREWORD, pos, wlen, cs + pos);

/* now we need to look inside what we good for "." and "`"
 * and see if what is before is a keyword or not
 */
for (i =0; i < kwlen; ++i) {
    delim = sf->current->val[i];
    if (delim == '.' || delim == '`') {
        ch = sf->lookup(sf, LOOKUP_WORD, sf->current->val, i);
        if (ch != TYPE_NONE && ch != TYPE_BAREWORD) {
            /* needed for swig */
            st_clear(sf->current);
            /*
             * we got something like "SELECT.1"
             * or SELECT`column`
             */
            st_assign(sf->current, ch, pos, i, cs + pos);
            return pos + i;
        }
    }
}

/*
 * do normal lookup with word including '.'
 */
if (wlen < LIBINJECTION_SQLI_TOKEN_SIZE) {

    ch = sf->lookup(sf, LOOKUP_WORD, sf->current->val, wlen);
    if (ch == CHAR_NULL) {
        ch = TYPE_BAREWORD;
    }
    sf->current->type = ch;
}
return pos + wlen;

}

a sad day.

There is an infinite loop in some cases of using backquotes.

test and fix forthcoming..

…since it's rather implementation-specific and a little stop gap in porting.

(For reference: I discussed this with @client9 in a chat.)

FOO & FOO --> n
FOO FOO & FOO --> nnon

I can't understand why those two patterns show difference on tokenized 'FOO & FOO'.
Should latter tokenized to 'nn' ?

Thanks.