ssh matching not working about ndpi-netfilter

[adrienb4]

I have the same problem. I think you use ssh in version 2 and the filter work with version 1. If you want you can open an issue ntop/nDPI project.

[ziyilu]

Thanks, adrienb4.

I just tried http, does not work either.
iptables -A FORWARD -m ndpi --http -j DROP

The config below work...
iptables -A FORWARD -p tcp --dport 80 -j DROP

Have you enabled ndpi & iptables for live network? and which protocols you succeed?

Thanks.

[ziyilu]

I made another test, using ndpiReader (under nDPI/example), it can correctly detect the http/ssh, I think sth wrong with the ndpi+netfilter, as the ndpiReader itself worked correctly.

[adrienb4]

In deed http not matching :/

[adrienb4]

I test on my router this afternoon (french hour)

[kong156]

Try to match ssh against both incoming and outgoing streams... this way should work. I don't know if this is normal.
Also, HTTP matching is working well on my ubuntu machine - 4.2.0-18-generic #22-Ubuntu SMP x86_64 GNU/Linux with the latest sources.

[kong156]

Also consider to use nDPI matching/marking into PREROUTING/POSTROUTING chains and then drop into FORWARDING chain if needed.

[kong156]

Looking deeper into this issue, i got this working, not on FORWARDING chain as expected, but with both INPUT and OUTPUT rules set as follows:

iptables -nvL
Chain INPUT (policy ACCEPT 152 packets, 12828 bytes)
pkts bytes target prot opt in out source destination
3 768 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 protocol SSH

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 120 packets, 10073 bytes)
pkts bytes target prot opt in out source destination
2 144 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 protocol SSH

This means that for a match, you need to complete a ssh handshake which implies traffic inspection for both directions. In this particular situation connmark isn't needed because it's done internally, i think.
The previous assumption may be valid also for the FORWARDING chain, but as i said earlier you have to use ndpi matching alongside (not sure because i couldn't test this one) connmark rules on PREROUTING/POSTROUTING chains and DROP those connections on FORWARDING chain.
You'll find an example of how to do protocol matching by reviewing an older and closed issue with number #8.

[ziyilu]

Thanks, kong156.

I tried to add following rule in iptables,
iptables -t mangle -I PREROUTING -m ndpi --dpi_check
iptables -t mangle -I POSTROUTING -m ndpi --dpi_check

Still not working for both ssh and http matching. while ntop finished the job perfectly to recognize both applications.

I guess ntop can get full flow of applications and iptables cannot?

[kong156]

All i can say it's working on INPUT and OUTPUT chains. I didn't test it on FORWARDING chain, but you can check what works by marking those packets that are supposed to match SSH and HTTP at PREROUTING/POSTROUTING level and accepting those marks at FORWARDING level.

*mangle
-A PREROUTING -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A POSTROUTING -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A POSTROUTING -m mark ! --mark 0x0 -j ACCEPT
-A POSTROUTING -m mark --mark 0x0 -m ndpi --ssh -j MARK --set-xmark 0x1/0xffffffff
-A POSTROUTING -m mark --mark 0x0 -m ndpi --http -j MARK --set-xmark 0x2/0xffffffff
-A POSTROUTING -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
COMMIT
*filter
-A FORWARDING -m mark --mark 0x1 -j ACCEPT
-A FORWARDING -m mark --mark 0x2 -j ACCEPT
COMMIT

[kong156]

I've just tested with FORWARD chain and it's also working. Check this iptables config:

*mangle
-A PREROUTING -m ndpi--dpi_check
-A POSTROUTING -m ndpi--dpi_check
*filter
-A FORWARD -m ndpi--http -j ACCEPT
-A FORWARD -m ndpi--ssh -j ACCEPT

These are the counters after doing some tests with ssh and http:

Chain FORWARD (policy ACCEPT 216 packets, 180K bytes)
pkts bytes target prot opt in out source destination
145 524K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 protocol HTTP
44 11024 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 protocol SSH

So, basically the matching is functioning at least for this two protocols.

[kong156]

Hi Humberto, what's wrong with iptables-save because it's eating some spaces when DPI rules comes in (see above)?

[ziyilu]

Thanks, Kong156.

I figured out the reason. It is due to I add rule below as the 1st rule for FORWARD,
iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

This causes the packets matches the session and not go further to the iptable. After I move the rule above after ndpi matching, the ndpi works!

However, I do have another question:
assume the default rule for some chain is DROP, how can I implement the whitelist for ndpi?
e.g.
iptables -A my-chain -m ndpi --http -j ACCEPT
iptables -A my-chain -j DROP

the ndpi engine has to learn the application after couple msg back and forth, however, due to the default drop, the msg cannot continue and the dpi engine will have no chance to learn the http application, any suggestion for this?

Thanks.

[kong156]

As you know already, one way is to blacklist the traffic you don't want to pass, otherwise I don't know what you can do with ndpi-netfilter only. There is no transient state which ndpi reports, but you can do it outside iptables by dropping/shaping all unidentified/unmarked taffic using qos policers/shapers. But you still have to let all the traffic pass through your chain. That's not what you want, but maybe it helps somehow :(

[kong156]

ziyilu, you should open a new issue and ask betolj if he can do something about this.

For this particular situation it would be nice to have a layer 7 "incomplete" state for those established tcp or udp connections which are in progress to be identified at layer 7, so they can be accepted till layer 7 state turns into "complete".

[betolj]

Fix applied to iptables-save output.

[kong156]

thanks alot.