_grokparsefailure on some lines about logstash-pfsense

[patrickjennings]

The log lines that you provided are missing the fields in the PFSENSE_PROTOCOL_DATA grok pattern.

You could make these optional. In 11-pfsense.conf:

match => [ "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IP_SPECIFIC_DATA}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}?", "message", "%{PFSENSE_LOG_DATA}%{PFSENSE_IPv4_SPECIFIC_DATA_ECN}%{PFSENSE_IP_DATA}%{PFSENSE_PROTOCOL_DATA}?" ]

But you will have empty src and dest port and data length fields.

What version of pfsense are you running?

[a1466d44-d3dc-4c0b-90c7-315b088731d7]

Thank you for your fast reply
I'm on 2.4.1-RELEASE (amd64) built on Sun Oct 22 17:29:23 CDT 2017 FreeBSD 11.1-RELEASE-p2

PS: how are you debugging the grok pattern?
The online GROK debugger where not very helpfull :)

[patrickjennings]

This service works best for me: http://grokconstructor.appspot.com/do/match.

In pfsense under Status / System Logs / Firewall, are you seeing any source or destination ports attached to the log entrees?

[a1466d44-d3dc-4c0b-90c7-315b088731d7]

I dont have these exact entries anymore, but yes all the log entries have a source and destination.
But i think this issue can be closed, as your updated GROK Pattern works perfekt ;)
thx again