Not populating all values for tcp_flags about logstash-pfsense

[patrickjennings]

The match on tcp_flags is on the word boundaries. So it should match all of RA, FA, FPA, etc. The fact that you are seeing RA and FA, to me, seems like it's not an issue on the ELK side but on the pfSense side.

Are you sending all syslog events in pfSense or just a subset? Are you seeing any _grokparsefailure tagged messages in your indices?

[jvedman]

So, I am sending all of my traffic to logs, and I am not seeing _grokparsefailure tags on any of the traffic in question. I see the data (in the attached file, they are all PA or FPA) in the data message where it is expected to be, it just doesn't get tagged. But I'm with you in wondering if this is something weird in how pfSense is translating the actual data into human-readable format (maybe there's an ascii issue?) since the grok is for a pretty straight-forward word regex. I'll pursue with them as well.
All Clients - Outbound Blocked Traffic (pfSense).txt

Thank you, Patrick. Appreciate your attention!

[jvedman]

Oh, and because I'm curious, I'm wondering if you're seeing the same thing. If you want to check your own logs, look specifically for blocked outbound TCP traffic on your LAN interface(s).

I'm trying to parse out the causes for outbound traffic being blocked on the firewalls I manage. There are lots of possible reasons and it would be a lot easier if I could weed out based on tcp_flags... but... ;-)

[patrickjennings]

The data I have access to right now is from over a year ago. But I do see some documents for FA, RA, FPA, and SA.

I would be curious to see whether you can capture some raw syslog traffic from pfSense? Either by setting up a secondary output in logstash or tcpdump could be an option.

[jvedman]

Yes, I'm sure I can do that. It might take me a bit, but I'll get it as quickly as I can manage.