Kibana visualizations "Could not locate that index-pattern-field" about logstash-pfsense

[patrickjennings]

Have you created the index pattern in Kibana? The pattern to use is "logstash-pfsense-*". If so, do the src_ip.keyword and dest_port.keyword fields exist for the mapping?

[kabalman]

I had already created the index pattern logstash-pfsense-* but when I look in the fields, it doesn't hve src_ip.keyword or dest_port.keyword. Do I have to create these fields manually on the index pattern? Sorry I'm not very familiar with it, so would really appreciate some help.

[kabalman]

Ok - when I deleted the index and re-created it after I had imported the visualizations in Kibana, it created the index with additional fields, which included src_ip.keyword and dest_port.keyword. And the visualizations and dashboard work now.
So do I have to import the visualization first, so Kibana knows about the additional fields that the index requires? Before creating the index?

[patrickjennings]

The simplest steps would be to setup pfsense and logstash first and have events flow naturally into elasticsearch. This will create the index and fields automatically as they are inserted. It seems in your instance, the events that were pushed through the stack did not contain src_ip and dest_port data. Maybe pfsense was not pushing firewall events and only other syslog data at the time? Anyway, when you initially created the index pattern in kibana, it did not pick up those fields because they did not exist on the index.

Refreshing the index pattern fields in kibana is the correct solution. There is a refresh fields button in kibana that you can press at anytime to pick up any missing fields for the pattern.

[uniuuu]

Hi @patrickjennings

I have the same issue. Are you available to reproduce and check?
May it be due to different log format from pfsense ?