molu8bits / modsecurity-filebeat-kibana Goto Github PK

filebeat -e

2020-07-16T17:50:34.138+0900 ERROR [reload] cfgfile/list.go:96 Error creating runner from config: Error getting config for fileset modsecurity2/audit: Error interpreting the template of the input: template: text:7:8: executing "text" at <.tags>: map has no entry for key "tags"

Why do I get an error when running filebeat as above?

My filebeat configuration
Elasticsearch 7.8.0
Kibana 7.8.0

./filebeat modules list
Enabled:
modsecurity2

cat modsecurity.yml

• module: modsecurity2
  audit:
  enabled: true

  var.paths: ["/appstore/logs/modsec_apacheA.*"]

filebeat/module/modsecurity2 The folder exists, but there are no modules(modsecurity2).

Where can I download this module?

I get the above error message when deploying your Kibana dashboard with Filebeat according to README.md.

ES/Kibana Version: 8.5.2
Filebeat Version: 8.5.2

I have deployed the module, the module configuration and the nginx modsecurity configuration.

What can be wrong?

Best regards,
rforberger

sorry
I couldn't find the menu to delete

Hi Firts I want to thank you for this Dashboard, follow your instructions and found a couple if issues in my Kibana, I set Modsec2 and Suricata, Suricata is working fine but Modsec2 have disply problems. Any help will be highly appreciatted.

Shard Failures
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 1,
"skipped": 1,
"failed": 1,
"failures": [
{
"shard": 0,
"index": "filebeat-7.8.1-2020.07.28-000001",
"node": "avET7gESR7isL8CSrge11w",
"reason": {
"type": "illegal_state_exception",
"reason": "Rewrite first"
}
}
]
},
"hits": {
"total": 0,
"max_score": 0,
"hits": []
}
}

Request

{
"aggs": {
"3": {
"date_histogram": {
"field": "@timestamp",
"fixed_interval": "30m",
"time_zone": "America/Mexico_City",
"min_doc_count": 1
},
"aggs": {
"4": {
"filters": {
"filters": {
"Intercepted": {
"bool": {
"must": [
{
"query_string": {
"query": "exists:modsecurity2 AND exists:modsecurity2.audit.audit_data.action.intercepted",
"analyze_wildcard": true,
"time_zone": "America/Mexico_City"
}
}
],
"filter": [],
"should": [],
"must_not": []
}
},
"Passed": {
"bool": {
"must": [
{
"query_string": {
"query": "exists:modsecurity2 AND NOT exists:modsecurity2.audit.audit_data.action.intercepted",
"analyze_wildcard": true,
"time_zone": "America/Mexico_City"
}
}
],
"filter": [],
"should": [],
"must_not": []
}
}
}
}
}
}
}
},
"size": 0,
"stored_fields": [
"*"
],
"script_fields": {},
"docvalue_fields": [
{
"field": "@timestamp",
"format": "date_time"
},
{
"field": "event.created",
"format": "date_time"
},
{
"field": "event.end",
"format": "date_time"
},
{
"field": "event.start",
"format": "date_time"
},
{
"field": "file.accessed",
"format": "date_time"
},
{
"field": "file.created",
"format": "date_time"
},
{
"field": "file.ctime",
"format": "date_time"
},
{
"field": "file.mtime",
"format": "date_time"
},
{
"field": "kafka.block_timestamp",
"format": "date_time"
},
{
"field": "netflow.collection_time_milliseconds",
"format": "date_time"
},
{
"field": "netflow.exporter.timestamp",
"format": "date_time"
},
{
"field": "netflow.flow_end_microseconds",
"format": "date_time"
},
{
"field": "netflow.flow_end_milliseconds",
"format": "date_time"
},
{
"field": "netflow.flow_end_nanoseconds",
"format": "date_time"
},
{
"field": "netflow.flow_end_seconds",
"format": "date_time"
},
{
"field": "netflow.flow_start_microseconds",
"format": "date_time"
},
{
"field": "netflow.flow_start_milliseconds",
"format": "date_time"
},
{
"field": "netflow.flow_start_nanoseconds",
"format": "date_time"
},
{
"field": "netflow.flow_start_seconds",
"format": "date_time"
},
{
"field": "netflow.max_export_seconds",
"format": "date_time"
},
{
"field": "netflow.max_flow_end_microseconds",
"format": "date_time"
},
{
"field": "netflow.max_flow_end_milliseconds",
"format": "date_time"
},
{
"field": "netflow.max_flow_end_nanoseconds",
"format": "date_time"
},
{
"field": "netflow.max_flow_end_seconds",
"format": "date_time"
},
{
"field": "netflow.min_export_seconds",
"format": "date_time"
},
{
"field": "netflow.min_flow_start_microseconds",
"format": "date_time"
},
{
"field": "netflow.min_flow_start_milliseconds",
"format": "date_time"
},
{
"field": "netflow.min_flow_start_nanoseconds",
"format": "date_time"
},
{
"field": "netflow.min_flow_start_seconds",
"format": "date_time"
},
{
"field": "netflow.monitoring_interval_end_milli_seconds",
"format": "date_time"
},
{
"field": "netflow.monitoring_interval_start_milli_seconds",
"format": "date_time"
},
{
"field": "netflow.observation_time_microseconds",
"format": "date_time"
},
{
"field": "netflow.observation_time_milliseconds",
"format": "date_time"
},
{
"field": "netflow.observation_time_nanoseconds",
"format": "date_time"
},
{
"field": "netflow.observation_time_seconds",
"format": "date_time"
},
{
"field": "netflow.system_init_time_milliseconds",
"format": "date_time"
},
{
"field": "process.start",
"format": "date_time"
},
{
"field": "suricata.eve.flow.end",
"format": "date_time"
},
{
"field": "suricata.eve.flow.start",
"format": "date_time"
},
{
"field": "suricata.eve.timestamp",
"format": "date_time"
},
{
"field": "suricata.eve.tls.notafter",
"format": "date_time"
},
{
"field": "suricata.eve.tls.notbefore",
"format": "date_time"
}
],
"_source": {
"excludes": []
},
"query": {
"bool": {
"must": [],
"filter": [
{
"match_all": {}
},
{
"range": {
"@timestamp": {
"gte": "2020-07-30T05:00:00.000Z",
"lte": "2020-07-31T04:59:59.999Z",
"format": "strict_date_optional_time"
}
}
}
],
"should": [],
"must_not": []
}
}
}

Response
{
"took": 10,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 1,
"skipped": 1,
"failed": 1,
"failures": [
{
"shard": 0,
"index": "filebeat-7.8.1-2020.07.28-000001",
"node": "avET7gESR7isL8CSrge11w",
"reason": {
"type": "illegal_state_exception",
"reason": "Rewrite first"
}
}
]
},
"hits": {
"total": 0,
"max_score": 0,
"hits": []
}
}

Modsecurity2_Overview.ndjson file was imported to kibana

What items should be changed after that?

I see five in kibana's Visualize item, but I see a warning saying "There is a problem with the stored object".

The index name in use is "log_modsecurity_photo"

Hi @molu8bits,
any predictions for posting the Modsecurity2_Overview.json file? Can I contribute something?

Hello i have done all the steps that you mention on the readme.
I receive something (i see this on discover kibana)

but it was like a json problem (log sent from my server (modsecurity_log)??