[Bug] Cortex Data > Splunk HEC event line breaks missing about splunk-apps

[paulmnguyen]

Hey @0xC0FFEEEE,

Was this working before and suddenly changed?

[0xC0FFEEEE]

I believe this has always been the case, our Obs team has been trying to get to the bottom of this since the integration was originally set up and have had an ongoing case raised with Splunk on this issue.

My fix for this is a bit hacky, I've cloned the sourcetype and set the following:
LINE_BREAKER: \}\}()
TRUNCATE: 999999

For reference the installed version on our Splunk cloud environment is currently v8.1.0

[0xC0FFEEEE]

I've identified another issue which is impacting the ability to perform field extractions and normalization. similar to #325. This could be due to the events being sent to the incorrect HTTP endpoint (as there are several to choose from in the Splunk dev docs covering HTTP inputs).

What should happen is that the top level key/value pairs in the JSON sent to Splunk should be interpreted as internal fields (.e.g. host, source, time), and the nested JSON under the event key interpreted as the actual event and displayed accordingly.

What is actually happening is the following, the Palo event data lives under the event key at the top level and therefore all of the Palo knowledge objects are not being applied as the fields are extracted as event.<field_name>.

Example of a correctly interpreted event sent to the https://http-inputs.splunkcloud.com/services/collector/event endpoint. Note that the JSON displayed is the nested JSON from the top level event key, this was sent to the HTTP input endpoint as follows:

{
  "event": {"Accept":"*/*","Accept-Encoding":"gzip,deflate,br","Accept-Language":"en-GB,en; q=0.5", <snip>},
  "source": "mysource",
  "host": "myhost"
}

And is interpreted and displayed like this: