paloaltonetworks / splunk-apps Goto Github PK

We are looking for Splunk 6.6.0 support and would like to know when an upgraded version will be out to support this release.

Thank you

I'm a Splunk app for Enterprise Security user. The Intrusion_Detection datamodel has a signature field that the app is currently assigning using an eval expression in props.conf. It means the signature field looks like this typically "Vulnerability exploit detection(35364)"

My preference is to use the lookup table already supplied with the TA and assign signature from the field name threat:name populated from that lookup. So my props entry is:

[pan:threat]
LOOKUP-pan_threat_id = threat_lookup threat_id OUTPUT "threat:category" "threat:cve" "threat:name" AS signature "threat:severity"

And the signatures look more like this (IMO more descriptive): "Microsoft Windows SMB Negotiate Request"

If you guys agree I'm happy to do the git pull, but didn't want to before someone had a chance to disagree with me. There are obviously more ways to do this but I feel the lookup is most scalable.

Hi,

After upgrade to v4.2, I got
Error in model "pan_logs" : Error in 'DataModelEvaluator': JSON for model 'pan_logs' is invalid.
error message.

Currently, I am using
PanOS 5.0.x
Splunk 6.1.1

I check the firewall log forwarding setting which only provide syslog in current PanOS 5.0.x, is it the root cause of problem or I have missed any configuration ?

Thanks

I noticed that the wildfire dashboard was putting "unknown" for the user when the wildfire logs were for SMTP traffic and contained the email sender and recipient fields.

I've changed the pan:threat source so it captures the recipient or sender in the user field if src_user or dest_user are null as follows:

[pan:threat]
EVAL-user                           = coalesce(src_user,dest_user,recipient,sender,"unknown")

this makes the wildfire dashboard more accurate.

cheers

Just logging a case to say that in PANOS 6.1 there is a log format change that adds an ID number (i think) after the threat name. This is the only one I've noticed so far, but there could be others.

Hi Brian,

I've had some issues using the app when ingesting 100GB+ of PAN events a day and after speaking with other Palo Alto customers it seems I'm not alone.

The first issue is the default acceleration time, however, I noticed it's now been changed from 'all time' to '1w'. That's really good, but also reckon third-party apps like this shouldn't automatically enable datamodel acceleration. It's a bit like having indexes.conf in Splunkbase apps, it's just not cool. :) What are your thoughts on perhaps changing this? The pan_tstats macro should also probably not have 'summariesonly=t'.

The biggest issue however is the size and number of datamodels. In large environments they're huge and they've resulted in heavy indexer load and instability in our environment on more than one occation.

Hope this is helpful feedback mate.

Thanks,
Doksu

So, because almost all of the data, including items which do not summarize well, are included in the data models, the data models that get created by this app are way too big. By way to big, I mean, generally they are larger than the original data. This can cause some major retention issues.

The PA Wildfire Submissions log on a firewall can show multiple events for malware. The screenshot below shows a PA Anti-virus wildfire-virus event that has blocked some known malware from getting through via SMTP but the wildfire event only alerted. I assume it alerts so that the admin can see the analysis report associated with the malware.

I think this is due to the wildfire dashboard in the splunk app and the data model not considering the wildfire-virus events which results in the wildfire dashboard inaccurately showing the malware as being allowed (from the wildfire alert event) when it was in fact blocked in the wildfire-virus reset-both event.

It would be nice if it only showed the real result.

Version 3.3.1 of the app has bad comments in default/searchbnf.conf in the "Description" lines. There are EOL characters at the end of several lines of the descriptions, which give errors when splunk starts.

The full error:
"ERROR Could not get Splunk_TA_paloalto credentials from splunk. Error: 'str' object has no attribute 'os_startIndex' "

This happens when running any of the commands from the App/TA. Specifically for retrieveWildFireReport.py.

• The credentials are in the passwords.conf file under Splunk_TA_paloalto/local.
• The $SPLUNK_HOME environment is set.
• We are running Splunk with Search Head Clustering.

We can perform further troubleshooting if requested.

Thank you

sourcetype=pan:system signature="*fail" type events should be tagged as authentication. Should have a user, and a src, and an action at least.

The pan:config should have a "user" field, per CIM 4.6 for Change Analysis. Can be remedied by adding a field alias from the "admin":

[pan:config]
FIELDALIAS-user = admin AS user

Thanks for the awesome work! Is there any chance that you could add a dropdown menu to select what dvc you're seeing in addition to the vsys as well? Thanks!

The correlated events come in through the pan:log sourcetype but do not have the key fields extracted.

It would be great to have these fields extracted like they show up in the PA's Correlated Events screen.

match Time
Update Time
Object Name
Source Address
Source User
Severity
Summary
Match OID
Object ID

Would be great to see Destination Address as well if it becomes available.

PA version 8.0.1

Will not work with latest version of SplunkPA , docs say:

In a single node environment, the latest Add-on (TA) is installed automatically by the App, and does not need to be installed separately. In clustered environments, the App and Add-on should be installed separately. Both can be installed by a deployment server.

but click on configure app says cannot find TA.

when I run eventtype="pan_config" I get No results found. Try expanding the time range. (all time).

I have ran wire-shark and syslog data is hitting machine.

I also manually installed the TA app but still does not work.

BizBo

The datamodel constraints for Log.url are incorrect. So the web activity report and Content Dashboard do not work.

Current 3.7.1:
eventtype="pan_threat" log_subtype="url"

The correct constraints are:
eventtype="pan_url" log_subtype="url"

More information: https://answers.splunk.com/answers/354828/palo-alto-networks-app-for-splunk-50-why-does-the.html

Even if the API key is applied correctly, the cloud logs still do not appear in Splunk.

Splunk 6.4.0
App Version 5.2.0
Path - /en-US/app/SplunkforPaloAltoNetworks/threat_detail
Panel - Users by Kilobytes

When clicking the magnifying glass and opening the Users by Kilobytes panel there is a search parser error. This appears to be from an extra pipe (pictured below).

Splunk 6.4.0
App Version 5.2.0
Path - /en-US/app/SplunkforPaloAltoNetworks/threat_detail

No data is shown when clicking through on the drill down for a user with the domain "". This appears to be from the backslash not being escaped. The first pic shows no data when using a single slash and the second shows an escaped backslash.

The following needed to be added to make Splunk_TA_paloalto CIM compliant for web:

[pan:threat]
FIELDALIAS-http_user_agent_for_pan_threat        = user_agent as http_user_agent
FIELDALIAS-http_content_type_for_pan_threat      = content_type as http_content_type
FIELDALIAS-http_referrer_for_pan_threat          = referrer as http_referrer
EVAL-http_user_agent_length                      = len(user_agent)

[pan:traffic]
FIELDALIAS-http_user_agent_for_pan_traffic        = user_agent as http_user_agent
FIELDALIAS-http_content_type_for_pan_traffic      = content_type as http_content_type
FIELDALIAS-http_referrer_for_pan_traffic          = referrer as http_referrer
EVAL-http_user_agent_length                       = len(user_agent)

Another question: is it possible to get the http_method logged in PA? If so, what are the steps for configuring/enabling this?

http_method is needed by enterprise security to show the correct method associated with each event and notable otherwise every event shows a value of "unknown".

is it possible to limit the size of tsidx or remove the outdated data in tsidx?

The search contains Application field which does not exist in the table. (Tables could also use some titles)

It would be great if you included a GlobalProtect dashboard. I have built a few for myself but I'm sure the community would appreciate having one out of the box that they can either use or build upon.

I have updated the Splunk to 6.1 and using the latest Palo Alto App(4.1.1). Although the new App no longer have the unlimited TSIDX issue, it is take very long time to generated the data model.

I have already applied the solution from the following link.
http://answers.splunk.com/answers/138840/only-the-overview-dashboard-has-data-pan-app-v4-1-1-splunk-v6-1-1.html
But it still took more than 1 month to accelerated 11% of data model and the total size is 420G.

Regards,
Kevin

On the Endpoint Security dashboard the "Targeted Machines" panel shows the src field, but it should show the dest field. src is the ESM while dest is the endpoint.

We have a requirement for CIM 4.7/4.8 and it seems that the Palo Alto App has an issue with this version.

Is this an issue with CIM or is there something we can do around resolving it?

The data model acceleration building was at 99% for days and then we disabled and enabled acceleration and now it will not go past 53%.

Thanks in advance.

When clicking on a user from the Threat Overview dashboard no results are returned on the Threat Detail dashboard.

Issue is caused by escaping user with \ in it.

""log.user="tng\crusher"""

Advanced Endpoint Protection (Traps and ESM) logs changed in version 3.3.0 and 3.3.2. Need to support the new log format.

This will most likely mean older (3.2.x) log support will drop. This should be noted in the documentation, release notes, etc.

Hi,
I just noticed that the Splunk no longer able to retrieve log form Wildfire since the end of March.
s there any method to troubleshoot the problem?

Many Thanks!

Splunk + Enterprise Security + PaloAlto add-on + App
events are fed, parsed correctly and threats do appear in the PaloAlto App Threat dashboard.

I cannot figure out which correlation search need to be enabled to have threats create Enterprise Security Notable Events so that they appear in Splunk ES Threat Dashboard and Indicators as well.

pan:traffic does not match the CIM for Network Traffic:
http://docs.splunk.com/Documentation/CIM/4.8.0/User/NetworkTraffic

protocol expects OSI Layer3, not OSI Layer4. tcp/udp are transport
https://en.wikipedia.org/wiki/List_of_network_protocols_(OSI_model)#Layer_3_.28Network_Layer.29

add an eval to the TA to assume protocol is ip or include instructions to log this field from PA.

Currently the web proxy data is not being tagged correctly. It is being tagged as "web" but not "proxy", so while it does reach the correct data model, it doesn't reach the Proxy node. Here is what I am currently using for this:
eventtypes.conf

[pan_proxy]
search = sourcetype=pan:threat log_subtype="url"

tags.conf

[eventtype=pan_proxy]
web = enabled
proxy = enabled

Hey guys!

I was reviewing some of the GP logs in our deployment to get a better sense of what is being captured.

I see that the following are extracted from the description field:

• user
• src_ip
• agent_version
• agent_message

However, I took a quick pull of the different 'header' fields, and 'Message' doesn't seem to exist anymore.

Instead, here is a sample of what I'm able to pull from last 24h of GP logs:

• Auth type
• Client OS version
• Client version
• Config name
• Device name
• Login from
• Private IP
• Reason
• User name
• VPN type
• error

Only 4 of these are currently captured. Can we update for the rest of them? I'm particularly interested in the Device name, Reason, Auth type, and error clauses. I can customize myself, but wanted to share w you guys as well.

Thanks!

index=pan_userid
| panuserupdate panorama="10.0.0.1" serial="012345678901" ip_field="my_ip" user_field="my_user"

Status returned ERROR: Unable to determine user from field: user

index=pan_userid
| rename my_ip as ip
| rename my_user as user
| panuserupdate panorama="10.0.0.1" serial="012345678901"

Splunk search returned following:

External search command 'panuserupdate' returned error code 2. Script output = "ERROR local variable 'this_ip' referenced before assignment "

Current cron schedule for Applications - Retrieve New Apps scheduled search runs every minute from 00:00-00:59 and 12:00-12:59.

Suggest changing cron_schedule from * */12 * * * to 0 */12 * * * so that this search only runs once every 12 hours, as I believe was the intention

Modular input
Datamodel
Dashboard

Per Network Traffic DM, ids_type field is required, the TA presently does not populate this. Suggested following enhancement to the TA:

props.conf

[pan:threat]
EVAL-ids_type = "network"

This field is primarily exposed within ES -> Intrusion Center -> "New Attacks - Last 30 Days" panel

The savedsearches.conf file does not use the pan_index macro for the Wildfire and New Apps collects. Additionally, there should be a second macro for a summary index (e.g. pan_sum_index) which in large installs is likely to be a separate index for summary data instead of the main event index.

I would expect that there is a cascade of other searches that this affects, but since there are only 2 saved searches this is probably buried in dashboards, etc.

Lastly, in upgrading from an old version using the older TSIDX model, I had a local savedsearches that referenced tscollect instead of collect, so you should probably make a note to remove these from local and copy/modify new versions from default as appropriate.

Thanks Brian!

The error is:

Encountered the following error while trying to update: In handler 'localapps': Password cannot contain all * characters

I've verified this in 2 separate environments with new Splunk for Palo Alto Networks installations.

After upgrading from 6.x to 6.4.3 (with latest SplunkforPaloAltoNetworks App) i get the error:

Invalid key in stanza [WildFire Reports - Retrieve Report] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 7: cron_schedule (value: */1 * * * *). Invalid key in stanza [WildFire Reports - Retrieve Report] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 8: dispatch.earliest_time (value: -1m@m). Invalid key in stanza [WildFire Reports - Retrieve Report] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 9: displayview (value: flashtimeline). Invalid key in stanza [WildFire Reports - Retrieve Report] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 10: enableSched (value: 1). Invalid key in stanza [WildFire Reports - Retrieve Report] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 11: realtime_schedule (value: 0). Invalid key in stanza [WildFire Reports - Retrieve Report] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 12: request.ui_dispatch_view (value: flashtimeline). Invalid key in stanza [WildFire Reports - Retrieve Report] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 13: search (value:pan_wildfire| panwildfirereport | table wildfire_report | rename wildfire_report AS _raw | collect index=pan_logs sourcetype=pan_wildfire_report). Invalid key in stanza [Applications - Retrieve New Apps] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 24: cron_schedule (value: * */12 * * *). Invalid key in stanza [Applications - Retrieve New Apps] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 25: dispatch.earliest_time (value: -1mon). Invalid key in stanza [Applications - Retrieve New Apps] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 26: displayview (value: flashtimeline). Invalid key in stanza [Applications - Retrieve New Apps] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 27: enableSched (value: 1). Invalid key in stanza [Applications - Retrieve New Apps] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 28: realtime_schedule (value: 0). Invalid key in stanza [Applications - Retrieve New Apps] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 29: request.ui_dispatch_view (value: flashtimeline). Invalid key in stanza [Applications - Retrieve New Apps] in /opt/splunkforwarder/etc/apps/SplunkforPaloAltoNetworks/default/savedsearches.conf, line 30: search (value: index=pan_logs sourcetype=pan_newapps | table app{@name} | pannewapps | collect index=pan_logs sourcetype=pan_newapps).

Anybody else with the same issues?

"dropped" is now a valid action, per CIM 4.5. As such the lookup for vendor_action to action should output and action of "dropped" when the vendor_action is "drop"

Per http://docs.splunk.com/Documentation/CIM/4.7.0/User/NetworkSessions the Network_Sessions DM is for DHCP and VPN traffic. The current version includes these eventtypes:

[pan_traffic_start]
search = sourcetype=pan_traffic OR sourcetype=pan:traffic AND log_subtype="start"
#tags = network session start
[pan_traffic_end]
search = sourcetype=pan_traffic OR sourcetype=pan:traffic AND log_subtype="end"

This tags ALL palo traffic as Network Sessions, which makes the data model run for a very long time.

When a log has action of 'block-continue' (such as a file log), the CIM action for this is showing up as 'unknown'.

Need to figure out what CIM action it should translate to and then add that to the lookup table.

This is really more of an issue with the TA, but the tagging on the events is incorrect. The URL filtering items should go be tagged as "web" and "proxy". I would also suggest excluding the file identification events from the "ids" "attack" tags, as they create a ton of noise.