elastic / security-docs

Home for Elastic Security Documentation

Elastic Security docs

Elastic Security Docs home page: https://www.elastic.co/guide/en/security/current/index.html

Serverless docs: https://docs.elastic.co/serverless/security/what-is-security-serverless

Documentation Manager: Janeen Roberts (Github: @jmikell821)

Contributing to Elastic Security docs

If you're an Elastic employee, you can open an issue using the appropriate template. To contribute directly to Elastic Security documentation:

1. Please fork and clone the security-docs repo.
2. Check out the main branch and fetch the latest changes.
3. Check out a new branch and make your changes.
4. Save your changes and open a pull request.
5. Add the @elastic/security-docs team and any other appropriate members as reviewers.
6. Add the appropriate release version label, backport version label if appropriate, and team label to the PR.
7. If your PR changes any serverless docs content, add the label ci:doc-build to generate a preview of the serverless docs on the PR.
8. Once the docs team approves all changes, you can merge it. If a backport version label was added to a PR for stack versions 7.14.0 and newer, mergify will automatically open a backport PR.
9. Merge the backport PR once it passes all CI checks.

Preview documentation changes

When you open a pull request, preview links are automatically added as a comment in the PR. Once the CI check builds successfully, the links will be live and you can click them to preview your changes.

For stateful docs, you also might want to add targeted links to help reviewers find specific pages related to your PR. Preview URLs include the following pattern (replace <YOUR_PR_NUMBER_HERE> with the PR number):

https://security-docs_bk_<YOUR_PR_NUMBER_HERE>.docs-preview.app.elstc.co/guide/en/security/master/

Note

Serverless docs previews don't allow targeted links, because the id in the URL changes with each rebuild.

security-docs's Issues

[DOCS]: Manage hosts with installed endpoints (agents)

Meta issue: https://github.com/elastic/endpoint-app-team/issues/64

Users need a dedicated area of the application to manage their Endpoints. This area can just serve as an anchor point to get an overview for Endpoints. Views of Endpoints can be used throughout the application.

AC:

When entering the Endpoint app, users can clear see a tab or section for Endpoint Management
The landing view for the page/route contains the Endpoint Holistic view
Make a decision on a routing framework

[7.9] [Management] Host Details view

Description

[Management] Host Details META: https://github.com/elastic/endpoint-app-team/issues/61

The [managed] Host Details screen in the Endpoint Management section will allow users to take a detailed look into the state of a specific Endpoint. On this page, users can view high level host information and details around the state of an endpoint policy. (Eventually, users will be able to use this view to better understand endpoint health and metrics, and pivot to the Observability solution to take a look into the endpoint logs. These features will not be included in 7.9)

This view is in addition to the Host details view that you see on the Host tab of the Security app, and is specific to Hosts that are running Endpoint capabilities on their Agent.

Acceptance Test Criteria

• When a user selects the Host Name from the Managed Host List page, a flyout will open
• Flyout will show Host Details -
  • Host Name
  • OS
  • Last seen date/time
  • Alert Count
  • associated Policy Name
  • IP address
  • Domain Name
  • Endpoint version
  • associated Agent Configuration, and a link to that Agent Configuration in the Ingest Management application

Notes

• Screen shots to be added

[7.9] Endpoint ECS mapping

Description

Document ECS mapping for Endpoint
Similar to: https://www.elastic.co/guide/en/beats/auditbeat/master/exported-fields-ecs.html
We will link to this in our package definition

Acceptance Test Criteria

List all the ATC of each action and its intended result.
As a user, when [action (e.g., viewing, clicking, selecting, etc.)] the [insert the expected result].
If the doc issue includes a procedure, number the steps in sequential order.

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

[DOCS] Timeline Updates in 7.9

Description

Meta Issues: https://github.com/elastic/siem-team/issues/528 and elastic/kibana#68580

In 7.9, the Timeline feature will have the following updates:

• User ability to add a Timeline to existing Case
• Timeline Template

Acceptance Test Criteria

Docs are needed for the 2 enhancements to Timeline

User ability to add a Timeline to existing Case

User can add a Timeline to an existing Case now. Gif is included in issue ticket.

Timeline Template

User can create a new template or convert an existing timeline into a template to use for future investigations.

Workflow below helps show how timeline templates are created, and how they can be used for Detection Rules.
Timeline.Template.Workflow.1.pdf

Use case: When users open up alerts in the Timeline, it would be nice to have each of those Timeline pre-configured and populated with all the relevant fields in the relevant order. User can create templates which they can pre-configured fields and link them to the Detection Rule that will produce alert types that they want to use the Timeline Template for.

For example, if a user write a Detection Rule to seek out malicious file events in the environment, they would definitely want to see fields such as file name, file path, file size, username, and file hash. They can make a timeline template that has all of these fields set as the default view, and add it to the Detection Rule. When an alert is triggered on that rule, user opens up the alert in timeline, the timeline will show file name, file path, file size, username, and file hash fields on default.

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

[7.9] Endpoint Package/Integration documentation

Description

Similar to: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-crowdstrike.html

Acceptance Test Criteria

List all the ATC of each action and its intended result.
As a user, when [action (e.g., viewing, clicking, selecting, etc.)] the [insert the expected result].
If the doc issue includes a procedure, number the steps in sequential order.

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

Document exceptions and lists APIs

Add exceptions and lists APIs to the documentation.

PR: #35

[Docs][Endpoint]: Add instructions for enabling full disk access, mac

Description

According to bug https://github.com/elastic/endpoint-dev/issues/6635, any macOS 10.15 endpoint must enable full disk access in order to be protected by the Endpoint app. Write a set of instructions about how to enable this access, and document how the Endpoint app sensor checks for this access.

Acceptance Test Criteria

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

Example Issue:

Issue Name: Document the ability for users to disable artifact updates.

Description: Many organizations have major events where they lock down updates to their systems. They lock down the updates because the systems are highly tuned and they don't want to introduce error or risk during the major event (think World Series or All Star Game or a state government just before elections.) This feature will give admin users the ability to stop artifact updates until that event is over and then re-enable after.

Acceptance Test Criteria: Ensure the user is logged into the platform as an admin.

1. On the Left Navigation toolbar, click the Administration button, then select the PLATFORM tab.
2. In the "ARTIFACT UPDATE CONFIGURATION" section, click Disable artifact updates.
3. In the dialog box that says, "You are about to DISABLE artifact updates. This will prevent you from receiving updated artifacts..." click Disable. A 'Successfully updated configuration" confirmation appears.
4. Click Finish.

Note: To re-enable cloud updates, follow the aforementioned steps, except in Step 2, click Enable artifact updates.

Create kernel extension loading documentation for elastic endpoint

Description

We are currently missing documentation on how to approve the kernel extension for the elastic endpoint

Acceptance Test Criteria

Instructions and diagrams showing the user how to approve the elastic endpoint kernel extension on macOS

Make an issue template

Make an initial issue template.

[DOCS]: Unified Detection Alerts View

Description

Meta Issue: https://github.com/elastic/endpoint-app-team/issues/372
Link to Mocks https://www.figma.com/file/LsjbVEOGXX4iPHqoqQL8mc/Endpoint-Screens-and-Components?node-id=1269%3A125098

As an analyst, I want to be able to view all alerts on a page and search/filter/sort fields in the Detection Alerts view. As a user, I want to have a single unified place to see all alerts coming from Elastic Endpoints, 3rd party logs, and detection engine created so that I can have a holistic and uniformed triage process for all of my alerts.

Acceptance Test Criteria

Documentation required to point out following changes to the Detection Alert page in 7.9:

1. Open, In Progress, and Closed Alerts filters on alert list
2. Action overflow menu (elastic/kibana#65945)
   1. Actins not in overflow - Investigate in Timeline, Analyze Event
   2. Action in overflow - Mark In Progress, Close selected, Add Exceptions, Add Endpoint Exception, Edit Actions
3. Signals now called Detection Alerts (elastic/kibana#65944)
4. Remove External Alerts tabs
5. Toast success and failed messages on status change of alerts to closed, in progress, reopen (elastic/kibana#67406)
6. Sticky column preferences - Columns and row rendered preferences are stored per-user base (https://github.com/elastic/siem-team/issues/589)

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

[7.9] Editing the Endpoint Security Integration

Description

Background:
The advanced endpoint security integration settings allow operations managers to configure endpoint protections and event data collection and apply those configurations to one or more endpoints. By containing a number of specific settings, these configurations enable users to customize their endpoint protections as aggressively or conservatively as needed to meet the security requirements of their organization.

There are several entry points for editing the advanced options of the Endpoint Security integration:

In the security app:

• In Administration > Hosts, clicking the link under the "Integration" header will take the user to the advanced options (previously known as Policy details)
• In Administration > Hosts, opening up the Host Details flyout and clicking the Integration link will take the user to the advanced options (previously known as Policy details)

In Ingest Manager:

• In Ingest Manager > Configurations, locate the configuration that the integration is associated with and click the configuration name
  • Locate the Elastic Endpoint Security integration. Under Actions > click "Edit Integration"
  • There will be a link at the bottom of this page that will take the user to the advanced options (protections, event collection)

Acceptance Test Criteria

Notes

[7.9] Configuration Status and Response

Description

Policy Response META: https://github.com/elastic/endpoint-app-team/issues/309
Users need a detailed understanding of an agent configuration's application in order to know if the hosts in their environment are properly protected.

Configuration statuses:

• Green/Success:
  • policy applied correctly
• Yellow/Pending or Partially Applied
  • The Elastic Endpoint will send us a detailed message which shows what occurred on the Endpoint. In some cases, some actions taken on the Endpoint may fail during the config application, but are not recognized a critical failure (meaning, there may be a failure, but the hosts are still protected) These would be "Partially Applied". Other failures that are considered critical will result in a red, “Failed” status.
• Red/Failed
  • A critical failure; This means policy did not apply correctly and hosts are not properly protected.
• Unknown:
  • UI is either waiting for the API response to return or a fallback case the API returns error or a value that we don't recognize (unlikely - but just a protection in the code).

Important things to note:

• the status uses the Ingest Manager APIs. Users must have permissions to read/write to these APIs in order to make changes to their Policy
• Ingest Manager must be enabled in a Kibana Space in order for the Management section of the Security app to function properly

Acceptance Test Criteria

Notes

• Screen shots to be added

[DOCS]: Document exceptions guideline for prebuilt rules

Description

A sub-issue of #57. Via @dontcallmesherryli: "Per speaking with @mark-dufresne, we should document a guideline for users to help them identify fields for creating exceptions for each of the prebuilt rules, so that users are not shooting themselves in the foot and unknowingly hide critical alerts that the prebuilt rules are meant to detect."

Will add to this ticket as we get more info.

Trusted Applications

Meta Issue: https://github.com/elastic/security-team/issues/156

Description

Background:
Users expect to be able to use Elastic Security without any conflicts or compatibility issues with other installed applications on their system. They want to be able to intentionally exclude some processes from being monitored and make them completely trusted - or, conditionally trusted. This is easier on the user than adding multiple entries to their allowlist in order to ensure an application can be used.

User Story/Problem Statement(s):

As an elastic endpoint security user, I want to be able to trust applications by defining what process to trust to trust, so that I can ensure there are no compatibility or performance issues with applications I need in my environment.

Acceptance Test Criteria

• User can add trusted apps by OS
• User can add an entry by Process Path
• User can wild card the process path
• User can add entry by Hash
• User can view a list of Trusted Apps
• We do not automatically trust child processes of a trusted process
• Users are able to access Trusted Apps from the Fleet application through the "edit" integration page

Screenshots:

Notes

• Team: endpoint management (Kevin/Caitlin)

[DOCS] Rule Exceptions and Endpoint Exceptions

Description

Meta issue: https://github.com/elastic/endpoint-app-team/issues/377
Mock link: https://www.figma.com/file/jcCKnGXvOlFxMOpUjlTMMz/All-Exceptions?node-id=347%3A24834

As a user, I need the ability to add exceptions to alerts in the Elastic Security App so that I can tune out the false positive alerts and allow end users performance of their jobs on their computers.

Acceptance Test Criteria

Documentation is needed when user adds an exception to Endpoint, adds an exception to rule, creates an exception list, view exception items created, and edits an exception item.

Add an exception to Endpoint

1. User can add an exception to Endpoint by clicking on action overflow menu on the Detection Alert list.
2. User sees a modal they must complete. They can add desired fields and nested conditions for Endpoint exception (originally known as whitelisting). User can add comment to the exception and select to close alerts with matching attributes.
3. User clicks "Add Exception" when they complete the modal. The attributes they entered are added to the Endpoint Promotion Rule, as well as added to the package sent to all endpoints so the sensors on the endpoints will no longer alert and allow matching attributes to happen on the endpoint.

• Use case example - SOC analyst wants to allow for a file that is not malicious (False Positive). Analyst can "whitelist" the file by adding the hash and the path of the file as an endpoint exception item.

Add an exception to rule

1. User can add an except to a rule by clicking on action overflow menu on the Detection Alert list.
2. User sees a modal they must complete. They can add desired fields, operators, nested conditions, and exception lists for the rule exception. User can add comment to the exception entry and select to close alerts with matching attributes.
3. User clicks "Add Exception" when they complete the modal. The fields, operators, nested conditions, and exception lists from the exception entry will be applied to the detection rule.

• Use case example - SOC analyst doesn't want to see detection alerts on a group of hosts, they can add exception of these hosts to a detection rule

Create an exception list

1. User can add an value list to be used for exceptions on Rules Management page.
2. User click on "upload value list" button on top, and is prompted with a modal that has an upload section and a section that shows all uploaded value lists. User chooses from computer file to upload a list.
3. User can Export and Remove existing value lists.

• Use case example - SOC analyst needs to add a list of trusted hashes that should be ignored by detection rule.

View exception items created

1. User can view all exception items created in the Detection Rule page.
2. Endpoint Exceptions created are in the Endpoint Rule details page under the Exceptions tab, while exceptions applied to detection rules are found within individual detection rules details page under the Exceptions tab.

Edit an exception item

1. User can edit and remove exception items by going into Rules Details page under the Exceptions tab.
2. Clicking on Edit, user gets a modal that is similar to the "Add exceptions" modal and can edit all attributes there. User can also add more comments to the exception entry.

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

[DOC] Rule fields Override and Rule Building blocks

Description

**Meta issue: ** elastic/kibana#65941

User has the ability to map source fields to values of severity and risk score when they create a detection rule.

In advanced settings for rules creation, user also has the ability to override Rule Name and Timestamp, as well as check that the rule is a Building Block.

Acceptance Test Criteria

Documentation is required to instruct user to use Severity and Risk score override if they wish to use certain fields to map their rule severity and risk score to.

Use case example: User is using a data source where he wants to use a field called Priority Ranking (with the values of 1, 2, 3, 4, 5 as rank) is used for the SIEM Detection Rule Severity. He would select Priority Ranking as source field, and map severity value 1 to Critical, 2 to High, and so on. Now the user can use the source data to standardize into the SIEM severity mapping.

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

[DOCS]: Overview: Elastic Security

Write an overview topic that summarizes Elastic Security, its purpose and main functionality. Old intro from platform docs for inspo:

[7.9] Management View + Hosts Running Endpoint

Description

Management Tab META: https://github.com/elastic/endpoint-app-team/issues/461
[Managed] Host List META: https://github.com/elastic/endpoint-app-team/issues/60

Background:
With the decision to combine SIEM + Endpoint use cases into one security app, we've decided to create a section of the app dedicated to “Management” tasks - management of hosts running the Elastic Endpoint. The hope is that by having management related functions in one place, it will be clear to users which hosts will accept policy configurations, endpoint-exceptions, and other tasks that will eventually be sent to the endpoint. The main Host view will also show hosts running endpoint capabilities - but management actions will be taken through the Management section.

• Management will be a new tab in the Elastic Security app
• The Management section will allow users to manage various functions on hosts that contain an agent with Endpoint capabilities
• Things users can do in Security Management in 7.9:
  • User can view + monitor hosts running endpoint
  • View policy status and response
  • Apply a policy
  • Create a new policy

Acceptance Test Criteria

• When landing in the security app, users will see a Management tab
• Clicking into the Management tab will display a list of Hosts running the Elastic Endpoint
• Users will see the following data in the table:
  • Name
  • Policy
  • Policy Status
  • Alert count
  • OS
  • IP Address
  • Version (Endpoint)
  • Host Status
  • Last Active Date/Time

Notes

• 6/15/2020 Note: Use of "Policy" in the management section is subject to change. Will have final text this week.
• Screen shots to be added

[DOCS]: Terminology topic

Write a security, glossary-like topic on UI concepts, features, and terms.

References:

https://github.com/elastic/endpoint-app-team/issues/412

[Docs]Add a what's new/breaking changes sections

We need a section that tells users what's new in each release. We also need a breaking changes section, as a Cases API call has changed. I've added the breaking changes section in this PR: #64.

Since we now own the release notes for the Security app, I'd like to see if we can use the RNs section to highlight what's new (including terminology changes requested by @MikePaquette). @DonNateR - I think this is your decision as you're doing the RNs. Whatever we decide, we need to make sure users can easily find a (very :)) high-level list of changes.

Related to: #12, #53

cc @jmikell821

[DOCS]: Create an ad-hoc Protections Guide

Placeholder to (possibly) create a Protections Guide that describes all the endpoint protections, MITRE ATT&CK technique, etc. This may be tricky to do in Asciidoc, so will explore the option of creating a read-only Google doc. Release date would likely be 7.10+.

[DOCS]: Endpoint Rule for endpoint alerts

Description

Meta Issue: https://github.com/elastic/siem-team/issues/641
No Mock, UI changes scoped out of 7.9

Enable Users to add all Endpoint and 3rd party alerts to the SIEM Unified Alert View of the Elastic Security App

Acceptance Test Criteria

Documentation required to let user know of the pre-built Endpoint Rule (internally known as Promotion Rule) that is turned on by default with 7.9 release. The rule ensures that all Elastic Endpoint alerts populates inside the SIEM.

Promotion rule cannot be edited by users, but can be turned on or off and deleted by users inside Rules Management page.

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

[Docs]Detections API updates

Add the new fields and objects available when creating rules.

[DOCS]: Filter Endpoints list

Related issues: https://github.com/elastic/endpoint-app-team/issues/66, https://github.com/elastic/endpoint-app-team/issues/67

Link to mocks: https://zpl.io/2vp1YMj

Users need a way to filter their query of deployed Endpoints

AC:

API should be extendable to new filters
API should support KQL format
Curated filters are not required for this ticket. It's acceptable to only accept KQL
At the least, we can filter by with KQL:

All unique Endpoints (latest document for each)
Filter string search on Hostname
Filter on IP (CIDR)
Filter by OS type/platform (Windows, Mac, Linux), 3 discrete filters
Policy status - (Success (Green), Pending or Partially applied (Yellow), Utter Failure (Red))
String search on Endpoint version

[DOCS] Host Page Overview

Creating an issue for the host page overview topic.

[DOCS]: System Requirements

Draft a topic that includes hardware, system, and OS requirements for Elastic Security. If applicable, include these components:

[DOCS]: Create new docs repository instructions

Description

In order to add a new section to elastic.co, users must either add their book to a different elastic repository or create a new docs repository.

Either option requires a similar but slightly divergent set of steps. Some of the repo addition processes are documented in the "Creating a new book" page in the docs section of the wiki, but there's no information about configuring a new repository.

Verify the steps required to add a book to an existing Elastic repository and add the additional steps needed to create your own docs repository.

Acceptance Test Criteria

1. Add a docs directory with the required files to an existing elastic repo.

OR

1. Create a new repository with a docs directory and the required files.

2. Modify the conf.yaml and shared attributes files in the elastic/docs repo to call your existing or new repo.

3. (For new repositories), add your new repository to elastic/infra repo, in order to get the benefits of Elastic's CI/CD system Jenkins.

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

[DOCS] Wording needed to clarify Exceptions are linked to a Rule

Description

We received feedback from the Protections team that there's a lack of clarification that an exception is tied to the rule that the alert originated from. Text is needed inside the Endpoint Exception Modal to let users know "This exception will apply to XXX rule"

Acceptance Test Criteria

Text is needed inside the Endpoint Exception Modal to let users know "This exception will apply to XXX rule"

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add the version number label.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

Update prebuilt job descriptions for 7.9

Need to add new 7.9 jobs and update the rule changelog.

[DOCS] Tooltips needed for the timeline action dropdown and next to “add value list” button

Description

To help users use the new features, we need hoverover tooltips for when timeline action dropdown items are greyed out and for the "add value list" button in the Detection Rules page.:

Acceptance Test Criteria

Tooltips for:

1. timeline action dropdown menu when actions are greyed out - things are greyed out because user didn’t name their Timeline yet
2. next to “add value list” button on Detection Rules page to let users know what value lists are

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add the version number label.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

[DOC] EQL Rule Creation

Description

EQL Rules can be created by users to run on the Detection Engine. EQL is a language that can match events, generate sequences, stack data, build aggregations, and perform analysis. EQL gives analysts the advantage of correlation and sequence querying, allowing users create alerts based on event relationships.

Meta ticket: https://github.com/elastic/security-team/issues/42
Other related tickets: https://github.com/elastic/security-team/issues/43, https://github.com/elastic/siem-team/issues/809

Acceptance Test Criteria

Documentation is required around the process of EQL Rule Creation.

1. User can create a correlation based rule using EQL syntax to write a query.
2. User can preview the EQL query they are writing for the rule inside the Rule Creation steps.
   1. User selects date range to preview the EQL query on - last hour/day/month/year
   2. Histogram shows the alerts produced on the selected time range
   3. If number of alerts that shows up is greater than 1 alert per hour, user will get a warning message that this rule could be noisy and produce a lot of alerts. Warning message doesn't prevent users from creating the rule, only informs them.
      • Last Hour - 1 alert
      • Last Day - 24 alerts
      • Last Month - 730 alerts
      • Last Year - 8760 alerts
3. All other steps to EQL rule creation is the same as Custom Query rule.

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add the version number label.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

7.9 Cases updates

New IBM Resilient connector.

META Issue: Security Documentation TOC Outline

The purpose of this ticket is to record the final TOC/outline of our security documentation. Security docs are located here: https://www.elastic.co/guide/en/endpoint/current/index.html.

Test issue

[DOCS]: 7.9 Release Notes

Opening this issue to track release notes for 7.9.

Error in Artemis documentation search query examples

Description

There should not be a space between the number and the size indicator. For example, the query in the docs is:

Search network data for endpoints sending more than 100 MB
Artemis
Are you sure you want to run:

Search network for greater than 100B on active endpoints?

This is interpreted by Artemis as 100 Bytes rather than Megabytes

By removing the space, the query is interpreted properly.

Search network data for endpoints sending more than 100MB
Artemis
Are you sure you want to run:

Search network for greater than 100000000B on active endpoints?

Acceptance Test Criteria

Users should be able to copy/paste any Artemis example and get the expected results.

Notes

It's possible that this is really an artemis issue and it should properly interpret either entry, but it currently does not and can create confusion.

Create and configure new Policy

Issue: https://github.com/elastic/endpoint-app-team/issues/108

Background:
The endpoint policy allows operations managers to configure endpoint protections and event data collection and apply those configurations to one or more endpoints. By containing a number of specific settings, Endpoint policies enable users to customize their endpoint protections as aggressively or conservatively as needed to meet the security requirements of their organization. The user should be able to see a view of the current configuration of the policy and whether that policy was correctly applied to that endpoint.

User Story/Problem Statement:
As an operations user, I need to be able to fine tune my policy to meet my protection expectations and to manage the amount of noise in my environment. I need to be able to configure detect vs prevent actions, and the ability to display a user notification upon preventing a threat. I also need to be able to configure the collection of events running on my Windows, Linux, and macOS endpoints/hosts.

Reference Information

SMP policy enhancement requests
User Workflow Diagrams:
https://docs.google.com/presentation/d/1ydji-tTk1_FPmSoibQXwKF0YYEO75bZIaz9ZnLC3thI/edit#slide=id.p
Link to Mocks
https://app.zeplin.io/project/5df9464db94aa799eb8bf0e9/screen/5e31b9323d11857fb0ae02a5
https://zpl.io/2GdKMgY
https://zpl.io/b67Zyed

NOTE: if you turn on malware in policy, that creates External Alerts. In order for those alerts to be prioritized to Detection Alerts, you have to set up a Detection Rule. We will need to note this specifically in docs.

[DOCS]: Analyze Events Overview

Description

Meta issue: https://github.com/elastic/endpoint-app-team/issues/68, https://github.com/elastic/endpoint-app-team/issues/475
Key contacts: James Brown, Rob Austin, Michael Olurunnisola, Sherry Li, Lindsey Poli.
Mock: https://www.figma.com/file/WxBmwHu1dDwi0Z1HWkBUrq/GAH-Workflows-%E2%80%94-Timeline?node-id=2%3A1

Update - Final naming of feature: no name, but the action is call "Analyze Event" in the action menu in Alert list and Timeline.

Create a comprehensive topic on all the timeline features, whose main feature is to view processes, spawned processes, and other details in a graphic visualization to show the analyst what led up to and occurred after an attempted attack.

Once the front and back-end are complete, let's add some GIFs to this topic. I think customers will be able to benefit from some visualization.

Acceptance Test Criteria

Documentation is needed to guide users to using the Analyze Event feature.

1. User can see an event process tree with details on each events in a process of all endpoint alerts. User clicks on "Analyze Event" icon in Alert list (available only for endpoint alerts) to go into the the graphical view.
2. User can see an event process tree of file and process events within a Timeline. User clicks on "Analyze Event" icon in Timeline cards to go to the graphical view.
3. Within the Analyze Event feature, user can:
   • Zoom in and out of the graphic to see more event details
   • Open and close all event count drop downs to see number of events per process node
   • See time passed between each event node
   • Red color for Alert event node to focus users to the problem event
   • Left panel that allows users to drill down on information levels process, event type, single event, details on single event, with breadcrums to help users know where they are during their investigation of an alert.
   • Attach the URL of the rendered graphic view to a new or existing Case.

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

[7.9] "What's changed" - Security update overview

Description

We want to include documentation to help communicate some of the changes as a result of the combined security app in 7.9. We want to get ahead of questions like "where are my signals?" and help users better understand where new and old features now live in the unified app.

What's Changed

Terminology changes for 7.9:

Old → New

• Endpoint → Host
• Signal Detection Rules → Detection Rules
• Whitelist → Exception(s) / Exception List
• Elastic SIEM & Endpoint Security → Elastic Security
• Management → Administration
• Signals → Detection Alerts (See note below)
  • Detection Alerts: Alerts occurring within the Elastic Security from the Detection Engine / Detection Rules
  • External Alerts: Alerts originating outside of Elastic Security
  • Kibana Alerts: Alerts native to Kibana not necessarily security-related
• Resolver → No name, will be referred to as an action: "Analyze Event"
• Sensor → Endpoint

Note: Some navigation changes happened due to renaming of Signals

• Top Nav naming:

1. Alerts → Detections
2. URL will be app/security/detections

• Under "Detection":

1. Alert page title → Detection alerts
2. Alert count → Trend
3. Alert list → nothing (blank space)
4. URL will be app/security/detections

• Under "Overview":

1. Alert Count → Detection Alert Trend
2. External Alert Count → External Alert Trend

• Inside Timeline Event filter drop down

1. Alert Events → Detection Alerts

What's New

Administration Tab:

• New Administration tab is dedicated to managing Hosts that are running endpoint security. From this page you can view hosts, view and edit the advanced integration options,

Other stuff:

• Alert table customisations are now persistent, on both the Detections and Rule details pages (elastic/kibana#67156) cc @spong @cnasikas
• Exceptions and value lists
• Endpoint integration (policies and endpoint exceptions)
• Threshold rules
• Timeline templates
• IBM Resilient integration for Cases

Notes

[DOCS]: Host List Overview

Meta issue: https://github.com/elastic/endpoint-app-team/issues/63

AC:

Lists Endpoints in a tabular format
Contains fields: IP, Hostname (fully qualified?), OS, Endpoint Status (last seen), Policy Status, Domain
Unit Tests at a minimum
API response format agreement: #65
For this ticket, all Endpoint data can mocked/faked including Endpoint Status and Policy Status

[DOCS]: Prerequisites and System Requirements

Elastic Security customers need to know what prequisities are required before installing the app. They also need to know what system requirements are mandatory, what operating systems are supported, and if any firewalls or security settings need to be enabled/disabled. Some sections in this topic may benefit from a checklist.

[Docs]Add Elastic Security flowchart to the docs

Currently, there is no diagram illustrating the Elastic Security workflow. @MikePaquette and I started working on one for the 7.9 release (based on Mike's roadmap flow).

The draft flow is here: https://docs.google.com/drawings/d/1snRC0crIxcWdig3RsMkvKbIHq_lqZGpjymbpcT6O9rU/edit?usp=sharing

We'd like to get feedback on the flow from the other tech writers, and the product and UI teams. There's no need to worry about the diagram's aesthetics (unless you want to), as we'll send the approved flow to the design team.

Thanks,

cc @jmikell821, @DonNateR, @marrasherrier, @dontcallmesherryli, @caitlinbetz, @jamiehynds, @shimonmodi

[DOCS]: Configure malware protection and event collection settings

Once customers create an agent integration, they should enable and configure malware protection on protected hosts.

1. Go to the Administration page.
2. In the "Integration" column, select the appropriate one to configure.

Malware Settings
Ensure the "Malware protections enabled" toggle is turned on.

Protection Level
Detect: Detects malware on the host and generates an alert.
Prevent: Detects malware on the host, blocks it from executing, and generates an alert.

Note: Malware is enabled by default and set to Protect, but customers can change it to Detect if desired. However, if they DO set it to Detect, the customer needs to pay attention to and analyze the alerts coming in, as those malware alerts in detect mode won't block anything.

Customers can also configure event collection settings for macOS, Windows, and Linux - and can enabe/disable as appropriate.

Add Documentation Details for Malware Prevention

Description

Customers often ask - "how do you prevent malware?" we should describe malware prevention in more detail in the docs.

Acceptance Test Criteria

Something similar to <endgame_platform>/api/help/Content/Admin%20User%20Guide/Sensor%20Topics/malware_protection_ov.htm?Highlight=malwarescore

@jmikell821 , @JamieRButler , @cosiomoises , @mark-dufresne , @bfilar

[Security docs] 7.9 doc updates (Ben's stuff)

Detailed list of all the doc things that need doing for 7.9:

• ES Security overview flowchart: #31
• Cases/case connectors (Christos, Xavier):
  • API: new external service: #64
  • UI: new connector type: #64
• Detection rules (Garrett, Ryland)
  • API: update rule schema: #70
  • API: update signal (alert) schema: #70
  • UI: update create/modify rule procedure: #73
  • UI: new threshold rule type: #73
  • UI: new rule settings: #73
  • UI: screenshots: #73
  • Exception lists (Garrett, Frank, Yara, Ryland, Devin, Pedro):
    • New API: #35
    • UI conceptual stuff and procedures #73
  • Prebuild rule updates (Craig/Paul): #66
  • New ML jobs (Craig): elastic/stack-docs#1270
  • Rename ML jobs (Craig): elastic/stack-docs#1317
  • Fix prolematic terminology used in the prebuild rule descriptions (UI only, docs have been fixed): elastic/detection-rules#54
• Timeline/Timeline templates (Angela, Xavier)
  • Timeline object schema: #50
  • Timeline API: #50
  • New UI conceptual stuff and procedures: #84
• Update the SIEM section (including screenshots) in the Kibana UG: @DonNateR
• Rename SIEM > Elastic Security, Signals > Detection alerts (probably as part of a few different PRs)
• Restructure repo file system (probably as part of a few different PRs):
  • Move APIs and reference sections to first level: #18
• Update in-app links to the docs: elastic/kibana#71403, elastic/detection-rules#54
  • Map configuration
  • Prebuilt rule metadata
  • Others?
• Check if the ECS section needs updating (Jamie, Mike)

[DOCS] Create Threshold-based Rule type

Description

Meta issue: elastic/kibana#68409
Mocks: https://www.figma.com/file/yNMzeaGvLkyRvra8Bu2fD2/Threshold-Based-Rules?node-id=140%3A31

In 7.9, user will be given a 3rd type of rule to create in the Elastic Security app - Threshold-Based Rule. Threshold rule lets user select fields and set a threshold count to these fields to be alerted on in the SIEM.

Acceptance Test Criteria

Documentation needed to help instruct users on how to create Threshold-Based Rules with some use case examples.

1. User can select Threshold-Based rule when selecting a type of rule to create.
2. User will be prompted the steps to enter the Index Pattern, Custom Query, and Field and Count of the threshold rule that it applies to.
3. Just like the Detection Rule type, user can select configurations around the rule such as Timeline Template used and Schedule of the frequency of the rule to run on the Rule Engine.

• Use case example: Rule write wants to ensure an alert is generated to detect large number of failed login attempts, so they can select the desired index, query for registry or authorization types of event.type, and designate those fields to a threshold of 100 counts. This way when an end user attempts to log in 100 times within the time of the rule scheduled, the SIEM will receive an alert about the detected activity.

Notes

• Add the "Team:Docs" label to new issues.
• Be sure to add any necessary screenshots for clarity.
• Include any conditions or caveats that may affect customers.

[DOCS]: Policy List and Policy Details Page Overview

Description

NOTE: As of 7/27/2020 we've decided to remove the concept of Policy for the 7.9 Release. The current "edit" policy page will still exist, but as an extension of the Agent Configuration/Integration. Closing this ticket and will open another to outline how the edit policy/protections page will fit in the flow.

---

Background:
The endpoint policy allows operations managers to configure endpoint protections and event data collection and apply those configurations to one or more endpoints. By containing a number of specific settings, Endpoint policies enable users to customize their endpoint protections as aggressively or conservatively as needed to meet the security requirements of their organization.

Policy List:

• Users can find their security policies under the Management tab, and the Policies sub-tab.
• The policy table provides the following information:
  • Policy Name
    • The policy name is directly tied to the name of the associated Agent Configuration in the Ingest Manager.
  • Created By (user)
  • Created Date/Time
  • Last Updated By (user)
  • Last Updated Date/Time
  • Actions Menu
    • Actions Menu contains the options "View Agent Configuration" and "Delete Policy"

Policy Details:

• User can select a Policy name to be taken to the policy details page
  • Policy Details is split into two sections: Protections, and Settings
  • The protections section will list any available protections and their associated options. For 7.9, we are releasing the Malware protection for Mac and Windows. This can be toggled on or off. In addition, user can select two options: Detect or Prevent
  • Prevent with Notification will be available in the future, but not in 7.9
  • The settings section allows users to configure events that are collected by the endpoint
• The top right of the policy details page shows metrics related to the policy application. Users can see the number of hosts assigned to the policy, and how many have successfully accepted the policy, how many are pending, and how many failed to apply the policy

Acceptance Test Criteria

Notes

Important things to note:

• Policy uses the Ingest Manager APIs. Users must have permissions to read/write to these APIs in order to make changes to their Policy
• Ingest Manager must be enabled in a Kibana Space in order for the Management section of the Security app to function properly

[DOCS]: UI Overview

Topic should include user interface tour of app, including main page and what's in each section.

Document Timeline API and object schema

Add Timeline object schema to the documentation.

PR: #50

cc @angorayc @XavierM @MikePaquette

[7.9] Elastic Endpoint - Onboarding

Description

Elastic Endpoint Onboarding META ticket: https://github.com/elastic/endpoint-app-team/issues/16

This will outline the process of getting a user started with the Elastic Endpoint (installing Endpoint on the agent to get events flowing in the Security app).

(It would be good to have "get started"/quick guide for this process as I think we'll want to link the user to the docs in the onboarding flow.)

Prerequisites

• An Elasticsearch cluster and Kibana (running version 7.9) with a basic license.
• A user with the superuser role.
• Ingest Manager (and Fleet) enabled (ingest docs here)
• OS support: See Column B

Ingest Concepts:

Integration: Ingest Manager provides integrations that bundle various assets needed to ingest and visualize data. See ingest docs for more information. Note: this documention still references Data Source, which is a concept that will be removed in 7.9. Instead users will just see the word "Integration".

Agent Configuration: The Elastic Agent configuration allows you to add and use any number of integrations. You can apply the Elastic Agent configuration to multiple Agents, making it even easier to manage configuration at scale. Users may only add one instance of the Endpoint Security Integration to an Agent Configuration (so, if an Agent Configuration already has the Endpoint Integration associated with it, it cannot be added again). There is also one policy associated with an Endpoint Security integration (think of the Agent Configuration as an extension of Policy).

Our onboarding flow takes users through the process of adding the Elastic Endpoint Integration to an Agent Configuration (the default, or one they create). These steps can be done in a different order if they go through Ingest directly (if you create an Agent Configuration first, you can add the Endpoint Security integration later. Or, if you start with the Endpoint Security integration, it will ask you which Agent Configuration to associate it with.)

After the integration is associated with the Agent Configuration, we automatically create a Policy for them. The policy is configured with "recommended" settings (Malware turned on and set to Prevent, and all event collection set to Yes).

The next step is to enroll the elastic agent on their hosts. We show the instructions in the UI for this (also, instructions are in ingest docs, see here).

Acceptance Test Criteria

Notes