elastic / csp-security-policies

Cloud Security Posture security policies

[DEPRECATED] Cloud Security Posture - Rego policies

This repository has been merged into the cloudbeat repository: https://github.com/elastic/cloudbeat/tree/main/security-policies

Project structure

.
├── bundle
│   ├── compliance                         # Compliance policies
│   │   ├── cis_aws
│   │   │   ├── rules
│   │   │   │   ├── cis_1_8                # CIS AWS 1.8 rule package
│   │   │   │   │   ├── data.yaml          # Rule's metadata
│   │   │   │   │   ├── rule.rego          # Rule's rego
│   │   │   │   │   └── test.rego          # Rule's test
│   │   │   │   ...
│   │   ├── cis_eks
│   │   │   ├── rules
│   │   ├── cis_k8s
│   │   │   ├── rules
│   │   │   ├── schemas                    # Benchmark's schemas
│   │   ├── kubernetes_common
│   │   ├── lib
│   │   │   ├── common                     # Common functions and tests
│   │   │   ├── output_validations
│   │   ├── policy                         # Common audit functions per input
│   │   │   ├── kube_api
│   │   │   ...
├── dev
└── server

Local Evaluation

input.json

should contain a beat/agent output and the benchmark (not mandatory - without specifying benchmark all benchmarks will apply), e.g. k8s eks aws

{
  "type": "file",
  "benchmark": "cis_k8s",
  "sub_type": "file",
  "resource": {
    "mode": "700",
    "path": "/hostfs/etc/kubernetes/manifests/kube-apiserver.yaml",
    "owner": "etc",
    "group": "root",
    "name": "kube-apiserver.yaml",
    "gid": 20,
    "uid": 501
  }
}

Evaluate entire policy into output.json

opa eval data.main --format pretty -i input.json -b ./bundle > output.json

Evaluate findings only

opa eval data.main.findings --format pretty -i input.json -b ./bundle > output.json

Example output

{
  "result": {
    "evaluation": "failed",
    "evidence": {
      "containers": [
        {
          "name": "aws-node",
          "securityContext": {
            "capabilities": {
              "add": ["NET_ADMIN"]
            }
          }
        }
      ]
    }
  },
  "rule": {
    "audit": "Get the set of PSPs with the following command:\n\n```\nkubectl get psp\n```\n\nFor each PSP, check whether capabilities have been forbidden:\n\n```\nkubectl get psp \u003cname\u003e -o=jsonpath='{.spec.requiredDropCapabilities}'\n```",
    "benchmark": {
      "id": "cis_eks",
      "name": "CIS Amazon Elastic Kubernetes Service (EKS)",
      "rule_number": "4.2.9",
      "version": "v1.0.1"
    },
    "default_value": "By default, PodSecurityPolicies are not defined.\n",
    "description": "Do not generally permit containers with capabilities",
    "id": "b28f5d7c-3db2-58cf-8704-b8e922e236b7",
    "impact": "Pods with containers require capabilities to operate will not be permitted.",
    "name": "Minimize the admission of containers with capabilities assigned",
    "profile_applicability": "* Level 2",
    "rationale": "Containers run with a default set of capabilities as assigned by the Container Runtime.\nCapabilities are parts of the rights generally granted on a Linux system to the root user.\n\nIn many cases applications running in containers do not require any capabilities to operate, so from the perspective of the principal of least privilege use of capabilities should be minimized.",
    "references": "1. https://kubernetes.io/docs/concepts/policy/pod-security-policy/#enabling-pod-security-policies\n2. https://www.nccgroup.trust/uk/our-research/abusing-privileged-and-unprivileged-linux-containers/",
    "remediation": "Review the use of capabilites in applications runnning on your cluster.\nWhere a namespace contains applicaions which do not require any Linux capabities to operate consider adding a PSP which forbids the admission of containers which do not drop all capabilities.",
    "section": "Pod Security Policies",
    "tags": ["CIS", "EKS", "CIS 4.2.9", "Pod Security Policies"],
    "version": "1.0"
  }
}

Evaluate with input schema

opa eval data.main --format pretty -i input.json -b ./bundle -s bundle/compliance/cis_k8s/schemas/input_schema.json
1 error occurred: bundle/compliance/lib/data_adapter.rego:11: rego_type_error: undefined ref: input.filenames
        input.filenames
              ^
              have: "filenames"
              want (one of): ["command" "filename" "gid" "mode" "path" "type" "uid"]

Local Testing

Test entire policy

opa build -b ./bundle -e ./bundle/compliance

opa test -b bundle.tar.gz -v

Test specific rule

opa test -v bundle --run 'cis_4_1.test'  # Test the 4.1 rule
opa test -v bundle --run 'cis_(4|5)'     # Test all rules of CIS section 4 and 5

Pre-commit hooks

see pre-commit package

• Install the package brew install pre-commit
• Then run pre-commit install
• Finally pre-commit run --all-files --verbose

Running opa server with the compliance policy

docker run --rm -p 8181:8181 -v $(pwd):/bundle openpolicyagent/opa:0.36.1 run -s -b /bundle

Test it 🚀

curl --location --request POST 'http://localhost:8181/v1/data/main' \
--header 'Content-Type: application/json' \
--data-raw '{
    "input": {
        "type": "file",
        "resource": {
            "type": "file",
            "mode": "700",
            "path": "/hostfs/etc/kubernetes/manifests/kube-apiserver.yaml",
            "uid": "etc",
            "name": "kube-apiserver.yaml",
            "group": "root"
        }
    }
}'

Adding new rules

Add a new rule package to /bundle/compliance/<benchmark>/rules/<rule_name>

1. Add rule.rego file that will contain the rule evaluation logic.
2. Add test.rego file that will contain the rule tests.
3. Generate rule metadata (data.yaml) and templates following the steps in the README

csp-security-policies's Issues

[csp task] 5.2 Pod Security Policies Rules

Rule	Description	Type	Status	PR
5.2.1	Minimize the admission of privileged containers	Automated	Done	#41
5.2.2	Minimize the admission of containers wishing to share the host process ID namespace	Automated	Done	#41
5.2.3	Minimize the admission of containers wishing to share the host IPC namespace	Automated	Done	#41
5.2.4	Minimize the admission of containers wishing to share the host network namespace	Automated	Done	#41
5.2.5	Minimize the admission of containers with allowPrivilegeEscalation	Automated	Done	#41
5.2.6	Minimize the admission of root containers	Automated	Done	#41
5.2.7	Minimize the admission of containers with the NET_RAW capability	Automated	Done	#41
5.2.8	Minimize the admission of containers with added capabilities	Automated	Done	#41
5.2.9	Minimize the admission of containers with capabilities assigned	Manual	Done	#41

[CIS GCP] Implement rules 7.1-7.3

Definition of done

Implement the following rules:

• 7.1 Ensure That BigQuery Datasets Are Not Anonymously or Publicly Accessible
• 7.2 Ensure That All BigQuery Tables Are Encrypted With Customer-Managed Encryption Key
• 7.3 Ensure That a Default Customer-Managed Encryption Key (CMEK) Is Specified for All BigQuery Data Sets

Related tasks/epics

• elastic/cloudbeat#986

File permissions rules that look for exact permissions are currently implemented wrongfully

The current implementation that we have for the file permissions checks is implemented with maximum permissions.
This means that we verify that the permissions are equal or more restrictive.
Yet, we have a single rule 1.1.21 in the Kubernetes benchmark that goes by the name "Ensure that the Kubernetes PKI key file permissions are set to 600 (Manual)".
This rule wants to verify that the permissions are exactly 600 and not more restrictive like all of the other rules.
This means that we aren't compliant with this rule and evaluate the resources incorrectly.
Our code right now will pass any permission that is more restrictive than 600.

[csp task] 1.1 Master Node Configuration Files

Rule	Description	Type	Status	PR
1.1.1	Ensure that the API server pod specification file permissions are set to 644 or more restrictive	Automated	Done	#1
1.1.2	Ensure that the API server pod specification file ownership is set to root:root	Automated	Done	#1
1.1.3	Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive	Automated	Done	#2
1.1.4	Ensure that the controller manager pod specification file ownership is set to root:root	Automated	Done	#5
1.1.5	Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive	Automated	Done	#2
1.1.6	Ensure that the scheduler pod specification file ownership is set to root:root	Automated	Done	#5
1.1.7	Ensure that the etcd pod specification file permissions are set to 644 or more restrictive	Automated	Done	#2
1.1.8	Ensure that the etcd pod specification file ownership is set to root:root	Automated	Done	#5
1.1.9	Ensure that the Container Network Interface file permissions are set to 644 or more restrictive	Manual	TBD
1.1.10	Ensure that the Container Network Interface file ownership is set to root:root	Manual	TBD
1.1.11	Ensure that the etcd data directory permissions are set to 700 or more restrictive	Automated	Done	#7
1.1.12	Ensure that the etcd data directory ownership is set to etcd:etcd	Automated	Done	#7
1.1.13	Ensure that the admin.conf file permissions are set to 644 or more restrictive	Automated	Done	#2
1.1.14	Ensure that the admin.conf file ownership is set to root:root	Automated	Done	#5
1.1.15	Ensure that the scheduler.conf file permissions are set to 644 or more restrictive	Automated	Done	#2
1.1.16	Ensure that the scheduler.conf file ownership is set to root:root	Automated	Done	#5
1.1.17	Ensure that the controller-manager.conf file permissions are set to 644 or more restrictive	Automated	Done	#2
1.1.18	Ensure that the controller-manager.conf file ownership is set to root:root	Automated	Done	#5
1.1.19	Ensure that the Kubernetes PKI directory and file ownership is set to root:root	Automated	Done	#7
1.1.20	Ensure that the Kubernetes PKI certificate file permissions are set to 644 or more restrictive	Manual	TBD
1.1.21	Ensure that the Kubernetes PKI key file permissions are set to 600	Manual	TBD

[Bug] - CIS 4.2.7 when there is no process `--make-iptables-util-chains` flag rule should pass

Describe the bug
When a Kubernetes cluster runs against rule Vanilla rule 4.2.7, and tests its processes. When Kubelet process does not have --make-iptables-util-chains flag, and the value does not exist in the kubelet configuration file, the rule should pass.

To Reproduce

1. Run rule 4.2.7 against kubelet process resource with no --make-iptables-util-chainsflag and no corllating value in the config.
2. The rule return failed evaluation.

Expected result
The rule should pass.

Create the IAM Rego rules

Implement the automated rules of section 1 of the "CIS AWS" benchmark.

• 1.4 Ensure no 'root' user account access key exists
• 1.5 Ensure MFA is enabled for the 'root' user account
• 1.6 Ensure hardware MFA is enabled for the 'root' user account
• 1.7 Eliminate the use of the 'root' user for administrative and daily tasks
• 1.8 Ensure IAM password policy requires a minimum length of 14 or greater
• 1.9 Ensure IAM password policy prevents password reuse
• 1.10 Ensure multi-factor authentication (MFA) is enabled for all IAM users that have a console password
• 1.11 Do not setup access keys during initial user setup for all IAM users that have a console password
• 1.12 Ensure credentials unused for 45 days or greater are disabled
• 1.13 Ensure there is only one active access key available for any single IAM user
• 1.14 Ensure access keys are rotated every 90 days or less
• 1.15 Ensure IAM Users Receive Permissions Only Through Groups
• 1.16 Ensure IAM policies that allow full ":" administrative privileges are not attached
• 1.17 Ensure a support role has been created to manage incidents with AWS Support
• 1.19 Ensure that all the expired SSL/TLS certificates stored in AWS IAM are removed
• 1.20 Ensure that IAM Access analyzer is enabled for all regions

s3 data adapter produces rego errors

I noticed that one of our rules for s3 buckets produces some errors during evaluation:

cc @uri-weisman @oren-zohar

Improve k8s evaluation

Investigate the k8s evaluation and improve it.
Just like I did for file and process.

automate implemented rules mapping

Motivation

Adding automation to track our rule implementation progress can be nice

DoD

• Add a script that generates a markdown table with all the relevant rules and their status
• Add this to out pre-commit hooks
• Add it as a CI step

CIS v1.23 - Fix rule `4.2.12` default value

As part of the migration to CIS benchmark v1.23 rule, Rego rule 4.2.12 needs to be updated.
Default value of rule 4.2.12 has changed - this results in metadata and Rego changes.

4.2.12 Verify that the RotateKubeletServerCertificate argument is set to
true (Manual)

Default value change:
Old: By default, kubelet server certificate rotation is disabled.
New: By default, kubelet server certificate rotation is enabled.

Originally posted by @eyalkraft in https://github.com/elastic/security-team/issues/3719#issuecomment-1114276607

** DOD **

• update rego and metadata in csp-rego-policies
• update integration rule

(or delete)

Implement IAM rule 1.17: Ensure a support role has been created to manage incidents with AWS Support

Split from #178

[csp task] 1.4 Scheduler rules

Rule	Description	Type	Status	PR
1.4.1	Ensure that the --profiling argument is set to false	Automated	In Progress	#16
1.4.2	Ensure that the --bind-address argument is set to 127.0.0.1	Automated	In Progress	#16

Implement the RDS related Rego rules

Implement the following rules:

• 2.3.1 Ensure that encryption is enabled for RDS Instances
• 2.3.2 Ensure Auto Minor Version Upgrade feature is Enabled for RDS Instances
• 2.3.3 Ensure that public access is not given to RDS Instance

---

• Implement unit tests to ensure rules work as expected.

Implement CIS AWS S3 rule 2.1.5

Implement the following AWS CIS rule:

• Implement CIS AWS rule 2.1.5 - Ensure that S3 Buckets are configured with 'Block public
  access (bucket settings)'
• Implement required fetching logic in cloudbeat

Policy performance degradation

Background
As we transitioned to the new policy structure (introduced with v1.2.0), we encountered a major performance degradation in our cloudbeat functional tests (we had to edit our timeouts to work around it).

Issue Summary
As a result of @gurevichdmitry and @kfirpeled's work on benchmarking our different policy versions, we determined that there was a performance hit of approximately ~50% between v1.0.8 and v1.2.0.

Besides that, there's a performance hit of ~75% when not supplying the list of activated rules

Definition of Done

• Determine the root cause of the performance degradation
• Is there a way to quickly resolve the degradation? Let's fix it if that is the case.
• Define internal performance KPI for our policy

---

• #158
• Improve process evaluation #137
• Improve file-system evaluation #141

Out of scope

• elastic/cloudbeat#1507
• Does starting the next cycle before the previous one was evaluated make sense?

Benchmarks results:

Version	w/o rule config - req/s	w/ rule config - req/s
1.0.8	66.86	128.50
1.2.0	16.21	61.19

The higher the number the better (more requests per second)

1.0.8 k8s_obj:

❯ ab -c 10 -n 1000 -p ./input_k8sobj.json http://localhost:8182/v1/data/main
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:
Server Hostname:        localhost
Server Port:            8182

Document Path:          /v1/data/main
Document Length:        42869 bytes

Concurrency Level:      10
Time taken for tests:   14.957 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      42957000 bytes
Total body sent:        3755000
HTML transferred:       42869000 bytes
Requests per second:    66.86 [#/sec] (mean)
Time per request:       149.569 [ms] (mean)
Time per request:       14.957 [ms] (mean, across all concurrent requests)
Transfer rate:          2804.73 [Kbytes/sec] received
                        245.17 kb/s sent
                        3049.90 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       3
Processing:    56  148  34.8    146     347
Waiting:       55  145  34.6    144     346
Total:         56  148  34.8    146     347

Percentage of the requests served within a certain time (ms)
  50%    146
  66%    159
  75%    167
  80%    174
  90%    190
  95%    209
  98%    229
  99%    254
 100%    347 (longest request)

1.2.0 k8s_obj:

❯ ab -c 10 -n 1000 -p ./input_k8sobj.json http://localhost:8181/v1/data/main
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:
Server Hostname:        localhost
Server Port:            8181

Document Path:          /v1/data/main
Document Length:        42921 bytes

Concurrency Level:      10
Time taken for tests:   61.679 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      43009000 bytes
Total body sent:        3755000
HTML transferred:       42921000 bytes
Requests per second:    16.21 [#/sec] (mean)
Time per request:       616.788 [ms] (mean)
Time per request:       61.679 [ms] (mean, across all concurrent requests)
Transfer rate:          680.96 [Kbytes/sec] received
                        59.45 kb/s sent
                        740.42 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       2
Processing:   286  613  73.9    609     890
Waiting:      284  610  73.5    606     888
Total:        286  613  73.9    609     890

Percentage of the requests served within a certain time (ms)
  50%    609
  66%    637
  75%    658
  80%    671
  90%    709
  95%    740
  98%    781
  99%    796
 100%    890 (longest request)

1.0.8 k8s_obj with vanilla activated rules:

❯ ab -c 10 -n 1000 -p ./input_k8sobj.json http://localhost:8182/v1/data/main
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:
Server Hostname:        localhost
Server Port:            8182

Document Path:          /v1/data/main
Document Length:        24131 bytes

Concurrency Level:      10
Time taken for tests:   7.782 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      24219000 bytes
Total body sent:        5800000
HTML transferred:       24131000 bytes
Requests per second:    128.50 [#/sec] (mean)
Time per request:       77.823 [ms] (mean)
Time per request:       7.782 [ms] (mean, across all concurrent requests)
Transfer rate:          3039.12 [Kbytes/sec] received
                        727.81 kb/s sent
                        3766.93 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       3
Processing:    26   77  29.9     74     189
Waiting:       25   73  29.5     70     189
Total:         26   77  29.9     74     189

Percentage of the requests served within a certain time (ms)
  50%     74
  66%     86
  75%     94
  80%     98
  90%    116
  95%    134
  98%    152
  99%    167
 100%    189 (longest request)

1.2.0 k8s_obj with activated_rules:

❯ ab -c 10 -n 1000 -p ./input_k8sobj.json http://localhost:8181/v1/data/main
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:
Server Hostname:        localhost
Server Port:            8181

Document Path:          /v1/data/main
Document Length:        24157 bytes

Concurrency Level:      10
Time taken for tests:   16.341 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      24245000 bytes
Total body sent:        5800000
HTML transferred:       24157000 bytes
Requests per second:    61.19 [#/sec] (mean)
Time per request:       163.414 [ms] (mean)
Time per request:       16.341 [ms] (mean, across all concurrent requests)
Transfer rate:          1448.89 [Kbytes/sec] received
                        346.61 kb/s sent
                        1795.49 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.4      0      11
Processing:    61  161  39.1    157     336
Waiting:       61  159  38.4    155     336
Total:         61  162  39.1    157     337

Percentage of the requests served within a certain time (ms)
  50%    157
  66%    175
  75%    186
  80%    192
  90%    213
  95%    232
  98%    255
  99%    265
 100%    337 (longest request)

Implement IAM rule 1.19: Ensure that all the expired SSL/TLS certificates stored in AWS IAM are removed

Split from #178

Implement the EC2 related rego rules

• Implement rule 2.2.1 - Ensure EBS Volume Encryption is Enabled in all Regions
• Implement unit tests to ensure that the rule works as expected.

Implement CIS AWS RDS rule 2.3.3

Implement the following AWS CIS rule:

• Implement CIS AWS rule 2.3.3 - Ensure that public access is not given to RDS Instance
• Implement required fetching logic in cloudbeat

[CIS GCP] Implement rules 1.9-1.10 of the CIS GCP becnhmark

Definition of done

• 1.9 Ensure That Cloud KMS Cryptokeys Are Not Anonymously or Publicly Accessible
• 1.10 Ensure KMS Encryption Keys Are Rotated Within a Period of 90 Days

Related tasks/epics

• elastic/cloudbeat#985

Implement IAM rule 1.20: Ensure that IAM Access analyzer is enabled for all regions

Split from #178

[CIS AWS] Rule 3.3 fails on some valid bucket policies

Describe the bug

Rule 3.3 fails on some valid bucket policies, for example:

[
  {
    "Action": "s3:GetBucketAcl",
    "Resource": "arn:aws:s3:::elastic-inf-logs",
    "Effect": "Allow",
    "Principal": {
      "Service": "cloudtrail.amazonaws.com"
    },
    "Sid": "AWSCloudTrailAclCheck20150319"
  },
  {
    "Condition": {
      "StringEquals": {
        "s3:x-amz-acl": "bucket-owner-full-control"
      }
    },
    "Action": "s3:PutObject",
    "Resource": "arn:aws:s3:::elastic-inf-logs/AWSLogs/887624222702/*",
    "Effect": "Allow",
    "Principal": {
      "Service": "cloudtrail.amazonaws.com"
    },
    "Sid": "AWSCloudTrailWrite20150319"
  }
] 

Preconditions

Run an 8.7 elastic agent with a CSPM integration.

To Reproduce

Steps to reproduce the behavior:

1. Create an cloud trail with the above bucket policy
2. AWS CIS Rule 3.3 will fail

Expected behavior

The rule should pass

[csp task] 4.1.x Worker Node Configuration rules

Rule	Description	Type	Status	PR
4.1.1	Ensure that the kubelet service file permissions are set to 644 or more restrictive	Automated	In Progress	#20
4.1.2	Ensure that the kubelet service file ownership is set to root:root	Automated	In Progress	#20
4.1.3	If proxy kubeconfig file exists ensure permissions are set to 644 or more restrictive	Manual	TBD
4.1.4	If proxy kubeconfig file exists ensure ownership is set to root:root	Manual	TBD
4.1.5	Ensure that the --kubeconfig kubelet.conf file permissions are set to 644 or more restrictive	Automated	In Progress	#20
4.1.6	Ensure that the --kubeconfig kubelet.conf file ownership is set to root:root	Manual	TBD
4.1.7	Ensure that the certificate authorities file permissions are set to 644 or more restrictive	Manual	TBD
4.1.8	Ensure that the client certificate authorities file ownership is set to root:root	Manual	TBD
4.1.9	Ensure that the kubelet --config configuration file has permissions set to 644 or more restrictive	Automated	In Progress	#20
4.1.10	Ensure that the kubelet --config configuration file ownership is set to root:root	Automated	In Progress	#20

tests for common/util functions

Add tests to common/utils functions

Implement the remaining IAM rules

Motivation

Implement the remaining rules of the IAM fetcher.

Definition of done

• 1.16 Ensure IAM policies that allow full ":" administrative privileges are not attached
• 1.17 Ensure a support role has been created to manage incidents with AWS Support
• 1.19 Ensure that all the expired SSL/TLS certificates stored in AWS IAM are removed
• 1.20 Ensure that IAM Access analyzer is enabled for all regions

Implement S3 related Rego rules

• Implement the automated rules of section 2.1 of the "CIS AWS" benchmark
• Implement unit tests to ensure rules work as expected.

CIS AWS rule number 2.1.5 does not generate findings

Describe the bug

The rule does not generate any findings.

Preconditions

CSPM + 8.8 (BC3) env

To Reproduce

1. Install the CSPM integration and notice that no findings related to rule 2.1.5 are being generated.

Screenshots

Desktop (please complete the following information):

• Kibana Version: VERSION: 8.8.0, BUILD: 62994, COMMIT: 85b22d307ab93fca95c1698ede4cb61d85f3d314
• Endpoint Version:
• Other Version:

Related:

• #194

Implement the Networking rules part of the AWS CIS benchmark

• Implement the automated rules of section 5 of the "CIS AWS" benchmark
• Implement unit tests to ensure rules work as expected.

** Make sure to re-use relevant implemented audit functions

Implement the Logging rules in the AWS CIS benchmark

Implement the automated rules of section 3 of the "CIS AWS" benchmark:

• 3.1 Ensure CloudTrail is enabled in all regions.
• 3.2 Ensure CloudTrail log file validation is enabled
• 3.3 Ensure the S3 bucket used to store CloudTrail logs is not publicly accessible
• 3.4 Ensure CloudTrail trails are integrated with CloudWatch Logs
• 3.5 Ensure AWS Config is enabled in all regions (8.8)
• 3.6 Ensure S3 bucket access logging is enabled on the CloudTrail S3 bucket
• 3.7 Ensure CloudTrail logs are encrypted at rest using KMS CMKs
• 3.8 Ensure rotation for customer created symmetric CMKs is enabled (8.8)
• 3.9 Ensure VPC flow logging is enabled in all VPCs
• 3.10 Ensure that Object-level logging for write events is enabled for S3 bucket
• 3.11 Ensure that Object-level logging for read events is enabled for S3 bucket
• Implement unit tests to ensure rules work as expected.

Related Tasks:

• #180

Process input refactoring

• refactor data_adapter to is_process and process_command that will contain the command name e.g. kube-api
• command_args does the filtering implicitly -> need to split the filter into the rule explicitly
• Make process_input for all process types in test_data.rego:

Update EKS rules metadata test

Motivation
As a developer, I would like to have an updated rule metadata test.

In-depth
Lately, we have updated the schema of the EKS metadata, but we didn't update the relevant test to make sure all these new fields exist.

DOD

• Make sure all the relevant fields are being tested within the metadata schema test.

Rule Schema Validations

Following rule field schema defined in https://github.com/elastic/security-team/issues/2246 we need to do the following:

• Add Schema validation to policy inputs
• Add Rule Schema validation tests to the rules outputs
• Add CI validations

Rule numbering is wrong for rule 1.2.32

Looking to rule 1.2.32 the numbering of the rule is wrong.
https://github.com/elastic/csp-security-policies/blob/main/bundle/compliance/cis_k8s/rules/cis_1_2_32

I had a look to the CIS PDF and I have seen the rule should be 1.2.31 can we please correct that please?

A second issue I had is that while trying to mitigate this issue with api server by adding the flag I ended up with errors like that:
Cipher suite TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA not supported or doesn't exist

Turns out the whole mitigation would be that it needs added the flag that TLS version minimum should be TLSv1.2.
Maybe we could mention the flag in the rule content:
--tls-min-version=VersionTLS12

[csp task] 1.3 Controller Manager rules

Rule	Description	Type	Status	PR
1.3.1	Ensure that the --terminated-pod-gc-threshold argument is set as appropriate	Manual	TBD
1.3.2	Ensure that the --profiling argument is set to false	Automated	Done	#14
1.3.3	Ensure that the --use-service-account-credentials argument is set to true	Automated	Done	#14
1.3.4	Ensure that the --service-account-private-key-file argument is set as appropriate	Automated	Done	#14
1.3.5	Ensure that the --root-ca-file argument is set as appropriate	Automated	Done	#14
1.3.6	Ensure that the RotateKubeletServerCertificate argument is set to true	Automated	Done	#14
1.3.7	Ensure that the --bind-address argument is set to 127.0.0.1+	Automated	Done	#14

[CIS GCP] Implement the GCS rules

Definition of done

Implement the following rules:

• 5.1 Ensure That Cloud Storage Bucket Is Not Anonymously or Publicly Accessible
• 5.2 Ensure That Cloud Storage Buckets Have Uniform Bucket- Level Access Enabled

Related tasks/epics

• elastic/cloudbeat#987

[csp task] 1.2 API Server rules

Rule	Description	Type	Status	PR
1.2.1	Ensure that the --anonymous-auth argument is set to false	Manual	TBD
1.2.2	Ensure that the --basic-auth-file argument is not set	Automated	Done	#11
1.2.3	Ensure that the --token-auth-file parameter is not set	Automated	Done	#12
1.2.4	Ensure that the --kubelet-https argument is set to true	Automated	Done	#12
1.2.5	Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate	Automated	Done	#12
1.2.6	Ensure that the --kubelet-certificate-authority argument is set as appropriate	Automated	Done	#12
1.2.7	Ensure that the --authorization-mode argument is not set to AlwaysAllow	Automated	Done	#12
1.2.8	Ensure that the --authorization-mode argument includes Node	Automated	Done	#12
1.2.9	Ensure that the --authorization-mode argument includes RBAC	Automated	Done	#12
1.2.10	Ensure that the admission control plugin EventRateLimit is set	Manual	TBD
1.2.11	Ensure that the admission control plugin AlwaysAdmit is not set	Automated	Done	#12
1.2.12	Ensure that the admission control plugin AlwaysPullImages is set	Manual	TBD
1.2.13	Ensure that the admission control plugin SecurityContextDeny is set if PodSecurityPolicy is not used	Manual	TBD
1.2.14	Ensure that the admission control plugin ServiceAccount is set	Automated	Done	#12
1.2.15	Ensure that the admission control plugin NamespaceLifecycle is set	Automated	Done	#12
1.2.16	Ensure that the admission control plugin PodSecurityPolicy is set	Automated	Done	#12
1.2.17	Ensure that the admission control plugin NodeRestriction is set	Automated	Done	#12
1.2.18	Ensure that the --insecure-bind-address argument is not set	Automated	Done	#12
1.2.19	Ensure that the --insecure-port argument is set to 0	Automated	Done	#12
1.2.20	Ensure that the --secure-port argument is not set to 0	Automated	Done	#12
1.2.21	Ensure that the --profiling argument is set to false	Automated	Done	#12
1.2.22	Ensure that the --audit-log-path argument is set	Automated	Done	#12
1.2.23	Ensure that the --audit-log-maxage argument is set to 30 or as appropriate	Automated	Done	#12
1.2.24	Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate	Automated	Done	#12
1.2.25	Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate	Automated	Done	#12
1.2.26	Ensure that the --request-timeout argument is set as appropriate	Automated	Done	#12
1.2.27	Ensure that the --service-account-lookup argument is set to true	Automated	Done	#12
1.2.28	Ensure that the --service-account-key-file argument is set as appropriate	Automated	Done	#12
1.2.29	Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate	Automated	Done	#12
1.2.30	Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate	Automated	Done	#12
1.2.31	Ensure that the --client-ca-file argument is set as appropriate	Automated	Done	#12
1.2.32	Ensure that the --etcd-cafile argument is set as appropriate	Automated	Done	#12
1.2.33	Ensure that the --encryption-provider-config argument is set as appropriate	Manual	TBD
1.2.34	Ensure that encryption providers are appropriately configured	Manual	TBD
1.2.35	Ensure that the API Server only makes use of Strong Cryptographic Ciphers	Manual	TBD

Implement IAM rule 1.16: Ensure that IAM Access analyzer is enabled for all regions

Split from #178

[CIS AWS] Rule 1.12 fail when a new created IAM user didn't access console

Describe the bug

Rule 1.12 fail when evaluating a newly created IAM user that didn't access the console.

Preconditions

Run an 8.8 elastic agent with a CSPM integration.

To Reproduce

Write the exact actions one should perform in order to reproduce the bug.
Steps to reproduce the behavior:

1. Create an IAM user with an active access key.
2. Don't access the AWS console.
3. AWS CIS Rule 1.12 will fail.

Expected behavior

The rule should pass if the console password has changed in the last 45 days (Rego code).

The benchmark refers to a case where the password wasn’t used as No_Information while the AWS SDK actually returns no_information - this mismatch cause us to fail in the evaluation.

Resource:

"resource": {
    "sub_type": "aws-iam-user",
    "name": "cspm_testing",
    "raw": {
      "last_access": "no_information",
      "attached_policies": [
        {
          "PolicyArn": "arn:aws:iam::aws:policy/SecurityAudit",
          "PolicyName": "SecurityAudit"
        }
      ],
      "password_enabled": true,
      "mfa_active": false,
      "name": "cspm_testing",
      "arn": "arn:aws:iam::887624222702:user/cspm_testing",
      "inline_policies": [],
      "access_keys": [
        {
          "last_access": "2023-03-08T11:09:00+00:00",
          "has_used": true,
          "active": true,
          "rotation_date": "2023-03-07T16:36:06+00:00"
        },
        {
          "last_access": "N/A",
          "has_used": false,
          "active": false,
          "rotation_date": "N/A"
        }
      ],
      "password_last_changed": "2023-03-02T15:01:41+00:00"
    },
    "id": "arn:aws:iam::887624222702:user/cspm_testing",
    "type": "identity-management"
  }

use assert.is_false instead of == false

Some rules checks util.func() == "false" since we can't type not util.func(). a better solution would be to add assert.is_false

Implement config rule for CIS AWS

Motivation

Implement rule 3.5 of the CIS AWS benchmark.

Definition of done

• 3.5 Ensure AWS Config is enabled in all regions
• Unit tests for this rule.

Related tasks/epics

• elastic/cloudbeat#535

Implement Kubernetes v1.20 CIS v1.0.0 benchmark

The benchmark rules we implemented were of an older version of Kubernetes, this version is already deprecated. new version is Kubernetes v1.20 CIS v1.0.0 benchmark.

I would assume we will not split it to two benchmarks as done in the CIS site, since the older kube version is already EOL for more than a year now, just update the current implementation @amar-rama-elastic wdyt?.

From a quick look, it seems like the main difference is PSP rules are now automated, @yashtewari if we have a fetcher for this data already can you please share how will it look as an input for the policy?

• see https://github.com/elastic/security-team/issues/2510

Section	Status	Issue link	to-be-implemented	User-Input-Needed
5.1 RBAC and Service Accounts	Done	#64	4/8	4/8
5.2 Pod Security Policies	Done	#38	9/9	0/0
5.3 Network Policies and CNI	Missing Data		1/2	1/2
5.4 Secrets Management	TBD		1/2	1/2
5.5 Extensible Admission Control	TBD		0/0	1/1
5.7 General Policies	TBD		2/4	2/4

---

• sheet: https://docs.google.com/spreadsheets/d/1OIKbKSmDqcNtObb50mhse027hx0mfrQQwHepy2CoBa0/edit#gid=0

Migrate Template/Metadata generator to python

Motivation
Currently, we have an old script in TS that was implemented by Kanfer. This script is hard to maintain and has some undocumented assumptions. This led to many discrepancies in our benchmarks metadata, and as a result also in the templates.

Definitaiton of done

• Migrate Metadata generator - #160
• Migrate Templates generator - #172
• Add CI step to enforce consistent metadata - #166

Create CIS AWS rules templates

Motivation

Create the rules templates that will be presented in Kibana.

Definition of done

• Get the newest CSV version of the CIS benchmark.
• Use the generator to generate the policy metadata.
• Review metadata for mistakes.

Before closing this ticket

• Reference to tech-debts that shall be solved as we move forward

[csp task] 2.x Scheduler rules

Rule	Description	Type	Status	PR
2.1	Ensure that the --cert-file and --key-file arguments are set as appropriate	Automated	In Progress	#18
2.2	EEnsure that the --client-cert-auth argument is set to true	Automated	In Progress	#18
2.3	Ensure that the --auto-tls argument is not set to true	Automated	In Progress	#18
2.4	Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate	Automated	In Progress	#18
2.5	Ensure that the --peer-client-cert-auth argument is set to true	Automated	In Progress	#18
2.6	Ensure that the --peer-auto-tls argument is not set to true	Automated	In Progress	#18
2.7	Ensure that a unique Certificate Authority is used for etcd	Manual	TBD

[csp task] 4.2.x Kubelet Rules

Rule	Description	Type	Status	PR
4.2.1	Ensure that the --anonymous-auth argument is set to false	Automated	In Progress	#22
4.2.2	Ensure that the --authorization-mode argument is not set to AlwaysAllow	Automated	In Progress	#22
4.2.3	Ensure that the --client-ca-file argument is set as appropriate	Automated	In Progress	#22
4.2.4	Verify that the --read-only-port argument is set to 0	Manual	TBD
4.2.5	Ensure that the --streaming-connection-idle-timeout argument is not set to 0	Manual	TBD
4.2.6	Ensure that the --protect-kernel-defaults argument is set to true	Automated	In Progress	#22
4.2.7	Ensure that the --make-iptables-util-chains argument is set to true	Automated	In Progress	#22
4.2.8	Ensure that the --hostname-override argument is not set	Manual	TBD
4.2.9	Ensure that the --event-qps argument is set to 0 or a level which ensures appropriate event capture	Manual	TBD
4.2.10	Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate	Manual	TBD
4.2.11	Ensure that the --rotate-certificates argument is not set to false	Manual	TBD
4.2.12	Verify that the RotateKubeletServerCertificate argument is set to true	Manual	TBD
4.2.13	Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers	Manual	TBD

[Bug] - When there is `-request-timeout` argument Vanilla CIS 1.2.25 should pass

Describe the bug
When running the Vanilla CIS 1.2.25 with a kube-apiserver process as input with no -request-timeout flag, and no relevant value in the configuration file the rule fails.

To Reproduce

1. Run rule 1.2.25 with a kube-apiserver when there is no -request-timeout flag, and no relevant value in the configuration
2. rule will return result.evaluation: failed

Expected result
The rule should return result.evaluation: pass

Evidence from the CIS benchmark:

Audit
Run the following command on the control plane node:
ps -ef | grep kube-apiserver
Verify that the --request-timeout argument is either not set or set to an appropriate
value.

Repo polishing

• #34
• function docs alignment
• #30
• #33
• contains_key_with_value -> contains_key + has_value
• Check integration with opabeat after all is done

Implement the KMS rule for CIS AWS

Motivation

Implement rule 3.8 of the CIS AWS benchmark.

Definition of done

• 3.8 Ensure rotation for customer created symmetric CMKs is enabled
• Unit tests for this rule.

Out of scope

What should not be included in this task

Related tasks/epics

• elastic/cloudbeat#704

[CIS GCP] Create CIS GCP policy metadata

Definition of done

• Get the latest CSV version of the CIS GCP benchmark
• Use the generator to generate the policy metadata
• Review metadata for mistakes

Implement the monitoring rules in the AWS CIS benchmark

Implement the automated rules of section 4 of the "CIS AWS" benchmark:

• 4.1 Ensure a log metric filter and alarm exist for unauthorized API calls
• 4.2 Ensure a log metric filter and alarm exist for Management Console sign-in without MFA
• 4.3 Ensure a log metric filter and alarm exist for usage of 'root' account
• 4.4 Ensure a log metric filter and alarm exist for IAM policy changes
• 4.5 Ensure a log metric filter and alarm exist for CloudTrail configuration changes
• 4.6 Ensure a log metric filter and alarm exist for AWS Management Console authentication failures
• 4.7 Ensure a log metric filter and alarm exist for disabling or scheduled deletion of customer created CMKs
• 4.8 Ensure a log metric filter and alarm exist for S3 bucket policy changes
• 4.9 Ensure a log metric filter and alarm exist for AWS Config configuration changes
• 4.10 Ensure a log metric filter and alarm exist for security group changes
• 4.11 Ensure a log metric filter and alarm exist for changes to Network Access Control Lists (NACL)
• 4.12 Ensure a log metric filter and alarm exist for changes to network gateways
• 4.13 Ensure a log metric filter and alarm exist for route table changes
• 4.14 Ensure a log metric filter and alarm exist for VPC changes
• 4.15 Ensure a log metric filter and alarm exists for AWS Organizations changes
• 4.16 Ensure AWS Security Hub is enabled
• Implement unit tests to ensure rules work as expected.

fix cloudbeat CI when working with forks