jfrog / kubexray

JFrog KubeXray scanner on Kubernetes

Deprecation Notice

Note: KubeXray is no longer maintained or supported by JFrog. Feel free to review this code for your own POC concepts, but we are not continuing to update it or add features. For people looking for great tools to help for enforcement in Kubernetes, we do continue to have KubeNab which allows enforcement of what repositories a kubernetes cluster pulls from (which then can leverage enforcement of Xray policies in Artifactory).*

JFrog KubeXray scanner on Kubernetes

An open source software project that monitors pods in a Kubernetes cluster to help you detect security & license violations in containers running inside the pod.

KubeXray listens to events from Kubernetes API server, and leverages the metadata from JFrog Xray (commercial product) to ensure that only the pods that comply with your current policy can run on Kubernetes. As an example, KubeXray listens to these event streams:

• Deployment of a new service
• Upgrade of an existing service
• A new license policy, such as a new license type disallowed for runtime.
• A new security issue

And when an issue is detected, KubeXray responds according to the current policy that you have set.

You can select one of the following possible actions:

• Scaledown to 0. The desired state of a service's replica count is updated to 0, making the services inactive but still traceable.
• Delete the corresponding Kubernetes resource that’s pointing to a vulnerable container image(s)
• Ignore and leave the pod running

KubeXray also allows you to enforce policy for running applications that have not been scanned by JFrog Xray and whose risks are unknown.

Install Instructions

The easiest way to install KubeXray is using the Helm chart

Please follow install instruction from chart's readme

Local development and testing

Building binary

To build kubexray locally

make build

Docker

To build kubexray docker image locally (testing docker image build)

make image

Contributing Code

We welcome community contribution through pull requests.

License

This tool is available under the Apache License, Version 2.0.

(c) All rights reserved JFrog

kubexray's Issues

Add support for Environment variable to change logging level in helm chart.

Add daemonset support

it might be part of #21

Add support for SNS

We'd like to automate actions based on the scan results. Being able to push to SNS would be very useful.

Move to admission controller

To be more officiant and have more control of pods livecycle it makes more sense to move of using admission controllers.

Blogpost links to invalid url

Hello,

Currently, the blogpost announcing this project links to https://github.com/jfrog/kubexray/Read.me instead of https://github.com/jfrog/kubexray/blob/master/README.md, the specific section with the incorrect url is in the 5th paragraph from the bottom.

slice bounds out of range panic on pod from daemonset

When running kubexray on our cluster we get the following panic:

/usr/local/go/src/runtime/panic.go:522                                                                                                                                                 [39/267]
/usr/local/go/src/runtime/panic.go:54
/build/kubexray/handler.go:527
/build/kubexray/handler.go:359
/build/kubexray/controller.go:127
/build/kubexray/controller.go:56
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:134
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:88
/build/kubexray/controller.go:47
/usr/local/go/src/runtime/asm_amd64.s:1337
E0329 18:50:01.174685       1 runtime.go:69] Observed a panic: "slice bounds out of range" (runtime error: slice bounds out of range)
/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:76
/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:65
/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:51
/usr/local/go/src/runtime/panic.go:522
/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:58
/usr/local/go/src/runtime/panic.go:522
/usr/local/go/src/runtime/panic.go:54
/build/kubexray/handler.go:527
/build/kubexray/handler.go:359
/build/kubexray/controller.go:127
/build/kubexray/controller.go:56
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:134
/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:88
/build/kubexray/controller.go:47
/usr/local/go/src/runtime/asm_amd64.s:1337
panic: runtime error: slice bounds out of range [recovered]
        panic: runtime error: slice bounds out of range [recovered]
        panic: runtime error: slice bounds out of range

goroutine 24 [running]:
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:58 +0x105
panic(0x10b3d60, 0x1cf0b90)
        /usr/local/go/src/runtime/panic.go:522 +0x1b5
k8s.io/apimachinery/pkg/util/runtime.HandleCrash(0x0, 0x0, 0x0)
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:58 +0x105
panic(0x10b3d60, 0x1cf0b90)
        /usr/local/go/src/runtime/panic.go:522 +0x1b5
main.checkResource(0x13d4880, 0xc000112700, 0xc0002cdc00, 0x1, 0xb, 0xf)
        /build/kubexray/handler.go:527 +0x52c
main.(*HandlerImpl).ObjectCreated(0xc00026a480, 0x13d4880, 0xc000112700, 0x11d28c0, 0xc0002cdc00)
        /build/kubexray/handler.go:359 +0xce
main.(*Controller).processNextQueueItem(0xc000209ef0, 0xc000631e00)
        /build/kubexray/controller.go:127 +0x2cf
main.(*Controller).runWorker(0xc000209ef0)
        /build/kubexray/controller.go:56 +0xcb
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc00005d788)
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x54
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000631f88, 0x3b9aca00, 0x0, 0x1, 0xc00008a900)
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:134 +0xf8
k8s.io/apimachinery/pkg/util/wait.Until(...)
        /go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:88
main.(*Controller).Run(0xc000209ef0, 0xc00008a900)
        /build/kubexray/controller.go:47 +0x2f1
created by main.main
        /build/kubexray/main.go:165 +0x758

I believe this happens because we have a pod from a daemonset that doesn't adhere to a naming scheme that is expected in the checkResource function:

func checkResource(client kubernetes.Interface, pod *core_v1.Pod) (string, ResourceType) {
	subs1 := strings.LastIndexByte(pod.Name, '-')
	subs2 := strings.LastIndexByte(pod.Name[:subs1], '-')
	sets := client.AppsV1().StatefulSets(pod.Namespace)
	_, err := sets.Get(pod.Name[:subs1], meta_v1.GetOptions{})
	if err == nil {
		return pod.Name[:subs1], StatefulSet
	}
	log.Debugf("Resource for pod %s is not stateful set %s: %v", pod.Name, pod.Name[:subs1], err)
	deps := client.AppsV1().Deployments(pod.Namespace)
	_, err = deps.Get(pod.Name[:subs2], meta_v1.GetOptions{})
	if err == nil {
		return pod.Name[:subs2], Deployment
	}
	log.Debugf("Resource for pod %s is not deployment %s: %v", pod.Name, pod.Name[:subs2], err)
	return "", Unrecognized
}

From the debug logs:

time="2019-03-29T19:02:59Z" level=debug msg=HandlerImpl.ObjectCreated
time="2019-03-29T19:02:59Z" level=debug msg="Resource for pod falco-wzdl4 is not stateful set falco: statefulsets.apps \"falco\" not found"
E0329 19:02:59.457975       1 runtime.go:69] Observed a panic: "slice bounds out of range" (runtime error: slice bounds out of range)

The falco-wzdl4 pod doesn't contain two dashes. Please advise on this issue.

Enable securityContext in the helm chart by default & fix description

What?
securityContext should be enabled by default, with default values. The description needs to be fixed.

Why?
Security configurations (even remote ones) should be always enabled by default.

Whitelisting namespaces are not honored

The whitelisting is not honored and no matter the order in which its provided, there is not any help. Xray version is Xray 2.3.3 Revision: 6b3b534

Also the response from api has issues and licenses as separate attributes
{"artifacts":[{........},"issues":[],"licenses":[{"name":"Unknown","full_name":"Unknown license","components":[........]}]}]}

But the code looks for security and license under the issues. Is this due to any version mismatch.

Using the latest version of kubexray helm chart

whitelistNamespaces in values.yaml should be 1 level up

What?
whitelistNamespaces in values.yaml should be global and probably directly under scanPolicy.

Why?
whitelistNamespaces is applicable to all policies. In the default values.yaml it is not clear that this key is applicable to all policies or only the unscanned ones.

Existing sample -

scanPolicy:
unscanned:
# Set for unscanned deployments delete/scaledown/ignore
deployments: ignore
# Set for unscanned statefulsets delete/scaledown/ignore
statefulSets: ignore
# Whitelist namespaces
whitelistNamespaces: "kube-system,kube-xray"

When upgrading to a new "bad" version over a "good" version - service is down! Support rollback action

BUG

What happened:

• I have a good version running: 1.1
• I upgrade it using helm upgrade ..... to bad version 1.2
• KubeXray detects the bad version and scales deployment down to 0
• Service is down!

What should happen:

• KubeXray should block deployment of bad version 1.2 but allow version 1.1 to keep running (rollback)
• Service is still up (with old, good version 1.1)