Parse, explain, and probe network policies to understand their implications and help design policies that suit your needs!
Grab the latest release to get started using Cyclonus.
Create a job.yaml file:
apiVersion: batch/v1
kind: Job
metadata:
name: cyclonus
spec:
template:
spec:
restartPolicy: Never
containers:
- command:
- ./cyclonus
- generate
name: cyclonus
imagePullPolicy: IfNotPresent
image: mfenwick100/cyclonus:latest
serviceAccount: cyclonusThen create a namespace, service account, and job:
kubectl create ns netpol
kubectl create clusterrolebinding cyclonus --clusterrole=cluster-admin --serviceaccount=netpol:cyclonus
kubectl create sa cyclonus -n netpol
kubectl create -f job.yaml -n netpolUse kubectl logs -f to watch your job go!
Take a look at the kind directory:
ls hack/kind
antrea calico cilium ovn-kubernetes run-cyclonus.sh
Choose the right job for your CNI, then run:
cd hack/kind
CNI=calico RUN_FROM_SOURCE=false ./run-cyclonus.sh
This will:
- create a namespace, service account, and cluster role binding for cyclonus
- create a cyclonus job
Pull the logs from the job:
kubectl logs -f -n netpol cyclonus-abcde
Assuming you have a kube cluster and your kubectl is configured to point to it, you can run:
cd cmd/cyclonus
go run main.go generate
Images are available at mfenwick100/cyclonus:
docker pull docker.io/mfenwick100/cyclonus:latest
Cyclonus is available as a krew/kubectl plugin:
- Set up krew
- install cyclonus through krew:
kubectl krew install cyclonus - use cyclonus as a kubectl plugin:
kubectl cyclonus -h.
Cyclonus runs network policy tests for Antrea on a daily basis.
Cyclonus runs network policy tests for Cilium on a daily basis.
Run a connectivity probe against a Kubernetes cluster.
cyclonus probe
Kube results for:
policy y/allow-all-for-label:
policy y/allow-by-ip:
policy y/allow-label-to-label:
policy y/deny-all:
policy y/deny-all-for-label:
+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| - | X/A | X/B | X/C | Y/A | Y/B | Y/C | Z/A | Z/B | Z/C |
+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| x/a | . | . | . | X | . | X | . | . | . |
| x/b | . | . | . | X | . | X | . | . | . |
| x/c | . | . | . | X | . | X | . | . | . |
| y/a | . | . | . | X | . | X | . | . | . |
| y/b | . | . | . | X | . | X | . | . | . |
| y/c | . | . | . | . | . | X | . | . | . |
| z/a | . | . | . | X | . | X | . | . | . |
| z/b | . | . | . | X | . | X | . | . | . |
| z/c | . | . | . | X | . | X | . | . | . |
+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+
0 wrong, 0 no value, 81 correct, 0 ignored out of 81 total
For CNI conformance testing.
Generate network policy test scenarios, install the scenarios one at a time in kubernetes, and compare actual measured connectivity to expected connectivity using a truth table.
cyclonus generate \
--mode simple-fragments \
--include conflict,peer-ipblock \
--ignore-loopback \
--perturbation-wait-seconds 15
...
Tag results:
| Tag | Result |
|---|---|
| direction | 10 / 20 = 50% ❌ |
| - egress | 5 / 11 = 45% ❌ |
| - ingress | 5 / 11 = 45% ❌ |
| miscellaneous | 10 / 16 = 62% ❌ |
| - conflict | 10 / 16 = 62% ❌ |
| peer-ipblock | 0 / 4 = 0% ❌ |
| - IP-block-no-except | 0 / 2 = 0% ❌ |
| - IP-block-with-except | 0 / 2 = 0% ❌ |
| peer-pods | 4 / 4 = 100% ✅ |
| - all-namespaces | 4 / 4 = 100% ✅ |
| - all-pods | 4 / 4 = 100% ✅ |
| rule | 6 / 8 = 75% ❌ |
| - allow-all | 2 / 4 = 50% ❌ |
| - deny-all | 6 / 8 = 75% ❌ |
Groups policies by target, divides rules into egress and ingress, and gives a basic explanation of the combined policies. This clarifies the interactions between "denies" and "allows" from multiple policies.
cyclonus analyze \
--mode explain \
--policy-path ./networkpolicies/simple-example/
+---------+---------------+------------------------+---------------------+--------------------------+
| TYPE | TARGET | SOURCE RULES | PEER | PORT/PROTOCOL |
+---------+---------------+------------------------+---------------------+--------------------------+
| Ingress | namespace: y | y/allow-label-to-label | no ips | no ports, no protocols |
| | Match labels: | y/deny-all-for-label | | |
| | pod: a | | | |
+ + + +---------------------+--------------------------+
| | | | namespace: y | all ports, all protocols |
| | | | pods: Match labels: | |
| | | | pod: c | |
+ +---------------+------------------------+---------------------+ +
| | namespace: y | y/allow-all-for-label | all pods, all ips | |
| | Match labels: | | | |
| | pod: b | | | |
+ +---------------+------------------------+---------------------+--------------------------+
| | namespace: y | y/allow-by-ip | ports for all IPs | no ports, no protocols |
| | Match labels: | | | |
| | pod: c | | | |
+ + + +---------------------+--------------------------+
| | | | 0.0.0.0/24 | all ports, all protocols |
| | | | except [] | |
| | | | | |
+ + + +---------------------+--------------------------+
| | | | no pods | no ports, no protocols |
| | | | | |
| | | | | |
+ +---------------+------------------------+---------------------+ +
| | namespace: y | y/deny-all | no pods, no ips | |
| | all pods | | | |
+---------+---------------+------------------------+---------------------+--------------------------+
This takes the previous command a step further: it combines the rules from all the targets that apply to a pod.
cyclonus analyze \
--mode query-target \
--policy-path ./networkpolicies/simple-example/ \
--target-pod-path ./examples/targets.json
pod in ns y with labels map[pod:a]:
+---------+---------------+-----------------------------+---------------------+--------------------------+
| TYPE | TARGET | SOURCE RULES | PEER | PORT/PROTOCOL |
+---------+---------------+-----------------------------+---------------------+--------------------------+
| Ingress | namespace: y | y/allow-label-to-label | no ips | no ports, no protocols |
| | Match labels: | y/deny-all-for-label | | |
| | pod: a | y/deny-all | | |
+ + + +---------------------+--------------------------+
| | | | namespace: y | all ports, all protocols |
| | | | pods: Match labels: | |
| | | | pod: c | |
+---------+---------------+-----------------------------+---------------------+--------------------------+
| | | | | |
+---------+---------------+-----------------------------+---------------------+--------------------------+
| Egress | namespace: y | y/deny-all-egress | all pods, all ips | all ports, all protocols |
| | Match labels: | y/allow-all-egress-by-label | | |
| | pod: a | | | |
+---------+---------------+-----------------------------+---------------------+--------------------------+
Given arbitrary traffic examples (from a source to a destination, including labels, over a port and protocol), this command parses network policies and determines if the traffic is allowed or not.
cyclonus analyze \
--mode query-traffic \
--policy-path ./networkpolicies/simple-example/ \
--traffic-path ./examples/traffic.json
Traffic:
+--------------------------+-------------+---------------+-----------+-----------+------------+
| PORT/PROTOCOL | SOURCE/DEST | POD IP | NAMESPACE | NS LABELS | POD LABELS |
+--------------------------+-------------+---------------+-----------+-----------+------------+
| 80 (serve-80-tcp) on TCP | source | 192.168.1.99 | y | ns: y | app: c |
+ +-------------+---------------+ + +------------+
| | destination | 192.168.1.100 | | | pod: b |
+--------------------------+-------------+---------------+-----------+-----------+------------+
Is traffic allowed?
+-------------+--------+---------------+
| TYPE | ACTION | TARGET |
+-------------+--------+---------------+
| Ingress | Allow | namespace: y |
| | | Match labels: |
| | | pod: b |
+ +--------+---------------+
| | Deny | namespace: y |
| | | all pods |
+-------------+--------+---------------+
| | | |
+-------------+--------+---------------+
| Egress | Deny | namespace: y |
| | | all pods |
+-------------+--------+---------------+
| IS ALLOWED? | FALSE |
+-------------+--------+---------------+
Runs a simulated connectivity probe against a set of network policies, without using a kubernetes cluster.
cyclonus analyze \
--mode probe \
--policy-path ./networkpolicies/simple-example/ \
--probe-path ./examples/probe.json
Combined:
+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| | X/A | X/B | X/C | Y/A | Y/B | Y/C | Z/A | Z/B | Z/C |
+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+
| x/a | . | . | . | X | . | X | . | . | . |
| x/b | . | . | . | X | . | X | . | . | . |
| x/c | . | . | . | X | . | X | . | . | . |
| y/a | . | . | . | X | . | X | . | . | . |
| y/b | . | . | . | X | . | X | . | . | . |
| y/c | X | X | X | X | X | X | X | X | X |
| z/a | . | . | . | X | . | X | . | . | . |
| z/b | . | . | . | X | . | X | . | . | . |
| z/c | . | . | . | X | . | X | . | . | . |
+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+
Checks network policies for common problems.
cyclonus analyze \
--mode lint \
--policy-path ./networkpolicies/simple-example
+-----------------+------------------------------+-------------------+-----------------------------+
| SOURCE/RESOLVED | TYPE | TARGET | SOURCE POLICIES |
+-----------------+------------------------------+-------------------+-----------------------------+
| Resolved | CheckTargetAllEgressAllowed | namespace: y | y/allow-all-egress-by-label |
| | | | |
| | | pod selector: | |
| | | matchExpressions: | |
| | | - key: pod | |
| | | operator: In | |
| | | values: | |
| | | - a | |
| | | - b | |
| | | | |
+-----------------+------------------------------+-------------------+-----------------------------+
| Resolved | CheckDNSBlockedOnTCP | namespace: y | y/deny-all-egress |
| | | | |
| | | pod selector: | |
| | | {} | |
| | | | |
+-----------------+------------------------------+-------------------+-----------------------------+
| Resolved | CheckDNSBlockedOnUDP | namespace: y | y/deny-all-egress |
| | | | |
| | | pod selector: | |
| | | {} | |
| | | | |
+-----------------+------------------------------+-------------------+-----------------------------+
Check out our sonobuoy plugin!
-
clone this repo
git clone [email protected]:mattfenwick/cyclonus.git cd cyclonus -
set up a KinD cluster with a CNI that supports network policies
pushd kind/calico ./setup.sh popd -
run cyclonus
go run cmd/cyclonus/main.go generate --mode=example
See goreleaser's requirements here.
Get a GitHub Personal Access Token and add the repo scope.
Set GITHUB_TOKEN to this value:
export GITHUB_TOKEN=...See here for more information on github tokens.
Choose a tag/release name, create and push a tag:
TAG=v0.0.1
git tag $TAG
git push origin $TAGCut a release:
goreleaser release --rm-distMake a test release:
goreleaser release --snapshot --rm-distTesting network policies for CNI providers on Kubernetes has historically been very difficult, requiring a lot of boiler plate. This was recently improved upstream via truth table based tests (see KEP). Cyclonus is the next evolution of the truth table tests which are part of upstream Kubernetes. Cyclonus generates hundreds of network policies, their connectivity tables, and outputs results in the same, easy to read format.
