cndoit18 / lxcfs-on-kubernetes

This project will automatically deploy LXCFS while mounted to the container

lxcfs-on-kubernetes

This project will automatically deploy LXCFS while mounted to the container

Introduction

LXCFS is a small FUSE filesystem written with the intention of making Linux containers feel more like a virtual machine. It started as a side-project of LXC but is useable by any runtime.

LXCFS will take care that the information provided by crucial files in procfs such as:

/proc/cpuinfo
/proc/diskstats
/proc/meminfo
/proc/stat
/proc/swaps

/proc/uptime
/proc/slabinfo
/sys/devices/system/cpu
/sys/devices/system/cpu/online

are container aware such that the values displayed (e.g. in /proc/uptime) really reflect how long the container is running and not how long the host is running.

Prerequisites

1. Kubernetes cluster (v1.19+) is running. For local development purpose, check Kind installation.
2. cert-manager (v1.2+) is installed.
3. helm v3 is installed.

Deploy

Run the helm command to install the lxcfs-on-kubernetes to your cluster:

helm repo add lxcfs-on-kubernetes https://cndoit18.github.io/lxcfs-on-kubernetes/

you can then do

helm upgrade --install lxcfs lxcfs-on-kubernetes/lxcfs-on-kubernetes -n lxcfs --create-namespace

For what settings you can override with --set, --set-string, --set-file or --values, you can refer to the values.yaml file.

you can enable the namespace for injection.

kubectl label namespace default mount-lxcfs=enabled

You can change it by setting matchLabels during installation

License

lxcfs-on-kubernetes's Issues

[Feature Request] Optional lxcfs files to mount

Feature Request

Is your feature request related to a problem? Please describe:
Older docker releases use older runc, which has a smaller allow list of /proc files being mounted. Currently lxcfs-on-kubernetes webhook mounts a fixed set of /proc files. It would not work on node runs old docker(pod crashes with errors like cannot be mounted because it is located inside "/proc").

Describe the feature you'd like:
Add a CLI flag to allow specifying LXCFS proc files to be mounted. So users can tune the list without needing to recompile the webhook.

I can submit a PR if this feature is reasonable to you.

Drop the dependency on cert-manager

Kubernetes 1.19 introduced a feature which I believe can replace the need of cert-manager, at least for this chart.

https://kubernetes.io/docs/reference/access-authn-authz/certificate-signing-requests/

Instead of using cert-manager.io API, they can be changed to certificate.k8s.io.

Am I correct? I may send a PR if so.

About CPU Pin

use this template:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: web
          image: httpd:2.4.32
          imagePullPolicy: Always
          resources:
            requests:
              memory: "256Mi"
              cpu: "2"
            limits:
              memory: "256Mi"
              cpu: "2"

if we deploy in this metorthd, we still can view all the cpu in the pod.
Like:

root@web-5f859c87cf-5wzhm:/usr/local/apache2# top
top - 13:00:18 up  3:57,  0 users,  load average: 0.86, 0.69, 0.65
Tasks:   6 total,   1 running,   5 sleeping,   0 stopped,   0 zombie
%Cpu0  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  :  1.0 us,  0.0 sy,  0.0 ni, 99.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu4  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu5  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:    262144 total,    13336 used,   248808 free,        0 buffers
KiB Swap:        0 total,        0 used,        0 free.      264 cached Mem

if we add the cpu pin featuer in the kunbernetes cluster, we can get a good environment.
Like:

bash-4.2# taskset -cp 1
pid 1's current affinity list: 20,22,64,66
bash-4.2# free -mh 
              total        used        free      shared  buff/cache   available
Mem:            16G        3.4G        7.2G          0B        5.4G         12G
Swap:            0B          0B          0B
bash-4.2# top 
top - 13:01:43 up  6:57,  0 users,  load average: 17.88, 19.84, 20.87
Tasks:  40 total,   1 running,  39 sleeping,   0 stopped,   0 zombie
%Cpu0  : 20.9 us, 20.9 sy,  0.0 ni, 58.2 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  : 50.0 us, 50.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  : 50.0 us, 50.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  : 50.0 us, 50.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st

why not add a README.md? 😁

bump LXCFS to v5.0?

Hi, me again 😄

Feature Request

Is your feature request related to a problem? Please describe:
LXCFS is currently at v5, looks like new commits are not back ported to v4(e.g. fixes to /sys/devices/system/cpu virtualization). Is there a plan to update Dockerfile.agent file correspondingly?

v5 added meson to toolchain, so Dockerfile.agent will need some modifications. I wrote a Dockerfile.agent that builds v5 in ubuntu:20.04 (because LXCFS runs github action in ubuntu:20.04 runner). Will be happy to create a PR on this.