Kured memory usage seems high about kured

[patrickmslatteryvt]

No, that's not a daft question, here is what I get:

kubectl top pods --selector=app=kured
NAME          CPU(cores)   MEMORY(bytes)   
kured-7s22n   0m           14Mi            
kured-csbkv   0m           10Mi            
kured-dzrjq   0m           16Mi            
kured-f269j   0m           15Mi            
kured-g5xnm   0m           12Mi            
kured-lzjld   0m           12Mi            
kured-mf4gg   0m           14Mi            
kured-pc55p   9m           20Mi            
kured-tsxpc   0m           13Mi            

I'm running this version: quay.io/weaveworks/kured:1.1.0

I found recently that occasionally one of the pods would get OOMKilled, so I raised the deployment limits slightly

resources:
  limits:
    cpu: 5m
-   memory: 16Mi
+   memory: 24Mi
  requests:
    cpu: 1m
-   memory: 8Mi
+   memory: 16Mi

I originally chose the numbers off of observed usage values reported by Prometheus over a 48 hour period.
So far I've seen no issues with the updated values.

I should also point out that I have the Slack integration enabled but nothing else differs from the default deployment values.

[patrickmslatteryvt]

I did note some issues today on a new cluster, after turning off memory limits I saw the memory usage climb to 40Mi on the kured pod on the node that that was actively being drained.
Therefore I raised my memory limits to 48MB.

resources:
  limits:
    cpu: 5m
-   memory: 24Mi
+   memory: 48Mi
  requests:
    cpu: 1m
    memory: 16Mi

[ac-astuartkregor]

I've had a chance to run kured for a few weeks and it does seem to be behaving itself with more stringent memory limits. I've not noticed it running into problems. Perhaps it'd be worth me putting in a PR for limits and requests.

[ac-astuartkregor]

Thanks for the details of your experience @patrickmslatteryvt, I'll try and take a look at any version updates. I'm looking to apply requests and limits to the daemonset myself, and this is when I discovered the memory usage.

I'll bump the version and report back.
EDIT: Looks like we're already running 1.1.0, though we have it deployed in the default namespace currently, with the intention of moving it to a purpose-made namespace for admin-deployed cluster services. That might be affecting things, but I'm doubtful. If anyone has any ideas, that'd be grand.

Alternatively I might just try and apply memory limits and see what happens.

[patrickmslatteryvt]

We are using the default and deploying it in kube-system but I somehow doubt that change would cause an increase in RAM usage.

[ac-astuartkregor]

Indeed, I'd be tempted to agree. I've applied memory limits and it hasn't complained at me.. yet. Will keep an eye on things.

I'm thinking that perhaps it can use the extra memory if it needs to but has no issues being limited, though not sure.

[github-actions]

This issue was automatically considered stale due to lack of activity. Please update it and/or join our slack channels to promote it, before it automatically closes (in 7 days).