Comments (7)
I can take a look at addressing this when I bump the embedded spegel version. I will note that this currently only affects servers, as agents do not support the --bind-address flag, so the listener address cannot be configured. Might want to promote that to an agent flag as well I guess.
from k3s.
OK, I've modified that PR to always bind the supervisor on the loopback addresses, in addition to the requested address
from k3s.
Might want to promote that to an agent flag as well I guess.
Having the bind flag on agents would definitely be desirable (and make the overall config options more consistent).
from k3s.
there are already some changes staged in this space in
from k3s.
One of the problems here is that spegel needs to be able to identify which requests are from the local containerd instance and should proxy to other nodes, as opposed to those from other nodes that should not be reproxied. Right now this is done by identifying requests to localhost, and I can't think of another really good way to do that without opening more ports. We may just need to set it up so that it binds to localhost plus the configured bind address.
from k3s.
Note that the embedded registry still will not work on nodes where IPv6 is the primary address family due to #9897
from k3s.
Validated on master branch with version v1.30.2-rc2+k3s1
Environment Details
Infrastructure
- Cloud
- Hosted
Node(s) CPU architecture, OS, and Version:
$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.2 LTS"
$ uname -m
x86_64
Cluster Configuration:
HA: 3 server/ 1 agent
Config.yaml:
token: xxxx
cluster-init: true
write-kubeconfig-mode: "0644"
node-external-ip: 1.1.1.1
node-label:
- k3s-upgrade=server
embedded-registry: true
bind-address: x.x.x.x
registries.yaml:
mirrors:
private.registry.com:
endpoint:
- private.registry.com
docker.io:
endpoint:
- private.registry.com
k8s.gcr.io:
endpoint:
- private.registry.com
configs:
private.registry.com:
auth:
username: <username>
password: <password>
tls:
ca_file: /home/ubuntu/ca.pem
Testing Steps
- Copy config.yaml
$ sudo mkdir -p /etc/rancher/k3s && sudo cp config.yaml /etc/rancher/k3s && sudo cp registries.yaml /etc/rancher/k3s
- Install k3s
curl -sfL https://get.k3s.io | sudo INSTALL_K3S_VERSION='v1.30.2-rc2+k3s1' sh -s - server
- Verify Cluster Status:
kubectl get nodes -o wide
kubectl get pods -A
Replication Results:
- k3s version used for replication:
$ k3s -v
k3s version v1.30.1+k3s1 (80978b5b)
go version go1.22.2
$ journalctl -xeu k3s-agent | grep 'received image event'
Validation Results:
- k3s version used for validation:
$ k3s -v
k3s version v1.30.2-rc2+k3s1 (b4d4ed8f)
go version go1.22.4
Sample spegel logs from the journal logs on the agent node:
$ journalctl -xeu k3s-agent | grep 'received image event'
Jun 14 17:50:51 ip-172-31-24-105 k3s[208743]: time="2024-06-14T17:50:51Z" level=info msg="spegel 2024/06/14 17:50:51 \"level\"=0 \"msg\"=\"received image event\" \"image\"=\"docker.io/rancher/mirrored-pause:3.6@sha256:74bf6fc6be13c4ec53a86a5acf9fdbc6787b176db0693659ad6ac89f115e182c\" \"type\"=\"CREATE\""
Jun 14 17:50:51 ip-172-31-24-105 k3s[208743]: time="2024-06-14T17:50:51Z" level=info msg="spegel 2024/06/14 17:50:51 \"level\"=0 \"msg\"=\"received image event\" \"image\"=\"docker.io/rancher/mirrored-pause@sha256:74bf6fc6be13c4ec53a86a5acf9fdbc6787b176db0693659ad6ac89f115e182c\" \"type\"=\"CREATE\""
Jun 14 17:50:55 ip-172-31-24-105 k3s[208743]: time="2024-06-14T17:50:55Z" level=info msg="spegel 2024/06/14 17:50:55 \"level\"=0 \"msg\"=\"received image event\" \"image\"=\"private.registry.com/mytestcontainer:unprivileged@sha256:7e418465981575a9abef4ee16a80c562a2d2d171e591c1475c38347ef3ec2a72\" \"type\"=\"CREATE\""
Jun 14 17:50:55 ip-172-31-24-105 k3s[208743]: time="2024-06-14T17:50:55Z" level=info msg="spegel 2024/06/14 17:50:55 \"level\"=0 \"msg\"=\"received image event\" \"image\"=\"private.registry.com/mytestcontainer@sha256:7e418465981575a9abef4ee16a80c562a2d2d171e591c1475c38347ef3ec2a72\" \"type\"=\"CREATE\""
Jun 14 17:51:29 ip-172-31-24-105 k3s[208743]: time="2024-06-14T17:51:29Z" level=info msg="spegel 2024/06/14 17:51:29 \"level\"=0 \"msg\"=\"received image event\" \"image\"=\"private.registry.com/nginx:latest@sha256:80550935209dd7f6b2d7e8401b9365837e3edd4b047f5a1a7d393e9f04d34498\" \"type\"=\"CREATE\""
Jun 14 17:51:29 ip-172-31-24-105 k3s[208743]: time="2024-06-14T17:51:29Z" level=info msg="spegel 2024/06/14 17:51:29 \"level\"=0 \"msg\"=\"received image event\" \"image\"=\"private.registry.com/nginx@sha256:80550935209dd7f6b2d7e8401b9365837e3edd4b047f5a1a7d393e9f04d34498\" \"type\"=\"CREATE\""
from k3s.
Related Issues (20)
- How to add multiple parameter values for the --kube-apiserver-arg option in the K3s service. HOT 1
- Flannel Dualstack crash on 1.30.3 HOT 8
- Installation script is failing HOT 1
- Tailscale Auth Keys Expire, Causing Node Disconnection HOT 1
- Broken SQL Connection pooling HOT 1
- Bump stargz-snapshotter to address indirect dependency on `github.com/docker/docker`
- Bump runc to v1.1.14 HOT 1
- logs not found for nodes other than master HOT 4
- k3s second server node join fail with k3s token HOT 1
- [Release-1.31] - Failure to read certificates and key files during k3s certificate rotate-ca HOT 1
- [Release-1.30] - Failure to read certificates and key files during k3s certificate rotate-ca
- [Release-1.29] - Failure to read certificates and key files during k3s certificate rotate-ca
- [Release-1.28] - Failure to read certificates and key files during k3s certificate rotate-ca HOT 1
- [Release-1.31] - Bump runc to v1.1.13
- [Release-1.30] - Bump runc to v1.1.13
- [Release-1.29] - Bump runc to v1.1.14 HOT 1
- [Release-1.28] - Bump runc to v1.1.13
- metrics-server is unable to scrape node when using custom bind-address HOT 1
- [k3s-upgrade] MODIFIED_VERSION is undefined HOT 2
- Document for creating k3s cluster with etcd enabled or upgrading existing k3s to etcd. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from k3s.