GithubHelp home page GithubHelp logo

ldellisola / traefikkobling Goto Github PK

View Code? Open in Web Editor NEW
54.0 2.0 2.0 41 KB

A dynamic Traefik-to-Traefik discovery agent

License: MIT License

C# 95.16% Dockerfile 4.84%
selfhosted traefik docker dotnet redis

traefikkobling's Introduction

Traefik Kobling

A dynamic traefik to traefik discovery agent.

"Kobling" means "coupling" or "linking" in Norwegian. Traefik Kobling lets homelab users link Traefik instances on different hosts to a main, public-facing Traefik instance without using Docker Swarm or publishing ports.

Going forward, we will refer to the main, public-facing Traefik instance as the main instance and the others will be called local instances.

This is accomplished by using Traefik's API on the local instances to find out which rules are registered and then publish them to redis, so the main instance can read and use them.

Usage

For the main traefik instance, you have to configure it to use the redis provider in traefik.yml:

providers:
  redis:
    endpoints:
      - "localhost:6379" # change the address if the redis instance is not available in local host
  # other providers

Also, if you want to use HTTPS, it should be done in this instance.

Configure all your local traefik instances to have API access enabled in their traefik.yml:

  • If the local traefik instances are located within your own local network, then you can allow insecure access and connect to it over port 8080:
api:
  insecure: true
  • if your local traefik instances can be accessed through the internet, then you should not be allowing insecure access and you should set up some sort of authentication. Traefik Kobling only supports Basic Auth and here's a guide on how to set it up in your instance.

These instances do not (and probably should not) have HTTPS enabled.

We can write our config file config.yml:

servers:
  - name: "other-host"
    apiAddress: http://192.168.0.10:8080
    destinationAddress: http://192.168.0.10
    entryPoints:
      web: web
      websecure: web

And then finally, we can set up Traefik Kobling on your main instance:

services:
  traefik-kobling:
    image: ghcr.io/ldellisola/traefik-kobling
    volumes:
      - ./config.yml:/config.yml
    environment:
      REDIS_URL: "localhost:6379"

Configuration

There are two places where you can configure this application. First there are some environment variables:

  • REDIS_URL: Specify the URL of the redis database. The default is redis:6379.
  • CONFIG_PATH: It let's the user change the location of the config.yml file. The default location is /config.yml.
  • RUN_EVERY: It specifies how many seconds to wait before checking the local instances for changes. The default value is 60 seconds.

You must also create a config.yml. This file contains a list of servers with information about what the address of the local traefik instances are and where traffic should be redirected.

servers:
  - name: "host-1"
    apiAddress: http://192.168.0.10:8080
    destinationAddress: http://192.168.0.10
    entryPoints:
      web: web
      websecure: web
    
  - name: "host-2"
    apiAddress: http://192.168.0.11:8080
    destinationAddress: http://192.168.0.11
    entryPoints:
      web-tcp: local-tcp
    
  - name: "host-3"
    apiAddress: http://192.168.0.12:8080
    destinationAddress: http://192.168.0.12

The entryPoints mapping works in the following way:

The entrypoints of your main instance are on the left and the entrypoint of the local instance are on the right, where ultimately traffic will be forwarded.

This approach means we do not have to register more routers than necessary and it helps keep our main dashboard clean.

If no entrypoints are provided in the configuration, the default value http is used for both the main instance as well as local instances.

Connecting to Traefik instances with Basic Auth

If your local instance has basic auth enabled, then you have to specify it in the Kobling config:

servers:
  - name: "host-1"
    apiAddress: http://username:[email protected]
    apiHost: traefik.domain.tld
    destinationAddress: http://192.168.0.10
    entryPoints:
      web: web
      websecure: web

The address in apiAddress should include the username and password to access the api and apiHost should be the host name for that service.

Example

So what does this mean?

Let's say we have 2 machines in our home network:

  • Machine A is exposing port 80 and 443 to the internet and it is running the following containers:
    • traefik: this is our main instance and has the dashboard exposed in the FQDN main.domain.tld
    • redis: it's the storage for the main instance to use as a provider.
    • traefik koblink: it will read data from the traefik instance in machine B and provide redirect data for the main traefik instance.

For this machine we will deploy the following docker-compose.yml file:

networks:
  default:
    name: "web"

services:
  traefik:
    image: traefik:latest
    ports:
      - 80:80
      - 443:443
    environment:
      CF_API_EMAIL: ${CF_API_EMAIL}
      CF_API_KEY: ${CF_API_KEY}
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./traefik.yml:/traefik.yml
      - ./acme.json:/acme.json
      
  redis:
    image: redis:alpine

  traefik-kobling:
    image: ghcr.io/ldellisola/traefik-kobling
    volumes:
      - ./config.yml:/config.yml
    environment:
      REDIS_URL: "redis:6379"

The traefik.yml looks like this:

api:
  dashboard: true
  
entryPoints:
  web:
    address: ":80"
    http:
        redirections:
            entrypoint:
                to: web-secure
                scheme: https

  web-secure:
    address: ":433"
    http:
      tls:
        certResolver: "cloudflare"
        domains:
          - main: "domain.tld"
            sans:
              - "*.domain.tld"

serversTransport:
  insecureSkipVerify: true

providers:
  redis:
    endpoints:
      - "redis:6379"

certificatesResolvers:
  cloudflare:
    acme:
      email: [email protected]
      storage: /acme.json
      dnsChallenge:
        provider: cloudflare
        resolvers:
          - "1.1.1.1:53"
          - "1.0.0.1:53"

The config.yml looks like this:

servers:
  - name: "machine-b"
    apiAddress: http://192.168.0.10:8080
    destinationAddress: http://192.168.0.10
    entryPoints:
      web: local
      web-secure: local
  • Machine B is on IP 192.168.0.10 and it runs:
    • traefik: this is a local instance and has the FQDN local.domain.tld
    • Service B: another random service hosted in this server, with the FQDN serviceB.domain.tld

It is deployed with the following docker-compose.yml:

networks:
  web:
    name: "web"

services:
  traefik:
    image: traefik:latest
    ports:
      - 80:80
      - 8080:8080
    networks:
      - web
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./traefik.yml:/traefik.yml
    labels:
      traefik.enable: "true"
      traefik.http.routers.traefik.rule: "Host(`local.domain.tld`)
      traefik.http.routers.traefik.service: "api@internal"
      traefik.http.services.traefik.loadbalancer.server.port: "8080"

  service-b:
    image: service-b:latest
    networks:
      - web
    labels:
      traefik.enable: "true"
      traefik.http.routers.service-b.rule: "Host(`serviceB.domain.tld`)
      traefik.http.services.service-b.loadbalancer.server.port: "8080"

And the traefik.yml is:

api:
    insecure: true
    
entryPoints:
    local:
        address: ":80"

providers:
    docker:
        endpoint: "unix:///var/run/docker.sock"
        exposedByDefault: false
        network: web

The main traefik instance is set up with https but the local ones do not have to, and both instances are set up to redirect trafic to the services within the machine according to their domain name.

So, if we want to access serviceB.domain.tld, the request should be redirected like:

[internet] -- serviceB.domain.tld --> [main traefik]
[main traefik] -- serviceB.domain.tld --> [local traefik] on 192.168.0.10
[local traefik] --> [Service B]

License

  • Traefik Kobling: MIT, (c) 2022, Pixelcop Research, Inc.
  • traefik: MIT, Copyright (c) 2016-2020 Containous SAS; 2020-2023 Traefik Labs

traefikkobling's People

Contributors

ldellisola avatar agneevx avatar

Stargazers

fronix avatar Sorin Petrus avatar Mikhaël Regni avatar 冒牌月光 avatar Charlie Cortial avatar  avatar  avatar Corentin Delamotte avatar  avatar Palvir avatar  avatar  avatar Aaron avatar  avatar denNorske avatar Gonzalo Díez avatar Ben Epstein avatar icyleaf avatar renothing avatar Jan Juhar avatar Doni Halim avatar Tertius Stander avatar Gabriel Vanca avatar Christian Merlau avatar Markus Fenes avatar  avatar  avatar Mark Tearle avatar Robin Collins avatar  avatar Frederik avatar Cosme Charlier avatar Peter Buga avatar VampDev avatar Dave avatar  avatar Vinay Sastry avatar  avatar Jonas Myhr Refseth avatar Tawmu avatar Brenek Harrison avatar Pascal Riesinger avatar  avatar Luke Tainton avatar Thomas Chartron avatar Tomas Zaluckij avatar  avatar Lindley White avatar Dave Conroy avatar Can Evgin avatar  avatar  avatar  avatar  avatar

Watchers

 avatar Doni Halim avatar

Forkers

zaephor agneevx

traefikkobling's Issues

Change log level with env variable

Hi,

Is there a way to change the log level for the container?

I'm currently running this container

  traefik-kobling:
    image: ghcr.io/ldellisola/traefik-kobling:latest
    container_name: traefik-connector
    restart: always
    volumes:
      - /cat/docker/volumes/network_traefik_kobling/config.yml:/config.yml
    environment:
      REDIS_URL: "network_redis:6379"

But I get way too many logs, as you can see in the following screenshot. About 222Gb in a month.
image

Would be great if there was a LOG_LEVEL: [Info, Warning, Error] env variable.
Or is there another good way to handle the log size?

I get this error

time="2023-12-16T16:13:38+04:00" level=debug msg="WatchTree: traefik"

time="2023-12-16T16:13:38+04:00" level=debug msg="List: traefik"

time="2023-12-16T16:13:38+04:00" level=error msg="KV connection error: field not found, node: url, retrying in 1.614904743s" providerName=redis

Gateway Timeout issue

Thanks for the fast entrypoint fix!

I tried your new solution and it seems to be working (config in traefik looks good) but I always get a Gateway Timeout when I call a service on my sub machine.

MAIN docker compose:

version: '3.9'
services:
  traefik:
    container_name: traefik
    image: traefik:${TRAEFIK_IMAGE_TAG}
    networks:
      - net
    security_opt:
      - no-new-privileges:true
    restart: always
    environment:
      - TZ=${TZ}
      - CF_API_EMAIL=${CLOUDFLARE_EMAIL}
      - CF_API_KEY=${CLOUDFLARE_API_KEY}
    volumes:
      - ...
    ports:
      - 80:80
      - 443:443
      - ${TRAEFIK_DASHBOARD_PORT}:8080
    depends_on:
      - redis
    labels:
      - traefik.enable=true
      - traefik.http.routers.traefik-rtr.entrypoints=websecure
      - traefik.http.routers.traefik-rtr.rule=Host(`traefik.$DOMAINNAME_CLOUD_SERVER`)
      - traefik.http.routers.traefik-rtr.tls=true
      - traefik.http.routers.traefik-rtr.middlewares=chain-local@file
      - traefik.http.routers.traefik-rtr.service=api@internal

  traefik-kobling:
    image: ghcr.io/ldellisola/traefik-kobling
    container_name: traefik-connector
    volumes:
      - /cat/docker/volumes/network_traefik_kobling/config.yml:/config.yml
    environment:
      REDIS_URL: "network_redis:6379"
    depends_on:
      - redis
    networks:
      - net

networks:
  net:
    driver: bridge

traefik-kobling config file

servers:
  - name: "jupiter"
    apiAddress: http://jupiter:8080
    destinationAddress: http://192.168.72.255
    entryPoints:
      websecure: web

Main Traefik dashboard
image
image

Main Debug logging:

time="2023-04-25T11:01:08+02:00" level=debug msg="'504 Gateway Timeout' caused by: dial tcp 192.168.72.255:80: i/o timeout"
time="2023-04-25T11:01:09+02:00" level=debug msg="Configuration received: {\"http\":{\"routers\":{\"api_jupiter\":{\"entryPoints\":[\"websecure\"],\"service\":\"jupiter\",\"rule\":\"Host(`jupiter.domain.tld`)\"},\"dashboard_jupiter\":{\"service\":\"jupiter\",\"rule\":\"PathPrefix(`/`)\"},\"uptime-kuma-rtr\":{\"entryPoints\":[\"websecure\"],\"service\":\"jupiter\",\"rule\":\"Host(`statusdomain.tld`)\"},\"whoami-rtr\":{\"entryPoints\":[\"websecure\"],\"service\":\"jupiter\",\"rule\":\"Host(`whoami.domain.tld`)\"}},\"services\":{\"jupiter\":{\"loadBalancer\":{\"servers\":[{\"url\":\"http://192.168.72.255/\"}],\"passHostHeader\":true}}}},\"tcp\":{},\"udp\":{}}" providerName=redis
time="2023-04-25T11:01:09+02:00" level=debug msg="Skipping unchanged configuration." providerName=redis
time="2023-04-25T11:23:48+02:00" level=debug msg="'504 Gateway Timeout' caused by: dial tcp 192.168.72.255:80: i/o timeout"
time="2023-04-25T11:33:51+02:00" level=debug msg="'504 Gateway Timeout' caused by: dial tcp 192.168.72.255:80: i/o timeout"
time="2023-04-25T11:45:24+02:00" level=debug msg="Serving default certificate for request: \"\""
time="2023-04-25T11:45:24+02:00" level=debug msg="Serving default certificate for request: \"\""
time="2023-04-25T11:45:24+02:00" level=debug msg="Serving default certificate for request: \"\""
time="2023-04-25T11:45:34+02:00" level=debug msg="'499 Client Closed Request' caused by: context canceled"
time="2023-04-25T11:45:34+02:00" level=debug msg="Serving default certificate for request: \"\""
time="2023-04-25T11:53:48+02:00" level=debug msg="'504 Gateway Timeout' caused by: dial tcp 192.168.72.255:80: i/o timeout"

Sub machine docker compose:

version: '3.9'
services:
  traefik:
    image: traefik:$TRAEFIK_IMAGE_TAG
    container_name: traefik
    restart: always
    security_opt:
      - no-new-privileges:true
    ports:
      - 80:80
      - 8080:8080
    environment:
      - TZ=${TZ}
    volumes:
      - /home/reggi/volumes/network_traefik/logs:/logs
    labels:
      - traefik.enable=true
      - traefik.http.routers.traefik.rule=Host(`jupiter.$DOMAINNAME_CLOUD_SERVER`)
      - traefik.http.routers.traefik.service=api@internal
      - traefik.http.services.traefik.loadbalancer.server.port=8080
    command: 
      - --api=true
      - --api.insecure=true

      - --log.filePath=/logs/traefik.log
      - --log.level=DEBUG

      - --entryPoints.web.address=:80

      - --providers.docker=true
      - --providers.docker.endpoint=unix:tcp://dockerproxy:2375
      - --providers.docker.exposedByDefault=false
      - --providers.docker.network=network_network
    networks:
      - network

  whoami:
    image: traefik/whoami
    container_name: whoami
    restart: always
    networks:
      - network
    ports:
      - 2001:2001
    command:
      - --port=2001
    labels:
      - traefik.enable=true
      - traefik.http.routers.whoami-rtr.rule=Host(`whoami.$DOMAINNAME_CLOUD_SERVER`)
      - traefik.http.routers.whoami-router.entrypoints=web
      - traefik.http.services.whoami-rtr.loadbalancer.server.port=2001

  dockerproxy:
    ...

networks:
  network:
    driver: bridge

sub traefik dashboard
image

I'm not sure what is happening. I can see a gateway timeout on my main machine but nothing on my sub machine meaning the call to 192.168.72.225:80 is failing. Any idea why?

Ow, if I navigate to the whoami service directly, there is no issue. So it's not the docker service itself. Maybe I'm doing something wrong with the setup between the main and sub machine?

Thanks for the great work already!

Feature request: Make entrypoint names dynamic

Tried this container today and looks very promising! Not sure if you are open for feature requests and or pull requests?

If so, it would be nice to have dynamic named entrypoint names. You are using "web" and "web-secure" (Workers.cs lines 109 and 110) but I'm using "websecure" so that means I get a fail state in traefik.

Would be nice if we can set the names in the yml config file.
If you want, I can make a PR to get it started

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.