coredns / multicluster

CoreDNS plugin implementing K8s multi-cluster services DNS spec.

multicluster

Name

multicluster - implementation of Multicluster DNS

Description

This plugin implements the Kubernetes DNS-Based Multicluster Service Discovery Specification.

Syntax

multicluster [ZONES...] {
    kubeconfig KUBECONFIG [CONTEXT]
    noendpoints
    fallthrough [ZONES...]
}

• kubeconfig KUBECONFIG [CONTEXT] authenticates the connection to a remote k8s cluster using a kubeconfig file. [CONTEXT] is optional, if not set, then the current context specified in kubeconfig will be used. It supports TLS, username and password, or token-based authentication. This option is ignored if connecting in-cluster (i.e., the endpoint is not specified).
• noendpoints will turn off the serving of endpoint records by disabling the watch on endpoints. All endpoint queries and headless service queries will result in an NXDOMAIN.
• fallthrough [ZONES...] If a query for a record in the zones for which the plugin is authoritative results in NXDOMAIN, normally that is what the response will be. However, if you specify this option, the query will instead be passed on down the plugin chain, which can include another plugin to handle the query. If [ZONES...] is omitted, then fallthrough happens for all zones for which the plugin is authoritative. If specific zones are listed (for example in-addr.arpa and ip6.arpa), then only queries for those zones will be subject to fallthrough.

Startup

When CoreDNS starts with the multicluster plugin enabled, it will delay serving DNS for up to 5 seconds until it can connect to the Kubernetes API and synchronize all object watches. If this cannot happen within 5 seconds, then CoreDNS will start serving DNS while the multicluster plugin continues to try to connect and synchronize all object watches. CoreDNS will answer SERVFAIL to any request made for a Kubernetes record that has not yet been synchronized.

Examples

Handle all queries in the clusterset.local zone. Connect to Kubernetes in-cluster.

.:53 {
    multicluster clusterset.local
}

Installation

See CoreDNS documentation about Compile Time Enabling or Disabling Plugins.

Recompile coredns

Add the plugin to plugins.cfg file. The ordering of plugins matters, add it just below kubernetes plugin that has very similar functionality:

...
kubernetes:kubernetes
multicluster:github.com/coredns/multicluster
...

Follow the coredns README file to build it.

Modify cluster's corefile

To enable the plugin for clusterset.local zone, add multicluster configuration to the corefile. Resulting corefile may look like this:

.:53 {
    errors
    health
    multicluster clusterset.local
    kubernetes cluster.local in-addr.arpa ip6.arpa {
      pods insecure
      fallthrough in-addr.arpa ip6.arpa
    }
    prometheus :9153
    forward . /etc/resolv.conf
    cache 30
    loop
    reload
    loadbalance
}

multicluster's Issues

Zones cannot match qname

When I use multiycluster, and I andd a serviceImport, I found it cannot reslove the domain name.
And I had a doubt:

the code is following,and I add some log:

func (m MultiCluster) ServeDNS(ctx context.Context, w dns.ResponseWriter, r *dns.Msg) (int, error) {
	state := request.Request{W: w, Req: r}

	qname := state.QName()

	zone := plugin.Zones(m.Zones).Matches(qname)
	log.Warningf("haha===ServeDNS qname:%+v,m.Zones:%+v", qname, m.Zones)
	log.Warningf("haha===ServeDNS CompareDomainName:%+v,CountLabel:%+v", dns.CompareDomainName("cluster.local.", qname), dns.CountLabel("cluster.local."))
	log.Warningf("haha===ServeDNS dns.Msg:r(%+v)", r)
	log.Warningf("haha===ServeDNS state: %+v", state.Req)
	if zone == "" {
		return plugin.NextOrFailure(m.Name(), m.Next, ctx, w, r)
	}
	zone = qname[len(qname)-len(zone):] // maintain case of original query
....

And the logs are:

[WARNING] plugin/multicluster: haha===ServeDNS zone:cluster.local
[WARNING] plugin/multicluster: haha===ServeDNS qname:3629697582728363839.1269917840174598765.,m.Zones:[cluster.local.]
[WARNING] plugin/multicluster: haha===ServeDNS CompareDomainName:0,CountLabel:2
[WARNING] plugin/multicluster: haha===ServeDNS dns.Msg:r(;; opcode: QUERY, status: NOERROR, id: 21087
;; flags: rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;3629697582728363839.1269917840174598765.	IN	 HINFO
)

I'm confused about why qname is "3629697582728363839.1269917840174598765", not like "cluster.local"?

version: k8s v1.26, coredns v1.9.3

error: failed to list *v1alpha1.ServiceImport: the server could not find the requested resource (get serviceimports.multicluster.x-k8s.io)

Hello all,

I've successfully deployed the plugin on my cluster. But when the pod start, it shows the errors:
reflector.go:324] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:167: failed to list *v1alpha1.ServiceImport: the server could not find the requested resource (get serviceimports.multicluster.x-k8s.io) pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:167: Failed to watch *v1alpha1.ServiceImport: failed to list *v1alpha1.ServiceImport: the server could not find the requested resource (get serviceimports.multicluster.x-k8s.io )

When I just use serviceImport across the cluster, I cannot access the server

I deploy a server foo in member1, and use serviceImport to import this foo in another k8s cluster named member2.
Member2 donnot has foo or it's endpointSlice. But I cannot access "foo.default.svc.cluster.local" in member2, but ok in member1.

I've already done something:

1).Config the coredns corefile:

.:53 {
multicluster cluster.local.
}

2). Build and install the multicluster as

...
kubernetes:kubernetes
multicluster:github.com/coredns/multicluster
...

in plugin.cfg

3). Add the RBAC

- apiGroups:
  - multicluster.x-k8s.io
  resources:
  - serviceimports
  verbs:
  - list
  - watch

4). The serviceImport.yaml is :

apiVersion: multicluster.x-k8s.io/v1alpha1
kind: ServiceImport
metadata:
  name: foo
  namespace: default
spec:
  type: ClusterSetIP
  ports:
  - name: foo
    port: 80
    protocol: TCP
  ips:
  - "10.10.0.5"

version: k8s:1.21,coredns:v1.9.3 and above version

So, what's wrong with this?

Update for parity with changes in record functions

Plugin does not build with coredns main branch, I tried at coredns commit a929b0b1ecbb0f87b0bd0cd0a40ce38893dce9c0

lauralorenz@lauralorenz:~/coredns/coredns$ make
CGO_ENABLED=0  go build -v -ldflags="-s -w -X github.com/coredns/coredns/coremain.GitCommit=a929b0b1-dirty" -o coredns
github.com/coredns/multicluster
# github.com/coredns/multicluster
../../go/pkg/mod/github.com/coredns/[email protected]/multicluster.go:131:18: assignment mismatch: 2 variables but plugin.A returns 3 values
../../go/pkg/mod/github.com/coredns/[email protected]/multicluster.go:133:18: assignment mismatch: 2 variables but plugin.AAAA returns 3 values
../../go/pkg/mod/github.com/coredns/[email protected]/multicluster.go:135:18: assignment mismatch: 2 variables but plugin.TXT returns 3 values
../../go/pkg/mod/github.com/coredns/[email protected]/multicluster.go:148:12: assignment mismatch: 2 variables but plugin.A returns 3 values
make: *** [Makefile:16: coredns] Error 2

Appears to be due to function signature changes for records in coredns/coredns#4715 meaning this plugin is currently incompatible as of coredns 1.9.1.

error: cannot use e (type *Endpoints) as type "k8s.io/apimachinery/pkg/apis/meta/v1"

Hi guys,

I am having trouble compiling the project. When adding the plugin in the plugin.cfg list I get the following error:

# github.com/coredns/multicluster/object
/go/pkg/mod/github.com/coredns/[email protected]/object/endpoint.go:104:2: cannot use e (type *Endpoints) as type "k8s.io/apimachinery/pkg/apis/meta/v1".Object in return argument:
	*Endpoints does not implement "k8s.io/apimachinery/pkg/apis/meta/v1".Object (missing GetZZZ_DeprecatedClusterName method)
/go/pkg/mod/github.com/coredns/[email protected]/object/endpoint.go:152:2: cannot use e (type *Endpoints) as type "k8s.io/apimachinery/pkg/apis/meta/v1".Object in return argument:
	*Endpoints does not implement "k8s.io/apimachinery/pkg/apis/meta/v1".Object (missing GetZZZ_DeprecatedClusterName method)
/go/pkg/mod/github.com/coredns/[email protected]/object/namespace.go:31:2: cannot use n (type *Namespace) as type "k8s.io/apimachinery/pkg/apis/meta/v1".Object in return argument:
	*Namespace does not implement "k8s.io/apimachinery/pkg/apis/meta/v1".Object (missing GetZZZ_DeprecatedClusterName method)

go version go1.18.2

bad prefix in one of the setup tests(?): 'kubernetes .. ' instead of 'multicluster ..'

Hi guys,

I went through your setup_test.go file, and saw that the last test was config as 'kubernetes ..' instead of 'multicluster ...' in the plugin setup for the config map:
in setup_test.go , line 34:

 `kubernetes coredns.local clusterset.local {
            fallthrough
             }`,
			false,
			"",
			2,
			fall.Root,
		},

other test cases were with a multicluster prefix, as I thought should be (for example):

		{
			`multicluster coredns.local clusterset.local`,
			false,
			"",
			2,
			fall.Zero,
		},

Is there a reason for the different prefixes?
Thanks.

Add support for headless services

Currently, only ClusterIP is supported. Add support for headless services.

See Multi-Cluster DNS spec.

Affected part of the code
https://github.com/vanekjar/coredns-multicluster/blob/bdb34f77f97d6f50a37788613f5a6b1f516379fa/multicluster.go#L353