octodns / octodns-docker Goto Github PK

Right now it looks like tags only follow the YYYY-MM convention - would it be possible to also add a latest tag for folk who auto-update containers to stay on the latest version?

We have need for an octodns container image with a few cloud providers + octodns-bind, but were surprised to find that, despite being a first-party provider, octodns-bind isn't actually included in octodns/octodns:all!

Is there a reason for this that I'm failing to find, or is it simply an oversight we can correct?

also, thanks for helping make + maintain such a great product, octodns rules haha

Docker Hub is shutting down free organizations. https://blog.alexellis.io/docker-is-deleting-open-source-images/

This does not affect the octodns account since it is a user account instead of an organization.

We might want to ensure we have redundancy across multiple container registries anyway.

Cc @ross

This image is very practical for CI processes, but is missing curl (or wget, or even nc) for making API calls after finishing, with the results of the execution.

Also, when using CircleCI orbs, lots of the installation scripts (for example, for GitHub CLI, or gcloud CLI...) use curl to pull the binaries. This make it less practical for running such jobs, as it's needed to install curl on every run of the job.

• Does it make sense for you to add this tool to the base image?

I know it doesn't make sense to add every tool under the sun, but curl maybe helps people to bootstrap another tools or make basic curl calls easier.

Here's the error I'm getting:

2022-03-24T05:33:28  [140404123150144] INFO  Manager __init__: config_file=/octodns/test.yaml
2022-03-24T05:33:28  [140404123150144] INFO  Manager __init__:   max_workers=2
2022-03-24T05:33:28  [140404123150144] INFO  Manager __init__:   include_meta=False
2022-03-24T05:33:28  [140404123150144] ERROR Manager _get_{}_class: Unable to import module octodns_ns1.Ns1Provider
Traceback (most recent call last):
  File "/opt/octodns/env/src/octodns/octodns/manager.py", line 190, in _get_named_class
    module = import_module(module_name)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/octodns/env/lib/python3.10/site-packages/octodns_ns1/__init__.py", line 7, in <module>
    from collections import Mapping, OrderedDict, defaultdict
ImportError: cannot import name 'Mapping' from 'collections' (/usr/local/lib/python3.10/collections/__init__.py)
Traceback (most recent call last):
  File "/opt/octodns/env/src/octodns/octodns/manager.py", line 190, in _get_named_class
    module = import_module(module_name)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/octodns/env/lib/python3.10/site-packages/octodns_ns1/__init__.py", line 7, in <module>
    from collections import Mapping, OrderedDict, defaultdict
ImportError: cannot import name 'Mapping' from 'collections' (/usr/local/lib/python3.10/collections/__init__.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/octodns/env/bin/octodns-sync", line 33, in <module>
    sys.exit(load_entry_point('octodns', 'console_scripts', 'octodns-sync')())
  File "/opt/octodns/env/src/octodns/octodns/cmds/sync.py", line 37, in main
    manager = Manager(args.config_file)
  File "/opt/octodns/env/src/octodns/octodns/manager.py", line 115, in __init__
    _class = self._get_named_class('provider', _class)
  File "/opt/octodns/env/src/octodns/octodns/manager.py", line 194, in _get_named_class
    raise ManagerException(f'Unknown {_type} class: {_class}')
octodns.manager.ManagerException: Unknown provider class: octodns_ns1.Ns1Provider

Using python:3.9-slim instead works just fine. Code could be changed in octodns_ns1 to import Mapping from collections.abc (and fall back to collections in case ImportError is thrown).

This image is very handy (thank you!) it currently supports amd64 architecture.

Would you be open to a PR that added support to the publish-on-push-to-main action that adds multi-arch support via the additional steps of:

https://github.com/docker/setup-qemu-action & https://github.com/docker/setup-buildx-action

and modification to the build script: e.g.

docker buildx build --tag "$TAG" \
    --label org.opencontainers.image.created="$(date --rfc-3339=seconds)" \
    --label org.opencontainers.image.url="https://github.com/${GITHUB_REPOSITORY}" \
    --label org.opencontainers.image.documentation="https://github.com/octodns/octodns#readme" \
    --label org.opencontainers.image.source="https://github.com/${GITHUB_REPOSITORY}" \
    --label org.opencontainers.image.revision="$GITHUB_SHA" \
    --label org.opencontainers.image.licenses=MIT \
    --platform linux/amd64,linux/arm/v7,linux/arm64 \
    "$FLAVOR"

Update:

After some experimenting, I've found that this isn't going to work easily due to the decoupled push step and the cibuild / release as the images aren't loaded into the local store and there is some incompatibility around this: docker/roadmap#371 (basically can't load the image into local store to test and/or push later if it's multi-arch so would require larger rework of this ci/cd pipeline or waiting until docker supports this, perhaps..)

octodns 1.0 will be made up of a core at octodns/octodns and first-party providers in separate repos, e.g. CloudflareProvider in octodns/octodns-cloudflare. This means that at 1.0, you will need to install octodns==1.0 and octodns-cloudflare with pip to successfully use Cloudflare as your DNS provider.

The octodns org has around 14 first-party plugins for providers, so we have 15 versions to consider: one for octodns and one for each of the 14 providers. We can generate 15 flavors: one octodns flavor which contains all of them, and then one for each provider which just includes that one provider. The current approach is to manually manage requirements.txt files. I don't think this is going to work long-term (e.g. updating the octodns core package means updating all 15 requirements.txt files).

I'd propose a single file in the root of the repo which stores the versions for all the components used. We'd automatically copy just the versions we need to the flavor we want. One possibility is to store the versions as requirements.txt in the base of the repo, then filter and copy that file into the relevant flavor. We already copy the Dockerfile into each flavor since it's the same across all the flavors, so we can easily do this. I'd vote using grep or awk to make it as simple as possible.