astronomer / ap-airflow Goto Github PK

Our workflows currently require that the Trivy and Snyk scans all pass before pushing any builds at all, even dev builds.

But, at the moment, that's compounding our issues, because we not only have a bad constraints file that needs updating, we also now need to resolve a CVE in gzip/liblzma5 to push any images at all, even images for dev versions:

I think we should still run the Snyk and Trivy checks, but we shouldn't gate dev image pushes on them. Turn them into advisory checks.

We should still (eg: leave this alone) require those checks to pass when we push out images for actual release versions.

Edit: fix should -> shouldn't

We currently have no way of notifying people about failed builds (I'm not sure who all gets emailed about CircleCI nightly build failures, but I'm not included). Additionally, notifying a Slack channel would allow more people to more easily subscribe to these updates.

https://issues.apache.org/jira/browse/AIRFLOW-6854

1.10.10.* crashes when running with python 3.8:
File "/home/asadaoui/.virtualenvs/airflow/lib/python3.8/site-packages/airflow/serialization/json_schema.py", line 26, in
from typing_extensions import Protocol
ModuleNotFoundError: No module named 'typing_extensions'

The script clean-airflow-logs.sh performs log trimming inside an infinite while loop. The loop uses sleep to wait until the next scheduled cleanup and exits when SIGINT or SIGTERM is recieved. However, Bash does not execute traps until the current command has finished. [0] That means, when SIGTERM arrives, the signal handler will not be called until sleep has finished.

When this scripts runs as a container in a Kubernetes pod this behavior causes potentially large delays when terminating the Pod. Kubernetes sends SIGTERM to all containers of a Pod when terminating it. Since the container running the clean-airflow-logs.sh script is currently sleeping, it may take a long time for it to react to SIGTERM. As a result, the container exceeds the configured termination grace period and gets killed by the Kubernetes Pod controller.

I'll happily provide a patch to address the issue. Are you willing to accept a contribution for this?

[0] see Bash Manual section 3.7.6

I am not sure if this is the correct place for this request, so let me know if i need to move it somewhere better.
I recently started using tags in our Astronomer instance and while they are very useful, the way the UI is currently styled makes it feel very busy and cramped. I tried to find a way to just override some of the styling, which seems possible in base airflow but not in astronomer. Can we get some better styling for the DAG list, or atleast a better way to apply styles ourselves? Here's an example of my concern.

without styling

Changing the tags font-size to .75em and the DAG name font-size to 1.25em. I feel like this greatly improves the ability to quickly skim the names while keeping the tags readable.

If there is a way to do this and i just missed it, please let me know.

When run dev start with dockerfile image base quay.io/astronomer/ap-airflow:2.1.4-buster the image docker keeping restarting on ubuntu 20.04.3

We currently use the latest-main.build.json file to collect data from the package build environment when we add labels to the Docker images. However, we do not include any/much information from the CircleCI build environment in the built Docker image.

It would be smart to include more information from the CircleCI environment, possibly including a select few environment variables, in the Docker image labels. At the very least we should include a link to the CircleCI build workflow and/or build job so we and QA have an easier time "following the breadcrumbs" back if/when they are diagnosing issues possibly caused by the build environment.

Currently, it's not possible to add Python packages from internal pip indexes to the requirements.txt file and have them installed when building images FROM astronomerinc/ap-airflow:*-onbuild.

Adding the following line right before the ONBUILD RUN pip install -r requirements.txt command:

ONBUILD ARG PIP_INDEX_URL

would enable users of the Dockerfile to build an image specifying a custom PIP_INDEX_URL (e.g. pointing to an internal company pip index) by adding a --build-arg PIP_INDEX_URL https://username:[email protected]/ to the docker build command, and include internal packages in the requirements.txt file.

A use case for this is for example adding an on_failure_callback to DAGs, that uses internal Python libraries for custom logging and incident handling.

I am no longer able to build the latest Airflow image on Docker for Desktop version 4.6.1 (76265), running Mac OS Monterey v12.3.1.

I previously had been able to successfully build the 2.2.5 release, but that now fails. I also tried building the 2.3.0 release, and that also failed. Both due to pip being unable to find a version that satisfied requirements for astronomer-certified[.....]==2.2.5-3-* (and 2.3.0 similarly). Full stack trace:

> DOCKER_BUILDKIT=0 docker build --build-arg PYTHON_MAJOR_MINOR_VERSION=3.8 -t ap-airflow:py38 https://github.com/astronomer/ap-airflow.git#master:2.3.0/bullseye/
Sending build context to Docker daemon  30.21kB
Step 1/67 : ARG APT_DEPS_IMAGE="airflow-apt-deps"
Step 2/67 : ARG PYTHON_MAJOR_MINOR_VERSION="3.9"
Step 3/67 : ARG PYTHON_BASE_IMAGE="python:${PYTHON_MAJOR_MINOR_VERSION}-slim-bullseye"
Step 4/67 : FROM ${PYTHON_BASE_IMAGE} as airflow-apt-deps
 ---> 654e18c3f59c
Step 5/67 : LABEL maintainer="Astronomer <[email protected]>"
 ---> Using cache
 ---> 1c96172c8065
Step 6/67 : ARG ASTRONOMER_USER="astro"
 ---> Using cache
 ---> 204eabfc3b98
Step 7/67 : ARG ASTRONOMER_UID="50000"
 ---> Using cache
 ---> 83b112225176
Step 8/67 : LABEL io.astronomer.docker=true
 ---> Using cache
 ---> e46957e97ba2
Step 9/67 : LABEL io.astronomer.docker.distro="debian"
 ---> Using cache
 ---> 12ae0a3d4812
Step 10/67 : LABEL io.astronomer.docker.module="airflow"
 ---> Using cache
 ---> dc8791d042f7
Step 11/67 : LABEL io.astronomer.docker.component="airflow"
 ---> Using cache
 ---> 50829d950c62
Step 12/67 : LABEL io.astronomer.docker.uid="${ASTRONOMER_UID}"
 ---> Using cache
 ---> b9d8acf44b83
Step 13/67 : ARG ORG="astronomer"
 ---> Using cache
 ---> a999463fbeb3
Step 14/67 : ENV AIRFLOW_HOME="/usr/local/airflow"
 ---> Using cache
 ---> bb61da75bc45
Step 15/67 : ENV PYTHONPATH=${PYTHONPATH:+${PYTHONPATH}:}${AIRFLOW_HOME}
 ---> Using cache
 ---> b971365c26b6
Step 16/67 : ENV ASTRONOMER_USER=${ASTRONOMER_USER}
 ---> Using cache
 ---> af6e48608745
Step 17/67 : ENV ASTRONOMER_UID=${ASTRONOMER_UID}
 ---> Using cache
 ---> 71b33d71cab5
Step 18/67 : ARG PYTHON_BASE_IMAGE
 ---> Using cache
 ---> 3e5b784bb97b
Step 19/67 : ENV PYTHON_BASE_IMAGE=${PYTHON_BASE_IMAGE}
 ---> Using cache
 ---> 6d795160f3c2
Step 20/67 : ARG PYTHON_MAJOR_MINOR_VERSION
 ---> Using cache
 ---> 639c51da0550
Step 21/67 : ENV PYTHON_MAJOR_MINOR_VERSION=${PYTHON_MAJOR_MINOR_VERSION}
 ---> Using cache
 ---> d15a535a7564
Step 22/67 : ARG PIP_VERSION="21.2.4"
 ---> Using cache
 ---> 408e6dc75a18
Step 23/67 : ENV PYTHON_PIP_VERSION=${PIP_VERSION}
 ---> Using cache
 ---> c36cc72b62d8
Step 24/67 : ENV DEBIAN_FRONTEND=noninteractive LANGUAGE=C.UTF-8 LANG=C.UTF-8 LC_ALL=C.UTF-8     LC_CTYPE=C.UTF-8 LC_MESSAGES=C.UTF-8
 ---> Using cache
 ---> 40057402fe74
Step 25/67 : ARG DEPENDENCIES_EPOCH_NUMBER="2"
 ---> Using cache
 ---> 676cbb5c9126
Step 26/67 : ENV DEPENDENCIES_EPOCH_NUMBER=${DEPENDENCIES_EPOCH_NUMBER}
 ---> Using cache
 ---> 5f6cc1bed465
Step 27/67 : RUN apt-get update     && apt-get install -y --no-install-recommends            apt-utils            curl            libmariadb3            freetds-bin            gosu            libffi7            libkrb5-3            libpq5            libsasl2-2            libsasl2-modules            libssl1.1            locales             netcat            rsync            sasl2-bin            sudo            tini     && apt-get autoremove -yqq --purge     && apt-get clean     && rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 9cf0c3ac1a3b
Step 28/67 : RUN pip install --upgrade pip=="${PYTHON_PIP_VERSION}"
 ---> Using cache
 ---> 9461618a6cb2
Step 29/67 : RUN useradd --uid $ASTRONOMER_UID --create-home ${ASTRONOMER_USER}     && groupadd astrogroup --gid 101     && usermod --append --groups astrogroup ${ASTRONOMER_USER}
 ---> Using cache
 ---> a34cf6859437
Step 30/67 : FROM ${APT_DEPS_IMAGE} as devel
 ---> a34cf6859437
Step 31/67 : SHELL ["/bin/bash", "-o", "pipefail", "-e", "-u", "-x", "-c"]
 ---> Using cache
 ---> eefe33e545d3
Step 32/67 : ENV PIP_NO_CACHE_DIR="true"
 ---> Using cache
 ---> b26b8b1eec8d
Step 33/67 : RUN apt-get update     && curl -sL https://deb.nodesource.com/setup_14.x | sudo bash -     && apt-get install -y --no-install-recommends         build-essential         default-libmysqlclient-dev         libffi-dev         libkrb5-dev         libpq-dev         libsasl2-dev         libssl-dev     && apt-get clean     && rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 44d9a1d678b5
Step 34/67 : ARG VERSION="2.3.0-1-*"
 ---> Running in fe07adf2edb0
Removing intermediate container fe07adf2edb0
 ---> ce1b787845ca
Step 35/67 : ARG SUBMODULES="async,azure,amazon,elasticsearch,google,password,cncf.kubernetes,mysql,postgres,redis,slack,ssh,statsd,virtualenv"
 ---> Running in 466474de123b
Removing intermediate container 466474de123b
 ---> 3d5e855f1f0b
Step 36/67 : ARG AIRFLOW_MODULE="astronomer_certified[${SUBMODULES}]==$VERSION"
 ---> Running in f53f2b4b1a74
Removing intermediate container f53f2b4b1a74
 ---> 48d77d872f25
Step 37/67 : ARG AIRFLOW_VERSION="2.3.0"
 ---> Running in e295064c7e0f
Removing intermediate container e295064c7e0f
 ---> c91edae1de05
Step 38/67 : ARG ASTRONOMER_FAB_SECURITY_MANAGER_VERSION="1.9.1"
 ---> Running in 21bea3cde6ff
Removing intermediate container 21bea3cde6ff
 ---> 5459a53eb263
Step 39/67 : COPY include/pip.conf /etc/pip.conf
 ---> 70ee97bac288
Step 40/67 : COPY include/pip-constraints.txt /usr/local/share/astronomer-pip-constraints.txt
 ---> a0467429a7df
Step 41/67 : COPY build-time-pip-constraints.txt /tmp/build-time-pip-constraints.txt
 ---> 0a82e8991f77
Step 42/67 : RUN pip install "${AIRFLOW_MODULE}" celery flower         --constraint /tmp/build-time-pip-constraints.txt     && pip install "https://github.com/astronomer/astronomer-airflow-scripts/releases/download/v0.0.5/astronomer_airflow_scripts-0.0.5-py3-none-any.whl"     && pip install "astronomer-fab-security-manager==${ASTRONOMER_FAB_SECURITY_MANAGER_VERSION}"
 ---> Running in 7ac63e8d3338
+ pip install 'astronomer_certified[async,azure,amazon,elasticsearch,google,password,cncf.kubernetes,mysql,postgres,redis,slack,ssh,statsd,virtualenv]==2.3.0-1-*' celery flower --constraint /tmp/build-time-pip-constraints.txt
Looking in indexes: https://pypi.org/simple, https://pip.astronomer.io/simple/
ERROR: Could not find a version that satisfies the requirement astronomer_certified[amazon,async,azure,cncf.kubernetes,elasticsearch,google,mysql,password,postgres,redis,slack,ssh,statsd,virtualenv]==2.3.0-1-* (from versions: 1.10.7.post7, 1.10.7.post8, 1.10.7.post9, 1.10.7.post10, 1.10.7.post11, 1.10.7.post12, 1.10.7.post13, 1.10.7.post14, 1.10.7.post15, 1.10.7.post16, 1.10.7.post17, 1.10.7.post18, 1.10.7.post19, 1.10.10.post1, 1.10.10.post2, 1.10.10.post3, 1.10.10.post4, 1.10.10.post5, 1.10.10.post6, 1.10.10.post7, 1.10.10.post8, 1.10.10.post9, 1.10.10.post10, 1.10.12.post1, 1.10.12.post2, 1.10.12.post3, 1.10.12.post4, 1.10.12.post5, 1.10.12.post6, 1.10.13.post1, 1.10.14.post1, 1.10.14.post2, 1.10.14.post3, 1.10.14.post4, 1.10.14.post5, 1.10.15.post1, 1.10.15.post2, 1.10.15.post3, 1.10.15.post4, 1.10.15.post5.dev3, 1.10.15.post5, 1.10.15.post6.dev1, 1.10.15.post6, 1.10.15.post7.dev1, 1.10.15.post7, 2.0.0.post1, 2.0.0.post2, 2.0.0.post3, 2.0.0.post4, 2.0.0.post5, 2.0.0.post6, 2.0.0.post7, 2.0.0.post8, 2.0.0.post9, 2.0.0.post10, 2.0.1.post1, 2.0.2.post1, 2.0.2.post2, 2.0.2.post3, 2.0.2.post4, 2.0.2.post5, 2.0.2.post6, 2.1.0.post1, 2.1.0.post2, 2.1.0.post3, 2.1.0.post4, 2.1.0.post5, 2.1.0.post6, 2.1.0.post7.dev2, 2.1.0.post7.dev5, 2.1.0.post7, 2.1.1.post1, 2.1.1.post2, 2.1.1.post3, 2.1.1.post4, 2.1.1.post5, 2.1.1.post6.dev2, 2.1.1.post6.dev5, 2.1.1.post6, 2.1.3.post1, 2.1.3.post2, 2.1.3.post3, 2.1.3.post4.dev2, 2.1.3.post4.dev5, 2.1.3.post4, 2.1.4.post1, 2.1.4.post2, 2.1.4.post3, 2.1.4.post4.dev2, 2.1.4.post4.dev7, 2.1.4.post4, 2.1.4.post5.dev4, 2.1.4.post5, 2.1.4.post6.dev1, 2.1.4.post6, 2.1.4.post7.dev1, 2.1.4.post7.dev2, 2.1.4.post7, 2.2.0.post1, 2.2.0.post2, 2.2.0.post3, 2.2.0.post4, 2.2.0.post5.dev3, 2.2.0.post5, 2.2.1.post1, 2.2.1.post2, 2.2.1.post3.dev2, 2.2.1.post3, 2.2.2.post1, 2.2.2.post2.dev2, 2.2.2.post2, 2.2.3.post1.dev918, 2.2.3.post1.dev919, 2.2.3.post1.dev924, 2.2.3.post1.dev929, 2.2.3.post1.dev942, 2.2.3.post1, 2.2.3.post2.dev1, 2.2.3.post2, 2.2.4.post1.dev1180, 2.2.4.post1.dev1190, 2.2.4.post1.dev1192, 2.2.4.post1, 2.2.4.post2.dev1, 2.2.4.post2, 2.2.4.post3.dev1, 2.2.4.post3, 2.2.4.post4.dev1, 2.2.4.post4.dev2, 2.2.4.post4, 2.2.5.post1.dev1264, 2.2.5.post1.dev1265, 2.2.5.post1, 2.2.5.post2, 2.2.5.post3.dev2, 2.3.0.dev20220323, 2.3.0.dev20220325, 2.3.0.dev20220326, 2.3.0.dev20220329, 2.3.0.dev20220330, 2.3.0.dev20220331, 2.3.0.dev20220401, 2.3.0.dev20220402, 2.3.0.dev20220405, 2.3.0.dev20220406, 2.3.0.dev20220407, 2.3.0.dev20220408, 2.3.0.dev20220409, 2.3.0.dev20220411, 2.3.0.dev20220412, 2.3.0.dev20220413, 2.3.0.dev20220414, 2.3.0.dev20220416, 2.3.0.dev20220419, 2.3.0.dev20220420, 2.3.0.dev20220421, 2.3.0.dev20220422, 2.3.0.post1.dev308, 2.3.0.post1.dev334, 2.3.0.post1.dev356)
ERROR: No matching distribution found for astronomer_certified[amazon,async,azure,cncf.kubernetes,elasticsearch,google,mysql,password,postgres,redis,slack,ssh,statsd,virtualenv]==2.3.0-1-*
WARNING: You are using pip version 21.2.4; however, version 22.0.4 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
The command '/bin/bash -o pipefail -e -u -x -c pip install "${AIRFLOW_MODULE}" celery flower         --constraint /tmp/build-time-pip-constraints.txt     && pip install "https://github.com/astronomer/astronomer-airflow-scripts/releases/download/v0.0.5/astronomer_airflow_scripts-0.0.5-py3-none-any.whl"     && pip install "astronomer-fab-security-manager==${ASTRONOMER_FAB_SECURITY_MANAGER_VERSION}"' returned a non-zero code: 1

I currently use astro dev init, and then astro dev start to get airflow 2.4.3 up and running locally.

Have been looking at the parameters that can be set in Dockerfile (which basically only has a FROM line in it after astro dev init) - and found 2.4.3/bullseye/Dockerfile which is fully loaded. Is the intent behind that so we can use those parameters?

Seems like I should use the Dockerfile & requirements.txt files in this github repo locally to keep things constrained to that 2.4.3 version - is that the idea?

I have tried to deploy both locally on my mac and on circleci, but in both cases it never gets past the pip install command pip install --no-cache-dir -q -r requirements.txt. Not really sure how to debug this since it uses the quiet flag.

While testing out an upgrade from Airflow 2.1.4 to 2.2.2 in my dev environment I noticed a breaking change while the image was building.

The reason the image failed to build is because I am installing msodbcsql17 in my Dockerfile using the described method as shown here: https://www.astronomer.io/docs/enterprise/v0.26/astronomer-certified/install-packages. The build fails because it is unable to locate the package msodbcsql17. If I take out installing msodbcsql17 the image builds successfully. This was not an issue with Astronomer Certified 2.1.4.

I found out Astronomer Certified 2.2.0-1 upgrades from Debian 10 ("Buster") to Debian 11 ("Bullseye") (PR for this is #298), however the Microsoft ODBC Driver for SQL Server is not supported on Debian Bullseye (see https://github.com/MicrosoftDocs/sql-docs/issues/6804), so we can't use Airflow’s OdbcHook with Microsoft SQL Server anymore in Astronomer Certified 2.2.0+ until Microsoft releases a msodbcsql version that supports Bullseye (possibly in January 2022 according the twice a year release frequency).

Also, is it possible you guys might have prematurely upgraded to Debian Bullseye as Airflow seems like it currently does not support Debian Bullseye due to this same MSSQL issue? (see discussion on this Airflow PR: apache/airflow#18279)

Question: Is it possible to keep Astronomer Certified 2.2.0+ images for Buster until there is a msodbcsql version that supports Bullseye?

In the meantime I am switching all of my DAGs from using OdbcHook to MsSqlHook so that I can upgrade Airflow past 2.1.4.

The current Slack notifications/release approvals are sent in multiple separate messages for each potential version release. This eats up a lot of vertical space and makes it difficult to pick out individual versions.

It would be nice to either consolidate the multiple version approvals into a single Slack message, or to just include a link to the CircleCI workflow to manually approve releases.

I have used Slack's Block Kit builder to prototype a new message format. The content is the same, the only difference is dark/light theme.

It might be cool to render the Astronomer logo instead of an extra large ✋ emoji to differentiate and separate release approval notifications.

The JSON for this prototype is inlined below.

{
  "blocks": [
    {
      "type": "section",
      "fields": [
        {
          "type": "plain_text",
          "text": "New build request"
        },
        {
          "type": "mrkdwn",
          "text": "*New build request (Markdown)!*"
        }
      ]
    },
    {
      "type": "divider"
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "*CircleCI Workflow*:\n    <https://app.circleci.com/pipelines/github/astronomer/ap-airflow/5678/workflows/11112222-3333-4444-5555-666677778888|CircleCI#4321>\nPull Request:\n    <https://github.com/astronomer/ap-airflow/pull/1234|astronomer/ap-airflow#1234>"
      },
      "accessory": {
        "type": "image",
        "image_url": "https://i.imgur.com/3jf0wLj.png",
        "alt_text": "hand thumbnail"
      }
    },
    {
      "type": "context",
      "elements": [
        {
          "type": "image",
          "image_url": "https://api.slack.com/img/blocks/bkb_template_images/notificationsWarningIcon.png",
          "alt_text": "notifications warning icon"
        },
        {
          "type": "mrkdwn",
          "text": "Edge builds not included"
        }
      ]
    },
    {
      "type": "divider"
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "*Approve a release:*"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`1.10.10`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`1.10.12`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`1.10.14`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`1.10.15`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`2.0.0`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`2.0.2`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`2.1.0`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`2.1.1`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`2.1.3`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`2.1.4`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`2.2.0`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`2.2.1`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`2.2.2`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "`2.2.3`"
      },
      "accessory": {
        "type": "button",
        "text": {
          "type": "plain_text",
          "emoji": true,
          "text": "Approve :white_check_mark:"
        },
        "value": "click_me_123"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "*<fakelink.ToMoreTimes.com|Show more releases>*"
      }
    }
  ]
}

Some customers are having issues with their builds when running astro deploy and astro dev start. The error is.

Here is the ticket to these issues on Zendesk

WARNING: Requested dill>=0.2.2 from https://files.pythonhosted.org/packages/3e/ad/31932a4e2804897e6fd2f946d53df51dd9b4aa55e152b5404395d00354d1/dill-0.3.1.tar.gz#sha256=d3ddddf2806a7bc9858b20c02dc174396795545e9d62f243b34481fd26eb3e2c (from apache-airflow>=2.2.0->apache-airflow-providers-opsgenie==4.0.0->-r requirements.txt (line 3)), but installing version 0.3.1.dev0
WARNING: Discarding https://files.pythonhosted.org/packages/3e/ad/31932a4e2804897e6fd2f946d53df51dd9b4aa55e152b5404395d00354d1/dill-0.3.1.tar.gz#sha256=d3ddddf2806a7bc9858b20c02dc174396795545e9d62f243b34481fd26eb3e2c (from https://pypi.org/simple/dill/) (requires-python:>=2.6, !=3.0.*). Requested dill>=0.2.2 from https://files.pythonhosted.org/packages/3e/ad/31932a4e2804897e6fd2f946d53df51dd9b4aa55e152b5404395d00354d1/dill-0.3.1.tar.gz#sha256=d3ddddf2806a7bc9858b20c02dc174396795545e9d62f243b34481fd26eb3e2c (from apache-airflow>=2.2.0->apache-airflow-providers-opsgenie==4.0.0->-r requirements.txt (line 3)) has inconsistent version: filename has '0.3.1', but metadata has '0.3.1.dev0'

Another customer has said that it is stalling with this message

[#12](https://astronomer.zendesk.com/agent/tickets/12) 97.08 Collecting dill>=0.2.2
[#12](https://astronomer.zendesk.com/agent/tickets/12) 97.09   Downloading dill-0.3.5.1-py2.py3-none-any.whl (95 kB)
[#12](https://astronomer.zendesk.com/agent/tickets/12) 173.8 INFO: pip is looking at multiple versions of distro to determine which version is compatible with other requirements. This could take a while.
[#12](https://astronomer.zendesk.com/agent/tickets/12) 173.8   Downloading dill-0.3.5-py2.py3-none-any.whl (95 kB)
[#12](https://astronomer.zendesk.com/agent/tickets/12) 249.8   Downloading dill-0.3.4-py2.py3-none-any.whl (86 kB)
[#12](https://astronomer.zendesk.com/agent/tickets/12) 268.8 INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
[#12](https://astronomer.zendesk.com/agent/tickets/12) 325.8   Downloading dill-0.3.3-py2.py3-none-any.whl (81 kB)
[#12](https://astronomer.zendesk.com/agent/tickets/12) 400.2   Downloading dill-0.3.2.zip (177 kB)
[#12](https://astronomer.zendesk.com/agent/tickets/12) 474.7   Downloading dill-0.3.1.1.tar.gz (151 kB)
[#12](https://astronomer.zendesk.com/agent/tickets/12) 550.5   Downloading dill-0.3.1.tar.gz (151 kB)

Here are the respective requirements.txt files and Dockerfiles

First Customer

FROM quay.io/astronomer/ap-airflow:2.3.4-onbuild

# first error
--extra-index-url https://pip.astronomer.io/simple

apache-airflow-providers-opsgenie==4.0.0
apache-airflow-providers-snowflake==2.7.0
memory-profiler>=0.55.0
numpy>=1.17.4
pycurl>=7.43.0.2
jinja2==2.11.3
sqlalchemy==1.4.9
snowflake-sqlalchemy==1.2.4
packaging==21.0
pyyaml==5.4.1
colorlog==4.0.2
pyparsing==2.4.7
google-auth==1.34.0
types-python-dateutil==2.8.3
types-pytz==2021.3.3
types-PyYAML==6.0.1
types-croniter==1.0.3
types-requests==2.26.2
types-pycurl==7.44.6

Second Customer

FROM quay.io/astronomer/ap-airflow:2.3.4-onbuild

# stalling
apache-airflow-providers-hashicorp==3.1.0
apache-airflow-providers-sftp==4.0.0
apache-airflow-providers-microsoft-mssql==3.2.0
black==22.6.0
mandrill==1.0.60
python-gnupg==0.5.0
slackclient==2.9.4
user-agents==2.2.0
spectacles==2.2.0
dbt-core==1.2.1
dbt-snowflake==1.2.0
snowflake-connector-python[pandas]==2.7.12

One customer tested with Docker build and verified that worked, but could not get it working with astro deploy or astro dev start.

The issue seems to be with the version of pymssql included with either Airflow or these images. After a few days of poking around, we found a solution (thanks to this question).

FROM astronomerinc/ap-airflow:1.10.10-buster-onbuild
USER root
RUN apt-get update
RUN apt-get install -y \
      build-essential \
      freetds-bin \
      freetds-dev

RUN pip3 install Cython
RUN pip3 install --no-binary pymssql pymssql

I didn't put in a PR because I'm not sure if you'd want that backported to existing builds, or just wait until Airflow 2 is out (there's no issue with pyodbc).

It's not super clear what DEV_ALLOWLIST is intended for or whether or not it's still relevant, and it complicates a bunch of logic in our Jinja template (see below). We should either document it's use case(s), or we should remove it entirely.

1. ap-airflow/.circleci/config.yml.j2

   Line 50 in 5214a32

   {%- if "dev" in ac_version and airflow_version not in dev_allowlist %}

2. ap-airflow/.circleci/config.yml.j2

   Line 115 in 5214a32

   {%- if ("dev" not in ac_version or airflow_version in dev_allowlist) and not edge_build %}

3. ap-airflow/.circleci/config.yml.j2

   Line 162 in 5214a32

   {%- if ("dev" not in ac_version or airflow_version in dev_allowlist) and not edge_build %}

4. ap-airflow/.circleci/config.yml.j2

   Line 189 in 5214a32

   {%- if ("dev" not in ac_version or airflow_version in dev_allowlist) and not edge_build %}

5. ap-airflow/.circleci/config.yml.j2

   Line 232 in 5214a32

   {%- if airflow_version not in dev_allowlist %}

Hi, I was trying to use this image as the base image for our project and bumped into this issue. Here's the build log:

Step 1/18 : FROM quay.io/astronomer/ap-airflow:1.10.12-buster-onbuild
1.10.12-buster-onbuild: Pulling from astronomer/ap-airflow
852e50cd189d: Pulling fs layer
334ed303e4ad: Pulling fs layer
92c4d2410afa: Pulling fs layer
2ad460f59b0e: Pulling fs layer
029a44c5c02e: Pulling fs layer
44900377c3a4: Pulling fs layer
94e6315e644c: Pulling fs layer
0a3d218e3763: Pulling fs layer
d29d7c782ecb: Pulling fs layer
51e74141e244: Pulling fs layer
4547faff47df: Pulling fs layer
87430c4bbefd: Pulling fs layer
f8eb859db472: Pulling fs layer
a56073fffedd: Pulling fs layer
423471916159: Pulling fs layer
afd942d2534b: Pulling fs layer
30495f695d7a: Pulling fs layer
850a45aaed63: Pulling fs layer
eb3d3e6dbcba: Pulling fs layer
2ad460f59b0e: Waiting
029a44c5c02e: Waiting
44900377c3a4: Waiting
94e6315e644c: Waiting
0a3d218e3763: Waiting
d29d7c782ecb: Waiting
51e74141e244: Waiting
4547faff47df: Waiting
87430c4bbefd: Waiting
f8eb859db472: Waiting
a56073fffedd: Waiting
423471916159: Waiting
afd942d2534b: Waiting
30495f695d7a: Waiting
850a45aaed63: Waiting
eb3d3e6dbcba: Waiting
334ed303e4ad: Verifying Checksum
334ed303e4ad: Download complete
92c4d2410afa: Verifying Checksum
92c4d2410afa: Download complete
2ad460f59b0e: Verifying Checksum
2ad460f59b0e: Download complete
852e50cd189d: Verifying Checksum
852e50cd189d: Download complete
029a44c5c02e: Verifying Checksum
029a44c5c02e: Download complete
94e6315e644c: Verifying Checksum
94e6315e644c: Download complete
0a3d218e3763: Verifying Checksum
0a3d218e3763: Download complete
d29d7c782ecb: Verifying Checksum
d29d7c782ecb: Download complete
44900377c3a4: Verifying Checksum
44900377c3a4: Download complete
4547faff47df: Verifying Checksum
4547faff47df: Download complete
87430c4bbefd: Verifying Checksum
87430c4bbefd: Download complete
f8eb859db472: Verifying Checksum
f8eb859db472: Download complete
a56073fffedd: Verifying Checksum
a56073fffedd: Download complete
423471916159: Verifying Checksum
423471916159: Download complete
afd942d2534b: Verifying Checksum
afd942d2534b: Download complete
30495f695d7a: Verifying Checksum
30495f695d7a: Download complete
850a45aaed63: Verifying Checksum
850a45aaed63: Download complete
eb3d3e6dbcba: Verifying Checksum
eb3d3e6dbcba: Download complete
51e74141e244: Verifying Checksum
51e74141e244: Download complete
852e50cd189d: Pull complete
334ed303e4ad: Pull complete
92c4d2410afa: Pull complete
2ad460f59b0e: Pull complete
029a44c5c02e: Pull complete
44900377c3a4: Pull complete
94e6315e644c: Pull complete
0a3d218e3763: Pull complete
d29d7c782ecb: Pull complete
51e74141e244: Pull complete
4547faff47df: Pull complete
87430c4bbefd: Pull complete
f8eb859db472: Pull complete
a56073fffedd: Pull complete
423471916159: Pull complete
afd942d2534b: Pull complete
30495f695d7a: Pull complete
850a45aaed63: Pull complete
eb3d3e6dbcba: Pull complete
Digest: sha256:25c05d46fd4e6f00678bffd77f59fa046bdfc22bef873b0f42f55b7924317b1d
Status: Downloaded newer image for quay.io/astronomer/ap-airflow:1.10.12-buster-onbuild
# Executing 7 build triggers
---> Running in f5924208b34e
Removing intermediate container f5924208b34e
---> Running in 12e04f12762d
�[91m+ [[ -s packages.txt ]]
�[0mRemoving intermediate container 12e04f12762d
---> Running in 732e91387793
�[91m+ grep -Eqx 'apache-airflow\s*[=~>]{1,2}.*' requirements.txt
+ pip install --no-cache-dir -q -r requirements.txt
�[0m�[91mWARNING: You are using pip version 19.3.1; however, version 21.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
�[0mRemoving intermediate container 732e91387793
---> Running in c2cb9bd1d200
Removing intermediate container c2cb9bd1d200
---> 67f7e6cc9e06
Step 2/18 : RUN apt-get update && apt-get -y install build-essential sudo git && git clone https://github.com/ncopa/su-exec.git /tmp/su-exec && cd /tmp/su-exec && make && chmod 770 su-exec && mv ./su-exec /usr/local/sbin/
---> Running in 737a1ad5d22f
�[91m+ apt-get update
�[0mReading package lists...
�[91mE: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied)
�[0mThe command '/bin/bash -o pipefail -e -u -x -c apt-get update && apt-get -y install build-essential sudo git && git clone https://github.com/ncopa/su-exec.git /tmp/su-exec && cd /tmp/su-exec && make && chmod 770 su-exec && mv ./su-exec /usr/local/sbin/' returned a non-zero code: 100
build hook failed! (100)

How do I install pip installable modules? do i need to pass the requirement.txt in the dags folder?

This environment variable implies that the retention is only relevant to the worker logs. however this script is also used to clean the scheduler logs in the helm chart. These should either be controlled by separate variables or this name should change to be ASTRONOMER__AIRFLOW__LOG_RETENTION_DAYS

Hi, I'm using astronomerinc/ap-airflow:2.1.3-3-buster image.
It seems certificate of https://pip.astronomer.io/simple/ is expired.

Seems a little suspicious

Hello, do you plan support for images for Apple Silicon? Would be great to be able to run Astronomer on the new Macs! Right now the command astro dev start works but the containers (webserver, scheduler, database) refuse to work together.

Could you say when 2.3.3-buster-onbuild image will be available?

Since we have got such notification, we would like to update astronomer ASAP, but we need only buster version.

As discussed here, older versions of pip cause sqlalchemy to update beyond what we want when apache-airflow-providers-snowflake is installed. There is a workaround to pair apache-airflow-providers-snowflake with a restraint of sqlalchemy at the same time, but it would be simpler to just have pip figure it out.

Description

One thing that has been useful elsewhere and seems useful here is to add some info steps to CI after docker build. For instance we've done docker inspect afterwards to show metadata about the build. The build URL shows up in the metadata of a docker image, which makes a nice chain to follow to get info.

Example

https://quay.io/repository/astronomer/ap-airflow/manifest/sha256:23ccc960a0db114aadfa27d81f7bb7f947c576c45269340f0b7a8bf365ad9567 shows < io.astronomer.ci.build_url=https://circleci.com/gh/astronomer/ap-airflow/44315>, and if we go there, it would be great to see things like docker inspect and pip freeze and stuff, in order to know which components were included in that built docker image.

References

https://quay.io/repository/astronomer/ap-base/manifest/sha256:ba3b7812f0091cead3bc674d24ad9cce95a2d706f1be9c33cdac57f18139a221 shows < io.astronomer.build.url=https://circleci.com/gh/astronomer/ap-base/512> which shows docker inspect output https://app.circleci.com/pipelines/github/astronomer/ap-base/113/workflows/29aea806-ebbc-4203-95c7-f10e2eca93e1/jobs/512/parallel-runs/0/steps/0-103

Reasoning

Today I was trying to find a list of packages that were contained in a docker image, and this seems like a good method for having a historical record of that information.