Problem: The size of the Docker images has grown massively since >= v2.9.92 about semaphore

[tboerger]

I have done some investigation and the increase of the image size is related to all the packages that got to be installed for being able to install python modules which rely on things like cargo, rust, openssl-dev and so on:

❯ podman history localhost/semaphore-server
ID            CREATED        CREATED BY                                     SIZE        COMMENT
96e8da9d0108  5 minutes ago  /bin/sh -c #(nop) CMD [ "/usr/local/bin/se...  0B          FROM 346dfc23091c
<missing>     5 minutes ago  /bin/sh -c #(nop) ENTRYPOINT ["/sbin/tini"...  0B          FROM 96e8da9d0108
<missing>     5 minutes ago  /bin/sh -c mkdir /opt/semaphore/venv &&   ...  515MB       FROM 8f79f495a9f5
efda46108e15  6 minutes ago  /bin/sh -c #(nop) ENV ANSIBLE_VERSION 9.4.0    0B          FROM d231254cd63b
<missing>     6 minutes ago  /bin/sh -c #(nop) USER 1001                    0B          FROM 66bf78857ec4
<missing>     6 minutes ago  /bin/sh -c #(nop) WORKDIR /home/semaphore      0B          FROM efda46108e15
<missing>     6 minutes ago  /bin/sh -c chown -R semaphore:0 /usr/local...  26.3MB      FROM 15e486aa2feb
15e486aa2feb  6 minutes ago  /bin/sh -c #(nop) COPY file:9de15c57fc07d5...  26.3MB      FROM 71fa1182ee64
71fa1182ee64  6 minutes ago  /bin/sh -c #(nop) COPY file:73ee5cf0ce3774...  9.22kB      FROM 209c4601967b
209c4601967b  6 minutes ago  /bin/sh -c #(nop) COPY file:8c8e1eb063ffa2...  3.07kB      FROM 27e5a7ea7740
27e5a7ea7740  7 minutes ago  /bin/sh -c apk add --no-cache -U bash curl...  1.24GB      FROM docker.io/library/alpine:3.19
05455a08881e  4 months ago   /bin/sh -c #(nop)  CMD ["/bin/sh"]             0B
<missing>     4 months ago   /bin/sh -c #(nop) ADD file:37a76ec18f98877...  7.67MB

These tools are required to enable the option to install python packages on demand which many people require for their customization needs.

That's the downside the python modules like cryptography depend on rust:

4350ed42c294:~$ du -sh /usr/lib/* | grep rustlib
487.2M  /usr/lib/rustlib
4350ed42c294:~$ du -sh /usr/lib/* | grep python
122.5M  /usr/lib/python3.11
4350ed42c294:~$ du -sh /opt/semaphore/venv
475.1M  /opt/semaphore/venv

Some cleanup of __pycache__ folders reduces the storage some bits:

$ du -sh /opt/semaphore/venv
$ find /opt/semaphore/venv -iname __pycache__ -type d | xargs rm -rf
$ du -sh /opt/semaphore/venv

475.1M  /opt/semaphore/venv
270.6M  /opt/semaphore/venv

du -sh /usr/lib/python*
find /usr/lib/python* -iname __pycache__ -type d | xargs rm -rf
du -sh /usr/lib/python*

122.5M  /usr/lib/python3.11
97.0M   /usr/lib/python3.11

I will put this minor cleanup into a PR.

[tboerger]

It won't reduce the image size a lot, but better than nothing :)

[fiftin]

Reduced start from 2.10.2

[gschafra]

Hmmm... maybe becaus of arm support, see ae04954, right?

[tboerger]

Hmmm... maybe becaus of arm support, see ae04954, right?

This is not related to that, other architectures are simply separate tags which does not affect image sizes for other architectures.

[tboerger]

#2083

[fiftin]

Hi @tboerger
I splitted package installation to layers:

cargo takes 700 mb. I think we should remove it from base image. And build-base (200 mb) too.

[tboerger]

It can be possible to remove them, if the installation of ansible is done within the same RUN, but since rust is used by multiple python libraries it could be missing for requirements.txt installations at runtime which would require the installation of it based on packages.txt again.