PyChess Debian packages should avoid zstd compression for compatibility about pychess

[theofficialgman]

@Botspot a new release hasn't been made yet since the fix was implemented

[hartwork]

Hi @theofficialgman, thanks for bringing this to our attention!

When playing with Ubuntu 18.04 I noticed that their Python 3 is 3.6.x which is end-of-life and too old for PyChess 1.0.4 which requires >=3.7, so Ubuntu 18.04 won't work any way.

For Ubuntu 20.04, installation seems to work fine without changes. Here's the Dockerfile I used for proof:

FROM ubuntu:20.04
RUN apt-get update && apt-get install --no-install-recommends --yes -V \
        ca-certificates \
        wget
RUN wget https://github.com/pychess/pychess/releases/download/1.0.4/python3-pychess_1.0.4-1_all.deb
RUN apt-get install --yes ./python3-pychess_1.0.4-1_all.deb

So before we take any action here, I would like to better understand why you're getting that error "unknown compression", which system are affected, and if installing package zstd on your side would solve the issue. What do you think?

[theofficialgman]

@hartwork please re-read my post. The information you requested on the systems affected by this are contained. As for what systems do support tar.zst in dpkg, only Ubuntu systems (where Ubuntu backported support to all LTS versions), and debian bookworm (unstable) support tar.zst in dpkg. Debian has ZERO intention or will to backport tar.zst support in dpkg to stable releases (buster and bullseye). I have asked already and was shot down.

No the package you suggest has nothing to do with what compression formats dpkg supports.

You will find more information about this than you could ever care for here
The original bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=892664
My request to backport to stable which was shot down: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1033961

The long and short of it is you must start building on 20.04 or manually specify the compression method inorder to restore support with all debian stable versions

[hartwork]

Hi @theofficialgman, let's keep this constructive and collaborative please, I do want to help and understand your case but we also need to find a future-proof solution and downgrading to 20.04 in CI could be a short-term fix (where it's not yet clear if it would even help without a new release 1.0.5 also) but is not sustainable. I can think of many ways to address this but I'll first need to understand the situation better. To save your and my time, if you're up for jumping on a voice chat call with me — https://meet.jit.si/ or something — maybe that can save us hours of reading and ping-pong? My e-mail is in my profile.

[theofficialgman]

Your best source of information is the resources that I linked. When building debs, Ubuntu changed the default compression mechanism to tar.zst in 22.04. 20.04 remains with a default compression mechanism of tar.xz and will remain as such indefinitely. As I already mentioned, you can override dpkg-deb/dpkg-buildpackagd/debhelper/whatever debian helper you use and specify a desired compression format.

I suggest you refer to the manpage for your desired tool for how to change that as the flags/options differ slightly between them. (dpkg-deb 22.04 manpage for reference https://manpages.ubuntu.com/manpages/jammy/man1/dpkg-deb.1.html scroll down to compress-type where you will see the default was changed to zst which isn't supported on debian. At the top of the page change to 20.04 and you will see that the default is xz which is supported on debian).

[hartwork]

@theofficialgman installation of zstd-based .deb files seems to work just fine on Debian bullseye, in practice. Here's my Dockerfile:

FROM debian:bullseye
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install --no-install-recommends --yes -V \
        ca-certificates \
        wget
RUN wget https://github.com/pychess/pychess/releases/download/1.0.4/python3-pychess_1.0.4-1_all.deb
RUN apt-get install --yes ./python3-pychess_1.0.4-1_all.deb
RUN pychess --help

So what system is this an actual problem for?

I can confirm that…

dpkg-deb --raw-extract ./python3-pychess_1.0.4-1_all.deb ./python3-pychess_1.0.4-1_all.dir
dpkg-deb --build -Zxz ./python3-pychess_1.0.4-1_all.dir ./python3-pychess_1.0.4-1_all.deb

… works to re-compress the .deb file. If DPKG_DEB_COMPRESSOR_TYPE doesn't work in CI (where bdist_deb is used), that's what we could do manually in CI if needed.

[theofficialgman]

I have CI and users and package management understanding that says otherwise. It is an issue on debian bullseye and all older debian releases. You should go to the dockerfile maintainer of that debian image. They have likely tainted it with packages not from bullseye.

You can see here that it is failing on all the following distros https://github.com/Botspot/pi-apps/actions/runs/4745384249

[hartwork]

It turned out now that python3-pychess_1.0.4-1_all.deb does not finish the installation on bullseye successfully, the error shows quite late and I made a wrong assumption that it would need to error out right away. So that means you did not run my Dockerfile and the .deb is now confirmed to not be installable on Debian bullseye. Finally some hard results. I will work on a pull request.

[theofficialgman]

I gave you hard results in my very first message showing it fail in buster CI images but it doesn't matter. I'm just happy a fix is being implemented. Thanks for your time.

[hartwork]

@theofficialgman I was considering possible that installability and dpkg-pkg complaints were out of sync in bullseye earlier…

With some luck PR #2050 should produce artifacts for download at https://github.com/pychess/pychess/actions/runs/4746697027?pr=2050 when done and that zip file should contain an xz-compressed .deb file. Please let me know how that new file works for you.

[Botspot]

This issue seems to be occurring again. Has something changed?

[Botspot]

Anyone curious can check out our automated test run of installing PyCharm to see the errors directly.
See: https://github.com/Botspot/pi-apps/actions/runs/5733068492/job/15537171266#step:7:2773