icinga2 may be removed from Debian armel, mips64el, ppc64el, and riscv64 about icinga2

[julianbrost]

• armel:

/<>/lib/base/application.hpp:148:33: error: static assertion failed

buildd.debian.org/status/fetch.php?pkg=icinga2&arch=armel&ver=2.14.1-1&stamp=1703225119&raw=0

So looks like no lock-free implementation of std::atomic<uint32_t> available on that platform:

icinga2/lib/base/application.hpp

Lines 147 to 148 in 2c9117b

	typedef Atomic<std::conditional_t<Atomic<double>::is_always_lock_free, double, uint32_t>> AtomicTs;
	static_assert(AtomicTs::is_always_lock_free);

If someone wants to make it work, go ahead, but I'm not sure if it's worth the time for some old ARM systems.

• mips64el:

152 - base-methods_pluginnotificationtask/truncate_long_output (Failed)

buildd.debian.org/status/fetch.php?pkg=icinga2&arch=mips64el&ver=2.14.1-1&stamp=1704107092&raw=0

• ppc64el:

152 - base-methods_pluginnotificationtask/truncate_long_output (Failed)

buildd.debian.org/status/fetch.php?pkg=icinga2&arch=ppc64el&ver=2.14.1-1&stamp=1703225525&raw=0

That was recently added in #9887. Not sure what's going wrong on these architectures. I don't think we have easy access to such machines so help for debugging this is greatly appreciated.

• riscv64:

/usr/bin/ld: /usr/lib/riscv64-linux-gnu/libboost_coroutine.so.1.83.0: undefined reference to jump_fcontext' /usr/bin/ld: /usr/lib/riscv64-linux-gnu/libboost_coroutine.so.1.83.0: undefined reference to make_fcontext'

buildd.debian.org/status/fetch.php?pkg=icinga2&arch=riscv64&ver=2.14.1-1&stamp=1703250691&raw=0

That actually seems to be an issue in Boost which, according to Debian bug #1059046, should already be fixed there.

[aurel32]

The issue seems to be an confusion between MAX_ARG_STRLEN, which represents the maximum length of a single argument, and syconf(_SC_ARG_MAX), which is the maximum length of the command. Note that the maximum length defaults ot 1/4 of the stack size, and can therefore be smaller than MAX_ARG_STRLEN. This is the case on ppc64el which has a page size of 64kB. Therefore the fix for the plugin would be something like:

--- icinga2-2.14.1.orig/lib/methods/pluginnotificationtask.cpp
+++ icinga2-2.14.1/lib/methods/pluginnotificationtask.cpp
@@ -13,17 +13,12 @@
 #include "base/convert.hpp"

 #ifdef __linux__
-#      include <linux/binfmts.h>
 #      include <unistd.h>
-
-#      ifndef PAGE_SIZE
-//             MAX_ARG_STRLEN is a multiple of PAGE_SIZE which is missing
-#              define PAGE_SIZE getpagesize()
-#      endif /* PAGE_SIZE */
+#      define ARG_MAX sysconf(_SC_ARG_MAX)

 // Make e.g. the $host.output$ itself even 10% shorter to leave enough room
 // for e.g. --host-output= as in --host-output=$host.output$, but without int overflow
-const static auto l_MaxOutLen = MAX_ARG_STRLEN - MAX_ARG_STRLEN / 10u;
+const static auto l_MaxOutLen = ARG_MAX - ARG_MAX / 10u;
 #endif /* __linux__ */

 using namespace icinga;

Then the second issue, is that the test hard codes the way to trigger the issue by using a 1024*1024 size.

[noloader]

Thanks @julianbrost,

Just one comment...

base-methods_pluginnotificationtask/truncate_long_output (Failed)

That was recently added in #9887. Not sure what's going wrong on these architectures. I don't think we have easy access to such machines so help for debugging this is greatly appreciated.

It will probably be easiest to use a KVM/QEMU virtual machine, though emulation will be a little slow. I endure it for a couple of projects I help with, like when I need a s390x machine.

The Gnulib folks recently added instructions for QEMU at https://git.savannah.gnu.org/gitweb/?p=gnulib/maint-tools.git;a=tree;f=platforms/environments/qemu. Once you setup a Debian based machine, switch from Bookworm/Stable to Unstable, then apt-get update && apt-get upgrade. That should get you to a similar machine that exhibits the issue.

And if you don't have time, then I understand that, too. I understand managing a project takes a lot of time.

[aurel32]

Thanks for pointing at the commit that introduced the issue. Looking at it and the affected architectures, it could be linked to the fact that the page size on those architectures is not 4K.

[noloader]

@aurel32,

That looks like a good patch for Debian to carry until the Icinga folks determine their path forward. It will clear FTBFS on two platforms. And coupled with the Boost fix that @julianbrost provided, that takes the total to three platforms.

I hate to see armel and armv7 left behind, but I think its good progress none the less.

[aurel32]

The patch is only part of the fix, just the the plugin, not the testsuite. Also playing more with this, I noticed the problem is more complicated.

E2BIG happens when:

• One argument is bigger than MAX_ARG_STRLEN
• All arguments + environment is bigger than sysconf(_SC_ARG_MAX)

The default value of sysconf(_SC_ARG_MAX) on all modern Linux systems seems to be 2MiB. The value of MAX_ARG_STRLEN, changes with the page size, so it's 128KiB on systems with 4KiB pages (e.g. x86), 512KiB on systems with 16KiB pages (e.g. some mips systems), and 2MiB on systems with 64KiB pages (e.g. ppc64el).

So on an x86 system you can pass up to 16 arguments of length MAX_ARG_STRLEN, while on ppc64el, you can pass only one.

The current code and testsuite does not look at the number of arguments and assumes that MAX_ARG_STRLEN is the only limit.

[Al2Klimov]

So looks like no lock-free implementation of std::atomic<uint32_t> available on that platform:

icinga2/lib/base/application.hpp

Lines 147 to 148 in 2c9117b

	typedef Atomic<std::conditional_t<Atomic<double>::is_always_lock_free, double, uint32_t>> AtomicTs;
	static_assert(AtomicTs::is_always_lock_free);

If someone wants to make it work, go ahead, but I'm not sure if it's worth the time for some old ARM systems.

What about locking instead of requiring a lock-free number? https://github.com/Icinga/icinga2/pull/8430/files#diff-f60819a3bc65b59dde8810c55c02f627b457f605fd27445e523fbd44618c1f10R157

[julianbrost]

What about locking instead of requiring a lock-free number? #8430 (files)

The nice thing about atomic operations is that they can't block indefinitely. Using shared memory locking instead just adds a lot more room for error, especially given that we do so nice things like the following:

icinga2/lib/base/application.cpp

Line 120 in 2c9117b

_exit(rc); // Yay, our static destructors are pretty much beyond repair at this point.

So if that would happen while that process held the lock, it would remain locked indefinitely.

Is there really a lack of 32 bit atomic operations on these architectures or is it just that those are not implemented in std::atomic for these platforms or that they have additional memory alignment requirements so that the always part of the check is the problem?