Incremental GC collecting referenced objects (gctest fails with soft vdb if shared build by zig) about bdwgc

[ivmai]

Looks strange, let's figure out the root cause.

Which libgc version? Reproducible on fresh release-8_2 (or master)?

Which VDB mode? SOFT_VDB? The issue is not reproducible if compiled with NO_SOFT_VDB, right?

Also, try with NO_VDB_FOR_STATIC_ROOTS (instead of NO_SOFT_VDB) - based on your report, the issue is here.

(We need to understand why an object referenced from a static root directly is not marked.)

[ivmai]

If you confirm that SOFT_VDB is suspected then this might have the same root cause as issue #479

[plajjan]

I'm compiling using zig / clang and targetting glibc 2.28.

Using git commit daea2f1 so that's relatively close to HEAD of master.

I haven't explicitly picked a VDB mode. I think it defaults to SOFT_VDB on Linux, right? So yes, SOFT_VDB!?

And yes indeed, compiling with either -DNO_SOFT_VDB or -DNO_VDB_FOR_STATIC_ROOTS appears to fix the problem, I don't see any more crashes while running the acton test suite.

What does NO_VDB_FOR_STATIC_ROOTS actually mean? That we scan the static root ranges on every GC run, right? But for the rest we rely on VDB?

[ivmai]

I think it defaults to SOFT_VDB on Linux

Yes

And yes indeed, compiling with either -DNO_SOFT_VDB or -DNO_VDB_FOR_STATIC_ROOTS appears to fix the problem, I don't see any more crashes while running the acton test suite.

Got it. The issue relates to code within ifndef NO_VDB_FOR_STATIC_ROOTS block.

What does NO_VDB_FOR_STATIC_ROOTS actually mean?

With SOFT VDB (and PROC_VDB) facility the scanning of static roots could be optimized too (unless NO_VDB_FOR_STATIC_ROOTS is defined) to skip unmodified pages during collections other than full ones.

The algorithm of full collections should be re-inspected.

[ivmai]

I need your help with figuring out the root cause.

I ran the program with gdb, put a breakpoint at initialization of reQ_W_125

Could you please check that soft_set_grungy_pages(0x7f9412cca000, 0x7f9412cfc000, ...) is called? (without -D NO_VDB_FOR_STATIC_ROOTS) And when is it called among output lines above?

[ivmai]

It seems the information that reQ_W_125 has been written (i.e. page is dirty) is lost somehow.
-D DEBUG_DIRTY_BITS enables the relevant logging.

[ivmai]

I could debug this but I need a a guide how to setup the environment. Which host OS/arch?

[plajjan]

@ivmai I'm super busy at the moment preparing for a conference next week, I'll get back as soon as I can after that with more info! :)

[ivmai]

@michaellilltokiwa, I moved the issue reported by you in a separate one - #552

[ivmai]

@plajjan, I've implemented a checker (CHECK_SOFT_VDB) that if some page was identified as dirty by mprotect-based VDB then it should be also identified as dirty by SOFT_VDB implementation, commit b4e1ce5.
Please try to build bdwgc (master) passing -D CHECK_SOFT_VDB to CFLAGS_EXTRA. Just in case it would be useful in figuring out the root case.
PS. On my Ubuntu 22.04 x64 machine I see some rare inconsistency on gctest (i.e. some page is reported dirty by mprotect-based but pagemap file says it is clean) - I have not investigated it yet by looks like an issue in the kernel.

[ivmai]

Is the issue not observed if -D NO_SFT_VDB passed to CFLAGS?

[ivmai]

Reproduced with the following build command:
../zig/zig build -DBUILD_SHARED_LIBS -Dbuild_tests -Denable_gc_assertions -Denable_threads=false -Denable_gcj_support=false -Denable_disclaim=false -Denable_munmap=false -Ddisable_handle_fork test
Also reproduced if -Ddisable_single_obj_compilation option added.

Not reproduced with any of:

• -DBUILD_SHARED_LIBS=false option
• -D NO_VDB_FOR_STATIC_ROOTS added to C flags
• without -fvisibility=hidden in C flags (but only if -Denable_gc_assertions option is given)
• build with cmake
• build with configure

[ivmai]

The root cause seems to be the same as in #376 .
soft_set_grungy_pages is to be fixed.