lvc / abi-dumper Goto Github PK

The tool can't properly identify enumeration data types in objects compiled by GCC 5.2.

Allow setting a custom path for the vtable dumper binary. --vtable-dumper-path= maybe?

Hello,
I followed instructions in the "Usage with ABI Dumper" section of the index.html documentation, since this usage is very convenient. I built the library (libpetsc.so) with "-g -Og" on a Linux machine with gcc. Then,

~/abi-dumper/abi-dumper.pl ~/old/linux-gnu-dbg/lib/libpetsc.so -lver 0 -o old.dump
ERROR: missed type id 713704
<many similar errors>
Creating ABI dump
The object ABI has been dumped to:
  old.dump

~/abi-dumper/abi-dumper.pl ~/new/linux-gnu-dbg/lib/libpetsc.so -lver 1 -o new.dump
ERROR: missed type id 696722
<many similar errors>
Creating ABI dump
The object ABI has been dumped to:
  new.dump

~/abi-compliance-checker/abi-compliance-checker.pl -l libpetsc -old old.dump -new new.dump
Preparing, please wait ...
Comparing ABIs ...
Comparing APIs ...
Creating compatibility report ...
Binary compatibility: 100%
Source compatibility: 100%
Total binary compatibility problems: 0, warnings: 0
Total source compatibility problems: 0, warnings: 0
Report: compat_reports/libpetsc/0_to_0/compat_report.html

But actually, the new library is not API compatible with the old since it changed the name of an enum. So, what is the recommended way of using abi-compliance-checker.pl if I want very accurate results.
Thank you.

When dumping an ABI with abi-compliance-checker, it is possible to exclude namespaces with the directive. Would it be possible to do the same for abi-dumper? I'm getting lots of noise about added symbols in internal namespaces that have nothing to do with the public API. It would be nice to be able to suppress that.

Hi, I am trying to use abi-dumper but I am encountering the following error "ERROR: can't find "eu-readelf" from Elfutils".

I have installed the elfutils into a custom location but I don't know how to point the abi-dumper tool to it? I've also tried adding it to the LD_LIBRARY_PATH. Would you have any ideas? It works if I install Elfutils into the default system usr/ directory however it needs to be in a specified location.

Thanks in advance for your help!

command abi-dumper library/libmbedtls.a -o ABI-0.dump never finishes when run with attached lib.
libmbedtls.zip

I am having a project structure which includes both public and private headers. following is the example structure:
|-headerDirectory
|---Directory1
|---Directory2
|---Directory3
|---Directory4
|---HeaderFile.h
|---header.txt

Here headerDirectory contains header files for both public as well as private modules. for example Directory1, Directory2 are publicly exposed and Directory3 and Directory4 are for internal use only. My HeaderFile.h file contains all public header files.

If i use "-public-headers ~/headerDirectory", i am able to generate .dump file. But .dump contains info for both public as well as private symbols.

If i use "-public-headers ~/headerDirectory/header.txt" which basically have "HeaderFile.h" only, abi-dumper does not produce .dump file after following output over console:
Detect public symbols
Reading debug-info

I am not sure if its processing or stuck somewhere. Is there a option by which i can know progress made by abi-dumper. if -debug is the option to be used how i can enable it?

Please correct me if there is any gap in my understanding

We recently upgraded our "abi-dumper" build from fedora:33 -> fedora:34 and we started seeing the following errors with our project google-cloud-cpp

$ abi-dumper /var/tmp/google-cloud-cpp/lib64/libgoogle_cloud_cpp_spanner_protos.so
WARNING: module version is not specified (-lver NUM)
Reading debug-info
Integer overflow in hexadecimal number at /usr/bin/abi-dumper line 1496, <$INFO_fh> line 17646.
Integer overflow in hexadecimal number at /usr/bin/abi-dumper line 1496, <$INFO_fh> line 223879.
ERROR: invalid debug_loc section of object, please fix your elf utils
Integer overflow in hexadecimal number at /usr/bin/abi-dumper line 1496, <$INFO_fh> line 326184.
...
^C

Looking at the eu-readelf output for one of the offending lines above (17646) we see that the output is:

$ eu-readelf --debug-dump=info /var/tmp/google-cloud-cpp/lib64/libgoogle_cloud_cpp_spanner_protos.so | cat -n | grep -C10 -P "\b17646\b"
 17636                   declaration          (flag_present) yes
 17637                   sibling              (ref4) [  91af]
 17638   [  91a9]          formal_parameter     abbrev: 2
 17639                     type                 (ref4) [ 30cd3]
 17640   [  91af]        member               abbrev: 277
 17641                   name                 (strp) "_M_head_impl"
 17642                   decl_file            (data1) tuple (56)
 17643                   decl_line            (data1) 125
 17644                   decl_column          (data1) 39
 17645                   type                 (ref4) [  8ebc]
 17646                   data_member_location (sdata) 18446744073709551615  # <--- BIG NUM: 2^64-1
 17647   [  91bd]        template_value_parameter abbrev: 118
 17648                   name                 (strp) "_Idx"
 17649                   type                 (ref4) [  e806]
 17650                   const_value          (data1) 1
 17651   [  91c7]        template_type_parameter abbrev: 35
 17652                   name                 (strp) "_Head"
 17653                   type                 (ref4) [  8ebc]
 17654   [  91d1]      const_type           abbrev: 19
 17655                 type                 (ref4) [  90d3]
 17656   [  91d6]      structure_type       abbrev: 60

In the above output the 18446744073709551615 (== 0xFFFFFFFFFFFFFFFF) is the number that's too big for the perl call to hex($2) in abi-dumper (it's also not hex).

The issue may be a bug in elfutils on Fedora 34. Note that there is also a line saying ERROR: invalid debug_loc section of object, please fix your elf utils in the first block.

However, it would be nice if abi-dumper could workaround this issue. A diff like the following seems to quiet the Integer overflow warnings for me:

--- /usr/bin/abi-dumper 2021-01-26 00:20:00.000000000 +0000
+++ /tmp/abi-dumper     2021-06-08 16:54:45.905900421 +0000
@@ -1493,9 +1493,11 @@

                 if($Val=~/\A\(\w+\)\s*(-?)(\w+)\Z/)
                 { # (data1) 1c
-                    $Val = hex($2);
-                    if($1) {
-                        $Val = -$Val;
+                    if ($2 != 0xFFFFFFFFFFFFFFFF) {
+                        $Val = hex($2);
+                        if($1) {
+                            $Val = -$Val;
+                       }
                     }
                 }
                 else

abi-dumper is easy to use and combine with abi-compliance-checker and has it been used in several projects to track backwards compatibility. The way it was been used in the projects I was involved is keep a dump of old libraries, and then compare the current build against them.

Unfortunately at 1.1 abi-dumper broke compatibility with older files.

That caused the following issues.

1. The abi-compliance-checker suddently stopped working within updates in distributions [0]
2. Dumps of old libraries are no longer useful, and even worse, most of the times it is impossible to re-create (for the libidn2 library which I was trying the first versions are notoriously hard to compile on modern systems) - see [1]
3. abi-dumper and abi-compliance-checker can now be used only if the dump is generated in the same system as the compliance-checker is run.

(3) is the worst of them as it makes relying to abi-dumper for continuous testing purposes impossible. It is very hard to guarrantee indefinitely that the system where these files are generated is exactly the same as the one abi-compliance-checker is run (as I said old libraries cannot be compiled in any arbitrary system, I have to compile them on the old systems they were intended to be run).

The question is what are the best practices for running these tools for that purpose? Would it make sense to have a minimal baseline format which will be portable and be always supported?

[0]. https://bugzilla.redhat.com/show_bug.cgi?id=1502214
[1]. https://gitlab.com/libidn/libidn2/issues/42

Hi, I am creating an abi dump file and I added the -public-headers "/path/to/my/headers.txt" option. The content of headers.txt is something like:

/home/knitschi/CppLibraries/CppCodeBase/Sources/libQtUtilities/QObjectUtilities.h
/home/knitschi/CppLibraries/CppCodeBase/Sources/libQtUtilities/QPrintTo.h
/home/knitschi/CppLibraries/CppCodeBase/Sources/libQtUtilities/QStringUtilities.h

Now the tool gives me the error

Detect public symbols
ERROR: requires Universal Ctags to work properly

which does not occur when I do not add the -public-headers option. However the abi-dump is created and the abi-dumper does not return with an error.

Any ideas about what went wrong?

I found abi-dumper and abi-compliance-checker are depend on each other.
So how can we break the cyclic dependency?

Dear Andrey,
Is it possible to accelerate abi-dumper?
Profiling of one test library:

20 sec and 11041973 calls of line 1438:
if(defined $ID and $Line=~/\A\s*(\w+)\s+(.+?)\s*\Z/)

13 sec and 8939481 calls of line 1436:
while(my $Line = <$FH>)

7sec and 4881590 calls of line 1495:
if($Val=~/\A\s*\(([^()]*)\)\s*\[\s*(\w+)\]\s*\Z/)

It would be great to boost it up. But feel free to close the issue if this is impossible.
With appreciation of your hard work and best regards,
Ivan

I tested tesseract 4.1.0 and tesseract 4.1.1 with these steps:

git clone https://github.com/tesseract-ocr/tesseract.git
git checkout 4.1.0
./autogen.sh && ./configure --prefix=/opt/tesseract410 --enable-debug CXXFLAGS="-pedantic -Og"
make -j 4 && make -j 4 training
sudo make install && sudo make training-install

make clean
git checkout 4.1.1
./autogen.sh && ./configure --prefix=/opt/tesseract411 --enable-debug CXXFLAGS="-pedantic -Og"
make -j 4 && make -j 4 training
sudo make install && sudo make training-install


abi-dumper /opt/tesseract410/lib/libtesseract.so.4 -o tesseract-4.1.0.dump -lver tesseract-4.1.0 -public-headers /opt/tesseract410/include/tesseract/

abi-dumper /opt/tesseract411/lib/libtesseract.so.4 -o tesseract-4.1.1.dump -lver tesseract-4.1.1 -public-headers /opt/tesseract411/include/tesseract/

abi-compliance-checker -l tesseract-4.1.1 -old tesseract-4.1.0.dump -new tesseract-4.1.1.dump


chromium-browser compat_reports/tesseract-4.1.1/tesseract-4.1.0_to_tesseract-4.1.1/compat_report.html

According my report there is Compatibility 1 00%. But abi-laboratory.pro report shows only 96.4% backward
Compat. Why?

teseract_api_comp.tar.gz

This is related to #19 and to lvc/abi-compliance-checker#89:

• Create a .so with one symbol, library_version and dump it with abi-dumper
• Rename the symbol to libraryVersion and dump it with abi-dumper
• abi-dumper -compare shows there is a different
• abi-compliance-checker -l example -old 1.dump -new 3.dump shows 100% compatibility

1.txt
3.txt

We need to port the tool on Mac OS with the help of gobjdump or dwarfdump tools in order to simplify usage of the ABI Compliance Checker.

Is there a documentation somewhere about the semantics of what's output by abi-dumper? (so that anyone can write tools that analyze such files)

Sometimes debug info in the input objects misses information for some data types. In this case such data types are marked as unknown. The tool should warn the user if it will find such types in the debug info.

More of a question than an true issue, I suppose.

If I build with -g to get symbols, but without also using -Og (for example, in CMake nomenclature, RelWithDebInfo versus Debug), I get warnings.

But what exactly are the ABI incompatibilities that -Og will allow it to detect, which it cannot do otherwise?

I ask because the abi dump files from a full Debug build of a large library are enormous, but the ones from a RelWithDebInfo build are sufficiently smaller that it allows me to check them into the project as reference for CI checks of ABI conformance. I'd love to stick to the smaller dump files, unless by doing so I am disabling a part of the analysis that is actually important to me.

We are having problems with functions with the same name but different types in the public and private ABI. It seems abi-dumper considers the function in the private abi public as well.

For example:

// public/foo.hh
class Foo;
bool operator==(const Foo& a, const Foo& b);

// private/bar.hh
class Bar;
bool operator==(const Bar& a, const Bar& b);

$ abi-dumper.pl libfoo.so -lver 1 -public-headers public -o foo.dump

When we remove bool operator==(const Bar& a, const Bar& b) from the library it is marked as an ABI change by abi-compliance-checker.pl which I think it shouldn't be. Am I misunderstanding something?

I have uploaded a small repro case where I expect void foo1(int) (_Z4foo1i) and void foo3() (_Z4foo3v) to be private as void foo2() is. But instead they appear in the SymbolInfo map, not marked private. Note that void foo3() is made public because of void Foo::foo3()(!).

OS: RHEL 7.7
g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
ABI Dumper 1.1
Vtable-Dumper 1.2
Exuberant Ctags 5.8

From the Fedora Bugzilla report:

/usr/bin/abi-dumper contains a Perl expression that will stop working in Perl 5.30 (late May 2019):

$ abi-dumper --help >/dev/null
Unescaped left brace in regex is deprecated here (and will be fatal in Perl 5.30), passed through in regex; marked by <-- HERE in m/(\w){ <-- HERE / at /usr/bin/abi-dumper line 3200.

You need to apply this fix:

--- /usr/bin/abi-dumper 2019-01-31 13:54:11.000000000 +0100
+++ abi-dumper  2019-03-05 09:39:06.432000000 +0100
@@ -3197,7 +3197,7 @@
     
     if(defined $LambdaSupport)
     { # struct {lambda()}
-        $N=~s/(\w){/$1 {/g;
+        $N=~s/(\w)\{/$1 {/g;
     }
     
     return ($Cache{"formatName"}{$_[1]}{$_[0]} = $N);

abi-dumper-1.1-6.fc30.noarch is affected.

https://bugzilla.redhat.com/show_bug.cgi?id=1685441

When I tried to use abi-dumper to dump some shared library (.so). The abi-dumper will show missed type ERROR about __restrict type which was compiled by clang-3.6, but it works fine on compiling by gcc.

I've taken a look at the debug info of it, and It looks fine. There is my testing example compiled by clang-3.6:

• The type is int * __restrict QAQ
• The result from readelf --debug-dump=info is:

 <2><4d>: Abbrev Number: 3 (DW_TAG_formal_parameter)
    <4e> DW_AT_location : 2 byte block: 91 70 (DW_OP_fbreg: -16)
    <51> DW_AT_name : (indirect string, offset: 0xa9): QAQ
    <55> DW_AT_decl_file : 1
    <56> DW_AT_decl_line : 9
    <57> DW_AT_type : <0x83>

 <1><83>: Abbrev Number: 8 (DW_TAG_restrict_type)
    <84>   DW_AT_type        : <0x88>

 <1><88>: Abbrev Number: 9 (DW_TAG_pointer_type)
    <89>   DW_AT_type        : <0x7c>   

 <1><7c>: Abbrev Number: 7 (DW_TAG_base_type)
    <7d> DW_AT_name : (indirect string, offset: 0xa1): int
    <81> DW_AT_encoding : 5 (signed)
    <82> DW_AT_byte_size : 4 

And after I added "restrict_type"=>"Restrict" in my %TypeType(at line 352), then it looks work fine with __restrict keyword compiled by clang-3.6.

I tried using abi-dumper with kernel 3.18.5 compiled with CONFIG_DEBUG_INFO=y , but I'm getting:

ERROR: missed type name (66820236)
ERROR: missed type id 223437
ERROR: missed type name (11706)
ERROR: missed type name (11803)
ERROR: missed type id 16785117
ERROR: missed type name (16225)
ERROR: missed type id 35603
ERROR: missed type name (60824)
ERROR: missed type name (60704)
ERROR: missed type id 99833
ERROR: missed type id 18524
ERROR: missed type id 717
ERROR: missed type name (20971)
ERROR: missed type name (20918)
ERROR: missed type id 18556
ERROR: missed type name (20828)
ERROR: missed type name (20795)
ERROR: missed type name (20733)
ERROR: missed type id 35668251

and more...

What is the problem?

Under my Linux (CentOS-7), I built my library with -g -Og, and then dump the ABI.
After a lot of "ERROR: missed type id XXXX", it did generate the abi dump. But I saw may strange things like 'Header' => undef in the dump file.
Comparing dumps of two versions of the library with abi-compliance-checker.pl gives me "compatibility: 100%". Actually, it is not true.

Hi,

I'm opening this issue as an enhancement idea.

gcc >= 11 now defaults to DWARF 5, which makes abi-dumper fail due to the missing debug_loc section:

ERROR: invalid debug_loc section of object, please fix your elf utils

It seems debug_loc has been replaced with debug_loclists, which is considered a "more compact representation".

Thanks for open-sourcing abi-compliance-checker, it's an amazing toolset.

I have a request[1] to update abi-dumper for 1.3 for Fedora and it's even referenced in the README but there is no actual release published on github.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=2041550

Found while working on a package update for jack sound server on Rosa Fresh 2021.1 (x86_64).
In this distribution release, the build process make use of dwz to reduce the debugging info size; eg, for jack package, the build log contains the following line:
/usr/bin/find-debuginfo -j12 -m -i --build-id-seed 1.9.16-2 --unique-debug-suffix -1.9.16-2.x86_64 --unique-debug-src-base jack-1.9.16-2.x86_64 --run-dwz --dwz-low-mem-die-limit 10000000 --dwz-max-die-limit 110000000 -S debugsourcefiles.list /tmp/abf/rpmbuild/BUILD/jack2-1.9.16

But this causes abi-build to fail:

$ abi-dumper libjack.so -o ABI-0.dump -lver 1.9.16
Found link to libjack.so.0.1.0-1.9.16-2.x86_64.debug (gnu_debuglink)
Reading debug-info file libjack.so.0.1.0-1.9.16-2.x86_64.debug linked from gnu_debuglink
WARNING: can't access '../../.dwz/jack-1.9.16-2.x86_64'
ERROR: can't read gnu_debugaltlink

There is a workaround, though, according to the Fedora docs (https://src.fedoraproject.org/rpms/redhat-rpm-config/blob/rawhide/f/buildflags.md): adding "%undefine _find_debuginfo_dwz_opts" to the spec file for the interested package, so disabling the use of dwz... but it would be better teach abi-dumper to deal with the presence of this program.

Hello and thanks a lot for developing this great set of tools!

I am using this to check a C library that exposes a large number of symbols in its public interface.
Some of these symbols are struct definitions that are not directly used in any of the public functions.

For example, in one of my header files I have the following:

#define SIZE 16

struct data {
    int data[SIZE];
    char flag;
};

int set_value(int type, void *o);

The type of void *o depends on the value of the first argument type. The struct data is one of the possible argument types and it will be casted inside the function.

The problem is that non-backward compatible changes to struct data are not detected and flagged as error by the tool. In fact, it looks like struct data is not even part of the dump.
So far I tried to add the options -all -dump-static to the abi-dumper as well as -ext to the abi-compliance-checker. However these do not seem to work.

The only way to detect these changes is to add a dummy function that explicitly takes an argument of type struct data.

int foo(struct data *o);

This is not very friendly and convenient to use. Is there any other option that I should try?

Thanks!

for some binaries,
Ubuntu 18.04 (WSL2) gives output:

	'Memb' => {
		'0' => {
			'name' => 'CONST_NAME1',
			'value' => '-1'
		},
		'1' => {
			'name' => 'CONST_NAME2,
			'value' => '1073741824'
		},
	...
	}

Ubuntu 20.04 (native) on the other hand, gives:

	'Memb' => {
		'0' => {
			'name' => 'CONST_NAME1',
			'value' => '18446744073709551615 (-1)'  # <<<--- dublicated values. "Correct one" is in brackets?
		},
		'1' => {
			'name' => 'CONST_NAME2,
			'value' => '1073741824 (1073741824)' # <<<--- dublicated values. "Correct one" is in brackets?
		},
	...
	}

I've got the following trying to dump a trivial static library:

Unescaped left brace in regex is deprecated here (and will be fatal in Perl 5.30), passed through in regex; marked by <-- HERE in m/(\w){ <-- HERE / at /usr/bin/abi-dumper line 3200.
ERROR: analysis of static libraries is not supported, please dump ABIs of individual objects in the archive or compile a shared library

My perl version is:

$ perl --version

This is perl 5, version 26, subversion 1 (v5.26.1) built for x86_64-linux
(with 27 registered patches, see perl -V for more detail)

Hi @lvc , I want know if abi-dumper can create abidump for windows dlls? this way is more easy than descriptor way when using ABI checker, so I would like to know the feasibility.

Thank you!

Can abi-dumper only dump the ABI for a list of header files which we are concerned about?
For example,
target_file:

a.h
b.h

cmd:

abi-dumper liba.so -header-list target_file

And then the output dump file only contains the ABI information in a.h and b.h.
How can I achieve this?

Hello,

I've tried abi-dumper on a project to see if it could be used to detect API/ABI issues. The project uses hierarchical headers and it looks like abi-dumper flatten the hierarchy, which seems problematic to detect header changes.

Suppose the project has the following hierarchy:

• include/
  • foo/a.h
  • foo/b.h
  • bar/a.h
  • bar/b.h

The dump will only list a.h and b.h as the headers when giving include/ as the public headers folder. Am I missing something?