CDT needs support for DWARF v5 about cdt

[jld01]

@jonahgraham I have added support for DW_FORM_line_strp in #242, building on your own work in #204. There is more work required to fully support DWARF v5 in CDT but ElfTest is now passing and the Binaries virtual resource tree is populated as expected in Project Explorer for native GCC under Ubuntu 22.04 LTS. In recognition of the joint effort, I suggest we do not squash this one when merging.

[jonahgraham]

Eek. Regression or something we failed to handle in new tool chain versions? Sounds like something that is worth doing an 11.0.1 for if we can fix it.

[jonahgraham]

This looks like Bug 580413 too.

[jld01]

@jonahgraham I can check against older Ubuntu but not until Friday at earliest. Yes, looks like Bug 580413.

[jonahgraham]

something we failed to handle in new tool chain versions?

In that case AFAICT it is new tool chain versions somewhere as I already checked that gcc/toolchain with Ubuntu 20.04 works fine (see comment 2 specifically)

[jonahgraham]

Use the toolchain test builds to install newer GCC on Ubuntu 20.04, here is a short blog post to help https://lindevs.com/install-gcc-on-ubuntu. However using just that gcc-11 (gcc 11.1.0) was not sufficient to trigger the error, so it is either gcc 11.2 or I need binutils. I can trigger the error with the attachment in Bug 580413.

[jonahgraham]

dwarfdump falls over too:

$ dwarfdump nota_media 

.debug_info

dwarfdump ERROR:  dwarf_srcfiles:  DW_DLE_STRP_OFFSET_BAD(204). Attempting to continue.

CU Name = <unknown>
CU Producer = GNU C17 11.2.0 -mtune=generic -march=x86-64 -g -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection
DIE OFF = 0x0000000c GOFF = 0x0000000c, Low PC = unknown   , High PC = unknown   

COMPILE_UNIT<header overall offset = 0x00000000>:
< 0><0x0000000c>  DW_TAG_compile_unit
                    DW_AT_producer              GNU C17 11.2.0 -mtune=generic -march=x86-64 -g -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection
                    DW_AT_language              DW_LANG_C11
                    
dwarfdump ERROR:  Cannot get a formstr (or a formstrp)....:  DW_DLE_STRP_OFFSET_BAD(204)

CU Name = <unknown>
CU Producer = GNU C17 11.2.0 -mtune=generic -march=x86-64 -g -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection
DIE OFF = 0x0000000c GOFF = 0x0000000c, Low PC = unknown   , High PC = unknown   

[jonahgraham]

The issue is dwarf5, the default on GCC 11+:

To take full advantage of DWARF version 5 GCC needs to be built against binutils version 2.35.2 or higher. When GCC is built against earlier versions of binutils GCC will still emit DWARF version 5 for most debuginfo data, but will generate version 4 debug line tables (even when explicitly given -gdwarf-5).

[jonahgraham]

We are misreading the debug_info header because the layout of it changed in v5, here is one extract:

DWARF Version 5 is compatible with DWARF Version 4 except as follows:

• The compilation unit header (in the .debug_info section) has a new
  unit_type field. In addition, the debug_abbrev_offset and address_size
  fields are reordered

[jonahgraham]

I think supporting Dwarf5 is a non-trivial effort and I have added the workaround (use Dwarf4) to the release notes for CDT11 in e874e9b

[jonahgraham]

"The line number table header is substantially revised."

This is the next part that needs resolving. org.eclipse.cdt.utils.debug.dwarf.DwarfReader.parseSourceInCULineInfo(String, int) parse the header, that needs to be reworked.

[jonahgraham]

With the "simple-dwarf5.elf" in PR #204 added to a project, expanding the elf file in the project explorer leads to this stack trace:

full stack trace

!ENTRY org.eclipse.ui.navigator 4 0 2022-12-09 15:14:13.468
!MESSAGE An exception occurred invoking extension: org.eclipse.cdt.ui.navigator.content for object simple-dwarf5.elf
!STACK 0
java.lang.IndexOutOfBoundsException: Index 9 out of bounds for length 1
	at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
	at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
	at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266)
	at java.base/java.util.Objects.checkIndex(Objects.java:359)
	at java.base/java.util.ArrayList.get(ArrayList.java:427)
	at org.eclipse.cdt.utils.debug.dwarf.DwarfReader.parseSourceInCULineInfo(DwarfReader.java:377)
	at org.eclipse.cdt.utils.debug.dwarf.DwarfReader.processCompileUnit(DwarfReader.java:647)
	at org.eclipse.cdt.utils.debug.dwarf.DwarfReader.processDebugInfoEntry(DwarfReader.java:606)
	at org.eclipse.cdt.utils.debug.dwarf.Dwarf.parseDebugInfoEntry(Dwarf.java:610)
	at org.eclipse.cdt.utils.debug.dwarf.Dwarf.parseDebugInfo(Dwarf.java:496)
	at org.eclipse.cdt.utils.debug.dwarf.Dwarf.parse(Dwarf.java:449)
	at org.eclipse.cdt.utils.debug.dwarf.DwarfReader.getSourceFilesFromDebugInfoSection(DwarfReader.java:564)
	at org.eclipse.cdt.utils.debug.dwarf.DwarfReader.getSourceFiles(DwarfReader.java:542)
	at org.eclipse.cdt.internal.core.model.Binary.addSourceFiles(Binary.java:314)
	at org.eclipse.cdt.internal.core.model.Binary.computeChildren(Binary.java:284)
	at org.eclipse.cdt.internal.core.model.Binary.buildStructure(Binary.java:270)
	at org.eclipse.cdt.internal.core.model.Openable.generateInfos(Openable.java:264)
	at org.eclipse.cdt.internal.core.model.CElement.openWhenClosed(CElement.java:428)
	at org.eclipse.cdt.internal.core.model.CElement.getElementInfo(CElement.java:306)
	at org.eclipse.cdt.internal.core.model.CElement.getElementInfo(CElement.java:296)
	at org.eclipse.cdt.internal.core.model.Parent.getChildren(Parent.java:57)
	at org.eclipse.cdt.internal.ui.BaseCElementContentProvider.getChildren(BaseCElementContentProvider.java:282)
	at org.eclipse.cdt.internal.ui.cview.CViewContentProvider.getChildren(CViewContentProvider.java:92)
	at org.eclipse.cdt.internal.ui.navigator.CNavigatorContentProvider.getChildren(CNavigatorContentProvider.java:233)
	at org.eclipse.ui.internal.navigator.extensions.SafeDelegateTreeContentProvider.getChildren(SafeDelegateTreeContentProvider.java:98)
	at org.eclipse.ui.internal.navigator.extensions.SafeDelegateTreeContentProvider.getChildren(SafeDelegateTreeContentProvider.java:241)
	at org.eclipse.ui.internal.navigator.extensions.SafeDelegateTreeContentProvider.getChildren(SafeDelegateTreeContentProvider.java:96)
	at org.eclipse.ui.internal.navigator.NavigatorContentServiceContentProvider$1.run(NavigatorContentServiceContentProvider.java:160)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.ui.internal.navigator.NavigatorContentServiceContentProvider.internalGetChildren(NavigatorContentServiceContentProvider.java:146)
	at org.eclipse.ui.internal.navigator.NavigatorContentServiceContentProvider.getChildren(NavigatorContentServiceContentProvider.java:132)
	at org.eclipse.jface.viewers.AbstractTreeViewer.getRawChildren(AbstractTreeViewer.java:1438)
	at org.eclipse.jface.viewers.TreeViewer.getRawChildren(TreeViewer.java:350)
	at org.eclipse.jface.viewers.StructuredViewer.getFilteredChildren(StructuredViewer.java:850)
	at org.eclipse.jface.viewers.AbstractTreeViewer.getSortedChildren(AbstractTreeViewer.java:638)
	at org.eclipse.jface.viewers.AbstractTreeViewer.createChildren(AbstractTreeViewer.java:837)
	at org.eclipse.jface.viewers.TreeViewer.createChildren(TreeViewer.java:611)
	at org.eclipse.jface.viewers.AbstractTreeViewer.createChildren(AbstractTreeViewer.java:790)
	at org.eclipse.jface.viewers.AbstractTreeViewer.handleTreeExpand(AbstractTreeViewer.java:1565)
	at org.eclipse.jface.viewers.TreeViewer.handleTreeExpand(TreeViewer.java:897)
	at org.eclipse.jface.viewers.AbstractTreeViewer$3.treeExpanded(AbstractTreeViewer.java:1577)
	at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:136)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
	at org.eclipse.swt.widgets.Display.sendEvent(Display.java:5855)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1529)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1555)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1538)
	at org.eclipse.swt.widgets.Tree.gtk_test_expand_row(Tree.java:2633)
	at org.eclipse.swt.widgets.Widget.windowProc(Widget.java:2536)
	at org.eclipse.swt.widgets.Display.windowProc(Display.java:6169)
	at org.eclipse.swt.internal.gtk3.GTK3.gtk_main_do_event(Native Method)
	at org.eclipse.swt.widgets.Display.eventProc(Display.java:1597)
	at org.eclipse.swt.internal.gtk3.GTK3.gtk_main_iteration_do(Native Method)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4514)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1155)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1046)
	at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:155)
	at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:643)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:550)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:171)
	at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:152)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:136)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:402)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:659)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:596)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1467)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1440)

[jld01]

"The line number table header is substantially revised."

This is the next part that needs resolving. org.eclipse.cdt.utils.debug.dwarf.DwarfReader.parseSourceInCULineInfo(String, int) parse the header, that needs to be reworked.

@jonahgraham we discussed this on the project call 2022-12-14. Are you intending to look at this next step in the short term? Do you have any insight on whether there may be any further steps required beyond this one? I may be able to take a look if your time is limited in the short term. It seems that much of the complexity in the above method is in determining where the source records begin.

[jonahgraham]

@jonahgraham we discussed this on the project call 2022-12-14. Are you intending to look at this next step in the short term?

No. I can consider revisiting in the new year.

Do you have any insight on whether there may be any further steps required beyond this one?

No. I just took the approach of creating a test in #204 and seeing where it fails.

I may be able to take a look if your time is limited in the short term.

Great.

It seems that much of the complexity in the above method is in determining where the source records begin.

I think so. The DWARF spec is very long and on a quick look I found where the spec for header was that I fixed in #204 already. But this other header I didn't immediately understand its layout.

[jld01]

@jonahgraham I found some time to look at this today. On Ubuntu 22.04 LTS:

$ dwarfdump -b core/org.eclipse.cdt.core.tests/resources/elf/unit_test/simple-dwarf5.elf 

.debug_abbrev
<    1><0x00000000><code:   1> DW_TAG_compile_unit         DW_children_yes
       <0x00000003>              DW_AT_producer              DW_FORM_strp
       <0x00000005>              DW_AT_language              DW_FORM_data1
       <0x00000007>              DW_AT_name                  DW_FORM_line_strp
       <0x00000009>              DW_AT_comp_dir              DW_FORM_line_strp
       <0x0000000b>              DW_AT_low_pc                DW_FORM_addr
       <0x0000000d>              DW_AT_high_pc               DW_FORM_data8
       <0x0000000f>              DW_AT_stmt_list             DW_FORM_sec_offset

One of the issues is lack of support for data format DW_FORM_line_strp in org.eclipse.cdt.utils.debug.dwarf.Dwarf.readAttribute(int, ByteBuffer, CompilationUnitHeader). The DWARF v5 specification suggests that the data is an offset to a string held in the .debug_line_str section. It appears we should look up these strings in an equivalent manner to DW_FORM_strp, but there is no section named .debug_line_str in simple-dwarf5.elf.

The data is definitely in there somewhere:

$ dwarfdump core/org.eclipse.cdt.core.tests/resources/elf/unit_test/simple-dwarf5.elf 

.debug_info

COMPILE_UNIT<header overall offset = 0x00000000>:
< 0><0x0000000c>  DW_TAG_compile_unit
                    DW_AT_producer              GNU C17 11.3.0 -mtune=generic -march=x86-64 -g -gdwarf-5 -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection
                    DW_AT_language              DW_LANG_C11
                    DW_AT_name                  simple.c
                    DW_AT_comp_dir              /work
                    DW_AT_low_pc                0x00001129
                    DW_AT_high_pc               <offset-from-lowpc> 38 <highpc: 0x0000114f>
                    DW_AT_stmt_list             0x00000000

I will continue to investigate as time permits.

[emrekultursay]

Considering that there is still work needed to get full DWARF v5 support (e.g., new operand forms for attribute values such as DW_FORM_strx that requires reading .debug_str_offsets section), shall we reopen this issue, or fork to a different issue to track that?

[jld01]

Considering that there is still work needed to get full DWARF v5 support (e.g., new operand forms for attribute values such as DW_FORM_strx that requires reading .debug_str_offsets section), shall we reopen this issue, or fork to a different issue to track that?

@emrekultursay, feel free to create a new issue for the additional work and reference this one.

[jld01]

Further work to support DWARF v5 at #443.

[jld01]

@emrekultursay do you have an example ELF file that uses DW_FORM_strx and is causing problems in CDT?