eclipse-cdt / cdt-lsp Goto Github PK

I get lots of warnings which are written to stderr when a file is opened (only happens once per file after eclipse restart)

nested repeat operator '+' and '?' was replaced with '*' in regular expression /(using)(?:(?:\s)+)?(?!namespace)(\s*+((?:(?:(?:\[\[.*?\]\]|__attribute(?:__)?\s*\(\s*\(.*?\)\s*\))|__declspec\(.*?\))|alignas\(.*?\))(?!\)))?((?:((?:(?>(?:\s)+)|\/\*(?:[^\*]|(?:\*)++[^\/])*+(?:\*)++\/)+)|(?:\b)|(?=\W)|(?<=\W)|(?:\�)|(?:\Z)))(?:(?:(?:(?:unsigned)|(?:signed)|(?:short)|(?:long))|(?:(?:struct)|(?:class)|(?:union)|(?:enum)))((?:((?:(?>(?:\s)+)|\/\*(?:[^\*]|(?:\*)++[^\/])*+(?:\*)++\/)+)|(?:\b)|(?=\W)|(?<=\W)|(?:\�)|(?:\Z))))*((?:::)?(?:(?!\b(?:__has_cpp_attribute|reinterpret_cast|atomic_noexcept|atomic_commit|atomic_cancel|__has_include|synchronized|dynamic_cast|thread_local|static_cast|const_cast|co_return|constexpr|constexpr|constexpr|co_return|protected|namespace|consteval|noexcept|decltype|template|operator|noexcept|co_yield|co_await|reflexpr|continue|co_await|co_yield|requires|volatile|register|restrict|explicit|volatile|noexcept|typename|default|_Pragma|mutable|include|concept|alignas|virtual|alignof|__asm__|defined|mutable|typedef|warning|private|and_eq|define|pragma|typeid|switch|bitand|return|ifndef|export|struct|sizeof|module|static|public|extern|inline|friend|delete|xor_eq|import|not_eq|class|compl|bitor|throw|or_eq|while|catch|break|union|const|const|endif|ifdef|undef|error|using|else|line|goto|else|elif|this|enum|case|new|asm|not|try|for|and|xor|or|if|do|if)\b)(?<!\w)(?:[a-zA-Z_]|(?:\\u[0-9a-fA-F]{4}|\\U[0-9a-fA-F]{8}))(?:[a-zA-Z0-9_]|(?:\\u[0-9a-fA-F]{4}|\\U[0-9a-
...

The warnings originate from the joni library which parses the regular expressions. In org.eclipse.tm4e.core.internal.oniguruma.OnigRegExp, a regex is created like this

new Regex(pattern, 0, pattern.length, Option.CAPTURE_GROUP, UTF8Encoding.INSTANCE, Syntax.DEFAULT,
					WarnCallback.DEFAULT);

See https://github.com/jruby/joni/blob/master/src/org/joni/WarnCallback.java

public interface WarnCallback {
    WarnCallback DEFAULT = new WarnCallback() {
        public void warn(String message) {
            System.err.println(message);
        }
    };
    ...

The grammar has either to be fixed (in the https://github.com/jeff-hykin/better-cpp-syntax) or there has to be an option in https://github.com/eclipse/tm4e to disable the warnings being printed to stderr.

I am experiencing an issue with the C/C++ Editor (LSP). I have selected the C/C++ Editor (LSP) option in the Editor(LSP) Preferences. However, when I double click on a file, it does not open in the C/C++ Editor (LSP) as expected. Instead, it opens in the standard C/C++ Editor. However, if I choose the C/C++ Editor (LSP) from the menu list and click on it, then it opens in the C/C++ Editor (LSP).

Please correct me if I am wrong in my understanding of the C/C++ Editor (LSP) preference option.

Please find the screen recording for your reference.

E.g. C++ keywords in .h files are not highlighted.

One solution could be to always use the cpp grammar for header files by adding a content binding to org.eclipse.tm4e.registry.grammars

      <scopeNameContentTypeBinding
            contentTypeId="org.eclipse.cdt.core.cHeader"
            scopeName="cpp-grammar">
      </scopeNameContentTypeBinding>

After completion of #72 we need to repackage its test bundle accordingly

Let's also move 'org.eclipse.cdt.lsp.clangd.tests' from bundles to tests

When switching from a C/C++ Editor (LSP) tab to a C/C++ Editor (old) tab, the LS will be enabled for the file in this old editor as well.

This issue can be fixed when the LSP4E issue #393 has been fixed hopefully by PR #400

Test case would look like this:
GIVEN are several editor tabs opened in the C/C++ Editor as well as in the C/C++ Editor (LSP)
WHEN switching from an C/C++ Editor (LSP) editor tab to an C/C++ Editor tab with an external file
THEN the LS won't be enabled for the external file in the C/C++ Editor tab

Remove/replace the occurances of the old bundle path org.eclipse.cdt.lsp.editor.ui int the new org.eclipse.cdt.lsp.clangd bundle:

After completing #72 we need to remove empty org.eclipse.cdt.lsp.editor.ui bundle

Why do we create an additional editor type instead of just using generic editor?

https://github.com/Bachmann-electronic-GmbH/eclipse-cdt-lsp/blob/b3e75d76454aa89d2925a23bf43532afaf210eec/bundles/org.eclipse.cdt.lsp/plugin.xml#L6

The entry I added to the readme incorrectly stated to open with the generic editor:

https://github.com/Bachmann-electronic-GmbH/eclipse-cdt-lsp#open-a-file

Use case: A vendor wants to customize the command which is executed to start clangd.

There is an extension point to add a ICLanguageServerProvider, but in order to still be able to use the preferences page, CdtCLanguageServerProvider would have to be public API.

We need to publish sources as well, currently this is not the case.

When expanding a translation unit in the CDT Project Explorer the IScannerInfo provides information about the functions and included files:

These information should not be provided by the CDT internal build AST for a project which prefers the LSP based C/C++ Editor. The language server should provide these informations.

See discussion on #90 (comment) which applies to this code:

We need to add manifests to aggregate CDT LSP IUs and produce p2 repository.

Currently we have one composite repository defined right in the parent pom like

 <!-- this should probably be replaced by target files -->
  <repositories>
	<repository>
	  <id>eclipse.core</id>
	  <layout>p2</layout>
	  <url>https://download.eclipse.org/releases/2022-06/</url>
	</repository>
  </repositories>

However, a repository with eclipse bundles should have its dedicated target definition with explicitly listed required IUs

When the compile_commands.json changes and the C/C Editor (LSP) editor(s) has/have been closed until the refreshEditor has been scheduled after 5 seconds, a NPE occurs in org.eclipse.cdt.lsp.editor.ui.clangd.CompileCommandsMonitor:

!ENTRY org.eclipse.ui.workbench 4 0 2023-03-09 08:40:33.280
!MESSAGE An internal error has occurred.
!STACK 0
java.lang.NullPointerException: Cannot invoke "org.eclipse.jface.text.IDocument.getLineInformation(int)" because the return value of "org.eclipse.jface.text.TextViewer.getDocument()" is null
	at org.eclipse.jface.text.TextViewer.getBottomIndexEndOffset(TextViewer.java:3155)
	at org.eclipse.cdt.lsp.editor.ui.clangd.CompileCommandsMonitor.lambda$0(CompileCommandsMonitor.java:137)
	at org.eclipse.ui.progress.UIJob$2.runInUIThread(UIJob.java:76)
	at org.eclipse.ui.progress.UIJob.lambda$0(UIJob.java:148)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:132)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4029)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3645)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1155)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1046)
	at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:155)
	at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:643)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:550)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:171)
	at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:152)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:136)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:402)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:659)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:596)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1467)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1440)

Currently preferences are located in org.eclipse.cdt.lsp.clangd UI part, while some of them could be

• required from general headless part to decide if we need to launch LSP at all.
• required from clangd headless part to decide how we should launch (see #81 )

So , the preferences should be:

• split to have general and 'clangd' items
• reworked from UI to headless manner

In order to publish CDT LSP pre-release we need to complete the following tasks

• #57
• #59
• #66
• #100

The receiver that arrives in the property tester is sometimes just a simple Object instance.

I think that eclipse/lsp4e#393 is for the same issue, but with the current LSP4E snapshot build I made this change to unblock me. Things were working in e44c41f before the UI stuff went in.

$ git diff
diff --git a/org.eclipse.cdt.lsp/src/org/eclipse/cdt/lsp/server/HasLanguageServerPropertyTester.java b/org.eclipse.cdt.lsp/src/org/eclipse/cdt/lsp/server/HasLanguageServerPropertyTester.java
index 152dd7b..8081d47 100644
--- a/org.eclipse.cdt.lsp/src/org/eclipse/cdt/lsp/server/HasLanguageServerPropertyTester.java
+++ b/org.eclipse.cdt.lsp/src/org/eclipse/cdt/lsp/server/HasLanguageServerPropertyTester.java
@@ -24,9 +24,6 @@ public class HasLanguageServerPropertyTester extends PropertyTester {
 
        @Override
        public boolean test(Object receiver, String property, Object[] args, Object expectedValue) {
-               if (cLanguageServerProvider != null) {
-                       return cLanguageServerProvider.isEnabled(receiver);
-               }
                return true;
        }

So for now I raise this to track best workaround until LSP4E is resolved. (Is best workaround to use eclipse/lsp4e#400?)

The static log methods provided in org.eclipse.cdt.lsp.LspPlugin and org.eclipse.cdt.lsp.editor.ui.LspEditorUiPlugin can throw NPE when the plugin has been stopped.

@ruspl-afed mentioned as solution here #25 (comment)

Currently ClangdFallbackManager needs to access InitialFileManager and other services that makes code more and more statefull

We can use OSGi components to allow Platform resolve initialization order and other things for us.

When opening a *.hpp file, the C/C++ Editor (LSP) won't be used, because content types are missing in the org.eclipse.ui.editors extension point. The org.eclipse.tm4e.language_pack plug-in defines content types for C/C++ files as well ("lng.cpp" and "lng.c")

The content types "lng.cpp" and "lng.c" are associated to the ExtensionBasedTextEditor only.

Unfortunately is the extension type returned for a *.hpp file by org.eclipse.core.internal.resources.ContentDescriptionManager.getDescriptionFor(File, ResourceInfo, boolean) "lng.cpp" and NOT "org.eclipse.cdt.core.cxxHeader".

Because the "lng.cpp" extension type is only associated to the ExtensionBasedTextEditor, the editor registry only returns this editor for the "lng.cpp" content type.

This results in the need to associate the "lng.cpp" and "lng.c" to the C/C++ Editor and C/C++ Editor (LSP). By doing this, we are able to override the default editor in CEditorAssociationOverride for a *.hpp file with either the C/C++ Editor or C/C++ Editor (LSP)

As already mentioned here, the DefaultCLanguageServerProvider should be moved to the org.eclipse.cdt.lsp.clangd bundle, because it is the base implementation for the clangd language server. It should be renamed into BaseClangdLanguageServerProvider and abstract, because vendors should be able to extend this class according to their purposes.

When clangd cannot find a compile_commands.json the fallback values are used.

Solution proposal:
I assume that in most cases this external file has been opened by a hyperlink in the editor. During startup, the LS will be started, when the active editor is the LSP based C/C++ editor.
If the IDE keeps track of the origin file, the LS has to be started with this origin file, which must be a file in the current workspace. If the origin file is not a workspace file then get the origin of this file and so on until a project file (with a compile_commands.json) has been reached. When the LS server is started with the origin file, clangd will inherit the compile settings from the first opened file for the external file in focus.

The LSP4E based outline view does not work for files outside if the project:

The preferenceStore field in the LspEditorUiPlugin is not needed anymore since #117

Since the org.eclipse.cdt.lsp.server.enable.HasLanguageServerPropertyTester (which is used to enable the LSP based C/C++ editor and language server) relies on the isEnabledFor(IProject) from the ICLanguageServerProvider the enable expression and the enabledWhen element in the serverProvider schema is obsolet.

Currently the compile settings (examined from the found compile_commands.json) from the first opened project file which triggers the LS will be used workspace wide to examine (e.g. the C/C++ version to use) the included external (header) files.

Instead, the project specific compile settings should be used to examine external (#include) files included in a project file (e.g. a C++20 system header is included in a project where the C++20 standard has been defined). Another project in the workspace could use a different standard and system library.

Background info: The clangd LS searches for a compile_commands.json file in the source path and the parent folders. If no compile_commands.json could be determined for a file to be opened (this is the case for external files), clangd's fallback strategy is to use the compile_commands.json file from the first opened file.

This leads to new requirement: Run a language server instance per project.

The old cdt editor provides auto completion for (doxygen) multi line commands (typing e.g. /** and then pressing enter) and also for single line commands with doxygen (if enabled in the preferences).

Visual studio code does something similar by automatically adding */ after typing /*.

We can easily add auto-completion as in visual studio code, by modifying the language-configuration.json autoClosingPairs, e.g.:

	"autoClosingPairs": [
		{ "open": "[", "close": "]" },
		{ "open": "{", "close": "}" },
		{ "open": "(", "close": ")" },
		{ "open": "'", "close": "'", "notIn": ["string", "comment"] },
		{ "open": "\"", "close": "\"", "notIn": ["string"] },
		{ "open": "/*", "close": " */", "notIn": ["string"] }
	],

We can also use the org.eclipse.ui.genericeditor.autoEditStrategies extension point and add auto strategies, which trigger after pressing enter in the editor. We could even reuse the strategies from CDT, however as far as I understand, the cdt editor has some logic to select the strategies depending on the type of the current partition, so I'm not sure how we could glue this together.

What are your thoughts on that, especially @ruspl-afed as some of the doxygen code is written by you.

We need a feature to aggregate CDT LSP bundles. This feature then could be added to p2 repository (will follow soon)

When a C/C++ file has been opened and closed with the old C/C++ editor, the markers from the previous opening remain :

They should be deleted when opened by the new editor.

Only a closing of the project will remove these markers.

@jonahgraham has shown on Linux OS that several timeout errors occur on bigger C/C++ projects using the new LSP based C/C++ editor. The cause must be further investigated.

One proposal is to increase the number of async workers used by clangd by adding the -j <uint> option and to increase the background index priority by --background-index-priority=<value>, options are =background, =low, =normal.

If the issue can be localized or improved on LSP4E we should open an issue there.

A Language Server configuration file (e.g. .clangd) should be updated, when the projects build configuration has been changed.

See also this comment

Some toolchains will require the use of different clangds, or specific arguemnts to clangd, tohandle custom language extensions. There should be a way for the clangd & arguments used to be configured on a per-project basis

I noticed that the C/C++ Editor(LSP) icon is not transparent. I've attached a screenshot for your reference.

This is a minor issue, but I wanted to log it so that we don't forget to fix it.

Environment: macOS

Thanks to #46 it is possible to set breakpoints in the C/C++ Editor (LSP). When using the LSP editor no debug hovers are shown e.g. when moving over a variable. Instead the information from the language server gets displayed:
Current behavior:

Expected behavior:

Now we have bundle org.eclipse.cdt.lsp.clangd for clangd-related code

Let's move classes from org.eclipse.cdt.lsp.editor.ui there

Issues for moving tests and removing org.eclipse.cdt.lsp.editor.ui will follow.

When all files, which have been opened in the LSP based C/C++ editor, are closed, the LSP server shall be terminated. This won't happen if any document is still present in the file buffer manager. This happens if the children of a compilation unit are calculated while this unit is not yet opened in an editor:

Steps to reproduce:

1. Open Eclipse CDT with LSP plugin
2. Select project with enabled Prefer C/C++ Editor (LSP) property.
3. Show children of a compilation unit in that project in Project Explorer tree view (see screenshot above)
4. Open compilation unit in LSP based C/C++ Editor.
5. Close editor
6. The LS server won't be terminated after 5s

After change #117 the Pretty-print JSON output setting under Editor (LSP) preference settings are not saved persistent. The changes are missing after reopen the settings or restart of the workbench.

Steps to reproduce:

1. Enable Pretty-print JSON output under prefs or project properties: C/C++ > Editor (LSP)
2. Press Apply or Apply and Close
3. Reopen settings or restart IDE
4. Pretty-print JSON output is not enabled

When starting the IDE and changing the project properties of a cmake project in CDT it comes to an NPE in ICProjectDescriptionListener.handleEvent:

!ENTRY org.eclipse.jface 4 0 2023-05-09 09:56:08.059
!MESSAGE java.lang.NullPointerException: Cannot invoke "org.eclipse.core.runtime.IPath.toOSString()" because "cwdBuilder" is null
!STACK 0
java.lang.NullPointerException: Cannot invoke "org.eclipse.core.runtime.IPath.toOSString()" because "cwdBuilder" is null
	at org.eclipse.cdt.lsp.editor.ui.clangd.CProjectChangeMonitor$1.handleEvent(CProjectChangeMonitor.java:46)

Since the bundle org.eclipse.cdt.lsp.editor.ui has been replaced by org.eclipse.cdt.lsp.clangd there are several old class with LspEditor... names. They should be renamed to Clangd...

The plugin id should be adjusted as well: org.eclipse.cdt.lsp.editor.ui -> 'org.eclipse.cdt.lsp.clangd

The name of the LSP4E Console is currently "org.eclipse.lsp4e". This is a technical name that is not very user-friendly. It would be better to change the name to something like "Eclipse LSP Console". This would make it easier for users to identify the console and know what it is used for.

When using the gcc/g++ compiler from MSYS2 in a CMake project, the clangd fallbackFlags in the initilaizingOptions of the initialize jsonprc method cannot be correctly determined. The reason is, that the process to fetch the compiler builtins fails (see this CDT issue 407)

Steps to reproduce:

1. Windows 10 and MSYS2 with gcc and cmake installed.
2. Open Eclipse CDT with no opened editors (to prevent that the LS gets started)
3. Create a new Cmake project
4. Enable Prefer C/C++ Editor (LSP) in the project properties under C/C++ General > Editor (LSP)
5. Build it
6. Open a source file
7. In the org.eclipse.lsp4e console you should find an entry similar to:

"initializationOptions":{"fallbackFlags":["-isystemC:/TR/workspaces/runtime-cdt-development/Cmk1/build/default"]}

The system builtin are missing in the fallbackFlags array.

For clang it works. When using the clang compiler by adding:

set(CMAKE_C_COMPILER "clang")
set(CMAKE_CXX_COMPILER "clang++")

to the CMakeLists.txt

The result is

"initializationOptions":{"fallbackFlags":["-isystemC:/TR/workspaces/runtime-cdt-development/CmkOldGcc/build/default","-isystemC:/msys2/msys64/mingw64/include/c++/12.2.0","-isystemC:/msys2/msys64/mingw64/include/c++/12.2.0/x86_64-w64-mingw32"...

When the compile_commands.json changes due to modified compile settings, all affected language servers shall be restarted or notified that the compile_commands.json has changed.

I am currently setting up the project for the first time and I have noticed that org.eclipse.cdt.lsp.target.target have the latest version of Eclipse 4.27. However, it appears that we are using CDT 10.7, which is an older version, and orbit build is also outdated.

It's worth updating to the latest CDT 11.1 and eclipse orbit R20230302014618

https://download.eclipse.org/tools/orbit/downloads/drops/R20230302014618/

After PR #117 the query driver option is erroneous:

argv[4]: --query-driver==C:/msys2/msys64/mingw64/bin/*

There are two=

I'll provide a PR

Implement basic support of line breakpoints in the LSP bases C/C++ Editor.

Perspectively we should consider to switch to the DAP.

Provide an action to switch between source and header when a source/header file is opened in the editor. This can be done by using the clangd extension point https://clangd.llvm.org/extensions#switch-between-sourceheader.

When an editor gets opened by a double-click event on a symbol or include file of a C/C++ translation unit in the Project Explorer view:

then the old C/C++ Editor will be opened

Fixed by cdt issue 341

During discussion #56 regarding CDT LSP structure one of the common points was to split between "general" and "clangd" part

Let's introduce the bundle org.eclipse.cdt.lsp.clangd and then gradually move the clangd-specific content there.
Finally, org.eclipse.cdt.lsp.editor.ui bundle will became empty and we can remove it,

The org.eclipse.cdt.lsp.clangd bundle package structure should have clear separation between "headless" and "ui" part:
for "ui" part let's use org.eclipse.cdt.lsp.clangd.editor package and its children. The same prefix org.eclipse.cdt.lsp.clangd.editor should be used for all the UI contributions. So, later we can split "headless" and "ui" part if required.

Let's try to automate 3rd party license check