Multiple project folder error: Can't find file for formatting. about elm-language-client-vscode

[sentience]

I've spent a couple of hours learning about the Language Server Protocol, getting a debuggable copy of elm-language-client-vscode and elm-language-server running on my machine, and tracing source code for how the document formatting request is handled.

I haven't fully gotten my head around the layers of indirection, but here's what I've understood so far:

The Server at startup creates an ElmWorkspace for each elm.json file it finds in your project's directory tree. Each ElmWorkspace finds all *.elm files in the "source-directories" specified in the elm.json file, and follows the import statements they contain to also find all dependencies in elm-stuff and other directories.

It then registers each *.elm file, with a bunch of ”providers”, one for each major feature of the language server (e.g. the DocumentFormattingProvider for requests to format the document). I haven't yet found how the files get registered with the providers, but it seems to have something to do with TextDocumentEvents, which listens for "open" events (among others) for text documents.

When a request to format a document comes in, the request is supposed to be routed to an instance of DocumentFormattingProvider with a reference to a TextDocumentEvents object that knows about that specific document. In my debugging, all of my Elm files have their requests routed to a DocumentFormattingProvider that has an empty collection of documents! Why that is, I haven't yet been able to figure out.

I can see that on your WIP branch you're doing some rewrites of how the DocumentFormattingProvider is found when a document formatting request comes in. I don't yet understand the old mechanisms enough to see how your new approach differs, though.

One hunch I now have after all this debugging is that a quirk of my elm.json files may be responsible for most of the language server features not working for me: While a stereotypical elm.json file starts with "source-directories": [ "src" ] (Elm files in a src subdirectory), all of our elm.json files start with "source-directories": [ "." ] (Elm files in the root directory). When the Elm Language Server starts up, it reports all of our source files with /./ in the path (e.g. “Adding /Users/kyank/code/murmur/app/client/modules/ca-workflow-header/./WorkflowHeader/Decoders.elm”). And because many of our elm.json files refer to shared library directories whose paths begin with ../../, I see plenty of /../ in the reported source file paths, too (e.g. “Adding /Users/kyank/code/murmur/app/client/modules/ca-workflow-header/../../../../lib/client/modules/ca-ui/ProgressDialog.elm”).

Is it possible that all of our source files are getting registered in the server with these “non-normalized” paths, so that when the client later sends a request for a path like /Users/kyank/code/murmur/app/client/modules/ca-workflow-header/WorkflowHeader/Decoders.elm (without the /./), the server is unable to match it to any of its registered files? If so, might there be a possible fix of ensuring that the server uses path.resolve to resolve all of the source file paths to absolute, normalized paths before registering them?

[sentience]

As I suspect @razzeee may have predicted, this is a bigger job than I was expecting! Processing every client event in all detected workspaces, trusting that only one of them will actually do anything, is looking like it will generate too many potential problems.

This is more properly tracked as an issue of the language server (specifically, it looks like elm-tooling/elm-language-server#187 was already created for this). I'm going to start posting updates on my work there.

[razzeee]

I think I found and fixed the underlying problem (currently in a branch) but it also helped me to see some more problems, that are connected to multiple elmWorkspaces in one vscode workspace. So I will have to keep working on this some more.

[sentience]

We've just finally upgraded the last of our Elm 0.18 code here at Culture Amp, so we have a bunch of engineers eager to switch to this new editor extension. Unfortunately, in my testing, we're falling prey to issues like these that seem related to having more than one elm.json root in a single VSCode workspace (we have 27 of them).

Is there any way I can help, @razzeee?

[sentience]

OK, I think I have an idea of what's going on. Once each ElmWorkspace finds all of the *.elm files that it contains (and their dependencies), it records these in a “tree”. When the Server is told that all of the trees have been built, it calls registerInitializedProviders on each ElmWorkspace, which creates a set of providers. Each of those providers registers one or more event handlers on the connection, like this:

https://github.com/elm-tooling/elm-language-server/blob/88e5021e88b7861a9beefa6e5ecfdffeda401c64/src/providers/documentFormatingProvider.ts#L19

and this:

https://github.com/elm-tooling/elm-language-server/blob/88e5021e88b7861a9beefa6e5ecfdffeda401c64/src/providers/hoverProvider.ts#L20

The problem is, there's only one connection and multiple ElmWorkspace instances, so each call to onDocumentFormatting or onHover replaces any previously-registered handler with the new one. As a result, only the last ElmWorkspace's handlers actually remain registered!

I think the fix here will be to write a single onDocumentFormatting handler (and a single onHover handler, and so on) that calls the corresponding handlers for all of the active ElmWorkspaces (and only the one that recognises the file will handle it). Unless you have any better ideas, I'm happy to get started on a prototype of this!

[sentience]

Feel free to assign this to me. I'm planning to submit a fix in the next few days.

[razzeee]

Are both actual 0.19.0 projects?

[allanderek]

Yes, sorry, both are 0.19.0 projects.

[allanderek]

Okay that's great news. It's not an urgent show-stopper for me, I've just opened the two separate projects as two separate workspaces.

[julian-jelfs]

Same problem for me. The only reason I add a comment is that I similar behaviour when using the elm language server integration with nvim using coc. I was wondering whether that might point to a problem in the language server implementation itself rather than the vscode plugin?

[razzeee]

I'm not sure. As far as I understood, separating projects like this is how it's meant to be. CoC should (somehow) know that it needs to spin up multiple servers too.

If I got that wrong and we're meant to only have one server running, we need to redesign a bunch of things in the server. And I would have questions about the protocol, I previously was not able to resolve.

[razzeee]

But your case also is that each subdirectory has a elm.json and there is none at the root?

Feel free to look at the code. It seems the branch I talked about earlier is https://github.com/elm-tooling/elm-language-server/tree/improve-elm-workspaces but it has been quiet some time and I would likely start from scratch.

[razzeee]

You might also be able to get around this problem using vscode workspaces https://code.visualstudio.com/docs/editor/multi-root-workspaces

From a quick test it seems to work fine. But you will have to take a deeper look.

[sentience]

@razzeee That's right. No elm.json at the root of my VSCode workspace, but 27 of them nested in various subfolders. Perhaps importantly, some of our Elm files are in folders with no elm.json file in a parent directory, because they are shared modules that are imported by one (or more!) of our Elm applications that do have elm.json files.

For example:

|
|- apps
|  |- app1
|  |  |- elm.json (includes '../../lib' in its srcdirs)
|  |  `- App1.elm (imports Lib.elm)
|  `- app2
|     |- elm.json (includes '../../lib' in its srcdirs)
|     `- App2.elm (imports Lib.elm)
`- lib
   ` Lib.elm

Unfortunately breaking up our project into 27 separate workspaces (and getting all of our engineers to update their workspaces whenever we add a new one) would be somewhat impractical.

I'll take a look at the code and see if I have any ideas.

[razzeee]

I think workspaces are just files that can be checked in. So they should be shareable. Still a solution that works for non vscode folks is preferable.

…

On Wed, 25 Dec 2019, 01:49 Kevin Yank, ***@***.***> wrote: @razzeee <https://github.com/Razzeee> That's right. No elm.json at the root of my VSCode workspace, but 27 of them nested in various subfolders. Perhaps importantly, some of our Elm files are in folders with no elm.json file in a parent directory, because they are shared modules that are imported by one (or more!) of our Elm applications that do have elm.json files. For example: | |- apps | |- app1 | | |- elm.json (includes '../../lib' in its srcdirs) | | `- App1.elm (imports Lib.elm) | `- app2 | |- elm.json (includes '../../lib' in its srcdirs) | `- App2.elm (imports Lib.elm) `- lib ` Lib.elm Unfortunately breaking up our project into 27 separate workspaces (and getting all of our engineers to update their workspaces whenever we add a new one) would be somewhat impractical. I'll take a look at the code and see if I have any ideas. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#30?email_source=notifications&email_token=ABNLEZAD6MENBI4FSGKW5PTQ2KUYZA5CNFSM4IWPGLYKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHTW4IA#issuecomment-568815136>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNLEZGAS6DLATFLNLCUKDLQ2KUYZANCNFSM4IWPGLYA> .

[sentience]

I did some more tracing for hover requests (which are also not working in our multi-workspace project), and found that something similar is happening there: the hover request gets routed to a HoverProvider that contains a forest that does not include the file, and therefore the request is ignored.

I've experimented with hacking in a path.resolve(…) at the right place to register all files with normalized paths, but that hasn't made any difference to the behaviour for me.

[razzeee]

Yes, that's essentially, what my branch tried to do.

Nice job on understanding the code so far.

[sentience]

See elm-tooling/elm-language-server#187 if you want to test out an early version of a fix!

[razzeee]

Has been merged and released