GithubHelp home page GithubHelp logo

lexical-lsp / lexical Goto Github PK

View Code? Open in Web Editor NEW
788.0 20.0 77.0 3.2 MB

Lexical is a next-generation elixir language server

Elixir 91.06% Shell 0.49% Makefile 0.05% Erlang 8.24% Nix 0.12% Dockerfile 0.04%

lexical's Introduction

Discord GitHub Workflow Status (with event)

Lexical logo: Lexi the lynx

Lexical

Lexical is a next-generation language server for the Elixir programming language.




Features

  • Context aware code completion
  • As-you-type compilation
  • Advanced error highlighting
  • Code actions
  • Code Formatting
  • Go To Definition
  • Completely isolated build environment

What makes Lexical different?

There are a couple things that lexical does differently than other language servers. Let's look at what separates it from the pack.

Architecture

When lexical starts, it boots an erlang virtual machine that runs the language server and its code. It then boots a separate virtual machine that runs your project code and connects the two via distribution. This provides the following benefits:

  • None of lexical's dependencies will conflict with your project. This means that lexical can make use of dependencies to make developing in it easier without having to "vendor" them. It also means that you can use lexical to work on your project, even if lexical depends on your project.
  • Your project can depend on a different version of elixir and erlang than lexical itself. This means that lexical can make use of the latest versions of elixir and erlang while still supporting projects that run on older versions.
  • The build environment for your project is only aware of your project, which enables as-you-type compilation and error reporting.
  • In the future, there is a possibility of having the lexical vm instance control multiple projects

Ease of contribution

Lexical has been designed to be easy to contribute to and work on. It features:

  • A consistent data model that represents the Language Server Protocol and mix tasks to generate new Language Server models.
  • A clearly defined separation between the language server and project code
  • A set of utilities that deal with manipulating code
  • A set of unit tests and test cases that make testing new features easy.

Focus on developer productivity

Lexical is also built with an eye on increasing developer productivity, and approaches some common features a little bit differently. For example, Lexical's code completion is context aware, which means that if you type alias MyModule.| you will only receive completions for modules and not the names of functions in MyModule. This awareness will extend to other areas, which means:

  • You won't see completions for random functions and types in strings. In fact, when extended to string literals, Lexical will only show you completions if you're inside of an interpolation ("hello there #{na|}').
  • If you're inside of a struct reference (%StructModule.|), you will only see modules listed that define structs, or are the parents of modules that define structs.

Because of this focus, Lexical aims to deliver depth of features rather than breadth of them. We'll likely spend more time making sure each thing we add works and feels just right rather than adding a whole slew of features that people mostly won't use.

As you type compilation

Because your project is run in a separate virtual machine, we can compile the code that you're working on as you type. This means you see errors immediately rather than having to wait for a save. The result is you see and fix typos, warnings, unused variables and a whole host of errors when they occur, which makes your code better, faster.

Installation

Follow the Detailed Installation Instructions

mix package

Lexical will now be available in _build/dev/package/lexical

If you would like to change the output directory, you can do so with the --path option

mix package --path /path/to/lexical

Lexical will be available in /path/to/lexical.

Development

Lexical is intended to run on any version of Erlang 24+ and Elixir 1.13+. Before beginning development, you should install Erlang 24.3.4.12 and Elixir 1.13.4 and use those versions when you're building code.

You should also look at the complete compatibility matrix do see which versions are supported.

You're going to need a local instance in order to develop lexical, so follow the Detailed Installation Instructions first.

Then, install the git hooks with

mix hooks

These are pre-commit hooks that will check for correct formatting and run credo for you.

After this, you're ready to put together a pull request for Lexical!

Benchmarks

The remote_control project has a set of benchmarks that measure the speed of various internal functions and data structures. In order to use them, you first need to install git large file storage, and then run git pull. Benchmarks are stored in the benchmarks subdirectory, and can be run via

mix benchmark /benchmarks/<benchmark_file>.exs

Logging

When lexical starts up, it creates a .lexical directory in the root directory of a project. Inside that directory are two log files, lexical.log and project.log. The .lexical.log log file contains logging and OTP messages from the language server, while the project.log file contains logging and OTP messages from the project's node. While developing lexical, it's helpful to open up a terminal and tail both of these log files so you can see any errors and messages that lexical emits. To do that, run the following in a terminal while in the project's root directory:

tail -f .lexical/*.log

Note: These log files roll over when they reach 1 megabyte, so after a time, it will be necessary to re-run the above command.

Debugging

Lexical supports a debug shell, which will connect a remote shell to a currently-running language server process. To use it, cd into your lexical installation directory and run

./bin/debug_shell.sh <name of project>

For example, if I would like to run the debug server for a server running in your lexical project, run:

./bin/debug_shell.sh lexical

...and you will be connected to a remote IEx session inside my language server. This allows you to investigate processes, make changes to the running code, or run :observer.

While in the debugging shell, all the functions in Lexical.Server.IEx.Helpers are imported for you, and some common modules, like Lexical.Project and Lexical.Document are aliased.

You can also start the lexical server in interactive mode via ./bin/start_lexical.sh iex. Combining this with the helpers that are imported will allow you to run projects and do completions entirely in the shell.

Note: The helpers assume that all of your projects are in folders that are siblings with your lexical project.

Consider the example shell session:

./bin/start_lexical.sh iex
iex(1)> start_project :other
# the project in the ../other directory is started
compile_project(:other)
# the other project is compiled
iex(2)> complete :other, "defmo|"
[
  #Protocol.Types.Completion.Item<[
    detail: "",
    insert_text: "defmacro ${1:name}($2) do\n  $0\nend\n",
    insert_text_format: :snippet,
    kind: :class,
    label: "defmacro (Define a macro)",
    sort_text: "093_defmacro (Define a macro)"
  ]>,
  #Protocol.Types.Completion.Item<[
    detail: "",
    insert_text: "defmacrop ${1:name}($2) do\n  $0\nend\n",
    insert_text_format: :snippet,
    kind: :class,
    label: "defmacrop (Define a private macro)",
    sort_text: "094_defmacrop (Define a private macro)"
  ]>,
  #Protocol.Types.Completion.Item<[
    detail: "",
    insert_text: "defmodule ${1:module name} do\n  $0\nend\n",
    insert_text_format: :snippet,
    kind: :class,
    label: "defmodule (Define a module)",
    sort_text: "092_defmodule (Define a module)"
  ]>
]

The same kind of support is available when you run iex -S mix in the lexical directory, and is helpful for narrowing down issues without disturbing your editor flow.

Other resources

lexical's People

Contributors

03juan avatar bangalcat avatar blond11516 avatar dalugm avatar dimitarvp avatar distefam avatar edwardsmit avatar hauleth avatar jhwls avatar jollyjerr avatar jparise avatar kirillrogovoy avatar lukad avatar mdshamoon avatar mixwui avatar moosieus avatar nicd avatar philipgiuliani avatar reisub avatar scohen avatar scottming avatar sheldak avatar sleepful avatar soundmonster avatar viniciusmuller avatar wingyplus avatar wkirschbaum avatar x-ji avatar yerguden avatar zachallaun avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

lexical's Issues

Elixir mismatch breaks formatting

Due to the way load paths are calculated, if you have a version mismatch between the release version of mix and the project version of mix, formatting will fail with a BadFunctionError

Can't format the file in `server` app

When try to format this file lib/lexical/server/project/diagnostics.ex, we will encounter this error:

lexical: -32603: ** (ErlangError) Erlang error: {:exception, %Mix.Error{message: "Unknown dependency :proto given to :import_dep
s in the formatter configuration. The dependency is not listed in your mix.exs for environment :dev", mix: 1}, [{Mix, :raise, 2,
 [file: 'lib/mix.ex', line: 513]}, {Mix.Tasks.Format, :"-eval_deps_opts/1-fun-1-", 3, [file: 'lib/mix/tasks/format.ex', line: 36
4]}, {Enum, :"-reduce/3-lists^foldl/2-0-", 3, [file: 'lib/enum.ex', line: 2468]}, {Mix.Tasks.Format, :eval_deps_opts, 1, [file:
'lib/mix/tasks/format.ex', line: 363]}, {Mix.Tasks.Format, :"-eval_deps_and_subdirectories/4-fun-4-", 4, [file: 'lib/mix/tasks/f
ormat.ex', line: 310]}, {Mix.Tasks.Format, :eval_deps_and_subdirectories, 4, [file: 'lib/mix/tasks/format.ex', line: 308]}, {Mix
.Tasks.Format, :"-eval_subs_opts/3-fun-1-", 2, [file: 'lib/mix/tasks/format.ex', line: 387]}, {Enum, :"-flat_map_reduce/3-fun-1-
", 3, [file: 'lib/enum.ex', line: 1288]}]}
    (kernel 8.5.3) erpc.erl:702: :erpc.call/5
    (server 0.1.0) lib/lexical/server/provider/handlers/formatting.ex:12: Lexical.Server.Provider.Handlers.Formatting.handle/2
    (server 0.1.0) lib/lexical/server/provider/queue.ex:95: anonymous fn/2 in Lexical.Server.Provider.Queue.State.as_task/2
    (elixir 1.14.3) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2
    (elixir 1.14.3) lib/task/supervised.ex:34: Task.Supervised.reply/4
    (stdlib 4.2) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

The interesting thing is if you try to format the file in the remote_control app, it can be formatted. I guess that the app/remote_control/.formatter.exs rewrited the format config of common_protocol, but app/server/.formatter.exs not.

And this bug is introduced by #48, But I don't think #48 is doing it wrong because we've actually been unable to get deps_paths on the remote node:

iex(4)> project_path = Path.expand("apps/server")
"/Users/scottming/Code/lexical/apps/server"
iex(5)> project_uri = "file://#{project_path}"
"file:///Users/scottming/Code/lexical/apps/server"
iex(6)> project = Lexical.Project.new(project_uri)
%Lexical.Project{
  root_uri: "file:///Users/scottming/Code/lexical/apps/server",
  mix_exs_uri: "file:///Users/scottming/Code/lexical/apps/server/mix.exs",
  mix_project?: true,
  mix_env: nil,
  mix_target: nil,
  env_variables: %{}
}
iex(7)> {:ok, node} = start_project project
{:ok, #PID<0.389.0>}
iex(manager-64444@127.0.0.1)8> RemoteControl.call(project, Mix.Project, :deps_paths, [])
%{}                                                                                                                          

But we can get that in local

~/Code/lexical main โฏ cd apps/remote_control                                                          3m 23s erl 25.2.1 ๏ˆ™ 2.7.2

~/Code/lexical/apps/remote_control main โฏ iex -S mix                                                         erl 25.2.1 ๏ˆ™ 2.7.2
Erlang/OTP 25 [erts-13.1.4] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] [jit:ns]

Interactive Elixir (1.14.3) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Mix.Project.deps_paths
%{
  common: "/Users/scottming/Code/lexical/apps/common",
  common_protocol: "/Users/scottming/Code/lexical/apps/common_protocol",
  elixir_sense: "/Users/scottming/Code/lexical/deps/elixir_sense",
  nimble_parsec: "/Users/scottming/Code/lexical/deps/nimble_parsec",
  patch: "/Users/scottming/Code/lexical/deps/patch",
  path_glob: "/Users/scottming/Code/lexical/deps/path_glob"
}

Possible solutions

We should get the deps_paths in the remote node.

`String.|` will crash neovim lsp

I haven't quite figured out how this repository works, but I've found some bugs from a user's perspective; this is the one.

When I type String.|, neovim will crash, seems like the content format is not right.

image

After encountering the error, LSP cannot resume anymore.

image

This is my config of neovim:

local lspconfig = require("lspconfig")
local configs = require("lspconfig.configs")

local lexical = {
	cmd = { "/Users/scottming/Code/lexical/_build/prod/rel/lexical/start_lexical.sh" },
	filetypes = { "elixir", "eelixir", "heex", "surface" },
	settings = {},
}

local custom_attach = function(client)
	print("Lexical has started.")
end

if not configs.lexical then
	configs.lexical = {
		default_config = {
			cmd = lexical.cmd,
			filetypes = lexical.filetypes,
			settings = lexical.settings,
		},
	}
end

lspconfig.lexical.setup({
	--[[ capabilities = require("user.lsp.handlers").capabilities, ]]
	on_attach = custom_attach,
	root_dir = lspconfig.util.root_pattern("mix.exs", ".git") or vim.loop.os_homedir(),
})

-- log config
require("vim.lsp.log").set_format_func(vim.inspect)
vim.lsp.set_log_level("debug")

I use the nightly neovim(installed by brew install neovim --head), because nvim0.8.3 cannot start the lexical.

If you have the vs code config, I can test this in vs code.

Duplicated diagnostics / code actions

When a file is compiled via the as-you-type compiler and again during project compilation, code actions are duplicated.
I'm not sure if the diagnostic is being duplicated or the code action is being duplicated, but you see two identical code actions in the editor.

Protocol improvements

Some ideas for improvements in the protocol:

  • 1. Have requests and notifications point directly to their params structs rather than having to spell them out.
  • 2. Make it so requests know about and can generate their responses.
  • 3. Maybe use ecto schemas for validation of parameters.
  • 4. Remove the :exclusive keyword on defrequest
  • 5. Have responses convert their elixir positions automatically.

Documentation suggestions

It would be cool if we had the following suggestions:

@moduledoc suggests @moduledoc false or a snippet of

@moduledoc """
$0
"""

@doc suggests @doc false or a snippet of

@doc """
$0
"""

` Lexical.RemoteControl.Api.formatter_for_file` can not find the deps

17:05:43.382 [error] ** (ErlangError) Erlang error: {:exception, %Mix.Error{message: "Could not find an SCM for dependency :jason from Lexical.Server.MixProject", mix: 1}, [{Mix, :raise, 2, [file: 'lib/mix.ex', line: 513]}, {Mix.Dep.Loader, :with_scm_and_app, 4, [file: 'lib/mix/dep/loader.ex', line: 195]}, {Mix.Dep.Loader, :to_dep, 3, [file: 'lib/mix/dep/loader.ex', line: 141]}, {Enum, :"-map/2-lists^map/1-0-", 2, [file: 'lib/enum.ex', line: 1658]}, {Enum, :"-map/2-lists^map/1-0-", 2, [file: 'lib/enum.ex', line: 1658]}, {Mix.Dep.Loader, :mix_children, 2, [file: 'lib/mix/dep/loader.ex', line: 358]}, {Mix.Dep.Loader, :children, 0, [file: 'lib/mix/dep/loader.ex', line: 18]}, {Mix.Dep.Converger, :all, 4, [file: 'lib/mix/dep/converger.ex', line: 80]}]}
    (kernel 8.5.3) erpc.erl:702: :erpc.call/5
    (remote_control 0.1.0) lib/lexical/remote_control/api.ex:17: Lexical.RemoteControl.Api.formatter_for_file/2
    (server 0.1.0) lib/lexical/server/code_mod/format.ex:38: Lexical.Server.CodeMod.Format.formatter_for/2
    (server 0.1.0) lib/lexical/server/code_mod/format.ex:24: Lexical.Server.CodeMod.Format.do_format/2
    (server 0.1.0) lib/lexical/server/code_mod/format.ex:14: Lexical.Server.CodeMod.Format.text_edits/2
    (server 0.1.0) lib/lexical/server/provider/handlers/formatting.ex:12: Lexical.Server.Provider.Handlers.Formatting.handle/2
    (server 0.1.0) lib/lexical/server/provider/queue.ex:95: anonymous fn/2 in Lexical.Server.Provider.Queue.State.as_task/2
    (elixir 1.14.3) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2

You can reproduce this issue in the lexical server app.

Diagnostics persistence issues

I've noticed the following issues with diagnostics:

  1. Presently, they disappear when you save your file, and then reappear when you type. This is due to the fix that swapped lines 39 and 42 in diagnostics.ex. Swapping these lines back fixes this issue.
  2. as-you-type diagnostics aren't reliably cleared when you save. I get a lot of function Logger.in doesn't exist at a specific line, even though i've saved the file with the correct code of Logger.info.

Switch to one-based positions and ranges

When I started lexical, I thought that I could eliminate the conversion between elixir's positions (one-based everything) and the LSP's positions (zero-based everything) by setting compiler options. However this turned out to be a documentation issue; starting an elixir file on line 0 and column 0 is not possible consistently.

Knowing this, we should give up on this approach and instead work on handling these conversions at the protocol layer. There, when we read data, we'll add 1 to positions and on the way out, we'll subtract one.

The goal here is to not have character = pos.character + 1 in the user-land code.

Opening mix.exs displays diagnostic

diagnostic message:

"Trying to load Lexical.MixProject from \"/Users/jose/Code/lexical/mix.exs\" but another project with the same name was already defined at \"/Users/jose/Code/lexical/mix.exs\"

client:

 โ”‚   ๏—  Trying to load Lexical.MixProject from "/Users/jose/Code/lexical/mix.exs" but another project with the same name was already defined at "/Users/jose/Code/lexical/mix.exs"

project.log:

2023-03-20T03:58:31.975969-06:00 info: Emitting [%Mix.Task.Compiler.Diagnostic{file: "/Users/jose/Code/lexical/mix.exs", severity: :error, message: "Trying to load Lexical.MixProject from \"/Users/jose/Code/lexical/mix.exs\" but another project with the same name was already defined at \"/Users/jose/Code/lexical/mix.exs\"", position: 0, compiler_name: "Elixir", details: nil}]

`module undefined` warning diagnostic until the file is opened and compiled

defmodule A
	def hello(), do: IO.puts("hello)
end
defmodule B
	def say_hello(), do: A.hello() # diagnostic here
	# "Module A is not defined blabla"
end

This is because even if we compile the project, the modules are loaded dynamically, perhaps try to load a module when it isn't found before displaying diagnostic.

Formatting crashes sometimes when there are errors

formatting crashes because of errors, and then the project gets all wonky

symptoms include constant LSP timeout after the crash.

this is what it looks like in logs (using random ex project):

project.log

2023-04-06T14:42:35.512837-06:00 error: Supervisor: {local,'Elixir.LXRelease.RemoteControl.Supervisor'}. Context: child_terminated. Reason: {shutdown,1}. Offender: id='Elixir.LXRelease.RemoteControl.Build',pid=<0.972.0>.

lexical.log

14:42:30.925 [error] Formatter failed %BadArityError{function: &Code.format_string!/2, args: ["defmodule M.Application do\n  # See https://hexdocs.pm/elixir/Application.html\n  # for more information on OTP Applications\n  @moduledoc false\n\n  use Application\n\n  @impl true\n  def start(_type, _args) do\n    children = [\n      # Start the Telemetry supervisor\n      MWeb.Telemetry,\n      # Start the PubSub system\n      {Phoenix.PubSub, name: M.PubSub},\n      # Start Finch\n      {Finch, name: M.Finch},\n      # Start the Endpoint (http/https)\n      MWeb.Endpoint,\n      # Start a worker by calling: M.Worker.start_link(arg)\n      M.Data\n    ]\n\n    # See https://hexdocs.pm/elixir/Supervisor.html\n    # for other strategies and supported options\n    opts = [strategy: :one_for_one, name: M.Supervisor]\n    Supervisor.start_link(children, opts)\n  end\n\n  # Tell Phoenix to update the endpoint configuration\n  # whenever the application is updated.\n  @impl true\n  def config_change(changed, _new, removed) do\n    MWeb.Endpoint.config_change(changed, removed)\n    :ok\n  end\nend\n"]}

happened to me with file save.

Then it starts to do this:

14:46:25.925 [error] Process #PID<0.2436.0> terminating
** (exit) {:exception, {:shutdown, 1}}
    (kernel 8.5.3) erpc.erl:700: :erpc.call/5
    (server 0.1.0) lib/lexical/server/provider/handlers/formatting.ex:12: LXRelease.Server.Provider.Handlers.Formatting.handle/2
    (server 0.1.0) lib/lexical/server/provider/queue.ex:99: anonymous fn/2 in LXRelease.Server.Provider.Queue.State.as_task/2
    (elixir 1.14.3) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2
    (elixir 1.14.3) lib/task/supervised.ex:34: Task.Supervised.reply/4
    (stdlib 4.2) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

Port over existing tests

  • code_unit_test.exs
  • line_parser_test.exs
  • process_cache_test.exs
  • project_test.exs
  • diff_test.exs
  • formatting_test.exs
  • goto_definition_test.exs
  • find_references_test.exs
  • server/state_text.exs
  • source_file/conversions_test.exs

Module purging in build

there was another bug where snippets would insert code that was valid.
So if you add a module and type its name, say this is your snippet:

defmodule <module_name> do 
end

Then you make it called Stuff

defmodule Stuff do 
end

the as-you-type compiler would create a module named S, one named St one named, Stu and so on, i'm thinking about having the as-you-type compiler wait for a 250ms pause in typing. or some amount of time

solutions:

we might need to associate the modules with the files that created them
and then then delete any modules that a file created before we compile it

it's more than that
we need to associate file_uri -> [modules]
and when we compile a file, first delete all the modules that the file defined before we compile
the problem isn't with modules that are updated
it's with modules that are deleted

Project compilation produces no diagnostics

When lexical starts, it triggers a project-wide build. For some reason, this doesn't produce diagnostics any more.

For example, we use the :slave module, which always produces a deprecation warning, yet this is not emitted in the initial builds (it used to be)

Remote control needs to detect umbrella apps

When remote control starts, it needs to ensure that it is in the correct root for the project. If the project is an umbrella, it needs to make sure it's not in one of the subapps, and if it is, it needs to cd to the umbrella's root before it begins builds.

Time out (crash) for code-format when using `lexical` as project context

example, using method = "textDocument/formatting", in file lexical/apps/common/test/support/range.ex gives me a Timeout warning message on editor when /Users/jose/Code/lexical is used as root.

Client's log:

[DEBUG][2023-03-20 03:36:29] .../lua/vim/lsp.lua:1397	"LSP[lexical]"	"client.request"	1	"textDocument/formatting"	{  options = {    insertSpaces = true,    tabSize = 2  },  textDocument = {    uri = "file:///Users/jose/Code/lexical/apps/common/test/support/range.ex"  }}	<function 1>	161
[DEBUG][2023-03-20 03:36:29] .../vim/lsp/rpc.lua:284	"rpc.send"	{  id = 7,  jsonrpc = "2.0",  method = "textDocument/formatting",  params = {    options = {      insertSpaces = true,      tabSize = 2    },    textDocument = {      uri = "file:///Users/jose/Code/lexical/apps/common/test/support/range.ex"    }  }}
[DEBUG][2023-03-20 03:36:30] .../vim/lsp/rpc.lua:284	"rpc.send"	{  jsonrpc = "2.0",  method = "$/cancelRequest",  params = {    id = 7  }}

In a smaller project I can use textDocument/formatting without issue.

Test env doesn't seem to know about extra elixirc_paths

I noticed that when editing unit tests, a couple of the imports that were in test/support were flagged as not being found. It's likely because the code paths that bootstrap sets up don't take into account the test environment's elixirc_paths.

Feature: Code action - Organize aliases

This is actually a tough one. I tried implementing this in elixir-ls but gave up because it didn't have good enough source code editing tooling.

Initially, we should enforce the following:

Flatten nested aliases Foo.Bar{Baz, Quux}
Sort aliases alphabetically

Be aware though; Aliases can depend on one another

alias Lexical.RemoteControl
alias RemoteControl.Build 
alias Build.Server

This should result in:

alias Lexical.RemoteControl
alias Lexical.RemoteControl.Build
alias Lexical.RemoteControl.Build.Server

Similarly, nested aliases can have internal dependencies and external dependencies

Also, aliases can appear anywhere in a source file. Aliases inside function definitions cannot be pulled out of those definitions. Aliases that are present in an outer context are visible in an inner context. So

defmodule Outer do 
  alias Lexical.RemoteControl 

  defmodule Inner do 
    def my_fun do 
      RemoteControl.foobar()
   end
end

is valid.

Roadmap to v0.1

Just pinning @scohen thoughts somewhere more traceable.

I'd like to get all the bugs fixed before we move on to more features
Project wise , Iโ€™m going to put together a feature list. Then we work towards getting that done. Then we eliminate elixir sense
Maybe the rule is elixir sense only does completion
Right now, I want better completion. Go to definition, find references and dialyzer.

Remove Build locks?

I can't for the life of me remember why I added the build locks.
Builds presently go through the build genserver, which means that only one thing will progress at a time, so we shouldn't need locks.
I was noticing some stalling in the app yesterday, and disabled locks which cured the stalling and didn't seem to have any side effects.

Dialyzer Diagnostics

This is a big feature, but we should start small

Dialyzer creates files called persistent lookup tables or .plts. These tables are extremely specific to code. We should generate .plt files for

  1. The erlang release
  2. The elixir release
  3. Your application's code

I believe these tables can be updated via code, but we need to be extremely careful about when we do so and how we do so. They're effectively caches, and caches are evil.
This is also an area where elixir_ls doesn't do so well, and one of the reasons why deleting your .elixir-ls directory is the way to fix problems.

So to start small, we should create a dialyzer subdirectory in .elixir-ls and in that directory, store various .plt files. These files will need to be named after the elixir and erlang releases they index.

Then we need to create a plt for our app. This is going to be tough, since the code there changes so much. I'm wide open to ideas, but we should probably walk the module tree when we compile the first time and keep md5s of each module. Then, when a module is compiled, we need to write the md5 to the cache, and update that module in dialyzer.

Move off of `:slave` in remote control.

The erlang :slave module has the following drawbacks:

  1. It shares IO and file servers with the node that started it. This means the server's cwd is constantly changing
  2. It's started by the server node, which means the version of erlang is the same as the one on the server node. This means that projects that declare a different erlang version won't be running in the correct version
  3. It's deprecated
  4. It has a very regrettable name.

I'd suggest moving over to :peer, which fixes the first, third and fourth problems, but it still will have the second and it's new in erlang 25. I'd like to have at least two erlang versions of compatibility.

We should investigate starting up our own node and bootstrapping it. We already do this to some extent with remote_control so I wonder how hard it would be to launch a blank node and then add the libraries to it.

replace with underscore will change the function name if they are the same.

image

In principle, the third argument of an unused variable, when it converts to AST, should be nil.

image

And the parent of the function name would be the {:def, meta, _} block.

image

This should not be a very urgent bug, and it is not often that function names match variable names. I can take this. If you don't mind that I use Sourceror.Zipper to rewrite some code.

Improve completion of text leading `alias`, `%`, `&` ...

def print(s) do
  IO.puts(s)
end

Enum.map(["a", "b"], &print| should complete ``Enum.map(["a", "b"], &print/1)`

Currently, it gives:

[{"detail":"Dummy.print/1(s)","insertText":"print($0)","insertTextFormat":2,"kind":3,"label":"print/1","sortText":"print/1"}]

Can not start two LSP Clients that connect to Lexical at the same time

Repro:

Open your editor with lexical ON.
Open another instance of your editor with lexical ON.

Log from LSP Client:

[INFO][2023-03-20 03:12:09] .../vim/lsp/rpc.lua:662	"Starting RPC client"	{  args = {},  cmd = "/Users/jose/Code/lexical/_build/dev/rel/lexical/start_lexical.sh",  extra = {    cwd = "/Users/jose/Code/lexical"  }}
[DEBUG][2023-03-20 03:12:09] .../vim/lsp/rpc.lua:284	"rpc.send"	{  id = 1,  jsonrpc = "2.0",  method = "initialize",  params = {    capabilities = {      textDocument = {        callHierarchy = {          dynamicRegistration = false        },        codeAction = {          codeActionLiteralSupport = {            codeActionKind = {              valueSet = { "", "quickfix", "refactor", "refactor.extract", "refactor.inline", "refactor.rewrite", "source", "source.organizeImports" }            }          },          dataSupport = true,          dynamicRegistration = false,          isPreferredSupport = true,          resolveSupport = {            properties = { "edit" }          }        },        completion = {          completionItem = {            commitCharactersSupport = true,            deprecatedSupport = true,            documentationFormat = { "markdown", "plaintext" },            insertReplaceSupport = true,            insertTextModeSupport = {              valueSet = { 1, 2 }            },            labelDetailsSupport = true,            preselectSupport = true,            resolveSupport = {              properties = { "documentation", "detail", "additionalTextEdits" }            },            snippetSupport = true,            tagSupport = {              valueSet = { 1 }            }          },          completionItemKind = {            valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25 }          },          completionList = {            itemDefaults = { "commitCharacters", "editRange", "insertTextFormat", "insertTextMode", "data" }          },          contextSupport = true,          dynamicRegistration = false,          insertTextMode = 1        },        declaration = {          linkSupport = true        },        definition = {          linkSupport = true        },        documentHighlight = {          dynamicRegistration = false        },        documentSymbol = {          dynamicRegistration = false,          hierarchicalDocumentSymbolSupport = true,          symbolKind = {            valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26 }          }        },        hover = {          contentFormat = { "markdown", "plaintext" },          dynamicRegistration = false        },        implementation = {          linkSupport = true        },        publishDiagnostics = {          relatedInformation = true,          tagSupport = {            valueSet = { 1, 2 }          }        },        references = {          dynamicRegistration = false        },        rename = {          dynamicRegistration = false,          prepareSupport = true        },        semanticTokens = {          augmentsSyntaxTokens = true,          dynamicRegistration = false,          formats = { "relative" },          multilineTokenSupport = false,          overlappingTokenSupport = true,          requests = {            full = {              delta = true            },            range = false          },          serverCancelSupport = false,          tokenModifiers = { "declaration", "definition", "readonly", "static", "deprecated", "abstract", "async", "modification", "documentation", "defaultLibrary" },          tokenTypes = { "namespace", "type", "class", "enum", "interface", "struct", "typeParameter", "parameter", "variable", "property", "enumMember", "event", "function", "method", "macro", "keyword", "modifier", "comment", "string", "number", "regexp", "operator", "decorator" }        },        signatureHelp = {          dynamicRegistration = false,          signatureInformation = {            activeParameterSupport = true,            documentationFormat = { "markdown", "plaintext" },            parameterInformation = {              labelOffsetSupport = true            }          }        },        synchronization = {          didSave = true,          dynamicRegistration = false,          willSave = true,          willSaveWaitUntil = true        },        typeDefinition = {          linkSupport = true        }      },      window = {        showDocument = {          support = true        },        showMessage = {          messageActionItem = {            additionalPropertiesSupport = false          }        },        workDoneProgress = true      },      workspace = {        applyEdit = true,        configuration = true,        didChangeWatchedFiles = {          dynamicRegistration = false,          relativePatternSupport = true        },        semanticTokens = {          refreshSupport = true        },        symbol = {          dynamicRegistration = false,          hierarchicalWorkspaceSymbolSupport = true,          symbolKind = {            valueSet = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26 }          }        },        workspaceEdit = {          resourceOperations = { "rename", "create", "delete" }        },        workspaceFolders = true      }    },    clientInfo = {      name = "Neovim",      version = "0.9.0"    },    initializationOptions = vim.empty_dict(),    processId = 92588,    rootPath = "/Users/jose/Code/lexical",    rootUri = "file:///Users/jose/Code/lexical",    trace = "off",    workspaceFolders = { {        name = "/Users/jose/Code/lexical",        uri = "file:///Users/jose/Code/lexical"      } }  }}
[ERROR][2023-03-20 03:12:11] .../vim/lsp/rpc.lua:734	"rpc"	"/Users/jose/Code/lexical/_build/dev/rel/lexical/start_lexical.sh"	"stderr"	"Protocol 'inet_tcp': the name manager-1093"
[ERROR][2023-03-20 03:12:11] .../vim/lsp/rpc.lua:734	"rpc"	"/Users/jose/Code/lexical/_build/dev/rel/lexical/start_lexical.sh"	"stderr"	"[email protected] seems to be in use by another Erlang node\r\n"

Refactor completions.ex

Completions is fairly complex, ugly and centralized. This needs to change.

Goals:

  1. Make the code cleaner, place the responsibility for completing something into different modules
  2. Develop a plugin architecture so other libraries can implement completion plugins for lexical
  3. Dogfood that architecture

`NAMESPACE=1 mix release lexical --overwrite` won't work at the first time.

use rm -rf _build deps .lexical && mix deps.get && NAMESPACE=1 mix release lexical build the release.

Then open a project, write some incorrect code, the diagnostics won't show, and the completion also won't work, but if you use rm -rf _build deps .lexical && mix deps.get && mix release lexical && NAMESPACE=1 mix release lexical --overwrite to build the release, everything works well.

Server crash by code action

10:15:31.104 [error] Child Lexical.Server of Supervisor Lexical.Server.Supervisor terminated
** (exit) an exception was raised:
    ** (FunctionClauseError) no function clause matching in Lexical.Server.Provider.Queue.add/2
        (server 0.1.0) lib/lexical/server/provider/queue.ex:146: Lexical.Server.Provider.Queue.add(%Lexical.Protocol.Requests.CodeAction{lsp: %Lexical.Protocol.Requests.CodeAction.LSP{id: 917, jsonrpc: "2.0", method: "textDocument/codeAction", context: %Protocol.Types.CodeAction.Context{%{diagnostics: []}}, range: %Protocol.Types.Range{%{end: %Protocol.Types.Position{%{character: 55, line: 99}}, start: %Protocol.Types.Position{%{character: 55, line: 99}}}}, text_document: %Protocol.Types.TextDocument.Identifier{%{uri: "file:///Users/scottming/Code/lexi/test/lexi/tracer/builder_test.exs"}}}, source_file: nil, range: nil, id: 917, jsonrpc: "2.0", method: "textDocument/codeAction", context: nil, text_document: nil}, nil)
        (server 0.1.0) lib/lexical/server.ex:106: Lexical.Server.handle_message/2
        (server 0.1.0) lib/lexical/server.ex:46: Lexical.Server.handle_cast/2
        (stdlib 4.2) gen_server.erl:1123: :gen_server.try_dispatch/4
        (stdlib 4.2) gen_server.erl:1200: :gen_server.handle_msg/6
        (stdlib 4.2) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Pid: #PID<0.3264.0>

Project builds seem to be broken

On start, I'm not seeing diagnostics being emitted by lexical when the project compiles.
Individual file compilation works though

Can't format any files of any project.

image

This is the source message:

[DEBUG][2023-03-11 22:07:47] .../vim/lsp/rpc.lua:284	"rpc.send"	{
  id = 34,
  jsonrpc = "2.0",
  method = "textDocument/formatting",
  params = {
    options = {
      insertSpaces = true,
      tabSize = 2
    },
    textDocument = {
      uri = "file:///Users/scottming/Code/dummy/lib/dummy.ex"
    }
  }
}
[DEBUG][2023-03-11 22:07:47] .../vim/lsp/rpc.lua:387	"rpc.receive"	{
  error = {
    code = -32603,
    message = "** (ErlangError) Erlang error: {:exception, %Mix.Error{message: \"Trying to load Dummy.MixProject from \\\"/Users/scottming/Code/dummy/mix.exs\\\" but another project with the same name was already defined at \\\"/Users/scottming/Code/dummy/mix.exs\\\"\", mix: 1}, [{Mix, :raise, 2, [file: 'lib/mix.ex', line: 513]}, {Mix.Project, :load_project, 2, [file: 'lib/mix/project.ex', line: 785]}, {Mix.Project, :in_project, 4, [file: 'lib/mix/project.ex', line: 389]}, {File, :cd!, 2, [file: 'lib/file.ex', line: 1607]}, {Lexical.RemoteControl.Formatter, :for_file, 1, []}]}\n    (kernel 8.5.3) erpc.erl:702: :erpc.call/5\n    (remote_control 0.1.0) lib/lexical/remote_control/api.ex:17: Lexical.RemoteControl.Api.formatter_for_file/2\n    (server 0.1.0) lib/lexical/server/code_mod/format.ex:53: Lexical.Server.CodeMod.Format.formatter_for/2\n    (server 0.1.0) lib/lexical/server/code_mod/format.ex:31: Lexical.Server.CodeMod.Format.do_format/2\n    (server 0.1.0) lib/lexical/server/code_mod/format.ex:14: Lexical.Server.CodeMod.Format.text_edits/2\n    (server 0.1.0) lib/lexical/server/provider/handlers/formatting.ex:12: Lexical.Server.Provider.Handlers.Formatting.handle/2\n    (server 0.1.0) lib/lexical/server/provider/queue.ex:95: anonymous fn/2 in Lexical.Server.Provider.Queue.State.as_task/2\n    (elixir 1.14.3) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2\n"
  },
  id = 34
}

And I think we should use the message type ERROR instead of DEBUG when can not format file.

Enhance `undefined` function/variable error diagnostics

image

When a compilation error occurs indicating that a value does not exist, it can be more precise. There are three messages here, but none of them are accurate enough. It is not difficult to be precise about errors like this down to the column level and make it simpler and clearer.

{:error,
 [
   {"/Users/scottming/Code/lexical/apps/remote_control/test/lexical/remote_control/code_intelligence/definition_test.exs",
    240,
    "** (CompileError) test/lexical/remote_control/code_intelligence/definition_test.exs:240: undefined function referenced_uri/0 (expected Lexical.RemoteControl.CodeIntelligence.DefinitionTest to define such a function or for it to be imported, but no
 are available)\n\n"}
 ],
 [
   {"/Users/scottming/Code/lexical/apps/remote_control/test/lexical/remote_control/code_intelligence/definition_test.exs",
    238,
    "variable \"ctx\" is unused (if the variable is not meant to be used, prefix it with an underscore)"},
   {"/Users/scottming/Code/lexical/apps/remote_control/test/lexical/remote_control/code_intelligence/definition_test.exs",
    240,
    "variable \"referenced_uri\" does not exist and is being expanded to \"referenced_uri()\", please use parentheses to remove the ambiguity or change the variable name"}
 ]}

Errors that occur when code is formatted need to be emitted as diagnostics

Say you're editing this code

    parser_options = [path: source_file.path] ++ parser_options

    compile = fn ->
      result =
        case Code.string_to_quoted(SourceFile.to_string(), ) do
        end
   end

The comma in the call to string_to_quoted causes the formatter to fail, and for some reason diagnostics fail to get updated to tell you there's a syntax error.
This makes the server feel "dead" and is annoying since you don't know where to look for the error.

Should generate two diagnostics for `missing terminator` error

defmodule Dummy do
  @moduledoc """
  Documentation for `Dummy`.
  """

  @doc """
  Hello world.

  ## Examples

      iex> Dummy.hello()
      :world

  """
  def hello do
    :world
  end

  def foo(foo) do
    .
  end
end
$ mix compile
Compiling 1 file (.ex)

== Compilation error in file lib/dummy.ex ==
** (TokenMissingError) lib/dummy.ex:23:1: missing terminator: end (for "do" starting a
t line 1)
    (elixir 1.14.0) lib/kernel/parallel_compiler.ex:346: anonymous fn/5 in Kernel.Para
llelCompiler.spawn_workers/7

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.