leanprover-community / lean-client-python Goto Github PK

The newest version of the lean server now has widgets. Our current code breaks. I think the refactor proposed in #6 (or just adding to the PR #5) could make the lean client more future proof.

As we talked about in #3, commands.py needs unit tests. In particular, it's easy to mess up both converting to and from the correct JSON. (This is even more important if we go with more high level tests for the Lean server that don't depend on the specific JSON. If we do that, there is nothing currently testing our handling of the JSON correctly.) There is no reason to work on this until we complete #6 (or better it could be part of #6).

While we could add methods like state for every possible use case, we don't need to since we can directly use the send method and the corresponding Request object. I'd like to make some examples of this (to head off requests like, can you implement XXX functionality?).

Now that we have a testing framework, #3, we should turn on continuous integration. Any objections?

Here are some examples I'd like to add to the trio server:

• A version of our current example, but with all the lines annotated so the user understands what it is doing.
• A simple example of using InfoCommand, maybe just an example of running info on each token and printing the results (and maybe also printing messages after the sync).
• An example or running eval and check programmatically to do some calculation and using messages to find the results.
• An example of BFS in tactic state with a very simple tactic API.
• A tool to refactor all references to a particular definition name with a new name (e.g. replacing all group.subgroup with group.sub) or something like that. (I made a tool like this once already and I think this is one of the most useful applications of the Lean server.)
• An example combining the lean server and simple meta-programming, like making a tactic or user command to print some custom information to messages.

Also, I'd like to make better instructions on how to set up and run the Trio server on your own Lean project or for mathlib development.

Could this lean client be use to write an IPython kernel that supports Lean?

Watchdog error: Cannot read LSP request: A Lean 3 request was received. Please ensure that your editor has a Lean 4 compatible extension installed. For VSCode, this is

As you can see I installed Lean4 on the right of the screenshot using Elan.

Please make this useful tool work. Thanks.

If you sync the same file again (with no changes) the response is of the form

{"message":"file unchanged","response":"ok","seq_num":3}

It will not send an additional current tasks response, so full_sync is left waiting forever. I'm having a bit of trouble following your code to fix it. You are throwing away the message field (not sure why) and so I don't have access to it to check for this case.

I often use the command line parameter -D pp.all=true. I'd like a way to set Lean command-line parameters.

This relates to Issue #8 (a more comprehensive trio example), but is a little different. In short, I'd like to document all the requests in commands.py. This will provide a form of documentation of the Lean server interface, probably the best so far. I think I'll be able to incorporate it into the PR for Issue #6, but just in case, I've made it a separate issue.

As for the commands.py, I think it suffers from the following possible problem. parse_response is trying (with limited success) to figure out what type a response is from the JSON alone.

I think there should be two levels of parse_response. The first level just turns the JSON into one of fours types: AllMessagesResponse, CurrentTasksResponse, CommandResponse, ErrorResponse. If it is a CommandResponse (or we could call it OkResponse), most of the dictionary should be stored in some field called response_data or something. In the Trio server at least, this is what will be processed and stored by the receiver. Then later the trio send method, when it gets that CommandResponse (or ErrorResponse) can further process it since it knows the type of the request. The commands.py file can have another function called refine_command_response (or it can be a method under command response) which takes in the CommandResponse and the Request object (or type) and returns a ResponseObject of the specific command type (SyncResponse, InfoResponse, etc). This will ensure, for example, if we send an InfoRequest, we get an InfoResponse (or ErrorResponse), even if the InfoRecord is empty. This should simplify code like the state method. Thoughts?

If a message is large, the stdout buffer of the lean server may only contain half a message. Here is code to reproduce the problem:

from pathlib import Path

import trio # type: ignore
from lean_client.trio_server import TrioLeanServer

async def main():
    async with trio.open_nursery() as nursery:
        server = TrioLeanServer(nursery, debug=False)
        await server.start()

        for i in range(1000):
            path = f'/tmp/lean_tmp_{i}.lean'
            with open(path, 'w') as f:
                f.write(f"example : {i}={i} :=\nbegin end")
            
            await server.full_sync(path)
            state = await server.state(path, 2, 0)
            print(state)
        nursery.cancel_scope.cancel()

if __name__ == '__main__':
    trio.run(main)

The fix is to change the loops in receiver as follows:

    async def receiver(self):
        """This task waits for Lean responses, updating the server state
        (tasks and messages) and triggering events when a response comes."""
        if not self.process:
            raise ValueError('No Lean server')
        unfinished_message = b''
        async for data in self.process.stdout:
            lines = (unfinished_message + data).split(b'\n')
            unfinished_message = lines.pop()  # usually empty, but can be half a message
            for line in lines:
                resp = parse_response(line.decode())
                if self.debug:
                    print(f'Received {resp}')
                if isinstance(resp, CurrentTasksResponse):
                    self.current_tasks = resp.tasks
                    if not resp.is_running:
                        self.is_fully_ready.set()
                elif isinstance(resp, AllMessagesResponse):
                    self.messages = resp.msgs
                if hasattr(resp, 'seq_num'):
                    self.responses[resp.seq_num] = resp
                    self.response_events[resp.seq_num].set()