jupyterlab / jupyter-chat

An extension to add a chat panel

jupyter-chat

This project is a monorepo containing the frontend components and extensions to build a chat in Jupyter.

Many components of this chat project come from jupyter-ai.

Composition

Typescript package

The typescript package is located in packages/jupyter-chat and builds an NPM package named @jupyter/chat.

This package provides a frontend library (using react), and is intended to be used by a jupyterlab extension to create a chat.

Jupyterlab extensions

Chat extension based on shared document: packages/jupyterlab-collaborative-chat

This extension is an implementation of the @jupyter/chat package, relying on shared document (see jupyter_ydoc).

It is composed of:

• a Python package named jupyterlab_collaborative_chat, which register the YChat shared document in jupyter_ydoc
• a NPM package named jupyterlab-collaborative-chat.

Chat extension based on websocket: packages/jupyterlab-ws-chat

This extension is an implementation of the @jupyter/chat package, relying on websocket for the communication between server and front end.

It is composed of a Python package named jupyterlab_ws_chat for the server side and a NPM package named jupyterlab-ws-chat.

Contributing

See the contributing part of each package for details.

jupyter-chat's Issues

Rename package @jupyter/chat

As follow up of #6, rename the npm package @jupyter/chat if the repo ends up in an official Jupyter repo.

Releases will have two version bumps

Hi @brichet it looks like we independently arrived at somewhat similar monorepo implementations. I am working on jupyterlab/jupyter-collaboration#280 and in there I noticed that I have to short-circuit if any changes were made to the git, see the bump_version.py script. This is because of jupyter-releaser running the bump action for each of the python packages (jupyter-server/jupyter_releaser#567). This is just to let you know that currently the release check in this repo also bumps two versions, taking an extract from logs:

Changes:
 - chat-jupyter: 0.1.0 => 0.1.1
 - jupyterlab-collaborative-chat: 0.1.0 => 0.1.1
 - jupyterlab-ws-chat: 0.1.0 => 0.1.1

lerna info auto-confirmed
lerna info execute Skipping git tag/commit
lerna info execute Skipping git push
lerna info execute Skipping releases
lerna success version finished
Setting up build environment for missing dependencies
0.1.1
0.1.1
COMMAND: ../../scripts/bump-version.sh patch
lerna notice cli v6.4.1
lerna info ci enabled
lerna info current version 0.1.1
lerna notice FYI git repository validation has been skipped, please ensure your version bumps are correct
lerna WARN force-publish all packages
lerna info Assuming all packages changed
lerna WARN version Skipping working tree validation, proceed at your own risk

Changes:
 - chat-jupyter: 0.1.1 => 0.1.2
 - jupyterlab-collaborative-chat: 0.1.1 => 0.1.2
 - jupyterlab-ws-chat: 0.1.1 => 0.1.2

Add documentation

Localize more user facing strings

Problem

Briefly looking at the code, it looks like there are a couple more strings could be localized using the JupyterLab translator, for example:

jupyter-chat/packages/jupyterlab-collaborative-chat/src/index.ts

Line 341 in f3ba6e0

label: 'Open a chat',

Proposed Solution

Use the ITranslator in more places.

Consider making the list of params for `buildChatSidebar` an object

Currently parameters must be passed as regular parameters to the buildChatSidebar helper function:

jupyter-chat/packages/jupyter-chat/src/widgets/chat-sidebar.tsx

Lines 14 to 18 in f3ba6e0

	export function buildChatSidebar(
	chatModel: IChatModel,
	themeManager: IThemeManager | null,
	rmRegistry: IRenderMimeRegistry
	): ReactWidget {

Maybe it could be useful to provide an options object instead, so it's easier to grow the list of parameters in the long run?

Something that can be called as follows by user code:

buildChatSidebar({ chatModel, themeManager });

Improve the `dev-install` script

The top-level package.json has a dev-install script here:

jupyter-chat/package.json

Line 36 in 025efd1

"dev-install": "lerna run dev-install --stream",

However the underlying packages don't seem to be exposing such script in their own package.json.

It could be useful to add such script in the different packages to speed up the dev install, which would for example run the jupyter labextension develop . --overwrite command for the lab extensions.

Question about collaborative chat shared document

The shared document in collaborative chat is a JSON formatted as the following:

{
  "messages": {
    "msg_id1": {
      "type": "msg",
      "id": "f7843bee-7e40-4659-84f6-552d492befd3",
      "body": "Hello World!",
      "time": 1714057210.97,
      "sender": "4480f9e7ffc847b3b657d8681ba3a4c8"
    }
  },
  "users": {
    "4480f9e7ffc847b3b657d8681ba3a4c8": {
      "name": "Anonymous Thyone",
      "avatar_url": null,
      "initials": "AT",
      "username": "4480f9e7ffc847b3b657d8681ba3a4c8",
      "color": "var(--jp-collaborator-color7)",
      "display_name": "Anonymous Thyone"
    }
  }
}

Problem

Structuring the messages as a Map is better for message manipulation (edition, deletion), but bring some drawback in comparison with an Array:

• the file is hardly readable without the widget, since the messages will not be ordered.
• if we expect to do some lazy loading for large file (loading only the last messages), it will be less performant if the messages are not ordered.

Proposed Solution

Switch back to an Array for the messages.
This structure has been changed in #18 (the first idea of the PR was to add the users section).
The part about the messages could be reverted for better readability.

Path configuration for .chat files

Problem

The .chat files are created in the home directory of JupyterLab. It would be nice to be able to specify the path for chat files so users can put them into a chats folder, or admins (for instance using JupyterHub) can store them in a different path to minimize confusion for novice users.

Proposed Solution

Enable configuration or something like a jupyterlab_collaborative_chat_config.py that is processed on startup, where the path can be specified.

Add autocomplete feature

As suggested by @jtpio, the autocomplete feature of jupyter-ai could be ported to jupyter-chat.

Problem

There is no auto-completion in jupyter-chat and no way setup one for a specific chat.

Proposed Solution

Include a "plugin system" like to allow customize auto-completion in the chat input.

Python SDK to add or receive jupyter-chat message events in the jupyter notebook

Problem

When developing LLM-based chat applications, people need a chat interface to see how it works. Many times people use a library like StreamLit in a separate file and which runs in a separate process.

This works, but there would be advantages if one were able to prototype a chat application right in JupyterLab:

• a single JupyterLab instance can be shared by multiple people; once it's stood up it can be re-used
• the JupyterLab instance doesn't require knowledge of how to use the terminal CLI; it's more friendly for non-technical users, which is growing in importance as Red Teaming and AI UX experts need to be able to be included in the AI development process, even though they don't necessarily know how to run and modify a python application from the CLI
• a Jupyter notebook tends to be a single file encapsulating everything needed to prototype a particular behavior, so an individual user could clone a Jupyter notebook and change parts with a feeling of "safety."

Given that there's not an idiomatic chat widget built into the Jupyter notebooks yet, I think it might be nice if a Jupyter notebook situated in a JupyterLab instance had access to be able to use the jupyter-chat extension panel as an interface to prototype chat applications.

Proposed Solution

Provide a jupyter_chat SDK that makes it possible to

• listen to message events
• write message events
• read all messages
• clear all messages

By calling the SDK from inside the cells of a jupyter notebook, one could control the chat interface so as to render an LLM-powered AI chat.

Timestamp server side in collaborative chat

Problem

The timestamp of messages in the collaborative chat is set up client side.

Proposed Solution

For greater reliability, it should be configured on the server side. This would avoid time-stamping errors due to incorrect client configuration.

Update links to the repo

As a follow-up to the move to the JupyterLab organization (jupyterlab/frontends-team-compass#247), we should look into:

• updating the various links to the repo
• if published to PyPI, make sure the trusted publisher is set up properly

Add notifications in collaborative chat

Problem

There is currently no notification on new message or new chat.

Proposed Solution

Add notifications when a chat is created, or a new message received.

It could use the notification system of Jupyterlab, and add a * in the tab title.

Set up the releaser workflow

Task list to set up the releaser workflow:

• Add the jupyterlab releaser bot Github App (done by default)
• Create a "release" environment
• Add APP_ID variable, NPM_TOKEN and APP_PRIVATE_KEY secrets to that environment (coming from the Github app)
• Configure the ruleset on branch and tag to allow the Github app to bypass it

cc @jtpio

Add binder link

Problem

There is no binder link in README (actually there is one not working in the collaborative chat README).

Proposed Solution

• add binder links in the main README to try both the extensions.
• add corresponding binder link in the README of each extension
• add binder links on PR

Collaborative chat models manager

It may be interesting to have a manager of collaborative chat models, to dissociate the model from the widget.

Problems

• notifications (after #43) will be enabled only if a widget is open for the chat.
• moving a widget from main area to the side panel (and vice versa) recreates a model from scratch. Moving the widget uses app commands, and there is no way to pass on complex arguments (model for instance).

Proposed Solution

Add a model manager, that would allow having a model running without any widget. That way we could:

• reference it when moving a widget from one area to an other
• add a context menu command to a chat file to enable notification without any opened widget
• add the manager to the running session panel, to let user manage its running models

Stack messages in the UI

Problem

When a user send several messages in a row, the username is repeated for each message.

Proposed Solution

The messages could be stacked to avoid repeated useless information.

Additional context

It could be true by default but configurable using the config of the model.

JSON schema for the shared document

Problem

The chat shared document is a JSON file, but does not have associated schema yet.

Proposed Solution

We should provide a versioned JSON schema of the chat file, for better backward compatibility in future change of the schema.

Support for streaming messages

Problem

Currently messages are added to the chat in one go. See this screencast as an example:

jupyterlab-codestral-demo-lite.webm

However most of the AI chat UIs stream responses to the user, for example:

mistral-ai-chat.webm

Proposed Solution

Add optional support for streaming messages, which could be enabled via the settings editor or by extensions.

Additional context

This would be useful for AI extensions using jupyter-chat for interacting with some online AI providers, such as https://github.com/jtpio/jupyterlab-codestral