jupyter / jupyter_events

Configurable event system for Jupyter applications and extensions.

Jupyter Events

An event system for Jupyter Applications and extensions.

Jupyter Events enables Jupyter Python Applications (e.g. Jupyter Server, JupyterLab Server, JupyterHub, etc.) to emit events—structured data describing things happening inside the application. Other software (e.g. client applications like JupyterLab) can listen and respond to these events.

Install

Install Jupyter Events directly from PyPI:

pip install jupyter_events

or conda-forge:

conda install -c conda-forge jupyter_events

Documentation

Documentation is available at jupyter-events.readthedocs.io.

About the Jupyter Development Team

The Jupyter Development Team is the set of all contributors to the Jupyter project. This includes all of the Jupyter subprojects.

The core team that coordinates development on GitHub can be found here: https://github.com/jupyter/.

Our Copyright Policy

Jupyter uses a shared copyright model. Each contributor maintains copyright over their contributions to Jupyter. But, it is important to note that these contributions are typically only changes to the repositories. Thus, the Jupyter source code, in its entirety is not the copyright of any single person or institution. Instead, it is the collective copyright of the entire Jupyter Development Team. If individual contributors want to maintain a record of what changes/contributions they have specific copyright on, they should indicate their copyright in the commit message of the change, when they commit the change to one of the Jupyter repositories.

With this in mind, the following banner should be used in any source code file to indicate the copyright and license terms:

# Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License.

jupyter_events's Issues

Feature request: add modifier API to mutate an event before emitting

The modifier API would provide a hook for extension authors to modify an event in-place before it is validated and emitted.

A modifier would a be callable that would take the raw event data, modify it in-place, and return the modified version.

def modifier(event_data: dict) -> dict:
    ...

The immediate use-case for the modifier API would be to apply "redactors" to the EventLogger that remove properties from the event before they were emitted.

The EventLogger would need a new method to add these modifiers, e.g.

class EventLogger(...):

    ...

    def add_modifier(self, modifier: Callable[[dict], dict]):
        # Adds modifier to list of modifiers
        ...

    def emit(self, ..., data: dict, ...):
        # Should modify the data in-place.
        for modifier in self._modifiers:
            data = modifier(data)

        # Validate after. Ensures that validators return a valid schema to emit.
        self.validate(..., data)

        # Emit the event
        self.emit(...)

The public API would look something like:

def my_redactor(event_data: dict) -> dict:
    # Redact sensitive data.
    if "username" in event_data:
        event_data["username"] = "<masked>"
    return event_data


logger = EventLogger()
logger.add_modifier(my_redactor)

`jupyter-events` break JupyterLab due to lack of `jsonschema` version pin/incompatibility with newer versions

See:

• jupyter/jupyter_core#369
• jupyterlab/jupyterlab#14507 (comment)

But briefly:

Traceback (most recent call last):
  File "/usr/bin/jupyter", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/lib/python3.11/site-packages/jupyter_core/command.py", line 261, in main
    mod = __import__(package)
          ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/jupyterlab/__init__.py", line 8, in <module>
    from .handlers.announcements import (
  File "/usr/lib/python3.11/site-packages/jupyterlab/handlers/announcements.py", line 14, in <module>
    from jupyter_server.base.handlers import APIHandler
  File "/usr/lib/python3.11/site-packages/jupyter_server/base/handlers.py", line 24, in <module>
    from jupyter_events import EventLogger
  File "/usr/lib/python3.11/site-packages/jupyter_events/__init__.py", line 3, in <module>
    from .logger import EVENTS_METADATA_VERSION, EventLogger
  File "/usr/lib/python3.11/site-packages/jupyter_events/logger.py", line 19, in <module>
    from .schema import SchemaType
  File "/usr/lib/python3.11/site-packages/jupyter_events/schema.py", line 18, in <module>
    from .validators import draft7_format_checker, validate_schema
  File "/usr/lib/python3.11/site-packages/jupyter_events/validators.py", line 44, in <module>
    JUPYTER_EVENTS_SCHEMA_VALIDATOR = Draft7Validator(
                                      ^^^^^^^^^^^^^^^^
TypeError: create.<locals>.Validator.__init__() got an unexpected keyword argument 'registry'

As per:

Downgrading jsonschema from the latest 4.20.0 to 4.19.2 solved the issue for me, fyi.

Originally posted by @gokceneraslan in jupyter/jupyter_core#369 (comment)

Current version is 4.22

Feature Request: add a command line tool to ensure a custom schema is valid

We should provide schema authors a simple CLI tool to check that their schema is valid.

Feature request: add API for event listeners

One way to extend Jupyter's Event system would be to enable extensions/plugins of Jupyter Applications to listen for specific events and trigger a callback function. One common use-case would be to enable an extension to emit additional event data that immediately follows the source event.

I think the easiest way to achieve this is to add a simple API to the main EventLogger object, something like:

class EventLogger:
    listeners: dict
    ...
    def add_listener(self, schema_id: str, version: int, listener) -> None:
        ...

    def emit(self, schema_id: str, version: int, data: dict) -> dict:
        ...
        # Event has already been validated and emitted
        for listener in self.listeners[(schema_id, version)]:
            listener(self, schema_id, version, data)

where the signature of the callback is :

def my_listener(logger: EventLogger, schema_id:str, version: int, data: dict) -> None:
    ...

data is the raw (validated) event data.

Using more JSON Schema format checkers

With the default jsonschema, many of the extended format arguments to string schema, such as uri, uuid, date, email would be silently ignored, but all seem rather relevant in an event sourcing system.

By adding a dependency on jsonschema[format] or [format-nogpl], these will actually be validated.

Additionally, custom validators can be added when the schema is instantiated: this could be useful, and more secure, than relying on the potentially dangerous YAML tags allowed by #26.

0.5.0: pytest is failing in two units

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix>
• run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-events-0.5.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-jupyter-events-0.5.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.15, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/jupyter_events-0.5.0, configfile: pyproject.toml, testpaths: tests/
plugins: anyio-3.6.2, console-scripts-1.3.1, asyncio-0.20.2
asyncio: mode=auto
collected 49 items

tests/test_cli.py .......                                                                                                                                            [ 14%]
tests/test_listeners.py .....                                                                                                                                        [ 24%]
tests/test_logger.py .........F...                                                                                                                                   [ 51%]
tests/test_modifiers.py ......                                                                                                                                       [ 63%]
tests/test_schema.py ..F............                                                                                                                                 [ 93%]
tests/test_traits.py ...                                                                                                                                             [100%]

================================================================================= FAILURES =================================================================================
________________________________________________________________________ test_emit_badschema_format ________________________________________________________________________

    def test_emit_badschema_format():
        """
        Fail fast when an event doesn't conform to a specific format
        """
        schema = {
            "$id": "http://test/test",
            "version": 1,
            "type": "object",
            "properties": {
                "something": {"type": "string", "title": "test", "format": "date-time"},
            },
        }

        el = EventLogger(handlers=[logging.NullHandler()])
        el.register_event_schema(schema)

        with pytest.raises(jsonschema.ValidationError) as excinfo:
>           el.emit(schema_id="http://test/test", data={"something": "chucknorris"})
E           Failed: DID NOT RAISE <class 'jsonschema.exceptions.ValidationError'>

tests/test_logger.py:264: Failed
_______________________________________________________ test_bad_validations[bad-id.yaml-'not-a-uri' is not a 'uri'] _______________________________________________________

schema_file = 'bad-id.yaml', validation_error_msg = "'not-a-uri' is not a 'uri'"

    @pytest.mark.parametrize("schema_file,validation_error_msg", BAD_SCHEMAS)
    def test_bad_validations(schema_file, validation_error_msg):
        """
        Validation fails because the schema is missing
        a redactionPolicies field.
        """
        # Read the schema file
        with open(SCHEMA_PATH / "bad" / schema_file) as f:
            schema = yaml.loads(f)
        # Assert that the schema files for a known reason.
        with pytest.raises(ValidationError) as err:
>           validate_schema(schema)
E           Failed: DID NOT RAISE <class 'jsonschema.exceptions.ValidationError'>

tests/test_schema.py:38: Failed
=========================================================================== slowest 10 durations ===========================================================================
0.27s call     tests/test_cli.py::test_cli_version[subprocess]
0.27s call     tests/test_cli.py::test_cli_invalid[subprocess]
0.27s call     tests/test_cli.py::test_cli_good_raw[subprocess]
0.27s call     tests/test_cli.py::test_cli_good[subprocess]
0.26s call     tests/test_cli.py::test_cli_missing[subprocess]
0.26s call     tests/test_cli.py::test_cli_malformed[subprocess]
0.26s call     tests/test_cli.py::test_cli_help[subprocess]
0.01s setup    tests/test_listeners.py::test_listener_function
0.01s teardown tests/test_modifiers.py::test_modifier_function

(1 durations < 0.005s hidden.  Use -vv to show these durations.)
========================================================================= short test summary info ==========================================================================
FAILED tests/test_logger.py::test_emit_badschema_format - Failed: DID NOT RAISE <class 'jsonschema.exceptions.ValidationError'>
FAILED tests/test_schema.py::test_bad_validations[bad-id.yaml-'not-a-uri' is not a 'uri'] - Failed: DID NOT RAISE <class 'jsonschema.exceptions.ValidationError'>
======================================================================= 2 failed, 47 passed in 2.28s =======================================================================

Here is list of installed modules in build env

Package                       Version
----------------------------- -----------------
alabaster                     0.7.12
anyio                         3.6.2
appdirs                       1.4.4
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
attrs                         22.1.0
Babel                         2.11.0
beautifulsoup4                4.11.1
bleach                        5.0.0
Brlapi                        0.8.3
build                         0.9.0
cffi                          1.15.1
charset-normalizer            3.0.1
click                         8.1.3
cloudpickle                   2.2.0
commonmark                    0.9.1
contourpy                     1.0.6
cssselect                     1.1.0
cycler                        0.11.0
defusedxml                    0.7.1
distro                        1.8.0
dnspython                     2.2.1
docutils                      0.19
doit                          0.36.0
editables                     0.3
entrypoints                   0.4
exceptiongroup                1.0.0
extras                        1.0.0
fastjsonschema                2.16.1
fixtures                      4.0.0
fonttools                     4.38.0
gpg                           1.17.1-unknown
hatchling                     1.11.1
html5lib                      1.1
idna                          3.4
imagesize                     1.4.1
importlib-metadata            5.1.0
importlib-resources           5.9.0
iniconfig                     1.1.1
Jinja2                        3.1.2
json5                         0.9.9
jsonschema                    4.17.3
jupyter_client                7.4.8
jupyter_core                  5.1.0
jupyter-server                1.23.3
jupyterlab-pygments           0.1.2
jupyterlab_server             2.16.3
kiwisolver                    1.4.4
libcomps                      0.1.19
louis                         3.23.0
lxml                          4.9.1
markdown-it-py                2.1.0
MarkupSafe                    2.1.1
matplotlib                    3.6.2
mdit-py-plugins               0.3.3
mdurl                         0.1.2
mistune                       2.0.4
myst-parser                   0.18.1
nbclient                      0.7.2
nbconvert                     7.2.6
nbformat                      5.7.0
nest-asyncio                  1.5.6
numpy                         1.23.1
olefile                       0.46
packaging                     21.3
pandocfilters                 1.5.0
pathspec                      0.10.2
pbr                           5.9.0
pep517                        0.13.0
Pillow                        9.3.0
pip                           22.3.1
pkgutil_resolve_name          1.3.10
platformdirs                  2.5.2
pluggy                        1.0.0
ply                           3.11
prometheus-client             0.15.0
ptyprocess                    0.7.0
pycparser                     2.21
Pygments                      2.13.0
PyGObject                     3.42.2
pyparsing                     3.0.9
pyrsistent                    0.19.2
pytest                        7.2.0
pytest-asyncio                0.20.2
pytest-console-scripts        1.3.1
python-dateutil               2.8.2
python-json-logger            2.0.4
pytz                          2022.4
PyYAML                        6.0
pyzmq                         24.0.0
requests                      2.28.1
rich                          12.6.0
rpm                           4.17.0
scour                         0.38.2
Send2Trash                    1.8.0
setuptools                    65.6.3
six                           1.16.0
sniffio                       1.2.0
snowballstemmer               2.2.0
soupsieve                     2.3.2.post1
Sphinx                        5.3.0
sphinxcontrib-applehelp       1.0.2.dev20221204
sphinxcontrib-devhelp         1.0.2.dev20221204
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1.dev20221204
sphinxcontrib-qthelp          1.0.3.dev20221204
sphinxcontrib-serializinghtml 1.1.5
terminado                     0.15.0
testtools                     2.5.0
tinycss2                      1.2.1
tomli                         2.0.1
tornado                       6.2
traitlets                     5.4.0
typing_extensions             4.4.0
urllib3                       1.26.12
webencodings                  0.5.1
websocket-client              1.4.2
wheel                         0.38.4
zipp                          3.11.0

Attribute Error for python 3.11

Trying to start jupyterlab on python 3.11 I see the following

AttributeError: module 'collections' has no attribute 'Hashable'

whole stack

python3 -m jupyterlab
Traceback (most recent call last):
  File "<frozen runpy>", line 189, in _run_module_as_main
  File "<frozen runpy>", line 148, in _get_module_details
  File "<frozen runpy>", line 112, in _get_module_details
  File "/opt/py/py/lib/python3.11/site-packages/jupyterlab/__init__.py", line 7, in <module>
    from .handlers.announcements import (  # noqa
  File "/opt/py/py/lib/python3.11/site-packages/jupyterlab/handlers/announcements.py", line 14, in <module>
    from jupyter_server.base.handlers import APIHandler
  File "/opt/py/py/lib/python3.11/site-packages/jupyter_server/base/handlers.py", line 23, in <module>
    from jupyter_events import EventLogger
  File "/opt/py/py/lib/python3.11/site-packages/jupyter_events/__init__.py", line 3, in <module>
    from .logger import EVENTS_METADATA_VERSION, EventLogger
  File "/opt/py/py/lib/python3.11/site-packages/jupyter_events/logger.py", line 19, in <module>
    from .schema_registry import SchemaRegistry
  File "/opt/py/py/lib/python3.11/site-packages/jupyter_events/schema_registry.py", line 3, in <module>
    from .schema import EventSchema
  File "/opt/py/py/lib/python3.11/site-packages/jupyter_events/schema.py", line 9, in <module>
    from .validators import draft7_format_checker, validate_schema
  File "/opt/py/py/lib/python3.11/site-packages/jupyter_events/validators.py", line 17, in <module>
    EVENT_METASCHEMA = yaml.load(EVENT_METASCHEMA_FILEPATH)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/py/py/lib/python3.11/site-packages/jupyter_events/yaml.py", line 25, in load
    return loads(data)
           ^^^^^^^^^^^
  File "/opt/py/py/lib/python3.11/site-packages/jupyter_events/yaml.py", line 15, in loads
    return yload(stream, Loader=SafeLoader)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/py/py/lib/python3.11/site-packages/yaml/__init__.py", line 72, in load
    return loader.get_single_data()
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/py/py/lib/python3.11/site-packages/yaml/constructor.py", line 37, in get_single_data
    return self.construct_document(node)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/py/py/lib/python3.11/site-packages/yaml/constructor.py", line 46, in construct_document
    for dummy in generator:
  File "/opt/py/py/lib/python3.11/site-packages/yaml/constructor.py", line 398, in construct_yaml_map
    value = self.construct_mapping(node)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/py/py/lib/python3.11/site-packages/yaml/constructor.py", line 204, in construct_mapping
    return super().construct_mapping(node, deep=deep)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/py/py/lib/python3.11/site-packages/yaml/constructor.py", line 126, in construct_mapping
    if not isinstance(key, collections.Hashable):
                           ^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'collections' has no attribute 'Hashable'

I understand the issue is likely in pyyaml, leaving this here for visibility, feel free to close

Published wheel has invalid metadata

The published wheel for jupyter_events 0.6.3 has an invalid line in jupyter_events-0.6.3.dist-info/METADATA:

Requires-Dist: jsonschema[format-nongpl]>=3.2.0

See pypa/hatch#801

pyyaml dependency

After completing the effort for #58 (via #59), this now exposes another issue with kfp's capping policy. In this case, they are capping pyyaml<6 (with a floor >=5.3). For applications using both Jupyter Server (which pulls in jupyter_events) and KFP, this generates an impossible resolution.

Similar to #59, is it possible to decrease the floor of pyyaml to >=5.3?

After doing so in my env (i.e., pip install pyyaml==5.3), both pytests and pre-commit succeed.

I'll go ahead and submit a PR and we can discuss it further.

Defining event schema with well-known paths?

I've taken a look at the code and the docs here and upstream in Server, and only see reference to the python-side event registration.

For cases like a labextension, is it possible to register an event by putting a declarative file in a place at packaging time?

I could imagine two approaches:

event-schema-at-rest

{sys.prefix}/
  share/
    jupyter/
      events/
        my-custom-event.yaml

event-schema-in-traitlets

{sys.prefix}/
  etc/
    jupyter/
      jupyter_server_config.d/
        my-custom-event.json

{
  "EventLogger": {
    "extra_schemas": {
      "my-custom-event": {
        "$id": "http://event.jupyter.org/test",
        "version": 1,
        "title": "Simple Test Schema",
        "description": "A simple schema for testing",
        "type": "object",
        "properties": {
          "prop": {
            "title": "Test Property",
            "description": "Test property.",
            "type": "string"
          }
        }
      }
    }
  }
}

My preference, I guess, would be to the former, as nesting schema inside anything is kinda nightmarish, but there is a certain simplicity in keeping everything in traitlets, for good or bad.

Version 0.7.0 is breaking jupyter lab build

Hi

I tried to run jupyter lab build in my Ubuntu 22 container - it was working on Friday, but now it stopped working, after the release of version 0.7.0 of jupyter_events

First I had this error:

0.783 Traceback (most recent call last):
0.783 File "/opt/conda/bin/jupyter-lab", line 6, in
0.783 from jupyterlab.labapp import main
0.783 File "/opt/conda/lib/python3.10/site-packages/jupyterlab/init.py", line 7, in
0.783 from .handlers.announcements import ( # noqa
0.783 File "/opt/conda/lib/python3.10/site-packages/jupyterlab/handlers/announcements.py", line 14, in
0.783 from jupyter_server.base.handlers import APIHandler
0.783 File "/opt/conda/lib/python3.10/site-packages/jupyter_server/base/handlers.py", line 23, in
0.784 from jupyter_events import EventLogger
0.784 File "/opt/conda/lib/python3.10/site-packages/jupyter_events/init.py", line 3, in
0.784 from .logger import EVENTS_METADATA_VERSION, EventLogger
0.785 File "/opt/conda/lib/python3.10/site-packages/jupyter_events/logger.py", line 18, in
0.785 from .schema import SchemaType
0.785 File "/opt/conda/lib/python3.10/site-packages/jupyter_events/schema.py", line 7, in
0.786 from referencing import Registry
0.786 ModuleNotFoundError: No module named 'referencing'

After running pip install referencing, I had the following error:

0.950 Traceback (most recent call last):
0.950 File "/opt/conda/bin/jupyter-lab", line 6, in
0.950 from jupyterlab.labapp import main
0.950 File "/opt/conda/lib/python3.10/site-packages/jupyterlab/init.py", line 7, in
0.950 from .handlers.announcements import ( # noqa
0.950 File "/opt/conda/lib/python3.10/site-packages/jupyterlab/handlers/announcements.py", line 14, in
0.950 from jupyter_server.base.handlers import APIHandler
0.950 File "/opt/conda/lib/python3.10/site-packages/jupyter_server/base/handlers.py", line 23, in
0.950 File "/opt/conda/lib/python3.10/site-packages/jupyter_events/init.py", line 3, in
0.951 from .logger import EVENTS_METADATA_VERSION, EventLogger
0.951 File "/opt/conda/lib/python3.10/site-packages/jupyter_events/logger.py", line 18, in
0.951 from .schema import SchemaType
0.951 File "/opt/conda/lib/python3.10/site-packages/jupyter_events/schema.py", line 18, in
0.951 from .validators import draft7_format_checker, validate_schema
0.951 File "/opt/conda/lib/python3.10/site-packages/jupyter_events/validators.py", line 41, in
0.951 JUPYTER_EVENTS_SCHEMA_VALIDATOR = Draft7Validator( # type: ignore

My installation of Jupyter Lab is made on miniconda, over a nvidia-cuda container, using the following libraries:
jupyterlab==3.6.3
jupyterlab-git==0.41.0
ipywidgets==8.0.7
nb_conda_kernels==2.3.1
jupyter_contrib_nbextensions==0.7.0

After uninstalling jupyter_events 0.7.0 and reinstalling 0.6.3 the build worked fine.

jsonschema dependency

In schema.py you import from jsonschema.protocols import Validator for the purposes of adding a type hint. As far as I can see, this is the only reason why you need to depend on jsonschema >=4. Unfortunately, the community has not been speedy on adopting jsonschema >=4 in many, many places (for instance here and here).

I was wondering if it makes sense to remove the type hint or at least have something like the following, and then drop the dependency for jsonschema back down to 3.x?

try:
    from jsonschema.protocols import Validator
except ImportError:
    from typing import Any
    Validator = Any

While not ideal, the way it is currently means that there is a lot of packages that cannot co-exist with jupyter in the conda ecosystem right now.

Use `yaml.safe_dump` and `_load`?

Using the default load and dump features of pyyaml can have unintended consequences, as they can use YAML tags to execute arbitrary code.

For an application, this is great as it provides a low-code way to use complex structures.

For a library, this is kinda bad, as it means any yaml source is immediately an arbitrary code execution engine.

Likely, this would just need from yaml import safe_dump, safe_load.

Additionally, when reading yaml, using read_text(encoding="utf-8") is a good way to avoid future issues.