chainyo / tensorshare

🤝 Trade any tensors over the network

TensorShare

🤝 Trade any tensors over the network

---

TensorShare is a powerful tool enabling ⚡ lightning-fast tensor sharing across networks.

This project leverages the best open-source tools to provide a simple and easy-to-use interface for sharing tensors:

• safetensors for secure tensor serialization and deserialization.
• pydantic for data validation and settings management.
• fastapi for building APIs (and because it's too good to avoid it).

This project is heavily in development and is not ready for production use. Feel free to contribute to the project by opening issues and pull requests.

Usage

Example of tensors serialization with torch:

import torch
from tensorshare import TensorShare

tensors = {
    "embeddings": torch.zeros((2, 2)),
    "labels": torch.zeros((2, 2)),
}
ts = TensorShare.from_dict(tensors, backend="torch")
print(ts)
# tensors=b'gAAAAAAAAAB7ImVtYmVkZGluZ3MiOnsiZHR5cGUiO...' size=168

You can now freely send the tensors over the network via any means (e.g. HTTP, gRPC, ...).

On the other side, when you receive the tensors, you can deserialize them in any supported backend:

from tensorshare import Backend

np_tensors = ts.to_tensors(backend=Backend.NUMPY)
print(np_tensors)
# {
# 	'embeddings': array([[0., 0.], [0., 0.]], dtype=float32),
# 	'labels': array([[0., 0.], [0., 0.]], dtype=float32)
# }

For more examples and details, please refer to the Usage section.

Roadmap

• Pydantic schema for sharing tensors ~> TensorShare
• Serialization and deserialization ~> tensorshare.serialization
• TensorProcessor for processing tensors ~> TensorProcessor
• Server for sharing tensors ~> TensorShareServer
• Client for sharing tensors ~> TensorShareClient
• New incredible features! (Check the issues)

tensorshare's Issues

Add serialization method to TensorShare

The TensorShare schema needs to have a serialize method that uses the TensorConverter (rename it to TensorSerializer).

Prepare deserialization for any tensors

Now we have a TensorProcessor with a serialization method, we need to add a deserialization method.

Add Streaming support

This would be lit to add streaming support for large workloads.

Leverage ByteSize from pydantic

Add inner functions to the TensorShare schema to return the size in different formats depending on the size.

Fix TensorShareClient import

Make TensorShareClient importable from tensorshare and make aiohttp as an optional deps.

Grammar/typos correction

Correct the readme/documentation from typos and grammar mistakes

Add task tracking

For production environments, keeping track of the tasks sent to a server with a tracking tool, including a lightweight database system, could be nice.

Add `pydantic` v1 compatibility

This project is built on top of pydantic>=2.1.1 but it could be helpful to make it compatible with v1 too.

Add client and router usage

Add the new TensorShareClient usage to the documentation and provide a code snippet to share FastAPI router integration.

Prepare client sending requests

To send any TenshorShare object across the network, we need prepared functions that allow this in one line.

• takes a dict of tensors + metadata + backend as inputs
• include headers with tokens
• prepare payload
• send request
• sync + async?

Add Tensorflow

Tensorflow can be added when the v2.14.0 is released for pydanticv2 compatibility.

Prepare v0.1.1

Add router server implementation

Users should be able to easily integrate into their FastAPI application the required endpoints from TensorShare with custom return functions and logic.

-> Leverages the APIRouter and Depends from FastAPI

Add a notebook for quick tests

Add logging

This package lacks logs!

Fix the package imports for next release

At the moment the project requires all the backends to be installed, fix that.

Prepare v0.1.0

Format tensors to dict function

Create a utility function that takes any tensors and formats them in a Dict.

• use embeddings or embeddings_N when no key is provided
• ensure keys are different

TensorConverter interface

To allow the user to convert the tensors without using a specific method, we will create an interface class that handles the conversion process automatically based on the input tensors.

The input tensors could have different formats like np.ndarray and torch.Tensor.

Add a method to deserialize to TensorShare

The TensorShare object should have a method to_tensors to convert the serialized tensors to any backend tensors (the from_dict reverse function).

Add TensorShare usage to docs

Show multi-backend usage of the TensorShare schema.

Fix dependencies naming

Fix the optional dependencies naming in the pyproject.toml file.

Add authentication to pre-built endpoints

Add authentication as a parameter to the pre-built FastAPI endpoints.

Add codecov

Add a coverage tool and a badge in the README.