GithubHelp home page GithubHelp logo

upstash / redis-py Goto Github PK

View Code? Open in Web Editor NEW
26.0 8.0 3.0 443 KB

HTTP based Python Redis Client for Serverless and Edge Functions

Home Page: https://docs.upstash.com/redis

License: MIT License

Python 100.00%
redis upstash upstash-sdk

redis-py's Introduction

Upstash Redis Python SDK

Note

This project is in GA Stage.

The Upstash Professional Support fully covers this project. It receives regular updates, and bug fixes. The Upstash team is committed to maintaining and improving its functionality.

upstash-redis is a connectionless, HTTP-based Redis client for Python, designed to be used in serverless and serverful environments such as:

  • AWS Lambda
  • Vercel Serverless
  • Google Cloud Functions
  • and other environments where HTTP is preferred over TCP.

Inspired by other Redis clients like @upstash/redis and redis-py, the goal of this SDK is to provide a simple way to use Redis over the Upstash REST API.

The SDK is currently compatible with Python 3.8 and above.

Quick Start

Install

PyPI

pip install upstash-redis

Usage

To be able to use upstash-redis, you need to create a database on Upstash and grab UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN from the console.

# for sync client
from upstash_redis import Redis

redis = Redis(url="UPSTASH_REDIS_REST_URL", token="UPSTASH_REDIS_REST_TOKEN")

# for async client
from upstash_redis.asyncio import Redis

redis = Redis(url="UPSTASH_REDIS_REST_URL", token="UPSTASH_REDIS_REST_TOKEN")

Or, if you want to automatically load the credentials from the environment:

# for sync use
from upstash_redis import Redis
redis = Redis.from_env()

# for async use
from upstash_redis.asyncio import Redis
redis = Redis.from_env()

If you are in a serverless environment that allows it, it's recommended to initialise the client outside the request handler to be reused while your function is still hot.

Running commands might look like this:

from upstash_redis import Redis

redis = Redis.from_env()

def main():
  redis.set("a", "b")
  print(redis.get("a"))

# or for async context:

from upstash_redis.asyncio import Redis

redis = Redis.from_env()

async def main():  
  await redis.set("a", "b")
  print(await redis.get("a"))

BITFIELD and BITFIELD_RO

One particular case is represented by these two chained commands, which are available as functions that return an instance of the BITFIELD and, respectively, BITFIELD_RO classes. Use the execute function to run the commands.

redis.bitfield("test_key") \
  .incrby(encoding="i8", offset=100, increment=100) \
  .overflow("SAT") \
  .incrby(encoding="i8", offset=100, increment=100) \
  .execute()

redis.bitfield_ro("test_key_2") \
  .get(encoding="u8", offset=0) \
  .get(encoding="u8", offset="#1") \
  .execute()

Custom commands

If you want to run a command that hasn't been implemented, you can use the execute function of your client instance and pass the command as a list.

redis.execute(command=["XLEN", "test_stream"])

Pipelines & Transactions

If you want to submit commands in batches to reduce the number of roundtrips, you can utilize pipelining or transactions. The difference between pipelines and transactions is that transactions are atomic: no other command is executed during that transaction. In pipelines there is no such guarantee.

To use a pipeline, simply call the pipeline method:

pipeline = redis.pipeline()

pipeline.set("foo", 1)
pipeline.incr("foo")
pipeline.get("foo")

result = pipeline.exec()

print(result)
# prints [True, 2, '2']

For transaction, use mutli:

pipeline = redis.multi()

pipeline.set("foo", 1)
pipeline.incr("foo")
pipeline.get("foo")

result = pipeline.exec()

print(result)
# prints [True, 2, '2']

You can also chain the commands:

pipeline = redis.pipeline()

pipeline.set("foo", 1).incr("foo").get("foo")
result = pipeline.exec()

print(result)
# prints [True, 2, '2']

Encoding

Although Redis can store invalid JSON data, there might be problems with the deserialization. To avoid this, the Upstash REST proxy is capable of encoding the data as base64 on the server and then sending it to the client to be decoded.

For very large data, this can add a few milliseconds in latency. So, if you're sure that your data is valid JSON, you can set rest_encoding to None.

Retry mechanism

upstash-redis has a fallback mechanism in case of network or API issues. By default, if a request fails it'll retry once, 3 seconds after the error. If you want to customize that, set rest_retries and rest_retry_interval (in seconds).

Contributing

Preparing the environment

This project uses Poetry for packaging and dependency management. Make sure you are able to create the poetry shell with relevant dependencies.

You will also need a database on Upstash.

Running tests

To run all the tests, make sure the poetry virtual environment activated with all the necessary dependencies. Set the UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN environment variables and run:

poetry run pytest

redis-py's People

Contributors

burak-upstash avatar cahidarda avatar enesakar avatar fahreddinozcan avatar mdumandag avatar tudorzgimbau avatar ytkimirti avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

redis-py's Issues

Include a timeout for `session.post` in `sync_execute`

In sync_execute, commands are executed by making requests to the Upstash REST API like this...

response = session.post(url, headers=headers, json=command).json()

...where session is a requests.Session instance.

As indicated by the requests documentation, this call should include a timeout to avoid blocking for long periods in the event of networking trouble:

Nearly all production code should use this parameter [timeout] in nearly all requests. Failure to do so can cause your program to hang indefinitely

For the Redis workloads I typically have, no response from Redis is better than a slow response from Redis. The client should at least allow users to configure a timeout for these requests, and in my opinion should also have a default timeout enabled without user intervention.

tests

The current test suite is not enough. There are lots of commands which we don't have even a single test for. And, in my local environment, most of the tests are failing. We should fix this, and improve the test suite.

Python version

  • The minimum supported Python version of 3.11 is too recent. Most of the ecosystem have support for Python 3.8 and even for 3.7. Given that we are very close to the end-of-life of Python 3.7, I believe we can set to minimum Python version to 3.8.

naming

  • The implementation of the library currently deviates from the Redis command definitions on the parameter names. For example, we currently have the following API:

        async def set(
            self,
            key: str,
            value: Any,
            nx: bool = False,
            xx: bool = False,
            get: bool = False,
            seconds: int | None = None,
            milliseconds: int | None = None,
            unix_time_seconds: int | None = None,
            unix_time_milliseconds: int | None = None,
            keep_ttl: bool = False,
        ) -> str | None: ...

    However, the command definition on the Redis is as follows:

    SET key value [NX | XX] [GET] [EX seconds | PX milliseconds |
      EXAT unix-time-seconds | PXAT unix-time-milliseconds | KEEPTTL]
    

    IMO we should follow the command definitions. I gave an example for this specific command, but there might be other deviations.

Some comments on the current state of the library

Here are my initial views on the current state of the library

  • I believe the name of the library should be changed from upstash_py to upstash_redis. The py suffix does not add any value, and we have other products, so giving a more descriptive name with redis suffix would be better.
  • The minimum supported Python version of 3.11 is too recent. Most of the ecosystem have support for Python 3.8 and even for 3.7. Given that we are very close to the end-of-life of Python 3.7, I believe we can set to minimum Python version to 3.8.
  • Asyncio is not dominant yet. Providing an API only for it might hurt the adoption of the library. I think we should do what most of the other libraries do, and provide sync + async APIs on the same package, and the default implementation should have sync APIs: Basically, upstash_redis.Redis and upstash_redis.aio.Redis, which provides the sync and async versions of the same API.
  • The implementation of the library currently deviates from the Redis command definitions on the parameter names. For example, we currently have the following API:
        async def set(
            self,
            key: str,
            value: Any,
            nx: bool = False,
            xx: bool = False,
            get: bool = False,
            seconds: int | None = None,
            milliseconds: int | None = None,
            unix_time_seconds: int | None = None,
            unix_time_milliseconds: int | None = None,
            keep_ttl: bool = False,
        ) -> str | None: ...
    However, the command definition on the Redis is as follows:
    SET key value [NX | XX] [GET] [EX seconds | PX milliseconds |
      EXAT unix-time-seconds | PXAT unix-time-milliseconds | KEEPTTL]
    
    IMO we should follow the command definitions. I gave an example for this specific command, but there might be other deviations.
  • The current test suite is not enough. There are lots of commands which we don't have even a single test for. And, in my local environment, most of the tests are failing. We should fix this, and improve the test suite.
  • Current type hints have some problems. Running mypy shows 33 errors. In the strict mode, the error count is currently at 219.
  • We don't have any CI/CD set. We should make sure that the tests are passing in Github Actions.
  • This one is optional, but we should be using static analysis and linter tools to have a better codebase. A popular stack is: mypy, black, ruff, and isort. We can make use of these tools.
  • We should re-export our public APIs in __init__.py so that what is public could be detemined easily and the public APIs could be used easier.

asyncio

  • Asyncio is not dominant yet. Providing an API only for it might hurt the adoption of the library. I think we should do what most of the other libraries do, and provide sync + async APIs on the same package, and the default implementation should have sync APIs: Basically, upstash_redis.Redis and upstash_redis.aio.Redis, which provides the sync and async versions of the same API.

reexport __init__.py

We should re-export our public APIs in init.py so that what is public could be detemined easily and the public APIs could be used easier.

package name

I believe the name of the library should be changed from upstash_py to upstash_redis. The py suffix does not add any value, and we have other products, so giving a more descriptive name with redis suffix would be better.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.