UnSecureDataError when reading JSON value about cashews

[rkrell]

Oh, great, haven't even noticed that possibility :-)
The community may forgive you the breaking change ;-)

[AIGeneratedUsername]

Maybe all signature checks must be skipped if a user did not provide a hash_key.

[rkrell]

What is the background here with this hash key? I couldn't find any hint in the documentation, except the mentioning in https://github.com/Krukov/cashews#redis:
This backend uses pickle module to store values, but the cashes can store values with sha1-keyed hash. Use hash_key parameter to protect your application from security vulnerabilities.

If it is necessary, how should I use it? Why is the value split on '_' chars, is this a standard?

The point here is that the key is provided to Redis by a different system not using cashews.

[rkrell]

I found the basic meaning in serializer.py, but isn't that a proprietary approach to add a prefix separated by '_' to the user's value?

The main disadvantage is that interprocess communication will not be compatible with software not using exactly the same rule.

[AIGeneratedUsername]

Are you trying to use 'cashews' instead of 'aioredis' directly only to have a Pickle serializer to pickle/unpickle your data when working with Redis? Just trying to understand your use case when you already have some data set without 'cashews', but then you get this data back with 'cashews' and not with 'aioredis' directly.

I know this is not an answer to your question, but in my own project I use 'aioredis' directly with my own serializer:

class PickleSerializer(_Base):
    @staticmethod
    async def loads(value: bytes | None, key: AnyKeyT, *, default: Any = None) -> Any:
        if value is None:
            return None
        if value == default:
            return default
        if value.isdigit():
            return int(value)
        with suppress(zlib.error):  # The error indicates that the value is shorter than `compress_min_size_bytes`.
            value = zlib.decompress(value)
        unpickled = pickle.loads(value, fix_imports=False, encoding="bytes")
        try:
            repr(unpickled)  # https://github.com/Krukov/cashews/issues/26#issuecomment-1030613323
        except AttributeError:
            await super().delete(key)  # type: ignore[misc]
            return default
        return unpickled

    @staticmethod
    def dumps(value: Any) -> bytes:
        compress_min_size_bytes = 100  # Compression smaller objects may give either the same or even larger size.
        dumped = pickle.dumps(value, protocol=pickle.HIGHEST_PROTOCOL, fix_imports=False)
        return zlib.compress(dumped) if sys.getsizeof(dumped) > compress_min_size_bytes else dumped

Sorry, I do not have time now to investigate 'cashews' source code. Perhaps Krukov will take a look soon.

[rkrell]

Are you trying to use 'cashews' instead of 'aioredis' directly only to have a Pickle serializer to pickle/unpickle your data when working with Redis? Just trying to understand your use case when you already have some data set without 'cashews', but then you get this data back with 'cashews' and not with 'aioredis' directly.

...

The data are written by a completely different system to Redis, using redis-py. Redis is used for inter-process communication between these two systems. It cannot be expected the "foreign" system adds a hash key as expected by cashews.

To simulate the problem I just use cashews as shown in the simple example.
What I wanted is just unpickling the value back to a string as saved in Redis.

Generally, cashews should not have any "opinion" about the value itself by default. At the moment, it checks for a '_' char in the value by default, which means a '_' could not be used in a cached value without getting in conflict with this parsing, am I wrong? The value I use is a valid value for Redis.

[rkrell]

Just to show how the Redis message is created by a non-cashews system, not using asyncio:

        redis_message = SomePydanticObjectBasedOnBaseModel(
             ...
        )
        redis_key = '{}:product_scan_{}'.format("packing", barcode)
        redis_expiration = timedelta(days=7)

        redis_client.setex(redis_key, redis_expiration,
                           redis_message.json())

The project uses redis==3.5.3.

[rkrell]

Nevertheless, if cashews is actually mentioned not to be used for interoperation between different implementations due to a special pickling/unpickling it's easy to use plain aioredis for my purpose.

Pydantic's BaseModel.parse_raw does the job here and establishes an object from the encoded JSON without any custom unpickler.

[Krukov]

@rkrell

With a shame I can say that it is possible to get value without any serialization. I did very silly mistake with naming (english is not native for me, I don't know how it can happened) - set_row , get_row. It should be raw of course

await cache.get_row("test")

So I going to fix this naming without backward compatibility ( for versions 4.x.x)