Incompatibility with cattrs 23.2, application/json response contents, and `decode_content=True` about requests-cache

[JWCook]

Okay, I think I understand what's happening:

• cattrs: JsonConverter now uses the new function configure_union_passthrough() here to register a Union type containing all JSON primitive types.
  • This is to handle results from a JSON deserializer that produces values with these types. Each value will be structured using its individual type (say, a float) instead of treating the Union itself as a special type.
• requests-cache: CachedResponse._decoded_content is annotated with DecodedContent, a Union type alias for decoded content from a response with either application/json or text/*.
• requests-cache: the converter used by CattrStage registers a structure hook for DecodedContent.
  • This is handled specially to allow being unstructured into a fully human-readable JSON (or YAML, etc.) file instead of a binary blob, mainly for use with the Filesystem, MongoDB, and DynamoDB backends.
• cattrs: The "union passthrough" strategy kicks in before this hook has a chance to run.
  • It handles JSON primitive types, but not JSON document root types (dict and list).
  • The diff between the registered Union and the DecodedContent Union becomes Union[List, Dict].
  • It treats this as a "spillover type" here, and looks for a converter for Union[List, Dict], which doesn't exist, raising an error.

This may be more of a problem with how I'm using cattrs; maybe modifying/extending the pre-configured converters isn't an intended use case? The docs for the preconf module do say they're "specifically adjusted for particular serialization libraries" (in this case json/ujson), i.e. they expect output directly from one of those libraries.

This did work prior to 23.2, though.

[JWCook]

So the workaround I mentioned above turns out to be a reasonable solution for this.

I think a better long-term solution would be to implement support for RFC 7159 (#868), and handle the logic for the decode_content option after the cattrs converter runs, instead of as part of the converter.

[JWCook]

Fixed in main, and patched in v1.1.1.