handle from __future__ import annotations / ForwardRefs better about typedload

[ltworf]

#296

But it doesn't work with attrs, so I guess it could use some more work.

[ltworf]

But it looks like if you use get_type_hints you won't get a problem either way

lol :)

The original one loses the context, so for anything that is not module level, it is impossible later on to go and "eval" because the context is simply lost forever and there is no way to get it back.

Which by the way implies that most of my test suite fails irreparably when that thing is enabled.

Which is why I think it's a terrible idea, since it's really really really not backwards compatible and there are no workarounds to make it compatible, except for putting everything top-level and manually fixing all the inevitable name collisions.

[ltworf]

True, but it's api breaking and requires a version bump, and since nobody has complained about this so far I think it's best to keep things as they are.

[ltworf]

What version of python are you using?

This thing was postponed and they were talking about redesigning it.

[phiresky]

python 3.10.

I need to use from __future__ import annotations to prevent circular imports (together with if TYPE_CHECKING) as well as improve startup speeds. So far, the following fix seems to work for me:

def _forwardrefload(l: Loader, value: Any, type_: type) -> Any:
    """
    This resolves a ForwardRef.
    """
    if l.frefs is None:
        raise TypedloadException("ForwardRef resolving is disabled for the loader", value=value, type_=type_)
    tname = type_.__forward_arg__  # type: ignore
    t = type_._evaluate(None, None, set())
    if t is None:
        raise TypedloadValueError("ForwardRef '%s' unknown" % tname, value=value, type_=type_)
    return l.load(value, t, annotation=Annotation(AnnotationType.FORWARDREF, tname))

[ltworf]

I still think this is the way to go https://peps.python.org/pep-0649/ and they are incompatible, but this one is faster and can work with nested types.

So I can maybe make a temporary flag to enable forwardref stuff, since this other PEP seems to not be implemented, but it will be deprecated from day 1 and unsupported.

[phiresky]

That PEP looks like a good idea. But it looks like if you use get_type_hints you won't get a problem either way:

Code using typing.get_type_hints() would continue to work unchanged, though future use of that function would become optional in most cases.

And I'd guess the same would be done for forward_ref._evaluate() to keep it compatible with the existing behaviour

[phiresky]

Thanks for fixing the issue.

I'm not trying to be a dick here but I don't really understand why using get_type_hints is worse than not using it. You don't need to have pep563 enabled to use get_type_hints. get_type_hints is a no-op for non-forward type refs so everything stays the same except it also resolves most forward refs fine. and it doesn't make anything worse other than that it resolves types that are explicitly forward refs, which you can't "correctly" resolve in any other way either until pep649 happens.

Looking at your test suite, with your new pep563=True flag force-enabled everywhere (but without from future import annotations) only 2 of 138 tests fail:

• test_add_fref: This one fails because you explicitly add a "fake" forward ref that is manually added to frefs. I don't understand why you would want to do this though - it seems better to just fix the type resolution by moving the type to not need a forward ref instead of having to remember to manually add it to the frefs array

• test_known_refs: This one only fails because I see you add your own "hacky" handling for forward refs:

  typedload/typedload/dataloader.py

  Lines 275 to 279 in 5e7ebfd

  	# Add type to known types, to resolve ForwardRef later on
  	if self.frefs is not None and hasattr(type_, '__name__'):
  	typename = type_.__name__
  	if typename not in self.frefs:
  	self.frefs[typename] = type_

  
  That honestly doesn't really seem much better than what Pep563 does at all since it also interprets local definitions as completely global, which is very easy to break:

   def test_collision(self):
       from typedload import dataloader
  
       class Tree1:
           class Node(NamedTuple):
               a: int
               next: Optional["Node"] = None
  
       class Tree2:
           class Node(NamedTuple):
               b: int
               next: Optional["Node"] = None
  
       loader = dataloader.Loader()
       print(loader.load({"a": 1, "next": {"a": 1}}, Tree1.Node))  # works
       print(loader.load({"b": 1, "next": {"b": 2}}, Tree2.Node))  # breaks unexpectedly, tries to construct the wrong Node type

  So really you get the same problem as you described pep563 has above:

  [...] putting everything top-level and manually fixing all the inevitable name collisions.

  At least you should probably use type_.__module__ + "." + type_.__qualname__ instead of type_.__name__ in that code?