Give an error if a class extends two classes about mypy

[JukkaL]

@datnamer Obviously this depends on the specifics of the compiler. Mypy still has some of the hooks I added originally for creating a compiler, but these were never quite complete. I suspect that the type information is almost -- but not quite -- enough for a compiler, at least for some value of 'compiler' :-)

I'm not opposed to somebody tweaking mypy to better support this use case, assuming that it doesn't make the code significantly messier or more difficult to maintain for pure type checking needs, but this would have to be considered on a case by case basis.

[JukkaL]

PEP 484 types clearly aren't optimal for compilers, but reinterpreting them as something closer to Cython types should be possible. For example, if something is annotated as int, the compiled code could only accept int values (and subclasses) and it could directly access the internals of the objects and store the integer value in an internal, more efficient representation. Other changes to Python/PEP 484 semantics would likely be helpful for significant performance gains -- again Cython is an interesting precedent. This wouldn't be a Python compiler any more but a Python variant, but a Cython-like approach would make it possible to run together with regular Python code in the same runtime.

Fixed-width integer types is one big thing missing from PEP 484 but it should be possible to provide type aliases such as Int32 (that PEP 484 compliant implementations would just treat these as aliases for int) that would result in fixed-width integers in compiled code.

There are a ton of details that I'm leaving out and I'm not suggesting that this would be easy (or necessarily even possible) but I think it might be feasible if we don't insist on 100% compatibility with Python semantics. Performance would likely be worse than Cython for a lot of stuff, but I think that's okay if it would still be significantly faster than CPython for an interesting set of programs.

I haven't thought about web assembly much -- can't really help there, sorry.

[gvanrossum]

Please move this discussion to a new issue with a more appropriate subject. This issue has been closed since 2013 and it keeps receiving new comments that are not related to the subject. Please stop that!

(Also, there isn't any mention of mypy in the doc you linked and I found only one mention in the gitter -- if you want us to pay attention please summarize what you want from the mypy project. Finally, if this is about PEP 484, please use the tracker at https://github.com/python/typing/

[Varriount]

Why should this be an error? Regular python allows for multiple extension.

[JukkaL]

The JVM does not have multiple implementation inheritance, and we want to be able to support efficient JVM back ends (in the future). Mypy can support multiple inheritance of interfaces, though. As interfaces can have method implementations, you get much of the flexibility of Python multiple inheritance.

Also, CPython does not support multiple C extension classes as bases; in some respects mypy classes act like CPython extension classes.

[ashleyh]

Could you give more details on the inheritance model? Some questions:

If an interface can have method implementations, what is the difference from a(n abstract) class? No constructor or fields?

How would that be implemented in a JVM backend? While it supports implementing multiple interfaces, I was under the impression that Java interfaces can contain no code at all.

How do we handle the case that we want to implement two interfaces which share a method name and each provide an implementation? Scala handles this by a fairly complicated linearisation, but I believe Ceylon simply prohibits this situation (except possibly for the 'diamond' case, I can't remember). Personally, I would try to avoid the linearisation.

[JukkaL]

An interface cannot have a constructor. An interface can have fields, but they are only declared; a class has to implement the fields explicitly (though fields in interfaces are not supported currently). For example:

interface I:
    int x

class A(I):
    int x # required; otherwise field is not implemented

Fields are similar to C# properties by default; a subclass can override a field as a property. There will probably be 'final' fields as well which cannot be overridden.

In the JVM, default implementations can be supported by copying the default method implementations to the class. This leads to some code duplication, but there seems to be no way around it.

If two implemented interfaces share a method with a default implementation, the class must implement the method explicitly. This is most likely a rare event for most programmers, so it makes sense to require the programmer to handle it explicitly.

Finally, methods inherited from a base class take precedence over interface methods with default implementations.

Note that not everything works quite correctly in the current implementation. Feel free to add issues related to any deficiencies.

[JukkaL]

On second thought, there may be a better way of supporting default method implementations on the JVM. They could be implemented as static methods (with an explicit this/self argument), and the method in the class could simply be a wrapper that calls the static method.

Right now I can't see why this wouldn't work. This would make it possible to update default method implementations in a library without having to recompile all programs that use the library. Also, this would fix the code duplication issue.

[ashleyh]

That is exactly how scala implements traits :)

On 24 January 2013 21:16, Jukka Lehtosalo [email protected] wrote:

On second thought, there may be a better way of supporting default method
implementations on the JVM. They could be implemented as static methods
(with an explicit this/self argument), and the method in the class could
simply be a wrapper that calls the static method.

Right now I can't see why this wouldn't work. This would make it possible
to update default method implementations in a library without having to
recompile all programs that use the library. Also, this would fix the code
duplication issue.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/14#issuecomment-12673923.

[ztane]

Note that the JVM couldn't still support you having the properties in the interface definition, and the X.prop and Y.prop would be incompatible on JVM if the prop wouldnt be declared in the same base class. Thus for pretty much every platform I think having an "int x" in the interface it would essentially mean on all target languages (except Python proper) that these would be accessed through accessor functions.

[JukkaL]

ztane: Data attributes will be accessed through accessor functions on non-Python back ends even if they are not declared in an interface, unless they are final. Final attributes can be accessed directly through a class type, but they will have to be accessed using an accessor through an interface type. So if an attribute is declared in an interface and accessed through the interface type, an accessor function is always used.

However, a clever compiler can often inline calls to accessor functions (even an ahead-of-time compiler can do it much of the time; the JVM JIT compiler can do it more often). There may need to be a type guard, but it's more efficient than calling a virtual method.

[datnamer]

@JukkaL @gvanrossum

Following up on this blogpost, are there any plans for static ahead of time compilation with mypy (if we opt in to restrict our semantics)?

Static typing is great, but the benefits for performance and portability of having slim and fast binaries would be huge, particularly with the ability to potentially run in the browser with web assembly.

This would undermine the need for javascript and put huge interesting into M(almost everyone I talk to would rather program the web in python, but the thought of forcing clients to get a a ~10mb interpreter to serve slower code does not seem particularly inviting ).

https://news.ycombinator.com/item?id=11289345

I'm sure you know this, but this would also allow for first class python development for IOT, mobile, per the WASM vision etc

[refi64]

No; Mypy is now targeting "just" being a static type checker for Python, not an AOT (or any kind of) compiler.

[datnamer]

Gotcha. Does it provide enough type information that a 3rd party compiler can consume to do compilation?

[refi64]

I'm not sure...maybe? I was actually thinking of the same thing a while back!

[datnamer]

Mypy's awesome type system/ inference/ typing/limited semantics/ typed standard library and @kayhayen Nuitka compiler sounds like a match made in heaven.

[JukkaL]

Mypy is not going to do this, but the original mypy plan might still be feasible -- though I'm unlikely to continue with it. However, I've been toying with the related idea of compiling statically typed Python to CPython C extension modules, similar to Cython. It would be a separate project that would use mypy as part of a compiler. It wouldn't really help produce slim binaries (and I won't have time to work on it at least in the near future, so don't get your hopes up yet).

[datnamer]

Thanks @JukkaL . Would the original mypy plan have produced slim binaries?

[JukkaL]

Not sure what slim binaries are exactly, but the original plan would at least have made it possible to generate reasonably small statically linked binaries that don't have any external dependencies, other than maybe libc.

[datnamer]

@JukkaL - Yes, thats what I meant.

Thanks. Is it possible for an external compiler to consume type information from Mypy, without current mods to mypy, in pursuit for this original plan? Or would it need major mypy surgery.

[datnamer]

@njsmith You might be interested in the tail end of this discussion as it related to the issue of utilizing libraries within JIT compilers like Numba without rewriting all the functionality. I totally agree that we need a better alternative than rewriting all our libraries for JIT compilers.

Perhaps Mypy or type hint annotated code can be consumed by Numba or Nuitka etc.

This and datashape for arrays and maybe one of the FAT python series of peps (like function guard or opt in restrict semantics) can get us there. http://faster-cpython.readthedocs.org/fat_python.html

Edit: There is also this pep for adding a JIT endpoint to cpython https://github.com/Microsoft/Pyjion/blob/master/pep.rst

[njsmith]

Unfortunately, it's not clear to me how a compiler can get much mileage out of PEP 484 types -- the problem is that the constraints they expression (IIUC) generally don't actually constrain behavior. So e.g. saying that x : List[Int] implies that isisintance(x, list) and isinstance(x[0], int), but it tells you nothing about the semantics of calling any particular method on x or x[0] -- subclasses can have arbitrarily different behavior than superclasses. This is why the PEP 484 types and Cython types are mostly orthogonal. Some relevant excerpts from the PEP 484 discussion (there's more, this is just a few I found on a quick search...):

• https://mail.python.org/pipermail/python-ideas/2015-January/031109.html
• https://mail.python.org/pipermail/python-ideas/2015-January/031121.html

That said, the idea of defining a python-plus-something-sorta-like-cython's-type-annotations and using it as an import format for Numba/Nuitka/PyPy/etc. is high on my list of things that seem like they're worth thinking about :-).

(For those wondering what the context here is: see https://python-compilers-workshop.github.io/ , esp. the end of the "Motivation" section)

[datnamer]

@njsmith I think mypy's behavior is constrained, though I could be wrong.

Are you thinking about this for any arbitrary python module or only ones with numeric data structures and algorithms?

I would love to be able to run compiled python on the web when web assembly compilation is more mature. Do you think that is a pipe dream?

[datnamer]

@JukkaL There was some discussion of using a Mypy with strict mode to compile to a python IR to be consumed by JIT compilers.

It seems there were some reservations about mypy semantics etc

If you are interested, the notes are here: https://docs.google.com/document/d/1jGksgI96LdYQODa9Fca7EttFEGQfNODphVmbCX0DD1k/edit#heading=h.v4f52j6z2px6

and mpyp was mentioned here in the gitter: https://gitter.im/python-compilers-workshop/chat

I personally would rather not wanted to see further fragmentation in the community by standardizing on something besides type hints (also cython doesn't have generics).

[JukkaL]

Agreed that isn't the best place for this discussion. (But thanks for the links -- unfortunately I couldn't make it to the workshop.)