Comments (25)
@datnamer Obviously this depends on the specifics of the compiler. Mypy still has some of the hooks I added originally for creating a compiler, but these were never quite complete. I suspect that the type information is almost -- but not quite -- enough for a compiler, at least for some value of 'compiler' :-)
I'm not opposed to somebody tweaking mypy to better support this use case, assuming that it doesn't make the code significantly messier or more difficult to maintain for pure type checking needs, but this would have to be considered on a case by case basis.
from mypy.
PEP 484 types clearly aren't optimal for compilers, but reinterpreting them as something closer to Cython types should be possible. For example, if something is annotated as int
, the compiled code could only accept int
values (and subclasses) and it could directly access the internals of the objects and store the integer value in an internal, more efficient representation. Other changes to Python/PEP 484 semantics would likely be helpful for significant performance gains -- again Cython is an interesting precedent. This wouldn't be a Python compiler any more but a Python variant, but a Cython-like approach would make it possible to run together with regular Python code in the same runtime.
Fixed-width integer types is one big thing missing from PEP 484 but it should be possible to provide type aliases such as Int32
(that PEP 484 compliant implementations would just treat these as aliases for int
) that would result in fixed-width integers in compiled code.
There are a ton of details that I'm leaving out and I'm not suggesting that this would be easy (or necessarily even possible) but I think it might be feasible if we don't insist on 100% compatibility with Python semantics. Performance would likely be worse than Cython for a lot of stuff, but I think that's okay if it would still be significantly faster than CPython for an interesting set of programs.
I haven't thought about web assembly much -- can't really help there, sorry.
from mypy.
Please move this discussion to a new issue with a more appropriate subject. This issue has been closed since 2013 and it keeps receiving new comments that are not related to the subject. Please stop that!
(Also, there isn't any mention of mypy in the doc you linked and I found only one mention in the gitter -- if you want us to pay attention please summarize what you want from the mypy project. Finally, if this is about PEP 484, please use the tracker at https://github.com/python/typing/
from mypy.
Why should this be an error? Regular python allows for multiple extension.
from mypy.
The JVM does not have multiple implementation inheritance, and we want to be able to support efficient JVM back ends (in the future). Mypy can support multiple inheritance of interfaces, though. As interfaces can have method implementations, you get much of the flexibility of Python multiple inheritance.
Also, CPython does not support multiple C extension classes as bases; in some respects mypy classes act like CPython extension classes.
from mypy.
Could you give more details on the inheritance model? Some questions:
If an interface can have method implementations, what is the difference from a(n abstract) class? No constructor or fields?
How would that be implemented in a JVM backend? While it supports implementing multiple interfaces, I was under the impression that Java interfaces can contain no code at all.
How do we handle the case that we want to implement two interfaces which share a method name and each provide an implementation? Scala handles this by a fairly complicated linearisation, but I believe Ceylon simply prohibits this situation (except possibly for the 'diamond' case, I can't remember). Personally, I would try to avoid the linearisation.
from mypy.
An interface cannot have a constructor. An interface can have fields, but they are only declared; a class has to implement the fields explicitly (though fields in interfaces are not supported currently). For example:
interface I:
int x
class A(I):
int x # required; otherwise field is not implemented
Fields are similar to C# properties by default; a subclass can override a field as a property. There will probably be 'final' fields as well which cannot be overridden.
In the JVM, default implementations can be supported by copying the default method implementations to the class. This leads to some code duplication, but there seems to be no way around it.
If two implemented interfaces share a method with a default implementation, the class must implement the method explicitly. This is most likely a rare event for most programmers, so it makes sense to require the programmer to handle it explicitly.
Finally, methods inherited from a base class take precedence over interface methods with default implementations.
Note that not everything works quite correctly in the current implementation. Feel free to add issues related to any deficiencies.
from mypy.
On second thought, there may be a better way of supporting default method implementations on the JVM. They could be implemented as static methods (with an explicit this/self argument), and the method in the class could simply be a wrapper that calls the static method.
Right now I can't see why this wouldn't work. This would make it possible to update default method implementations in a library without having to recompile all programs that use the library. Also, this would fix the code duplication issue.
from mypy.
That is exactly how scala implements traits :)
On 24 January 2013 21:16, Jukka Lehtosalo [email protected] wrote:
On second thought, there may be a better way of supporting default method
implementations on the JVM. They could be implemented as static methods
(with an explicit this/self argument), and the method in the class could
simply be a wrapper that calls the static method.Right now I can't see why this wouldn't work. This would make it possible
to update default method implementations in a library without having to
recompile all programs that use the library. Also, this would fix the code
duplication issue.—
Reply to this email directly or view it on GitHubhttps://github.com//issues/14#issuecomment-12673923.
from mypy.
Note that the JVM couldn't still support you having the properties in the interface definition, and the X.prop and Y.prop would be incompatible on JVM if the prop wouldnt be declared in the same base class. Thus for pretty much every platform I think having an "int x" in the interface it would essentially mean on all target languages (except Python proper) that these would be accessed through accessor functions.
from mypy.
ztane: Data attributes will be accessed through accessor functions on non-Python back ends even if they are not declared in an interface, unless they are final. Final attributes can be accessed directly through a class type, but they will have to be accessed using an accessor through an interface type. So if an attribute is declared in an interface and accessed through the interface type, an accessor function is always used.
However, a clever compiler can often inline calls to accessor functions (even an ahead-of-time compiler can do it much of the time; the JVM JIT compiler can do it more often). There may need to be a type guard, but it's more efficient than calling a virtual method.
from mypy.
Following up on this blogpost, are there any plans for static ahead of time compilation with mypy (if we opt in to restrict our semantics)?
Static typing is great, but the benefits for performance and portability of having slim and fast binaries would be huge, particularly with the ability to potentially run in the browser with web assembly.
This would undermine the need for javascript and put huge interesting into M(almost everyone I talk to would rather program the web in python, but the thought of forcing clients to get a a ~10mb interpreter to serve slower code does not seem particularly inviting ).
https://news.ycombinator.com/item?id=11289345
I'm sure you know this, but this would also allow for first class python development for IOT, mobile, per the WASM vision etc
from mypy.
No; Mypy is now targeting "just" being a static type checker for Python, not an AOT (or any kind of) compiler.
from mypy.
Gotcha. Does it provide enough type information that a 3rd party compiler can consume to do compilation?
from mypy.
I'm not sure...maybe? I was actually thinking of the same thing a while back!
from mypy.
Mypy's awesome type system/ inference/ typing/limited semantics/ typed standard library and @kayhayen Nuitka compiler sounds like a match made in heaven.
from mypy.
Mypy is not going to do this, but the original mypy plan might still be feasible -- though I'm unlikely to continue with it. However, I've been toying with the related idea of compiling statically typed Python to CPython C extension modules, similar to Cython. It would be a separate project that would use mypy as part of a compiler. It wouldn't really help produce slim binaries (and I won't have time to work on it at least in the near future, so don't get your hopes up yet).
from mypy.
Thanks @JukkaL . Would the original mypy plan have produced slim binaries?
from mypy.
Not sure what slim binaries are exactly, but the original plan would at least have made it possible to generate reasonably small statically linked binaries that don't have any external dependencies, other than maybe libc.
from mypy.
@JukkaL - Yes, thats what I meant.
Thanks. Is it possible for an external compiler to consume type information from Mypy, without current mods to mypy, in pursuit for this original plan? Or would it need major mypy surgery.
from mypy.
@njsmith You might be interested in the tail end of this discussion as it related to the issue of utilizing libraries within JIT compilers like Numba without rewriting all the functionality. I totally agree that we need a better alternative than rewriting all our libraries for JIT compilers.
Perhaps Mypy or type hint annotated code can be consumed by Numba or Nuitka etc.
This and datashape for arrays and maybe one of the FAT python series of peps (like function guard or opt in restrict semantics) can get us there. http://faster-cpython.readthedocs.org/fat_python.html
Edit: There is also this pep for adding a JIT endpoint to cpython https://github.com/Microsoft/Pyjion/blob/master/pep.rst
from mypy.
Unfortunately, it's not clear to me how a compiler can get much mileage out of PEP 484 types -- the problem is that the constraints they expression (IIUC) generally don't actually constrain behavior. So e.g. saying that x : List[Int]
implies that isisintance(x, list)
and isinstance(x[0], int)
, but it tells you nothing about the semantics of calling any particular method on x
or x[0]
-- subclasses can have arbitrarily different behavior than superclasses. This is why the PEP 484 types and Cython types are mostly orthogonal. Some relevant excerpts from the PEP 484 discussion (there's more, this is just a few I found on a quick search...):
- https://mail.python.org/pipermail/python-ideas/2015-January/031109.html
- https://mail.python.org/pipermail/python-ideas/2015-January/031121.html
That said, the idea of defining a python-plus-something-sorta-like-cython's-type-annotations and using it as an import format for Numba/Nuitka/PyPy/etc. is high on my list of things that seem like they're worth thinking about :-).
(For those wondering what the context here is: see https://python-compilers-workshop.github.io/ , esp. the end of the "Motivation" section)
from mypy.
@njsmith I think mypy's behavior is constrained, though I could be wrong.
Are you thinking about this for any arbitrary python module or only ones with numeric data structures and algorithms?
I would love to be able to run compiled python on the web when web assembly compilation is more mature. Do you think that is a pipe dream?
from mypy.
@JukkaL There was some discussion of using a Mypy with strict mode to compile to a python IR to be consumed by JIT compilers.
It seems there were some reservations about mypy semantics etc
If you are interested, the notes are here: https://docs.google.com/document/d/1jGksgI96LdYQODa9Fca7EttFEGQfNODphVmbCX0DD1k/edit#heading=h.v4f52j6z2px6
and mpyp was mentioned here in the gitter: https://gitter.im/python-compilers-workshop/chat
I personally would rather not wanted to see further fragmentation in the community by standardizing on something besides type hints (also cython doesn't have generics).
from mypy.
Agreed that isn't the best place for this discussion. (But thanks for the links -- unfortunately I couldn't make it to the workshop.)
from mypy.
Related Issues (20)
- Should mypy check existence of module attributes for unannotated modules? HOT 1
- False positive: "overloaded signature will never be matched" with Self
- module name assumed is __main__ for .pyw files on windows
- False positive when using `Self` as the return type with multiple base classes HOT 1
- torch-nn-functional-mypy-bug-when-torchvision-installed
- dmypy crashes from RecursionError for some complex expressions (fastparse) HOT 1
- PEP 695 variance inferred as simultaneously covariant and contravariant HOT 2
- False negative for invalid abstractmethod implementation
- False negative when accessing dataclass instance attribute as class attribute HOT 2
- Basic checking of format string compatibility in f-strings HOT 2
- Mypy errors on code that's trivial syntactic transformation of code that it does not error on
- AssertionError: Must not defer during final iteration
- [good-first-issue] Expose the change log via Sphinx docs
- [maintenance] Upgrade the use of `pip-tools` to leverage its modern features
- support for easy limiting of literals to entrypoint names and/or the entrypoint lookup value
- False Positive when assigning to union type as dictionary key
- Feature Request: Support for Enabling/Disabling Specific Error Codes similar to Pylint
- Passing self to a generic function that accepts Self
- `stubgen --inspect-mode` fails while non-inspect mode works ok
- Another circular import in `mypy/test/testreports.py` HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mypy.