ceylon / ceylon-js

DEPRECATED

Ceylon

This is the 1.3.4-SNAPSHOT "You'll Thank Me Later" release of the Ceylon command line tools. This is a production version of the platform.

Ceylon is a modern, modular, statically typed programming language for the Java and JavaScript virtual machines. The language features a flexible and very readable syntax, a unique and uncommonly elegant static type system, a powerful module architecture, and excellent tooling, including an awesome Eclipse-based IDE.

Ceylon enables the development of cross-platform modules which execute portably in both virtual machine environments. Alternatively, a Ceylon module may target one or the other platform, in which case it may interoperate with native code written for that platform.

Read more about Ceylon at http://ceylon-lang.org.

Distribution layout

• cmr - Ceylon Module Resolver module
• common - Common code module
• compiler-java - JVM compiler module
• compiler-js - JS compiler module
• dist - Build files
• language - Ceylon language module
• model - Type model module
• runtime - Runtime module
• typechecker - Typechecker module
• langtools-classfile - Java tools classfile module fork
• tool-provider - Ceylon tool provider module
• LICENSE-ASL - The Ceylon ASL license
• LICENSE-GPL-CP - The Ceylon GPL/CP license
• LICENSE-LGPL - The Ceylon LGPL license
• README.md - This file

Building the distribution

Go to the dist folder and follow the instructions in the BUILD.md file.

Source code

Source code is available from GitHub:

http://github.com/ceylon

Issues

Bugs and suggestions may be reported in GitHub's issue tracker.

http://github.com/ceylon/ceylon/issues

Systems where Ceylon is known to work

Since Ceylon is running on the JVM it should work on every platform that supports a Java 7 or 8 compatible JVM. However we have tested the following platforms to make sure it works:

Linux

• Ubuntu "wily" 15.10 (64 bit) JDK 1.7.0_95 (IcedTea) Node 0.10.25
• Fedora 23 (64 bit), JDK 1.8.0_77 (OpenJDK)
• Fedora 22 (64 bit), JDK 1.8.0_72 (OpenJDK)
• Fedora 22 (64 bit), JDK 1.7.0_71 (Oracle)

Windows

• Windows 10 Home (64 bit) 1.8.0_77
• Windows 7 (64 bit) 1.7.0_05 (Oracle)
• Windows Server 2008 R2 SP1 JDK 1.7.0_04

OSX

• OSX 10 Lion (10.8.5) JDK 1.7.0_40 (Oracle) Node 0.10.17
• OSX 11 El Capitan (10.11.6) JDK 1.7.0_80 (Oracle) Node 0.10.35

License

The Ceylon distribution is and contains work released

• partly under the ASL v2.0 as provided in the LICENSE-ASL file that accompanied this code, and
• partly under the GPL v2 + Classpath Exception as provided in the LICENSE-GPL-CP file that accompanied this code.

License terms for 3rd Party Works

This software uses a number of other works, the license terms of which are documented in the NOTICE file that accompanied this code.

Repository

The content of this code repository, available here on GitHub, is released under the ASL v2.0 as provided in the LICENSE-ASL file that accompanied this code.

By submitting a "pull request" or otherwise contributing to this repository, you agree to license your contribution under the license mentioned above.

Acknowledgement

We're deeply indebted to the community volunteers who contributed a substantial part of the current Ceylon codebase, working often in their own spare time.

Ceylon is a project of the Eclipse Foundation.

ceylon-js's Issues

Inner type with same name as outer type

As mentioned in #41, there are problems with nested types when inner types have the same name as the containing type. For instance, I tested with the following code:

class NameTest() {
    shared Integer x = 1; 
    shared class NameTest() {
        shared Integer x = 2;
    }
    shared Integer f() { return this.NameTest().x; }
}
object nameTest {
    shared Integer x = 1; 
    shared object nameTest {
        shared Integer x = 2;
    }
    shared Integer f() { return this.nameTest.x; }
}

The class seems to work in prototype style but in closure style calling f() causes a runtime error. The object definition even causes a runtime error immediately while loading the package, in both styles.

And of course there are much more complex examples. For instance it looks like you are allowed to write something like this:

class C1() {
    shared class C1() {}
    shared class C3() extends C1() {}
}
class C2() extends C1() {
    shared class C2() extends super.C1() {
        class C2() extends C3() {}
    } 
}

We should make the generated code more robust against name conflicts and define additional tests.
Note that this is somewhat related to #18.

Update to new Spec

Equality is gone, hash and equals moved to Object.

case (is...) is broken

This currently doesn't work:

class C(Integer? x) {
    switch (x)
    case (is Integer) {print(x.string);}
    else {}
}

Here is the relevant part of the resulting JS code (capture style):

function C(x, $$c){
    if ($$c===undefined)$$c=new C.$$;
    //Switch
    var tmpvar$1=x;
    if ($$$cl15.isOfType(tmpvar$1,'ceylon.language.Integer')===$$$cl15.getTrue()) {
        $$$cl15.print(getX().getString());
    }
    else {}

    return $$c;
}

That fails because getX does not exist. The problem is that case (is...) introduces a new local variable x which shadows the outer x. The code inside the case block expects a getter for that variable, but we currently don't define any. Actually we have already solved this same problem for if (is...)/while (is...) so maybe we can reuse some of that logic.

super refers to the wrong object in some situations

The following test demonstrates the problem:

class Type1() {
    shared default String test() { return "1"; }
}
class Type2() extends Type1() {
    shared actual default String test() {
        return super.test() + "2";
    } 
}
class Type3() extends Type2() {
    shared actual default String test() {
        return super.test() + "3";
    } 
}
void run() {
    value tst = Type3();
    print(tst.test());
}

This results in an endless loop because the call super.test() in Type2.test calls Type2.test again instead of Type1.test.

Sequenced parameters undefined instead of empty

Functions declared as f(Type...) can be called f() but in js this gets translated exactly to a call as f() when it really should be f(empty) so that the function receives an empty Sequence, otherwise errors occur when trying to use the sequenced parameter because it's undefined.

Reified types

This is currently a big gap in the JS backend. To completely and correctly support reified types we need the cross-platform reified types implementation (see https://groups.google.com/group/ceylon-dev/browse_thread/thread/67ab29438d330bad).
But maybe we can already implement something simple for the straight-forward cases. This way we could provide a preliminary version of if (is...) and maybe try...catch.

Object, IdentifiableObject and Equality

Instead of just having the global CeylonObject, the definitions of Object, IdentifiableObject and Equality should be correctly implemented in the language module.

Overcomplicating the node module?

The ceylon.language module is define like this in js:

(function (define) {
    define('ceylon.language', function (require, exports) {

 //the Ceylon language module
...
exports.bla=bla;
});
}(typeof define==='function' && define.amd ? 
    define : function (id, factory) {
    if (typeof exports!=='undefined') {
        factory(require, exports);
    } else {
        throw "no module loader";
    }
}));

What is all that stuff for? I was doing some tests for issue #36 and it turns out that if I remove all that stuff and just leave the code between "the Ceylon language module" up to the last exports.bla, it works perfectly in node.

Turns out this is the CommonJS format as described here http://wiki.commonjs.org/wiki/Modules/1.1.1#Module_Context

And to make it work in a browser, using require.js it's just a matter of adding a couple of lines of code:

//this as first line of ceylon.language.js
define(function (require, exports, module) {
...
//this as the last line of the file
});

So... what is the idea behind all that extra stuff? Looks like the ability to just throw an exception if there is no module loader... but if there is no module loader, well loading this file will just result in an exception when it hits the first assign to exports.

I haven't pushed any of these changes to the repo because there might be a good reason I'm not aware of. But if having it work with node.js is OK and it leaves the file prepared for require.js (just need to add two lines) is more desirable, then I guess we should just remove all that unnecessary encapsulation...

Missing getters for constructor parameters

This Ceylon class:

class C(Float f) satisfies Comparable<C> {
    shared actual Comparison compare(C other) {
        return f<=>other.f;
    }
    shared actual String string = "C(" f.string ")";
}

turns into this js code:

function $C(){}
$C.T$all={'C':$C}
for($ in $$$cl15.$IdentifiableObject.T$all){$C.T$all[$]=$$$cl15.$IdentifiableObject.T$all[$]}
for($ in $$$cl15.$Comparable.T$all){$C.T$all[$]=$$$cl15.$Comparable.T$all[$]}
for(var $ in CeylonObject.prototype){
    var $m=CeylonObject.prototype[$];
    $C.prototype[$]=$m;
    if($.charAt($.length-1)!=='$'){$C.prototype[$+'$CeylonObject$']=$m}
}
for(var $ in $$$cl15.$Comparable.prototype){
    var $m=$$$cl15.$Comparable.prototype[$];
    $C.prototype[$]=$m;
    if($.charAt($.length-1)!=='$'){$C.prototype[$+'$$$cl15.$Comparable$']=$m}
}
function C(f, $$c){
    if ($$c===undefined)$$c=new $C;
    $$$cl15.Comparable($$c);

    //MethodDefinition compare at comparables.ceylon (2:4-4:4)
    function compare(other){
        return f.compare(other.f);
    }
    $$c.compare=compare;

    //AttributeDeclaration string at comparables.ceylon (5:4-5:51)
    var $string=$$$cl15.StringBuilder().appendAll($$$cl15.ArraySequence([$$$cl15.String("C(",2),f.getString().getString(),$$$cl15.String(")",1)])).getString();
    function getString(){
        return $string;
    }
    $$c.getString=getString;
    return $$c;
}

So the compare method fails because other.f is undefined. I think the class definition is missing the getF() and of course assigning f as a property to $$c.

for loops don't return properly

This Ceylon code:

Integer x() {
  for (i in 1..10) {
    if (i==5) return i;
  }
return 0;
}

when compiled to JS, always returns 0. This is because the for loop is enclosed in an anonymous function which is executed immediately, but this has the side-effect of ignoring the return inside the loop.

Error in Range.span if range is out of bounds

The code (1..2).span(4, 5) causes an error at runtime: TypeError: Cannot call method 'compare' of null.

Package and module descriptors

Support package and module descriptors.

Split ceylon.language.js

The ceylon.language.js file is becoming unmanageable. Splitting it into several files will make it easier to handle and reduce conflicts in commits.

File-per-class might be a little too much, but we can take out some of the stuff that isn't used by the language module itself, only exported.

Make compiler executable

We need a script to call the compiler as an executable, similar to ceylonc, which runs a Java program that calls the typechecker and then the JsCompiler.

Complete String and Character classes

Check what's missing for the string-related classes String, Character and StringBuilder, and finish them. Some parts, like determining if a char is a Unicode titlecase or letter character, may be a bit tricky to achieve because there is only limited Unicode support in JavaScript.

Local getters get overwritten

This Ceylon code is valid:

void imprimeNumero(Integer|Float numero) {
    if (is Integer numero) {
        print("Entero: " numero "");
    } else if (is Float numero) {
        print("Flotante: " numero "");
    }
    switch (numero)
    case (is Integer) { print("es entero "); }
    case (is Float) { print("es flotante "); }
}

However the generated js has problems: There is a function getNumero() inside each of the ifs and also inside each case. The getNumero() inside the cases references a tmpvar created for the switch; the fourth getNumero is called in all cases, and since the tmpvar doesn't exist inside the first if, it returns undefined.

A temporary fix is to implement the local getter in case(is) just like for if (is) but it's not a very good solution. This is a problem with scopes in js which can occur in certain situations; perhaps a solution would be to put the local getter functions inside a tmp object or to be able to directly reference the tmpvar created instead (and use that pattern also for if (is)).

Range.contains doesn't check type

Range.contains(x) needs to check if the type of x is the type specified for the Range (I think reified generics are needed for this)

Implement nullsafe invoke

This should work:

class C() {
    shared Integer f() {return 1;}
}
void test() {
    C? obj = null;
    Integer? i = obj?.f();  
}

Collection-related interfaces

Define the collection-related interfaces and helper classes in the language module implementation (a few of them already exist). This includes Category, Cloneable, Container, Correspondence, Empty, Entry, Finished, Iterable, Iterator, Ordered, Range, Ranged, Sequence, SequenceBuilder, Singleton, Sized, plus the types currently being added by Gavin: https://groups.google.com/group/ceylon-dev/browse_thread/thread/377bcf6484caa776.

Simplify method calls

With the latest change to allow for higher-class functions, many method calls are now being made in an unnecessarily convoluted way: a.b(c) in many cases is being generated with the JsCallable mechanism instead of just a.b(c), degrading performance.

nonempty breaks with null

ceylon.language.js has a nonempty function, however it not being used anymore but it should; this code:

void x(String? p) {
    if (nonempty p) {
        print("p es " p "");
    }
}

generates if (p.getEmpty()===$$$cl15.getFalse()) when it should generate if ($$$cl15.nonempty(f)===$$$cl15.getTrue), because nonempty checks that the value is not null and then that it is not empty. Doing just p.getEmpty() breaks when passed a null (which is valid because it's optional type).

Safe index operation does not work

The safe index operation ?[] currently fails at runtime.

Initialization of shared non-variable attribute

This currently doesn't work in capture style:

class A() {
    shared String z;
    z = "ok";
}

Here is the relevant part of the generated code:

var $z;
function getZ(){
    return $z;
}
$$a.getZ=getZ;
$$a.$z=$$$cl15.String("ok",2)

The value has to be assigned to $z, not $$a.$z.

Unified language module test suite

Instead of having our own set of tests for the language module, we should test against the ones already in ceylon.language and in any case complement them. This will avoid redundancy and both compilers will benefit from having one robust set of tests.

Prototype style vs. closure style in ceylon.language

We can generate either prototype style or closure style JS code from Ceylon code, but we only have one version of the language module implementation. That's particularly problematic for classes or interfaces which are derived/implemented by Ceylon code outside the language module because inheriting and overriding methods works differently for the two styles.

We should figure out how to deal with this before implementing larger parts of the language module.

Decimal and Whole

We don't have BigDecimal and BigInteger as in Java. So we need a JS implementation if we want to support these types.

method refs with spread

This is not working:

String[] full = { "hello", "world" };
Character?[] spread(Integer x) = full[].item;

spread(1) should return {e,o} but instead returns world. This is because the expression full[].item is not being correctly evaluated (spread op is ignored) and this is what gets generated:

var spread=(function(){var $=getFull();return $$$cl15.JsCallable($, $.item)})();

Roadmap

We really need a roadmap, to know what's been done, what's still missing, etc. Or even just a checklist w/language features and the language module (I think those are the two things that this project needs to address, right? Or am I missing something?)

Simplification of setters used in assignments

The short: change the implementation of setters to return their new value, eg:

setFoo(newfoo) { return foo=newfoo; }

so we can simplify the generation of code for arithmetic assignment operators (like +=) and when using setters in assingments (like a = b = c).

The long: this started out as a mail to @gavinking but I decided to turn it into an issue:

Right now setters are defined like this (Java-ish style):

void setTitle(String title) { }

we do this in Java so attributes are compatible with Java Bean properties which is a really nice feature.
It does make some code more difficult though. Let's take this example in Ceylon :

foo = bar = baz;

Now to make this work we need all kinds of compiler trickery, while if we could make the setter return its new value we could just generate this:

setFoo(setBar(getBaz));

Now in Java we can't because if we would change the setter signature we wouldn't be compatible with the Java Bean spec anymore, so we're stuck with the trickery.

But looking at the generated JavaScript code I see the same thing!
Trickery involving anonymous functions just to get this to work because they've adopted the same limitations as Java has but without any good reason.
Was it done this way because that's what the code translation examples in the spec show?
Anyway, IMHO each backend should be free to implement Ceylon's features in whatever way they want to get the most out of their platform.

Make generated code work in web browsers

I've been trying for a couple of days now to get the generated code to work in a browser using RequireJS but without any luck.
Maybe some of you guys with more JavaScript experience can take a look at it?

http://requirejs.org

There is a special section within RequireJS for Node as well: http://requirejs.org/docs/node.html

Lexer/Parser errors get filtered out

The visitor used by the compiler to check for existing errors in nodes is filtering out the errors by class, this is why lexer/parser errors are never shown.

Remaining issues with nullsafe invoke

We have a working implementation of nullsafe invoke now, see #22. But there are still some issues with this which we should address:

Method reference (capture style)

This code:

Callable<Void>? f2 = obj?.f;

generates the following JS in capture style:

var $f2=function($){return $===null?$$$cl15.nullsafe:$.f}(getObj());

So if obj is null, f2 will be assigned nullsafe instead of null. The nullsafe member operator should return nullsafe only if it occurs in the context of an invocation expression, otherwise it should return null instead.

Method reference (prototype style)

The same Ceylon code generates the following JS in prototype style:

var $f2=function(){var $=getObj();return $.f.apply($,arguments)};

Here the null check is missing completely.

Simple call statement

This line:

obj?.f();

generates the following line in JS:

function($){return $===null?$$$cl15.nullsafe:$.f}(getObj())();

Unfortunately this appears to be not a valid JS statement. node.js complains about an unexpected token at the first (.

Lost this in prototype style

For the following code:

class C() {
    String s = "hi";
    shared String f() {return s;}
}
...
C? obj = C();
String? s = obj?.f();

the last line becomes:

var $s=function($){return $===null?$$$cl15.nullsafe:$.f}(getObj())();

That looks ok, but doesn't work in prototype style: TypeError: Object #<Object> has no method 'getS$C$'. The problem is that the method gets "disconnected" from the object and doesn't have a proper this when it is executed.

Manage identifier names in generated code

The identifier names in the generated code play an important role in solving problems like #44 and #18 (and probably also #34): we have to give them pre/postfixes to model the scopes of the Ceylon program. To effectively implement a consistent naming scheme for the generated identifiers we should implement a separate class/function in the compiler which is used whenever an identifier name is emitted. This should include identifiers corresponding to members, parameters, local declarations, toplevel declarations and even package names.

Make sure that types in ceylon.language can be refined

Many classes and interfaces in ceylon.language may be extended/implemented by user code. We have to make sure that the way they are defined and exported in ceylon.language.js actually allows that, and add test cases correspondingly. But note that we first have to sort out #11.

Char literals with escape sequences

Char literals with escape sequences like \t currently don't work. I think they are incorrectly interpreted as the character \.

Float.string without decimals

This works OK in the JVM:

(1.0).string == "1.0"

but it fails in js, because Float.getString() returns just "1" instead of "1.0" when there are no decimals.

Multiline strings are not escaped for js

This is valid Ceylon code, taken from the Ceylon tour page:

print("Hello,
       World!");

But it gets translated to something like this:

    $$$cl15.print($$$cl15.String("Hello,
world!");

Which simply doesn't run. We need to escape multiline strings in whatever way js expects them to be (if js supports multiline strings; otherwise just replace newlines with \n).

expanded sequences don't work

if you have a function f(Integer...) (that is, with a sequenced argument), passing a sequence with the "treat the sequence as varargs" syntax doesn't work. Having:

void f(Integer...nums);
Sequence<Integer> s;

then f(s...) should directly pass s as the param to f(); instead of that, the compiler generates code for the equivalent of f({s}) which obviously doesn't work.

Test and fix namespace problems

I am pretty sure that there are still problems with referencing elements in other packages in some situations. In particular, see my comments on #9 and #10 concerning remaining problems. We should add set of tests.

Run tests as individual subsets

Each test subset (directory under test) should be run separately, instead of just calling run.js. This way if a subset breaks, it doesn't stop the whole test suite.

Ceylon scopes vs. JavaScript scopes

Scopes of variables in JavaScript are different than in Ceylon. For instance, consider the following JS code:

function test() {
    var f;
    if (true) {
       var x = 1;
       f = function() {return x}
    }
    if (!false) {
        var x = 2;
    }
    return f();
}

This function returns 2. The reason is that the scope of a variable is always the whole function, not just the enclosing block. This can most likely cause problems with the current implementation of if, while, try/catch and perhaps others.

We should write some test cases to check where such problems exist.

An easy solution would be to wrap everything that may cause trouble in an additional function. But especially for simple ifs that would probably be rather inefficient. If only captured variables cause problems then we could check whether a block actually contains any captured variables, and only wrap it in a function if that is the case.

Call to function with defaulted and sequenced parameter

For the following Ceylon code

void f1(String s="", String... more) {}
void f2() { f1(); }

we generate incorrect JS code for the call f1():

function f2(){
    f1($$$cl15.empty);
}

Implement defaulted parameters

We can probably make use of the fact that JS allows calling functions with any number of arguments, setting the missing ones to undefined. Note that this also has to work properly with named argument method invocation.

Inner types in prototype style

Inner types are currently only partly functional in prototype style. For instance, this doesn't work:

class Outer() {
    class Inner() {
        shared String s = "hello";
    }
    shared String test() { return Inner().s; }
}

In the generated JS code test can't access Outer.Inner() properly because it is defined inside Outer and is not exported. That could be fixed of course. But perhaps the real problem is that we mix prototype style and capture style here: while methods are defined separately and added to the prototype, inner types are defined inside the body of the outer type, creating a closure. It seems that this mixing of concepts is not completely trouble-free.

It would be more consistent, and probably work out better in the long term, to define inner types in the same way as other class members in prototype style. It would also definitely be more efficient because currently inner types are completely initialized every time an instance of the outer class is created.
Implementing this is definitely not uncomplicated either though.

Nested try...catch

Normal try...catch seems to work fine now. But the exceptions test from ceylon.language fails while executing this nested try...catch:

try {
    try {
        throw Exception(null, null);
    }
    catch (MyException me) {
        caught:=true;
    }
}

I'm getting this error:

/home/ivo/src/ceylon2/ceylon-js/build/test/node_modules/default/default.js:413
                function getMe(){return $ex$}
                                 ^
ReferenceError: $ex$ is not defined

Here is the corresponding JS code:

    try{
        try{
            throw $$$cl15.Exception($$$cl15.getNull(),$$$cl15.getNull());
        }
        catch($ex$){
            if($$$cl15.isOfType($ex$,'MyException')===$$$cl15.getTrue()){
                function getMe(){return $ex$}
                setCaught($$$cl15.getTrue());
            }
            else{throw $ex$}
        }

    }
    catch($ex$){
        if($$$cl15.isOfType($ex$,'ceylon.language.Exception')===$$$cl15.getTrue()){}else{throw $ex$}
    }

Interfaces for numeric types

Define the interfaces for numeric types in the language module (a few of them already exist). This includes Float, Integer, Castable, Comparable, Integral, Invertable, Number, Numeric, Ordinal, Summable, Slots and FixedSlots. Some of those interfaces are also relevant for other types like strings or sets.

if (is...) can't handle unions of intersections

This works in Ceylon on the JVM:

interface I1 {}
interface I2 {}
interface I3 {}
interface I4 {}

class C1() satisfies I1&I2{}
class C3() satisfies I3{}

shared void test() {
  C1|C3 c1 = C1();
  if (is I1&I2|I3&I4 c1) {
    print("yay! 1");
  }
}

But not in JS. I modified the "is" function in ceylon.language.js (there are two, actually: one for simple types and another one for union/intersection types). The structure that is generated looks right: { t:'u', l:[{ t:'i', l:['tipos.I1','tipos.I2']},{ t:'i', l:['tipos.I3','tipos.I4']}]} (a union of two intersections), but the algorithm could be flawed, or the C1 type does not have the correct info.

Equality test fails

The equality test from ceylon.language fails

Derive ceylon.language.Boolean from JavaScript Boolean

I just pushed a change (17095f0) which modifies the implementation of ceylon.language.Boolean so that it is derived from JavaScript's Boolean and getTrue/getFalse just return true/false. Because JavaScript does autoboxing for primitive boolean values this means that the following should work:

var s = true.getString();  // "true"
var b = isOfType(false, 'ceylon.language.Boolean'); // true
var b2 = !(2==1).equals(true); // true

So Ceylon Booleans and JavaScript booleans are basically interchangeable. I kept the old code for now (commented out) but as far as I can see nothing has been broken. The idea is that this could allow us to simplify the JS code we write and generate.
What do you think? Am I missing something?

Doing the same for other types would be conceivable but probably more complicated. For instance, ceylon.language.String has methods (replace, split) that have identical names as methods of JavaScript's String class so it would be tricky to unify them.

Private members overwriting each other (prototype version)

With the "prototype" version, the following program prints 2 although it should print 1:

class Type1() {
    Integer i = 1;
    shared void test() { print(i); }
}
class Type2() extends Type1() {
    Integer i = 2;
}
void run() {
    Type2 tst = Type2();
    tst.test();
}

That's because both members i get mapped to a variable this.i of the same JS object.