ceylon / ceylon-common

DEPRECATED

Ceylon

This is the 1.3.4-SNAPSHOT "You'll Thank Me Later" release of the Ceylon command line tools. This is a production version of the platform.

Ceylon is a modern, modular, statically typed programming language for the Java and JavaScript virtual machines. The language features a flexible and very readable syntax, a unique and uncommonly elegant static type system, a powerful module architecture, and excellent tooling, including an awesome Eclipse-based IDE.

Ceylon enables the development of cross-platform modules which execute portably in both virtual machine environments. Alternatively, a Ceylon module may target one or the other platform, in which case it may interoperate with native code written for that platform.

Read more about Ceylon at http://ceylon-lang.org.

Distribution layout

• cmr - Ceylon Module Resolver module
• common - Common code module
• compiler-java - JVM compiler module
• compiler-js - JS compiler module
• dist - Build files
• language - Ceylon language module
• model - Type model module
• runtime - Runtime module
• typechecker - Typechecker module
• langtools-classfile - Java tools classfile module fork
• tool-provider - Ceylon tool provider module
• LICENSE-ASL - The Ceylon ASL license
• LICENSE-GPL-CP - The Ceylon GPL/CP license
• LICENSE-LGPL - The Ceylon LGPL license
• README.md - This file

Building the distribution

Go to the dist folder and follow the instructions in the BUILD.md file.

Source code

Source code is available from GitHub:

http://github.com/ceylon

Issues

Bugs and suggestions may be reported in GitHub's issue tracker.

http://github.com/ceylon/ceylon/issues

Systems where Ceylon is known to work

Since Ceylon is running on the JVM it should work on every platform that supports a Java 7 or 8 compatible JVM. However we have tested the following platforms to make sure it works:

Linux

• Ubuntu "wily" 15.10 (64 bit) JDK 1.7.0_95 (IcedTea) Node 0.10.25
• Fedora 23 (64 bit), JDK 1.8.0_77 (OpenJDK)
• Fedora 22 (64 bit), JDK 1.8.0_72 (OpenJDK)
• Fedora 22 (64 bit), JDK 1.7.0_71 (Oracle)

Windows

• Windows 10 Home (64 bit) 1.8.0_77
• Windows 7 (64 bit) 1.7.0_05 (Oracle)
• Windows Server 2008 R2 SP1 JDK 1.7.0_04

OSX

• OSX 10 Lion (10.8.5) JDK 1.7.0_40 (Oracle) Node 0.10.17
• OSX 11 El Capitan (10.11.6) JDK 1.7.0_80 (Oracle) Node 0.10.35

License

The Ceylon distribution is and contains work released

• partly under the ASL v2.0 as provided in the LICENSE-ASL file that accompanied this code, and
• partly under the GPL v2 + Classpath Exception as provided in the LICENSE-GPL-CP file that accompanied this code.

License terms for 3rd Party Works

This software uses a number of other works, the license terms of which are documented in the NOTICE file that accompanied this code.

Repository

The content of this code repository, available here on GitHub, is released under the ASL v2.0 as provided in the LICENSE-ASL file that accompanied this code.

By submitting a "pull request" or otherwise contributing to this repository, you agree to license your contribution under the license mentioned above.

Acknowledgement

We're deeply indebted to the community volunteers who contributed a substantial part of the current Ceylon codebase, working often in their own spare time.

Ceylon is a project of the Eclipse Foundation.

ceylon-common's Issues

Support scripts installed in the Ceylon system folder

Right now we support scripts installed in ~/.ceylon/bin we should also support /etc/ceylon/bin (and their equivalents on Windows and MacOS)

tools: Replace `@PostConstruct` with `Tool.initialize()`

The tools framework has always supported a Tool with nullary method(s) annotated @PostConstruct which will be called after the setters and before run(). This is intended for tools which need to do something special with their properties which cannot be done in a setter alone, e.g. because it depends on two properties.

Such logic could go in run() of course, but that makes it harder for the framework to distinguish errors due to invalid arguments (exception from setters and @PostConstruct) from errors in actually running the tool with correct arguments.

Using @PostConstruct makes embedding a particular tool (e.g. I want to new-up a CeylonCompileTool and run it 'by hand') because you have to go about finding what the @PostConstruct methods are.

So I think we should have some kind of initialize() method on Tool itself and do away with support for @PostConstruct.

Java7Checker is too strict, should allow 7+

Now that there are early releases of Java 8 we should make this check smarter.

CeylonConfig silently ignores errors in configuration file

Unable to pass --help argument to Ceylon tools

Originally posted as ceylon/ceylon-runtime#58 by @lucaswerkmeister

ceylon run foo.bar --help will print the ceylon run help, even though the option belongs to the module being run, not ceylon run. This means that a Ceylon program started via the command line can’t print its own --help message.

Probably related to ceylon/ceylon-compiler#1228 (although --version works fine).

Add support for a `timeout` option

The tools should support a --timeout option and it should be possible to set it in the config file.
Where necessary connection code should set the timeout to the value returned by DefaultToolOptions.getDefaultTimeout()

Support `SCRIPT` and `SCRIPT_DIR` variables in plugin scripts

The plugin scripts could use a couple of extra variables to be even more useful: SCRIPT and SCRIPT_DIR. The former is the absolute path to the script we're currently executing and the latter is the folder it's in (to make it easy to get to support files in the same folder).

Config file remotes used after Herd (causing slow build due to network access)

[repositories]
remote=aether:/Users/julien/.m2/settings.xml

[ceylon-compile] [Looking for io.netty.netty-all-4.0.0.CR3.car]
[ceylon-compile] [ Trying repository /Users/julien/java/ceylon-0.5/repo]
[ceylon-compile] [ -> Not Found]
[ceylon-compile] [ Trying repository /Users/julien/.ceylon/repo]
[ceylon-compile] [ -> Not Found]
[ceylon-compile] [ Trying repository /Users/julien/.ceylon/cache]
[ceylon-compile] [ Trying repository http://modules.ceylon-lang.org/test]
[ceylon-compile] [Got 404 for url: http://modules.ceylon-lang.org/test/io/netty/netty-all/4.0.0.CR3/io.netty.netty-all-4.0.0.CR3.car]
[ceylon-compile] [ -> Not Found]
[ceylon-compile] [ Trying repository [Maven] Aether]
[ceylon-compile] [ -> Found at /io/netty/netty-all/4.0.0.CR3/netty-all-4.0.0.CR3.jar]
[ceylon-compile] [Adding module to classpath: io.netty.netty-all/4.0.0.CR3]
[ceylon-compile] [Found module at : /Users/julien/.m2/repository/io/netty/netty-all/4.0.0.CR3/netty-all-4.0.0.CR3.jar]
[ceylon-compile] [Trying to look up module from io.netty.netty-all.module]
[ceylon-compile] [Trying to look up module from io.netty.module]
[ceylon-compile] [Trying to look up module from io.module]

Make a Writer for configuration files

This Writer should be able to be its output on an existing configuration file and leave that file as much as possible the same as it was, only removing items that don't exist anymore, adding new items to the end and replacing values leaving white space and comments intact!

tools: WordWrap and punctuation

At the moment the synopses generated by the help tool sometimes look bad because WordWrap thinks it can wrap on a hyphen, or a ] etc. WordWrap needs to support being put in different 'modes' so that it can treat various punctuation differently.

tools: bug with indentation

for:

@Description("Examples:\n" +
            "\n" +
            "    --link https://modules.ceylon-lang.org/\n" +
            "    --link ceylon.math=https://modules.ceylon-lang.org/\n")

is printed:

Examples:

--link https://modules.ceylon-lang.org/
 --link ceylon.math=https://modules.ceylon-lang.org/

cey compile doesn't work on sdk

in the SDK's collection folder:

atlantis stephane 12:39 collection $ rm -rf modules/
atlantis stephane 12:39 collection $ ../../ceylon-dist/dist/bin/ceylonc ceylon.collection
atlantis stephane 12:39 collection $ rm -rf modules/
atlantis stephane 12:39 collection $ ../../ceylon-dist/dist/bin/cey compile ceylon.collection
[-src, source, -out, modules, ceylon.collection]
error: Unable to find class file for ceylon.language.String
/home/stephane/src/java-eclipse/ceylon-sdk/collection/source/ceylon/collection/makeCellArray.ceylon:2: error: type declaration does not exist or is ambiguous: Integer
Array<Cell<Key->Item>?> makeCellEntryArray<Key,Item>(Integer size) 
                                                     ^
/home/stephane/src/java-eclipse/ceylon-sdk/collection/source/ceylon/collection/makeCellArray.ceylon:2: error: ExpressionVisitor caused an exception visiting BaseType node: java.lang.NullPointerException at com.redhat.ceylon.compiler.typechecker.analyzer.ExpressionVisitor.defaultType(ExpressionVisitor.java:3857)
[followed by 100 errors]

That could be a tool issue, or a repo issue, so not sure if @quintesse or @tombentley is the target.

Dynamically loaded tool-chain based command plugins

We now have the script plugins to extend the ceylon command, it would be nice to have the same for Ceylon/legacy modules (containing classes) implementing the Tool interface.

Split list of lookup repositories in two

The idea here is to split the current list of lookup repositories into two different lists, either of them separately overridable. The current situation is (simplified):

[repositories]
bootstrap=...
cache=...
output=...
lookup=./modules
lookup=~/.ceylon/repo
lookup=http://modules.ceylon-lang.org

The new situation would be:

[repositories]
bootstrap=...
cache=...
output=...
lookup=./modules
global=~/.ceylon/repo
global=http://modules.ceylon-lang.org

When supplying -rep arguments on the command line only the lookup list will be overridden, leaving the global list alone. (But you can still override the global list if you want in any configuration file, just not on the command line).

Refactor: move model and CMR stubs here

To be able to run Ceylon code at runtime without JBoss Modules nor CMR support we need to:

• move the model from the typechecker to here
• move the model loader from the jvm compiler to here
• move some CMR interfaces to here (those used by the language module and the two above components)

Then we can make it so the language module can run without typechecker/compiler/cmr/runtime when using a flat classpath.

Introduce possibility to include the current directory name in a configuration value

We should introduce the possibility of adding something like ${DIR} to a value that would be replaced with the current directory the config file lives in.

ConfigWriter does not handle escape characters

multivalued-valued option arguments with a separator

To support ceylon/ceylon-compiler#745 we should support stuff like --verbose=foo,bar corresponding to a setter setVerbose(List<String>) or possibly setVerbose(List<EnumOfFooAndBar>).

Right now such a setter only works if you pass --verbose=foo --verbose=bar, which is, er, verbose. We also currently support passing a String "foo,bar" to setVerbose(), but then you have to write --verbose= to pass the empty set, and that's ugly: It would be better to support --verbose for that case.

The @OptionArgument() will need to grow a separator parameter.

Note: This will blur the distinction between options and option arguments, because now we can't tell simply from seeing --verbose whether it's option or an option argument.

Test issue

Don't ask. Really.
#36

#36
an issue link to #36 or #36 in the middle
not a link ceylon-common#36 #36asd or "#36"?

Ceylon command suggests `--null` option

$ ceylon compile -h org.codejive.ceylon.httpd
ceylon compile: Unrecognised short option '-h' to command 'compile'

Usage:
ceylon compile [--d] [--javac=<option>...] [--out=<url>] [--pass=<secret>] [--rep=<url>...] [--src=<dirs>...] [--sysrep=<url>] [--
               user=<name>] [--verbose[=<flags>]] [--] [<moduleOrFile...>]

Did you mean?
    --null

Run 'ceylon help compile' for more help

tools: allow throw validation exception and print its message

Add git commit id to full CEYLON_VERSION

It would be very nice that once we have the OBR we include the HEAD commit id of the code that was used to build the distribution. That way we can be sure exactly which compiler somebody uses (especially useful for dealing with non-released versions).

Maybe there's some ideas we could use here: http://stackoverflow.com/questions/3442874/in-git-how-can-i-write-the-current-commit-hash-to-a-file-in-the-same-commit

`ceylon --version` doesn't work anymore

Fix long option parsing

As per ceylon/ceylon-compiler#756 we need to fix the parsing of long options that take mandatory arguments to permit omission of the =.

Optional arguments to long options will still require an = and the warning message should make this clear if an = is missing.

See:

• http://www.gnu.org/software/libc/manual/html_node/Using-Getopt.html#Using-Getopt
• http://www.gnu.org/software/tar/manual/html_node/Long-Options.html

Note: At this time we're not going to implement the unambiguous abbreviated long option features.

CeylonConfig instances should not be static

That's just not thread-safe at all. We can have multiple instances of the compiler/tools running on the same jvm.

`ceylon compile default` gives exception if no code is found

$ ceylon compile default
ceylon compile: no source files or modules
Usage: ceylon compile <options> <source files> <module names>
use --help for a list of possible options
ceylon compile: Fatal error: The compiler exited abnormally (2) due to a bug in
the compiler.
Please report it:
 https://github.com/ceylon/ceylon-compiler/issues/new
Please include:

* a description of what you were trying to compile.

Thank you!
com.redhat.ceylon.compiler.CompilerBugException: Bug
    at com.redhat.ceylon.compiler.CeylonCompileTool.handleExitCode(CeylonCompileTool.java:496)
    at com.redhat.ceylon.compiler.CeylonCompileTool.run(CeylonCompileTool.java:478)
    at com.redhat.ceylon.common.tools.CeylonTool.run(CeylonTool.java:409)
    at com.redhat.ceylon.common.tools.CeylonTool.execute(CeylonTool.java:346)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.redhat.ceylon.launcher.Launcher.runInJava7Checked(Launcher.java:113)
    at com.redhat.ceylon.launcher.Launcher.run(Launcher.java:40)
    at com.redhat.ceylon.launcher.Launcher.run(Launcher.java:33)
    at com.redhat.ceylon.launcher.Launcher.main(Launcher.java:26)

Improve help for sub tools

Right now if you type ceylon config you get:

ceylon config: Argument 'action' to command 'config' should appear at least 1
time(s)

Usage:
ceylon config [--cwd=<dir>] [--define=<key>=<value>...] [--file=<file>] [--
              local] [--system] [--user] [--verbose[=<flags>]] get [--] <key>
ceylon config [--cwd=<dir>] [--define=<key>=<value>...] [--file=<file>] [--
              local] [--system] [--user] [--verbose[=<flags>]] keystore [--
              store-password[=<value>]] get-password [--] <alias>
ceylon config [--cwd=<dir>] [--define=<key>=<value>...] [--file=<file>] [--
              local] [--system] [--user] [--verbose[=<flags>]] keystore [--
              store-password[=<value>]] set-password [--] <alias> [<password>]
ceylon config [--cwd=<dir>] [--define=<key>=<value>...] [--file=<file>] [--
              local] [--system] [--user] [--verbose[=<flags>]] keystore [--
              store-password[=<value>]] unset-password [--] <alias>
ceylon config [--cwd=<dir>] [--define=<key>=<value>...] [--file=<file>] [--
              local] [--system] [--user] [--verbose[=<flags>]] list
ceylon config [--cwd=<dir>] [--define=<key>=<value>...] [--file=<file>] [--
              local] [--system] [--user] [--verbose[=<flags>]] remove [--] <key
              >
ceylon config [--cwd=<dir>] [--define=<key>=<value>...] [--file=<file>] [--
              local] [--system] [--user] [--verbose[=<flags>]] remove-section
              [--] <name>
ceylon config [--cwd=<dir>] [--define=<key>=<value>...] [--file=<file>] [--
              local] [--system] [--user] [--verbose[=<flags>]] rename-section
              [--] <old-name> <new-name>
ceylon config [--cwd=<dir>] [--define=<key>=<value>...] [--file=<file>] [--
              local] [--system] [--user] [--verbose[=<flags>]] set [--] <key>
              <values...>

Run 'ceylon help config' for more help

which is basically impossible to read because of the repetition of all those options. Maybe we should see if we can do something nicer, like for example:

Usage:
ceylon config [--cwd=<dir>] [--define=<key>=<value>...] [--file=<file>] [--
              local] [--system] [--user] [--verbose[=<flags>]] <subtool-arguments>

Where <subtool-arguments> can be:
    get [--] <key>
    keystore [--store-password[=<value>]] get-password [--] <alias>
    keystore [--store-password[=<value>]] set-password [--] <alias> [<password>]
    keystore [--store-password[=<value>]] unset-password [--] <alias>
    list
    remove [--] <key>
    remove-section [--] <name>
    rename-section [--] <old-name> <new-name>
    set [--] <key> <values...>

I guess this should be doable, right @tombentley ?

`mavenoverrides` default option is in the wrong config file section

It was added to compiler but it is in fact a general option available to all tools using repositories so it should be moved to defaults.

Improve module not found error message in case of missing version

On the mailing list a user got misled by the message that a module in a HTTP repository could not be found while it was clearly there. They were used to the fact that you can leave out a version and Ceylon will either automatically choose the only one available or give you the list of possibilities.

Thing is this doesn't work for repositories that can't be searched, like plain (non Herd) HTTP repositories.

We should improve this error in those cases by suggesting to specify a version.

build.xml doesn't create or delete build directory

The clean task deletes "${build}" instead of "${build.dir}" and the compile task does not ensure that "${build.classes}" exists.

move jboss-modules.jar from ceylon-runtime/lib to ceylon-common/lib

Currently, ceylon-module-resolver depends on ceylon-runtime only because of a jar in its lib directory. This results, via a complex chain of dependencies, in the pathalogical outcome that ceylon-spec eventually depends on ceylon-compiler!

We should move that jar to ceylon-common to break the chain of dependencies.

@quintesse would you take a look at that, please?

"run" tool doesn't give error for unknown option

Opened here instead of "runtime" because it's almost surely a problem with the underlying framework (probably related to the use of the annotation @Rest)

Found when trying to execute ceylon run --no-default-repos default (the flag is called --no-default-repositories)

Error in tool arguments doesn't show usage anymore

Typing something like ceylon config q results in:

ceylon config: Invalid value 'q' given for argument 'action' to command
'config': q

Usage:

Run 'ceylon help config' for more help

The usage section is empty.

cey script stutters

$ dist/bin/cey -version
cey: Unrecognised option(s): [-version, -version, -version, -version, -version, -version, -version]atlantis stephane 12:14 ceylon-dist $

Note that it also forgot to add a new line

Terminating `ceylon help` with `less` pager leaves terminal in dirty state

On Linux, with default pager settings, run ceylon help, then terminate the process with Ctrl+C. This will leave the terminal in a weird state where:

• less is terminated, but its output still visible
• your prompt is printed (right after the less prompt ‘:’)
• the next character will be swallowed (for some meaning of “character” – Esc and Enter have the same effect, while Ctrl+C will re-print the prompt)
• upon entering that character, the terminal will reset to show the regular output before less was started
• the terminal is “silent”, characters typed are not echoed – blind-type reset and hit Enter to fix this

I have no idea what’s going on here; I tried to investigate it, but didn’t arrive at any conclusions. Notes:

• less is supposed to stay open when its input is closed, and ignore Ctrl+C itself. This is useful, for instance, when running git log -Ssearch (or another expensive Git operation) – you’ll terminate git after the first few results are there, then look through those results with less still up.

  • in the above scenario, git actually continues running after terminated – not sure what’s happening here.

• piping ceylon help into less manually works without problems.

• this only happens for less, CEYLON_PAGER=more ceylon help works without problems.

• it happens no matter how much you hide less:

  echo -e "#!/bin/bash\nsleep 1; bash -c 'sleep 1; less'" > pager;
  chmod +x ./pager;
  CEYLON_PAGER=./pager ceylon help

CC ceylon/ceylon-compiler#1506.

tools: Improve `cey` script

Improve the cey script so if there is no Java tool corresponding to cey foo (which we can tell from the exit code), we try to run cey-foo.

At the moment the Java top level tool (Tool) prints out a list of possible tool names in this situation before exiting. This is nice functionality, but we don't really want it to start looking at $PATH, to try to figure out other suggestions, do we? I guess the alternative would be to leave that to the script, which could get a list of the nearly-matching Java tools by calling another low-level tool.

tools: Injecting standard streams

The tools framework currently doesn't do anything special for the standard streams (stdout, stderr & stdin): If a tool wants to print to standard output or error it just uses System.out.println()/System.err.println(). I'm wondering if we should change this. If the top level tool was responsible for injecting the PrintStreams (and InputStream in the case of standard input) it would:

• simplify testing (in particular test isolation when testing concurrently, which prevents the TopLevelToolTest running in the compiler tests),
• work better in those cases where the tool is being embedded in another program (rather than the more usual case of command line execution).

It would mean that tools would need to use the injected streams instead of reaching immediately for System.*.

The other way of looking at this is that tools "shouldn't" be reaching for raw streams at all, but should be using logging. In which case this becomes a question about whether it is the top level tool's job to configure logging, or the individual tools', or some combination.

cey newline issue

$ ../../ceylon-dist/dist/bin/cey compile 
cey compile: Argument modules should appear at least 1 time(s)atlantis stephane 12:36 collection $

Also note that we don't only accept modules but also file names

Support module wildcards in ant compile tasks

The ant ceylon-compile and ceylon-compile-js tasks should support compiling all modules in the given source folders that can be compiled for the given backend, just like '*' on the CLI (ceylon compile '*').

Make CACHE repository default to ~/.ceylon/cache

S.O.

Add project directory option to all tools

We need to be able to tell the tools which is their project directory, where the tools will look for config file and source/modules folders. This is different from the CWD of the user code that we're running or compiling.

This needs support in all the run tools (for now), and in those that use the RepositoryBuilder directly it needs to call cwd to set the project path for the CMR.

Then the IDE needs to be updated to call this for the JS run tool.

I'm sure implementing this can wait for later for the compiler/doc tools.

Don't store passwords in plaintext

Currently passwords can only be stored in plain text. We can easily do a couple of things to make this situation less dire:

1. Provide a way to prompt the user for the password for a particular repo. On the command line we can prompt via the console. In the IDE we'd need to be able to plug in a SWT password dialog.
2. Once we can prompt for passwords, it's not too difficult to store encrypted passwords in the config file. We could have an [encryption] section for the algorithm parameters and prompt for the decryption password when the system needs to access a password that is encrypted.

`ModuleWildcardsHelper` should ignore modules for other backends

So we can do ceylon compile \* without getting errors for modules that shouldn't have been compiled in the first place. Related to ceylon/ceylon-spec#946

tools: Bash completion

The bash completion is very ugly at the moment. It really needs someone with a lot more bash skills then I have to improve it. In particular there's lots of ugliness due to the way = (think long option arguments) is one of the $COMP_WORDBREAKS, which git manages to work around via majik which I don't entirely understand.

Logging

Currently a whole bunch of things each use their own logging:

• ceylon-module-resolver has its own Logger interface. Because everything else already does its own logging there are already with 7 implementations (delegating to JUL, System.err, javac Log, plus a couple of No-op loggers)
• ceylon-js seems to just write to System.err directly
• ceylon-runtime uses JUL, I think
• At some point there will be a ceylon.logging module.

It seems like ceylon-common is a good place to define a common API (and some implementations).

tools: Ambiguity in option names

At the moment the option parsing is very strict about the order of options with respect to the tool it belongs to. For instance the top level ceylon tool (the one which runs subcommands) accepts a --stacktraces option, so

ceylon --stacktrace compile ...

is ok, but

ceylon compile --stacktraces ...

is not. Likewise for (sub)*tools.

The benefit of this is I can later add an option to a tool without it conflicting with options of sub- or parent tools. The drawbacks are:

• it's really not very user friendly,
• it's not even obvious and
• conventionally the order of options shouldn't matter.

So I think we should change it. To do so, we'll need a way to add options to the top level tool without causing conflicts with subtools.

Support configuration of proxies

As a prerequisite for ceylon/ceylon-module-resolver#38, we need a configuration section for proxies

Tool arguments of type URI don't have full range of possibilities on Windows

An argument like --rep C:\\TEST will give an invalid value error on Windows.

Create "ceylon copy" command

We need a ceylon copy command that can do the following:

• copy a single or list of modules
• have an --include-dependencies / --recursive option
• have an --explode option to expand sources
• have a --create option to create a new repository if the target doesn't exist yet

Tools accepting URIs don't handle Windows paths correctly

They only accept POSIX style paths and not Windows style paths.