meamanusername / pie

Pipelines for Interactive Environments

PIE: Pipelines for Interactive Environments

PIE is an API and runtime for developing interactive software development pipelines and incremental build scripts.

Copyright and License

Copyright © 2018-2019 Delft University of Technology

The code and files in this project are licensed under the Apache License, Version 2.0. You may use the files in this project in compliance with the license.

Questions and Issues

If you have a question, enhancement, feature request, or bug report, please search the issue tracker for a solution or workaround, or create a new issue.

User's guide

Installation

PIE is deployed as a set of Maven artifacts to the MetaBorg artifact server, which you can consume with Maven or Gradle. To be able to get artifacts from the MetaBorg artifact servers, add the following to your build.gradle file:

repositories {
  jcenter()
  maven { url "https://artifacts.metaborg.org/content/repositories/releases/" }
  maven { url "https://artifacts.metaborg.org/content/repositories/snapshots/" }
}

or build.gradle.kts file:

repositories {
  jcenter()
  maven { url = uri("https://artifacts.metaborg.org/content/repositories/releases/") }
  maven { url = uri("https://artifacts.metaborg.org/content/repositories/snapshots/") }
}

or add the following to your Maven pom.xml file:

<repositories>
  <repository>
    <id>metaborg-release-repo</id>
    <url>https://artifacts.metaborg.org/content/repositories/releases/</url>
    <releases>
      <enabled>true</enabled>
    </releases>
    <snapshots>
      <enabled>false</enabled>
    </snapshots>
  </repository>
  <repository>
    <id>metaborg-snapshot-repo</id>
    <url>https://artifacts.metaborg.org/content/repositories/snapshots/</url>
    <releases>
      <enabled>false</enabled>
    </releases>
    <snapshots>
      <enabled>true</enabled>
    </snapshots>
  </repository>
</repositories>

To depend on version 0.6.0 of the runtime to build and execute pipelines, add the following Gradle dependency to your build.gradle file:

implementation 'org.metaborg:pie.runtime:0.6.0'

or build.gradle.kts file:

implementation("org.metaborg:pie.runtime:0.6.0")

or add the following to your Maven pom.xml file:

<dependency>
  <groupId>org.metaborg</groupId>
  <artifactId>pie.runtime</artifactId>
  <version>0.6.0</version>
</dependency>

The latest stable version is listed at the top of this file. The development version in the develop branch is published on every commit with version develop-SNAPSHOT which you can depend on as follows:

implementation 'org.metaborg:pie.runtime:develop-SNAPSHOT'

implementation("org.metaborg:pie.runtime:develop-SNAPSHOT")

<dependency>
  <groupId>org.metaborg</groupId>
  <artifactId>pie.runtime</artifactId>
  <version>develop-SNAPSHOT</version>
</dependency>

Components

PIE consists of several components:

• pie.api: The PIE API for developing reusable interactive pipelines or incremental build scripts
• pie.runtime: The PIE runtime for incrementally executing pipelines developed with the API
• pie.taskdefs.guice: Guice dependency injection support for task definitions
• pie.dagger: Dagger dependency injection support

Developer's guide

Building

As prerequisites for building PIE, you need a Java Development Kit (JDK) of version 8 or higher. To build PIE, run the Gradle wrapper as follows in the root directory of this repository:

./gradlew buildAll

Development

PIE can developed by importing this repository into IntelliJ IDEA or Eclipse as Gradle projects. Alternatively, any code editor in conjunction with local builds described above should work.

Continuous integration

PIE is automatically built, tested, and published on our buildfarm, configured in the Jenkinsfile. It uses the gradlePipeline shared pipeline from the metaborg.jenkins.pipeline shared pipeline library.

Publishing

To publish PIE, run ./gradlew publishAll.

To publish a new release version of PIE, first Git commit your changes and tag the commit in the form of release-<version>, e.g., release-0.3.0, and then run ./gradlew publishAll.

pie's Issues

`walk` code generation does not add imports

[reparseTest(file) | file <- wal]; compiles to the correct code but it does not add imports

Generate dagger module for generated tasks

For each PIE file, generate a Dagger module that provides all the tasks into a set

Optional: make it configurable

Generate pie file with spoofax 3 project tasks

Should contain PIE funcs and data for the generated PIE tasks so that they can be used with just import myProject:spoofax:task instead of needing to define all of them as foreign java

supplier constructor call

add generics diamond (<>) to generated code for supplier(...)

Default super type

Every data type should have a default super type. When generating to Java, that should be Object, with predefined methods hashCode, equals, toString.
Probably shouldn't make Object the default super type, how should user define what the default super type is?
Option: pass a data definition to the compiler which will be used as default super type.

Add concrete Java syntax

Would speed up development quite a bit

Allow injecting values into tasks

see https://slde.slack.com/archives/G014YGMK65U/p1592228343051500

Function overloading

Allow overloading functions, i.e. have functions with the same names

code generation

1. no code generation, all functions with less parameters get desugared to calls with default parameters
2. generate to same task. May be possible by creating an abstract Input class that has subtypes for the possible input types. Could apply strategy pattern in subclasses to implement different behavior for each input type.

class someTask implements TaskDef<someTask.Input, Output> {
  <I extends Input<I>> private static interface Input<I> extends Serializable {
    // private so that other classes can't implement this and provide something else
    // can be made public?
    Output exec(ExecContext context, I input);
  }

  public static class Input1 extends TupleX implements someTask.Input<someTask.Input1> {
    public Input1(...args...) {
      ... assign fields ...
    }
    
    public Output exec(ExecContext context, Input1 input) { ... }
  }
  
  public exec(ExecContext context, Input input) {
    return input.exec(context, input);
  }
}

3. generate to different tasks. Each task takes only one input and has a distinct name.

Options

Easy: default parameters

last parameters can have defaults. Generate functions that pass the default values.
Code generation: 1, 2 or 3.

Medium: user-defined implementation, one function per arity

allow each overload to have its own implementation. Each function combined with its number of parameters must be unique.
Code generation: 2 or 3

Hard: user defined implementations, arbitrary overloads

Allow arbitrary overloads. They are basically completely separate functions except that they have the same name.
Requires some way to choose which function to use when they have the same arity. Use most specific signature? Follows Java semantics, but don't know if that works in NaBL2.
Code generation: 2 or 3

fix add-import

fix add-import (does not actually tokenize the string, it just returns the string as is, which works because prettyprinting accepts it)

take path relative from other relative path

Allow something like

val tmp = ./build/tmp;
val relative = $tmp/services/createFoo;

Can this be done by adding PathStart = "$" Ref-CF "/"?
Should probably also add PathStart = "${" Exp-CF "}/" to allow ${projectRoot()}/tmp

Higher order functions

Allow creating lambdas, passing lambdas, referring to functions by name and applying functions.

Use cases

Some of the options in #15

Call multiple functions on a file and then handle all results the same afterward

val foo1 = print(postProcess(transform1(preProcess(file))));
val foo2 = print(postProcess(transform2(preProcess(file))));
val foo3 = print(postProcess(transform3(preProcess(file))));
val foo4 = print(postProcess(transform4(preProcess(file))));
... etc

becomes

val transformations = [transform1, transform2, transform3, transform4, ...];
val foos = [print(postProcess(transform(preProcess(file)))) | transform <- transformations];

Reparsing can be made generic over which specific parse and prettyprint are used

Could create a spoofax 3 project data with references to specific parse, style, prettyprint etc. functions (depends on #33). This would make the reparse test function above just take such a spoofaxProject data.

data spoofaxProject = {
  parse: func(ast: IStrategoTerm) -> IStrategoTerm
  prettyprint: func(ast: IStrategoTerm) -> string
  // etc.
}
func createReparseTest(project: spoofaxProject) -> IStrategoTerm -> unit = {
  ast1: IStrategoTerm => {
    val ast2 = project.parse(project.prettyprint(ast1));
    assertEquals(ast1, ast2, "Expected prettyprinting and reparsing an AST to yield the same AST")
  }
}

// requires #62
val reparseJavaTest = createReparseTest(javaProject)

Allows threading values in list comprehensions, i.e. reduceLeft. See #61.

Syntax

Lambdas

Exp.Lambda = Lambda
Lambda.SingleParam = [[TBind] => [Exp]] // e.g. x: int => x + 5
Lambda.MultiParam = [([{TBind ", "}*]) => [Exp]] // e.g. (x: int, y: int) => x + y

Extra: allow _ as parameter name multiple times, means that parameter is ignored.
Bonus extra: _ does not require type? Could use TopTy in that case?

Function Types

Type.FuncTyHeader = FuncHeader, e.g. val parse: func(ast: IStrategoTerm) -> IStrategoTerm = example:cpp:parse;
Option: allow leaving out the func and parameter name, e.g. val plus: (int, int) -> int
Requires that FuncHeader becomes right associative so that a function can return a function:

val curriedPlus: int -> int -> int = (x: int) => (y: int) => x + y;
val add1 = curriedPlus(1);
val two = add1(1);
val three = curriedPlus(1)(2);

Application

Exp.Application = <<Exp>(<{<Exp> ", "}*>)> // or Exp.Apply; e.g. plus(a, b)
This is ambiguous with Call (foo() could be application of variable foo or call to function foo.
Remove function calls, application can be used on both Func and Var?
See https://slde.slack.com/archives/C7254SF60/p1594220454206400

val const5 = () => 5;
val five = const5();

Should be left associative, see curriedPlus above

Should generate STaskDef instead of TaskDef?

Better way to define commands in spoofax 3

Reported by Guido.

Right now it has a lot of boilerplate, especially for commands specified in PIE.
Options:

• Simplify definitions in spoofax 3 project build.gradle to only different info. Is it enough to define a function that factors out the common parts at the top of the build file?
• Integration in PIE DSL so that full command definition (including displayname and description) can be written in DSL. Commands in PIE are automatically added to spoofax 3
• Create separate DSL to define commands (like ESV files)

Create Dagger Module in lang.test/src/test/java/mb/pie/lang/test/imports/subModule/a/b/c

Remove the tasks defined in lang.test/src/test/java/mb/pie/lang/test/imports/subModule/a/b/c from the Modules in imports.subModule

remove try-p2j-ast-exp

It turns errors at stratego compile time into errors at Java compile time / Java runtime, which is bad.

Lexical syntax

• QID cannot have multiple periods after each other (foo..bar)
• IDs should not be able to be keywords
• keywords cannot be followed by letters (see #3)

Reference types with import

module foo

import bar

func fun() -> bar:someType = {
  bar:createSomeType()
}

Only show error on last line of block when block type does not match declared type

Might be possible by creating a separate rule in NaBL2 for a function with a Block

Type equality

data should be considered equal if: both are foreign java of the same Java class

DSL generics

Generics for PIE DSL
Full generics cannot be implemented in NaBL2, but some parts can.
This proposal aims to implement generics for data types.

use cases

• foreign functions may return generics, e.g. JSGLR1ParseResult.getAst() returns an Optional<T>
  • in particular, spoofax 3 tasks return Result<T, E>
• Create generic functions, e.g. assertEquals(expected: T, actual: T, message: string) -> unit

syntax

New type: Type.GenericTy = [[ID]<{[ID] ", "}+>]. Exactly the same as Java, e.g. Result<T, E>. Can be used everywhere a type is used.

Functions gain the ability to specify generic type parameters:
func createOk<T: Object>(val: T) -> Result<T, Exception> = foreign java mb.common.result.Result#ofOk
func id<T>(val: T) -> T = val

Call a generic function:
val result = createOk<Foo*>([createFoo()]);
Requirement to explicitly add types may be lifted in the future (presumably requires statix analysis though).

Static semantics

For now, data Foo<Bar>: Bar must be a valid data type, it is not a generic data type.
Later: allow data Foo<T: Language>. If the generic type parameter adds a supertype it is a generic parameter, without a parameter it is specific. Allow T: _ to use TopTy as parameter.
Even later: deprecate data Foo<T>. After deprecation, data Foo<T> becomes syntactic sugar for Foo<T: _>
func createOk<T: Object>(...)

Code generation:

[todo]

Issues

[todo]

Ideal code to run pie task

Ideally, all that is required to run PIE with the DSL is to specify the task and pass it the input, everything else is generated by PIE. Generating a Component and a Module should be possible.

Ideal code

In all cases there should be a pie file.

Java

Project should contain a metaborg.yaml specifiying pie as a compile dependency.
Java code to run a task

class Runner {
  public static void main(String[] args) throws ExecException {
    PipelineComponent pipelineComponent = DaggerPipelineComponent.create(); // generated from `pipeline.pie`
    pipelineComponent.executeTask(Main.class);
  }
}

Here, the pipelineComponent is generated by PIE. It contains an @ElementsIntoSet that provides all taskDefs of the pie file. It extends from a library interface that has methods to call a task and a method to get Pie. The methods that take a String will not give errors when nothing has been build yet but will also only give errors at runtime instead of build time.

• Pie getPie()
• <O extends Serializable> O executeTask(Class<? extends TaskDef<None, O>> task)
• <I extends Serializable, O extends Serializable> O executeTask(Class<? extends TaskDef<I, O>> task, I input)
• Serializable executeTask(String task)
• Serializable executeTask(String task, Serializable input)

Gradle plugin

Use a gradle plugin to handle everything (don't even need to add a metaborg.yaml), just create pie file and specify which task to run

plugins {
  id 'bla.bla:pie.dsl:0.1.0'
}
pie {
  main = 'example:root:main' // fully qualified name of main task
}

It should set up compilation and stuff
It should automatically add the right directories to sourceSets, i.e. read config and add that directory
It would be nice if it only compiles files from the right sourceSets, e.g. if there is a file in src/test/pie and it doesn't need to compile the test sourceSet it does not compile the pie file.
Would be great if it can also generate a CLI (with picoCLI), either as jar or as executable.

Sub issues

#8
Missing issues

• create gradle plugin
• create superclass for generated Components with the utility methods to call a task by class

Star import

Is this possible in NaBL2?
everything in file: import example:c:task:*
everything in module (non-recursive): import example:c:*
everything in module with extra qualifiers: import example:*:task:*
everything in module recursively: import example:c:**
everything in module recursively with extra qualifiers: import example:**:parse

Questions: how to clarify between importing from file and from module? (e.g. function foo:bar() and module foo:baz would both be caught under foo:*. How to specify which one you want?

Generate files in configurable location

Generate files in location specified in /pie/config.txt

fix stratego imports

Stratego files all have hundreds of errors because the editor does not import the Java constructors correctly. It slows down development because it is hard to see real errors.

Create example multi-project parse-transform-prettyprint pipeline

To provide an example of the problems and requirements for Eelco and Gabriel

Goal:
project 1: language A. Includes PIE task to parse.
project 2: compiler: provides strategy strategy to compile A to B. Includes PIE tasks to invoke the strategy.
project 3: language B. Includes PIE task to prettyprint.
project 4: pipelines. Calls all tasks from the different projects.

Languages could be PIE and Java?

Code generation fails sometimes when you select a part of a pie file and then generate

Solution: use third argument instead of first argument in compilation strategy (see https://slde.slack.com/archives/C7254SF60/p1589975383014100)

import functions

Allow specifying functions directly in the import so that they do not have to be called with a qualified call:

module example
import foo:exampleFunc

func bar() -> int = {
  exampleFunc()
}

If there is a name conflict with another imported function or a local function it gives a duplicate name static analysis error

import data

Allow importing data directly instead of requiring to import the module and then using qualified data

module example
import foo:exampleData
func bar() -> exampleData = {
  foo:createExampleData()
}

Should give a duplicate names error in case there are conflicts between the imported data type, other imported data types or locally declared data types.

compiler configuration format

Specify configuration file format for the compiler.

Options that are likely to exist

• output directory (where to generate files)
• generate Dagger module?
• generate Dagger component?
• Add header / footer

Requirements

• no need to specify everything
• may need to specify some things
• can easily be read into Stratego
• handles types
  • boolean
  • string
  • int
  • Path
  • float?
  • enum?
  • composite?
• allows comments
• allows whitespace

formats

options

• JSON
• XML
• properties file
• custom format

decision

JSON? Can be read with Jackson, but need to figure out how to read into stratego. See discussion and example implementation

Improve compiler development

It is currently a slog, because it is easy to make mistakes when implementing a transformation and a lot of work to add tests

Goals

compiler

easily add transformations. Ideally: just write PIE code and corresponding Java code using concrete syntax.

Tests

Easily add new tests for new features or discovered bugs.
Best case scenario is just adding a pie file, but I will settle for adding pie file and a Java test file.
Test names should / could all be main once #4 is closed

Sub issues

• compiler
  • #42
  • #43
  • #44
• tests
  • #22

Handle dead code

Dead code is allowed (not checked for) in PIE but not in Java.
There are also cases like { fail "error" } that get compiled to code that doesn't compile due to dead code

Options:

• disallow dead code in PIE. Probably not a good idea, language design now depends on how advanced the compiler analysis for dead code is.
• check for dead code and remove it before code generation. Requires DynSem analysis, does not solve fail-in-block error.
• remove dead code after generating java. Could be done as a larger java optimization round. There are probably existing tools but getting them to work with spoofax might be a pain.

name generation

• Should only generate legal names. If name is illegal (a keyword), prepend $ to make it legal
• Should follow Java conventions: class starts with uppercase, variable starts with lowercase
• fix duplicate name generation (something like val supplier0 = supplier(""); would fail because code gen generates supplier0 twice)

data declarations

Goals

Use cases

• configuration settings in PIE
• semantic tuples (keep related data together)

Requirements

• define the structure of data
• instantiate data in PIE
• read the data from file
• get the data from CLI parameters
  • would be nice if there is some way to provide defaults
• access data fields
  • in PIE
  • in Java

Solution

A data defition has only constant fields. An aditional data type allows
referring to the data type to create an instance in PIE or to call library
methods to read from a file.

Definition

Declare data with Fields (Tbinds).

data Optimization = {
  flag: string
  strategy: string
  alternativeStrategy: string?
}
data OptimizationProfilingInfo = {
  optimization: Optimization
  start: Time
  end: Time
}

Data type

Add a new type data<T>. Refers to a data type.
Refering to a data type is possible by name (variable declaration at module level).
The declaration has type data<name>.
Should it be allowed to shadow a data type name? If yes, should it shadow backward?
Example:

data Config = {...} // declares Var{"Config} : DataClassTy(<occurence of Var{"Config"}>)
func example() -> data<Config> = {
  Config
}
func shadowing() -> int = {
  val Config = 5; // allowed or Error?
  Config
}

Instantiate in PIE

A data definition implicitly has a create method that takes all the fields in the order they are defined.

data Config = {
  name: string
  files: path*
}
func example() -> Config = {
  Config.create("example", [./foo.txt, ./bar.txt])
}

read data from file

Provide library functions that take a data class type and a file to read from

import pie:std:readData
data Config = {...}
func readConfig() -> Config = {
  val config: Config = readData:readJSON(Config, ./config.txt); // assign to val for illustration purposes
  // readJSON requires the file
  config
}

get data from CLI parameters

picoCLI has functions to parse parameters with a custom parser.
How to parse them can be defined in Java.
Automatic splitting of data definitions would be nice but is not important now,
see future work.

access configuration settings

in PIE

Add syntax for field access.

data IntPair = {
  x: int
  y: int
}
func getX(pair: IntPair) -> int = pair.x

in Java

Generated code has private fields with getters.

Code generation

Data class is final (no subclasses).
Fields are private final with public getters. getter names: get_fieldName

Includes toString, equals, hashCode.
Equals compares equality for each of the fields.

Full Example

module example

import oracle:greenmarl:project

data Config = {
  algorithms: path*
  outputDir: path
}

func main(configPath: path?) -> unit = {
  val config = if (configPath == null)
    Config.create(list ./algorithms with extension "gmi", ./build/output)
  else
    pie:std:readJSON(Config, configPath!);
  run(config)
}

func run(config: Config) -> unit = {
  // Spoofax 3: command
  val results: (string, string) = [(project:getName(algorithm), compile(mb:spoofax:supplierFrom(algorithm))) | algorithm <- config.algorithms];
  [pie:io:writeFile(config.outputDir/name, algorithm) | (name, algorithm) <- results];
  unit
}

func compile(program: supplier<string>) -> string {
  project:prettyprint(project:compile(project:parse(program)));
}

About this proposal

Open questions

Should add syntax to define data on single line?

Better name for DataClassTy?

• struct, class, etc.

create method takes all the fields in the order they are defined: is this possible?

probably

  [ DataImpl(fields) ^ (s, name) : ty ] :=
    Map2T [[ fields ^ (s, constructor_params) ]], // processing a Field binds constructor_param
    ty_constructor := FuncTy(constructor_params, DataTy(name)),
    ... // ty etc.

shadowing

Should shadowing be legal? Should use before shadowing be legal?

data config = {}
func example() -> int = {
  val conf: data<config> = config;
  val config = 8; // shadows config as data definition
  config
}

Shadowing probably does not give issues.

should data be equal?

Use nominal types for now, structural types are complicated to implement and probably not necessary.

Future work (i.e. not included in this proposal)

Default values.

There are no default values as that would increase the scope.
Default values could be implemented in a simple way by creating constructors
with less parameters. See #20.
They could be implemented more complicated by creating a partial type to
represent data that is partially filled out.

functions on data declarations

This is related to the spoofaxProject use case on #16, but this defines constant functions on the data while that defines functions as fields.

• custom function (can use self to access own fields)
• static function (defined on data type instead of on instance)
• custom constructor (just a sub type of static function that returns an instance?)
• validator: function that is called during creation to validate fields, e.g. check that int is between 0 and 10.
• suppress implicit create method (only allow construction via custom constructor)
• override default equality (question: how to handle hashcode?)
  • simple override: declare which fields should be considered
  • custom override: custom equality function. Hashcode is just 0 to be sure?

data Config = {
  name: string
  files: path*
  constructor func createNamed(name: string) -> Config = { // params can be anything, return type must be (subtype of) Config
    Config.create(name, [])
  }
  func finagle() -> string = finagle(self.name, self.files) // calls externally defined finagle function with fields
}

Split data definitions when used in CLI

Example:
data Config = {name: string, files: path*, level: int}
call with myapp --name="bob" --files=./foo;./bar --level=8

Failure handling

I'm trying to figure out how to do error handling in PIE. I mostly think that if there is an error the pipeline should just fail (which is then handled by whatever called PIE).
I see two use cases:

• try to compile all files so that you can report errors on all files instead of one at a time:
  val parsed: result<CompilationResult>* = try {parse(file)} | file <- files] (Note: result<T> is either Result(T) or Error(Exception))
• test framework where tests can fail without throwing

Currently blocked waiting for Gabriel to finish prototyping failure handling in the PIE framework prototyping done, PIE now uses Result for failure handling. See #84

Renaming on imports

import foo:bar as baz

Use cases:

• When multiple modules have the same last package, e.g. oracle.cpp.spoofax.task, oracle.gm.spoofax.task. Can now import without name clashes: import oracle:gm:spoofax:task as gm
• When java package name does not make much sense in PIE, e.g. mb:spoofax:pie can be imported using import mb:spoofax:pie as spoofax

Should call generation inject a `Provider<X>` or just `X`?

If just X, it may be better to use Lazy<X>

Improve compiler tests

• create master Module that imports all other modules? (Is it possible to take all modules from a package?)
• remove Components.
• Extend SimpleRunner to take a string (task name), input and expected output.
• Rename pie names (remove TestGen from names)
  • File names
  • Function names

Automatically add tasks to spoofax 3

When pie tasks are defined in a spoofax 3 project, automatically add them to the spoofax 3 project (or at least add them in a few lines instead of adding tasks manually one by one)

Nice to have

• make it configurable to add or not to add (add by default)
• some way to add commands without specifying them manually

logging in DSL

There needs to be some way to log from within the DSL
Cannot be made a task because it may not get executed. Additionally, tasks should not have side effects.

Syntax

• as function: log(severity, message)
• built-in: log <severity> <message>, e.g. log ERROR "Could not open file $file"
  As function seems most logical.

Create libraries

pie standard library

module pie:std

// IO, like write file (should be build in?)
func projectRoot() -> path = ... // provides the root of the project, see https://slde.slack.com/archives/C7254SF60/p1593869119193700
func repositoryRoot() -> path = ... // provides the root of the repository. Will need to determine the repository type and then get the root. For git: walk up the tree until you find a .git folder.

Spoofax commons and spoofax 3 are one module because most spoofax commons stuff (strategoRuntime etc) is still spoofax 3

module mb:spoofax

data IStrategoTerm = foreign java org.spoofax.interpreter.terms.IStrategoTerm {}
transient data StrategoRuntime = foreign java mb.stratego.common.StrategoRuntime {
  func invoke(strategyId: string, term: IStrategoTerm) -> IStrategoTerm?
}

func buildStrategoRuntime() -> StrategoRuntime = foreign mb.spoofax.BuildStrategoRuntime

func invokeStrategoStrategy(strategy: string, ast: IStrategoTerm) -> IStrategoTerm = {
  val strategoRuntime = buildStrategoRuntime();
  val result = strategoRuntime.invoke(strategy, ast);
  if (result == null)
    fail "Executing Stratego strategy '$strategy' failed";
  result!
}

data ResourceKey = foreign java mb.resource.ResourceKey {}
data Region = foreign java mb.common.region.Region {}
data CommandOutput = foreign java mb.spoofax.core.language.command.CommandOutput {}

func supplierFrom(key: ResourceKey) -> supplier<string> = foreign java constructor mb.pie.api.ResourceStringSupplier
func getSmallestTermEncompassingRegion(ast: IStrategoTerm, region: Region) -> IStrategoTerm = foreign java mb.jsglr.common.TermTracer#getSmallestTermEncompassingRegion
func termToString(term: IStrategoTerm) -> string = foreign java mb.stratego.common.StrategoUtil#toString
func singleResultCommmandOutput(result: string, description: string) -> CommandOutput = foreign java 

func restrictToRegion(ast: IStrategoTerm, region: Region?) -> IStrategoTerm = {
  if (region != null)
    getSmallestTermEncompassingRegion(ast, region!)
  else
    ast  
}

testing

• func assertEquals(expected: TopTy, actual: TopTy, message: string)

Add PIE concrete syntax

Would make the code easier to understand.

list comprehension does not compile

func example() -> int* = [x + 1 | x <- [1, 2, 3]]

Better way to create PIE `func`s for stratego strategies

Right now adding a compileFoo(ast: IStrategoTerm) -> IStrategoTerm = invokeStrategoStrategy("generate-foo") for every strategy you want to use seems like boilerplate.
Naming: transform strategy name generate-foo-names --> generateFooNames

Options

with auto-generated names

Questions:

• how to discover strategies?
• how to determine which strategies to use?
  • automatically, from ESV file for example: very non-trivial
  • user specifies a list
  • all strategies
• requires IDE support to suggest function names / some other way for user to discover the strategies

preprocessor

Allow to define own shorthand, provide library function to desugar STRATEGO_FUNC("generate-foo-names", "generateFooNames") --> func generateFooNames(ast: IStrategoTerm) -> IStrategoTerm = invokeStrategoStrategy("generate-foo-names").

First class functions

stratego(strategy: string) -> func(ast: IstrategoTerm) -> IStrategoTerm = (ast: IstrategoTerm) => invokeStrategoStrategy(strategy, ast)
Call as val generateFooNames = strategoFunc("generate-foo-names");
Call only works in a function body, how to bind this to a function in the module scope?
Still has a bit of boilerplate

first class functions with compile time execution

Add syntax func <ID> = <ID>. Second ID refers to a function that returns a lambda and is executed at compile time, e.g.
func generateFooNames = strategoFunc("generate-foo-names");
Almost no boilerplate, but seems like design bloat.

Reference foreign method that returns `void`

It's not possible right now, because PIE doesn't have void but unit. An option is to use unit: func print(text: string) -> unit = foreign java java.lang.System.out#println. But this means that given a signature func foo() -> unit = foreign java pkg.Foo#foo it is not possible to know if foo returns unit or void. My solution would be to ignore the return value and always use None.instance, so the generated code becomes

foo();
final None result1 = None.instance;

(Note: yes, that second line is not required most of the time. Figuring out when it is needed does not have priority right now)
Another option is to explicitly say when a method is void: func print(text: string) -> unit = foreign java void java.lang.System.out#println. This is similar to the way constructors are handled (foreign java constructor pkg.Foo), but I feel like the distinction here is not something the developer should have to care about (static vs constructor is an important difference in java, the return value of a method not that much, although a void method is special in that it does not have a return value)

Generate header / footer

Add an option to add a header / footer to the generated PIE code

Invoke super method

Invoke a method on the superclass of a class

This should compile:

data Foo : Object = mb.pie.Foo
func test(foo: Foo) -> string = foo.toString()

Disallow following keywords with a letter

Add this snippet to syntax somewhere

lexical restrictions
  
  template options keyword -/- [a-zA-Z0-9\_\-]

Give warning when adding `.` to extension matcher

walk directory with extension ".cpp" should give a warning on the . that that is probably not what you want (should be walk directory with extension "cpp").

import multiple things at once

Allows things like import example:{c, cpp}:task:{parse, prettyprint}

Adds syntactic sugar {<names>} to imports. This is desugared to the Cartesian product of all the elements, i.e. the example is equivalent to

import example:c:task:parse
import example:c:task:prettyprint
import example:cpp:task:parse
import example:cpp:task:prettyprint

concrete object syntax

Add concrete object syntax to decrease mistakes and increase development speed