Problem building with bazel 0.3.2 about rules_scala

[petemounce]

I can reproduce this error with bazel 0.3.2 on OSX Sierra, in my own project, within this https://github.com/petemounce/rules_scala/tree/90fca498df4570ec8cb00a57e7cc9a8c44a854b4 by running bazel test //.... This is the link to compare the two repos:

$ git ll -3
90fca498df4570ec8cb00a57e7cc9a8c44a854b4 3 hours ago Peter Mounce Merge remote-tracking branch 'upstream/master' into our_total_hacks
f6b36b3317f1c78bef861f64607a118d7b3ba7e8 21 hours ago GitHub Merge pull request #98 from damienmg/master
deff6c2806df8fbfc98330c06c1444140664daa2 5 days ago Peter Mounce Need to separate args from binary with a space

$ bazel test //...
WARNING: /Users/peter/src/oss/rules_scala/twitter_scrooge/twitter_scrooge.bzl:208:33: Argument `cfg = "host"` or `cfg = "data"` is required if `executable = True` is provided for a label.
INFO: Found 67 targets and 1 test target...
INFO: From Building src/java/io/bazel/rulesscala/scalac/scalac.jar (2 source files) [for host]:
Note: src/java/io/bazel/rulesscala/scalac/CompileOptions.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
INFO: From Building src/java/io/bazel/rulesscala/scalac/scalac.jar (2 source files):
Note: src/java/io/bazel/rulesscala/scalac/CompileOptions.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
ERROR: /Users/peter/src/oss/rules_scala/test/BUILD:154:1: scala //test:MixJavaScalaLib failed: scalac failed: error executing command bazel-out/host/bin/src/java/io/bazel/rulesscala/scalac/scalac @bazel-out/local-fastbuild/bin/test/MixJavaScalaLib_worker_input: com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
Exception in thread "main" java.lang.RuntimeException: nope
    at io.bazel.rulesscala.scalac.ScalaCInvoker.main(ScalaCInvoker.java:338)
Caused by: java.io.IOException: Cannot run program "external/local_jdk/bin/javac ": error=2, No such file or directory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
    at io.bazel.rulesscala.scalac.ScalaCInvoker.compileJavaSources(ScalaCInvoker.java:293)
    at io.bazel.rulesscala.scalac.ScalaCInvoker.processRequest(ScalaCInvoker.java:237)
    at io.bazel.rulesscala.scalac.ScalaCInvoker.main(ScalaCInvoker.java:334)
Caused by: java.io.IOException: error=2, No such file or directory
    at java.lang.UNIXProcess.forkAndExec(Native Method)
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
    at java.lang.ProcessImpl.start(ProcessImpl.java:134)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
    ... 3 more
INFO: Elapsed time: 64.537s, Critical Path: 7.76s
//test:HelloLibTest                                                   NO STATUS

Executed 0 out of 1 test: 1 was skipped.

[petemounce]

This happens when, in WORKSPACE, the io_bazel git_repository rule is changed like in #100, and also, similarly, when it's tag = "0.3.2". This is, again, on petemounce/rules_scala@90fca498df4570ec8cb00a57e7cc9a8c44a854b4 with either of those changes applied.

Here is the output where tag = "0.3.2", this time with verbose_failures:

$ bazel test --verbose_failures //...
WARNING: /Users/peter/src/oss/rules_scala/twitter_scrooge/twitter_scrooge.bzl:208:33: Argument `cfg = "host"` or `cfg = "data"` is required if `executable = True` is provided for a label.
INFO: Found 67 targets and 1 test target...
ERROR: /Users/peter/src/oss/rules_scala/test/BUILD:154:1: scala //test:MixJavaScalaLib failed: scalac failed: error executing command
  (cd /private/var/tmp/_bazel_peter/cb0f8a6254f19f3f9fe9f1ab970ecbf6/execroot/rules_scala && \
  exec env - \
  bazel-out/host/bin/src/java/io/bazel/rulesscala/scalac/scalac @bazel-out/local-fastbuild/bin/test/MixJavaScalaLib_worker_input): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
Exception in thread "main" java.lang.RuntimeException: nope
    at io.bazel.rulesscala.scalac.ScalaCInvoker.main(ScalaCInvoker.java:338)
Caused by: java.io.IOException: Cannot run program "external/local_jdk/bin/javac ": error=2, No such file or directory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
    at io.bazel.rulesscala.scalac.ScalaCInvoker.compileJavaSources(ScalaCInvoker.java:293)
    at io.bazel.rulesscala.scalac.ScalaCInvoker.processRequest(ScalaCInvoker.java:237)
    at io.bazel.rulesscala.scalac.ScalaCInvoker.main(ScalaCInvoker.java:334)
Caused by: java.io.IOException: error=2, No such file or directory
    at java.lang.UNIXProcess.forkAndExec(Native Method)
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
    at java.lang.ProcessImpl.start(ProcessImpl.java:134)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
    ... 3 more
INFO: Elapsed time: 11.205s, Critical Path: 11.10s

[johnynek]

So, we are hitting this branch of the code:

https://github.com/bazelbuild/rules_scala/blob/master/src/java/io/bazel/rulesscala/scalac/ScalaCInvoker.java#L274

which is just shelling out to javac to support mixed targets.

If you add ctx.file._javac to this list:
https://github.com/bazelbuild/rules_scala/blob/master/scala/scala.bzl#L200

does it work for you?

This looks like yet another hygiene issue that we don't see in tests because we test with docker. Very frustrating situation.

[petemounce]

@johnynek Unfortunately not. If I add that to the list (diff included so you can see where I did), I get the same error.

peter at petermounce-mbp-imp in ~/src/oss/rules_scala (our_total_hacks●●)
$ gd
diff --git i/WORKSPACE w/WORKSPACE
index 4ab9958..2cc5005 100644
--- i/WORKSPACE
+++ w/WORKSPACE
@@ -23,5 +23,5 @@ maven_jar(
 git_repository(
     name = "io_bazel",
       remote = "git://github.com/bazelbuild/bazel.git",
-      commit = "0.3.1",
+      tag = "0.3.2",
 )
diff --git i/scala/scala.bzl w/scala/scala.bzl
index bf3b490..14e3b67 100644
--- i/scala/scala.bzl
+++ w/scala/scala.bzl
@@ -211,6 +211,7 @@ SourceJars: {srcjars}
            [ctx.outputs.manifest,
             ctx.file._ijar,
             ctx.file._java,
+            ctx.file._javac,
             argfile])
     ctx.action(
         inputs=ins,

Output:

peter at petermounce-mbp-imp in ~/src/oss/rules_scala (our_total_hacks●●)
$ bazel test --verbose_failures //...
.........
WARNING: /Users/peter/src/oss/rules_scala/twitter_scrooge/twitter_scrooge.bzl:208:33: Argument `cfg = "host"` or `cfg = "data"` is required if `executable = True` is provided for a label.
INFO: Found 67 targets and 1 test target...
ERROR: /Users/peter/src/oss/rules_scala/test/BUILD:154:1: scala //test:MixJavaScalaLib failed: scalac failed: error executing command
  (cd /private/var/tmp/_bazel_peter/cb0f8a6254f19f3f9fe9f1ab970ecbf6/execroot/rules_scala && \
  exec env - \
  bazel-out/host/bin/src/java/io/bazel/rulesscala/scalac/scalac @bazel-out/local-fastbuild/bin/test/MixJavaScalaLib_worker_input): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
Exception in thread "main" java.lang.RuntimeException: nope
    at io.bazel.rulesscala.scalac.ScalaCInvoker.main(ScalaCInvoker.java:338)
Caused by: java.io.IOException: Cannot run program "external/local_jdk/bin/javac ": error=2, No such file or directory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
    at io.bazel.rulesscala.scalac.ScalaCInvoker.compileJavaSources(ScalaCInvoker.java:293)
    at io.bazel.rulesscala.scalac.ScalaCInvoker.processRequest(ScalaCInvoker.java:237)
    at io.bazel.rulesscala.scalac.ScalaCInvoker.main(ScalaCInvoker.java:334)
Caused by: java.io.IOException: error=2, No such file or directory
    at java.lang.UNIXProcess.forkAndExec(Native Method)
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
    at java.lang.ProcessImpl.start(ProcessImpl.java:134)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
    ... 3 more
INFO: Elapsed time: 39.435s, Critical Path: 9.73s
//test:HelloLibTest                                                   NO STATUS

Executed 0 out of 1 test: 1 was skipped.

[petemounce]

Ah, but... If I undo a change I made around handling jvmflags, then the build and test passes within scala_rules, but fails within my own project.

$ gd
diff --git i/WORKSPACE w/WORKSPACE
index 4ab9958..2cc5005 100644
--- i/WORKSPACE
+++ w/WORKSPACE
@@ -23,5 +23,5 @@ maven_jar(
 git_repository(
     name = "io_bazel",
       remote = "git://github.com/bazelbuild/bazel.git",
-      commit = "0.3.1",
+      tag = "0.3.2",
 )
diff --git i/scala/scala.bzl w/scala/scala.bzl
index bf3b490..3d7e68b 100644
--- i/scala/scala.bzl
+++ w/scala/scala.bzl
@@ -184,7 +184,7 @@ SourceJars: {srcjars}
         javac_path=ctx.file._javac.path,
         java_files=",".join([f.path for f in java_srcs]),
         #  these are the flags passed to javac, which needs them prefixed by -J
-        jvm_flags=" " + " ".join(["-J" + flag for flag in ctx.attr.jvm_flags]),
+        jvm_flags=" ".join(["-J" + flag for flag in ctx.attr.jvm_flags]),
         resource_src=",".join([f.path for f in ctx.files.resources]),
         resource_dest=",".join(
           [_adjust_resources_path(f.path)[1] for f in ctx.files.resources]
@@ -211,6 +211,7 @@ SourceJars: {srcjars}
            [ctx.outputs.manifest,
             ctx.file._ijar,
             ctx.file._java,
+            ctx.file._javac,
             argfile])
     ctx.action(
         inputs=ins,

My project error:

ERROR: /Users/peter/src/everything/Fabric/Scala/fabric/BUILD:5:1: Couldn't build file Fabric/Scala/fabric/fabric.jar: scala //Fabric/Scala/fabric:fabric failed: scalac failed: error executing command
  (cd /private/var/tmp/_bazel_peter/ca0e6c99a8d9bbee51f9cc27ea38a414/execroot/everything && \
  exec env - \
  bazel-out/host/bin/external/io_bazel_rules_scala/src/java/io/bazel/rulesscala/scalac/scalac '--jvm_flag=-Xms1G' '--jvm_flag=-Xmx4G' @bazel-out/local-fastbuild/bin/Fabric/Scala/fabric/fabric_worker_input): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
warning: there were 84 deprecation warnings; re-run with -deprecation for details
one warning found
Exception in thread "main" java.lang.RuntimeException: nope
    at io.bazel.rulesscala.scalac.ScalaCInvoker.main(ScalaCInvoker.java:338)
Caused by: java.io.IOException: Cannot run program "external/local_jdk/bin/javac-J-Xms1G -J-Xmx4G": error=2, No such file or directory
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
    at io.bazel.rulesscala.scalac.ScalaCInvoker.compileJavaSources(ScalaCInvoker.java:293)
    at io.bazel.rulesscala.scalac.ScalaCInvoker.processRequest(ScalaCInvoker.java:237)
    at io.bazel.rulesscala.scalac.ScalaCInvoker.main(ScalaCInvoker.java:334)
Caused by: java.io.IOException: error=2, No such file or directory
    at java.lang.UNIXProcess.forkAndExec(Native Method)
    at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
    at java.lang.ProcessImpl.start(ProcessImpl.java:134)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
    ... 3 more

The Cannot run program "external/local_jdk/bin/javac-J-Xms1G -J-Xmx4G": error=2, No such file or directory lacks a space between javac and the options to it, which is, now I recall, why I made the jvmflags change. Hopefully there's a different way to address that issue?

The target looks like

scala_library(
  name = "fabric",
  visibility = ["//Fabric:fabric-internal"],
  resources = glob(["src/main/resources/**/*"]),
  srcs = glob(["src/main/scala/**/*.scala"]) + glob(["src/main/scala/**/*.java", "src/main/java/**/*.java"]),
  deps = [ ... ],
  runtime_deps = [ ... ],
  exports = [ ... ]
)

scala_export_to_java(
  name = "as_java",
  exports = [":fabric"],
  runtime_deps = [],
)

[johnynek]

ahh yeah, this looks like a bug:
https://github.com/bazelbuild/rules_scala/blob/master/src/java/io/bazel/rulesscala/scalac/ScalaCInvoker.java#L277

I guess we are not exercising with flags. That is just directly appending the jvm flags. We should fix that be adding a space I guess.

[petemounce]

@johnynek I've added a PR with repro, but no fix. I'm off to bed at this point; thanks for the pointer in the right direction :)

[johnynek]

I think this is fixed by #104 please reopen if I'm mistaken.