java_library() breaks package path when resource is genfile() about buck

[bolinfest]

I just pushed a new (undocumented) rule named sh_binary() to the repo:

6b66b36

It behaves a little differently from genrule(), so I'm curious to get your thoughts.

Basically, within the script that you specify as the main argument to sh_binary(), you can assume the existence of the environment variable $BUCK_PROJECT_ROOT. That is the root of the directory that should contain all of the resources you passed to sh_binary(). (It's a read-only symlink to the actual file.) This directory should be a unified view of buck-out/gen/ and your main tree, filtered by the resources you declared in sh_binary().

(Though as I type this, as things are currently implemented for generated files, I think you would have $BUCK_PROJECT_ROOT/buck-out/gen/foo/bar.txt instead of $BUCK_PROJECT_ROOT/foo/bar.txt, which is a bug.)

Given this approach for sh_binary(), do you think we should do something similar for genrule()? We currently have $SRCDIR, but that [roughly] refers to the directory that contains the build file rather than the root of the repo.

[spearce]

I expected genrule() to have $SRCDIR setup as a tree of symlinks to the named srcs, with the path prefix of the BUCK file containing the genrule() stripped off. i.e. the current behavior of genrule(). Which doesn't always work thanks to some bugs in the way canonical vs. absolute paths vs. "." as the project root are handled, but that is issue #39.

For a genfile() used as src for a genrule() I expect the same thing. If this BUCK file says genfile("foo/bar.txt") then my genrule should see $SRCDIR/foo/bar.txt. I shouldn't need to write a sh_binary() to be invoked from a genrule() to get there.

But this is fairly hideous either way. What I am really trying to do is generate an XML file in a genrule() and then package that into a JAR that I can pass through to another java_library() as a dep. I do that right now by just feeding the genfile(...) path as part of the resources list of a java_library() and carry on. Looks like 2aa814b broke this, so I need to rewrite the rule. If I have to fall down to a sh_binary() I might as well just replace the java_library() rule with an additional step in my genrule() to produce a JAR output instead of an XML file as output.

So then my example becomes:

genrule(
name = 'gen',
cmd = 'mkdir -p $TMP/com/example/;' +
  'echo foo>$TMP/com/example/foo.properties;' +
  'cd $TMP; zip -qr $OUT .',
deps = [],
out = 'foo.jar',
)
prebuilt_jar(
  name = 'lib',
  binary_jar = genfile('foo.jar'),
  deps = [':gen'],
)

we already have a few such rules in our tree. I guess I should convert this java_library() rule to this form instead?

[bolinfest]

Short answer: no decision has been made, so don't go rewriting all of your
gen rules just yet! Use .buckversion to stay at a point that doesn't have
the regression for now (though I realize that may not allow you to get the
recent python speed wins).
On Jul 24, 2013 9:29 PM, "Shawn O. Pearce" [email protected] wrote:

I expected genrule() to have $SRCDIR setup as a tree of symlinks to the
named srcs, with the path prefix of the BUCK file containing the genrule()
stripped off. i.e. the current behavior of genrule(). Which doesn't always
work thanks to some bugs in the way canonical vs. absolute paths vs. "." as
the project root are handled, but that is issue #39#39
.

For a genfile() used as src for a genrule() I expect the same thing. If
this BUCK file says genfile("foo/bar.txt") then my genrule should see
$SRCDIR/foo/bar.txt. I shouldn't need to write a sh_binary() to be invoked
from a genrule() to get there.

But this is fairly hideous either way. What I am really trying to do is
generate an XML file in a genrule() and then package that into a JAR that I
can pass through to another java_library() as a dep. I do that right now by
just feeding the genfile(...) path as part of the resources list of a
java_library() and carry on. Looks like 2aa814bhttps://github.com/facebook/buck/commit/2aa814be5a646e5cd8c0480bb68e20007b5df4e7broke this, so I need to rewrite the rule. If I have to fall down to a
sh_binary() I might as well just replace the java_library() rule with an
additional step in my genrule() to produce a JAR output instead of an XML
file as output.

So then my example becomes:

genrule(
name = 'gen',
cmd = 'mkdir -p $TMP/com/example/;' +
'echo foo>$TMP/com/example/foo.properties;' +
'cd $TMP; zip -qr $OUT .',
deps = [],
out = 'foo.jar',
)
prebuilt_jar(
name = 'lib',
binary_jar = genfile('foo.jar'),
deps = [':gen'],
)

we already have a few such rules in our tree. I guess I should convert
this java_library() rule to this form instead?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/41#issuecomment-21529831
.

[spearce]

Gerrit already requires a patched version of buck that is 4 commits ahead of your tree. I just keep rebasing and fixing the same damn merge conflicts. Now we have 5 commits ahead, the latest one replaces "." as the top of the source tree with the absolute path because "." as the project root late becomes "/./" in some path strings and buck isn't fully prepared for this. sigh

The java_library() not accepting a genfile() in a package name is now patched around by having the genrule() spit out a JAR by running jar itself. sigh

Despite things like this, buck is still better than Maven.

[bolinfest]

"still better than Maven" is our motto
On Jul 26, 2013 12:27 AM, "Shawn O. Pearce" [email protected]
wrote:

Gerrit already requires a patched version of buck that is 4 commits ahead
of your tree. I just keep rebasing and fixing the same damn merge
conflicts. Now we have 5 commits ahead, the latest one replaces "." as the
top of the source tree with the absolute path because "." as the project
root late becomes "/./" in some path strings and buck isn't fully prepared
for this. sigh

The java_library() not accepting a genfile() in a package name is now
patched around by having the genrule() spit out a JAR by running jar
itself. sigh

Despite things like this, buck is still better than Maven.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/41#issuecomment-21602181
.

[sdwilsh]

Is this still an issue for you @spearce?

[Coneko]

genfile doesn't even exist anymore.