iconara / puck

Packs your JRuby app as a standalone Jar file

Puck

Puck takes your app and packs it along with all your gems and a complete JRuby runtime in a standalone Jar file that can be run with just java -jar …

Installation

Add puck and jruby-jars to your Gemfile, like this:

group :development do
  gem 'puck'
  gem 'jruby-jars', '= 1.7.18'
end

Make sure you add them to a group, otherwise they will be included in the Jar, which isn't needed and only adds to its already considerable size.

Make sure you specify a specific version of JRuby, and that it's the same as the one you're using locally. If you don't want to depend on jruby-jars for some reason there are ways to specify a path to jruby-complete.jar, see below for instructions.

Requirements

Puck uses Bundler to figure out your gem dependencies, so if you're not using Bundler, Puck will not work.

Puck also requires you to specify the version of JRuby you want to bundle. The easiest way to do that is to add jruby-jars as a dependency, as described above, or provide your own jruby-complete.jar, as described below.

Usage

You can use Puck either from the command line, or from your own code, for example in a Rakefile. See below for examples, and configuration.

Once you have a Jar file you can run your application like this:

java -jar path/to/app.jar name-of-bin-script arg1 arg2

where name-of-bin-script is the name of a file from your app's bin directory. Any subsequent arguments will be passed to this script as if it was called directly from the command line. Everything your application needs to run will be included in the Jar, all gems and a complete JRuby runtime.

Puck also exposes all gem's bin files, as well as JRuby's, so if you have Pry among your (included) gems you can do this

java -jar path/to/app.jar pry

to get a Pry session that can access your application's code.

Creating a Jar from the command line

Just run puck and it will build a Jar file from your app. The Jar will be placed in a directory called build in your application's root directory.

Creating a Jar with Rake

Probably the best way to use Puck is to create a Rake task:

task :dist do
  Puck::Jar.new.create
end

As you can see the code to create a Jar file is tiny, you can easily integrate this with other tools like Thor or your custom build toolchain (for an example of this see Jarå).

Configuration

Puck has some sane defaults, and let's you override most of them. It will determine the name of your application from the current working directory (which will be the directory containing your Rakefile if you run it from a Rake task) and it will include all files in bin and lib automatically.

If you want to include files that are not in bin or lib you can pass in the :extra_files option:

Puck::Jar.new(extra_files: Dir['config/*.yml']).create

or using the command line:

puck --extra-files config/*.yml

The more advanced :extra_files feature, where the destination paths within the Jar file is specified as described below, is not available as a command line option.

In some circumstances you need to merge another Jar or zip file with the Jar created by Puck. This can be useful if have Java libraries, like JDBC drivers, that need to be available to the top level class loader. This is possible using the :merge_archives option:

Puck::Jar.new(merge_archives: Dir['external.jar']).create

or using the command line:

puck --merge-archives external.jar

The more advanced :merge_archives feature, where the destination paths within the Jar file is specified as described below, is not available as a command line option.

These are the options that you can set (check the API documentation for Puck::Jar#initialize for full documentation):

• :gem_groups: the groups from your Gemfile to include, defaults to :default (which is all gems that aren't in an explicit group). Don't include the group that contains puck and jruby-jars.
• :app_dir: your application's root directory, useful to set if it isn't the current working directory (and you're not using Rake).
• :app_name: the name of your application, it defaults to the name of the current working directory (and if you change that you don't need to change this too, you only need this option if you want a different name than the base directory's).
• :build_dir: defaults to build, but if you want the Jar file to end up somewhere else you can change it with this option.
• :jruby_complete: if you don't want to depend on the jruby-jars gem for some reason you can provide the path to your own jruby-complete.jar
• :extra_files: a list of files to include in the Jar. The option can be either an Array, in which case paths must be below the :app_dir, or a Hash, in which case the file specified by the key is included at the path specified by the corresponding value.
• :merge_archives: a list of Jars to be merged into the Jar. The option can be either an Array, in which case the source Jar or zip file will be merged at the root, or a Hash, in which case the Jar specified by the key is merged at the path specified by its value. Signature files ('META-INF/*.SF', 'META-INF/*.RSA', 'META-INF/*.DSA' and 'META-INF/SIG-*') are discarded in the merge, since they describe the source Jar and will not match the Jar produced by Puck.

They can also be specified on the command line (e.g. puck --build-dir dist).

Gotchas

Bundler

You can't have require 'bundler' or require 'bundler/setup' or anything else that references Bundler in your code.

Even if Puck uses Bundler to determine which gems to pack into the Jar it doesn't pack Bundler. Bundler is extremely opinionated, and assumes that it has full control, so creating an environment where it can run is not easy (just see the tests for this project to get an idea of what lengths you need to go to to get a clean environment to load Bundler). Bundler is also not need since the environment in the Jar is frozen – there are no dependencies to resolve or Gem paths to set up. Bundler, just like Puck, is a build time tool that shouldn't be required at run time.

Future versions of Puck may change this and make require 'bunder/setup' work, even though Bundler is not included, just to make it easier to run your code outside of the Jar – in JRuby using require 'bundler/setup' is many, many times faster than doing bundle exec … since it doesn't start an extra process.

I get “Cannot run program "ant"”

You need to install Ant, most systems with a JDK come with Ant already installed, so the ant integration in JRuby seems to assume that it's always installed. brew install ant, yum install ant or apt-get install ant should all work.

Puck uses Ant primarily for its easy-to-use ZIP file merging abilities. Just like Puck, Ant is only a build-time dependency and is not included in artifact.

Answers

Why not just use Warbler?

I've found Warbler to be opinionated in an unhelpful way. For example, if you have a config.ru in your application's root directory but don't want to create a War file you need to monkeypatch two classes, one of them seemingly unrelated to War files.

If Warbler works for you, you should continue using it.

The Jar file is huge, is there something I can do to slim it down?

First off make sure that the jruby-jars gem is not in a group that gets included in the Jar. Puck uses one of the Jars from this gem as the basis for the Jar it creates, but it's purely a development dependency, just like Puck itself. Including it in the Jar is essentially including itself.

But if that is not the issue, the short answer is unfortunately: probably not.

The JRuby runtime with all its dependencies clocks in at 20 MiB, you could probably slim it down a little bit by removing the 1.8 standard library, but apart from that you should probably leave it. Your gems also take up quite a lot of space, but usually you don't notice because they're tucked away in some directory that you never see. The gems are bundled as-is and include everything that the gem author thought necessary to include, tests, documentation, etc. You can probably strip this, but it's included by default because many gems do weird things (I'm looking at you jruby-openssl).

Also, you're not going to put it on a floppy, you're going to send it over a network that handles megabytes per second, it's probably ok that you app is 50 MiB.

Will it work with Rails?

I have no idea. Puck should be able to package anything that has a bin file that starts it, but Rails makes a lot of assumptions. Try it (and report back), but if it doesn't work you're probably better off with Warbler. Puck wasn't designed with Rails in mind, it was designed primarily for headless services, but it runs Rack applications just fine, the integration tests package a Rack app, launches it with Puma and throws a request at it to make sure it works.

Copyright

Copyright 2013-2015 Theo Hultberg/Iconara

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

puck's Issues

Getting load error when requiring lib/somefile

I'm getting a load error when requiring my lib/somefile.
I'm not sure if I'm missing something or doing something wrong.

System specs:
OS:
Ubuntu 14.04 x64
3.13.0-27-generic

Java:
java version "1.7.0_76"
Java(TM) SE Runtime Environment (build 1.7.0_76-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.76-b04, mixed mode)

rbenv:
rbenv 0.4.0-129-g7e0e85b

jruby:
jruby 1.7.18 (1.9.3p551) 2014-12-22 625381c on Java HotSpot(TM) 64-Bit Server VM 1.7.0_76-b13 +jit [linux-amd64]

Background:
I'm creating a jruby app on ubuntu and running it on Windows 7.
When I run the app, the lib require (in bin/mytestapp) fails with

LoadError: no such file to load -- mytestapp

This happens on both Java 7 and 8 runtimes.

I'm using the following require statement in bin/mytestapp

    begin
      require 'mytestapp'
    rescue LoadError
      $LOAD_PATH.unshift File.expand_path("../../lib", __FILE__)
      puts 'LOAD_PATH:'
      $LOAD_PATH.each do |p|
        puts "  #{p}"
      end
      require 'mytestapp'
    end

I run the app on the build machine (linux) with

java -jar build/mytestapp.jar mytestapp

I see the following output:

LOAD_PATH:
  classpath:/META-INF/app.home/lib
  file:/home/jeff/projects/ruby/mytestapp/build/mytestapp.jar!/META-INF/jruby.home/lib/ruby/1.9/site_ruby
  file:/home/jeff/projects/ruby/mytestapp/build/mytestapp.jar!/META-INF/jruby.home/lib/ruby/shared
  file:/home/jeff/projects/ruby/mytestapp/build/mytestapp.jar!/META-INF/jruby.home/lib/ruby/1.9
  classpath:META-INF/gem.home/rake-10.1.0/lib
  classpath:META-INF/gem.home/builder-3.2.2/lib
  classpath:META-INF/gem.home/dbd-jdbc-0.1.6-java/lib
  classpath:META-INF/gem.home/deprecated-2.0.1/lib
  ...
  classpath:META-INF/gem.home/xml-simple-1.1.2/lib
  classpath:META-INF/gem.home/user-choices-1.1.6.1/lib

The app runs as expected (no errors) but note the first listing:

classpath:/META-INF/app.home/lib

compared to the first gem.home listing:

classpath:META-INF/gem.home/rake-10.1.0/lib

the gem.home listing doesn't contain a leading slash.

When I run the app on windows, I see the following:

C:\Apps\MyTestApp>java -jar mytestapp.jar mytestapp
LOAD_PATH:
  classpath:C:/META-INF/app.home/lib
  file:/C:/Apps/MyTestApp/mytestapp.jar!/META-INF/jruby.home/lib/ruby/1.9/site_ruby
  file:/C:/Apps/MyTestApp/mytestapp.jar!/META-INF/jruby.home/lib/ruby/shared
  file:/C:/Apps/MyTestApp/mytestapp.jar!/META-INF/jruby.home/lib/ruby/1.9
  classpath:META-INF/gem.home/rake-10.1.0/lib
  classpath:META-INF/gem.home/builder-3.2.2/lib
  classpath:META-INF/gem.home/dbd-jdbc-0.1.6-java/lib
  classpath:META-INF/gem.home/deprecated-2.0.1/lib
  ...
  classpath:META-INF/gem.home/xml-simple-1.1.2/lib
  classpath:META-INF/gem.home/user-choices-1.1.6.1/lib
LoadError: no such file to load -- mytestapp
  require at org/jruby/RubyKernel.java:1071
  require at /C:/Apps/MyTestApp/mytestapp.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
  (root) at classpath:/META-INF/app.home/bin/mytestapp:20
    load at org/jruby/RubyKernel.java:1087
  (root) at classpath:jar-bootstrap.rb:50
    each at org/jruby/RubyArray.java:1613
  (root) at classpath:jar-bootstrap.rb:46

Now the app.home path is prefixed with C:/:

classpath:C:/META-INF/app.home/lib

and it fails, as you can see.

Question: Why is the classpath prefixed with a file path? Shouldn't it just
be META-INF/whatever?

During my testing, I modified puck/lib/puck/jar.rb#create_jar_bootstrap! to
add the lib dir to the LOAD_PATH (line 148):

    def create_jar_bootstrap!(tmp_dir, gem_dependencies)
      File.open(File.join(tmp_dir, 'jar-bootstrap.rb'), 'w') do |io|
        io.puts(%(PUCK_BIN_PATH = ['/#{JAR_APP_HOME}/bin', '/#{JAR_JRUBY_HOME}/bin']))
        gem_dependencies.each do |spec|
          io.puts("PUCK_BIN_PATH << '/#{JAR_GEM_HOME}/#{spec[:versioned_name]}/#{spec[:bin_path]}'")
        end
        io.puts

        #
        # Add JAR_APP_HOME/lib to LOAD_PATH
        #
        io.puts(%($LOAD_PATH << 'classpath:#{JAR_APP_HOME}/lib'))

        gem_dependencies.each do |spec|
          spec[:load_paths].each do |load_path|
            io.puts(%($LOAD_PATH << 'classpath:#{JAR_GEM_HOME}/#{spec[:versioned_name]}/#{load_path}'))
          end
        end
        io.puts
        io.puts(File.read(File.expand_path('../bootstrap.rb', __FILE__)))
      end
    end

My app now works in both systems, and I don't have to add the lib dir to the
LOAD_PATH in bin/mytestapp. I can just do:

require 'mytestapp'

Question: Is there a reason to NOT add the lib dir to the LOAD_PATH within create_jar_bootstrap?
Will bad things happen?

Question: Maybe it would be better to make this a configuration option?

If I'm just losing it, please point me in the right direction.

Other than this issue, puck is working out great!

Gemfiles with gemspec references fail

When Gemfile contains gemspec the packaging fails.

Cannot run program "ant"

When I just run require 'puck' I get this error:

jruby-1.7.18 :001 > require 'puck'
IOError: Cannot run program "ant" (in directory "/Users/jeremy/Development/games/maze_craze"): error=2, No such file or directory
    from org/jruby/RubyIO.java:4345:in `popen'
    from /Users/jeremy/.rvm/rubies/jruby-1.7.18/lib/ruby/shared/ant.rb:5:in `load_from_ant'
    from /Users/jeremy/.rvm/rubies/jruby-1.7.18/lib/ruby/shared/ant.rb:38:in `load'
    from /Users/jeremy/.rvm/rubies/jruby-1.7.18/lib/ruby/shared/ant.rb:61:in `Ant'
    from /Users/jeremy/.rvm/rubies/jruby-1.7.18/lib/ruby/shared/ant.rb:3:in `(root)'
    from org/jruby/RubyKernel.java:1071:in `require'
    from /Users/jeremy/.rvm/rubies/jruby-1.7.18/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /Users/jeremy/.rvm/rubies/jruby-1.7.18/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:53:in `require'
    from /Users/jeremy/.rvm/gems/jruby-1.7.18@waves/gems/puck-1.0.0.pre4/lib/puck/jar.rb:1:in `(root)'
    from org/jruby/RubyKernel.java:1071:in `require'
    from /Users/jeremy/.rvm/rubies/jruby-1.7.18/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /Users/jeremy/.rvm/rubies/jruby-1.7.18/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:53:in `require'
    from /Users/jeremy/.rvm/gems/jruby-1.7.18@waves/gems/puck-1.0.0.pre4/lib/puck/jar.rb:7:in `(root)'
    from org/jruby/RubyKernel.java:1071:in `require'
    from /Users/jeremy/.rvm/gems/jruby-1.7.18@waves/gems/puck-1.0.0.pre4/lib/puck.rb:1:in `(root)'
    from /Users/jeremy/.rvm/gems/jruby-1.7.18@waves/gems/puck-1.0.0.pre4/lib/puck.rb:10:in `(root)'
    from org/jruby/RubyKernel.java:1107:in `eval'
    from /Users/jeremy/.rvm/rubies/jruby-1.7.18/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1:in `(root)'
    from /Users/jeremy/.rvm/rubies/jruby-1.7.18/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:135:in `require'
    from (irb):1:in `evaluate'
    from org/jruby/RubyKernel.java:1507:in `loop'
    from org/jruby/RubyKernel.java:1270:in `catch'
    from org/jruby/RubyKernel.java:1270:in `catch'
    from /Users/jeremy/.rvm/rubies/jruby-1.7.18/bin/irb:13:in `(root)'jruby-1.7.18 :002 >

example run fails

Iv tried to run the example run used in the integration spec, it does run outside of the jar but fails when being run via:

java -jar example_app.jar server

Channels.java:67:in `checkNotNull': java.lang.NullPointerException: "in" is null!
    from Channels.java:347:in `newChannel'
    from ChannelDescriptor.java:829:in `open'
    from RubyFile.java:1242:in `sysopen'
    from RubyFile.java:1199:in `sysopenInternal'
    from RubyFile.java:1150:in `openFile19'
    from RubyFile.java:334:in `initialize19'
    from RubyFile$INVOKER$i$0$2$initialize19.gen:-1:in `call'
    from CachingCallSite.java:286:in `cacheAndCall'
    from CachingCallSite.java:81:in `callBlock'
    from CachingCallSite.java:85:in `call'
    from RubyClass.java:876:in `newInstance'
    from RubyIO.java:1144:in `open'
    from RubyIO$INVOKER$s$0$0$open.gen:-1:in `call'
    from DynamicMethod.java:217:in `call'
    from CachingCallSite.java:356:in `cacheAndCall'
    from CachingCallSite.java:213:in `callBlock'
    from CachingCallSite.java:222:in `callIter'
    from CallTwoArgBlockNode.java:62:in `interpret'
    from NewlineNode.java:105:in `interpret'
    from ASTInterpreter.java:74:in `INTERPRET_METHOD'
    from InterpretedMethod.java:182:in `call'
    from DefaultMethod.java:186:in `call'
    from CachingCallSite.java:326:in `cacheAndCall'
    from CachingCallSite.java:170:in `call'
    from CallOneArgNode.java:57:in `interpret'
    from LocalAsgnNode.java:123:in `interpret'
    from NewlineNode.java:105:in `interpret'
    from BlockNode.java:71:in `interpret'
    from RootNode.java:129:in `interpret'
    from ASTInterpreter.java:121:in `INTERPRET_ROOT'
    from Ruby.java:834:in `runInterpreter'
    from Ruby.java:2674:in `loadFile'
    from ExternalScript.java:66:in `load'
    from LoadService.java:359:in `load'
    from RubyKernel.java:1081:in `loadCommon'
    from RubyKernel.java:1073:in `load19'
    from RubyKernel$INVOKER$s$0$1$load19.gen:-1:in `call'
    from DynamicMethod.java:209:in `call'
    from DynamicMethod.java:205:in `call'
    from CachingCallSite.java:326:in `cacheAndCall'
    from CachingCallSite.java:170:in `call'
    from classpath:jar-bootstrap.rb:39:in `block_0$RUBY$__file__'
    from classpath_3a_jar_minus_bootstrap$block_0$RUBY$__file__:-1:in `call'
    from CompiledBlock19.java:135:in `yield'
    from Block.java:142:in `yield'
    from RubyArray.java:1610:in `eachCommon'
    from RubyArray.java:1617:in `each'
    from RubyArray$INVOKER$i$0$0$each.gen:-1:in `call'
    from CachingCallSite.java:316:in `cacheAndCall'
    from CachingCallSite.java:145:in `callBlock'
    from CachingCallSite.java:154:in `callIter'
    from classpath:jar-bootstrap.rb:36:in `__file__'
    from classpath:jar-bootstrap.rb:-1:in `load'
    from Ruby.java:807:in `runScript'
    from Ruby.java:800:in `runScript'
    from Ruby.java:669:in `runNormally'
    from Ruby.java:518:in `runFromMain'
    from Main.java:390:in `doRunFromMain'
    from Main.java:279:in `internalRun'
    from Main.java:221:in `run'
    from Main.java:201:in `main'
    from JarBootstrapMain.java:57:in `main'

Im using:

java version "1.7.0_25"
OpenJDK Runtime Environment (IcedTea 2.3.10) (7u25-2.3.10-1ubuntu0.13.04.2)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)

LocalJumpError when running simple script

A LocalJumpError is raised when running a script. I see this error for both JRuby 1.7.19 and 9.0.0.0. It looks like e537f0f is to blame. I suspect we have not seen this before because most processes are long running, and if the script explicitly exits the return statement will never be reached.

$ java -jar build/foo.jar foo
Hello
LocalJumpError: unexpected return
  block in classpath:jar-bootstrap.rb at classpath:jar-bootstrap.rb:11
                                 each at org/jruby/RubyArray.java:1560
                                <top> at classpath:jar-bootstrap.rb:6
$ cat bin/foo
#!/usr/bin/env ruby
puts 'Hello'

[Feature Request] Integration with l4j ?

I was wondering if puck could be integrated with launch4j to enable users to produce .exe and all, like other similar libraries offer. I am ready to write codes for it.

Puck for library jar creation

I'm working on a Jruby Vertx project and because of Vertx fatjar creation I have concluded that I need a way of packing up gems into a jar in such a way that those gems are in Jruby loadpath, to be clear the final jar which is driven by the shadowJar plugin is meant to pack up all java dependencies along with a jar that represents needed Jruby gems

I have struggled to find a beast that does all of that.

Your readme does not directly mention whether the resulting jar from the Puck packaging process could indeed be used as a library jar but maybe you could elaborate ?

thanks once again

quick question

sorry for the intrusion, just thought I ask for your perspective
I'm trying to get away from the use of Warbler and its deployment intoJava app servlet containers such as Jetty

I have a Sinatra app for which I have currently defined a config.ru again packed to run in Jetty by Warbler but instead I want to explore running Puma , from checking out your code it seems then that the config.ru is irrelevant but rather the starting endpoint should be something like your server script i.e.

#!/usr/bin/env ruby

$LOAD_PATH << File.expand_path('../../lib', FILE)

require 'rack/handler/puma'
require 'example_app/hello_world'

config_path = File.expand_path('../../config/app.yml', FILE)
config = YAML.load_file(config_path)
port = config['port'].to_i
app = ExampleApp::HelloWorld.new(File.basename($0) + ": " + config['message'])
Rack::Handler::Puma.run(app, :Port => port)

I also will then assume that somewhere in the packaging thru Puma you specify this file to be your start point and thus I will scan your repo to glean how that is setup?

Am I missing something ? any words of advice to facilitate are appreciated.

Finally, thank you for writing this potentially very useful utility.

License missing from gemspec

Some companies will only use gems with a certain license.
The canonical and easy way to check is via the gemspec
via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

Bundler now generates gems with a default 'MIT' license. There is even a License Finder
to help companies ensure all gems they use meet their licensing needs. This tool depends on license information being available in the gemspec.
Including a license in your gemspec is a good practice, in any case.

If you need help choosing a license, github has created a license picker tool

How did I find you?

I'm using a script to collect stats on gems, originally looking for download data, but decided to collect licenses too,
and make issues for gemspecs not specifying a license as a public service :)
So far it's going pretty well.
I've written a blog post about it

Exit from jar-bootstrap.rb not a good idea

v1.2.0 via #18 changed it so that jar-bootstrap.rb ran Kernel.exit once the bin script had run. For the majority of applications this won't be a problem since this is roughly what Ruby does anyway. However, if an application starts Java (non-daemon) threads and relies on Java's behaviour of not exiting until the last non-daemon thread has finished, this will cause problems. One example where it does is in Rubydoop, see iconara/rubydoop#38.

The behaviour was changed in v1.2.1 via #26.

Because v1.2.0 is broken for certain applications, and to avoid gems like Rubydoop to have to specify a dependency on v1.2.1 or greater (thus disallowing v1.1.x and earlier which also work fine), it should be yanked.

NameError: uninitialized constant Puck::DependencyResolver::StringIO

The require "stringio" was placed after it was actually called. Hence it threw an error saying "NameError: uninitialized constant Puck::DependencyResolver::StringIO".

Refer #10

Filter out puck and jruby-jars

If you put puck and jruby-jars in the default group, or in a group that you explicitly include in the gem, they will get bundled.

Maybe it's not bad if puck gets bundled, I guess there might be a use case where you build a stand-alone app that can build other stand-alone apps. But to bundle jruby-jars doesn't make much sense since everything is bundled inside of the JAR from that gem by Puck.

including third party jar

I didn't find a way to include third party jars it would be great to be able to do so

Thanks