sous-chefs / kafka

Development repository for the kafka cookbook

kafka cookbook

Installs and configures Kafka >= v0.8.1.

Initially based on the Kafka cookbook released by Webtrends (thanks!), but with a few notable differences:

• does not depend on runit cookbook.
• does not depend on zookeeper cookbook, thus it will not search for nodes with a specific role or such, that is left up to you to decide.
• intended to be used by wrapper cookbooks.

Maintainers

This cookbook is maintained by the Sous Chefs. The Sous Chefs are a community of Chef cookbook maintainers working together to maintain important cookbooks. If you’d like to know more please visit sous-chefs.org or come chat with us on the Chef Community Slack in #sous-chefs.

Requirements

This cookbook does not depend on any specific cookbooks, but it requires that java is installed on the system, thus the java cookbook is recommended.

Furthermore, Kafka requires ZooKeeper for coordination, and this cookbook does not install or manage ZooKeeper to any extent. A general recommendation is to not run Kafka and ZooKeeper on the same hardware.

Chef 13+

Platform

• Amazon Linux
• CentOS 6 and 7
• Debian 7 and 8
• Fedora 21
• Ubuntu 14.04

The platforms / versions listed above are the ones that are included in .kitchen.yml and/or tested in the wild, so it might work on other platforms as well, YMMV.

Attributes

In order to keep the README in some kind of manageable state (and thus in sync with attributes), attributes are documented inline (in the attribute files that is).

Attributes concerning configuration of a Kafka broker are to be set under the broker namespace, as such:

node.default['kafka']['broker']['log.dirs'] = %w[/tmp/kafka-logs]
node.default['kafka']['broker']['num.io.threads'] = 4
node.default['kafka']['broker']['zookeeper.connect'] = %w[localhost:2181]

The attribute names match the configuration names that Kafka uses to make it easier to support new and old versions of Kafka simultaneously, by avoiding "hardcoded" attribute names for configuration options, so please refer to the official documentation for the release at your hand.

Refer to the official documentation for the version of Kafka that you're installing. Documentation for the latest release can be found over here.

Recipes

This section describes the different recipes that are available.

default

Downloads and installs Kafka from the official binary releases. Defaults to installing v1.1.1 of Kafka.

Controlling restart of Kafka brokers in a cluster

Any changes made to the broker configuration could result in a restart of the Kafka broker, if the kafka.automatic_restart attribute is set to true. If Chef runs as a daemon on all of the nodes this could result in all of the Kafka brokers being brought down at the same time, resulting in unavailability of service.

If unavailability is an issue, this cookbook provides an option to implement custom logic to control the restart of Kafka brokers so that not all of the brokers in a cluster are stopped at the same time. For example the custom logic can be something along the lines of acquiring a lock in ZooKeeper and when held the broker is allowed to restart. Be aware that a restart might take quite some time if you're using controlled shutdown and have a lot of partitions, and Chef usually have some timeout for each resource.

By default the resources in the _coordinate recipe performs the start/restart of the kafka service. If custom logic needs to be implemented, this recipe can be replaced with another recipe, but don't forget to update the kafka.start_coordination.recipe attribute.

The only requirement is that the new recipe has a ruby_block resource with 'coordinate-kafka-start' as ID. The following is a sample recipe that shows roughly what one can do with this feature.

ruby_block 'coordinate-kafka-start' do
  block do
    Chef::Log.info 'Custom recipe to coordinate Kafka start/restart'
  end
  action :nothing
  notifies :create, 'ruby_block[restart-coordination]', :delayed
end

ruby_block 'restart-coordination' do
  block do
    Chef::Log.info 'Implement the process to coordinate the restart, like using ZK'
  end
  action :nothing
  # `kafka_service_resource` is a helper method that will return the correct
  # resource name depending on whether you're using `runit` or not.
  notifies :restart, kafka_service_resource, :delayed
  notifies :create, 'ruby_block[restart-coordination-cleanup]', :delayed
end

ruby_block 'restart-coordination-cleanup' do
  block do
    Chef::Log.info 'Implement any cleanup logic required after restart like releasing locks'
  end
  action :nothing
end

Please refer to issue #58 for background of this feature.

FAQ

Kafka dies for no apparent reason (ulimit)

Depending on your system / infrastructure setup you might run into issues with Kafka just sporadically dying for no obvious reason. One thing you might want to look into is the file handle limit as Kafka tend to keep a lot file handles open due to socket connections (depends on the number of brokers, producers and consumers) and the actual data log files (depends on the number of partitions and log segment and/or log roll settings).

It's possible to set a specific ulimit for Kafka using the node['kafka']['ulimit_file'] attribute. If this value is not set, Kafka will use whatever the system default is, which as stated previously might not be enough, so it might be wise to set a higher limit.

How do I get started locally? (minimal required setup)

If you want to hit the ground running and just setup a single broker (or a cluster for that matter) locally, these are the necessary broker attributes that needs to be set (assumes that you have ZooKeeper running on port 2181 locally):

node.default['kafka']['broker']['zookeeper.connect'] = 'localhost:2181'
# This shouldn't normally be necessary, but might need to be set explicitly
# if you're having trouble connecting to the brokers.
node.default['kafka']['broker']['hostname'] = '127.0.0.1' # or perhaps 'localhost'

If you plan on running a cluster locally you will want to set separate values for the following configuration options:

node.default['kafka']['broker']['broker.id'] = <id>
node.default['kafka']['broker']['port'] = <port>
node.default['kafka']['broker']['log.dirs'] = <dir path>

Other than that things should work as expected, though depending on what platform you're running on, you might want to change the install and config directories as well. See attributes/default.rb and recipes/_defaults.rb for the default path regarding directories that Kafka will use.

Kafka killed prematurely (kill timeout)

When using controlled shutdown and either systemd or upstart as init system you might run into issues with Kafka being killed before it has managed to shutdown completely, resulting in long recovery times.

Not sure if it's possible to configure either systemd or upstart to not automatically kill processes, but a workaround is to set kafka.kill_timeout to a sufficiently high value.

Contributing

• Fork the repository on Github
• Create a named feature branch (like add-component-x)
• Write your change
• Check that your change works, for example with Vagrant
• Submit a Pull Request using Github

Contributors

This project exists thanks to all the people who contribute.

Backers

Thank you to all our backers!

Sponsors

Support this project by becoming a sponsor. Your logo will show up here with a link to your website.

kafka's Issues

Why doesn't the cookbook start kafka service?

I'm working on a pull request to add support for Ubuntu.

The configure.rb recipe enables, but doesn't start the service.

service 'kafka' do
  supports start: true, stop: true, restart: true
  action [:enable]
end

Is there a reason why the start action isn't included by default?

Override download url

Currently it's possible to override the base_url attribute, but the recipes still enforce some form of standard, i.e. base_url/version/filename, where filename actually has a specific convention.

Has anyone tried to see if this will work with 0.8.2 beta?

Just wondering if anyone has tried this cookbook to deploy 0.8.2 Beta?

Proposal for combining some attributes

Howdy. We're starting to play with this cookbook and I'd like to propose renaming a few attributes to make it a bit easier to combine like-parameters. I was thinking something like,

• node[:kafka][:broker]
• node[:kafka][:topic]
• node[:kafka][:consumer]
• node[:kafka][:producer]

This opens the door for replacing templates with methods which just render the config correctly. It's my first time deploying Kafka, so the above organization might not make complete sense. Something like node[:kafka][:config] under which everything else is buried might be good, alternatively.

Anyway, would be willing to put in the time to PR this, but wanted to check if this is something you're interested in first.

Enhancement req: enable support to control Kafka server restarts

One of the scenarios which Kafka users will come across often is the restart of Kafka servers in a cluster to deploy new changes with minimal down time. When chef-client is run in batch on all the nodes (scheduled as daemons), there is a possibility that many kafka servers are brought down at the same time due to restart for config changes resulting in availability/performance issue. The worst case scenario would be all but one Kafka server is brought down.
Another way to alleviate this scenario will be to control the number of servers which can be restarted at a time which can vary by users based on how many replication they set for the topic partitions. Can we provide some feature to be able to control the restart of the particular Kafka node. One way would be to

• conditionally include a new recipe file instead of the "service" resource in " _configure.rb
• The default "new recipe" file will have a dummy recipe which will get executed on any condition that currently triggers the Kafka restart.
• When the dummy recipe gets executed, it notifies the restart "kafka service" resource to perform the restart.

This will help users to override the default "new" recipe file with their own recipe file and logic to control whether the Kafka server can be restarted or not. Let me know if you have any questions/thoughts.

Refactor / reduce duplication between init scripts

It should be possible to extract some common logic into partials to DRY things up.

Default broker config. options are defined using `underscore` notation

Realized that, currently, host_name and broker_id are defined as follows in attributes/default.rb:

default.kafka.broker.broker_id = node.ipaddress.gsub('.', '').to_i % 2**31

default.kafka.broker.host_name = node.hostname

To override these one is forced to use the underscore notation, or otherwise they'll appear twice in the rendered configuration file.
Obviously this should be oblivious to how one defines their configuration, or there should be one and only one way to structure the broker configuration attributes.

v0.1.0 tag references Oct 27 2013 commit

currently points at c014151

expected it to point at 9611b4d

default recipe does not force install_method to a Symbol

When overriding install_method attribute in .kitchen.yml it's defined as a string, which will make the default recipe explode. Sure, it's possible to enforce symbols in Ruby YAML, but nevertheless it should use #to_sym to make sure.

`source` recipe broken on 0.8.2-beta

Something to do with gradle:

---- Begin output of tar zxf /tmp/kitchen/cache/kafka-0.8.2-beta-src.tgz &&     cd kafka-0.8.2-beta-src &&     ./gradlew -PscalaVersion=2.9.2 releaseTarGz -x signArchives
----
STDOUT:
STDERR: Error: Could not find or load main class org.gradle.wrapper.GradleWrapperMain
---- End output of tar zxf /tmp/kitchen/cache/kafka-0.8.2-beta-src.tgz &&     cd kafka-0.8.2-beta-src &&     ./gradlew -PscalaVersion=2.9.2 releaseTarGz -x signArchives
----

Start the zookeeper as part of kafka service

I am using this recipe in vagrant VM to install and run kafka as a service.
I notice that kafka fails since zookeeper was never started on that VM using service.

[2015-03-04 19:24:01,515] FATAL Fatal error during KafkaServerStable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
        at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)

If I manually run following

/opt/kafka/bin/zookeeper-server-start.sh /opt/kafka/config/zookeeper.properties &

then it works.

What should be the right approach for bringing the zookeeper automatically as well ?
I only have following in my vagrant file:
config.vm.provision "chef_solo" do |chef|
chef.cookbooks_path = "cookbooks"
chef.add_recipe "kafka::default"
end

DRY integration tests

Use shared_examples_for and symlink a support directory or something.

Question - How does this cookbook differ from others

There are a couple of kafka cookbooks on the supermarket

For example

https://github.com/MathYourLife/apache_kafka

Does this cookbook exist because of a deficiency in the other kafka cookbooks? Are you planning on maintaining this one for a while?

DRY up {source,binary} recipes

There's quite some duplication between the recipes that could be DRY'ed up using a mix of definitions and "library" methods.

Any plans for a changelog?

I'm running cookbook 0.7.1 and I'm upgrading to cookbook 0.8.0. It isn't easy to tell exactly what changed just by looking at the commit history.

Any plans to include a changelog file that lists what changed?

User/group management issues

I think user/group management should be optional. Usually I deal with user management beforehand in a wrapping recipe.

Especially if you define node[:kafka][:user] = root or some other existing user.

Then you end up with noshell...I had restore shell (sudo chsh -s /bin/bash root)

A quick and simple workaround could be to add a not_if

group node[:kafka][:group] do
  not_if {node[:kafka][:user] == "root" }
end

user node[:kafka][:user] do
  gid   node[:kafka][:group]
  shell '/sbin/nologin'
  supports(manage_home: false)
  not_if {node[:kafka][:user] == "root" }
end

Set a default zookeeper.connect value

I'm not very familiar with kafka, but wanted to test out a local cluster. I make a quick wrapper cookbook to get a zookeeper cluster set up, but then it wasn't straightforward to get kafka to connect to it. I had to google around to figure out the right statement to add to the server.properties. What do you think about adding the following attribute?
default[:kafka][:broker][:'zookeeper.connect'] = 'localhost:2181'

Broker log.dirs does not create correct string

With the following attribute set:

set[:kafka][:num_partitions] = 8
set[:kafka][:log][:dirs] = node[:kafka][:num_partitions].times.inject([]) {|dirs, num| dirs << "/var/kafka/logs/#{num}"}

Which I've confirmed is an array in debugging by adding a chef warn log ends up with the output during a chef-client run:

# node[:kafka][:log][:dirs].inspect
WARN: Doing logs in: ["/var/kafka/logs/0", "/var/kafka/logs/1", "/var/kafka/logs/2", "/var/kafka/logs/3", "/var/kafka/logs/4", "/var/kafka/logs/5", "/var/kafka/logs/6", "/var/kafka/logs/7"]

The output in server.properties is:

log.dirs=/var/kafka/logs/0=/var/kafka/logs/1=/var/kafka/logs/2=/var/kafka/logs/3=/var/kafka/logs/4=/var/kafka/logs/5=/var/kafka/logs/6=/var/kafka/logs/7

So the .join(',') in the server.properties.erb is not being evaluated before the .join('=') in partials/option.erb.

Perhaps doing the same thing you did with the zookeeper_connect_string would be a good way.

README should include note about coordination recipe

Currently there's no mention of how the "coordination" recipe is supposed to work or how to work with it.

runit support

A lot of people seem to use runit, so why not include support for that as well.

Run kafka as configured user

Kafka currently runs as root, though it would be desirable to have it run as the user configured in node[:kafka][:user] instead.

Requires changes to init scripts, which I'm not particularly interested in doing right now. It would be easy for the upstart script, but not for the sysv one(s).

Restart coordination - is called everytime

Hello,

I'm using the restart coordination as described in the readme, e.g.

ruby_block 'coordinate-kafka-start' do
  block do
    Chef::Log.info 'Custom recipe to coordinate Kafka start/restart'
  end
  action :create
  notifies :create, 'ruby_block[restart-coordination]', :delayed
end

ruby_block 'restart-coordination' do
  block do
    Chef::Log.info 'Implement the process to coordinate the restart, like using ZK'
  end
  action :nothing
  notifies :restart, 'service[kafka]', :delayed
  notifies :create, 'ruby_block[restart-coordination-cleanup]', :delayed
end

service 'kafka' do
  provider kafka_init_opts[:provider]
  supports start: true, stop: true, restart: true, status: true
  action kafka_service_actions
end

ruby_block 'restart-coordination-cleanup' do
  block do
    Chef::Log.info 'Implement any cleanup logic required after restart like releasing locks'
  end
  action :nothing
end

The problem I'm seeing is that no matter if any changes have been made to the kafka config, kafka is being restarted. Should this be happening? Has anyone else seen this?

It would be preferable if kafka was only notified to restart when an aspect of the configuration which required a restart was changed.

Configurable init-style

Would be good to support regular init.d scripts, upstart, systemd and what else people are using. Preferably using the "regular" approach of an attribute and switch statement.

Furthermore runit support should be considered as well.

Add Fedora support

Override default log path for kafka GC logs

GC logs are currently placed in /opt/kafka/logs and should rather be in /var/log/kafka.

Can we add support for kafka version 0.8.2 ?

Kafka version 0.8.2 is the latest.
Can we add support to install that ?

Controlled shutdown and actual shutdown

I feel like there should be some special logic in the init scripts for when one has enabled controlled shutdown as sometimes it's going to take way more than 5 seconds to shutdown a broker.

Allow auto setting of broker id

The latest version of kafka allows the auto configuration of broker id, by not providing one in the configuration file.

As per https://thisdataguy.com/2016/02/04/about-big-kafka-broker-id/

Currently the cookbook will generate a broker id if none is defined with the following:

 node.default.kafka.broker.broker_id = node.ipaddress.gsub('.', '').to_i % 2**31

However, with the latest version of kafka, this auto generated number is higher than the default for:

reserved.broker.max.id

Which means kafka fails to start.

So either the cookbook needs to allow this parameter to be configured, or if no broker id is provided, do not attempt to automatically generate one via the above method using the IP address.

Support for Kafka v0.8.1

Should go through if there's been any changes to start up scripts or whatever. This also relates to #18.

Incomprehensible unit tests

Due to attempting to support various configuration parameters available in the three (!) last versions of Kafka, the unit tests are quite... messy.

Let's try to clean 'em up.

Set ulimit number by default

I think the default value for the ulimit should be set by default.

After days of crashes, and various conversations in the IRC chanel, I discovered that kafka was exceeding the maximum number of open files.

Someone pointed out that this cookbook can set the ulimit right inside of the upstart file.

bmhatfield/chef-ulimit#38

Even users with simple configurations will run out of open files. Should the limit be set by default?

Support for additional server logs (state-change.log controller.log etc)

The current cookbook log4j.properties template is fairly minimal and just logs stdout to a file. The stock kafka distribution includes a few other logging configs that are pretty useful. Is there any interest in supporting those by default in the cookbook? If so, I can throw together a PR. Otherwise, it's something that I'll put in a separate wrapper cookbook.

See: https://github.com/apache/kafka/blob/0.8/config/log4j.properties

Need to change JMX port attribute as broker attribute

Currently the attribute for JMX port is not a broker attribute even though JMX port is one of the broker properties. Need to change the attribute default.kafka.jmx_port to default.kafka.broker.jmx_port to make it consistent with other attributes related to broker properties.

Restart service when config files change.

When any of the configs change a deferred restart should be sent to the service so it can pick up on the config.

Otherwise, we get a current configuration on disk while the process runs with an out of date config.

kafka doesn't get enabled when using upstart

ulimit file descriptor support

The default ulimit for file descriptors on many Linux systems is not sufficient for production workloads, various scripts set it high, e.g. https://github.com/linkedin/sin/blob/master/start_script/linux/kafka#L61 sets it to 65535

Let's make this something we can set

md5 is not case sensitive

The md5 provided by kafka is now entirely upper case. If you provide a md5 sum with uppper case letters, the download will fail.

For example, with 8.2.1 the md5 is

 446eab1f5329eb03662926aa1cb0845d

Changing one or more letters to uppercase will throw the following error.

           ================================================================================
           Error executing action `create` on resource 'kafka_download[/tmp/kitchen/cache/kafka_2.10-0.8.2.1.tgz]'
           ================================================================================

           SystemExit
           ----------
           ruby_block[kafka-validate-download] (/tmp/kitchen/cookbooks/kafka/providers/download.rb line 20) had an error: SystemExit: exit

           Cookbook Trace:
           ---------------
           /tmp/kitchen/cookbooks/kafka/providers/download.rb:24:in `block (3 levels) in class_from_file'

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cookbooks/kafka/recipes/_install.rb

             6: kafka_download kafka_local_download_path do
             7:   source kafka_download_uri(kafka_tar_gz)
             8:   checksum node.kafka.checksum
             9:   md5_checksum node.kafka.md5_checksum
            10:   not_if { kafka_installed? }
            11: end
            12:

           Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cookbooks/kafka/recipes/_install.rb:6:in `from_file'

           kafka_download("/tmp/kitchen/cache/kafka_2.10-0.8.2.1.tgz") do
             action :create
             updated true
             updated_by_last_action true
             retries 0
             retry_delay 2
             default_guard_interpreter :default
             declared_type :kafka_download
             cookbook_name :kafka
             recipe_name "_install"
             source "https://archive.apache.org/dist/kafka/0.8.2.1/kafka_2.10-0.8.2.1.tgz"
             checksum "89ede9ae0f51f7163c4140d8ab43fcedf8eb3066bb8058f3d97f75e9868899ce"
             md5_checksum "446eab1f5329eb03662926aa1cb0845D"
             mode "644"
             not_if { #code block }
           end


       Running handlers:
       [2015-04-03T21:29:08+00:00] ERROR: Running exception handlers
       Running handlers complete
       [2015-04-03T21:29:08+00:00] ERROR: Exception handlers complete
       [2015-04-03T21:29:08+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       Chef Client failed. 18 resources updated in 76.431007827 seconds
       [2015-04-03T21:29:08+00:00] ERROR: kafka_download[/tmp/kitchen/cache/kafka_2.10-0.8.2.1.tgz] (kafka::_install line 6) had an error: SystemExit: ruby_block[kafka-validate-download] (/tmp/kitchen/cookbooks/kafka/providers/download.rb line 20) had an error: SystemExit: exit
       [2015-04-03T21:29:08+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Explicit support for systemd

Fedora uses systemd and correctly converts the current init.d script, but it would be cool to support "pure" systemd as well.

How to allocate memory to Kafka broker?

I have just manage to setup multi-node multi-broker (3 broker for each) Kafka cluster. So far I didn't test it on heavy dataflow, but I like to specify memory limits for each broker lets say 30% for each. Is it possible or Kafka manage this thing effectively for us?

Version bump, conflict with supermarket kafka'

In issue #89, Mathias mentioned that he was planning a 1.0.0 release. Unfortunately, the 'kafka' cookbook on chef supermarket is already at version 1.x.x.

Seeing as the supermarket cookbook has already been deprecated by its author, would it be reasonable to bump this kafka cookbook to version 2.0.0, so that specifying depends 'kafka', '~> 2' in a wrapper cookbook cannot possibly resolve to the supermarket-published 'kafka' ?

It seems not compatible with Chef 11.4.4 and Ruby 2.0.0p451 on OpsWorks

I try to deploy Kafka on AWS by OpsWorks. However it throws an error as follows:

root@kafka2:~# opsworks-agent-cli run_command setup
[2014-05-24 23:40:09] INFO [opsworks-agent (4054)] About to re-run 'setup' from 2014-05-24T23:39:53
Waiting for process 4057
[Sat 24 May 2014 11:40:09 PM UTC] About to execute (with LD_PRELOAD=): "/opt/aws/opsworks/current/bin/chef-solo" -j "/var/lib/aws/opsworks/chef/2014-05-24-23-40-09-01.json" -c "/opt/aws/opsworks/current/conf/solo.rb" 2>&1
[2014-05-24T23:40:10+00:00] INFO: *** Chef 11.4.4 ***
[2014-05-24T23:40:10+00:00] DEBUG: Building node object for kafka2.localdomain
[2014-05-24T23:40:10+00:00] DEBUG: Extracting run list from JSON attributes provided on command line
[2014-05-24T23:40:10+00:00] INFO: Setting the run_list to ["opsworks_custom_cookbooks::load", "opsworks_custom_cookbooks::execute"] from JSON
[2014-05-24T23:40:10+00:00] DEBUG: Applying attributes from json file
[2014-05-24T23:40:10+00:00] DEBUG: Platform is ubuntu version 12.04
[2014-05-24T23:40:10+00:00] INFO: Run List is [recipe[opsworks_custom_cookbooks::load], recipe[opsworks_custom_cookbooks::execute]]
[2014-05-24T23:40:10+00:00] INFO: Run List expands to [opsworks_custom_cookbooks::load, opsworks_custom_cookbooks::execute]
[2014-05-24T23:40:10+00:00] INFO: Starting Chef Run for kafka2.localdomain
[2014-05-24T23:40:10+00:00] INFO: Running start handlers
[2014-05-24T23:40:10+00:00] INFO: Start handlers complete.
[2014-05-24T23:40:10+00:00] DEBUG: No chefignore file found at /opt/aws/opsworks/releases/20140514153150_227/cookbooks/chefignore no files will be ignored
[2014-05-24T23:40:10+00:00] DEBUG: No chefignore file found at /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/chefignore no files will be ignored
[2014-05-24T23:40:11+00:00] DEBUG: Cookbooks to compile: ["gem_support", "packages", "opsworks_bundler", "opsworks_rubygems", "ruby", "ruby_enterprise", "dependencies", "opsworks_commons", "scm_helper", :opsworks_custom_cookbooks]
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook gem_support's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/gem_support/libraries/current_gem_version.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook packages's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/packages/libraries/packages.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook dependencies's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/dependencies/libraries/current_gem_version.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook opsworks_commons's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/libraries/activesupport_blank.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook opsworks_commons's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/libraries/monkey_patch_chefgem_resource.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook opsworks_commons's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/libraries/monkey_patch_gem_package_resource.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook opsworks_commons's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/libraries/monkey_patch_rubygems_provider.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook opsworks_commons's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/libraries/shellout.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook scm_helper's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/scm_helper/libraries/archive.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook scm_helper's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/scm_helper/libraries/git.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook scm_helper's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/scm_helper/libraries/package.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook scm_helper's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/scm_helper/libraries/s3.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook scm_helper's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/scm_helper/libraries/svn.rb
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook packages's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/packages/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute packages::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook packages's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/packages/attributes/packages.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute packages::packages
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file packages::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_bundler's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_bundler/attributes/bundler.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_bundler::bundler
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_bundler::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_bundler's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_bundler/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_bundler::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_rubygems's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_rubygems/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_rubygems::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_rubygems's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_rubygems/attributes/rubygems.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_rubygems::rubygems
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_initial_setup::default
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_initial_setup::customize
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_rubygems::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ruby's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ruby/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute ruby::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ruby's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ruby/attributes/ruby.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute ruby::ruby
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_initial_setup::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_commons::default
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_commons::customize
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file ruby::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ruby_enterprise's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ruby_enterprise/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute ruby_enterprise::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ruby_enterprise's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ruby_enterprise/attributes/ruby_enterprise.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute ruby_enterprise::ruby_enterprise
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_commons::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_rubygems::rubygems, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file ruby_enterprise::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook dependencies's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/dependencies/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute dependencies::default
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file ruby::ruby, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute dependencies::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook dependencies's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/dependencies/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file dependencies::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_commons's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_commons::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_commons's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_commons::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_custom_cookbooks's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_custom_cookbooks/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_custom_cookbooks::default
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_initial_setup::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_custom_cookbooks::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_custom_cookbooks's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_custom_cookbooks/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_custom_cookbooks::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook opsworks_commons's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/definitions/fallback.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_custom_cookbooks::load via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe load in cookbook opsworks_custom_cookbooks
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_custom_cookbooks::checkout via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe checkout in cookbook opsworks_custom_cookbooks
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_custom_cookbooks::execute via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe execute in cookbook opsworks_custom_cookbooks
[2014-05-24T23:40:11+00:00] INFO: OpsWorks Custom Run List: ["opsworks_initial_setup", "ssh_host_keys", "ssh_users", "mysql::client", "dependencies", "ebs", "opsworks_ganglia::client", "opsworks_stack_state_sync", "kafka::default", "deploy::default", "test_suite", "opsworks_cleanup"]
[2014-05-24T23:40:11+00:00] DEBUG: Loading from cookbook_path: /opt/aws/opsworks/releases/20140514153150_227/cookbooks, /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks
[2014-05-24T23:40:11+00:00] DEBUG: Converging node kafka2.localdomain
[2014-05-24T23:40:11+00:00] INFO: Processing package[git-core] action install (opsworks_custom_cookbooks::checkout line 9)
[2014-05-24T23:40:11+00:00] DEBUG: package[git-core] checking package status for git-core
[2014-05-24T23:40:11+00:00] DEBUG: package[git-core] current version is 1:1.7.9.5-1
[2014-05-24T23:40:11+00:00] DEBUG: package[git-core] candidate version is 1:1.7.9.5-1
[2014-05-24T23:40:11+00:00] DEBUG: package[git-core] is already installed - nothing to do
[2014-05-24T23:40:11+00:00] INFO: Processing directory[/root/.ssh] action create (opsworks_custom_cookbooks::checkout line 8)
[2014-05-24T23:40:11+00:00] INFO: Processing file[/root/.ssh/config] action touch (opsworks_custom_cookbooks::checkout line 16)
[2014-05-24T23:40:11+00:00] INFO: file[/root/.ssh/config] updated atime and mtime to Sat May 24 23:40:11 +0000 2014
[2014-05-24T23:40:11+00:00] INFO: Processing execute[echo 'StrictHostKeyChecking no' > /root/.ssh/config] action run (opsworks_custom_cookbooks::checkout line 23)
[2014-05-24T23:40:11+00:00] DEBUG: Skipping execute[echo 'StrictHostKeyChecking no' > /root/.ssh/config] due to not_if command `grep '^StrictHostKeyChecking no$' /root/.ssh/config`
[2014-05-24T23:40:11+00:00] INFO: Processing template[/root/.ssh/id_dsa] action create (opsworks_custom_cookbooks::checkout line 27)
[2014-05-24T23:40:11+00:00] DEBUG: Current content's checksum:  20157f713adbba1bf2f48d4b9c6ac4d701473e2d7b0eea25ab55f2538877b279
[2014-05-24T23:40:11+00:00] DEBUG: Rendered content's checksum: 20157f713adbba1bf2f48d4b9c6ac4d701473e2d7b0eea25ab55f2538877b279
[2014-05-24T23:40:11+00:00] DEBUG: template[/root/.ssh/id_dsa] content has not changed.
[2014-05-24T23:40:11+00:00] INFO: Processing git[Download Custom Cookbooks] action checkout (opsworks_custom_cookbooks::checkout line 29)
[2014-05-24T23:40:11+00:00] DEBUG: Skipping git[Download Custom Cookbooks] due to not_if ruby block
[2014-05-24T23:40:11+00:00] INFO: Processing ruby_block[Move single cookbook contents into appropriate subdirectory] action run (opsworks_custom_cookbooks::checkout line 64)
[2014-05-24T23:40:11+00:00] DEBUG: Skipping ruby_block[Move single cookbook contents into appropriate subdirectory] due to only_if ruby block
[2014-05-24T23:40:11+00:00] INFO: Processing execute[ensure correct permissions of custom cookbooks] action run (opsworks_custom_cookbooks::checkout line 80)
[2014-05-24T23:40:11+00:00] INFO: execute[ensure correct permissions of custom cookbooks] ran successfully
[2014-05-24T23:40:11+00:00] INFO: Processing ruby_block[Compile Custom OpsWorks Run List] action run (opsworks_custom_cookbooks::execute line 3)
[2014-05-24T23:40:11+00:00] DEBUG: No chefignore file found at /opt/aws/opsworks/releases/20140514153150_227/cookbooks/chefignore no files will be ignored
[2014-05-24T23:40:11+00:00] DEBUG: No chefignore file found at /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/chefignore no files will be ignored
[2014-05-24T23:40:11+00:00] INFO: New Run List expands to ["opsworks_initial_setup", "ssh_host_keys", "ssh_users", "mysql::client", "dependencies", "ebs", "opsworks_ganglia::client", "opsworks_stack_state_sync", "kafka::default", "deploy::default", "test_suite", "opsworks_cleanup"]
[2014-05-24T23:40:11+00:00] DEBUG: Cookbooks to compile: ["gem_support", "packages", "opsworks_bundler", "opsworks_rubygems", "ruby", "ruby_enterprise", "dependencies", "opsworks_commons", :opsworks_initial_setup, :ssh_host_keys, "opsworks_initial_setup", :ssh_users, :mysql, :dependencies, :ebs, "apache2", :opsworks_ganglia, :opsworks_stack_state_sync, "kafka-cookbook", :kafka, "mod_php5_apache2", "nginx", "opsworks_agent_monit", "opsworks_java", "ssh_users", "unicorn", "deploy", "rails", "passenger_apache2", :deploy, :test_suite, :opsworks_cleanup]
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook gem_support's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/gem_support/libraries/current_gem_version.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook packages's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/packages/libraries/packages.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook dependencies's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/dependencies/libraries/current_gem_version.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook opsworks_commons's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/libraries/activesupport_blank.rb
/opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/libraries/activesupport_blank.rb:92: warning: already initialized constant NON_WHITESPACE_REGEXP
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook opsworks_commons's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/libraries/monkey_patch_chefgem_resource.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook opsworks_commons's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/libraries/monkey_patch_gem_package_resource.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook opsworks_commons's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/libraries/monkey_patch_rubygems_provider.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook opsworks_commons's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/libraries/shellout.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook ssh_host_keys's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ssh_host_keys/libraries/host_key.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook ssh_users's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ssh_users/libraries/user.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook dependencies's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/dependencies/libraries/current_gem_version.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook ebs's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ebs/libraries/block_device.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook kafka-cookbook's library file: /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/libraries/kafka_helpers.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook kafka-cookbook's library file: /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/libraries/matchers.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook ssh_users's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ssh_users/libraries/user.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook rails's library file: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/rails/libraries/rails_configuration.rb
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook packages's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/packages/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file packages::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook packages's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/packages/attributes/packages.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file packages::packages, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_bundler's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_bundler/attributes/bundler.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_bundler::bundler, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_bundler's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_bundler/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_bundler::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_rubygems's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_rubygems/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_rubygems::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_rubygems's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_rubygems/attributes/rubygems.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_rubygems::rubygems, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ruby's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ruby/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file ruby::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ruby's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ruby/attributes/ruby.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file ruby::ruby, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ruby_enterprise's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ruby_enterprise/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file ruby_enterprise::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ruby_enterprise's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ruby_enterprise/attributes/ruby_enterprise.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file ruby_enterprise::ruby_enterprise, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook dependencies's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/dependencies/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file dependencies::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook dependencies's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/dependencies/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file dependencies::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_commons's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_commons::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_commons's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_commons::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_initial_setup's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_initial_setup/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_initial_setup::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_initial_setup's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_initial_setup/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_initial_setup::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_initial_setup's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_initial_setup/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_initial_setup::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_initial_setup's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_initial_setup/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_initial_setup::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ssh_users's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ssh_users/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute ssh_users::default
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_initial_setup::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute ssh_users::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ssh_users's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ssh_users/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file ssh_users::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook mysql's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/mysql/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute mysql::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook mysql's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/mysql/attributes/server.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute mysql::server
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file mysql::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook dependencies's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/dependencies/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file dependencies::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook dependencies's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/dependencies/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file dependencies::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ebs's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ebs/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute ebs::default
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute ebs::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ebs's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ebs/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file ebs::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook apache2's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/apache2/attributes/apache.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute apache2::apache
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute apache2::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook apache2's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/apache2/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file apache2::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_ganglia's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_ganglia/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_ganglia::default
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_initial_setup::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file apache2::apache, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_ganglia::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_ganglia's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_ganglia/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_ganglia::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook kafka-cookbook's attribute file /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute kafka-cookbook::default
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook kafka-cookbook's attribute file /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/attributes/zookeeper.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute kafka-cookbook::zookeeper
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook mod_php5_apache2's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/mod_php5_apache2/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute mod_php5_apache2::default
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute mod_php5_apache2::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook mod_php5_apache2's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/mod_php5_apache2/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file mod_php5_apache2::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook nginx's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/nginx/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute nginx::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook nginx's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/nginx/attributes/nginx.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute nginx::nginx
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file nginx::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_agent_monit's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_agent_monit/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_agent_monit::default
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_initial_setup::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_agent_monit::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_agent_monit's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_agent_monit/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_agent_monit::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_java's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_java/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_java::default
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_java::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_java's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_java/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_java::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ssh_users's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ssh_users/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file ssh_users::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook ssh_users's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ssh_users/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file ssh_users::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook unicorn's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/unicorn/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute unicorn::default
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute rails::rails
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute deploy::default
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute deploy::deploy
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute deploy::logrotate
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute deploy::customize
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute deploy::rails_stack
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file deploy::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file deploy::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file deploy::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute rails::customize
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute unicorn::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook unicorn's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/unicorn/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file unicorn::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook deploy's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file deploy::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook deploy's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file deploy::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook deploy's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/attributes/deploy.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file deploy::deploy, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook deploy's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/attributes/logrotate.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file deploy::logrotate, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook deploy's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/attributes/rails_stack.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file deploy::rails_stack, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook rails's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/rails/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file rails::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook rails's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/rails/attributes/rails.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file rails::rails, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook passenger_apache2's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/passenger_apache2/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute passenger_apache2::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook passenger_apache2's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/passenger_apache2/attributes/passenger.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute passenger_apache2::passenger
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_initial_setup::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file rails::rails, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file packages::packages, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file passenger_apache2::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook deploy's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file deploy::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook deploy's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file deploy::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook deploy's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/attributes/deploy.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file deploy::deploy, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook deploy's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/attributes/logrotate.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file deploy::logrotate, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook deploy's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/attributes/rails_stack.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file deploy::rails_stack, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook test_suite's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/test_suite/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute test_suite::default
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute test_suite::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook test_suite's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/test_suite/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file test_suite::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_cleanup's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_cleanup/attributes/default.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_cleanup::default
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_initial_setup::default, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Loading Attribute opsworks_cleanup::customize
[2014-05-24T23:40:11+00:00] DEBUG: Node kafka2.localdomain loading cookbook opsworks_cleanup's attribute file /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_cleanup/attributes/customize.rb
[2014-05-24T23:40:11+00:00] DEBUG: I am not loading attribute file opsworks_cleanup::customize, because I have already seen it.
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook kafka-cookbook's providers from /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/providers/download.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loaded contents of /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/providers/download.rb into a provider named kafka-cookbook_download defined in Chef::Provider::KafkaCookbookDownload
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook kafka-cookbook's providers from /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/providers/install.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loaded contents of /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/providers/install.rb into a provider named kafka-cookbook_install defined in Chef::Provider::KafkaCookbookInstall
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook kafka-cookbook's resources from /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/resources/download.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loaded contents of /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/resources/download.rb into a resource named kafka-cookbook_download defined in Chef::Resource::KafkaCookbookDownload
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook kafka-cookbook's resources from /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/resources/install.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loaded contents of /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/resources/install.rb into a resource named kafka-cookbook_install defined in Chef::Resource::KafkaCookbookInstall
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook opsworks_commons's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/definitions/fallback.rb
[2014-05-24T23:40:11+00:00] INFO: Overriding duplicate definition fallback, new definition found in /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_commons/definitions/fallback.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook apache2's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/apache2/definitions/apache_conf.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook apache2's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/apache2/definitions/apache_module.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook apache2's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/apache2/definitions/apache_site.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook apache2's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/apache2/definitions/web_app.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook nginx's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/nginx/definitions/nginx_web_app.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook unicorn's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/unicorn/definitions/unicorn_web_app.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook deploy's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_deploy.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook deploy's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_deploy_dir.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook deploy's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_deploy_user.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook deploy's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_nodejs.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook deploy's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_rails.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook passenger_apache2's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/passenger_apache2/definitions/passenger_web_app.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook deploy's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_deploy.rb
[2014-05-24T23:40:11+00:00] INFO: Overriding duplicate definition opsworks_deploy, new definition found in /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_deploy.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook deploy's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_deploy_dir.rb
[2014-05-24T23:40:11+00:00] INFO: Overriding duplicate definition opsworks_deploy_dir, new definition found in /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_deploy_dir.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook deploy's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_deploy_user.rb
[2014-05-24T23:40:11+00:00] INFO: Overriding duplicate definition opsworks_deploy_user, new definition found in /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_deploy_user.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook deploy's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_nodejs.rb
[2014-05-24T23:40:11+00:00] INFO: Overriding duplicate definition opsworks_nodejs, new definition found in /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_nodejs.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading cookbook deploy's definitions from /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_rails.rb
[2014-05-24T23:40:11+00:00] INFO: Overriding duplicate definition opsworks_rails, new definition found in /opt/aws/opsworks/releases/20140514153150_227/cookbooks/deploy/definitions/opsworks_rails.rb
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe default in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::sysctl via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe sysctl in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::limits via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe limits in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::bind_mounts via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe bind_mounts in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::autofs via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe autofs in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::vol_mount_point via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe vol_mount_point in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::remove_landscape via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe remove_landscape in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::ldconfig via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe ldconfig in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::yum_conf via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe yum_conf in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::tweak_chef_yum_dump via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe tweak_chef_yum_dump in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::setup_rhel_repos via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe setup_rhel_repos in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::package_procps via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe package_procps in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::package_ntpd via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe package_ntpd in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::package_vim via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe package_vim in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::package_sqlite via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe package_sqlite in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_initial_setup::package_screen via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe package_screen in cookbook opsworks_initial_setup
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe ssh_host_keys via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe default in cookbook ssh_host_keys
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe ssh_users via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe default in cookbook ssh_users
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe mysql::client via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe client in cookbook mysql
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe dependencies via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe default in cookbook dependencies
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe packages via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe default in cookbook packages
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe gem_support via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe default in cookbook gem_support
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe ruby via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe default in cookbook ruby
[2014-05-24T23:40:11+00:00] INFO: Userspace Ruby version is 2.0.0p451 - up-to-date
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_rubygems via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe default in cookbook opsworks_rubygems
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_bundler via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe default in cookbook opsworks_bundler
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe ebs via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe default in cookbook ebs
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe ebs::volumes via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe volumes in cookbook ebs
[2014-05-24T23:40:11+00:00] WARN: Cloning resource attributes for mount[/data] from prior resource (CHEF-3694)
[2014-05-24T23:40:11+00:00] WARN: Previous mount[/data]: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ebs/recipes/volumes.rb:19:in `from_file'
[2014-05-24T23:40:11+00:00] WARN: Current  mount[/data]: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ebs/recipes/volumes.rb:26:in `from_file'
[2014-05-24T23:40:11+00:00] WARN: Cloning resource attributes for mount[/var/log] from prior resource (CHEF-3694)
[2014-05-24T23:40:11+00:00] WARN: Previous mount[/var/log]: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ebs/recipes/volumes.rb:19:in `from_file'
[2014-05-24T23:40:11+00:00] WARN: Current  mount[/var/log]: /opt/aws/opsworks/releases/20140514153150_227/cookbooks/ebs/recipes/volumes.rb:26:in `from_file'
[2014-05-24T23:40:11+00:00] DEBUG: Loading Recipe opsworks_ganglia::client via include_recipe
[2014-05-24T23:40:11+00:00] DEBUG: Found recipe client in cookbook opsworks_ganglia
[2014-05-24T23:40:12+00:00] DEBUG: Loading Recipe opsworks_ganglia::monitor-fd-and-sockets via include_recipe
[2014-05-24T23:40:12+00:00] DEBUG: Found recipe monitor-fd-and-sockets in cookbook opsworks_ganglia
[2014-05-24T23:40:12+00:00] DEBUG: Loading Recipe opsworks_ganglia::monitor-disk via include_recipe
[2014-05-24T23:40:12+00:00] DEBUG: Found recipe monitor-disk in cookbook opsworks_ganglia
[2014-05-24T23:40:12+00:00] INFO: Adding EBS volumes ["/dev/xvdi", "/dev/xvdj"] to monitoring
[2014-05-24T23:40:12+00:00] INFO: Installing Monitoring for disk xvdap1
[2014-05-24T23:40:12+00:00] INFO: Installing Monitoring for disk xvdbp
[2014-05-24T23:40:12+00:00] INFO: Installing Monitoring for disk xvdip
[2014-05-24T23:40:12+00:00] INFO: Installing Monitoring for disk xvdjp
[2014-05-24T23:40:12+00:00] DEBUG: Loading Recipe opsworks_stack_state_sync via include_recipe
[2014-05-24T23:40:12+00:00] DEBUG: Found recipe default in cookbook opsworks_stack_state_sync
[2014-05-24T23:40:12+00:00] DEBUG: Loading Recipe opsworks_stack_state_sync::hosts via include_recipe
[2014-05-24T23:40:12+00:00] DEBUG: Found recipe hosts in cookbook opsworks_stack_state_sync
[2014-05-24T23:40:12+00:00] DEBUG: Loading Recipe opsworks_stack_state_sync::motd via include_recipe
[2014-05-24T23:40:12+00:00] DEBUG: Found recipe motd in cookbook opsworks_stack_state_sync
[2014-05-24T23:40:12+00:00] DEBUG: Loading Recipe kafka::default via include_recipe
[2014-05-24T23:40:12+00:00] DEBUG: Found recipe default in cookbook kafka
[2014-05-24T23:40:12+00:00] DEBUG: Loading Recipe kafka-cookbook::binary via include_recipe
[2014-05-24T23:40:12+00:00] DEBUG: Found recipe binary in cookbook kafka-cookbook
[2014-05-24T23:40:12+00:00] DEBUG: filtered backtrace of compile error:
[2014-05-24T23:40:12+00:00] DEBUG: filtered backtrace of compile error:
[2014-05-24T23:40:12+00:00] DEBUG: backtrace entry for compile error: '/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_platform_map.rb:126:in `get''
[2014-05-24T23:40:12+00:00] DEBUG: Line number of compile error: '126'

================================================================================
Recipe Compile Error in /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka/recipes/default.rb
================================================================================


NameError
---------
Cannot find a resource for kafka_download on ubuntu version 12.04


Cookbook Trace:
---------------
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_platform_map.rb:126:in `get'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource.rb:772:in `resource_for_platform'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource.rb:789:in `resource_for_node'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/dsl/recipe.rb:55:in `method_missing'
  /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/recipes/binary.rb:9:in `from_file'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_version.rb:346:in `load_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:151:in `load_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:132:in `include_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:131:in `each'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:131:in `include_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/dsl/include_recipe.rb:26:in `include_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka/recipes/default.rb:2:in `from_file'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_version.rb:346:in `load_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:151:in `load_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:139:in `compile_recipes'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:137:in `each'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:137:in `compile_recipes'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:74:in `compile'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:86:in `load'
  /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_custom_cookbooks/recipes/execute.rb:15:in `from_file'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider/ruby_block.rb:33:in `call'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider/ruby_block.rb:33:in `action_run'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/why_run.rb:52:in `call'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/why_run.rb:52:in `add_action'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider.rb:151:in `converge_by'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider/ruby_block.rb:32:in `action_run'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider.rb:114:in `send'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider.rb:114:in `run_action'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource.rb:617:in `run_action'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:50:in `run_action'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:82:in `converge'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:82:in `each'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:82:in `converge'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection.rb:94:in `execute_each_resource'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection.rb:92:in `execute_each_resource'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:81:in `converge'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:404:in `converge'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:469:in `do_run'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:200:in `run'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application.rb:190:in `run_chef_client'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:239:in `run_application'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:231:in `loop'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:231:in `run_application'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application.rb:73:in `run'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/chef-solo:25
  /opt/aws/opsworks/current/bin/chef-solo:16:in `load'
  /opt/aws/opsworks/current/bin/chef-solo:16


Relevant File Content:
----------------------
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_platform_map.rb:

119:          end
120:        end
121:
122:        def get(short_name, platform=nil, version=nil)
123:          resource_klass = platform_resource(short_name, platform, version) ||
124:                           resource_matching_short_name(short_name)
125:
126>>         raise NameError, "Cannot find a resource for #{short_name} on #{platform} version #{version}" if resource_klass.nil?
127:
128:          resource_klass
129:        end
130:
131:        private
132:
133:        def platform_resource(short_name, platform, version)
134:          pmap = filter(platform, version)
135:          rtkey = short_name.kind_of?(Chef::Resource) ? short_name.resource_name.to_sym : short_name



[2014-05-24T23:40:12+00:00] ERROR: Caught exception while compiling OpsWorks custom run list: NameError - Cannot find a resource for kafka_download on ubuntu version 12.04 - /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_platform_map.rb:126:in `get'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource.rb:772:in `resource_for_platform'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource.rb:789:in `resource_for_node'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/dsl/recipe.rb:55:in `method_missing'
/opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/recipes/binary.rb:9:in `from_file'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_version.rb:346:in `load_recipe'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:151:in `load_recipe'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:132:in `include_recipe'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:131:in `each'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:131:in `include_recipe'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/dsl/include_recipe.rb:26:in `include_recipe'
/opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka/recipes/default.rb:2:in `from_file'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_version.rb:346:in `load_recipe'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:151:in `load_recipe'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:139:in `compile_recipes'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:137:in `each'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:137:in `compile_recipes'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:74:in `compile'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:86:in `load'
/opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_custom_cookbooks/recipes/execute.rb:15:in `from_file'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider/ruby_block.rb:33:in `call'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider/ruby_block.rb:33:in `action_run'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/why_run.rb:52:in `call'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/why_run.rb:52:in `add_action'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider.rb:151:in `converge_by'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider/ruby_block.rb:32:in `action_run'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider.rb:114:in `send'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider.rb:114:in `run_action'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource.rb:617:in `run_action'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:50:in `run_action'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:82:in `converge'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:82:in `each'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:82:in `converge'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection.rb:94:in `execute_each_resource'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection.rb:92:in `execute_each_resource'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:81:in `converge'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:404:in `converge'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:469:in `do_run'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:200:in `run'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application.rb:190:in `run_chef_client'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:239:in `run_application'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:231:in `loop'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:231:in `run_application'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application.rb:73:in `run'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/chef-solo:25
/opt/aws/opsworks/current/bin/chef-solo:16:in `load'
/opt/aws/opsworks/current/bin/chef-solo:16

================================================================================
Error executing action `run` on resource 'ruby_block[Compile Custom OpsWorks Run List]'
================================================================================


NameError
---------
Cannot find a resource for kafka_download on ubuntu version 12.04


Resource Declaration:
---------------------
# In /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_custom_cookbooks/recipes/execute.rb

  3: ruby_block("Compile Custom OpsWorks Run List") do
  4:   block do
  5:     begin
  6:
  7:       # Reload cookbooks after they're available on local filesystem
  8:       cl = Chef::CookbookLoader.new(Chef::Config[:cookbook_path])
  9:       cl.load_cookbooks
 10:       self.run_context.instance_variable_set(:@cookbook_collection, Chef::CookbookCollection.new(cl))
 11:
 12:       # Expand run list with custom cookbooks and load them into the current run_context
 13:       opsworks_run_list = Chef::RunList.new(*node[:opsworks_custom_cookbooks][:recipes])
 14:       Chef::Log.info "New Run List expands to #{opsworks_run_list.run_list_items.map(&:name).inspect}"
 15:       self.run_context.load(opsworks_run_list)
 16:
 17:     rescue Exception => e
 18:       Chef::Log.error "Caught exception while compiling OpsWorks custom run list: #{e.class} - #{e.message} - #{e.backtrace.join("\n")}"
 19:       raise e
 20:     end
 21:
 22:   end
 23: end



Compiled Resource:
------------------
# Declared in /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_custom_cookbooks/recipes/execute.rb:3:in `from_file'

ruby_block("Compile Custom OpsWorks Run List") do
  retries 0
  block #<Proc:0x00007fbc40b40290@/opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_custom_cookbooks/recipes/execute.rb:4>
  cookbook_name :opsworks_custom_cookbooks
  block_name "Compile Custom OpsWorks Run List"
  recipe_name "execute"
  action "run"
  retry_delay 2
end



[2014-05-24T23:40:12+00:00] INFO: Running queued delayed notifications before re-raising exception
[2014-05-24T23:40:12+00:00] DEBUG: Re-raising exception: NameError - ruby_block[Compile Custom OpsWorks Run List] (opsworks_custom_cookbooks::execute line 3) had an error: NameError: Cannot find a resource for kafka_download on ubuntu version 12.04
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_platform_map.rb:126:in `get'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource.rb:772:in `resource_for_platform'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource.rb:789:in `resource_for_node'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/dsl/recipe.rb:55:in `method_missing'
  /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/recipes/binary.rb:9:in `from_file'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_version.rb:346:in `load_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:151:in `load_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:132:in `include_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:131:in `each'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:131:in `include_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/dsl/include_recipe.rb:26:in `include_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka/recipes/default.rb:2:in `from_file'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_version.rb:346:in `load_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:151:in `load_recipe'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:139:in `compile_recipes'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:137:in `each'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:137:in `compile_recipes'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:74:in `compile'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:86:in `load'
  /opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_custom_cookbooks/recipes/execute.rb:15:in `from_file'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider/ruby_block.rb:33:in `call'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider/ruby_block.rb:33:in `action_run'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/why_run.rb:52:in `call'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/why_run.rb:52:in `add_action'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider.rb:151:in `converge_by'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider/ruby_block.rb:32:in `action_run'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider.rb:114:in `send'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider.rb:114:in `run_action'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource.rb:617:in `run_action'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:50:in `run_action'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:82:in `converge'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:82:in `each'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:82:in `converge'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection.rb:94:in `execute_each_resource'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection.rb:92:in `execute_each_resource'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:81:in `converge'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:404:in `converge'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:469:in `do_run'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:200:in `run'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application.rb:190:in `run_chef_client'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:239:in `run_application'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:231:in `loop'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:231:in `run_application'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application.rb:73:in `run'
  /opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/chef-solo:25
  /opt/aws/opsworks/current/bin/chef-solo:16:in `load'
  /opt/aws/opsworks/current/bin/chef-solo:16
[2014-05-24T23:40:12+00:00] ERROR: Running exception handlers
[2014-05-24T23:40:12+00:00] ERROR: Exception handlers complete
[2014-05-24T23:40:12+00:00] FATAL: Stacktrace dumped to /opt/aws/opsworks/releases/20140514153150_227/chef-stacktrace.out
[2014-05-24T23:40:12+00:00] DEBUG: NameError: ruby_block[Compile Custom OpsWorks Run List] (opsworks_custom_cookbooks::execute line 3) had an error: NameError: Cannot find a resource for kafka_download on ubuntu version 12.04
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_platform_map.rb:126:in `get'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource.rb:772:in `resource_for_platform'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource.rb:789:in `resource_for_node'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/dsl/recipe.rb:55:in `method_missing'
/opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka-cookbook/recipes/binary.rb:9:in `from_file'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_version.rb:346:in `load_recipe'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:151:in `load_recipe'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:132:in `include_recipe'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:131:in `each'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:131:in `include_recipe'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/dsl/include_recipe.rb:26:in `include_recipe'
/opt/aws/opsworks/releases/20140514153150_227/site-cookbooks/kafka/recipes/default.rb:2:in `from_file'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/cookbook_version.rb:346:in `load_recipe'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:151:in `load_recipe'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:139:in `compile_recipes'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:137:in `each'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:137:in `compile_recipes'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context/cookbook_compiler.rb:74:in `compile'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/run_context.rb:86:in `load'
/opt/aws/opsworks/releases/20140514153150_227/cookbooks/opsworks_custom_cookbooks/recipes/execute.rb:15:in `from_file'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider/ruby_block.rb:33:in `call'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider/ruby_block.rb:33:in `action_run'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/why_run.rb:52:in `call'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/mixin/why_run.rb:52:in `add_action'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider.rb:151:in `converge_by'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider/ruby_block.rb:32:in `action_run'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider.rb:114:in `send'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/provider.rb:114:in `run_action'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource.rb:617:in `run_action'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:50:in `run_action'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:82:in `converge'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:82:in `each'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:82:in `converge'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection.rb:94:in `execute_each_resource'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/resource_collection.rb:92:in `execute_each_resource'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/runner.rb:81:in `converge'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:404:in `converge'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:469:in `do_run'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/client.rb:200:in `run'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application.rb:190:in `run_chef_client'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:239:in `run_application'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:231:in `loop'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application/solo.rb:231:in `run_application'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/../lib/chef/application.rb:73:in `run'
/opt/aws/opsworks/releases/20140514153150_227/vendor/gems/chef-11.4.4/bin/chef-solo:25
/opt/aws/opsworks/current/bin/chef-solo:16:in `load'
/opt/aws/opsworks/current/bin/chef-solo:16
[2014-05-24T23:40:12+00:00] FATAL: NameError: ruby_block[Compile Custom OpsWorks Run List] (opsworks_custom_cookbooks::execute line 3) had an error: NameError: Cannot find a resource for kafka_download on ubuntu version 12.04
pid 4057 exited with status 256, exit code: 1 (time=3 sec)
[2014-05-24 23:40:12] ERROR [opsworks-agent (4054)] Chef run failed with exitcode 1
root@kafka2:~# ruby -v
ruby 2.0.0p451 (2014-02-24 revision 45167) [x86_64-linux]

Overriding default.kafka.log_dir doesn't change the log_dir in the Appender blocks

If I override default.kafka.log_dir = '/var/log/kafka' in a wrapper cookbook to something else like:
node.default[:kafka][:log_dir] ='/data/logs/kafka' it updates that attribute but the *Appender attriibutes (like the line node.default[:kafka][:log_dir] ='/data/logs/kafka') still get set to /var/log/kafka

Not sure how to fix that. I've always had problems with compounding attributes in attribute files though its something I always want!

Separate variables in environment file

Just do it.

Add support for advertised.host.name

It would be nice if the config file could also set the advertised host name in the settings.properties

 advertised.host.name=foobar.example.com

Add upstart support

See #1.

Install kafka into version specific directory

Right now, there's no way of upgrading / downgrading (or even reinstalling kafka for that matter), as there's an if installed statement guarding the download and install resources.

It would be nice if kafka was installed into a version specific directory, i.e. /opt/kafka-0.8.0 which would then be symlinked to /opt/kafka, much like the Chef ark cookbook works.

How to run with chef-client zero (-z, local mode, or solo)

Hello,

Could you help me with this? I've tried to run it through bundler, and did bekshelf install too.

I want to clone this onto a server (or preferably not) and issue a few one liners to get the dependencies (on both sides) and have a running kafka 20 minutes later.

Thanks!

Starting Chef Client, version 11.12.8
resolving cookbooks for run list: ["kafka-cookbook::binary"]
Synchronizing Cookbooks:
  - kafka-cookbook
Compiling Cookbooks...

================================================================================
Recipe Compile Error in /root/.chef/local-mode-cache/cache/cookbooks/kafka-cookbook/recipes/binary.rb
================================================================================


NoMethodError
-------------
No resource or method named `kafka_download' for `Chef::Recipe "binary"'


Cookbook Trace:
---------------
  /root/.chef/local-mode-cache/cache/cookbooks/kafka-cookbook/recipes/binary.rb:9:in `from_file'


Relevant File Content:
----------------------
/root/.chef/local-mode-cache/cache/cookbooks/kafka-cookbook/recipes/binary.rb:

  2:  # Cookbook Name:: kafka
  3:  # Recipe:: binary
  4:  #
  5:  
  6:  kafka_tar_gz = [kafka_base, kafka_archive_ext].join('.')
  7:  local_file_path = ::File.join(Chef::Config.file_cache_path, kafka_tar_gz)
  8:  
  9>> kafka_download local_file_path do
 10:    source kafka_download_uri(kafka_tar_gz)
 11:    checksum node.kafka.checksum
 12:    md5_checksum node.kafka.md5_checksum
 13:    not_if { kafka_installed? }
 14:  end
 15:  
 16:  execute 'extract-kafka' do
 17:    user 'root'
 18:    group 'root'

Add Ubuntu support

See #1 and #2.

Plans to release to supermarket?

This is a great cookbook. I can't find it on the supermarket though, so I end up having to do the following in my wrapper cookbook berksfile.

source "https://supermarket.getchef.com"

metadata

cookbook 'kafka', github: "mthssdrbrg/kafka-cookbook", tag: "v0.8.0"
cookbook 'cron'

Any plans to upload to the supermarket?

Checksum fails

When running kitchen create the download will fail on the checksum verification step.

I've verified that the checksum is correct by downloading from https://archive.apache.org/dist/kafka/0.8.1.1/kafka_2.9.2-0.8.1.1.tgz" and comparing the md5 and sha256 with the versions in the cookbook.

MD5 (kafka_2.9.2-0.8.1.1.tgz) = 7541ed160f1b3aa1a5334d4e782ba4d3
cb141c1d50b1bd0d741d68e5e21c090341d961cd801e11e42fb693fa53e9aaed kafka_2.9.2-0.8.1.1.tgz

For some reason, Checksum on resource () does not match checksum on content (cb141c) has an empty value inside the brackets.

Is this a problem with the manifest? Or is a parameter missing on my part? I'm pretty new to chef.

       ================================================================================
           Error executing action `create` on resource 'kafka_download[/tmp/kitchen/cache/kafka_2.9.2-0.8.1.1.tgz]'
           ================================================================================

       Chef::Exceptions::ChecksumMismatch
           ----------------------------------
           remote_file[/tmp/kitchen/cache/kafka_2.9.2-0.8.1.1.tgz] (/tmp/kitchen/cookbooks/kafka/providers/download.rb line 12) had an error: Chef::Exceptions::ChecksumMismatch: Checksum on resource () does not match checksum on content (cb141c)

           Resource Declaration:
           ---------------------
           # In /tmp/kitchen/cookbooks/kafka/recipes/_install.rb

         6: kafka_download kafka_local_download_path do
             7:   source kafka_download_uri(kafka_tar_gz)
             8:   checksum node.kafka.checksum
             9:   md5_checksum node.kafka.md5_checksum
            10:   not_if { kafka_installed? }
            11: end
        12:

       Compiled Resource:
           ------------------
           # Declared in /tmp/kitchen/cookbooks/kafka/recipes/_install.rb:6:in `from_file'

           kafka_download("/tmp/kitchen/cache/kafka_2.9.2-0.8.1.1.tgz") do
             action :create
             retries 0
         retry_delay 2
             default_guard_interpreter :default

             cookbook_name :kafka
             recipe_name "_install"
         source "https://archive.apache.org/dist/kafka/0.8.1.1/kafka_2.9.2-0.8.1.1.tgz"
         mode "644"
             not_if { #code block }
           end


       Running handlers:
       [2015-02-10T20:48:43+00:00] ERROR: Running exception handlers
       Running handlers complete
       [2015-02-10T20:48:43+00:00] ERROR: Exception handlers complete
       [2015-02-10T20:48:43+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
       Chef Client failed. 1 resources updated in 65.305166066 seconds
       [2015-02-10T20:48:43+00:00] ERROR: kafka_download[/tmp/kitchen/cache/kafka_2.9.2-0.8.1.1.tgz] (kafka::_install line 6) had an error: Chef::Exceptions::ChecksumMismatch: remote_file[/tmp/kitchen/cache/kafka_2.9.2-0.8.1.1.tgz] (/tmp/kitchen/cookbooks/kafka/providers/download.rb line 12) had an error: Chef::Exceptions::ChecksumMismatch: Checksum on resource () does not match checksum on content (cb141c)
       [2015-02-10T20:48:43+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
>>>>>> Converge failed on instance <init-style-upstart-ubuntu-1404>.
>>>>>> Please see .kitchen/logs/init-style-upstart-ubuntu-1404.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sh -c '
sudo -E /opt/chef/bin/chef-solo --config /tmp/kitchen/solo.rb --log_level auto --force-formatter --no-color --json-attributes /tmp/kitchen/dna.json
']
>>>>>> ----------------------