michaelklishin / cassandra-chef-cookbook Goto Github PK

View Code? Open in Web Editor NEW

161.0 32.0 230.0 1003 KB

Chef cookbook for Apache Cassandra, DataStax Enterprise (DSE) and DataStax agent

License: Apache License 2.0

Ruby 65.42% Shell 12.42% HTML 22.16%

cassandra chef-cookbook datastax-enterprise chef cookbook cookbooks

cassandra-chef-cookbook's People

Contributors

Stargazers

Watchers

Forkers

samstokes iconara ifesdjeen ryandub kwarter-cookbooks bryanwb blakehilscher keenlabs dlaube nabeelio mbiermann buger alecnmk pragnesh krone castro1688 skyebook tobyhede jfelectron opsmatic gansbrest hayesmp lum yaitskov cevaris dholbrook silviorelli senseobservationsystems leonidas tnosaj cantenesse joshpasqualetto mdxp scullxbones grizo dpkp vidyar maasg bryncooke garrygu brenttheisen cwjohnston aimtheory rajpalparyani silviud varecham sidr klamontagne mydrive erwin32 fedesilva zznate acqant tjnicholas yieldbot benkershner amlee larsp pfleidi petecheslock mikebthun mimperatore frankwis rakibulislam pkoz lejacobs naturalatlas addictedcs benpillet awgneo igorfrenkel igorshp apptopia willejs wadehundley mdbishop chokha dbxfb adityagodbole dcparker88 joshvar88 rubikloud cxrebollar lrdcasimir lithiumtech rnekkanti zeroto alexrudd exg77 kamaradclimber al3xdm eyeem jankows2 vervas restless-et malinik markharding aferdous richid jamesawesome

cassandra-chef-cookbook's Issues

Provision 2.1.0 by default?

Opening this to start a discussion. Should we now provision 2.1.0 by default?

@vkhatri @arr-ee

Add option to use OS packages to install datastax-agent

It's be nice to be able to stick to packages rather than use the tar.
http://www.datastax.com/documentation/opscenter/4.1/opsc/install/opscManualInstallAgentPkgDeb.html
http://www.datastax.com/documentation/opscenter/4.1/opsc/install/opscManualInstallPkgRpm.html

rhel/centos package version

version node[:cassandra][:version] would be nice in datastax.rb @105:

package "#{node[:cassandra][:package_name]}" do
  action :install
end

current workaround:

chef_gem 'chef-rewind'
require 'chef/rewind'
rewind "package[#{node[:cassandra][:package_name]}]" do
  action :install
  version node[:cassandra][:version]
end

Failed join call on seeds, seed data already joined

[2013-11-28T00:33:52+00:00] INFO: Processing template[/etc/cassandra/cassandra.yaml] action create (cassandra::tarball line 97)

================================================================
Error executing action `create` on resource 'template[/etc/cassandra/cassandra.yaml]'

Chef::Mixin::Template::TemplateError

undefined method `join' for "192.168.2.10,192.168.2.11,192.168.2.12":String

Resource Declaration:

In /tmp/vagrant-chef-1/chef-solo-1/cookbooks/cassandra/recipes/tarball.rb

97: template File.join(node.cassandra.conf_dir, f) do
98: source "#{f}.erb"
99: owner node.cassandra.user
100: group node.cassandra.user
101: mode 0644
102: end
103: end

Compiled Resource:

Declared in /tmp/vagrant-chef-1/chef-solo-1/cookbooks/cassandra/recipes/tarball.rb:97:in `block in from_file'

template("/etc/cassandra/cassandra.yaml") do
provider Chef::Provider::Template
action "create"
retries 0
retry_delay 2
path "/etc/cassandra/cassandra.yaml"
backup 5
source "cassandra.yaml.erb"
cookbook_name :cassandra
recipe_name "tarball"
owner "cassandra"
group "cassandra"
mode 420
end

Template Context:

on line #235
233: # Ex: ",,"
234: #- seeds: "127.0.0.1"
235: - seeds: "<%= node[:cassandra][:seeds].join(",") %>"
236:
237: # emergency pressure valve: each time heap usage after a full (CMS)

Incorrect yum release specified for 2.0.8 RPM

The yum RPM package for dsc20 has a release of 2 instead of 1 so it yum package installation fails.

[ec2-user@cassandra1 ~]$ yum info dsc20
Available Packages
Name        : dsc20
Arch        : noarch
Version     : 2.0.8
Release     : 2
Size        : 1.6 k
Repo        : datastax
Summary     : Meta RPM for installation of the DataStax DSC platform
URL         : http://www.datastax.com/products/community
License     : ASL 2.0
Description : DataStax Community Edition is a free packaged distribution of the Apache
            : Cassandra database.
            :
            : This package depends on the other components of DSC.

Version 2.0.1 no longer hosted on apache dist

Need to update to 2.0.3

Chef 11.10 compatibility issue

Hello,

I've just tried to install Cassandra using your recipe. Unfortunately it does not compile using Chef 11.10.4.

Here's my environment:

Chef-Solo: 11.10.4
Vagrant: 1.6.3
Berkshelf: 3.1.4
Ubuntu ubuntu-14.04

This is the error message I get while runing vagrant provision:

==> default: ================================================================================
==> default: Recipe Compile Error in /tmp/vagrant-chef-3/chef-solo-1/cookbooks/cassandra-node/recipes/default.rb
==> default: ================================================================================
==> default:
==> default:
==> default: ArgumentError
==> default: -------------
==> default: You must supply a name when declaring a default resource
==> default:
==> default:
==> default: Cookbook Trace:
==> default: ---------------
==> default:   /tmp/vagrant-chef-3/chef-solo-1/cookbooks/cassandra/recipes/datastax.rb:23:in `from_file'
==> default:   /tmp/vagrant-chef-3/chef-solo-1/cookbooks/cassandra/recipes/default.rb:20:in `from_file'
==> default:   /tmp/vagrant-chef-3/chef-solo-1/cookbooks/cassandra-node/recipes/cassandra.rb:9:in `from_file'
==> default:   /tmp/vagrant-chef-3/chef-solo-1/cookbooks/cassandra-node/recipes/default.rb:9:in `from_file'
==> default:
==> default:
==> default: Relevant File Content:
==> default: ----------------------
==> default: /tmp/vagrant-chef-3/chef-solo-1/cookbooks/cassandra/recipes/datastax.rb:
==> default:
==> default:  16:  # See the License for the specific language governing permissions and
==> default:  17:  # limitations under the License.
==> default:  18:  #
==> default:  19:
==> default:  20:  # This recipe relies on a PPA package and is Ubuntu/Debian specific. Please
==> default:  21:  # keep this in mind.
==> default:  22:
==> default:  23>> default[:cassandra][:conf_dir] = '/etc/cassandra/'
==> default:  24:
==> default:  25:  include_recipe "java"
==> default:  26:
==> default:  27:  Chef::Application.fatal!("attribute node['cassandra']['cluster_name'] not defined") unless node.cassandra.cluster_name
==> default:  28:
==> default:  29:  include_recipe "cassandra::user" if node.cassandra.setup_user
==> default:  30:
==> default:  31:  [node.cassandra.root_dir,
==> default:  32:    node.cassandra.log_dir,
==> default:
==> default:
==> default:
==> default: [2014-08-01T15:31:18+00:00] ERROR: Running exception handlers
==> default: [2014-08-01T15:31:18+00:00] ERROR: Exception handlers complete
==> default: [2014-08-01T15:31:18+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
==> default: [2014-08-01T15:31:18+00:00] ERROR: You must supply a name when declaring a default resource
==> default: [2014-08-01T15:31:18+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Chef never successfully completed! Any errors should be visible in the
output above. Please fix your recipes so that they properly complete.

Multiple Changes

Hello!

I have made some changes to the cookbook, some are just an addition but some are more of a change in cookbook existing resources.

like:

use ulimit cookbook for ulimits, add node.cassandra.limits.nproc
make jna recipe configurable by an attribute
make datastax/tarball recipe setup configurable by an attribute, so that we would just add recipe[cassandra], configure node.cassandra.flavor = 'datastax', default tarball
use node.cassandra.service_name to create service & init.d template resource in tarball
separate cassandra user setup to a recipe and make user setup configurable by attribute, node.cassandra.setup_user = true, default false
add node.cassandra.group to have both user and group, current recipe uses node.cassandra.user for group ownership
change init.d script resource for node.patform_family instead of family
add RHEL platform_family init.d script
update bash resource which does chown -R everytime chef runs which is not required and could create problem for large data sets. when C* process is run by node.cassandra user it will create system/data other data location folders with right ownership and does not require chown -R bash resource on every chef run
change install/extract flow to run chown one time on extracted folder
update node.cassandra.cluster_name = nil, it could/would create problem if node is accidentally provisioned without correct cluster_name, though default cluster is not meant for production. But for production node provisioning one would have to spend extra time to flush the new node and would be a problem if dont know the steps (saying from experience)
added check for node.cassandra.cluster_name
removed user/group resource from datastax & tarball recipe and created a separate recipe

These changes might not add much of value to running C* cluster and would make setup a bit better.

Feedback is appreciated.

Cheers

Cassandra cluster_name needs description in README

Including the cassandra::datastax cookbook requires you to set the cluster_name attribute or else it fails with an exception. However, this attribute isn't even mentioned as existing in the README. Should be added with a mention that it's required.

Consider changing default rpc_address to "0.0.0.0"

The current setting for rpc_address is the ip address of the box.
However this prevents local clients from connecting to the server (although they will connect using the node ip address) as it won't be listening on the loopback adapter.

Property changes in cassandra 2.0.6

Hello,

It looks like cassandra recently got updated to v2.0.6 (I am using the cassandra::tarball recipe with the "package_name": "dsc20" attribute) and there are issues when starting the service with the provided configuration template:

first with the row_cache_provider option

org.apache.cassandra.exceptions.ConfigurationException: Invalid yaml. Please remove properties [row_cache_provider] from your cassandra.yaml
    at org.apache.cassandra.config.YamlConfigurationLoader$MissingPropertiesChecker.check(YamlConfigurationLoader.java:132)
    at org.apache.cassandra.config.YamlConfigurationLoader.loadConfig(YamlConfigurationLoader.java:95)
    at org.apache.cassandra.config.DatabaseDescriptor.loadConfig(DatabaseDescriptor.java:135)
    at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:111)
    at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:153)
    at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:471)
    at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:560)
Invalid yaml. Please remove properties [row_cache_provider] from your cassandra.yaml
Fatal configuration error; unable to start. See log for stacktrace.

secondly commitlog_sync_batch_window_in_ms

ERROR 12:19:42,554 Fatal configuration error
org.apache.cassandra.exceptions.ConfigurationException: commitlog_sync_period_in_ms specified, but commitlog_sync_batch_window_in_ms found.  Only specify commitlog_sync_period_in_ms when using periodic sync.
    at org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:170)
    at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:111)
    at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:153)
    at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:471)
    at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:560)
commitlog_sync_period_in_ms specified, but commitlog_sync_batch_window_in_ms found.  Only specify commitlog_sync_period_in_ms when using periodic sync.
Fatal configuration error; unable to start. See log for stacktrace.

Default recipe wants `--force-yes` to install 1.2.9

This appears to be the error:
STDERR: E: There are problems and -y was used without --force-yes

[2013-10-22T21:26:52+00:00] DEBUG: package[cassandra] current version is 2.0.1
[2013-10-22T21:26:52+00:00] DEBUG: package[cassandra] candidate version is 2.0.1
[2013-10-22T21:26:52+00:00] DEBUG: Executing apt-get -q -y install cassandra=1.2.9
[2013-10-22T21:26:53+00:00] DEBUG: ---- Begin output of apt-get -q -y install cassandra=1.2.9 ----
[2013-10-22T21:26:53+00:00] DEBUG: STDOUT: Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
jsvc libcommons-daemon-java
Suggested packages:
java-virtual-machine
The following NEW packages will be installed:
jsvc libcommons-daemon-java
The following packages will be DOWNGRADED:
cassandra
0 upgraded, 2 newly installed, 1 downgraded, 0 to remove and 83 not upgraded.
Need to get 13.8 MB of archives.
After this operation, 87.0 kB disk space will be freed.
[2013-10-22T21:26:53+00:00] DEBUG: STDERR: E: There are problems and -y was used without --force-yes
[2013-10-22T21:26:53+00:00] DEBUG: ---- End output of apt-get -q -y install cassandra=1.2.9 ----
[2013-10-22T21:26:53+00:00] DEBUG: Ran apt-get -q -y install cassandra=1.2.9 returned 100

================================================================================
Error executing action `install` on resource 'package[cassandra]'

Chef::Exceptions::Exec

apt-get -q -y install cassandra=1.2.9 returned 100, expected 0
---- Begin output of apt-get -q -y install cassandra=1.2.9 ----
STDOUT: Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
jsvc libcommons-daemon-java
Suggested packages:
java-virtual-machine
The following NEW packages will be installed:
jsvc libcommons-daemon-java
The following packages will be DOWNGRADED:
cassandra
0 upgraded, 2 newly installed, 1 downgraded, 0 to remove and 83 not upgraded.
Need to get 13.8 MB of archives.
After this operation, 87.0 kB disk space will be freed.STDERR: E: There are problems and -y was used without --force-yes
---- End output of apt-get -q -y install cassandra=1.2.9 ----

Resource Declaration:

In /root/chef-solo/cookbooks-1/cassandra/recipes/datastax.rb

69:     package "cassandra" do
70:       action :install
71:       version node[:cassandra][:version]
72:     end
73:   end

Compiled Resource:

Declared in /root/chef-solo/cookbooks-1/cassandra/recipes/datastax.rb:69:in `from_file'

package("cassandra") do
action [:install]
retries 0
retry_delay 2
package_name "cassandra"
version "1.2.9"
cookbook_name :cassandra
recipe_name "datastax"
end

[2013-10-22T21:26:53+00:00] INFO: Running queued delayed notifications before re-raising exception
[2013-10-22T21:26:53+00:00] DEBUG: Re-raising exception: Chef::Exceptions::Exec - package[cassandra](cassandra::datastax line 69) had an error: Chef::Exceptions::Exec: apt-get -q -y install cassandra=1.2.9 returned 100, expected 0

Invalid yaml (native_transport_min_threads & native_transport_max_threads)

Getting an issue provisioning since the latest commit (b818e3c)

Commenting out native_transport_min_threads and native_transport_max_threads fixes the issue.

ERROR [main] 2014-12-03 16:29:36,574 DatabaseDescriptor.java (line 116) Fatal configuration error
org.apache.cassandra.exceptions.ConfigurationException: Invalid yaml. Please remove properties [native_transport_min_threads] from your cassandra.yaml
    at org.apache.cassandra.config.YamlConfigurationLoader$MissingPropertiesChecker.check(YamlConfigurationLoader.java:132)
    at org.apache.cassandra.config.YamlConfigurationLoader.loadConfig(YamlConfigurationLoader.java:95)
    at org.apache.cassandra.config.DatabaseDescriptor.loadConfig(DatabaseDescriptor.java:135)
    at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:111)
    at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:156)
    at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
    at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)

E: The method driver /usr/lib/apt/methods/https could not be found.

On a fresh install of Debian 7.4, I get the following error with the datastax recipe :

E: The method driver /usr/lib/apt/methods/https could not be found.

This comes from the datastax repository, declared with "https://debian.datastax.com/...". I see two solutions : declare the repository with http (like in the documentation) or add a dependency to apt-transport-https.

The work-around is to install apt-transport-https (manually / with chef).

Publish the cookbook to the Chef Supermarket

I use cassandra I get https://github.com/gbatalski/cassandra

I do berks search cassandra I get

cassandra (0.2.4)
cassandra-opscenter (0.1.0)
cassandra-priam (0.1.3)

fix rubocop offenses

19 files inspected, 364 offenses detected ...

Make the yum repo creation a configurable option

The part where the yum datastax repo is created-is it possible to do this only if a node attribute is setup to explicitly specify that the yum repo needs to be created. That way for those of us who have the dse rpm's hosted in our private yum repo, we get to bypass this step

DSE recipe installs the wrong package

While it may be possible to install dsc20 from a Datastax Enterprise Repo there is no way that the DSE cassandra can be installed and configured using the templates and recipes in this repository. There are no configuration templates for DSE services. The 'DSE' integration test does not try to install the 'dse' package either - it installs the 'dsc20' instead.

Please update the README to clearly state that the Datastax Enterprise distribution of Cassandra is NOT supported so people don't waste time trying to make it work.

fix foodcritic

$ foodcritic .
FC002: Avoid string interpolation where not required: ./recipes/opscenter_agent_datastax.rb:72
FC002: Avoid string interpolation where not required: ./recipes/opscenter_agent_tarball.rb:22
FC002: Avoid string interpolation where not required: ./recipes/opscenter_server.rb:61
FC003: Check whether you are running with chef server before using server-specific features: ./recipes/opscenter_agent_datastax.rb:64
FC003: Check whether you are running with chef server before using server-specific features: ./recipes/opscenter_agent_tarball.rb:31
FC019: Access node attributes in a consistent manner: ./recipes/datastax.rb:39
FC019: Access node attributes in a consistent manner: ./recipes/datastax.rb:43
FC019: Access node attributes in a consistent manner: ./recipes/datastax.rb:84
FC019: Access node attributes in a consistent manner: ./recipes/datastax.rb:109
FC019: Access node attributes in a consistent manner: ./recipes/opscenter_agent_datastax.rb:24
FC019: Access node attributes in a consistent manner: ./recipes/opscenter_agent_datastax.rb:27
FC019: Access node attributes in a consistent manner: ./recipes/opscenter_agent_datastax.rb:28
FC019: Access node attributes in a consistent manner: ./recipes/opscenter_server.rb:24
FC019: Access node attributes in a consistent manner: ./recipes/opscenter_server.rb:27
FC019: Access node attributes in a consistent manner: ./recipes/opscenter_server.rb:28
FC023: Prefer conditional attributes: ./recipes/datastax.rb:202
FC023: Prefer conditional attributes: ./recipes/tarball.rb:144
FC023: Prefer conditional attributes: ./recipes/tarball.rb:155
FC039: Node method cannot be accessed with key: ./templates/default/cassandra-topology.properties.erb:7
FC043: Prefer new notification syntax: ./recipes/opscenter_server.rb:70

'2.5.0-pre' does not match 'x.y.z' or 'x.y'

When I use the latest version of the cookbook I get the following error:

[2014-07-16T22:42:59+00:00] ERROR: '2.5.0-pre' does not match 'x.y.z' or 'x.y'

Googling around it seems this value from metadata.rb is invalid?

Consider adding git tags on released versions

Would you please consider adding git tags for released versions of this cookbook (e.g. "v2.2.0")?

This makes it much easier to understand what version of a cookbook is required when adding entries to a Berksfile or Cheffile.

Compare:

cookbook "cassandra", git: "git://github.com/michaelklishin/cassandra-chef-cookbook.git", ref: "7809857663c3e8bb2e1c869120b53923b9b662ea"

vs.

cookbook "cassandra", git: "git://github.com/michaelklishin/cassandra-chef-cookbook.git", ref: "v2.2.0"

Thanks!

Cassandra fails to restart after setting the cluster name

Hello there,

I've set up a small Cassandra Cluster using your cookbook. I'm using the default datastax installation method to install Cassandra 2. Unfortunately, I ran into a problem when setting a custom cluster name:

node.set[:cassandra][:cluster_name] = 'my-little-cassandra-cluster'
node.set[:cassandra][:start_native_transport] = true
...
include_recipe 'cassandra::default'

Here's my environment:

Chef: 11.10.4
Vagrant: 1.6.3 and Current Opsworks stack
Berkshelf: 3.1.4
Ubuntu ubuntu-14.04

When deploying Cassandra on my node, the installation works as expected. But when rebooting Cassandra to activate the changes (or with node[:cassandra][:notify_restart] enabled), Cassandra dies after the restart. The logs contain the following Stacktrace:

ERROR [main] 2014-08-08 11:12:56,619 CassandraDaemon.java (line 265) Fatal exception during initialization
org.apache.cassandra.exceptions.ConfigurationException: Saved cluster name Test Cluster != configured name hockey-telemetry-cassandra
        at org.apache.cassandra.db.SystemKeyspace.checkHealth(SystemKeyspace.java:564)
        at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:261)
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:496)
        at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:585)

As far as I can tell the problem is caused by Cassandra being started automatically after installing the .deb-package. Therefore, the cluster-name is alrady set to "Test Cluster" and changing it results in the error above.

I've found a way to circumvent this problem, but it feels like a hack to me:

# Work around cassandra starting with the wrong cluster name
execute 'cqlsh' do
  command %(
  listen_address = `lsof -iTCP:#{node[:cassandra][:rpc_port]} -sTCP:LISTEN -n -F n | grep -oE "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}"`
  cqlsh $listen_address -e "update system.local set cluster_name='#{node[:cassandra][:cluster_name]}' where key='local';"
  nodetool flush
  )

  not_if %(cqlsh #{node[:ipaddress]} -e 'select cluster_name from system.local' | grep #{node[:cassandra][:cluster_name]}$)

  action :run
end

Is there any way you can prevent Cassandra from starting before deploying the config file?

Cheers,
pfleidi

Init script on Centos 6.4

When I install using the cookbook everything works until cassandra is attempting to be started and I get the following.

Expected process to exit with [0], but received '1'
---- Begin output of /sbin/service cassandra start ----
STDOUT:
STDERR: /etc/init.d/cassandra: line 35: /lib/init/vars.sh: No such file or directory
---- End output of /sbin/service cassandra start ----
Ran /sbin/service cassandra start returned 1

Now I have confirmed that the file indeed does not exist on the server. What needs to be in this file? Do I even need it?

FC048 should not evaluate 'system' in user and group resources

Foodcritic reads users.rb and fails on "system" keyword. FC048 has an exception for user and group resources, but only when they has a boolean value. Once node attribute assign it fails.
I opened issue in foodcritic to disable FC048 evaluation in user and group.

You must set the CASSANDRA_CONF and CLASSPATH vars

You'll have to excuse my ignorance on this. I've been able to find zero documentation on a) what these need to be set to or b) how to set them in my cassandra recipe.

EDIT: I'm using the tarball installation. I'm overridng the version to 1.2.10 (since 9 doesn't exist in the specified location anymore).

repo_name is not longer supported by yum 3.x recipe

Yum 3.0.0 release removed repo_name support

-# name of the repo, used for .repo filename
-attribute :repo_name, kind_of: String, name_attribute: true
-attribute :description, kind_of: String # long description

But we are still using it here

I think we need some check like this, or only support Yum 3.x by remove repo_name

if Chef::Resource::YumRepository.respond_to?(:repo_name)

Invalid yaml when installing dsc v2.1.2

After installing using the default recipe the cassandra service fails to start. Upon closer inspection of the logs, it is throwing an exception.

My attributes:

default[:cassandra][:cluster_name] = 'ClusterName'
default[:cassandra][:version] = '2.1.2'
default[:cassandra][:package_name] = 'dsc21'
default[:cassandra][:setup_jamm] = true

Error:

Invalid yaml. Please remove properties [multithreaded_compaction, memtable_flush_queue_size, compaction_preheat_key_cache, in_memory_compaction_limit_in_mb] from your cassandra.yaml

Trace:

ERROR [main] 2014-11-19 04:45:53,433 DatabaseDescriptor.java:138 - Fatal configuration error
org.apache.cassandra.exceptions.ConfigurationException: Invalid yaml. Please remove properties [multithreaded_compaction, memtable_flush_queue_size, compaction_preheat_key_cache, in_memory_compaction_limit_in_mb] from your cassandra.yaml
        at org.apache.cassandra.config.YamlConfigurationLoader$MissingPropertiesChecker.check(YamlConfigurationLoader.java:162) ~[apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.config.YamlConfigurationLoader.loadConfig(YamlConfigurationLoader.java:115) ~[apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.config.YamlConfigurationLoader.loadConfig(YamlConfigurationLoader.java:84) ~[apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.config.DatabaseDescriptor.loadConfig(DatabaseDescriptor.java:158) ~[apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.config.DatabaseDescriptor.<clinit>(DatabaseDescriptor.java:133) ~[apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:96) [apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:448) [apache-cassandra-2.1.2.jar:2.1.2]
        at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:537) [apache-cassandra-2.1.2.jar:2.1.2]

Any ideas?

Usage os init.d script in DataStax recipe

Hi!

Just started to use your cookbook.
As far I want, I've fixed some chef issues with users and groups on Ubuntu 12.04 in my fork.
When it will finally work, I will send you pull request.

I've stumbled on setting rpc address using datastax recipe, but four that that recipe does not use init.d template at all.

Is it intentional or I can fix it in coming pull request?

Xmx and Xms not set via :jvm:xmx :jvm:xms

The documentation describes the following options:

node[:cassandra][:jvm][:xms](default: 32) and node[:cassandra][:jvm][:xmx](default: 512) control JVM -Xms and -Xms flag values, in megabytes (no need to add the m suffix)

However, I cannot find any place that these settings are actually used within the recipe. I am able to control the Xmx via the :max_heap_size setting.

Recipe stopped compiling in Chef 11

We recently upgraded our gem to chef 11 and the cookbook (which we're all really fond of) stopped working. It seems to be that the dot notation used i.e. node.cassandra.user is failing where bracket notation is working i.e. node["cassandra"]["user"]. I'm starting to refactor now. I was wondering if anyone else has reported this or not.

Regards,

Bob

cassandra rpm puts configs in soft linked directory for centos

And possibly other distros.
The import part of the rpm script install is this
alternatives --install /etc/cassandra/conf cassandra /etc/cassandra/default.conf/ 0

The cookbook installs the yaml file to /etc/cassandra/cassandra.yaml and the rpm init script look at

export CASSANDRA_CONF=/etc/cassandra/conf

[root@cass01 cassandra]# ls -sl /etc/cassandra/
total 48K
 12K -rw-r--r-- 1 cassandra cassandra 9.5K Aug  7 01:57 cassandra-env.sh
 32K -rw-r--r-- 1 cassandra cassandra  31K Aug  7 01:57 cassandra.yaml
   0 lrwxrwxrwx 1 root      root        27 Aug  7 01:54 conf -> /etc/alternatives/cassandra
4.0K drwxr-xr-x 2 cassandra cassandra 4.0K Aug  7 02:00 default.conf

rpm -q --scripts cassandra output

preinstall scriptlet (using /bin/sh):
getent group cassandra >/dev/null || groupadd -r cassandra
getent passwd cassandra >/dev/null || \
useradd -d /usr/share/cassandra -g cassandra -M -r cassandra
exit 0
postinstall scriptlet (using /bin/sh):
alternatives --install /etc/cassandra/conf cassandra /etc/cassandra/default.conf/ 0
# alternatives --install /etc/default/cassandra cassandra /etc/cassandra/default.conf/cassandra.default 0
cd /usr/share/cassandra/lib
grep "release 5" /etc/redhat-release > /dev/null 2> /dev/null
if [ $? -eq 0 ]; then
  # Put old snappy file in place for old Linux distros. Basically
  # rename the newer version out of the way.
  if [ -f snappy-java-1.0.5.jar ]; then
    if [ -f snappy-java-1.0.5.jar.backup ]; then
      /bin/rm -f snappy-java-1.0.5.jar.backup
    fi
    /bin/mv snappy-java-1.0.5.jar snappy-java-1.0.5.jar.backup
  fi
else
  # Move old version of snappy out of the way on modern Linux versions
  if [ -f snappy-java-1.0.5.jar ]; then
    if [ -f snappy-java-1.0.4.1.jar ]; then
      if [ -f snappy-java-1.0.4.1.jar.backup ]; then
        /bin/rm -f snappy-java-1.0.4.1.jar.backup
      fi
      /bin/mv snappy-java-1.0.4.1.jar snappy-java-1.0.4.1.jar.backup
    fi
  fi
fi
exit 0
preuninstall scriptlet (using /bin/sh):
# only delete alternative on removal, not upgrade
if [ "$1" = "0" ]; then
    alternatives --remove cassandra /etc/cassandra/default.conf/
fi
if [ "$1" = "0" ]; then
  # restore original snappy files so package management is happy on removal
  cd /usr/share/cassandra/lib
  if [ -f snappy-java-1.0.5.jar.backup ]; then
    if [ -f snappy-java-1.0.5.jar ]; then
      /bin/rm -f snappy-java-1.0.5.jar
    fi
    /bin/mv snappy-java-1.0.5.jar.backup snappy-java-1.0.5.jar
  fi
  if [ -f snappy-java-1.0.4.1.jar.backup ]; then
    if [ -f snappy-java-1.0.4.1.jar ]; then
      /bin/rm -f snappy-java-1.0.4.1.jar
    fi
    /bin/mv snappy-java-1.0.4.1.jar.backup snappy-java-1.0.4.1.jar
  fi
fi
exit 0

metadata.rb missing depends "apt"

I kept receiving the below error. Once I added depends "apt" to metadata.rb, it worked.

[2013-08-15T15:37:57+00:00] DEBUG: Line number of compile error: '23'

================================================================================
Recipe Compile Error in /var/chef/cache/cookbooks/cassandra/recipes/default.rb
================================================================================


NameError
---------
Cannot find a resource for apt_repository on ubuntu version 12.04


Cookbook Trace:
---------------
  /var/chef/cache/cookbooks/cassandra/recipes/datastax.rb:23:in `from_file'
  /var/chef/cache/cookbooks/cassandra/recipes/default.rb:1:in `from_file'


Relevant File Content:
----------------------
/var/chef/cache/cookbooks/cassandra/recipes/datastax.rb:

 16:  # See the License for the specific language governing permissions and
 17:  # limitations under the License.
 18:  #
 19:
 20:  # This recipe relies on a PPA package and is Ubuntu/Debian specific. Please
 21:  # keep this in mind.
 22:
 23>> apt_repository "datastax" do
 24:    uri          "http://debian.datastax.com/community"
 25:    distribution "precise"
 26:    components   ["main"]
 27:    key          "http://debian.datastax.com/debian/repo_key"
 28:
 29:    action :add
 30:  end
 31:
 32:  # DataStax Server Community Edition package will not install w/o this

Versions / Tags / Releases

This is a great cookbook! Do you have any plans to bump the version to 2.3? More generally, it would be really nice if you included tags for specific version releases to make it easier to use w/ chef server deploys.

If I set 'listen_address' to 'localhost' I can't get Cassandra to broadcast on localhost.

If I set:
node.override['cassandra']['listen_address'] = 'localhost'

I cannot get Usergrid to see Cassandra on localhost. I can telnet:

telnet <hostname> 9160 -YES
telnet <ip address> 9160 -YES
telnet localhost 9160 - NO

Add multiple data_dir locations (JBOD Cassandra 1.2+)

For C* 1.2+ it will be nice to have option to define array of data_dirs and template will populate multiple locations.

Is the datastax.rb standalone recipe supported on Ubuntu 12.10?

I'm upgrading from 12.04 and wanted to see if I could install the datastax recipe w/out running the other chef recipe? Is this supported on 12.10?

Thank you in advance

Missing depends on java

Looks like a depends for java in metadata.rb is missing.

I would expect this line at the bottom of metadata.rb

depends "java"

Not having this while installing the cookbook results in the following role:

FATAL: Saving node information to /var/chef/cache/failed-run-data.json
[2013-03-26T23:13:23+00:00] ERROR: Exception handlers complete
Chef Client failed. 4 resources updated
[2013-03-26T23:13:23+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2013-03-26T23:13:23+00:00] FATAL: Chef::Exceptions::CookbookNotFound: Cookbook java not found. If you're loading java from another cookbook, make sure you configure the dependency in your metadata

Bug: JAR manifest missing : /usr/share/cassandra/lib/jamm-0.2.5.jar

Hey guys,

Noticed that when installing dsc21, the log gives an error for Error opening zip file or JAR manifest missing : /usr/share/cassandra/lib/jamm-0.2.5.jar. This can be fixed by simply checking if we're using dsc21 or greater, and doing a remote_file.

Will try to make a PR now to do that, and will add tests.

The init script does not work on centos

Hi,

The init script you have given in the templates directory does not work for centos as the ". /lib/init/vars.sh" , ". /lib/lsb/init-functions" , "start-stop-daemon" do not exist on centos .. We need to do a case "platform" in the recipe which them uses that particular platforms init script.

I will write a new init script for it and submit it for review later ..

Please let me know if you have any questions.

Thanks,
Raj

Thrift cookbook fails to provision

[2014-10-03T20:06:32+00:00] ERROR: remote_file[/tmp/kitchen/cache/thrift-0.9.0.tar.gz] (thrift::default line 37) had an error: Net::HTTPServerException: 404 "Not Found"
[2014-10-03T20:06:32+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
>>>>>> Converge failed on instance <default-ubuntu-1404>.
>>>>>> Please see .kitchen/logs/default-ubuntu-1404.log for more details
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: SSH exited (1) for command: [sudo -E chef-solo --config /tmp/kitchen/solo.rb --json-attributes /tmp/kitchen/dna.json  --log_level info]
>>>>>> ----------------------

Basically the thrift cookbook you are including has a broken remote file fetch attempt, that no longer links to anything and is breaking this cookbooks build. Not sure what you can do about it, other than be aware it exists.

RFC: put all template values into attributes

The cassandra.yaml file is heavily templated and has many hard coded values in it. The config file is large, and has many possible values and combinations.

I don't want to submit a bunch of pull requests for these, and i don't care about the comments in the file either. (Others might care about these comments, we can put them into the config.rb attributes file if people wish.)

I am proposing that we do away with the keys, and comments in the erb template and instead generate the yaml from a hash in the attributes.

This way we can easily manage the hundred or so config values in attributes, and add additional settings that are not in the template that can be set.

What are peoples thoughts on this? I will get to work on a PR if people are in agreement.

Cassandra datastax installation issue

Hi,
I am trying to provision my machine with with cassandra and the datastax enteprise. I am also use vagrant to create multiple machines.

Vagrant script

(https://github.com/dholbrook/vagrant-cassandra)

chef.add_recipe "cassandra::datastax"
chef.json = {
  :java =>  {
    'install_flavor' => 'oracle',
    'jdk_version' => '7',
    'oracle' => {
      'accept_oracle_download_terms' => true
    }
  },
  :cassandra => {
    'version' => CASSANDRA_VERSION,
    'dse' => {
        'credentials' => {
            'username' => 'username',
            'password' => 'password'
        }
    },
    'seeds' => SEEDS,
    'listen_address' => n['addr'],
    'broadcast_address' => n['addr'],
    'rpc_address' => '0.0.0.0',
    'vnodes' => VNODES,
    'cluster_name' => CLUSTER_NAME  
  }  
}

Error

ERROR: template[/etc/cassandra/cassandra.yaml] (cassandra::datastax line 181) had an error: Chef::Exceptions::EnclosingDirectoryDoesNotExist: Parent directory /etc/cassandra does not exist.

syntax error in repository.rb

Hello, I'm trying to install cassandra following the instruction here https://github.com/calebgroom/vagrant-cassandra and using the following vagrant box
https://dl.dropbox.com/u/14292474/vagrantboxes/precise64-ruby-1.9.3-p194.box

While executing vagrant up I get the following error:

================================================================================
Recipe Compile Error in /tmp/vagrant-chef-1/chef-solo-1/cookbooks/yum/providers/repository.rb
================================================================================


SyntaxError
-----------
compile error
/tmp/vagrant-chef-1/chef-solo-1/cookbooks/yum/providers/repository.rb:108: syntax error, unexpected ':', expecting ')'
      repo_name: new_resource.repo_name,
                ^
/tmp/vagrant-chef-1/chef-solo-1/cookbooks/yum/providers/repository.rb:109: syntax error, unexpected ':', expecting '='
      description: new_resource.description,
                  ^
/tmp/vagrant-chef-1/chef-solo-1/cookbooks/yum/providers/repository.rb:110: syntax error, unexpected ':', expecting '='
      url: new_resource.url,
          ^
/tmp/vagrant-chef-1/chef-solo-1/cookbooks/yum/providers/repository.rb:111: syntax error, unexpected ':', expecting '='
      mirrorlist: new_resource.mirrorlist,
                 ^
/tmp/vagrant-chef-1/chef-solo-1/cookbooks/yum/providers/repository.rb:112: syntax error, unexpected ':', expecting '='
      key: new_resource.key,
          ^
/tmp/vagrant-chef-1/chef-solo-1/cookbooks/yum/providers/repository.rb:113: syntax error, unexpected ':', expecting '='
      enabled: new_resource.enabled,
              ^
/tmp/vagrant-chef-1/chef-solo-1/cookbooks/yum/providers/repository.rb:114: syntax error, unexpected ':', expecting '='
      type: new_resource.type,
           ^
/tmp/vagrant-chef-1/chef-solo-1/cookbooks/yum/providers/repository.rb:115: syntax error, unexpected ':', expecting '='
      failovermethod: new_resource.failovermethod,
                     ^
/tmp/vagrant-chef-1/chef-solo-1/cookbooks/yum/providers/repository.rb:116: syntax error, unexpected ':', expecting '='
      bootstrapurl: new_resource.bootstrapurl,
                   ^
/tmp/vagrant-chef-1/chef-solo-1/cookbooks/yum/providers/repository.rb:117: syntax error, unexpected ':', expecting '='
      includepkgs: new_resource.includepkgs,

The box has ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux]

Thanks in advance
Andrea

[Improved] Cassandra 2.1.0 support

Before we can proceed with #107, as @vkhatri points out, there's a bunch of improvements we have to make that are 2.1.0-specific. Quoting @vkhatri from #107:

jna is now part of C* package, jna setup needs to be modified for backward compatibility
logging framework used in 2.1.0 is logback (instead of log4j), need to add template and version dependency
some new cassandra.yaml configuration parameters

Arch for java already determined by java cookbook

In attributes/default.rb, the java architecture is set. For i686 architectures (and above), this causes problems:

NoMethodError
-------------
undefined method `[]' for nil:NilClass

Cookbook Trace:
---------------
  /var/chef/cache/cookbooks/java/recipes/oracle.rb:38:in `from_file'
  /var/chef/cache/cookbooks/java/recipes/default.rb:28:in `from_file'

Relevant File Content:
----------------------
/var/chef/cache/cookbooks/java/recipes/oracle.rb:

 31:
 32:  case node['java']['jdk_version'].to_s
 33:  when "6"
 34:    tarball_url = node['java']['jdk']['6'][arch]['url']
 35:    tarball_checksum = node['java']['jdk']['6'][arch]['checksum']
 36:    bin_cmds = node['java']['jdk']['6']['bin_cmds']
 37:  when "7"
 38>>   tarball_url = node['java']['jdk']['7'][arch]['url']
 39:    tarball_checksum = node['java']['jdk']['7'][arch]['checksum']
 40:    bin_cmds = node['java']['jdk']['7']['bin_cmds']
 41:  when "8"
 42:    tarball_url = node['java']['jdk']['8'][arch]['url']
 43:    tarball_checksum = node['java']['jdk']['8'][arch]['checksum']
 44:    bin_cmds = node['java']['jdk']['8']['bin_cmds']
 45:  end
 46:
 47:  if tarball_url =~ /example.com/

The java cookbook already sets it to an appropriate value that it can handle:

default['java']['arch'] = kernel['machine'] =~ /x86_64/ ? "x86_64" : "i586"

Is there any reason this line can't be removed?

p.s. This is a little bizarre of a request I know... given no one sane will be running cassandra 32bit, hah.

Use semver correctly with the cookbook

The issue I have is that there have been many changes to the cookbook without bumping the versions as per semver.
This causes dependency management issues.

Going forward, can you maintain the cookbook to the Cookbook Versioning Policy?

prep for lint tests and ci

add files for various lints and travis ci.

Submit to the community site

This is a great cookbook. You should submit it to the community site: http://community.opscode.com/

Use of incorrect cookbook version format prevents berkshelf from being able to install the cookbook

Getting the following error while attempting to use berkshelf to install this cookbook. The reason why is the current version of the cookbook is 2.5.0.pre when really it should be 2.5.0-pre in order to conform with the Chef cookbook versioning standard (see this Berkshelf issue).

wasted:womply-kitchen brent$ berks
Resolving cookbook dependencies...
Fetching 'cassandra' from https://github.com/michaelklishin/cassandra-chef-cookbook.git (at master)
/Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/semverse-1.1.0/lib/semverse/version.rb:29:in `split': '2.5.0.pre' did not contain a valid version string: 'x.y.z' or 'x.y'. (Semverse::InvalidVersionFormat)
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/semverse-1.1.0/lib/semverse/version.rb:64:in `initialize'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/semverse-1.1.0/lib/semverse/version.rb:10:in `new'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/semverse-1.1.0/lib/semverse/version.rb:10:in `coerce'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/semverse-1.1.0/lib/semverse/constraint.rb:240:in `satisfies?'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/berkshelf-3.1.3/lib/berkshelf/locations/base.rb:64:in `validate_cached!'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/berkshelf-3.1.3/lib/berkshelf/locations/git.rb:67:in `install'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/berkshelf-3.1.3/lib/berkshelf/installer.rb:179:in `block in download_locations'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/berkshelf-3.1.3/lib/berkshelf/installer.rb:176:in `each'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/berkshelf-3.1.3/lib/berkshelf/installer.rb:176:in `download_locations'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/berkshelf-3.1.3/lib/berkshelf/installer.rb:149:in `install_from_universe'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/berkshelf-3.1.3/lib/berkshelf/installer.rb:39:in `run'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/berkshelf-3.1.3/lib/berkshelf/berksfile.rb:371:in `install'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/berkshelf-3.1.3/lib/berkshelf/cli.rb:143:in `install'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/berkshelf-3.1.3/lib/berkshelf/cli.rb:52:in `dispatch'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/berkshelf-3.1.3/lib/berkshelf/cli.rb:27:in `execute!'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/gems/berkshelf-3.1.3/bin/berks:5:in `<top (required)>'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/bin/berks:23:in `load'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/bin/berks:23:in `<main>'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/bin/ruby_executable_hooks:15:in `eval'
        from /Users/brent/.rvm/gems/ruby-2.0.0-p451@womply-kitchen-gemset/bin/ruby_executable_hooks:15:in `<main>'

Cassandra >= 2.0 requires Java 7

When installing Cassandra 2.0, need to make sure that Java 7 is installed even if the default for the java recipe is to install Java 6.