no security group, and vagrant-mesos can't set `hostname` to its public dns name. about vagrant-mesos

[everpeace]

@hanskoff Hi! Thank you for trying this project :-)

Do you use VPC?? If so, did you confirm that sg-81eaf2e5 corresponds to the VPC id which you try to use??

[johnykov]

Hi, thanks for response ;)

Yeah, definitely I didn't configure VPC. I will read and work on it ;) Thanks for hint.

[johnykov]

Hello again,
I've confirmed that sg-81eaf2e5 corresponds to the VPC id I want to use, but problem remains ;(

btw. when I try to run my configuration
first message I get is

There are errors in the configuration of this machine. Please fix
the following errors and try again:

AWS Provider:
* vagrant_aws.config.ami_required

but this one I manage to fix with mitchellh/vagrant-aws#275

[everpeace]

@hanskoff oh, I recently updated amis.
So could you try to update your vagrant-mesos box image before vagrant up

$ vagrant box update
$ vagrant up --provider=aws

In default configuration, vagrant pulls ami metadata from vagrant cloud and launch vm instances with correct amis.

[johnykov]

I'll do that. Do you have any additional idea about not found security group?
16 cze 2015 9:02 PM "Shingo Omura" [email protected] napisał(a):

@hanskoff https://github.com/hanskoff oh, I recently updated amis.
So could you try to update your vagrant-mesos box image before vagrant up

$ vagrant box update
$ vagrant up --provider=aws

In default configuration, vagrant pulls ami metadata from vagrant cloud
and launch vm instances with correct amis.

—
Reply to this email directly or view it on GitHub
#67 (comment)
.

[everpeace]

@hanskoff hmm, I'm not sure about that. perhaps, name of security group possibly might work instead of id...

[johnykov]

ad 1. box update didn't resolve ami problem ;)
ad 2. providing group name helped instead of group id... strange
new 3 problem. chef doesn't complete successfully
I see

==> default: Ran apt-cache policy update-notifier-common returned 100
==> default: [2015-06-17T06:54:18+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Chef never successfully completed! Any errors should be visible in the
output above. Please fix your recipes so that they properly complete.

[everpeace]

@hanskoff Yay! you successfully ssh to the vm! Could you kindly paste all vagrant output??

[johnykov]

I've tried vagrant provision this time
and got same error message

Generating chef JSON and uploading...
==> default: Running chef-solo...
==> default: stdin: is not a tty
==> default: [2015-06-17T06:54:14+00:00] INFO: Forking chef instance to converge...
==> default: [2015-06-17T06:54:14+00:00] INFO: *** Chef 12.3.0 ***
==> default: [2015-06-17T06:54:14+00:00] INFO: Chef-client pid: 2524
==> default: [2015-06-17T06:54:16+00:00] INFO: Setting the run_list to ["recipe[apt]", "recipe[mesos::master]", "recipe[mesos::slave]"] from CLI options
==> default: [2015-06-17T06:54:16+00:00] INFO: Run List is [recipe[apt], recipe[mesos::master], recipe[mesos::slave]]
==> default: [2015-06-17T06:54:16+00:00] INFO: Run List expands to [apt, mesos::master, mesos::slave]
==> default: [2015-06-17T06:54:16+00:00] INFO: Starting Chef Run for ip-10-157-159-177.ec2.internal
==> default: [2015-06-17T06:54:16+00:00] INFO: Running start handlers
==> default: [2015-06-17T06:54:16+00:00] INFO: Start handlers complete.
==> default: [2015-06-17T06:54:17+00:00] INFO: node[:mesos][:prefix] is ignored. prefix will be set with /usr/local .
==> default: [2015-06-17T06:54:17+00:00] WARN: Cloning resource attributes for template[/etc/init/mesos-master.conf] from prior resource (CHEF-3694)
==> default: [2015-06-17T06:54:17+00:00] WARN: Previous template[/etc/init/mesos-master.conf]: /tmp/vagrant-chef/600f65399f86813f58e0a48b48726273/cookbooks/mesos/libraries/helpers.rb:177:in `deploy_service_scripts'
==> default: [2015-06-17T06:54:17+00:00] WARN: Current  template[/etc/init/mesos-master.conf]: /tmp/vagrant-chef/600f65399f86813f58e0a48b48726273/cookbooks/mesos/libraries/helpers.rb:200:in `activate_master_service_scripts'
==> default: [2015-06-17T06:54:17+00:00] INFO: node[:mesos][:prefix] is ignored. prefix will be set with /usr/local .
==> default: [2015-06-17T06:54:17+00:00] WARN: Cloning resource attributes for directory[/usr/local/var/mesos/deploy] from prior resource (CHEF-3694)
==> default: [2015-06-17T06:54:17+00:00] WARN: Previous directory[/usr/local/var/mesos/deploy]: /tmp/vagrant-chef/600f65399f86813f58e0a48b48726273/cookbooks/mesos/recipes/master.rb:21:in `from_file'
==> default: [2015-06-17T06:54:17+00:00] WARN: Current  directory[/usr/local/var/mesos/deploy]: /tmp/vagrant-chef/600f65399f86813f58e0a48b48726273/cookbooks/mesos/recipes/slave.rb:21:in `from_file'
==> default: [2015-06-17T06:54:17+00:00] WARN: Cloning resource attributes for template[/etc/init/mesos-slave.conf] from prior resource (CHEF-3694)
==> default: [2015-06-17T06:54:17+00:00] WARN: Previous template[/etc/init/mesos-slave.conf]: /tmp/vagrant-chef/600f65399f86813f58e0a48b48726273/cookbooks/mesos/libraries/helpers.rb:186:in `deploy_service_scripts'
==> default: [2015-06-17T06:54:17+00:00] WARN: Current  template[/etc/init/mesos-slave.conf]: /tmp/vagrant-chef/600f65399f86813f58e0a48b48726273/cookbooks/mesos/libraries/helpers.rb:213:in `activate_slave_service_scripts'
==> default: [2015-06-17T06:54:17+00:00] WARN: Cloning resource attributes for template[/etc/mesos/zk] from prior resource (CHEF-3694)
==> default: [2015-06-17T06:54:17+00:00] WARN: Previous template[/etc/mesos/zk]: /tmp/vagrant-chef/600f65399f86813f58e0a48b48726273/cookbooks/mesos/recipes/master.rb:87:in `from_file'
==> default: [2015-06-17T06:54:17+00:00] WARN: Current  template[/etc/mesos/zk]: /tmp/vagrant-chef/600f65399f86813f58e0a48b48726273/cookbooks/mesos/recipes/slave.rb:59:in `from_file'
==> default: [2015-06-17T06:54:17+00:00] WARN: Cloning resource attributes for template[/etc/default/mesos] from prior resource (CHEF-3694)
==> default: [2015-06-17T06:54:17+00:00] WARN: Previous template[/etc/default/mesos]: /tmp/vagrant-chef/600f65399f86813f58e0a48b48726273/cookbooks/mesos/recipes/master.rb:97:in `from_file'
==> default: [2015-06-17T06:54:17+00:00] WARN: Current  template[/etc/default/mesos]: /tmp/vagrant-chef/600f65399f86813f58e0a48b48726273/cookbooks/mesos/recipes/slave.rb:69:in `from_file'
==> default:
==> default: ================================================================================
==> default: Error executing action `install` on resource 'apt_package[update-notifier-common]'
==> default: ================================================================================
==> default:
==> default:
==> default: Mixlib::ShellOut::ShellCommandFailed
==> default: ------------------------------------
==> default: Expected process to exit with [0], but received '100'
==> default:
==> default: ---- Begin output of apt-cache policy update-notifier-common ----
==> default:
==> default: STDOUT:
==> default:
==> default: STDERR: E: Encountered a section with no Package: header
==> default:
==> default: E: Problem with MergeList /var/lib/apt/lists/us-east-1.ec2.archive.ubuntu.com_ubuntu_dists_trusty_main_i18n_Translation-en
==> default:
==> default: E: The package lists or status file could not be parsed or opened.
==> default:
==> default: ---- End output of apt-cache policy update-notifier-common ----
==> default:
==> default: Ran apt-cache policy update-notifier-common returned 100
==> default:
==> default:
==> default: Resource Declaration:
==> default: ---------------------
==> default: # In /tmp/vagrant-chef/600f65399f86813f58e0a48b48726273/cookbooks/apt/recipes/default.rb
==> default:
==> default:
==> default:
==> default:  75: package 'update-notifier-common' do
==> default:
==> default:  76:   notifies :run, 'execute[apt-get-update]', :immediately
==> default:
==> default:  77:   only_if { apt_installed? }
==> default:
==> default:  78: end
==> default:
==> default:  79:
==> default:
==> default:
==> default:
==> default: Compiled Resource:
==> default: ------------------
==> default: # Declared in /tmp/vagrant-chef/600f65399f86813f58e0a48b48726273/cookbooks/apt/recipes/default.rb:75:in `from_file'
==> default:
==> default:
==> default:
==> default: apt_package("update-notifier-common") do
==> default:
==> default:   action :install
==> default:
==> default:   retries 0
==> default:
==> default:   retry_delay 2
==> default:
==> default:   default_guard_interpreter :default
==> default:
==> default:   package_name "update-notifier-common"
==> default:
==> default:   timeout 900
==> default:
==> default:   declared_type :package
==> default:
==> default:   cookbook_name :apt
==> default:
==> default:   recipe_name "default"
==> default:
==> default:   only_if { #code block }
==> default:
==> default: end
==> default:
==> default:
==> default:
==> default: [2015-06-17T06:54:18+00:00] INFO: Running queued delayed notifications before re-raising exception
==> default: [2015-06-17T06:54:18+00:00] ERROR: Running exception handlers
==> default: [2015-06-17T06:54:18+00:00] ERROR: Exception handlers complete
==> default: [2015-06-17T06:54:18+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
==> default: [2015-06-17T06:54:18+00:00] ERROR: apt_package[update-notifier-common] (apt::default line 75) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '100'
==> default: ---- Begin output of apt-cache policy update-notifier-common ----
==> default: STDOUT:
==> default: STDERR: E: Encountered a section with no Package: header
==> default: E: Problem with MergeList /var/lib/apt/lists/us-east-1.ec2.archive.ubuntu.com_ubuntu_dists_trusty_main_i18n_Translation-en
==> default: E: The package lists or status file could not be parsed or opened.
==> default: ---- End output of apt-cache policy update-notifier-common ----
==> default: Ran apt-cache policy update-notifier-common returned 100
==> default: [2015-06-17T06:54:18+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
Chef never successfully completed! Any errors should be visible in the
output above. Please fix your recipes so that they properly complete.

[everpeace]

@hanskoff hmm... log said that 'apt-get update' failed.

• Are you sure to use my ami?? In Vagrantfile, what you need is aws.ami="". you don't need to set concrete ami.
• Your security groups in/out-bound rules are clean??

[johnykov]

This one helped to clean and provision again http://askubuntu.com/a/30199
;)

yeap, I've added aws.ami="" line

only mesos is up and running, marathon and chronos are down:( why is that?
when I use provide=virtualbox marathon and chronos are down as well :(

[johnykov]

when I tried to fire marathon from hand it prints

/opt/marathon$ ./bin/start
MESOS_NATIVE_JAVA_LIBRARY is not set. Searching in /usr/lib /usr/local/lib.
MESOS_NATIVE_LIBRARY, MESOS_NATIVE_JAVA_LIBRARY set to '/usr/lib/libmesos.so'
Error: Unable to access jarfile ./bin/../target/scala-2.11/marathon-assembly-0.8.2.jar

[johnykov]

I made it work ;)
sudo chmod 777 target/scala-2.11/marathon-assembly-0.8.2.jar
all 3 services are accessible by ip:port

[everpeace]

@hanskoff Great to hear that!!

By the way, in my environment, I successfully spin up mesos vm and can access all three web UIs. something seemed to go wrong in your environment, I'm not sure though.

For now, I close this issue. If you have some problem on this issue, please feel free to reopen.

[johnykov]

ok, the last thingie is when I deploy an example play app - mesos can't boot it up...
I've noticed that mesos set hostname to ip-10-170-85-220.ec2.internal which is not accessible from outside, isn't this a common issue?
how do I change mesos config?

when I ssh to ec2 instance and type hostname it's set to ip-10-170-85-220

[johnykov]

I can't reopen this ticket

[everpeace]

@hanskoff could you tell me a little bit more detail about what your essential problem is?

By the way it is difficult for vagrant to configure its hostname before provisioning the vms.

[johnykov]

for virtualbox configuration I can use marathon to deploy an app and it works.
for aws configuration, chef provider establishes mesos & marathon resolving hostname from aws ec2 private dns which is not accessible from outside and because of that - in the end no app can be deployed successfully with marathon... I wish to provide hostname parameter being public dns of aws instance... did you manage successfully to deploy app with marathon on aws config?

[everpeace]

@hanskoff oh, ok. I now see what you meant.

did you manage successfully to deploy app with marathon on aws config?

I always get public dns via this small script

$ vagrant ssh -- 'echo http://`curl --silent http://169.254.169.254/latest/meta-data/public-hostname`:8080'

Deployment itself was successfully finished by calling public dns name provided in above. And accessing deployed apps, I always manually convert internal hostname which is displayed in marathon ui to its public dns....

As I mentioned, it is hard to configure vm's hostname with its public dns from vagrant provisioner...
If you found nice way to configure vm's hostname to public dns, please give me feedback and I will be waiting for your PR!!

[everpeace]

Hi @hanskoff , I have worked how to configure hostname with its public dns name. Unfortunately, I have some bad update...

According to the ec2 document "Changing the Hostname of Your Linux Instance", to change hostname, we have to reboot the vm after changing /etc/hostname. However, as you know, default public dns name changes when the vm is booted because it contains public ip.....

Thus, I think we could have two options:

• Use your domain: If you owned some your own domain, you can set your own static hostname by userdata and can create DNS record in Route53 or similar.
• Use elastic IP: allocate elastic IPs first, then vagrant-aws set elastic ip which is already allocated. We can have static public dns name for the vm. we can set the static public dns name by userdata. however, this function now has some issue and PR hasn't been merged yet. please refer to mitchellh/vagrant-aws#129

[johnykov]

You are right, even when I do sudo hostname newname and restart network service - it obtains internal domain name from somewhere. I think this is not the way to do it.
The real problem is (I found in /var/log/mesos/mesos-slave) this message Checkpointing ACK for status update TASK_FAILED - it doesn't arise on virtualbox.

I've figure out how to setup mesos-[master | slave] and experimenting with ip and hostname settings. I'm trying to set public ip or public dns name... no progress so far.

[johnykov]

strange, I can successfully deploy this one:
https://github.com/mesosphere/marathon/blob/master/examples/PythonSimpleHTTPServer.json
but not this one: https://github.com/mesosphere/marathon/blob/master/examples/Play.json

[johnykov]

I think I found the guy There is insufficient memory for the Java Runtime Environment to continue. I should switch t1.micro to t2.medium...
and changing it, it brought me back to VPC security groups may not be used for a non-VPC launch problem, and after configuring that vpc, I see I need to provide subnet id (which I did) and visualization type where I stop cuz I don't see this param in vagrant aws plugin...

looks like m3.medium works