ansible / lightbulb

Lightbulb has been deprecated and replaced by Ansible Workshops

NOTICE

Lightbulb has been deprecated and replaced by Ansible Workshops https://ansible.github.io/workshops/

Ansible Lightbulb

The Ansible Lightbulb project is an effort to provide a content toolkit and educational reference for effectively communicating and teaching Ansible topics.

Lightbulb began life as the content that supported Ansible's training program before it joined the Red Hat family focused solely on Linux server automation.

This content is now taking on a new life as a multi-purpose toolkit for effectively demonstrating Ansible's capabilities or providing informal workshop training in various forms -- instructor-led, hands-on or self-paced.

Over time Lightbulb will be expanded to include advanced and developer topics in addition to expanding beyond linux server automation and into Windows and network automation.

To support these objectives, the project provides a lab provisioner tool for creating an environment to present and work with Lightbulb content.

What's Provided

The Ansible Lightbulb project has been designed to be used as a toolkit and best practices reference for Ansible presentations ranging from demos thru self-paced learning thru hands-on workshops. Here you will find:

• Examples
• Workshops
• Presentation Decks
• Guides
• Lab Provisioner
• Facilitator Documentation

Examples

The content in examples/ is the heart of what Lightbulb has to offer. They are complete Ansible playbooks that demonstrate the most fundamental features and most common use patterns.

These examples are an excellent educational reference for communicating how Ansible works in a clear, focused and consistent manner using recommended best practices.

This content is a great source for canned demos or something you can walk-thru to illustrate automating with Ansible to a group. Some of the examples serve as the solutions to the workshops.

Workshops

The content of workshops/ are a collection of Markdown documents and applicable resources for providing hands-on assignments for learning how to automate with Ansible. The workshops are set up as exercises to be done by the participants, and are most suitable for smaller audiences.

Instructor notes on the execution and solution to all workshops can be found in facilitator/solutions/.

Presentation Decks

The content of decks/ are collection of presentation decks using the reveal.js framework for delivering instructor-led or hands-on instruction.

The presentations can be viewed at ansible.github.io/lightbulb

Guides

The guides/ provide closely guided exercises with a lower barrier to entry. These are suitable for beginners or larger audiences. People can follow the guides on their own pace, and usually need very limited support is required during the execution of such labs.

Lab Provisioner

Lightbulb provides a lab provisioner utility for creating a personal lab environment for each student. Currently only Amazon Web Services (AWS) is supported in us-east-1 and us-west-1 with the foundation to support other regions in place.

The provisioner and the documentation how to use it can be found in tools/aws_lab_setup/.

Coming Soon. Vagrant support for self-paced learning is planned. Legacy support from the previous generation of Lightbulb remains, but is in need of an overhaul.

Facilitator Documentation

facilitator/ includes documentation on recommended ways Lightbulb content can be assembled and used for a wide range of purposes and scenarios.

If you are planning on using Lightbulb for some sort of informal training on automating with Ansible this documentation should be your next stop.

Requirements

True to its philosophy and The Ansible Way, Lightbulb has been developed so that using Lightbulb is as simple and low-overhead as possible. Requirements depend on the format and delivery of the Lightbulb content.

• Modern HTML5 Standard Compliant Web Browser
• A recent stable version of Python 2.7 and the latest stable version of the boto libraries.
• The latest stable versions of Ansible.
• A SSH client such as PuTTY or Mac OSX Terminal.
• An AWS account or local Vagrant setup.

Assumed Knowledge

For hands-on or self-paced training, students should have working knowledge of using SSH and command line shell (BASH). The ability to SSH from their personal laptop to a lab environment hosted in a public cloud can also be required based on the format and presentation of the context.

For demos and instructor-led exercises, conceptual understanding of linux system admin, DevOps and distributed application architecture is all that is required.

Reference

• Ansible Documentation
• Ansible Best Practices: The Essentials

License

Red Hat, the Shadowman logo, Ansible, and Ansible Tower are trademarks or registered trademarks of Red Hat, Inc. or its subsidiaries in the United States and other countries.

All other parts of Ansible Lightbulb are made available under the terms of the MIT License.

lightbulb's Issues

lab provisioner tags are not documented and seem excessive.

[This comment refers to the refactored lab provisioner being developed in #81.]

There seems to be an excessive use of tags that are not documented. Given the level of configuration the various lightbulb v2 labs are expected to see, I recommend we implement configuration options as feature switches (variables) so they can be shared as in files and versioned controlled unlike command line tags as implemented.

Fork influxdb role so we have ownership

Example of ignore_errors and failed_when

Check to see if postgres is installed, if so, do some things. It is not imperative that the actual command be successful.

Don't introduce Jinja Templates until the apache_centos role lesson.

Control should default to rhel7-tower not rhel7

lightbulb/aws_lab_setup/roles/manage_ec2_instances/defaults/main.yml

Should be:
ec2_lab_node_types:

• name: control
  type: rhel7-tower

Currently set to:
ec2_lab_node_types:

• name: control
  type: rhel7

Which uses a t2.micro instance

Replace web server example static and homepage templates with something more esthetically pleasing and on brand

The static home page in the "simple" examples and the templates in the "basic" and "role" examples of apache and nginx are rather pain and pretty ugly. They should be replaced with something that is more esthetically pleasing though simple and on brand. Something like this one: https://github.com/tima/8ball-service/blob/master/app/templates/8ball.j2

stop disabling selinux

a horrible practice to show customers, especially in a scripted demo

Split AMIs data out from AWS instance size and disk information

[This comment refers to the refactored lab provisioner being developed in #81.]

Since AMIs are only available in one region and we want to support different operating systems (RHEL and CentOS to start), AMIs meta data should be separate from the instance sizing and lab topology definition. The OS type can be static.

Which regions to support initially needs to be determined. Prior to this, lightbulb has only traditionally used us-east-1.

Create User Group tasks fails on web nodes with v2.2.1

Running the lab provisioner locally from my Mac with ansible 2.2.1, all my web nodes failed the Create User Group task in the user_accounts role. When I downgraded to 2.2.0 everything worked as expected.

TASK [user_accounts : Create User Group] ***************************************
fatal: [lightbulb-tima-node1]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'username' is undefined\n\nThe error appears to have been in '/Users/tima/projects/ansible-lightbulb/aws_lab_setup/roles/user_accounts/tasks/main.yml': line 1, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Create User Group\n  ^ here\n"}
fatal: [lightbulb-tima-node2]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'username' is undefined\n\nThe error appears to have been in '/Users/tima/projects/ansible-lightbulb/aws_lab_setup/roles/user_accounts/tasks/main.yml': line 1, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Create User Group\n  ^ here\n"}
fatal: [lightbulb-tima-node3]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'username' is undefined\n\nThe error appears to have been in '/Users/tima/projects/ansible-lightbulb/aws_lab_setup/roles/user_accounts/tasks/main.yml': line 1, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Create User Group\n  ^ here\n"}
changed: [lightbulb-tima-control]

It appears the username var is going out of scope or something. Lacking time I was unable to find how that var is getting set. username is in the users dictionary, but the task isn't referencing it that way.

Show how to use inventory ranges.

instructor_inventory.txt template does not build groups properly when using generic users ranges.

When generating labs such using user names such as student1, student2... student10... student 20, a groups such as student1 has the nodes for student10, student11 and so on in it.

Sendgrid fails to send mail

TASK [training_infra : Email Hosts file] *************************************** task path: /root/lightbulb/aws_lab_setup/roles/training_infra/tasks/main.yml:21 An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: string indices must be integers, not str failed: [localhost -> localhost] (item={u'username': u'thisdavejohnson', u'email': u'[email protected]'}) => {"failed": true, "item": {"email": "[email protected]", "username": "thisdavejohnson"}, "module_stderr": "Traceback (most recent call last):\n File \"/tmp/ansible_QZnZ7a/ansible_module_sendgrid.py\", line 254, in <module>\n main()\n File \"/tmp/ansible_QZnZ7a/ansible_module_sendgrid.py\", line 246, in main\n module.fail_json(msg=\"unable to send email through SendGrid API: %s\" % info['message'])\nTypeError: string indices must be integers, not str\n", "module_stdout": "", "msg": "MODULE FAILURE", "parsed": false}

Provisioner should assure/create vpc and subnet etc and not require they exist.

Presenter view gives a "403 Forbidden" HTTP error

The (presenter) notes plugin in reveal.js uses an HTML page. Apparently HTML pages are blocked from being served from the CDN we are using. Need to figure out the work around.

Create a single vagrant hosts file and ansible.cfg sample rather than liter the repo with copies.

Modify user static inventory path

The lessons/ folder is going away in v2. The inventory file should be moved to ~/hosts.

implement install_ansible option in lab provisioner

Some forms of lightbulb training have an installing ansible workshop however the provisioner (#81) installs ansible without the option of skipping that step.

The default would be yes (install)

install pip and install ansible with it

[This comment refers to the refactored lab provisioner being developed in #81.]

Some forms of hands on training use pip in their exercises to install python dependencies. The provisioner used to install pip, but was changed(?) to use yum exclusively.

Installing using PIP is the recommend/preferred means of installing ansible. Not using pip can be problematic because of the occasional lag new releases see getting into other channels that PyPi does not suffer from.

Implement default lab instance profile function

[This comment refers to the refactored lab provisioner being developed in #81.]

Related to #82, the idea here is to have a default instance profile to cut down on unnecessary repetitive definitions of lab machines.

Tower lab license

We need to document and/or automate licenses to run Tower in a lab environments.

Suggest a different practice around passwords

Currently, aws_lab_setup both stores the participants passwords in inventory, <username>-instances.txt as well as sends it to the participant via sendgrid email.

I would suggest removing that password from both the inventory and not sending it out via email.

A workshop instructor can simply tell participants the password on the day of the workshop/training.

Implement optional git version and url to lightbulb fetch task

For times when a facilitator needs to specify a certain branch or tag or a their own fork of lightbulb. Defaults to HEAD (param omit) and the standard ansible/lightbulb repo URL otherwise.

Have deck accept config file via reveal.js init

This optional configure should enable a facilitator/instructor to insert their name, title and company without having to modify HTML. They should optionally should also be able to insert their email, github and twitter contact info.

Since this configuration file is user-specific it should live in a standard location and be ignored by git for version control.

Format: JSON?

Clone telegraf/influxdb/grafana repos

We need to manage the binaries, let's put a mirror on S3 so we can be self-reliant.

apache and nginx basic and role examples should allow for the port to defined.

Add a variable for the port the web server (nginx or apache) in the playbook and configuration template to which port the web server will bind. This effects basic and role versions of the apache and nginx examples.

aws_lab_setup needs to pull v2 branch

Currently with the aws_lab_setup, it's cloning the master branch into student's control node. Need to change this to version: v2

lightbulb/facilitator v2

Introduction to Ansible Tower Hands-On Workshop

Believe the prerequisites content is for non 'hands-on' workshops. Might need to have similar wording to the Ansible Hands-On workshop. Currently, it looks like it's worded to be a lecture / non hands-on.

pin telegraf and influxdb versions

Let's be explicit of the versions of the packages we are installing.

git clone locally and sync files to student control hosts

Currently the provisioner does a git clone on each machine and removes files students do not need. That's fine unless you re-run the provisioner. A secondary git clone fails because of uncommitted changes. Since students aren't making changes to lightbulb and committing them, the provisioner should check out a copy locally and then synchronize file to each students control machine.

Address AWS inventory in Tower lab(s)

In v2 we have not addressed how to handle credentials for AWS dynamic inventory in Tower for students.

Document that student inventory file is on their controller as is part of bootstrapping process.

Apache to Nginx transition (need to stop service)

If running through the examples in order, students will error on nginx examples as the http port is busy. Need to include a playbook to stop httpd and or have the students perform that before moving onto nginx.

Cleanup inline CSS styles and abstract to ansible.css file

Need to use play naming.

Add mechanism to email out Tower license file to students if provided

stop disabling firewalld

a horrible practice to show customers, even for demos

apache_dirs undefined

I am getting below error while executing playbook for apache
{"failed": true, "msg": "'apache_dirs' is undefined"}
so I defined apache_dirs: /etc/httpd in webservers under group_vars folder but the issue remains.

Configure sshd and sudoers - error

Error --->>

TASK [common : Configure sshd and sudoers] *************************************
fatal: [TRAINING-LAB-rmarcandier-node1]: FAILED! => {"failed": true, "msg": "[{u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?Port', u'line': u\"Port {{ ssh_port | default('22') }}\", u'validate': u'sshd -t -f %s'}, {u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?PasswordAuthentication', u'line': u'PasswordAuthentication yes', u'validate': u'sshd -t -f %s'}, {u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?UseDNS', u'line': u'UseDNS no', u'validate': u'sshd -t -f %s'}, {u'dest': u'/etc/sudoers', u'regexp': u'^{{ username }}', u'line': u'{{ username }} ALL=(ALL) NOPASSWD: ALL', u'validate': u'visudo -cf %s'}, {u'dest': u'/etc/sudoers', u'regexp': u'^Defaults.*requiretty', u'line': u'Defaults    !requiretty', u'validate': u'visudo -cf %s'}]: 'username' is undefined"}
fatal: [TRAINING-LAB-rmarcandier-node3]: FAILED! => {"failed": true, "msg": "[{u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?Port', u'line': u\"Port {{ ssh_port | default('22') }}\", u'validate': u'sshd -t -f %s'}, {u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?PasswordAuthentication', u'line': u'PasswordAuthentication yes', u'validate': u'sshd -t -f %s'}, {u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?UseDNS', u'line': u'UseDNS no', u'validate': u'sshd -t -f %s'}, {u'dest': u'/etc/sudoers', u'regexp': u'^{{ username }}', u'line': u'{{ username }} ALL=(ALL) NOPASSWD: ALL', u'validate': u'visudo -cf %s'}, {u'dest': u'/etc/sudoers', u'regexp': u'^Defaults.*requiretty', u'line': u'Defaults    !requiretty', u'validate': u'visudo -cf %s'}]: 'username' is undefined"}
fatal: [TRAINING-LAB-rmarcandier-node2]: FAILED! => {"failed": true, "msg": "[{u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?Port', u'line': u\"Port {{ ssh_port | default('22') }}\", u'validate': u'sshd -t -f %s'}, {u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?PasswordAuthentication', u'line': u'PasswordAuthentication yes', u'validate': u'sshd -t -f %s'}, {u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?UseDNS', u'line': u'UseDNS no', u'validate': u'sshd -t -f %s'}, {u'dest': u'/etc/sudoers', u'regexp': u'^{{ username }}', u'line': u'{{ username }} ALL=(ALL) NOPASSWD: ALL', u'validate': u'visudo -cf %s'}, {u'dest': u'/etc/sudoers', u'regexp': u'^Defaults.*requiretty', u'line': u'Defaults    !requiretty', u'validate': u'visudo -cf %s'}]: 'username' is undefined"}
fatal: [TRAINING-LAB-rmarcandier-haproxy]: FAILED! => {"failed": true, "msg": "[{u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?Port', u'line': u\"Port {{ ssh_port | default('22') }}\", u'validate': u'sshd -t -f %s'}, {u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?PasswordAuthentication', u'line': u'PasswordAuthentication yes', u'validate': u'sshd -t -f %s'}, {u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?UseDNS', u'line': u'UseDNS no', u'validate': u'sshd -t -f %s'}, {u'dest': u'/etc/sudoers', u'regexp': u'^{{ username }}', u'line': u'{{ username }} ALL=(ALL) NOPASSWD: ALL', u'validate': u'visudo -cf %s'}, {u'dest': u'/etc/sudoers', u'regexp': u'^Defaults.*requiretty', u'line': u'Defaults    !requiretty', u'validate': u'visudo -cf %s'}]: 'username' is undefined"}
ok: [TRAINING-LAB-rmarcandier-control] => (item={u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?Port', u'line': u'Port 22', u'validate': u'sshd -t -f %s'})
ok: [TRAINING-LAB-rmarcandier-control] => (item={u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?PasswordAuthentication', u'line': u'PasswordAuthentication yes', u'validate': u'sshd -t -f %s'})
ok: [TRAINING-LAB-rmarcandier-control] => (item={u'dest': u'/etc/ssh/sshd_config', u'regexp': u'^#?UseDNS', u'line': u'UseDNS no', u'validate': u'sshd -t -f %s'})
ok: [TRAINING-LAB-rmarcandier-control] => (item={u'dest': u'/etc/sudoers', u'regexp': u'^rmarcandier', u'line': u'rmarcandier ALL=(ALL) NOPASSWD: ALL', u'validate': u'visudo -cf %s'})
ok: [TRAINING-LAB-rmarcandier-control] => (item={u'dest': u'/etc/sudoers', u'regexp': u'^Defaults.*requiretty', u'line': u'Defaults    !requiretty', u'validate': u'visudo -cf %s'})

PLAY RECAP *********************************************************************
TRAINING-LAB-rmarcandier-control : ok=18   changed=2    unreachable=0    failed=0   
TRAINING-LAB-rmarcandier-haproxy : ok=8    changed=0    unreachable=0    failed=1   
TRAINING-LAB-rmarcandier-node1 : ok=8    changed=0    unreachable=0    failed=1   
TRAINING-LAB-rmarcandier-node2 : ok=8    changed=0    unreachable=0    failed=1   
TRAINING-LAB-rmarcandier-node3 : ok=8    changed=0    unreachable=0    failed=1   
localhost                  : ok=9    changed=0    unreachable=0    failed=0  

---

Playbook

vim /home/rmarcand/Documents/lightbulb/aws_lab_setup/roles/common/tasks/main.yml

1 - name: Include Red Hat tasks
2   include: "{{ ansible_os_family }}.yml"
3   static: no
4   when: ansible_os_family == 'RedHat'
5 
6 - name: Include Ubuntu tasks
7   include: "{{ ansible_distribution }}.yml"
8   static: no
9   when: ansible_distribution == 'Ubuntu'
10 
11 - name: Configure sshd and sudoers
12   lineinfile:
13     dest: "{{ item.dest }}"
14     regexp: "{{ item.regexp }}"
15     line: "{{ item.line }}"
16     state: "{{ item.state | default('present') }}"
17     validate: "{{ item.validate | default(omit) }}"
18     backup: no
19   with_items: "{{ common_node_config_options }}"
20   notify: restart ssh
21   tags:
22     - ssh
23     - sudo
24     - common
25 
26 - meta: flush_handlers
27   tags:
28     - common

---

Config file

vim /home/rmarcand/Documents/lightbulb/aws_lab_setup/roles/common/defaults/main.yml

 1 common_node_config_options:
 2   - dest: /etc/ssh/sshd_config
 3     regexp: '^#?Port'
 4     line: Port {{ ssh_port | default('22') }}
 5     validate: sshd -t -f %s
 6 
 7   - dest: /etc/ssh/sshd_config
 8     regexp: '^#?PasswordAuthentication'
 9     line: 'PasswordAuthentication yes'
10     validate: sshd -t -f %s
11 
12   - dest: /etc/ssh/sshd_config
13     regexp: '^#?UseDNS'
14     line: 'UseDNS no'
15     validate: sshd -t -f %s
16 
17   - dest: /etc/sudoers
18     regexp: '^{{ username }}'
19     line: '{{ username }} ALL=(ALL) NOPASSWD: ALL'
20     validate: visudo -cf %s
21 
22   - dest: /etc/sudoers
23     regexp: '^Defaults.*requiretty'
24     line: 'Defaults    !requiretty'
25     validate: visudo -cf %s

---

users.yml

vim users.yml

[rmarcand@ThinkPad aws_lab_setup]$ cat users.yml 
users:
  - name: Renato Marcandier
    username: rmarcandier
    email: [email protected]
[rmarcand@ThinkPad aws_lab_setup]$

---

Ansible version

[rmarcand@ThinkPad aws_lab_setup]$ ansible --version
ansible 2.2.1.0
  config file = /home/rmarcand/Documents/lightbulb/aws_lab_setup/ansible.cfg
  configured module search path = Default w/o overrides
[rmarcand@ThinkPad aws_lab_setup]$ 

---

I notice that the error is on "username' is undefined" but you already have with_items on your playbook, because my idea was to use {{ item.username }} ...

Do you have an idea how can I fix this issue?

Regards
Renato Marcandier

Install wget on the control machines

It makes grabbing files (like the Tower installer) much easier so let's just load it up.

Remove haproxy and grafana group generation from user inventory file templates

Configure firewalld will all ports needed for lab exercise and workshops rather than stop it

Thinking ports 22, 80, 443, 8080, 8888. Currently firewalld (RHEL7) blocks everything but port 22. Others?

instructor_inventory should create groups by node type such as control, web etc.

In the event an instructor needs to perform an operation on student nodes such as all web nodes, they should be able to target this easily with a group.

vim-ansible-syntax installs under root and not user

vim-ansible-syntax role is run with become:yes that the user is root so that ansible_env.HOME /var/root and not /home/username.

Apache config path incorrect on Ubuntu

TASK: [apache | copy httpd.conf] **********************************************
failed: [10.42.0.8] => {"failed": true}
msg: Destination directory /etc/httpd does not exist

This affects the examples from apache_centos_ubuntu onward. The correct directory for Ubuntu is /etc/apache2.

Tower 3.0.1 fails to install on control machine

The control machine is set up with epel-release installed..

The tower installer writes its own epel repo file called epel-7 and so the ./setup.sh fails because of a duplicate EPEL repo.

Fundamentally I think this is a problem with the tower installer. But to resolve this I did a
yum erase epel-release

Implement install_ansible_tower option to install or not install tower in a lab environment

[This comment refers to the refactored lab provisioner being developed in #81.]

Sometimes you are teaching how to install Tower, sometimes you are doing a demo where you just need it up and running. The lab provisioner should support a configuration option for both.

Default would be: no (don't install)

Install python-pip on the control machines

It's just handy to have around and the ansible install workshop is setup to use pip to install Ansible.

Create IAM user(s) and assign to instance profile to facilitate dynamic inventory in Tower labs

Currently our Tower workshops do not incorporate dynamic inventory though it's a recommended and common usage pattern. We need to address this with our giving away full access to the AWS account being used by the AWS lab provisioner.

This issue is directly tied to #91.

Remove sendgrid module from aws_lab_setup once tested on v2.0.

document pip install passlib

Another small thing I ran into using the provisioner being developed in #81: We should make mention of needing passlib in the lab provisioner docs: http://docs.ansible.com/ansible/faq.html#how-do-i-generate-crypted-passwords-for-the-user-module