learnlinux / tuxlab-infra Goto Github PK

Similar to this one for Ubuntu:
https://github.com/gauthierc/DockerSpiceLxde

Because of the operation of RedRouter, we no longer need to do SSL key syncing across the SSHD Docker containers. We do, however, need to manage key syncing across the RedRouter proxy containers, such as to enable HTTPS/SSL via Let's Encrypt. Some key considerations:

1. This code resides in the tuxlab-ssh-proxy branch (it is really obvious in proxy.js)
2. Remember that Let's Encrypt keys expire after 90 days. So this key needs to be updated and propagated.
3. These keys cannot be generated separately on each host. They need to be generated once and synced.

WE need to make sure that docker containers (RedRouter, Swarm) run forever- restarting if they fail.

• Put Proxy and DNS on Registry
• Allow for Lab Images from Registry
• Use enclosejs to minify native includes (particularly for RedRouter)

The release of centos/atomic we use does not include the latest version of docker-py, which means that we get the following error:

ansible/ansible-modules-core#5515

As a temporary fix, I pull the new images using the shell module prior to running the docker_container module. These temporary fix blocks should be removed once the bug above is resolved. I have marked these blocks as follows:

#TODO https://github.com/ansible/ansible-modules-core/issues/5515

SSH tests, were they run, would now currently fail. The only reason for this is that tux_pass is not copied over to meteor. We just need to do this.

Task sometimes fails asking for user authentication; this task does not need to be run if pexpect is already installed on local machine, so the bug can be avoided by manually installing pexpect and commenting out the task for now.
https://github.com/learnlinux/tuxlab-infra/blob/master/roles/tls-ca/tasks/main.yml#L3

• Move RUN commands in the Dockerfile into a separate shell file
• Put the password write command at the top of the file (resolves issues @cemersoz is having)

For the most part, no free platform allows for running a Vagrant VM (whether VirtualBox, Libvirt, VMWare) inside. We will need to setup a dedicated Jenkins CI Server inside an AWS Instance

Need to update tuxlab-reverse-proxy to allow for HTTPS connections.

The beta version of the infrastructure makes some critical changes to the microservice architecture. We need to recreate test.yml to validate the state of the services TuxLab relies on:

• Docker Swarm / HTTPS (:4000)
• Docker Engine / HTTPS (:2375)
• ETCD / HTTPS (:2379)
• TuxLab Proxy (Docker Container on Hosts)
• Meteor (:80 => :443 and :443)

I would like the automate the following processes:

(On CoreOS Hosts)
https://coreos.com/os/docs/latest/configuring-date-and-timezone.html

(On CentOS Hosts use Chronyd)
https://www.certdepot.net/rhel7-set-ntp-service/

After revoking all guest permissions to make ETCD secure, I discovered that helixDNS no longer works. We need to find a way to pass ETCD authentication info into helixdns. I created an issue in the helixdns repo asking about this. For now, we should probably disable ETCD authentication.

The biggest problem right now is that the RedRouter container needs to be given access to the host docker process. I believe that it is possible to give it access via the binding at 172.17.0.1, but this needs to be tested. In addition, we need to be sure to block this access from all other containers so that LabVMs can't access services on the host:

• Ensure RedRouter is given access to Docker, ETCD
• Ensure LabVMs dont have access to any ports on the host (alpha.2)

There are then going to be some refactoring changes since we have moved to RedRouter:

• Delete the variables currently associated with the template in the config.yml file
• Needs to sync keys (#13) across RedRouter instances. (This task can be delayed until alpha.2)

We need to perform some updates on the standard Alpine LabVM:

• Update the Maintainer Field in the Dockerfile
• Change the default shell to Bash

The final SSH testing of proxy doesn't work, and needs to be fundamentally changed. The only way to really test the session is to create an interactive session, which requires "abusing" Ansible by adding a host (which is actually the LabVM) and trying to connect to it.

Create a Red Hat LabVM Container

The new version of Meteor has been released officially. However, I am not sure that it is installed as part of the default installer. In any case though, it seems we need to add at least the chown step from the temporary Meteor fix into the Ansible role for tuxlab-app so that the app will function properly:

learnlinux/tuxlab-app#130

Test it with just the chown step first, and add the other rm steps only if necessary.

Use ETCD TLS for Peer-to-Peer communications:

@trunkatedpig we need to use TLS for external communications:

https://coreos.com/etcd/docs/latest/security.html

https://github.com/learnlinux/tuxlab-infra/blob/master/tests.yml#L166 breaks... From this point, running ssh [email protected] from localhost and entering the password logged by a previous task should result in a successful connection if the infrastructure is set up properly.

We need to create a simple nodejs application which runs inside the proxy container which listens to ETCD records and stops the docker containers if needed.

https://docs.docker.com/engine/security/https/

Another concern we haven't thought a ton about is the idea of actual HDD storage space- if we are automatically pulling whatever images the instructors specify, this could be a problem. How are we going to handle running out of storage space?

Containers get created twice right now; just need to fix issues as they come up until vagrant provision builds cleanly

We would like to add the ability to connect to a remote host via SPICE. This requires modifying RedRouter to include the functionality of WebSockify in forwarding these requests.

http://www.spice-space.org/spice-html5.html

We need to re-implement the code which performs health checks on the Docker Hosts. Ultimately, it needs the following functionality;

• Checks RedRouter for WebSSH and SSH Use
• Checks RAM and Processor use of the Docker Host

With this data, it can make a scaling request to AWS to scale using SQS.

In order for this to work down the road with etcd authentication, we need to pass in the correct etcd credentials during Ansible configuration:

• tuxlab-app
  Currently the tuxlab-app ansible role creates a file called settings.env.json from a template. It only includes the etcd ip address, but ultimately needs to include the etcd authentication information (username, password) for the meteor role.
• RedRouter
  The RedRouter backend object takes this input as the etcd_conn_opts object, which is just a raw node-etcd auth object. Ideally, this is imported from a file.

When we switched to skyDNS, the way to create and dig(?) for DNS records changed. We need to modify our infrastructure tests to account for this.

Create a resolver which uses the etcd tuxlab session object instead of a separate redrouter etcd object.

etcd-io/etcd#1016

• Use Ansible Tower 3
• Verify Atomic AMI
• Create DNS Automatically

When SSHing into lab containers, we currently always do so as the root user. The code upon which labs are built is written such that we can only SSH as the root user, even if other users are created. We want to be able to create a new user with a unique password and SSH as that user.

• The banner you created will only display on regular SSH (and not WebSSH connections). I think this is good in the long run- as it identifies our server. However, we need to create a separate MOTD for users once they login. This resides in /etc/motd and needs to be edited from the alpine default.
• Can we rename from docker-sshd? We have made too many changes to call it the same thing.
• Remove key syncing (#13)
• Post to the Docker Hub