gluster / softserve

Flask web app for Gluster cluster over AWS cloud.

Softserve

Softserve is a web based service which enables users to self-provision virtual machines for a fixed duration. These virtual instances can be used to debug issues arising in various topics within the Gluster project. The provisioned VMs will get automatically removed after a fixed number of hours (at present this is set to 4 hrs)

The Gluster Infrastructure Project offers an instance of this service. To deploy this service outside of the Infrastructure, please follow the instruction set below.

Installation

• Before you start installation make sure you have a working installation of python2
• Create and activate virtualenv
• In your virtualenv run pip install -r dev-requirements.txt
• Now you are all ready with the installation and development

Goals

• Launch the VMs as per the need of the user.
• Tear down automatically after a specific number of hours(4 hours). This is a requirement so that the machine time is used judicially and tied down to a specific bug
• Allow only those users who are in the Gluster organization on Github to request VMs.
• Users should be able to upload their SSH public key and they will get access to the machines when it is created.
• Maximum allowance of 5 VMs at a time across all the user.
• Send the notification to the user half an hour before terminating the VM ( when meeting the deadline of the 4 hours)

Usage

• Run migrations and setup database flask db upgrade
• To run the softserve app on localhost run flask run
• Create application.cfg and add DEBUG=True to it so as to enable debug mode for your flask app
• By default this application reads and writes the data into sqlite but it can be overriden in your application.cfg

TODO

• Write down the test cases

Instance configuration details

• Operating system: CentOS 7(template with dependencies installed for glusterfs)
• RAM: 2GB
• CPU: 2 vCPUs
• System Disk: 40GB
• Network: 400Mb/s

softserve's Issues

Create a directory structure that looks like fstat

Let's split it into one folder called softserve with an __init__.py where everything is imported, a controller.py file with all the view functions, a lib.py with all the shared functions, especially the ones that create VMs.

Hours is unused

We have a field for the hours of renting, but this is unused at the moment. Shouldn't it be removed then ?

Unable to connect to vm

After creating vm on softserve , not able to ssh to instance
It shows the following

ssh: connect to host x.x.x.x port 22: Connection timed out

Abstract pyrax calls into functions in lib.py

We'll need to do two things at Rackspace - Create VMs and delete VMs. Let's abstract this out into two functions that will live in lib.py. In views.py, we'll call these functions.

Create dummy functions for VM creation and deletion

Add a new column deleted_at

This should be updated when a VM is deleted.

Report page for admins

We need a report page for the admins. The report page should at the least show the following information

• List of each request with name, start time, end time.
• Ability to filter loans by person and by date/time.

Build out the page first with dummy data. And we can figure out how to go from there. Admins should be a list that we give as the application config with github usernames.

Initialize the rackspace key only inside the task.

Do not initialize the rackspace api when starting the app. This limits being able to test the app without a valid rackspace key.

Project missing LICENSE file

Validate the SSH key before sending request off to the task queue.

Otherwise the task item will silently fail.

Create .travis.yml for running flake8

Create UI with Flask-Bootstrap

Health check end point

We want an end point that checks that the github API credentials work, the rackspace API connection works, and the celery queue is processing requests correctly.

So if you hit /health-check

It should tell us in page what works, what fails, and possible an error message.

Reuse the ssh key from github instead of asking user

So, the current form is asking the user for its ssh key. It would be better to reuse the one from the user on github, since this can be downloaded.

For example, this url give mine:
https://github.com/mscherer.keys

After logging in, redirect user directly to dashboard

Call it "Create Nodes" and "Delete Nodes"

"Create Lab" does not sound right.

Provide an option to install FreeBSD VMs

I'd like to debug a build issue on FreeBSD and it would be most helpful to have Softserve give me a VM for that.

After create_node call redirect to dashboard

Deleted VMs still appearing on the soft-serve dashboard.

If I click the delete button again, it says "Deleting softserve-ravi1.1 machine" but after that, it still appears on the dashboard. I'm not able to ssh into it though.

Create a centos 7 devel image

This image will be on top of the clean image. It will have all the dependencies enabled. The only missing pieces should be

• Networking. Let's not mess with ipv6 vs ipv4.
• Loopback device.

Add functionality for centos8 machine

Could we have the hardware detail in the README section?

I think it would be useful to have hardware detail in README section. Could we update README with these details?

Add travis integration

We should add some travis integration, at least to verify the syntax and maybe more.

machine name only has to be unique for currently running machines

I should be able to re-use the same name if the machine has been destroyed.

Allow only 1 VM per user

At this point, we should be enabling just regression runs. So one node at a time is perfectly good.

Add a favicon

Twitter bootstrap should have an ideal one.

Two view functions rendering same template

get_node_data and home. This should probably be unified into a create_node view.

Unable to instantiate softserve VM

A few weeks back, I had trouble installing gluster on the CentOS-7 VM template (#64) but now I am not able to instantiate any VM on softserve. After filling up the details and hitting 'CREATE', the VMs do not show up on my dashboard. Help is appreciated.

Track request state

We have no way to track a request state. If I open a request for X servers, there's no easy way to check the DB and find out whether this request succeeded or not. I propose that we have a column in the NodeRequest model that goes from Requested -> Processing -> Delivered. This way if we have a bug that causes celery failures, we can retrigger them after we deploy a fix.

Clean up requirements.txt

It should only have direct dependencies.

Increase the instance RAM to 2GB

The popped machines are using 1GB General Purpose flavor. Need to upgrade to 2GB General Purpose flavor.

Add an option for requesting the extension of machine duration

It will be good if we have a option like request for extension of the VM duration, and the option will be automatically activated after 3 hours of usage of VM. If somebody is using the VM after 3 hours and they feel like they need it for 2 more hours they will request to extend the duration by 1 more hour. It will save the time of engineering since if a machine is expired, one has to configure the machine and all other stuff from the beginning.

shutdown check does not work

Usage: flask [OPTIONS] COMMAND [ARGS]...

Error: No such command "shutdown_check".

This means that stale machines are still online without ever being deleted.

Migrating from Boto to Boto3

Boto3 is more pythonic and also is consistent with any changes made to the AWS services. Plus it has native support for python version 2.7+ and 3.4+.

Create CLI command to delete VMs when time is up

Unable to setup a CentOS-7 VM instantiated from softserve.

Followed the instructions 'to the letter' as mentioned at https://github.com/gluster/softserve/wiki/Running-Regressions-on-loaned-Softserve-instances. This was done on a Fedora 33 machine.

When I run the playbook, I get the following error:

TASK [Make sure firewalld allows traffic from the machine to itself] ***********************************************************************************************************************************************************************
fatal: [builder500.cloud.gluster.org]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: 'ansible_eth0' is undefined\n\nThe error appears to be in '/home/ravi/workspace/softserve/playbooks/regressions-final.yml': line 74, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n  - name: Make sure firewalld allows traffic from the machine to itself\n    ^ here\n"}

The only change I have made in the cloned softserve repo is as per the notes.

[ravi@linuxpad playbooks]$ git diff
diff --git a/playbooks/inventory b/playbooks/inventory
index 39ed5eb..b025102 100644
--- a/playbooks/inventory
+++ b/playbooks/inventory
@@ -1,5 +1,5 @@
 [softserve]
-<hostname> ansible_host=<ip_address>
+builder500.cloud.gluster.org ansible_host=18.221.142.198

Extending reporting capabilities

#32 lists the basic minimum set of reports based on usage. I'd like to propose that we also add to backlog a set of extended reports which are focused on consumption and thus billing/metering. These could be used to add extra information for:

• peak usage hours

• off-peak usage hours

• user who has used Softserve most in one week/fortnight/month

• lifetime of each instance being requested (highest/lowest/average)

• instance type (instance related meta-data)

I am not sure if the corresponding host machine data can also be collected to check how the hosting infrastructure behaves at peak etc

Create a README.md

For now this can include the goals of the project.

Add functionality for centos8 machine

Add the ability to select multiple images

Ideally, I want us to have a clean Centos 7 image and a Centos 7 developer image with all the dependencies installed and configured. We should just leave the network configuration and the loop device configuration to the end.

Do not have catch all exceptions

Do not have Catch all exceptions in the code.

The type that goes like this

try:
    do_something()
except:
    do_something_else()

Catch specific exceptions and log them. It's easier to debug the application this way.