openebs-archive / elves Goto Github PK

The proposed changes to reduce CI time by making use of pre-pulled images uses a single tar bundle to hold all the test images. While this does help with improving CI time, a better approach would be to maintain the image tars separately and load them individually. Reasons :

• Currently the application specification YAML files use "latest" tag for some images, and other version-based tags for others. It is necessary to identify changes to the images using "version"-based tags using a separate logic on the test harness machine

• With current logic, even on changes to any one application/test image, the whole tar bundle will be re-applied

https://github.com/segmentio/stack
https://segment.com/blog/the-segment-aws-stack/

Refer #20

We need to include the logic to verify presence of versioned image tarballs before attempting to fetch/tar the test images. If the image for the specified version/tag is available, the image fetch/tar operations should be skipped.

A base box with all the dependencies needs to be created. Only check-out of the code should be dynamic. This should help in reducing the time taken to setup new development VM.

Vagrant: 1.9
Provider: Virtual Box

• Write scrap configuration to monitor kubernetes-cluster

references - https://github.com/gruntwork-io/terragrunt

Refer #19 , #20, #21, #22 for background on this issue/enhancement

The following need to be created :

• bash script to trigger the setup-images playbook when there is no ongoing CI run and place the results into a logfile
• cronjob to trigger the above bash script

longhorn make fails with

Step 1 : FROM longhorn:master
---> cc02572e4272
Step 2 : COPY . /go/src/github.com/openebs/longhorn/
---> 6cb2dba4761b
Removing intermediate container ee831656eeeb
Successfully built 6cb2dba4761b
frontend/gotgt/frontend.go:11:2: cannot find package "github.com/openebs/gotgt/pkg/config" in any of:
/go/src/github.com/openebs/longhorn/vendor/github.com/openebs/gotgt/pkg/config (vendor tree)
/usr/local/go/src/github.com/openebs/gotgt/pkg/config (from $GOROOT)
/go/src/github.com/openebs/gotgt/pkg/config (from $GOPATH)
frontend/gotgt/frontend.go:12:2: cannot find package "github.com/openebs/gotgt/pkg/port/iscsit" in any of:
/go/src/github.com/openebs/longhorn/vendor/github.com/openebs/gotgt/pkg/port/iscsit (vendor tree)
/usr/local/go/src/github.com/openebs/gotgt/pkg/port/iscsit (from $GOROOT)
/go/src/github.com/openebs/gotgt/pkg/port/iscsit (from $GOPATH)
frontend/gotgt/frontend.go:13:2: cannot find package "github.com/openebs/gotgt/pkg/scsi" in any of:
/go/src/github.com/openebs/longhorn/vendor/github.com/openebs/gotgt/pkg/scsi (vendor tree)
/usr/local/go/src/github.com/openebs/gotgt/pkg/scsi (from $GOROOT)
/go/src/github.com/openebs/gotgt/pkg/scsi (from $GOPATH)
frontend/gotgt/frontend.go:14:2: cannot find package "github.com/openebs/gotgt/pkg/scsi/backingstore" in any of:
/go/src/github.com/openebs/longhorn/vendor/github.com/openebs/gotgt/pkg/scsi/backingstore (vendor tree)
/usr/local/go/src/github.com/openebs/gotgt/pkg/scsi/backingstore (from $GOROOT)
/go/src/github.com/openebs/gotgt/pkg/scsi/backingstore (from $GOPATH)
FATA[0478] exit status 1
make: *** [ci] Error 1

OpenEBS VSM comprises of two types of containers: frontend/controller and backend/replica. backend/replica are associated with a minion node and store the data onto a disk attached to the minions. A VSM can have one or more backend/replica for providing data availability in case of node loss.

The yaml specs for launching the jiva controller and replica are at : https://github.com/openebs/elves/tree/master/k8s-dev/examples/openebs

Launching a VSM involves three steps:

• Jiva controller/frontend is launched as a deployment with replica=1. (demo-vsm-fe.yaml)
• A service is created to obtain a cluster IP for the jiva frontend. (demo-vsm-fe-service.yaml)
• Jiva replica are launched by passing the jiva frontend cluster ip. (demo-vsm-be.yaml)

With k8s pods, the IP address to the pod is assigned at runtime, so the yaml file doesnt pass the the IP address (param "listen"). Some of the issues hit when listen parameter is not specified:
(a) replica passes an empty address to controller for auto-registration. the controller can't reach the replica.
(b) controller provides an empty ip address for the iscsi discovery request, so the client can't connect to it.

• Setting up the vagrant boxes to setup
  - [x] Will be Single Node Kolla Ansible
• Wait = Configuring firewall things( firewalld -> ufw )
• libvirt on my bare host gives some errors -> http://dpaste.com/2YQK719.txt
• Need for setting up NFS? Why not just rely on /vagrant to give me folders when it only has a single node??? -> Doesn't exist /etc/exports
• Is it required to share folders with host when I am setting up a single node? Should I simply set up a multi-node?
• Since it is all containers+ansible is it okay to run on bare metal? the docs speak about etting it up via the vagrant
• WIP : Getting the kolla-build

• Fixed the issue with the container not getting pulled

• Explore how the following project can be useful from OpenEBS-CI perspective (Allows creation of kubernetes clusters on different cloud platforms)

  https://github.com/crosscloudci/cross-cloud/

Explore the way the iSCSI volumes get attached to application using the PV/PVC concept.