plunder-app / plunder Goto Github PK

With yaml support now part of plunder, the documentation needs updating to reflect how to make use of it. :)

This will require a re-architecture of the way things are parsed due to additional functionality being implemented.

The first function implemented in this plugin will be allowing the plunder server to push Docker images over ssh to remote nodes.

The VMware toolbox is a "replacement" for open-vm-tools that provides enhanced remote execution functionality.

Note the VMtoolbox will not start if open-vm-tools are running. A plunder automate vmware will take the same "parley" deployment files and use the toolbox agent to handle remote execution through the vCenter API.

Currently a yaml deployment file will generate an error.

The use of the --logfile flag has introduced a bug in the automate ssh

An excellent suggestion today, where plunder will register a new server and then power-it off. It will then use WoL to power the server on when being provisioned..

NOTE: some servers can take 8-10 minutes to power on (Looks at HPE)

d-i grub-installer/bootdev string /dev/sda or something there about needs adding to the boot config

Docker plugin can now automate the movement of images with the inclusion of internal piping of data.

Modify the plugin so it can take an array of files?

Add a /parlay api endpoint with base64 encoded response for remote execution.

Add a -o or —output flag to output example configuration for the config option, also move the pre-populating into the output of data here. (And make an option)

We're using a Mac address xx:xx:xx:xx etc.. it should be xx-xx-xx-xx

Currently we've builtin:

• preseed
• booty
• kickstart

pldrctl create deployment -a 172.16.1.15 -m 00:50:56:a5:29:2c -c booty

These have generally hardcoded configuration and can only be used once, this Issue is to track the capability to create a configuration based upon these as templates.

Something like:

Would configure a BOOTy configuration from the template with some settings and name it pullImage

pldrctl create config --template booty \
--sourceDevice /dev/sda
--destinationAddress http://192.168.0.12/images
--name pullImage

pldrctl create deployment -a 172.16.1.15 -m 00:50:56:a5:29:2c -c pullImage

pldrctl create config --template preseed \
--packages openssh cloud-utils
--noSwap
--name kubernetes

pldrctl create deployment -a 172.16.1.16 -m 00:50:56:a5:f9:24 -c kubernetes

If a user forgets to build a system that doesn't have sudo enabled then tasks will error with ->

WARN[0000] Command Task [Reset Kubernetes] on node [192.168.0.100] failed (execution will continute) 
DEBU[0000] Command Results ->
sudo: a password is required 

Add logic that looks for "sudo" "password" and report a fatal error.

https://github.com/bmc-toolbox/bmclib

This in theory should work quite happily on a raspberry Pi.

Test:

• go get -u github.com/plunder-app/plunder
• cd ~/go/src/github.com/plunder-app/plunder
• make build
• plunder config server -o yaml > config.yaml
• Edit config.yaml and enable DHCP etc. and ensure IP addresses are OK
• plunder server --config ./config.yaml --logLevel 5
• Boot something and check DHCP requests appear

Command written from memory so ymmv ...

Using https://github.com/hooklift/iso9660 to read an iso in place and map it through the httpHandler... might work?

If a run has already been completed then the first generation will reuse the previously existing certificates, the pki directory will need cleaning first.

Using https://github.com/AlecAivazis/survey/blob/master/examples/longmulti.go add the ability to select hosts, deployments and actions from an existing map.

Following the instructions here:

https://xenappblog.com/2018/automatically-install-vmware-esxi-6-7/

Work Required:

• pkg/utils/ipxe.go will need updating with the details for vSphere.
• pkg/server/templateUtils.go will need updating to parse a deployment type vSphere

Other steps will be required.

This issue is to track moving to utilising the docker socket directly (locally) to provide an API method (using the docker go pkg) to call the API. This will allow the pulling of images and transmitting the data over the same ssh session parlay has started.

This should allow issues to be fixed and a deployment resumed. With a simple --resume flag passed to automate ssh

Add an --endpoint to the plunder configuration.

Plunder won't match MAC addresses that are in different cases.

Currently plunder will continue to attempt to provision even with bad SSH keys which will result in a failed installation.

golang/go#7175 <- The functionality for the low-level packet requirements for DHCP is missing for windows.

The change required is here -> https://go-review.googlesource.com/c/net/+/76394/

Currently there is a hard-coded subnet, this should be modified (as should the lease timeout).

This will provide the capability to have online updates of the configuration without restarting the services (which can potentially leave servers half initialised.

The flags:

      --overrideAddress string    (Optional) Override address to automate against
      --overrideKeypath string    (Optional) Override path to a key
      --overrideUsername string   (optional) Override Username

require documentation.

At the moment you can add a bad path to the boot configurations.

Additions and modifications to the types mean that a server will ignore the defaultBoot until a correct deployment/config is applied.

https://github.com/denji/golang-tls
https://medium.com/@johnteckert/building-a-restful-api-with-go-part-1-9e234774b14d

^ Deets

A quick test of https://www.json2yaml.com shows that a general configuration of 520 lines is reduced to 360, which makes configurations significantly more easy to manage. So adding yaml as an option makes sense to implement (also yaml supports comments etc..)

Currently the kubeadm/{X} and load balancer actions are embedded code, the plan is to split the this code out into plugins that can be started at runtime.

Perhaps a ~./plugin directory when a plugin is invoked, it will start a socket ~./plugin/plugin_name.sock which can then be interacted with via HTTP calls. Plunder should be able to detect the plugin if it's running via a /live API call, and be able to kill or restart a plugin via an /exit API call.

Go Version

go version go1.13.4 linux/amd64

Go Env

GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/yiran/.cache/go-build"
GOENV="/home/yiran/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/yiran/go"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/yiran/go/src/github.com/plunder-app/plunder/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build454146169=/tmp/go-build -gno-record-gcc-switches"

Error messages

yiran@t480:~/go/src/github.com/plunder-app/plunder 
master ✔ $ make 
vet: invoking "go tool vet" directly is unsupported; use "go vet"
make: *** [Makefile:87：check] Error 1

Maybe we need to use go vet instead of go tool vet

A bug spotted by @errordeveloper was introduced by adding functionality to manage the internal state and provide on-line updates to developments through the /deployment api end point.

The server will always attempt to parse a byte[] even when no config is parsed, to fix this the updateConfiguration() should create an empty configuration if no json/yaml is passed.

Give a warning, but drop to globalconfig or the public ssh key.

Only the command action type generates any messages, log.Info() needs adding for additional types.

Something like:

	if conn, err := c.Connection.Dial("unix", "/var/run/docker.sock"); err != nil {
		return "", err
	}

Then use the go SDK to connect to the existing connection

The configuration structure needs re-organising to allow external interaction to create new deployments that point to different a kernel and initrd and make the API integration much more simple.