GithubHelp home page GithubHelp logo

bravetools / bravetools Goto Github PK

View Code? Open in Web Editor NEW
149.0 9.0 14.0 25.48 MB

A tool to build, deploy, and release any environment using System Containers.

Home Page: https://bravetools.github.io/bravetools/

License: Apache License 2.0

Makefile 0.54% Go 95.66% Shell 3.13% Python 0.68%
devops containers lxc-containers lxc-lxd

bravetools's Introduction

Gitter Go Report Card

Bravetools

Bravetools provides a simple and consistent interface to declare, build, and deploy system containers.

Features

How it works

Configuring image environments

The steps to build and deploy an image are configured using a Bravefile.
View sample Bravefiles on GitHub to explore common use cases.

template

Build images

Build new images or layer existing ones using brave build
Resulting image can be stored locally or remotely

Deploy images

System containers can be deployed to a variety of environments with brave deploy, including local development machines, test environments, and remote production systems.

Image management

Manage images with multiple versions and architectures

Deploy multi-unit systems

Systems with multiple units can be declared in a brave compose file.
Build and deploy the system defined in a compose file with the brave compose command.

template

Installation

Prerequisites:

  • Mac/Windows: Multipass
  • Linux:
    • LXD
    • Ensure your user belongs to the lxd group: sudo usermod --append --groups lxd $USER
    • You may also need zfsutils: sudo apt install zfsutils-linux
  1. Download the latest stable release for your host platform and add it to your $PATH.

  2. Run brave init to get started.

Installing from source

Linux/MacOS

git clone https://github.com/bravetools/bravetools
cd bravetools
make [ubuntu]/[darwin]

Windows

git clone https://github.com/bravetools/bravetools
cd bravetools
go build -ldflags="-s -w" -o brave.exe -trimpath main.go

Getting Started

Run brave init to start if you haven't yet.

brave init
...

Create an example Bravefile

brave template

This creates a file named "Bravefile" in the current working directory. The contents should look like this:

image: example-image/v1.0

base:
  image: alpine/3.16

packages:
  manager: apk
  system:
    - curl

run: 
  - command: echo
    args:
      - hello world

copy:
  - source: ./Bravefile
    target: /root/

service:
  name: example-container
  ports:
    - 8888:8888
  resources:
    ram: 2GB
    cpu: 2
    disk: 10GB

Create an image using the Bravefile

brave build
...

Check images

brave images

IMAGE           VERSION ARCH    CREATED         SIZE    HASH
example-image   v1.0    x86_64  just now        4MB     ef28b49bf36f0b4b9cbad89ff67ef0ee

Deploy image as container

brave deploy

Check running units

brave units

NAME                    STATUS  IPV4            MOUNTS  PORTS     
example-container       Running 10.148.59.45            8888:8888

Add a remote

brave remote add example-remote https://20.0.0.20:8443 --password [PASSWORD]

Certificate fingerprint:  ...

Deploy to remote

brave deploy --name example-remote:example-container
brave units

NAME                                    STATUS  IPV4            MOUNTS  PORTS
example-remote:example-container        Running 20.0.0.7                8888:8888

example-container                       Running 10.148.59.45            8888:8888

Command Reference

A complete System Container management platform

Usage:
  brave [command]

Available Commands:
  base        Pull a base image from LXD Image Server or public Github Bravefile
  build       Build an image from a Bravefile
  compose     Compose a system from a set of images
  configure   Configure local host parameters
  deploy      Deploy Unit from image
  help        Help about any command
  images      List images
  import      Import LXD image tarballs into local Bravetools image repository
  info        Display workspace information
  init        Create a new Bravetools host
  mount       Mount a directory to a Unit
  publish     Publish deployed Units as images
  remote      Manage remotes
  remove      Remove Units or Images
  start       Start Units
  stop        Stop Units
  template    Generate a template Bravefile
  umount      Unmount <disk> from UNIT
  units       List Units
  version     Show current bravetools version

Flags:
  -h, --help   help for brave

Use "brave [command] --help" for more information about a command.

bravetools's People

Contributors

adrozdovbering avatar beringresearch avatar deknos avatar dependabot[bot] avatar idroz avatar jaybusch avatar szubie avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bravetools's Issues

brave units doesn't list multiple forwarded ports - only one port is shown

Describe the bug
Bravetools is able to forward multiple ports to host, however brave units will only show one port in its output

To Reproduce

brave base alpine/edge/amd64
cat >Bravefile <<EOL
service:
  image: brave-base-alpine-edge-1.0
  name: brave-base-alpine-edge
  version: 1.0
  ip: 10.0.0.10
  ports:
    - 8888:8888
    - 7777:7777
  resources:
    ram: 2GB
    cpu: 1
EOL

Deploy unit brave deploy

Expected behaviour
All forwarded ports are listed by brave units.

Environment (please complete the following information):

  • Your operating system name and version: Mac OS Mojave
  • Bravetools version: 1.55-dev
  • LXD version: 4.6

Bravetools image naming schema

At the moment Bravetools image definitions span several fields, and may possibly expand in future as bravetools moves towards multiple remotes. These fields are generally just concatenated together internally anyway to uniquely identify an image. It would be nice to allow the users to specify the image they want using just one field.

The lxc tool and LXD public image store already has a schema which we may want to emulate to better integrate with the existing ecosystem. It is also much briefer, which is nice. Here's the schema:

[remote:]name[/version][/arch][/type]

The public LXD image store hierarchical directories can be browsed here. These public images are associated with the "images:" remote in the lxc tool.

Bravefile Service section required at build time

Currently the Service section of the Bravefile is required during build since it defines the name and version of the image to be built.
It's also necessary to provide the name of the service container that will be deployed from the image at build time.

To better separate these steps, it would be better to define the image (name+version) that's being built within the build section of the Bravefile. This would make it possible to have a Bravefile without a Service section, and clearly splits the build and deploy stages.

To avoid too much unnecessary repetition of fields and maintain backward compatibility perhaps we can read the Image name from Service section if it is not provided in build section.

Support for a list of ports in brave deploy --port

Currently, brave deploy --port supports only a single port forwarding option.

Enable bravetools to forward multiple ports from command line e.g:
brave deploy brave-base-ubuntu-bionic-1.0 --name test --port 8501:8501 --port 8888:8888

brave compose subcommands

Currently the brave compose command does a build and deploy of a system. This is convenient in many places, but additional flexibility would be enabled by enabling separate subcommands that do just one of these steps.

Suggested subcommands for this would be:

  • brave compose build
  • brave compose deploy

This is more flexibile as it allows using the tool to execute just one step at a time.

Handle Image Alias not found error

Sometimes deleting a Brave unit results in an Image alias not found error. The error is harmless and does not actually prevent removal of an image. It should be tracked down and handled appropriately to reduce confusion.

Data corruption - brave build deletes LXD images from wrong storage pool !

Describe the bug
It looks like a really serious and nasty bug - "brave build" removes (deletes!) images stored in the LXD storage pool NOT managed by Brave. If you do not have backup of the images, they are gone...

To Reproduce

  1. I have an already configured and working LXD environment, with its own storage pool - just one (named: default, on ZFS)
  2. installed bravetools (the latest: release-1.55)
  3. performed brave init with the defaults
  4. so, now my LXD has 2 storage pools:
lxc storage ls
+----------------------+-------------+--------+---------------------------------------------------------+---------+
|         NAME         | DESCRIPTION | DRIVER |                         SOURCE                          | USED BY |
+----------------------+-------------+--------+---------------------------------------------------------+---------+
| brave-20210314184522 |             | zfs    | /var/snap/lxd/common/lxd/disks/brave-20210314184522.img | 1       |
+----------------------+-------------+--------+---------------------------------------------------------+---------+
| default              |             | zfs    | rpool/lxd                                               | 8       |
+----------------------+-------------+--------+---------------------------------------------------------+---------+
  1. on ZFS level everything seems to be fine, zpool list:
NAME                   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
...
brave-20210314184522     9G  7,64M  8,99G        -         -     0%     0%  1.00x    ONLINE  -
...
rpool                  920G   236G   684G        -         -     2%    25%  1.00x    ONLINE  -
  1. zfs list (the "default" LXD storage pool is on rpool/LXD - this is Ubuntu 20.10 with root on ZFS):
NAME                                                                                                       USED  AVAIL     REFER  MOUNTPOINT
bpool                                                                                                      575M  1,19G       96K  /boot
bpool/BOOT                                                                                                 573M  1,19G       96K  none
bpool/BOOT/ubuntu_n9nlq9                                                                                   573M  1,19G      251M  /boot
brave-20210314184522                                                                                      7,50M  8,71G       24K  none
brave-20210314184522/containers                                                                             24K  8,71G       24K  none
brave-20210314184522/custom                                                                                 24K  8,71G       24K  none
brave-20210314184522/deleted                                                                               120K  8,71G       24K  none
brave-20210314184522/deleted/containers                                                                     24K  8,71G       24K  none
brave-20210314184522/deleted/custom                                                                         24K  8,71G       24K  none
brave-20210314184522/deleted/images                                                                         24K  8,71G       24K  none
brave-20210314184522/deleted/virtual-machines                                                               24K  8,71G       24K  none
brave-20210314184522/images                                                                               6,88M  8,71G       24K  none
brave-20210314184522/images/22da9387f1687a576a78d36df79db0179e1c64dad4ba3c91618f94aabde27e5d              6,86M  8,71G     6,86M  /var/snap/lxd/common/lxd/storage-pools/brave-20210314184522/images/22da9387f1687a576a78d36df79db0179e1c64dad4ba3c91618f94aabde27e5d
brave-20210314184522/virtual-machines                                                                       24K  8,71G       24K  none
...
...
rpool/lxd                                                                                                 32,8G   650G       96K  none
rpool/lxd/containers                                                                                      1,16G   650G       96K  none
rpool/lxd/containers/c1                                                                                   37,0M   650G     3,02G  /var/snap/lxd/common/lxd/storage-pools/default/containers/c1
rpool/lxd/containers/netdata-test                                                                          162M   650G     2,08G  /var/snap/lxd/common/lxd/storage-pools/default/containers/netdata-test
rpool/lxd/containers/pytorch                                                                               384M   650G     20,6G  /var/snap/lxd/common/lxd/storage-pools/default/containers/pytorch
rpool/lxd/containers/u2004-2                                                                              38,2M   650G     3,02G  /var/snap/lxd/common/lxd/storage-pools/default/containers/u2004-2
rpool/lxd/containers/u2004base                                                                             243M   650G     3,03G  /var/snap/lxd/common/lxd/storage-pools/default/containers/u2004base
rpool/lxd/containers/u2010base                                                                             322M   650G     2,24G  /var/snap/lxd/common/lxd/storage-pools/default/containers/u2010base
rpool/lxd/custom                                                                                            96K   650G       96K  none
rpool/lxd/deleted                                                                                         28,6G   650G       96K  none
rpool/lxd/deleted/containers                                                                                96K   650G       96K  none
rpool/lxd/deleted/custom                                                                                    96K   650G       96K  none
rpool/lxd/deleted/images                                                                                  28,6G   650G       96K  none
rpool/lxd/deleted/images/08065ab9c22f02216a1f2850d8236f8aae9838cdd496e2f0f40a265123a04d68                 2,01G   650G     2,01G  /var/snap/lxd/common/lxd/storage-pools/default/images/08065ab9c22f02216a1f2850d8236f8aae9838cdd496e2f0f40a265123a04d68
rpool/lxd/deleted/images/19213e6ab97e101d3ba3035315fc2dff67d2f160467f3c45d4f94c497ac0ea28                 3,02G   650G     3,02G  /var/snap/lxd/common/lxd/storage-pools/default/images/19213e6ab97e101d3ba3035315fc2dff67d2f160467f3c45d4f94c497ac0ea28
rpool/lxd/deleted/images/3d3c2e10034ed9a817cfd9da6cf011506b481819ddc5bc05930e8cebf519968e                 3,18G   650G     3,18G  /var/snap/lxd/common/lxd/storage-pools/default/images/3d3c2e10034ed9a817cfd9da6cf011506b481819ddc5bc05930e8cebf519968e
rpool/lxd/deleted/images/79991761556f650e1b4856644e43df47f81a8ccde02f8220a210c0e064b83528                 20,4G   650G     20,4G  /var/snap/lxd/common/lxd/storage-pools/default/images/79991761556f650e1b4856644e43df47f81a8ccde02f8220a210c0e064b83528
rpool/lxd/deleted/virtual-machines                                                                          96K   650G       96K  none
rpool/lxd/images                                                                                          3,02G   650G       96K  none
rpool/lxd/images/cab10c166185a82f3582eb0174fb8ee31df7962bd86d6c9120168246908570c1                         3,02G   650G     3,02G  /var/snap/lxd/common/lxd/storage-pools/default/images/cab10c166185a82f3582eb0174fb8ee31df7962bd86d6c9120168246908570c1
rpool/lxd/virtual-machines                                                                                  96K   650G       96K  none
  1. created a new image and published it:
lxc image ls
+------------------+--------------+--------+------------------------------------+--------------+-----------+-----------+------------------------------+
|      ALIAS       | FINGERPRINT  | PUBLIC |            DESCRIPTION             | ARCHITECTURE |   TYPE    |   SIZE    |         UPLOAD DATE          |
+------------------+--------------+--------+------------------------------------+--------------+-----------+-----------+------------------------------+
| ubuntu-2004-base | 6a0498845280 | no     | Ubuntu 20.04 LTS server (20210201) | x86_64       | CONTAINER | 2532.26MB | Mar 14, 2021 at 6:23pm (UTC) |
+------------------+--------------+--------+------------------------------------+--------------+-----------+-----------+------------------------------+
  1. created a simple Bravefile:
base:
  image: alpine/edge/amd64
  location: public
packages:
  manager: apk
  system:
    - python3-dev
    - python3
service:
  name: alpine-edge-python3
  image: alpine-edge-python3-1.0
  version: "1.0"
  resources:
    ram: "4GB"
    cpu: 2
  1. after brave build, the lxc image ls gives this:
+-------+--------------+--------+------------------------------------+--------------+-----------+--------+------------------------------+
| ALIAS | FINGERPRINT  | PUBLIC |            DESCRIPTION             | ARCHITECTURE |   TYPE    |  SIZE  |         UPLOAD DATE          |
+-------+--------------+--------+------------------------------------+--------------+-----------+--------+------------------------------+
|       | 22da9387f168 | no     | Alpine edge amd64 (20210314_13:00) | x86_64       | CONTAINER | 4.38MB | Mar 14, 2021 at 6:24pm (UTC) |
+-------+--------------+--------+------------------------------------+--------------+-----------+--------+------------------------------+

I.e. all the images in the non-bravetools managed pool are gone ! bravetools seem to overwrite the existing LXD images in the 'default' storage pool, which is NOT managed by bravetools...

  1. things are even more "fun" - doing brave deploy now, makes the result of lxc image ls empty:
lxc image ls
+-------+-------------+--------+-------------+--------------+------+------+-------------+
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCHITECTURE | TYPE | SIZE | UPLOAD DATE |
+-------+-------------+--------+-------------+--------------+------+------+-------------+
  1. brave units and lxc ls show the bravetools-managed container working fine. 'lxc exec alpine-edge-python3 -- sh' works fine too...
  2. I have not tested bravetools standalone or with more than 1 non-bravetools storagepool.

Expected behavior
Bravetools - as promised in the docs - should not "touch" any other LXD components and operate only on its own profiles, storage, network, etc.

Environment (please complete the following information):

  • Ubuntu 20.10 Desktop on x86, ZFS on root
  • release-1.55
  • 4.12 (snap)

Mounting multiple volumes to unit results in brave units displaying only the last mount

Describe the bug
When mounting multiple volumes to a unit, brave units displays only the volume that was mounted last

To Reproduce

mkdir testA
touch testA/test.file

mkdir testB
touch testB/test.file

brave base alpine/edge/amd64
brave deploy brave-base-alpine-edge-1.0 --name alpine
brave mount $PWD/testA alpine:/root/testA
brave mount $PWD/testB alpine:/root/testB
brave units

The output will be testB disk information.

Expected behavior
Both testA and testB should be displayed

Environment (please complete the following information):

  • Your operating system name and version: Ubuntu 18.04
  • Bravetools version: 1.54
  • LXD version: 4.6

brave import does not support absolute paths

Describe the bug
brave import PATH_TO_IMAGE produces an import error

Expected behavior
bravetools should be able to import images from arbitrary locations

Environment (please complete the following information):

  • Your operating system name and version: Mac OS Mojave
  • Bravetools version: 1.55
  • LXD version: 4.3

brave deploy from command line results in a network error

failed to delete unit: cannot delete unit test due to mounted disks. Umount them and try again
2022/09/14 16:30:32 failed to attach network: Network not found
Ignats-MBP:bravetools (master) $ brave units
NAME      	STATUS 	IPV4	VOLUMES 	PORTS
local:test	Stopped	    	root:->/

To reproduce:

  1. Fresh install of Bravetools with Multipass backend. Local remote.
  2. brave base alpine/edge
  3. brave deploy brave-base-alpine-edge-1.0 --name test

In bravetools there's a check that prevents a unit being deleted if there's storage attached to it. Need to address as VOLUMES at the moment is reserved for mounted volumes rather than ZFS storage volumes.

Also for some reason local remove network is not fetched correctly when it isn't specified in the CLI. Default network name should be $USERbr0

Show image architecture in brave images

With remote builds enabled, it is possible to build images with a different architecture to the host system.

It would be good to be able to show image architecture and keep track of it in brave tools.

Maybe we could implement an image naming schema e.g. image_name-version-arch

brave deploy fails silently if attempting to forward a port that's already in use on the host

Describe the bug
'brave deploy UNIT --port 8888:8888 --name UNIT' will fail if the port 8888 on the host is already taken up by another process. Result

To Reproduce

  1. Run Jupyter Notebook: jupyter notebook
  2. In another window, build mlbase-1.0 image brave base github.com/beringresearch/bravefiles/ubuntu/ubuntu-bionic-mlbase
  3. Deploy mlbase-1.0 image: `brave deploy mlbase-1.0 --name mlbase --port 8888:8888'

The output is:

Importing mlbase-1.0.tar.gz
Unit launched:  mlbase
Service started:  mlbase
2020/09/17 10:36:30 <nil>

`

Expected behavior
An error should be raised before image begins to be imported to alert the user that a port is in use.

Environment (please complete the following information):

  • Your operating system name and version: Ubuntu 18.04
  • Bravetools version: 1.53
  • LXD version: 4.4

Move unit table creation to brave init

unit database table is created when the user initialises their first container. Empty Database should be created with brave init, otherwise, if there is an error in brave deploy, the database is not initialised properly and bravetools cannot operate as intended.

brave build fails on multipass client if image size exceeds ZFS storage pool size

Describe the bug
brave build fails on multipass client if image size exceeds ZFS storage pool size

To Reproduce
On default multipass settings run:

brave base github.com/beringresearch/bravefiles/ubuntu/ubuntu-bionic-mlbase

The image completes a build, but export to host fails

Expected behaviour
Image should complete the build succesfully

Environment (please complete the following information):

  • Your operating system name and version: Mac OS Mojave
  • Bravetools version: 1.53
  • LXD version: 4.6

Brave init

Hi guys!
Thanks for sharing this spectacular tool. I hope that it can grow in the future.

I follow step by step the Quick Tour in the Readme, but fails because it never mention the brave init command.

Once I ran it, the Quick tour works like a charm.
Another thing is that we (users) don't know what brave init does and why we need to run this first. Or how to remove that brave init does in the future, etc. Also I checked in the https://bravetools.github.io/ docs and you only mention it one time, explaining only this: Create a new Bravetools host.
Can you add more info about init?

BTW, I think that the Readme is a little messy, because the Quick Tour is in the first place, then the Installation part and finally the Command Reference.
If you order it diferrently like:

  • Installation
  • Command Reference
  • Quick Tour

Will be easy to follow the guides.

Thanks!!

EDIT: Even in the https://bravetools.github.io/bravetools/intro/quickstart/ you never mention the brave init.

Brave base builds a new image

Instead of taking a remote image and importing it into bravetools untouched, brave base spins up a container based on that remote image and creates a new image from that container.

Not only is this unnecessary/slower, it means that the fingerprint will no longer match the remote LXD image as it is actually a snapshot of a different state.

A side-effect of this is that running brave base multiple times results in images with different fingerprints each time.

It would be better to copy the remote image into bravetools store unchanged.

Trying to get a CentOS Brave container to run on Kubuntu 20.10, but its throwing an issue?

Hello,

So I was introduced to Brave by the reddit user /u/bering_team in this thread: https://www.reddit.com/r/LXD/comments/l7lpvi/i_was_suggested_to_check_out_lxd_since_i_used_a/gmk84cs/

So far he has helped me in installing and creating a Bravefile, however when I try to deploy I get an error

So if I run brave deploy I get:

si@linux:~/Documents/Brave-LXD/CentOS-7$ brave deploy
2021/02/10 00:59:58 Unit centos already exists on host

If I run brave units I get this:

si@linux:~/Documents/Brave-LXD/CentOS-7$ brave units
NAME    STATUS  IPV4    VOLUMES PORTS 
centos  Stopped  

So I run brave start centos and I get this:

si@linux:~/Documents/Brave-LXD/CentOS-7$ brave start centos
Starting unit: centos2021/02/10 01:04:42 Failed to start unit: Failed to run: /snap/lxd/current/bin/lxd forkstart centos /var/snap/lxd/common/lxd/containers /var/snap/lxd/common/lxd/logs/centos/lxc.conf: 

And finally I get this if I run lxc info --show-log centos:

si@linux:~/Documents/Brave-LXD/CentOS-7$ lxc info --show-log centos
Name: centos
Location: none
Remote: unix://
Architecture: x86_64
Created: 2021/02/09 03:41 UTC
Status: Stopped
Type: container
Profiles: brave

Log:

lxc centos 20210210060441.580 WARN     cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1129 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc.monitor.centos"
lxc centos 20210210060441.723 WARN     cgfsng - cgroups/cgfsng.c:mkdir_eexist_on_last:1129 - File exists - Failed to create directory "/sys/fs/cgroup/cpuset//lxc.payload.centos"
lxc centos 20210210060441.138 WARN     cgfsng - cgroups/cgfsng.c:fchowmodat:1550 - No such file or directory - Failed to fchownat(17, memory.oom.group, 1000000000, 0, AT_EMPTY_PATH | AT_SYMLINK_NOFOLLOW )
lxc centos 20210210060441.745 ERROR    conf - conf.c:run_buffer:314 - Script exited with status 1
lxc centos 20210210060441.745 ERROR    conf - conf.c:lxc_setup:3388 - Failed to run mount hooks
lxc centos 20210210060441.745 ERROR    start - start.c:do_start:1218 - Failed to setup container "centos"
lxc centos 20210210060441.746 ERROR    sync - sync.c:__sync_wait:36 - An error occurred in another process (expected sequence number 5)
lxc centos 20210210060441.758 WARN     network - network.c:lxc_delete_network_priv:3185 - Failed to rename interface with index 0 from "eth0" to its initial name "vethf0aa3a51"
lxc centos 20210210060441.759 ERROR    lxccontainer - lxccontainer.c:wait_on_daemonized_start:860 - Received container state "ABORTING" instead of "RUNNING"
lxc centos 20210210060441.759 ERROR    start - start.c:__lxc_start:1999 - Failed to spawn container "centos"
lxc centos 20210210060441.759 WARN     start - start.c:lxc_abort:1013 - No such process - Failed to send SIGKILL via pidfd 30 for process 3982
lxc centos 20210210060442.400 WARN     cgfsng - cgroups/cgfsng.c:cgfsng_monitor_destroy:1086 - Success - Failed to initialize cpuset /sys/fs/cgroup/cpuset//lxc.pivot/lxc.pivot
lxc 20210210060442.408 WARN     commands - commands.c:lxc_cmd_rsp_recv:126 - Connection reset by peer - Failed to receive response for command "get_state"

It shows that its failing to load the container? Anyone have any ideas?

A bit more info about my system,

OS: Kubuntu 20.10
LXD version: 4.11
Brave version: 1.55 (the precompiled version found on the github)

Hopefully this is sufficient info to help me.

Remote builds

Is it possible to use a Bravefile to specify that an image should be built on a remote LXD instance?

Bravetools multiple remotes

For the moment bravetools can only build/deploy to a single LXD instance, the local one that was set up during brave init. It would be very useful to be able to build images locally and deploy them into production using bravetools.

Bravetools recently switched remote backend to make preperations for this change. #125

To allow users to specify remotes during build/deploy time it may be good to add new fields to the bravefile. Alternatively, there is a suggestion here #140 to bake the remote info into the Bravefile "Image" field.

Adding new remotes

There's also the question of how to allow users to add new remotes. For now, the user can manually add the required file to the ~/.bravetools/remotes directory and the corresponding server cert with matching filename to the ~/.bravetools/servercerts dir. They would then have to manually add their client keys to the remote LXD server manually (perhaps using SSH).

This works and is very secure, but we could add a CLI command that creates and authenticates the new remote automatically if a Trust Password or Token for the server is known (see LXD docs for remote auth).

Alternatively if the manaul workflow is encouraged we could provide documentation for it.

Multipass VM IP is not static

Bravetools saves the IP address of the multipass VM when it first initializes it and reuses it in future. This assumes that the IP address of the VM is static - however, multipass VM IP addresses are not static by default on Windows or on Mac (I haven't tried on Mac). This means that bravetools cannot access the multipass VM nor the LXD instance running on it when the IP switches.

A potential solution mentioned here is to use <instance_name>.mshome.net rather than the IP address directly. This appears to address the issue on Windows, but won't work on Mac. Nevertheless I'll make a pull request with this change (#112) as an easy fix.

An alternative way of solving the problem might be to just not store the IP address of multipass at all and request it dynamically each time, but would require restructing of the code.

Until this issue is fixed it is possible to manually fix things by running multipass list to get the new multipass VM IP address and entering that IP in the ~/.bravetools/config.yml file under the backendsettings.resources.ip field.

Environment:

  • Your operating system name and version: Windows 10 Pro 19044.1706
  • multipass 1.9.1+win

Cannot use bravetools with certain usernames

Usernames that are too long or include special characters cause issues when using bravetools.

To reproduce, make a username longer than 12 characters. If using multipass, special characters cause issues when creating VM (including the underscore). Calling brave init will result in errors.

Environment:

  • multipass 1.9.1+win
  • Windows 10 Pro 19044.1706

I've made a pull request (#90) that attempts to address the issue by sanitizing the username. However, there are still a few issues that remain after the fix - there may be a better solution.

Debian Packaging?

Hi, is there any chance you'll provide packaging for debian or ubuntu?

Make remote network and storage names configurable in settings

Current remote settings:

{
    "name": "bravetools",
    "url": "https://192.168.64.51:8443",
    "protocol": "lxd",
    "public": false,
    "profile": "profile_name"
}

Suggested remote settings:

{
    "name": "bravetools",
    "url": "https://192.168.64.51:8443",
    "protocol": "lxd",
    "public": false,
    "storage":  "storage_name",
    "network":  "profile_namebr0",
    "profile": "profile_name"
}

This would allow to have fine-grained control over remote deployments and won't force user to have identical local and remote configurations.

Brave init should check for any IP address conflicts when creating an LXD bridge.

Expected behavior
If internal network conflict exits, brave init is expected to roll back initialisation and return an error or resolve the conflict by changing the conflicting IP address internally

Environment (please complete the following information):

  • Your operating system name and version: Ubuntu 18.04
  • Bravetools version: 1.55
  • LXD version: 4.3

'brave base' command fails on Windows when exporting image

Describe the bug
brave base command fails on Windows on final step, when exporting image.

To Reproduce
Run brave base github.com/beringresearch/bravefiles/ubuntu/ubuntu-bionic-py3 on Windows.

After all dependencies are installed using apt, on the final step, the build will fail.

Expected behavior
Expected brave image to be built successfully.

Command line log
Publishing image ubuntu-bionic-py3
Exporting image ubuntu-bionic-py3
Failed to export image: remove ubuntu-bionic-py3-1.0.root: The process cannot access the file because it is being used by another process.

Environment (please complete the following information):

  • Your operating system name and version: Windows 10 Pro version 1903 (OS Build 18362.1082) 64-bit
  • Bravetools version: 1.53
  • LXD version: 4.4

Remote Bravetools LXD profile init

For the moment Bravetools creates a local LXD profile with resources for itself using Exec commands. A bravetools profile must exist at any deploy location bravetools uses.

Using the LXD API instead would be easier and also opens up opportunities to create a bravetools-managed profile remotely.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.