hydrajump / vyos-build-ami

vyos-build-ami is a fully automated build system for VyOS AMIs (Amazon Machine Images)

vyos-build-ami

vyos-build-ami is a fully automated build system for VyOS AMIs (Amazon Machine Images)

Builds the official VyOS AMIs available on the AWS Marketplace

vyos-build-ami is a fully automated build system for VyOS AMIs defined in Ansible playbooks. The build process is self-contained in an EC2 instance launched by the playbooks and the resulting AMIs are never booted prior to release. This is no different from installing VyOS via a release ISO image on a non-AWS (Amazon Web Services) platform.

Background and Motivation

VyOS is a fork of Vyatta, which came in two flavours: open-source and commerical. Unfortunately, the only official AMI available on the AWS Marketplace is the latter. Until now...UPDATE 11th April 2014 the official VyOS AMIs are now available.

Initially, I created a recipe to build a VyOS AMI based on j3tm0t0's blog post with some linguistic help from Google Translate. However, the process required manual intervention and is not suitable for building and releasing dependable AMIs.

vyos-build-ami is a culmination of the lessons I learnt from creating my manual build recipe and a lot of head bashing. The build process is 100% automated through all phases: building, testing and releasing. In addition, build parameters are easily set in a single configuration file to accomodate new VyOS releases.

I've built this project using Ansible, which offers a number of advantages over custom scripting including:

• Simplicity and maximum ease of use.
• Strong focus on security and reliability.
• Uses a language that is designed around auditability by humans – even those not familiar with the program.

AMI Features

• Architecture: x86_64
• Virtualization type: HVM (Hardware Virtual Machine)
  • Only current generation instances e.g. M3 instances providing better and more consistent performance at lower prices than M1 instances.
• EBS root volume: 4 GB
• Enhanced networking (higher packet per second performance, lower latencies and lower network jitter).
  • Only available with C3 and I2 instances.
• Only SSH passwordless access using SSH key pair.

Cost Comparison

A quick comparison between running a VyOS AMI and the commercial Vyatta VSE AMI.

• Region: N. Virginia (us-east-1)
• Comparable instance type for each AMI
• 4 GB EBS volume
• 1 month run time ~730 hours

Vyatta VSE AMI m1.medium (PV)

Software: $0.60/Hour
Instance m1.medium (1 vCPU, 2 ECU, 3.75 GiB Memory): $0.087/Hour
Storage Standard EBS: $0.05/ GB /Month

Total = ( $0.60 + $0.087 ) * 730 + ( $0.05 * 4 ) = $501.71/Month

VyOS AMI m3.medium (HVM)

Software: FREE
Instance m3.medium (1 vCPU, 3 ECU, 3.75 GiB Memory): $0.070/Hour
Storage Standard EBS: $0.05/ GB /Month

Total = ( FREE + $0.070 ) * 730 + ( $0.05 * 4 ) = $51.3/Month

Note: Calculations do not account for data transfer costs. Amazon EC2 prices effective April 1st, 2014.

Requirements

You will need the following:

• AWS CLI (AWS Command Line Interface)
  • An AWS account
  • Your AWS access key ID and secret access key
• Ansible
• vyos-build-ami playbooks

Installation

Follow these simple steps if you are missing any of the requirements.

1. Sign up for an AWS account and get your access key ID and secret access key

Sign Up
http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-set-up.html#cli-signup

2. Install AWS CLI

Install the AWS CLI Using the MSI Installer (Windows)
http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-set-up.html#install-msi-on-windows

Install the AWS CLI Using the Bundled Installer (Linux, OS X, or Unix)
http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-set-up.html#install-bundle-other-os

Install the AWS CLI Using pip (Windows, Linux, OS X, or Unix)
http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-set-up.html#install-with-pip

Install the AWS CLI Using the Homebrew package manager on Mac OS X
brew install awscli

3. Test the AWS CLI installation

$ aws help

If the test is successful, you will see the help displayed.

4. Install Ansible

Install Ansible (all platforms)
http://docs.ansible.com/intro_installation.html

Mac OS X using Homebrew
brew install ansible

5. Test the Ansible installation

$ ansible --help

If the test is successful, you will see the help displayed.

6. Get vyos-build-ami playbooks

$ git clone https://github.com/hydrajump/vyos-build-ami.git

Configuration

1. Configure AWS CLI

You need to configure the AWS CLI with your AWS access key ID and secret access key from step 1 in the installation section above.

$ aws configure
AWS Access Key ID [None]: YOUR_AWS_ACCESS_KEY_ID
AWS Secret Access Key [None]: YOUR_AWS_SECRET_ACCESS_KEY
Default region name [None]: SELECT_A_REGION_FROM_THE_LIST_BELOW
Default output format [None]: text

Note: Default output format must be set to text for vyos-build-ami.

You can select a Default region name from the following list:

US East (Northern Virginia) Region    us-east-1
US West (Oregon) Region               us-west-2
US West (Northern California) Region  us-west-1
EU (Ireland) Region                   eu-west-1
Asia Pacific (Singapore) Region       ap-southeast-1
Asia Pacific (Sydney) Region          ap-southeast-2
Asia Pacific (Tokyo) Region           ap-northeast-1
South America (Sao Paulo) Region      sa-east-1

Note: It doesn't matter which region you choose, but you must set a region.

2. Configure vyos-build-ami

Coming soon...

Usage

Run the vyos-build-ami playbooks.

$ cd vyos-build-ami
$ ./vyos-build-ami.sh

VyOS AMIs built, tested and released.

Have fun ;)

How It Works

Coming soon...

Contact

hydrajump

@hydrajump

License

vyos-build-ami is available under the MIT license. See the LICENSE file for more info.

vyos-build-ami's Issues

version_string missing

Anybody saw this issue before?

['ec2'][0]]['version_string']['stdout'] }}: 'dict object' has no attribute 'version_string'\n\nThe error appears to have been in '/build-ami/playbooks/build-ami.yml': line 19, column 7

(I could not find issue tracker on vyos/build-ami)

AWS AMI SSH not working out of the box

EC2: -----BEGIN FETCH SSH PUBLIC KEY-----.
EC2: Requesting ssh public key from EC2 instance metadata.
EC2: Downloaded ssh public key from EC2 instance metadata.
EC2: Loaded ssh public key for user vyos.

################################################################### 100.0%

################################################################### 100.0%

################################################################### 100.0%

Unknown key type AAAAB3NzaC1yc2EAAAADAQABAAABAQDH9lrtekBNEjVnyIdVgWKbut/IHJ8pHDm5NKPbAizlETMMI+3iraBPqEV7l8p3i1aXLeNYvBESee/9J9pJe5lKLMSpgpQSRTEvT4br8r4m7KNmHRVna47heCSM6jZjfd1y0ctpNRnkaEcVxesVVIookSJpBDg/ok8h4DxMuFgmknvJrFFRCwdSUTX0pyN4IEuBRPqxXPXrHvSA5vGQjjzhXwCrPGFSq2GevyL+jv6npf74qG2SKubWY6GmqN8Oxeb+2vM12gYlm6Ppfoz21WLpLUbOYPxzmtUdxTveVNxeriOv6x5HeB2bW3vnrZ6sKPTBBOMteKj3cq1WKGKPahZh : must be ssh-rsa or ssh-dss
Saving configuration to '/config/config.boot'...
Done
EC2: -----END FETCH SSH PUBLIC KEY-----.

Vyos-AMI change hostname

Note to improve build.

After loading the AMI and logging in with default vyos@ user I switched to configure mode to change the hostname. The commit kept failing as "vyos" user. After changing root password and then switching to root I was able to commit the change and save the configuration.

vyos@VyOS-AMI# set system host-name cmnet04

Set failed

[edit]

After changing permissions From:

root@cmnet04:/opt/vyatta/config# ls -la active/
total 0
drwxrwxr-x 5 root vyattacfg 100 Jul 2 18:03 .
drwxrwxr-x 4 root vyattacfg 100 Jul 2 17:48 ..
drwxrwxr-x 4 root root 80 Jul 2 18:03 interfaces
drwxrwxr-x 3 root root 60 Jul 2 18:03 service
drwxrwxr-x 10 root root 200 Jul 2 18:03 system

TO:

vyos@cmnet04:/opt/vyatta/config$ ls -la active/
total 0
drwxrwxr-x 5 root vyattacfg 100 Jul 2 18:03 .
drwxrwxr-x 4 root vyattacfg 100 Jul 2 17:48 ..
drwxrwxr-x 4 root vyattacfg 80 Jul 2 18:03 interfaces
drwxrwxr-x 3 root vyattacfg 60 Jul 2 18:03 service
drwxrwxr-x 10 root vyattacfg 200 Jul 2 18:03 system

The VyOS user is now able to save configuration changes.

AWS Instance Block Storage not able to Create

TASK: [Launch an EC2 instance ami-020db2c14939a8efb t3.medium] ****************
failed: [localhost] => {"changed": true, "cmd": ["aws", "ec2", "run-instances", "--image-id", "ami-020db2c14939a8efb", "--key-name", "vyos-build-ami", "--security-groups", "vyos-build-ami", "--instance-type", "t3.medium", "--block-device-mappings", "[{"DeviceName":"/dev/sda1","Ebs":{"DeleteOnTermination":"true","VolumeType":"standard"}},{"DeviceName":"/dev/sdf","Ebs":{"VolumeSize":4,"DeleteOnTermination":"true","VolumeType":"standard"}}]", "--query", "Instances[*].InstanceId"], "delta": "0:00:01.560837", "end": "2022-01-11 11:07:11.955229", "rc": 252, "start": "2022-01-11 11:07:10.394392"}
stderr:
Parameter validation failed:
Invalid type for parameter BlockDeviceMappings[0].Ebs.DeleteOnTermination, value: true, type: <class 'str'>, valid types: <class 'bool'>
Invalid type for parameter BlockDeviceMappings[1].Ebs.DeleteOnTermination, value: true, type: <class 'str'>, valid types: <class 'bool'>

FATAL: all hosts have already failed -- aborting