spacecase123 / packages

Cloud Posse DevOps distribution of linux packages for native apps, binaries, alpine packages, debian packages, and redhat packages.

Packages

Cloud Posse distribution of awesome apps.

---

This project is part of our comprehensive "SweetOps" approach towards DevOps.

It's 100% Open Source and licensed under the APACHE2.

Introduction

Use this repo to easily install releases of popular Open Source apps. We provide a few ways to use it.

1. Make Based Installer. This installer works regardless of your OS and distribution. It downloads packages directly from their GitHub source repos and installs them to your INSTALL_PATH.
2. Alpine Linux Packages. Use our Alpine repository to install prebuilt packages that use the original source binary (where possible) from the maintainers' official GitHub repo releases.
3. Docker Image. Use our docker image as a base-image or as part of a multi-stage docker build. The docker image always distributes the latest linux binaries for x86_64 architectures.

See examples below for usage.

Is one of our packages out of date?

Open up an issue or submit a PR (preferred). We'll review quickly!

Sponsorship

Package repository hosting is graciously provided by cloudsmith. Cloudsmith is the only fully hosted, cloud-native, universal package management solution, that enables your organization to create, store and share packages in any format, to any place, with total confidence. We believe there’s a better way to manage software assets and packages, and they’re making it happen!

Usage

Alpine Repository (recommended)

A public Alpine repository is provided by Cloud Posse. The repository is hosted on Amazon S3 and fronted by CloudFlare's CDN with end-to-end TLS. This ensures insane availability with DDoS mitigation and low-cost hosting. Using this alpine repository is ultimately more reliable than depending on GitHub for availability and provides an easier way to manage dependencies pinned at multiple versions.

The repository itself is managed using alpinist, which takes care of the heavy lifting of building repository indexes. You can self-host your own Alpine repository using this strategy.

Configure the alpine repository:

The Easy Way

We provide a bootstrap script to configure the alpine repository for your version of alpine.

curl -sSL https://apk.cloudposse.com/install.sh | bash

NOTE: Requires bash and curl to run:

For Docker

Add the following to your Dockerfile near the top.

# Install the cloudposse alpine repository
ADD https://apk.cloudposse.com/[email protected] /etc/apk/keys/
RUN echo "@cloudposse https://apk.cloudposse.com/3.11/vendor" >> /etc/apk/repositories

NOTE: we support alpine 3.7, 3.8, 3.9, 3.10, and 3.11 packages at this time

Installing Alpine Packages

When adding packages, we recommend using apk add --update $package to update the repository index before installing packages.

Simply install any package as normal:

apk add gomplate

But we recommend that you use version pinning:

apk add gomplate==3.0.0-r0

And maybe even repository pinning, so you know that you get our versions:

apk add gomplate@cloudposse==3.0.0-r0

Makefile Interface

The Makefile interface works on OSX and Linux. It's a great way to distribute binaries in an OS-agnostic way which does not depend on a package manager (e.g. no brew or apt-get).

This method is ideal for local development environments (which is how we use it) where you need the dependencies installed natively for your OS/architecture, such as installing a package on OSX.

See all available packages:

make -C install help

Install everything...

make -C install all

Install specific packages:

make -C install aws-vault chamber

Install to a specific folder:

make -C install aws-vault INSTALL_PATH=/usr/bin

Uninstall a specific package

make -C uninstall yq

Rebuilding GitHub Action Workflows

The GitHub Action workflows are compiled from the .github/package-template.yml file by running make -C .github workflows. It's also run automatically when rebuilding the README.md with make readme.

Run this make target anytime the package-template.yml changes or any new packages are added to the vendor/ folder.

IMPORTANT: The package-template.yml supports a single macro for interpolation %PACKAGE_NAME% which is replaced using a sed expression. Since the workflow uses a combation of gotemplate-like interpolations as well as inlines shell scripts, we used the %VAR% form of interpolation to avoid the need for endless escaping of interpolation specifiers.

Testing Locally

Alpine

$ make docker/build/apk/shell
$ make -C vendor/<package> apk

Debian

$ make docker/build/deb/shell
$ make -C vendor/<package> deb

Mac

$ make -C vendor/<package> install

Examples

Docker Multi-stage Build

Add this to a Dockerfile to install packages using a multi-stage build process:

FROM cloudposse/packages:latest AS packages

COPY --from=packages /packages/bin/kubectl /usr/local/bin/

Docker with Git Clone

Or... add this to a Dockerfile to easily install packages on-demand:

RUN git clone --depth=1 -b master https://github.com/cloudposse/packages.git /packages && \
    rm -rf /packages/.git && \
    make -C /packages/install kubectl

Makefile Inclusion

Sometimes it's necessary to install some binary dependencies when building projects. For example, we frequently rely on gomplate or helm to build chart packages.

Here's a stub you can include into a Makefile to make it easier to install binary dependencies.

export PACKAGES_VERSION ?= master
export PACKAGES_PATH ?= packages/
export INSTALL_PATH ?= $(PACKAGES_PATH)/bin

## Install packages
packages/install:
        @if [ ! -d $(PACKAGES_PATH) ]; then \
          echo "Installing packages $(PACKAGES_VERSION)..."; \
          rm -rf $(PACKAGES_PATH); \
          git clone --depth=1 -b $(PACKAGES_VERSION) https://github.com/cloudposse/packages.git $(PACKAGES_PATH); \
          rm -rf $(PACKAGES_PATH)/.git; \
        fi

## Install package (e.g. helm, helmfile, kubectl)
packages/install/%: packages/install
        @make -C $(PACKAGES_PATH)/install $(subst packages/install/,,$@)

## Uninstall package (e.g. helm, helmfile, kubectl)
packages/uninstall/%:
        @make -C $(PACKAGES_PATH)/uninstall $(subst packages/uninstall/,,$@)

Contributing Additional Packages

In addition to following the Contributing section, the following steps can be used to add new packages for review (via a PR). If possible (and it usually is), you want to find an existing package with similarly packaged release (.tar, .gz, uncompressed binary, etc.), and copy and edit its Makefile.

1. Copy the Makefile from an existing, similar, package within the vendors directory. Name the new folder with the same name as the binary package being installed.
2. Edit the Makefile, ensuring the DOWNLOAD_URL is properly formatted
3. Run make init from within the directory to create the DESCRIPTION, LICENSE, RELEASE, and VERSION files.
4. Ensure that a test task exists in the package Makefile. It should check the version number of the installed binary if possible.
5. Test the install and ensure that it downloads and runs as expected (make -C install <your_package> INSTALL_PATH=/tmp)
6. Test the apk build (see below)
7. Update the README.md (make init readme/deps readme)

Testing apk builds

To validate that a new package will build into an apk you can use the following steps;

make docker/build/apk/shell
make -C vendor/<appname> apk
# Some temp build files in the volume mount set user/group to nobody/nobody for apk building.
# It is easier to remove them while within the docker container.
rm -rf ./tmp/build.*
exit

Troubleshooting Package Addition

Here are some solutions to several common problems that may occur when adding a new package:

1. When adding a new app, the `make -C vendor/ apk` command fails, claiming it can't find the app's binary file, even though it is in the expected place.

   Part of the make -C vendor/<app> apk command is building a package for the binary file inside an Alpine Linux container. Since Alpine Linux uses musl as its C library, this often leads to situations where binaries built against libc might not function on Alpine. What's more, binaries from projects written in Go will not be found by the Alpine package builder at all if they are missing any necessary libraries, like libc. The solution to this problem is to add an export APKBUILD_DEPENDS += libc6-compat line to the top of your new package's associated Makefile.

2. When adding a new binary, the `make builder TARGETS=readme` command fails with `Unable to find image 'cloudposse/build-harness:sha-[some_SHA_stub]' locally`.

   This can occur when you have the cloudposse/build-harness repository checked out somewhere on your machine. make builder TARGETS=readme will end up looking for a docker image tagged with the SHA that the HEAD ref of your buld-harness points to. To correct this behavior, just run make init in the cloudposse/packages directory prior to running make builder TARGETS=readme.

Package Build Status

Build Status	Version	Description
	0.6.0	Automatically gets credentials for Amazon ECR on docker push/docker pull
	0.24.0	Tool for interacting with the Alertmanager API
	2.4.0	Declarative GitOpts for Kubernetes
	0.3.2	Easily assume AWS roles in your terminal.
	0.19.4	Terraform For Teams
	1.4.20	Universal Tool for DevOps and Cloud Automation
	0.1.11	A Mighty CLI for AWS
	0.5.7	A tool to use AWS IAM credentials to authenticate to a Kubernetes cluster
	6.6.0	A vault for securely storing and accessing AWS credentials in development environments
	1.6.1	Cloudflare's PKI and TLS toolkit
	1.6.1	Cloudflare's PKI and TLS toolkit json parser
	2.10.10	CLI for managing secrets
	0.8.18	Command line tool for Amazon Route 53
	2022.5.3	Argo Tunnel client
	0.9.0.3	Terraform For Teams, enhanced by Cloud Posse
	0.79.1	Codefresh CLI
	0.32.1	Test your configuration files using Open Policy Agent
	1.12.2	Hashicorp consul
	0.7.7	Top-like interface for container metrics
	2.32.0	Unclutter your .profile
	1.76.2	A command line tool for DigitalOcean services
	0.3.5b1	CNAB installer
	2.3.0	A CLI tool and go library which recommends instance types based on resource criteria like vcpus and memory
	1.0.3	Command line email sending client written in Go.
	0.7.1	A simple wrapper that allows you to run commands within ethereal docker containers
	0.3.2	CLI for AWS Fargate
	0.4.4	fetch makes it easy to download files, folders, and release assets from a specific public git commit, branch, or tag
	1.2.0	Print your name in style
	0.23.1	A command-line fuzzy finder
	2.12.1	The GitHub CLI
	0.14.0	Upload multiple artifacts to GitHub Releases in parallel
	0.10.1	Command line utility for creating GitHub comments on Commits, Pull Request Reviews or Issues
	0.10.0	Commandline app to create and edit releases on Github (and upload artifacts)
	0.7.0	Command line utility for updating GitHub commit statuses and enabling required status checks for pull requests
	1.2.0	Audit git repos for secrets 🔑
	0.18.0	This an implementation of Jsonnet in pure Go.
	3.11.0	A flexible commandline tool for template rendering. Supports lots of local and remote datasources.
	1.0.2	A stand-alone alternative to git2consul
	0.24.0	a high-performance, POSIX-ish Amazon S3 file system written in Go
	1.14.0	Simple Go-based setuid+setgid+setgroups+exec
	3.0.0	A terminal based graphical activity monitor inspired by gtop and vtop
	1.8.6	Like cURL, but for gRPC: Command-line tool for interacting with gRPC servers
	0.2.4	A command line editor for HCL
	3.9.0	The Kubernetes Package Manager
	2.17.0	The Kubernetes Package Manager
	3.9.0	The Kubernetes Package Manager
	0.144.0	Deploy Kubernetes Helm Charts
	0.16.0	✅ Test generated HTML for problems
	0.100.2	The world’s fastest framework for building websites.
	0.10.3	Cloud cost estimates for Terraform
	0.2.1	Command line interface to JMESPath
	1.6.0	Command-line JSON processor
	0.0.6	Convert JSON to HCL, and vice versa
	3.2.346	Jenkins-X
	5.4.3	Little helper to run Rancher Lab's k3s in Docker
	0.38.3	A modern load testing tool, using Go and JavaScript - https://k6.io
	0.25.18	Kubernetes CLI To Manage Your Clusters In Style
	0.8.3	K8s continuous backup to git
	1.2.0	Machine Learning Toolkit for Kubernetes
	0.14.0	A tool for running local Kubernetes clusters using Docker
	1.23.2	Kubernetes Operations (kops) - Production Grade K8s Installation, Upgrades, and Management
	0.4.1	Kubectl plugin manager
	2.0.0	Utilities to manage kubernetes cronjobs. Run a CronJob manually for test purposes. Suspend/unsuspend a CronJob
	1.24.1	Production-Grade Container Scheduling and Management
	1.13.12	Production-Grade Container Scheduling and Management (v1.13)
	1.14.10	Production-Grade Container Scheduling and Management (v1.14)
	1.15.12	Production-Grade Container Scheduling and Management (v1.15)
	1.16.15	Production-Grade Container Scheduling and Management (v1.16)
	1.17.17	Production-Grade Container Scheduling and Management (v1.17)
	1.18.20	Production-Grade Container Scheduling and Management (v1.18)
	1.19.16	Production-Grade Container Scheduling and Management (v1.19)
	1.20.15	Production-Grade Container Scheduling and Management (v1.20)
	1.21.13	Production-Grade Container Scheduling and Management (v1.21)
	1.22.10	Production-Grade Container Scheduling and Management (v1.22)
	0.9.4	Switch faster between clusters and namespaces in kubectl
	0.9.4	Switch faster between clusters and namespaces in kubectl
	0.16.1	Validate your Kubernetes configuration files, supports multiple Kubernetes versions
	0.18.1	The lazier way to manage everything docker
	0.21.0	Script to check issued certificates by Let's Encrypt on CTL (Certificate Transparency Log) using https://crt.sh
	1.25.2	Run Kubernetes locally
	0.3.4	Correct commonly misspelled English words in source files
	0.41.0	An open source project to policy-enable your service.
	0.26.0	Create cloud native Buildpacks
	1.8.1	Packer is a tool for creating identical machine images for multiple platforms from a single source configuration.
	2.18	Universal markup converter
	1.13.0	Postgres metrics
	5.8.0	A cli tool to help discover deprecated apiVersions in Kubernetes
	0.10.0	A Kubernetes cluster resource sanitizer
	2.36.1	Prometheus CLI tool
	1.2.1	Tasty rainbows for your terminal! (lolcat clone)
	0.5.0	Review Access - kubectl plugin to show an access matrix for all available resources
	2.6.5	Rancher CLI
	0.8.3	Find Kubernetes roles and cluster roles bound to any user, service account, or group name.
	2.35.0	CLI tool which enables you to login and retrieve AWS temporary credentials using a SAML IDP
	0.1.5	A Terraform plan output prettifier
	2.1.0	A command line utility to work with Sentry.
	0.8.0	ShellCheck, a static analysis tool for shell scripts
	3.5.1	A shell parser, formatter and interpreter (POSIX/Bash/mksh)
	0.4.0	Command line utility to send messages with attachments to Slack channels via Incoming Webhooks
	3.7.3	Secrets management stinks, use some sops!
	0.10.0	Spacelift.io client and CLI
	0.25.0	A unified CLI to manage your Spot resources.
	1.2.2	Easy connect on EC2 instances thanks to AWS System Manager Agent
	1.11.0	⎈ Multi pod and container log tailing for Kubernetes
	0.3.0	Shell wrapper to run a login shell with sudo as the current user for the purpose of audit logging
	9.3.5	Secure Access for Developers that doesn't get in the way.
	4.2.12	Privileged access management for elastic infrastructure.
	4.3.10	Privileged access management for elastic infrastructure.
	4.4.12	Privileged access management for elastic infrastructure.
	5.0.2	Secure Access for Developers that doesn't get in the way.
	1.2.2	Terraform is a tool for building, changing, and combining infrastructure safely and efficiently.
	0.11.15	Terraform is a tool for building, changing, and combining infrastructure safely and efficiently.
	0.12.31	Terraform is a tool for building, changing, and combining infrastructure safely and efficiently.
	0.13.7	Terraform is a tool for building, changing, and combining infrastructure safely and efficiently.
	0.14.11	Terraform is a tool for building, changing, and combining infrastructure safely and efficiently.
	0.15.5	Terraform is a tool for building, changing, and combining infrastructure safely and efficiently.
	1.2.2	Terraform enables you to safely and predictably create, change, and improve infrastructure.
	0.0.20211115214459+git90acf1ca460f	A helper library for shallow inspection of Terraform configurations
	0.16.0	Generate docs from terraform modules
	3.1.10	CLI tool that checks Terraform code for module updates. Single binary, no dependencies. linux, osx, windows.
	0.11.15	Terraform (Deprecated package. Use terraform-0.11 instead)
	0.12.31	Terraform (Deprecated package. Use terraform-0.12 instead)
	0.13.7	Terraform (Deprecated package. Use terraform-0.13 instead)
	0.37.4	Terragrunt is a thin wrapper for Terraform that provides extra tools for working with multiple Terraform modules.
	0.7.5	Terrahelp is as a command line utility that provides useful tricks like masking of terraform output.
	0.4.0	Transform environment variables for use with Terraform (e.g. HOSTNAME ⇨ TF_VAR_hostname)
	0.37.0	A Pluggable Terraform Linter
	0.7.0	Terraform utility to mask select output from terraform plan and terraform apply
	0.7.2	A schema inspector for Terraform providers
	1.24.4	Security scanner for your Terraform code
	0.26.0	Highly available Prometheus setup with long term storage capabilities. CNCF Sandbox project.
	0.28.1	A Simple and Comprehensive Vulnerability Scanner for Containers, Suitable for CI
	0.17.0	Turf is Cloud Posse's command-line automation helper.
	0.38.0	Variant is a Universal CLI tool that works like a task runner
	0.38.0	Second major version of Variant, a Universal CLI tool that works like a task runner
	1.10.4	Hashicorp vault
	0.27.0	Easy way to vendor portions of git repos, github releases, helm charts, docker image contents, etc. declaratively.
	1.7.4	Codefresh runtime-environment agent
	0.1.0	Simple CLI for comparing two or more versions
	1.4.0	Yet Another JSON Schema Validator [CLI]
	4.25.2	yq is a portable command-line YAML processor

Share the Love

Like this project? Please give it a ★ on our GitHub! (it helps us a lot)

Are you using this project or any of our other projects? Consider leaving a testimonial. =)

Related Projects

Check out these related projects.

• build-harness - Collection of Makefiles to facilitate building Golang projects, Dockerfiles, Helm charts, and more
• geodesic - Geodesic is the fastest way to get up and running with a rock solid, production grade cloud platform built on strictly Open Source tools.

Help

Got a question? We got answers.

File a GitHub issue, send us an email or join our Slack Community.

DevOps Accelerator for Startups

We are a DevOps Accelerator. We'll help you build your cloud infrastructure from the ground up so you can own it. Then we'll show you how to operate it and stick around for as long as you need us.

Work directly with our team of DevOps experts via email, slack, and video conferencing.

We deliver 10x the value for a fraction of the cost of a full-time engineer. Our track record is not even funny. If you want things done right and you need it done FAST, then we're your best bet.

• Reference Architecture. You'll get everything you need from the ground up built using 100% infrastructure as code.
• Release Engineering. You'll have end-to-end CI/CD with unlimited staging environments.
• Site Reliability Engineering. You'll have total visibility into your apps and microservices.
• Security Baseline. You'll have built-in governance with accountability and audit logs for all changes.
• GitOps. You'll be able to operate your infrastructure via Pull Requests.
• Training. You'll receive hands-on training so your team can operate what we build.
• Questions. You'll have a direct line of communication between our teams via a Shared Slack channel.
• Troubleshooting. You'll get help to triage when things aren't working.
• Code Reviews. You'll receive constructive feedback on Pull Requests.
• Bug Fixes. We'll rapidly work with you to fix any bugs in our projects.

Slack Community

Join our Open Source Community on Slack. It's FREE for everyone! Our "SweetOps" community is where you get to talk with others who share a similar vision for how to rollout and manage infrastructure. This is the best place to talk shop, ask questions, solicit feedback, and work together as a community to build totally sweet infrastructure.

Discourse Forums

Participate in our Discourse Forums. Here you'll find answers to commonly asked questions. Most questions will be related to the enormous number of projects we support on our GitHub. Come here to collaborate on answers, find solutions, and get ideas about the products and services we value. It only takes a minute to get started! Just sign in with SSO using your GitHub account.

Newsletter

Sign up for our newsletter that covers everything on our technology radar. Receive updates on what we're up to on GitHub as well as awesome new projects we discover.

Office Hours

Join us every Wednesday via Zoom for our weekly "Lunch & Learn" sessions. It's FREE for everyone!

Contributing

Bug Reports & Feature Requests

Please use the issue tracker to report any bugs or file feature requests.

Developing

If you are interested in being a contributor and want to get involved in developing this project or help out with our other projects, we would love to hear from you! Shoot us an email.

In general, PRs are welcome. We follow the typical "fork-and-pull" Git workflow.

1. Fork the repo on GitHub
2. Clone the project to your own machine
3. Commit changes to your own branch
4. Push your work back up to your fork
5. Submit a Pull Request so that we can review your changes

NOTE: Be sure to merge the latest changes from "upstream" before making a pull request!

Copyright

Copyright © 2017-2022 Cloud Posse, LLC

License

See LICENSE for full details.

Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.

Trademarks

All other trademarks referenced herein are the property of their respective owners.

About

This project is maintained and funded by Cloud Posse, LLC. Like it? Please let us know by leaving a testimonial!

We're a DevOps Professional Services company based in Los Angeles, CA. We ❤️ Open Source Software.

We offer paid support on all of our projects.

Check out our other projects, follow us on twitter, apply for a job, or hire us to help with your cloud strategy and implementation.

Contributors

Erik Osterman	Igor Rodionov	Andriy Knysh