This project is developed for my personal practice. Not recommented to be used in your productions.
Famo is a command line tool for caching builds on S3 similer to sccache. Famo is assumed to be used in many CI/CD platforms and for many languages.
Still under super development.
- ✔️ Quite easy and simple to use.
- ✔️ Detect languages automatically.
- ✔️ [WIP] Single Binary (For Linux, MacOS) which could be easily installed on CI/CD platforms.
- ✔️ [WIP] Asyncronous uploading.
To use famo, you need
- ✔️
famo
command - ✔️ S3 or its compatibles with access key and secret access key.
TBD
You need below information about S3.
- Access Key ID (e.g. AKIAXXXXXXXXXXXXXXXX)
- Secret Access Key (e.g. HLn2U3xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx)
- Region (e.g. ap-northeast-1)
- Endpoint (e.g. s3-ap-northeast-1.amazonaws.com)
- Bucket (e.g. famo-cache)
You could pass them as command line arguments. But access key id and secret access key should be passed as environment variables to hide their actual values. Use systems called "secrets" or "credentials" on your CI/CD platforms to set them.
export FAMO_ACCESS_KEY=AKIAXXXXXXXXXXXXXXXX
export FAMO_SECRET_ACCESS_LEY=HLn2U3xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
famo --bucket=famo-cache --region=ap-northeast-1 --endpoint=s3-ap-northeast-1.amazonaws.com
famo -h
to show other options.
- Watch package files (like Cargo.toml, Gemfile, package.json ...).
- Calculate unique hash (hex) from them.
- Check the existance of the cache on S3.
- If the cache exists, download and unpack it into current directory.
- Builds project. If the cache was hit, it's very fast.
- Upload an archive of the cache if cache didn't exist on step 4.
So basically cache is effective until package files are not changed. The cache will reduce the build time especially for the big projects.
There are several works which Famo was inspired. sscache
developed by Mozilla is also caching builds onto cloud storages but supported languages (build tools) are limited.
sccache
is inspired by ccache which only support gcc
.
CircleCI, TravisCI and other famous platforms have their caching system. Famo is used in platforms which doesn't have such system.
They are targets of auto detection. You can use Famo to other languages (not be listed below) by specifying --watches
and --archive
options.
- Rust (cargo)
- Node.js (npm, yarn)
- Ruby (gem)
- Crystal (shards)
Basically, Famo can be used in every platforms. Here are basic steps to use famo in your platform.
- Pre-download
famo
or includefamo
in build image. - Setting up S3 environment on your build platform.
- Insert famo step into your build process.
Currently only S3 and its compatibles are supported. Please suggest on issues when you need others. (e.g. Redis)
We need your supports especially for
- Adding another languages as a target of auto detection.
- Adding another awesome option. (e.g. Asyncronous uploading)
- Bug fixes
We don't need issues for each pull request. Please submit pull request directly. 😄