GithubHelp home page GithubHelp logo

godaddy / bffs Goto Github PK

View Code? Open in Web Editor NEW
1.0 12.0 3.0 649 KB

handles higher level operations around the warehouse data model and build pipeline.

License: MIT License

JavaScript 100.00%

bffs's Introduction

bffs

Version npm License npm Downloads Build Status Dependencies

BFFS (Build files finder service) is a module that helps with the storage and retrieval of build files for warehouse.ai.

It stores the build files and gzip version of the build file. In addition to that it also stores all meta for a given build so it can be retrieved again by searching for the spec it was build with (version, name and environment)

Install

npm install --save bffs

Usage

In all API examples we assume that you've already created a new BFFS instance, this is how you setup a new instance:

'use strict';

const Redis = require('ioredis');
const BFFS = require('bffs');

const bffs = new BFFS({
  store: new Redis()
});

As you can see in the example above, the constructor accepts a single options argument this options object can contain the following properties:

  • store an ioredis client instance that can be used
  • cache Options that are directly passed in to the hotpath module for hotpath cache optimization.
  • cdn The root or prefix of the build files URL's.
  • env An array of environments that we support.

API

bffs.build

Fetch a completed build file. The method requires the following arguments:

  • The fingerprint + extension of the buildfile that you're trying to retrieve.
  • Boolean that indicates if you want to have a pre-gzipped version instead.
  • Completion callback which follows the error first callback pattern.
bffs.build('0810984t019823098d08a.js', false, function (err, data) {

});

bffs.search

Search for previously published builds of modules. This method accepts 3 arguments:

  • Object with a build specification that needs to be fetched. It requires the name, version and env properties to be set.
  • Optional argument which will be passed in to the key method to generate a specific a key. Only used internally
  • Completion callback which follows the error first callback pattern.
bffs.search({
  name: 'wsb-pancakes',
  version: '1.2.4',
  env: 'test'
}, function (err, meta) {

});

bffs.publish

Publish a new build, this does a couple of things. It stores the content and compressed keys as "build" files using the fingerprint key as file name. The rest of the data is stored as meta build data which should give some detailed information about the build it self.

This method requires 3 arguments:

  • Object with a build specification that needs to be fetched. It requires the name, version and env properties to be set.
  • Object with a files array that contains compressed, content and fingerprint as minimum requirements. You don't need to add the name, version and env properties to this object as we will merge those in from the first supplied argument.
  • Completion callback which follows the error first callback pattern.
bffs.publish({
  name: 'wsb-pancakes',
  version: '1.2.5',
  env: 'test'
}, {
  promote: false, // prevents creating BuildHead based on the created Build
  files: [{
    content: fs.readFileSync('file.js'),
    compressed: fs.readFileSync('file.js.gz'),
    fingerprint: fingerprinter(fs.readFileSync('file.js')).id
  }]
}, function (err) {

});

bffs.promote

Promotes the Build defined by the given spec (name, env, version) to be the BuildHead for every locale it was built for. Useful for when a build was published using promote: false functionality.

bffs.promote({
  name: 'my-package',
  env: 'test',
  version: '3.4.5'
}, (err) => {
  if (err) return /* handle me */
})

bffs.rollback

Sets the current BuildHead to the given name, env, and version while also setting keeping record of the prior build via rollbackBuildId.

bffs.rollback({
  name: 'my-package',
  env: 'test'
}, /* optional /* '1.5.6', (err) => {
  if (err) return /* handle me */
})

bffs.meta

Get all the meta data from a given build for every support environment. This method requires 2 arguments:

  • Object with a build specification that needs to be fetched. It requires the name, version properties to be set.
  • Completion callback which follows the error first callback pattern.
bffs.meta({
  name: 'wsb-pancakes',
  version: '1.1.1'
}, function (err, meta) {

});

bffs.start

Store the build id as indication that a given package is already building. The build will stay active until the supplied timeout or stop method has been called. This method requires 4 arguments:

  • Object with a build specification that needs to be fetched. It requires the name, version and env properties to be set.
  • The id of the build.
  • Timeout of the build as number.
  • Completion callback which follows the error first callback pattern.
bffs.start({
  name: 'wsb-pancakes',
  version: '1.2.5',
  env: 'test'
}, '98798ad0-afd7a0-afasdfas901', 27E2, function (err) {

});

bffs.active

Check if a given build set is active and returns the jobs of the builds if this is the case. It requires 2 arguments:

  • Object with a build specification that needs to be fetched. It requires the name, version and env properties to be set.
  • Completion callback which follows the error first callback pattern.
bffs.active({
  name: 'wsb-pancakes',
  version: '1.2.5',
  env: 'test'
}, function (err, jobs) {
  // jobs is an array with a `key` and `value` property.
  // The `value` is the `id` of the job.
});

bffs.stop

Stop and remove the indication that a build is running. It requires 2 arguments:

bffs.stop({
  name: 'wsb-pancakes',
  version: '1.2.5',
  env: 'test'
}, function (err) {

});

bffs.partial

Get running build information. It requires 2 arguments:

bffs.partial({
  name: 'wsb-pancakes',
  version: '1.2.5',
  env: 'test'
}, function (err, job) {

});

bffs.wipe

Stop the build and clear it from cache. It requires 2 arguments:

bffs.wipe({
  name: 'wsb-pancakes',
  version: '1.2.5',
  env: 'test'
}, function (err) {

});

Tests

A key/value database (i.e Redis or similar) should be running local.

docker pull redis:latest
npm run redis

Also run an AWS local cloud stack, pull latest localstack. This requires docker to be setup.

docker pull localstack/localstack:latest
npm run localstack

Run tests in a separate terminal.

npm test

bffs's People

Contributors

agerard-godaddy avatar dependabot[bot] avatar fritzmonkey avatar indexzero avatar jcrugzz avatar msluther avatar mswaagman-godaddy avatar sharrison-godaddy avatar swaagie avatar thumbsupep avatar wjarvis-godaddy avatar wjarvis1 avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bffs's Issues

DynamoDB Migration

To support the DynamoDB migration outlined in this proposal:

Once warehouse-models is updated

  • bffs uses the new DynamoDB version of warehouse-models
  • Tests are updated and use localstack if necessary
  • Update documentation
  • Published as a new major version

Usage of Redis should be deprecated

๐Ÿ› Bug Report

bffs still requires both Redis and another database (DynamoDB or similar) to work. This seems to be limited to unit tests. For example, #partial is not used by any other service https://github.com/search?q=org%3Awarehouseai+partial&type=Code. Other functions that can be deprecated.

  • partial
  • stop
  • start
  • wipe
  • active

BFFS.defaults seems to be unused. #keys can be simplified to only support the default and/or file variation.

To Reproduce

No specific steps, unit test still use Redis so I guess npm test

Expected Behavior

It should only use a single database to store status and build stats.

Code Example

See internal unit tests that still use bffs.store.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.