lukeed / gittar

:guitar: Download and/or Extract git repositories (GitHub, GitLab, BitBucket). Cross-platform and Offline-first!

gittar

🎸 Download and/or Extract git repositories (GitHub, GitLab, BitBucket). Cross-platform and Offline-first!

Gittar is a Promise-based API that downloads *.tar.gz files from GitHub, GitLab, and BitBucket.

All archives are saved to the $HOME/.gittar directory with the following structure:

{HOSTNAME}/{USER}/{REPO}/{BRANCH-TAG}.tar.gz
#=> github/lukeed/mri/v1.1.0.tar.gz
#=> gitlab/Rich-Harris/buble/v0.15.2.tar.gz
#=> github/vuejs-templates/pwa/master.tar.gz

By default, new gittar.fetch requests will check the local filesystem for a matching tarball before intiating a new remote download!

Important: Please see gittar.fetch for exceptions & behaviors!

Install

$ npm install --save gittar

Usage

const gittar = require('gittar');

gittar.fetch('lukeed/gittar').then(console.log);
//=> ~/.gittar/github/lukeed/gittar/master.tar.gz

gittar.fetch('lukeed/tinydate#v1.0.0').then(console.log);
//=> ~/.gittar/github/lukeed/tinydate/v1.0.0.tar.gz

gittar.fetch('https://github.com/lukeed/mri').then(console.log);
//=> ~/.gittar/github/lukeed/mri/master.tar.gz

gittar.fetch('gitlab:Rich-Harris/buble#v0.15.2').then(console.log);
//=> ~/.gittar/gitlab/Rich-Harris/buble/v0.15.2.tar.gz

gittar.fetch('Rich-Harris/buble', { host:'gitlab' }).then(console.log);
//=> ~/.gittar/gitlab/Rich-Harris/buble/master.tar.gz

const src = '...local file or repo pattern...';
const dest = '/path/to/foobar';

gittar.extract(src, dest, {
  filter(path, entry) {
    if (path.includes('package.json')) {
      let pkg = '';
      entry.on('data', x => pkg += x).on('end', _ => {
        const devDeps = JSON.parse(pkg).devDependencies || {};
        console.log('~> new devDependencies:', Object.keys(devDeps));
      });
    }

    if (path.includes('.babelrc')) {
      return false; // ignore this file!
    }

    return true; // keep all other files
  }
});

API

gittar.fetch(repo, options)

Type: Promise
Returns: String

Behavior

Parses the repo name, then looks for matching tarball on filesystem. Otherwise, a HTTPS request is dispatched and then, if successful, the archive is saved to ~/.gittar at the appropriate path.

Exceptions

• If Gittar detects that your machine is not connected to the internet, or if useCache is true, then it will only attempt a local fetch.
• If repo is (or is assumed to be) a master branch, or if force is true, then the archive will be downloaded over HTTPS before checking local cache.

repo

Type: String

The name, link, or pattern for a git repository.

Optionally provide a tag or branch name, otherwise master is assumed.

parse('user/repo');
//=> ~/.gittar/github/user/repo/master.tar.gz
//=> https://github.com/user/repo/archive/master.tar.gz

parse('user/repo#v1.0.0');
//=> ~/.gittar/github/user/repo/v1.0.0.tar.gz
//=> https://github.com/user/repo/archive/v1.0.0.tar.gz

parse('user/repo#production');
//=> ~/.gittar/github/user/repo/production.tar.gz
//=> https://github.com/user/repo/archive/production.tar.gz

parse('bitbucket:user/repo');
//=> ~/.gittar/bitbucket/user/repo/master.tar.gz
//=> https://bitbucket.org/user/repo/get/master.tar.gz

parse('https://gitlab.com/user/repo');
//=> ~/.gittar/gitlab/user/repo/master.tar.gz
//=> https://gitlab.com/user/repo/repository/archive.tar.gz?ref=master

Note: The parse function does not exist -- it's only demonstrating the parsed values of the varying patterns.

options.host

Type: String
Default: github

The hostname for the repository.

Specifying a "hint" in the repo will take precedence over this value.

fetch('gitlab:user/repo', { host:'bitbucket' });
//=> ~/.gittar/gitlab/user/repo/master.tar.gz
//=> https://gitlab.com/user/repo/repository/archive.tar.gz?ref=master

options.force

Type: Boolean
Default: false

Force a HTTPS download. If there is an error with the network request, a local/cache lookup will follow.

Note: A network request is also forced if the repo is (or is assumed to be) a master branch!

options.useCache

Type: Boolean
Default: false

Only attempt to use an existing, cached file. No network requests will be dispatched.

Note: Gittar enacts this option if it detects that there is no internet connectivity.

gittar.extract(file, target, options)

file

Type: String

A filepath to extract. Also accepts a repo pattern!

Important: A repo pattern will be parsed (internally) as a filepath. No network requests will be dispatched.

target

Type: String
Default: process.cwd()

The target directory to place the archive's contents.

Note: Value will be resolved (see path.resolve) if not already an absolute path.

options

Type: Object
Default: { strip:1 }

All options are passed directly to tar.extract.

Note: The cwd and file options are set for you and cannot be changed!

options.strip

Type: Integer
Default: 1

By default, gittar will strip the name of tarball from the extracted filepath.

const file = 'lukeed/mri#master';

// strip:1 (default)
gittar.extract(file, 'foo');
//=> contents: foo/**

// strip:0 (retain tarball name)
gittar.extract(file, 'foo', { strip:0 });
//=> contents: foo/mri-master/**

License

MIT © Luke Edwards

gittar's Issues

Save in the correct OS cache location

Home folder dot files are out of hand! It would be better if this module instead saved to the preferred operating system locations for cache content.

A module like https://github.com/sindresorhus/env-paths or https://github.com/sindresorhus/xdg-basedir makes it very easy to find that path. Would you be open to changing the default to one of these alternate locations?

Private Repos

Might be its own module // decorator 🤔

• Use private: prefix or private:true
• Spawn a git clone process
• Sidestep .git dir with extraction/tmpdir

Does gittar depend on `git` to work?

I have used the awesome degit from Rich Harris, and included it on my little framework to allow the installation of optional modules. What I don't realized was that it depends on git.

As degit, does gittar depend on git to be installed to work?

Lookup `master` Ref

Run a quick check to see if the latest remote ref on <user/repo#master> is newer than what we have locally. Only if it's more recent, begin downloading the archive.

Requires saving/storing the last-used ref locally.

Feat: Configure `fetch` destination

Should add a new target option for the gittar.fetch method.

It should be a function that receives all the components from the parser ({ site, repo, type }) so that you have all the same information that gittar would use to compose a location.

When left unspecified, gittar will default to its current destination(s). But this gives a window for the caller to fully customize where the tarball is placed.

gittar.fetch('foo/bar', {
  target(info) {
    return path.join(__dirname, info.repo + '.tar.gz')
  }
})

This is a non-breaking way to handle #7. It also handles #5 in a different way.

Better Errors

Specifically for local/file-based errors.

Right now:

• fetch (local) Not-Found rejects with undefined
• extract Not-Found rejects with undefined

Not good -- but at least it triggers catch 😅

Would like to reject with ENONET somehow... or maybe need a custom hash of error codes & messages?

Likely rework run helper to recognize errors-vs-nothing before initiating next attempt.