GithubHelp home page GithubHelp logo

mozilla / oghliner Goto Github PK

View Code? Open in Web Editor NEW
115.0 13.0 17.0 753 KB

template and tool for deploying Offline Web Apps to GitHub Pages

Home Page: https://mozilla.github.io/oghliner/

License: Apache License 2.0

HTML 5.35% JavaScript 91.41% CSS 3.23%

oghliner's Introduction

Build Status dependencies devdependencies

Oghliner

Oghliner is a Node tool to deploy Offline Web Apps to GitHub Pages.

Offline Web Apps are web apps that cache their files (HTML, JavaScript, CSS, images, etc.) on the client, so they work even when your network doesn't. GitHub Pages is a simple host for static files.

Oghliner offlines an app by generating a Service Worker for it. It deploys the app by committing it to the gh-pages branch of the repository being deployed.

Oghliner includes commands for bootstrapping an app from scratch or integrating its functionality into your existing app. It can also configure Travis CI to automatically deploy your app when you merge changes to its master branch.

Oghliner has both command-line and module interfaces and provides five commands: bootstrap, integrate, offline, deploy, and configure. All commands are available via both interfaces, but the CLI is more appropriate for the bootstrap, integrate, and configure commands, which you'll invoke at most once per app; while the module is more appropriate for the offline and deploy commands, which you'll call each time you change your app and rebuild it.

Getting Started

To use the CLI, install Oghliner globally:

npm install --global oghliner

To use the module, install Oghliner locally and save it to your dependencies:

npm install --save oghliner

Then require it in your script(s):

var oghliner = require('oghliner');

Bootstrap

The bootstrap command creates an initial set of directories and files for a new app. To use it, first create a repository on GitHub and clone it to your local machine:

git clone [email protected]:mykmelez/offline-app.git

Then change to its working directory and invoke oghliner bootstrap:

cd offline-app/
oghliner bootstrap

Oghliner will bootstrap your app by copying files from its app template to the current directory:

Bootstrapping current directory as Oghliner app…

Your app's configuration is:

Name: offline-app
Repository: [email protected]:mykmelez/offline-app.git
Description: A template app bootstrapped with oghliner.
License: Apache-2.0

Would you like to change its configuration (y/N)? n

Creating files…
✓ Creating README.md
✓ Creating app
✓ Creating .gitignore
✓ Creating gulpfile.js
✓ Creating package.json
✓ Creating app/favicon.ico
✓ Creating app/fonts
✓ Creating app/images
✓ Creating app/index.html
✓ Creating app/robots.txt
✓ Creating app/scripts
✓ Creating app/styles
✓ Creating app/images/apple-touch-icon-114x114.png
✓ Creating app/images/apple-touch-icon-120x120.png
✓ Creating app/images/apple-touch-icon-144x144.png
✓ Creating app/images/apple-touch-icon-152x152.png
✓ Creating app/images/apple-touch-icon-57x57.png
✓ Creating app/images/apple-touch-icon-60x60.png
✓ Creating app/images/apple-touch-icon-72x72.png
✓ Creating app/images/apple-touch-icon-76x76.png
✓ Creating app/images/favicon-128x128.png
✓ Creating app/images/favicon-16x16.png
✓ Creating app/images/favicon-196x196.png
✓ Creating app/images/favicon-32x32.png
✓ Creating app/images/favicon-96x96.png
✓ Creating app/images/mstile-144x144.png
✓ Creating app/images/mstile-150x150.png
✓ Creating app/images/mstile-310x150.png
✓ Creating app/images/mstile-310x310.png
✓ Creating app/images/mstile-70x70.png
✓ Creating app/scripts/main.js
✓ Creating app/scripts/offline-manager.js
✓ Creating app/styles/stylesheet.css

✓ Creating files… done!
✓ Installing npm dependencies… done!

Your app has been bootstrapped! Just commit the changes and push the commit
to the origin/master branch:

git add --all && git commit -m"initial version of Oghliner app"
git push origin master

Then you can build, offline, and deploy the app using gulp commands.

ℹ For more information about building, offlining and deployment, see:
    https://mozilla.github.io/oghliner/

Then just commit the changes and push the commit to the origin/master branch:

git add --all && git commit -m"initial version of Oghliner app"
git push origin master

Note: To bootstrap an app into a different directory than the current one, specify its path when invoking bootstrap:

oghliner bootstrap path/to/another/clone/

Integrate

The integrate command adds Oghliner functionality into an existing app. To use it, invoke oghliner integrate, passing it the path to the directory containing your app's scripts:

oghliner integrate app/scripts/

Oghliner will copy the offline-manager.js script to that directory:

Integrating Oghliner into the app in the current directory…

✓ Copying offline-manager.js to app/scripts/… done!

Oghliner has been integrated into the app!

The app needs to load the offline-manager.js script in order to register
the service worker that offlines the app. To load the script, add this line
to the app's HTML page(s)/template(s):

<script src="app/scripts/offline-manager.js"></script>

Then commit the changes and push the commit to the origin/master branch:

git commit -m"integrate Oghliner" --all
git push origin master

Then you can offline and deploy the app using the offline and deploy commands.

ℹ For more information about offlining and deployment, see:
    https://mozilla.github.io/oghliner/

Then add a <script> tag referencing that script to your app's HTML page(s)/template(s). Oghliner will suggest one, but it doesn't necessarily know your app's directory structure, so make sure it contains the correct path to the file!

Offline

The offline command generates a service worker that caches your app's files on the client.

To use it via the command line, invoke oghliner offline, passing it a rootDir argument specifying the directory containing the files to offline:

oghliner offline dist/

To use it via the module, call oghliner.offline, passing it an options object with a rootDir property specifying the directory containing the files to offline:

var promise = oghliner.offline({
  rootDir: 'dist/',
});

The call returns a Promise that resolves once Oghliner finishes generating the service worker. If left unspecified, the default value of rootDir is ./, i.e. the current directory.

Note: rootDir should be the target directory containing the output of your build process, not the source directory containing the original files. For example, if your source files are in app/, and your build process outputs into dist/, then you should specify the dist/ directory.

Options

The offline command takes the following options:

  • --file-globs glob,… or fileGlobs: ['glob', …] - a comma-separated list of globs identifying the files to offline (default: **/*). The globs are matched inside rootDir.
  • --import-scripts script,… or importScripts: ['script', …] - a comma-separated list of additional scripts to evaluate in the service worker (no default value). This is useful, for example, when you want to use the Push API.

Deploy

The deploy command deploys your app to GitHub Pages by committing its files to the gh-pages branch of the repository and pushing the commit to GitHub.

To use it via the command line, invoke oghliner deploy, passing it a rootDir argument specifying the directory containing the files to deploy:

oghliner deploy dist/

To use it via the module, call oghliner.deploy, passing it an options object with a rootDir property specifying the directory containing the files to deploy:

var promise = oghliner.deploy({
  rootDir: 'dist/',
});

The call returns a Promise that resolves once Oghliner finishes deploying the app. If left unspecified, the default value of rootDir is ./, i.e. the current directory.

Note: rootDir should be the target directory containing the output of your build process, not the source directory containing the original files. For example, if your source files are in app/, and your build process outputs into dist/, then you should specify the dist/ directory.

Options

The deploy command takes the following options via both the CLI and the module:

  • -m, --message message or message: 'message' - the message for the commit to the gh-pages branch

The deploy command takes the following options via the module only:

  • cloneDir: 'dir' - the directory into which Oghliner will create a temporary clone of the repository while deploying the app (default is implementation detail). This is mostly useful internally, for implementation of the CLI.
  • fileGlobs: ['glob', …] - a comma-separated list of globs identifying the files to offline (default: **/*). The globs are matched inside rootDir.
  • remote: 'remote' - the Git remote to which to push the gh-pages branch (default: origin).

Configure

The configure command configures Travis to automatically deploy an app to GitHub Pages when you push to its master branch.

To use it, invoke oghliner configure within your local working directory:

oghliner configure

Oghliner will create a GitHub token that authorizes Travis to push changes to your repository, then configure Travis to use the token to deploy changes.

Configuring Travis to auto-deploy to GitHub Pages…

Your repository has a single remote, origin.
Ok, I'll configure Travis to auto-deploy the origin remote (mykmelez/offline-app).

To check the status of your repository in Travis and authorize Travis to push
to it, I'll create GitHub personal access tokens, for which I need your GitHub
username and password (and two-factor authentication code, if appropriate).

ℹ For more information about GitHub personal access tokens, see:
    https://github.com/settings/tokens

Username: mykmelez
Password:

✗ Checking credentials… error!

You're using two-factor authentication with GitHub.
Please enter the code provided by your authentication software.

Auth Code: 123456

✓ Checking credentials… done!
✓ Getting Travis token… done!
✓ Creating permanent GitHub token for Travis to push to the repository… done!
✓ Checking the status of your repository in Travis… done!
✓ I didn't find your repository in Travis; syncing Travis with GitHub… done!
✓ Checking the status of your repository in Travis… done!
✓ Your repository isn't active in Travis yet; activating it… done!
✓ Encrypting permanent GitHub token… done!
✓ Writing configuration to .travis.yml file… done!

⚠ You didn't already have a .travis.yml file, so I created one for you.
  For more information about the file, see:
    http://docs.travis-ci.com/user/customizing-the-build/

You're ready to auto-deploy using Travis!  Just commit the changes
in .travis.yml and push the commit to the origin/master branch:

git add .travis.yml
git commit -m"configure Travis to auto-deploy to GitHub Pages" .travis.yml
git push origin master

Then visit https://travis-ci.org/mykmelez/offline-app/builds to see the build status.

After configuring the repository, add and commit the changes to .travis.yml and push the master branch to the origin remote on GitHub to make Travis build and auto-deploy your app:

git add .travis.yml
git commit -m"configure Travis to auto-deploy to GitHub Pages" .travis.yml
git push origin master

You can see the status of a build/deployment at https://travis-ci.org/USERNAME/REPOSITORY/builds. For example, the status of builds for https://github.com/mykmelez/eggtimer/ is at https://travis-ci.org/mykmelez/eggtimer/builds.

Once configured, Travis deploys successful builds via gulp deploy. You can change the deploy command by editing your .travis.yml file.

Note: Oghliner needs your GitHub credentials to create the token, and the token gives Travis limited access to your GitHub account. Specifically, the token provides the public_repo scope, which gives Travis "read/write access to code, commit statuses, collaborators, and deployment statuses for public repositories and organizations." For more information about personal access tokens, see Creating an access token for command-line use.

Build Process Integration

You can integrate Oghliner into your Node-based build process via its module interface. This is particularly helpful for the offline and deploy commands, which you'll call each time you change your app and rebuild it.

To do so, first install Oghliner locally and save it to your dependencies:

npm install --save oghliner

Then require the module in your build script and call its offline function, passing options to configure its behavior. For example, if you use Gulp to build your app, you could add code like this to your gulpfile.js:

var oghliner = require('oghliner');

gulp.task('offline', function() {
  return oghliner.offline({
    rootDir: 'dist/',
    fileGlobs: [
      '**/*.html',
      'js/**/*.js',
    ],
  });
});

gulp.task('deploy', function() {
  return oghliner.deploy({
    rootDir: 'dist/',
  });
});

Then you could invoke gulp offline && gulp deploy to offline and deploy your app.

Gulp Integration

If you used Oghliner to bootstrap your app, then it already has a gulpfile.js with tasks for building, offlining, and deploying your app. To use it, install Gulp globally:

npm install --global gulp

Then invoke gulp build && gulp offline && gulp deploy to build, offline, and deploy your app:

gulp build && gulp offline && gulp deploy

oghliner's People

Contributors

brendandahl avatar darkwing avatar delapuente avatar marco-c avatar mykmelez avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

oghliner's Issues

figure out how to test the deploy command

We need to figure out how to test the deploy command.

We can create a local Git repository and bootstrap an app in it to deploy, but then there's the question of how to test that deployment pushes it to its origin remote, which is normally on GitHub.

But deploy doesn't actually depend on GitHub itself, only on there being an origin "remote", which could presumably be another local repository. So it should be possible to test deploy without a dependency on a GitHub server.

We would just need to create two local repositories, one of which has the other as its origin; build the app in the first repository; and then test that the deploy command pushes the build output to the gh-pages branch in the origin repository.

include git revision/tag in template app

Building the template app should interject the Git revision ID (and tag, if the current revision is tagged) into the app somewhere subtle but visible for those who know where to look for it.

Then it'll be easier to see when the template app (and any app based on the template that doesn't remove the feature) has been updated in GitHub Pages, which could be useful when debugging the update flow (#5), especially since there's a delay of up to ten minutes (although it's usually less than a minute) between when you deploy an app and when it shows up on GitHub Pages.

Since the template app has a package.json file, and npm version updates the version in the file (while also tagging the current revision, per https://docs.npmjs.com/getting-started/publishing-npm-packages#updating-the-package), gulpfile.js could use var packageJson = require('./package.json'); to get the version number from the package for folks who are using npm version to version their apps, although that requires a manual step, and it'd be better to do this automagically.

Of note:

git rev-parse HEAD: the current revision ID
git describe --abbrev=0 --tags: the tag on the branch
git describe --abbrev=0 --tags --exact-match: the most recent tag on the current revision

(If we wanted to get fancy, we could distinguish between deployment of a clean working directory, where git status shows no changes, and deployment of a dirty one, where git status shows that there are changes that have not been committed yet.)

release template files to public domain

We should release the template files to the public domain, so web developers who use them to bootstrap an app can choose their own license for the app. In some cases, this will require replacing template code that we obtained from other sources (like the GitHub Pages generator). The headers for files in the public domain are at the bottom of this page: https://www.mozilla.org/en-US/MPL/headers/.

tell users using CLI deploy to add .gh-pages-cache to .gitignore

When you deploy an app with the CLI, Oghliner configures gh-pages to clone the repository to .gh-pages-cache, so we should tell users to add that directory to .gitignore.

That's because gh-pages clones to a canonical location by default, but it leaves the clone around, and it fails if you try to deploy two different apps using the same clone dir.

That isn't a problem when you install Oghliner locally and use it from a build script, because the canonical location is within the app's own directory, f.e. node_modules/gh-pages/.cache.

But it is a problem when you install Oghliner globally and use the CLI, because then the canonical location is a system directory, f.e. /usr/local/lib/node_modules/oghliner/node_modules/gh-pages/.cache, so gh-pages would fail when you tried to deploy a second app.

Thus the custom local clone directory. Which shows up as an untracked directory in git status, and which we should therefore tell users to .gitignore.

await sync if it's already in progress when configure tells Travis to sync

When the configure command doesn't find your repository in Travis, it tells Travis to sync it and then waits for the sync to complete (periodically polling Travis to find out the status of the sync). But if Travis is already syncing your repository when configure tells it to sync, then the travis.users.sync.post call throws { message: 'Sync already in progress. Try again later.' }, and the command doesn't catch that error, so it simply reports the error and dies:

Checking the status of your repository in Travis…
I didn't find your repository in Travis. Syncing Travis with GitHub…
{ message: 'Sync already in progress. Try again later.' }

It should instead catch that error and call the travisAwaitSyncing polling function in that case, just as it does in the case of the repository not being found.

make service worker ignore query parameters when matching URL to resource in cache

Oghliner doesn't set sw-precache's ignoreUrlParametersMatching option, and that option's default value of [/^utm_/] means that sw-precache takes most query parameters into account when deciding whether or not to serve a resource from its cache.

That's a problem for apps like presentations using Shower, which set query parameters like full and lang in client code to store state in a way that is bookmarkable. If a user loads an app page from a URL with a query parameter that wasn't present when the app was initially offlined, then the service worker will not serve the page from the cache, so the load will fail if the user is offline.

To solve that problem, offline should set sw-precache's ignoreUrlParametersMatching option to [/./] so it ignores all query parameters when matching URLs to fetch against URLs in its cache.

(Being able to do this for all apps automagically is one of the benefits of focusing on the GitHub Pages use case. Since GitHub Pages sites are always static, and query parameters never affect the result of a GET request for a GitHub Pages resource, we can always do this when offlining apps specifically for deployment on GitHub Pages.)

use yargs (or minimist) after all

c19f3ff introduced gulp.env usage to gulpfile.js, but gulp complains: "gulp.env has been deprecated. Use your own CLI parser instead. We recommend using yargs or minimist." So I guess we should use yargs (or minimist) after all. Note that cli.js uses commander, but that seems like overkill and a mismatch for this use case.

set sw-precache cacheId option to unique name

sw-precache allows you to specify a cacheId to distinguish between apps hosted at the same origin/path. That isn't a problem for apps hosted on GitHub Pages, since each app is hosted at a unique path (assuming a one to one relationship between an app and a GitHub repository). But it can be a problem when testing an app locally, if you typically start a local web server whose root dir is an app's build output dir, such that you load it from http://localhost/.

To avoid that problem, we should set cacheId to a unique name, like the name of the repository on GitHub.

specify commit message on deploy/auto-deploy

When deploying an app, it should be possible to specify a message for the commit to the gh-pages branch. And when auto-deploying from Travis, the auto-deploy script should derive that message from the one for the commit to master that triggered the deployment.

support specifying files to deploy

When developing a simple app that doesn't require a build step, it's tempting to put the app's files at the top-level of its repository, since that is the simplest possible directory layout:

  • my-project/
    • gulpfile.js
    • index.html
    • script.js
    • style.css

And Oghliner can do a reasonable job of offlining such an app, since you can specify that rootDir is the current directory (./) and put the list of files to offline into fileGlobs.

But deployment is problematic, since the gh-pages submodule that Oghliner uses to deploy to GitHub Pages has a more limited mechanism for specifying the files to deploy, a src option that is a single minimatch string, and Oghliner doesn't currently expose it.

So if you deploy the current directory for such an app, then gh-pages will also deploy gulpfile.js and any other meta-file that is present in the directory even though it isn't actually part of the app itself.

Of course it's a good practice to put app files into a subdirectory like app/ and then build them into another directory like dist/ (even if "build" simply means copying the files). And the template does that by default.

But Oghliner should nevertheless support specifying the files to deploy at least to the extent that supports specifying the files to offline, i.e. via a set of file globs, to satisfy the "simple app" use case (and presumably others).

make it easier to configure auto-deploy for *upstream* repo

The configure command assumes that you want to auto-deploy merges to the master branch on the origin remote.

But for projects with multiple contributors, it's common for a user to fork the original repo and clone their fork, such that the origin remote points to their fork, and then create an upstream remote for the original repo, as described in GitHub's Fork A Repo documentation.

And it's actually more interesting to configure auto-deploy for the original, upstream repo, since that's the one in which people will more often merge changes via pull requests and want those merges to trigger auto-deploy.

But it's hard to do that right now. You have to either temporarily rename your remotes so the original remote is named origin (which actually sounds like a better name for it, although it is unconventional) or clone the original remote and configure that clone.

We should make it easier to configure auto-deploy for the upstream repo.

For example, we could prompt the user to choose the remote for which to configure auto-deploy, selecting the upstream remote by default if one is defined.

(A related problem is that you can't configure a repo to auto-deploy both the original repo and a fork, since the encrypted GitHub auth token in .travis.yml is repo-specific, and there can be only one auth token in the Travis build environment variables. But that's a different issue.)

require Node 0.12+

At least the child_process.execSync call in the configure command requires Node 0.12+, and there may be other code that assumes Node 0.12+, so Oghliner's package.json file should explicitly require that version.

figure out how to test the configure command

We need to figure out how to test the configure command. I confess to having no good idea how to do this.

The command depends on the GitHub and Travis APIs, so perhaps we could stub those.

Or I guess we could create a GitHub user specifically for testing, then encrypt that user's username and password for Travis. That still wouldn't allow us to test 2FA, nor perhaps some other aspects of the configuration flow, but it could test many aspects. However, it would be a live test, dependent on the status of the GitHub and Travis API endpoints.

consider supporting conditional/lazy caching of some assets

Sometimes images are conditionally downloaded, particularly responsive images that use <img srcset> to make the browser determine which one of a set to load, as described in Responsive Images: If you’re just changing resolutions, use srcset..

I'm not sure how often they're used in apps generally, much less offline-able apps hosted on GitHub Pages, but they're mentioned in Introduction to Service Worker at least, so at least some folks have spent some time thinking about how to handle them when using Service Worker to cache app assets.

Since the promise of Oghliner is to simply cache your whole app offline, I'm loathe to complicate the interface with conditional/lazy caching of assets. But perhaps it would make sense to include some support for caching one of several variants of an image.

For example, we might let users specify assets that shouldn't be cached immediately but should be cached once requested, something like:

{
    "filesToCacheLazily": [
        "images/logo-lo-res.png",
        "images/logo-hi-res.png",
    ],
}

Then the worker can cache whichever variant the browser ends up requesting.

Before we do something like this, however, we should have better use cases for it, ideally in the field.

consider supporting Grunt

Oghliner currently supports building with Gulp, but a lot of webdevs use Grunt, so we should consider supporting it. I'm only a little hesitant because it would be easy for support for the two systems to get out of sync, such that we support features in one that we don't support in the other.

Alternately, we might support the same set of features, but one system is a second-class citizen (whether explicitly or implicitly), such that it doesn't get the same level of testing and maintenance.

Note that sw-precache's README - Example describes the project as having a "sample gulpfile.js", with many changes over the history of the project, but only a "basic Gruntfile.js", with only four commits in its history as of this moment.

use formatting to make command output clearer

Oghliner's output is quite simplistic, just plain text without any special formatting. By contrast, other popular command-line tools, like Yeoman (npm install -g yo && yo), use color, boldness, and other stylistic formatting to clarify the text they output to the console. (Yeoman even prints ASCII art, although that's a bit extraneous.)

We should evaluate the output of the various Oghliner commands for opportunities to use formatting to make them clearer.

  • Use gutil.log to log messages (#131)
  • Print colored warnings and errors (#131)
  • Clarify output of configure (#148)
  • Print large file warning in yellow (#151)
  • Print all suggestions in blue (#156)
  • Clarify output of bootstrap (#160)
  • Clarify output of integrate (#161)
  • Clarify output of offline (#162)
  • Clarify output of deploy (#163)

redesign Oghliner website

The Oghliner website at https://mozilla.github.io/oghliner/ was designed using the GitHub Pages generator, and it could use a redesign (both graphic design and information design) in order to better convey what the project is about and how to go about using it.

This used to be identical to #7, because we were using the website as the template for bootstrapping new apps. But now we've separated those two things, so the website is in app/, while the bootstrap command uses the template in templates/. Thus redesigning the website is now a separate concern from redesigning the template.

configure Travis to configure Git with name/email of user

Travis auto-deployment requires Git to be configured with the name and email address of the user who will commit the deployed files to the gh-pages branch, and while the name can be arbitrary, the email address has to be the one associated with the user whose token is being used to authorize the deployment.

But the configure task doesn't configure Travis to configure Git with this info. It should do so.

For example, in my .travis.yml file, I manually configure Travis to configure Git via a section like the following:

before_script:
  - 'git config --global user.name "Travis-CI"'
  - 'git config --global user.email "[email protected]"'

The configure task should determine the user's email address and use it to configure .travis.yml similarly.

make Travis show output of deploy command by default

The output of the command that configure adds to the after_success script in .travis.yml is collapsed by default in Travis build logs. So you only see a single line like this when you first view a log (like this one):

2.87s$ [ "${TRAVIS_PULL_REQUEST}" = "false" ] && [ "${TRAVIS_BRANCH}" = "master" ] && gulp deploy

You can see the rest of the log by clicking on the disclosure widget (expander arrow) to the left of that line item, after which you see the output of the command, which might look something like this:

[23:31:35] Using gulpfile ~/build/mykmelez/mc-choir/gulpfile.js
[23:31:35] Starting 'deploy'...
[23:31:37] Finished 'deploy' after 1.91 s

But it would be better if Travis showed you that output by default, so you didn't have to expand the command to see it.

simplify process of integrating offline-manager.js into existing app

When integrating offlining/deployment into an existing app, the clunkiest part of the process is integrating the offline-manager.js script into the app. You have to copy the script from an obscure location in local or global node_modules/ into the appropriate place in the app's source tree:

> cp node_modules/oghliner/app/scripts/offline-manager.js app/scripts/

Then you have to add a <script> tag for it to the appropriate place in your HTML page(s) or page template(s):

<script src="scripts/offline-manager.js"></script>

By contrast, when bootstrapping an app from the template, this happens automagically.

We can't make integration as automagic as bootstrapping, but it can be easier than it is now.

First, we should add a command that copies offline-manager.js to the appropriate place without requiring the user to know its source location, something like oghliner *some-command* *destination*, i.e.:

> oghliner copy-offline-manager-to app/scripts/
Copying offline-manager.js to app/scripts/… done.

Second, we should consider auto-detecting the HTML page(s)/template(s) for the app and offering to auto-add the <script> tag to them, if we can guess the right location reliably enough to make it more helpful than a hindrance:

I've detected your app at app/index.html.  It needs to load the script offline-manager.js
in order to register the service worker that offlines your app.  I can add a <script> tag
that loads that script into app/index.html.

Add &lt;script tag to app/index.html (Y/n)?

Otherwise, we should at least tell the user they need to add the tag themselves and print it to the console so they know what to add:

Your app needs to load the script offline-manager.js in order to register the service
worker that offlines your app. To load the script, add this line to your app's HTML
page(s)/template(s):

    <script src="scripts/offline-manager.js"></script>

Perhaps we should do both of those things with a single command, f.e. integrate, which first prompts the user to specify the destination for offline-manager.js and then offers to add the <script> tag to the right file(s) if it thinks it can determine those reliably.

(An integrate command for existing apps, like its bootstrap counterpart for new apps, might eventually prompt the user to do other optional things, like configure Travis auto-deploy.)

Travis auto-deploy fails with "invalid username or password"

Travis auto-deploy appears to have regressed. After reconfiguring this repository, an auto-deploy attempt fails with "Invalid username or password":

2.05s$ [ "${TRAVIS_PULL_REQUEST}" = "false" ] && [ "${TRAVIS_BRANCH}" = "master" ] && gulp deploy
[20:39:53] Using gulpfile ~/build/mozilla/oghliner/gulpfile.js
[20:39:53] Starting 'deploy'...
[20:39:53] Deploying "update GitHub token for Travis auto-deploy"
[20:39:54] 'deploy' errored after 917 ms
[20:39:54] ProcessError: remote: Invalid username or password.
fatal: Authentication failed for 'https://github.com/mozilla/oghliner.git/'
at ChildProcess. (/home/travis/build/mozilla/oghliner/node_modules/gh-pages/lib/git.js:48:23)
at ChildProcess.emit (events.js:110:17)
at maybeClose (child_process.js:1015:16)
at Process.ChildProcess._handle.onexit (child_process.js:1087:5)
Done. Your build exited with 0.

flesh out Oghliner template app

The built-in Oghliner template app in its app/ subdirectory, which GitHub publishes to https://mozilla.github.io/oghliner/, is too basic. It doesn't contain any images nor CSS files, so there's no guidance about where to put those files in the source directory structure. Nor are there other common files, like favicon.ico or robots.txt.

Nor does it even explain the Oghliner project or point to more info about it. I might not expect that info in a template that gets created by a command-line tool, but I would expect it in a template that is itself published as an app, which is the case here.

By contrast, compare to the app/ subdirectory of the Yeoman gulp-webapp template (mkdir my-app && cd my-app && yo gulp-webapp), which contains:

apple-touch-icon.png
favicon.ico
fonts
images
index.html
robots.txt
scripts
scripts/main.js
styles
styles/main.scss

Or the Angular template (mkdir my-app && cd my-app && yo angular), whose app/ subdirectory has:

404.html
favicon.ico
images
images/yeoman.png
index.html
robots.txt
scripts
scripts/app.js
scripts/controllers
scripts/controllers/about.js
scripts/controllers/main.js
styles
styles/main.scss
views
views/about.html
views/main.html

Or the template built into Ember (npm install -g ember-cli && ember new my-app), whose app/ subdirectory includes:

app.js
index.html
router.js
styles
styles/app.css
templates
templates/application.hbs

We should flesh out the Oghliner template app with a set of files that is more representative of the apps that users are going to build. Also, since the template app is itself published by GitHub as an app, we should include content that is useful to users who browse that site.

Perhaps we can use GitHub's Automatic Page Generator to generate an attractive basic page for the template, per Instantly Beautiful Project Pages. And then add content similar to the content in the README (or a basic description of the project along with a link to the README for further info).

consider maintaining list of files to offline in separate file

This is an interesting request to be able to maintain a list of files to offline that is separate from the service worker: https://jakearchibald.com/2014/offline-cookbook/#comment-2189777576.

It's basically the appcache use case. I think i recall @marco-c suggesting that we could enable users of our various offlining tools to maintain a list of assets in a separate file. We'd still need to modify the service worker somehow on each update, since service worker updates depend on that file changing; but at least we could minimize such changes.

On the other hand, perhaps the list of file globs in the build file is sufficient to satisfy users who want to maintain such a list, since it is essentially that list of files, and it can consist entirely of specific files; it doesn't have to use globbing characters.

npm install fails with 'shasum check failed' for multiparty module

This recent build on Travis failed a hash check for the multiparty module:

make: Entering directory /home/travis/build/mozilla/oghliner/node_modules/travis-encrypt/node_modules/ursa/build' CXX(target) Release/obj.target/ursaNative/src/ursaNative.o CXX(target) Release/obj.target/ursaNative/src/asprintf.o SOLINK_MODULE(target) Release/obj.target/ursaNative.node COPY Release/ursaNative.node make: Leaving directory/home/travis/build/mozilla/oghliner/node_modules/travis-encrypt/node_modules/ursa/build'
npm ERR! Linux 3.13.0-40-generic
npm ERR! argv "/home/travis/.nvm/versions/node/v0.12.7/bin/node" "/home/travis/.nvm/versions/node/v0.12.7/bin/npm" "install"
npm ERR! node v0.12.7
npm ERR! npm v2.11.3

npm ERR! shasum check failed for /tmp/npm-2066-5cbdf385/registry.npmjs.org/multiparty/-/multiparty-2.2.0.tgz
npm ERR! Expected: a567c2af000ad22dc8f2a653d91978ae1f5316f4
npm ERR! Actual: 6055a8df1315811def83ca6585d46cd476845953
npm ERR! From: https://registry.npmjs.org/multiparty/-/multiparty-2.2.0.tgz
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR! https://github.com/npm/npm/issues

npm ERR! Please include the following file with any support request:
npm ERR! /home/travis/build/mozilla/oghliner/npm-debug.log

The command "npm install" failed and exited with 1 during .

Your build has been stopped.

A rebuild succeeded, so you won't see that error on Travis now, but here's the log file:

log.txt

Presumably this is a rare build failure, and nothing to be concerned about, but filing it here just in case it happens more frequently, and we should dig into it further.

automagically enable travis for repo in config task

The "config" task to configure Travis auto-deploy currently requires you to go to your Travis profile page and activate the repository. But the Travis API for Hooks appears to be a way to do this programmatically, so the config task should be able to do it automagically.

It's a bit complicated to use that API, because the task would first have to request a GitHub token with a particular set of scopes (that are different from the single "public_repo" scope for the GitHub token we create to enable auto-deploy), then use that token to request a Travis token, which it could then use to authenticate with the Travis API.

Fortunately, there's the travis-ci module, which simplifies the process. After prompting you for your GitHub username/password, the task could use the authenticate method to get the Travis token, then use that to get the hooks, find the one for the repository in question, and activate it.

The config task already prompts you for your username/password, however, and it shouldn't do it twice. Unfortunately, the current prompt is generated by the get-github-token module, which doesn't expose the credentials. So we may have to reimplement that module's functionality in order to be able to prompt once to generate both tokens.

(We could also request a single token with all of the scopes that are required by either use case, but that would make a token compromise more damaging. Also, I think the GitHub token for getting a Travis token is a temporary one, while the GitHub token for auto-deploy is permanent.)

turn off Jekyll processing with .nojekyll file

GitHub Pages processes output with Jekyll by default, but it isn't clear how likely it is for offline web apps to use this feature, and processing files with Jekyll can introduce unexpected behavior. It may also cause GitHub Pages to take longer to deploy your app than it otherwise would.

So the template should include a .nojekyll file to turn off Jekyll processing. And it should configure the gh-pages module to include that file when deploying to GitHub Pages.

Of course we should document this feature so folks know about it (and how to disable it after bootstrapping apps that are going to use Jekyll).

support an app update user flow

Oghliner should support an app update user flow, so an app that uses Oghliner receives a message when an update is available and can prompt the user to update the app (if it chooses to do so, and at the appropriate time).

The flow would be optional, so an app that wants updates to be completely automatic and silent can simply ignore the message and never prompt the user to update the app, which will then update itself automatically whenever the service worker is updated by the browser (which is currently the default and only supported flow).

See sw-precache's demo service-worker-registration.js file for inspiration, but ideally we would not have apps add their custom logic to our own equivalent (service-manager.js) but rather expose the registration to their code in a way that makes it possible for them to listen for its events and handle them in their own script.

cache large files with warning (and option for excluding them)

sw-precache sets its maximumFileSizeToCacheInBytes to 2MiB by default, which breaks offlining of large image files, even if you explicitly include them via file globs. I understand the reasoning behind avoiding the caching of large files, especially if they aren't actually necessary for your app to function. But it does complicate and undermine the promise of Oghliner: that you can point it at your app's assets, and it'll ensure they're available offline.

Instead of simply excluding large files, we should warn the user when offlining them and give them the opportunity to exclude them from the cache, f.e. by allowing entries in fileGlobs to exclude files instead of merely including files. Then a user who is warned about a large file can choose to cache it anyway by doing nothing or explicitly exclude it to prevent it from being cached, f.e. by using a leading exclamation point to negate the match à la minimatch:

{
    "fileGlobs": [
        "!images/my-large-file.png",
    ]
}

We might even let users explicitly include a file to suppress the "large file" warning:

{
    "fileGlobs": [
        "images/my-large-file.png",
    ]
}

Remove boostrap npmInstall option (or enable it on Travis)

It was introduced in #63, because without it the test is too slow or fails on Travis.
We should either remove the option altogether and always install dependencies (which is the option I prefer) or enable it only when running tests on Travis, where we don't really need the tests to be too fast (and installing dependencies is usually very fast on Travis, so it shouldn't really be a problem).

figure out how to test the *offline* command

We need to figure out how to test all the commands. deploy and configure might be harder, because they interact with servers; but offline at least should be doable, so let's start there.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.