torus-tools / cli

An easy to full feature command line interface for Torus Tools. Made with OCLIF

Torus CLI

Mutli CLI for torus tools

• Usage
• Commands

Usage

$ npm install -g @torus-tools/cli
$ torus COMMAND
running command...
$ torus (-v|--version|version)
@torus-tools/cli/0.0.124 linux-x64 node-v14.4.0
$ torus --help [COMMAND]
USAGE
  $ torus COMMAND
...

Commands

• torus content ACTION [FILES]
• torus help [COMMAND]
• torus init
• torus stack ACTION [SETUP]

torus content ACTION [FILES]

List/download/upload/delete all of your content (or the specified files).

USAGE
  $ torus content ACTION [FILES]

ARGUMENTS
  ACTION  (list|download|upload|delete) given action to carry out with the content of your site

  FILES   local paths or object keys of the files/directories you want to upload/download to/from your bucket. For
          example suppose theres a directory img inside the cwd the path of image1.jpg would be img/image1.jpg. For
          local files the root is the current working directory unless specifiecd otherwise with the -i flag. By
          default, if no paths are provided all files/dirs in the root will be used.

OPTIONS
  -a, --all            Upload all files. By default only updated files are uploaded.
  -d, --domain=domain  Domain of your site (i.e. yoursite.com).
  -i, --input=input    Path of the root directory of your project (if different to the current working driectory).
  -o, --output=output  [default: ./] Path to save downloaded content into. Default is the current working directory.
  -r, --reset          Reset the cache in all edge locations for the given files.
  -s, --sort           Sorts listed files by last modified date.

DESCRIPTION
  ...
  By default only modified files are uploaded; to upload all files provide the --all flag. To automatically update cache 
  in cloudfront for the given files add the --reset flag.

See code: src/commands/content.js

torus help [COMMAND]

display help for torus

USAGE
  $ torus help [COMMAND]

ARGUMENTS
  COMMAND  command to show help for

OPTIONS
  --all  see all commands in CLI

See code: @oclif/plugin-help

torus init

Configure torus globally in your machine, or on a per-project basis

USAGE
  $ torus init

OPTIONS
  -d, --domain=domain
      The valid desired/existing domain of your site i.e. yoursite.com

  -g, --global
      Create a global torus configuration file. The command will guide you through the steps to generate the required API 
      keys for each of your desired providers, set up your global environment variables and your deisred default settings.

  -p, --providers=aws|godaddy
      Desired cloud/domain providers to be used with torus. You must have an existing account in all of the providers you 
      choose.

  -r, 
  --region=us-east-2|us-east-1|us-west-1|us-west-2|af-south-1|ap-east-1|ap-south-1|ap-northeast-2|ap-southeast-1|ap-sout
  heast-2|ap-northeast-1|ca-central-1|cn-north-1|cn-northwest-1|eu-central-1|eu-west-1|eu-west-2|eu-west-3|eu-north-1|me
  -south-1|sa-east-1|us-gov-east-1|us-gov-west-1
      [default: us-east-1] Desired AWS region

  -u, --user=user
      [default: torus_admin] Desired name for the AWS IAM user

DESCRIPTION
  ...
  The init command helps you configure torus in your site/project. Providing the -g (--global) flag helps you configure 
  torus globally (for all of your projects). When using the torus CLI, you can always overwrite global settings by 
  including a project config file. You can also overwrite global environment variables by including a .env file. If you 
  are using the init command without the -g flag make sure to run it from the root of your project.

See code: src/commands/init.js

torus stack ACTION [SETUP]

Deploy static sites to AWS

USAGE
  $ torus stack ACTION [SETUP]

ARGUMENTS
  ACTION  (create|update|import|delete|pull|push) choose an action to perform. you can create, update, import your stack
          or upload files to your bucket.

  SETUP   (dev|test|prod|custom) [default: dev] setup for the site - dev, test, production or custom

OPTIONS
  -b, --bucket=aws|true                   Enables a cloud storage bucket to be used as the websites origin. You can
                                          provide this flag without the =string to use aws s3.

  -c, --cdn=aws|true                      Add a CDN to your site. CDNs enable faster website load times by caching your
                                          content around the globe (the edge). You can provide this flag without the
                                          =string to use aws Cloudfront.

  -d, --domain=domain                     The domain of your site i.e. yoursite.com

  -e, --error=error                       name of the error document. Default is error.html

  -i, --index=index                       name of the index document. Default is index.html

  -o, --overwrite                         By default, torus always reads your template in the cloud and only adds
                                          changes (updated resources or additional resources). If you want to eliminate
                                          the resources that arent prvoided in the CLI flags you can add this flag.

  -p, --publish                           Publish the sites content

  -r, --registrar=aws|godaddy|other|true  The current domain name registrar of your sites domain. Using AWS or godaddy
                                          enables automatic namserver updates if the DNS provider is different to the
                                          registrar. Selecting other will require manual nameserver updates. true
                                          evaluates to other.

  -s, --ssl=aws|true                      Enables https by creating and validating an SSL certificate for your site. You
                                          can provide this flag without the =string to use aws certificate manager.

  -w, --www=true                          creates a www reroute bucket.

  --dns=aws|godaddy|other|true            Desired DNS provider for your site. The aws option adds a route53 hosted zone
                                          to your stack. You can provide this flag without the =string to use aws.

DESCRIPTION
  ...
  Deploy static sites to the AWS Cloud using Cloudformation templates.

See code: src/commands/stack.js

cli's Issues

file upload loop exits before finishing

The forEach loop used is synchronous and the last index is usually returned before the end of the loop.

instead use a for(i=0;i<files.length;i++) classic loop

make a test for the createPaths function

it recieves a series of files and then creates a bucnh of folders.

us fs-mock to mock the fs for tests.
https://www.npmjs.com/package/mock-fs

Add custom help

Add aski art of Torus Host in the help command.

The aski art should only have the text "Torus Host" no need to include the logo

https://oclif.io/docs/help_classes

start command creates an unnecessary directory called lib

the live reload server creates unnecessary directory called directory called lib

create an arjan_config directory

problem: arjan creates a bunch of folders and files when it is ran. this can be confusing for users. istead arjan should only create 2 things. dep_pack and arjan_config

arjan_config will contain everything for arjan including:

• defualt config for optimize
• default config for audit
• localize locales
• localize exports
• temaplate for deploy
• changesets for deploy

Add a quick setup for AWS iam user in the init function

copy the build function from super easy forms that opens aws console in the browser.

Use non-blocking methods from oclif

For errors use this.error which exist the command. Also this.log is a lot better than console.log because its non blocking.

https://oclif.io/docs/commands#other-command-options

Add the stop and close commands

If the commands
arjan stop or arjan close are used they stop the development server.

Upload command path error

Upload command when provided the directory flag still uploads the original directory; should only upload files inside that directory.

the upload method has a parameter dir for this. just need to supply the parameter in the upload command

Analytics for Nodejs

Adding analytics to Nodejs to collect data on how users are using the command line.

https://www.npmjs.com/package/analytics-node

Add a warning when deleting objects or stacks

I want to be warned when im going to delete a stack or delte objects just in case i miss-typed something or ran the command by accident...

open index document in the browser when read

For the deploy command, when ready open the index document.

the url to open will be different depending on dev/test/prod.

in the case of arjan localize open the translated document if theres a translation.

Local project cant find babel dependency and some of the loaders

This can be solved by using the resolveLoaders property and setting the modules attribute to use an absolute path (path(__dirname, ''))

https://webpack.js.org/configuration/resolve/#resolvemodules

------IGNORE-------
create a package.json file in the project like the following one

{
  "name": "project,
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "directories": {
    "lib": "lib"
  },
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/gkpty/project.git"
  },
  "author": "",
  "license": "ISC",
  "bugs": {
    "url": "https://github.com/gkpty/project/issues"
  },
  "devDependencies": {
    "css-loader": "^3.6.0",
    "style-loader": "^1.2.1",
    "@babel/core": "^7.10.3",
    "@babel/preset-env": "^7.10.3",
    "babel-loader": "^8.1.0"
  },
  "homepage": "https://github.com/gkpty/project#readme"
}

1. run npm install programmatically
   or
2. recursively copy all files and directories

Feature: Add quality parameter

I want to be able to vary the quality of the compression of my images.

add a flag called quality q type string

add command description for start and init

Print report when error happens

Describe the bug
When an error occurs in on of the parsers, Arja Optimize doesn't print the report of what had al ready optimized.

To Reproduce
Steps to reproduce the behavior:
Get an error from one of the modules.

Expected behavior
Even if there is an error, arjan optimize should to print the report of what it has optimized as well as the error.

Desktop (please complete the following information):

• OS: mac os
• Browser: chrome

Add notifications for when the deploysite command has finished running.

Oclif has a plugin for this. sounds pretty cool

https://oclif.io/docs/notifications

HTTPS for local server

Have HTTPS in local server for better testing.

I don't know if it's possible with webpack server. Would need to check

option to run dev server on different dirs

i want to run the dev server for other directories, precisely for dep_pack.

Prettify JSON

The task consists of creating a new method that stringifies and pretyfies the JSON

before saving the JSON locale files:

1. plit at every comma
2. map the array adding a '\n' at the end of each element
3. stringify the JSON and save

then the method can be used inside of fs.writeFile(prettyJsonStringify(JSON))

this method will be used in:

• localize - createLocale
• init - webpack config
• init - arjan ignore
• optimizeConfig

Add the serve command

has 1 argument: index document
1 flag: port, default is 8080

Save and read from a local webpack config inside the arjan_config

To fix this need to place a condition wherever webpack.config is used.
If webpack.config isnt present in the root of the cwd then use the absolute path(__dirname) of webpack.config inside the webpack dir.

the webpack config object should be saved locally. in each project

to do this we will have to require an absolute path.
path.join(__dirname, 'arjan_config/webpack.prod.js')

optimize and serve should both read from the locally saved webpack config and then modify the object based on flag values

remove dotenv dependency of CLI

CLI has a dependency on dotenv that is not being used.. remove this unnecessary dependency.

add conversion of images to webp

should edit the config object in optimize command depedning on the flags provided either add the imagemin webpack plugin or not.

if(css) //add the optimize css plugin
if(images && webp || webp) //add the imagming plugin with the webp plugin
else if (images) //add the imagemin plugin with the pngquant and the mozjpeg settings

ImageminWebpackPlugin = require("imagemin-webpack-plugin").default;
ImageminWebP = require("imagemin-webp");
CopyWebpackPlugin = require("copy-webpack-plugin");

module.exports = {
  // Omitted required entry, output, and loader configs for brevity...
  plugins: [
    new CopyWebpackPlugin([{
      from: "./src/images/*.png",
      to: "./images/[name].webp"
    }]),
    new ImageminWebpackPlugin({
      plugins: [
        ImageminWebP({
          quality: 75
        })
      ]
    })
  ]
};

Bug: Image loader needed for classes with images

Describe the bug
error when running arjan start

ERROR in ./img/port.jpg 1:0
Module parse failed: Unexpected character '�' (1:0)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders

had an image background class inside the stylesheet.

to solve this install file-loader and add the following in webpack.dev

test: /\.(gif|svg|jpg|png)$/,
loader: "file-loader",

Add the audit command

has 1 argument: index document
1 flag: port, default is 8080

Ad webP replacement

parse the HTML and replace images for pictures

Add hot reload in start

Add arjan_config directory

arjan creates a bunch of files. having them laying around in the root dir individually is messy. should create everything in a dir called arjan_config

Delete Responsive command from Arjan Optimize

The command -r responvess is not ready yet.

Remove from the CLI in arjan optimize.

Save stylesheets for audit reports separately

The HTML report produced by lighthouse has the css inline.

When making reports for multiple pages its a better practice to sepparate the stylesheet and have all the HTML reports read from the same stylesheet.

am software

Separate live reload into its own plugin/repo

Sepparate live reload into its own repository

Add domain command

the domain command will make syncing a given domain name registrar easier.

Webpack adds unecessary commonJS headers to scripts

Describe the bug
Webpack adds an unecessary commonJS header to all js files. This is not needed for the regular scripts written in vanilla JS

In optimize, after making the build, should remove all of the commonJS headers on the scripts. (replace with "")

Check for other processes running on the port

https://www.npmjs.com/package/tcp-port-used

simply try to start the server and listen then check for the error message if its in use.

https://stackoverflow.com/questions/19129570/how-can-i-check-if-port-is-busy-in-nodejs

Arjan audit good scores should be green

Describe the bug
In arjan Audit, even if you get a 90+ score, it's still in red.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

prettify all JSON

before saving the JSON locale files:

1. split at every comma
2. map the array adding a '\n' at the end of each element
3. stringify the JSON and save

Integrate CLI with Travis CI

Return detail of the recommnedations in the audit

Return the details from the lighthause audit for certain items like longest contentful paint

reload browser window with the serve command

https://github.com/cyrus-and/chrome-remote-interface

const CDP = require('chrome-remote-interface');

async function example() {
    let client;
    try {
        // connect to endpoint
        client = await CDP();
        // extract domains
        const {Network, Page} = client;
        // setup handlers
        Network.requestWillBeSent((params) => {
            console.log(params.request.url);
        });
        // enable events then start!
        await Network.enable();
        await Page.enable();
        await Page.navigate({url: 'https://github.com'});
        await Page.loadEventFired();
    } catch (err) {
        console.error(err);
    } finally {
        if (client) {
            await client.close();
        }
    }
}

example();

const chromeLauncher = require('chrome-launcher');
const CDP = require('chrome-remote-interface');

/**
 * Launches a debugging instance of Chrome.
 * @param {boolean=} headless True (default) launches Chrome in headless mode.
 *     False launches a full version of Chrome.
 * @return {Promise<ChromeLauncher>}
 */
function launchChrome(headless=true) {
  return chromeLauncher.launch({
    port: 9222,
    chromeFlags: [
      '--disable-gpu',
      headless ? '--headless' : ''
    ]
  });
}

(async function() {

const chrome = await launchChrome();
const protocol = await CDP({port: chrome.port});

const {Page, Runtime} = protocol;
await Promise.all([Page.enable(), Runtime.enable()]);

const url = "https://www.google.com/";

Page.navigate({url: url});

// Create a value to track which page we are on
let pageNum = 0;

// Wait for window.onload before doing stuff.
Page.loadEventFired(async () => {

  if (pageNum === 0) {
    const result1 = await Runtime.evaluate({expression: "document.querySelector('title').textContent"});

    // Prints "Google"
    console.log('Title of page: ' + result1.result.value);

    // Navigate to the About page
    const result2 = await Runtime.evaluate({expression: "document.querySelector('#fsl a:nth-child(3)').click()"});
    pageNum = 1;

  } else if (pageNum === 1) {
    const result3 = await Runtime.evaluate({expression: "document.querySelector('title').textContent"});

    // Prints "About Us | Google"
    console.log('Title of page: ' + result3.result.value);

    protocol.close();
  }

});

})();

Errors in stylesheets

Theres errors in stylesheets in the live reload server. it mergers all stylesheets. and the stylesheets are larger than before in optimize

Create a default arjan_ignore.json file

the arjan_ignore file gets created with arjan init and is read by all of the methods that use it.

the arjan_ignore file should include:

• arjan_config
• node_modules
• .env
• .gitignore
  & others

Add Auto Fix Feature

Arjan should be able to automatically fix the recommendations made by Webhint in your website.

Each fix in Webhint should be mapped to a specific function in Arjan optimize.

Webhint should run in Arjan Audit, then optimize would execute the fixes.

Add HTML report for audit

the audit should return an html report and the command should save this report with fs

Open the webpack dev server as a child process

Open webpack dev server as a child process on a new terminal.

This will allow you to close the child process with a specific command while on the parent process.

also you will be able to check if a child process is running at a particular port before starting the audit.

use execa:

https://www.npmjs.com/package/execa

webpack/webpack-dev-server#2168

Bug: arjan localize help command

Describe the bug
When you are in the root folder of a project and run 'arjan localize help' you get a response o f "setting up ... done"

Include Webpack inside the CLI dependencies

Add multiple providers

the provider flag will help the user choose how he wants to deploy their site.

options:

1. arjan
2. netlify
3. vercel

Arjan

Arjan will allow a user to mix and match cloud providers. Sort of like a terraform specially made for websites.
the arjan template will include the following definitions

{
    Domain: Namecheap | AWS | godaddy | google | other
    DNS: AWS | Domain | google | other
    Sotrage: AWS | tardigrade | google
    CDN: AWS | Cloudflare | google | keyCDN
    Certificate: AWS | letsEncrypt
    IPFS: true | false
}

Netlify

the deploy to netlify option will work using either

1. the netlify API
2. Netlify's AWS cloudformation custom Resource

• after creating the site in netlify you must create the deployment:
• for generating SHA1 hashes check out the netlify JS SDK

Vercel

The deploy to vercel option will work using the vercel REST API
In vercel you must:

1. upload all of the necessary files for deployment
2. Create a deployment

example of deploying a static site to vercel https://vercel.com/guides/deploying-charge-with-vercel

Save audits by date

Save audits by date in a JSON file that contains the dates as keys and the json report data as a value for each report.

then this data will be easy to plot.

Also save the html reports by date (with the same exact date_time as the id of the object).