GithubHelp home page GithubHelp logo

indigounited / automaton Goto Github PK

View Code? Open in Web Editor NEW
214.0 18.0 10.0 2.24 MB

Task automation tool built in JavaScript

Home Page: http://indigounited.com/automaton

License: MIT License

JavaScript 100.00%

automaton's Introduction

Automaton Build Status Bitdeli Badge

Task automation tool built in JavaScript.

Automaton

Why?

You often find yourself needing to do some repetitive operation, and this is usually the time to quickly bake some ad-hoc script. Still, from project to project you find yourself needing to reuse some task you had already previously created.

Automaton eases this process, allowing you to quickly set up an autofile, which describes what you want to do, by means of an ordered list of tasks that need to run for the task as a whole to be complete.

A little detail that makes Automaton a powerful tool, is that every autofile you create can itself be used by another autofile, turning the first one into a single task (imagine boxes within boxes). If you are curious, you can take a look at the source code, and check for yourself that even the tasks that Automaton provides built-in are simple autofiles.

Built-in tasks

automaton comes bundled with a few tasks to ease your own tasks.

Filesystem

  • chmod: Change mode of files
  • cp: Copy files and directories
  • mv: Move files and directories
  • mkdir: Make directories recursively
  • rm: Remove several files or directories
  • symlink: Create symlink

Scaffolding

Scaffolding tasks help you perform some typical tasks, like appending, replacing, and others, to placeholders in a template file. Any text file can be a template. These tasks will look for a {{placeholder_name}} inside the file, and perform the operation on it.

Miscellaneous

  • run: Run a shell command
  • init: Initialise an empty autofile
  • uglify (soon)
  • minify (soon)
  • concat (soon)

Installing

You can simply install Automaton through NPM, by running npm install -g automaton. This will install Automaton globally, and you will be able to execute automaton in your terminal.

If you only plan to use automaton programmatically, you can just install it locally.

Creating a task

An automaton task is a simple object, describing what the task will do.

Simple task

For illustration purposes, here's a simple autofile that just creates a folder and copies a file into it:

module.exports = function (task) {
    task
    .id('my-task')
    .name('My task')
    .do('mkdir', {
        description: 'Create the project root folder',
        options: {
            dirs: ['some_dir']
        }
    })
    .do('cp', {
        description: 'Copy some file',
        options: {
            files: {
                'some_file': 'some_dir/dest_file'
            }
        }
    });
};

More complete task

To illustrate most of the capabilities of automaton, here's a complete autofile with comments along the file:

module.exports = function (task) {
    // Everything bellow is optional
    task
    // Allows to reference this task by id
    .id('example_task')
    // A user friendly name
    .name('Example task')
    // The task author
    .author('IndigoUnited')
    // The task description
    .description('My example task')

    // Options definition: name, description, default value
    .option('dir1', 'The name of the folder')
    .option('dir2', 'Another folder name', 'automaton')
    .option('run_all', 'Run everything', false)
    .option('debug', 'Enable/disable', false)

    // Function to run before everything
    .setup(function (options, ctx, next) {
        // You can change existing options
        options.dir2 = options.dir2 + '_indigo';

        // And even define additional options; in this case we're defining a 'dir3' option,
        // which will be used by one of the subtasks.
        options.dir3 = 'united';

        next();
    })

    // Function to run afterwards
    // Note that this function will run, even if some of the substask failed
    .teardown(function (options, ctx, next) {
        // do something afterwards (e.g.: cleanup something)
        next();
    })

    // A list of subtasks that will run
    .do('mkdir', {
        description: 'Create the root and second folder',
        options: {
            // the option below
            // will have its placeholders replaced by
            // the value that it receives.
            dirs: ['{{dir1}}/{{dir2}}']
        }
    })
    .do('mkdir', {
        // Description messages can be generated according to the options
        // by using a string with placeholders or a function.
        description: function (opt) {
            return 'Creating ' + opt.dir1 + '/' + opt.dir2 + '/' + opt.dir3
        },
        options: {
            dirs: ['{{dir1}}/{{dir2}}/{{dir3}}']
        },
        // This 'on' attributes allows you to enable/disable a subtask just by
        // setting it to a falsy value.
        // In this case, we even used a placeholder, allowing us tto skip this subtask
        // depending on the run_all option.
        // Of course, you have can set it to something like 'false'.
        on: '{{run_all}}',
        // This 'fatal' attribute allows to bypass tasks that fail, just by setting
        // it to a falsy value.
        // In this case, we even used a placeholder, allowing us to skip this subtask depending
        // on the debug option.
        fatal: '{{debug}}',
        // This 'mute' attribute allows you to completely mute log calls made inside
        // this task as well its subtasks.
        // In this case, we used 'false' but it could have been a placeholder.
        mute: false
    })
    // If you find yourself looking for something a bit more custom,
    // you can just provide a function as the task.
    // More details about inline functions below in the "Inline functions" section.
    .do(function (options, ctx, next) {
        // 'options' is a list of the options provided to the task.

        // ctx.log gives you access to the Logger.
        // The Logger should be used to perform any logging information, and is
        // preferred to any console.* methods, as this gives additional control.
        // More information on ctx in the "Inline Functions" section.
        ctx.log.writeln('I can do whatever I want', 'even works with multiple args');

        // When the task is done, you just call next()
        // not like the MTV show, though…
        // (- -')
        next();
    });
};

Note that placeholders can be escaped with backslashes: '\\{\\{dir1\\}\\}'

Inline functions

If you find yourself trying to do something that is not supported by the existing tasks, you can just provide a function, instead of the task name, and it will be used as the task.

This task will receive 3 arguments, an options object (the options that were provided to the subtask), a context object (more on this later), and a callback that must be called once the subtask is over, giving you full flexibility, since your function can do whatever you like.

The second argument, the context, is used to provide you with a tool belt that will aid you while developing automaton tasks. It currently provides you a log object, which is an instance of Logger, and can be used to handle logging. Using the automaton logger is preferred to using the traditional console.* methods, since it gives additional control over logging to whoever is running the task.

The Logger provides the following methods:

  • Normal logging: write(), writeln()
  • Information logging: info(), infoln()
  • Warnings logging: warn(), warnln()
  • Error logging: error(), errorln()
  • Success logging: success(), successln()
  • Debug logging: debug(), debugln() (These will only be outputted when in debug mode)

The ln variants of each method output a new line (\n) in the end. Note that these methods work just like your typical console.* methods, so you can pass multiple arguments, and they will all get logged.

Here's an example usage:

var inspect = require('util').inspect;

module.exports = function (task) {
    task
    .id('bogus')
    .do(function (options, ctx, next) {
        ctx.log.writeln(
            'hello,',
            'here\'s the process',
            inspect(process)
        );

        next();
    });
};

Grunt tasks

You are able to run grunt tasks in automaton. It's actually very simple:

module.exports = function (task) {
    task
    .id('bogus')
    .do('mincss', {
        grunt: true,
        options: {
            files: {
                'path/to/output.css': 'path/to/input.css'
            }
        }
    });
};

By default, automaton autoload tasks located in tasks/ and npm tasks (that start with grunt-). If your task lives in a different folder, you can specify it in the grunt config. Other grunt options like force and verbose can also be specified:

module.exports = function (task) {
    task
    .id('bogus')
    .do('some-grunt-task', {
        grunt: {
            tasks: ['lib/tasks/'],                // array of folders to load tasks from
            force: true,
            verbose: true
            // other grunt config goes here
        },
        options: {
            some: 'option'
        }
    });
};

Loading tasks

Once you start building your own tasks, you will probably find yourself wanting to use some custom task within another task you're working on. In order to do this, you have a few options, depending on how you are using automaton.

If you are using automaton in the CLI, you have the --task-dir. This tells automaton to load all the tasks in that folder, making them available to you, just like the built-in tasks.

If you are using automaton programmatically, you have a bigger range of possibilities:

  1. Run automaton.loadTasks(/some/folder/with/tasks). This is the equivalent to what you would to in the CLI, with the exception that you can call this multiple times, loading multiple folders.

  2. require() the task yourself, just like you would with any NodeJS module, and then call automaton.addTask(your_task). Just like loadTasks(), this will make the task you just added available on automaton as if it is a built-in task.

  3. require() the task in the task that depends on it, and use it directly in the subtask list, where you would typically put a task name, or an inline function.

Usage

CLI

All you need to use the CLI can be found by executing automaton -h. This will show you how to use automaton, and any of the loaded tasks.

In order to run an autofile, you simply run automaton. This will look for autofile.js in the current working dir. Instead, you can also run automaton some_dir/my_autofile.js, enabling you to specify what autofile you want to run.

Node.js

automaton can also be used programmatically as a node module. Here's a quick example of its usage:

var automaton = require('automaton').create(/*options go here*/);

// Since autofiles are node modules themselves,
// you can just require them
// Note that you could have instead declared
// the module inline, in JSON.
var myTask = require('my_autofile');

// Note that we're running a task that you have loaded using node's
// require, and passing it as the first argument of the run() function.
// Instead, you can load the task using loadTask(), and then simply
// pass its id (a string), as the first argument of run. You can find an
// example of this below, in the Logging section.
automaton.run(myTask, { 'some_option': 'that is handy' }, function (err) {
    if (err) {
        console.log('Something went wrong: ' + err.message);
    } else {
        console.log('All done!');
    }
});

Logging

automaton.run() returns a readable stream that is used for outputting log information. The depth of this log can be controlled by a verbosity option, provided upon instantiation.

There are also a few other options. Here's a full list:

  • verbosity: Controls the depth of the log. Remember the box in a box analogy? This controls how many boxes deep you want to go, regarding logging information.
    • 0: no logging
    • 1: 1 level deep (default)
    • n: n levels deep
    • -1: show all levels
  • debug: If you want to receive debug logging messages. false by default.
  • color: If you want the logging information to contain colors. true by default.

Here's an example of the usage of the stream, with custom options:

var automaton = require('automaton').create({
    verbosity: 3, // show me 3 level deep logging info
    debug: true,  // show me debug logging
    color: false  // disable colors
});

// run some task
automaton
.run('run', { cmd: 'echo SUCCESS!' })
.pipe(process.stdout);

As you can see, we've tweaked all the automaton options, and even piped the logging information to STDOUT. What you do exactly with the stream, it's completely up to you.

Acknowledgements

Should be noted that this tool was inspired by already existing tools, and you should definitely take a look at them before deciding what is the right tool for the job at hand:

To these guys, a big thanks for their work.

Also, big thanks to Ricardo Pereira, for his awesome work with the mascot.

Contributing

Should be noted that Automaton is an open source project, and also work in progress. Feel free to contribute to the project, either with questions, ideas, or solutions. Don't forget to check out the issues page, as there are some improvements planned.

Thanks, and happy automation!

License

Released under the MIT License.

automaton's People

Contributors

filipediasf avatar marcelombc avatar marcooliveira avatar satazor avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

automaton's Issues

Scaffolding tasks should have a replacements object

At the moment we specify the replacements with 'what' and 'with' or 'what' and 'where'.
We could instead support an object:

options: {
  replacements: {
    'abc': 'def'
    'foo': 'bar'
  }
}

With this strategy we also get rid of with which is a reserved keyword.

Task: rsync

Should we provide an rsync task? Ideally, this would be a pure JS implementation, not requiring rsync to be installed, although I'm not sure it is possible. If not, and we decide to go through with this, there is a wrapper module for rsync, https://npmjs.org/package/rsyncwrapper.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/98445-task-rsync?utm_campaign=plugin&utm_content=tracker%2F39683&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F39683&utm_medium=issues&utm_source=github).

Task argument handler

The developer should be able to specify an argument handler that would receive all the task arguments, and do some work on them, for the subtasks.

Also, the subtasks should be able to reference task arguments, using a simple syntax, like {{some_arg}}.

Support non-fatal errors

Following the discussion initiated in 48632c5#commitcomment-2397444, we should probably allow users to define a fatal: false on the subtask level, which makes that subtask able to fail without killing the whole task, allowing it to proceed to the next subtask.

Verbose mode

The verbose mode would make all the subtasks also log their description. Verbosity could have a depth, which would indicate how deep the feedback should go.

The depth would be a number, which would indicate how many levels deep the subtask description is used to give feedback (0 being no feedback at all, 1 feedback from only the task that the user executed, and so forth)

Add ability to run sync tasks

We could provide something similar to mocha. If the callback has a next, we assume it async, otherwise we call next under the hood.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/98463-add-ability-to-run-sync-tasks?utm_campaign=plugin&utm_content=tracker%2F39683&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F39683&utm_medium=issues&utm_source=github).

"on" attribute for subtasks

Add support for a on attribute that allows the developer to enable/disable a subtask just by setting it to something that evaluates to false. This attribute must support {{placeholders}}, and must only be parsed after the task filter is run. This gives additional flexibility over the task.

Take the following example:

  • You are creating a build task
  • Your build task is supposed to support building for dev and prod environments
  • Although you want to run all the subtasks in prod, you want to disable uglify in dev

In order to do this, you can just define the task like so:

var myBuilder = {
    filter: function (opt) {
        // you could put whatever logic you needed here
        opt.not_dev = opt.env !== 'dev';
    }
    tasks: [
        {
            task: 'lint',
            // some configs
        },
        {
            task: 'concat'
            // some configs
        },
        {
            task: 'uglify',
            on: '{{not_dev}}'
            // some configs
        },
    ]
};

module.exports = myBuilder;

Or even something like this:

var myBuilder = {
    options: {
        not_dev: {
            'default': false
        }
    },
    tasks: [
        {
            task: 'lint',
            // some configs
        },
        {
            task: 'concat'
            // some configs
        },
        {
            task: 'uglify',
            on: '{{not_dev}}'
            // some configs
        },
    ]
};

module.exports = myBuilder;

Create a proper CLI tool

The current CLI is just a hack, to test the inner engine.

Should create support for:

  • Initialising an empty task
  • Getting the usage of a task
  • Pass arguments to a task

Discuss logging

We should provide a log interface for tasks to use (analyze grunt's one).
We should also improve how logs are presented in the CLI.

Ability to abort task with an error in the filter

I had a use case in which I needed to pass an error when the options didn't met some combined requirements.

This could be supported if the filter function also receives the next function.
It would be called with the error (e.g.: next(new Error('whattttt')).

Automaton should be an emitter and emit meaningful events.

It would increate the flexibility to use automaton programatically.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/98455-automaton-should-be-an-emitter-and-emit-meaningful-events?utm_campaign=plugin&utm_content=tracker%2F39683&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F39683&utm_medium=issues&utm_source=github).

Scaffolding: Allow operations based on line number/regex besides placeholders

You don't always have control over the files you are trying to operate on, so having the ability to work with line numbers or regular expressions would be really useful. We could simply have some match_type option that changed what the keys in the data option meant, defaulting to placeholder.

This option would support the following:

  • placeholder (default)
  • line - note that this one can create some issues, since as stuff gets appended/replaced inside the file, the line number will be changing, so we need to think this through, and plan how we really want it to perform
  • regex


Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Alternative way to construct a task object

This is a suggestion made by @mklabs and its actually pretty interesting. I like it.
His suggestion is basically to provide a programatic API to construct the task object that automaton already knows. Here's an example:

var task = module.exports = require('task-builder').create();
var otherTask = require('other-task');

 task
  .id('my-task')
  .name('My awesome task')
  .option('a', 'Option a', 'foo')
  .subtask('cp', { files: { 'foo/' : 'bar/' })
  .subtask(function (opts, ctx, next) { next(); })
  .subtask(otherTask, { foo: 'bar' })
  // ..etc;

Needs further discussion. This alternative syntax is much less verbose.

Improve exclusion of files in minimatch

Excluding files from a minimatch expression can be tricky, and the syntax can become a nightmare to read. Maybe we should come up with a simpler syntax?

If we take the cp example, we could support something like this:

files: {
  'src/**/*.js,!*.min.js': 'dst'
}

This would copy all the .js files, except those that end with .min.js, the exclamation mark being a negation (exclusion) operator, and the comma separating these. This expression could be composed of several comma separated expressions and, in order for a file to match, it would have to match ALL the expressions.

These sort of features should be provided in something like ctx.file, so that task creators don't have to do this by hand repetitively, and also it should be part of the automaton-lib. Check #45 for more information on this.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/98439-improve-exclusion-of-files-in-minimatch?utm_campaign=plugin&utm_content=tracker%2F39683&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F39683&utm_medium=issues&utm_source=github).

Add debug messages to the tasks

Now that we have this.log done, built in tasks can print logs (preferably with debug true).

The cp and mv tasks are good candidates to debug messages. They could debug each file being copied/moved.

Load and execution of tasks that are located in the tasks/ dir

In my opinion, automaton should load the tasks located in tasks/ automatically.
It would be nice to be able to do automaton server to run the tasks/server.js (if there is not a server.js in the cwd).

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/98459-load-and-execution-of-tasks-that-are-located-in-the-tasks-dir?utm_campaign=plugin&utm_content=tracker%2F39683&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F39683&utm_medium=issues&utm_source=github).

Unit tests

Create unit tests for the main engine, and each of the built-in tasks, as well as for the CLI.

Ability to pass options to autofile task

Should be able to pass options to the autofile task. As of now, the task must be static, and only subtasks can receive options. Also consider receiving the whole task in the run method instead of only the subtask list, since it will allow for filters, option validation and improved feedback.

Document and test ability to use other tasks

At the moment tasks can use another tasks using node require like so:

var otherTask = require('./other-task');

var myTask = {
  tasks: [
    {
      task: otherTask,
      options: {}
    }
  ]
};

Add unit tests for this.
Also document this feature.

Support Gruntjs tasks

It would be great if Automaton also could take Gruntjs tasks. The goal here would be to achieve a state in which the user could invoke some grunt task just like any other autofile. From my understanding, Grunt tasks have a set of options as input, just like automaton. The main difference is that they have targets, while we simply invoke the task separate times, with separate options, if necessary.

This needs some proper planning before implementation, so if anyone wants to pick this, the first thing to do would be to discuss what you are planning to do.

Mute option to silence tasks

Sometimes, it makes sense to use a task but completely prevent it from logging things.

Having this in mind we could add a mute or silent option to enable this.

Create automaton-lib that holds support functions

There are several util functions on the horizon, so it might be a good time to think about creating an automaton-lib to hold all this stuff. What do you think? Should we create a separate repository and separate nodejs module?

I'll be referencing this issue in issues that relate to this.


Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Add ability to specify options types to automatically assert them

One possible solution:

options: {
    opt1: {
        description: 'Opt1 description',
        'default': 'foo',
        type: String
    },
    opt2: {
        description: 'Opt2 description',
        'default': false,
        type: Boolean
    },
    opt3: {
        description: 'Opt3 description',
        type: [Array, Object]
    }
}

Then automaton would assert any options automatically based on their types.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/98457-add-ability-to-specify-options-types-to-automatically-assert-them?utm_campaign=plugin&utm_content=tracker%2F39683&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F39683&utm_medium=issues&utm_source=github).

Speed up tasks by only running on changed files

I don't know the best way to implement it, but from being involved in grunt early on, we've gotten that request a lot.

It makes sense too. It would be way faster to only execute on a changed file instead of everything.

Let's say you have a sass and coffescript task. You run Automaton, and it takes some time. You then change one of the CoffeScript files and run Automaton again. I now recompiles all Sass and CoffeScript files for this little change. This is extremely inefficient.

What it however should have done, somehow, is to only recompile the changed CoffeScript file.

This is a fairly common pattern, so it shouldn't require a lot of boilerplate in the task.


Related grunt issue: gruntjs/grunt#212


Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.