michaelcurrin / unicron

The tenacious but efficient scheduler for daily tasks 🔂 ⏳ 🦄

Unicron 🕐

The tenacious but efficient scheduler for daily tasks 🔂⏳🦄

• The easy way to take control of your scheduled tasks.
• Configure as many tasks as you want that will each run once a day.
• For example, scrape HTML pages or Twitter API data - without running excessively, so you save on storage (avoiding duplicates) and avoid hitting rate limits.
• Unicron integrates with crontab on macOS or Linux.
• Made with Python, but zero packages needed!
• Just clone, configure and run. 🐍 🎉 🦄

License

Released under MIT by @MichaelCurrin.

unicron's Issues

Update cmd handling

Use PY cmd splitting such as with shlex for multi-word commands. Setup unit tests for this.

Not needed when only file names apply

Create lock file for main script

Exit if it already locked - print to stderr and give exit code.

Else lock file gets remove always with try finally perhaps

Make docs site.

Note this is low priority as the docs are simple enough and functionality is more important at this stage.

Create docs directory and use docsify.

Having dynamic sidebar but fixed navbar could work.

Add force run mode.

Similar to #11 in identifying executables.

With a flag, run all executables using the main script (and associating logging and last_run updates). A similar approach would be a simple script to run everything in the targets directory (or targets config, if there is one). This would run everything but sidestep the main application.

This is useful say you want to run everything manually before you turn off the machine for the day.

Remove verbose and test as global variables

Handle error case of broken symlink

Convert main Bash script to Python

#22

With unit tests and pipeline.
And docs to setup.

Allow disabling of configured tasks

Send error mail if nothing ran the day before

As an extension of #5, if no scripts could be executed yesterday because the machine errored on all, was off or was offline, then send a mail today before starting tasks.

Perhaps use the log of the app or create a run file for the app, to see when it last run to completion with at least one successful task.

There is of course the limitation that the script can't watch itself because it might get an error and exit before it can even send the warning. So splitting the warning into a separate script which even runs maybe on a different crontab could help.
Though what watches that? If it is simple enough and robust (independent of the rest of the system) then it will help.
Alternatively, use a script to check yesterday's date appears in the main app's log along with successful end, provided this is logged.

Also ensure there are at least instructions to view the main app log and mainly check runs. This avoids code.

Add logo to sidebar

Small image and maybe title (see similar sites).

Add loading bar or spinner for main script in verbose mode

This must not appear for cron

Groom issues

Prioritize and point

Remove use of is_task in favor of checking "unicron" or using more functions

Also make logging easier to use - app logging always needs extra, including task name

Send mail on error

Use system mail. Check if is the same on mac and linux

Include script output - all of it.

Send one mail for each failure. Although noisy it helps to see a pattern. Create an issue for aggregating more than X errors into one email with count in subject. If that is even needed. As failure rate is expected to be low.

Add dry run mode

This must not actually execute anything just print out names of executables that would be executed.
This can run against main and test.
Send a mail listing them as well.

Version the test files

Quiet and noisy running

Usecases

• Run quietly. Only log error messages for tasks and show error in main script itself.
• Run verbosely when running locally. DEBUG and ERROR messages.

The first point can be achieved using /dev/null redirection.
Or just running without verbose flag.

Allow running on other schedules e.g. once a week

Add instructions for rotating app.log and other task logs

Update ignore rules

Convert todo items into issues

Rebuild in Go

This seems like a simple app that can be rebuilt in Go, so it can be compiled and distributed as a binary without Python needed (though, any Linux or mac should easily have Python available or installed and this is intended for crontab so not Windows but maybe)

Update logo

Does not render properly on Github.

<img src="logo.png" alt="logo" style="width:100px;">

Resizing locally means a blurred finished image.

Docs - crontab

Include shell, user, mail to etc. and how to mail to dev null

Add emojis to doc site

These render except the unicorn. Maybe use a plugin.

:repeat_one::hourglass_flowing_sand::unicorn:

Move var directory to user location

Include make command to open the directory with open or vscode.

Make CLI more streamlined from anywhere

Ideas:

• Add setup.sh or install.sh script - run it locally or with curl?
• Clones the repo to ~/.unicron
  • for dev, should this be the same? Or symlink? Or separate? I've had issues before where something (including cron) was broken for a few days and I didn't know. Unfinished unstashed work can cause issues.

As part of install, create actual script in /usr/local/bin default, or ~/bin with arg. Which will do this.

• Script:

  SCRIPT_DIR=...
  cd $SCRIPT_DIR
  make $@

Maybe it is a symlink rather to a script in unicron which does the same thing. (symlink is better for keeping logic in version control).

This could run the install script.

make install

Will also need a way of doing git pull. Keep it like a repo cloned with https for updates rather than trying to download.

Also make sure to checkout to latest tag when running unicron update. Unless a flag is used to indicated latest/master.

What to do about unicron data? Keep it separate in a var directory so the repo can be deleted. Maybe ~/.unicron/unicron-repo can be the repo

See how Oh my ZSH handles this

This setup could be done manually first to see if I like it and then all brought into scripts (automate second).

Move tasks to config file

This means no command line work for symlinks.

It also prevents scripts from being added to the project which can be good. They then live in bin or other user directory that may or may not be executed from anywhere.

The config could even include shell commands in one item for arguments but then it lives in this project.

Benefit is text editor or even jekyll manager can be used to edit the file.

Note that syntax of file can be bad so gui helps.
And a broken symlink is the same on the old and new method.

Remember each task will be run separately. And maybe even may or may not have arguments. may be a full path or just name of bin script

Extend unit tests to handle state before and after

Separate code from data

Major upgrade in functionality, to be handled by several issues but possibly one in one PR (since changes are all large and work together and backwards behavior can be lost) or series of PRs.

• Make unicron installable as a python file (with venv in bin path or install globally as long as dev dependencies are left out)
• Use ~/.unicron for output and run data
• Use config file there too. Editable through IDE or maybe through terminal like crontab -e and validate as YAML or TOML, which have having to write an editor like git config a.b value. Maybe given option of open $(unicron -e) or simpler unicron -e open or ... code etc
• commandline will have to be revised for new commands
• any native bash commands like tail are best not handled with python otherwise its extra code and potential lost in functionality with streams and exit codes. If unicron is a bash script this is easier and maybe some commands go through unicron python script internally. Though the architecture is different for install bash than Python scripts.
• maybe a web server or python gui will help for running or viewing files but it takes away from the simplicity of unicron

Include task length in logging

Add tag to readme

Make installable with pip

Use Github URL and layer pypi

If there are no dependencies, global env instructions are fine.

Separate dev clone and user install instructions.

Use 3rd party service for mail

As an extension of #5

Later, extend to send with gmail etc.
The benefit being you do not need to look at the terminal or even be at the machine to see errors.

But you will need to look up a tutorial to see easy modern way to do it with keys. Check if this still works for 2FA enabled account.

Note that the risk is that the machine will offline and the mail fails. So the standard mail should be kept. See if there is an easy way to add to message queue for gmail etc. Maybe have to rely on a background queue process configured to start at startup.