GithubHelp home page GithubHelp logo

reflex's Introduction

Reflex

Reflex is a small tool to watch a directory and rerun a command when certain files change. It's great for automatically running compile/lint/test tasks and for reloading your application when the code changes.

A simple example

# Rerun make whenever a .c file changes
reflex -r '\.c$' make

Installation

You can download binaries from the Releases page.

To compile from source, you'll need Go 1.13+ installed.

If you have Go 1.16 or later, you can download and install the latest module version directly with

go install github.com/cespare/reflex@latest

Reflex is only tested on Linux and macOS.

Usage

The following is given by running reflex -h:

Usage: reflex [OPTIONS] [COMMAND]

COMMAND is any command you'd like to run. Any instance of {} will be replaced
with the filename of the changed file. (The symbol may be changed with the
--substitute flag.)

OPTIONS are given below:
      --all=false:
            Include normally ignored files (VCS and editor special files).
  -c, --config="":
            A configuration file that describes how to run reflex
            (or '-' to read the configuration from stdin).
  -d, --decoration="plain":
            How to decorate command output. Choices: none, plain, fancy.
  -g, --glob=[]:
            A shell glob expression to match filenames. (May be repeated.)
  -G, --inverse-glob=[]:
            A shell glob expression to exclude matching filenames.
            (May be repeated.)
  -R, --inverse-regex=[]:
            A regular expression to exclude matching filenames.
            (May be repeated.)
      --only-dirs=false:
            Only match directories (not files).
      --only-files=false:
            Only match files (not directories).
  -r, --regex=[]:
            A regular expression to match filenames. (May be repeated.)
  -e, --sequential=false:
            Don't run multiple commands at the same time.
  -t, --shutdown-timeout=500ms:
            Allow services this long to shut down.
  -s, --start-service=false:
            Indicates that the command is a long-running process to be
            restarted on matching changes.
      --substitute="{}":
            The substitution symbol that is replaced with the filename
            in a command.
  -v, --verbose=false:
            Verbose mode: print out more information about what reflex is doing.

Examples:

    # Print each .txt file if it changes
    $ reflex -r '\.txt$' echo {}

    # Run 'make' if any of the .c files in this directory change:
    $ reflex -g '*.c' make

    # Build and run a server; rebuild and restart when .java files change:
    $ reflex -r '\.java$' -s -- sh -c 'make && java bin/Server'

Overview

Reflex watches file changes in the current working directory and re-runs the command that you specify. The flags change what changes cause the command to be rerun and other behavior.

Patterns

You can specify files to match using either shell glob patterns (-g) or regular expressions (-r). If you don't specify either, reflex will run your command after any file changes. (Reflex ignores some common editor and version control files; see Ignored files, below.)

You can specify inverse matches by using the --inverse-glob (-G) and --inverse-regex (-R) flags.

If you specify multiple globs/regexes (e.g. -r foo -r bar -R baz -G x/*/y), only files that match all patterns and none of the inverse patterns are selected.

The shell glob syntax is described here, while the regular expression syntax is described here.

The path that is matched against the glob or regular expression does not have a leading ./. For example, if there is a file ./foobar.txt that changes, then it will be matched by the regular expression ^foobar. If the path is a directory, it has a trailing /.

--start-service

The --start-service flag (short version: -s) inverts the behavior of command running: it runs the command when reflex starts and kills/restarts it each time files change. This is expected to be used with an indefinitely-running command, such as a server. You can use this flag to relaunch the server when the code is changed.

Substitution

Reflex provides a way for you to determine, inside your command, what file changed. This is via a substitution symbol. The default is {}. Every instance of the substitution symbol inside your command is replaced by the filename.

As a simple example, suppose you're writing Coffeescript and you wish to compile the CS files to Javascript when they change. You can do this with:

reflex -r '\.coffee$' -- coffee -c {}

In case you need to use {} for something else in your command, you can change the substitution symbol with the --substitute flag.

Configuration file

What if you want to run many watches at once? For example, when writing web applications I often want to rebuild/rerun the server when my code changes, but also build SCSS and Coffeescript when those change as well. Instead of running multiple reflex instances, which is cumbersome (and inefficient), you can give reflex a configuration file.

The configuration file syntax is simple: each line is a command, and each command is composed of flags and arguments -- just like calling reflex but without the initial reflex. Lines that start with # are ignored. Commands can span multiple lines if they're \-continued, or include multi-line strings. Here's an example:

# Rebuild SCSS when it changes
-r '\.scss$' -- \
   sh -c 'sass {} `basename {} .scss`.css'

# Restart server when ruby code changes
-sr '\.rb$' -- \
    ./bin/run_server.sh

If you want to change the configuration file and have reflex reload it on the fly, you can run reflex inside reflex:

reflex -s -g reflex.conf -- reflex -c reflex.conf

This tells reflex to run another reflex process as a service that's restarted whenever reflex.conf changes.

--sequential

When using a config file to run multiple simultaneous commands, reflex will run them at the same time (if appropriate). That is, a particular command can only be run once a previous run of that command finishes, but two different commands may run at the same time. This is usually what you want (for speed).

As a concrete example, consider this config file:

-- sh -c 'for i in `seq 1 5`; do sleep 0.1; echo first; done'
-- sh -c 'for i in `seq 1 5`; do sleep 0.1; echo second; done'

When this runs, you'll see something like this:

[01] second
[00] first
[01] second
[00] first
[00] first
[01] second
[01] second
[00] first
[01] second
[00] first

Note that the output is interleaved. (Reflex does ensure that each line of output is not interleaved with a different line.) If, for some reason, you need to ensure that your commands don't run at the same time, you can do this with the --sequential (-e) flag. Then the output would look like (for example):

[01] second
[01] second
[01] second
[01] second
[01] second
[00] first
[00] first
[00] first
[00] first
[00] first

Decoration

By default, each line of output from your command is prefixed with something like [00], which is simply an id that reflex assigns to each command. You can use --decoration (-d) to change this output: --decoration=none will print the output as is; --decoration=fancy will color each line differently depending on which command it is, making it easier to distinguish the output.

Ignored files

Reflex ignores a variety of version control and editor metadata files by default. If you wish for these to be included, you can provide reflex with the --all flag.

You can see a list of regular expressions that match the files that reflex ignores by default here.

Notes and Tips

If you don't use -r or -g, reflex will match every file.

Reflex only considers file creation and modification changes. It does not report attribute changes nor deletions.

For ignoring directories, it's easiest to use a regular expression: -R '^dir/'.

Many regex and glob characters are interpreted specially by various shells. You'll generally want to minimize this effect by putting the regex and glob patterns in single quotes.

If your command has options, you'll probably need to use -- to separate the reflex flags from your command flags. For example: reflex -r '.*\.txt' -- ls -l.

If you're going to use shell things, you need to invoke a shell as a parent process:

reflex -- sh -c 'sleep 1 && echo {}'

If your command is running with sudo, you'll need a passwordless sudo, because you cannot enter your password in through reflex.

It's not difficult to accidentally make an infinite loop with certain commands. For example, consider this command: reflex -r '\.txt' cp {} {}.bak. If foo.txt changes, then this will create foo.txt.bak, foo.txt.bak.bak, and so forth, because the regex \.txt matches each file. Reflex doesn't have any kind of infinite loop detection, so be careful with commands like cp.

The restart behavior works as follows: if your program is still running, reflex sends it SIGINT; after 1 second if it's still alive, it gets SIGKILL. The new process won't be started up until the old process is dead.

Batching

Part of what reflex does is apply some heuristics to batch together file changes. There are many reasons that files change on disk, and these changes frequently come in large bursts. For instance, when you save a file in your editor, it probably makes a tempfile and then copies it over the target, leading to several different changes. Reflex hides this from you by batching some changes together.

One thing to note, though, is that the the batching is a little different depending on whether or not you have a substitution symbol in your command. If you do not, then updates for different files that all match your pattern can be batched together in a single update that only causes your command to be run once.

If you are using a substitution symbol, however, each unique matching file will be batched separately.

Argument list splitting

When you give reflex a command from the commandline (i.e., not in a config file), that command is split into pieces by whatever shell you happen to be using. When reflex parses the config file, however, it must do that splitting itself. For this purpose, it uses this library which attempts to match sh's argument splitting rules.

This difference can lead to slightly different behavior when running commands from a config file. If you're confused, it can help to use --verbose (-v) which will print out each command as interpreted by reflex.

Open file limits

Reflex currently must hold an open file descriptor for every directory it's watching, recursively. If you run reflex at the top of a big directory tree, you can easily run into file descriptor limits. You might see an error like this:

open some/path: too many open files

There are several things you can do to get around this problem.

  1. Run reflex in the most specific directory possible. Don't run reflex -g path/to/project/*.c ... from $HOME; instead run reflex in path/to/project.
  2. Ignore large subdirectories. Reflex already ignores, for instance, .git/. If you have other large subdirectories, you can ignore those yourself: reflex -R '^third_party/' ... ignores everything under third_party/ in your project directory.
  3. Raise the fd limit using ulimit or some other tool. On some systems, this might default to a restrictively small value like 256.

See issue #6 for some more background on this issue.

The competition

Why you should use reflex instead

  • Reflex has no dependencies. No need to install Ruby or anything like that.
  • Reflex uses an appropriate file watching mechanism to watch for changes efficiently on your platform.
  • Reflex gives your command the name of the file that changed.
  • No DSL to learn -- just give it a shell command.
  • No plugins.
  • Not tied to any language, framework, workflow, or editor.

Authors

reflex's People

Contributors

cespare avatar hartshorne avatar hollow avatar pjeby avatar robx avatar sethwklein avatar vanackere avatar wildeyes avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

reflex's Issues

multiple globs doesn't seem to work

I have a little build script that compiles a rust program. I want reflex to rebuild whenever files in src or example/src change. I can set up globs for each case, but it doesn't work with both together.

# this doesn't work.
# reflex -g 'example/src/*.rs' -g 'src/*.rs' -s -- sh -c 'cd example && clear && cargo build'

# this works.
reflex -g 'src/*.rs' -s -- sh -c 'cd example && clear && cargo build'

# this one works also.
# reflex -g 'example/src/*.rs' -s -- sh -c 'cd example && clear && cargo build'

Not working with Go 1.15.0

Upgraded Go to 1.15.0 in alpine linux 3.12 and I'm getting this error: "Setctty set but Ctty not valid in child" for a http application.

The command used: reflex -r '.\.go' -R '._test\.go' -d 'none' -s my-binary start

If I downgrade to Go 1.14.7 it works as expected.

As far as I can tell there were some modifications made in the syscall package in Go 1.15.0, specifically from the docs:

On Unix systems, functions that use SysProcAttr will now reject attempts to set both the Setctty and Foreground fields, as they both use the Ctty field but do so in incompatible ways. We expect that few existing programs set both fields.

Setting the Setctty field now requires that the Ctty field be set to a file descriptor number in the child process, as determined by the ProcAttr.Files field. Using a child descriptor always worked, but there were certain cases where using a parent file descriptor also happened to work. Some programs that set Setctty will need to change the value of Ctty to use a child descriptor number.

Source: https://golang.org/doc/go1.15

Run multiple commands

I assume this is just something I havent figured out yet, but is it possible to run multiple commands, something like cmd1 && cmd2 && cmd3?

Using sed to change filepath

Hey there,
This isn't an issue but i was hoping to get some advice.
I currently have a golang repository consisting of lots of lambda functions. Currently when i make a change to one lambda function, i have to rebuild them all.

build: clean
		@for dir in `ls handler`; do \
			echo 'building...' $$dir; \
			CGO_ENABLED=0 GOOS=linux go build -o dist/handler/$$dir gitlab.com/repo/code/handler/$$dir; \
		done
		echo "All built"

gomon:
	reflex -R '^node_modules/' -R '^output/' -r '\.go' -s -- sh -c 'make build'

File structure being

-/
   -handler/
       -apiHandler1/
       -apiHandler2/

I think I am extremely close, in rebuilding just one file when it's changed. But am having trouble getting sed to work. When i try my variations with sed, i often get
sed: 1: "s|[^/]+|g": unterminated substitute in regular expression

{} has the output of handler/approvedGet/approvedGet.go, but i need handler/approvedGet/ without the file name. For the command
CGO_ENABLED=0 GOOS=linux go build -o dist/handler/approvedGet/approvedGet.go gitlab.com/repo/code/handler/approvedGet/

A sed like sed "s|[^/]+|g" would work but i'm having trouble getting it to execute.

I'm hoping to get to the point where i have both {} and another variable in the command without the filename, so i can easily rebuild just the files & packages that have been changed.
Something like would be ideal

gomon:
	reflex -R '^node_modules/' -R '^output/' -r '\.go' -s -- 
        sh -c 'CGO_ENABLED=0 GOOS=linux go build -o dist/handler/{} gitlab.com/repo/code/{}_withoutfilename'

Thanks

Add ability to listen for only specific events

Combined with --only-events, this could make it easy to listen to make slightly more programmable script triggering. For instance, you might not want to trigger your asset pipeline on file create, but only file delete and file updated.

Polling instead of inotify

I am currently using reflex to recompile go application when something changes. Unfortunately I noticed reflex is not working properly when inside a VM with filesystems like vboxsf. Is there any chance that reflex will support somethink like force-polling or polling as a fallback?

Support manual triggers

First off, thanks for the awesome tool, I use it almost every day now.

One feature I would like is the ability to manually trigger reflex by hitting enter (or maybe ctrl-g) key.

Today I have to touch a file to get an initial run, a key-binding would be more clean.

Running reflex in Docker

Does someone do a multi stage builds with Reflex? I would like to keep the container size as low as possible.
I've try to find some implementation, but without success.

Entr

Hello, there is a similar program called Entr, I think it would be helpful to do some comparisons and explain why reflex was made and how it is different / better than Entr.

Handling `Chmod` seems to be needed when using dinghy

Hey there, sorry for bringing this up out of nowhere but in order to get reflex to play fine with dinghy I had to delete the code that ignored fsnotify.Chmod because that's the event we get when the tool that is used under the hood forwards events to the boot2docker VM by touching the file on the VM 😢

Sorry but this is my first time looking at reflex sources and I was wondering what would be the reason for having that logic around. Would it make sense to provide a flag to consider those events?

Not sure if you are familiar with Docker / Dinghy / Boot2Docker so let me know if you need any additional context.

/cc @codekitchen so that you know that there might be other tools that might have a hard time detecting changes right away when using dinghy, feel free to unsubscribe to the issue in case you don't want to get updates about it.

Reflex using 100% CPU

This has happened to me a few times on OSX. I'm using the latest reflex and Go 1.9.

I'm not sure yet how to reproduce but I'll update if I get more information. I've pasted a goroutine dump below.

The config file is just watching resource files and calling make.

-r '^resources/assets/.*\.(coffee|js)$' -- make js
-r '^resources/assets/.*\.(less|css)$' -- make css

Goroutine dump:

PC=0x1056bd3 m=2 sigcode=0

goroutine 0 [idle]:
runtime.mach_semaphore_timedwait(0x3c00001e03, 0x0, 0x70000f831cf4, 0x3c, 0x0, 0xc420000900, 0x70000f831d28, 0x1051173, 0xdf8475800, 0x0, ...)
	/usr/local/Cellar/go/1.9/libexec/src/runtime/sys_darwin_amd64.s:455 +0x13
runtime.semasleep1(0xdf8475800, 0x0)
	/usr/local/Cellar/go/1.9/libexec/src/runtime/os_darwin.go:402 +0xdf
runtime.semasleep.func1()
	/usr/local/Cellar/go/1.9/libexec/src/runtime/os_darwin.go:432 +0x33
runtime.systemstack(0x70000f831d50)
	/usr/local/Cellar/go/1.9/libexec/src/runtime/asm_amd64.s:360 +0xab
runtime.semasleep(0xdf8475800, 0x1033b73)
	/usr/local/Cellar/go/1.9/libexec/src/runtime/os_darwin.go:431 +0x44
runtime.notetsleep_internal(0x11f0b18, 0xdf8475800, 0xc420000900, 0x742c47f3603, 0xc400000014)
	/usr/local/Cellar/go/1.9/libexec/src/runtime/lock_sema.go:218 +0x112
runtime.notetsleep(0x11f0b18, 0xdf8475800, 0x7188aae1b01)
	/usr/local/Cellar/go/1.9/libexec/src/runtime/lock_sema.go:269 +0x75
runtime.sysmon()
	/usr/local/Cellar/go/1.9/libexec/src/runtime/proc.go:3866 +0x14d
runtime.mstart1()
	/usr/local/Cellar/go/1.9/libexec/src/runtime/proc.go:1172 +0x11e
runtime.mstart()
	/usr/local/Cellar/go/1.9/libexec/src/runtime/proc.go:1142 +0x64

goroutine 1 [chan receive]:
main.main()
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/main.go:201 +0x8ff

goroutine 5 [syscall]:
os/signal.signal_recv(0x0)
	/usr/local/Cellar/go/1.9/libexec/src/runtime/sigqueue.go:131 +0xa7
os/signal.loop()
	/usr/local/Cellar/go/1.9/libexec/src/os/signal/signal_unix.go:22 +0x22
created by os/signal.init.0
	/usr/local/Cellar/go/1.9/libexec/src/os/signal/signal_unix.go:28 +0x41

goroutine 6 [select, locked to thread]:
runtime.gopark(0x11473a0, 0x0, 0x113e929, 0x6, 0x18, 0x1)
	/usr/local/Cellar/go/1.9/libexec/src/runtime/proc.go:277 +0x12c
runtime.selectgo(0xc420046f50, 0xc4200244e0)
	/usr/local/Cellar/go/1.9/libexec/src/runtime/select.go:395 +0x1138
runtime.ensureSigM.func1()
	/usr/local/Cellar/go/1.9/libexec/src/runtime/signal_unix.go:511 +0x1fe
runtime.goexit()
	/usr/local/Cellar/go/1.9/libexec/src/runtime/asm_amd64.s:2337 +0x1

goroutine 7 [chan receive]:
main.main.func1(0xc42006a540)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/main.go:175 +0x4b
created by main.main
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/main.go:174 +0x55b

goroutine 8 [syscall]:
syscall.Syscall6(0x16b, 0x3, 0x0, 0x0, 0xc42005fe88, 0xa, 0x120ede0, 0x0, 0x0, 0x0)
	/usr/local/Cellar/go/1.9/libexec/src/syscall/asm_darwin_amd64.s:41 +0x5
github.com/cespare/reflex/vendor/golang.org/x/sys/unix.kevent(0x3, 0x0, 0x0, 0xc42005fe88, 0xa, 0x120ede0, 0x0, 0x0, 0x0)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/vendor/golang.org/x/sys/unix/zsyscall_darwin_amd64.go:207 +0x83
github.com/cespare/reflex/vendor/golang.org/x/sys/unix.Kevent(0x3, 0x0, 0x0, 0x0, 0xc42005fe88, 0xa, 0xa, 0x120ede0, 0x0, 0x0, ...)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/vendor/golang.org/x/sys/unix/syscall_bsd.go:448 +0x71
github.com/cespare/reflex/vendor/github.com/fsnotify/fsnotify.read(0x3, 0xc42005fe88, 0xa, 0xa, 0x120ede0, 0xc42005fe88, 0x0, 0xa, 0x0, 0x0)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/vendor/github.com/fsnotify/fsnotify/kqueue.go:493 +0x78
github.com/cespare/reflex/vendor/github.com/fsnotify/fsnotify.(*Watcher).readEvents(0xc42006a600)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/vendor/github.com/fsnotify/fsnotify/kqueue.go:284 +0x572
created by github.com/cespare/reflex/vendor/github.com/fsnotify/fsnotify.NewWatcher
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/vendor/github.com/fsnotify/fsnotify/kqueue.go:62 +0x293

goroutine 9 [select]:
main.watch(0x113e2d8, 0x1, 0xc42006a600, 0xc4200246c0, 0xc420024720, 0xc42004e7e0, 0x2, 0x2)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/watch.go:23 +0x2b0
created by main.main
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/main.go:193 +0x7d2

goroutine 10 [chan receive]:
main.broadcast(0xc42004e800, 0x2, 0x2, 0xc4200246c0)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/main.go:205 +0xa0
created by main.main
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/main.go:194 +0x818

goroutine 11 [chan receive]:
main.printOutput(0xc42006a060, 0x11da520, 0xc42000e018)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/print.go:65 +0xa7
created by main.main
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/main.go:195 +0x854

goroutine 12 [chan receive]:
main.(*Reflex).filterMatching(0xc4200a0a00, 0xc420024840, 0xc420024780)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/reflex.go:130 +0x4e
created by main.(*Reflex).Start
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/reflex.go:298 +0x9d

goroutine 13 [chan receive]:
main.(*Reflex).batch(0xc4200a0a00, 0xc4200248a0, 0xc420024840)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/reflex.go:156 +0x292
created by main.(*Reflex).Start
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/reflex.go:299 +0xd3

goroutine 14 [chan receive]:
main.(*Reflex).runEach(0xc4200a0a00, 0xc4200248a0)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/reflex.go:184 +0x4b
created by main.(*Reflex).Start
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/reflex.go:300 +0xff

goroutine 15 [chan receive]:
main.(*Reflex).filterMatching(0xc4200a0be0, 0xc420024900, 0xc4200247e0)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/reflex.go:130 +0x4e
created by main.(*Reflex).Start


[gumtwo] Fix tests
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/reflex.go:298 +0x9d

goroutine 16 [chan receive]:
main.(*Reflex).batch(0xc4200a0be0, 0xc420024960, 0xc420024900)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/reflex.go:156 +0x292
created by main.(*Reflex).Start
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/reflex.go:299 +0xd3

goroutine 18 [chan receive]:
main.(*Reflex).runEach(0xc4200a0be0, 0xc420024960)
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/reflex.go:184 +0x4b
created by main.(*Reflex).Start
	/Users/mikeq/dev/go/src/github.com/cespare/reflex/reflex.go:300 +0xff

rax    0xe
rbx    0x3c
rcx    0x70000f831cc8
rdx    0x0
rdi    0x1e03
rsi    0x3c
rbp    0x70000f831d00
rsp    0x70000f831cc8
r8     0x70000f831dc0
r9     0x6d
r10    0x603d27dc
r11    0x202
r12    0x6029408d5610
r13    0xb03
r14    0x102cf10
r15    0x1400000
rip    0x1056bd3
rflags 0x202
cs     0x7
fs     0x0
gs     0x0

"too many open files" error

When I run a reflex command within my $GOPATH, I get a bunch of errors related to "too many open files". Here is an example:

[info] Error while watching new path src/code.google.com/p/go.net/websocket: open src/code.google.com/p/go.net/websocket: too many open files

I am on Mac OS X 10.9. I installed with go get (and then moved the binary into my ~/bin). I can run reflex in other directories that only have a few files. Here is the reflex command I am running:

reflex -g '*.go' -- go test github.com/jgoodall/test

It actually doesnt seem to matter what the command is, I get the same errors.

stdin is ignored

I'm using reflex to compile a Java program that gets input from stdin. I'm running the command using:

# Build and run a server; rebuild and restart when .java files change:
    $ reflex -r '\.java$' -s -- sh -c 'make && java bin/Server'

However, the input from stdin gets ignored.

Is there something I can do? If not, where, in the code, can I try to fix it?

1.6beta2 on OSX

I can't seem to get reflex to work.. perhaps I'm using it incorrectly.. im doing reflex -g * -- echo hi I've also tried reflex -g ./* echo hi .. nothing happens when I change those files

Clear screen before re running a command.

When the watched files in a folder changes, it runs the command without clearing the screen. It makes results from the latest run commands very dirty.

Can a simple clear screen command be invoked, before a new command is run?

Installation takes hours

Hi there, I am trying to install reflex on a Centos 6 using go get github.com/cespare/reflex. It has been running now for hours. This is what I see on the box:

root      6929  2750  0 11:18 pts/0    00:00:01 go get github.com/cespare/reflex
root      7213  6929  0 11:18 pts/0    00:00:00 git clone https://gopkg.in/fsnotify.v1 /root/golang/src/gopkg.in/fsnotify.v1
root      7214  7213  0 11:18 pts/0    00:00:00 git-remote-https https://gopkg.in/fsnotify.v1 https://gopkg.in/fsnotify.v1
root      7216  7214  0 11:18 pts/0    00:00:00 git fetch-pack --stateless-rpc --lock-pack --thin --no-progress https://gopkg.in/fsnotify.v1  efs/heads/master  efs/heads/v0  efs/tags/v0.8.06  efs/tags/v0.8.07  efs/tags/v0.8.08  efs/tags/v0.8.09  efs/tags/v0.8.10  efs/tags/v0.8.11  efs/tags/v0.8.12  efs/tags/v0.8.13  efs/tags/v0.9.0  efs/tags/v0.9.1  efs/tags/v0.9.2  efs/tags/v0.9.3  efs/tags/v1.0.0  efs/tags/v1.0.2  efs/tags/v1.0.3  efs/tags/v1.0.4  efs/tags/v1.1.0  efs/tags/v1.1.1  efs/tags/v1.2.0  efs/tags/v1.2.1  efs/tags/v1.2.10  efs/tags/v1.2.11  efs/tags/v1.2.5  efs/tags/v1.2.8  efs/tags/v1.2.9  efs/tags/v1.3.0
root      8368  7916  0 14:04 pts/2    00:00:00 grep go

Is this normal or am I missing something related to this installation?

Support shared inverse regexes

I was running into a too many open files issue due to having a lot of stuff in my project's vendor/ folder, which was odd because my configuration specified --inverse-regex='^vendor/'. After some investigation, I realized that Reflex only excludes files from its watcher as long as it knows that all of the specified configurations should ignore that regex; if any of them don't include it explicitly, then Reflex will still walk the tree just in case something needs it. Since I use Reflex for several different purposes, I had to add --inverse-regex='^vendor/' to about 5 different lines before the issue went away, even though the rest of them include a more specific --regex that would never match a vendored file anyway.

A couple possible solutions are:

  1. Allow both a config file and command-line inverse regexes. If an inverse regex is provided as an argument, it should apply to all watchers found in the config.
  2. Update the config file syntax to support some form of reusable arguments, even if it's just environment variable expansion.

Discussion: Option to coalesce backlog

It is sometimes advantageous to have a command digest all changed files, rather than being invoked separately several times. Reflex's internal backlog maps a burst of changes into multiple commands, such that running touch 1 2 3 will generate the following output:

$ reflex echo {}
[00] 1
[00] 2
[00] 3

As implemented, any system would have to be tolerant of rapid-fire invokations and do its own batching. It would be nice to have a non-default option to run the command once per backlog flush:

$ reflex --combined-changes echo {}
[00] 1 2 3

I have implemented a quick & dirty proof of concept on my fork's combined-changes branch. Hopefully this can be used as a base to get a better implementation merged in.

The good:

  • Basic functionality works, defaults are unchanged
  • Config option, config subsitution check, and help text added

The bad / ugly:

  • Backlog interface modified; an All() function was added.
  • The functions batch and runEach had to be modified, so right now they're copies for combined & default behavior. Not exactly DRY.
  • Due to the particular way select is used for signaling, this could inflate a set of strings arbitrarily many times. This could be a performance problem if I'm correctly understanding the RTL implications of case out <- r.backlog.All().
  • Code style guide / gofmt not applied yet.
  • Right now, this passes the paths set as a single space-delimited parameter to the command. Arguably it should pass N many parameters instead.

Not working with http server

I'm trying to use reflex to compile, stop and then restart a http server written in Go. But after http server listening in some port, suppose :8080 the reflex does nothing. In a simple hello-world function works fine, but after start a http server not working.

I have a Make file:

build:
	GOOS=linux go build -o ./build/api ./main.go

start:
	make build
	echo "Starting build/api"
	./build/api
	reflex -r \.go make restart

restart: stop build start

stop:
	echo "Stopping ./build/api if it's running"
	kill -9 `cat ./tmp/.pid`

So I just need to run make start and server will start fine, but never will execute the line reflex -r \.go make restart.

The .pid file on stop command is generated by my go program before start the server. And is working fine. Just the restart command is never executed by reflex.

Someone already had this behavior before?

--sequential runs --start-service invocations sequentially

Hi,

I understand that strictly speaking, this makes sense, but practically I think it is not that useful. I have something like the following in a config file:

-r '.*\.go' -- go build -v -i ./cmd/foo ./cmd/bar
-s -r '.*\.go' -- go run ./cmd/foo/main.go
-s -r '.*\.go' -- go run ./cmd/bar/main.go

The goal is to rebuild all dependencies of foo and bar, then start them as services. Under normal (parallel) operation there is no way to guarantee that the build is to be completed before restarting the services. This often results in the services restarting before the dependencies have been rebuilt.

I tried to use --sequential, but in that case the first line completes, then the second line runs and prevents the third from running. I actually expected both --start-service lines to execute because I interpreted "service" as a background service. IMO, practically, this makes sense. What do you think?

Alec

Binary downloads

One of this repository's proposed benefits as written in the readme: "Reflex has no dependencies. No need to install Ruby or anything like that." Yet the instillation reads:

$ go get github.com/cespare/reflex

I don't want to install go to use this library. Could this dependency be removed with binary downloads?

Command not being run in Linux

Reflex is able to watch files for changes but it doesn't run the commands in my Linux computer. Here is the verbose output of a simple ./bin/reflex -v -g '.' echo {} command:

  λ  ./bin/reflex -v  -g '.' echo {}                                                                                                                                         
Globals set at commandline                                                                                                                                                   
| --glob (-g) '[.]' (default: '[]')
| --verbose (-v) 'true' (default: 'false')
+---------
Reflex from [commandline]
| ID: 0
| Inverted regex match: "(^|/)\\.git/"
| Inverted regex match: "(^|/)\\.hg/"
| Inverted regex match: "~$"
| Inverted regex match: "\\.swp$"
| Inverted regex match: "\\.#"
| Inverted regex match: "(^|/)#.*#$"
| Inverted regex match: "(^|/)\\.DS_Store$"
| Glob match: "."
| Substitution symbol {}
| Command: [echo <filename>]
+---------

[info] fsnotify event: "./Makefile": WRITE 

Report remove events

This was reported by @maysunfaisal on #58.

We should notify on remove events. For example, if you delete a .c file, you probably want to rerun make to see if the build broke.

It's tricky because we also need to know whether an event was for a directory or not. This is exposed to the user in two ways:

  1. The path matched against the user's glob/regex is documented as having a trailing / if and only if the path is a directory.
  2. The --only-dirs and --only-files flags allow the user to only match against one or the other.

Right now we don't know this information when we get a remove event because the file/directory is already gone. To implement this, we'd need to keep our own in-memory representation of the watched tree (or at least a set of known directories); we can reference this when we get a remove event.

I anticipate annoying corner cases where this cache gets out of sync with the filesystem.

This is related to #13 and possibly #39.

Use a default config filename

Forgive me if this is a duplicate.

It would be much appreciated if a default config filename was used, such that, if I have a config file in the root directory, like reflex.conf, I do not have to specify --config='reflex.conf' when invoking reflex. Just running reflex would automatically use my default config. (Maybe also log that reflex.conf was found, and is using that.)

If such a thing has already been implemented, I apologize, I could not find it. Please point me to it.
If such a thing can't/won't be implemented, can we discuss the reasons?

executable file not found in $PATH error for an alias I have

I have an alias that I want to run when a file changes so I enter this command :

reflex -g '*.go' afl where afl is an alias that runs a command. I get an error. that executable file is not found in $PATH even though the alias is in the path.

Auto-ignore certain editor files

Tv` asked for this for emacs on IRC. Looking at this emacs wiki page, it looks like I should ignore:

  • \.#.*$
  • ^#.*#$

We also might cover vim's backup and swap files:

  • .*~$
  • .*\.swp$

I suppose we would need a flag to un-ignore them, but maybe I won't bother unless someone complains.

Reflex does not remove file watches

While reading of the code I noticed:

func watch(...) { 
...
        case e := <-watcher.Event:
            path := strings.TrimPrefix(e.Name, "./")
            if e.IsDelete() {
                watcher.RemoveWatch(path)
            }
...
}

This might not be entirely correct. RemoveWatch checks if path exists in a map. Because it's been trimmed earlier (TrimPrefix), it likely will not find it and just return an error. I believe watcher.RemoveWatch(e.Name) would have a better chance of succeeding. Also perhaps checking the returned error. An error will be returned on many occasions though, because if path is not a folder, a watch was never issued for it. So RemoveWatch will probably fail for those cases.

I'm also not sure how some platforms handle it, but deleting a file might automatically delete the associated watcher (if any) on the kernel side. Calling RemoveWatch could still be beneficial though, to clean up the structure on the Go side.

Intelligently limit watches to certain directories

It's quite common for people to run into open file limits problems because their working directory has a subdirectory with a giant tree in it. (Today's example was a node_modules dir.) There are workarounds listed at https://github.com/cespare/reflex#open-file-limits, but we could also do better directly without adding any other flags.

As I mentioned in #29 (comment), we could do something like this:

If the user passes something like -g api/*.py, instead of watching ./ and filtering against api/*.py, we can watch ./api/ and filter against *.py.

This can be extended to multiple globs and even regexps, where we can extract prefix directories.

If the user passes -g api/*.py -g tests/foo/bar/*.py, reflex can issue two watches, one in ./api/ and one in ./tests/foo/bar/.

In this way, we can automatically watch the most specific directories possible.

There are many caveats here, including:

  • We need to be careful about overlapping watches causing duplicates
  • Inverse matches need to be adjusted

Trigger events only on file contents changing

Raising this here for discussion (because I use reflex extensively and would welcome thoughts on the thinking).

In this CL I proposed updating stringer to only write to its output file if the contents changed.

But @mvdan pointed out that to do so would break the isolation rule Russ mentions in golang/go#24356 (comment)

Could we therefore have reflex operate in a mode where it only fires events when the contents of a file have changed?

Add an '--ignore' flag or similar

Reflex doesn't seem to have an easy way of ignoring certain paths. Many package management solutions for various languages (npm for node, composer for php, etc.) pull the project's dependencies into a "vendor" directory (by purpose, not necessarily by name - e.g. node uses node_modules). Since these can often be large or recursive (node) it makes sense to be able to ignore these folders for performance and FS (#6) reasons.

Imagined usage would be like so:

reflex -g '*.c' --ignore vendor/ make
# or
reflex -g '*.c' -i vendor/ make

If you'd accept a PR I can put one together.

Basic use case, "executable file not found in $PATH"

Terminal input:

$ reflex file.html "/usr/bin/firefox file.html" &
$ echo "content" > file.html
$ [00] exec: "file.html": executable file not found in $PATH

Expected result:
I was expecting "/usr/bin/firefox file.html" to run (exec /usr/bin/firefox file.html works just fine) instead of getting this error.

Why am I getting this problem? Should the unexpected output perhaps be added to README.md? Since this is presumably one of the primary use cases for this tool (which seems great if it worked as I expected).

/usr/bin is in my $PATH.

The above example works perfectly fine in when-changed, which is a similar program (written in Python).

watch the watchers

I'm having trouble running multiple watchers without regexs using the config file. The following config file thrashes, constantly restarting the processes:

-s -- python -m SimpleHTTPServer
-s -- coffee -cw js

My workaround has been to add dummy regexes like so:

-sr "asdfjkle" -- python -m SimpleHTTPServer
-sr "asdfjkle" -- coffee -cw js

Why the negative 1?

For gosh sake's I've been trying to debug my own code for a while now, and I look to your code for inspiration, and the thing that fixed it was changing

syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)
syscall.Kill(-1*cmd.Process.Pid, syscall.SIGKILL)

as you did in your code. I just thought it looked odd and there must be a reason for it, so I tried it and it fixed my code!!!

So, I just have to ask why on earth. I guess it's casting it to an int? But why does that matter?

go command complains about Ctty not valid

whenever I use the go command on MacOS X Catalina I get the following error:

jum@isis > ~/player > master > reflex -s -- go run player.go [00] Starting service [00] fork/exec /Users/jum/go/bin/go: Setctty set but Ctty not valid in child ^CInterrupted (interrupt). Cleaning up children... jum@isis > ~/player > master > go version go version devel +8194187a2d Sun May 24 01:19:35 2020 +0000 darwin/amd64 jum@isis > ~/player > master >

Anything I can do about this?

Reflex sometimes misses changes

@dmac noticed this one. He's on a Mavericks OS X laptop. From watching the debug logs, it looks like a single change can arrive that batches a write and a chmod; right now we're ignoring all attribute change events so it gets ignored.

This may be a recent fsnotify change.

Cleanup/tests

Reflex is spaghetti-ish and needs cleanup.

  • Remove global state
  • Split up long functions a bit
  • Get some basic tests going
  • Get rid of the TODOs
  • More function docs

-bash: reflex: command not found

Hey, I am new to Go so don't blame me for asking a silly question [:
This is probably even not entirely related to reflex, but to any Go package.
I have installed reflex using go get github.com/cespare/reflex and it worked. But when I reopened terminal it stopped working and it returns -bash: reflex: command not found when I enter reflex in a terminal. Writing go get github.com/cespare/reflex once again does nothing. Should I set a path correctly or something?

Crash with error message read: interrupted system call

How to reproduce

Rapidly save changes to a file, reflex crashes fairly soon (After 5-10 saves).

Error message

read: interrupted system call

Expected behavior

Reflex shouldn’t crash on interrupted system calls

Environment

Value
OS Debian GNU/Linux 7 (wheezy)
Architecture i386
Go version 1.4
Hosting provider BlueVM

Tag a release

The current release is almost four years old. Can you tag a new release?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.