bazelbuild / bazel-watcher Goto Github PK

View Code? Open in Web Editor NEW

414.0 414.0 110.0 837 KB

Tools for building Bazel targets when source files change.

License: Apache License 2.0

Python 0.19% Go 83.49% JavaScript 2.65% Shell 0.56% Starlark 13.11%

bazel build fast

bazel-watcher's People

Contributors

Stargazers

Watchers

Forkers

steren dougkoch mattmoor vladmos kamalmarhubi alexeagle jmhodges gonzojive mrmeku jin markuskobler mprobst gregmagolan pkozlowski-opensource brendandouglas lucidsoftware brendanjryan ecosia globegitter kalbasit clintharrison jchw-forks schroederc dln wardmike meteorcloudy renovate-tests igorminar justbuchanan jdban john2912 renovate-bot laurentlb zaucy yext kyliau rerion ubehebe a-ignatov-parc libsamek whilp kindlyops pward123 girtsf fjgauthier josephperrott tea-ebook mariusgrigoriu zoidyzoidzoid mbrukman excavador lewish trironkk devversion akonradi joeljeske anothergroupchat flexport franciscocpg pascalfleury fmeum djmarcin smklein isabella232 matttproud lukasholzer dymart mistic sgammon arnavk efyx testfailed tryweirdier heartless-clown kmicklas xuehuichao alex-torok armbiant philsc adamscybot wearymonkey beyondevil yousefhindy anima-os-dev xinbinhuang lorite sxlijin pauldraper jakeleventhal sluongng purkhusid ericsong xzhang-brex chenrui333 ankit-agarwal1999 gzm0 moisesvega jstuder-gh vorburger davidzbarsky-at

bazel-watcher's Issues

bazel test fails locally and on TravisCI

Not sure where to begin to track this down... failure on Travis looked like this on my PR.

INFO: Found 14 targets and 5 test targets...
INFO: Building...
[0 / 1] BazelWorkspaceStatusAction stable-status.txt
[2 / 35] Writing file external/org_golang_x_sys/unix/libgo_default_library.cgo\
_c_lib.lo-2.params
[5 / 55] Compiling external/com_github_google_protobuf/src/google/protobuf/com\
piler/java/java_map_field.cc [for host]
[26 / 74] Compiling external/com_github_google_protobuf/src/google/protobuf/em\
pty.pb.cc [for host]
[57 / 105] Compiling external/com_github_google_protobuf/src/google/protobuf/c\
ompiler/java/java_message_field.cc [for host]
[84 / 132] Compiling external/com_github_google_protobuf/src/google/protobuf/u\
til/internal/utility.cc [for host]
[188 / 243] Still waiting for 2 jobs to complete:
      Running (local):
        SkylarkAction external/go_stdlib_linux_amd64_cgo/bin/go [for host], 10\
 s
      Scheduling:
        Linking external/com_github_google_protobuf/protoc [for host], 10 s
[215 / 291] GoCompile external/io_bazel_rules_go/go/tools/builders/~stripped~a\
sm~/github.com/bazelbuild/rules_go/go/tools/builders/asm.a [for host]
[222 / 291] GoCompile e2e/simple/~stripped~sample~/github.com/bazelbuild/bazel\
-watcher/e2e/sample.a
[244 / 291] Compiling external/org_golang_x_sys/unix/go_default_library.cgo.di\
r/external_org_golang_x_sys_unix_syscall_linux.cgo2.c
PASS: //bazel:go_default_test
PASS: //ibazel/command:go_default_test
PASS: //ibazel:go_default_test
PASS: //e2e:go_default_test
INFO: Elapsed time: 242.429s, Critical Path: 42.66s
java.lang.IllegalStateException
	at com.google.common.base.Preconditions.checkState(Preconditions.java:429)
	at com.google.devtools.build.lib.util.Preconditions.checkState(Preconditions.java:196)
	at com.google.devtools.build.lib.runtime.TestResultAnalyzer.differentialAnalyzeAndReport(TestResultAnalyzer.java:115)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.analyzeTestResults(TestCommand.java:172)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.doTest(TestCommand.java:145)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.exec(TestCommand.java:104)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:600)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:351)
	at com.google.devtools.build.lib.runtime.BlazeRuntime.batchMain(BlazeRuntime.java:755)
	at com.google.devtools.build.lib.runtime.BlazeRuntime.main(BlazeRuntime.java:551)
	at com.google.devtools.build.lib.bazel.BazelMain.main(BazelMain.java:57)
java.lang.IllegalStateException
	at com.google.common.base.Preconditions.checkState(Preconditions.java:429)
	at com.google.devtools.build.lib.util.Preconditions.checkState(Preconditions.java:196)
	at com.google.devtools.build.lib.runtime.TestResultAnalyzer.differentialAnalyzeAndReport(TestResultAnalyzer.java:115)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.analyzeTestResults(TestCommand.java:172)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.doTest(TestCommand.java:145)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.exec(TestCommand.java:104)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:600)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:351)
	at com.google.devtools.build.lib.runtime.BlazeRuntime.batchMain(BlazeRuntime.java:755)
	at com.google.devtools.build.lib.runtime.BlazeRuntime.main(BlazeRuntime.java:551)
	at com.google.devtools.build.lib.bazel.BazelMain.main(BazelMain.java:57)
The command "bazel \
  --output_base=$HOME/.cache/bazel \
  --batch \
  --host_jvm_args=-Xmx500m \
  --host_jvm_args=-Xms500m \
  --bazelrc=.test-bazelrc \
  test \
  --experimental_repository_cache="$HOME/.bazel_repository_cache" \
  --local_resources=400,1,1.0 \
  --test_tag_filters=-dev \
  //...
" exited with 37.

Locally I can something similar but different using bazel --bazelrc=.test-bazelrc test --test_tag_filters=-dev ... on both master and on my live-reload branch:

NFO: Found 14 targets and 5 test targets...
INFO: Elapsed time: 6.972s, Critical Path: 0.74s
java.lang.IllegalStateException
	at com.google.common.base.Preconditions.checkState(Preconditions.java:429)
	at com.google.devtools.build.lib.util.Preconditions.checkState(Preconditions.java:196)
	at com.google.devtools.build.lib.runtime.TestResultAnalyzer.differentialAnalyzeAndReport(TestResultAnalyzer.java:115)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.analyzeTestResults(TestCommand.java:176)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.doTest(TestCommand.java:149)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.exec(TestCommand.java:104)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:600)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:351)
	at com.google.devtools.build.lib.runtime.BlazeRuntime.batchMain(BlazeRuntime.java:755)
	at com.google.devtools.build.lib.runtime.BlazeRuntime.main(BlazeRuntime.java:551)
	at com.google.devtools.build.lib.bazel.BazelMain.main(BazelMain.java:57)
java.lang.IllegalStateException
	at com.google.common.base.Preconditions.checkState(Preconditions.java:429)
	at com.google.devtools.build.lib.util.Preconditions.checkState(Preconditions.java:196)
	at com.google.devtools.build.lib.runtime.TestResultAnalyzer.differentialAnalyzeAndReport(TestResultAnalyzer.java:115)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.analyzeTestResults(TestCommand.java:176)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.doTest(TestCommand.java:149)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.exec(TestCommand.java:104)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:600)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:351)
	at com.google.devtools.build.lib.runtime.BlazeRuntime.batchMain(BlazeRuntime.java:755)
	at com.google.devtools.build.lib.runtime.BlazeRuntime.main(BlazeRuntime.java:551)
	at com.google.devtools.build.lib.bazel.BazelMain.main(BazelMain.java:57)

Any ideas? I'd like to write a test for the live reload but this is blocking me.

ibazel does not pass arguments to Bazel

I want to run my tests in parallel and ibazel adds --test_output=streamed to the args when running bazel test, which is unexpected and undocumented. I would expect it to just run exactly the same had the command line been bazel test.

Versioning needed

It would be good to get some some version info so you could type ibazel --version and see what you're running. There doesn't seem to be a standard way to version go project however.

Where would the version live? Should it be sematic versioning? Right now there is an version number npm/package.json. Should this be the source of truth for the version number? If so, how to consume that number and get it into the go binary?

How would releases work? Update version number, tag commit and publish to npm? Noticed that there is a version 0.0.1 published to npm.

ibazel watch stops working after 2 refreshes

Originally reported at:
angular/angular#21770

Has been observed by several engineers on the Angular team.

ibazel doesn't support multiple target patterns even though it is documented to do so

Usage:

ibazel build|test|run targets...

yet from main.go

	target := os.Args[2]

PKGBUILD for Arch Linux

Just FYI, I added an unofficial package for Arch Linux at: https://aur.archlinux.org/packages/ibazel-git/.

If you have any suggestions or issues with it, please let me know.

Note: It currently doesn't build without the changes from #61.

Error asm while building ibazel

At the current HEAD, bazel build ibazel breaks:

ERROR: /home/thomas/.cache/bazel/_bazel_thomas/ab22553f6e662f61e6ff0f9adcaf6149/external/io_bazel_rules_go/go/tools/builders/BUILD.bazel:22:1: in go_tool_binary rule @io_bazel_rules_go//go/tools/builders:asm: 


The following files have no generating action:
external/io_bazel_rules_go/go/tools/builders/asm
.
ERROR: Analysis of target '//ibazel:ibazel' failed; build aborted: Analysis of target '@io_bazel_rules_go//go/tools/builders:asm' failed; build aborted.

publish gh release binaries

something like https://github.com/bazelbuild/buildtools/releases would be ideal. using a naming convention what would allow for conditional download based on platform using standard uname conventions would be even better

ibazel writes IBAZEL_BUILD_COMPLETED FAILURE to stdin even for successful builds

In my prototype devserver that uses ibazel from HEAD, I had to listen for FAILURE
https://github.com/alexeagle/angular-bazel-example/blob/98af46d27bab378e4ed5f14799c80578f0287d55/airplane/devserver.js#L9
because ibazel never reports SUCCESS

ibazel doesn't watch changes in .bzl files that affect the build

I have a .bzl file that defines a rule, then ibazel build the rule. That .bzl file isn't watched for changes.

broken when running from WORKSPACE/subdirectory

I get lstat errors about no such file or directory, it seems like the paths being watched are still workspace-relative but then don't exist relative to the pwd when in a subdirectory of the workspace.

Likely leaking file descriptors

We are moving core Angular development to an ibazel workflow. (First critical adoption on our team)

@mhevery notices that after running ibazel test on Mac for many iterations, it errors:

State: WAIT
Changed: "packages/core/src/render3/di.ts". Rebuilding...
State: DEBOUNCE_RUN
State: RUN
Testing packages/core/test/render3
INFO: Analysed target //packages/core/test/render3:render3 (0 packages loaded).
INFO: Found 1 test target...
ERROR: Caught I/O exception: java.io.FileNotFoundException: /private/var/tmp/_bazel_misko/1126e726ab9a0d0ed3e2392a7bbe5a3c/execroot/angular_src/bazel-out/darwin-fastbuild/testlogs/packages/core/test/render3/render3/test.cache_status (Too many open files in syst
em)
ERROR: /Users/misko/work/angular/packages/core/test/render3/BUILD.bazel:22:1:  failed: unexpected I/O exception: /private/var/tmp/_bazel_misko/1126e726ab9a0d0ed3e2392a7bbe5a3c/execroot/angular_src/bazel-out/darwin-fastbuild/testlogs/packages/core/test/render3/r
ender3/test.cache_status (Too many open files in system)
ERROR: I/O error while writing action log: /private/var/tmp/_bazel_misko/1126e726ab9a0d0ed3e2392a7bbe5a3c/action_cache/action_cache_v12.tmp (Too many open files in system)
INFO: Elapsed time: 5.182s, Critical Path: 4.60s
FAILED: Build did NOT complete successfully
//packages/core/test/render3:render3                                  NO STATUS

Executed 0 out of 1 test: 1 was skipped.
Build error: exit status 1
State: WAIT

and that restarting ibazel fixes it for a while.

Maybe we should add some instrumentation to help us see how many file handles are used, maybe both in end-user mode and also under unit tests, to help confirm if there is a leak?

Link error "-pie and -r are incompatible"

..../external/org_golang_x_sys/unix/BUILD.baze
bazel build ibazel:ibazel. Got link error:
l:3:1: Linking ../org_golang_x_sys/unix/go_default_library.cgo.dir/_all.o failed (Exit 1)
/usr/bin/ld.gold: fatal error: -pie and -r are incompatible
collect2: error: ld returned 1 exit status

Environment info
Ubuntu 17.04
bazel version 0.5.4

When reverse the go build rule to 0.5.2, it worked.

Add Windows support

Currently things seem pretty unhappy on Windows, and it'd be really awesome if support was added. fsnotify already supports Windows, which from a brief scan of the code looks like the most platform-specific thing.

Windows 10
Go 1.9.2
Bazel 0.8.1

Attempting bazel build //ibazel from cmd leads to:

λ bazel build //ibazel
WARNING: C:/users/ben/appdata/local/temp/_bazel_ben/fa1bxbsu/external/com_github_bazelbuild_bazel_integration_testing/WORKSPACE:1: Workspace name in C:/users/ben/appdata/local/temp/_bazel_ben/fa1bxbsu/external/com_github_bazelbuild_bazel_integration_testing/WORKSPACE (@build_bazel_integration_testing) does not match the name given in the repository's definition (@com_github_bazelbuild_bazel_integration_testing); this will cause a build error in future versions
ERROR: D:/tools/bazel-watcher/ibazel/BUILD:30:1: no such package '@com_github_fsnotify_fsnotify//': failed to fetch com_github_fsnotify_fsnotify: 2017/12/19 18:02:56 --remote should be used with the --vcs flag. If this is an import path, use --importpath instead.
 and referenced by '//ibazel:go_default_library'
ERROR: Analysis of target '//ibazel:ibazel' failed; build aborted: no such package '@com_github_fsnotify_fsnotify//': failed to fetch com_github_fsnotify_fsnotify: 2017/12/19 18:02:56 --remote should be used with the --vcs flag. If this is an import path, use --importpath instead.
INFO: Elapsed time: 1.763s
FAILED: Build did NOT complete successfully (19 packages loaded)

This may be a go_repository issue.

If running from the msys bash the above doesn't trigger but instead nothing happens:

$ bazel build //ibazel
WARNING: C:/users/ben/appdata/local/temp/_bazel_ben/fa1bxbsu/external/com_github_bazelbuild_bazel_integration_testing/WORKSPACE:1: Workspace name in C:/users/ben/appdata/local/temp/_bazel_ben/fa1bxbsu/external/com_github_bazelbuild_bazel_integration_testing/WORKSPACE (@build_bazel_integration_testing) does not match the name given in the repository's definition (@com_github_bazelbuild_bazel_integration_testing); this will cause a build error in future versions
INFO: Analysed 0 targets (0 packages loaded).
INFO: Found 0 targets...
INFO: Elapsed time: 0.345s, Critical Path: 0.00s
INFO: Build completed successfully, 1 total action

Interestingly a git prompt window will pop up for a split second, which looks like it's running the workplace_status.sh script.

I'm not sure what issues may lie beyond those. I am but a simple C++ developer and the world of go packaging frightens and confuses me :)

Notify long-running processes when a build finishes

After #29 a server process can stay running while ibazel rebuilds its runtime file resources. But sometimes the process needs a callback each time the build finishes, so it knows to go read the resources off disk again.

For example, Karma JS test runner depends on this to know to trigger the tests again in watch mode. It should not actually watch the files on disk (in my case there are 10k files and it's not practical to watch them all)

There should be a protocol that uses stdin of the process to send lines like IBAZEL_BUILD_COMPLETED FAILED, like
http://google3/devtools/iblaze/daemon.go?l=349
for best internal/external portability.

Call hotswap when ibazel run'ing Java binaries

I don't run any Java binaries so this is a FR for someone who does. It would be possible to detect that you're running a java_binary under the hood and leave the command running in the background then hot swap in the newest version of the code to prevent having to restart the server.

Crash when passing --profile

$ ibazel test --profile=myprofile.out  test:golden_test
State: QUERY
Querying for BUILD files...
Error running Bazel exit status 2
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x40ec1e3]

goroutine 1 [running]:
main.(*IBazel).queryForSourceFiles(0xc42009a000, 0xc420018240, 0x3f, 0x0, 0xc4200cc000, 0xc4200ce000)
	ibazel/ibazel.go:316 +0x1b3
main.(*IBazel).watchFiles(0xc42009a000, 0xc420018240, 0x3f, 0xc420062060)
	ibazel/ibazel.go:345 +0x5a
main.(*IBazel).iteration(0xc42009a000, 0x413b590, 0x4, 0xc420055e18, 0xc42000c060, 0x2, 0x2, 0xc42001c0f0, 0x28)
	ibazel/ibazel.go:204 +0x91e
main.(*IBazel).loop(0xc42009a000, 0x413b590, 0x4, 0xc420055e18, 0xc42000c060, 0x2, 0x2, 0xc420055e20, 0xc420010170)
	ibazel/ibazel.go:176 +0xf6
main.(*IBazel).Test(0xc42009a000, 0xc42000c060, 0x2, 0x2, 0x2, 0x2)
	ibazel/ibazel.go:168 +0x80
main.handle(0xc42009a000, 0x7fff5fbff7ff, 0x4, 0xc420010160, 0x2, 0x2)
	ibazel/main.go:110 +0x22e
main.main()
	ibazel/main.go:99 +0x1bf

Home for ibazel-benchmark-runner code?

@achew22 @alexeagle

Looking for a home for this code? https://github.com/gregmagolan/ibazel-benchmark-runner

Would it make sense to house it in a folder under this repo since its tied to ibazel? Thinking it would get published to npm at @bazel/benchmark-runner or something like that.

Make it possible to not restart the server and only rebuild the data dependencies of a job (while leaving it running)

To quote @mprobst,

I found that to be more useful than respawning the binary. In
particular for servers that often take a long time to start, it's useful to
rebuild their assets (think: JS, TypeScript, etc) without restarting the
server.

ibazel occasionally stops running on changes

I haven't got much of a repro. I have ibazel run //some:target or ibazel build //some:target going, and it works for a while and eventually stops. I have fallen back to my earlier general purpose watching tool, but would like to be able to advise colleagues to use ibazel as it matures.

Is this a familiar issue to anyone?

Seen with ibazel built from commit 15a1ebd on Linux:

$ lsb_release --description                                               
Description:    Debian GNU/Linux testing (buster)   
$ uname -r           
4.11.0-1-amd64

ibazel doesn't watch changes in BUILD files that affect the build

If I run ibazel on some targets, and then totally mess up the BUILD file defining those targets (removing needed dependencies), they are not rebuilt/retested.

Add explanation of where the ibazel binary lives to the README

These are the current installation instructions:

Compile the //ibazel target inside this repo and copy the source file onto your $PATH.

For a developer new to the bazel ecosystem, this isn't very approachable. There are a lot of missing steps.

This would be more helpful:

Install bazel from the GET BAZEL link on bazel.build
Clone this repo
git clone https://github.com/bazelbuild/bazel-watcher.git
Build this repo
```
cd bazel-watcher
bazel build //ibazel
```
Add the ibazel binary to your path (for instance, in ~/.bashrc).

The binary will be in an architecture-specific folder inside bazel-out; for instance: /Projects/bazel-watcher/bazel-out/darwin_x86_64-fastbuild/bin/ibazel/

file watcher triggers too much

If I carefully write to a file with a simple unix tool, like
echo "" >> file then I see exactly one rebuild when watching file.
But with any editor, after saving a file I see a bunch of rebuilds. On linux with VScode, I saw exactly two rebuilds, but on Mac it triggers endlessly and the ibazel command needs to be killed.

e2e non-determinism after addition of lifecycle hooks

I've run the e2e tests repeatedly with --cache_test_results=no on every recent commit, and I've concluded that both the live_reload and simple tests started failing non-deterministically on the commit that added lifecycle hooks (b87e958).

Your platform/architecture combination NaN is not yet supported. Windows 10

Hello

all prerequisites installed

cannot start devserver, Windows 10

$ ./node_modules/.bin/ibazel run src:devserver
FATAL: Your platform/architecture combination NaN is not yet supported.
    Follow install instructions at https://github.com/bazelbuild/bazel-watcher/blob/master/README.md to compile for your system.

package.json

{
  "scripts": {
    "build": "bazel run @yarn//:yarn",
    "prebuildifier": "bazel build @com_github_bazelbuild_buildtools//buildifier",
    "buildifier": "find . -type f \\( -name BUILD -or -name BUILD.bazel \\) ! -path \"./node_modules/*\" | xargs $(bazel info bazel-bin)/external/com_github_bazelbuild_buildtools/buildifier/buildifier"
  },
  "dependencies": {
    "@bazel/ibazel": "^0.2.0",
    "@bazel/typescript": "^0.6.0",
    "rxjs": "^5.5.6",
    "typescript": "^2.6.2"
  }
}

Thank you

Replace the "MAGIC" tag with something permanent; document it

bazel-watcher/ibazel/ibazel.go

Line 265 in 35e574c

if contains(attr.StringListValue, "IBAZEL_MAGIC_TAG") {

@achew22 says the term IBAZEL_MAGIC_TAG was meant as a placeholder. I want to start depending on this feature so we should mature it to something we can live with, and document how it's used.

Too many files error

I tried using the tool on my project, and get the following error message many times:

Error: too many open files

Leaking watchers will lead to resource exhaustion

By creating watchers and not removing them we will inch closer to hitting the inode watcher limit. Normally this is in the neighborhood of 10k but it is not impossible that you would have a project near that limit and get to it with 8 hours of iteration.

Additionally we should detect when this limit is near (or reached) and notify the user how to fix it.

avoid version problems for node users

There are two ways:

recommend PATH=./node_modules/.bin:$PATH so that if you are in a project directory where [email protected] is installed, you get that version. @vsavkin suggests this. But if you're in a child directory (common for bazel users since it does a good job in this case) it doesn't work
when ibazel runs, it checks if there is a locally installed copy, and exec()s that one instead. gulp and angular-cli do it this way. @hansl suggests this

Fix CI status badges

The status badges in README.md that point to ci.bazel.io are currently pointing to dead links. Can we update those? Here's the page for that build: https://ci.bazel.io/job/bazel-watcher/

Document how this compares with the `--watchfs` bazel option

I just noticed there's already an option to watch the input files.

eg. here's a commit where @damienmg improved how it's wired on Mac:
bazelbuild/bazel@990fa00

But IIUC that option doesn't actually put Bazel in watch mode.

does ibazel benefit from using that flag as well?
should we add docs on that flag to clarify that it's not a watch mode and maybe point here?

installs error

I definitly don't understand what's wrong here:

PS C:\Users\ee\Desktop\r\bazel-watcher-master> bazel build //ibazel
WARNING: C:/users/ee/appdata/local/temp/_bazel_ee/vzadcu2y/external/com_github_bazelbuild_bazel_integration_testing/WORKSPACE:1: Workspace name in C:/users/ee/appdata/local/temp/_bazel_ee/vzadcu2y/external/com_github_bazelbuild_bazel_integration_testing/WORKSPACE (@build_bazel_integration_testing) does not match the name given in the repository's definition (@com_github_bazelbuild_bazel_integration_testing); this will cause a build error in future versions.
DEBUG: C:/users/ee/appdata/local/temp/_bazel_ee/vzadcu2y/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:37:3:
Auto-Configuration Warning: 'BAZEL_VC' is not set, start looking for the latest Visual C++ installed.
.
DEBUG: C:/users/ee/appdata/local/temp/_bazel_ee/vzadcu2y/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:37:3:
Auto-Configuration Warning: Looking for VS%VERSION%COMNTOOLS environment variables,eg. VS140COMNTOOLS
.
DEBUG: C:/users/ee/appdata/local/temp/_bazel_ee/vzadcu2y/external/bazel_tools/tools/cpp/lib_cc_configure.bzl:37:3:
Auto-Configuration Warning: Visual C++ build tools found at C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\
.
DEBUG: C:/users/ee/appdata/local/temp/_bazel_ee/vzadcu2y/external/io_bazel_rules_go/go/private/toolchain.bzl:97:5: java.io.IOException: CreateProcess(): Le fichier spÚcifiÚ est introuvable.
.
ERROR: C:/users/ee/appdata/local/temp/_bazel_ee/vzadcu2y/external/io_bazel_rules_go/go/toolchain/BUILD.bazel:8:1: every rule of type _go_toolchain implicitly depends upon the target '@go_sdk//:packages.txt', but this target could not be found because of: no such package '@go_sdk//': Traceback (most recent call last):
        File "C:/users/ee/appdata/local/temp/_bazel_ee/vzadcu2y/external/io_bazel_rules_go/go/private/toolchain.bzl", line 45
                _prepare(ctx)
        File "C:/users/ee/appdata/local/temp/_bazel_ee/vzadcu2y/external/io_bazel_rules_go/go/private/toolchain.bzl", line 98, in _prepare
                fail("failed to list standard package...")
failed to list standard packages.
ERROR: Analysis of target '//ibazel:ibazel' failed; build aborted.
INFO: Elapsed time: 97,087s

It seems bazel's working cause I succeed to install bazel build //examples/cpp:hello-world from bazel-master/examples

Watch for new files in all packages

If a BUILD file declares something using a glob, a new file should result in a rebuild. The creation of a BUILD file in some already-existing directory affects the BUILD graph.

All directories containing BUILD files and all their descendent directories should be watched to detect new files. When an ordinary file is created, the watch list should be updated. When a BUILD file is created, the build file watch list should be updated.

Automatically apply buildozer commands from warnings

Hi,
I've heard that the internal version of Bazel-watcher (iBazel) knows to watch for log warnings in Bazel outputs which are buildozer commands and to automatically apply them.
The need for this (for me) is to be able to automatically apply strict java/scala deps warnings.
This means a developer can use code from transitive targets, the rule will warn against it with a buildozer command (saying which target needs to be added explicitly where) and ibazel will automatically apply it.
Is there an interest to extend ibazel to that area?

Request: add CLI option to set debounce delay value

It would be nice to set the debounce delay value through a CLI option.
100ms is hard-coded today, but some set ups may require longer values.

Resource limit hit on OSX

In order to get iBazel to work on OSX I needed to increase the resource limit for open files.

Used the approach suggested here: https://superuser.com/questions/827984/open-files-limit-does-not-work-as-before-in-osx-yosemite

It seems like there is an entirely different method for changing the open files limit for each version of OS X!

For OS X Sierra (10.12.X) you need to:

In Library/LaunchDaemons create a file named limit.maxfiles.plist and paste the following in (feel free to change the two numbers (which are the soft and hard limits, respectively):

Label limit.maxfiles ProgramArguments launchctl limit maxfiles 64000 524288 RunAtLoad ServiceIPC 2. Change the owner of your new file:
sudo chown root:wheel /Library/LaunchDaemons/limit.maxfiles.plist
3. Load these new settings:

sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist
4. Finally, check that the limits are correct:

launchctl limit maxfiles

This works for OSX 10.13 (High Sierra), however I get warnings whenever running iBazel that look like this: Warning: failed to raise resource limit 8 to 524288: Invalid argument

Not sure what the best solution is. If a manual resource limit change on OSX is needed then perhaps it should be suggested in the README. From the warning above it seems that iBazel is attempting to increase the resource limit but failing?

There are also side-effects to increasing the resource limit manually as it may break other applications. Adobe Illustrator, for example, doesn't open up anymore for me with the error: unable to set maximum number of files to be opened.

Watches 0 files when target is in a different workspace

$ ibazel test @angular//src/sw-config:test
State: QUERY
Querying for BUILD files...
WARNING: Switched off --watchfs again... temporarily falling back to manually checking files for changes
Watching: 0 files
Querying for source files...
Watching: 0 files
State: RUN
Testing @angular//src/sw-config:test
INFO: Analysed target @angular//src/sw-config:test (0 packages loaded).
INFO: Found 1 test target...
Target @angular//src/sw-config:test up-to-date:
  /private/var/tmp/_bazel_arick/c12f4ea4eaf7a974edf6cb95dd136227/execroot/angular_src/bazel-out/darwin-fastbuild/bin/external/angular/src/sw-config/darwin_amd64_stripped/test
INFO: Elapsed time: 0.270s, Critical Path: 0.00s
INFO: Build completed successfully, 1 total action
@angular//src/sw-config:test                                    (cached) PASSED in 0.1s

Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
State: WAIT

$ bazel query 'deps(@angular//src/sw-config:test)' | sort | grep _test
go_test rule @angular//src/sw-config:test
source file @angular//src/sw-config:glob_test.go
source file @angular//src/sw-config:manifest_test.go
source file @com_github_google_protobuf//:src/google/protobuf/arena_test_util.h
source file @com_github_google_protobuf//:src/google/protobuf/map_lite_test_util.h
source file @com_github_google_protobuf//:src/google/protobuf/map_test_util.h
source file @com_github_google_protobuf//:src/google/protobuf/map_test_util_impl.h
source file @com_github_google_protobuf//:src/google/protobuf/util/internal/type_info_test_helper.cc
source file @com_github_google_protobuf//:src/google/protobuf/util/internal/type_info_test_helper.h

Changing manifest_test.go failed to rerun the tests.

process is left running after ctrl-c

I'm on the latest npm release. I notice that a single ctrl-c kills the ibazel process now, where it used to require two. And now the process being run is not shut down, so I have to go find and kill it before I can run again.

/cc @gregmagolan

alexeagle@alexeagle:~/Projects/angular-bazel-example$ ./node_modules/.bin/ibazel
iBazel - Version v0.2.0
...
alexeagle@alexeagle:~/Projects/angular-bazel-example$ ps aux | grep devserver
alexeag+ 191626  0.0  0.0  23776  2192 pts/42   S+   09:30   0:00 grep --color=auto devserver
alexeagle@alexeagle:~/Projects/angular-bazel-example$ ./node_modules/.bin/ibazel run src:devserver
State: QUERY
Querying for BUILD files...
Watching: 5 files
Querying for source files...
Watching: 3871 files
State: RUN
Runing src:devserver
Launching with live reload
Launching with notifications
[live reload] listening on :35729
INFO: Analysed target //src:devserver (44 packages loaded).
INFO: Found 1 target...
Target //src:devserver up-to-date:
  dist/bin/src/devserver.MF
  dist/bin/src/scripts_devserver.MF
  dist/bin/src/devserver
INFO: Elapsed time: 3.476s, Critical Path: 1.57s
INFO: Build completed successfully, 1 total action

Starting...Triggering live reload
[live reload] requesting reload: reload
State: WAIT
Serving livereload script from http://localhost:35729/livereload.js?snipver=1
Server listening on http://alexeagle.mtv.corp.google.com:5432/
[live reload] connected
Changed: "src/lib/file.ts". Rebuilding...
State: DEBOUNCE_RUN
State: DEBOUNCE_RUN
State: RUN
Runing src:devserver
Notifying of changes
INFO: Analysed target //src:devserver (0 packages loaded).
INFO: Found 1 target...
Target //src:devserver up-to-date:
  dist/bin/src/devserver.MF
  dist/bin/src/scripts_devserver.MF
  dist/bin/src/devserver
INFO: Elapsed time: 2.491s, Critical Path: 0.22s
INFO: Build completed successfully, 2 total actions
SUCCESS
Triggering live reload
[live reload] requesting reload: reload
State: WAIT
[live reload] websocket: close 1001 (going away)
[live reload] connected
Changed: "src/lib/file.ts". Rebuilding...
State: DEBOUNCE_RUN
State: DEBOUNCE_RUN
State: RUN
Runing src:devserver
Notifying of changes
INFO: Analysed target //src:devserver (0 packages loaded).
INFO: Found 1 target...
Target //src:devserver up-to-date:
  dist/bin/src/devserver.MF
  dist/bin/src/scripts_devserver.MF
  dist/bin/src/devserver
INFO: Elapsed time: 2.285s, Critical Path: 0.20s
INFO: Build completed successfully, 2 total actions
SUCCESS
Triggering live reload
[live reload] requesting reload: reload
State: WAIT
[live reload] websocket: close 1001 (going away)
[live reload] connected
^C
Subprocess killed from getting SIGINT

alexeagle@alexeagle:~/Projects/angular-bazel-example$ 
alexeagle@alexeagle:~/Projects/angular-bazel-example$ ps aux | grep devserver
alexeag+ 192013  0.1  0.0  17976 11324 pts/42   Sl   09:32   0:00 /usr/local/google/home/alexeagle/Projects/angular-bazel-example/node_modules/@bazel/ibazel/bin/linux_amd64/ibazel run src:devserver
alexeag+ 195149  0.0  0.0  23776  2144 pts/42   S+   09:33   0:00 grep --color=auto devserver

query fails quietly, easy to overlook and hard to debug

When running ibazel on the tsickle repro, I get:

State: QUERY
Querying for BUILD files...
Error running Bazel exit status 2

And goes on to do a single run, making it easy to overlook.

I can give you a better repro, or maybe error reporting could be improved between ibazel and bazel to report the query error.

Sad error reporting: panic

Built from head
db7d755

$ ibazel test examples/ts
State: QUERY
Querying for BUILD files...
Error running Bazel exit status 7
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x40eb463]

goroutine 1 [running]:
main.(*IBazel).queryForSourceFiles(0xc42009e000, 0xc42001e120, 0x22, 0x0, 0xc4200c4000, 0xc4200c6000)
	ibazel/ibazel.go:316 +0x1b3
main.(*IBazel).watchFiles(0xc42009e000, 0xc42001e120, 0x22, 0xc420066060)
	ibazel/ibazel.go:345 +0x5a
main.(*IBazel).iteration(0xc42009e000, 0x413a810, 0x4, 0xc420057e18, 0xc4200483b0, 0x1, 0x1, 0x7fff5fbffa04, 0xb)
	ibazel/ibazel.go:204 +0x91e
main.(*IBazel).loop(0xc42009e000, 0x413a810, 0x4, 0xc420057e18, 0xc4200483b0, 0x1, 0x1, 0xc420057e20, 0xc4200100e0)
	ibazel/ibazel.go:176 +0xf6
main.(*IBazel).Test(0xc42009e000, 0xc4200483b0, 0x1, 0x1, 0x1, 0x1)
	ibazel/ibazel.go:168 +0x80
main.handle(0xc42009e000, 0x7fff5fbff9ff, 0x4, 0xc4200100e0, 0x1, 0x1)
	ibazel/main.go:110 +0x22e
main.main()
	ibazel/main.go:99 +0x1bf
alexeagle-macbookpro:rules_karma alexeagle$ bazel build examples/ts
ERROR: error loading package 'examples/ts': Encountered error while reading extension file 'defs.bzl': no such package '@build_bazel_rules_typescript//': /private/var/tmp/_bazel_alexeagle/94bb2099ab3cc28ce303572fe6ffae30/external/build_bazel_rules_typescript must be an existing directory.
INFO: Elapsed time: 0.125s

Communicate to a long-running process that it will get stdin notifications

There should also be an environment variable so the long-running process knows that it will get input on stdin.
Otherwise the program cannot know if it was launched under bazel or ibazel, if it expects the latter, then under bazel it will hang waiting for stdin that never arrives

if (process.env.IBAZEL_NOTIFY_CHANGES == 'y') {
  // rely on stdin to find out about changes
} else {
  // do our own file watching, perhaps
}

Deceptive warning message when running iBazel from npm dist

WARNING: no ibazel version found in your node_modules.
        We recommend installing a devDependency on ibazel so you use the same
        version as other engineers on this project.
        Using the globally installed version at ibazel-benchmark-runner/node_modules/@bazel/ibazel

Can be seen here: https://github.com/gregmagolan/ibazel-benchmark-runner

Run:

yarn
yarn test

query for tags doesn't read through alias

I thought it would be clever to improve ergonomics like this:

BUILD

alias(name = "devserver", actual = "//src:devserver")

src/BUILD

my_rule(name = "devserver", tags=["ibazel_live_reload", "ibazel_notify_changes"])

but if I now ibazel run :devserver it doesn't recognize the tags.

/cc @gregmagolan

ibazel not exiting on Ctrl-C on Mac OS after shutting down child process

Not sure what's going on elsewhere, but when I run a server with ibazel, and hit ctrl-c, I get

 Subprocess killed from getting SIGINT

but the prompt freezes and the ibazel process does not exit.

This is happening with current HEAD, e8cbba3

Publish 0.3.0 to npm

@achew22 Can you make a release that includes the profiler?

ibazel run not forwarding ctrl-c to processes

I have a server I use ibazel run on that uses Go's Server.Shutdown on syscall.SIGTERM and os.Interrupt on mac OS 10.12.6. Sometimes, ibazel seems to leak a process out the side, causing the server on the next reboot to fail because the port is already claimed by that leaked process.

Haven't dug into the code yet to figure out the deal, but dealt and fixed with similar issues in http://github.com/jmhodges/justrun. (The fix, without looking at ibazel's code, might be some stuff like https://github.com/jmhodges/justrun/blob/master/cmdwrap.go#L23-L25 and https://github.com/jmhodges/justrun/blob/master/cmdwrap.go#L41-L46 to work probably)

data race in ibazel

Here's the stacktrace of a data race with the latest ibazel e8cbba3. It came up in normal use.

It's the same one I mentioned in #35

==================
WARNING: DATA RACE
Read at 0x00c42008c788 by goroutine 11:
  github.com/bazelbuild/bazel-watcher/ibazel/command.(*defaultCommand).IsSubprocessRunning()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/command/command.go:75 +0xae
  main.(*IBazel).handleSignals()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/ibazel.go:96 +0x478
  main.New.func1()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/ibazel.go:83 +0x38

Previous write at 0x00c42008c788 by main goroutine:
  os/exec.Command()
      /usr/local/Cellar/go/1.9.1/libexec/src/os/exec/exec.go:143 +0x157
  github.com/bazelbuild/bazel-watcher/ibazel/command.start()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/command/command.go:58 +0x397
  github.com/bazelbuild/bazel-watcher/ibazel/command.(*defaultCommand).Start()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/command/default_command.go:64 +0x161
  github.com/bazelbuild/bazel-watcher/ibazel/command.(*defaultCommand).NotifyOfChanges()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/command/default_command.go:75 +0x46
  main.(*IBazel).run()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/ibazel.go:281 +0x155
  main.(*IBazel).(main.run)-fm()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/ibazel.go:158 +0x5f
  main.(*IBazel).iteration()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/ibazel.go:218 +0x656
  main.(*IBazel).loop()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/ibazel.go:176 +0x126
  main.(*IBazel).Run()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/ibazel.go:158 +0x169
  main.handle()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/main.go:130 +0x1cf
  main.main()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/main.go:116 +0x38a

Goroutine 11 (running) created at:
  main.New()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/ibazel.go:81 +0x1d0
  main.main()
      /Users/jmhodges/src/github.com/bazelbuild/bazel-watcher/ibazel/main.go:109 +0x202
==================

Track files who are replaced (by mv operations) when being saved

Some editors writes files by writing them to a file then moving them to replace the file they are writing. I presume this is for atomicity reasons on some file systems. Since the inode changes files are occasionally lost by ibazel which prevents it from working when you write the file the second time.

To fix this we can readd a watch whenever a file is moved, replacing the original file.

Don't write all the watched file locations

In my build graph, there are hundreds of source files transitively depended on by the target I iblaze

Printing the list of files is spamming my terminal. Also, I'm not sure how this is useful to an end user, if they need to debug what is being watched you could just print an ibazel query ... command that would show them.

Perhaps if you do think it's useful to print the list, it should be a CLI option? (but I think increasing the CLI surface should be done sparingly)

When initial build fails, ibazel should exit

$ ibazel build //example:all
Error running Bazel exit status 7
Building //example:all
Files changed rebuilding...
ERROR: while parsing '//example:all': no such package 'example': BUILD file not found on package path.
INFO: Elapsed time: 0.182s
Build error: exit status 1
[ tool is now apparently hung ]

I think it's better to exit in this case (like iblaze does)

bazelbuild / bazel-watcher Goto Github PK

bazel-watcher's People

Contributors

Stargazers

Watchers

Forkers

bazel-watcher's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs