google / kati

An experimental GNU make clone

kati

kati is an experimental GNU make clone. The main goal of this tool is to speed-up incremental build of Android.

Currently, kati does not offer a faster build by itself. It instead converts your Makefile to a ninja file.

Development

Building:

$ make ckati

The above command produces a ckati binary in the project root.

Testing (best ran in a Ubuntu 22.04 environment):

$ make test
$ go test --ckati
$ go test --ckati --ninja
$ go test --ckati --ninja --all

The above commands run all cKati and Ninja tests in the testcases/ directory.

Alternatively, you can also run the tests in a Docker container in a prepared test enviroment:

$ docker build -t kati-test . && docker run kati-test

If you are working on a machine that does not provide make in the same version as kati is currently compatible with, you might want to download a prebuilt version instead. For example to use the prebuilt version of Ubuntu 20.04 LTS:

  $ mkdir tmp/ && cd tmp/
  $ wget http://mirrors.kernel.org/ubuntu/pool/main/m/make-dfsg/make_4.2.1-1.2_amd64.deb
  $ ar xv make_4.2.1-1.2_amd64.deb
  $ tar xf data.tar.xz
  $ cd ..
  $ PATH=$(pwd)/tmp/usr/bin/:$PATH make test

How to use for Android

For Android-N+, ckati and ninja is used automatically. There is a prebuilt checked in under prebuilts/build-tools that is used.

All Android's build commands (m, mmm, mmma, etc.) should just work.

kati's Issues

How to exclude directory from kati scan

I have a temp directory under AOSP root dir, its subdir created and removed freqquently,
How can I exclude it from kati scan ?

build
out
vendor/mydir

The problem I encounter:
Running kati to generate build-xxx.ninja...
out/build-xxx.ninja is missing, regenerating...
opendir failed: vendor/delivery/obj/mydir: No such file or directory
make[2]: *** [out/build-xxx.ninja] Error 1
make error!!

Ninja mkdir elision doesn't handle non-canonical paths

I'll take a look at this next week, but logging this before I forget:

test: a/b/../c

a/b/../c:
	@mkdir -p $(dir $@)
	touch a/b/../c

Ninja canonicalizes the path before making the directories, so only a is created. In make, mkdir -p a/b/../c would create a and a/b.

"make -j10 test" incorrectly fails all tests

Test case:

$ make clean
...
$ git clean -xfd
...
# See issue #59 why we need to build "kati" first
$ make -j10 kati
...
$ GOPATH=/usr/share/gocode make -j10 test
g++ -c -std=c++11 -g -W -Wall -MMD -MP -O -DNOLOG -march=native -o affinity.o affinity.cc
g++ -c -std=c++11 -g -W -Wall -MMD -MP -O -DNOLOG -march=native -o command.o command.cc
...
multi_explicit_output_patterns.mk: FAIL
--- out.make    2016-04-27 10:50:45.200593757 +0300
+++ out.kati    2016-04-27 10:50:45.200593757 +0300
@@ -1,5 +1,4 @@
 === test ===
-warning: jobserver unavailable: using -j1.  Add "+" to parent make rule.
 echo azz
 azz
 echo zza
multi_explicit_output_patterns_double_colon.mk: FAIL (expected)
multi_implicit_output_patterns.mk has a string "FAIL" in its expectation
Makefile:21: recipe for target 'test' failed
make: *** [test] Error 1

[PATCH] --ignore_optional_include ignores non-matching include files

Test case:

$ cat Makefile.minus-include-main
.PHONY: default
default:

-include Makefile.minus-include-?
$ cat Makefile.minus-include-a
$(info include a...)
$ cat Makefile.minus-include-b
$(info include b...)

$ ckati --warn --gen_all_targets --regen --ninja -f Makefile.minus-include-main
include b...
include a...
$ ckati --warn --gen_all_targets --regen --ninja -f Makefile.minus-include-main --ignore_optional_include=Makefile.minus-include-a
arguments changed, regenerating...
include b...
$ ckati --warn --gen_all_targets --regen --ninja -f Makefile.minus-include-main --ignore_optional_include=Makefile.minus-include-b
arguments changed, regenerating...

This is caused by an erroneous return statement in eval.cc:EvalInclude().

[PATCH] Discrepancy between $(shell) parsing and $(shell) check

There is a discrepancy in $(shell) execution which can cause kati to unnecessarily trigger a full re-parse of the makefile code:

• when makefile code is parsed the $(shell) command lines are passed to RunCommand($(SHELL))
• during startup stored $(shell) command lines are passed to RunCommand("/bin/sh")

This hard-codes the assumption that $(SHELL) and /bin/sh are the same shell. Counter-example:

• on Ubuntu 12.04 /bin/sh means "dash"
• Android build/core/main.mk sets $(SHELL) to /bin/bash

Test case for systems where /bin/sh means "bash" (requires part of Android source code for the command line to do something):

$ cat Makefile.shell
SHELL  := /bin/dash
_files := $(shell cd libcore && ls -d */src/test/{java,resource} 2> /dev/null)

.PHONY: default
default:

$ rm build.ninja
$ echo $(cd libcore && ls -d */src/test/{java,resource} 2> /dev/null) | sha1sum -
c3a0d5b86b91c763c2f3689b8454dda45007d3d7  -
$ build/kati/ckati --warn --gen_all_targets --regen --ninja -f Makefile.shell
./build.ninja is missing, regenerating...

$ echo $(cd libcore && ls -d */src/test/{java,resource} 2> /dev/null) | sha1sum -
c3a0d5b86b91c763c2f3689b8454dda45007d3d7  -
$ build/kati/ckati --warn --gen_all_targets --regen --ninja -f Makefile.shell
$(shell cd libcore && ls -d */src/test/{java,resource} 2> /dev/null) was changed, regenerating...

# comment out SHELL definition in Makefile.shell
$ rm build.ninja
$ echo $(cd libcore && ls -d */src/test/{java,resource} 2> /dev/null) | sha1sum -
c3a0d5b86b91c763c2f3689b8454dda45007d3d7  -
$ build/kati/ckati --warn --gen_all_targets --regen --ninja -f Makefile.shell
./build.ninja is missing, regenerating...

$ echo $(cd libcore && ls -d */src/test/{java,resource} 2> /dev/null) | sha1sum -
c3a0d5b86b91c763c2f3689b8454dda45007d3d7  -
$ build/kati/ckati --warn --gen_all_targets --regen --ninja -f Makefile.shell
No need to regenerate ninja file

i.e. although there are no changes to the source tree or build configuration kati triggers a reparse

I think the only correct solution is to record the value of $(SHELL) along with command line into the kati stamp file, so that the correct shell command is available during checking phase.

wildcard expansion in targets and in prerequisites.

as used in kati's Makefile

go_src_stamp: $(GO_SRCS) cmd//.go

cmd//.go should be expanded as wildcard.
for now, kati doesn't expand the wildcard, so fails as

*** No rule to make target "cmd//.go", needed by "go_src_stamp".

for ninja generator, we might not need to generate `build cmd//.go: phony" (and remove cmd//.go from build go_src_stamp) ?

Missing endif silently ignored

When a line continuation would eat the endif of a conditional:

main:
        echo "FAIL"

ifeq (0,1)
A := 1 \
endif

$(error $(A))

Make returns an error about a missing endif:

Makefile:9: *** missing `endif'.  Stop.

But kati just silently ignores the rest of the file:

echo "FAIL"
FAIL

ckati differs from make when appending a variable to itself

ckati fails this testcase:
x := one
x += two $(x)
$(info $(x))

$ ./runtest.rb -c testcase/append_self_reference.mk
append_self_reference.mk: FAIL
--- out.make 2016-03-08 17:46:40.017690761 -0800
+++ out.kati 2016-03-08 17:46:40.017690761 -0800
@@ -1,5 +1,5 @@
=== ===
-one two one
+one two one two
*** No targets.

=== FILES ===

=== Failures ===
append_self_reference.mk

FAIL! (1 fails 0 passes)

Anyway to completely rebuild a new system image?

Hi,

I ecountered an issue when I want to rebuild a system image:

I built a system.img use command:
make -j4

then there has system.img in out/target/product/....
I found if I deleted the system.img then used:
make systemimage
It only copy a system.img from:
target/product/obj/PACKAGING/systemimage_intermediates/
Once I deleted the target/product/obj/PACKAGING/systemimage_intermediates/system.img
then run
make systemimage,
a error occurred:
cannot find target/product/obj/PACKAGING/systemimage_intermediates/system.img
make failed

The question is, it can't find the system.img, why the Android can't rebuilt a new one? the system.img is come from the system dictionary.

Even I have modify the Makefile in buile/core, it also don't rebuilt ,just copy

"VAR:=<value>" is not supported on the command line

GNU make allows "VAR:=" on the command line. Test case:

$ cat Makefile
$(info VAR: '$(VAR)')

$ make VAR:=old -f Makefile
VAR: 'old'
make: *** No targets.  Stop.

$ ckati -c --warn VAR:=old -f Makefile
VAR: ''

add basic follows_symlink support

It'd be nice to be able to use symlinks for (sub)directories.

Hard code Android OUT_DIR as out in find emulator

If Android OUT_DIR is not called out, find emulator will traverse it and waste some time.
Maybe need to check OUT_DIR environment or bypass the folder marked with --ignore_dirty.

segmentation fault when compiling on MAC OS 10.12

We got segmentation fault when compiling Android on MAC OS 10.12.

Here is the crash-report:
crash-report.txt

How to call a shell command and use its return value in Makefile when using Kati?

I have a Makefile in which some statements relies on the $(shell ...) function. For example:

$(eval SYSTEM_FS_SIZE := $(shell du -sm $(TARGET_OUT)))

But the "commit 28da237" (description: [C++] Explicitly disallow $(shell) in other make constructs ) explicitly disallows such usage. And the following error occurs:

kati doesn't support passing results of $(shell) to other make constructs: du -sm out/target/product/aa997/system, Expr(du -sm , SymRef(TARGET_OUT))
build/core/ninja.mk:159: recipe for target 'out/build-aa997_native.ninja' failed

So my question is: How to call a shell command and use its return value in Makefile when using Kati?

[PATCH] [C++] Ignore white space around $(call) function name

Regression when compared to GNU make behaviour.

Test case:

$ cat Makefile.call-func-name
func = $(info called with '$(1)')
test = $(call $(1),$(1))

$(call test,func)
$(call test, func)
$(call test,func )
$(call test, func )

$ make -f Makefile.call-func-name
called with 'func'
called with ' func'
called with 'func '
called with ' func '
make: *** No targets.  Stop.

$ ckati -c --warn -f Makefile.call-func-name
called with 'func'
Makefile.call-func-name:5: *warning*: undefined user function:  func
Makefile.call-func-name:6: *warning*: undefined user function: func
Makefile.call-func-name:5: *warning*: undefined user function:  func
*** No targets.

0002-C-Ignore-white-space-around-call-function-name.txt

Multiple ninja targets generated for same file

The test case is stripped-down from real-life case of including .P files in Android build processed by kati. Please keep this in mind when reading the example :-)

Test case:

$ mkdir -p external
$ touch dummy.h
$ cat Makefile.canonical-path
.PHONY: default

default: dummy1.o dummy2.o

dummy1.o: dummy.h
          touch $@

dummy.h:

dummy2.o: external/../dummy.h
          touch $@

external/../dummy.h:

.PHONY: clean
clean:
        rm -f dummy1.o dummy2.o

GNU make:

$ rm -f dummy?.o
$ make -f Makefile.canonical-path
touch dummy1.o
touch dummy2.o

kati + ninja:

$ rm -f build.ninja dummy?.o
$ ckati --warn --gen_all_targets --regen --ninja -f Makefile.canonical-path
./build.ninja is missing, regenerating...
$ ninja
ninja: warning: multiple rules generate dummy.h. builds involving this target will not be correct; continuing anyway [-w dupbuild=warn]
...
$ fgrep dummy.h build.ninja
build dummy1.o: rule2 dummy.h
build dummy.h: phony
build dummy2.o: rule3 external/../dummy.h
build external/../dummy.h: phony

GNU make and ninja consider both paths to be pointing to same file. kati should probably canonicalize the paths to detect targets pointing to the same file.

Issue when dealing with multiline variable.

For the following line, ninja report error 'inline_value1' doesn't match the target pattern.

VAR := initvalue

VAR += \
    some_value1 \
   some_value2 \

VAR += inline_value1 inline_value2

Mixing multiline with single line will cause this issue. And if I put single line ahead, the problem gone.

Bad error message for something that's questionably valid

This isn't what people want to do anyway, but the error message looks like a kati problem, when it's really a Makefile problem (and maybe a kati problem if what is breaking is technically valid):

Makefile:

define var
VAR:=1
endef

$(call var)

$(info SUCCESS)

make output:

SUCCESS
make: *** No rule to make target `=1', needed by `VAR'.  Stop.

ckati output:

parser.cc:584: sep != 0

ifdef is still buggy

ifdef is still buggy when there is an embedded space in the evaluated variable name. This snippet works in make, but not with kati:

A := a # comment

ifdef $(A)
$(error FAIL)
else
$(info PASS)
endif

a := b
ifdef $(A)
$(info PASS)
else
$(error FAIL)
endif

I don't think we're hitting this anywhere, but I noticed it when looking at the latest change.

clang-format?

It would be nice to have a code formatter available when editing the code, but we don't have a defined style guide right now. From some quick checks with clang-format, it looks like we're closest to the Chromium style (not surprising), but still have a decent number of changes:

Chromium diffstat: 48/68 files changed,  853 insertions(+),  903 deletions(-)
  Google diffstat: 52/68 files changed, 1104 insertions(+), 1632 deletions(-)
    LLVM diffstat: 64/68 files changed, 1965 insertions(+), 2224 deletions(-)
 Mozilla diffstat: 66/68 files changed, 2846 insertions(+), 1747 deletions(-)
  Webkit diffstat: 65/68 files changed, 8747 insertions(+), 8072 deletions(-)

Any opinions @shinh / @colincross? Should we adopt the Chromium style? Or a modified version? Should we reformat all the files? Not do anything?

From a quick look through the diffs, I didn't find anything that we were consistently doing one way that's now different.

ninja mode ckati behaves strange when environment variable modified

I am not sure if it is a bug, but ckati's behavior is quite strange for me to understand.
My code sample test.mk is abstracted and modified from Android build system:

ifeq "" "$(BUILD_NUMBER)"
BUILD_NUMBER := eng.$(USER).$(shell date +%Y%m%d.%H%M%S)
endif
BUILD_NUMBER := $(shell date +%s)
$(info origin=$(origin BUILD_NUMBER))

all:
ifndef BUILDING_WITH_NINJA
all: ninja_wrapper
ninja_wrapper:
./ckati --ninja --regen --detect_android_echo --use_find_emulator -f test.mk --gen_all_targets BUILDING_WITH_NINJA=true

endif

Then if environment variable BUILD_NUMBER is set, each time running "make -f test.mk" will cause build.ninja regenerated:
origin=file
./ckati --ninja --regen --detect_android_echo --use_find_emulator -f test.mk --gen_all_targets BUILDING_WITH_NINJA=true
Environment variable BUILD_NUMBER was modified (?????????? => ??????????), regenerating...
origin=file

It seems only recursive make with ckati will have such problem. if I change from make to ckati, like "ckati -f test.mk", it will never cause build.ninja always regenerated.
Similarly, unset BUILD_NUMBER from environment variable, or remove the line of "BUILD_NUMBER := $(shell date +%s)", will also make the problem disappeared.

[PATCH] "override VAR := <value>" only works once

Regression compared to GNU make behaviour. Test case

$ cat Makefile.override-failure
$(info VAR: '$(VAR)')
override VAR := test
$(info VAR: '$(VAR)')
override VAR := test-new
$(info VAR: '$(VAR)')
VAR := test-no-override
$(info VAR: '$(VAR)')
VAR := test-no-override-new
$(info VAR: '$(VAR)')

$ make VAR=old -f Makefile.override-failure
VAR: 'old'
VAR: 'test'
VAR: 'test-new'
VAR: 'test-new'
VAR: 'test-new'
make: *** No targets.  Stop.

$ ckati -c --warn VAR=old -f Makefile.override-failure
VAR: 'old'
VAR: 'test'
VAR: 'test'
VAR: 'test'
VAR: 'test'
*** No targets.

multi_implicit_output_patterns test fails for GNU make 4.0

When executed with GNU make 4.0 it prints "FAIL" in the expected output which causes runtest.rb to abort the whole test suite.

I'm not sure what the correct approach is to fix version dependencies in tests. It seems that GNU make 4.0 always executes the recipe for the dependency of b.c.z.

No test target for ckati

AFAIU ruby runtest.rb -c tests ckati instead of kati, but there is no makefile target for it.

CRLF line terminators are not supported

For the case in kati/testcase/crlf.mk, if "\r\n" is assigned in more than 1 lines, for example:
PASS :=
PASS
PASS

ckati still fails:
echo PASS
PASS
PASS
/bin/sh: PASS: command not found
*** [test] Error 127

while make 3.8.1 has no error:
echo PASS PASS
PASS PASS

Comment in macro trips up $(eval)

Regression when compared to GNU make

$ cat Makefile.comment-in-macro
 .PHONY: all
all: test
        @echo "TARGET: $@"

define _define_rule
# this is a test
.PHONY: test
test:
        @echo "TARGET: $$@"

endef

$(eval $(call _define_rule))

$ make -f Makefile.comment-in-macro
TARGET: test
TARGET: all

$ ckati --ninja --ninja_dir . --gen_all_targets --warn -f Makefile.comment-in-macro
$ ninja all
ninja: error: 'test', needed by 'all', missing and no known rule to make it

After removing the comment in _define_rule:

$ make -f Makefile.comment-in-macro
TARGET: test
TARGET: all

$ ckati --ninja --ninja_dir . --gen_all_targets --warn -f Makefile.comment-in-macro
$ ninja all
[1/2] build test
TARGET: test
[2/2] build all
TARGET: all

GNU make 4.0 causes tests to fail

Test case:

$ make --version
GNU Make 4.0
...
$ make test
...
err_semicolon.mk: FAIL
--- out.make    2016-04-27 11:22:41.786489640 +0300
+++ out.kati    2016-04-27 11:22:41.786489640 +0300
@@ -1,5 +1,5 @@
 ===  ===
-Makefile:2: *** missing rule before recipe.
+Makefile:2: *** missing rule before commands.

 === FILES === 
...

Another text that has changed compared to older GNU make.

Error for comparing wildcard results

Test case:
$ cat Android.mk
...
include $(LOCAL_PATH)/*/Android.mk

and suppose we have several sub-dirs which contain Android.mk file.
(e.g. a/Android.mk, b/Android.mk, c/Android.mk ... z/Android.mk)

Kati have been comparing a previous result of wildcard after sorted to a new result of wildcard without sorted. (https://github.com/google/kati/blob/master/regen.cc#L266)
The new result of wildcard should be also sorted before comparing with the previous one.

For now, kati dected "wildcard" results in the above cases as changed so always regenerate .ninja.

-cp src dst, generate command "cp src dst ;", is this right?

If write a - at beginning of a command line
ckati will generate a command and write a ; at the end of a command line
should ckati write a || (true) at the end of a command line?

take build/core/Makefile for example, if command (rsync -a --exclude=etc --exclude=sdcard out/target/product/xxxxx/root out/target/product/xxxxx/recovery) fail, then build system will continue (rm -f out/target/product/xxxxx/recovery/root/init*.rc) and result in a wrong recovery image

(mkdir -p out/target/product/xxxxx/recovery) &&
(mkdir -p out/target/product/xxxxx/recovery/root/etc out/target/product/xxxxx/recovery/root/sdcard out/target/product/xxxxx/recovery/root/tmp) && 
(echo Copying baseline ramdisk...) && 
(rsync -a --exclude=etc --exclude=sdcard out/target/product/xxxxx/root out/target/product/xxxxx/recovery) &&
(echo Modifying ramdisk contents...) ;
(rm -f out/target/product/xxxxx/recovery/root/init*.rc)

CI test run does not reflect real production systems

No wonder that I can't even get passing tests on Ubuntu 14.04. The CI test has a hard-coded

ln -sf /bin/bash /bin/sh

which normal users aren't allowed on production systems. It's a bad idea anyway, because it would corrupt the Ubuntu installation.

IMHO the test cases and/or test runner needs to be updated to use /bin/bash everywhere.

Allow to echo @commands

I was wondering why @echo isn't treated like normal echo
With this small patch I was able to get it working properly
Are there any downsides allowing @echo ?
Thanks

support for `else conditional-directive`

https://www.gnu.org/software/make/manual/html_node/Conditional-Syntax.html

conditional-directive-one
text-if-one-is-true
else conditional-directive-two
text-if-two-is-true
else
text-if-one-and-two-are-false
endif

But if I use that syntax with kati:

build/core/target_test_internal.mk:12: warning for parse error in an unevaluated line: *** invalid syntax in conditional.                                                                                 [62/9586]
build/core/target_test_internal.mk:14: warning for parse error in an unevaluated line: *** only one `else' per conditional.
build/core/target_test_internal.mk:16: warning for parse error in an unevaluated line: *** only one `else' per conditional.

looser throw specifier

Checked out the code and tried compiling on Ubuntu 12.10 Quantal 64bit and got the following error

rm -rf out/{src,pkg/}/github.com/google/kati
mkdir -p out/{src,pkg/}/github.com/google/kati
cp -a ast.go bootstrap.go buf.go buf_test.go dep.go depgraph.go doc.go evalcmd.go eval.go exec.go expr.go expr_test.go fileutil.go flags.go func.go func_test.go log.go ninja.go ninja_test.go parser.go pathutil.go pathutil_test.go query.go rule_parser.go rule_parser_test.go serialize.go shellutil.go shellutil_test.go stats.go strutil.go strutil_test.go symtab.go var.go version.go worker.go cmd out/src/github.com/google/kati
GOPATH=$(pwd)/out:${GOPATH} go get github.com/google/kati/cmd/kati
touch go_src_stamp
rm -f out/bin/kati
GOPATH=$(pwd)/out:${GOPATH} go install -ldflags "-X github.com/google/kati.gitVersion c9b0aca" github.com/google/kati/cmd/kati
cp out/bin/kati kati
g++ -c -std=c++11 -g -W -Wall -MMD -MP -O -DNOLOG -o ast.o ast.cc
g++ -c -std=c++11 -g -W -Wall -MMD -MP -O -DNOLOG -o command.o command.cc
g++ -c -std=c++11 -g -W -Wall -MMD -MP -O -DNOLOG -o dep.o dep.cc
g++ -c -std=c++11 -g -W -Wall -MMD -MP -O -DNOLOG -o eval.o eval.cc
g++ -c -std=c++11 -g -W -Wall -MMD -MP -O -DNOLOG -o exec.o exec.cc
g++ -c -std=c++11 -g -W -Wall -MMD -MP -O -DNOLOG -o file.o file.cc
g++ -c -std=c++11 -g -W -Wall -MMD -MP -O -DNOLOG -o file_cache.o file_cache.cc
g++ -c -std=c++11 -g -W -Wall -MMD -MP -O -DNOLOG -o fileutil.o fileutil.cc
g++ -c -std=c++11 -g -W -Wall -MMD -MP -O -DNOLOG -o find.o find.cc
find.cc:162:3: error: looser throw specifier for ‘virtual {anonymous}::DirentDirNode::~DirentDirNode()’
find.cc:113:11: error: overriding ‘virtual {anonymous}::DirentNode::~DirentNode() noexcept (true)’
find.cc: In member function ‘void {anonymous}::DirentDirNode::Add(const string&, {anonymous}::DirentNode_)’:
find.cc:208:24: error: ‘class std::mapstd::basic_string<char, {anonymous}::DirentNode_>’ has no member named ‘emplace’
find.cc:208:39: error: unable to deduce ‘auto’ from ‘’
find.cc: At global scope:
find.cc:564:11: error: looser throw specifier for ‘virtual {anonymous}::FindEmulatorImpl::~FindEmulatorImpl()’
In file included from find.cc:17:0:
find.h:45:11: error: overriding ‘virtual FindEmulator::~FindEmulator() noexcept (true)’
make: *** [find.o] Error 1

ckati differs from make in target variable assign

$ cat Makefile
all: A ?= 0
all: A += 1
all:
@echo A=$(A)

$ A=2 make
A=2 1

$ A=2 ckati
A=2

Doesn't seem to be working for me. But I don't totally know what I'm doing...

I followed the instructions by copying and pasting. I think I got the $PATH correct too.

if [ -d "$HOME/src/kati" ] ; then
PATH="$HOME/src/kati:$PATH"
fi

When I run the script ./ninja.sh I get
./ninja.sh: 5: exec: ninja: not found

Can you tell me what I did wrong?

NetBSD fixes

Please take a look at https://github.com/0-wiz-0/kati
Most of the patches should be ok for merging; the last three definitely are not and are just there to show issues:

• /bin/bash is hardcoded, but might not exist
• make is expected to be GNU make, but on BSD's that is BSD make; GNU make there is called gmake
• NetBSD's gcc does not support -march=native (not sure why)

As of today, the fork is down to 24 test failures. Here's the test output:

ruby runtest.rb -c -n
abspath.mk: PASS
addprefix.mk: PASS
addsuffix.mk: PASS
and.mk: PASS
append_self_reference.mk: PASS
assign_after_tab.mk: PASS
assign_in_parens.mk: PASS
assign_types.mk: PASS
assign_with_trailing_space.mk: PASS
auto_var_suffixes.mk: FAIL
--- out.make	2017-02-06 11:22:51.486406875 +0100
+++ out.kati	2017-02-06 11:22:51.486447732 +0100
@@ -24,8 +24,8 @@
 afile bfile
 
 === FILES ===
+tdir
 adir
 bdir
 afile
 bfile
-tdir
auto_vars.mk: PASS
autovar_assign.mk: FAIL (expected)
backslash_before_empty_line.mk: PASS
backslash_before_special_character.mk: PASS
backslash_in_rule_command.mk: PASS
basename.mk: PASS
basic_dep.mk: PASS
basic_rule.mk: PASS
basic_var.mk: PASS
blank_line_before_command.mk: FAIL (expected)
build_once.mk: PASS
builtin_rules.mk: FAIL
--- out.make	2017-02-06 11:22:52.008918486 +0100
+++ out.kati	2017-02-06 11:22:52.008966676 +0100
@@ -6,7 +6,7 @@
 === test2 ===
 
 === FILES ===
+foo.o
 foo.c
 bar.cc
-foo.o
 bar.o
builtin_vars.mk: PASS
call.mk: PASS
call_with_many_args.mk: PASS
call_with_whitespace.mk: PASS
canned_recipes.mk: PASS
circular_dep.mk: FAIL (expected)
close_paren_without_open.mk: PASS
colon_ws_in_file.mk: FAIL (expected)
colon_ws_in_target.mk: FAIL (expected)
command_vars.mk: PASS
comment.mk: PASS
comment_in_command.mk: PASS
comment_in_define.mk: PASS
cond_syntax.mk: PASS
crlf.mk: PASS
crlf_after_directive.mk: PASS
curdir.mk: PASS
curdir_implicit_rule.mk: PASS
curdir_var.mk: PASS
default_rule.mk: PASS
define.mk: PASS
define_newline.mk: PASS
define_verbatim.mk: PASS
define_with_comments.mk: PASS
delete_on_error.mk: FAIL (expected)
dir.mk: PASS
directive_after_tab.mk: PASS
dollar_in_file.mk: PASS
dot_rule.mk: PASS
double_colon_rule.mk: PASS
else_if.mk: PASS
empty_line_in_define.mk: PASS
empty_output.mk: PASS
empty_target_specific_var.mk: FAIL
--- out.make	2017-02-06 11:22:53.721179237 +0100
+++ out.kati	2017-02-06 11:22:53.721225611 +0100
@@ -1,5 +1,5 @@
 === test ===
-Makefile:3: *** empty variable name.
+PASS
 
 === FILES ===
 
empty_target_specific_var2.mk: FAIL
--- out.make	2017-02-06 11:22:53.735892871 +0100
+++ out.kati	2017-02-06 11:22:53.735932680 +0100
@@ -1,5 +1,5 @@
 ===  ===
-Makefile:7: *** empty variable name.
+PASS
 
 === FILES ===
 
empty_var_cmd.mk: PASS
envvar.mk: PASS
equal_and_semi_in_rule.mk: PASS
equal_in_target.mk: FAIL (expected)
err_both_colon.mk: PASS
err_empty_output.mk: PASS
err_empty_var_name.mk: PASS
err_error.mk: PASS
err_error_in_recipe.mk: FAIL
--- out.make	2017-02-06 11:22:53.835878730 +0100
+++ out.kati	2017-02-06 11:22:53.835918958 +0100
@@ -1,4 +1,5 @@
 === test ===
+/usr/pkg/bin/bash -c "echo -e \"Makefile:2: *** foo.\" 2>&1 && false"
 Makefile:2: *** foo.
 
 === FILES ===
err_export_override.mk: PASS (unexpected)
err_extra_else.mk: PASS
err_extra_endif.mk: PASS
err_ifdef_two_args.mk: PASS
err_ifdef_two_args2.mk: PASS
err_ifdef_with_leading_space.mk: PASS
err_include.mk: PASS
err_invalid_else.mk: PASS
err_invalid_ifeq.mk: PASS
err_invalid_ifeq2.mk: PASS
err_invalid_ifeq3.mk: PASS
err_invalid_ifeq4.mk: PASS
err_invalid_ifeq5.mk: FAIL (expected)
err_keyword_in_rule.mk: PASS
err_missing_endef.mk: PASS
err_missing_endif.mk: PASS
err_missing_sep.mk: PASS
err_no_rule.mk: PASS
err_no_target_commands.mk: PASS
err_no_target_commands2.mk: PASS
err_override.mk: PASS
err_override_export.mk: PASS (unexpected)
err_pattern_rule_only.mk: PASS
err_semicolon.mk: PASS
err_semicolon2.mk: PASS
err_semicolon3.mk: PASS
err_semicolon4.mk: PASS
err_semicolon_in_output.mk: PASS
err_suffixes.mk: FAIL (expected)
err_suffixes2.mk: PASS
err_two_else.mk: PASS
err_unmatched_endef.mk: PASS
err_unterminated_func.mk: PASS
err_unterminated_var.mk: PASS
err_word_non_numeric.mk: PASS
err_word_zero.mk: PASS
escape_for_shell_in_recipe.mk: FAIL (expected)
escaped_backslash.mk: PASS
escaped_comment.mk: PASS
escaped_wildcard.mk: FAIL
--- out.make	2017-02-06 11:22:54.497896571 +0100
+++ out.kati	2017-02-06 11:22:54.497946717 +0100
@@ -3,7 +3,7 @@
 === FILES ===
 foo
 === test2 ===
-
+foo
 
 === FILES ===
 foo
eval.mk: FAIL
--- out.make	2017-02-06 11:22:54.768044711 +0100
+++ out.kati	2017-02-06 11:22:54.768086406 +0100
@@ -12,13 +12,13 @@
 client.o client_api.o client_mem.o -o client
 
 === FILES ===
+server.o
 server.c
 server_priv.c
 server_access.c
 client.c
 client_api.c
 client_mem.c
-server.o
 server_priv.o
 server_access.o
 client.o
eval_assign.mk: PASS
eval_starts_with_comment.mk: PASS
excl_in_shell.mk: PASS
expanded_nl.mk: PASS
explicit_pattern_rule.mk: PASS
export.mk: PASS
export_export.mk: FAIL (expected)
fail_ignore_error.mk: FAIL
--- out.make	2017-02-06 11:22:54.931814117 +0100
+++ out.kati	2017-02-06 11:22:54.931863983 +0100
@@ -1,5 +1,6 @@
 === test ===
 *** [test] Error 1
+/usr/pkg/bin/bash -c "(false ) && (false ; true ) && (echo FAIL )"
 
 === FILES ===
 
fail_subshell_in_recipe.mk: FAIL
--- out.make	2017-02-06 11:22:54.947846205 +0100
+++ out.kati	2017-02-06 11:22:54.947885944 +0100
@@ -1,5 +1,6 @@
 === test ===
 *** [test] Error 1
+/usr/pkg/bin/bash -c "(false ) && ((true) ; echo FAIL )"
 
 === FILES ===
 
filter-out.mk: PASS
filter.mk: PASS
find_command.mk: FAIL (expected)
find_command_sorted.mk: FAIL (expected)
findstring.mk: PASS
first_rule.mk: PASS
firstword.mk: PASS
flavor.mk: PASS
foreach.mk: PASS
func_backslash.mk: PASS
func_nop.mk: PASS
func_with_backslash.mk: PASS
hash_in_var.mk: PASS
if.mk: PASS
if_recipe.mk: PASS
ifdef_rec_var.mk: PASS
ifdef_ret_in_arg.mk: PASS
ifdef_with_comments.mk: PASS
ifdef_with_trailing_space.mk: PASS
ifeq_without_parens.mk: PASS
ignore_error.mk: PASS
implicit_pattern_rule.mk: PASS
implicit_pattern_rule_chain.mk: PASS
implicit_pattern_rule_chain2.mk: FAIL (expected)
implicit_pattern_rule_for_no_commands.mk: PASS
implicit_pattern_rule_phony.mk: PASS
implicit_pattern_rule_prefix.mk: PASS
include.mk: PASS
include_glob.mk: PASS
include_glob_order.mk: PASS
include_var.mk: PASS
info.mk: PASS
join.mk: PASS
last_resort.mk: PASS
lastword.mk: PASS
lineno_in_call.mk: PASS
makecmdgoals.mk: PASS
makefile_list.mk: FAIL
--- out.make	2017-02-06 11:22:57.854456066 +0100
+++ out.kati	2017-02-06 11:22:57.854501602 +0100
@@ -7,17 +7,17 @@
 Makefile foo.mk foo.mk foo.mk
 
 === FILES ===
-foo.mk
 bar.mk
+foo.mk
 === test3 ===
 Makefile foo.mk bar.mk bar.mk foo.mk foo.mk
 
 === FILES ===
-foo.mk
 bar.mk
+foo.mk
 === test4 ===
 PASS
 
 === FILES ===
-foo.mk
 bar.mk
+foo.mk
merge_inputs.mk: PASS
merge_output_pattern.mk: PASS
merge_target_specific_vars.mk: PASS
multi_explicit_output_patterns.mk: PASS
multi_explicit_output_patterns_double_colon.mk: FAIL (expected)
multi_implicit_output_patterns.mk: PASS
multi_outputs.mk: PASS
multi_pattern_rule.mk: PASS
multi_rule.mk: PASS
multi_rule_order_only.mk: PASS
multi_suffix_rule.mk: PASS
multiline_and_leading_space.mk: PASS
multiline_and_lineno.mk: PASS
multiline_arg.mk: FAIL
--- out.make	2017-02-06 11:22:58.346097909 +0100
+++ out.kati	2017-02-06 11:22:58.346139953 +0100
@@ -5,7 +5,7 @@
 INFO:  foo
 INFO:  \
 foo
-\
+
 
 === FILES ===
 
multiline_define.mk: PASS
multiline_recipe.mk: FAIL (expected)
multiple_output_patterns.mk: PASS
nested_call.mk: PASS
nested_define.mk: PASS
ninja_normalized_path.mk: FAIL (expected)
no_last_newline.mk: PASS
not_command_with_tab.mk: PASS
notdir.mk: PASS
nothing_to_do.mk: PASS
or.mk: PASS
order_only.mk: FAIL
--- out.make	2017-02-06 11:22:59.681289871 +0100
+++ out.kati	2017-02-06 11:22:59.681343230 +0100
@@ -7,6 +7,6 @@
 PASS_foo
 
 === FILES ===
+baz
 foo
 bar
-baz
order_only2.mk: FAIL (expected)
origin.mk: PASS
override.mk: PASS
override_define.mk: PASS
override_export.mk: FAIL (expected)
override_override.mk: PASS
override_rule.mk: PASS
param.mk: FAIL (expected)
patsubst.mk: PASS
pattern_rules_priority.mk: PASS
phony.mk: PASS
posix_var.mk: PASS
preserve_single_dot.mk: PASS
quine.mk: PASS
realpath.mk: PASS
recipe_in_rule.mk: PASS
recipe_var.mk: PASS
recursive_command_expansion.mk: PASS
recursive_marker.mk: PASS
rule_in_var.mk: PASS
rule_with_extra_ws.mk: PASS
semi_in_var.mk: PASS
semicolon.mk: PASS
semicolon_in_var.mk: PASS
shell.mk: FAIL
--- out.make	2017-02-06 11:23:00.541626427 +0100
+++ out.kati	2017-02-06 11:23:00.541680344 +0100
@@ -1,13 +1,19 @@
 === test ===
 /disk/6/archive/foreign/kati/out/shell.mk
 
-na n b n
-na n b n X
-X-e nnY
-X-e annY
-X-e nnbY
-X-e nnbY
-X-e nnnbY
+
+a 
+ b 
+
+
+a 
+ b 
+ X
+X-e \n\nY
+X-e a\n\nY
+X-e \n\nbY
+X-e \n\nbY
+X-e \n\n\nbY
 X-e bY
 X-e b Y
 
shell_arith_in_recipe.mk: PASS
shell_stderr.mk: PASS
shell_var.mk: PASS
shell_var_with_args.mk: FAIL
--- out.make	2017-02-06 11:23:00.749555728 +0100
+++ out.kati	2017-02-06 11:23:00.749596934 +0100
@@ -3,10 +3,9 @@
             [-C directory] [-D variable] [-d flags] [-f makefile]
             [-I directory] [-J private] [-j max_jobs] [-m directory] [-T file]
             [-V variable] [variable=value] [target ...]
-": not found
-
-": not found
-*** [test] Error 127
+foo
+cmd: echo baz
+baz
 
 === FILES ===
 
silent.mk: PASS
silent_ignore_error.mk: PASS
silent_multiline.mk: PASS
sort.mk: PASS
static_pattern.mk: PASS
stem.mk: PASS
strip.mk: PASS
strip_and_shell.mk: FAIL (expected)
submake_basic.mk: FAIL (expected)
subshell_in_recipe.mk: PASS
subst.mk: PASS
subst2.mk: PASS
suffix.mk: PASS
suffix_rule.mk: PASS
suffix_subst.mk: PASS
suffix_subst_pat.mk: PASS
tab_comment.mk: PASS
tab_only_line.mk: PASS
target_specific_var.mk: PASS
target_specific_var_append.mk: PASS
target_specific_var_in_var.mk: PASS
target_specific_var_ref.mk: PASS
target_specific_var_simple.mk: PASS
target_specific_var_timing.mk: PASS
target_specific_var_var_name.mk: PASS
target_specific_var_with_pattern.mk: PASS
target_specific_var_with_semi.mk: PASS
terms_in_parens.mk: PASS
trim_leading_curdir.mk: PASS
unmatched_paren.mk: PASS
unmatched_paren2.mk: PASS
value.mk: PASS
var_append.mk: PASS
var_cond_assign.mk: PASS
var_eval.mk: PASS
var_target.mk: PASS
var_with_space.mk: FAIL
--- out.make	2017-02-06 11:23:01.675907240 +0100
+++ out.kati	2017-02-06 11:23:01.675951519 +0100
@@ -3,7 +3,8 @@
             [-C directory] [-D variable] [-d flags] [-f makefile]
             [-I directory] [-J private] [-j max_jobs] [-m directory] [-T file]
             [-V variable] [variable=value] [target ...]
-Makefile:10: *** missing separator.
+PASS
+PASS2
 
 === FILES ===
 
vardef_in_call.mk: PASS
vpath.mk: FAIL (expected)
vpath_directive.mk: FAIL (expected)
warn_extra_trailings.mk: FAIL (expected)
warn_output_pattern_mismatch.mk: PASS
warning.mk: PASS
warning_in_eval.mk: PASS
whitespace_in_cmd.mk: PASS
wildcard.mk: PASS
wildcard_cache.mk: PASS (unexpected)
wildcard_multi.mk: PASS
wildcard_target.mk: FAIL (expected)
wildcard_with_commas.mk: PASS
wildcard_with_var.mk: PASS
word.mk: PASS
wordlist.mk: PASS
words.mk: PASS
cmdline_var.sh: PASS
cmdline_var_makeflags.sh: PASS
cmdline_var_modify.sh: PASS
cmdline_var_override.sh: PASS
file_func.sh: PASS
kati_cache.sh: PASS
ninja_mkdir.sh: PASS
./build.ninja is missing, regenerating...
./build.ninja is missing, regenerating...
./build.ninja is missing, regenerating...
ninja_phony_targets.sh: PASS
ninja_pool.sh: PASS
ninja_regen.sh: PASS
ninja_regen_filefunc_read.sh: FAIL
--- out.make	2017-02-06 11:23:10.003122773 +0100
+++ out.kati	2017-02-06 11:23:10.003176970 +0100
@@ -0,0 +1,5 @@
+foo
+foo
+foo
+foo
+foo
ninja_regen_filefunc_write.sh: PASS
ninja_regen_find_link.sh: FAIL
--- out.make	2017-02-06 11:23:10.109914960 +0100
+++ out.kati	2017-02-06 11:23:10.109969366 +0100
@@ -1,3 +1,3 @@
 linkdir/d/link linkdir/d/link/file1
-linkdir/d/link linkdir/d/link/file1 linkdir/d/link/file1_2
-linkdir/d/link linkdir/d/link/file2
+linkdir/d/link linkdir/d/link/file1
+linkdir/d/link linkdir/d/link/file1
ninja_regen_glob.sh: PASS
phony_targets.sh: PASS
readonly_global.sh: FAIL
--- out.make	2017-02-06 11:23:10.240105409 +0100
+++ out.kati	2017-02-06 11:23:10.240219459 +0100
@@ -1,24 +1,16 @@
 Testcase: = = =
-Nothing to be done for "all".
-Clean exit
+Makefile:3: *** cannot assign to readonly variable: FOO
 Testcase: = += =
-Nothing to be done for "all".
-Clean exit
+Makefile:3: *** cannot assign to readonly variable: FOO
 Testcase: = := =
-Nothing to be done for "all".
-Clean exit
+Makefile:3: *** cannot assign to readonly variable: FOO
 Testcase: = := :=
-Nothing to be done for "all".
-Clean exit
+Makefile:3: *** cannot assign to readonly variable: FOO
 Testcase: = := +=
-Nothing to be done for "all".
-Clean exit
+Makefile:3: *** cannot assign to readonly variable: FOO
 Testcase: := := :=
-Nothing to be done for "all".
-Clean exit
+Makefile:3: *** cannot assign to readonly variable: FOO
 Testcase: := := +=
-Nothing to be done for "all".
-Clean exit
+Makefile:3: *** cannot assign to readonly variable: FOO
 Testcase: := := =
-Nothing to be done for "all".
-Clean exit
+Makefile:3: *** cannot assign to readonly variable: FOO
readonly_global_missing.sh: FAIL
--- out.make	2017-02-06 11:23:10.251811768 +0100
+++ out.kati	2017-02-06 11:23:10.251862473 +0100
@@ -1,2 +1 @@
-Nothing to be done for "all".
-Clean exit
+Makefile:2: *** unknown variable: FOO
readonly_rule.sh: FAIL
--- out.make	2017-02-06 11:23:10.263493404 +0100
+++ out.kati	2017-02-06 11:23:10.263545017 +0100
@@ -1,3 +1,2 @@
 Testcase: := := +=
-Nothing to be done for "all".
-Clean exit
+Makefile:4: *** cannot assign to readonly variable: FOO
readonly_rule_missing.sh: FAIL
--- out.make	2017-02-06 11:23:10.275079987 +0100
+++ out.kati	2017-02-06 11:23:10.275123708 +0100
@@ -1,2 +1 @@
-Nothing to be done for "all".
-Clean exit
+Makefile:2: *** unknown variable: FOO

=== Expected failures ===
autovar_assign.mk
blank_line_before_command.mk
circular_dep.mk
colon_ws_in_file.mk
colon_ws_in_target.mk
delete_on_error.mk
equal_in_target.mk
err_invalid_ifeq5.mk
err_suffixes.mk
escape_for_shell_in_recipe.mk
export_export.mk
find_command.mk
find_command_sorted.mk
implicit_pattern_rule_chain2.mk
multi_explicit_output_patterns_double_colon.mk
multiline_recipe.mk
ninja_normalized_path.mk
order_only2.mk
override_export.mk
param.mk
strip_and_shell.mk
submake_basic.mk
vpath.mk
vpath_directive.mk
warn_extra_trailings.mk
wildcard_target.mk
=== Unexpected passes ===
err_export_override.mk
err_override_export.mk
wildcard_cache.mk
=== Failures ===
auto_var_suffixes.mk
builtin_rules.mk
empty_target_specific_var.mk
empty_target_specific_var2.mk
err_error_in_recipe.mk
escaped_wildcard.mk
eval.mk
fail_ignore_error.mk
fail_subshell_in_recipe.mk
makefile_list.mk
multiline_arg.mk
order_only.mk
shell.mk
shell_var_with_args.mk
var_with_space.mk
ninja_regen_filefunc_read.sh
ninja_regen_find_link.sh
readonly_global.sh
readonly_global_missing.sh
readonly_rule.sh
readonly_rule_missing.sh

FAIL! (24 fails 232 passes)
Makefile:21: recipe for target 'test' failed
gmake: *** [test] Error 1

Do you have suggestions on how to fix the issues and the failing tests?

Thank you!

Another missing endef silently ignored

define test1
# Typo below, endif instead of endef
endif
define test2
endef

foo:
    echo FAIL

$ make -f test.mk
test.mk:1: *** missing `endef', unterminated `define'.  Stop.

$ ckati -f test.mk
echo FAIL
FAIL

This is related to kati not supporting nested defines:

define outer
define inner
PASS
endef
endef

A := $(inner)
$(eval $(outer))

foo:
    echo $(A)
    echo $(inner)

$ make -f test.mk 
echo 

echo PASS
PASS

$ ckati -f test.mk
test.mk:5: *** missing separator.

Properly supporting nested defines doesn't seem important, but parsing them correctly does.

[PATCH] Ignore .POSIX:

GNU make 3.81 silently ignores .POSIX:, so kati should too.

Only effect would be adding "-e" parameter to SHELL command line.

0001-C-Ignore-.POSIX.txt

Missing repository description in GitHub

The title is self-descriptive. There is no description to this repository in GitHub.

Suggestion of description from the README file: kati is an experimental GNU make clone. The main goal of this tool is to speed-up incremental build of Android.

[PATCH] kati should ignore recursive marker in recipes

Test case:

$ cat Makefile.recursive 
.PHONY: default
default:
        +echo DONE

$ ckati --warn --gen_all_targets --regen --ninja -f Makefile.recursive
Makefile.recursive was modified, regenerating...
$ fgrep DONE build.ninja
 command = /bin/sh -c "+echo DONE"

$ ninja
[1/1] build default
FAILED: default
/bin/sh -c "+echo DONE"
/bin/sh: + : invalid option
Usage:  /bin/sh [GNU long option] [option] ...
...

It should be safe to simply ignore the marker during parsing as ninja doesn't set GNU make magic recursion variables anyway.

Multi-word MAKEFLAGS doesn't work properly

We ran into this issue when upgrading Kati on AOSP, but I've solved this on our side separately by clearing MAKEFLAGS, since we don't want the MAKEFLAGS override behavior and the variables are already available in the environment.

Simple shell testcase to drop into testcase/cmdline_var_spaces.sh:

set -e

mk="$@"

cat <<EOF > Makefile
ifdef SUBMAKE
all:
       +@\$(SUBMAKE) SUBMAKE=
else
CLVAR := FAIL
CLVAR_RESULT := FAIL
ifeq (\$(CLVAR),multi word)
CLVAR_RESULT := PASS
endif
all:
       @echo \$(CLVAR_RESULT) \$(CLVAR) \$(origin CLVAR)
endif
EOF

make SUBMAKE="${mk}" CLVAR='multi word' 2> /dev/null

Which prints out:

(make)  PASS multi word command line
(ckati) FAIL multi command line

If you print out $(MAKEFLAGS), the makeflags is written as:

CLVAR=multi\ word SUBMAKE=../../ckati\ SHELL=/bin/bash

So kati's MAKEFLAGS parsing needs to handle the escaped spaces (I'm not sure if anything else can be escaped)

Force regen if ninja file is missing

I told a coworker to force a ninja file regeneration (to test a different bug) by removing the generated ninja file. But kati doesn't realize that it needs to regenerate the ninja file in that case:

$ rm out/build-aosp_flounder.ninja
$ m
.....
make: Entering directory `aosp'
Running kati to generate build-aosp_flounder.ninja...
No need to regenerate ninja file
Starting build with ninja
ninja: Entering directory `.'
ninja: error: loading 'out/build-aosp_flounder.ninja': No such file or directory
make: *** [ninja_wrapper] Error 1
make: Leaving directory `aosp'

make target "ckati_clean" does not remove test output

make doesn't work

almasrymina@almasrymina:~/src/kati$ make
rm -rf out/{src,pkg/}/github.com/google/kati
mkdir -p out/{src,pkg/}/github.com/google/kati
cp -a ast.go bootstrap.go buf.go buf_test.go dep.go depgraph.go doc.go evalcmd.go eval.go exec.go expr.go expr_test.go fileutil.go flags.go func.go func_test.go log.go ninja.go ninja_test.go parser.go pathutil.go pathutil_test.go query.go rule_parser.go rule_parser_test.go serialize.go shellutil.go shellutil_test.go stats.go strutil.go strutil_test.go symtab.go var.go version.go worker.go cmd out/src/github.com/google/kati
GOPATH=$(pwd)/out:${GOPATH} go get github.com/google/kati/cmd/kati
import cycle not allowed
package github.com/google/kati/cmd/kati
imports bytes
imports errors
imports runtime
imports runtime/internal/atomic
imports runtime
make: *** [go_src_stamp] Error 1

Fail early for unsupported automatic variables

Kati doesn't support all automatic variables -- for the variables it doesn't support, it should error out early instead of acting like they are undefined.

Example: $? -- this should be the prerequisites that are newer than the target. Ninja has no way of supporting this, but it's not often used either.

Could not build Linux kernel

I guess this is a very complex issue because the kernel has a lot of Makefiles and uses a lot of specific features that may not be supported.
Now if I run kati in the kernel source tree, I get this:

./scripts/Makefile.build:423: kati doesn't support .PRECIOUS
./arch/x86/entry/syscalls/Makefile:49: Automatic variable `$?' isn't supported yet
*** [archheaders] Error 1

It is at least two features that should be supported in order to build Linux, but I believe there are more.

"make test" fails on clean build

Test case:

$ make clean
...
$ git clean -xfd
...

# GOPATH is required on my system to access glog package
$ GOPATH=/usr/share/gocode make test
g++ -c -std=c++11 -g -W -Wall -MMD -MP -O -DNOLOG -march=native -o affinity.o affinity.cc
g++ -c -std=c++11 -g -W -Wall -MMD -MP -O -DNOLOG -march=native -o command.o command.cc
...
GOPATH=$(pwd)/out:${GOPATH} go test *.go
ok      command-line-arguments  0.011s
ruby runtest.rb
abspath.mk: FAIL
...
multi_explicit_output_patterns.mk: FAIL
--- out.make    2016-04-27 10:31:55.930860676 +0300
+++ out.kati    2016-04-27 10:31:55.930860676 +0300
@@ -1,10 +1,5 @@
 === test ===
-echo azz
-azz
-echo zza
-zza
-echo zzz
-zzz
+sh: ../../kati: No such file or directory

 === FILES ===

multi_explicit_output_patterns_double_colon.mk: FAIL (expected)
multi_implicit_output_patterns.mk has a string "FAIL" in its expectation
Makefile:21: recipe for target 'test' failed
make: *** [test] Error 1

I have to work-around this with

$ make clean
...
$ git clean -xfd
...
$ make kati
...
$ GOPATH=/usr/share/gocode make test
...

BTW: simply adding the missing target won't work if you have to use GOPATH, because

$ GOPATH=/usr/share/gocode make kati test 2>&1 | tee build.log
rm -rf out/src out/pkg
mkdir -p out/src/github.com/google/kati
cp -a .... cmd out/src/github.com/google/kati
GOPATH=$(pwd)/out:${GOPATH} go get github.com/google/kati/cmd/kati
go install github.com/golang/glog: mkdir /usr/share/gocode/pkg: permission denied
Makefile.kati:29: recipe for target 'go_src_stamp' failed
make: *** [go_src_stamp] Error 1

I have no idea why Go wants to install kati in my system directory.

Tag a release?

Hello,
It would be great to have a tagged release so we could include kati in the Homebrew core.
Thanks!

ckati differs from make in $(join x,x x)

For $(join x,x x)
make 3.8.1 will get: xx x
ckati will get: xx
so that GNU Make Standard Library cannot be used with ckati.

ckati differs from make in ifdef

code sample:

define foo
$1
endef

ifdef foo
$(info defined)
else
$(info undefined)
endif

all:

make 3.8.1 shows "defined", but ckati shows "undefined".