1# Ninja File Canonicalizer 2 3Suppose we have a tool that generates a Ninja file from some other description (think Kati and makefiles), and during 4the testing we discovered a regression. Furthermore, suppose that the generated Ninja file is large (think millions of 5lines). And, the new Ninja file has build statements and rules in a slightly different order. As the tool generates the 6rule names, the real differences in the output of the `diff` command are drowned in noise. Enter Canoninja. 7 8Canoninja renames each Ninja rule to the hash of its contents. After that, we can just sort the build statements, and a 9simple `comm` command immediately reveal the essential difference between the files. 10 11## Example 12 13Consider the following makefile 14 15```makefile 16second := 17first: foo 18foo: 19 @echo foo 20second: bar 21bar: 22 @echo bar 23``` 24 25Depending on Kati version converting it to Ninja file will yield either: 26 27``` 28$ cat /tmp/1.ninja 29# Generated by kati 06f2569b2d16628608c000a76e3d495a5a5528cb 30 31pool local_pool 32 depth = 72 33 34build _kati_always_build_: phony 35 36build first: phony foo 37rule rule0 38 description = build $out 39 command = /bin/sh -c "echo foo" 40build foo: rule0 41build second: phony bar 42rule rule1 43 description = build $out 44 command = /bin/sh -c "echo bar" 45build bar: rule1 46 47default first 48``` 49 50or 51 52``` 53$ cat 2.ninja 54# Generated by kati 371194da71b3e191fea6f2ccceb7b061bd0de310 55 56pool local_pool 57 depth = 72 58 59build _kati_always_build_: phony 60 61build second: phony bar 62rule rule0 63 description = build $out 64 command = /bin/sh -c "echo bar" 65build bar: rule0 66build first: phony foo 67rule rule1 68 description = build $out 69 command = /bin/sh -c "echo foo" 70build foo: rule1 71 72default first 73``` 74 75This is a quirk in Kati, see https://github.com/google/kati/issues/238 76 77Trying to find out the difference between the targets even after sorting them isn't too helpful: 78 79``` 80diff <(grep '^build' /tmp/1.ninja|sort) <(grep '^build' /tmp/2.ninja | sort) 811c1 82< build bar: rule1 83--- 84> build bar: rule0 853c3 86< build foo: rule0 87--- 88> build foo: rule1 89``` 90 91However, running these files through `canoninja` yields 92 93``` 94$ canoninja /tmp/1.ninja 95# Generated by kati 06f2569b2d16628608c000a76e3d495a5a5528cb 96 97pool local_pool 98 depth = 72 99 100build _kati_always_build_: phony 101 102build first: phony foo 103rule R2f9981d3c152fc255370dc67028244f7bed72a03 104 description = build $out 105 command = /bin/sh -c "echo foo" 106build foo: R2f9981d3c152fc255370dc67028244f7bed72a03 107build second: phony bar 108rule R62640f3f9095cf2da5b9d9e2a82f746cc710c94c 109 description = build $out 110 command = /bin/sh -c "echo bar" 111build bar: R62640f3f9095cf2da5b9d9e2a82f746cc710c94c 112 113default first 114``` 115 116and 117 118``` 119~/go/bin/canoninja /tmp/2.ninja 120# Generated by kati 371194da71b3e191fea6f2ccceb7b061bd0de310 121 122pool local_pool 123 depth = 72 124 125build _kati_always_build_: phony 126 127build second: phony bar 128rule R62640f3f9095cf2da5b9d9e2a82f746cc710c94c 129 description = build $out 130 command = /bin/sh -c "echo bar" 131build bar: R62640f3f9095cf2da5b9d9e2a82f746cc710c94c 132build first: phony foo 133rule R2f9981d3c152fc255370dc67028244f7bed72a03 134 description = build $out 135 command = /bin/sh -c "echo foo" 136build foo: R2f9981d3c152fc255370dc67028244f7bed72a03 137 138default first 139``` 140 141and when we extract only build statements and sort them, we see that both Ninja files define the same graph: 142 143```shell 144$ diff <(~/go/bin/canoninja /tmp/1.ninja | grep '^build' | sort) \ 145 <(~/go/bin/canoninja /tmp/2.ninja | grep '^build' | sort) 146``` 147 148# Todo 149 150* Optionally output only the build statements, optionally sorted 151* Handle continuation lines correctly 152