[Ninja author here] Nice post, cool to see the deep dive! I also appreciate the details on how they produced their numbers.
As they observe, Ninja gets to be fast mostly by cheating: it avoids a lot of work by saying many things are just out of scope for Ninja to do, and that means it is a useful a target to race against. (Funny thing: when I wrote Ninja I was misremembering how fast an earlier build system was so I kept trying to make it faster. So don't treat it as a lower bound, I just made it up!)
I comment here to say I find the explanation for 'why' in this post unsatisfying. They mention three design decisions.
The first one is a criticism of CMake, not Ninja (?), so I don't think it can be why. I might have misunderstood?
The second reason given is doing some work like header dependencies in multiple threads. This is the most plausible reason to me but it still feels unlikely. It's a very small amount of work: the post mentions 300 compiles, so maybe parsing 300 small text files?
The third is that they run the compiler up front an additional time to gather headers, which is strictly more work than Ninja. There is some hand waving about file access patterns but I am skeptical; if the end-to-end build time is 3 seconds then the project is small enough to all fit in kernel caches. They also mention doing other things like invoking the compiler to get version information. This seems like it would dwarf any performance gain from number 2.
Maybe it's just my own curiosity, I think this post would be better if it had a better explanation for the reason. I'm not disputing the result, I just think the result should make you suspicious that something else is going on, and you might learn something from that! You could for example explore whether it's the header dependency thing by profiling the Ninja invocation and seeing if it's waiting for CPU or waiting for tasks to execute.
(If I had to guess without looking at any of the involved code, I would predict it's something about how CMake generates the build, like it introduces serialization in a place where build2 is parallel, or it adds some extra build steps like gathering the current git hash into a header file or something.)
[build2 author here] Thanks for the feedback! Some additional details:
> The first one is a criticism of CMake, not Ninja (?), so I don't think it can be why.
Fair enough. The point I was making is that if you want to compete with Ninja, you cannot leave any potential performance gains on the table.
> The second reason given is doing some work like header dependencies in multiple threads. This is the most plausible reason to me but it still feels unlikely.
We are talking about ~2% performance difference here. Parallelizing even a small amount of work across 24 threads rather that doing it serially saving a percent or two feels plausible to me.
> There is some hand waving about file access patterns but I am skeptical; if the end-to-end build time is 3 seconds then the project is small enough to all fit in kernel caches.
It fits into the system's file cache unless there is memory pressure, like one would expect from having 24 C++ compiler jobs running in parallel. We actually measured this in isolation (with more detailed results in the linked article) and it has a measurable effect.
> They also mention doing other things like invoking the compiler to get version information. This seems like it would dwarf any performance gain from number 2.
I measured this, it costs 70ms or ~2% of the overall time.
> It fits into the system's file cache unless there is memory pressure, like one would expect from having 24 C++ compiler jobs running in parallel. We actually measured this in isolation (with more detailed results in the linked article) and it has a measurable effect.
300 TUs is not much. If they build in 3 seconds then they are trivial (small). If the machine is 24-thread (I assume some sort of heterogeneous 12-core), how little RAM does the machine have for the kernel to start evicting page cache during the build?
> > They also mention doing other things like invoking the compiler to get version information. This seems like it would dwarf any performance gain from number 2.
I measured this, it costs 70ms or ~2% of the overall time.
Couldn’t you amortize this to 0 by just caching the result and only changing it if the binary timestamp changed?
> Couldn’t you amortize this to 0 by just caching the result and only changing it if the binary timestamp changed?
Yes, that would be nice, but the tricky question is can any of this information change without the compiler binary mtime changing? First off, GCC's gcc/g++ binaries are drivers and are not what does the actual compilation, it's private cc1/cc1plus binaries that do the job. Can one of these change but not the driver? I think it's plausible (some package manager optimization where the file is not touched if it hasn't changed). So at a minimum we would need to discover where those are located (probably by invoking gcc/g++) and checking them as well. Could there be something else? Who knows. We value speed very much but we value correctness even more.
I think a more fruitful direction to explore is to improve GCC itself to dump all this information in a single invocation and in a machine-readable format (JSON). I think if we go from 70ms to 14ms (and perhaps even lower because this special GCC mode could conceivably do things faster than how we do it now), it would be good enough.
Simple CMake projects using the Ninja generator are very efficient, unless you create generated files.
And if they are in their own targets, they are not really an issue (they would serialize everything that depends on them as you'd expect), but if you have them in a library grouped with other files to compile, then the whole library compilation is serialized.
And obviously worse if you also have to build the generator for the generated files, but that's not a big surprise, you can observe that in full builds of Chromium or its libraries too waiting for protoc if you crank the parallelization a lot.
Yes, I don’t remember the details but vaguely remember that CMake tends to group things together that could in principle be made more parallel, as you mention with generates files in a library. On the other hand if build2 makes it easier for authors to express these kinds of patterns without the serialization then I count that as a win for build2!
I feel like make/ninja suffer from the same output modstamp > input modstamps to rebuild. Really a build system should track modstmaps on all inputs (apps included), and the rebuild on not-equal.
Reading about manifests immediately reminded be of Clearcase/Clearmake. My memory may be not correct after all the years passed, but configuration records for derived objects contained the same information as proposed manifests + some more metadata about source (versions, at least).
If I remember correctly SCons allows to implement a custom "checker" (or whatever it is called there) to compare files based on any criteria that are useful, not just timestamps. But this is about the only visible advantage of SCons over Make, at least for our case.
Why not just embrace Content Based Addressing and the Bazel action cache? Not necessarily adopt the RBE protos, but you could have on-disk cache that are similar or even identical.
Nowadays, I think a build tool that doesn't natively support distributed caching (and possibly remote execution) is a weird choice. And I don't think that spawning layers of processes allows for good parallelization as you don't know if an action is going to be network bound or compute bound and the job slot is then spent. So you either oversubscribe or undersubscribe.
Sadly, I think things like distcc or ccache have mostly gone out of mode. Vertically scaling (multi core, multi thread, multi TB RAM build machines) and simple artifact caching have won for now. Artifact caching with dependency tracking in the midst of job scheduling was solved “good enough” by OS based mechanisms. Having used and managed the basics of a build farm, removing the need for networking, build farm management and build job coordination is a huge burden relief when your source tree doesn’t require it.
Part of the motivation to use processes is because their structure helps to keep the job generic, uni
I stumbled upon the very experimental nix-ninja [0] recently.
They seem to create a dynamic nix derivation per compilation unit, which would be very similar to what you describe as manifests in your post as it also creates a hash of all inputs.
Would be interesting to here your opinion on that approach
I really like the ideas you have in n2. Hashing the entire build string is a great idea. And it really is as simple as just having all the timestamps of the inputs in the hash. And you have the output timestamp as a quick test.
Always glad to see authors on here. Thanks for your insight!
Since they had to rewrite the build file for their program I also assume that something is missing. Didn’t see any mention of verifying that.
I also really didn’t like their denigrating tone. It totally turns me off trying build2, because it seems they don’t understand the point of separating build stages like environment setup (getting dependencies), configure, native build, cross build, packaging. I am a very happy ninja user instead of a batteries-included solution because it does its one job well and can be used very flexibly. I personally detest cmake, so I use nix + own configure script + ninja.
The blog is also wrong about ninja being unable to call configure, but I intentionally don’t want that (I want the build stages to communicate in one direction for sanity).
> I want the build stages to communicate in one direction for sanity
That's easy to do if you control the whole pipeline and can integrate all the features together, not so much with the CMake model unfortunately. I think it would be nice if CMake had Ninja integrated as a library, it could lead to some nice optimizations later.
Great that there are performance benefits, but one thing that I think both meson/ninja and cmake (or cmake/ninja) got wrong, and GNU configure got right is "./configure --help". Why do neither of these tools simply add --help here? Yes, their syntax is different, but the issue is not only about --help. Often I could disable documentation or man page via --disable-man or --disable-doc or something like that. I recently had a discussion with a guy who transitioned into ninja, and I reasoned that there should be an option to skip installing man-pages. He thinks everyone needs manpages. I told him I never look at any local man page ever; I only look online for help. And have been doing so for almost 30 years. I understand the 1970s era of man-pages, but I have no use for them (I do gather local documentation, just not via manpages). He did not want to add a way to install his software without having available xmlto, docbook variants (which are a pain to install, just look at the LFS/BLFS instructions and even then they do not work cleanly). Meson and cmake are better than the GNU autotools, but the few things GNU autotools got right, were lost in both meson and cmake, which is interesting. It seems we can only select between different trade-offs here. New is not automatically better. And being 5% faster really is pointless if functionality is removed or not available anymore.
This is not purely a CMake problem, this is a maintainer problem. It's absolutely possible to not define the man page targets if the related tools are not found, or to add a variable to disable that.
In practice, it does show the limitations of the CMake model though. You should not be paying the price to generate the build files for bits you don't care about. You should be able to just load the CMake files and build a target, ignoring the rest.
This is how tools like Bazel function and that allows them to scale to very large monorepos. Build files are only processed when a target from that folder is built directly or used. CMake has to load everything as it does not know about the build tree and the maintainer has to manage exclusion lists or inclusion lists.
> He thinks everyone needs manpages. I told him I never look at any local man page ever; I only look online for help...
Everyone's workflow is different. Your's obviously works for you and many others.
But I like local man pages. They get updated at the same time as the software is updated. Sometimes I have an older version of software and the online version is for a newer version, or vice-versa. Local man pages are always available and generally quicker than accessing them online.
If the machines in question are mult user, you should always install local documentation regardless of your personal preferences.
I think what you're talking about is part of a much larger pattern. Autotools imagines that the most important thing is the "user" in a world where people want to download your software and build it on their machine. They don't care (as much) about fast build times, the semantic clarity of the build files themselves, the ease with which one may change or extend the build sequence, test harnesses . . . Those are all developer worries.
They do care that the code can compile on their (possibly old) machine. They do care about the ability to include some features and not others (including their ability to not have to make decisions on this if they don't want to). And they certainly want documentation that helps them to achieve their goal (which, again, is to build the software).
The newer tools make substantial inroads in making the developer's life easier. If your software is not of the sort where people download it and try to build it (because it only ships as a closed binary, is entirely internal, etc.) then maybe that makes sense. But until all these fancy gadgets start seriously caring about the use case above, I'm going to stick with autotools.
> They do care that the code can compile on their (possibly old) machine.
In my experience is it doesn't work. Sure it can be made to work and autotools asks all the right questions. However in the real world I have yet to see an autotools project that works out of the box in my custom environment.
Meanwhile cmake just worked every time when I plugged in the toolchain file that I created for the very first project I needed to build for my environment.
Again, it isn't that autotools cannot do what is claims. However nobody gets the details right and so something doesn't work. That something is different for every project. In the end what works is either what someone takes the time to make work and those things that are "very close". My project is cross compiling linux, which cmake does well (autotools also messed something up, often failing to use the sysroot paramater where needed)
I wish that when people posted a performance win, they would include a theory explaining it. Unexpected speedups are usually noise, bugs, or harness issues. No program is faster due to the superior manly virtue of its programmer. Every effect has a cause.
When, from first principles, something shouldn't be faster, yet it is, you dig and dig until you understand. You don't just say your shit rules and the other thing sux0rz. If you do, you'll regret it: if you don't understand how you got a win, you don't understand how to keep it.
Cmake spend 15 seconds to generate this project? I find it unlikely build2 is doing the same work as cmake is. Now I will grant cmake is single threaded and slow, so there is a lot of room to do the things it does better (the language sucks, and is part of what forces single threaded). It also wouldn't surprise me if cmake is doing things that are not really needed (odds are the default compiler works - most of the time it isn't valuable to check the version)
Xerces is an old library, probably ported from autotools and will do a lot of autotools like checks which for CMake means: creating a CMake project with the probe, generating the build files from it, building it, gathering results. Once for each test. It's SLOOOOW.
I recently (a few weeks ago) made a project to batch those type of commands together and parallelize tests as much as possible. You can check a merge request for CMake upstream here: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/1226... and the associated issue for more details.
In general, CMake is fast enough, the only slow bit on the critical path is detecting the toolchain. The rest is quick, but if your build scripts are using slow serialized functionality, you'll be paying that price. But only once as the slow results are cached, so next incremental run should be pretty fast!
Not only is CMake single threaded but the generation step time scales almost quadratically* with the number of targets in some cases. Probably not what happens here but this is what forced us to move from CMake — generation times were surpassing 5 minutes, and this is not counting configuring step.
* Because in CMake there are several target properties that may be affected by the "parent" (another target which added this target as dependency), meaning that these properties need to be re-evaluated in the context of each root target dependent on this one. If I remember correctly, exact mechanism is that some properties can have generator expressions, and these expressions can reference "parent" target. Now we imagine an app which has some std lib with 1k targets, and 100 top-level executables dependent on this std lib - suddenly we have 100k target evaluations in generation step.
I'm not trying to overlook cmake's issues... However my first thought on reading about your problem is "you shouldn't do that anyway - write simple code that someone else can understand". A little complexity is sometimes needed, but if your generations times are more than 30 seconds you probably should step back and do something different anyway.
Important detail here is that the problem disappers if you use PRIVATE dependencies — but you can't do that with a heavily templated C++ codebase. Templates (and complexity they bring) are there for good reasons (primarily performance at all costs) so we did not manage to find a path forward with CMake.
> Let's see if we can go even faster. Next, we disable compression in the file cache. We will discuss the file cache in more detail a bit later but for now let's just say that by disabling compression we trade temporary disk space usage for speed:
Something is wrong here. Which compression algorithm is being used here and how much has it been tuned? A core hypothesis of the likes of zram is that disk access is so slow (even NVME), that you can often beat it with the bit-rate of decompression.
1. Is something slow like gzip being used?
2. Is the compression effort over-tuned for size? Do some space benchmarks and make sure that you aren't saving a few dozen MB on GBs of data.
zstd, with 1-3 effort (you may even find negative is a overall win), and a trained dictionary (your data does all look identical) is probably a good start.
It is lz4. While all that you say about disk being slow makes sense, measurements show that in this specific situation disabling compression improves performance by a little. My hand-wavy explanation is that freeing CPU to doing compilation rather than [de]compression is a better use of it, with all the uncompressed disk writes probably ending up in the system's file cache.
[Ninja author here] Nice post, cool to see the deep dive! I also appreciate the details on how they produced their numbers.
As they observe, Ninja gets to be fast mostly by cheating: it avoids a lot of work by saying many things are just out of scope for Ninja to do, and that means it is a useful a target to race against. (Funny thing: when I wrote Ninja I was misremembering how fast an earlier build system was so I kept trying to make it faster. So don't treat it as a lower bound, I just made it up!)
I comment here to say I find the explanation for 'why' in this post unsatisfying. They mention three design decisions.
The first one is a criticism of CMake, not Ninja (?), so I don't think it can be why. I might have misunderstood?
The second reason given is doing some work like header dependencies in multiple threads. This is the most plausible reason to me but it still feels unlikely. It's a very small amount of work: the post mentions 300 compiles, so maybe parsing 300 small text files?
The third is that they run the compiler up front an additional time to gather headers, which is strictly more work than Ninja. There is some hand waving about file access patterns but I am skeptical; if the end-to-end build time is 3 seconds then the project is small enough to all fit in kernel caches. They also mention doing other things like invoking the compiler to get version information. This seems like it would dwarf any performance gain from number 2.
Maybe it's just my own curiosity, I think this post would be better if it had a better explanation for the reason. I'm not disputing the result, I just think the result should make you suspicious that something else is going on, and you might learn something from that! You could for example explore whether it's the header dependency thing by profiling the Ninja invocation and seeing if it's waiting for CPU or waiting for tasks to execute.
(If I had to guess without looking at any of the involved code, I would predict it's something about how CMake generates the build, like it introduces serialization in a place where build2 is parallel, or it adds some extra build steps like gathering the current git hash into a header file or something.)
[build2 author here] Thanks for the feedback! Some additional details:
> The first one is a criticism of CMake, not Ninja (?), so I don't think it can be why.
Fair enough. The point I was making is that if you want to compete with Ninja, you cannot leave any potential performance gains on the table.
> The second reason given is doing some work like header dependencies in multiple threads. This is the most plausible reason to me but it still feels unlikely.
We are talking about ~2% performance difference here. Parallelizing even a small amount of work across 24 threads rather that doing it serially saving a percent or two feels plausible to me.
> There is some hand waving about file access patterns but I am skeptical; if the end-to-end build time is 3 seconds then the project is small enough to all fit in kernel caches.
It fits into the system's file cache unless there is memory pressure, like one would expect from having 24 C++ compiler jobs running in parallel. We actually measured this in isolation (with more detailed results in the linked article) and it has a measurable effect.
> They also mention doing other things like invoking the compiler to get version information. This seems like it would dwarf any performance gain from number 2.
I measured this, it costs 70ms or ~2% of the overall time.
> It fits into the system's file cache unless there is memory pressure, like one would expect from having 24 C++ compiler jobs running in parallel. We actually measured this in isolation (with more detailed results in the linked article) and it has a measurable effect.
300 TUs is not much. If they build in 3 seconds then they are trivial (small). If the machine is 24-thread (I assume some sort of heterogeneous 12-core), how little RAM does the machine have for the kernel to start evicting page cache during the build?
> > They also mention doing other things like invoking the compiler to get version information. This seems like it would dwarf any performance gain from number 2. I measured this, it costs 70ms or ~2% of the overall time.
Couldn’t you amortize this to 0 by just caching the result and only changing it if the binary timestamp changed?
> Couldn’t you amortize this to 0 by just caching the result and only changing it if the binary timestamp changed?
Yes, that would be nice, but the tricky question is can any of this information change without the compiler binary mtime changing? First off, GCC's gcc/g++ binaries are drivers and are not what does the actual compilation, it's private cc1/cc1plus binaries that do the job. Can one of these change but not the driver? I think it's plausible (some package manager optimization where the file is not touched if it hasn't changed). So at a minimum we would need to discover where those are located (probably by invoking gcc/g++) and checking them as well. Could there be something else? Who knows. We value speed very much but we value correctness even more.
I think a more fruitful direction to explore is to improve GCC itself to dump all this information in a single invocation and in a machine-readable format (JSON). I think if we go from 70ms to 14ms (and perhaps even lower because this special GCC mode could conceivably do things faster than how we do it now), it would be good enough.
Simple CMake projects using the Ninja generator are very efficient, unless you create generated files.
And if they are in their own targets, they are not really an issue (they would serialize everything that depends on them as you'd expect), but if you have them in a library grouped with other files to compile, then the whole library compilation is serialized.
And obviously worse if you also have to build the generator for the generated files, but that's not a big surprise, you can observe that in full builds of Chromium or its libraries too waiting for protoc if you crank the parallelization a lot.
Yes, I don’t remember the details but vaguely remember that CMake tends to group things together that could in principle be made more parallel, as you mention with generates files in a library. On the other hand if build2 makes it easier for authors to express these kinds of patterns without the serialization then I count that as a win for build2!
Thank you for your work on Ninja. It's just really good.
Thanks for saying this! I am close enough to it that I mostly remember all of the bad decisions I made that are now unfixable, haha.
I would like to read blogpost about this and what could be done better.
Better to have made imperfectly and learned than to not have made at all. :)
I feel like make/ninja suffer from the same output modstamp > input modstamps to rebuild. Really a build system should track modstmaps on all inputs (apps included), and the rebuild on not-equal.
Would be nice to see build2 go this route.
I did some exploration of this idea in a followup build system! See https://neugierig.org/software/blog/2022/03/n2.html . (It's not really production-ready.)
Reading about manifests immediately reminded be of Clearcase/Clearmake. My memory may be not correct after all the years passed, but configuration records for derived objects contained the same information as proposed manifests + some more metadata about source (versions, at least). If I remember correctly SCons allows to implement a custom "checker" (or whatever it is called there) to compare files based on any criteria that are useful, not just timestamps. But this is about the only visible advantage of SCons over Make, at least for our case.
Why not just embrace Content Based Addressing and the Bazel action cache? Not necessarily adopt the RBE protos, but you could have on-disk cache that are similar or even identical.
Nowadays, I think a build tool that doesn't natively support distributed caching (and possibly remote execution) is a weird choice. And I don't think that spawning layers of processes allows for good parallelization as you don't know if an action is going to be network bound or compute bound and the job slot is then spent. So you either oversubscribe or undersubscribe.
Sadly, I think things like distcc or ccache have mostly gone out of mode. Vertically scaling (multi core, multi thread, multi TB RAM build machines) and simple artifact caching have won for now. Artifact caching with dependency tracking in the midst of job scheduling was solved “good enough” by OS based mechanisms. Having used and managed the basics of a build farm, removing the need for networking, build farm management and build job coordination is a huge burden relief when your source tree doesn’t require it.
Part of the motivation to use processes is because their structure helps to keep the job generic, uni
I stumbled upon the very experimental nix-ninja [0] recently.
They seem to create a dynamic nix derivation per compilation unit, which would be very similar to what you describe as manifests in your post as it also creates a hash of all inputs.
Would be interesting to here your opinion on that approach
[0]: https://github.com/pdtpartners/nix-ninja
I really like the ideas you have in n2. Hashing the entire build string is a great idea. And it really is as simple as just having all the timestamps of the inputs in the hash. And you have the output timestamp as a quick test.
Thank you for Ninja!
The speed (reading the build.ninja file) was never a concern for us. If I could share a wish-list, it will be:
- Fix the possibility of a segmentation fault when the build file is damaged;
- Use a better order of execution: https://github.com/ninja-build/ninja/issues/2157
Always glad to see authors on here. Thanks for your insight!
Since they had to rewrite the build file for their program I also assume that something is missing. Didn’t see any mention of verifying that.
I also really didn’t like their denigrating tone. It totally turns me off trying build2, because it seems they don’t understand the point of separating build stages like environment setup (getting dependencies), configure, native build, cross build, packaging. I am a very happy ninja user instead of a batteries-included solution because it does its one job well and can be used very flexibly. I personally detest cmake, so I use nix + own configure script + ninja.
The blog is also wrong about ninja being unable to call configure, but I intentionally don’t want that (I want the build stages to communicate in one direction for sanity).
> I want the build stages to communicate in one direction for sanity
That's easy to do if you control the whole pipeline and can integrate all the features together, not so much with the CMake model unfortunately. I think it would be nice if CMake had Ninja integrated as a library, it could lead to some nice optimizations later.
Great that there are performance benefits, but one thing that I think both meson/ninja and cmake (or cmake/ninja) got wrong, and GNU configure got right is "./configure --help". Why do neither of these tools simply add --help here? Yes, their syntax is different, but the issue is not only about --help. Often I could disable documentation or man page via --disable-man or --disable-doc or something like that. I recently had a discussion with a guy who transitioned into ninja, and I reasoned that there should be an option to skip installing man-pages. He thinks everyone needs manpages. I told him I never look at any local man page ever; I only look online for help. And have been doing so for almost 30 years. I understand the 1970s era of man-pages, but I have no use for them (I do gather local documentation, just not via manpages). He did not want to add a way to install his software without having available xmlto, docbook variants (which are a pain to install, just look at the LFS/BLFS instructions and even then they do not work cleanly). Meson and cmake are better than the GNU autotools, but the few things GNU autotools got right, were lost in both meson and cmake, which is interesting. It seems we can only select between different trade-offs here. New is not automatically better. And being 5% faster really is pointless if functionality is removed or not available anymore.
This is not purely a CMake problem, this is a maintainer problem. It's absolutely possible to not define the man page targets if the related tools are not found, or to add a variable to disable that.
In practice, it does show the limitations of the CMake model though. You should not be paying the price to generate the build files for bits you don't care about. You should be able to just load the CMake files and build a target, ignoring the rest.
This is how tools like Bazel function and that allows them to scale to very large monorepos. Build files are only processed when a target from that folder is built directly or used. CMake has to load everything as it does not know about the build tree and the maintainer has to manage exclusion lists or inclusion lists.
> He thinks everyone needs manpages. I told him I never look at any local man page ever; I only look online for help...
Everyone's workflow is different. Your's obviously works for you and many others.
But I like local man pages. They get updated at the same time as the software is updated. Sometimes I have an older version of software and the online version is for a newer version, or vice-versa. Local man pages are always available and generally quicker than accessing them online.
If the machines in question are mult user, you should always install local documentation regardless of your personal preferences.
I think what you're talking about is part of a much larger pattern. Autotools imagines that the most important thing is the "user" in a world where people want to download your software and build it on their machine. They don't care (as much) about fast build times, the semantic clarity of the build files themselves, the ease with which one may change or extend the build sequence, test harnesses . . . Those are all developer worries.
They do care that the code can compile on their (possibly old) machine. They do care about the ability to include some features and not others (including their ability to not have to make decisions on this if they don't want to). And they certainly want documentation that helps them to achieve their goal (which, again, is to build the software).
The newer tools make substantial inroads in making the developer's life easier. If your software is not of the sort where people download it and try to build it (because it only ships as a closed binary, is entirely internal, etc.) then maybe that makes sense. But until all these fancy gadgets start seriously caring about the use case above, I'm going to stick with autotools.
> They do care that the code can compile on their (possibly old) machine.
In my experience is it doesn't work. Sure it can be made to work and autotools asks all the right questions. However in the real world I have yet to see an autotools project that works out of the box in my custom environment.
Meanwhile cmake just worked every time when I plugged in the toolchain file that I created for the very first project I needed to build for my environment.
Again, it isn't that autotools cannot do what is claims. However nobody gets the details right and so something doesn't work. That something is different for every project. In the end what works is either what someone takes the time to make work and those things that are "very close". My project is cross compiling linux, which cmake does well (autotools also messed something up, often failing to use the sysroot paramater where needed)
I wish that when people posted a performance win, they would include a theory explaining it. Unexpected speedups are usually noise, bugs, or harness issues. No program is faster due to the superior manly virtue of its programmer. Every effect has a cause.
When, from first principles, something shouldn't be faster, yet it is, you dig and dig until you understand. You don't just say your shit rules and the other thing sux0rz. If you do, you'll regret it: if you don't understand how you got a win, you don't understand how to keep it.
Cmake spend 15 seconds to generate this project? I find it unlikely build2 is doing the same work as cmake is. Now I will grant cmake is single threaded and slow, so there is a lot of room to do the things it does better (the language sucks, and is part of what forces single threaded). It also wouldn't surprise me if cmake is doing things that are not really needed (odds are the default compiler works - most of the time it isn't valuable to check the version)
Xerces is an old library, probably ported from autotools and will do a lot of autotools like checks which for CMake means: creating a CMake project with the probe, generating the build files from it, building it, gathering results. Once for each test. It's SLOOOOW.
I recently (a few weeks ago) made a project to batch those type of commands together and parallelize tests as much as possible. You can check a merge request for CMake upstream here: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/1226... and the associated issue for more details.
In general, CMake is fast enough, the only slow bit on the critical path is detecting the toolchain. The rest is quick, but if your build scripts are using slow serialized functionality, you'll be paying that price. But only once as the slow results are cached, so next incremental run should be pretty fast!
Not only is CMake single threaded but the generation step time scales almost quadratically* with the number of targets in some cases. Probably not what happens here but this is what forced us to move from CMake — generation times were surpassing 5 minutes, and this is not counting configuring step.
* Because in CMake there are several target properties that may be affected by the "parent" (another target which added this target as dependency), meaning that these properties need to be re-evaluated in the context of each root target dependent on this one. If I remember correctly, exact mechanism is that some properties can have generator expressions, and these expressions can reference "parent" target. Now we imagine an app which has some std lib with 1k targets, and 100 top-level executables dependent on this std lib - suddenly we have 100k target evaluations in generation step.
I'm not trying to overlook cmake's issues... However my first thought on reading about your problem is "you shouldn't do that anyway - write simple code that someone else can understand". A little complexity is sometimes needed, but if your generations times are more than 30 seconds you probably should step back and do something different anyway.
Important detail here is that the problem disappers if you use PRIVATE dependencies — but you can't do that with a heavily templated C++ codebase. Templates (and complexity they bring) are there for good reasons (primarily performance at all costs) so we did not manage to find a path forward with CMake.
I wonder how it compares to Tup ( https://gittup.org/tup/ ).
I keep forgetting about it. It's alien technology, the road not taken, etc.
> Let's see if we can go even faster. Next, we disable compression in the file cache. We will discuss the file cache in more detail a bit later but for now let's just say that by disabling compression we trade temporary disk space usage for speed:
Something is wrong here. Which compression algorithm is being used here and how much has it been tuned? A core hypothesis of the likes of zram is that disk access is so slow (even NVME), that you can often beat it with the bit-rate of decompression.
1. Is something slow like gzip being used?
2. Is the compression effort over-tuned for size? Do some space benchmarks and make sure that you aren't saving a few dozen MB on GBs of data.
zstd, with 1-3 effort (you may even find negative is a overall win), and a trained dictionary (your data does all look identical) is probably a good start.
It is lz4. While all that you say about disk being slow makes sense, measurements show that in this specific situation disabling compression improves performance by a little. My hand-wavy explanation is that freeing CPU to doing compilation rather than [de]compression is a better use of it, with all the uncompressed disk writes probably ending up in the system's file cache.
> And now we are 2.2% faster than Ninja!
This claim isn't supported by the author's measurements?
1 - 3.355/3.429 = 0.021581 = 2,2%
no?
At least you know it was written by a fallible human. But if you point out my mistake, I will be happy to fix it.