> In real life, using a hermetic build system for tests does not actually scale. The dependency graph of your repo becomes entwined with how much of your repo gets tested. Change a deeply nested component often? Even though you only want to deploy one out of all your services, you’ll get full rebuilds of your repo.I still think a hermetic build system is the way to go; but I wouldn’t gate merges on testing the full repo. Instead, let’s give some flexibility to our engineers.
Scaling a monorepo also involves scaling with the number of engineers at the company. The strategy mentioned here, of putting the trust in your engineers to determine what tests need to be run after platform-wide changes does not scale. Eventually there will be enough product teams who care more about speed than quality. Likely resulting in increased outages, because changes won't be guaranteed to be fully tested in CI.
A more reliable way to solve this problem would be to scale out testing in CI. For example with Bazel's remote execution or any type of CI sharding strategy. Optimizing CI build and test times for changes impacting many targets is a scaling problem any monorepo will eventually need to tackle.
In a perfect world, you can scale CI indefinitely. However, I don't think it's a simple as that. As mentioned in the post, even with a hermetic build system your CI times become entwined with your dependency graph, no matter if you're able to shard it over remote executors or not.
The block you've quoted specifically mentions gating _merges_. I still think its prudent to run ~all tests after merge, with automatic reverts if tests start failing. I really want to make sure that people think of CI, merges, and deploys as pieces of a larger puzzle and not a monolith.
I updated the language around this in [0], hope that makes it clearer as to my intention.
Thanks for the clarification! fwiw Uber's monorepo treats CI and merges as the same, and deployments separately [0].
What it comes down to is engineering a solution to "mitigate a broken main" (automatic reverts) or a solution to "keep main green" (gating commits on a green build/test).
Scaling a monorepo also involves scaling with the number of engineers at the company. The strategy mentioned here, of putting the trust in your engineers to determine what tests need to be run after platform-wide changes does not scale. Eventually there will be enough product teams who care more about speed than quality. Likely resulting in increased outages, because changes won't be guaranteed to be fully tested in CI.
A more reliable way to solve this problem would be to scale out testing in CI. For example with Bazel's remote execution or any type of CI sharding strategy. Optimizing CI build and test times for changes impacting many targets is a scaling problem any monorepo will eventually need to tackle.