You can make a system better by making it worse.
An engineer on the Starlink team told me a story:
The solar panels on a satellite are tightly folded accordion-style during transport from earth to space, in order to minimize the volume inside the rocket.
It’s industry standard to build a dampener into the mechanism that unfolds the solar panels. The solar panels are under high tension while folded, and without a dampener would slam out at great speed. Satellites are delicate electronic devices, and the physical stress of an un-dampened deployment breaks some satellites.
However, the dampener is heavy, and weight reduces the number of satellites that can be launched in a single rocket.
The Starlink team tried removing the dampener as an experiment. A couple satellites did break due to the increased physical strain, but the weight savings allowed many more satellites to be packed into the rocket, more than enough to replace the broken ones.
The satellites themselves were relatively cheap, but the rocket was very expensive, so this trade-off was worth it.
This is an interesting story, because it illustrates a trade-off. The correctness of the system was clearly degraded by removing the dampeners. The solar panels deploy in a haphazard and uncontrolled fashion. More satellites break.
However, by the objective function of “maximum number of working satellites, per dollar”, this is also clearly the correct choice.
In the design of any system, from an API interface to a sales commission plan, there is a fundamental trade-off between:
Designing a system to maximize Completeness will result in worse Correctness, as the additional flexibility creates more edge cases to cover, and worse Simplicity as the additional flexibility requires more concepts and abstractions to be introduced.
Similarly, maximizing Correctness degrades Completeness and Simplicity. And maximizing Simplicity degrades Completeness and Correctness.
Many engineering minded people make the mistake of maximizing correctness and completeness, and accepting the resulting complexity as inherent to the domain. This is the wrong approach.
The objective function of a system is “maximal success for users”.
There is no difference between a low-correctness system that’s full of bugs, and a low-simplicity system that users implement wrong. A bug introduced by the system designer, and a bug introduced by the system user, have an identical impact on the objective function.
Kubernetes is an example of a system which maximizes correctness and completeness at the expense of simplicity. I would guess that the vast majority Kubernetes failures stem from high-complexity, resulting in user-error. There is a whole ecosystem of products offering “Kubernetes, but simplified”.
Why do engineering minded people maximize correctness and completeness, at the expense of overall success?
One reason is that correctness and completeness are easy to reason about in isolation, and can be objectively measured. It’s straightforward to list, quantify, and address bugs, edge-cases, and user stories in a way that feels like forward progress.
By contrast, the benefits of simplicity are felt weakly by the implementor of a system, who is maximally familiar with its construction. The benefits of simplicity are most realized by a novice in the system. It’s hard to understand the perspective of a novice as an expert. It often requires talking to real users.