Tree Mapping, Maintenance Debt, and the Datacenter Squeeze

Last month, unusually, there was very little time left for reading between running my household, taking care of my health and making progress on my work projects. I only picked up just a few noteworthy pieces.

Refreshingly, a non-LLM-related piece of sound engineering wisdom: The third hard problem by Roman Kashitsyn. Besides the two classic “hard problems of computer science”, namely cache invalidation and naming, Roman identifies tree mapping as another general area where practitioners often stumble into pitfalls. Tree mapping is the problem of taking a graph and structuring tree views of it for algorithms that prefer to work on trees. An example is the choice of whether to structure data in a row-major or column-major representation in a database schema. Another example is the choice to write a documentation guide with one chapter per theme, versus one chapter per educational level. Tree mapping problems do not have generally optimal solutions and good enough approaches are often contextual and a matter of experiential instinct.

❦❦❦

In You Need AI That Reduces Maintenance Costs, James Shore explains that LLM-assisted engineering does not reduce the overall volume of followup/maintenance work. Given that velocity is not infinite (even with evermore powerful/efficient inference, there's always some ceiling on token throughput), mathematically we must always account for the overheads of taking care of maintenance, in the future, for work that's being done today. The more work we do today, the more overhead we have in the future. It is possible to increase the velocity “today” so much that the maintenance overhead “tomorrow” becomes unsustainable. Unless we evolve our preference to start favoring LLMs that are better at maintenance specifically.

❦❦❦

One piece that generated some heated discussion online was Cut off by Anton Leich. It is very US-centric and, in my opinion, too enthusiastic about current trends. It gave me pause, however, and I felt compelled to revisit its main arguments 1) from the perspective of EU policymakers (which is close to my role now) and 2) taking into account additional risks and opportunities that Anton Leich did not recognize. You can find my write-up here: AI/LLM Access Policy for the World Outside Washington.

One of the arguments that both Anton Leich and I agree on is that current geo-political trends would make each regional power want, strategically, to increase its own compute capability, i.e. build more data centers. This is, in my opinion, true even if we end up not increasing LLM usage over time (i.e. we will want geographically local data centers even beyond the current AI/LLM hype) because governments are increasingly interested in increasing their sovereignty over data storage and processing.

However, there is political work to do. Datacenter buildout competes for hard resources (space, energy supply) with other land uses. In the Netherlands, this discussion is ongoing. From another angle, the politics of the job market evolution caused by LLMs are not being handled particularly gracefully and we can already see signs of knee-jerk political blowback. This tension will probably become one of the key debates of this decade.

❦❦❦

References: