Can there be downsides to making a system more measurable? It seems like such a central assumption in analytics that better measurability is always better, but can it lead to a system being worse off?

To answer this question, I want to draw on some ideas from James C Scott’s Seeing Like a State. The book is about how schemes introduced to measure behaviour have changed the societies that they’ve been applied to, and the major idea Scott introduces from the study of these schemes is the idea of legibility. The short version is that if there’s a drive to understand something, then systems will often be reorganised by their administrators in order to be more understandable and measurable - but especially more understandable according to what the administrators think is most useful. The basis for this idea stems from how governments reorganise societies in order to make them more governable, such as introducing standardised names, or standardising measurements. These cause a change in a system in order to make it more measurable (or legible, to the adminstrators).

I think this is a powerful idea, and it has me thinking about how his idea of legibility applies in the analytics space. I think it’s worth applying to questions of how designing measurements can affect systems that we control, and how this can drive designing the system so we can better measure it.

First, I’ll give a brief summary of Scott’s discussion of the ‘cameral sciences’ and 18th century German scientific forestry, before talking about some potential lessons for analytics.

Getting into the trees

During the 18th century, the development of scientific forestry in Europe was driven by a need for understanding what resources were available to states. The cameral sciences (or Cameralism) was the German form of scientific administration.

Part of these natural resources were the forests and what they produced - chiefly wood. Forests were extremely useful not only for state revenue, but for strategic resources such as shipbuilding. As such, there was a desire to catalogue and maintain the forests This was all the more pressing by the fear of Holznot (wood crisis), in which there would be a lack of available wood supply caused by depletion.

As a consequence, the forest was conceptualised as a managed resource that had to be maintained to supply the necessities for the state’s industries, which led to the development of scientific forestry as a field. The marshalling of various disciplines meant that the status and expected yield of a forest could be measured to a very high degree, and this was not only true in knowing what resources were currently available, but even forecasting what would be available in the future.

But the ‘lens’ through which the forest was seen was reorganised around aspects particularly useful for the state, such as the type of trees necessary for industry. The other aspects of the forest’s use were downgraded in their focus:

“The best way to appreciate how heroic was this constriction of vision is to notice what fell outside its field of vision. Lurking behind the number indicating revenue yield were not so much forests as commercial wood, representing so many thousands of board feet of saleable timber and so many cords of firewood fetching a certain price. Missing, of course, were all those trees, bushes, and plants holding little or no potential for state revenue. Missing as well were all those parts of trees, even revenue-bearing trees, which might have been useful to the population but whose value could not be converted into fiscal receipts.”

In essence, these parts of the forest were ignored since they weren’t considered useful to the overall central aim. But the simplification of measurement did not just standardise the measurements of the forest - it standardised the forest itself.

“The achievement of German forestry science in standardizing techniques for calculating the sustainable yield of commercial timber and hence revenue was impressive enough. What is decisive for our purposes, however, was the next logical step in forest management. That step was to attempt to create, through careful seeding, planting, and cutting, a forest that was easier for state foresters to count, manipulate, measure, and assess. The fact is that forest science and geometry, backed by state power, had the capacity to transform the real, diverse, and chaotic old-growth forest into a new, more uniform forest that closely resembled the administrative grid of its techniques”

Images reproduced from Scott (1998)

Mixed temperate forest, part managed,	One aisle of a managed poplar forest in Tuscany
part natural regeneration

At first, this seemed like a massive success, and “in the short run, this experiment in the radical simplification of the forest to a single commodity was a resounding success.” But in the long run, there were unintended consequences:

“A new term, Waldsterben (forest death), entered the German vocabulary to describe the worst cases. An exceptionally complex process involving soil building, nutrient uptake, and symbiotic relations among fungi, insects, mammals, and flora-which were, and still are, not entirely understood-was apparently disrupted, with serious consequences. Most of these consequences can be traced to the radical simplicity of the scientific forest.”

Lessons for analytics

We can see a potential analogous process in research and analytics - the focus on what is considered most important for certain needs can lead to a neglect of other parts. The ‘success’ of some forms of data can reflect as much as the biases of the ultimate consumer of the data as much as the object they’re trying to measure. By choosing particular measurements and focusing on certain attributes, we miss other important factors. This is doubly so for factors that are not so easily quantified or straight-forward in their benefit. In short, what we see is that making a system more measurable is not without its effects. By imposing structure onto the system, we make measurability easier and seemingly give more insight into how it’s used, but are at the risk of changing the system itself.

I think it’s worth stressing that this can happen as a factor of wanting to make a system better, and there are clear benefits that drive these choices. These benefits can give the appearance of an overall more manageable system, but have lasting long-term consequences.

Experimentation

The German forestry example has such a factor that is especially relevant for analytics - because of the simplification of the forest, experimentation became better:

“The new legible forest was also easier to manipulate experimentally. Now that the more complex old-growth forest had been replaced by a forest in which many variables were held constant, it was a far simpler matter to examine the effects of such variables as fertilizer applications, rainfall, and weeding, on same-age, single-species stands. It was the closest thing to a forest laboratory one could imagine at the time. The very simplicity of the forest made it possible, for the first time, to assess novel regimens of forest management under nearly experimental conditions.”

This, I think, is a key blindspot for analytics. While it’s readily understood that imposing unsuitable metrics from above can be damaging, putting metrics in place to progress towards better experimentation seems innocent, if not praise-worthy. It appears like a reasonable trade-off if we can change the system to make experiments better, so that we can better give the customer what they want. After all, what could be wrong with wanting to be able to make more data-driven decisions?

But it’s worth to keep in mind that better experimentation makes things more amenable to the analyser, but not necessarily the end-user. Such an example is wanting to reduce the number of experiences available to users, and then experiment until the best variant is found. At first this might be pitched as ensuring that the optimal user experience is eventually settled on, even if the changes are made for the end-user’s benefit, this is only possible as long as the analyser’s choices of metrics ultimately match what end-users actually want.

This is not only an issue on the user-side, but on the backend as well - engineering choices can be influenced by the needs of data collection. An engineering team needs to be mindful that supporting some experiment types can lead to being locked into a particular path that may be hard to walk away from.

The desire for better experimentation—among other goals in analytics—can drive legibility in a negative way. We have ideas about how we want to improve a system for its users. We develop metrics for them: experimentation is how we validate our metrics, and metrics are how we measure success. Although seemingly paradoxical, the process of changing things so that they are more amenable to experimentation (and, ultimately, towards shaping the system to meet the user’s desires) can lead to the potential of the system becoming overall less suited for its purpose. And, in the worst case scenario, becoming the antithesis of what was desired.

Seeing the forest

So how might be avoid these issues? First of all, it’s good to keep in mind that a metric is only useful in as far as it accurately reflects what we want it to reflect. In many ways, we choose metrics not because we want that specific metric, but because it’s the best measure of the actual outcome we want. We can only see as far as users intersect with our measurements, and so while what you pay attention to is important, just as important is how that attention affects what you think is important. In short, we don’t see how the user sees the product, we see how we think the user uses the product.

But while that would seem to suggest that we should alter the system in order to make it more measurable, I think this leads exactly to a potential trap. The major thing here is knowing that increasing measurability may involve a trade-off in another aspect, and that there’s no perfect measurement system. A good start is to consider multiple metrics and to reduce overreliance on a single metric. By balancing various needs and trying to keep an independent set of metrics, we can help identify our blindspots and what we might be missing, while also reducing the impulse to change the system to be more suited for a single metric.

While there’s a general lesson here for any push to make something more measurable and the dangers involved in pursuing metrics single-mindedly, I think there’s also a deeper issue here. There is a potential feedback loop, where advances in analytics lead to more data being available, and this larger availability of data increases the appetite for data – the increase drives the expectation of more data, which incentivises the production of more data. The more data available, the greater need for simplifying assumptions to interpret the data. This increases the relative importance of this data, but this is built on perhaps an oversimplified view of what users are interacting with.

Measurement is a powerful tool, but it has its own tradeoffs. If we keep this in mind, we should be able to avoid unintended consequences in pursuit of better measurement.