Research commentary · Stochastic models

When the state is too small.

A dashboard can give two machines the same label while concealing different prospects. The choice of what counts as the present state determines how much of the past a model can afford to forget.

Imagine a service whose dashboard shows only two conditions: working and failed. A newly serviced machine and a badly worn machine both display green. The label is correct. The forecast made from that label may be wrong.

My public working paper Markov, Not Memoryless: A Clarification of the Blockchain Folk Theorem examines a related distinction in mining economics. A Markov model can carry history in its current state. Calling a process Markov therefore does not establish that its economically relevant history has disappeared. The paper's public abstract identifies the block tree, miner allocations and accumulated block counts among the state information relevant to that discussion.

The practical issue extends beyond the terminology. Once a researcher compresses a detailed state into a convenient summary, the resulting summary may cease to support the same predictions. A small invented example makes the information loss visible.

Two green lights, different next steps

Suppose a machine has three underlying conditions: fresh, worn and failed. For this illustration, a fresh machine has a 90% probability of remaining fresh over the next period and a 10% probability of becoming worn. A worn machine has an 80% probability of remaining worn and a 20% probability of failing. A failed machine stays failed until an intervention outside this example. These probabilities are assumptions, not measurements of a particular machine.

Fresh moves to worn with probability ten percent; worn moves to failed with probability twenty percent per period.FreshWornFailed10% →20% →90% stays80% stays100% stays
An illustrative three-state model. Each transition depends on the current detailed state.

The detailed model is Markov: once the current condition is known, an earlier maintenance diary adds nothing to the next-step probabilities specified here. Nevertheless, replacing fresh and worn with the single word “working” removes information. A fresh machine has zero probability of failing in the next step under these assumptions. A worn machine has a 20% probability. The same green light now represents two different forecasts.

The mixture changes while the label survives

Begin with every machine fresh. After one period, 90% remain fresh and 10% are worn. All still work. Their collective failure probability during the following period is 2%: one tenth are worn, and one fifth of those fail.

After the second period, the original population consists of 81% fresh, 17% worn and 2% failed. Among the machines still working, the worn share is 17 divided by 98. Their next-period failure probability is therefore approximately 3.47%. The dashboard still says “working”, but the composition behind that label has changed.

Conditional next-period failure risk among working machines rises from zero initially to two percent after one period and about three point four seven percent after two periods.Initially fresh0%After one period2%After two periods3.47%
Risk conditional on still working. The calculation follows the stated transition assumptions; it is not an empirical reliability estimate.

This example does not say that every reduced state fails. It identifies the test that a reduction must survive. For a single time-homogeneous “working” transition rule to apply across arbitrary starting mixtures, the detailed states placed in that category must agree about the probabilities of entering the retained categories. Here they plainly disagree about failure.

Keep the state needed by the decision

An engineer need not keep every scrap of history. Keeping the fresh/worn distinction is sufficient for this model. If that distinction cannot be observed directly, an estimate of the hidden mixture may be needed instead. Its uncertainty should remain visible rather than being converted into a fictitiously precise green light.

The same discipline applies when interpreting economic models. A compact description can be perfectly useful for one question and inadequate for another. Information needed to price a near-term interruption may differ from information needed to value a long-lived commitment. An elegant transition rule is no substitute for specifying what the state contains.

The research connection is precise: a Markov representation can retain consequential history through the present variables. The machine calculation adds a separate explanatory lesson about compression. Before treating two observations as the same state, ask whether they support the same next-step predictions and the same decision. If they do not, the convenience of the label has become a cost.