BSV decentralisation series · Capacity, recovery & responsibility
The capacity left
after a failure.
A service can survive a failure and still leave its customers waiting. For BSV, the useful scaling question is not only how much work a system completes when everything is healthy. It is how much capacity remains when something breaks, and who takes responsibility for the work already accepted.
Imagine a BSV service selling transaction processing to a business with a busy trading day. Its demonstration is impressive. The machines remain responsive, the throughput counter rises, and adding workers improves the result. Then one dependency slows down. Requests still enter the system, but completion falls behind. The service is technically running. The customer's work is accumulating somewhere inside it.
This is an invented situation, not an incident report about a named operator. It exposes a distinction that a peak-rate demonstration cannot settle. A machine returning to service is one event. Clearing the work that accumulated during the interruption is another. Restoring the customer's ability to make decisions can take longer still.
The previous articles considered implementation disagreements and the scope of payment evidence. Here I want to examine the capacity behind a service promise. Independent implementations matter because they allow different engineering answers. Those answers become commercially meaningful when their failure and recovery behaviour can be compared without requiring every supplier to adopt the same internals.
1. Fixed rules do not prescribe one machine
The BSV protocol's policy-versus-consensus notes distinguish transaction and block validity from an operator's relay, admission and mining choices. They also describe component boundaries as an implementation choice rather than a protocol requirement. That distinction leaves room for different operating designs without making each design decision a change to the base rules.
I favour a fixed foundation on which businesses can invest and compete. A supplier should be able to improve its scheduling, storage layout or deployment without asking the rest of the market to adopt its preferred software. Equally, a supplier's difficulty handling a workload does not establish that the workload violates the protocol. Capacity limits belong in the account of the service being offered.
There is no contradiction between a common validity contract and competing implementations. The common contract specifies what their results must respect. It does not guarantee equal cost, equal speed, identical availability or interchangeable commercial terms.
The practical question is therefore not whether every supplier uses the same architecture. It is whether each can state its obligations precisely enough for a customer, another implementer or an independent evaluator to check them.
2. Count the right kind of independence
The Teranode architecture documentation describes work distributed across machines and separate services for functions including propagation, transaction validation, block assembly and block validation. This is an architectural account, not a performance result that I have reproduced here. It illustrates why a system's useful capacity cannot be inferred merely by counting its processes.
Ten workers may depend on one storage service. Several separately managed providers may depend on the same upstream data source. Independently written applications may share a library whose behaviour determines the outcome that matters. Those arrangements can be sensible. They simply provide different kinds of independence.
I would distinguish at least three questions. Are the businesses free to choose and change suppliers? Are the implementations independently capable of finding a defect? Are their operating failures sufficiently separate that one failure does not remove every alternative at once? An affirmative answer to one is not an answer to the others.
A distributed deployment inside one operator's business is not, by itself, evidence of competition between independently operated businesses. Conversely, a business with a tightly managed internal deployment can participate in a competitive market. Organisational independence, implementation diversity and operational resilience should be described separately rather than collected under a reassuring count of nodes.
This matters particularly during an incident. A fallback provider is useful only to the extent that it can perform the required work under the circumstances that disabled the first one. A different logo is not a recovery mechanism.
3. The minute after recovery
Consider a deliberately simple fluid model of a BSV processing service. The workload is homogeneous, the queue begins empty, and new work arrives steadily at 9,000 processing jobs per second. Healthy service capacity is 10,000 jobs per second. For 60 seconds, a failure reduces capacity to 8,000. The server uses its full available capacity whenever work is waiting. All arriving work is retained, nothing expires, and there is no retry traffic, prioritisation, startup delay or other bottleneck.
The backlog grows at 1,000 jobs per second. After the minute of reduced capacity, it contains 60,000 jobs. Restoring capacity to 10,000 does not remove that backlog immediately, because 9,000 new jobs still arrive each second. Only the spare 1,000 jobs per second can clear the accumulated work. Under these assumptions, draining the queue takes another 60 seconds.
- Accumulated work: (9,000 − 8,000) × 60 = 60,000 jobs.
- Capacity available for recovery: 10,000 − 9,000 = 1,000 jobs per second.
- Time to drain: 60,000 ÷ 1,000 = 60 seconds after capacity returns.
If restored capacity merely equals the continuing arrival rate, this model never clears the backlog. If it remains below that rate, the backlog continues to grow. A green process-health indicator can therefore coexist with a service that has not recovered its original operating position.
These are invented rates and elementary queue arithmetic, not a BSV network benchmark, a measurement of Teranode, or a prediction of confirmation time. A processing job is the unit of work chosen for this illustration; it is not a claim that completing that job mines or settles a transaction. Real workloads have differing costs, dependencies and bursts. Those complications require measurement rather than turning the illustration into a universal capacity rule.
The important procurement question is the spare capacity available under continuing demand. A demonstration that stops the arrivals before measuring recovery answers a different question from the one faced by a service whose customers keep working.
4. Restarting is not reconciliation
Teranode's Kafka documentation separately discusses reliability, recovery, consumer lag, processing time, retries and queue size. Those categories are useful precisely because a message-processing system has more than one observable stage. Documentation and configuration are evidence about the intended design; neither substitutes for exercising the exact deployed version under a specified failure.
For a proposed test, I would keep a ledger of unique submitted work and its observed state. Distinguish an initial attempt from a repeated request for the same work. Distinguish a service's acknowledgement from completion of the processing stage being purchased. Identify the observation deadline, and retain cases whose outcome remains unknown.
A client timeout is especially instructive. It may mean the service did no work. It may mean that the work completed but its reply was lost. Repeating the request without a defined reconciliation rule can create additional load and confusing records. A useful service contract explains how a customer queries uncertain work and what the service does with duplicates. It does not need every operator to use the same internal queue.
Nor does an empty queue automatically prove recovery. Work can leave a queue because it completed, was refused, expired, was moved elsewhere or was lost. The test should account for those dispositions. Where the service promises retained processing, reconcile the accepted population against completed work, explicit failures and work still pending.
These are proposed evaluation requirements, not a claim that a particular implementation lacks them. They make a recovery claim inspectable. Without them, a provider can appear to recover quickly simply by ceasing to count the difficult work.
5. Standardise the evidence, not the architecture
A voluntary reporting format could help customers compare these outcomes. It could identify the workload, software revision, configuration, failure interval, observation horizon, completion definition and unresolved population. A provider could publish enough non-sensitive material for another team to repeat the comparison while withholding customer information and credentials.
This is a proposal for evaluation practice, not a new BRC or a compulsory BSV interface. The BRC repository guidance offers a place to discuss technical proposals and describes its standards as vendor-neutral. Its guidance also separates registry metadata from permission to use a protocol. A shared vocabulary can support cooperation without assigning one registry or reference implementation authority over every commercial service.
The standard worth adopting would define what a result means. It would not require an evaluator to approve a vendor's choice of database, language or internal message transport. If two architectures fulfil the stated contract, a customer can compare their cost, recovery behaviour and support rather than treating architectural conformity as the product.
Independence also applies to the evaluation itself. A common workload makes results comparable, but a single workload can miss a shared weakness. I would retain the common cases, then allow independent teams to contribute additional distributions, dependency patterns and failure scenarios. Publish the reason for each test and the limits of what passing it establishes.
For BSV services, the comparison should keep base-rule compliance distinct from local operating policy. A refusal caused by a provider's configured capacity limit is not evidence of invalidity. A completed processing stage is not a promise about every later network event. Correctly naming the boundary is part of the engineering.
6. Sell a recovery promise someone can check
Spare capacity costs money. So do retained records, independent verification and incident response. Customers may reasonably purchase different levels of protection. A business running a small periodic workload need not buy the same recovery arrangement as one whose operations depend on a continuous flow.
What should not be negotiable is whether the promise describes the service actually supplied. If a provider advertises recovery under continuing demand, it should state the demand and failure conditions under which that claim was established. If its fallback shares a dependency with the primary route, that qualification belongs in the explanation. If it has not tested a scenario, it should say so.
A customer changing providers also needs a controlled way to reconcile work still in flight. Exporting a list is not enough if neither side can distinguish completed requests from uncertain ones. I would test a planned handover using synthetic work before relying on it during an incident. That is an operational proposal, not an assertion that any current service can be exchanged without preparation.
Responsibility remains with the business making the promise. It may use excellent open-source components and still configure or operate them poorly. It may build independent software and still make a shared mistake. Naming a dependency helps explain an incident; it does not answer the customer's question about what happens to the work already accepted.
BSV decentralisation should leave room for different implementations, voluntary standards and competing commercial judgement on a dependable base. That freedom is strengthened, not weakened, by precise accounts of capacity and failure. The supplier is free to build differently. The customer is free to demand evidence that the difference works.
The next useful demonstration is therefore not only a higher counter. Keep the arrivals running, remove a stated part of the capacity, restore it, and account for the work until the observation ends. Show what recovered, what did not and who is responsible for the remainder. The capacity left after a failure is part of the service being sold.