Did the payment go through? Handling uncertain outcomes with Ballarch

Picture a checkout. The app sends the order, the server creates it and charges the card, and then the response gets lost on the way back.

The HTTP client gives up after its timeout. The repository turns that into a Failure, and the screen shows "Payment failed." So the user taps Pay again.

Each layer behaved correctly by its own rules. The client really did time out, and the repository really did get an error. Somewhere along the way, though, "I stopped waiting" turned into "the payment didn't happen", and nobody in the chain actually knows that.

I built Ballarch for this situation: the app made a call that changes something on another system, and now it can't tell whether the change happened. In this post I'll go through the outcome model, a Kotlin checkout example, and the harder case where a multi-step workflow only partly succeeds.

TL;DR: a call with side effects ends in one of three states: Done, NotDone (only when there's evidence that nothing happened), or Unknown. The Ballarch runtime handles retries, idempotency keys, crash recovery and reconciliation in one place, and feature code has to deal with Unknown explicitly.

Everything here is about Ballarch Core 2.0.0-rc.1, a release candidate that's open for public review. The Kotlin code is a JVM reference implementation. Limitations and costs are listed near the end.

1. What the app actually knows

Here's the whole sequence:

1. The app sends POST /orders.
2. The server creates the order and charges the card.
3. The response is lost, or it arrives after the client has stopped waiting.
4. The app tells the user that the payment failed.
5. The user tries again. The new request has no idempotency key, or a freshly generated one, so the server creates a second order.

After step 3, the only thing the client knows for sure is that it stopped waiting. It has no evidence that the server did nothing.

Getting this right takes several pieces that have to work together:

• an idempotency key that's created before the request goes out and survives an app crash;
• retries that reuse that key;
• a known limit on how long the server remembers the key;
• a way to look up the operation by its key;
• a UI state for "we don't know yet".

None of these is new. Ballarch puts all of them under one owner and adds a contract you can test against.

2. Three outcomes

In Ballarch, every external call ends with exactly one of three outcomes:

Done only says that the request was processed. The value can still carry a business answer, such as a declined card, and the app has to interpret it. NotDone needs real evidence. A timeout after the request may have left usually gives you neither, which is what Unknown is for.

A third outcome is an old idea. RPC has "maybe" semantics, Jepsen records :info operations, MongoDB has UnknownTransactionCommitResult, and XA has heuristic outcomes. Ballarch turns it into a type and spells out what evidence each of the three outcomes requires.

What happens if a request arrives twice

A port is how a Ball (more on those below) talks to an external system. Each port has an effect class. The application declares it, because it describes how the destination behaves, and only you know that.

A Safe call can't end Unknown. If it gets no answer, it ends NotDone(NoAnswer).

Idempotent here is narrower than the HTTP term. It specifically means deduplication by key. A "set value" request can still be NonIdempotent in Ballarch, because a late delivery might overwrite a newer value.

If you don't know how the destination behaves, declare NonIdempotent.

Two moments that matter

For every call, the runtime tracks two specific points in time:

1. Commit horizon. The decision and the calls it requests are saved together, atomically. Nothing is sent before this point. From here on, each call is an obligation that will get exactly one outcome. Under the Durable profile, where the runtime persists its state, that obligation survives a crash.
2. Send horizon. The first moment when the request might have left the runtime. Before it, cancelling the call gives a clean NotDone. After it, a call with side effects that never got an answer may end Unknown.

For a NonIdempotent call under Durable, the runtime writes down "possibly sent" before it dispatches the request. If the app crashes, the runtime keeps that uncertainty after the restart and doesn't send the call again.

Adapters only report what happened at the transport level. They don't retry and they don't generate idempotency keys. Deciding which outcome a call gets is the runtime's job.

3. A checkout as one Ball

A Ball owns a piece of state and is the only thing that changes it. Its decider is a pure function: it takes the current state and an input and returns the next state plus the calls it wants to make. The runtime saves the result and does the actual I/O.

The checkout example assumes three things about the orders server:

• it deduplicates requests by Idempotency-Key for 24 hours;
• it can find an order by that key;
• it won't apply a request older than the declared delivery horizon.

The server has to keep these promises, and it's on the application to make sure it does. Ballarch can't check them.

The ports and the "we don't know" state

The code below comes from PlaceOrder.kt, with lines wrapped to fit.

data class Cart(val items: List<String>) data class Order(val id: String) data class Place(val cart: Cart) val Orders = AdapterPort<Cart, Order>( "orders.place", EffectClass.Idempotent ) val OrderLookup = AdapterPort<String, Order?>( "orders.lookup", EffectClass.Safe ) sealed interface PlaceOrderState { data object Idle : PlaceOrderState data class Placing( val cart: Cart, val intent: Intent, val call: CallRef<Order>, val repeat: Boolean ) : PlaceOrderState data class Placed(val orderId: String) : PlaceOrderState data class NotPlaced(val reason: String) : PlaceOrderState data class Checking( val cart: Cart, val intent: Intent ) : PlaceOrderState } 

Checking records one specific fact: the order might exist. It also keeps the intent, which identifies the business action itself, so a later call can repeat it safely.

There are two kinds of identity here. A call reference identifies a single call, and an intent identifies the effect across several calls. All attempts of one call use the same key, and different calls with the same intent share that key too.

Save the request before sending it

Here's the part of the Ball that handles requests. Imports and the completion handler are in the linked source.

object PlaceOrder : Ball<PlaceOrderState, Place, PlaceOrderState> { override val type = "place-order" override fun initial(key: String): PlaceOrderState = Idle override fun decide( state: PlaceOrderState, request: Place, ctx: RequestContext ): Decision<PlaceOrderState, PlaceOrderState> = when (state) { Idle, is NotPlaced -> place(request.cart, ctx.newIntent(), repeat = false, ctx) is Checking -> if (request.cart == state.cart) place(state.cart, state.intent, repeat = true, ctx) else reject(state) is Placing -> if (request.cart == state.cart) accept(state, state) else reject(state) is Placed -> accept(state, state) } private fun place( cart: Cart, intent: Intent, repeat: Boolean, ctx: Context ): Decision<PlaceOrderState, PlaceOrderState> { val (call, send) = ctx.call( Orders, cart, timeoutMillis = 60_000, intent = intent ) val placing = Placing(cart, intent, call, repeat) return accept(placing, placing, send) } // complete(...) is shown below. } 

ctx.call doesn't touch the network. It returns a call reference and an output that describes the call. The runtime commits placing and send together, and only then sends the request.

Most of the behavior lives in the state branches:

• First submit: create a new intent.
• Double tap while placing: return the current state and send nothing.
• Try again from Checking: reuse the original intent and cart.
• Different cart while the result is unknown: reject the request, so the old intent never gets attached to a different order.

Handle every outcome

This method completes the Ball:

override fun complete( state: PlaceOrderState, completion: Completion<*>, ctx: Context ): Step<PlaceOrderState, PlaceOrderState> { if (state !is Placing) return step(state) return when (val outcome = completion.outcomeFor(state.call)) { is Outcome.Done -> step(Placed(outcome.value.id)) is Outcome.NotDone -> step( if (state.repeat) Checking(state.cart, state.intent) else NotPlaced("${outcome.reason}") ) is Outcome.Unknown -> step(Checking(state.cart, state.intent)) null -> step(state) // Completion of a different call. } } 

Outcome is sealed, so the when has to be exhaustive. Delete the Unknown branch and the code stops compiling.

The tricky case is NotDone on a repeat. It proves that this call didn't place the order. An earlier call ended Unknown and might have placed it, so the state goes back to Checking.

With a generic Result.Success / Result.Failure, this detail is easy to lose.

4. Let the runtime handle the plumbing

The composition root connects the Ball to real adapters and sets the retry and reconciliation policy:

const val ORDERS_HORIZON_MILLIS: Long = 30_000 fun Composition.placeOrder( orders: Adapter<Cart, Order>, lookup: Adapter<String, Order?> ): Composition = ball(PlaceOrder) .adapter( Orders, orders, RetryPolicy( maxAttempts = 3, backoffMillis = 1_000, attemptTimeoutMillis = 10_000 ), dedupWindowMillis = 24 * 60 * 60 * 1_000L - ORDERS_HORIZON_MILLIS, reconcile = Reconciliation( status = OrderLookup, horizonMillis = ORDERS_HORIZON_MILLIS, statusTimeoutMillis = 5_000, query = { it.idempotencyKey }, verdict = { order -> if (order != null) StatusVerdict.Applied(order) else StatusVerdict.NotApplied } ) ) .adapter(OrderLookup, lookup) 

Retries keep the same key

With this setup, the runtime makes up to three attempts. Each one has a 10-second timeout and uses the same key. If a late answer arrives before the call has settled, it can still produce Done.

A NonIdempotent port can't have a retry policy or an attempt timeout. If you set one, the composition fails at assembly time.

Idempotency keys expire

An idempotency key only protects you while the server still remembers it.

Here the declared window is 24 hours minus the 30-second delivery horizon. Before sending, the runtime checks whether the call could finish after that window closes, counting from the moment the intent was created. If it could, the runtime refuses the call and sends nothing.

So if the user taps "Try again" a week later, the app can't create a second order on a server that has already forgotten the first key.

Reconciliation needs a delivery deadline

Once all attempts have gone unanswered, the runtime switches to reconciliation. It waits until the delivery horizon has passed since the last send, then looks the order up by key:

• order found → Done(order);
• the server says for certain that it wasn't applied → NotDone(NotApplied);
• nothing conclusive → Unknown.

The delivery horizon is the age after which the server will never apply a request. Without that limit, the lookup could report "no such order" right before a delayed request arrives and creates one.

The runtime can't verify the horizon and takes your declaration on trust. If the server doesn't keep it, the reconciliation result can be wrong.

Crash recovery needs a single owner

Under Durable, the saved state includes each call and its current phase. After a restart, a call keeps its identity and resumes according to its effect class and the attempts it has left.

Recovery also assumes that only one runtime owns the store. The reference FileStore uses a lock that works on a single host. Running on several servers needs a store that fences writes, and the reference implementation doesn't include one.

5. Show the user what you actually know

The screen reads the last committed state through runtime.read or watch. Reads don't create commits and don't go through the Ball's input queue.

In this checkout, each state maps straight to a message:

When Placing repeats an earlier intent, it can say "Checking your order…".

If the destination is idempotent, Try again reuses the same intent and stays available only while the deduplication window is open. After that, offer a way to check the order status or order history.

A non-idempotent payment gateway needs a different action in the unknown state:

We're checking your payment. Please don't pay again.

Show a retry button only when you have evidence that the original payment didn't go through. The same goes for styling: a red color or an error icon on an Unknown state tells the user that something failed, and the state doesn't support that.

On the backend, an operation with an unknown result can be returned as 202 Accepted with a status resource. The API should let the caller keep the operation's identity and check its result later.

6. Workflows need more than success and failure

Now take a trip booking with two steps: a flight and a hotel.

The flight call ends Unknown. The hotel call ends NotDone. What can the workflow say about the trip?

The trip is definitely incomplete. A flight might still be booked.

Ballarch describes what's known about a multi-step operation as a pair (all, any) in strong Kleene three-valued logic:

• all: did every effect happen?
• any: did at least one effect happen?

Each value is Yes, No, or Maybe. That leaves exactly seven possible pairs:

The trip above is NotAllDone: at least one required step didn't happen, and we don't know whether anything happened at all. Partial means some effects happened and others didn't. SomeDone means something definitely happened, and we don't know whether everything did.

NoEffects is for an operation that requested no effects at all. "All" is vacuously true, and "any" is false.

Keeping the answer honest while the workflow runs

Three rules keep these values honest:

1. Any effect that can still happen, including a compensation, counts as Maybe. Before the hotel request goes out, the workflow can't claim as a final answer that no hotel will be booked. A status read in the middle of the workflow has to be labeled as a snapshot of the current state.
2. A compensation counts as undoing an effect only when it's absorbing. That means the destination fences off the original request, or the original request's delivery horizon has passed. Otherwise a delayed booking could arrive after its cancellation.
3. A settled workflow returns its knowledge value. A workflow is settled when none of its calls are open and it won't make any more. Under Durable and the composition rules, the caller can treat it as a single honest call and combine it with other settled workflows.

The three-valued logic itself is classical. What I'm proposing is to apply it to settled, nested operations with explicit uncertainty and absorbing compensations.

You can always collapse Partial, NotAllDone and SomeDone into Unknown and stay honest. You just lose information the caller might need.

7. What the tests show

The reference tests go through failure scenarios systematically and compare what the code reports with what the simulated servers actually did. The numbers below are asserted in the repository's tests. They come from bounded simulations, and there are no production measurements behind them.

Checkout. PlaceOrderTest covers lost requests, lost responses, refused connections and a crash. It checks that an order is never placed twice and that NotPlaced appears only when no order exists. A retry after three lost responses keeps the original key, and a retry a week later is refused before anything is sent.

One trip. TripFlowTest goes through 6,438 runs with different call outcomes and crash points. In none of them does a status or a settled claim contradict the actual bookings. A simple two-state "booked / nothing reserved" status would be wrong in 1,190 of those runs.

Nested trips. ItineraryFlowTest combines two trip Flows. It finds no false claims in 130,321 runs without a crash, or in 29,774 runs with faults and a crash point. The crash actually happens in 28,933 of those runs.

The repository also has algebra checks, runtime trace oracles, C firmware tests, SystemVerilog testbenches, and a bounded Alloy model with seeded bugs. Core Appendix F has the full evidence, and the README has the commands to reproduce it.

These checks show that the contracts hold within their models and bounds. They say nothing about productivity or about bug rates in production.

8. When to use it and what it costs

Use Ballarch at the points where the result of an effect can become uncertain:

• No calls with side effects: keep your usual code.
• One effect: use one Ball with explicit stages.
• Several effects: use a Flow, which is a Ball whose state is the whole multi-step operation.

Screens, ViewModels, navigation, caches, paging, pure mappers, Safe reads, and CRUD that fits in one local database transaction can stay as they are. You can move one effectful use case at a time into a Ball and keep the existing read path.

Clean Architecture, hexagonal design, MVI, actors and durable execution systems already give you useful pieces: dependency boundaries, ports, pure reducers, ownership and durable progress. A careful implementation in any of these styles can handle the checkout scenario correctly. Ballarch makes the outcome rules part of a shared runtime contract with conformance tests. I haven't measured whether that trade-off pays off for a team.

Costs

• Under Durable, an effectful call that ends Done costs three durable writes, plus one per retry and one when it enters reconciliation.
• Each key has a single writer, so a hot key turns into a queue.
• Every step needs an explicit stage, and under Durable that stage has to be serializable.
• You host the runtime and the store yourself. The reference has no operations UI and no worker scaling.
• Invariants that span several Balls need a coordinating Ball, a reservation, or a database constraint.

Limitations

Nothing checks your declarations mechanically: effect classes, deduplication windows, delivery horizons, absorbing compensations and ownership boundaries. The runtime relies on them being true.

The Kotlin reference runs only on the JVM and isn't published as a library. It has no Android module and no mobile store. The text and diagrams are licensed under CC BY 4.0. The code has no software license yet.

The C and RTL references check the outcome and knowledge rules at their own levels, and no reference connects those levels end to end. Driver integration is only a sketch. Applying these rules to AI agent tools doesn't make an agent secure on its own, and the agent skill that ships with the repo hasn't been evaluated.

Ballarch is a poor fit for stateless code, CRUD inside one transaction, consensus, and loops that can't afford a commit for every input.

What I'd like feedback on

Most of the building blocks are well known: single owners, pure deciders, commit before dispatch, transactional outboxes, idempotency keys and three-way outcomes.

What Ballarch proposes is connecting them through an executable outcome contract, together with a way to combine what's known about multi-step effects.

The release candidate is open for review. The most useful feedback is a concrete counterexample: a failure sequence where the rules claim more than they know, an assumption in the examples that doesn't hold, or prior work that already does the same thing.

And a question for you: when a response never comes back, what can your system prove about what happened?

Code and specification

• Ballarch repository and reproduction commands
• Core 2.0.0-rc.1 specification
• Tutorial: place an order once
• Structuring an application
• Honest UI guidelines
• Comparison with Clean Architecture

The links point to the feature/architecture-readiness-user-identity branch, which is the version under review. Up to Core 1.5, the project was called Pokeball.