The Agent Is Not the Control Plane (I/III)

A field note on auditing a professional FX options manuscript with Claude, Codex and Kimi. Published on orbaos.com.

What Claude, Codex and Kimi taught me while auditing the mathematics of an FX options book.

I recently gave three advanced AI systems a task that sounded straightforward:

Audit the formulas, calculations and worked examples in my forthcoming professional book on FX options.

This was not a writing competition.

I was not asking which model could produce the smoothest explanation of delta, simplify a difficult chapter or rewrite technical prose in a more confident voice. The book had already been written and typeset.

The question was whether its technical claims could survive scrutiny.

Could an agent establish that:

the formula was mathematically correct;
the stated inputs reproduced the published result;
units and market conventions remained consistent;
the accompanying table or figure represented the same calculation;
and the practitioner interpretation followed from what had actually been calculated?

That is a very different test.

It is also a practical illustration of the ideas behind OrbaOS and the Post-Project World.

The most interesting result was not simply that one model performed better than the others. It was that the task only began to work properly when I stopped treating the AI as a clever individual contributor and started treating the whole exercise as an operating system.

At the model level, Kimi is the winner so far.

At the system level, the winner is governed orchestration.

The book and the scale of the problem

The manuscript under review is:

FX Options and Volatility Trading: Pricing, Hedging and Volatility Strategies

It is Book 2 of the Foreign Exchange Markets Professional Series and is intended as a practitioner-focused reference covering institutional market structure, pricing, volatility, Greeks, vanilla and exotic strategies, hedging and risk management.

The current 7×10-inch pre-publication build is substantial.

Current manuscript audit surface:

Major parts — 5
Chapters — 30
Typeset pages — 764
Formula and equation objects — approximately 473
Worked examples — approximately 235
Tables — approximately 217
Figures and technical diagrams — approximately 152

These counts refer to the current pre-publication build and may change as findings are implemented and duplicate or damaged objects are consolidated.

More information about the book is available at rondanini.com/fx-options.

The scale explains why this could not be treated as conventional proofreading.

A proofreader can find an inconsistent heading, a missing word or an awkward page break. A technical auditor must determine whether hundreds of mathematical and financial objects agree with the prose, assumptions and interpretation surrounding them.

That is a systems problem.

A technical book is not a document

A professional finance book may appear to be a sequence of chapters, paragraphs and equations.

Operationally, it is something else.

It is a network of dependent representations.

A paragraph introduces a market convention. An equation formalises it. A worked example supplies numerical inputs. A calculation produces a result. A table presents the result. A figure illustrates its behaviour. A final paragraph explains the economic meaning.

The object being published is therefore not the prose alone. It is the alignment of the complete chain:

Text → convention → formula → inputs → calculation → table or figure → interpretation

A defect can occur at any transition.

The formula may be correct, but the worked example may use a different maturity.

The arithmetic may be correct, but the result may be expressed per unit volatility while the surrounding prose interprets it per one volatility point.

A payoff diagram may be correct at expiry but used to support a statement about pre-expiry mark-to-market value.

Domestic and foreign rates may be defined correctly at the beginning of a chapter and reversed in a later equation.

A sensitivity figure may have been produced from an earlier version of an example and remain visually plausible even though it no longer corresponds to the printed inputs.

Each component can look credible in isolation while the system as a whole is wrong.

This was the real audit target: not isolated correctness, but semantic and numerical alignment across representations.

Why “review this chapter” was the wrong instruction

My earlier prompts often asked a model to review one chapter at a time.

That instruction naturally encouraged editorial behaviour.

A model reads a difficult passage, infers what the author probably intended and produces a cleaner version. The result usually sounds better. It may even be technically better than the original.

But it is not necessarily an audit.

An auditor must resist rewriting until the original claim has been identified and tested.

It must preserve:

the exact statement being examined;
the formula under review;
the stated inputs;
the applicable convention;
the independent calculation;
the discrepancy;
the proposed correction;
and the other locations affected by the same issue.

A rewritten sentence can conceal the history of a defect.

A findings register preserves it.

A fluent explanation asks for confidence.

A reproducible calculation permits challenge.

That distinction shaped my experience with Claude, Codex and Kimi.

Claude: the strongest conceptual reader

Claude was often the strongest natural reader.

It understood what a section was trying to achieve. It could identify passages that felt incomplete, contradictory or poorly sequenced. It was particularly useful when material had been damaged through repeated conversions among Markdown, Word, PDF and LaTeX.

Its rewritten explanations were frequently excellent.

That was also the problem.

Claude naturally moved towards editorial resolution. It often inferred the intended meaning and repaired the prose before creating a complete evidential chain.

The chapter became cleaner, but polish is not proof.

In a technical finance book, an elegant correction can be more dangerous than obvious corruption. Once a wrong statement has been made fluent, readers have fewer reasons to question it.

The issue was not that Claude lacked financial understanding. Its default behaviour was simply closer to that of a strong technical editor than a forensic auditor.

It could often tell me what the section ought to say.

It was less consistent at proving precisely why the published version was wrong and how the error propagated through the example, figure and interpretation.

Claude remains an excellent explanation and editorial layer.

I would still use it for conceptual clarity, restructuring and prose refinement.

I would not now use it as the sole control mechanism for a whole-book mathematical audit.

Codex: the strongest implementer

Codex was better at operating on the machinery.

It could inspect LaTeX sources, write Python scripts, modify files, compile documents and apply precise local corrections. It understood the work as a file system and build pipeline rather than merely as a conversation.

That made it highly useful.

It also revealed a different weakness.

Codex could become absorbed in whether the source compiled or whether the calculation executed successfully.

But successful execution does not prove that the calculation corresponds to the prose.

A script may correctly calculate forward delta while the chapter claims to demonstrate spot delta.

A table may be reproduced perfectly from code while both the code and the table use a different maturity from the example.

A figure may be regenerated without error from assumptions that no longer appear in the text.

This is the difference between syntactic execution and semantic admissibility.

The fact that something can be calculated does not establish that it is the correct thing to calculate.

Codex was therefore valuable as an execution layer, particularly for deterministic checks and controlled source changes. But it required strong external governance to keep the task centred on the financial meaning of the book rather than on the mechanics of the repository.

Kimi: the strongest auditor so far

Kimi approached the exercise differently.

I gave it:

the complete typeset PDF;
the LaTeX source;
the figures;
previous chapter reviews;
the book’s technical conventions;
and a detailed audit protocol.

The protocol required Kimi to:

inspect every PDF page visually;
compare the rendering with the source;
identify every material formula and worked example;
reproduce calculations using Python;
distinguish confirmed errors from editorial preferences;
assign severity and confidence;
maintain a master findings register;
and write progress continuously to disk.

The difference appeared quickly.

For Chapter 2, Kimi produced a deterministic Python script containing 35 audit assertions. These checked premiums, breakevens, payoff values, percentage calculations and time-decay relationships.

A passing test did not always mean that the book was correct.

Some assertions were deliberately designed to prove that a suspected defect existed.

That distinction was recorded explicitly.

A mature audit process must distinguish among:

a printed value independently confirmed;
a suspected error disproved;
a defect successfully demonstrated;
and a result that remains unresolved.

Chapter 3 provided the more revealing test.

Kimi identified:

damaged Garman–Kohlhagen material;
duplicated explanatory prose;
dropped mathematical symbols;
malformed time notation;
a numerical paragraph apparently inserted into the wrong example;
a corrupted formula box;
and sensitivities in a figure that appeared mathematically impossible or incorrectly signed.

It did not merely state that the figure looked suspicious.

It attempted to reconstruct the sensitivities and tested whether the illustrated losses were possible under the stated assumptions.

It also corrected one of its own earlier findings after deterministic calculation contradicted its initial interpretation.

That is important.

A weak audit process searches for evidence supporting the agent’s first conclusion.

A stronger process allows the evidence to overturn it.

By the end of Chapter 3, the audit had produced 43 traceable findings, chapter-specific reports, reproducible calculation scripts and a precise continuation point for Chapter 4.

The wider audit then moved into Chapters 4, 5 and 6 and is intended to continue across the complete book.

The full exercise is not yet finished, so the verdict remains provisional.

But on the evidence generated so far, Kimi is the clear winner for this particular task.

The crashes revealed the architecture

Kimi also crashed repeatedly.

At first, that appeared to undermine the experiment.

The task involved hundreds of pages, extensive tool use and a context window approaching one million tokens. Connections disappeared. Sessions terminated. Work had to be resumed through the command line.

But the failures exposed the most valuable architectural lesson of the exercise:

The session is not the system.

Conventional AI interaction places too much importance on conversational continuity.

The model is expected to remember what it was doing, what it had discovered and what should happen next.

That is not a reliable operating model.

It is dependency on a temporary cognitive state.

The audit became resilient only when Kimi was instructed to write durable artefacts:

a master findings register;
individual chapter audit files;
Python calculation scripts;
calculation outputs;
extracted text for every PDF page;
rendered images of every page;
canonical technical conventions;
a coverage ledger;
and an audit_state.json file containing the exact recovery point.

When a session failed, the replacement session did not need to reconstruct the work from conversation history.

It read the state from disk and resumed from the first incomplete page, equation, worked example, table or figure.

The distinction is central:

The agent performed the work. The control plane preserved the work.

This is an OrbaOS principle in practice.

Autonomy without state, evidence and recovery is not operational autonomy.

It is merely extended improvisation.

Orchestra remembered why; the files remembered where

My Orchestra memory layer played a complementary role.

Orchestra preserved the wider project context:

why the book existed;
the professional standard it was expected to meet;
the technical conventions already established;
previous issues identified in the manuscript;
earlier failures by different models;
and the intended relationship with the wider Foreign Exchange Markets Professional Series.

The local audit workspace preserved execution state:

which pages had been inspected;
which findings had been saved;
which calculations had been completed;
which external claims still required verification;
and where the next session should resume.

These are different forms of memory.

Orchestra remembered why the work existed and what it was trying to achieve.

The filesystem remembered what had been completed and what remained.

A functioning machine-mediated organisation needs both.

Semantic memory without execution state produces intelligent repetition.

Execution state without semantic memory produces efficient work that may drift away from its purpose.

The control plane has to preserve the relationship between the two.

The Post-Project World in miniature

A traditional approach to auditing this book could involve:

a quantitative reviewer;
a senior FX-options practitioner;
a technical editor;
a fact-checker;
a proofreader;
a typesetter;
a project manager;
and the author.

The work would be coordinated through meetings, spreadsheets, tracked documents, marked-up PDFs and email chains.

It would be organised as a temporary project.

The AI-based approach did not eliminate those responsibilities.

It redistributed them across agents, deterministic tools, persistent memory and explicit operating rules.

Claude contributed interpretation.

Codex contributed execution.

Kimi contributed audit orchestration.

Python provided deterministic calculation.

Rendered pages provided visual evidence.

The findings register provided traceability.

Orchestra preserved wider context.

The author retained adjudication authority.

This is the Post-Project World in practical form.

It does not mean that work becomes unstructured or that projects simply disappear.

It means coordination migrates from temporary human arrangements into persistent operational infrastructure.

The project plan becomes a state model.

The status meeting becomes a coverage ledger.

The handover document becomes a machine-readable checkpoint.

The quality gate becomes an executable test.

The audit trail becomes a native output of the work rather than something assembled afterwards.

Post-project does not mean post-discipline.

It means discipline is encoded closer to execution.

The model explains; tools calculate

One principle became non-negotiable during the audit:

The model explains. Tools calculate.

A language model can identify the relevant financial convention, interpret the result and explain why a discrepancy matters.

It should not be trusted to perform material financial arithmetic from conversational intuition.

The calculation must be reproduced by deterministic software with visible:

inputs;
formulas;
assumptions;
intermediate values;
tolerances;
outputs;
and pass-or-fail conditions.

This division of labour is not a concession to model weakness.

It is governance.

The model is strongest where interpretation is required.

The deterministic tool is strongest where reproducibility is required.

The control plane decides which form of output is admissible for each part of the task.

That matters particularly in derivatives because a number can be plausible and still be economically wrong.

The calculation engine must therefore sit inside a larger system that also tests:

quotation direction;
domestic and foreign currency definitions;
discounting;
volatility units;
delta conventions;
premium adjustment;
position direction;
expiry versus pre-expiry value;
and the interpretation of the result.

Determinism does not remove judgement.

It makes judgement inspectable.

The real benchmark was alignment

Most AI benchmarks isolate capabilities.

They ask whether a model can solve an equation, retrieve a fact, write code or explain a concept.

The book audit required something more operationally realistic: maintaining alignment across different forms of representation.

Could the model recognise that the prose described one delta convention while the equation used another?

Could it detect that a figure and worked example came from different assumptions?

Could it distinguish an expiry payoff from a pre-expiry valuation?

Could it reproduce a number and then verify that the accompanying paragraph interpreted the sign correctly?

Could it identify that a vega value was calculated per unit volatility while the prose described a one-percentage-point move?

Could it trace a notation defect across multiple chapters rather than treating each occurrence as an isolated typo?

This is not merely mathematical reasoning.

It is coordination across representations.

That is why the quality of the result depended as much on operating architecture as on model intelligence.

Who won?

For conceptual reading and prose, Claude was excellent.

For file operations, implementation and calculation infrastructure, Codex was highly capable.

For the combined audit of prose, formulas, numerical examples, figures and rendered pages, Kimi performed best.

At the model level:

Kimi is the winner so far.

But this is not a universal benchmark.

It does not establish that Kimi is superior for every form of reasoning, writing or software work.

It establishes that, in this particular experiment, Kimi came closest to behaving like a technical audit agent rather than a writer or programmer.

More importantly, Kimi did not succeed because it possessed some magical form of mathematical certainty.

It succeeded because it was placed inside a better-governed process.

The strongest configuration was:

Kimi + deterministic calculations + rendered-page inspection + persistent state + structured findings + human adjudication

Remove those controls and the output could easily become another persuasive conversation rather than a reliable audit.

The model can change.

The evidence must survive it.

The governance lesson

The common question is whether an AI model can be trusted.

That is the wrong level of analysis.

A more useful question is:

What claims is this model permitted to make, what evidence must accompany them, and what system ensures that the work remains recoverable and reviewable?

Trust should not be assigned to a model as a general property.

It should emerge from an operating architecture.

For this audit:

narrative conclusions required traceable findings;
numerical conclusions required deterministic reproduction;
visual conclusions required page inspection;
factual claims required authoritative sources;
interrupted work required durable checkpoints;
disputed findings required confidence levels;
and material corrections remained subject to human adjudication.

This is governance as infrastructure.

It does not prevent the agent from acting.

It is what makes meaningful autonomy possible.

The lesson for OrbaOS

The most useful discovery was not simply that Kimi outperformed Claude and Codex in this particular exercise.

It was that no model should be asked to carry complete responsibility for a technical process inside its context window.

A model is a participant in the system.

It is not the system.

Claude helped interpret.

Codex helped execute.

Kimi helped audit.

Python verified the calculations.

The rendered PDF supplied visual evidence.

The files preserved execution state.

Orchestra preserved intent.

The control plane held the components together.

That is the OrbaOS lesson from this experiment:

Intelligence produces possibilities. Governance turns them into admissible work.

In a technical publication, admissibility means far more than a formula that looks correct.

It means that the text, market convention, formula, assumptions, calculation, result, figure and practitioner interpretation all describe the same financial reality.

That is what the audit is testing.

And that—not the fluency of the model—is where trust begins.