Ten caveats in every audit: what the engine writes about its own numbers

2026-09-04. Every passage quoted below is stored in the audit file itself, and every number comes out of the same files.

An audit that prints only numbers is selling a certainty it does not have. Ours prints ten passages beside them, written by the engine, that state what each block of numbers can and cannot support. Seven sit in the Monte Carlo section under the heading What these numbers can and cannot say. Three sit in the withdrawal replay under the heading Assumptions and their direction. They are in every one of the 33 audit files that hold a measured run, and they are the part of the page nobody reads.

Ten passages, two boxes

The seven Monte Carlo passages cover the section as a whole and then each method inside it, the permutation, the bootstrap, the block bootstrap, the ruin sweep, the fan chart and the prop challenge simulation. The three replay passages cover the monthly withdrawal, the balance scaled sizing mode and the capital what if table. In the audit file they sit at mc.honesty and replay.honesty, and the page renders them at the foot of the section they belong to, not in a notes appendix at the end.

The file count is exact rather than approximate. Our audits folder holds 38 files, 33 of them carry both blocks and 5 do not, and those 5 are the comparison files that hold the difference between two runs instead of a run of their own.

Four of them name the direction of their own error

A caveat that says a number is uncertain is close to useless, because every number is. Four of the ten say which way their number is wrong, which is the version a reader can act on.

The ruin passage says that firm rule sets for daily loss, consistency and position loss are stricter than the single floor we model, so treat every probability as a lower bound, and the safe sizing answer as an upper bound on the truly safe size. The challenge passage says its breach checks remain a lower bound on breaches even where the audit reconstructed intraday equity. The balance scaled passage says min lot granularity and balance coupled EA defaults do not scale linearly, and that the direction is optimistic for small accounts. The capital what if passage says margin requirements are not modeled, that a small account may be unable to hold the positions at all, and that the direction is optimistic.

Two of those four point at the same reader, the one with a small account, and that is where the passages stop being decoration.

Lizard reads two ways inside one file

Lizard is the clearest case in the catalog. Its audited run ends 183.99 USD ahead on our 100,000 USD frame, after 1,721.92 USD of retrofitted commission against 1,905.91 USD gross, for a final balance of 100,183.99 USD. On that frame the ruin sweep reports a floor breach probability of 0.0 against the six percent trailing profile at all seven tested position sizes, from full size down to one eighth, and it names full size as the smallest size that already meets the ruin target.

The same file re-expresses the same trade path against smaller capital, because the run capital is ours and not the vendor's advice. The deposit the vendor recommends for Lizard is 250 USD, recorded in the audit as the evaluation deposit with the source vendor recommended deposit. Against 250 USD the maximum drawdown of that identical path is 1,018.32 percent and the floor breach probability against the same profile is 1.0, with the observed path breaching as well. Against 1,000 USD it is 254.58 percent and still 1.0.

Neither reading is wrong and the caveat is what joins them. A ruin probability of 0.0 is a statement about a 100,000 USD account and carries no promise about any other. Then the what if passage adds the part that matters more than either number. Margin is not modeled, so 1,018.32 percent is itself the optimistic version. An account of 250 USD could not have held the positions that produced it, so the figure is not a loss anyone would have taken. It is proof that the question does not survive the change of capital.

The same words in every audit

The limit of these passages is that they are constants. In the engine both blocks are module level dictionaries, one in the Monte Carlo module and one in the replay module, and the code assigns each one unchanged into the audit it builds. Exactly one value inside them varies, the block length of five trading days, and that is itself a fixed constant in the same module rather than a fitted parameter.

So the passages describe the method and never the EA in front of you. The audit that needs the capital what if warning most, Lizard at 1,018.32 percent, receives the same sentence as an audit whose recommended deposit and tested capital are the same number. Reading them as a judgement about this product is the one mistake they invite, and the reason we say so here.

Three limits stay outside the lock

The free version of every audit page keeps a section called What this audit cannot tell you, and it names three fundamental limits, in sample bias, regime dependence and the simulation gap. It opens by saying that a good backtest cannot prove an edge, which is the least commercial sentence on the site and the truest one.

The ten passages above are not in that free part. They live inside the Monte Carlo and replay sections, and the free version replaces those sections with a question and a lock. The reason is not that the caveats are the product. It is that a caveat without the number it qualifies is a slogan, and the number is what the page sells. A reader who wants the assumptions is welcome to the three fundamental ones for nothing.

How to read them in thirty seconds

Two questions get the value out of both boxes. The first is whether the passage names a direction, because a bound is worth more than a warning and four of the ten give you one. The second is whether the number it qualifies sits near a decision you are actually making, and the capital what if is usually that number, because the capital you would trade is almost never the capital we tested.