How to distinguish an economic simulator error from a business model error

One negative chart, three possible causes

Suppose an economic simulator shows insufficient funds to meet obligations. What has actually been found?

There are at least three possible explanations. The algorithm calculates incorrectly. It calculates correctly, but the original business rules are incompatible. Or the rules are consistent, but the calculation relies on unrealistic assumptions about users and revenue.

The result may look identical. The appropriate remedy is fundamentally different.

Cause 1: an implementation defect

Symptoms include incorrect debits or credits, counting an operation twice, incorrect event ordering, rounding errors, faulty constraints and dependence on an unintended state.

Useful diagnostic methods include unit tests, reference examples calculated by hand, invariant checks and independent reconciliation of individual operations.

For example, if a resource can only move between participants in a closed system, its total quantity must not increase spontaneously. Violating that invariant points to a calculation error or an incomplete description of issuance sources.

The remedy is technical: correct the defect, add a regression test and rerun the affected calculations.

Cause 2: conflicting business rules

The second situation is harder: each algorithm works correctly, but the combined rules prevent the system from meeting its financial goals.

For example, the system is obliged to make payouts under conditions where expected inflows do not cover them. Adjusting the program loop cannot eliminate that gap without changing assumptions, funding sources or the obligations themselves.

Here, the result must be traced back to specific requirements, the conflict demonstrated reproducibly, and alternative designs proposed.

It is essential not to call that change a “bug fix”. It is a new business decision.

Cause 3: unsupported assumptions

In the third situation, the model correctly implements consistent rules, but parameters have been chosen without sufficient evidence.

For example, projected revenue depends on an expected conversion rate not yet supported by actual statistics. Or user churn is assumed constant, even though it may vary substantially over time and across user groups in the real product.

One test cannot resolve these issues. They require data, uncertainty ranges, alternative scenarios and sensitivity analysis.

When actual data is unavailable, the correct conclusion is “the result is conditional”, not “the model has proved profitability”.

The diagnostic sequence

In practice, I begin with result provenance: which engine version was used, which configuration was loaded, which input data was applied, and whether the run can be reproduced.

Next, I check reference operations and invariants. If defects appear here, economic conclusions must wait until they are corrected.

I then compare implemented rules with the current specification. Deviations belong to implementation; contradictions within the specification belong to requirements.

Only then do I assess how realistic the assumptions are and how robust the conclusions remain when parameters change.

This sequence avoids spending time debating the business model before the calculation tool has passed the necessary checks.

Why independent verification matters

A simulator's author understands its internal logic well but may overlook their own assumptions. Independent verification treats a result as a claim that requires evidence.

It checks not only the final figures but also input data, versions, criteria, test artifacts and methodological limitations.

Even an independent technical PASS must not be interpreted beyond its scope. It confirms specific properties of the calculations and process, not the product's future commercial success.

An example from FIGHTEX

The A6.3-R series used engine 0.8.1 for 72 repeat runs. The technical status was PASS WITH ACCEPTED LIMITATIONS. The economic status was NO QUALIFIED CANDIDATES.

These results are compatible. The first describes the technical verification of the calculations under investigation, with stated limitations. The second reports that the tested options did not satisfy the established economic criteria.

This does not automatically mean that every possible economic design is unsuitable. Equally, the economic problem cannot be declared solved merely because the simulator works correctly.

How to write the final conclusion

A useful report separates four elements: what was checked, what was established, what remains unknown and what decision is proposed.

For example: “No qualifying options were found for the fixed parameters and mechanisms investigated; technical checks passed with accepted limitations; the impact of an alternative economic design has not yet been assessed.”

That wording is less dramatic than “the model works” or “the project loses money”, but substantially more precise.

Conclusion

An economic simulator is a measuring instrument. To trust its conclusions, verify the instrument, the rules it reproduces and the assumptions behind its calculations.

Only by separating code defects, requirements errors and unsupported hypotheses can you choose the right next step: fix the implementation, revise the rules or collect more data.

Frequently asked questions

How can you tell whether the simulator or the business model is wrong?

First check reproducibility, reference operations and invariants. Then compare the rules with the specification and assess assumptions and result sensitivity.

Does a technical PASS prove profitability?

No. It confirms only the technical properties checked, subject to stated limitations. Economic criteria and business hypotheses must be assessed separately.

What if there is no actual user data yet?

State the uncertainty, use alternative scenarios and perform sensitivity analysis. The financial result remains conditional.