Model validation

How to spot fake model results and prediction claims

Audit high hit rates, profit curves and “live AI” claims with an evidence checklist.

Updated 2026-09-20

This guide uses testable rule assumptions and does not infer any platform’s implementation from its name. It is educational material, not betting advice.

Key definition

Ask whether raw data can be downloaded, predictions were preregistered and failed rounds remain in the record.

How to verify it

A smooth profit curve cannot be checked without round-level details, costs and a defined risk measure.

Limits and conclusion

“95% accuracy” is incomplete without task difficulty, baseline, sample size and confidence interval.

Practical checklist

Promises to always win, recover losses or know the next round are warning signs. Credible reports disclose limits and uncertainty.

Practical checklist

  • Ask whether raw data can be downloaded, predictions were preregistered and failed rounds remain in the record.
  • “95% accuracy” is incomplete without task difficulty, baseline, sample size and confidence interval.
  • Fix the rules, preserve a complete record, separate training from testing, and only then ask whether a model adds information beyond a simple baseline.