Fase 4 — Backtest e laboratorio #90

Merged
Alby96 merged 7 commits from fase-4 into main 2026-11-11 14:18:20 +01:00
Owner

Motore di backtest a eventi, convalide del protocollo anti-illusioni, registro delle prove e schermata Laboratorio.

Cosa entra

  • Motore a eventi, una seduta alla volta: decisione sulla chiusura, esecuzione all'apertura dopo, stessi costi del simulatore (moltiplicabili ×2 e ×3), stop, frazionamenti e dividendi, nessun prestito. Correttivi e rischio hanno già il loro posto e fino alla Fase 5 lasciano passare tutto.
  • Metriche del rapporto e confronto con "comprare e tenere".
  • Statistica scritta in casa: distribuzione normale, momenti, Deflated Sharpe Ratio, PBO (CSCV), K-fold purgato combinatorio, finestre che avanzano, regola dell'altopiano, test contro lo sguardo in avanti.
  • Registro delle prove: piano e impronta SHA-256 scritti prima dei risultati e mai modificati; il DSR conta tutti i tentativi della stessa strategia.
  • Laboratorio su un thread a priorità bassa, con ripresa dopo un riavvio e coda che sopravvive a un errore del database.
  • Schermata Laboratorio: elenco, nuovo test, rapporto completo, confronto tra due prove; sezioni per le migliorie (Fase 9) e le stime di costo (Fase 8).

Criteri di accettazione

  • Strategia giocattolo con risultato calcolato a mano (AToyStrategyGivesExactlyTheResultComputedByHand, TheMetricsOfTheToyRunAreThoseComputedByHand)
  • Test anti-sguardo-avanti che prende una strategia scorretta (TheLookAheadTestCatchesAStrategyThatReadsTomorrow)
  • DSR e PBO sui casi noti (TheDeflatedSharpeRatioReproducesTheExampleOfTheArticle, i tre test del PBO)
  • Il rapporto si apre dall'interfaccia (TheReportOpensFromTheLaboratory)

Scelte nell'ADR 0010. Revisione indipendente fatta: 3 difetti e 3 imprecisioni corretti in f278ff4.

Closes #34
Closes #35
Closes #36
Closes #37

🤖 Generated with Claude Code

Motore di backtest a eventi, convalide del protocollo anti-illusioni, registro delle prove e schermata Laboratorio. **Cosa entra** - Motore a eventi, una seduta alla volta: decisione sulla chiusura, esecuzione all'apertura dopo, stessi costi del simulatore (moltiplicabili ×2 e ×3), stop, frazionamenti e dividendi, nessun prestito. Correttivi e rischio hanno già il loro posto e fino alla Fase 5 lasciano passare tutto. - Metriche del rapporto e confronto con "comprare e tenere". - Statistica scritta in casa: distribuzione normale, momenti, Deflated Sharpe Ratio, PBO (CSCV), K-fold purgato combinatorio, finestre che avanzano, regola dell'altopiano, test contro lo sguardo in avanti. - Registro delle prove: piano e impronta SHA-256 scritti prima dei risultati e mai modificati; il DSR conta tutti i tentativi della stessa strategia. - Laboratorio su un thread a priorità bassa, con ripresa dopo un riavvio e coda che sopravvive a un errore del database. - Schermata Laboratorio: elenco, nuovo test, rapporto completo, confronto tra due prove; sezioni per le migliorie (Fase 9) e le stime di costo (Fase 8). **Criteri di accettazione** - [x] Strategia giocattolo con risultato calcolato a mano (`AToyStrategyGivesExactlyTheResultComputedByHand`, `TheMetricsOfTheToyRunAreThoseComputedByHand`) - [x] Test anti-sguardo-avanti che prende una strategia scorretta (`TheLookAheadTestCatchesAStrategyThatReadsTomorrow`) - [x] DSR e PBO sui casi noti (`TheDeflatedSharpeRatioReproducesTheExampleOfTheArticle`, i tre test del PBO) - [x] Il rapporto si apre dall'interfaccia (`TheReportOpensFromTheLaboratory`) Scelte nell'ADR 0010. Revisione indipendente fatta: 3 difetti e 3 imprecisioni corretti in f278ff4. Closes #34 Closes #35 Closes #36 Closes #37 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Alby96 added 6 commits 2026-11-11 10:18:24 +01:00
The strategy contract of SPEC 5.3 (IStrategy, StrategyContext, Intent) is
shared by backtest and live: the context exposes bars only up to the
decision, adjusted and anchored to that day, and reading past it throws.
The engine runs each session as ordered events (corporate actions, open,
session, close), decides on the close and executes at the next open with
the simulator's price lists, slippage tiers, tick and lot rounding,
minimums and no leverage. Corrective and risk hooks are wired in and pass
everything through until Phase 5. One decision cycle can restart from a
recorded book (SPEC 12.3).

A scripted toy strategy reproduces a result computed by hand to the cent
on OKX and Alpaca costs. The Phase 3 class Trading.Strategies becomes
StrategyNames so the Anubis.Strategies namespace can exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Metrics give what the report of SPEC 12.2 point 6 needs: return, CAGR,
annualised volatility and Sharpe, max drawdown with its days, trades,
win rate, profit factor, fees, exposure and results per calendar year.

Statistics are written in house as SPEC 12.2 asks: the normal CDF
(Hart 5666, double precision) and quantile (Acklam plus a Halley step),
moments, the probabilistic and the Deflated Sharpe Ratio. The DSR test
reproduces the example of Bailey and López de Prado (SR0 0.1132, DSR
0.9004).

Validation works on the daily returns of every grid variant:
walk-forward with a growing window, combinatorial purged CV (6 groups,
2 tested: 15 splits, 5 paths, purge and embargo of 1% of the series)
and PBO by CSCV (16 groups). Pure noise gives a PBO near 0.5, a
dominating variant 0, a winner that always loses out of sample 1.
Choices follow the plateau rule: a variant scores its own Sharpe but no
more than the mean of its zone, so a lonely peak loses to a broad hill
and a poor variant does not win by standing next to a good one.

The look-ahead check reruns the strategy on data cut at six days and
requires every decision up to each cut to be unchanged; it fails on a
strategy that reads tomorrow and passes on the average cross. The
one-step comparison of SPEC 12.3 classifies intents and flags action
differences and sizes more than 10% apart. A golden run guards the
engine against unintended changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A trial is planned before it runs: strategy and version, grid, data,
initial capital, cost multipliers, validation settings, thresholds and
engine version go into backtest_trials as JSON with the SHA-256 of the
text. The store has no way to change a plan; later writes touch only
status, verdict, results and times, and a test reads the raw row back
to prove it. Every finished trial stays and counts towards the number
of tries the Deflated Sharpe Ratio deflates for.

The trial runner is pure computation: every grid variant on the whole
period, plateau choice, the chosen variant again at double and triple
costs, buy and hold, walk-forward, purged CV, PBO, DSR and the
look-ahead check, then the five thresholds of SPEC 12.2 and the report
of point 6 with the limitations the backtest declares (no correctives
without history, no risk engine before Phase 5).

The laboratory writes the plan, queues the trial and runs one at a time
on a dedicated thread with below-normal priority; only the async side
reads and writes the database. On start, trials left running go back
to planned and run again. Data comes from the daily bars and corporate
actions in the database, or from the Binance research history with
estimated order rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The laboratory starts with the host (off in demo mode) and runs its
queue next to market data and trading. The page lists every trial with
status, verdict and headline figures, refreshing itself while one is
queued or running, and offers the form for a new test: strategy, data
origin and market, instruments, period, initial capital and the grid,
with the thresholds stated before the run. The form posts with the
antiforgery token; errors come back as a short code.

The report page shows the protocol checks, the chosen variant against
buy and hold (table and chart), results per year, cost sensitivity,
the Deflated Sharpe Ratio, PBO, walk-forward windows, purged CV paths,
the look-ahead result, every variant with its zone score, what the
backtest does not model, the data used and the trades. A comparison
page puts two finished tests side by side and says whether they read
the same data. The improvements of 8.7 and the cost estimates of 11.6
have their sections, empty until Phases 9 and 8.

The equity chart takes amounts in their own currency and its labels as
parameters. The profit factor has no value without a losing trade.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ADR 0010 writes down what the specification leaves open: the event
order of a session, the costs shared with the simulator, the pass-through
correctives and risk until Phase 5, the in-house statistics, validations
on the daily returns of the variants, the plateau rule as the minimum of
own and neighbourhood Sharpe, the sizes of walk-forward, CPCV and CSCV,
the trial count of the DSR, the plan written first and never changed,
the low-priority laboratory and the two data origins (the research
history stays in USDT; a trial uses a single currency).

ROADMAP ticks the four acceptance criteria with the tests that prove
them; STATE moves to Phase 5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fix the defects of the Phase 4 review (refs #34 #35 #36)
ci / test (postgres) (push) Successful in 2m41s
ci / test (sqlite) (push) Successful in 1m56s
ci / test (postgres) (pull_request) Successful in 3m45s
ci / test (sqlite) (pull_request) Successful in 2m22s
ci / image (push) Successful in 1m10s
ci / image (pull_request) Successful in 13s
f278ff4f67
- An error of the database (PostgreSQL restarting) no longer stops the
  laboratory until the next start: the queue logs it, waits 30 seconds
  on the injectable clock and tries the same trial again.
- Buy and hold, the comparison of every report, buys an instrument
  whose history starts later at its first close, with the share kept
  free for it; before, it was never bought and the others took its
  weight. Its version goes to 2.
- The comparison page reads a and b as text: a malformed id shows
  "two finished tests needed" instead of an error page.
- A trial re-checks the hash of its plan before running and refuses a
  plan that no longer matches it.
- Parameters that count days take whole numbers only (10.5 would run
  as 10 and count twice in the DSR); the capital starts from 1, as the
  message says.

ADR 0010 says so, and corrects the estimated minimum of the research
history: 5 does refuse smaller orders, like the exchanges' minimums.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Alby96 added 1 commit 2026-11-11 11:54:46 +01:00
Add the VS Code debug configuration
ci / test (postgres) (push) Successful in 3m18s
ci / test (sqlite) (push) Successful in 2m1s
ci / test (postgres) (pull_request) Successful in 3m50s
ci / test (sqlite) (pull_request) Successful in 2m26s
ci / image (push) Successful in 8s
ci / image (pull_request) Successful in 6s
9db95ea79a
Four launch entries for the server: a data folder of its own under
artifacts/ (ignored by git), the demo mode, the user's real data folder,
and attach to a running process. Each builds in Debug first and opens the
browser once the server listens. tasks.json also exposes check, test and
dev. SVILUPPO.md explains which entry to pick.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Alby96 merged commit a6227793c3 into main 2026-11-11 14:18:20 +01:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: BalboSoft/Anubis#90