diff --git a/.dockerignore b/.dockerignore index cf448c2..baa4069 100644 --- a/.dockerignore +++ b/.dockerignore @@ -7,6 +7,7 @@ artifacts/ TestResults/ docs/ !docs/design/logo/ +!docs/backtests/*.outcome.json **/*.md .env .env.* diff --git a/deploy/docker/Dockerfile b/deploy/docker/Dockerfile index d0a17b6..f1bc33a 100644 --- a/deploy/docker/Dockerfile +++ b/deploy/docker/Dockerfile @@ -11,6 +11,8 @@ COPY src/Anubis/ src/Anubis/ COPY src/Anubis.Server/ src/Anubis.Server/ # The logo is embedded from docs/design/logo, its only source (ADR 0005). COPY docs/design/logo/ docs/design/logo/ +# The backtest outcomes are read at the first start (SPEC 12.3); .dockerignore lets only their JSON files through. +COPY docs/backtests/ docs/backtests/ RUN dotnet publish src/Anubis.Server/Anubis.Server.csproj -c Release --no-restore -o /app FROM mcr.microsoft.com/dotnet/aspnet:10.0 diff --git a/docs/backtests/crypto_trend.md b/docs/backtests/crypto_trend.md new file mode 100644 index 0000000..94d925e --- /dev/null +++ b/docs/backtests/crypto_trend.md @@ -0,0 +1,256 @@ +# Backtest di `crypto_trend` (Trend crypto) + +**Esito: non superato.** Protocollo non superato. Universo ampio: calo massimo 29,1%, serve ≤ 25%. BTC, ETH e SOL: probabilità di sovra-adattamento (pbo) 0,839, serve ≤ 0,40. Resta in osservazione. + +Calcolato il 2026-11-11 con la versione 1 della strategia. Soglie, griglia dei parametri, costi e date erano fissati prima del calcolo (SPEC 12.2) e non sono stati cambiati dopo. + +Per rifarlo: `anubis backtest base --to 2026-10-31`. Il file dell'esito, letto dal bot al primo avvio, è `docs/backtests/crypto_trend.outcome.json`. + +## Dati + +- Archivi giornalieri di data.binance.vision in USDT, ogni file controllato con la sua impronta SHA-256 prima dell'uso. +- Prezzi convertiti in euro con il cambio EUR/USD della BCE del giorno (7133 giorni di cambi), contando un USDT come un dollaro. Le coppie in euro di OKX esistono solo da novembre 2023: questo storico è un'approssimazione (SPEC 8.1). +- 47 coppie nell'universo ampio, comprese quelle poi chiuse da Binance: ogni mese la strategia sceglie le 10 con più scambi tra quelle quotate da almeno 400 giorni. +- Lo spread non si può misurare su questo storico: il filtro dello spread vale solo dal vivo. +- Senza archivi nel periodo, quindi non usate: BCCEUR, VENEUR. + +## Riepilogo + +| Prova | Variante scelta | Rendimento annuo | Calo massimo | Netto a costi doppi | DSR | PBO | Esito | +|---|---|---:|---:|---:|---:|---:|---| +| Universo ampio | `coin_vol=10;max_stop=20` | 11,2% | −29,1% | 8.053,52 € | 0,960 | 0,143 | non superata | +| BTC, ETH e SOL | `coin_vol=20;max_stop=15` | 13,8% | −17,3% | 12.845,05 € | 0,971 | 0,839 | non superata | + +## Cosa conserva il bot + +- Calo massimo del backtest: −29,1%. Dal vivo, un calo oltre 1,5 volte questo riporta la strategia indietro (SPEC 12.3). +- Scostamento di prezzo previsto: 11,91 punti base; costo stimato di andata e ritorno 0,64%. +- Percorso atteso al 5° percentile (ricampionamento a blocchi dei rendimenti giornalieri): −4,9% dopo 30 giorni, −7,6% dopo 90, −13,2% dopo 365. Venti sedute di fila sotto questo percorso riportano la strategia indietro. + +## Universo ampio + +Le 10 monete con più scambi, scelte ogni mese tra 47 coppie (anche quelle poi chiuse). + +Dal 01/01/2019 al 31/10/2026, 2861 chiusure giornaliere, capitale iniziale 10.000,00 €. Variante scelta: `coin_vol=10;max_stop=20` (la zona della griglia dove il risultato cambia meno, non la migliore). Impronta del piano `737ca3a07b9c`, dei dati `8486b858321b`. + +| | Strategia | Comprare e tenere | +|---|---:|---:| +| Risultato netto | 12.984,56 € | 82.750,41 € | +| Rendimento totale | 129,8% | 827,5% | +| Rendimento annuo | 11,2% | 32,9% | +| Volatilità annua | 14,0% | 74,2% | +| Calo massimo | −29,1% (11/2021 → 10/2023) | −85,8% | +| Sharpe annuo | 0,83 | 0,76 | +| Operazioni chiuse | 1826 | 0 | +| Operazioni in guadagno | 14,1% | — | +| Rapporto guadagni/perdite | 1,51 | — | +| Commissioni pagate | 2.975,10 € | 20,00 € | +| Tempo investito | 16,9% | 92,0% | + +**Per anno** + +| Anno | Strategia | Comprare e tenere | +|---|---:|---:| +| 2019 | 17,8% | 4,4% | +| 2020 | 34,9% | 103,9% | +| 2021 | 24,9% | 1.603,5% | +| 2022 | −10,5% | −79,9% | +| 2023 | 14,6% | 56,4% | +| 2024 | 26,8% | 34,5% | +| 2025 | −8,8% | −33,1% | +| 2026 | −2,4% | −9,4% | + +**Sensibilità ai costi** (commissioni e scostamento moltiplicati insieme) + +| Costi | Risultato netto | Rendimento totale | Sharpe annuo | Calo massimo | Commissioni | +|---|---:|---:|---:|---:|---:| +| ×1 | 12.984,56 € | 129,8% | 0,83 | −29,1% | 2.975,10 € | +| ×2 | 8.053,52 € | 80,5% | 0,61 | −33,9% | 5.005,99 € | +| ×3 | 4.349,38 € | 43,5% | 0,40 | −36,0% | 6.495,29 € | + +**Convalide** + +- Finestre che avanzano (4, con 29 giorni di stacco): rendimento fuori campione 67,6%, Sharpe annuo 0,65. +- K-fold purgato combinatorio (6 gruppi, 2 di prova, 15 divisioni): Sharpe annuo medio dei percorsi 0,83, peggiore 0,83, percorsi positivi 100,0%. +- Deflated Sharpe Ratio 0,96, contando 9 varianti provate in tutto. +- Probabilità di sovra-adattamento (PBO) 0,14, su 12870 combinazioni di 9 varianti. +- Sguardo in avanti: 8583 decisioni rifatte tagliando i dati in 6 punti, 0 cambiate. + +**Soglie** + +| Controllo | Valore | Soglia | Esito | +|---|---:|---:|---| +| Risultato netto con costi doppi | 8.053,52 € | > 0,00 € | superato | +| Deflated Sharpe Ratio | 0,960 | ≥ 0,9 | superato | +| Probabilità di sovra-adattamento (PBO) | 0,143 | ≤ 0,4 | superato | +| Calo massimo | 29,1% | ≤ 25% | **non superato** | +| Decisioni cambiate aggiungendo dati successivi | 0 | = 0 | superato | + +**Varianti della griglia** + +| Parametri | Sharpe annuo | Stabilità della zona | Risultato netto | Calo massimo | Operazioni | +|---|---:|---:|---:|---:|---:| +| `coin_vol=10;max_stop=10` | 0,58 | 0,58 | 6.383,12 € | −22,9% | 2243 | +| `coin_vol=10;max_stop=15` | 0,83 | 0,71 | 12.327,24 € | −24,6% | 1994 | +| `coin_vol=10;max_stop=20` (scelta) | 0,83 | 0,78 | 12.984,56 € | −29,1% | 1826 | +| `coin_vol=15;max_stop=10` | 0,53 | 0,53 | 6.035,84 € | −25,0% | 2257 | +| `coin_vol=15;max_stop=15` | 0,74 | 0,67 | 10.656,85 € | −29,4% | 2002 | +| `coin_vol=15;max_stop=20` | 0,74 | 0,74 | 11.167,92 € | −30,8% | 1837 | +| `coin_vol=20;max_stop=10` | 0,44 | 0,44 | 4.826,58 € | −25,0% | 2258 | +| `coin_vol=20;max_stop=15` | 0,65 | 0,63 | 8.943,30 € | −30,5% | 2008 | +| `coin_vol=20;max_stop=20` | 0,66 | 0,66 | 9.921,78 € | −31,1% | 1838 | + +**Interventi di correttivi e rischio sulla variante scelta** + +| Motivo | Volte | +|---|---:| +| `residue_written_off` | 1823 | +| `below_min_notional` | 387 | + +**Strumenti** + +| Coppia | Prima barra | Ultima barra | Barre | +|---|---|---|---:| +| ADAEUR | 2018-04-17 | 2026-10-31 | 3120 | +| APEEUR | 2022-03-17 | 2026-10-31 | 1690 | +| ARBEUR | 2023-03-23 | 2026-10-31 | 1319 | +| ATOMEUR | 2019-04-29 | 2026-10-31 | 2743 | +| AVAXEUR | 2020-09-22 | 2026-10-31 | 2231 | +| BCHABCEUR | 2018-11-16 | 2019-11-28 | 378 | +| BCHEUR | 2019-11-28 | 2026-10-31 | 2530 | +| BCHSVEUR | 2018-11-16 | 2019-04-22 | 158 | +| BNBEUR | 2017-11-06 | 2026-10-31 | 3282 | +| BTCEUR | 2017-10-08 | 2026-10-31 | 3311 | +| DASHEUR | 2019-03-28 | 2026-10-31 | 2775 | +| DOGEEUR | 2019-07-05 | 2026-10-31 | 2676 | +| DOTEUR | 2020-08-18 | 2026-10-31 | 2266 | +| EOSEUR | 2018-05-28 | 2025-05-26 | 2556 | +| ETCEUR | 2018-06-12 | 2026-10-31 | 3064 | +| ETHEUR | 2017-10-08 | 2026-10-31 | 3311 | +| FILEUR | 2020-10-15 | 2026-10-31 | 2208 | +| FTTEUR | 2019-12-20 | 2022-11-15 | 1062 | +| GMTEUR | 2022-03-09 | 2026-10-31 | 1698 | +| ICXEUR | 2018-06-13 | 2026-09-03 | 3005 | +| IOTAEUR | 2018-05-31 | 2026-10-31 | 3076 | +| LINKEUR | 2019-01-16 | 2026-10-31 | 2846 | +| LTCEUR | 2017-12-13 | 2026-10-31 | 3245 | +| LUNAEUR | 2020-08-21 | 2022-05-13 | 631 | +| MATICEUR | 2019-04-26 | 2024-09-10 | 1965 | +| NEAREUR | 2020-10-14 | 2026-10-31 | 2209 | +| NEOEUR | 2017-11-20 | 2026-10-31 | 3268 | +| ONTEUR | 2018-06-08 | 2026-10-31 | 3068 | +| OPEUR | 2022-06-01 | 2026-10-31 | 1614 | +| ORDIEUR | 2023-11-07 | 2026-10-31 | 1090 | +| PEPEEUR | 2023-05-05 | 2026-10-31 | 1276 | +| POLEUR | 2024-09-13 | 2026-10-31 | 779 | +| QTUMEUR | 2018-03-19 | 2026-10-31 | 3149 | +| SANDEUR | 2020-08-14 | 2026-10-31 | 2270 | +| SHIBEUR | 2021-05-10 | 2026-10-31 | 2001 | +| SOLEUR | 2020-08-11 | 2026-10-31 | 2273 | +| SUIEUR | 2023-05-03 | 2026-10-31 | 1278 | +| TONEUR | 2024-08-08 | 2026-06-30 | 692 | +| TRUMPEUR | 2025-01-19 | 2026-10-31 | 651 | +| TRXEUR | 2018-06-11 | 2026-10-31 | 3065 | +| UNIEUR | 2020-09-17 | 2026-10-31 | 2236 | +| VETEUR | 2018-07-25 | 2026-10-31 | 3021 | +| WIFEUR | 2024-03-05 | 2026-10-31 | 971 | +| XLMEUR | 2018-05-31 | 2026-10-31 | 3076 | +| XMREUR | 2019-03-15 | 2024-02-20 | 1804 | +| XRPEUR | 2018-05-04 | 2026-10-31 | 3103 | +| ZECEUR | 2019-03-21 | 2026-10-31 | 2782 | + +## BTC, ETH e SOL + +Solo le tre coppie in euro con la liquidità migliore. + +Dal 01/01/2019 al 31/10/2026, 2861 chiusure giornaliere, capitale iniziale 10.000,00 €. Variante scelta: `coin_vol=20;max_stop=15` (la zona della griglia dove il risultato cambia meno, non la migliore). Impronta del piano `3ba7de18ca85`, dei dati `f8e752dabf31`. + +| | Strategia | Comprare e tenere | +|---|---:|---:| +| Risultato netto | 17.515,31 € | 215.787,99 € | +| Rendimento totale | 175,2% | 2.157,9% | +| Rendimento annuo | 13,8% | 48,9% | +| Volatilità annua | 12,8% | 71,7% | +| Calo massimo | −17,3% (11/2021 → 10/2023) | −86,7% | +| Sharpe annuo | 1,07 | 0,92 | +| Operazioni chiuse | 452 | 0 | +| Operazioni in guadagno | 15,7% | — | +| Rapporto guadagni/perdite | 2,29 | — | +| Commissioni pagate | 2.282,94 € | 20,00 € | +| Tempo investito | 14,9% | 95,2% | + +**Per anno** + +| Anno | Strategia | Comprare e tenere | +|---|---:|---:| +| 2019 | 15,0% | 28,6% | +| 2020 | 30,5% | 221,5% | +| 2021 | 20,9% | 643,3% | +| 2022 | −7,4% | −81,4% | +| 2023 | 29,4% | 256,5% | +| 2024 | 22,8% | 93,3% | +| 2025 | −0,3% | −31,6% | +| 2026 | 3,5% | −16,1% | + +**Sensibilità ai costi** (commissioni e scostamento moltiplicati insieme) + +| Costi | Risultato netto | Rendimento totale | Sharpe annuo | Calo massimo | Commissioni | +|---|---:|---:|---:|---:|---:| +| ×1 | 17.515,31 € | 175,2% | 1,07 | −17,3% | 2.282,94 € | +| ×2 | 12.845,05 € | 128,5% | 0,89 | −20,9% | 4.070,32 € | +| ×3 | 9.959,70 € | 99,6% | 0,75 | −22,7% | 5.605,48 € | + +**Convalide** + +- Finestre che avanzano (4, con 29 giorni di stacco): rendimento fuori campione 90,3%, Sharpe annuo 0,91. +- K-fold purgato combinatorio (6 gruppi, 2 di prova, 15 divisioni): Sharpe annuo medio dei percorsi 1,00, peggiore 0,94, percorsi positivi 100,0%. +- Deflated Sharpe Ratio 0,97, contando 18 varianti provate in tutto. +- Probabilità di sovra-adattamento (PBO) 0,84, su 12870 combinazioni di 9 varianti. +- Sguardo in avanti: 8583 decisioni rifatte tagliando i dati in 6 punti, 0 cambiate. + +**Soglie** + +| Controllo | Valore | Soglia | Esito | +|---|---:|---:|---| +| Risultato netto con costi doppi | 12.845,05 € | > 0,00 € | superato | +| Deflated Sharpe Ratio | 0,971 | ≥ 0,9 | superato | +| Probabilità di sovra-adattamento (PBO) | 0,839 | ≤ 0,4 | **non superato** | +| Calo massimo | 17,3% | ≤ 25% | superato | +| Decisioni cambiate aggiungendo dati successivi | 0 | = 0 | superato | + +**Varianti della griglia** + +| Parametri | Sharpe annuo | Stabilità della zona | Risultato netto | Calo massimo | Operazioni | +|---|---:|---:|---:|---:|---:| +| `coin_vol=10;max_stop=10` | 1,01 | 1,00 | 9.436,53 € | −10,9% | 532 | +| `coin_vol=10;max_stop=15` | 0,94 | 0,94 | 9.790,38 € | −13,0% | 452 | +| `coin_vol=10;max_stop=20` | 1,05 | 1,01 | 12.078,19 € | −13,0% | 388 | +| `coin_vol=15;max_stop=10` | 1,04 | 1,03 | 13.029,82 € | −13,3% | 532 | +| `coin_vol=15;max_stop=15` | 1,01 | 1,01 | 14.386,02 € | −15,8% | 452 | +| `coin_vol=15;max_stop=20` | 1,05 | 1,03 | 15.995,77 € | −15,0% | 388 | +| `coin_vol=20;max_stop=10` | 1,09 | 1,05 | 15.920,52 € | −13,9% | 532 | +| `coin_vol=20;max_stop=15` (scelta) | 1,07 | 1,06 | 17.515,31 € | −17,3% | 452 | +| `coin_vol=20;max_stop=20` | 1,08 | 1,05 | 18.543,32 € | −15,5% | 388 | + +**Interventi di correttivi e rischio sulla variante scelta** + +| Motivo | Volte | +|---|---:| +| `residue_written_off` | 450 | +| `below_min_notional` | 89 | + +**Strumenti** + +| Coppia | Prima barra | Ultima barra | Barre | +|---|---|---|---:| +| BTCEUR | 2017-10-08 | 2026-10-31 | 3311 | +| ETHEUR | 2017-10-08 | 2026-10-31 | 3311 | +| SOLEUR | 2020-08-11 | 2026-10-31 | 2273 | + +## Cosa il backtest non rappresenta + +- I correttivi senza una storia ricostruibile (notizie, calendario economico, trimestrali, modello) non ricevono dati e non intervengono. +- Le decisioni si prendono sulla chiusura e si eseguono all'apertura della seduta dopo, con lo scostamento stimato: nessun inseguimento con ordini limite. +- Sharpe e volatilità sono calcolati senza tasso privo di rischio. +- Limiti di rischio e regime crypto applicati come dal vivo. La strategia è il 40% del bot: il resto conta come liquidità ferma. Ogni chiusura vale come un giorno; se scatta l'interruttore, supponiamo che tu riparta dopo sette giorni. Età dei prezzi, prezzo dell'ordine e orologio esistono solo dal vivo. +- Prezzi dello storico Binance in USDT convertiti in euro con il cambio BCE del giorno, contando un USDT come un dollaro: è un'approssimazione delle coppie in euro di OKX. Su questo storico lo spread non si può misurare: l'universo sono le monete con più scambi, passo di prezzo e quantità minima sono stime, i costi sono quelli di OKX. diff --git a/docs/backtests/crypto_trend.outcome.json b/docs/backtests/crypto_trend.outcome.json new file mode 100644 index 0000000..9325ceb --- /dev/null +++ b/docs/backtests/crypto_trend.outcome.json @@ -0,0 +1,376 @@ +{ + "passed": false, + "verdict": "Protocollo non superato. Universo ampio: calo massimo 29,1%, serve \u2264 25%. BTC, ETH e SOL: probabilit\u00E0 di sovra-adattamento (pbo) 0,839, serve \u2264 0,40. Resta in osservazione.", + "report": "docs/backtests/crypto_trend.md", + "plan_hash": "737ca3a07b9c67574013581d6ebc395e60b8edfa0681707804e67601112f1c12", + "max_drawdown": 0.2910, + "expected_slippage_bp": 11.91, + "round_trip_cost": 0.006382, + "expected_path_p5": [ + -0.0099, + -0.0133, + -0.0175, + -0.0205, + -0.0231, + -0.0234, + -0.0242, + -0.026, + -0.0281, + -0.0291, + -0.031, + -0.031, + -0.0334, + -0.034, + -0.0352, + -0.037, + -0.0382, + -0.0396, + -0.0412, + -0.0405, + -0.0401, + -0.0415, + -0.0421, + -0.0437, + -0.044, + -0.0446, + -0.0458, + -0.0467, + -0.0479, + -0.0489, + -0.0503, + -0.0517, + -0.0528, + -0.0525, + -0.0529, + -0.0539, + -0.0552, + -0.0557, + -0.0563, + -0.0568, + -0.0583, + -0.0585, + -0.0595, + -0.0591, + -0.0583, + -0.0587, + -0.0593, + -0.0595, + -0.0596, + -0.0613, + -0.0609, + -0.0618, + -0.0629, + -0.0638, + -0.0641, + -0.0634, + -0.0646, + -0.0649, + -0.066, + -0.0659, + -0.067, + -0.0676, + -0.0693, + -0.0692, + -0.0697, + -0.0686, + -0.0687, + -0.0693, + -0.0693, + -0.0697, + -0.0696, + -0.069, + -0.069, + -0.0716, + -0.0725, + -0.0727, + -0.073, + -0.073, + -0.0734, + -0.0735, + -0.0738, + -0.074, + -0.0747, + -0.074, + -0.0757, + -0.0741, + -0.0742, + -0.0746, + -0.0757, + -0.0756, + -0.0761, + -0.0757, + -0.0761, + -0.0771, + -0.0785, + -0.0807, + -0.0819, + -0.0827, + -0.0823, + -0.083, + -0.0823, + -0.0831, + -0.0829, + -0.0853, + -0.0849, + -0.0836, + -0.0841, + -0.0835, + -0.084, + -0.0857, + -0.0856, + -0.0852, + -0.0861, + -0.0861, + -0.0861, + -0.0848, + -0.0861, + -0.0866, + -0.0876, + -0.0887, + -0.0897, + -0.0906, + -0.0902, + -0.0916, + -0.0917, + -0.0922, + -0.0937, + -0.0922, + -0.0931, + -0.0926, + -0.0922, + -0.0914, + -0.0914, + -0.0901, + -0.0916, + -0.0917, + -0.0937, + -0.0937, + -0.0955, + -0.0971, + -0.0958, + -0.0969, + -0.0972, + -0.0992, + -0.1007, + -0.101, + -0.1004, + -0.1008, + -0.1003, + -0.1005, + -0.1025, + -0.1024, + -0.1034, + -0.1027, + -0.1034, + -0.1042, + -0.1034, + -0.105, + -0.1037, + -0.1054, + -0.1066, + -0.1053, + -0.1055, + -0.1062, + -0.1057, + -0.1057, + -0.1072, + -0.1071, + -0.108, + -0.108, + -0.1087, + -0.1102, + -0.1118, + -0.1117, + -0.111, + -0.1112, + -0.111, + -0.1132, + -0.1108, + -0.1112, + -0.1119, + -0.1113, + -0.1127, + -0.1129, + -0.1129, + -0.1127, + -0.1143, + -0.1139, + -0.1162, + -0.1156, + -0.1159, + -0.1154, + -0.1173, + -0.116, + -0.1147, + -0.1162, + -0.1139, + -0.1135, + -0.1135, + -0.1138, + -0.1133, + -0.1144, + -0.1136, + -0.1136, + -0.1139, + -0.114, + -0.1151, + -0.1147, + -0.1148, + -0.1163, + -0.1161, + -0.1169, + -0.1164, + -0.1179, + -0.1179, + -0.1201, + -0.1206, + -0.1211, + -0.122, + -0.1227, + -0.1218, + -0.1221, + -0.1222, + -0.1222, + -0.1233, + -0.123, + -0.123, + -0.1229, + -0.1231, + -0.1225, + -0.1212, + -0.1195, + -0.1201, + -0.1202, + -0.1203, + -0.121, + -0.1228, + -0.1225, + -0.1221, + -0.1222, + -0.1222, + -0.1223, + -0.1223, + -0.1223, + -0.1241, + -0.1238, + -0.126, + -0.1252, + -0.1269, + -0.1266, + -0.1265, + -0.1258, + -0.1245, + -0.1261, + -0.1256, + -0.1258, + -0.1261, + -0.1263, + -0.1254, + -0.1272, + -0.1266, + -0.1266, + -0.1268, + -0.1273, + -0.1287, + -0.1275, + -0.1288, + -0.1269, + -0.1268, + -0.1273, + -0.1272, + -0.1269, + -0.1262, + -0.1259, + -0.1273, + -0.1267, + -0.1268, + -0.1259, + -0.1266, + -0.1257, + -0.1267, + -0.1272, + -0.1267, + -0.1268, + -0.126, + -0.1269, + -0.1267, + -0.1256, + -0.1256, + -0.1254, + -0.1258, + -0.1267, + -0.127, + -0.1275, + -0.1291, + -0.1262, + -0.1273, + -0.1287, + -0.129, + -0.1292, + -0.1278, + -0.125, + -0.1259, + -0.1273, + -0.1278, + -0.1278, + -0.1274, + -0.1253, + -0.1257, + -0.1237, + -0.1237, + -0.1244, + -0.1236, + -0.1242, + -0.1242, + -0.1248, + -0.1258, + -0.1266, + -0.127, + -0.126, + -0.126, + -0.1264, + -0.1263, + -0.1263, + -0.1268, + -0.1263, + -0.1249, + -0.1281, + -0.1282, + -0.1265, + -0.126, + -0.1262, + -0.1259, + -0.1281, + -0.1275, + -0.1296, + -0.1299, + -0.1286, + -0.1299, + -0.1292, + -0.128, + -0.1304, + -0.1309, + -0.1283, + -0.1313, + -0.1319, + -0.1308, + -0.1308, + -0.1307, + -0.1297, + -0.1303, + -0.1298, + -0.1322, + -0.1323, + -0.1323, + -0.1316, + -0.1311, + -0.1325, + -0.1332, + -0.1339, + -0.1339, + -0.1336, + -0.1332, + -0.1318, + -0.132 + ] +} diff --git a/docs/backtests/equity_meanrev.md b/docs/backtests/equity_meanrev.md new file mode 100644 index 0000000..db3757f --- /dev/null +++ b/docs/backtests/equity_meanrev.md @@ -0,0 +1,10 @@ +# Backtest di `equity_meanrev` (Ritorno alla media azioni) + +**Esito: in attesa.** In attesa delle chiavi Alpaca: senza lo storico delle azioni il backtest non si può fare, quindi la strategia resta in osservazione. + +Per provare le strategie sulle azioni servono gli storici giornalieri di Alpaca, con frazionamenti e dividendi, che si scaricano solo con le chiavi di un conto Alpaca (anche dimostrativo). Le chiavi non sono ancora state inserite. + +Finché manca il backtest la strategia resta in osservazione: guarda il mercato e scrive le sue decisioni nel diario, ma non opera, nemmeno in simulazione. + +Quando avrai inserito le chiavi nelle Impostazioni, il bot raccoglie gli storici. Poi dal Laboratorio lanci la prova della strategia con l'origine "barre raccolte": se supera le soglie di SPEC 12.2, la porti in simulazione dalla pagina Strategie. + diff --git a/docs/backtests/equity_meanrev.outcome.json b/docs/backtests/equity_meanrev.outcome.json new file mode 100644 index 0000000..dce192e --- /dev/null +++ b/docs/backtests/equity_meanrev.outcome.json @@ -0,0 +1,9 @@ +{ + "passed": false, + "verdict": "In attesa delle chiavi Alpaca: senza lo storico delle azioni il backtest non si pu\u00F2 fare, quindi la strategia resta in osservazione.", + "report": "docs/backtests/equity_meanrev.md", + "max_drawdown": 0, + "expected_slippage_bp": 0, + "round_trip_cost": 0, + "expected_path_p5": [] +} diff --git a/docs/backtests/equity_momentum.md b/docs/backtests/equity_momentum.md new file mode 100644 index 0000000..9f182a7 --- /dev/null +++ b/docs/backtests/equity_momentum.md @@ -0,0 +1,10 @@ +# Backtest di `equity_momentum` (Momentum azioni) + +**Esito: in attesa.** In attesa delle chiavi Alpaca: senza lo storico delle azioni il backtest non si può fare, quindi la strategia resta in osservazione. + +Per provare le strategie sulle azioni servono gli storici giornalieri di Alpaca, con frazionamenti e dividendi, che si scaricano solo con le chiavi di un conto Alpaca (anche dimostrativo). Le chiavi non sono ancora state inserite. + +Finché manca il backtest la strategia resta in osservazione: guarda il mercato e scrive le sue decisioni nel diario, ma non opera, nemmeno in simulazione. + +Quando avrai inserito le chiavi nelle Impostazioni, il bot raccoglie gli storici. Poi dal Laboratorio lanci la prova della strategia con l'origine "barre raccolte": se supera le soglie di SPEC 12.2, la porti in simulazione dalla pagina Strategie. + diff --git a/docs/backtests/equity_momentum.outcome.json b/docs/backtests/equity_momentum.outcome.json new file mode 100644 index 0000000..87af21a --- /dev/null +++ b/docs/backtests/equity_momentum.outcome.json @@ -0,0 +1,9 @@ +{ + "passed": false, + "verdict": "In attesa delle chiavi Alpaca: senza lo storico delle azioni il backtest non si pu\u00F2 fare, quindi la strategia resta in osservazione.", + "report": "docs/backtests/equity_momentum.md", + "max_drawdown": 0, + "expected_slippage_bp": 0, + "round_trip_cost": 0, + "expected_path_p5": [] +} diff --git a/src/Anubis.Server/BacktestCommand.cs b/src/Anubis.Server/BacktestCommand.cs new file mode 100644 index 0000000..49096cc --- /dev/null +++ b/src/Anubis.Server/BacktestCommand.cs @@ -0,0 +1,142 @@ +using System.Collections; +using System.Globalization; +using Anubis.Backtest; +using Anubis.Brokers; +using Anubis.Core; +using Anubis.Data; +using Anubis.Strategies; +using Anubis.Trading; + +namespace Anubis.Server; + +/// +/// "anubis backtest base": the honest backtests of the base strategies (SPEC 12.2, issue #44). It downloads the +/// Binance daily archives (each checked against its SHA-256) and the ECB rates into the research history, converts +/// them to euro, runs the protocol of crypto_trend on both universes and writes reports and outcome files. +/// The stock strategies need the Alpaca history: without it their files say they wait, and they stay in observation. +/// +internal static class BacktestCommand +{ + private const string Usage = + "Uso: anubis backtest base [--to AAAA-MM-GG] [--out docs/backtests] [--offline]\n" + + " --offline usa solo gli archivi già scaricati (i cambi BCE li scarica solo se mancano). Opzioni comuni: [--platform docker|desktop] [--data-dir ]"; + + public static async Task RunAsync(string[] args, TextWriter? output = null, IReadOnlyDictionary? environment = null, + HttpMessageHandler? handler = null) + { + output ??= Console.Out; + if (args.Length < 1 || args[0] != "base") + { + await output.WriteLineAsync(Usage).ConfigureAwait(false); + return 2; + } + + var rest = args[1..].ToList(); + environment ??= Environment.GetEnvironmentVariables().Cast().ToDictionary(e => (string)e.Key, e => e.Value as string, StringComparer.Ordinal); + var options = ServerOptions.Parse([.. rest.Where(a => a != "--offline")], environment); + var config = options.LoadConfig(environment); + var clock = new SystemClock(); + var now = clock.UtcNow; + var to = Option(rest, "--to") is { } t + ? DateOnly.ParseExact(t, "yyyy-MM-dd", CultureInfo.InvariantCulture) + : new DateOnly(now.Year, now.Month, 1).AddDays(-1); // The last month whose archive is published. + var outDir = Path.GetFullPath(Option(rest, "--out") ?? Path.Combine("docs", "backtests")); + try + { + config.Data.Ensure(); + Directory.CreateDirectory(outDir); + var store = new ResearchStore(config.Data.Research); + IReadOnlyList rates; + using (var http = handler is null ? new HttpClient() : new HttpClient(handler, disposeHandler: false)) + { + http.Timeout = TimeSpan.FromMinutes(5); + http.DefaultRequestHeaders.UserAgent.ParseAdd($"{AnubisInfo.Name}/{AnubisInfo.Version}"); + rates = await RatesAsync(new ApiHttp(http, clock, "ecb"), config.Data.Research, rest.Contains("--offline"), output).ConfigureAwait(false); + if (!rest.Contains("--offline")) + { + var importer = new BinanceImporter(new ApiHttp(http, clock, "binance"), store, clock); + foreach (var symbol in CryptoTrendProtocol.Archives) + { + var result = await importer.ImportAsync(symbol, "1d", 2017, 8, CancellationToken.None).ConfigureAwait(false); + await output.WriteLineAsync(string.Create(CultureInfo.InvariantCulture, $"{symbol}: {result.Bars} barre giornaliere")).ConfigureAwait(false); + } + } + } + + var converted = await EuroHistory.ConvertAllAsync(store, rates, CancellationToken.None).ConfigureAwait(false); + await output.WriteLineAsync($"Convertite in euro {converted.Count} coppie.").ConfigureAwait(false); + + var outcome = await CryptoTrendProtocol.RunAsync(config.Data.Research, to, now, text => output.WriteLine(text), CancellationToken.None).ConfigureAwait(false); + var notes = DataNotes(outcome, rates); + await File.WriteAllTextAsync(Path.Combine(outDir, "crypto_trend.md"), + ProtocolReport.Markdown(outcome, "crypto_trend", notes, "anubis backtest base --to " + to.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture))).ConfigureAwait(false); + await File.WriteAllTextAsync(Path.Combine(outDir, "crypto_trend.outcome.json"), BacktestOutcomes.ToJson(outcome.Qualification)).ConfigureAwait(false); + foreach (var run in outcome.Runs) + { + // The full reports (curve, trades) stay with the data: the repository keeps the summary and the outcome. + await File.WriteAllTextAsync(Path.Combine(config.Data.Research, "crypto_trend-" + run.Report.PlanHash[..12] + ".report.json"), run.Report.ToJson()) + .ConfigureAwait(false); + } + + foreach (var (strategy, name) in new[] { (StrategyNames.EquityMomentum, "Momentum azioni"), (StrategyNames.EquityMeanrev, "Ritorno alla media azioni") }) + { + var waiting = EquityProtocol.Waiting(strategy); + await File.WriteAllTextAsync(Path.Combine(outDir, strategy + ".md"), ProtocolReport.Waiting(strategy, name, waiting, EquityProtocol.Explanation)).ConfigureAwait(false); + await File.WriteAllTextAsync(Path.Combine(outDir, strategy + ".outcome.json"), BacktestOutcomes.ToJson(waiting)).ConfigureAwait(false); + } + + await output.WriteLineAsync((outcome.Qualification.Passed ? "Superato: " : "Non superato: ") + outcome.Qualification.Verdict).ConfigureAwait(false); + await output.WriteLineAsync("Rapporti scritti in " + outDir).ConfigureAwait(false); + return 0; + } + catch (Exception ex) when (ex is IOException or MarketDataException or HttpRequestException or FormatException or InvalidOperationException + or UnauthorizedAccessException or TaskCanceledException or System.IO.InvalidDataException) + { + await output.WriteLineAsync("Non riuscito: " + Redactor.Redact(ex.Message)).ConfigureAwait(false); + return 1; + } + } + + /// The ECB history of EUR/USD, kept next to the research data so an offline run uses the same rates. + private static async Task> RatesAsync(ApiHttp http, string researchRoot, bool offline, TextWriter output) + { + var path = Path.Combine(researchRoot, "ecb-eurofxref-hist.zip"); + if (!offline || !File.Exists(path)) + { + var zip = await http.GetBytesAsync(Ecb.History, null, null, CancellationToken.None).ConfigureAwait(false) + ?? throw new MarketDataException("BCE: lo storico dei cambi non è disponibile.", retryable: true); + await File.WriteAllBytesAsync(path, zip).ConfigureAwait(false); + } + + var rates = Ecb.ParseHistoryCsv(Ecb.ExtractHistoryCsv(await File.ReadAllBytesAsync(path).ConfigureAwait(false)), ["USD"]); + await output.WriteLineAsync(string.Create(CultureInfo.InvariantCulture, $"Cambi BCE EUR/USD: {rates.Count} giorni, dal {rates[0].Date:yyyy-MM-dd} al {rates[^1].Date:yyyy-MM-dd}.")) + .ConfigureAwait(false); + return rates; + } + + private static List DataNotes(ProtocolOutcome outcome, IReadOnlyList rates) + { + var broad = outcome.Runs[0].Report.Instruments; + var it = CultureInfo.GetCultureInfo("it-IT"); + var missing = CryptoTrendProtocol.Archives.Select(EuroHistory.SymbolOf).Where(s => broad.All(i => i.Symbol != s)).ToList(); + var notes = new List + { + "Archivi giornalieri di data.binance.vision in USDT, ogni file controllato con la sua impronta SHA-256 prima dell'uso.", + string.Create(it, $"Prezzi convertiti in euro con il cambio EUR/USD della BCE del giorno ({rates.Count} giorni di cambi), contando un USDT come un dollaro. Le coppie in euro di OKX esistono solo da novembre 2023: questo storico è un'approssimazione (SPEC 8.1)."), + string.Create(it, $"{broad.Count} coppie nell'universo ampio, comprese quelle poi chiuse da Binance: ogni mese la strategia sceglie le 10 con più scambi tra quelle quotate da almeno 400 giorni."), + "Lo spread non si può misurare su questo storico: il filtro dello spread vale solo dal vivo.", + }; + if (missing.Count > 0) + { + notes.Add("Senza archivi nel periodo, quindi non usate: " + string.Join(", ", missing) + "."); + } + + return notes; + } + + private static string? Option(List args, string name) + { + var index = args.IndexOf(name); + return index >= 0 && index < args.Count - 1 ? args[index + 1] : null; + } +} diff --git a/src/Anubis.Server/Program.cs b/src/Anubis.Server/Program.cs index f8c172c..c145661 100644 --- a/src/Anubis.Server/Program.cs +++ b/src/Anubis.Server/Program.cs @@ -12,6 +12,12 @@ if (args.Length >= 1 && args[0] == "data") return await DataCommand.RunAsync(args[1..]).ConfigureAwait(false); } +// "anubis backtest base": the honest backtests of the base strategies, written to docs/backtests (Phase 5). +if (args.Length >= 1 && args[0] == "backtest") +{ + return await BacktestCommand.RunAsync(args[1..]).ConfigureAwait(false); +} + var app = ServerApp.Build(args); await app.RunAsync().ConfigureAwait(false); return 0; diff --git a/src/Anubis/Anubis.csproj b/src/Anubis/Anubis.csproj index 7cc3c05..7ee364e 100644 --- a/src/Anubis/Anubis.csproj +++ b/src/Anubis/Anubis.csproj @@ -10,6 +10,8 @@ + + diff --git a/src/Anubis/Backtest/Laboratory.cs b/src/Anubis/Backtest/Laboratory.cs index e9a2c63..0309736 100644 --- a/src/Anubis/Backtest/Laboratory.cs +++ b/src/Anubis/Backtest/Laboratory.cs @@ -46,7 +46,7 @@ public static class TrialRequestErrors /// public sealed partial class Laboratory : IDisposable { - public const int MaxSymbols = 20; + public const int MaxSymbols = 60; public const int MaxVariants = 200; @@ -264,29 +264,45 @@ public sealed partial class Laboratory : IDisposable { var instrument = await catalog.FindAsync(data.Venue, symbol, cancellationToken).ConfigureAwait(false) ?? throw new InvalidOperationException($"Strumento sconosciuto: {symbol} su {data.Venue}."); - var daily = await bars.DailyAsync(instrument.Id, data.From, data.To, cancellationToken).ConfigureAwait(false); + var daily = await bars.DailyAsync(instrument.Id, data.HistoryStart(), data.To, cancellationToken).ConfigureAwait(false); var events = instrument.IsCrypto ? [] : await actions.ForSymbolAsync(symbol, cancellationToken).ConfigureAwait(false); - instruments.Add(new BacktestInstrument(instrument.Spec, daily, [.. events.Where(a => a.ExDate > data.From && a.ExDate <= data.To)], "barre raccolte")); + instruments.Add(new BacktestInstrument(instrument.Spec, daily, [.. events.Where(a => a.ExDate > data.HistoryStart() && a.ExDate <= data.To)], "barre raccolte")); } } else { - var store = new ResearchStore(researchRoot); - var from = new DateTimeOffset(data.From.ToDateTime(TimeOnly.MinValue), TimeSpan.Zero); - var to = new DateTimeOffset(data.To.AddDays(1).ToDateTime(TimeOnly.MinValue), TimeSpan.Zero); - foreach (var symbol in data.Symbols) - { - var daily = new List(); - await foreach (var b in store.ReadAsync(data.Venue, symbol, Timeframes.Day, from, to, cancellationToken).ConfigureAwait(false)) - { - daily.Add(new DailyBar(DateOnly.FromDateTime(b.OpensAt.UtcDateTime), (double)b.Open, (double)b.High, (double)b.Low, (double)b.Close, (double)b.Volume)); - } - - instruments.Add(new BacktestInstrument(ResearchSpec(data.Venue, symbol), daily, null, "storico di ricerca")); - } + return await LoadResearchAsync(researchRoot, data, cancellationToken).ConfigureAwait(false); } - var empty = instruments.FirstOrDefault(i => i.Bars.Count == 0); + return Checked(instruments, data); + } + + /// The bars of a trial on the research history: no database needed (the protocol runs from the command line). + public static async Task LoadResearchAsync(string researchRoot, TrialData data, CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(data); + var instruments = new List(); + var store = new ResearchStore(researchRoot); + var from = new DateTimeOffset(data.HistoryStart().ToDateTime(TimeOnly.MinValue), TimeSpan.Zero); + var to = new DateTimeOffset(data.To.AddDays(1).ToDateTime(TimeOnly.MinValue), TimeSpan.Zero); + var source = data.Venue == EuroHistory.Venue ? "storico Binance convertito in euro" : "storico di ricerca"; + foreach (var symbol in data.Symbols) + { + var daily = new List(); + await foreach (var b in store.ReadAsync(data.Venue, symbol, Timeframes.Day, from, to, cancellationToken).ConfigureAwait(false)) + { + daily.Add(new DailyBar(DateOnly.FromDateTime(b.OpensAt.UtcDateTime), (double)b.Open, (double)b.High, (double)b.Low, (double)b.Close, (double)b.Volume)); + } + + instruments.Add(new BacktestInstrument(ResearchSpec(data.Venue, symbol), daily, null, source)); + } + + return Checked(instruments, data); + } + + private static BacktestDataset Checked(List instruments, TrialData data) + { + var empty = instruments.FirstOrDefault(i => !i.Bars.Any(b => b.Day >= data.From && b.Day <= data.To)); if (empty is not null) { throw new InvalidOperationException($"Nessuna barra giornaliera per {empty.Symbol} tra il {data.From:dd/MM/yyyy} e il {data.To:dd/MM/yyyy}."); diff --git a/src/Anubis/Backtest/Protocol.cs b/src/Anubis/Backtest/Protocol.cs new file mode 100644 index 0000000..0fa9cfc --- /dev/null +++ b/src/Anubis/Backtest/Protocol.cs @@ -0,0 +1,376 @@ +using System.Globalization; +using System.Security.Cryptography; +using System.Text; +using System.Text.Json; +using Anubis.Core; +using Anubis.Data; +using Anubis.Strategies; +using Anubis.Trading; + +namespace Anubis.Backtest; + +/// +/// The Binance USDT archives turned into euro (SPEC 8.1): the EUR pairs of OKX start in November 2023, so the years +/// before use the USDT history divided by the ECB EUR/USD rate of the day, one USDT counted as one dollar. The +/// converted series live in the research history under their own venue, so a trial on them can be planned again. +/// +public static class EuroHistory +{ + public const string Venue = "binance_eur"; + + /// A rate older than this is not used: the ECB never skips more than a long holiday weekend. + public static readonly int MaxRateAgeDays = 7; + + /// + /// A hole longer than this in a coin's daily series ends it: what comes after is a relaunch under the same ticker + /// (LUNA after May 2022), not the coin the earlier bars describe. + /// + public static readonly int MaxGapDays = 7; + + /// BTCUSDT becomes BTCEUR. + public static string SymbolOf(string usdtSymbol) + { + ArgumentNullException.ThrowIfNull(usdtSymbol); + return usdtSymbol.EndsWith("USDT", StringComparison.Ordinal) && usdtSymbol.Length > 4 + ? usdtSymbol[..^4] + "EUR" + : throw new ArgumentException("Not a USDT pair: " + usdtSymbol, nameof(usdtSymbol)); + } + + /// + /// Divides prices and traded value by the EUR/USD rate of the bar's day, or of the last day before with a rate. A + /// bar is dropped when no rate of the last days exists. The rate of day d is fixed at + /// 16:00 CET, before the daily bar closes at 00:00 UTC, so the close in euro uses nothing from the future. The + /// series stops at the first hole longer than days. + /// + public static IReadOnlyList Convert(IEnumerable bars, IReadOnlyList eurUsd) + { + ArgumentNullException.ThrowIfNull(bars); + ArgumentNullException.ThrowIfNull(eurUsd); + var rates = eurUsd.Where(r => r.Base == "EUR" && r.Quote == "USD" && r.Rate > 0m).OrderBy(r => r.Date).ToList(); + var result = new List(); + var k = -1; + DateTimeOffset? previous = null; + foreach (var bar in bars.OrderBy(b => b.OpensAt)) + { + if (previous is { } p && bar.OpensAt - p > TimeSpan.FromDays(MaxGapDays)) + { + break; + } + + previous = bar.OpensAt; + var day = DateOnly.FromDateTime(bar.OpensAt.UtcDateTime); + while (k + 1 < rates.Count && rates[k + 1].Date <= day) + { + k++; + } + + if (k < 0 || rates[k].Date < day.AddDays(-MaxRateAgeDays)) + { + continue; + } + + var rate = rates[k].Rate; + result.Add(new ResearchBar(bar.OpensAt, Round(bar.Open / rate), Round(bar.High / rate), Round(bar.Low / rate), Round(bar.Close / rate), bar.Volume) + { + QuoteVolume = bar.QuoteVolume is { } q ? Round(q / rate) : null, + Trades = bar.Trades, + }); + } + + return result; + } + + /// Converts every daily USDT series of the Binance archives in the store. Returns the symbols written. + public static async Task> ConvertAllAsync(ResearchStore store, IReadOnlyList eurUsd, CancellationToken cancellationToken) + { + ArgumentNullException.ThrowIfNull(store); + var written = new List(); + foreach (var (_, symbol, _) in store.Series().Where(s => s.Venue == Venues.Binance && s.Timeframe == Timeframes.Day && s.Symbol.EndsWith("USDT", StringComparison.Ordinal))) + { + var bars = new List(); + await foreach (var bar in store.ReadAsync(Venues.Binance, symbol, Timeframes.Day, Ever.From, Ever.To, cancellationToken).ConfigureAwait(false)) + { + bars.Add(bar); + } + + var converted = Convert(bars, eurUsd); + if (converted.Count > 0) + { + await store.WriteAsync(Venue, SymbolOf(symbol), Timeframes.Day, converted, cancellationToken).ConfigureAwait(false); + written.Add(SymbolOf(symbol)); + } + } + + return written; + } + + // Twelve decimals keep seven significant digits even for coins priced at a hundred-thousandth of a euro. + private static decimal Round(decimal value) => Math.Round(value, 12, MidpointRounding.ToEven); + + private static class Ever + { + public static readonly DateTimeOffset From = new(2009, 1, 1, 0, 0, 0, TimeSpan.Zero); + + public static readonly DateTimeOffset To = new(2100, 1, 1, 0, 0, 0, TimeSpan.Zero); + } +} + +/// A trial of the protocol: which universe, its report and the daily returns of the chosen variant. +public sealed record ProtocolRun(string Name, string Description, TrialPlan Plan, BacktestReport Report, IReadOnlyList DailyReturns, decimal ExpectedSlippageBp); + +/// What the protocol of a strategy concluded: the outcome file and the trials behind it. +public sealed record ProtocolOutcome(Qualification Qualification, IReadOnlyList Runs); + +/// +/// The anti-illusion protocol of SPEC 12.2 for crypto_trend, run from the command line on the research +/// history: it must pass twice (SPEC 8.1), on the broad universe and on BTC, ETH and SOL alone. Everything is fixed +/// before the run: dates, capital, grid, costs, thresholds. The result is written whatever it is. +/// +public static class CryptoTrendProtocol +{ + public const string Strategy = StrategyNames.CryptoTrend; + + public const string ReportPath = "docs/backtests/crypto_trend.md"; + + public const decimal Capital = 10_000m; + + /// Two years of Binance history before this day are enough for the longest period (360) and the 400 days of age. + public static readonly DateOnly From = new(2019, 1, 1); + + public static readonly IReadOnlyList Narrow = ["BTCEUR", "ETHEUR", "SOLEUR"]; + + /// + /// The USDT pairs downloaded from the Binance archives: every coin that was among the ten most traded at some time + /// since 2018, including those later delisted (LUNA, FTT, BCC, VEN, XMR...), so the past universe is not chosen + /// with hindsight. Stablecoins are left out: the strategy excludes them anyway. A pair without archives is skipped. + /// + public static readonly IReadOnlyList Archives = + [ + "BTCUSDT", "ETHUSDT", "BNBUSDT", "XRPUSDT", "ADAUSDT", "SOLUSDT", "DOGEUSDT", "LTCUSDT", "BCCUSDT", "BCHABCUSDT", "BCHUSDT", "BCHSVUSDT", + "EOSUSDT", "TRXUSDT", "ETCUSDT", "LINKUSDT", "DOTUSDT", "MATICUSDT", "POLUSDT", "AVAXUSDT", "LUNAUSDT", "FTTUSDT", "SHIBUSDT", "FILUSDT", + "ATOMUSDT", "UNIUSDT", "NEOUSDT", "XLMUSDT", "VENUSDT", "VETUSDT", "SANDUSDT", "APEUSDT", "GMTUSDT", "NEARUSDT", "PEPEUSDT", "WIFUSDT", + "SUIUSDT", "ARBUSDT", "OPUSDT", "ORDIUSDT", "ONTUSDT", "IOTAUSDT", "QTUMUSDT", "ICXUSDT", "XMRUSDT", "DASHUSDT", "ZECUSDT", "TONUSDT", "TRUMPUSDT", + ]; + + /// Length of the expected path of the outcome: one year of daily closes. + public const int PathDays = 365; + + public static async Task RunAsync(string researchRoot, DateOnly to, DateTimeOffset now, Action? progress, CancellationToken cancellationToken) + { + var store = new ResearchStore(researchRoot); + var broad = new List(); + foreach (var (_, symbol, _) in store.Series().Where(s => s.Venue == EuroHistory.Venue && s.Timeframe == Timeframes.Day)) + { + // Every converted pair goes in: the strategy picks its top 10 month by month, among coins that died too. + if (await HasBarsAsync(store, symbol, to, cancellationToken).ConfigureAwait(false)) + { + broad.Add(symbol); + } + } + + if (Narrow.Any(s => !broad.Contains(s))) + { + throw new InvalidOperationException("Mancano gli storici di BTC, ETH o SOL in euro: scarica prima gli archivi Binance e il cambio BCE."); + } + + var family = StrategyCatalog.Find(Strategy) ?? throw new InvalidOperationException("crypto_trend non è nel catalogo."); + var first = await RunOneAsync(family, "Universo ampio", $"Le 10 monete con più scambi, scelte ogni mese tra {broad.Count} coppie (anche quelle poi chiuse).", + broad, to, now, researchRoot, PriorTrials.None, progress, cancellationToken).ConfigureAwait(false); + var prior = new PriorTrials(1, first.Report.Variants.Count, [.. first.Report.Variants.Select(v => v.Sharpe)]); + var second = await RunOneAsync(family, "BTC, ETH e SOL", "Solo le tre coppie in euro con la liquidità migliore.", + Narrow, to, now, researchRoot, prior, progress, cancellationToken).ConfigureAwait(false); + List runs = [first, second]; + return new ProtocolOutcome(QualificationOf(runs), runs); + } + + /// The outcome file: passed only when every trial passed; measures from the broad universe, the one traded live. + public static Qualification QualificationOf(IReadOnlyList runs) + { + ArgumentNullException.ThrowIfNull(runs); + var main = runs[0]; + var passed = runs.All(r => r.Report.Passed); + var it = CultureInfo.GetCultureInfo("it-IT"); + string verdict; + if (passed) + { + verdict = "Protocollo superato su entrambi gli universi: parte in simulazione."; + } + else + { + var failures = runs.Where(r => !r.Report.Passed) + .Select(r => r.Name + ": " + string.Join("; ", r.Report.Checks.Where(c => c.Passed == false).Select(c => Describe(c, it)))); + verdict = "Protocollo non superato. " + string.Join(". ", failures) + ". Resta in osservazione."; + } + + var taker = PaperOptions.Okx().TakerFee; + return new Qualification + { + Passed = passed, + Verdict = verdict, + Report = ReportPath, + PlanHash = TrialPlan.Hash(main.Plan.ToJson()), + MaxDrawdown = Math.Round((decimal)main.Report.Metrics.MaxDrawdown, 4), + ExpectedSlippageBp = main.ExpectedSlippageBp, + RoundTripCost = Math.Round(2m * (taker + (main.ExpectedSlippageBp / 10_000m)), 6), + ExpectedPathP5 = [.. PathBootstrap.Percentile(main.DailyReturns, PathDays, 0.05, PathBootstrap.Paths, PathBootstrap.Block, PathBootstrap.Seed) + .Select(x => Math.Round(x, 4))], + }; + } + + /// A failed check in plain Italian, e.g. "calo massimo 38,2%, oltre il 25%". + public static string Describe(ReportCheck check, IFormatProvider culture) + { + ArgumentNullException.ThrowIfNull(check); + var value = check.Value is not { } v ? "non calcolabile" + : check.Unit switch + { + "percent" => string.Create(culture, $"{v * 100:0.0}%"), + "money" => string.Create(culture, $"{v:#,0.00} €"), + "count" => string.Create(culture, $"{v:0}"), + _ => string.Create(culture, $"{v:0.000}"), + }; + var threshold = check.Unit == "percent" ? string.Create(culture, $"{check.Threshold * 100:0}%") : string.Create(culture, $"{check.Threshold:0.00}"); + return $"{check.Name.ToLower(CultureInfo.GetCultureInfo("it-IT"))} {value}, serve {check.Comparison} {threshold}"; + } + + private static async Task RunOneAsync(IStrategyFamily family, string name, string description, IReadOnlyList symbols, DateOnly to, + DateTimeOffset now, string researchRoot, PriorTrials prior, Action? progress, CancellationToken cancellationToken) + { + var request = new TrialRequest(family.Id, TrialSources.Research, EuroHistory.Venue, symbols, From, to, Capital, null); + var (plan, _) = Laboratory.BuildPlan(request, now); + var json = plan.ToJson(); + var hash = TrialPlan.Hash(json); + var data = await Laboratory.LoadResearchAsync(researchRoot, plan.Data, cancellationToken).ConfigureAwait(false); + + // The id comes from the plan, so the same plan on the same data writes the same report. + var id = new Guid(SHA256.HashData(Encoding.UTF8.GetBytes(hash)).AsSpan(0, 16)); + var (report, _) = TrialRunner.Run(id, plan, hash, family, data, prior, now, text => progress?.Invoke(name + ": " + text), cancellationToken); + + // The chosen variant again, for its daily returns and the slippage of what it traded. + var variant = ParameterGrid.Of(family, plan.Grid).First(v => v.ToString() == report.Chosen); + var options = StrategyPipeline.For(family.Id, data, new BacktestOptions { InitialCapital = plan.InitialCapital, From = plan.Data.From, To = plan.Data.To }, plan.ReferenceShare); + var run = BacktestEngine.Run(family.Create(variant.Parameters), data, options); + var specs = data.Instruments.ToDictionary(i => i.Symbol, i => i.Spec, StringComparer.Ordinal); + var traded = run.Trades.Sum(t => t.Invested); + var slippage = traded > 0m + ? Math.Round(run.Trades.Sum(t => t.Invested * PaperBroker.SlippageBp(specs.GetValueOrDefault(t.Symbol))) / traded, 2) + : PaperBroker.SlippageBp(null); + return new ProtocolRun(name, description, plan, report, run.DailyReturns(), slippage); + } + + private static async Task HasBarsAsync(ResearchStore store, string symbol, DateOnly to, CancellationToken cancellationToken) + { + var from = new DateTimeOffset(From.ToDateTime(TimeOnly.MinValue), TimeSpan.Zero); + var end = new DateTimeOffset(to.AddDays(1).ToDateTime(TimeOnly.MinValue), TimeSpan.Zero); + await foreach (var _ in store.ReadAsync(EuroHistory.Venue, symbol, Timeframes.Day, from, end, cancellationToken).ConfigureAwait(false)) + { + return true; + } + + return false; + } +} + +/// +/// The expected path of SPEC 12.3: a stationary block bootstrap of the backtest's daily returns (blocks keep the +/// clustering of volatility), and for each day from the start the given percentile of the cumulative result. The seed +/// is fixed, so the same returns always give the same path. +/// +public static class PathBootstrap +{ + public const int Paths = 2000; + + public const int Block = 20; + + public const int Seed = 20_260_101; + + public static IReadOnlyList Percentile(IReadOnlyList returns, int days, double percentile, int paths, int block, int seed) + { + ArgumentNullException.ThrowIfNull(returns); + ArgumentOutOfRangeException.ThrowIfLessThan(days, 1); + ArgumentOutOfRangeException.ThrowIfLessThan(paths, 1); + ArgumentOutOfRangeException.ThrowIfLessThan(block, 1); + if (returns.Count == 0) + { + return [.. Enumerable.Repeat(0d, days)]; + } + + var random = new Random(seed); + var cumulative = new double[days][]; + for (var d = 0; d < days; d++) + { + cumulative[d] = new double[paths]; + } + + for (var p = 0; p < paths; p++) + { + var value = 1d; + var index = random.Next(returns.Count); + for (var d = 0; d < days; d++) + { + // A new block starts with probability 1/block: blocks of geometric length, mean "block" days. + if (d > 0 && random.NextDouble() < 1d / block) + { + index = random.Next(returns.Count); + } + else if (d > 0) + { + index = (index + 1) % returns.Count; + } + + value *= 1d + returns[index]; + cumulative[d][p] = value - 1d; + } + } + + var result = new double[days]; + for (var d = 0; d < days; d++) + { + Array.Sort(cumulative[d]); + var rank = Math.Clamp((int)Math.Floor(percentile * (paths - 1)), 0, paths - 1); + result[d] = cumulative[d][rank]; + } + + return result; + } +} + +/// +/// The outcome files of docs/backtests/, built into the program: loaded at the first start, they decide +/// whether a base strategy starts in simulation or stays in observation (SPEC 12.3). +/// +public static class BacktestOutcomes +{ + public const string ResourcePrefix = "Anubis.Backtests."; + + public static Qualification? Find(string strategy) + { + using var stream = typeof(BacktestOutcomes).Assembly.GetManifestResourceStream(ResourcePrefix + strategy + ".outcome.json"); + return stream is null ? null : JsonSerializer.Deserialize(stream, Qualification.Json); + } + + private static readonly JsonSerializerOptions Indented = new(Qualification.Json) { WriteIndented = true }; + + public static string ToJson(Qualification qualification) => JsonSerializer.Serialize(qualification, Indented) + "\n"; +} + +/// +/// The stock strategies wait for the Alpaca history (SPEC 8.2, 8.3): without the keys there are no adjusted daily +/// bars of the S&P 500 members, so the protocol cannot run and they stay in observation. The outcome file says so. +/// +public static class EquityProtocol +{ + public static readonly IReadOnlyList Explanation = + [ + "Per provare le strategie sulle azioni servono gli storici giornalieri di Alpaca, con frazionamenti e dividendi, che si scaricano solo con le chiavi di un conto Alpaca (anche dimostrativo). Le chiavi non sono ancora state inserite.", + "Finché manca il backtest la strategia resta in osservazione: guarda il mercato e scrive le sue decisioni nel diario, ma non opera, nemmeno in simulazione.", + "Quando avrai inserito le chiavi nelle Impostazioni, il bot raccoglie gli storici. Poi dal Laboratorio lanci la prova della strategia con l'origine \"barre raccolte\": se supera le soglie di SPEC 12.2, la porti in simulazione dalla pagina Strategie.", + ]; + + public static Qualification Waiting(string strategy) => new() + { + Passed = false, + Verdict = "In attesa delle chiavi Alpaca: senza lo storico delle azioni il backtest non si può fare, quindi la strategia resta in osservazione.", + Report = "docs/backtests/" + strategy + ".md", + }; +} diff --git a/src/Anubis/Backtest/ProtocolReport.cs b/src/Anubis/Backtest/ProtocolReport.cs new file mode 100644 index 0000000..798d1e4 --- /dev/null +++ b/src/Anubis/Backtest/ProtocolReport.cs @@ -0,0 +1,206 @@ +using System.Globalization; +using System.Text; +using Anubis.Trading; + +namespace Anubis.Backtest; + +/// +/// The report of docs/backtests/ in plain Italian (SPEC 12.2 point 6): for each trial the figures of the chosen +/// variant against buy and hold, by year, at single, double and triple costs, the validations, the checks and what +/// the backtest does not model. It is written whatever the outcome. +/// +public static class ProtocolReport +{ + private static readonly CultureInfo It = CultureInfo.GetCultureInfo("it-IT"); + + public static string Markdown(ProtocolOutcome outcome, string title, IReadOnlyList dataNotes, string command) + { + ArgumentNullException.ThrowIfNull(outcome); + ArgumentNullException.ThrowIfNull(dataNotes); + var q = outcome.Qualification; + var first = outcome.Runs[0].Report; + var b = new StringBuilder(); + b.Append("# Backtest di `").Append(first.Strategy).Append("` (").Append(first.StrategyName).AppendLine(")").AppendLine(); + b.Append("**Esito: ").Append(q.Passed ? "superato" : "non superato").Append(".** ").AppendLine(q.Verdict).AppendLine(); + b.AppendLine(Text($"Calcolato il {first.ComputedAt:yyyy-MM-dd} con la versione {first.StrategyVersion} della strategia. Soglie, griglia dei parametri, costi e date erano fissati prima del calcolo (SPEC 12.2) e non sono stati cambiati dopo.")) + .AppendLine(); + b.Append("Per rifarlo: `").Append(command).AppendLine("`. Il file dell'esito, letto dal bot al primo avvio, è `docs/backtests/" + first.Strategy + ".outcome.json`.").AppendLine(); + + b.AppendLine("## Dati").AppendLine(); + foreach (var note in dataNotes) + { + b.Append("- ").AppendLine(note); + } + + b.AppendLine(); + b.AppendLine("## Riepilogo").AppendLine(); + b.AppendLine("| Prova | Variante scelta | Rendimento annuo | Calo massimo | Netto a costi doppi | DSR | PBO | Esito |"); + b.AppendLine("|---|---|---:|---:|---:|---:|---:|---|"); + foreach (var run in outcome.Runs) + { + var r = run.Report; + var doubled = r.Costs.FirstOrDefault(c => c.Multiplier == 2m); + b.Append("| ").Append(run.Name).Append(" | `").Append(r.Chosen).Append("` | ").Append(Pct(r.Metrics.AnnualReturn)).Append(" | ").Append(Pct(-r.Metrics.MaxDrawdown)) + .Append(" | ").Append(doubled is null ? "—" : Money(doubled.NetResult, r.Currency)).Append(" | ").Append(r.DeflatedSharpe.Value.ToString("0.000", It)) + .Append(" | ").Append(r.Overfitting is { } o ? o.Probability.ToString("0.000", It) : "—").Append(" | ").Append(r.Passed ? "superata" : "non superata").AppendLine(" |"); + } + + b.AppendLine(); + b.AppendLine("## Cosa conserva il bot").AppendLine(); + b.AppendLine(Text($"- Calo massimo del backtest: {Pct(-(double)q.MaxDrawdown)}. Dal vivo, un calo oltre 1,5 volte questo riporta la strategia indietro (SPEC 12.3).")); + b.AppendLine(Text($"- Scostamento di prezzo previsto: {q.ExpectedSlippageBp.ToString("0.##", It)} punti base; costo stimato di andata e ritorno {Pct((double)q.RoundTripCost, 2)}.")); + if (q.ExpectedPathP5.Count > 0) + { + b.AppendLine(Text($"- Percorso atteso al 5° percentile (ricampionamento a blocchi dei rendimenti giornalieri): {Pct(q.ExpectedPathP5[Math.Min(29, q.ExpectedPathP5.Count - 1)])} dopo 30 giorni, {Pct(q.ExpectedPathP5[Math.Min(89, q.ExpectedPathP5.Count - 1)])} dopo 90, {Pct(q.ExpectedPathP5[^1])} dopo {q.ExpectedPathP5.Count}. Venti sedute di fila sotto questo percorso riportano la strategia indietro.")); + } + + b.AppendLine(); + foreach (var run in outcome.Runs) + { + Trial(b, run); + } + + b.AppendLine("## Cosa il backtest non rappresenta").AppendLine(); + foreach (var line in first.Limitations) + { + b.Append("- ").AppendLine(line); + } + + return b.ToString(); + } + + /// The report of a strategy whose backtest cannot run yet, with what it waits for. + public static string Waiting(string strategy, string name, Qualification qualification, IReadOnlyList lines) + { + ArgumentNullException.ThrowIfNull(qualification); + ArgumentNullException.ThrowIfNull(lines); + var b = new StringBuilder(); + b.Append("# Backtest di `").Append(strategy).Append("` (").Append(name).AppendLine(")").AppendLine(); + b.Append("**Esito: in attesa.** ").AppendLine(qualification.Verdict).AppendLine(); + foreach (var line in lines) + { + b.AppendLine(line).AppendLine(); + } + + return b.ToString(); + } + + private static void Trial(StringBuilder b, ProtocolRun run) + { + var r = run.Report; + var m = r.Metrics; + var h = r.Benchmark; + // Validations and variants keep the Sharpe of one session: shown per year, like the metrics above. + var annual = Math.Sqrt(r.PeriodsPerYear); + b.Append("## ").AppendLine(run.Name).AppendLine(); + b.AppendLine(run.Description).AppendLine(); + b.AppendLine(Text($"Dal {m.From:dd/MM/yyyy} al {m.To:dd/MM/yyyy}, {r.Sessions} chiusure giornaliere, capitale iniziale {Money(m.InitialCapital, r.Currency)}. Variante scelta: `{r.Chosen}` (la zona della griglia dove il risultato cambia meno, non la migliore). Impronta del piano `{r.PlanHash[..12]}`, dei dati `{r.DataFingerprint[..12]}`.")) + .AppendLine(); + + b.AppendLine("| | Strategia | Comprare e tenere |").AppendLine("|---|---:|---:|"); + Row(b, "Risultato netto", Money(m.NetResult, r.Currency), Money(h.NetResult, r.Currency)); + Row(b, "Rendimento totale", Pct(m.TotalReturn), Pct(h.TotalReturn)); + Row(b, "Rendimento annuo", Pct(m.AnnualReturn), Pct(h.AnnualReturn)); + Row(b, "Volatilità annua", Pct(m.Volatility), Pct(h.Volatility)); + Row(b, "Calo massimo", Pct(-m.MaxDrawdown) + Text($" ({m.DrawdownPeak:MM/yyyy} → {m.DrawdownLow:MM/yyyy})"), Pct(-h.MaxDrawdown)); + Row(b, "Sharpe annuo", Num(m.Sharpe), Num(h.Sharpe)); + Row(b, "Operazioni chiuse", m.Trades.ToString(It), h.Trades.ToString(It)); + Row(b, "Operazioni in guadagno", Pct(m.WinRate), "—"); + Row(b, "Rapporto guadagni/perdite", m.ProfitFactor is { } pf ? Num(pf) : "—", "—"); + Row(b, "Commissioni pagate", Money(m.Fees, r.Currency), Money(h.Fees, r.Currency)); + Row(b, "Tempo investito", Pct(m.Exposure), Pct(h.Exposure)); + b.AppendLine(); + + b.AppendLine("**Per anno**").AppendLine(); + b.AppendLine("| Anno | Strategia | Comprare e tenere |").AppendLine("|---|---:|---:|"); + var held = h.Years.ToDictionary(y => y.Year); + foreach (var y in m.Years) + { + Row(b, y.Year.ToString(CultureInfo.InvariantCulture), Pct(y.Return), held.TryGetValue(y.Year, out var hy) ? Pct(hy.Return) : "—"); + } + + b.AppendLine(); + b.AppendLine("**Sensibilità ai costi** (commissioni e scostamento moltiplicati insieme)").AppendLine(); + b.AppendLine("| Costi | Risultato netto | Rendimento totale | Sharpe annuo | Calo massimo | Commissioni |").AppendLine("|---|---:|---:|---:|---:|---:|"); + foreach (var c in r.Costs) + { + b.AppendLine(Text($"| ×{c.Multiplier:0} | {Money(c.NetResult, r.Currency)} | {Pct(c.TotalReturn)} | {Num(c.Sharpe)} | {Pct(-c.MaxDrawdown)} | {Money(c.Fees, r.Currency)} |")); + } + + b.AppendLine(); + b.AppendLine("**Convalide**").AppendLine(); + if (r.WalkForward is { } wf) + { + b.AppendLine(Text($"- Finestre che avanzano ({wf.Windows.Count}, con {wf.Purge} giorni di stacco): rendimento fuori campione {Pct(wf.Return)}, Sharpe annuo {Num(wf.Sharpe * annual)}.")); + } + + if (r.Combinatorial is { } cv) + { + b.AppendLine(Text($"- K-fold purgato combinatorio ({cv.Groups} gruppi, {cv.TestGroups} di prova, {cv.Splits} divisioni): Sharpe annuo medio dei percorsi {Num(cv.MeanSharpe * annual)}, peggiore {Num(cv.WorstSharpe * annual)}, percorsi positivi {Pct(cv.PositiveShare)}.")); + } + + b.AppendLine(Text($"- Deflated Sharpe Ratio {Num(r.DeflatedSharpe.Value)}, contando {r.TrialsCounted} varianti provate in tutto.")); + b.AppendLine(r.Overfitting is { } pbo + ? Text($"- Probabilità di sovra-adattamento (PBO) {Num(pbo.Probability)}, su {pbo.Combinations} combinazioni di {pbo.Variants} varianti.") + : "- Probabilità di sovra-adattamento: non calcolabile con una sola variante."); + b.AppendLine(Text($"- Sguardo in avanti: {r.LookAhead.Decisions} decisioni rifatte tagliando i dati in {r.LookAhead.Cutoffs.Count} punti, {r.LookAhead.Changed} cambiate.")); + b.AppendLine(); + + b.AppendLine("**Soglie**").AppendLine(); + b.AppendLine("| Controllo | Valore | Soglia | Esito |").AppendLine("|---|---:|---:|---|"); + foreach (var c in r.Checks) + { + var value = c.Value is not { } v ? "—" : c.Unit switch + { + "percent" => Pct(v), + "money" => Money((decimal)v, r.Currency), + "count" => v.ToString("0", It), + _ => v.ToString("0.000", It), + }; + var threshold = c.Unit == "percent" ? Pct(c.Threshold, 0) : c.Unit == "money" ? Money((decimal)c.Threshold, r.Currency) : c.Threshold.ToString("0.##", It); + b.AppendLine(Text($"| {c.Name} | {value} | {c.Comparison} {threshold} | {(c.Passed is null ? "non si applica" : c.Passed.Value ? "superato" : "**non superato**")} |")); + } + + b.AppendLine(); + b.AppendLine("**Varianti della griglia**").AppendLine(); + b.AppendLine("| Parametri | Sharpe annuo | Stabilità della zona | Risultato netto | Calo massimo | Operazioni |").AppendLine("|---|---:|---:|---:|---:|---:|"); + foreach (var v in r.Variants) + { + b.AppendLine(Text($"| `{v.Parameters}`{(v.Chosen ? " (scelta)" : "")} | {Num(v.Sharpe * annual)} | {Num(v.PlateauScore * annual)} | {Money(v.NetResult, r.Currency)} | {Pct(-v.MaxDrawdown)} | {v.Trades} |")); + } + + if (r.Notes.Count > 0) + { + b.AppendLine(); + b.AppendLine("**Interventi di correttivi e rischio sulla variante scelta**").AppendLine(); + b.AppendLine("| Motivo | Volte |").AppendLine("|---|---:|"); + foreach (var (reason, count) in r.Notes.OrderByDescending(n => n.Value).ThenBy(n => n.Key, StringComparer.Ordinal)) + { + b.AppendLine(Text($"| `{reason}` | {count} |")); + } + } + + b.AppendLine(); + b.AppendLine("**Strumenti**").AppendLine(); + b.AppendLine("| Coppia | Prima barra | Ultima barra | Barre |").AppendLine("|---|---|---|---:|"); + foreach (var i in r.Instruments) + { + b.AppendLine(Text($"| {i.Symbol} | {i.First:yyyy-MM-dd} | {i.Last:yyyy-MM-dd} | {i.Bars} |")); + } + + b.AppendLine(); + } + + private static void Row(StringBuilder b, string label, string strategy, string benchmark) => + b.Append("| ").Append(label).Append(" | ").Append(strategy).Append(" | ").Append(benchmark).AppendLine(" |"); + + private static string Pct(double fraction, int decimals = 1) => + (fraction < 0 ? "−" : "") + Math.Abs(fraction * 100).ToString("N" + decimals.ToString(CultureInfo.InvariantCulture), It) + "%"; + + private static string Num(double value) => (value < 0 ? "−" : "") + Math.Abs(value).ToString("0.00", It); + + private static string Money(decimal value, string currency) => + (value < 0 ? "−" : "") + Math.Abs(value).ToString("#,0.00", It) + " " + (currency == "EUR" ? "€" : currency); + + private static string Text(FormattableString text) => text.ToString(It); +} diff --git a/src/Anubis/Backtest/Report.cs b/src/Anubis/Backtest/Report.cs index f4e627c..94201cc 100644 --- a/src/Anubis/Backtest/Report.cs +++ b/src/Anubis/Backtest/Report.cs @@ -106,9 +106,10 @@ public static class TrialRunner ArgumentNullException.ThrowIfNull(data); ArgumentNullException.ThrowIfNull(prior); var watch = Stopwatch.StartNew(); - if (data.Sessions.Count < 30) + var sessions = data.Sessions.Count(d => d >= plan.Data.From && d <= plan.Data.To); + if (sessions < 30) { - throw new InvalidOperationException($"Dati insufficienti: {data.Sessions.Count} sedute, ne servono almeno 30."); + throw new InvalidOperationException($"Dati insufficienti: {sessions} sedute, ne servono almeno 30."); } var variants = ParameterGrid.Of(family, plan.Grid); @@ -118,7 +119,7 @@ public static class TrialRunner } // The strategy goes through its correctives and the risk engine; buy and hold, the comparison, never does. - var plain = new BacktestOptions { InitialCapital = plan.InitialCapital }; + var plain = new BacktestOptions { InitialCapital = plan.InitialCapital, From = plan.Data.From, To = plan.Data.To }; var options = StrategyPipeline.For(family.Id, data, plain, plan.ReferenceShare); var runs = new BacktestRun[variants.Count]; for (var i = 0; i < variants.Count; i++) @@ -204,7 +205,7 @@ public static class TrialRunner data.Fingerprint(), data.Currency, data.PeriodsPerYear, - data.Sessions.Count, + sessions, [.. data.Instruments.Select(i => new ReportInstrument(i.Symbol, i.Spec.Venue, i.Bars.Count > 0 ? i.Bars[0].Day : null, i.Bars.Count > 0 ? i.Bars[^1].Day : null, i.Bars.Count, i.Actions.Count, i.Source))], chosen.ToString(), @@ -258,7 +259,12 @@ public static class TrialRunner result.Add("Strategia d'esempio: correttivi e limiti di rischio non si applicano, ogni intenzione passa così com'è."); } - if (plan.Data.Source == TrialSources.Research) + if (plan.Data.Source == TrialSources.Research && plan.Data.Venue == EuroHistory.Venue) + { + result.Add("Prezzi dello storico Binance in USDT convertiti in euro con il cambio BCE del giorno, contando un USDT come un dollaro: è un'approssimazione delle coppie in euro di OKX. " + + "Su questo storico lo spread non si può misurare: l'universo sono le monete con più scambi, passo di prezzo e quantità minima sono stime, i costi sono quelli di OKX."); + } + else if (plan.Data.Source == TrialSources.Research) { result.Add("Prezzi dello storico Binance in USDT: passo di prezzo e quantità minima sono stime, i costi sono quelli di OKX."); } diff --git a/src/Anubis/Backtest/Trials.cs b/src/Anubis/Backtest/Trials.cs index 62bdc38..96cd14a 100644 --- a/src/Anubis/Backtest/Trials.cs +++ b/src/Anubis/Backtest/Trials.cs @@ -17,8 +17,19 @@ public static class TrialSources public const string Research = "research"; } -/// The data a trial reads: one venue, a few instruments, a period (both days included). -public sealed record TrialData(string Source, string Venue, IReadOnlyList Symbols, DateOnly From, DateOnly To); +/// +/// The data a trial reads: one venue, a few instruments, a period (both days included). The bars of the +/// before are read too, only so the indicators and the listing age are ready on +/// the first day: the simulation, the measures and buy and hold start at . +/// +public sealed record TrialData(string Source, string Venue, IReadOnlyList Symbols, DateOnly From, DateOnly To) +{ + /// Calendar days of history before the start: a year of 252 sessions, a 360-day lookback, 400 days of listing. + public const int WarmupDays = 450; + + /// The first day read from the history. + public DateOnly HistoryStart() => From.AddDays(-WarmupDays); +} /// /// The thresholds of SPEC 12.2, fixed in the plan before the trial runs: changing them after seeing the results needs diff --git a/src/Anubis/Trading/TradingService.cs b/src/Anubis/Trading/TradingService.cs index 48ee059..b3929de 100644 --- a/src/Anubis/Trading/TradingService.cs +++ b/src/Anubis/Trading/TradingService.cs @@ -216,6 +216,7 @@ public sealed partial class TradingService : IDisposable var alpaca = await AlpacaKeysAsync(cancellationToken).ConfigureAwait(false); var okx = await OkxKeysAsync(cancellationToken).ConfigureAwait(false); var accounts = await AccountSetup.EnsureAsync(_db, _clock, alpaca is not null, okx is not null, cancellationToken).ConfigureAwait(false); + await LoadOutcomesAsync(cancellationToken).ConfigureAwait(false); await RefreshSpecsAsync(cancellationToken).ConfigureAwait(false); await RefreshLimitsAsync(cancellationToken).ConfigureAwait(false); foreach (var account in accounts) @@ -260,6 +261,21 @@ public sealed partial class TradingService : IDisposable await GuardAsync("patrimonio", () => PhotographAsync(cancellationToken), cancellationToken).ConfigureAwait(false); } + /// + /// The backtest outcomes built into the program (docs/backtests): at the first start a base strategy that passed + /// moves to simulation, one that did not stays in observation with the reason (SPEC 12.3). + /// + public async Task LoadOutcomesAsync(CancellationToken cancellationToken) + { + foreach (var (strategy, _, _) in StrategyNames.Base) + { + if (Backtest.BacktestOutcomes.Find(strategy) is { } outcome) + { + await Stages.ApplyQualificationAsync(strategy, outcome, cancellationToken).ConfigureAwait(false); + } + } + } + public IReadOnlyList Jobs() => [ new ScheduledJob("riconciliazione", "Riconciliazione dei conti", Schedule.Every(TimeSpan.FromMinutes(5)), ReconcileAllAsync), diff --git a/tests/Anubis.Tests/LaboratoryTests.cs b/tests/Anubis.Tests/LaboratoryTests.cs index 2ddfcc7..6b962e9 100644 --- a/tests/Anubis.Tests/LaboratoryTests.cs +++ b/tests/Anubis.Tests/LaboratoryTests.cs @@ -209,7 +209,8 @@ public sealed class LaboratoryTests var data = await Laboratory.LoadAsync(null!, SystemClock.Instance, root, new TrialData(TrialSources.Research, Venues.Binance, ["ETHUSDT"], From.AddDays(10), From.AddDays(29)), CancellationToken.None).ConfigureAwait(false); Assert.AreEqual("USDT", data.Currency); - Assert.HasCount(20, data.Sessions); + Assert.HasCount(30, data.Sessions, "the days before the start are read too, for the indicators"); + Assert.AreEqual(20, data.Sessions.Count(d => d >= From.AddDays(10)), "the trial itself covers its 20 days"); Assert.AreEqual("ETH", data["ETHUSDT"]!.Spec.BaseCurrency); Assert.IsTrue(data.IsCrypto); } diff --git a/tests/Anubis.Tests/ProtocolTests.cs b/tests/Anubis.Tests/ProtocolTests.cs new file mode 100644 index 0000000..5ed19da --- /dev/null +++ b/tests/Anubis.Tests/ProtocolTests.cs @@ -0,0 +1,216 @@ +using Anubis.Backtest; +using Anubis.Data; +using Anubis.Platform; +using Anubis.Trading; + +namespace Anubis.Tests; + +/// +/// The honest backtests of the base strategies (issue #44; SPEC 8.1, 12.2, 12.3): the Binance history in euro, the +/// protocol on both universes of crypto_trend, the expected path, the report and the outcome files read at the first +/// start. +/// +[TestClass] +public sealed class ProtocolTests +{ + private static readonly CancellationToken _none = CancellationToken.None; + + private static ResearchBar Bar(DateOnly day, decimal close) => + new(new DateTimeOffset(day.ToDateTime(TimeOnly.MinValue), TimeSpan.Zero), close, close * 1.01m, close * 0.99m, close, 100m) { QuoteVolume = close * 100m }; + + private static FxRate Usd(DateOnly day, decimal rate) => new("EUR", "USD", day, rate, "ecb"); + + [TestMethod] + public void UsdtPricesBecomeEurosWithTheRateOfTheDayOrTheLastBefore() + { + var friday = new DateOnly(2024, 3, 1); + var bars = new[] { Bar(friday, 110m), Bar(friday.AddDays(1), 120m), Bar(friday.AddDays(3), 100m) }; + var rates = new[] { Usd(friday, 1.10m), Usd(friday.AddDays(3), 1.25m) }; + + var euro = EuroHistory.Convert(bars, rates); + + Assert.HasCount(3, euro); + Assert.AreEqual(100m, euro[0].Close); + Assert.AreEqual(109.090909090909m, euro[1].Close); // Saturday: Friday's rate. + Assert.AreEqual(80m, euro[2].Close); + Assert.AreEqual(100m, euro[0].Volume, "the volume is in coins: it does not change"); + Assert.AreEqual(10_000m, euro[0].QuoteVolume); + Assert.AreEqual("BTCEUR", EuroHistory.SymbolOf("BTCUSDT")); + } + + [TestMethod] + public void ABarWithoutARecentRateIsDropped() + { + var day = new DateOnly(2024, 3, 1); + var bars = Enumerable.Range(-1, EuroHistory.MaxRateAgeDays + 3).Select(i => Bar(day.AddDays(i), 1m)).ToList(); + + var euro = EuroHistory.Convert(bars, [Usd(day, 1m)]); + + Assert.HasCount(EuroHistory.MaxRateAgeDays + 1, euro, "before the first rate and after a week without rates there is no price in euro"); + Assert.AreEqual(day, DateOnly.FromDateTime(euro[0].OpensAt.UtcDateTime)); + Assert.AreEqual(day.AddDays(EuroHistory.MaxRateAgeDays), DateOnly.FromDateTime(euro[^1].OpensAt.UtcDateTime)); + } + + [TestMethod] + public void AHoleOfMoreThanAWeekEndsTheSeriesBecauseWhatFollowsIsAnotherCoin() + { + var day = new DateOnly(2022, 5, 10); + var bars = new[] { Bar(day, 1m), Bar(day.AddDays(1), 0.0001m), Bar(day.AddDays(1 + EuroHistory.MaxGapDays + 1), 5m), Bar(day.AddDays(2 + EuroHistory.MaxGapDays + 1), 6m) }; + var rates = Enumerable.Range(0, 20).Select(i => Usd(day.AddDays(i), 1m)).ToList(); + + var euro = EuroHistory.Convert(bars, rates); + + Assert.HasCount(2, euro, "the relaunch under the same ticker is not the coin that collapsed"); + Assert.AreEqual(0.0001m, euro[^1].Close); + } + + [TestMethod] + public void TheExpectedPathIsTheCompoundedPercentileAndRepeatsWithTheSameSeed() + { + var steady = PathBootstrap.Percentile([0.01, 0.01, 0.01], 3, 0.05, 50, 5, 1); + Assert.AreEqual(0.01, steady[0], 1e-12); + Assert.AreEqual(0.030301, steady[2], 1e-12); + + var random = new Random(7); + var returns = Enumerable.Range(0, 500).Select(_ => (random.NextDouble() - 0.48) * 0.06).ToArray(); + var p5 = PathBootstrap.Percentile(returns, 100, 0.05, 500, 20, 3); + var median = PathBootstrap.Percentile(returns, 100, 0.50, 500, 20, 3); + CollectionAssert.AreEqual(p5.ToList(), PathBootstrap.Percentile(returns, 100, 0.05, 500, 20, 3).ToList()); + Assert.IsTrue(p5.Zip(median).All(x => x.First <= x.Second), "the 5th percentile is never above the median"); + Assert.IsLessThan(p5[9], p5[99], "the bad tail widens with time"); + } + + [TestMethod] + public async Task TheProtocolRunsBothUniversesAndWritesTheReportWhateverTheOutcome() + { + var folder = TempFolder.Create("protocol"); + try + { + var store = new ResearchStore(folder); + var random = new Random(11); + var start = new DateOnly(2017, 8, 1); + var end = new DateOnly(2020, 12, 31); + foreach (var coin in new[] { "BTC", "ETH", "SOL", "XRP", "LTC" }) + { + var price = 100m; + var bars = new List(); + for (var day = start; day <= end; day = day.AddDays(1)) + { + price *= 1m + (decimal)((random.NextDouble() - 0.49) * 0.08); + bars.Add(Bar(day, Math.Round(price, 6)) with { Volume = 1_000_000m }); + } + + await store.WriteAsync(Venues.Binance, coin + "USDT", Timeframes.Day, bars, _none); + } + + var rates = new List(); + for (var day = start; day <= end; day = day.AddDays(1)) + { + if (day.DayOfWeek is not (DayOfWeek.Saturday or DayOfWeek.Sunday)) + { + rates.Add(Usd(day, 1.15m)); + } + } + + var converted = await EuroHistory.ConvertAllAsync(store, rates, _none); + Assert.HasCount(5, converted); + + var outcome = await CryptoTrendProtocol.RunAsync(folder, end, new DateTimeOffset(2021, 1, 2, 0, 0, 0, TimeSpan.Zero), null, _none); + + Assert.HasCount(2, outcome.Runs); + Assert.HasCount(5, outcome.Runs[0].Report.Instruments); + CollectionAssert.AreEqual(CryptoTrendProtocol.Narrow.ToList(), outcome.Runs[1].Report.Instruments.Select(i => i.Symbol).ToList()); + Assert.AreEqual("EUR", outcome.Runs[0].Report.Currency); + Assert.AreEqual(outcome.Runs[0].Report.Variants.Count + outcome.Runs[1].Report.Variants.Count, outcome.Runs[1].Report.TrialsCounted, + "the second universe counts the variants of the first as tries"); + Assert.IsTrue(outcome.Runs.All(r => r.Report.LookAhead.Passed), "no decision changes when later data is added"); + + var q = outcome.Qualification; + Assert.AreEqual(outcome.Runs.All(r => r.Report.Passed), q.Passed); + Assert.AreEqual(CryptoTrendProtocol.ReportPath, q.Report); + Assert.HasCount(CryptoTrendProtocol.PathDays, q.ExpectedPathP5); + Assert.AreEqual(Math.Round((decimal)outcome.Runs[0].Report.Metrics.MaxDrawdown, 4), q.MaxDrawdown); + Assert.IsTrue(q.ExpectedSlippageBp is >= 5m and <= 15m); + Assert.AreEqual(2m * (0.002m + (q.ExpectedSlippageBp / 10_000m)), q.RoundTripCost); + if (!q.Passed) + { + StringAssert.Contains(q.Verdict, "Resta in osservazione"); + } + + var markdown = ProtocolReport.Markdown(outcome, "crypto_trend", ["dati di prova"], "anubis backtest base"); + foreach (var part in new[] { "# Backtest di `crypto_trend`", "## Universo ampio", "## BTC, ETH e SOL", "Comprare e tenere", "×2", "×3", "**Per anno**", + "Deflated Sharpe Ratio", "convertiti in euro con il cambio BCE", "dati di prova" }) + { + StringAssert.Contains(markdown, part); + } + + var json = BacktestOutcomes.ToJson(q); + Assert.AreEqual(q.Verdict, System.Text.Json.JsonSerializer.Deserialize(json, Qualification.Json)!.Verdict); + StringAssert.Contains(json, "\"expected_path_p5\""); + } + finally + { + TempFolder.Delete(folder); + } + } + + [TestMethod] + public void AFailedCheckIsDescribedInPlainItalian() + { + var check = new ReportCheck(ReportCheckKeys.Drawdown, "Calo massimo", 0.382, "≤", 0.25, "percent", false); + + Assert.AreEqual("calo massimo 38,2%, serve ≤ 25%", CryptoTrendProtocol.Describe(check, System.Globalization.CultureInfo.GetCultureInfo("it-IT"))); + } + + [TestMethod] + public void EveryBaseStrategyHasItsOutcomeFileBuiltIn() + { + foreach (var (strategy, _, _) in StrategyNames.Base) + { + var outcome = BacktestOutcomes.Find(strategy); + Assert.IsNotNull(outcome, strategy); + Assert.AreEqual("docs/backtests/" + strategy + ".md", outcome.Report); + Assert.IsTrue(File.Exists(Path.Combine(RepositoryRoot(), outcome.Report)), "the report of " + strategy + " is in the repository"); + } + + foreach (var strategy in new[] { StrategyNames.EquityMomentum, StrategyNames.EquityMeanrev }) + { + var outcome = BacktestOutcomes.Find(strategy)!; + Assert.IsFalse(outcome.Passed, "without the Alpaca history the stock strategies cannot pass"); + StringAssert.Contains(outcome.Verdict, "chiavi Alpaca"); + } + } + + [TestMethod] + public async Task AtTheFirstStartTheStockStrategiesWaitInObservationAndCryptoFollowsItsOutcome() + { + await using var rig = await TradingRig.CreateAsync(); + await AccountSetup.EnsureAsync(rig.Db, rig.Clock, alpacaDemo: false, okxDemo: false, _none); + var stages = new StageBook(rig.Db, rig.Clock, TradingConsent.None, (_, _, _) => Task.FromResult(false)); + foreach (var (strategy, _, _) in StrategyNames.Base) + { + await stages.ApplyQualificationAsync(strategy, BacktestOutcomes.Find(strategy)!, _none); + } + + foreach (var strategy in new[] { StrategyNames.EquityMomentum, StrategyNames.EquityMeanrev }) + { + var stage = await stages.GetAsync(strategy, _none); + Assert.AreEqual(StrategyStages.Observation, stage!.Stage); + StringAssert.Contains(stage.Qualification!.Verdict, "chiavi Alpaca"); + } + + var crypto = (await stages.GetAsync(StrategyNames.CryptoTrend, _none))!; + Assert.AreEqual(BacktestOutcomes.Find(StrategyNames.CryptoTrend)!.Passed ? StrategyStages.Simulation : StrategyStages.Observation, crypto.Stage); + } + + private static string RepositoryRoot() + { + var folder = new DirectoryInfo(AppContext.BaseDirectory); + while (folder is not null && !File.Exists(Path.Combine(folder.FullName, "Anubis.slnx"))) + { + folder = folder.Parent; + } + + return folder?.FullName ?? throw new InvalidOperationException("Repository root not found"); + } +}