38 Commits
Author SHA1 Message Date
Alby96 25e71b679f Merge pull request 'Raccolta dei dati per l'analisi e release di prova per Windows' (#96) from diagnostica into main
ci / test (postgres) (push) Failing after 5m55s
ci / test (sqlite) (push) Successful in 4m18s
ci / image (push) Skipped
2026-11-14 18:55:56 +01:00
Alby96andClaude Opus 5.5 9cf386d08b Count WebSocket upgrades as successful calls and tidy the report (refs #95)
ci / test (postgres) (pull_request) Successful in 9m14s
ci / test (sqlite) (push) Successful in 4m11s
ci / test (sqlite) (pull_request) Successful in 4m12s
ci / test (postgres) (push) Successful in 9m40s
ci / image (pull_request) Successful in 2m14s
ci / image (push) Successful in 54s
Trying the packaged release showed three things to fix: a 101 answer that opens a
WebSocket was counted as a failure, the duration buckets were too coarse around
the usual 200-500 ms of the brokers, and the report mixed CRLF and LF lines and
named SQLite only by its version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 18:05:57 +01:00
Alby96andClaude Opus 5.5 6e365673b8 Collect diagnostics for the analysis of a long run and package a Windows trial release (refs #95 #55)
ci / test (postgres) (push) Successful in 6m25s
ci / test (sqlite) (push) Successful in 4m1s
ci / test (postgres) (pull_request) Failing after 6m4s
ci / image (push) Canceled after 0s
ci / test (sqlite) (pull_request) Successful in 4m34s
ci / image (pull_request) Skipped
The 48-hour soak run at home kept being cut short by the host, never by Anubis.
The user will instead run a release for as long as they like and paste back a
report, so the bot now records what that report needs (ADR 0013).

- Every 5 minutes and at each start, a process sample goes to diagnostic_samples:
  memory, GC, threads, handles, CPU, exceptions, database size, log warnings and
  errors, and outbound HTTP calls by destination with duration buckets, read from
  the System.Net.Http meter without any new package.
- Every scheduled job run goes to job_runs. Both tables keep 30 days.
- The Markdown report (at most 60 KB, redacted, user folders masked) adds
  findings, the soak checks, starts and silences, jobs, calls, grouped log
  warnings and errors, and resources over time with their slopes. It comes from
  "anubis diagnostics", GET /api/diagnostica.md and a button in Diario e sistema.
- New {long} column token: SQLite STRICT tables reject BIGINT.
- "tasks release-win" builds the trial zip with a start script, a script that
  writes the report, and Italian instructions. Version 0.7.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 17:57:03 +01:00
Alby96 32673007a2 Merge pull request 'Prova di 48 ore: stop di Docker di 15 minuti e come evitarlo' (#94) from prova-48h-docker into main
ci / test (postgres) (push) Successful in 6m6s
ci / test (sqlite) (push) Successful in 3m14s
ci / image (push) Successful in 1m6s
2026-11-13 09:08:03 +01:00
Alby96andClaude Opus 5.5 e561245d4e Keep the 48-hour run going after Docker stopped with the Claude app (refs #53 #55)
ci / test (sqlite) (pull_request) Successful in 3m37s
ci / test (sqlite) (push) Successful in 4m1s
ci / test (postgres) (pull_request) Successful in 6m4s
ci / test (postgres) (push) Successful in 6m27s
ci / image (pull_request) Successful in 1m3s
ci / image (push) Successful in 1m5s
On 12 November at 23:37 Docker Desktop stopped for 15 minutes. It had
been started from the Claude app's shell after the PC switched off, so it
lived in the app's process job and died when the app updated itself. The
bot resumed alone and recorded the absence.

The run is not restarted from zero: a fresh start would no longer span
two US sessions before the weekend. Instead two reports are read: the 48
uninterrupted hours from 23:57, which must be regular, and the whole run
from 21:02, whose only allowed failure is that 15-minute stop. Docker is
now started through WMI, outside the app, and SVILUPPO.md says how.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 23:59:49 +01:00
Alby96 829fbd2eba Merge pull request 'Prova di 48 ore ripartita dopo lo spegnimento del PC' (#93) from prova-48h-ripartita into main
ci / test (postgres) (push) Successful in 7m4s
ci / test (sqlite) (push) Failing after 1m0s
ci / image (push) Skipped
2026-11-12 23:23:08 +01:00
Alby96andClaude Opus 5.5 f59eee8632 Restart the 48-hour run after the PC switched off (refs #53 #55)
ci / test (postgres) (pull_request) Successful in 8m29s
ci / test (sqlite) (pull_request) Successful in 4m18s
ci / image (pull_request) Successful in 1m25s
ci / test (postgres) (push) Successful in 7m25s
ci / test (sqlite) (push) Successful in 4m6s
ci / image (push) Successful in 1m32s
The first attempt ran 22 hours: on 12 November at 19:49 the PC switched
off without a clean shutdown (power or a machine hang, no bug check).
When it came back the bot resumed by itself and wrote the absence in
the diary, but a 48-hour run needs no gaps, so it started again from
zero at 21:02. STATE gives the new window and command; the first
attempt's report and database are kept under artifacts/prova-48h.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 21:03:56 +01:00
Alby96 7291f5d15b Merge pull request 'Fase 6 — Motore dal vivo in simulazione' (#92) from fase-6 into main
ci / test (postgres) (push) Successful in 10m0s
ci / test (sqlite) (push) Successful in 4m48s
ci / image (push) Successful in 1m31s
2026-11-11 22:30:53 +01:00
Alby96andClaude Opus 5.5 9a4e31aa48 Record the start of the 48-hour run and how to read its report (refs #53)
ci / test (postgres) (push) Successful in 9m13s
ci / test (sqlite) (push) Successful in 4m46s
ci / image (push) Successful in 1m50s
ci / test (sqlite) (pull_request) Successful in 4m57s
ci / test (postgres) (pull_request) Successful in 11m4s
ci / image (pull_request) Successful in 1m2s
The run started on 11 November 2026 at 21:44 Italian time in the
anubis-prova container, from the image of this branch. STATE says when
and how to make the report so that it spans both US sessions; its check
is the first step of Phase 7 (#55).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 21:49:12 +01:00
Alby96andClaude Opus 5.5 1c7205024f Count short gaps at the edges of the soak window as covered (refs #53)
The scheduler's first jobs finish about a minute after the bot starts, so
a run started exactly at the window's start was reported as covering
slightly less than 48 hours. A gap shorter than the allowed silence
(15 minutes) at either edge now counts as covered, like one in the middle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 21:40:41 +01:00
Alby96andClaude Opus 5.5 be6d5fb91f Let a late decision wait for its bars; read the soak run's continuity from jobs and absences (refs #49 #53)
Starting the 48-hour run showed two defects.

After an absence, a decision made past 12:00 UTC was written off as
"no data" at the first pulse, because the data step had not yet brought
the bars of the close: the recovered decision was lost. A late decision
now asks for its bars and waits up to 30 minutes for them. The 20-hour
absence test pulses before the bar arrives and fails without the fix.

The soak report measured coverage and silences from reconciliation
rows, but a clean round writes none, so a healthy run would have been
reported as never covered. It now reads when the scheduled jobs first
ran and last finished, the absences written at each restart, the guard
interventions, and each account's status (reconciled within the last
15 minutes and normal). The test no longer fakes a row every 5 minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 21:35:03 +01:00
Alby96andClaude Opus 5.5 2185a0b36b Fix the defects found by the Phase 6 review (refs #47 #48 #49 #51 #52 #54)
An independent review of the phase found nine defects. The ones that
could lose money or orders:

- Exits at the close of equity_meanrev are now planned orders written in
  the same transaction that claims the decision, so a restart can neither
  lose them nor send them twice. A sell with a deadline starts aggressive.
- Resume also finds a planned order not yet linked to its execution
  (restart between sending and linking) and closes it before re-planning.
- An account is "ready" only after adopting the guard's stops, checking
  the key, reconciling, resuming and aligning stops. Until then nothing is
  decided on it or sent to it, and the loop retries its startup each
  minute. The guard's note is deleted only when every stop it placed is in
  the ledger.
- Each account's turn is cut after 45 s (60 s for a retried startup), so a
  slow broker no longer silences the heartbeat until the guard kills a
  healthy engine; the beat is also withheld when the scheduler has been
  silent for 30 minutes.

Smaller ones: a stock decision made after the open is marked late; the
Telegram loops survive unexpected errors; a long absence is shown on the
Panoramica until "Ho visto" (SPEC 5.6); plain-http heartbeats are only
accepted to home-network addresses; the soak report gives a daily close
30 minutes to be decided before expecting it.

ADR 0012 point 12 records these choices; #51 and the two screen criteria
are ticked in the roadmap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 21:10:28 +01:00
Alby96andClaude Opus 5.5 3d174931e9 Add the endurance run report and record the Phase 6 choices (refs #53 #54)
"anubis soak" reads the bot's records over a time window and says whether
the run kept control. It changes nothing, so it can run beside the working
bot with docker exec. It exits with 4 when something needs a look. It
checks:
- the hours covered and silences longer than 15 minutes;
- orders or decisions made twice;
- unexplained reconciliation differences and each account's last outcome;
- absences and guard interventions;
- one crypto decision per daily close.

ADR 0012 records what the specification leaves open in this phase: the
live engine shares the backtest's decision call, run keys, recovery after
an absence, the guard and its drill seams, the diary templates, the
"senza bot" comparison, Telegram and the heartbeat, the screens and the
endurance run. ROADMAP ticks the Phase 6 items and criteria with the tests
that prove them, STATE moves on to Phase 7, and SVILUPPO explains the
guard and the report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 20:36:22 +01:00
Alby96andClaude Opus 5.5 9ae107000b Ignore late messages from a replaced engine; list only moves in the morning summary (refs #48 #52)
The guard drill failed once under load. The pipe of the killed engine
delivered a late "running" beat after the restart. The drill took the new
engine as ready and stopped it before it had cancelled the half-done sale
and put the stop back.

The same late message could have made a crash of the new engine look like
a planned stop, which the guard never restarts. Each engine the guard starts
now has a number, and messages from a replaced one are ignored. The drill
also waits until the restart's work is in the database before it checks.

The morning Telegram summary listed every job due today, including the
background ones that run every minute. It now lists only the bot's moves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 20:34:25 +01:00
Alby96andClaude Opus 5.5 e96ca70a1a Build the Panoramica and Diario e sistema screens as in the mockups (refs #51)
Panoramica now follows mockup 01. Below "Finora" and "Adesso e dopo" it
shows:
- the two account cards with the invested/cash bar and a 30-day trend;
- the four risk limits;
- the open positions by value;
- the strategies;
- the news, learning and system tiles.

On the phone it follows mockup 09: Conti as a list, four positions with
"Vedi le altre", and two limits. Outside the demo, news and learning say
they are not active yet (Phases 7 and 8). The next moves are the scheduler's
moves only, no longer every interval job.

Diario e sistema follows mockup 08:
- the diary grouped by day, with search and filter chips (plain links, no
  JavaScript) and older days on request;
- how each component is doing;
- the scheduled jobs;
- the host's processor, memory and disk;
- the technical logs as a zip, for signed-in users only.

In demo mode the screens show the values of appendix F.10. Tests cover the
demo figures, the filters and search, and the logs download.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 20:32:19 +01:00
Alby96andClaude Opus 5.5 84c540786c Send Telegram notices, answer phone commands and call the heartbeat (refs #52)
Every notice is written to the queue in the same transaction as what it
tells, then sent every ten seconds. Each has a state: pending, sent, or
failed with its error. Network failures and 429s are retried with growing
pauses (Telegram's own wait when it gives one). Refusals are final. A notice
that could not leave within a day is given up instead of arriving late. Every
fill now also reaches the phone, and so do pause, resume and restart.

Commands are read by long polling and obeyed only from the configured chat
and only when fresh, so yesterday's /pausa never pauses today. They are
/stato, /posizioni, /oggi, /pausa, /riprendi, /ferma (confirmed with
/conferma within two minutes) and /aiuto. Restarting after "Ferma tutto",
real money, limits and keys stay in the interface.

Also added:
- the heartbeat call every minute when an address is configured;
- the morning summary at 08:00, caught up within four hours;
- an optional hourly or evening status;
- a Telegram section in Impostazioni: test-before-save, a test button, the
  queue's state and the last successful heartbeat;
- on desktop, the first-run page recommends the heartbeat.

Tests run against a fake Telegram server, including the commands through
the real server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 19:53:55 +01:00
Alby96andClaude Opus 5.5 d00dfbd32f Add the guard that watches the engine and protects positions (refs #48)
The guard (SPEC 9.3, 9.4) is now the container's main process and the parent
of the engine. It listens on a named pipe for the engine's heartbeat and
startup steps, and steps in after two minutes of silence, ten minutes of a
startup that does not move, or an exit without notice. It stops the engine,
cancels our purchases and sales left open on demo accounts, puts back the
stops the positions need from the state file the engine writes every ten
seconds, writes a note, warns the user and restarts the engine without any
injected fault. The third forced restart within an hour writes the STOP
file. The container's stop is passed on to the engine, which closes in order.

At restart the engine adopts the guard's stops before reconciling, cancels
the orders it left open and plans again the exits already decided, so the
rules of SPEC 5.6 apply to them.

The acceptance test blocks the main loop right after a sale leaves: in
process on a demo-like broker, and with real processes in the drill, which
also runs inside the Docker image (tasks ci and the CI workflow). The guard
uses the ASP.NET shared runtime so both programs share /app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 19:36:32 +01:00
Alby96andClaude Opus 5.5 680cab4bb6 Recover missed work after an absence by the rules of SPEC 5.6 (refs #49)
At the start the host measures how long the bot was away from the last
job that finished, records the absence and writes it in the diary; one
longer than a day is kept for the Panoramica until the user has seen it.

A job can now say how a run missed while the bot was off is treated: the
nightly backup, the exchange rates, the market calendar and the cleanup
run once at the restart, queued, however late; a job with a deadline is
skipped past it and the diary says so. Corporate actions are read before
the reconciliation puts any stop back. A decision the engine makes late
because of an absence is written in the diary as recovered, as are the
orders it releases late; entries that moved too far or expired are
dropped with their reason.

Tests switch the bot off for 3 hours, 20 hours and 3 days with a manual
clock and check each of those rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 18:38:50 +01:00
Alby96andClaude Opus 5.5 41e067e7d1 Run the strategies live in simulation with the live engine (refs #47)
The live engine makes each decision at its time of SPEC 5.5 through the
same DecideOnce call the backtest uses, so live and backtest cannot drift:
the crypto close at 00:05 UTC, the stock signals 20 minutes after the
close, the meanrev exits 5 minutes before it, the stock orders and the
daily stops at the open. Each decision has a run key written in the same
transaction as its decision rows, planned orders and diary lines, and each
planned order's id gives the order key, so a restart never decides or
sends anything twice. Entries recovered late are dropped when the price
has moved too far.

The engine runs from the scheduler once the accounts are reconciled. The
IEX stream follows held stocks, planned orders and SPY during the session,
and the stock simulator's day orders now expire at the close.

Tests: parity with the backtest for crypto_trend (40 days) and
equity_meanrev (6 weeks), and a 14-day run with the network down, a lost
answer, a restart mid-order and the database unreachable, ending
reconciled with no duplicate orders, on SQLite and PostgreSQL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 18:28:08 +01:00
Alby96andClaude Opus 5.5 98ee504dab Write the diary from fixed templates and add the "senza bot" comparison (refs #50)
The diary needs sentences in the tone of appendix B, built only from what
happened. DiaryText holds the templates the live engine will use: fills
with the result of the trade they close, raised stops, the daily crypto
close, the stock signals, observation signals, recovery and absence. The
reason-code sentences move from the server into the library
(ReasonCatalog) so the engine and the screens share them; a test checks
that every reason constant has its sentence. The diary gets the kinds of
the Diario screen: Ordini, Decisioni, Sistema, Notizie, Apprendimento.

The comparison of SPEC 7.5 invests each capital flow on its day, 40% in
BTC/EUR and 60% in SPY adjusted and converted at the ECB rate, and never
touches it; a withdrawal sells both in the proportion they reached. Prices
are the closes of the day before, so a deposit is never a gain. Without
the SPY history (no Alpaca keys) the photograph leaves it empty rather
than show half a comparison. The Panoramica now takes the flows out of
the "Senza bot" result, as it already did for the bot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 17:46:28 +01:00
Alby96 e3c9eae65b Merge pull request 'Fase 5 — Strategie, portafoglio e rischio' (#91) from fase-5 into main
ci / test (postgres) (push) Successful in 5m34s
ci / test (sqlite) (push) Successful in 3m11s
ci / image (push) Successful in 5s
2026-11-11 17:30:08 +01:00
Alby96andClaude Opus 5.5 fc5df0146e Prepare strategies before the host is ready; stop demo servers first (refs #44 #45)
ci / test (postgres) (push) Successful in 4m39s
ci / test (sqlite) (push) Successful in 2m57s
ci / image (push) Successful in 56s
ci / test (postgres) (pull_request) Successful in 5m27s
ci / test (sqlite) (pull_request) Successful in 3m14s
ci / image (pull_request) Successful in 10s
Loading the outcomes before "ready" was not enough: on a fresh database the
strategy rows did not exist yet, so the load did nothing and CI run 220
failed the same way on PostgreSQL. The new TradingService.PrepareAsync
creates accounts and strategies, then loads the outcomes; the host runs it
before it is ready and logs a failure without blocking the interface.
StartAsync repeats it, which changes nothing.

The demo tests deleted the demo folder while the server still held its
SQLite file, which Windows refuses at random. They now stop the server
first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 17:19:15 +01:00
Alby96andClaude Opus 5.5 dc1ba4e701 Load the backtest outcomes before the host is ready (refs #44 #45)
ci / test (postgres) (push) Failing after 4m53s
ci / test (sqlite) (push) Successful in 3m4s
ci / image (push) Skipped
The trading service loaded the outcomes in the background after the host
reported "ready", so for a moment Strategie showed every strategy as
without a backtest. CI run 219 caught it on SQLite. The host now loads
them before it is ready; the call is idempotent, so the one at trading
start changes nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 17:01:25 +01:00
Alby96andClaude Opus 5.5 9ac159a7db Fix the defects of the Phase 5 review (refs #41 #42 #43 #44 #45 #46)
ci / test (postgres) (push) Successful in 3m52s
ci / test (sqlite) (push) Failing after 2m41s
ci / image (push) Skipped
An independent review found these, now fixed and tested:

- Risk limits could be raised through PUT /api/settings without the
  phrase and the password. The API now refuses risk keys.
- The reference equity counted results twice after reduced to real and
  after a demotion. Each ledger now keeps a baseline, and carried results
  stay in the ledger's currency until the reference is computed.
- An instrument could be bought by a second strategy in the same account
  (SPEC 8.6), and purchases under way did not hold their cash, so several
  purchases sent together could exceed it. Both are refused or shrunk now.
- After an automatic trip of the switch there was no way to restart from
  the interface. Rischio now shows the trip of both worlds and the restart.
- The fresh password of SPEC 12.4 could be tried without limit. Five wrong
  attempts in 15 minutes now lock it.
- Observation to simulation skipped the 90 days of live signals.
- A new build changed the qualification of a strategy with real money,
  "Ferma tutto" counted as a critical incident, raising risk ignored
  demotion conditions in progress, and weekends counted as stock sessions.
- The 12.4 confirmation now summarises capital, first purchases and limits,
  and live strategies show the gap with their simulated twin.

The backtest converts each bar with the rate fixed before it opens, the
day before, and the report now counts how often correctives and risk
changed a decision. crypto_trend is still not passed: the broad universe
now falls 32.8%. The outcome file also keeps the qualified parameters.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 16:48:41 +01:00
Alby96andClaude Opus 5.5 17d6b8db38 Record the Phase 5 choices and close the roadmap items (refs #46)
ADR 0011 collects what the specification leaves open or refines in this
phase: strategies shared by backtest and live, correctives and risk inside
the backtests, the risk engine that only refuses or shrinks, the stages as
the only way to real money, the euro history and the protocol of the base
backtests, and the honest "not passed" of crypto_trend.

ROADMAP ticks the Phase 5 items and criteria with the tests that prove
them; STATE moves to Phase 6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 16:19:35 +01:00
Alby96andClaude Opus 5.5 13f9c42f61 Add the Operazioni, Strategie and Rischio screens (refs #45)
The three screens follow mockups 03, 04 and 07.

Operazioni shows the balance of the closed trades, filters, a CSV export,
and for each trade why it was bought and sold. The reasons come from the
decision codes through fixed Italian templates.

Strategie shows the four stages towards real money, a card per strategy
with its requirements, the procedure of SPEC 12.4, and why each strategy in
observation waits. A strategy that failed its backtest cannot be moved into
the simulation.

Rischio shows the emergency switch, the automatic blocks, the use of each
limit, the fall from the peak, the scenarios and the risk events. Stop,
close and restart now go through the risk engine. A restart is refused
while the STOP file exists.

Impostazioni gains the limits, the day-trade guard and the scenarios.
Lowering a limit is always allowed. Raising a limit or the entrusted capital
while a strategy uses real money needs the phrase and the password.

The demo now holds the decisions behind its trades, the 58 days of
simulation and the indices of its curves, so it matches the mockups.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 16:14:44 +01:00
Alby96andClaude Opus 5.5 7041533ce2 Add the honest backtests of the base strategies (refs #44)
crypto_trend now has a backtest report and an outcome file, computed by
"anubis backtest base" on the Binance daily archives from 2019 to October
2026. Prices are converted from USDT to euro with the ECB rate of the day,
and delisted coins stay in the universe, so survivors do not flatter the
result. The protocol runs the broad universe and BTC, ETH and SOL, with
costs doubled and tripled and buy and hold beside. Both universes must pass.

The outcome is "not passed": the broad universe falls 29.1% (limit 25%) and
the narrow one has a probability of overfitting of 0.84 (limit 0.40). The
strategy therefore stays in observation. The stock strategies wait for the
Alpaca keys and stay in observation too. The bot reads the outcome files at
every start.

Trials now read 450 days of history before their start, only so the
indicators and the 400-day listing age are ready on the first day. Before
this, 2019 was spent warming up and BTC and ETH could not be chosen until
2020. A daily series also stops at a hole longer than a week, so the LUNA
relaunched in 2022 is not mistaken for the coin that collapsed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 16:13:23 +01:00
Alby96andClaude Opus 5.5 8def84e122 Add the risk engine, correctives and strategy stages (refs #41 #42 #43)
Every intent of a strategy now crosses a risk engine that only refuses or
shrinks: loss limits per world with the 24-hour block, the weekly halving,
the halving and the switch on the fall from the high, the size limits on
invested capital, single position, crypto share and trade risk, the
technical checks, no short sales and no leverage, the day-trade guard
(off by default). The switch, "Ferma tutto", "Ferma e chiudi", the STOP
file and the orders-per-minute anomaly stop the bot until the user
confirms a restart, and a restart after a trip measures the high from
then on.

The correctives of SPEC 8.5 (crypto regime, economic calendar) and the
risk rules also run inside the backtests of the base strategies, so the
reports show what would happen live. Each stage of a decision is written
to a new decisions table with reason codes.

The stage book is the only way to real money: an order on a real account
needs the startup consent, every requirement of SPEC 12.3 met, the typed
phrase and a fresh password and second factor. Order managers on real
accounts fail closed and ask again before each order leaves. A demoted or
revoked strategy keeps its real positions in exit only, with stops, until
they are at zero. The simulated twin stays out of totals and of the
reference equity.

Tests cover each rule of 9.1 and 9.2, with property tests for "never
larger than asked" and "never larger after a loss", and both proofs on a
fake real account: nothing reaches it without the procedure, and a
demoted strategy manages exits and stops down to zero.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 15:22:35 +01:00
Alby96andClaude Opus 5.5 2409e74b26 Add the three base strategies and their indicators (refs #38 #39 #40)
crypto_trend follows nine breakout periods with Donchian-middle stops,
volatility-targeted weights under the 35%/20% caps and a monthly top-10
universe. equity_momentum reviews on the last close of each week with the
SPY filter, inverse-volatility weights and an ATR trailing stop.
equity_meanrev buys RSI(2) dips above the 200-session average, sized for
a fixed risk, and exits at the close.

The engine gains what the rules need: exits at the same close, a stop set
at a distance from the price obtained, data-only instruments such as SPY,
the trading calendar, the reference equity of the strategy's share and the
equity history. Every strategy passes the look-ahead check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:43:19 +01:00
Alby96 a6227793c3 Merge Phase 4: backtest and laboratory (#90)
ci / test (postgres) (push) Successful in 2m58s
ci / test (sqlite) (push) Successful in 2m11s
ci / image (push) Successful in 9s
2026-11-11 14:18:18 +01:00
Alby96andClaude Opus 5.5 9db95ea79a Add the VS Code debug configuration
ci / test (postgres) (push) Successful in 3m18s
ci / test (sqlite) (push) Successful in 2m1s
ci / test (postgres) (pull_request) Successful in 3m50s
ci / test (sqlite) (pull_request) Successful in 2m26s
ci / image (push) Successful in 8s
ci / image (pull_request) Successful in 6s
Four launch entries for the server: a data folder of its own under
artifacts/ (ignored by git), the demo mode, the user's real data folder,
and attach to a running process. Each builds in Debug first and opens the
browser once the server listens. tasks.json also exposes check, test and
dev. SVILUPPO.md explains which entry to pick.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 11:53:49 +01:00
Alby96andClaude Opus 5.5 f278ff4f67 Fix the defects of the Phase 4 review (refs #34 #35 #36)
ci / test (postgres) (push) Successful in 2m41s
ci / test (sqlite) (push) Successful in 1m56s
ci / test (postgres) (pull_request) Successful in 3m45s
ci / test (sqlite) (pull_request) Successful in 2m22s
ci / image (push) Successful in 1m10s
ci / image (pull_request) Successful in 13s
- An error of the database (PostgreSQL restarting) no longer stops the
  laboratory until the next start: the queue logs it, waits 30 seconds
  on the injectable clock and tries the same trial again.
- Buy and hold, the comparison of every report, buys an instrument
  whose history starts later at its first close, with the share kept
  free for it; before, it was never bought and the others took its
  weight. Its version goes to 2.
- The comparison page reads a and b as text: a malformed id shows
  "two finished tests needed" instead of an error page.
- A trial re-checks the hash of its plan before running and refuses a
  plan that no longer matches it.
- Parameters that count days take whole numbers only (10.5 would run
  as 10 and count twice in the DSR); the capital starts from 1, as the
  message says.

ADR 0010 says so, and corrects the estimated minimum of the research
history: 5 does refuse smaller orders, like the exchanges' minimums.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:17:18 +01:00
Alby96andClaude Opus 5.5 af7528560f Record the Phase 4 choices and close the roadmap items (refs #37)
ADR 0010 writes down what the specification leaves open: the event
order of a session, the costs shared with the simulator, the pass-through
correctives and risk until Phase 5, the in-house statistics, validations
on the daily returns of the variants, the plateau rule as the minimum of
own and neighbourhood Sharpe, the sizes of walk-forward, CPCV and CSCV,
the trial count of the DSR, the plan written first and never changed,
the low-priority laboratory and the two data origins (the research
history stays in USDT; a trial uses a single currency).

ROADMAP ticks the four acceptance criteria with the tests that prove
them; STATE moves to Phase 5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:05:05 +01:00
Alby96andClaude Opus 5.5 953ffe5e9b Add the Laboratorio screen: new test, report, comparison (refs #36)
The laboratory starts with the host (off in demo mode) and runs its
queue next to market data and trading. The page lists every trial with
status, verdict and headline figures, refreshing itself while one is
queued or running, and offers the form for a new test: strategy, data
origin and market, instruments, period, initial capital and the grid,
with the thresholds stated before the run. The form posts with the
antiforgery token; errors come back as a short code.

The report page shows the protocol checks, the chosen variant against
buy and hold (table and chart), results per year, cost sensitivity,
the Deflated Sharpe Ratio, PBO, walk-forward windows, purged CV paths,
the look-ahead result, every variant with its zone score, what the
backtest does not model, the data used and the trades. A comparison
page puts two finished tests side by side and says whether they read
the same data. The improvements of 8.7 and the cost estimates of 11.6
have their sections, empty until Phases 9 and 8.

The equity chart takes amounts in their own currency and its labels as
parameters. The profit factor has no value without a losing trade.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:01:05 +01:00
Alby96andClaude Opus 5.5 5efb149fd6 Add the trial register, the report and the laboratory queue (refs #35 #36)
A trial is planned before it runs: strategy and version, grid, data,
initial capital, cost multipliers, validation settings, thresholds and
engine version go into backtest_trials as JSON with the SHA-256 of the
text. The store has no way to change a plan; later writes touch only
status, verdict, results and times, and a test reads the raw row back
to prove it. Every finished trial stays and counts towards the number
of tries the Deflated Sharpe Ratio deflates for.

The trial runner is pure computation: every grid variant on the whole
period, plateau choice, the chosen variant again at double and triple
costs, buy and hold, walk-forward, purged CV, PBO, DSR and the
look-ahead check, then the five thresholds of SPEC 12.2 and the report
of point 6 with the limitations the backtest declares (no correctives
without history, no risk engine before Phase 5).

The laboratory writes the plan, queues the trial and runs one at a time
on a dedicated thread with below-normal priority; only the async side
reads and writes the database. On start, trials left running go back
to planned and run again. Data comes from the daily bars and corporate
actions in the database, or from the Binance research history with
estimated order rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 09:49:05 +01:00
Alby96andClaude Opus 5.5 ccb9602950 Add backtest metrics, Deflated Sharpe, PBO, purged CV and the look-ahead test (refs #34 #35)
Metrics give what the report of SPEC 12.2 point 6 needs: return, CAGR,
annualised volatility and Sharpe, max drawdown with its days, trades,
win rate, profit factor, fees, exposure and results per calendar year.

Statistics are written in house as SPEC 12.2 asks: the normal CDF
(Hart 5666, double precision) and quantile (Acklam plus a Halley step),
moments, the probabilistic and the Deflated Sharpe Ratio. The DSR test
reproduces the example of Bailey and López de Prado (SR0 0.1132, DSR
0.9004).

Validation works on the daily returns of every grid variant:
walk-forward with a growing window, combinatorial purged CV (6 groups,
2 tested: 15 splits, 5 paths, purge and embargo of 1% of the series)
and PBO by CSCV (16 groups). Pure noise gives a PBO near 0.5, a
dominating variant 0, a winner that always loses out of sample 1.
Choices follow the plateau rule: a variant scores its own Sharpe but no
more than the mean of its zone, so a lonely peak loses to a broad hill
and a poor variant does not win by standing next to a good one.

The look-ahead check reruns the strategy on data cut at six days and
requires every decision up to each cut to be unchanged; it fails on a
strategy that reads tomorrow and passes on the average cross. The
one-step comparison of SPEC 12.3 classifies intents and flags action
differences and sizes more than 10% apart. A golden run guards the
engine against unintended changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 09:40:06 +01:00
Alby96andClaude Opus 5.5 d97304d7b9 Add the event-driven backtest engine and the strategy interface (refs #34)
The strategy contract of SPEC 5.3 (IStrategy, StrategyContext, Intent) is
shared by backtest and live: the context exposes bars only up to the
decision, adjusted and anchored to that day, and reading past it throws.
The engine runs each session as ordered events (corporate actions, open,
session, close), decides on the close and executes at the next open with
the simulator's price lists, slippage tiers, tick and lot rounding,
minimums and no leverage. Corrective and risk hooks are wired in and pass
everything through until Phase 5. One decision cycle can restart from a
recorded book (SPEC 12.3).

A scripted toy strategy reproduces a result computed by hand to the cent
on OKX and Alpaca costs. The Phase 3 class Trading.Strategies becomes
StrategyNames so the Anubis.Strategies namespace can exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 09:29:01 +01:00
Alby96 6d3ef8e7b5 Fase 3 — Conti, ordini, contabilità (#89)
ci / test (postgres) (push) Successful in 4m3s
ci / test (sqlite) (push) Successful in 2m49s
ci / image (push) Successful in 13s
2026-11-10 22:32:02 +01:00
166 changed files with 26897 additions and 395 deletions

No files matched your search

+1
View File
@@ -7,6 +7,7 @@ artifacts/
TestResults/
docs/
!docs/design/logo/
!docs/backtests/*.outcome.json
**/*.md
.env
.env.*
+3
View File
@@ -3,6 +3,9 @@
*.sh text eol=lf
.githooks/* text eol=lf
Dockerfile text eol=lf
# Windows scripts and the instructions in the release zip keep CRLF (ADR 0013).
*.cmd text eol=crlf
deploy/windows/*.txt text eol=crlf
*.png binary
*.ico binary
*.ttf binary
+3
View File
@@ -63,3 +63,6 @@ jobs:
git checkout -q FETCH_HEAD
- name: Build image (no publish)
run: docker build -f deploy/docker/Dockerfile -t anubis:ci .
# The guard drill inside the image (issue #48): offline, its own SQLite database in a temporary folder.
- name: Guard drill in the image
run: docker run --rm anubis:ci drill
+4
View File
@@ -0,0 +1,4 @@
{
// The C# extension provides the "coreclr" debugger used by launch.json.
"recommendations": ["ms-dotnettools.csharp"]
}
+70
View File
@@ -0,0 +1,70 @@
{
// Debugging Anubis from VS Code (needs the C# extension, ms-dotnettools.csharp).
// Each configuration builds the server in Debug first and opens the browser once it listens.
"version": "0.2.0",
"configurations": [
{
"name": "Anubis: server, dati di prova",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "build-server",
"program": "${workspaceFolder}/src/Anubis.Server/bin/Debug/net10.0/Anubis.Server.dll",
// A data folder of its own, ignored by git: the real one in %APPDATA%\Anubis stays untouched.
"args": ["--data-dir", "${workspaceFolder}/artifacts/debug-data"],
"cwd": "${workspaceFolder}/src/Anubis.Server",
"env": {
"ASPNETCORE_ENVIRONMENT": "Development"
},
"console": "internalConsole",
"stopAtEntry": false,
"serverReadyAction": {
"action": "openExternally",
"pattern": "Now listening on: (https?://[^\\s\"]+)"
}
},
{
"name": "Anubis: modalità dimostrativa",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "build-server",
"program": "${workspaceFolder}/src/Anubis.Server/bin/Debug/net10.0/Anubis.Server.dll",
// The data of SPEC appendix F, in a folder and a database apart (the same as "tasks seed-demo").
"args": ["--demo"],
"cwd": "${workspaceFolder}/src/Anubis.Server",
"env": {
"ASPNETCORE_ENVIRONMENT": "Development"
},
"console": "internalConsole",
"stopAtEntry": false,
"serverReadyAction": {
"action": "openExternally",
"pattern": "Now listening on: (https?://[^\\s\"]+)"
}
},
{
"name": "Anubis: server, dati dell'utente",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "build-server",
"program": "${workspaceFolder}/src/Anubis.Server/bin/Debug/net10.0/Anubis.Server.dll",
// The desktop data folder (%APPDATA%\Anubis), the same one "tasks dev" uses.
"args": [],
"cwd": "${workspaceFolder}/src/Anubis.Server",
"env": {
"ASPNETCORE_ENVIRONMENT": "Development"
},
"console": "internalConsole",
"stopAtEntry": false,
"serverReadyAction": {
"action": "openExternally",
"pattern": "Now listening on: (https?://[^\\s\"]+)"
}
},
{
"name": "Anubis: aggancia a un processo",
"type": "coreclr",
"request": "attach",
"processId": "${command:pickProcess}"
}
]
}
+45
View File
@@ -0,0 +1,45 @@
{
// The build the debugger needs, plus the commands of tasks.ps1 (Terminal > Run Task).
"version": "2.0.0",
"tasks": [
{
"label": "build-server",
"type": "process",
"command": "dotnet",
"args": ["build", "${workspaceFolder}/src/Anubis.Server", "-c", "Debug", "-nologo", "-consoleLoggerParameters:NoSummary"],
"group": { "kind": "build", "isDefault": true },
"presentation": { "reveal": "silent", "clear": true },
"problemMatcher": "$msCompile"
},
{
"label": "check",
"type": "shell",
"command": "./tasks.ps1 check",
"windows": { "command": "./tasks.ps1 check" },
"linux": { "command": "sh ./tasks.sh check" },
"osx": { "command": "sh ./tasks.sh check" },
"group": "test",
"problemMatcher": "$msCompile"
},
{
"label": "test",
"type": "shell",
"command": "./tasks.ps1 test",
"windows": { "command": "./tasks.ps1 test" },
"linux": { "command": "sh ./tasks.sh test" },
"osx": { "command": "sh ./tasks.sh test" },
"group": { "kind": "test", "isDefault": true },
"problemMatcher": "$msCompile"
},
{
"label": "dev",
"type": "shell",
"command": "./tasks.ps1 dev",
"windows": { "command": "./tasks.ps1 dev" },
"linux": { "command": "sh ./tasks.sh dev" },
"osx": { "command": "sh ./tasks.sh dev" },
"isBackground": true,
"problemMatcher": []
}
]
}
+1
View File
@@ -20,6 +20,7 @@ Dalla radice del repository: `./tasks.ps1 <comando>` (Windows) o `sh ./tasks.sh
- `hooks`: attiva `.githooks/pre-commit`, che esegue `check` prima di ogni commit. Si fa una volta per copia del repository.
- `seed-demo`: la modalità dimostrativa con i dati dell'appendice F, in una cartella e un database a parte, solo dal computer stesso.
- `screenshots`: fotografa Panoramica, Conti, Diario e Impostazioni a 1440 px (chiaro e scuro) e a 390 px in `artifacts/screenshots/`. Usa Edge o Chrome già installati.
- `release-win`: lo zip di prova per Windows in `artifacts/release/` (ADR 0013), da allegare alla release `v0.N.0` su Gitea.
- `desktop-win` (Fase 10): non ancora disponibile.
- `docker compose up --build`: l'applicazione su http://localhost:8787 con PostgreSQL, `/health` risponde `ok`. Primo accesso con `ANUBIS_ADMIN_PASSWORD` del compose.
+1 -1
View File
@@ -13,7 +13,7 @@
<SatelliteResourceLanguages>it;en</SatelliteResourceLanguages>
<Company>BalboSoft</Company>
<Product>Anubis</Product>
<Version>0.0.1</Version>
<Version>0.7.0</Version>
</PropertyGroup>
<!-- Only the packages of ADR 0004, with versions in Directory.Packages.props and lock files committed. -->
+10 -2
View File
@@ -6,12 +6,19 @@ WORKDIR /src
COPY global.json Directory.Build.props Directory.Packages.props ./
COPY src/Anubis/Anubis.csproj src/Anubis/packages.lock.json src/Anubis/
COPY src/Anubis.Server/Anubis.Server.csproj src/Anubis.Server/packages.lock.json src/Anubis.Server/
RUN dotnet restore src/Anubis.Server/Anubis.Server.csproj --locked-mode
COPY src/Anubis.Guard/Anubis.Guard.csproj src/Anubis.Guard/packages.lock.json src/Anubis.Guard/
RUN dotnet restore src/Anubis.Server/Anubis.Server.csproj --locked-mode && \
dotnet restore src/Anubis.Guard/Anubis.Guard.csproj --locked-mode
COPY src/Anubis/ src/Anubis/
COPY src/Anubis.Server/ src/Anubis.Server/
COPY src/Anubis.Guard/ src/Anubis.Guard/
# The logo is embedded from docs/design/logo, its only source (ADR 0005).
COPY docs/design/logo/ docs/design/logo/
# The backtest outcomes are read at the first start (SPEC 12.3); .dockerignore lets only their JSON files through.
COPY docs/backtests/ docs/backtests/
RUN dotnet publish src/Anubis.Server/Anubis.Server.csproj -c Release --no-restore -o /app
# The guard (SPEC 9.3, ADR 0012) is the main process and starts the engine next to it, from the same folder.
RUN dotnet publish src/Anubis.Guard/Anubis.Guard.csproj -c Release --no-restore -o /app
FROM mcr.microsoft.com/dotnet/aspnet:10.0
WORKDIR /app
@@ -25,7 +32,8 @@ ENV ANUBIS_PLATFORM=docker \
PGID=100
EXPOSE 8787
VOLUME /data
# Starts as root only to give /data to PUID:PGID, then runs the app as that user (entrypoint.sh).
# Starts as root only to give /data to PUID:PGID, then runs the guard as that user (entrypoint.sh); the guard starts
# the engine. "docker run --rm <image> drill" runs the guard drill instead (issue #48).
# Health through bash's /dev/tcp: no curl or wget in the image. 503 (database down) counts as unhealthy.
HEALTHCHECK --interval=30s --timeout=5s --start-period=60s --retries=3 \
CMD ["bash", "-c", "exec 3<>/dev/tcp/127.0.0.1/${PORT:-8787} && printf 'GET /health HTTP/1.0\r\nHost: localhost\r\n\r\n' >&3 && head -n 1 <&3 | grep -q ' 200 '"]
+3 -2
View File
@@ -1,9 +1,10 @@
#!/bin/sh
# Container start (SPEC 15.2, issue #16). As root: prepare /data for PUID:PGID (99:100 on Unraid), then drop to that
# user with setpriv, which is part of the base image (no gosu). Started as another user (docker run --user): go straight on.
# The process is the guard (SPEC 9.3, ADR 0012): it starts the engine, watches it and passes the stop on to it.
set -eu
app="/app/Anubis.Server.dll"
app="/app/Anubis.Guard.dll"
if [ "$(id -u)" != "0" ]; then
exec dotnet "$app" "$@"
@@ -19,7 +20,7 @@ case "$uid$gid" in
esac
mkdir -p /data
for folder in db models research import backups logs web-keys; do
for folder in db models research import backups logs web-keys guard; do
mkdir -p "/data/$folder"
done
# Only what does not already belong to PUID:PGID, so a large data folder does not slow every start.
+8
View File
@@ -0,0 +1,8 @@
@echo off
rem Starts Anubis on this computer (desktop platform) and opens it in the browser. ADR 0013, point 6.
cd /d "%~dp0"
echo Anubis si avvia su http://localhost:8787
echo Questa finestra resta aperta mentre il bot lavora. Per fermarlo premi Ctrl+C.
start "" /b cmd /c "timeout /t 8 /nobreak >/dev/null & start http://localhost:8787"
"%~dp0Anubis.Server.exe" --platform desktop
if errorlevel 1 pause
+44
View File
@@ -0,0 +1,44 @@
ANUBIS - versione di prova per Windows
======================================
Che cos'è
---------
Il bot di trading Anubis da far girare sul tuo computer per qualche giorno.
Lavora solo in simulazione e sui conti di prova: non può usare soldi veri.
Raccoglie da solo i dati per l'analisi (memoria, lavori, chiamate, errori),
così chi segue lo sviluppo può capire come si è comportato.
Cosa serve
----------
Il runtime ASP.NET Core 10 di Microsoft. Se hai già l'SDK .NET 10, c'è già.
Altrimenti lo scarichi da https://dotnet.microsoft.com/download/dotnet/10.0
(sezione "ASP.NET Core Runtime", versione per Windows x64).
Come si avvia
-------------
1. Estrai lo zip in una cartella qualsiasi, per esempio C:\Anubis.
2. Fai doppio clic su "Avvia Anubis.cmd".
Si apre una finestra nera: lasciala aperta, il bot lavora lì dentro.
Dopo qualche secondo il browser apre http://localhost:8787.
3. La prima volta una procedura guidata ti chiede dove tenere i dati e una
password. I dati vanno in %APPDATA%\Anubis.
4. Per fermarlo premi Ctrl+C nella finestra nera.
Chiudere la finestra lo ferma di colpo: funziona, ma il diario lo segnerà
come uno spegnimento improvviso.
Mentre gira
-----------
Usalo come vuoi. Se il computer va in sospensione, il rapporto lo vedrà come
un silenzio: va bene, ma per una prova pulita conviene lasciarlo acceso.
Come mandare i dati
-------------------
Quando hai finito (anche con il bot ancora acceso):
- fai doppio clic su "Prepara i dati da incollare.cmd": si apre il Blocco
note con il rapporto. Seleziona tutto (Ctrl+A), copia (Ctrl+C) e incolla
il testo nella chat;
- oppure, dall'interfaccia: Diario e sistema, pulsante "Scarica i dati per
l'analisi".
Il rapporto copre tutto ciò che è stato raccolto, fino a 30 giorni. Non
contiene chiavi, password né il tuo nome utente.
@@ -0,0 +1,10 @@
@echo off
rem Writes the diagnostics report (issue #95) next to this file and opens it, ready to copy. It only reads.
cd /d "%~dp0"
"%~dp0Anubis.Server.exe" diagnostics --platform desktop --out "%~dp0anubis-diagnostica.md" >nul
if errorlevel 1 (
echo Non riuscito: avvia prima Anubis almeno una volta.
pause
exit /b 1
)
start "" notepad "%~dp0anubis-diagnostica.md"
+2
View File
@@ -7,6 +7,8 @@ services:
context: .
dockerfile: deploy/docker/Dockerfile
image: gitea.encke-hake.ts.net/balbosoft/anubis:dev
# The guard gives the engine up to 55 seconds to close in order (SPEC 9.3).
stop_grace_period: 60s
ports:
- "8787:8787"
environment:
+45 -44
View File
@@ -13,9 +13,9 @@ aperta la fase. Se questo file e Gitea divergono, vale Gitea e si corregge quest
| 1 | [Fondamenta](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/milestone/28) | Completata |
| 2 | [Dati di mercato](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/milestone/29) | Completata (azioni in attesa delle chiavi Alpaca) |
| 3 | [Conti, ordini, contabilità](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/milestone/30) | Completata (conti di prova in attesa delle chiavi) |
| 4 | [Backtest e laboratorio](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/milestone/31) | Da fare |
| 5 | [Strategie, portafoglio, rischio](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/milestone/32) | Da fare |
| 6 | [Motore dal vivo in simulazione](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/milestone/33) | Da fare |
| 4 | [Backtest e laboratorio](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/milestone/31) | Completata |
| 5 | [Strategie, portafoglio, rischio](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/milestone/32) | Completata (`crypto_trend` non superato, azionarie in attesa delle chiavi Alpaca) |
| 6 | [Motore dal vivo in simulazione](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/milestone/33) | Completata (prova di 48 ore in corso) |
| 7 | [Notizie](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/milestone/34) | Da fare |
| 8 | [Apprendimento](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/milestone/35) | Da fare |
| 9 | [Rifinitura e consegna in Docker](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/milestone/36) | Da fare |
@@ -136,17 +136,17 @@ Un motore di prova onesto, riproducibile e protetto dall'auto-inganno.
**Lavori**
- [ ] [#34](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/34) Motore di backtest a eventi, modelli di costo e metriche
- [ ] [#35](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/35) Convalide: finestre che avanzano, K-fold purgato combinatorio, DSR, PBO; registro delle prove
- [ ] [#36](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/36) Rapporto di backtest e schermata Laboratorio
- [ ] [#37](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/37) Chiusura della fase, revisione indipendente e tag `fase-4`
- [x] [#34](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/34) Motore di backtest a eventi, modelli di costo e metriche
- [x] [#35](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/35) Convalide: finestre che avanzano, K-fold purgato combinatorio, DSR, PBO; registro delle prove
- [x] [#36](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/36) Rapporto di backtest e schermata Laboratorio
- [x] [#37](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/37) Chiusura della fase, revisione indipendente e tag `fase-4`
**Criteri di accettazione**
- [ ] Una strategia giocattolo con risultato noto dà esattamente il risultato atteso
- [ ] Il test anti-sguardo-avanti esiste e fallisce su una strategia volutamente scorretta
- [ ] DSR e PBO riproducono i casi noti di 12.2
- [ ] Il rapporto si apre dall'interfaccia
- [x] Una strategia giocattolo con risultato noto dà esattamente il risultato atteso (`AToyStrategyGivesExactlyTheResultComputedByHand`, `TheMetricsOfTheToyRunAreThoseComputedByHand`: patrimonio, commissioni ed esecuzioni calcolati a mano, al centesimo)
- [x] Il test anti-sguardo-avanti esiste e fallisce su una strategia volutamente scorretta (`TheLookAheadTestCatchesAStrategyThatReadsTomorrow`)
- [x] DSR e PBO riproducono i casi noti di 12.2 (`TheDeflatedSharpeRatioReproducesTheExampleOfTheArticle`: DSR 0,9004 dell'articolo; PBO circa 0,5 sul rumore, 0 con una variante dominante, 1 quando la migliore perde sempre fuori campione)
- [x] Il rapporto si apre dall'interfaccia (`TheReportOpensFromTheLaboratory`: test pianificato dal modulo, eseguito, rapporto e confronto aperti)
## Fase 5 — Strategie, portafoglio, rischio
@@ -156,25 +156,25 @@ Le tre strategie di base dentro un motore del rischio che nessuno può aggirare.
**Lavori**
- [ ] [#38](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/38) Strategia `crypto_trend`
- [ ] [#39](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/39) Strategia `equity_momentum`
- [ ] [#40](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/40) Strategia `equity_meanrev`
- [ ] [#41](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/41) Correttivi comuni, ripartizione del capitale e dimensionamento
- [ ] [#42](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/42) Motore del rischio completo con interruttore d'emergenza
- [ ] [#43](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/43) Stadi delle strategie, requisiti, gemello simulato, sola uscita e conferma per i soldi veri
- [ ] [#44](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/44) Backtest onesti delle tre strategie in docs/backtests/
- [ ] [#45](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/45) Schermate Strategie, Rischio e Operazioni con la scheda "perché"; Impostazioni su strategie, limiti e scenari
- [ ] [#46](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/46) Chiusura della fase, revisione indipendente e tag `fase-5`
- [x] [#38](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/38) Strategia `crypto_trend`
- [x] [#39](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/39) Strategia `equity_momentum`
- [x] [#40](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/40) Strategia `equity_meanrev`
- [x] [#41](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/41) Correttivi comuni, ripartizione del capitale e dimensionamento
- [x] [#42](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/42) Motore del rischio completo con interruttore d'emergenza
- [x] [#43](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/43) Stadi delle strategie, requisiti, gemello simulato, sola uscita e conferma per i soldi veri
- [x] [#44](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/44) Backtest onesti delle tre strategie in docs/backtests/
- [x] [#45](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/45) Schermate Strategie, Rischio e Operazioni con la scheda "perché"; Impostazioni su strategie, limiti e scenari
- [x] [#46](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/46) Chiusura della fase, revisione indipendente e tag `fase-5`
**Criteri di accettazione**
- [ ] Ogni regola di 9.1 e 9.2 ha il suo test
- [ ] Un test dimostra che senza la procedura di 12.4 nessun ordine raggiunge un conto reale
- [ ] Un test dimostra che una strategia retrocessa continua a gestire uscite e stop delle sue posizioni reali fino a zero
- [ ] Per ciascuna strategia un rapporto di backtest onesto in `docs/backtests/` (costi doppi e tripli, confronto con comprare e tenere), qualunque sia l'esito, più il file versionato con l'esito per il primo avvio
- [ ] Senza chiavi Alpaca i backtest azionari restano in attesa dell'utente e le due strategie azionarie restano in osservazione
- [ ] I test anti-sguardo-avanti passano
- [ ] Le schermate Strategie, Rischio e Operazioni corrispondono ai mockup
- [x] Ogni regola di 9.1 e 9.2 ha il suo test (`RiskTests` e `RiskEngineTests`: perdita del giorno, della settimana, calo dal massimo a 10 e 15%, capitale investito, posizione singola, quota crypto, rischio della singola operazione, controlli tecnici, niente vendite allo scoperto, protezione per le operazioni in giornata, avviso all'80%, ordini al minuto, file STOP; più `NoRuleEverEnlargesWhatTheStrategyAsked` e `AfterALossNoPurchaseIsLarger` come test di proprietà)
- [x] Un test dimostra che senza la procedura di 12.4 nessun ordine raggiunge un conto reale (`NoOrderReachesARealAccountWithoutTheProcedure`)
- [x] Un test dimostra che una strategia retrocessa continua a gestire uscite e stop delle sue posizioni reali fino a zero (`ADemotedStrategyKeepsManagingItsRealPositionsDownToZero`)
- [x] Per ciascuna strategia un rapporto di backtest onesto in `docs/backtests/` (costi doppi e tripli, confronto con comprare e tenere), qualunque sia l'esito, più il file versionato con l'esito per il primo avvio — `crypto_trend` **non superato** (calo del 32,8% sull'universo largo, PBO 0,84 su BTC, ETH e SOL): resta in osservazione (ADR 0011, punto 13). `EveryBaseStrategyHasItsOutcomeFileBuiltIn`, `TheProtocolRunsBothUniversesAndWritesTheReportWhateverTheOutcome`
- [x] Senza chiavi Alpaca i backtest azionari restano in attesa dell'utente e le due strategie azionarie restano in osservazione (`AtTheFirstStartTheStockStrategiesWaitInObservationAndCryptoFollowsItsOutcome`; i due rapporti azionari dicono cosa manca)
- [x] I test anti-sguardo-avanti passano (`CryptoTrendDoesNotLookAhead`, `EquityMomentumDoesNotLookAhead`, `EquityMeanrevDoesNotLookAhead`)
- [x] Le schermate Strategie, Rischio e Operazioni corrispondono ai mockup (`TheDemoShowsTradesStrategiesAndRiskAsInTheMockups` e confronto a 1440 e 375 px con i mockup 03, 04 e 07)
## Fase 6 — Motore dal vivo in simulazione
@@ -184,25 +184,25 @@ Il bot che lavora da solo, in simulazione, giorno e notte, e lo racconta.
**Lavori**
- [ ] [#47](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/47) Orchestrazione della giornata (5.5) e motore dal vivo in simulazione
- [ ] [#48](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/48) Processo di guardia, avvio e ripartenza sicuri
- [ ] [#49](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/49) Recupero dopo un'assenza (5.6)
- [ ] [#50](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/50) Diario con frasi in italiano dai codici motivo e confronto "senza bot"
- [ ] [#51](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/51) Schermate Panoramica e Diario e sistema
- [ ] [#52](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/52) Telegram e segnale di vita
- [ ] [#53](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/53) Prova di durata di 48 ore in tempo reale
- [x] [#47](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/47) Orchestrazione della giornata (5.5) e motore dal vivo in simulazione
- [x] [#48](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/48) Processo di guardia, avvio e ripartenza sicuri
- [x] [#49](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/49) Recupero dopo un'assenza (5.6)
- [x] [#50](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/50) Diario con frasi in italiano dai codici motivo e confronto "senza bot"
- [x] [#51](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/51) Schermate Panoramica e Diario e sistema
- [x] [#52](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/52) Telegram e segnale di vita
- [x] [#53](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/53) Prova di durata di 48 ore in tempo reale
- [ ] [#54](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/54) Chiusura della fase, revisione indipendente e tag `fase-6`
**Criteri di accettazione**
- [ ] Il test di parità tra backtest e motore in simulazione passa, con correttivi e rischio attivi
- [ ] Una prova accelerata con orologio simulato copre almeno 10 giorni di mercato con guasti iniettati e termina con registro e conti riconciliati e senza ordini duplicati
- [ ] Un test blocca il ciclo principale con una vendita a metà e verifica che dopo la guardia la posizione abbia di nuovo i suoi stop e il motore sia ripartito; la stessa prova passa dentro l'immagine Docker
- [ ] Un test con orologio simulato spegne il bot per 3 ore, 20 ore e 3 giorni e verifica le righe di 5.6 i cui lavori esistono
- [ ] A fine fase parte una prova di durata in tempo reale di almeno 48 ore a cavallo di due sedute USA (solo crypto senza chiavi Alpaca)
- [ ] In modalità dimostrativa la Panoramica corrisponde agli screen 01 (1440 px) e 09 (390 px)
- [ ] Il Diario corrisponde allo screen 08 a 1440 px e a 390 px non scorre in orizzontale
- [ ] I comandi Telegram funzionano (o, senza token, superano i test con un finto server)
- [x] Il test di parità tra backtest e motore in simulazione passa, con correttivi e rischio attivi (`TheLiveEngineMakesTheSameTradesAsTheBacktest` su 40 giorni di `crypto_trend`, `TheStockSideMakesTheSameTradesAsTheBacktest` su 6 settimane di `equity_meanrev`)
- [x] Una prova accelerata con orologio simulato copre almeno 10 giorni di mercato con guasti iniettati e termina con registro e conti riconciliati e senza ordini duplicati (`TenDaysWithFaultsEndReconciledWithoutDuplicateOrders`: 14 giorni con rete giù, risposta persa, ripartenza con un ordine a metà e database irraggiungibile, su SQLite e PostgreSQL)
- [x] Un test blocca il ciclo principale con una vendita a metà e verifica che dopo la guardia la posizione abbia di nuovo i suoi stop e il motore sia ripartito; la stessa prova passa dentro l'immagine Docker (`ALoopBlockedWithASaleHalfwayGetsItsStopBackAndTheEngineRestarts`, `TheGuardDrillPassesWithTheRealEngineAsChildProcess`; `docker run --rm <immagine> drill` in `tasks ci` e nella CI: 7 controlli su 7)
- [x] Un test con orologio simulato spegne il bot per 3 ore, 20 ore e 3 giorni e verifica le righe di 5.6 i cui lavori esistono (`ThreeHoursOffRecoverTheCryptoDecisionAndDropAnEntryThatMovedTooFar`, `TwentyHoursOffLetTheLateDecisionsUpdateOnlyExitsAndDropExpiredEntries`, `ThreeDaysOffAreDeclaredUntilSeenAndMissedRunsAreMadeOnce`)
- [x] A fine fase parte una prova di durata in tempo reale di almeno 48 ore a cavallo di due sedute USA (solo crypto senza chiavi Alpaca) — partita nel container `anubis-prova`, ripartita da zero il 12/11/2026 alle 21:02 dopo uno spegnimento improvviso del PC; il rapporto si verifica in Fase 7 (#55)
- [x] In modalità dimostrativa la Panoramica corrisponde agli screen 01 (1440 px) e 09 (390 px) (`TheDemoPanoramicaShowsAccountsPositionsAndLimits`; fotografie di `tasks screenshots`, confronto nella prova di #51)
- [x] Il Diario corrisponde allo screen 08 a 1440 px e a 390 px non scorre in orizzontale (`TheDiaryFiltersAndSearches`; a 390 px `scrollWidth` uguale alla larghezza su `/`, `/diario` e `/diario?cerca=…`)
- [x] I comandi Telegram funzionano (o, senza token, superano i test con un finto server) — senza token, con un finto server: `TheCommandsWorkFromThePhoneThroughTheRealServer`, `FermaWantsAConfirmationWithinTwoMinutes`, `OnlyTheChatOfTheSettingsIsObeyedAndOldMessagesAreNot` e gli altri di `TelegramTests`
## Fase 7 — Notizie
@@ -212,7 +212,8 @@ Le notizie usate per evitare guai, mai per comprare.
**Lavori**
- [ ] [#55](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/55) Verifica del rapporto della prova di durata
- [ ] [#95](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/95) Raccolta dei dati per l'analisi e release di prova su Gitea (ADR 0013)
- [ ] [#55](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/55) Verifica del rapporto della prova di durata: dal rapporto della release incollato dall'utente (ADR 0013)
- [ ] [#56](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/56) Raccolta, deduplica e collegamento delle notizie agli strumenti
- [ ] [#57](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/57) Classificatore locale ONNX e regole sulle parole chiave
- [ ] [#58](https://gitea.encke-hake.ts.net/BalboSoft/Anubis/issues/58) Analista con modello linguistico facoltativo, sicurezza e tetto di spesa
@@ -222,7 +223,7 @@ Le notizie usate per evitare guai, mai per comprare.
**Criteri di accettazione**
- [ ] Il rapporto della prova di durata è in `docs/` e non riporta ordini duplicati né differenze non spiegate
- [ ] Il rapporto della prova di durata è in `docs/` e non riporta ordini duplicati né differenze non spiegate (dalla release, ADR 0013)
- [ ] Su un archivio di notizie registrate la deduplica e il collegamento superano i test
- [ ] Un testo con istruzioni malevole non produce alcun effetto fuori dallo schema (test dedicato)
- [ ] Senza chiave del modello linguistico tutto funziona
+45 -8
View File
@@ -1,10 +1,10 @@
# Stato del progetto
Aggiornato: 2026-11-10
Aggiornato: 2026-11-14
## Fase corrente
**Fase 3 — Conti, ordini, contabilità: completata**, con le prove sui conti di prova in attesa delle chiavi demo di OKX e di Alpaca. La prossima è la Fase 4 (Backtest e laboratorio), da iniziare con il via dell'utente.
**Fase 7 — Notizie: in corso.** La Fase 6 è completata. La sua prova di durata di 48 ore in casa è stata interrotta più volte dal computer, mai da Anubis, e il 14 novembre l'utente ha scelto un'altra strada (ADR 0013). Ora il bot raccoglie da solo i dati per l'analisi (issue #95). L'utente fa girare la release di prova `v0.7.0` pubblicata su Gitea per il tempo che vuole, poi incolla in chat il rapporto. Il controllo della prova (#55) si fa su quel rapporto. Nessuna strategia manda ordini: `crypto_trend` non ha superato il backtest e decide solo in osservazione, le due azionarie aspettano le chiavi di Alpaca.
## Fatto
@@ -62,6 +62,33 @@ Aggiornato: 2026-11-10
- Riconciliazione ogni 5 minuti con modalità prudente, scelta "allinea" o "escludi", blocco delle chiavi che permettono i prelievi.
- Schermata Conti e posizioni come il mockup, esportazione CSV delle posizioni; in Impostazioni chiavi (verificate col broker prima di salvarle) e capitale affidato; la Panoramica indica il capitale come primo passo.
- Batteria di conformità sul simulatore e sulle risposte d'esempio della documentazione; sui conti di prova parte da sola quando ci sono le chiavi.
- **Fase 4** (scelte nell'ADR 0010):
- Motore di backtest a eventi, una seduta alla volta: si decide sulla chiusura e si esegue all'apertura dopo, con gli stessi costi del simulatore (moltiplicabili per 2 e 3), stop, frazionamenti e dividendi, nessun prestito. Le strategie sono le stesse classi che girano dal vivo; correttivi e rischio hanno già il loro posto e fino alla Fase 5 lasciano passare tutto.
- Metriche del rapporto (rendimento, volatilità, Sharpe, calo massimo, operazioni, esposizione, risultati per anno) e confronto con "comprare e tenere".
- Convalide scritte in casa: finestre che avanzano, K-fold purgato combinatorio, Deflated Sharpe Ratio, PBO, regola dell'altopiano, test contro lo sguardo in avanti. DSR e PBO riproducono i casi noti.
- Registro delle prove: il piano (con le soglie) e la sua impronta si scrivono prima dei risultati e non cambiano; il numero di prove del DSR conta tutti i tentativi della stessa strategia.
- Laboratorio su un thread a priorità bassa, una prova alla volta, con ripresa dopo un riavvio; dati dalle barre raccolte o dallo storico di ricerca Binance.
- Schermata Laboratorio: elenco delle prove, modulo per un nuovo test, rapporto completo, confronto tra due prove. Le sezioni "Migliorie in osservazione" e "Stime di costo da confermare" aspettano le Fasi 9 e 8.
- Strategia d'esempio `sma_cross` (incrocio di medie) per provare il laboratorio; le strategie di base arrivano con la Fase 5.
- **Fase 5** (scelte nell'ADR 0011):
- Le tre strategie di base: `crypto_trend` (rottura su nove periodi, stop a metà del canale, pesi a volatilità obiettivo, universo dei primi dieci del mese), `equity_momentum` (revisione settimanale con il filtro di SPY, pesi a volatilità inversa, stop che segue il prezzo) e `equity_meanrev` (RSI a 2 sopra la media a 200 sedute, rischio fisso, uscita alla chiusura). Le stesse classi girano in prova e dal vivo e passano il test contro lo sguardo in avanti.
- Correttivi comuni (regime crypto, calendario economico), dimensionamento sulla quota del bot e motore del rischio completo: limiti di perdita per mondo, calo dal massimo con dimezzamento e interruttore, limiti di dimensione, controlli tecnici, niente vendite allo scoperto né leva. Toglie o riduce, mai aggiunge. Gira anche dentro i backtest.
- Ogni passo di una decisione resta scritto con i suoi codici motivo (tabella `decisions`).
- Stadi verso i soldi veri: osservazione, simulazione, reale ridotto, reale. Senza la procedura di 12.4 nessun ordine raggiunge un conto reale; una strategia retrocessa gestisce le uscite delle posizioni reali fino a zero; il gemello simulato resta fuori dai totali. In simulazione entra solo una strategia che ha superato il backtest.
- Backtest onesti in `docs/backtests/` con `anubis backtest base`: storico Binance convertito in euro con il cambio BCE, monete tolte dal listino comprese, due universi, costi doppi e tripli, comprare e tenere accanto. **`crypto_trend` non supera** (calo massimo del 32,8% contro il 25% ammesso; probabilità di sovra-adattamento 0,84 su BTC, ETH e SOL) e resta in osservazione. Le due azionarie aspettano le chiavi di Alpaca. Gli esiti sono inclusi nell'applicazione e letti a ogni avvio.
- Schermate Operazioni (con la scheda "perché" e l'esportazione CSV), Strategie e Rischio come i mockup 03, 04 e 07; in Impostazioni limiti, protezione per le operazioni in giornata e scenari. Fermare, chiudere e ripartire passano dal motore del rischio; alzare un limite mentre una strategia usa soldi veri chiede frase e password.
- **Fase 6** (scelte nell'ADR 0012):
- Motore dal vivo: ogni decisione passa dalla stessa chiamata del backtest (`DecideOnce`), agli orari di 5.5. Ogni decisione ha una chiave scritta nella stessa transazione dei suoi ordini pianificati e delle righe del diario, quindi una ripartenza non decide né manda niente due volte. Parità con il backtest provata su 40 giorni di `crypto_trend` e 6 settimane di `equity_meanrev`; 14 giorni con guasti iniettati finiscono riconciliati e senza ordini doppi.
- Recupero dopo un'assenza (5.6): l'assenza si misura dall'ultimo lavoro finito. I lavori persi si rifanno una volta o si saltano con il motivo. Le decisioni in ritardo aggiornano solo le uscite. Un'assenza di più di un giorno resta in Panoramica finché non la vedi.
- Processo di guardia: in Docker è il processo principale e il genitore del motore. Interviene dopo due minuti di silenzio, un avvio fermo dieci minuti o un'uscita senza preavviso. Annulla gli ordini a metà, rimette gli stop, avvisa e fa ripartire il motore. La terza ripartenza forzata in un'ora scrive il file STOP. La prova (`drill`) gira anche dentro l'immagine, in `tasks ci` e nella CI.
- Diario con le frasi fisse dell'appendice B e confronto "senza bot" (40% BTC, 60% SPY, ogni versamento investito nel suo giorno).
- Telegram: coda con stato, nuovi tentativi e abbandono dopo un giorno; comandi dal telefono solo dalla chat configurata e solo se recenti, `/ferma` con `/conferma`; riepilogo del mattino, stato facoltativo, segnale di vita ogni minuto. Sezione in Impostazioni.
- Schermate Panoramica e Diario e sistema come i mockup 01, 08 e 09.
- `anubis soak`: il rapporto di una prova di durata, letto dai registri del bot mentre lavora.
- **Fase 7, in corso** (ADR 0013):
- Raccolta dei dati per l'analisi (#95). Ogni 5 minuti, e a ogni avvio, il bot scrive una fotografia del processo in `diagnostic_samples`: memoria, GC, thread, handle, CPU, eccezioni, dimensione del database, avvisi ed errori dei registri, chiamate HTTP in uscita per destinazione con le durate. Ogni esecuzione di ogni lavoro va in `job_runs`. Si tiene tutto per 30 giorni.
- Il rapporto da incollare, in Markdown fino a 60 KB e senza segreti, si ottiene in tre modi: dal pulsante «Scarica i dati per l'analisi» in Diario e sistema, da `/api/diagnostica.md` e da `anubis diagnostics`.
- Release di prova per Windows con `tasks release-win`: uno zip con il server, `Avvia Anubis.cmd`, `Prepara i dati da incollare.cmd` e un `LEGGIMI.txt`.
## In corso
@@ -69,18 +96,27 @@ Nulla.
## Prossimo passo
Fase 4, backtest e laboratorio: motore di prova sullo storico, con gli stessi costi e le stesse regole d'ordine della Fase 3, e il confronto "senza bot". Con le chiavi dei conti di prova si possono chiudere i due criteri in attesa (azioni della Fase 2, conti di prova della Fase 3).
1. L'utente scarica lo zip della release `v0.7.0` da Gitea (Release), lo fa girare e poi incolla il rapporto: doppio clic su `Prepara i dati da incollare.cmd`, oppure il pulsante in Diario e sistema. Il rapporto si analizza e si salva in `docs/diagnostica/`; da lì si chiude #55.
2. Intanto la Fase 7 va avanti sul branch `fase-7`: raccolta delle notizie (#56), poi classificatore locale (#57), analista facoltativo (#58), stato per strumento (#59) e schermata (#60).
3. Con le chiavi del conto paper di Alpaca si possono anche fare i backtest azionari (`anubis backtest base`) e chiudere i criteri delle Fasi 2 e 3 in attesa.
## Da sapere
- Le immagini .NET 10 sono su Ubuntu 24.04, non Debian (ADR 0006).
- La modalità dimostrativa mostra solo le parti già costruite: strategie e notizie della Panoramica arrivano con le loro fasi. Nella demo il motore di trading è spento.
- La prova nel container `anubis-prova` è chiusa (ADR 0013). I compiti programmati che la sorvegliavano sono spenti; il container si può fermare con `docker stop anubis-prova`.
- La modalità dimostrativa mostra le schermate costruite fin qui. Notizie e apprendimento appaiono solo con i dati dell'appendice F: fuori dalla demo arrivano con le Fasi 7 e 8. Nella demo il motore di trading e Telegram sono spenti.
- In Docker il processo principale è la guardia; fuori da Docker (`tasks dev`, VS Code) il server gira da solo.
- La guardia oggi apre solo i conti di prova: avrà i conti reali con la fase dei soldi veri (ADR 0012, punto 4).
- Le chiavi dei conti di prova si inseriscono da Impostazioni → Conti e chiavi oppure nel file `.env`; quelle nelle variabili d'ambiente valgono di più. Una chiave OKX che permette i prelievi viene rifiutata.
- Il capitale affidato si decide in euro in Impostazioni → Conti e chiavi. Finché non c'è, le strategie osservano soltanto.
- `tasks dev` fuori da Docker usa la cartella dei dati vera dell'utente: per prove usa `tasks seed-demo`.
- Alpaca non ha più conti senza margine: il divieto di leva lo impone Anubis (API-NOTES A14).
- OKX: WebSocket sulla porta 443, la 8443 è chiusa (API-NOTES O3).
- Le coppie in EUR di OKX esistono da novembre 2023: prima di allora la storia crypto viene dagli archivi Binance in USDT, da convertire in euro nella Fase 4.
- Le coppie in EUR di OKX esistono da novembre 2023: prima di allora la storia crypto viene dagli archivi Binance in USDT. Il Laboratorio li usa nella loro valuta (ADR 0010, punto 13); i backtest delle strategie di base li convertono in euro con il cambio BCE del giorno (ADR 0011, punto 10).
- `crypto_trend` resta in osservazione perché il suo backtest non è superato. Non ho ritoccato regole o soglie dopo aver visto il risultato: una nuova versione della strategia è una nuova prova, che il DSR conterà insieme alle precedenti.
- Il motore dal vivo fa girare ogni strategia con i parametri scritti nel suo file dell'esito (`parameters`), non con quelli predefiniti: sono quelli che si sono qualificati.
- Per rifare i backtest di base: `anubis backtest base --to AAAA-MM-GG --out docs/backtests` (con `--offline` usa gli archivi già scaricati).
- Nella modalità dimostrativa il Laboratorio è spento: mostra l'elenco ma non esegue prove.
- I dati di mercato si spengono con `ANUBIS_MARKET_DATA=off`; nella modalità dimostrativa sono sempre spenti.
- Gli universi passati usano i flag ETF e "frazionabile" di oggi: è un limite delle fonti gratuite (API-NOTES, Fase 2).
- Gitea ha un esecutore delle Actions attivo: la CI gira a ogni push (prima esecuzione: run 169 del branch `fase-0`, test verdi su SQLite e PostgreSQL). `tasks ci` resta per provarla in locale.
@@ -90,11 +126,12 @@ Fase 4, backtest e laboratorio: motore di prova sullo storico, con gli stessi co
| Cosa | Da quando serve | Senza |
|---|---|---|
| Via libera per la Fase 4 | Subito | Non si parte |
| Chiavi del conto paper di Alpaca (`ALPACA_API_KEY_DEMO`, `ALPACA_API_SECRET_DEMO`) | Già dalla Fase 2 (dati azionari) | Parte azionaria spenta; storico azionario dal 2016 in attesa |
| Far girare la release `v0.7.0` e incollare il rapporto in chat | Ora, per il tempo che vuoi | La prova di durata (#55) resta aperta |
| Chiavi del conto paper di Alpaca (`ALPACA_API_KEY_DEMO`, `ALPACA_API_SECRET_DEMO`) | Già dalla Fase 2 (dati azionari) | Parte azionaria spenta; storico azionario dal 2016 e backtest delle due strategie azionarie in attesa |
| Chiavi demo di OKX Europe, senza permesso di prelievo | Fase 3 (ordini sui conti di prova) | Batteria di conformità su risposte d'esempio; criterio della Fase 3 in attesa |
| Un token Gitea con permesso `write:package` come segreto `REGISTRY_TOKEN` | Fase 9 (pubblicazione dell'immagine) | Pubblicazione a mano |
| Token del bot Telegram e chat autorizzata (facoltativi) | Fase 6 | Test con un finto server |
| Token del bot Telegram e chat autorizzata (facoltativi), da Impostazioni → Telegram | Fase 6 | Nessun avviso sul telefono; comandi provati con un finto server |
| Un indirizzo per il segnale di vita (facoltativo, per esempio un controllo gratuito di Healthchecks.io) | Fase 6 | Nessuno avvisa se bot e guardia si spengono insieme |
| Chiavi FRED e Finnhub (facoltative, gratuite) | Già dalla Fase 2 | Calendario limitato alla settimana corrente; filtro trimestrali spento |
| Chiave di un modello linguistico (facoltativa) | Fase 7 | Solo classificatore locale |
| Prova del template su Unraid | Fase 9 | Criterio in attesa |
+24
View File
@@ -79,6 +79,30 @@ Un criterio che dipende solo da qualcosa che deve dare l'utente (chiavi dei cont
Per ogni schermata: implementa, apri sui dati dimostrativi (`tasks seed-demo`), fotografa con Edge senza finestra (`msedge --headless --screenshot`) a 1440 e 390 pixel, confronta zona per zona con lo screen di `docs/design/`, correggi, ripeti finché restano differenze trascurabili. I testi fissi si leggono dalle immagini, i dati dall'appendice F, colori e misure dall'appendice C. Guarda uno screen per volta e ritaglia la zona che ti serve: le immagini sono a doppia densità.
## Debug con Visual Studio Code
Apri la radice del repository in VS Code con l'estensione C# (`ms-dotnettools.csharp`, suggerita da `.vscode/extensions.json`), poi Esegui e debug (F5). Ogni voce compila il server in Debug (attività `build-server` di `.vscode/tasks.json`) e apre il browser quando il server ascolta:
- **Anubis: server, dati di prova**: una cartella dati a parte, `artifacts/debug-data`, ignorata da git. Al primo avvio chiede la configurazione iniziale; per ripartire da zero basta cancellare la cartella.
- **Anubis: modalità dimostrativa**: i dati dell'appendice F, come `tasks seed-demo`.
- **Anubis: server, dati dell'utente**: la cartella vera, `%APPDATA%\Anubis`, come `tasks dev`.
- **Anubis: aggancia a un processo**: per un server già avviato, per esempio con `tasks dev`.
Le attività `check`, `test` e `dev` sono anche in Terminale → Esegui attività. L'estensione è dell'editor, non del progetto: non entra nell'elenco dell'ADR 0004.
## Guardia e prova di durata
- In Docker il processo principale è la guardia (`Anubis.Guard.dll`), che avvia il server come processo figlio (ADR 0012). Fuori da Docker, con `tasks dev` o da VS Code, il server gira da solo, senza guardia.
- La prova della guardia blocca davvero il motore con una vendita a metà e controlla che tutto torni a posto:
- `dotnet run --project src/Anubis.Guard -- drill` sul computer;
- `docker run --rm <immagine> drill` dentro l'immagine.
La fanno anche `tasks ci` e la CI.
- Il rapporto di una prova di durata si legge mentre il bot lavora: `docker exec -u 99:100 -e HOME=/tmp <container> dotnet /app/Anubis.Server.dll soak --hours 48 --platform docker`, con l'utente del bot (`PUID:PGID`) perché i file del database restino suoi. Esce con 0 se tutto è regolare e con 4 se qualcosa va guardato. Senza `--from` la finestra sono le ultime `--hours` ore; con `--from` comincia lì e dura `--hours` ore, che sono anche la durata minima richiesta.
- Docker Desktop non si avvia dalla shell dell'app Claude (`Start-Process`, `docker desktop start`): il processo resta legato all'app e si chiude con lei quando si aggiorna o si chiude. Il 12 novembre 2026 è successo durante la prova di durata. Se va avviato da lì, si passa da WMI, che lo fa partire da un servizio di Windows:
`Invoke-CimMethod -ClassName Win32_Process -MethodName Create -Arguments @{CommandLine='"C:\Program Files\Docker\Docker\Docker Desktop.exe"'}`
- Gli appigli per le prove (`ANUBIS_FAULT`, `ANUBIS_DRILL_OKX_URL`) non si usano mai su un'installazione vera.
## Pacchetti e consegne
- **Docker**: immagine `gitea.encke-hake.ts.net/balbosoft/anubis`, pubblicata dalla CI sui tag `v*` nella sezione Packages del repository (ADR 0002). I tag `fase-N` non pubblicano nulla.
+68
View File
@@ -0,0 +1,68 @@
# 0010 — Scelte della Fase 4: backtest e laboratorio
Data: 2026-11-11 · Stato: accettata · Precisa la specifica (sezioni 5.3, 8.7, 12.1, 12.2, 13) e l'ADR 0009, senza aggiungere componenti esterni
## Contesto
La Fase 4 costruisce il motore di prova sullo storico, le convalide del protocollo anti-illusioni (12.2), il registro delle prove e la schermata Laboratorio. Le strategie di base, i correttivi e il motore del rischio arrivano con la Fase 5: per provare il laboratorio c'è una strategia d'esempio, l'incrocio di due medie (`sma_cross`). Alcune scelte non sono scritte nella specifica, o la precisano. Le raccolgo qui.
## Decisione
**1. Il motore va a eventi, una seduta alla volta.**
- Ordine degli eventi in una seduta: azioni societarie prima dell'apertura; all'apertura gli stop che aprono sotto il livello, poi gli ordini decisi alla chiusura prima, vendite per prime; durante la seduta gli stop toccati dal minimo; alla chiusura la valutazione e la decisione.
- Si decide sulla chiusura e si esegue all'apertura dopo: è il ritardo tra decidere e agire. Niente inseguimento con ordini limite (quello lo fa il gestore degli ordini dal vivo).
- Le strategie sono le stesse classi che girano dal vivo e non sanno se sono in prova.
- Il denaro è `decimal`; le barre sono `double` (dati d'analisi) e diventano `decimal` una volta sola, quando servono come prezzo.
- Il motore ha una versione (`BacktestEngine.Version`) scritta nel piano: un rapporto dice quale motore l'ha prodotto.
**2. Gli stessi costi del simulatore.** Commissioni e scostamento vengono dai listini del simulatore interno (ADR 0009 punto 5), arrotondati a nostro sfavore. Un moltiplicatore li scala tutti insieme per la sensibilità ai costi (×1, ×2, ×3). Gli acquisti non prendono mai in prestito; un ordine sotto il minimo non parte e lascia una nota.
**3. Correttivi e rischio sono già al loro posto, ma lasciano passare tutto.** Il motore chiama i correttivi (8.5) e il motore del rischio (9) per ogni intenzione; fino alla Fase 5 sono "passanti". Un test controlla che vedano ogni intenzione e possano cambiarla. Il rapporto lo scrive tra le cose che il test non considera.
**4. Statistica scritta in casa.** Nessuna libreria esterna (ADR 0004):
- funzione di ripartizione normale con l'algoritmo 5666 di Hart, quantile con il metodo di Acklam più un passo di Halley;
- momenti del campione: deviazione standard campionaria, asimmetria e curtosi (non in eccesso) di popolazione, come nell'articolo di Bailey e López de Prado;
- il Deflated Sharpe Ratio riproduce l'esempio dell'articolo (soglia 0,1132, DSR 0,9004).
**5. Sharpe senza tasso privo di rischio.** Sharpe e volatilità sono annualizzati con le sedute di un anno (365 per le crypto, 252 per le azioni) e non tolgono un tasso privo di rischio. Il rendimento annuo usa i giorni di calendario (365,25). Il rapporto guadagni/perdite non ha valore senza un'operazione in perdita: un "0" o un "infinito" ingannerebbero.
**6. Le convalide lavorano sui rendimenti giornalieri delle varianti.** Ogni variante della griglia gira una volta sull'intero periodo; finestre che avanzano, K-fold purgato combinatorio e PBO scelgono e giudicano su pezzi di quelle serie. È più semplice e veloce che rifare ogni volta il backtest su ogni pezzo, e la scelta nel pezzo d'addestramento vede comunque solo i rendimenti di quel pezzo. Il prezzo: le posizioni aperte prima dell'inizio di un pezzo di prova restano aperte, come lo sarebbero dal vivo.
**7. La regola dell'altopiano.** Il punteggio di una variante è il più basso tra il suo Sharpe e la media del suo intorno (lei e le vicine a distanza 1 su ogni parametro della griglia). Con la sola media dell'intorno vincevano gli angoli della griglia accanto a un picco isolato; con il minimo vince una collina larga, non una punta. A parità vince la prima variante nell'ordine della griglia.
**8. Le misure delle convalide.**
- Finestre che avanzano: addestramento che si allarga, 4 finestre di prova.
- K-fold purgato combinatorio: 6 gruppi, 2 in prova per volta, quindi 15 divisioni e 5 percorsi.
- Purga ed embargo: l'1% delle sedute, almeno una.
- PBO con il metodo CSCV: 16 gruppi, ridotti a un numero pari con almeno 4 sedute per gruppo. Un logit uguale o sotto zero conta come sovra-adattamento; nei pareggi il rango è la media. Con una sola variante il PBO non si applica e il rapporto lo dice.
**9. Il numero di prove del DSR.** N è la somma delle varianti di tutte le prove concluse della stessa strategia più quelle della prova corrente; la varianza degli Sharpe è quella di tutte queste varianti. Ogni nuovo tentativo alza l'asticella dei successivi.
**10. Il piano si scrive prima e non cambia.**
- Prima di qualsiasi calcolo si salvano il piano in JSON (strategia e versione, griglia, dati, capitale, moltiplicatori dei costi, misure delle convalide, soglie, versione del motore) e la sua impronta SHA-256. Il codice non aggiorna mai piano e impronta: un test controlla che dopo la corsa testo e impronta siano identici. Prima di partire la prova ricalcola l'impronta: se il piano non le corrisponde più, non parte.
- Su PostgreSQL la colonna è `json`, non `jsonb`, così il testo resta identico e l'impronta si ricontrolla.
- Le soglie sono nel piano: risultato netto positivo con costi doppi, DSR almeno 0,90, PBO non oltre 0,40, calo massimo non oltre il 25%, nessuna decisione cambiata dai dati futuri. La prova è superata se nessun controllo è fallito; un controllo che non si applica non la fa fallire.
**11. Il test contro lo sguardo in avanti.** Si taglia lo storico a 6 date e si rifà la decisione su dati che finiscono lì: le intenzioni devono essere identiche a quelle della corsa completa. Una strategia volutamente scorretta, che legge il giorno dopo, viene presa. Lo stesso confronto "un passo" servirà nella Fase 6 per paragonare dal vivo e backtest (12.3), con una tolleranza del 10% sulla dimensione.
**12. Il laboratorio non disturba il bot.**
- Una prova alla volta, su un thread suo a priorità bassa. Solo la parte asincrona tocca il database.
- All'avvio le prove rimaste "in corso" tornano "pianificate" e ripartono con lo stesso piano.
- Una prova fallita tiene il motivo.
- Massimo 20 strumenti e 200 varianti per prova, almeno 30 sedute, capitale tra 1 e 10.000.000. I parametri che contano giorni accettano solo numeri interi: 10,5 girerebbe come 10 e conterebbe due volte nel DSR.
- Un errore del database (per esempio PostgreSQL che riparte) non ferma la coda: si riprova dopo 30 secondi.
- Nella modalità dimostrativa il laboratorio è spento.
**13. Le due origini dei dati.**
- "Barre raccolte": le barre giornaliere del database, con frazionamenti e dividendi per le azioni.
- "Storico di ricerca": gli archivi Binance (Fase 2), nella loro valuta (USDT). Le regole d'ordine non ci sono negli archivi: passo di prezzo e di quantità di 0,00000001 e controvalore minimo di 5 (come i minimi tipici delle borse crypto: un ordine più piccolo non parte e un residuo sotto i 5 si perde all'uscita); i costi sono quelli di OKX. Il rapporto lo scrive. La conversione in euro (annunciata nello STATE della Fase 2) non serve: una prova lavora in una sola valuta e rifiuta strumenti quotati in valute diverse.
**14. Il rapporto.** Contiene i controlli del protocollo, la variante scelta contro "comprare e tenere" (pesi uguali, stessi costi; uno strumento la cui storia comincia dopo si compra alla sua prima chiusura, con la quota rimasta libera per lui), i risultati per anno, la sensibilità ai costi, DSR, PBO, finestre, percorsi, sguardo in avanti, tutte le varianti con il loro punteggio, ciò che il test non considera, l'impronta dei dati e le operazioni. La curva tiene al massimo 500 punti e l'elenco le ultime 300 operazioni, per restare leggero nel database. Il confronto tra due prove dice se hanno letto gli stessi dati.
**15. Le sezioni che arrivano dopo.** "Migliorie in osservazione" (8.7) e "Stime di costo da confermare" (11.6) sono già nella schermata, vuote, con la fase che le riempie (9 e 8).
## Conseguenze
- Una prova è riproducibile: stesso piano e stessi dati danno lo stesso rapporto, e il registro non si può ritoccare dopo aver visto i risultati.
- I backtest delle tre strategie di base (Fase 5) useranno questo laboratorio così com'è; i correttivi e il rischio veri entreranno nei punti già pronti.
- La prova su PostgreSQL della tabella delle prove passa dalla CI, perché su questo PC Docker non era disponibile.
+90
View File
@@ -0,0 +1,90 @@
# 0011 — Scelte della Fase 5: strategie, portafoglio, rischio
Data: 2026-11-11 · Stato: accettata · Precisa la specifica (sezioni 7.1, 7.5, 8, 9, 12, 13.1–13.6) e l'ADR 0010, senza aggiungere componenti esterni
## Contesto
La Fase 5 porta le tre strategie di base, i correttivi comuni, il motore del rischio, gli stadi verso i soldi veri, i backtest onesti delle strategie e le schermate Operazioni, Strategie e Rischio. Alcune scelte non sono scritte nella specifica, o la precisano. Le raccolgo qui.
## Decisione
**1. Le strategie sono le stesse in prova e dal vivo.**
- Le tre strategie (`crypto_trend`, `equity_momentum`, `equity_meanrev`) sono classi senza stato proprio: leggono le barre, il loro portafoglio e il patrimonio di riferimento, e restituiscono intenzioni (peso voluto e stop). Ogni intenzione ha un codice motivo, mai un testo libero.
- Il motore di backtest ha imparato ciò che le regole chiedono: l'uscita alla stessa chiusura (`equity_meanrev`, che dal vivo vende alle 15:55), lo stop messo a una distanza dal prezzo ottenuto, gli strumenti di solo dato come SPY, il calendario di borsa e la storia del patrimonio.
- Il patrimonio di riferimento di una strategia è quello della sua quota del bot (`ReferenceShare`): i limiti in percentuale si misurano sul bot, non sulla sola strategia.
- Tutte e tre passano il test contro lo sguardo in avanti dell'ADR 0010 (punto 11).
**2. Correttivi e rischio girano anche nei backtest.** I correttivi di 8.5 (regime crypto: metà delle nuove entrate quando Bitcoin chiude sotto la media a 200 giorni o la sua volatilità supera il 90° percentile di due anni; calendario economico: niente entrate da 30 minuti prima a 30 dopo un dato USA importante, fino a 60 dopo una decisione di Fed o BCE) e le regole di 9.1 e 9.2 sono passi della decisione, gli stessi dal vivo e in prova. Nel backtest di una strategia:
- la strategia è una parte del bot e il resto conta come liquidità ferma, così una sua perdita pesa sul mondo quanto la sua quota;
- ogni chiusura è una giornata italiana a sé (mezzogiorno UTC);
- dopo che l'interruttore scatta si suppone che l'utente riparta una settimana dopo, e il massimo si misura da lì;
- età del prezzo, prezzo dell'ordine e orologio esistono solo dal vivo.
**3. Il motore del rischio toglie o riduce, mai aggiunge.**
- Ogni intenzione lo attraversa prima di diventare un ordine; la protezione (stop e uscite) passa sempre.
- Le tre misure di 9.2 (perdita del giorno da mezzanotte italiana, degli ultimi sette giorni, calo dal massimo) vengono dall'indice ponderato nel tempo di ciascun mondo, così versamenti e prelievi non sembrano guadagni o perdite. Simulato e reale sono due mondi: un limite ferma solo il mondo dove è stato superato.
- Il monitor è una macchina a stati pura, la stessa in prova e dal vivo, e il suo moltiplicatore è sempre uno o meno.
- L'avviso parte all'80% di una soglia, una volta per superamento, senza blocchi.
- La protezione per le operazioni in giornata (`pdt_guard`) è spenta in partenza: Anubis non usa il margine.
- I test di proprietà controllano che il risultato non sia mai più grande del richiesto, e mai più grande dopo una perdita.
**4. Fermare e ripartire passa sempre dal motore del rischio.** "Ferma tutto", "Ferma e chiudi", il file `STOP` nella cartella dei dati e l'anomalia di ordini al minuto fermano il bot finché l'utente non conferma la ripartenza. La ripartenza è rifiutata finché il file `STOP` esiste. Dopo uno scatto il massimo si misura dalla ripartenza.
**5. Ogni passo di una decisione resta scritto.** La nuova tabella `decisions` tiene una riga per passo (strategia, correttivi, dimensionamento, rischio, ordine) con codici motivo e numeri usati. La scheda "perché" di Operazioni collega decisioni ed esecuzioni con l'id d'ordine deterministico (`ClientOrderIds.Key(decision_id)`) e scrive le frasi da modelli italiani fissi.
**6. Gli stadi sono l'unica strada verso i soldi veri.**
- Quattro stadi: osservazione, simulazione, reale ridotto, reale. Da osservazione a simulazione servono il backtest superato e almeno 90 giorni di segnali raccolti dal vivo (12.3); solo il primo avvio porta in simulazione una strategia di base che ha superato il backtest, come dice la specifica.
- Un ordine su un conto reale richiede il consenso della configurazione d'avvio, tutti i requisiti di 12.3 soddisfatti, la frase scritta a mano e password (più il secondo fattore, se attivo) inseriti da poco. I gestori degli ordini sui conti reali, in caso di dubbio, rifiutano e richiedono la conferma prima di ogni ordine.
- Da osservazione a simulazione si passa con un pulsante, ma il libro degli stadi ricontrolla i due requisiti: una strategia che non li ha resta in osservazione e la schermata dice perché.
- La password inserita per la procedura conta i tentativi come l'accesso: dopo cinque errori in 15 minuti è rifiutata anche quella giusta, finché la finestra non passa.
- Prima della conferma la schermata riassume cosa cambia: capitale usato, acquisti del primo ciclo (stimati da ciò che la simulazione tiene in quel momento), limiti e la loro base. Per una strategia con soldi veri mostra la differenza di risultato con il gemello simulato, in percentuale del capitale usato.
- Con soldi veri in gioco l'esito del backtest con cui la strategia si è qualificata non cambia, nemmeno con un aggiornamento del programma: lo riceve quando torna in simulazione. L'esito tiene anche i parametri della variante qualificata, che sono quelli che girano dal vivo (Fase 6).
- Una strategia retrocessa o revocata tiene le posizioni reali in sola uscita, con i loro stop, fino a zero.
- Il gemello simulato di una strategia con soldi veri resta fuori dai totali e dal patrimonio di riferimento.
- Il patrimonio di riferimento è il capitale affidato più i risultati, contati una volta sola. A ogni cambio di stadio il registro che la strategia lascia porta con sé ciò che ha prodotto da quando era in uso (per quattro se viene dal reale ridotto), e il registro che prende il suo posto parte da ciò che ha in quel momento. Così il passaggio da reale ridotto a reale, sullo stesso conto, e il ritorno al gemello dopo una retrocessione non contano due volte gli stessi guadagni. Gli importi restano nella valuta del registro e diventano euro al momento del calcolo.
- Nel calcolo della retrocessione per il percorso atteso le azioni non contano i fine settimana, in cui non ci sono sedute.
**7. Alzare il rischio costa la stessa conferma.** Abbassare un limite è sempre permesso. Alzare un limite o il capitale affidato mentre una strategia usa soldi veri chiede la frase di conferma (`RaisePhrase`) e la password, come la procedura di 12.4. Non passa comunque finché una strategia con soldi veri ha una condizione di retrocessione in corso (12.4). Ogni cambio finisce nel registro delle impostazioni. I limiti non si cambiano dall'API generale delle impostazioni (`PUT /api/settings`): solo dal modulo di Impostazioni, che fa questi controlli.
**7 bis. Liquidità, strumenti e fermate.**
- Un acquisto in corso tiene impegnata la sua liquidità finché non è eseguito, e ogni acquisto conta al prezzo più alto a cui il suo limite può arrivare (la distanza ammessa dall'ultimo prezzo): più acquisti inviati insieme non superano la liquidità della strategia.
- In uno stesso conto uno strumento appartiene a una sola strategia alla volta (8.6): se un'altra lo tiene, o lo sta comprando, l'ingresso salta con il suo motivo.
- "Ferma tutto", il file `STOP` e l'anomalia di ordini al minuto sono eventi di gravità alta, non critici: sono il bot che fa il suo lavoro, non un controllo perso, e non bloccano per 30 giorni il passaggio di stadio (12.3).
- Quando l'interruttore scatta da solo in un mondo, il bot resta "attivo" per l'altro: la pagina Rischio mostra lo scatto di entrambi i mondi e il pulsante "Riparti" anche in questo caso. La ripartenza toglie lo scatto e lascia una pausa com'era.
- Nella pagina Rischio peso di una posizione, quota crypto e rischio di un'operazione si misurano sul patrimonio di riferimento, come nel motore del rischio.
**8. Gli scenari sono una somma semplice.** "Se succedesse domani" applica la variazione al valore delle posizioni del momento. "Con gli stop" prende per ogni posizione la perdita più piccola tra lo scenario e la distanza dal suo stop, arrotondata a 5 € e scritta come "circa", perché uno stop non garantisce il prezzo. Lo scenario del dollaro tocca il conto in dollari e gli stop non lo riducono.
**9. I risultati per strategia si arrotondano come in Conti.** Ogni posizione si arrotonda al centesimo prima di sommare, così Strategie e Conti mostrano lo stesso numero.
**10. Lo storico crypto in euro, per i backtest delle strategie.** L'ADR 0010 (punto 13) lasciava i dati Binance in USDT; per le strategie di base serve l'euro, perché il bot pensa in euro.
- Le barre giornaliere USDT si dividono per il cambio BCE EUR/USD dello stesso giorno, o dell'ultimo precedente fino a 7 giorni prima; altrimenti la barra si scarta. Si salvano come sede `binance_eur`.
- Una serie si ferma al primo buco più lungo di 7 giorni (`MaxGapDays`): ciò che segue è un'altra moneta. Il LUNA rilanciato nel 2022 non si confonde con quello crollato, che si taglia al 13/05/2022.
- L'elenco degli archivi comprende le monete tolte dal listino (49 coppie, con LUNA, FTT, BCC, VEN, XMR…): l'universo passato non è scelto tra i sopravvissuti.
- Ogni barra usa l'ultimo cambio fissato prima della sua apertura, cioè quello del giorno prima: il cambio del giorno d è fissato alle 16:00, dopo l'apertura della barra di d alle 00:00 UTC, e il backtest esegue all'apertura.
- `anubis backtest base --offline` scarica i cambi BCE solo se il file manca.
**11. Il protocollo dei backtest di base.**
- Periodo dal 01/01/2019 al 31/10/2026. Le prove leggono anche 450 giorni prima dell'inizio (`TrialData.WarmupDays`), solo perché indicatori ed età minima di quotazione (400 giorni) siano pronti dal primo giorno. Il controllo "strumento vuoto" e il conteggio delle sedute guardano solo dentro il periodo.
- `crypto_trend` si prova su due universi: quello largo e solo BTC, ETH e SOL. Supera solo se superano entrambi; le misure mostrate vengono dall'universo largo.
- Nel DSR il secondo universo conta come prove precedenti anche le varianti del primo. L'id della prova viene dall'impronta del piano, così rifare il calcolo non crea prove nuove.
- Per gli strumenti dell'universo largo il limite del Laboratorio sale a 60 (`Laboratory.MaxSymbols`).
- Il costo di andata e ritorno è 2 × (commissione taker + scostamento); lo scostamento è pesato sul controvalore scambiato.
- Il percorso atteso a un anno viene da un bootstrap a blocchi stazionario: 2.000 percorsi, blocchi medi di 20 giorni, seme 20260101, 365 giorni.
- Il rapporto mostra lo Sharpe annuo (× √sedute dell'anno) e DSR e PBO con tre decimali, perché "0,90 serve ≥ 0,90" con due decimali confondeva.
- Il rapporto conta quante volte, per strumento e giorno, un correttivo o una regola di rischio ha cambiato la richiesta della strategia, separate dalle note del motore (ordini sotto il minimo, residui).
- "Comprare e tenere" dell'universo largo compra fin dall'inizio tutte le 47 monete che sono state tra le prime dieci in qualche momento: è un paniere scelto col senno di poi, quindi favorisce il confronto e rende il giudizio sulla strategia più severo, non più indulgente.
**12. L'esito dei backtest viaggia con il codice.** I file `docs/backtests/*.outcome.json` sono inclusi nell'applicazione e letti a ogni avvio. Gli esiti azionari sono "non superato, in attesa delle chiavi Alpaca": senza storico azionario le due strategie azionarie restano in osservazione.
**13. L'esito di `crypto_trend` è "non superato".** Con l'universo largo il calo massimo è del 32,8% (limite 25%); con BTC, ETH e SOL la probabilità di sovra-adattamento è 0,84 (limite 0,40). Rendimento annuo 10,4% contro il 32,9% di comprare e tenere. La strategia resta in osservazione. Non ho cambiato regole o soglie dopo aver visto il risultato: farlo renderebbe il test inutile.
Una prima versione del calcolo usava il cambio dello stesso giorno della barra e dava un calo del 29,1%. Spostare il cambio di un giorno (punto 10) ha mosso il calo di quasi quattro punti: anche questo dice che il risultato è fragile.
**14. La demo è coerente con i mockup.** La modalità dimostrativa semina le decisioni dietro le sue operazioni, 58 giorni di simulazione, una qualifica d'esempio e gli indici delle curve, così Operazioni, Strategie e Rischio mostrano le cifre dei mockup 03, 04 e 07. Nella demo nessuno stadio cambia e i moduli del rischio non ci sono.
## Conseguenze
- Nessuna strategia può arrivare alla simulazione senza un backtest superato, né ai soldi veri senza la procedura di 12.4. Oggi tutte e tre restano in osservazione: è l'esito onesto, non un difetto.
- Le prove dei backtest azionari partono quando l'utente fornisce le chiavi del conto paper di Alpaca.
- Cambiare una regola di `crypto_trend` per farla passare richiede una nuova versione della strategia e una nuova prova, che il DSR conterà insieme a quelle già fatte.
+148
View File
@@ -0,0 +1,148 @@
# 0012 — Scelte della Fase 6: motore dal vivo in simulazione
Data: 2026-11-11 · Stato: accettata · Precisa la specifica (sezioni 5.5, 5.6, 7.4, 7.5, 9.3, 9.4, 13, 14) e gli ADR 0009–0011, senza aggiungere componenti esterni
## Contesto
La Fase 6 porta il bot che lavora da solo in simulazione: la giornata di 5.5, il recupero dopo un'assenza, il processo di guardia, il diario con le frasi dell'appendice B, Telegram con il segnale di vita, le schermate Panoramica e Diario e sistema, e la prova di durata in tempo reale. Alcune scelte non sono scritte nella specifica, o la precisano. Le raccolgo qui.
## Decisione
**1. Il motore dal vivo decide con la stessa chiamata del backtest.**
- Ogni decisione passa da `DecideOnce`, la stessa funzione che usa il motore di backtest, così prova e simulazione non possono divergere. Il test di parità lo controlla su 40 giorni di `crypto_trend` e su 6 settimane di `equity_meanrev`, con correttivi e rischio attivi.
- Gli orari sono quelli di 5.5:
- chiusura crypto alle 00:05 UTC;
- segnali azionari 20 minuti dopo la chiusura di Wall Street;
- uscite di `equity_meanrev` 5 minuti prima della chiusura;
- ordini azionari e stop del giorno all'apertura.
- Una strategia gira con i parametri scritti nel suo file dell'esito (`parameters`), cioè quelli che si sono qualificati, non con quelli predefiniti.
- Il motore parte dal pianificatore solo dopo che i conti sono stati riconciliati.
**2. Ogni decisione si prende una volta sola, anche attraverso le ripartenze.**
- La decisione ha una chiave (`run_key`: strategia e chiusura su cui decide). Questa chiave viene scritta nella stessa transazione delle righe di decisione, degli ordini pianificati e delle righe del diario: o c'è tutto o non c'è niente.
- L'id di ogni ordine pianificato è l'id della decisione dell'intenzione, e da lì viene la chiave dell'ordine. Una ripartenza che trova l'esecuzione già presente non la manda una seconda volta.
- Gli ordini azionari decisi dopo la chiusura aspettano l'apertura nella tabella `planned_orders`. Ognuno ha una scadenza e un prezzo di riferimento.
- La prova accelerata copre 14 giorni con un guasto diverso nei giorni indicati: rete giù, una risposta persa, una ripartenza con un ordine a metà e il database irraggiungibile. Su SQLite e PostgreSQL termina riconciliata e senza ordini doppi.
**3. Il recupero dopo un'assenza (5.6).**
- L'assenza si misura all'avvio dall'ultimo lavoro finito (`jobs.last_finished_at`). Sotto i 5 minuti è una ripartenza, non un'assenza.
- Ogni assenza va nel diario. Una più lunga di un giorno resta in cima alla Panoramica finché l'utente non l'ha vista.
- Ogni lavoro dichiara cosa fare con le esecuzioni perse:
- salvataggio, cambi, calendario di borsa e pulizia girano una volta alla ripartenza, comunque in ritardo;
- un lavoro con una scadenza si salta dopo quella scadenza, e il diario lo dice.
- Le azioni societarie si leggono prima che la riconciliazione rimetta gli stop.
- Una decisione presa in ritardo per l'assenza va nel diario come recuperata. Un'entrata il cui prezzo si è mosso troppo o che è scaduta si lascia cadere, con il suo motivo.
**4. La guardia è il processo principale e il genitore del motore (9.3, 9.4).**
- In Docker il punto d'ingresso è `Anubis.Guard.dll`, che avvia `Anubis.Server.dll` come processo figlio con gli stessi argomenti.
- I due processi parlano su una pipe locale con nome (`ANUBIS_GUARD_PIPE`), un messaggio JSON per riga:
- il motore manda il battito, i passi dell'avvio, le chiavi dei conti (tenute solo in memoria) e gli stop che decide;
- la guardia manda solo "stop".
- La guardia interviene in tre casi:
- due minuti senza battito;
- un avvio fermo sullo stesso passo per dieci minuti;
- un'uscita del motore senza preavviso.
- Un salto dell'orologio oltre 30 secondi (computer sospeso) non conta come silenzio.
- Quando interviene, la guardia:
1. ferma il motore;
2. annulla i nostri acquisti e le nostre vendite rimasti aperti sui conti di prova;
3. rimette gli stop che le posizioni richiedono, leggendoli dal file di stato che il motore scrive ogni dieci secondi;
4. scrive una nota e avvisa l'utente;
5. fa ripartire il motore.
- La terza ripartenza forzata in un'ora scrive il file STOP.
- Gli stop rimessi dalla guardia saltano 500 numeri di sequenza (`GuardRepair.SequenceGap`), così il loro id non si scontra mai con uno che il motore stava per mandare.
- Alla ripartenza il motore, prima di riconciliare:
- adotta gli stop della guardia;
- annulla gli ordini che aveva lasciato aperti (motivo `restart`);
- pianifica di nuovo le uscite già decise, che seguono così le regole di 5.6.
- Lo stop del container (SIGTERM) passa al motore, che chiude in ordine e non viene fatto ripartire. Il compose e il template Unraid concedono 60 secondi; la guardia ne aspetta 55.
- **I conti reali per ora sono fuori dalla guardia.** La guardia apre solo conti di prova. Avrà i conti reali con la fase dei soldi veri, quando riceverà anche la procedura di avvio di 12.4. Oggi nessuna strategia usa soldi veri.
- La guardia usa il runtime condiviso di ASP.NET (`FrameworkReference`): server e guardia stanno così nella stessa cartella `/app` dell'immagine, con le stesse librerie.
**5. La prova della guardia ha due appigli, solo per la prova.**
- `ANUBIS_FAULT=hang-mid-sale` blocca il ciclo principale subito dopo l'invio di una vendita. La guardia non lo passa mai al motore che fa ripartire.
- `ANUBIS_DRILL_OKX_URL` indirizza il conto OKX della prova a un finto OKX. Funziona solo con indirizzi locali (loopback).
- La prova (`Anubis.Guard drill`) gira con processi veri in una cartella temporanea, anche dentro l'immagine Docker: lo fanno `tasks ci` e la CI. Fa sette controlli:
- la guardia è intervenuta;
- il motore è ripartito;
- la vendita era partita;
- la vendita rimasta a metà è stata annullata;
- lo stop è tornato per l'intera quantità;
- la vendita già decisa è ripartita con un ordine nuovo;
- la nota è nel diario.
**6. Il diario si scrive con frasi fisse.**
- Le frasi nel tono dell'appendice B vengono da modelli (`DiaryText`) riempiti solo con ciò che è successo: esecuzioni con il risultato dell'operazione che chiudono, stop alzati, chiusura crypto, segnali azionari, segnali in osservazione, recupero e assenza.
- Le frasi dei codici motivo stanno nella libreria (`ReasonCatalog`), condivise da motore e schermate. Un test controlla che ogni codice abbia la sua frase.
- I generi del diario sono quelli della schermata: Ordini, Decisioni, Sistema, Notizie, Apprendimento.
**7. Il confronto "senza bot" (7.5).**
- Ogni movimento di capitale viene investito nel suo giorno: 40% in BTC/EUR e 60% in SPY rettificato, convertito con il cambio BCE. Poi non si tocca più.
- Un prelievo vende i due nella proporzione che hanno raggiunto.
- I prezzi sono le chiusure del giorno prima, così un versamento non sembra mai un guadagno.
- Senza lo storico di SPY (niente chiavi Alpaca) la fotografia lascia vuoto il confronto, invece di mostrarne metà.
**8. Telegram (14).**
- Ogni avviso va prima in coda (`notifications`), nella stessa transazione di ciò che racconta, e poi si spedisce. Il passo di spedizione gira ogni dieci secondi e manda al massimo 20 messaggi.
- Ogni messaggio ha uno stato: in attesa, spedito, oppure non riuscito con il suo errore.
- Rete giù e 429 si riprovano dopo 1, 2, 4… fino a 60 minuti, o dopo l'attesa indicata da Telegram se è più lunga; al massimo 8 tentativi.
- Un rifiuto di Telegram è definitivo.
- Un messaggio che non è partito entro un giorno si abbandona con il motivo, invece di arrivare tardi come se fosse una notizia.
- I comandi si leggono con il long polling. Valgono solo:
- dalla chat configurata (le altre si ignorano in silenzio);
- se hanno meno di 5 minuti, così un /pausa di ieri non mette in pausa oggi.
- `/ferma` chiede `/conferma` entro due minuti; qualsiasi altro comando nel frattempo la annulla.
- Ripartire dopo "Ferma tutto", i soldi veri, i limiti e le chiavi restano nell'interfaccia.
- Lavori in programma:
- riepilogo del mattino alle 08:00, recuperabile entro quattro ore;
- stato ogni ora o alle 20:00, solo se l'impostazione `telegram.status` lo chiede.
- Nella modalità dimostrativa Telegram è spento.
- Il passo Telegram del primo avvio resta in Impostazioni → Telegram: la pagina del primo avvio non ha ancora un database dove tenere i segreti. Quella pagina lo dice, e sul desktop raccomanda il segnale di vita.
**9. Il segnale di vita.** Ogni minuto, se c'è un indirizzo, il bot chiama un servizio esterno (per esempio un controllo di Healthchecks.io).
- L'indirizzo viene da `ANUBIS_HEARTBEAT_URL`, oppure dal segreto `HEARTBEAT_URL` impostato da Impostazioni.
- È il servizio esterno ad avvisare se le chiamate smettono: copre il caso in cui bot e guardia sono spenti insieme.
**10. Panoramica e Diario e sistema (13.6).**
- Le prossime mosse vengono dai lavori del pianificatore che sono mosse del bot; i lavori a intervallo (ogni minuto, ogni 5 minuti) non compaiono. Gli eventi del calendario economico tra le mosse ci sono solo nella demo, che usa i valori dell'appendice F.10; fuori dalla demo la riga "Prossima decisione" dice "tra X, alle HH:MM".
- Notizie e Apprendimento, fuori dalla demo, dicono "non ancora attivo" fino alle Fasi 7 e 8.
- Il Diario filtra per genere con semplici collegamenti, senza JavaScript. Il filtro Rischio raccoglie i codici del motore del rischio e quelli di ordini o stop rifiutati, chiavi con prelievo, conto prudente, intervento della guardia e salto dell'orologio. La ricerca ignora maiuscole e accenti, guarda le ultime 5.000 righe e ne mostra al massimo 200. "Mostra i giorni precedenti" aggiunge 7 giorni e compare solo se ci sono righe più vecchie.
- Non esiste ancora una pagina per una singola operazione: "Perché l'ho comprata e venduta" porta a Operazioni filtrata su quel titolo, dove la scheda "perché" si apre.
- Il computer che lo ospita: il processore è la quota di questo processo dall'ultima lettura; la memoria è quella del processo sul limite che .NET vede (nel container, il limite del container); lo spazio è quello del disco della cartella dei dati.
- "Scarica i registri tecnici" (`/api/registri.zip`) chiede l'accesso e mette nello zip solo i file `anubis-*.log` della cartella dei registri, già oscurati quando sono stati scritti.
**11. La prova di durata.**
- `anubis soak` legge i registri del bot su una finestra di tempo e scrive un rapporto, senza cambiare niente. Può girare mentre il bot lavora, in Docker con `docker exec`. I suoi controlli:
- durata coperta;
- bot al lavoro dall'inizio alla fine (vedi il punto 12);
- nessun ordine né decisione doppia;
- nessuna differenza inspiegata e ultima riconciliazione in regola per ogni conto;
- nessuno spegnimento;
- nessun intervento della guardia;
- una decisione `crypto_trend` per ogni chiusura giornaliera.
- La prova gira sull'immagine Docker di questo PC, con un volume suo, senza chiavi: conto crypto sul simulatore interno con prezzi veri di OKX, nessuna parte azionaria.
- `crypto_trend` non ha superato il backtest (ADR 0011, punto 13), quindi resta in osservazione: decide ogni notte, ma non manda ordini. Non la porto in simulazione per far vedere ordini: sarebbe aggirare il suo esito. Gli ordini, le ripartenze e i guasti sono coperti dalla prova accelerata (punto 2) e dalla prova della guardia (punto 5).
**12. Correzioni dopo la revisione indipendente.** Una revisione del codice della fase ha trovato nove difetti; le correzioni scelte sono queste.
- **Un conto parte solo quando è in ordine.** All'avvio ogni conto prima adotta gli stop messi dalla guardia, poi controlla la chiave, si riconcilia, riprende o annulla ciò che era rimasto a metà e allinea gli stop. Solo se tutto riesce il conto è "pronto". Finché non lo è, il motore non decide su di lui e non gli manda ordini pianificati; il ciclo riprova l'avvio ogni minuto. Prima, un broker che non rispondeva all'avvio lasciava il conto lavorare su un registro non riconciliato.
- **Gli stop della guardia si adottano prima di tutto il resto.** La nota della guardia si cancella solo quando ogni stop che ha messo risulta nel registro degli ordini; se l'adozione si interrompe, la nota resta e si riprova.
- **Un turno di conto ha un tempo massimo**: 45 secondi per il giro degli ordini, 60 per un nuovo avvio. Un broker lento non tiene più fermo il segnale di vita per i due minuti dopo i quali la guardia interviene: il turno si interrompe, si scrive nel registro e si riprova al giro dopo. Un motore davvero bloccato resta in silenzio e la guardia lo sente.
- **Il segnale di vita guarda anche il pianificatore.** Se il pianificatore, dove girano le decisioni, tace da 30 minuti (i suoi lavori più lunghi durano pochi minuti), il ciclo degli ordini smette di battere e la guardia interviene.
- **Le uscite alla chiusura di `equity_meanrev` sono ordini pianificati**, scritti nella stessa transazione che prende la chiave della decisione, con scadenza alla chiusura della sessione. Il rilascio li manda subito; una vendita con scadenza parte già aggressiva (limite vicino al prezzo opposto). Prima venivano mandate direttamente: un riavvio a metà poteva perderle o mandarle due volte.
- **La ripresa trova anche l'ordine pianificato non ancora collegato** alla sua esecuzione (riavvio tra l'invio e la scrittura del collegamento), dalla chiave dell'ordine, e lo chiude prima di ripianificare il resto.
- **Una decisione azionaria fatta dopo l'apertura** (per esempio dopo un'assenza) è segnata in ritardo e segue le regole del recupero.
- **I cicli di Telegram non si fermano per un errore inatteso** (database occupato, rete): scrivono nel registro e riprovano.
- **Lo spegnimento lungo si vede in Panoramica** (SPEC 5.6): un avviso in cima dice per quanto il bot è rimasto spento e rimanda al diario, finché l'utente non preme "Ho visto".
- **Una decisione in ritardo aspetta i suoi dati.** Dopo uno spegnimento, al primo battito i dati del giorno prima non sono ancora arrivati: una decisione fatta dopo le 12:00 UTC li chiede e li aspetta fino a 30 minuti, invece di chiudersi subito come "senza dati" e andare persa. Il difetto si è visto all'avvio della prova di durata.
- **Il rapporto della prova di durata non conta più le righe di riconciliazione**, che si scrivono solo quando c'è una differenza. Legge invece:
- quando i lavori in programma hanno cominciato e quando hanno finito l'ultima volta;
- gli spegnimenti, scritti a ogni ripartenza dopo almeno 5 minuti;
- gli interventi della guardia, per un ciclo fermo con il processo vivo;
- lo stato di ogni conto: riconciliato negli ultimi 15 minuti della finestra e in modalità normale.
- Il segnale di vita in `http://` è accettato solo verso indirizzi della rete di casa (10.x, 172.16-31.x, 192.168.x, locali e i loro equivalenti IPv6); il rapporto della prova di durata non conta una chiusura giornaliera avvenuta meno di 30 minuti prima della fine della finestra.
## Conseguenze
- Un riavvio, un crash o un'assenza non producono decisioni né ordini doppi: le chiavi e le transazioni lo garantiscono, e i test lo provano su entrambi i motori.
- I conti reali, quando arriveranno, avranno bisogno che anche la guardia sappia aprirli: è scritto qui per non dimenticarlo.
- Il rapporto della prova di durata si controlla come primo passo della Fase 7.
@@ -0,0 +1,84 @@
# 0013 — Raccolta dei dati per l'analisi al posto della prova di durata in casa
Data: 2026-11-14 · Stato: accettata · Precisa il criterio della prova di durata (SPEC 17, Fasi 6 e 7, issue #55) e aggiunge l'issue #95, senza aggiungere componenti esterni
## Contesto
La prova di durata di 48 ore della Fase 6 girava nel container `anubis-prova` di questo PC. In tre giorni il computer l'ha interrotta più volte, mai Anubis:
- uno spegnimento improvviso del PC;
- Docker chiuso insieme all'app Claude durante un suo aggiornamento;
- un blocco del motore di Docker durato 18 ore.
Il 14 novembre 2026 l'utente ha scelto un'altra strada. Eseguirà l'ultima release pubblicata su Gitea per il tempo che vuole, sul suo computer e con il suo uso normale. Poi incollerà in chat un rapporto, da cui si analizza come si è comportato il bot. Serve quindi una raccolta di dati più ricca di quella che il rapporto `soak` legge dai soli registri del trading, e serve una release da scaricare.
## Decisione
**1. La verifica della prova di durata passa al rapporto della release.**
- Il criterio della Fase 7 "Il rapporto della prova di durata è in `docs/` e non riporta ordini duplicati né differenze non spiegate" resta com'è.
- Il rapporto non viene più dal container di prova. È quello che l'utente incolla dopo aver fatto girare la release, salvato in `docs/diagnostica/`.
- Il rapporto contiene gli stessi controlli di `anubis soak` (sezione «Controlli del lavoro»). Uno spegnimento voluto dall'utente compare come ✗ della continuità e va spiegato, non è un difetto.
- La prova nel container `anubis-prova` è chiusa. Il suo primo tratto resta in `artifacts/prova-48h/primo-tratto/`. I due compiti programmati che la sorvegliavano sono spenti o cancellati.
**2. Che cosa si raccoglie (issue #95).** Tutto resta nel database del bot e niente esce dal computer.
- Ogni 5 minuti, e una volta a ogni avvio, una fotografia del processo nella tabella `diagnostic_samples`:
- memoria (privata, di lavoro, gestita, impegnata dal GC);
- raccolte del GC per generazione e tempo di pausa;
- thread, handle e coda del pool;
- tempo di processore e contese sui lock;
- eccezioni lanciate;
- dimensione del database;
- righe di avviso e di errore dei registri tecnici;
- chiamate HTTP in uscita.
- I contatori sono totali dall'avvio del processo: il rapporto fa le differenze. Il lavoro "diagnostica" non ha recupero: un campione perso mentre il bot era spento non si rifà.
- Ogni esecuzione di ogni lavoro pianificato va in `job_runs`, con scadenza, inizio, fine, esito ed errore. La tabella `jobs` tiene solo l'ultima esecuzione.
- Le chiamate HTTP si misurano ascoltando lo strumento che .NET già pubblica per ogni richiesta in uscita (`System.Net.Http`, `http.client.request.duration`) con un `MeterListener`. Non serve nessun pacchetto e non cambia nessun client.
- Si tengono per destinazione: numero, esiti, fallite.
- Le durate stanno in fasce fisse, così i tratti si sommano senza tenere ogni chiamata.
- Si contano anche le eccezioni di primo lancio. Molte sono normali, ma un conto che sale sempre indica un ciclo che fallisce di continuo.
- Le due tabelle tengono 30 giorni. La pulizia gira una volta al giorno dentro lo stesso lavoro.
- Nella modalità dimostrativa non si raccoglie nulla.
**3. Il rapporto.**
- Si ottiene in tre modi:
- dal pulsante «Scarica i dati per l'analisi» in Diario e sistema;
- da `GET /api/diagnostica.md?ore=N`, che richiede l'accesso;
- dal comando `anubis diagnostics [--hours N | --from … --to …] [--out file.md]`, che legge soltanto e quindi gira anche mentre il bot lavora.
- Senza periodo indicato copre tutto ciò che è stato raccolto, fino a 30 giorni.
- È in Markdown e pesa al massimo 60 KB. Se serve, si tagliano per prime le righe della tabella delle risorse.
- Le sezioni:
- **In breve**: i segnali trovati da regole fisse, per esempio memoria o handle che crescono, lavori falliti o saltati, chiamate che falliscono o sono lente, errori nei registri, silenzi a processo acceso, ripartenze e spegnimenti;
- **Installazione**: versione, commit, sistema, runtime e motore del database;
- **Controlli del lavoro**: quelli di `soak`;
- **Avvii, spegnimenti e silenzi**;
- **Lavori pianificati**: esecuzioni, fallite, ritardi e durate;
- **Chiamate esterne**: per destinazione, con mediana, 95° percentile e massimo;
- **Registri tecnici**: avvisi ed errori raggruppati per messaggio, con numeri e id tolti;
- **Risorse nel tempo**: le pendenze dopo la prima mezz'ora e una tabella di al massimo 48 righe.
**4. Niente di riservato nel rapporto.**
- Tutto il testo passa dal `Redactor`.
- La cartella dell'utente diventa `%USERPROFILE%` e il suo nome nei percorsi diventa `‹utente›`.
- Delle variabili `ANUBIS_*` si mostra il valore solo se il nome non può contenere un segreto: niente `KEY`, `SECRET`, `PASS`, `TOKEN` o `URL`.
- Il rapporto lascia il computer solo quando l'utente lo incolla.
**5. Un tipo di colonna in più: `{long}`.**
- I byte di memoria e i contatori di una lunga esecuzione superano 2^31. SQLite in modalità `STRICT` non accetta `BIGINT`.
- Il segnaposto `{long}` diventa `BIGINT` su PostgreSQL e `INTEGER` su SQLite, che è già a 64 bit.
- `BIGINT` scritto a mano non è più ammesso nelle migrazioni.
**6. Release di prova su Gitea, prima della Fase 10.**
- Prima che esista l'installatore della Fase 10, una release `v0.N.0` porta uno zip per Windows. Dentro ci sono:
- il server, che richiede il runtime ASP.NET Core 10;
- `Avvia Anubis.cmd`;
- un `LEGGIMI.txt` in italiano.
- Il numero di versione segue la fase in corso: `0.7.0` durante la Fase 7.
- Lo zip usa la piattaforma desktop e la cartella dei dati dell'utente (`%APPDATA%\Anubis`). Il primo avvio apre la procedura guidata.
- Mancano le chiavi dei conti reali e non c'è modo di abilitarli (SPEC 2.2).
- L'immagine Docker della stessa versione si costruisce dal tag con `tasks build`. La sua pubblicazione nel registro aspetta il `REGISTRY_TOKEN` (Fase 9).
## Conseguenze
- La Fase 7 non aspetta più una prova in casa: il rapporto arriva quando l'utente lo incolla.
- Fino ad allora l'issue #55 resta aperta.
- Le analisi future partono dal rapporto e da tre tabelle: `diagnostic_samples`, `job_runs` e `absences`.
- Il costo è una riga ogni 5 minuti e una per esecuzione di lavoro. A un mese sono circa 9.000 campioni e qualche decina di migliaia di esecuzioni, una frazione del database.
+286
View File
@@ -0,0 +1,286 @@
# Backtest di `crypto_trend` (Trend crypto)
**Esito: non superato.** Protocollo non superato. Universo ampio: calo massimo 32,8%, serve ≤ 25%. BTC, ETH e SOL: probabilità di sovra-adattamento (pbo) 0,835, serve ≤ 0,40. Resta in osservazione.
Calcolato il 2026-11-11 con la versione 1 della strategia. Soglie, griglia dei parametri, costi e date erano fissati prima del calcolo (SPEC 12.2) e non sono stati cambiati dopo.
Per rifarlo: `anubis backtest base --to 2026-10-31`. Il file dell'esito, letto dal bot al primo avvio, è `docs/backtests/crypto_trend.outcome.json`.
## Dati
- Archivi giornalieri di data.binance.vision in USDT, ogni file controllato con la sua impronta SHA-256 prima dell'uso.
- Prezzi convertiti in euro con il cambio EUR/USD della BCE del giorno (7133 giorni di cambi), contando un USDT come un dollaro. Le coppie in euro di OKX esistono solo da novembre 2023: questo storico è un'approssimazione (SPEC 8.1).
- 47 coppie nell'universo ampio, comprese quelle poi chiuse da Binance: ogni mese la strategia sceglie le 10 con più scambi tra quelle quotate da almeno 400 giorni.
- Lo spread non si può misurare su questo storico: il filtro dello spread vale solo dal vivo.
- Senza archivi nel periodo, quindi non usate: BCCEUR, VENEUR.
## Riepilogo
| Prova | Variante scelta | Rendimento annuo | Calo massimo | Netto a costi doppi | DSR | PBO | Esito |
|---|---|---:|---:|---:|---:|---:|---|
| Universo ampio | `coin_vol=10;max_stop=20` | 10,4% | −32,8% | 7.264,22 € | 0,924 | 0,085 | non superata |
| BTC, ETH e SOL | `coin_vol=20;max_stop=10` | 12,2% | −15,0% | 10.274,96 € | 0,961 | 0,835 | non superata |
## Cosa conserva il bot
- Calo massimo del backtest: −32,8%. Dal vivo, un calo oltre 1,5 volte questo riporta la strategia indietro (SPEC 12.3).
- Scostamento di prezzo previsto: 11,94 punti base; costo stimato di andata e ritorno 0,64%.
- Percorso atteso al 5° percentile (ricampionamento a blocchi dei rendimenti giornalieri): −5,1% dopo 30 giorni, −8,0% dopo 90, −14,1% dopo 365. Venti sedute di fila sotto questo percorso riportano la strategia indietro.
## Universo ampio
Le 10 monete con più scambi, scelte ogni mese tra 47 coppie (anche quelle poi chiuse).
Dal 01/01/2019 al 31/10/2026, 2861 chiusure giornaliere, capitale iniziale 10.000,00 €. Variante scelta: `coin_vol=10;max_stop=20` (la zona della griglia dove il risultato cambia meno, non la migliore). Impronta del piano `b5f566d59bfa`, dei dati `840015607150`.
| | Strategia | Comprare e tenere |
|---|---:|---:|
| Risultato netto | 11.628,28 € | 82.820,55 € |
| Rendimento totale | 116,3% | 828,2% |
| Rendimento annuo | 10,4% | 32,9% |
| Volatilità annua | 14,2% | 74,7% |
| Calo massimo | −32,8% (11/2021 → 10/2023) | −85,8% |
| Sharpe annuo | 0,77 | 0,76 |
| Operazioni chiuse | 1842 | 0 |
| Operazioni in guadagno | 13,1% | — |
| Rapporto guadagni/perdite | 1,47 | — |
| Commissioni pagate | 2.967,95 € | 20,00 € |
| Tempo investito | 16,9% | 92,0% |
**Per anno**
| Anno | Strategia | Comprare e tenere |
|---|---:|---:|
| 2019 | 17,0% | 4,8% |
| 2020 | 36,2% | 103,2% |
| 2021 | 19,8% | 1.603,8% |
| 2022 | −11,2% | −79,9% |
| 2023 | 12,2% | 56,4% |
| 2024 | 26,2% | 33,8% |
| 2025 | −7,4% | −32,9% |
| 2026 | −2,6% | −9,4% |
**Sensibilità ai costi** (commissioni e scostamento moltiplicati insieme)
| Costi | Risultato netto | Rendimento totale | Sharpe annuo | Calo massimo | Commissioni |
|---|---:|---:|---:|---:|---:|
| ×1 | 11.628,28 € | 116,3% | 0,77 | −32,8% | 2.967,95 € |
| ×2 | 7.264,22 € | 72,6% | 0,56 | −35,8% | 5.039,47 € |
| ×3 | 3.522,91 € | 35,2% | 0,34 | −38,9% | 6.471,34 € |
**Convalide**
- Finestre che avanzano (4, con 29 giorni di stacco): rendimento fuori campione 58,4%, Sharpe annuo 0,58.
- K-fold purgato combinatorio (6 gruppi, 2 di prova, 15 divisioni): Sharpe annuo medio dei percorsi 0,77, peggiore 0,77, percorsi positivi 100,0%.
- Deflated Sharpe Ratio 0,92, contando 9 varianti provate in tutto.
- Probabilità di sovra-adattamento (PBO) 0,09, su 12870 combinazioni di 9 varianti.
- Sguardo in avanti: 8583 decisioni rifatte tagliando i dati in 6 punti, 0 cambiate.
**Soglie**
| Controllo | Valore | Soglia | Esito |
|---|---:|---:|---|
| Risultato netto con costi doppi | 7.264,22 € | > 0,00 € | superato |
| Deflated Sharpe Ratio | 0,924 | ≥ 0,9 | superato |
| Probabilità di sovra-adattamento (PBO) | 0,085 | ≤ 0,4 | superato |
| Calo massimo | 32,8% | ≤ 25% | **non superato** |
| Decisioni cambiate aggiungendo dati successivi | 0 | = 0 | superato |
**Varianti della griglia**
| Parametri | Sharpe annuo | Stabilità della zona | Risultato netto | Calo massimo | Operazioni |
|---|---:|---:|---:|---:|---:|
| `coin_vol=10;max_stop=10` | 0,54 | 0,54 | 5.784,07 € | −24,0% | 2269 |
| `coin_vol=10;max_stop=15` | 0,86 | 0,66 | 13.191,01 € | −27,8% | 2008 |
| `coin_vol=10;max_stop=20` (scelta) | 0,77 | 0,75 | 11.628,28 € | −32,8% | 1842 |
| `coin_vol=15;max_stop=10` | 0,39 | 0,39 | 3.932,83 € | −26,3% | 2281 |
| `coin_vol=15;max_stop=15` | 0,71 | 0,61 | 10.147,48 € | −31,9% | 2015 |
| `coin_vol=15;max_stop=20` | 0,68 | 0,68 | 9.971,55 € | −33,4% | 1853 |
| `coin_vol=20;max_stop=10` | 0,35 | 0,35 | 3.437,00 € | −26,8% | 2284 |
| `coin_vol=20;max_stop=15` | 0,59 | 0,55 | 7.855,25 € | −33,4% | 2021 |
| `coin_vol=20;max_stop=20` | 0,57 | 0,57 | 7.673,22 € | −34,6% | 1856 |
**Interventi di correttivi e rischio sulla variante scelta**
Quante volte, per strumento e giorno, un correttivo o una regola di rischio ha cambiato ciò che la strategia chiedeva.
| Motivo | Volte |
|---|---:|
| `crypto_regime_below_average` | 3565 |
| `risk_reduction` | 1590 |
| `crypto_regime_high_volatility` | 151 |
| `trade_risk_limit` | 43 |
| `switch_tripped` | 34 |
| `daily_loss_block` | 14 |
| `stop_above_price` | 13 |
**Note del motore di prova**
Ordini non eseguiti come chiesto: sotto il minimo, residui lasciati dopo un'uscita, liquidità.
| Motivo | Volte |
|---|---:|
| `residue_written_off` | 1837 |
| `below_min_notional` | 449 |
**Strumenti**
| Coppia | Prima barra | Ultima barra | Barre |
|---|---|---|---:|
| ADAEUR | 2018-04-17 | 2026-10-31 | 3120 |
| APEEUR | 2022-03-17 | 2026-10-31 | 1690 |
| ARBEUR | 2023-03-23 | 2026-10-31 | 1319 |
| ATOMEUR | 2019-04-29 | 2026-10-31 | 2743 |
| AVAXEUR | 2020-09-22 | 2026-10-31 | 2231 |
| BCHABCEUR | 2018-11-16 | 2019-11-28 | 378 |
| BCHEUR | 2019-11-28 | 2026-10-31 | 2530 |
| BCHSVEUR | 2018-11-16 | 2019-04-22 | 158 |
| BNBEUR | 2017-11-06 | 2026-10-31 | 3282 |
| BTCEUR | 2017-10-08 | 2026-10-31 | 3311 |
| DASHEUR | 2019-03-28 | 2026-10-31 | 2775 |
| DOGEEUR | 2019-07-05 | 2026-10-31 | 2676 |
| DOTEUR | 2020-08-18 | 2026-10-31 | 2266 |
| EOSEUR | 2018-05-28 | 2025-05-26 | 2556 |
| ETCEUR | 2018-06-12 | 2026-10-31 | 3064 |
| ETHEUR | 2017-10-08 | 2026-10-31 | 3311 |
| FILEUR | 2020-10-15 | 2026-10-31 | 2208 |
| FTTEUR | 2019-12-20 | 2022-11-15 | 1062 |
| GMTEUR | 2022-03-09 | 2026-10-31 | 1698 |
| ICXEUR | 2018-06-13 | 2026-09-03 | 3005 |
| IOTAEUR | 2018-05-31 | 2026-10-31 | 3076 |
| LINKEUR | 2019-01-16 | 2026-10-31 | 2846 |
| LTCEUR | 2017-12-13 | 2026-10-31 | 3245 |
| LUNAEUR | 2020-08-21 | 2022-05-13 | 631 |
| MATICEUR | 2019-04-26 | 2024-09-10 | 1965 |
| NEAREUR | 2020-10-14 | 2026-10-31 | 2209 |
| NEOEUR | 2017-11-20 | 2026-10-31 | 3268 |
| ONTEUR | 2018-06-08 | 2026-10-31 | 3068 |
| OPEUR | 2022-06-01 | 2026-10-31 | 1614 |
| ORDIEUR | 2023-11-07 | 2026-10-31 | 1090 |
| PEPEEUR | 2023-05-05 | 2026-10-31 | 1276 |
| POLEUR | 2024-09-13 | 2026-10-31 | 779 |
| QTUMEUR | 2018-03-19 | 2026-10-31 | 3149 |
| SANDEUR | 2020-08-14 | 2026-10-31 | 2270 |
| SHIBEUR | 2021-05-10 | 2026-10-31 | 2001 |
| SOLEUR | 2020-08-11 | 2026-10-31 | 2273 |
| SUIEUR | 2023-05-03 | 2026-10-31 | 1278 |
| TONEUR | 2024-08-08 | 2026-06-30 | 692 |
| TRUMPEUR | 2025-01-19 | 2026-10-31 | 651 |
| TRXEUR | 2018-06-11 | 2026-10-31 | 3065 |
| UNIEUR | 2020-09-17 | 2026-10-31 | 2236 |
| VETEUR | 2018-07-25 | 2026-10-31 | 3021 |
| WIFEUR | 2024-03-05 | 2026-10-31 | 971 |
| XLMEUR | 2018-05-31 | 2026-10-31 | 3076 |
| XMREUR | 2019-03-15 | 2024-02-20 | 1804 |
| XRPEUR | 2018-05-04 | 2026-10-31 | 3103 |
| ZECEUR | 2019-03-21 | 2026-10-31 | 2782 |
## BTC, ETH e SOL
Solo le tre coppie in euro con la liquidità migliore.
Dal 01/01/2019 al 31/10/2026, 2861 chiusure giornaliere, capitale iniziale 10.000,00 €. Variante scelta: `coin_vol=20;max_stop=10` (la zona della griglia dove il risultato cambia meno, non la migliore). Impronta del piano `1ce210e9cbcb`, dei dati `3b890d6f2ef3`.
| | Strategia | Comprare e tenere |
|---|---:|---:|
| Risultato netto | 14.642,97 € | 216.831,80 € |
| Rendimento totale | 146,4% | 2.168,3% |
| Rendimento annuo | 12,2% | 49,0% |
| Volatilità annua | 11,8% | 72,2% |
| Calo massimo | −15,0% (11/2021 → 10/2023) | −86,7% |
| Sharpe annuo | 1,03 | 0,92 |
| Operazioni chiuse | 539 | 0 |
| Operazioni in guadagno | 16,7% | — |
| Rapporto guadagni/perdite | 1,76 | — |
| Commissioni pagate | 2.866,85 € | 20,00 € |
| Tempo investito | 12,9% | 95,2% |
**Per anno**
| Anno | Strategia | Comprare e tenere |
|---|---:|---:|
| 2019 | 20,8% | 29,3% |
| 2020 | 29,0% | 219,6% |
| 2021 | 14,1% | 647,5% |
| 2022 | −6,7% | −81,5% |
| 2023 | 27,3% | 257,8% |
| 2024 | 20,9% | 92,3% |
| 2025 | −5,5% | −31,3% |
| 2026 | 2,3% | −16,1% |
**Sensibilità ai costi** (commissioni e scostamento moltiplicati insieme)
| Costi | Risultato netto | Rendimento totale | Sharpe annuo | Calo massimo | Commissioni |
|---|---:|---:|---:|---:|---:|
| ×1 | 14.642,97 € | 146,4% | 1,03 | −15,0% | 2.866,85 € |
| ×2 | 10.274,96 € | 102,7% | 0,82 | −18,9% | 5.161,91 € |
| ×3 | 6.273,26 € | 62,7% | 0,59 | −21,3% | 6.782,83 € |
**Convalide**
- Finestre che avanzano (4, con 29 giorni di stacco): rendimento fuori campione 82,4%, Sharpe annuo 0,85.
- K-fold purgato combinatorio (6 gruppi, 2 di prova, 15 divisioni): Sharpe annuo medio dei percorsi 0,90, peggiore 0,82, percorsi positivi 100,0%.
- Deflated Sharpe Ratio 0,96, contando 18 varianti provate in tutto.
- Probabilità di sovra-adattamento (PBO) 0,83, su 12870 combinazioni di 9 varianti.
- Sguardo in avanti: 8583 decisioni rifatte tagliando i dati in 6 punti, 0 cambiate.
**Soglie**
| Controllo | Valore | Soglia | Esito |
|---|---:|---:|---|
| Risultato netto con costi doppi | 10.274,96 € | > 0,00 € | superato |
| Deflated Sharpe Ratio | 0,961 | ≥ 0,9 | superato |
| Probabilità di sovra-adattamento (PBO) | 0,835 | ≤ 0,4 | **non superato** |
| Calo massimo | 15,0% | ≤ 25% | superato |
| Decisioni cambiate aggiungendo dati successivi | 0 | = 0 | superato |
**Varianti della griglia**
| Parametri | Sharpe annuo | Stabilità della zona | Risultato netto | Calo massimo | Operazioni |
|---|---:|---:|---:|---:|---:|
| `coin_vol=10;max_stop=10` | 0,98 | 0,95 | 8.962,80 € | −11,2% | 539 |
| `coin_vol=10;max_stop=15` | 0,91 | 0,91 | 9.287,93 € | −13,3% | 457 |
| `coin_vol=10;max_stop=20` | 0,97 | 0,95 | 10.893,96 € | −13,9% | 393 |
| `coin_vol=15;max_stop=10` | 1,00 | 0,97 | 12.333,91 € | −13,5% | 539 |
| `coin_vol=15;max_stop=15` | 0,91 | 0,91 | 12.048,69 € | −16,1% | 457 |
| `coin_vol=15;max_stop=20` | 0,99 | 0,96 | 14.421,53 € | −16,5% | 393 |
| `coin_vol=20;max_stop=10` (scelta) | 1,03 | 0,98 | 14.642,97 € | −15,0% | 539 |
| `coin_vol=20;max_stop=15` | 0,97 | 0,97 | 14.521,59 € | −17,1% | 457 |
| `coin_vol=20;max_stop=20` | 1,02 | 0,97 | 16.651,30 € | −16,6% | 393 |
**Interventi di correttivi e rischio sulla variante scelta**
Quante volte, per strumento e giorno, un correttivo o una regola di rischio ha cambiato ciò che la strategia chiedeva.
| Motivo | Volte |
|---|---:|
| `crypto_regime_below_average` | 1104 |
| `crypto_regime_high_volatility` | 52 |
| `trade_risk_limit` | 41 |
| `stop_above_price` | 3 |
| `daily_loss_block` | 1 |
**Note del motore di prova**
Ordini non eseguiti come chiesto: sotto il minimo, residui lasciati dopo un'uscita, liquidità.
| Motivo | Volte |
|---|---:|
| `residue_written_off` | 539 |
| `below_min_notional` | 105 |
**Strumenti**
| Coppia | Prima barra | Ultima barra | Barre |
|---|---|---|---:|
| BTCEUR | 2017-10-08 | 2026-10-31 | 3311 |
| ETHEUR | 2017-10-08 | 2026-10-31 | 3311 |
| SOLEUR | 2020-08-11 | 2026-10-31 | 2273 |
## Cosa il backtest non rappresenta
- I correttivi senza una storia ricostruibile (notizie, calendario economico, trimestrali, modello) non ricevono dati e non intervengono.
- Le decisioni si prendono sulla chiusura e si eseguono all'apertura della seduta dopo, con lo scostamento stimato: nessun inseguimento con ordini limite.
- Sharpe e volatilità sono calcolati senza tasso privo di rischio.
- Limiti di rischio e regime crypto applicati come dal vivo. La strategia è il 40% del bot: il resto conta come liquidità ferma. Ogni chiusura vale come un giorno; se scatta l'interruttore, supponiamo che tu riparta dopo sette giorni. Età dei prezzi, prezzo dell'ordine e orologio esistono solo dal vivo.
- Prezzi dello storico Binance in USDT convertiti in euro con il cambio BCE del giorno, contando un USDT come un dollaro: è un'approssimazione delle coppie in euro di OKX. Su questo storico lo spread non si può misurare: l'universo sono le monete con più scambi, passo di prezzo e quantità minima sono stime, i costi sono quelli di OKX.
+377
View File
@@ -0,0 +1,377 @@
{
"passed": false,
"verdict": "Protocollo non superato. Universo ampio: calo massimo 32,8%, serve \u2264 25%. BTC, ETH e SOL: probabilit\u00E0 di sovra-adattamento (pbo) 0,835, serve \u2264 0,40. Resta in osservazione.",
"report": "docs/backtests/crypto_trend.md",
"plan_hash": "b5f566d59bfae84f7c0821705d243dc0707803db36f74c1eecd130fb14fed8ed",
"parameters": "coin_vol=10;max_stop=20",
"max_drawdown": 0.3278,
"expected_slippage_bp": 11.94,
"round_trip_cost": 0.006388,
"expected_path_p5": [
-0.01,
-0.0144,
-0.0179,
-0.0199,
-0.023,
-0.0252,
-0.0256,
-0.0269,
-0.0284,
-0.0294,
-0.0313,
-0.0316,
-0.0327,
-0.0346,
-0.0355,
-0.0373,
-0.0381,
-0.0396,
-0.0411,
-0.0412,
-0.0413,
-0.0426,
-0.0429,
-0.0445,
-0.0457,
-0.0457,
-0.0468,
-0.0481,
-0.0499,
-0.0512,
-0.0507,
-0.0519,
-0.0538,
-0.0548,
-0.0558,
-0.0566,
-0.0582,
-0.0585,
-0.0583,
-0.0594,
-0.0599,
-0.06,
-0.06,
-0.0616,
-0.0619,
-0.0619,
-0.061,
-0.0626,
-0.0634,
-0.0655,
-0.0646,
-0.0651,
-0.0674,
-0.0681,
-0.0683,
-0.0678,
-0.0684,
-0.0691,
-0.0705,
-0.0724,
-0.0733,
-0.0725,
-0.0736,
-0.0737,
-0.0746,
-0.0729,
-0.074,
-0.0737,
-0.0744,
-0.0739,
-0.0751,
-0.0742,
-0.0749,
-0.076,
-0.0754,
-0.0755,
-0.0757,
-0.0774,
-0.0781,
-0.0782,
-0.0776,
-0.0791,
-0.0786,
-0.0785,
-0.0781,
-0.0785,
-0.0789,
-0.0788,
-0.0806,
-0.0802,
-0.0802,
-0.0802,
-0.0813,
-0.0826,
-0.0842,
-0.0841,
-0.087,
-0.0859,
-0.0855,
-0.0868,
-0.0868,
-0.0875,
-0.0872,
-0.0872,
-0.0876,
-0.0868,
-0.0866,
-0.0865,
-0.0867,
-0.0875,
-0.0877,
-0.0884,
-0.0882,
-0.0887,
-0.0882,
-0.0888,
-0.0895,
-0.0912,
-0.0919,
-0.0915,
-0.0929,
-0.0933,
-0.0938,
-0.0941,
-0.0943,
-0.0957,
-0.096,
-0.0957,
-0.0958,
-0.0957,
-0.0973,
-0.0965,
-0.0957,
-0.0941,
-0.0956,
-0.0958,
-0.0969,
-0.0992,
-0.0988,
-0.0996,
-0.1003,
-0.0996,
-0.101,
-0.1006,
-0.1041,
-0.1051,
-0.1045,
-0.1032,
-0.1035,
-0.1038,
-0.1048,
-0.1045,
-0.1067,
-0.1051,
-0.1065,
-0.1061,
-0.1062,
-0.1082,
-0.1073,
-0.1078,
-0.1088,
-0.109,
-0.1097,
-0.1116,
-0.1089,
-0.1105,
-0.1103,
-0.1083,
-0.1116,
-0.1108,
-0.1116,
-0.1132,
-0.1133,
-0.1131,
-0.1132,
-0.114,
-0.1141,
-0.1144,
-0.1159,
-0.116,
-0.1165,
-0.1157,
-0.1165,
-0.1191,
-0.1195,
-0.1198,
-0.1205,
-0.12,
-0.1216,
-0.1213,
-0.1219,
-0.1204,
-0.12,
-0.1217,
-0.1194,
-0.1206,
-0.1194,
-0.1184,
-0.1195,
-0.119,
-0.1195,
-0.1201,
-0.121,
-0.1206,
-0.1195,
-0.1217,
-0.1212,
-0.1204,
-0.1217,
-0.1229,
-0.124,
-0.1231,
-0.1232,
-0.1247,
-0.1256,
-0.1277,
-0.1286,
-0.1292,
-0.1271,
-0.1276,
-0.1301,
-0.1289,
-0.1273,
-0.1289,
-0.1285,
-0.1272,
-0.1265,
-0.1257,
-0.1273,
-0.127,
-0.1244,
-0.1244,
-0.125,
-0.126,
-0.1261,
-0.1262,
-0.1249,
-0.1243,
-0.1255,
-0.1267,
-0.1269,
-0.1283,
-0.1288,
-0.1296,
-0.1314,
-0.1302,
-0.1313,
-0.1311,
-0.1322,
-0.1315,
-0.1337,
-0.1319,
-0.1309,
-0.1341,
-0.1328,
-0.1323,
-0.1346,
-0.1315,
-0.1328,
-0.1353,
-0.1347,
-0.1338,
-0.1365,
-0.1356,
-0.1338,
-0.1346,
-0.1343,
-0.134,
-0.1345,
-0.1331,
-0.1332,
-0.1347,
-0.1339,
-0.136,
-0.1347,
-0.1348,
-0.1368,
-0.136,
-0.1356,
-0.1313,
-0.1334,
-0.134,
-0.1349,
-0.1341,
-0.1342,
-0.1341,
-0.135,
-0.1353,
-0.1319,
-0.1345,
-0.137,
-0.1358,
-0.1369,
-0.1368,
-0.1372,
-0.1347,
-0.1376,
-0.1345,
-0.1346,
-0.1343,
-0.1344,
-0.132,
-0.1331,
-0.133,
-0.1345,
-0.1347,
-0.1355,
-0.1349,
-0.1346,
-0.1341,
-0.1339,
-0.1334,
-0.1326,
-0.1343,
-0.134,
-0.1342,
-0.1351,
-0.135,
-0.1352,
-0.1353,
-0.1382,
-0.1366,
-0.1363,
-0.1368,
-0.1354,
-0.1352,
-0.1383,
-0.1382,
-0.1356,
-0.1354,
-0.1349,
-0.1379,
-0.1382,
-0.1364,
-0.1364,
-0.1348,
-0.1345,
-0.1346,
-0.1353,
-0.1342,
-0.1372,
-0.1386,
-0.1385,
-0.1388,
-0.1402,
-0.1399,
-0.1397,
-0.1397,
-0.1401,
-0.1393,
-0.1394,
-0.1417,
-0.1402,
-0.1399,
-0.1409,
-0.1417,
-0.1404,
-0.1408,
-0.1418,
-0.1422,
-0.1423,
-0.1444,
-0.1423,
-0.1399,
-0.1405
]
}
+10
View File
@@ -0,0 +1,10 @@
# Backtest di `equity_meanrev` (Ritorno alla media azioni)
**Esito: in attesa.** In attesa delle chiavi Alpaca: senza lo storico delle azioni il backtest non si può fare, quindi la strategia resta in osservazione.
Per provare le strategie sulle azioni servono gli storici giornalieri di Alpaca, con frazionamenti e dividendi, che si scaricano solo con le chiavi di un conto Alpaca (anche dimostrativo). Le chiavi non sono ancora state inserite.
Finché manca il backtest la strategia resta in osservazione: guarda il mercato e scrive le sue decisioni nel diario, ma non opera, nemmeno in simulazione.
Quando avrai inserito le chiavi nelle Impostazioni, il bot raccoglie gli storici. Poi il backtest si rifà con gli stessi controlli di crypto_trend e il nuovo esito arriva con un aggiornamento del programma: se supera le soglie, la strategia parte in simulazione da sola al primo avvio dopo l'aggiornamento.
@@ -0,0 +1,9 @@
{
"passed": false,
"verdict": "In attesa delle chiavi Alpaca: senza lo storico delle azioni il backtest non si pu\u00F2 fare, quindi la strategia resta in osservazione.",
"report": "docs/backtests/equity_meanrev.md",
"max_drawdown": 0,
"expected_slippage_bp": 0,
"round_trip_cost": 0,
"expected_path_p5": []
}
+10
View File
@@ -0,0 +1,10 @@
# Backtest di `equity_momentum` (Momentum azioni)
**Esito: in attesa.** In attesa delle chiavi Alpaca: senza lo storico delle azioni il backtest non si può fare, quindi la strategia resta in osservazione.
Per provare le strategie sulle azioni servono gli storici giornalieri di Alpaca, con frazionamenti e dividendi, che si scaricano solo con le chiavi di un conto Alpaca (anche dimostrativo). Le chiavi non sono ancora state inserite.
Finché manca il backtest la strategia resta in osservazione: guarda il mercato e scrive le sue decisioni nel diario, ma non opera, nemmeno in simulazione.
Quando avrai inserito le chiavi nelle Impostazioni, il bot raccoglie gli storici. Poi il backtest si rifà con gli stessi controlli di crypto_trend e il nuovo esito arriva con un aggiornamento del programma: se supera le soglie, la strategia parte in simulazione da sola al primo avvio dopo l'aggiornamento.
@@ -0,0 +1,9 @@
{
"passed": false,
"verdict": "In attesa delle chiavi Alpaca: senza lo storico delle azioni il backtest non si pu\u00F2 fare, quindi la strategia resta in osservazione.",
"report": "docs/backtests/equity_momentum.md",
"max_drawdown": 0,
"expected_slippage_bp": 0,
"round_trip_cost": 0,
"expected_path_p5": []
}
+2
View File
@@ -4,6 +4,8 @@
<RootNamespace>Anubis.Guard</RootNamespace>
</PropertyGroup>
<ItemGroup>
<!-- The same shared runtime as the engine: in the image both live in /app (ADR 0012). -->
<FrameworkReference Include="Microsoft.AspNetCore.App" />
<ProjectReference Include="..\Anubis\Anubis.csproj" />
</ItemGroup>
</Project>
+49 -3
View File
@@ -1,3 +1,49 @@
// Placeholder: the guard process (SPEC 9.3) arrives in phase 6 (ADR 0003).
await Console.Error.WriteLineAsync("Anubis.Guard non è ancora disponibile: arriva nella Fase 6.").ConfigureAwait(false);
return 2;
using System.Collections;
using System.Runtime.InteropServices;
using Anubis.Core;
using Anubis.Platform;
using Anubis.Supervision;
// The guard (SPEC 9.3, ADR 0012): the main process of the container and the parent of the engine. Its arguments go to
// the engine as they are. "drill" runs the guard drill in a temporary folder instead.
if (args is ["drill", ..])
{
return await GuardDrill.RunAsync(GuardLaunch.Server(), Console.Out, CancellationToken.None).ConfigureAwait(false);
}
var environment = Environment.GetEnvironmentVariables().Cast<DictionaryEntry>()
.ToDictionary(e => (string)e.Key, e => e.Value as string, StringComparer.Ordinal);
var paths = new GuardPaths(DataRoot(args, environment));
var logger = new LineLogger(Console.Out.WriteLine);
using var http = new HttpClient { Timeout = TimeSpan.FromSeconds(20) };
var clock = SystemClock.Instance;
var guardian = new Guardian(paths, clock, new GuardOptions(),
start => ChildEngine.Start(GuardLaunch.Dotnet(), [GuardLaunch.Server(), .. args], environment, start),
(account, keys) => GuardBrokers.Create(http, clock, account, keys),
(text, keys, ct) => GuardBrokers.AlertAsync(http, logger, keys, text, ct),
logger);
// The stop of the container (SIGTERM) or Ctrl+C: passed on to the engine, which closes in order and is not restarted.
using var shutdown = new CancellationTokenSource();
using var terminate = PosixSignalRegistration.Create(PosixSignal.SIGTERM, Stop);
using var interrupt = PosixSignalRegistration.Create(PosixSignal.SIGINT, Stop);
return await guardian.RunAsync(shutdown.Token).ConfigureAwait(false);
void Stop(PosixSignalContext context)
{
context.Cancel = true;
shutdown.Cancel();
}
// The engine's data folder: --data-dir when given, /data in Docker, the user's folder on desktop (SPEC 15.2, 15.6).
static string DataRoot(string[] args, Dictionary<string, string?> environment)
{
var index = Array.IndexOf(args, "--data-dir");
if (index >= 0 && index + 1 < args.Length)
{
return Path.GetFullPath(args[index + 1]);
}
var platform = Array.IndexOf(args, "--platform") is var p and >= 0 && p + 1 < args.Length ? args[p + 1] : environment.GetValueOrDefault("ANUBIS_PLATFORM");
return platform == "docker" ? "/data" : DataDirectory.ForCurrentUser().Root;
}
+1 -1
View File
@@ -136,7 +136,7 @@ internal sealed record AccountsView(
var (pending, others, alpacaStops) = PendingRows(orders.Where(o => shown.Contains(o.Account)).ToList(), byId, instruments.ToDictionary(i => i.Key, i => i.Value.Symbol),
positions, market.Open, market.Next, clock.UtcNow, zone);
var slices = Strategies.Base.Select(b => new StrategySlice(Strategies.Label(b.Name, withMarket: true), "var(--s-" + b.Name.Replace('_', '-') + ")",
var slices = StrategyNames.Base.Select(b => new StrategySlice(StrategyNames.Label(b.Name, withMarket: true), "var(--s-" + b.Name.Replace('_', '-') + ")",
rows.Where(r => r.Row.Strategy == b.Name).Sum(r => r.Row.ValueEur))).ToList();
decimal? fxEffect = null;
if (cards.Count > 0)
+173 -11
View File
@@ -1,9 +1,12 @@
using Anubis.Access;
using Anubis.Backtest;
using Anubis.Core;
using Anubis.Data;
using Anubis.Db;
using Anubis.Demo;
using Anubis.Diagnostics;
using Anubis.Platform;
using Anubis.Supervision;
using Anubis.Trading;
namespace Anubis.Server;
@@ -37,7 +40,7 @@ internal sealed record HostChanged(HostMode Mode);
/// <summary>Everything that needs the database, built once it is reachable and the installation checks out.</summary>
internal sealed record AppServices(Database Db, SettingsStore Settings, SecretStore Secrets, AccessStore Access, Backups Backups, Scheduler Scheduler,
MarketDataService? MarketData = null, TradingService? Trading = null);
MarketDataService? MarketData = null, TradingService? Trading = null, Laboratory? Lab = null, TelegramServices? Telegram = null);
/// <summary>
/// The startup state machine of the server (SPEC 15.1, 15.6): reads the startup configuration, opens the configured
@@ -54,6 +57,9 @@ internal sealed partial class AnubisHost(
ILogger<AnubisHost> logger,
ServerOverrides? overrides = null) : BackgroundService
{
/// <summary>A scheduler silent this long is stuck: the engine stops beating, so the guard steps in.</summary>
public static readonly TimeSpan SchedulerSilence = TimeSpan.FromMinutes(30);
private readonly TimeSpan _interval = overrides?.DatabaseCheckInterval ?? TimeSpan.FromSeconds(5);
private readonly Lock _gate = new();
@@ -65,15 +71,23 @@ internal sealed partial class AnubisHost(
private CancellationTokenSource? _schedulerStop;
private MarketDataService? _market;
private TradingService? _trading;
private TelegramServices? _telegram;
private Laboratory? _lab;
private DiagnosticSampler? _sampler;
private volatile HostStatus _status = new(HostMode.Starting, null, null, clock.UtcNow, 0);
private volatile AppServices? _services;
public StartupConfig Config { get; private set; } = initialConfig;
/// <summary>The channel to the guard that started this engine (SPEC 9.3); null without one.</summary>
public GuardLink? Guard { get; init; }
public bool IsDemo => options.Demo;
public HostStatus Status => _status;
public IClock Clock => clock;
/// <summary>The running services; null until the first successful start.</summary>
public AppServices? Services => _status.Mode == HostMode.Ready ? _services : null;
@@ -144,6 +158,10 @@ internal sealed partial class AnubisHost(
LogDemo(logger, Config.Data.Root);
}
// From the first moment: every exception and every outbound call of the process is counted (issue #95).
ExceptionCounter.Start();
_ = HttpCallMeter.Shared;
while (!stoppingToken.IsCancellationRequested)
{
Task wake;
@@ -174,6 +192,12 @@ internal sealed partial class AnubisHost(
SetStatus(HostMode.DatabaseUnavailable, Redactor.Redact(ex.Message));
}
// With trading the beat of a running engine comes from the trading loop, so a blocked loop is noticed.
if (_status.Mode != HostMode.Ready || _trading is null)
{
Guard?.Beat();
}
await Task.WhenAny(wake, Task.Delay(_interval, stoppingToken)).ConfigureAwait(false);
}
@@ -211,6 +235,7 @@ internal sealed partial class AnubisHost(
}
Config.Data.Ensure();
Guard?.Step("database");
// A data folder that belongs to an installation never gets a new empty SQLite file by itself (trap 20).
_db ??= Database.Open(Config.Db!, allowCreate: !File.Exists(Config.Data.InstallationFile));
@@ -232,6 +257,8 @@ internal sealed partial class AnubisHost(
return;
}
Guard?.Step("migrazioni e identità");
// Identity first, migration after: a database that belongs elsewhere is never written to, and an empty one stays
// empty, so a backup taken with an older schema can still be restored into it.
var backups = new Backups(Config.Data, clock);
@@ -269,6 +296,7 @@ internal sealed partial class AnubisHost(
return;
}
Guard?.Step("servizi");
var key = Config.SecretKey.GetOrCreateKey();
var settings = new SettingsStore(_db, clock, bus, StartupOverrides());
var secrets = new SecretStore(_db, clock, key, Config.EnvironmentKeys);
@@ -286,14 +314,76 @@ internal sealed partial class AnubisHost(
var scheduler = new Scheduler(clock, bus);
_market = MarketDataEnabled
? new MarketDataService(_db, clock, bus, settings, secrets, Config.Data, overrides?.MarketHttp, overrides?.MarketSockets, logger)
? new MarketDataService(_db, clock, bus, settings, secrets, Config.Data, overrides?.MarketHttp, overrides?.MarketSockets, logger,
Guard is null ? null : GuardDrill.DrillOkx(environment))
: null;
_trading = TradingEnabled && (identity.File ?? Installation.ReadFile(Config.Data)) is { } file
? new TradingService(_db, clock, bus, secrets, _market, new FxStore(_db, clock), TradingConsent.From(Config), file.InstallationId,
overrides?.MarketHttp, overrides?.MarketSockets, logger)
overrides?.MarketHttp, overrides?.MarketSockets, logger, settings, (password, code, ct) => access.ReauthenticateAsync(password, code, ct),
Path.Combine(Config.Data.Root, "STOP"))
{
Guard = Guard,
GuardFiles = Guard is null ? null : new GuardPaths(Config.Data.Root),
// The drill's fault is honored only under a guard, which never passes it to an engine it restarts.
Fault = Guard is null ? null : environment.GetValueOrDefault(GuardChannel.FaultVariable),
// The decisions run in the scheduler: a scheduler silent for half an hour is stuck (its longest jobs take
// minutes), and the guard must hear it (SPEC 9.3).
OthersAlive = () => scheduler.LastBeatUtc is not { } beat || clock.UtcNow - beat < SchedulerSilence,
}
: null;
_services = new AppServices(_db, settings, secrets, access, backups, scheduler, _market, _trading);
StartScheduler(scheduler, backups);
if (_trading is not null)
{
// Before "ready": the Strategie page must never show a strategy without its backtest outcome (SPEC 12.3).
// A failure here is logged and left to the trading start, which repeats it: the interface still opens.
try
{
await _trading.PrepareAsync(cancellationToken).ConfigureAwait(false);
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
LogTradingFailed(logger, ex);
}
}
// The laboratory computes on its own thread and reads only local data; the demo shows appendix F and runs none.
_lab = options.Demo ? null : new Laboratory(_db, clock, Config.Data.Research, logger);
// Telegram and the sign of life (SPEC 14): not in the demo, and in tests only with their fake network.
_telegram = options.Demo || (overrides is not null && overrides.TelegramHttp is null)
? null
: new TelegramServices(this, _db, clock, settings, secrets, Config.HeartbeatUrl, overrides?.TelegramHttp, logger);
_services = new AppServices(_db, settings, secrets, access, backups, scheduler, _market, _trading, _lab, _telegram);
// The data for the analysis of a long run (issue #95, ADR 0013); the demo has nothing to analyse.
_sampler = options.Demo ? null : new DiagnosticSampler(_db, clock, HttpCallMeter.Shared);
// Before the scheduler writes its first run: how long the bot was away, and which runs it missed (SPEC 5.6).
var lastDues = new Dictionary<string, DateTimeOffset>(StringComparer.Ordinal);
try
{
if (!options.Demo)
{
await new AbsenceBook(_db, clock) { Zone = Config.TimeZone }.RecordAsync(cancellationToken).ConfigureAwait(false);
}
foreach (var (name, due) in await _db.ReadAsync(s => s.QueryAsync("SELECT name, last_due_at FROM jobs WHERE last_due_at IS NOT NULL",
r => (r.Text("name"), r.Moment("last_due_at"))), cancellationToken).ConfigureAwait(false))
{
lastDues[name] = due;
}
if (_sampler is not null)
{
await _sampler.TakeAsync(SampleKinds.Start, cancellationToken).ConfigureAwait(false);
}
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
LogStepFailed(logger, ex);
}
StartScheduler(scheduler, backups, lastDues);
SetStatus(HostMode.Ready, null);
}
@@ -412,26 +502,31 @@ internal sealed partial class AnubisHost(
!options.Demo && (overrides is null ? true : overrides.Trading == true) &&
environment.GetValueOrDefault("ANUBIS_TRADING") is not ("0" or "off" or "false");
private void StartScheduler(Scheduler scheduler, Backups backups)
private void StartScheduler(Scheduler scheduler, Backups backups, Dictionary<string, DateTimeOffset> lastDues)
{
scheduler.Add(new ScheduledJob("backup", "Salvataggio del database", Schedule.Daily(new TimeOnly(3, 30), Config.TimeZone),
// A backup missed while the bot was off is taken once at the restart, whenever it comes (SPEC 5.6).
var backup = new ScheduledJob("backup", "Salvataggio del database", Schedule.Daily(new TimeOnly(3, 30), Config.TimeZone),
async ct => await backups.TakeAsync(_db!, "notturno", ct).ConfigureAwait(false))
{ MaxLateness = TimeSpan.FromHours(2) });
foreach (var job in (_market?.Jobs() ?? []).Concat(_trading?.Jobs() ?? []))
{ MaxLateness = TimeSpan.FromHours(2), CatchUpWithin = TimeSpan.FromDays(3650) };
foreach (var job in new[] { backup }.Concat(_sampler is { } sampler ? new[] { sampler.Job() } : []).Concat(_market?.Jobs() ?? []).Concat(_trading?.Jobs() ?? []).Concat(_telegram?.Jobs() ?? []))
{
scheduler.Add(job);
scheduler.Add(job, lastDues.TryGetValue(job.Name, out var due) ? due : null);
}
_schedulerStop = new CancellationTokenSource();
var token = _schedulerStop.Token;
var market = _market;
var trading = _trading;
var lab = _lab;
var telegram = _telegram;
_schedulerLoop = Task.Run(async () =>
{
using var runs = bus.Subscribe<JobRun>();
var recorder = RecordJobsAsync(runs, token);
var collecting = market is null ? Task.CompletedTask : RunMarketDataAsync(market, token);
var dealing = trading is null ? Task.CompletedTask : RunTradingAsync(trading, token);
var testing = lab is null ? Task.CompletedTask : RunLabAsync(lab, token);
var writing = telegram is null ? Task.CompletedTask : RunTelegramAsync(telegram, token);
try
{
await scheduler.RunAsync(token).ConfigureAwait(false);
@@ -442,6 +537,8 @@ internal sealed partial class AnubisHost(
await collecting.ConfigureAwait(false);
await dealing.ConfigureAwait(false);
await testing.ConfigureAwait(false);
await writing.ConfigureAwait(false);
await recorder.ConfigureAwait(false);
}, CancellationToken.None);
}
@@ -461,6 +558,36 @@ internal sealed partial class AnubisHost(
}
}
private async Task RunLabAsync(Laboratory lab, CancellationToken cancellationToken)
{
try
{
await lab.RunAsync(cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
}
catch (Exception ex)
{
LogLabFailed(logger, ex);
}
}
private async Task RunTelegramAsync(TelegramServices telegram, CancellationToken cancellationToken)
{
try
{
await telegram.RunAsync(cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
}
catch (Exception ex)
{
LogTelegramFailed(logger, ex);
}
}
private async Task RunTradingAsync(TradingService trading, CancellationToken cancellationToken)
{
try
@@ -488,7 +615,13 @@ internal sealed partial class AnubisHost(
{
await _db!.WriteAsync(async s =>
{
var description = _services?.Scheduler.Upcoming(100).FirstOrDefault(u => u.Name == run.Name)?.Description ?? run.Name;
var description = run.Description ?? _services?.Scheduler.Upcoming(100).FirstOrDefault(u => u.Name == run.Name)?.Description ?? run.Name;
if (AbsenceBook.MissedLine(run with { Description = description }, Config.TimeZone) is { } line)
{
await Diary.WriteAsync(s, clock, DiaryKinds.System, run.Outcome == JobOutcome.Completed ? "job_recovered" : "job_skipped", line,
new { job = run.Name, due = run.DueUtc }, cancellationToken).ConfigureAwait(false);
}
var updated = await s.ExecuteAsync(
"UPDATE jobs SET last_due_at = @d, last_started_at = @s, last_finished_at = @f, last_outcome = @o, last_error = @e, updated_at = @f WHERE name = @n",
("d", run.DueUtc), ("s", run.StartedUtc), ("f", run.FinishedUtc), ("o", run.Outcome.ToString().ToLowerInvariant()), ("e", run.Error), ("n", run.Name)).ConfigureAwait(false);
@@ -499,6 +632,12 @@ internal sealed partial class AnubisHost(
("id", Ids.NewId(clock)), ("n", run.Name), ("desc", description), ("d", run.DueUtc), ("s", run.StartedUtc), ("f", run.FinishedUtc),
("o", run.Outcome.ToString().ToLowerInvariant()), ("e", run.Error)).ConfigureAwait(false);
}
// Every run, for the lateness and the durations in the diagnostics report (issue #95).
await s.ExecuteAsync(
"INSERT INTO job_runs (id, job, due_at, started_at, finished_at, outcome, missed, error, created_at) VALUES (@id, @n, @d, @s, @f, @o, @m, @e, @f)",
("id", Ids.NewId(clock)), ("n", run.Name), ("d", run.DueUtc), ("s", run.StartedUtc), ("f", run.FinishedUtc),
("o", run.Outcome.ToString().ToLowerInvariant()), ("m", run.Missed), ("e", run.Error is null ? null : Redactor.Redact(run.Error))).ConfigureAwait(false);
}, cancellationToken).ConfigureAwait(false);
}
catch (Exception ex) when (ex is not OperationCanceledException)
@@ -532,6 +671,10 @@ internal sealed partial class AnubisHost(
_market = null;
_trading?.Dispose();
_trading = null;
_lab?.Dispose();
_lab = null;
_telegram?.Dispose();
_telegram = null;
}
// Values of the startup configuration that also exist as settings are read-only in the interface (SPEC 13.5).
@@ -543,6 +686,19 @@ internal sealed partial class AnubisHost(
var attempts = previous.Mode == mode ? previous.Attempts + 1 : 1;
var since = previous.Mode == mode ? previous.Since : clock.UtcNow;
_status = new HostStatus(mode, detail, identity, since, attempts);
if (mode == HostMode.Ready)
{
// Trading tells the guard by itself when its own startup steps are done.
if (_trading is null || _trading.IsStarted)
{
Guard?.Set(GuardStages.Running);
}
}
else
{
Guard?.Set(mode == HostMode.Starting ? GuardStages.Starting : GuardStages.Waiting);
}
if (previous.Mode != mode)
{
LogMode(logger, mode);
@@ -571,6 +727,12 @@ internal sealed partial class AnubisHost(
[LoggerMessage(Level = LogLevel.Error, Message = "Market data stopped")]
private static partial void LogMarketDataFailed(ILogger logger, Exception exception);
[LoggerMessage(Level = LogLevel.Error, Message = "Telegram stopped")]
private static partial void LogTelegramFailed(ILogger logger, Exception exception);
[LoggerMessage(Level = LogLevel.Error, Message = "Laboratory stopped")]
private static partial void LogLabFailed(ILogger logger, Exception exception);
[LoggerMessage(Level = LogLevel.Error, Message = "Trading stopped")]
private static partial void LogTradingFailed(ILogger logger, Exception exception);
+291
View File
@@ -224,6 +224,7 @@ textarea {
.panel > h2 { display: flex; align-items: center; gap: 10px; margin-bottom: 12px; }
.panel > h3, .panel h3 { margin: 20px 0 8px; }
.note { color: var(--text-3); font-size: 13px; line-height: 18px; }
h2 .note, h3 .note { font-weight: 400; letter-spacing: 0; }
.hint { color: var(--text-3); font-size: 13px; line-height: 18px; }
.empty { display: flex; flex-direction: column; gap: 8px; color: var(--text-2); max-width: 70ch; }
.empty-title { color: var(--text); font-weight: 600; font-size: 15px; }
@@ -256,6 +257,7 @@ textarea {
.form { display: flex; flex-direction: column; gap: 8px; max-width: 420px; }
.form label { font-weight: 550; margin-top: 6px; }
.form .btn { align-self: flex-start; margin-top: 10px; }
.form fieldset { display: flex; flex-direction: column; gap: 8px; margin-top: 10px; }
.form-inline { display: flex; align-items: center; gap: 12px; flex-wrap: wrap; margin: 8px 0; }
.form-inline label { min-width: 80px; font-weight: 550; }
.row { display: grid; grid-template-columns: 1fr 1fr; gap: 12px; }
@@ -349,6 +351,8 @@ legend { display: flex; align-items: center; gap: 10px; font-size: 17px; font-we
.account-head { display: flex; align-items: center; justify-content: space-between; gap: 12px; flex-wrap: wrap; }
.account-head h2 { display: flex; align-items: center; gap: 10px; }
.tag { padding: 2px 8px; border-radius: 6px; background: var(--surface-2); color: var(--text-2); font-size: 13px; font-weight: 500; line-height: 20px; }
.tag.gain { background: var(--gain-bg); color: var(--gain); }
.tag.loss { background: var(--loss-bg); color: var(--loss); }
.link-state { display: inline-flex; align-items: center; gap: 8px; color: var(--text-2); font-size: 13.5px; }
.link-state.down .dot { background: var(--loss); }
.account-value { display: flex; align-items: baseline; gap: 14px; flex-wrap: wrap; margin: 14px 0 12px; }
@@ -386,6 +390,255 @@ legend { display: flex; align-items: center; gap: 10px; font-size: 17px; font-we
.currency-lines dd { margin: 0; text-align: right; }
.fx-warning { margin-top: 16px; }
/* ---- Operazioni ---- */
.visually-hidden { position: absolute; width: 1px; height: 1px; overflow: hidden; clip: rect(0 0 0 0); white-space: nowrap; }
.trades-head .lead { margin-top: 0; }
.trades-head .big-number { margin-top: 8px; }
.trades-head .note { margin-top: 10px; }
.trades-head h2 { display: flex; align-items: baseline; gap: 12px; flex-wrap: wrap; }
.cells { margin-bottom: 0; }
.cells > div { display: flex; flex-direction: column; gap: 2px; min-width: 0; }
.cells dt { color: var(--text-3); font-size: 13px; }
.cells dd { margin: 0; display: flex; flex-direction: column; gap: 2px; }
.cells dd strong { font-size: 18px; line-height: 24px; font-weight: 650; letter-spacing: -0.015em; font-variant-numeric: tabular-nums; }
.cells dd span { color: var(--text-3); font-size: 13px; }
.result-bars { display: block; width: 100%; height: 170px; margin-top: 16px; }
.result-bars .axis { stroke: var(--rule); stroke-width: 1; vector-effect: non-scaling-stroke; }
.result-bars rect.gain { fill: var(--gain); }
.result-bars rect.loss { fill: var(--loss); }
.bars-scale { display: flex; justify-content: space-between; gap: 12px; flex-wrap: wrap; }
.filters { display: flex; align-items: center; gap: 12px; flex-wrap: wrap; margin-bottom: 16px; }
.filters .btn:last-child { margin-left: auto; }
.search { position: relative; flex: 1 1 240px; display: flex; align-items: center; }
.search .icon { position: absolute; left: 14px; color: var(--text-3); pointer-events: none; }
.search input { width: 100%; padding-left: 40px !important; }
.trades { display: flex; flex-direction: column; margin: 0 -22px; }
.trade-row {
display: grid; grid-template-columns: 116px minmax(0, 1fr) 128px 70px 176px 146px minmax(0, 1.7fr) 20px;
gap: 12px; align-items: center; padding: 14px 22px; border-bottom: 1px solid var(--rule-inner); font-variant-numeric: tabular-nums;
}
.trade-header { color: var(--text-3); font-size: 13px; padding-top: 4px; padding-bottom: 8px; border-bottom: 1px solid var(--rule); }
.trade-header::after { content: ""; }
.trade-row .num { text-align: right; }
.trade summary { list-style: none; cursor: pointer; }
.trade summary::-webkit-details-marker { display: none; }
.trade summary::after {
content: ""; justify-self: end; width: 8px; height: 8px; margin-right: 4px;
border-right: 2px solid var(--text-2); border-bottom: 2px solid var(--text-2); transform: rotate(45deg) translateY(-3px);
}
.trade[open] summary::after { transform: rotate(-135deg) translateY(-3px); }
.trade summary:hover { background: var(--surface-2); }
.trade[open] { background: var(--surface-2); }
.trade[open] summary { border-bottom: 0; }
.why { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 20px; padding: 4px 22px 22px; border-bottom: 1px solid var(--rule-inner); }
.why-card { background: var(--surface); border: 1px solid var(--rule); border-radius: 12px; padding: 18px 20px; display: flex; flex-direction: column; gap: 10px; }
.why-card h3 { margin: 0 !important; }
.why-card .account-lines { margin-top: 4px; }
.checks { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 8px; }
.checks li { display: flex; align-items: flex-start; gap: 10px; }
.checks .icon { margin-top: 2px; }
.checks .ok .icon { color: var(--gain); }
.checks .warn { color: var(--attention); }
.checks .wait .icon { color: var(--text-2); }
.pager { display: flex; align-items: center; gap: 12px; flex-wrap: wrap; padding-top: 18px; }
.pager .note { margin-right: auto; }
/* ---- Strategie ---- */
.stages { list-style: none; margin: 0; padding: 0; display: grid; grid-template-columns: repeat(4, minmax(0, 1fr)); gap: 16px; }
.stages li { border: 1px solid var(--rule); border-radius: 12px; padding: 16px 18px; display: flex; flex-direction: column; gap: 6px; }
.stages li > strong { font-size: 15px; }
.stages li > p { color: var(--text-2); font-size: 13.5px; }
.stages li.here { border: 2px solid var(--text); background: var(--surface-2); padding: 15px 17px; }
.stages .step { border: 1.5px solid var(--text); background: transparent; width: 24px; height: 24px; font-size: 13px; font-weight: 650; }
.stages .here .step { background: var(--text); color: var(--surface); }
.stage-count { display: flex; align-items: center; gap: 4px; margin-top: auto; padding-top: 6px; color: var(--text) !important; }
.stage-count .swatch { margin-right: 0; }
.stage-count strong { margin-left: 4px; }
.strategy { display: grid; grid-template-columns: minmax(0, 1.3fr) minmax(0, 1fr) minmax(0, 1.15fr); padding: 0; }
.strategy > div { padding: 24px 26px; min-width: 0; display: flex; flex-direction: column; gap: 12px; }
.strategy > div + div { border-left: 1px solid var(--rule); }
.strategy-about h2 { display: flex; align-items: center; gap: 4px; flex-wrap: wrap; font-size: 20px; line-height: 28px; }
.strategy-about h2 .tag { margin-left: 10px; background: var(--info-bg); color: var(--link); }
.strategy-about > p:first-of-type { font-size: 15px; line-height: 22px; }
.strategy-about ul { margin: 0; padding-left: 20px; color: var(--text-2); display: flex; flex-direction: column; gap: 4px; }
.strategy-about .note { margin-top: auto; }
.strategy-number { display: flex; align-items: center; gap: 14px; font-size: 30px; line-height: 36px; font-weight: 650; letter-spacing: -0.02em; font-variant-numeric: tabular-nums; }
.spark { display: block; width: 100%; height: 80px; }
.strategy-result .account-lines { grid-template-columns: max-content minmax(0, 1fr); column-gap: 16px; }
.strategy-result .account-lines dd { overflow-wrap: anywhere; }
.spark polyline { fill: none; stroke-width: 2; stroke-linejoin: round; }
.strategy-steps h3 { margin: 0 !important; }
.strategy-steps form { margin: 0; }
.confirm-live summary { list-style: none; align-self: flex-start; display: inline-flex; }
.confirm-live summary::-webkit-details-marker { display: none; }
.confirm-live[open] summary { margin-bottom: 12px; }
.confirm-summary { margin: 0; padding-left: 20px; display: grid; gap: 4px; }
.form-grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 12px 16px; align-items: end; }
.form-grid > p { grid-column: 1 / -1; }
.form-grid label { display: flex; flex-direction: column; gap: 6px; font-weight: 550; }
.form-grid .btn { justify-self: start; }
.confirm-live .form-grid { grid-template-columns: 1fr; }
.plain { border-top: 2px solid var(--text); padding-top: 18px; }
.plain > h2 { margin-bottom: 8px; }
.observing { display: grid; grid-template-columns: minmax(180px, max-content) minmax(0, 1fr); margin: 0; }
.observing dt, .observing dd { margin: 0; padding: 14px 0; border-bottom: 1px solid var(--rule-inner); }
.observing dt { font-weight: 650; padding-right: 32px; }
.observing dd { display: flex; align-items: center; justify-content: space-between; gap: 16px; color: var(--text-2); }
.observing form { margin: 0; flex: none; }
.attention { display: inline-flex; align-items: center; gap: 6px; }
/* ---- Rischio ---- */
.risk-top { display: grid; grid-template-columns: minmax(0, 1.6fr) minmax(0, 1fr); gap: 20px; align-items: stretch; }
.risk-top .panel { display: flex; flex-direction: column; gap: 12px; }
.risk-top .panel-head { margin-bottom: 0; }
.risk-top p { color: var(--text-2); }
.btn-danger-solid { background: var(--loss); border-color: var(--loss); color: var(--surface); }
.btn-danger-solid:hover { background: var(--loss); opacity: .9; }
.status-line.gain { display: flex; align-items: center; gap: 8px; color: var(--gain); font-weight: 600; font-size: 14px; white-space: normal; }
.blocks { list-style: none; margin: 0; padding: 0; display: flex; flex-direction: column; gap: 6px; }
.blocks li { display: flex; align-items: flex-start; gap: 10px; }
.blocks .icon { margin-top: 2px; }
.block-list { margin: 0; display: flex; flex-direction: column; }
.block-list dt { color: var(--text-3); font-size: 13px; padding-top: 12px; border-top: 1px solid var(--rule-inner); }
.block-list dd { margin: 0 0 12px; display: flex; flex-direction: column; gap: 2px; }
.block-list dd span { color: var(--text-2); font-size: 13.5px; }
.limits td.wide { width: 24%; vertical-align: middle; }
.usage { display: block; position: relative; height: 6px; border-radius: 3px; background: var(--track); overflow: hidden; }
.usage span { display: block; height: 100%; border-radius: 3px; background: var(--text-2); min-width: 2px; }
.usage.high span { background: var(--attention); }
.drawdown { display: block; width: 100%; height: auto; margin-top: 12px; overflow: visible; }
.drawdown .axis { stroke: var(--rule); stroke-width: 1; }
.drawdown .tick { fill: var(--text-3); font-size: 12px; }
.drawdown .area { fill: var(--loss); opacity: .1; }
.drawdown .line { fill: none; stroke: var(--loss); stroke-width: 2; stroke-linejoin: round; }
.drawdown .dot { fill: var(--loss); }
.drawdown .label { fill: var(--text); font-size: 12.5px; font-weight: 650; }
.columns .panel .notice { margin-top: 12px; }
.never { list-style: none; margin: 0; padding: 0; }
.never li { display: flex; align-items: center; gap: 12px; padding: 12px 0; border-bottom: 1px solid var(--rule-inner); }
.never .icon { color: var(--text-2); }
.events { display: grid; grid-template-columns: 140px minmax(0, 1fr); margin: 0 0 12px; }
.events dt, .events dd { margin: 0; padding: 12px 0; border-bottom: 1px solid var(--rule-inner); }
.events dt { color: var(--text-3); }
.limits-form { max-width: none; margin-bottom: 8px; }
/* ---- Panoramica, under the top block (mockup 01) ---- */
.phone-only { display: none !important; }
.next-decision { display: flex; align-items: flex-start; gap: 10px; color: var(--text-2); font-size: 14px; }
.next-decision .icon { margin-top: 2px; color: var(--text-3); }
.overview-row { display: grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 20px; align-items: stretch; }
.overview-row-wide { grid-template-columns: minmax(0, 1.85fr) minmax(0, 1fr); align-items: start; }
.overview-empty { grid-column: span 2; display: flex; flex-direction: column; gap: 8px; color: var(--text-2); }
.overview-account { display: flex; flex-direction: column; }
.overview-account h2 { font-size: 15px; line-height: 22px; }
.overview-account .account-value { margin: 12px 0 0; gap: 12px; }
.overview-account .account-equity { font-size: 28px; line-height: 34px; }
.split-bar.thin { height: 8px; margin-top: 16px; }
.split-key { display: flex; justify-content: space-between; gap: 12px; flex-wrap: wrap; margin-top: 10px; color: var(--text-2); font-size: 13px; }
.split-key > span { display: inline-flex; align-items: center; }
.split-key .swatch { width: 9px; height: 9px; margin-right: 4px; }
.split-key .swatch:last-of-type { margin-right: 8px; }
.split-key strong { color: var(--text); font-weight: 600; margin-left: 6px; font-variant-numeric: tabular-nums; }
.month { display: flex; align-items: flex-end; justify-content: space-between; gap: 12px; margin-top: 16px; padding-top: 14px; border-top: 1px solid var(--rule-inner); }
.month > div { display: flex; flex-direction: column; }
.month strong { font-weight: 650; font-variant-numeric: tabular-nums; }
.spark-mini { width: 120px; height: 36px; flex: none; overflow: visible; }
.spark-mini polyline { fill: none; stroke: var(--text-2); stroke-width: 1.5; stroke-linejoin: round; }
.spark-mini circle { fill: var(--text); }
.overview-account > .note { margin-top: 14px; }
.status-pill { display: inline-flex; align-items: center; gap: 6px; font-size: 13.5px; font-weight: 600; }
.status-pill.attention { color: var(--attention); }
.overview-risk { display: flex; flex-direction: column; }
.overview-risk .panel-head { margin-bottom: 10px; }
.overview-risk > .note { margin-top: 14px; }
.overview-risk .more { margin-top: 10px; align-self: flex-start; }
.limit-list { display: flex; flex-direction: column; gap: 12px; }
.limit { display: flex; flex-direction: column; gap: 6px; }
.limit-line { display: flex; justify-content: space-between; gap: 12px; color: var(--text-2); font-size: 13.5px; font-variant-numeric: tabular-nums; }
.limit-line strong { color: var(--text); font-weight: 650; }
.usage.notched { overflow: visible; }
.notch { position: absolute; top: -4px; width: 2px; height: 14px; margin-left: -1px; background: var(--text); border-radius: 1px; }
.panel-head h2 .note { margin-left: 8px; }
.overview-positions { padding-bottom: 6px; }
.strategy-list { list-style: none; margin: 0; padding: 0; }
.strategy-list li { display: flex; flex-direction: column; gap: 4px; padding: 14px 0; border-bottom: 1px solid var(--rule-inner); }
.strategy-list li:first-child { padding-top: 4px; }
.strategy-line { display: flex; justify-content: space-between; gap: 12px; font-variant-numeric: tabular-nums; }
.strategy-line strong { display: inline-flex; align-items: center; font-weight: 650; }
.strategy-bar { display: block; height: 8px; margin: 4px 0 6px; }
.strategy-bar span { display: block; height: 100%; border-radius: 4px; }
.info-note { margin-top: 18px; padding: 12px 16px; border-radius: 10px; background: var(--info-bg); color: var(--link); font-size: 13.5px; }
.overview-tiles { display: grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 28px; align-items: start; }
.tile .panel-head { margin-bottom: 8px; }
.tile h2 { font-size: 15px; }
.news-list { list-style: none; margin: 0; padding: 0; }
.news-list li { display: flex; flex-direction: column; gap: 4px; padding: 12px 0; border-bottom: 1px solid var(--rule-inner); }
.news-list li:first-child { padding-top: 4px; }
.news-list li:last-child { border-bottom: 0; }
.news-list strong { font-weight: 600; }
.news-list .attention { align-items: flex-start; font-size: 13.5px; }
.news-list .attention .icon { margin-top: 2px; }
.news-list .muted { font-size: 13.5px; }
.tile > p + .limit { margin-top: 14px; }
.tile-facts { display: grid; grid-template-columns: minmax(0, 1fr) auto; gap: 8px 12px; margin: 16px 0 0; padding-top: 12px; border-top: 1px solid var(--rule-inner); font-size: 13.5px; }
.tile-facts dt { color: var(--text-2); }
.tile-facts dd { margin: 0; text-align: right; font-weight: 600; }
.system-tiles { list-style: none; margin: 0; padding: 0; }
.system-tiles li { display: grid; grid-template-columns: 16px auto minmax(0, 1fr); gap: 10px; align-items: center; padding: 8px 0; border-bottom: 1px solid var(--rule-inner); font-size: 13.5px; }
.system-tiles li:last-child { border-bottom: 0; }
.system-tiles .muted { text-align: right; }
.ok > .icon { color: var(--gain); }
.warn > .icon { color: var(--attention); }
.off > .icon { color: var(--text-3); }
.phone-list { list-style: none; margin: 0; padding: 0; }
.phone-list li { display: flex; justify-content: space-between; gap: 12px; padding: 14px 0; border-top: 1px solid var(--rule-inner); }
.phone-list li > span { display: flex; flex-direction: column; gap: 4px; min-width: 0; }
.phone-list .num { align-items: flex-end; text-align: right; font-variant-numeric: tabular-nums; }
.phone-list .sub { display: inline-flex; align-items: center; gap: 6px; color: var(--text-2); font-size: 14px; }
.phone-list .sub.gain, .phone-list strong.gain { color: var(--gain); }
.phone-list .sub.loss, .phone-list strong.loss { color: var(--loss); }
.phone-list .swatch { margin-right: 0; }
.phone-list .dot.down { background: var(--loss); }
/* ---- Diario e sistema (mockup 08) ---- */
.journal-columns { display: grid; grid-template-columns: minmax(0, 1.75fr) minmax(0, 1fr); gap: 24px; align-items: start; }
.journal { padding: 24px 26px; }
.journal-head { display: flex; align-items: center; justify-content: space-between; gap: 12px 16px; flex-wrap: wrap; }
.journal-search { flex: 0 1 330px; }
.chips { display: flex; flex-wrap: wrap; gap: 8px; margin: 16px 0 8px; }
.chip { display: inline-flex; align-items: center; min-height: 32px; padding: 0 14px; border: 1px solid var(--control); border-radius: 999px; color: var(--text); text-decoration: none; font-size: 14px; }
.chip:hover { background: var(--surface-2); }
.chip.on { background: var(--text); border-color: var(--text); color: var(--surface); font-weight: 550; }
.journal-found { margin-top: 12px; }
.panel h3.journal-day { margin: 28px 0 0; padding-bottom: 10px; border-bottom: 2px solid var(--text); font-size: 14.5px; }
.journal-list { list-style: none; margin: 0; padding: 0; }
.journal-list li { display: grid; grid-template-columns: 44px 22px minmax(0, 1fr) auto; gap: 12px; align-items: start; padding: 12px 0; border-bottom: 1px solid var(--rule-inner); }
.journal-list time { color: var(--text-3); font-variant-numeric: tabular-nums; }
.journal-icon { color: var(--text-2); margin-top: 1px; }
.attention-entry .journal-icon { color: var(--attention); }
.journal-text { min-width: 0; display: flex; flex-direction: column; gap: 4px; overflow-wrap: anywhere; }
.journal-text strong { font-weight: 650; }
.journal-rest { color: var(--text-2); }
.journal-kind { color: var(--text-3); font-size: 13.5px; text-align: right; }
.journal-more { display: flex; justify-content: center; margin-top: 18px; }
.journal-side { display: flex; flex-direction: column; gap: 24px; min-width: 0; }
.journal-side .panel-head { margin-bottom: 4px; }
.system-parts { list-style: none; margin: 0; padding: 0; }
.system-parts li { display: flex; align-items: flex-start; gap: 12px; padding: 12px 0; border-top: 1px solid var(--rule-inner); }
.system-parts li > .icon { margin-top: 2px; }
.system-parts li > span { display: flex; flex-direction: column; min-width: 0; }
.system-parts strong { font-weight: 600; }
.system-parts li > span > span { color: var(--text-2); }
.job-list li { grid-template-columns: 96px minmax(0, 1fr); }
.job-list li:last-child { border-bottom: 0; }
.host { display: flex; flex-direction: column; gap: 12px; }
.host > h2 { margin-bottom: 0; }
.host .btn { align-self: flex-start; }
@media (max-width: 1180px) {
.overview-row { grid-template-columns: repeat(2, minmax(0, 1fr)); }
.overview-row-wide, .journal-columns { grid-template-columns: minmax(0, 1fr); }
}
/* ---- Service pages ---- */
.service { min-height: 100vh; display: flex; justify-content: center; padding: 48px 16px; }
.service-box { width: 100%; max-width: 720px; display: flex; flex-direction: column; gap: 16px; }
@@ -435,6 +688,25 @@ legend { display: flex; align-items: center; gap: 10px; font-size: 17px; font-we
.panel-tools { width: 100%; justify-content: space-between; }
.facts { grid-template-columns: 1fr; gap: 2px; }
.facts dd { margin-bottom: 8px; }
.panel > h2 { flex-direction: column; align-items: flex-start; gap: 4px; }
.trades-head .cells { grid-template-columns: 1fr 1fr; }
.trades { margin: 0 -18px; }
.trade-row { grid-template-columns: minmax(0, 1fr) auto 16px; gap: 4px 12px; padding: 12px 18px; }
.trade-row > span:first-child { grid-column: 1 / -1; color: var(--text-3); font-size: 13px; }
.trade-row .wide, .trade-header, .limits .wide, .filters .btn:last-child { display: none; }
.why { grid-template-columns: 1fr; padding: 4px 18px 18px; }
.filters > label:not(.search) { flex: 1 1 140px; }
.filters select { width: 100%; }
.stages { grid-template-columns: 1fr 1fr; gap: 10px; }
.strategy { grid-template-columns: 1fr; }
.strategy > div { padding: 18px; }
.strategy > div + div { border-left: 0; border-top: 1px solid var(--rule); }
.risk-top { grid-template-columns: 1fr; gap: 14px; }
.observing { grid-template-columns: 1fr; }
.observing dt { border-bottom: 0; padding-bottom: 0; }
.observing dd { flex-direction: column; align-items: flex-start; }
.events { grid-template-columns: 1fr; }
.events dt { border-bottom: 0; padding-bottom: 0; }
.bottombar {
display: grid; grid-template-columns: repeat(5, 1fr); gap: 4px; position: fixed; left: 0; right: 0; bottom: 0;
padding: 6px 6px calc(6px + env(safe-area-inset-bottom)); background: var(--surface); border-top: 1px solid var(--rule); z-index: 10;
@@ -446,3 +718,22 @@ legend { display: flex; align-items: center; gap: 10px; font-size: 17px; font-we
.bottom-item.active { background: var(--surface-2); color: var(--text); font-weight: 650; }
.nav-item, .btn, .more-list a { min-height: 44px; }
}
/* Panoramica and Diario on the phone: mockup 09, and mockup 08 stacked in one column */
@media (max-width: 760px) {
.desk-only { display: none !important; }
.cells > .cell-extra { display: none; }
section.phone-only, div.phone-only, ul.phone-only { display: block !important; }
a.phone-only { display: inline-block !important; }
span.phone-only { display: inline !important; }
.overview-row, .overview-row-wide, .overview-tiles { grid-template-columns: minmax(0, 1fr); gap: 14px; }
.half-next .moves { margin-top: 4px; border-top: 1px solid var(--rule); }
.phone-list li:last-child { padding-bottom: 4px; }
.journal { padding: 18px; }
.journal-search { flex: 1 1 100%; }
.journal-list li { grid-template-columns: 40px 18px minmax(0, 1fr); gap: 4px 10px; }
.journal-kind { grid-column: 3; text-align: left; font-size: 12.5px; line-height: 16px; }
.chip { min-height: 40px; }
.journal-side { gap: 14px; }
.job-list li { grid-template-columns: 84px minmax(0, 1fr); }
}
+142
View File
@@ -0,0 +1,142 @@
using System.Collections;
using System.Globalization;
using Anubis.Backtest;
using Anubis.Brokers;
using Anubis.Core;
using Anubis.Data;
using Anubis.Strategies;
using Anubis.Trading;
namespace Anubis.Server;
/// <summary>
/// "anubis backtest base": the honest backtests of the base strategies (SPEC 12.2, issue #44). It downloads the
/// Binance daily archives (each checked against its SHA-256) and the ECB rates into the research history, converts
/// them to euro, runs the protocol of <c>crypto_trend</c> on both universes and writes reports and outcome files.
/// The stock strategies need the Alpaca history: without it their files say they wait, and they stay in observation.
/// </summary>
internal static class BacktestCommand
{
private const string Usage =
"Uso: anubis backtest base [--to AAAA-MM-GG] [--out docs/backtests] [--offline]\n" +
" --offline usa solo gli archivi già scaricati (i cambi BCE li scarica solo se mancano). Opzioni comuni: [--platform docker|desktop] [--data-dir <cartella>]";
public static async Task<int> RunAsync(string[] args, TextWriter? output = null, IReadOnlyDictionary<string, string?>? environment = null,
HttpMessageHandler? handler = null)
{
output ??= Console.Out;
if (args.Length < 1 || args[0] != "base")
{
await output.WriteLineAsync(Usage).ConfigureAwait(false);
return 2;
}
var rest = args[1..].ToList();
environment ??= Environment.GetEnvironmentVariables().Cast<DictionaryEntry>().ToDictionary(e => (string)e.Key, e => e.Value as string, StringComparer.Ordinal);
var options = ServerOptions.Parse([.. rest.Where(a => a != "--offline")], environment);
var config = options.LoadConfig(environment);
var clock = new SystemClock();
var now = clock.UtcNow;
var to = Option(rest, "--to") is { } t
? DateOnly.ParseExact(t, "yyyy-MM-dd", CultureInfo.InvariantCulture)
: new DateOnly(now.Year, now.Month, 1).AddDays(-1); // The last month whose archive is published.
var outDir = Path.GetFullPath(Option(rest, "--out") ?? Path.Combine("docs", "backtests"));
try
{
config.Data.Ensure();
Directory.CreateDirectory(outDir);
var store = new ResearchStore(config.Data.Research);
IReadOnlyList<FxRate> rates;
using (var http = handler is null ? new HttpClient() : new HttpClient(handler, disposeHandler: false))
{
http.Timeout = TimeSpan.FromMinutes(5);
http.DefaultRequestHeaders.UserAgent.ParseAdd($"{AnubisInfo.Name}/{AnubisInfo.Version}");
rates = await RatesAsync(new ApiHttp(http, clock, "ecb"), config.Data.Research, rest.Contains("--offline"), output).ConfigureAwait(false);
if (!rest.Contains("--offline"))
{
var importer = new BinanceImporter(new ApiHttp(http, clock, "binance"), store, clock);
foreach (var symbol in CryptoTrendProtocol.Archives)
{
var result = await importer.ImportAsync(symbol, "1d", 2017, 8, CancellationToken.None).ConfigureAwait(false);
await output.WriteLineAsync(string.Create(CultureInfo.InvariantCulture, $"{symbol}: {result.Bars} barre giornaliere")).ConfigureAwait(false);
}
}
}
var converted = await EuroHistory.ConvertAllAsync(store, rates, CancellationToken.None).ConfigureAwait(false);
await output.WriteLineAsync($"Convertite in euro {converted.Count} coppie.").ConfigureAwait(false);
var outcome = await CryptoTrendProtocol.RunAsync(config.Data.Research, to, now, text => output.WriteLine(text), CancellationToken.None).ConfigureAwait(false);
var notes = DataNotes(outcome, rates);
await File.WriteAllTextAsync(Path.Combine(outDir, "crypto_trend.md"),
ProtocolReport.Markdown(outcome, "crypto_trend", notes, "anubis backtest base --to " + to.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture))).ConfigureAwait(false);
await File.WriteAllTextAsync(Path.Combine(outDir, "crypto_trend.outcome.json"), BacktestOutcomes.ToJson(outcome.Qualification)).ConfigureAwait(false);
foreach (var run in outcome.Runs)
{
// The full reports (curve, trades) stay with the data: the repository keeps the summary and the outcome.
await File.WriteAllTextAsync(Path.Combine(config.Data.Research, "crypto_trend-" + run.Report.PlanHash[..12] + ".report.json"), run.Report.ToJson())
.ConfigureAwait(false);
}
foreach (var (strategy, name) in new[] { (StrategyNames.EquityMomentum, "Momentum azioni"), (StrategyNames.EquityMeanrev, "Ritorno alla media azioni") })
{
var waiting = EquityProtocol.Waiting(strategy);
await File.WriteAllTextAsync(Path.Combine(outDir, strategy + ".md"), ProtocolReport.Waiting(strategy, name, waiting, EquityProtocol.Explanation)).ConfigureAwait(false);
await File.WriteAllTextAsync(Path.Combine(outDir, strategy + ".outcome.json"), BacktestOutcomes.ToJson(waiting)).ConfigureAwait(false);
}
await output.WriteLineAsync((outcome.Qualification.Passed ? "Superato: " : "Non superato: ") + outcome.Qualification.Verdict).ConfigureAwait(false);
await output.WriteLineAsync("Rapporti scritti in " + outDir).ConfigureAwait(false);
return 0;
}
catch (Exception ex) when (ex is IOException or MarketDataException or HttpRequestException or FormatException or InvalidOperationException
or UnauthorizedAccessException or TaskCanceledException or System.IO.InvalidDataException)
{
await output.WriteLineAsync("Non riuscito: " + Redactor.Redact(ex.Message)).ConfigureAwait(false);
return 1;
}
}
/// <summary>The ECB history of EUR/USD, kept next to the research data so an offline run uses the same rates.</summary>
private static async Task<IReadOnlyList<FxRate>> RatesAsync(ApiHttp http, string researchRoot, bool offline, TextWriter output)
{
var path = Path.Combine(researchRoot, "ecb-eurofxref-hist.zip");
if (!offline || !File.Exists(path))
{
var zip = await http.GetBytesAsync(Ecb.History, null, null, CancellationToken.None).ConfigureAwait(false)
?? throw new MarketDataException("BCE: lo storico dei cambi non è disponibile.", retryable: true);
await File.WriteAllBytesAsync(path, zip).ConfigureAwait(false);
}
var rates = Ecb.ParseHistoryCsv(Ecb.ExtractHistoryCsv(await File.ReadAllBytesAsync(path).ConfigureAwait(false)), ["USD"]);
await output.WriteLineAsync(string.Create(CultureInfo.InvariantCulture, $"Cambi BCE EUR/USD: {rates.Count} giorni, dal {rates[0].Date:yyyy-MM-dd} al {rates[^1].Date:yyyy-MM-dd}."))
.ConfigureAwait(false);
return rates;
}
private static List<string> DataNotes(ProtocolOutcome outcome, IReadOnlyList<FxRate> rates)
{
var broad = outcome.Runs[0].Report.Instruments;
var it = CultureInfo.GetCultureInfo("it-IT");
var missing = CryptoTrendProtocol.Archives.Select(EuroHistory.SymbolOf).Where(s => broad.All(i => i.Symbol != s)).ToList();
var notes = new List<string>
{
"Archivi giornalieri di data.binance.vision in USDT, ogni file controllato con la sua impronta SHA-256 prima dell'uso.",
string.Create(it, $"Prezzi convertiti in euro con il cambio EUR/USD della BCE del giorno ({rates.Count} giorni di cambi), contando un USDT come un dollaro. Le coppie in euro di OKX esistono solo da novembre 2023: questo storico è un'approssimazione (SPEC 8.1)."),
string.Create(it, $"{broad.Count} coppie nell'universo ampio, comprese quelle poi chiuse da Binance: ogni mese la strategia sceglie le 10 con più scambi tra quelle quotate da almeno 400 giorni."),
"Lo spread non si può misurare su questo storico: il filtro dello spread vale solo dal vivo.",
};
if (missing.Count > 0)
{
notes.Add("Senza archivi nel periodo, quindi non usate: " + string.Join(", ", missing) + ".");
}
return notes;
}
private static string? Option(List<string> args, string name)
{
var index = args.IndexOf(name);
return index >= 0 && index < args.Count - 1 ? args[index + 1] : null;
}
}
+32 -9
View File
@@ -5,8 +5,8 @@
HTML placed in percent, so nothing gets distorted at any width. *@
@if (Points.Count >= 2)
{
var values = Points.Select(p => Shell.Convert(p.Equity, Currency)).ToList();
var bench = Points.Select(p => p.Benchmark is { } b ? Shell.Convert(b, Currency) : (decimal?)null).ToList();
var values = Points.Select(p => Value(p.Equity)).ToList();
var bench = Points.Select(p => p.Benchmark is { } b ? Value(b) : (decimal?)null).ToList();
var all = values.Concat(bench.OfType<decimal>()).ToList();
var (lo, hi, step) = Scale(all.Min(), all.Max());
double X(int i) => i * 1000.0 / (Points.Count - 1);
@@ -15,7 +15,7 @@
var equityLine = Line(values.Select((v, i) => (i, v)));
var benchLine = Line(bench.Select((v, i) => (i, v)).Where(p => p.v is not null).Select(p => (p.i, p.v!.Value)));
var last = values[^1];
<figure class="chart" aria-label="Patrimonio nel tempo, con il bot e senza">
<figure class="chart" aria-label="@Label">
<div class="chart-box">
<div class="chart-axis" aria-hidden="true">
@for (var t = lo; t <= hi; t += step)
@@ -42,28 +42,28 @@
var gap = Math.Abs(Y(last) - Y(lastBench)) / 10;
var benchTop = gap < 7 ? Y(last) / 10 + (Y(lastBench) >= Y(last) ? 7 : -7) : Y(lastBench) / 10;
<span class="end-dot bench-dot" style="left:100%;top:@Pct(Y(lastBench) / 10)"></span>
<span class="end-label bench-label" style="top:@Pct(benchTop)">@Shell.Money(Points[^1].Benchmark!.Value, Currency, decimals: 0)</span>
<span class="end-label bench-label" style="top:@Pct(benchTop)">@Text(Points[^1].Benchmark!.Value, 0)</span>
}
<span class="end-dot" style="left:100%;top:@Pct(Y(last) / 10)"></span>
<span class="end-label" style="top:@Pct(Y(last) / 10)">@Shell.Money(Points[^1].Equity, Currency, decimals: 0)</span>
<span class="end-label" style="top:@Pct(Y(last) / 10)">@Text(Points[^1].Equity, 0)</span>
</div>
</div>
<figcaption class="legend">
<span class="key key-equity">Con Anubis</span>
<span class="key key-equity">@EquityLabel</span>
@if (benchLine.Length > 0)
{
<span class="key key-bench">Senza bot: Bitcoin e S&amp;P 500 comprati e tenuti</span>
<span class="key key-bench">@BenchmarkLabel</span>
}
</figcaption>
<details class="chart-table">
<summary>Vedi i numeri in una tabella</summary>
<div class="table-wrap">
<table class="table">
<thead><tr><th scope="col">Giorno</th><th scope="col" class="num">Con il bot</th><th scope="col" class="num">Senza bot</th></tr></thead>
<thead><tr><th scope="col">Giorno</th><th scope="col" class="num">@EquityColumn</th><th scope="col" class="num">@BenchmarkColumn</th></tr></thead>
<tbody>
@foreach (var p in Points.AsEnumerable().Reverse())
{
<tr><td>@Format.LongDate(p.Day)</td><td class="num">@Shell.Money(p.Equity, Currency)</td><td class="num">@(p.Benchmark is { } b ? Shell.Money(b, Currency) : "—")</td></tr>
<tr><td>@Format.LongDate(p.Day)</td><td class="num">@Text(p.Equity)</td><td class="num">@(p.Benchmark is { } b ? Text(b) : "—")</td></tr>
}
</tbody>
</table>
@@ -79,6 +79,29 @@
[Parameter]
public string Currency { get; set; } = "EUR";
/// <summary>True when the amounts are already in <see cref="Currency"/> (a backtest report): no conversion from euro.</summary>
[Parameter]
public bool Native { get; set; }
[Parameter]
public string Label { get; set; } = "Patrimonio nel tempo, con il bot e senza";
[Parameter]
public string EquityLabel { get; set; } = "Con Anubis";
[Parameter]
public string BenchmarkLabel { get; set; } = "Senza bot: Bitcoin e S&P 500 comprati e tenuti";
[Parameter]
public string EquityColumn { get; set; } = "Con il bot";
[Parameter]
public string BenchmarkColumn { get; set; } = "Senza bot";
private decimal Value(decimal amount) => Native ? amount : Shell.Convert(amount, Currency);
private string Text(decimal amount, int decimals = 2) => Native ? Format.Money(amount, Currency, decimals: decimals) : Shell.Money(amount, Currency, decimals: decimals);
private static string Fmt(double v) => v.ToString("0.##", CultureInfo.InvariantCulture);
private static string Pct(double v) => v.ToString("0.###", CultureInfo.InvariantCulture) + "%";
+9
View File
@@ -80,6 +80,15 @@
case "lock":
<rect x="5" y="10.5" width="14" height="10" rx="2.2" /><path d="M8.5 10.5V7.8a3.5 3.5 0 0 1 7 0v2.7" />
break;
case "search":
<circle cx="11" cy="11" r="6.5" /><path d="m16 16 4 4" />
break;
case "ban":
<circle cx="12" cy="12" r="8.5" /><path d="m6 6 12 12" />
break;
case "tick":
<path d="m5 12.5 4.5 4.5L19 7.5" />
break;
case "logout":
<path d="M14 4h4.5A1.5 1.5 0 0 1 20 5.5v13a1.5 1.5 0 0 1-1.5 1.5H14M10 16l-4-4 4-4M6 12h10" />
break;
@@ -221,7 +221,7 @@ else
<tr>
<th scope="row"><strong>@p.Symbol</strong> <span class="muted">@p.Name</span></th>
<td class="wide">@(p.Broker == Venues.Alpaca ? "Alpaca" : "OKX")</td>
<td class="nowrap wide"><span class="swatch round" style="background: var(--s-@p.Strategy.Replace('_', '-'))"></span>@Strategies.Label(p.Strategy)</td>
<td class="nowrap wide"><span class="swatch round" style="background: var(--s-@p.Strategy.Replace('_', '-'))"></span>@StrategyNames.Label(p.Strategy)</td>
<td class="num wide">@Format.Quantity(p.Quantity)</td>
<td class="num wide">@Format.Money(p.Average, p.Currency)</td>
<td class="num wide">@Format.Money(p.Last, p.Currency)</td>
+133 -60
View File
@@ -5,86 +5,159 @@
<PageTitle>Diario e sistema · Anubis</PageTitle>
<div class="columns">
<section class="panel">
<h2>Diario</h2>
@if (_entries.Count == 0)
@{
var d = _diary;
var s = _system;
}
<div class="journal-columns">
<section class="panel journal" aria-labelledby="diario">
<div class="journal-head">
<h2 id="diario">Diario</h2>
<form class="search journal-search" method="get" action="/diario" role="search">
@if (d.Filter != "tutto")
{
<input type="hidden" name="tipo" value="@d.Filter" />
}
<Icon Name="search" Size="16" />
<label class="visually-hidden" for="cerca-diario">Cerca nel diario</label>
<input id="cerca-diario" type="search" name="cerca" value="@d.Search" placeholder="Cerca nel diario" maxlength="100" />
</form>
</div>
<nav class="chips" aria-label="Tipo di voce">
@foreach (var (key, label, _) in DiaryView.Filters)
{
<a href="@d.Link(filter: key, range: DiaryView.DefaultRange)" class="chip @(key == d.Filter ? "on" : null)" aria-current="@(key == d.Filter ? "true" : null)">@label</a>
}
</nav>
@if (d.Search is { } search)
{
<p class="note">Ancora nessuna voce.</p>
<p class="note journal-found">
@(d.Days.Sum(x => x.Entries.Count) switch { 0 => "Nessuna voce", 1 => "Una voce", var n => $"{n} voci" }) con «@search»@(d.Truncated ? $", le {DiaryView.MaxSearchResults} più recenti" : "").
<a href="@(new DiaryView([], d.Filter, null, DiaryView.DefaultRange, false, false).Link())">Togli la ricerca</a>
</p>
}
else
@if (d.IsEmpty && d.Search is null)
{
<div class="table-wrap">
<table class="table">
<thead><tr><th scope="col">Quando</th><th scope="col">Tipo</th><th scope="col">Voce</th></tr></thead>
<tbody>
@foreach (var e in _entries)
{
<tr><td class="nowrap">@Format.ShortDate(e.At, Shell.Now, Shell.Zone)</td><td>@e.Kind</td><td>@e.Text</td></tr>
}
</tbody>
</table>
</div>
<p class="note">@(d.Filter == "tutto" ? "Ancora nessuna voce. Qui il bot scrive ogni decisione, ordine e controllo, in parole semplici, appena succede." : "Nessuna voce di questo tipo finora.")</p>
}
@foreach (var day in d.Days)
{
<h3 class="journal-day">@day.Heading</h3>
<ol class="journal-list">
@foreach (var e in day.Entries)
{
<li class="@(e.Attention ? "attention-entry" : null)">
<time datetime="@e.At.ToString("o", CultureInfo.InvariantCulture)">@TimeZoneInfo.ConvertTime(e.At, Shell.Zone).ToString("HH:mm", CultureInfo.InvariantCulture)</time>
<Icon Name="@e.Icon" Size="18" Class="journal-icon" />
<div class="journal-text">
<p>@if (e.Lead is { } lead) {<strong>@lead</strong>}@foreach (var piece in e.Main) {if (piece.Class is { } css) {<strong class="@css">@piece.Text</strong>} else {@piece.Text}}</p>
@if (e.Rest is { } rest)
{
<p class="journal-rest">@rest</p>
}
@if (e.Trade is { } trade)
{
<a href="/operazioni?cerca=@Uri.EscapeDataString(trade)">Perché l'ho comprata e venduta</a>
}
</div>
<span class="journal-kind">@e.Kind</span>
</li>
}
</ol>
}
@if (d.HasOlder)
{
<div class="journal-more"><a class="btn" href="@d.Link(range: d.Range + 7)">Mostra i giorni precedenti</a></div>
}
</section>
<section class="panel">
<h2>Sistema</h2>
<dl class="facts">
<dt>Versione</dt><dd>@AnubisInfo.Version</dd>
<dt>Piattaforma</dt><dd>@(Host.Config.Platform == PlatformKind.Docker ? "Docker" : "Desktop")</dd>
<dt>Database</dt><dd>@(Host.Config.Db?.Engine == DbEngine.Postgres ? "PostgreSQL" : "SQLite") · risponde</dd>
<dt>In funzione da</dt><dd>@Format.Ago(Shell.Now - _since).Replace(" fa", "", StringComparison.Ordinal)</dd>
</dl>
<h3 class="list-title">Lavori in programma</h3>
<ul class="moves">
@foreach (var job in _upcoming)
{
<li><span class="move-time">@Format.ShortDate(job.DueUtc, Shell.Now, Shell.Zone)</span><span>@job.Description</span></li>
}
</ul>
@if (_jobs.Count > 0)
{
<h3 class="list-title">Ultimi lavori</h3>
<ul class="moves moves-past">
@foreach (var job in _jobs)
<div class="journal-side">
<section class="panel" aria-labelledby="sistema">
<div class="panel-head">
<h2 id="sistema">Come sta il sistema</h2>
<span class="status-pill @(s.AllWell ? "gain" : "attention")"><Icon Name="@(s.AllWell ? "check" : "alert")" Size="16" />@(s.AllWell ? "Tutto regolare" : "Qualcosa da controllare")</span>
</div>
<ul class="system-parts">
@foreach (var part in s.Parts)
{
<li><span class="move-time">@(job.Finished is { } f ? Format.ShortDate(f, Shell.Now, Shell.Zone) : "—")</span><span>@job.Description: @Outcome(job.Outcome)</span></li>
<li class="@PartClass(part.State)">
<Icon Name="@PartIcon(part.State)" Size="16" />
<span><strong>@part.Name</strong><span>@part.Detail</span></span>
</li>
}
</ul>
}
</section>
</section>
<section class="panel" aria-labelledby="lavori">
<h2 id="lavori">Lavori in programma</h2>
@if (s.Jobs.Count == 0)
{
<p class="note">Nessun lavoro in programma: il pianificatore parte con il motore.</p>
}
else
{
<ul class="moves job-list">
@foreach (var job in s.Jobs)
{
<li><span class="move-time">@job.Time</span><span>@job.Text</span></li>
}
</ul>
}
</section>
<section class="plain host" aria-labelledby="computer">
<h2 id="computer">Il computer che lo ospita</h2>
@foreach (var r in s.Resources)
{
<div class="limit">
<div class="limit-line"><span>@r.Label</span><span><strong>@r.Value</strong>@if (r.Of is { } of) {<span class="muted"> su @of</span>}</span></div>
<span class="usage @(r.Share >= 0.85m ? "high" : null)"><span style="width: @Css(r.Share * 100m)%"></span></span>
</div>
}
<p class="note">@s.Running</p>
<a class="btn" href="/api/registri.zip" download><Icon Name="download" Size="18" />Scarica i registri tecnici</a>
<a class="btn" href="/api/diagnostica.md" download><Icon Name="download" Size="18" />Scarica i dati per l'analisi</a>
<p class="note">I dati per l'analisi sono un testo da incollare a chi segue lo sviluppo: risorse, lavori, chiamate ed errori, senza chiavi né password.</p>
</section>
</div>
</div>
@code {
private List<(DateTimeOffset At, string Kind, string Text)> _entries = [];
private List<(string Description, DateTimeOffset? Finished, string? Outcome)> _jobs = [];
private IReadOnlyList<UpcomingRun> _upcoming = [];
private DateTimeOffset _since;
private DiaryView _diary = null!;
private SystemView _system = null!;
[CascadingParameter]
public HttpContext? HttpContext { get; set; }
[SupplyParameterFromQuery(Name = "tipo")]
public string? Tipo { get; set; }
[SupplyParameterFromQuery(Name = "cerca")]
public string? Cerca { get; set; }
[SupplyParameterFromQuery(Name = "giorni")]
public string? Giorni { get; set; }
protected override async Task LoadAsync()
{
var ct = HttpContext?.RequestAborted ?? CancellationToken.None;
var services = Host.Services!;
(_entries, _jobs) = await services.Db.ReadAsync(async s =>
{
var entries = await s.QueryAsync("SELECT occurred_at, kind, message FROM journal ORDER BY occurred_at DESC LIMIT 200",
r => (r.Moment("occurred_at"), r.Text("kind"), r.Text("message")));
var jobs = await s.QueryAsync("SELECT description, last_finished_at, last_outcome FROM jobs ORDER BY last_finished_at DESC",
r => (r.Text("description"), r.MomentOrNull("last_finished_at"), r.TextOrNull("last_outcome")));
return (entries, jobs);
}, ct);
_upcoming = services.Scheduler.Upcoming(10);
_since = Host.Status.Since;
_diary = await DiaryView.LoadAsync(Host.Services!.Db, Tipo, Cerca, Giorni, Shell.Now, Shell.Zone, ct);
_system = await SystemView.LoadAsync(Host, Shell, ct);
}
private static string Outcome(string? outcome) => outcome switch
private static string PartClass(PartState state) => state switch
{
"succeeded" => "fatto",
"failed" => "non riuscito",
"skipped" => "saltato, orario superato",
_ => "in attesa",
PartState.Ok => "ok",
PartState.Warn => "warn",
_ => "off",
};
private static string PartIcon(PartState state) => state switch
{
PartState.Ok => "tick",
PartState.Warn => "alert",
_ => "clock",
};
private static string Css(decimal value) => Math.Round(value, 3).ToString("0.###", CultureInfo.InvariantCulture);
}
@@ -21,6 +21,14 @@
"chiavi" => "Chiavi verificate e salvate: possono fare ordini e non possono prelevare. Il motore è ripartito e le usa.",
"chiavi-tolte" => "Chiavi tolte: quel conto torna al simulatore interno per le strategie che non hanno ancora storia sul conto di prova.",
"capitale" => "Capitale affidato aggiornato e diviso tra le strategie.",
"limiti" => "Limiti di rischio salvati.",
"rischio" => "Impostazione del rischio salvata.",
"telegram" => "Telegram funziona: ti ho mandato un messaggio di prova e ho salvato token e chat.",
"telegram-prova" => "Messaggio di prova partito: controlla il telefono.",
"telegram-tolto" => "Telegram tolto: non ti scrivo più finché non lo rimetti.",
"segnale" => "Segnale di vita salvato: la prima chiamata è andata bene, da adesso lo chiamo ogni minuto.",
"segnale-tolto" => "Segnale di vita tolto.",
_ => "Fatto.",
})</p>
}
@@ -30,7 +38,7 @@
}
<nav class="tabs" aria-label="Sezioni delle impostazioni">
<a href="#aspetto">Aspetto</a><a href="#conti">Conti</a><a href="#dati">Dati</a><a href="#sicurezza">Sicurezza</a><a href="#sistema">Sistema</a><a href="#modifiche">Modifiche</a>
<a href="#aspetto">Aspetto</a><a href="#conti">Conti</a><a href="#rischio">Rischio</a><a href="#telegram">Telegram</a><a href="#dati">Dati</a><a href="#sicurezza">Sicurezza</a><a href="#sistema">Sistema</a><a href="#modifiche">Modifiche</a>
</nav>
<section class="panel" id="aspetto">
@@ -118,7 +126,7 @@
<ul class="plain">
@foreach (var s in b.Strategies)
{
<li>@Strategies.Label(s.Strategy, withMarket: true): quota @Format.Percent(s.Share * 100m, 0), capitale @Format.Money(s.Capital, s.Currency), liquidità @Format.Money(s.Cash, s.Currency)</li>
<li>@StrategyNames.Label(s.Strategy, withMarket: true): quota @Format.Percent(s.Share * 100m, 0), capitale @Format.Money(s.Capital, s.Currency), liquidità @Format.Money(s.Cash, s.Currency)</li>
}
</ul>
}
@@ -135,7 +143,143 @@
}
</section>
<section class="panel" id="rischio">
<h2>Rischio</h2>
<p class="note">I limiti valgono per tutte le strategie insieme. Abbassarli è sempre possibile. @(_anyLive ? "Una strategia usa soldi veri: per alzarne uno servono la frase e la password." : "Finché nessuna strategia usa soldi veri, puoi anche alzarli.")</p>
@if (!Shell.IsDemo)
{
<form method="post" action="/azioni/rischio/limiti" class="form-grid limits-form">
<AntiforgeryToken />
@foreach (var l in _limits)
{
<label>@l.Label<input name="@l.Key" value="@l.Value" inputmode="decimal" autocomplete="off" readonly="@l.ReadOnly" /></label>
}
@if (_anyLive)
{
<label>Per alzare un limite scrivi: <code>@StageBook.RaisePhrase</code><input name="frase" autocomplete="off" /></label>
<label>Password<input name="password" type="password" autocomplete="current-password" /></label>
<label>Codice a sei cifre (se attivo)<input name="codice" inputmode="numeric" autocomplete="one-time-code" /></label>
}
<button class="btn" type="submit">Salva i limiti</button>
</form>
<h3>Protezione day trade</h3>
<form method="post" action="/azioni/rischio/@Anubis.Risk.RiskSettings.PdtGuard.Key" class="form-inline">
<AntiforgeryToken />
<label for="pdt">Per un conto Alpaca sotto i 25.000 $: al massimo 3 operazioni aperte e chiuse in giornata ogni 5 sedute.</label>
<select id="pdt" name="valore" data-autosubmit>
<option value="off" selected="@(_pdt == "off")">Spenta</option>
<option value="on" selected="@(_pdt == "on")">Accesa</option>
</select>
<noscript><button class="btn btn-small" type="submit">Salva</button></noscript>
</form>
<h3>Scenari «Se succedesse domani»</h3>
<form method="post" action="/azioni/rischio/@Anubis.Risk.Scenarios.Setting.Key" class="form">
<AntiforgeryToken />
<label for="scenari">Uno per riga: descrizione; variazione delle crypto %; delle azioni USA %; del dollaro sull'euro %. Per esempio «Le crypto perdono il 20% in un giorno; -20; 0; 0».</label>
<textarea id="scenari" name="valore" rows="5" spellcheck="false">@_scenarios</textarea>
<button class="btn" type="submit">Salva gli scenari</button>
</form>
}
else
{
<dl class="facts">
@foreach (var l in _limits)
{
<dt>@l.Label</dt><dd>@l.Value</dd>
}
</dl>
}
</section>
<section class="panel" id="telegram">
<h2>Telegram e segnale di vita</h2>
<p class="hint">Su Telegram ti scrivo il riepilogo del mattino, ogni acquisto e vendita con il risultato, gli avvisi di rischio, i blocchi e gli errori. Dalla stessa chat accetto /stato, /posizioni, /oggi, /pausa, /riprendi, /ferma (con conferma) e /aiuto; da altre chat nessun comando. Soldi veri, limiti e chiavi non si cambiano da Telegram.</p>
<dl class="facts">
<dt>Token del bot</dt><dd>@KeyState(_telegramToken)</dd>
<dt>Chat</dt><dd>@KeyState(_telegramChat)</dd>
<dt>Ultimo messaggio partito</dt><dd>@(_queue?.LastSentAt is { } sent ? Format.ShortDate(sent, Shell.Now, Shell.Zone) : "Nessuno")</dd>
@if (_queue is { } q && (q.Pending > 0 || q.Retrying > 0 || q.GaveUp > 0))
{
<dt>In attesa</dt><dd>@q.Pending in coda@(q.Retrying > 0 ? $", {q.Retrying} da ritentare" : "")@(q.GaveUp > 0 ? $", {q.GaveUp} non partiti" : "")</dd>
}
@if (_queue?.LastError is { } lastError)
{
<dt>Ultimo errore</dt><dd class="loss">@lastError</dd>
}
</dl>
@if (Shell.IsDemo)
{
<p class="hint">Nella modalità dimostrativa Telegram non si usa.</p>
}
else if (_telegramToken.FromEnvironment || _telegramChat.FromEnvironment)
{
<p class="hint">Token e chat arrivano dalle variabili d'ambiente: si cambiano nella configurazione del container.</p>
}
else
{
<form method="post" action="/azioni/telegram" class="form" autocomplete="off">
<AntiforgeryToken />
<label for="tg-token">Token del bot (da @@BotFather)</label>
<input id="tg-token" name="token" type="password" autocomplete="off" spellcheck="false" maxlength="200" required />
<label for="tg-chat">Numero della chat</label>
<input id="tg-chat" name="chat" inputmode="numeric" autocomplete="off" spellcheck="false" maxlength="40" required />
<button class="btn" type="submit">Prova e salva</button>
</form>
<p class="hint">Prima di salvarli mando un messaggio di prova: se non parte, non salvo nulla. Il token non lo rimostro mai.</p>
}
@if (!Shell.IsDemo && _telegramToken.IsSet && _telegramChat.IsSet)
{
<form method="post" action="/azioni/telegram/prova" class="form-inline"><AntiforgeryToken /><button class="btn btn-small" type="submit">Manda un messaggio di prova</button></form>
@if (!_telegramToken.FromEnvironment && !_telegramChat.FromEnvironment)
{
<form method="post" action="/azioni/telegram/togli" class="form-inline"><AntiforgeryToken /><button class="btn btn-small" type="submit" data-confirm="Tolgo token e chat? Non ti scrivo più su Telegram finché non li rimetti.">Togli Telegram</button></form>
}
}
<h3>Stato periodico</h3>
<form method="post" action="/azioni/impostazioni/@SettingsStore.TelegramStatus.Key" class="form-inline">
<AntiforgeryToken />
<input type="hidden" name="ritorno" value="/impostazioni#telegram" />
<label for="tg-status">@SettingsStore.TelegramStatus.Label</label>
<select id="tg-status" name="valore" disabled="@(_telegramStatus.ReadOnly || Shell.IsDemo)" data-autosubmit>
@foreach (var value in SettingsStore.TelegramStatus.Allowed!)
{
<option value="@value" selected="@(value == _telegramStatus.Value)">@(value switch { "hourly" => "Ogni ora", "daily" => "Ogni sera alle 20:00", _ => "Mai" })</option>
}
</select>
<button class="btn btn-small" type="submit">Salva</button>
</form>
<h3>Segnale di vita</h3>
<p class="hint">Facoltativo. Se metti l'indirizzo di un servizio di controllo esterno (per esempio healthchecks.io), lo chiamo ogni minuto. Quando le chiamate si fermano è quel servizio ad avvisarti: è l'unico modo per sapere che il bot è spento quando è spento.</p>
<dl class="facts">
<dt>Indirizzo</dt><dd>@(Host.Services!.Telegram?.ConfiguredHeartbeat is not null ? "Dalla configurazione d'avvio (ANUBIS_HEARTBEAT_URL)" : KeyState(_heartbeat))</dd>
@if (Host.Services.Telegram is { } tg)
{
<dt>Ultima chiamata riuscita</dt><dd>@(tg.Heartbeat.LastOk is { } beat ? Format.ShortDate(beat, Shell.Now, Shell.Zone) : "Nessuna da quando il bot è partito")</dd>
@if (tg.Heartbeat.LastError is { } beatError)
{
<dt>Ultimo errore</dt><dd class="loss">@beatError</dd>
}
}
</dl>
@if (!Shell.IsDemo && Host.Services.Telegram is { ConfiguredHeartbeat: null })
{
<form method="post" action="/azioni/segnale-di-vita" class="form-inline" autocomplete="off">
<AntiforgeryToken />
<label for="hb-url">Indirizzo</label>
<input id="hb-url" name="indirizzo" type="url" autocomplete="off" spellcheck="false" maxlength="500" placeholder="https://hc-ping.com/…" required />
<button class="btn btn-small" type="submit">Prova e salva</button>
</form>
@if (_heartbeat.IsSet)
{
<form method="post" action="/azioni/segnale-di-vita/togli" class="form-inline"><AntiforgeryToken /><button class="btn btn-small" type="submit">Togli il segnale di vita</button></form>
}
}
</section>
<section class="panel" id="dati">
<h2>Dati</h2>
@if (Host.Services!.MarketData is not { } market)
{
@@ -302,11 +446,22 @@
private sealed record BrokerSetup(string Broker, string Title, string Short, IReadOnlyList<BrokerKey> Keys, AccountState? State, EntrustedCapital? Entrusted,
IReadOnlyList<StrategyCapital> Strategies);
private sealed record LimitField(string Key, string Label, string Value, bool ReadOnly);
private readonly List<LimitField> _limits = [];
private bool _anyLive;
private string _pdt = "off";
private string _scenarios = "";
private IReadOnlyList<Anubis.Data.SourceState> _sources = [];
private SettingValue _meetings = new(SettingsStore.CentralBankMeetings.Key, "", false, null);
private List<SettingChange> _history = [];
private IReadOnlyList<BackupInfo> _backups = [];
private bool _hasSecondFactor;
private SecretStatus _telegramToken = new(Anubis.Notifications.TelegramSettings.TokenSecret, false, false, null);
private SecretStatus _telegramChat = new(Anubis.Notifications.TelegramSettings.ChatSecret, false, false, null);
private SecretStatus _heartbeat = new(TelegramServices.HeartbeatSecret, false, false, null);
private SettingValue _telegramStatus = new(SettingsStore.TelegramStatus.Key, "never", false, null);
private Anubis.Notifications.NotificationSummary? _queue;
private string? _draft;
private string? _draftUri;
@@ -328,7 +483,22 @@
_history = await services.Settings.HistoryAsync(null, 50, ct);
_meetings = await services.Settings.GetAsync(SettingsStore.CentralBankMeetings.Key, ct);
_sources = await new Anubis.Data.SourceStateStore(services.Db, Clock).ListAsync(ct);
foreach (var definition in Anubis.Risk.RiskSettings.Limits)
{
var value = await services.Settings.GetAsync(definition.Key, ct);
_limits.Add(new LimitField(definition.Key, definition.Label, value.Value, value.ReadOnly));
}
_pdt = (await services.Settings.GetAsync(Anubis.Risk.RiskSettings.PdtGuard.Key, ct)).Value;
_scenarios = (await services.Settings.GetAsync(Anubis.Risk.Scenarios.Setting.Key, ct)).Value;
_anyLive = await (services.Trading?.Stages ?? new StageBook(services.Db, Clock, TradingConsent.None, (_, _, _) => Task.FromResult(false))).AnyLiveAsync(null, ct);
_backups = services.Backups.List();
_telegramToken = await services.Secrets.StatusAsync(Anubis.Notifications.TelegramSettings.TokenSecret, ct);
_telegramChat = await services.Secrets.StatusAsync(Anubis.Notifications.TelegramSettings.ChatSecret, ct);
_heartbeat = await services.Secrets.StatusAsync(TelegramServices.HeartbeatSecret, ct);
_telegramStatus = await services.Settings.GetAsync(SettingsStore.TelegramStatus.Key, ct);
_queue = services.Telegram is { } telegram ? await telegram.Sender.SummaryAsync(ct) : null;
var ledger = new Ledger(services.Db, Clock);
var capital = new Capital(services.Db, Clock, ledger, new FxStore(services.Db, Clock));
var reconciler = new Reconciler(services.Db, Clock, ledger);
@@ -372,6 +542,19 @@
"importo" => "Scrivi un importo in euro, per esempio 4000 o 1.500,50.",
"cambio" => "Manca ancora il cambio euro/dollaro della BCE: lo scarico ogni giorno, riprova più tardi.",
"capitale" => "Una strategia ha investito più di quanto le resterebbe: potrò ridurre il capitale quando avrà venduto.",
"limite" => "Un limite non è valido: controlla che ogni valore sia un numero dentro il suo intervallo.",
"ordine-limiti" => "Il dimezzamento sul calo dal massimo deve venire prima dell'interruttore, e la perdita di un giorno non può superare quella di una settimana.",
"frase-rischio" => $"Una strategia usa soldi veri: per aggiungere rischio scrivi la frase «{StageBook.RaisePhrase}». Non ho cambiato nulla.",
"password-rischio" => "Password o codice non corretti. Dopo cinque tentativi sbagliati devi aspettare 15 minuti. Non ho cambiato nulla.",
"retrocessione-rischio" => "Una strategia con soldi veri ha una condizione di retrocessione in corso: finché dura non si aggiunge rischio. Non ho cambiato nulla.",
"telegram-rifiutato" => "Telegram non ha accettato token o chat, o non ha risposto: il messaggio di prova non è partito e non ho salvato nulla. Controlla il token di @BotFather e il numero della chat.",
"telegram-prova" => "Il messaggio di prova non è partito: controlla token e chat, o riprova tra poco.",
"telegram-vuoto" => "Servono il token del bot e il numero della chat.",
"telegram-ambiente" => "Token e chat arrivano dalle variabili d'ambiente: si cambiano nella configurazione del container.",
"segnale" => "L'indirizzo non va bene: serve https:// (http solo nella rete di casa).",
"segnale-fallito" => "Il servizio di controllo non ha risposto bene alla prova: non ho salvato l'indirizzo.",
"scenari" => "Gli scenari non sono validi: una riga ciascuno, con descrizione e tre percentuali separate da punto e virgola.",
_ => PasswordMessages.For(Errore),
};
@@ -1,5 +1,239 @@
@inherits LoadedComponent
@page "/laboratorio"
@inject AnubisHost Host
@inject ShellData Shell
@inject IClock Clock
@using Anubis.Backtest
@using Anubis.Data
@using Anubis.Strategies
<PageTitle>Laboratorio · Anubis</PageTitle>
<Pending Title="Nessun test eseguito.">Il laboratorio dei test sul passato arriva con il motore dei backtest.</Pending>
@if (_running)
{
<HeadContent><meta http-equiv="refresh" content="5" /></HeadContent>
}
@if (_message is { } message)
{
<p class="notice notice-loss" role="alert"><Icon Name="alert" Size="16" />@message</p>
}
@if (_lab is null)
{
<Pending Title="Il laboratorio non esegue test in questa modalità.">Nella modalità dimostrativa i test sul passato non partono: avvia Anubis normalmente per provare una strategia.</Pending>
}
<section class="panel" aria-labelledby="prove">
<div class="panel-head">
<h2 id="prove">Test eseguiti</h2>
@if (_trials.Count(t => t.Status == TrialStatuses.Done) >= 2)
{
<form method="get" action="/laboratorio/confronto" class="form-inline">
<label for="a" class="sr-only">Primo test</label>
<select id="a" name="a">
@foreach (var t in _trials.Where(t => t.Status == TrialStatuses.Done))
{
<option value="@t.Id">@Title(t)</option>
}
</select>
<label for="b" class="sr-only">Secondo test</label>
<select id="b" name="b">
@foreach (var (t, i) in _trials.Where(t => t.Status == TrialStatuses.Done).Select((t, i) => (t, i)))
{
<option value="@t.Id" selected="@(i == 1)">@Title(t)</option>
}
</select>
<button class="btn btn-small" type="submit">Confronta</button>
</form>
}
</div>
@if (_trials.Count == 0)
{
<div class="empty">
<p class="empty-title">Nessun test eseguito.</p>
<p>Ogni test prova una strategia su un periodo passato, con tutte le combinazioni della sua griglia di parametri, costi realistici e poi doppi e tripli, e le convalide del protocollo anti-illusioni. Il piano si scrive prima di vedere i risultati e resta nel registro.</p>
</div>
}
else
{
<div class="table-wrap">
<table class="table">
<thead>
<tr>
<th scope="col">Pianificato</th>
<th scope="col">Strategia</th>
<th scope="col">Dati</th>
<th scope="col" class="num">Varianti</th>
<th scope="col">Stato</th>
<th scope="col">Esito</th>
<th scope="col" class="num">Risultato netto</th>
<th scope="col" class="num">Sharpe</th>
<th scope="col" class="num">Calo massimo</th>
</tr>
</thead>
<tbody>
@foreach (var t in _trials)
{
var s = t.Summary;
<tr>
<td class="nowrap"><a href="/laboratorio/@t.Id">@Format.ShortDate(t.CreatedAt, Shell.Now, Shell.Zone)</a></td>
<td>@Name(t.Strategy)</td>
<td>@LabText.Symbols(t.Plan.Data.Symbols)<br /><span class="muted">@LabText.Period(t.Plan.Data.From, t.Plan.Data.To)</span></td>
<td class="num">@t.Variants</td>
<td>@LabText.Status(t, _lab?.Progress(t.Id))</td>
<td>
@if (t.Verdict is { } verdict)
{
<span class="tag @(verdict == TrialVerdicts.Passed ? "gain" : "loss")">@LabText.Verdict(verdict)</span>
}
else
{
@LabText.Missing
}
</td>
<td class="num">@(s is null ? LabText.Missing : LabText.Money(s.NetResult, s.Currency, withSign: true))</td>
<td class="num">@(s is null ? LabText.Missing : LabText.Number(s.Sharpe))</td>
<td class="num">@(s is null ? LabText.Missing : LabText.Percent(s.MaxDrawdown))</td>
</tr>
}
</tbody>
</table>
</div>
}
</section>
@if (_lab is not null)
{
var family = _family;
<section class="panel" id="nuovo" aria-labelledby="nuovo-titolo">
<div class="panel-head">
<h2 id="nuovo-titolo">Nuovo test</h2>
<nav class="segmented" aria-label="Strategia">
@foreach (var f in StrategyCatalog.All)
{
<a href="/laboratorio?strategia=@f.Id#nuovo" class="@(f.Id == family.Id ? "on" : null)" aria-current="@(f.Id == family.Id ? "true" : null)">@f.Name</a>
}
</nav>
</div>
<p>@family.Description</p>
<form method="post" action="/azioni/laboratorio" class="form">
<AntiforgeryToken />
<input type="hidden" name="strategia" value="@family.Id" />
<label for="origine">Dati</label>
<select id="origine" name="origine">
<option value="@TrialSources.Database">Barre giornaliere raccolte da Anubis</option>
<option value="@TrialSources.Research">Storico di ricerca (archivi Binance dal 2017)</option>
</select>
<label for="mercato">Mercato, per le barre raccolte</label>
<select id="mercato" name="mercato">
<option value="@Venues.Okx">Crypto su OKX, in euro</option>
<option value="@Venues.Alpaca">Azioni USA su Alpaca, in dollari</option>
</select>
<label for="strumenti">Strumenti, separati da virgole</label>
<input id="strumenti" name="strumenti" autocomplete="off" spellcheck="false" maxlength="400" placeholder="BTC-EUR, ETH-EUR" required />
<p class="hint">Simboli del mercato scelto: BTC-EUR su OKX, AAPL su Alpaca, BTCUSDT nello storico di ricerca. @Research()</p>
<div class="row">
<div>
<label for="dal">Dal</label>
<input id="dal" name="dal" type="date" value="@_from.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)" required />
</div>
<div>
<label for="al">Al</label>
<input id="al" name="al" type="date" value="@_to.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture)" required />
</div>
</div>
<label for="capitale">Capitale iniziale, nella valuta degli strumenti</label>
<input id="capitale" name="capitale" inputmode="decimal" autocomplete="off" value="10.000" required />
@if (family.Parameters.Count > 0)
{
<fieldset>
<legend>Griglia dei parametri</legend>
@foreach (var p in family.Parameters)
{
<label for="griglia-@p.Name">@p.Label</label>
<input id="griglia-@p.Name" name="griglia_@p.Name" autocomplete="off" spellcheck="false" maxlength="200" value="@string.Join(", ", p.Grid.Select(v => v.ToString(CultureInfo.InvariantCulture)))" />
<p class="hint">Da @p.Min.ToString(CultureInfo.InvariantCulture) a @p.Max.ToString(CultureInfo.InvariantCulture), valori separati da virgole; ogni combinazione è una variante.</p>
}
</fieldset>
}
<p class="hint">Soglie del protocollo, fissate nel piano: risultato netto positivo anche con costi doppi, Deflated Sharpe Ratio almeno 0,90, probabilità di sovra-adattamento non oltre 0,40 con più varianti, calo massimo non oltre il 25%.</p>
<button class="btn btn-primary" type="submit">Scrivi il piano e avvia il test</button>
</form>
</section>
}
<section class="panel" aria-labelledby="migliorie">
<h2 id="migliorie">Migliorie in osservazione</h2>
<div class="empty">
<p class="empty-title">Nessuna miglioria in osservazione.</p>
<p>Le modifiche che potrebbero migliorare il risultato o ridurre i costi si calcolano in ombra accanto alle regole in vigore e compaiono qui con l'effetto che avrebbero avuto. Arrivano con la Fase 9.</p>
</div>
</section>
<section class="panel" aria-labelledby="stime">
<h2 id="stime">Stime di costo da confermare</h2>
<div class="empty">
<p class="empty-title">Nessuna stima da confermare.</p>
<p>Quando le esecuzioni con soldi veri diranno che lo scostamento è più basso di quello previsto, la nuova stima aspetterà qui la tua conferma. Arrivano con la Fase 8.</p>
</div>
</section>
@code {
private Laboratory? _lab;
private IReadOnlyList<Trial> _trials = [];
private IStrategyFamily _family = StrategyCatalog.All[0];
private bool _running;
private string? _message;
private DateOnly _from;
private DateOnly _to;
private IReadOnlyList<string> _research = [];
[CascadingParameter]
public HttpContext? HttpContext { get; set; }
[SupplyParameterFromQuery(Name = "strategia")]
public string? Strategia { get; set; }
[SupplyParameterFromQuery(Name = "errore")]
public string? Errore { get; set; }
protected override async Task LoadAsync()
{
var ct = HttpContext?.RequestAborted ?? CancellationToken.None;
var services = Host.Services!;
_lab = services.Lab;
_trials = await new TrialStore(services.Db, Clock).ListAsync(100, ct);
_running = _trials.Any(t => !t.IsFinished);
_family = StrategyCatalog.Find(Strategia ?? "") ?? StrategyCatalog.All[0];
var today = DateOnly.FromDateTime(Clock.UtcNow.UtcDateTime);
_to = today.AddDays(-1);
_from = _to.AddYears(-3);
_research = new ResearchStore(Host.Config.Data.Research).Series()
.Where(s => s.Timeframe == Timeframes.Day)
.Select(s => s.Symbol)
.Order(StringComparer.Ordinal)
.ToList();
_message = Errore switch
{
null => null,
"motore" => "Il laboratorio non è attivo in questo momento: riprova tra poco.",
TrialRequestErrors.Strategy => "Strategia sconosciuta: scegline una dall'elenco.",
TrialRequestErrors.Source => "Scegli da dove prendere i dati e, per le barre raccolte, il mercato.",
TrialRequestErrors.Symbols => $"Indica da 1 a {Laboratory.MaxSymbols} strumenti, separati da virgole.",
TrialRequestErrors.Dates => "Controlla le date: l'inizio deve venire prima della fine.",
TrialRequestErrors.Capital => "Il capitale iniziale deve essere un importo tra 1 e 10.000.000.",
TrialRequestErrors.Grid => "Un valore della griglia non si legge o è fuori dall'intervallo del parametro: usa numeri separati da virgole, con il punto per i decimali.",
TrialRequestErrors.Combinations => $"La griglia non dà combinazioni valide, oppure ne dà più di {Laboratory.MaxVariants}: restringila.",
_ => "Non è stato possibile avviare il test: riprova.",
};
}
private string Research() => _research.Count == 0
? "Lo storico di ricerca è vuoto: lo riempie la raccolta dei dati."
: "Nello storico di ricerca: " + LabText.Symbols(_research) + ".";
private static string Name(string strategy) => StrategyCatalog.Find(strategy)?.Name ?? strategy;
private string Title(Trial t) => $"{Name(t.Strategy)}, {LabText.Symbols(t.Plan.Data.Symbols)}, {Format.ShortDate(t.CreatedAt, Shell.Now, Shell.Zone)}";
}
@@ -0,0 +1,106 @@
@inherits LoadedComponent
@page "/laboratorio/confronto"
@inject AnubisHost Host
@inject ShellData Shell
@inject IClock Clock
@using Anubis.Backtest
@using Anubis.Strategies
<PageTitle>Confronto tra test · Anubis</PageTitle>
<p><a href="/laboratorio">← Laboratorio</a></p>
@if (_a is null || _b is null)
{
<Pending Title="Servono due test completati.">Scegli due test conclusi dall'elenco del <a href="/laboratorio">Laboratorio</a>.</Pending>
return;
}
@{
var a = _a.Value;
var b = _b.Value;
var ra = a.Report;
var rb = b.Report;
}
<section class="panel" aria-labelledby="confronto">
<h2 id="confronto">Confronto tra due test</h2>
@if (ra.DataFingerprint != rb.DataFingerprint)
{
<p class="notice notice-attention" role="note"><Icon Name="info" Size="16" />I due test hanno letto dati diversi (strumenti, periodo o valuta): il confronto è indicativo.</p>
}
else
{
<p class="notice notice-gain" role="note"><Icon Name="check" Size="16" />Stessi dati per entrambi: le differenze vengono dalla strategia, dalla griglia o dalle soglie.</p>
}
<div class="table-wrap">
<table class="table">
<thead>
<tr>
<th scope="col"></th>
<th scope="col"><a href="/laboratorio/@a.Trial.Id">@Name(a)</a></th>
<th scope="col"><a href="/laboratorio/@b.Trial.Id">@Name(b)</a></th>
</tr>
</thead>
<tbody>
@Row("Esito", x => LabText.Verdict(x.Trial.Verdict))
@Row("Dati", x => LabText.Symbols(x.Trial.Plan.Data.Symbols) + ", " + LabText.Period(x.Trial.Plan.Data.From, x.Trial.Plan.Data.To))
@Row("Variante scelta", x => LabText.Parameters(x.Report.Chosen) + $" (di {x.Trial.Variants})")
@Row("Risultato netto", x => LabText.Money(x.Report.Metrics.NetResult, x.Report.Currency, withSign: true))
@Row("Con costi doppi", x => x.Report.Costs.FirstOrDefault(k => k.Multiplier == 2m) is { } k ? LabText.Money(k.NetResult, x.Report.Currency, withSign: true) : LabText.Missing)
@Row("Rendimento annuo", x => LabText.Percent(x.Report.Metrics.AnnualReturn, 1, true))
@Row("Comprare e tenere, annuo", x => LabText.Percent(x.Report.Benchmark.AnnualReturn, 1, true))
@Row("Volatilità annua", x => LabText.Percent(x.Report.Metrics.Volatility))
@Row("Sharpe annuo", x => LabText.Number(x.Report.Metrics.Sharpe))
@Row("Calo massimo", x => LabText.Percent(x.Report.Metrics.MaxDrawdown))
@Row("Operazioni chiuse", x => x.Report.Metrics.Trades.ToString(CultureInfo.InvariantCulture))
@Row("Operazioni in guadagno", x => x.Report.Metrics.Trades == 0 ? LabText.Missing : LabText.Percent(x.Report.Metrics.WinRate, 0))
@Row("Rapporto guadagni/perdite", x => LabText.Number(x.Report.Metrics.ProfitFactor))
@Row("Deflated Sharpe Ratio", x => LabText.Number(x.Report.DeflatedSharpe.Value))
@Row("Probabilità di sovra-adattamento", x => x.Report.Overfitting is { } p ? LabText.Number(p.Probability) : "non si applica")
@Row("Finestre che avanzano, rendimento", x => x.Report.WalkForward is { } w ? LabText.Percent(w.Return, 1, true) : LabText.Missing)
@Row("Decisioni cambiate dai dati futuri", x => x.Report.LookAhead.Changed.ToString(CultureInfo.InvariantCulture))
</tbody>
</table>
</div>
</section>
@code {
private (Trial Trial, BacktestReport Report)? _a;
private (Trial Trial, BacktestReport Report)? _b;
[CascadingParameter]
public HttpContext? HttpContext { get; set; }
// Text, not Guid: a malformed value must give "servono due test", not an error page.
[SupplyParameterFromQuery(Name = "a")]
public string? A { get; set; }
[SupplyParameterFromQuery(Name = "b")]
public string? B { get; set; }
protected override async Task LoadAsync()
{
var ct = HttpContext?.RequestAborted ?? CancellationToken.None;
var store = new TrialStore(Host.Services!.Db, Clock);
_a = await ReadAsync(store, A, ct);
_b = await ReadAsync(store, B, ct);
}
private static async Task<(Trial, BacktestReport)?> ReadAsync(TrialStore store, string? id, CancellationToken ct)
{
if (!Guid.TryParse(id, out var key) || await store.GetAsync(key, ct) is not { } trial || await store.ReportJsonAsync(key, ct) is not { } json)
{
return null;
}
return (trial, BacktestReport.FromJson(json));
}
private string Name((Trial Trial, BacktestReport Report) x) =>
$"{StrategyCatalog.Find(x.Trial.Strategy)?.Name ?? x.Trial.Strategy}, {Format.ShortDate(x.Trial.CreatedAt, Shell.Now, Shell.Zone)}";
private RenderFragment Row(string label, Func<(Trial Trial, BacktestReport Report), string> value) => __builder =>
{
<tr><th scope="row">@label</th><td>@value(_a!.Value)</td><td>@value(_b!.Value)</td></tr>
};
}
@@ -0,0 +1,336 @@
@inherits LoadedComponent
@page "/laboratorio/{Id:guid}"
@inject AnubisHost Host
@inject ShellData Shell
@inject IClock Clock
@using Anubis.Backtest
@using Anubis.Strategies
<PageTitle>Test del laboratorio · Anubis</PageTitle>
@if (_trial is null)
{
<Pending Title="Test non trovato.">Torna al <a href="/laboratorio">Laboratorio</a> per l'elenco dei test.</Pending>
return;
}
@{
var t = _trial;
var plan = t.Plan;
var r = _report;
}
@if (!t.IsFinished)
{
<HeadContent><meta http-equiv="refresh" content="3" /></HeadContent>
}
<p><a href="/laboratorio">← Laboratorio</a></p>
<section class="panel" aria-labelledby="titolo">
<div class="panel-head">
<h2 id="titolo">@Name(t.Strategy) su @LabText.Symbols(plan.Data.Symbols)</h2>
@if (t.Verdict is { } verdict)
{
<span class="badge @(verdict == TrialVerdicts.Passed ? "badge-gain" : "badge-loss")"><Icon Name="@(verdict == TrialVerdicts.Passed ? "check" : "alert")" Size="18" />@LabText.Verdict(verdict)</span>
}
else
{
<span class="tag">@LabText.Status(t, _progress)</span>
}
</div>
<dl class="facts">
<dt>Dati</dt>
<dd>@LabText.Source(plan.Data), @LabText.Period(plan.Data.From, plan.Data.To)</dd>
<dt>Varianti provate</dt>
<dd>@t.Variants (@string.Join("; ", plan.Grid.Select(g => g.Key + " " + string.Join(", ", g.Value.Select(v => v.ToString(CultureInfo.InvariantCulture))))))</dd>
<dt>Capitale iniziale</dt>
<dd>@(r is null ? Format.Number(plan.InitialCapital, 2) : Format.Money(plan.InitialCapital, r.Currency))</dd>
<dt>Piano scritto</dt>
<dd>@Format.LongDateTime(t.CreatedAt, Shell.Zone), prima dei risultati. Impronta @Short(t.PlanHash)</dd>
@if (t.FinishedAt is { } finished)
{
<dt>Concluso</dt>
<dd>@Format.LongDateTime(finished, Shell.Zone)@(r is null ? "" : $", in {LabText.Number(r.Seconds, 1)} secondi")</dd>
}
</dl>
@if (t.Status == TrialStatuses.Failed)
{
<p class="notice notice-loss" role="alert"><Icon Name="alert" Size="16" />Il test non è riuscito: @t.Error</p>
}
else if (!t.IsFinished)
{
<p class="notice notice-attention" role="status"><Icon Name="clock" Size="16" />@(t.Status == TrialStatuses.Planned ? "In coda: parte appena finisce il test precedente." : "In corso" + (_progress is null ? "." : ": " + _progress + ".")) La pagina si aggiorna da sola.</p>
}
</section>
@if (r is not null)
{
var c = r.Currency;
var m = r.Metrics;
var b = r.Benchmark;
<section class="panel" aria-labelledby="esito">
<h2 id="esito">Esito del protocollo</h2>
<p>Variante scelta: <strong>@LabText.Parameters(r.Chosen)</strong>, la migliore zona della griglia e non il singolo punto migliore.</p>
<div class="table-wrap">
<table class="table">
<thead><tr><th scope="col">Controllo</th><th scope="col" class="num">Valore</th><th scope="col" class="num">Soglia</th><th scope="col">Esito</th></tr></thead>
<tbody>
@foreach (var check in r.Checks)
{
<tr>
<td>@check.Name</td>
<td class="num">@LabText.CheckValue(check, c)</td>
<td class="num">@LabText.CheckThreshold(check, c)</td>
<td>
@switch (check.Passed)
{
case true:
<span class="tag gain">Superato</span>
break;
case false:
<span class="tag loss">Non superato</span>
break;
default:
<span class="tag">Non si applica</span>
break;
}
</td>
</tr>
}
</tbody>
</table>
</div>
</section>
<section class="panel" aria-labelledby="risultato">
<h2 id="risultato">Risultato della variante scelta</h2>
<div class="table-wrap">
<table class="table">
<thead><tr><th scope="col"></th><th scope="col" class="num">Variante scelta</th><th scope="col" class="num">Comprare e tenere</th></tr></thead>
<tbody>
<tr><th scope="row">Risultato netto</th><td class="num">@LabText.Money(m.NetResult, c, withSign: true)</td><td class="num">@LabText.Money(b.NetResult, c, withSign: true)</td></tr>
<tr><th scope="row">Rendimento totale</th><td class="num">@LabText.Percent(m.TotalReturn, 1, true)</td><td class="num">@LabText.Percent(b.TotalReturn, 1, true)</td></tr>
<tr><th scope="row">Rendimento annuo</th><td class="num">@LabText.Percent(m.AnnualReturn, 1, true)</td><td class="num">@LabText.Percent(b.AnnualReturn, 1, true)</td></tr>
<tr><th scope="row">Volatilità annua</th><td class="num">@LabText.Percent(m.Volatility)</td><td class="num">@LabText.Percent(b.Volatility)</td></tr>
<tr><th scope="row">Sharpe annuo</th><td class="num">@LabText.Number(m.Sharpe)</td><td class="num">@LabText.Number(b.Sharpe)</td></tr>
<tr><th scope="row">Calo massimo</th><td class="num">@LabText.Percent(m.MaxDrawdown)</td><td class="num">@LabText.Percent(b.MaxDrawdown)</td></tr>
<tr><th scope="row">Operazioni chiuse</th><td class="num">@m.Trades</td><td class="num">@LabText.Missing</td></tr>
<tr><th scope="row">Operazioni in guadagno</th><td class="num">@(m.Trades == 0 ? LabText.Missing : LabText.Percent(m.WinRate, 0))</td><td class="num">@LabText.Missing</td></tr>
<tr><th scope="row">Rapporto guadagni/perdite</th><td class="num">@(m.ProfitFactor is { } pf ? LabText.Number(pf) : m.Trades > 0 ? "nessuna perdita" : LabText.Missing)</td><td class="num">@LabText.Missing</td></tr>
<tr><th scope="row">Guadagno e perdita medi</th><td class="num">@LabText.Money(m.AverageWin, c, true) / @LabText.Money(m.AverageLoss, c, true)</td><td class="num">@LabText.Missing</td></tr>
<tr><th scope="row">Commissioni</th><td class="num">@LabText.Money(m.Fees, c)</td><td class="num">@LabText.Money(b.Fees, c)</td></tr>
<tr><th scope="row">Tempo investito, in media</th><td class="num">@LabText.Percent(m.Exposure, 0)</td><td class="num">@LabText.Percent(b.Exposure, 0)</td></tr>
</tbody>
</table>
</div>
<EquityChart Points="_curve" Currency="@c" Native="true" Label="Patrimonio della variante scelta e di comprare e tenere" EquityLabel="Variante scelta" BenchmarkLabel="Comprare e tenere gli stessi strumenti" EquityColumn="Variante scelta" BenchmarkColumn="Comprare e tenere" />
</section>
<section class="panel" aria-labelledby="anni">
<h2 id="anni">Risultato per anno</h2>
<div class="table-wrap">
<table class="table">
<thead><tr><th scope="col">Anno</th><th scope="col" class="num">Risultato</th><th scope="col" class="num">Rendimento</th><th scope="col" class="num">Comprare e tenere</th></tr></thead>
<tbody>
@foreach (var y in m.Years)
{
var bh = b.Years.FirstOrDefault(x => x.Year == y.Year);
<tr><td>@y.Year</td><td class="num">@LabText.Money(y.Result, c, true)</td><td class="num">@LabText.Percent(y.Return, 1, true)</td><td class="num">@(bh is null ? LabText.Missing : LabText.Percent(bh.Return, 1, true))</td></tr>
}
</tbody>
</table>
</div>
</section>
<section class="panel" aria-labelledby="costi">
<h2 id="costi">Sensibilità ai costi</h2>
<p>Commissioni e scostamento moltiplicati: una strategia che muore con i costi doppi non passa.</p>
<div class="table-wrap">
<table class="table">
<thead><tr><th scope="col">Costi</th><th scope="col" class="num">Risultato netto</th><th scope="col" class="num">Rendimento</th><th scope="col" class="num">Sharpe</th><th scope="col" class="num">Calo massimo</th><th scope="col" class="num">Commissioni</th></tr></thead>
<tbody>
@foreach (var k in r.Costs)
{
<tr><td>@(k.Multiplier == 1m ? "realistici" : $"× {Format.Number(k.Multiplier)}")</td><td class="num">@LabText.Money(k.NetResult, c, true)</td><td class="num">@LabText.Percent(k.TotalReturn, 1, true)</td><td class="num">@LabText.Number(k.Sharpe)</td><td class="num">@LabText.Percent(k.MaxDrawdown)</td><td class="num">@LabText.Money(k.Fees, c)</td></tr>
}
</tbody>
</table>
</div>
</section>
<section class="panel" aria-labelledby="convalide">
<h2 id="convalide">Convalide</h2>
<h3>Sharpe corretto per il numero di prove</h3>
@{ var d = r.DeflatedSharpe; var annual = Math.Sqrt(r.PeriodsPerYear); }
<p>Deflated Sharpe Ratio <strong>@LabText.Number(d.Value)</strong>: la probabilità che lo Sharpe della variante scelta (@LabText.Number(d.Sharpe * annual) all'anno) superi quello che il migliore di @d.Trials tentativi mostrerebbe per caso (@LabText.Number(d.Benchmark * annual) all'anno). Contati tutti i test di questa strategia: @r.TrialsCounted varianti. Su @d.Observations sedute, asimmetria @LabText.Number(d.Skewness), curtosi @LabText.Number(d.Kurtosis).</p>
<h3>Probabilità di sovra-adattamento</h3>
@if (r.Overfitting is { } pbo)
{
<p>PBO <strong>@LabText.Number(pbo.Probability)</strong>: in @Format.Number(pbo.Combinations) divisioni del periodo in due metà, la quota di volte in cui la variante migliore in una metà è finita nella metà peggiore dell'altra. Con @pbo.Variants varianti e @pbo.Groups blocchi.</p>
}
else
{
<p>Con una sola variante non c'è nulla da scegliere, quindi nulla da sovra-adattare: il controllo non si applica.</p>
}
<h3>Finestre che avanzano</h3>
@if (r.WalkForward is { } wf)
{
<p>Ogni finestra sceglie la variante sui dati precedenti e la giudica su quelli successivi, mai visti. Rendimento fuori campione complessivo <strong>@LabText.Percent(wf.Return, 1, true)</strong>, Sharpe @LabText.Number(wf.Sharpe * annual) all'anno; @wf.Purge sedute tolte tra addestramento e prova.</p>
<div class="table-wrap">
<table class="table">
<thead><tr><th scope="col">Finestra</th><th scope="col">Scelta su</th><th scope="col">Provata su</th><th scope="col">Variante</th><th scope="col" class="num">Rendimento</th><th scope="col" class="num">Sharpe</th></tr></thead>
<tbody>
@foreach (var w in wf.Windows)
{
<tr><td>@w.Window</td><td class="nowrap">@LabText.Period(w.TrainFrom, w.TrainTo)</td><td class="nowrap">@LabText.Period(w.TestFrom, w.TestTo)</td><td>@LabText.Parameters(w.Chosen)</td><td class="num">@LabText.Percent(w.Return, 1, true)</td><td class="num">@LabText.Number(w.Sharpe * annual)</td></tr>
}
</tbody>
</table>
</div>
}
else
{
<p>Periodo troppo breve per le finestre.</p>
}
<h3>K-fold purgato combinatorio</h3>
@if (r.Combinatorial is { } cv)
{
<p>@cv.Groups blocchi, @cv.TestGroups alla volta in prova: @cv.Splits divisioni e @cv.PathSharpes.Count percorsi completi fuori campione, con @cv.Purge sedute tolte prima di ogni blocco di prova e @cv.Embargo dopo. Sharpe medio dei percorsi @LabText.Number(cv.MeanSharpe * annual) all'anno, il peggiore @LabText.Number(cv.WorstSharpe * annual); percorsi in positivo @LabText.Percent(cv.PositiveShare, 0).</p>
}
else
{
<p>Periodo troppo breve per i blocchi.</p>
}
<h3>Nessuno sguardo in avanti</h3>
<p>
Il test è stato rifatto con i dati tagliati al @string.Join(", ", r.LookAhead.Cutoffs.Select(x => LabText.Day(x))): delle @r.LookAhead.Decisions decisioni confrontate,
@(r.LookAhead.Changed == 0 ? "nessuna è cambiata." : $"{r.LookAhead.Changed} sono cambiate: la strategia guarda dati che al momento della decisione non esistevano.")
</p>
@if (r.LookAhead.Examples.Count > 0)
{
<ul>
@foreach (var e in r.LookAhead.Examples)
{
<li><code>@e</code></li>
}
</ul>
}
</section>
<section class="panel" aria-labelledby="griglia">
<h2 id="griglia">Tutte le varianti</h2>
<p>Ogni variante vale il suo Sharpe, ma non più della media della sua zona (lei e le vicine nella griglia): si preferisce una collina larga a un picco isolato.</p>
<div class="table-wrap">
<table class="table">
<thead><tr><th scope="col">Parametri</th><th scope="col" class="num">Sharpe annuo</th><th scope="col" class="num">Punteggio di zona</th><th scope="col" class="num">Risultato netto</th><th scope="col" class="num">Calo massimo</th><th scope="col" class="num">Operazioni</th></tr></thead>
<tbody>
@foreach (var v in r.Variants)
{
<tr>
<td>@LabText.Parameters(v.Parameters)@if (v.Chosen) {<span class="tag"> scelta</span>}</td>
<td class="num">@LabText.Number(v.Sharpe * annual)</td>
<td class="num">@LabText.Number(v.PlateauScore * annual)</td>
<td class="num">@LabText.Money(v.NetResult, c, true)</td>
<td class="num">@LabText.Percent(v.MaxDrawdown)</td>
<td class="num">@v.Trades</td>
</tr>
}
</tbody>
</table>
</div>
</section>
<section class="panel" aria-labelledby="limiti">
<h2 id="limiti">Cosa questo test non considera</h2>
<ul>
@foreach (var l in r.Limitations)
{
<li>@l</li>
}
</ul>
@if (r.Notes.Count > 0)
{
<h3>Decisioni e ordini non eseguiti come chiesto</h3>
<ul>
@foreach (var (reason, count) in r.Notes.OrderByDescending(n => n.Value))
{
<li>@LabText.Note(reason): @count</li>
}
</ul>
}
<h3>Dati usati</h3>
<div class="table-wrap">
<table class="table">
<thead><tr><th scope="col">Strumento</th><th scope="col">Origine</th><th scope="col">Periodo</th><th scope="col" class="num">Barre</th><th scope="col" class="num">Azioni societarie</th></tr></thead>
<tbody>
@foreach (var i in r.Instruments)
{
<tr><td>@i.Symbol</td><td>@i.Source</td><td class="nowrap">@LabText.Day(i.First) – @LabText.Day(i.Last)</td><td class="num">@i.Bars</td><td class="num">@i.Actions</td></tr>
}
</tbody>
</table>
</div>
<p class="hint">Impronta dei dati @Short(r.DataFingerprint): due test con la stessa impronta hanno letto esattamente gli stessi dati. Motore versione @plan.Engine, strategia versione @r.StrategyVersion.</p>
</section>
@if (r.Trades.Count > 0)
{
<section class="panel" aria-labelledby="operazioni">
<h2 id="operazioni">Operazioni della variante scelta</h2>
<details>
<summary>Vedi le ultime @r.Trades.Count operazioni</summary>
<div class="table-wrap">
<table class="table">
<thead><tr><th scope="col">Strumento</th><th scope="col">Aperta</th><th scope="col">Chiusa</th><th scope="col" class="num">Investito</th><th scope="col" class="num">Risultato</th><th scope="col" class="num">Commissioni</th></tr></thead>
<tbody>
@foreach (var tr in r.Trades.Reverse())
{
<tr><td>@tr.Symbol</td><td class="nowrap">@LabText.Day(tr.Opened)</td><td class="nowrap">@(tr.Closed is { } closed ? LabText.Day(closed) : "aperta")</td><td class="num">@LabText.Money(tr.Invested, c)</td><td class="num @(tr.Result >= 0 ? "gain" : "loss")">@LabText.Money(tr.Result, c, true)</td><td class="num">@LabText.Money(tr.Fees, c)</td></tr>
}
</tbody>
</table>
</div>
</details>
</section>
}
}
@code {
private Trial? _trial;
private BacktestReport? _report;
private string? _progress;
private IReadOnlyList<Anubis.Server.EquityPoint> _curve = [];
[Parameter]
public Guid Id { get; set; }
[CascadingParameter]
public HttpContext? HttpContext { get; set; }
protected override async Task LoadAsync()
{
var ct = HttpContext?.RequestAborted ?? CancellationToken.None;
var services = Host.Services!;
var store = new TrialStore(services.Db, Clock);
_trial = await store.GetAsync(Id, ct);
if (_trial is null)
{
return;
}
_progress = services.Lab?.Progress(Id);
if (await store.ReportJsonAsync(Id, ct) is { } json)
{
_report = BacktestReport.FromJson(json);
_curve = _report.Curve.Select(p => new Anubis.Server.EquityPoint(new DateTimeOffset(p.Day.ToDateTime(TimeOnly.MinValue), TimeSpan.Zero), p.Day, p.Equity, p.Benchmark)).ToList();
}
}
private static string Name(string strategy) => StrategyCatalog.Find(strategy)?.Name ?? strategy;
private static string Short(string hash) => hash.Length > 12 ? hash[..12] : hash;
}
@@ -1,5 +1,233 @@
@inherits LoadedComponent
@page "/operazioni"
@inject AnubisHost Host
@inject ShellData Shell
@inject IClock Clock
@using Anubis.Trading
<PageTitle>Operazioni · Anubis</PageTitle>
<Pending Title="Ancora nessuna operazione chiusa.">Quando il bot comincerà a lavorare, qui troverai ogni operazione con il suo risultato e il perché di ogni ingresso e uscita.</Pending>
@{
var v = _view;
var c = _currency;
var f = _filter;
}
@if (v.Total == 0)
{
<Pending Title="Ancora nessuna operazione chiusa.">Qui compare ogni operazione quando si chiude: quanto ha reso, quanto è durata e perché il bot ha comprato e venduto. Le strategie in osservazione non operano: le vedi in <a href="/strategie">Strategie</a>.</Pending>
}
else
{
<section class="panel head-block trades-head" aria-label="Il bilancio delle operazioni">
<div class="half">
<p class="lead">Per ogni euro perso ne ha guadagnati</p>
<p class="big-number">@(v.PerEuroLost is { } per ? Shell.Money(per, c) : "—")</p>
<p class="note">Guadagni incassati finora: <strong class="@(v.Net >= 0 ? "gain" : "loss")">@Shell.Money(v.Net, c, withSign: true)</strong>@(f.Period == "avvio" && f.Strategy is null && f.Account is null ? "" : " con i filtri scelti")</p>
<dl class="cells">
<div><dt>In guadagno</dt><dd><strong>@v.Winners su @v.Matching</strong><span>@Format.Percent(v.Matching == 0 ? 0 : (decimal)v.Winners / v.Matching * 100m)</span></dd></div>
<div><dt>Guadagno medio</dt><dd><strong>@(v.Winners == 0 ? "—" : Shell.Money(v.Gains / v.Winners, c, withSign: true))</strong><span>su @v.Winners @(v.Winners == 1 ? "operazione" : "operazioni")</span></dd></div>
<div><dt>Perdita media</dt><dd><strong>@(v.Losers == 0 ? "—" : Shell.Money(v.Losses / v.Losers, c, withSign: true))</strong><span>su @v.Losers @(v.Losers == 1 ? "operazione" : "operazioni")</span></dd></div>
<div><dt>Commissioni</dt><dd><strong>@Shell.Money(v.Fees, c)</strong><span>già tolte</span></dd></div>
</dl>
</div>
<div class="half">
<h2>Risultato di ogni operazione <span class="note">dalla prima all'ultima; in alto i guadagni, in basso le perdite</span></h2>
@if (v.Bars.Count > 0)
{
var max = Math.Max(1m, v.Bars.Max(b => Math.Abs(b.Eur)));
var width = 600m / v.Bars.Count;
var best = v.Bars.MaxBy(b => b.Eur);
var worst = v.Bars.MinBy(b => b.Eur);
<svg class="result-bars" viewBox="0 0 600 180" preserveAspectRatio="none" role="img" aria-label="Risultato di ogni operazione: la migliore @Shell.Money(best.Eur, c, withSign: true) su @best.Symbol, la peggiore @Shell.Money(worst.Eur, c, withSign: true) su @worst.Symbol">
<line x1="0" x2="600" y1="90" y2="90" class="axis" />
@for (var i = 0; i < v.Bars.Count; i++)
{
var b = v.Bars[i];
var h = Math.Abs(b.Eur) / max * 70m;
<rect x="@Css(i * width + width * 0.15m)" y="@Css(b.Eur >= 0 ? 90 - h : 90)" width="@Css(width * 0.7m)" height="@Css(Math.Max(h, 0.5m))" class="@(b.Eur >= 0 ? "gain" : "loss")" />
}
</svg>
<p class="note bars-scale"><span>@Day(v.Bars[0].At)</span><span>@Day(v.Bars[^1].At)</span></p>
<p class="note">La migliore <strong class="gain">@Shell.Money(best.Eur, c, withSign: true)</strong> su @best.Symbol, la peggiore <strong class="loss">@Shell.Money(worst.Eur, c, withSign: true)</strong> su @worst.Symbol.</p>
}
else
{
<p class="note">Nessuna operazione con questi filtri.</p>
}
</div>
</section>
<section class="panel" aria-labelledby="elenco">
<h2 id="elenco" class="visually-hidden">Operazioni chiuse</h2>
<form class="filters" method="get" action="/operazioni">
<label class="search"><Icon Name="search" Size="16" /><span class="visually-hidden">Cerca uno strumento</span><input type="search" name="cerca" value="@f.Search" placeholder="Cerca uno strumento" /></label>
<label><span class="visually-hidden">Conto</span>
<select name="conto" data-autosubmit>
<option value="">Tutti i conti</option>
@foreach (var (id, name) in v.Accounts)
{
<option value="@id" selected="@(f.Account == id)">@name</option>
}
</select>
</label>
<label><span class="visually-hidden">Strategia</span>
<select name="strategia" data-autosubmit>
<option value="">Tutte le strategie</option>
@foreach (var s in v.Strategies)
{
<option value="@s" selected="@(f.Strategy == s)">@StrategyNames.Label(s)</option>
}
</select>
</label>
<label><span class="visually-hidden">Esito</span>
<select name="esito" data-autosubmit>
@foreach (var (key, label) in TradeFilter.Results)
{
<option value="@key" selected="@((f.Result ?? "") == key)">@label</option>
}
</select>
</label>
<label><span class="visually-hidden">Periodo</span>
<select name="periodo" data-autosubmit>
@foreach (var (key, label) in TradeFilter.Periods)
{
<option value="@key" selected="@(f.Period == key)">@label</option>
}
</select>
</label>
<noscript><button class="btn" type="submit">Filtra</button></noscript>
<a class="btn" href="/api/operazioni.csv@(f.Query(1))" download><Icon Name="download" Size="18" />Esporta CSV</a>
</form>
@if (v.Rows.Count == 0)
{
<p class="note">Nessuna operazione con questi filtri.</p>
}
else
{
<div class="trades" role="list">
<div class="trade-row trade-header" aria-hidden="true">
<span>Chiusa</span><span>Strumento</span><span class="wide">Strategia</span><span class="wide">Durata</span><span class="num wide">Entrata e uscita</span><span class="num">Risultato</span><span class="wide">Perché è stata chiusa</span>
</div>
@foreach (var t in v.Rows)
{
var gain = t.RealizedEur >= 0;
<details class="trade" role="listitem">
<summary class="trade-row">
<span class="nowrap">@When(t.ClosedAt)</span>
<span><strong>@t.Symbol</strong> <span class="muted">@t.Name</span></span>
<span class="wide nowrap"><span class="swatch round" style="background: var(--s-@t.Strategy.Replace('_', '-'))"></span>@StrategyNames.Label(t.Strategy)</span>
<span class="wide">@Duration(t)</span>
<span class="num wide nowrap">@Format.Money(t.EntryPrice, t.Currency, decimals: t.EntryPrice >= 1m ? 2 : 4) → @Format.Money(t.ExitPrice, t.Currency, decimals: t.ExitPrice >= 1m ? 2 : 4)</span>
<span class="num nowrap @(gain ? "gain" : "loss")"><strong>@Shell.Money(t.RealizedEur, c, withSign: true)</strong> <span class="cell-extra">(@Format.Percent(t.Percent, 2, withSign: true))</span></span>
<span class="wide">@t.ExitReason</span>
</summary>
<div class="why">
<div class="why-card">
<h3>Perché l'ho comprata</h3>
@if (t.Bought.Count == 0)
{
<p class="note">Per questa operazione non c'è la traccia della decisione d'acquisto: è precedente al registro delle decisioni o è stata aperta senza strategia.</p>
}
else
{
<ul class="checks">
@foreach (var line in t.Bought)
{
<li class="@(line.Good ? "ok" : "warn")"><Icon Name="@(line.Good ? "check" : "alert")" Size="16" />@line.Text</li>
}
</ul>
}
</div>
<div class="why-card">
<h3>Perché l'ho venduta</h3>
@if (t.Sold.Count == 0)
{
<p>@t.ExitReason.</p>
}
else
{
<ul class="checks">
@foreach (var line in t.Sold)
{
<li class="@(line.Good ? "ok" : "warn")"><Icon Name="@(line.Good ? "check" : "alert")" Size="16" />@line.Text</li>
}
</ul>
}
<dl class="account-lines">
<dt>Comprata @Long(t.OpenedAt)</dt><dd><strong>@Format.Quantity(t.Quantity) a @Format.Money(t.EntryPrice, t.Currency, decimals: t.EntryPrice >= 1m ? 2 : 4)</strong></dd>
<dt>Venduta @Long(t.ClosedAt)</dt><dd><strong>a @Format.Money(t.ExitPrice, t.Currency, decimals: t.ExitPrice >= 1m ? 2 : 4)</strong></dd>
<dt>Differenza dal prezzo atteso</dt><dd><strong>@Slip(t.EntrySlipBp) e @Slip(t.ExitSlipBp)</strong></dd>
<dt>Commissioni</dt><dd><strong>@Format.Money(t.Fees, t.Currency)</strong></dd>
<dt>Conto</dt><dd>@t.Account</dd>
</dl>
</div>
</div>
</details>
}
</div>
var pages = (v.Matching + TradeFilter.PageSize - 1) / TradeFilter.PageSize;
<div class="pager">
<span class="note">Mostrate @v.Rows.Count di @v.Matching</span>
@if (f.Page > 1)
{
<a class="btn" href="/operazioni@(f.Query(f.Page - 1))">Più recenti</a>
}
@if (f.Page < pages)
{
<a class="btn" href="/operazioni@(f.Query(f.Page + 1))">Più vecchie</a>
}
</div>
}
</section>
}
@code {
private TradesView _view = null!;
private TradeFilter _filter = null!;
private string _currency = "EUR";
[CascadingParameter]
public HttpContext? HttpContext { get; set; }
[SupplyParameterFromQuery(Name = "cerca")] public string? Cerca { get; set; }
[SupplyParameterFromQuery(Name = "conto")] public string? Conto { get; set; }
[SupplyParameterFromQuery(Name = "strategia")] public string? Strategia { get; set; }
[SupplyParameterFromQuery(Name = "esito")] public string? Esito { get; set; }
[SupplyParameterFromQuery(Name = "periodo")] public string? Periodo { get; set; }
[SupplyParameterFromQuery(Name = "pagina")] public string? Pagina { get; set; }
protected override async Task LoadAsync()
{
var ct = HttpContext?.RequestAborted ?? CancellationToken.None;
var services = Host.Services!;
_currency = Shell.Effective((await services.Settings.GetAsync(SettingsStore.Currency.Key, ct)).Value);
_filter = TradeFilter.Parse(Cerca, Conto, Strategia, Esito, Periodo, Pagina);
_view = await TradesView.LoadAsync(services, Clock, _filter, null, ct);
}
private static string Css(decimal value) => value.ToString("0.##", CultureInfo.InvariantCulture);
private string Local(DateTimeOffset at, string format) => TimeZoneInfo.ConvertTime(at, Shell.Zone).ToString(format, CultureInfo.GetCultureInfo("it-IT"));
private string When(DateTimeOffset at) => Capital(Local(at, "ddd d MMM, HH:mm"));
private string Day(DateTimeOffset at) => Local(at, "d MMM");
private string Long(DateTimeOffset at) => Local(at, "dddd d MMMM, HH:mm");
private static string Capital(string text) => text.Length == 0 ? text : char.ToUpper(text[0], CultureInfo.GetCultureInfo("it-IT")) + text[1..];
private static string Duration(TradeRow t)
{
var days = (int)Math.Round((t.ClosedAt - t.OpenedAt).TotalDays);
return days switch
{
< 1 => "in giornata",
1 => "1 giorno",
_ => $"{days} giorni",
};
}
private static string Slip(decimal? bp) => bp is { } b ? Format.Percent(b / 100m, 2) : "—";
}
@@ -2,12 +2,28 @@
@page "/"
@inject AnubisHost Host
@inject ShellData Shell
@using Anubis.Trading
@using Anubis.Data
<PageTitle>Panoramica · Anubis</PageTitle>
@{
var o = _overview;
var p = _panorama;
var c = _currency;
var moves = p.DemoMoves ?? o.Next;
var last = p.DemoLast ?? o.Last;
// On the phone only the first two moves of the bot, without the calendar's events (SPEC 13.4).
var phoneMoves = moves.Select((m, i) => (m, i)).Where(x => !x.m.Calendar).Take(2).Select(x => x.i).ToHashSet();
var v = p.Accounts;
}
@if (_absence is { } away)
{
<div class="notice notice-attention absence" role="status">
<Icon Name="alert" Size="16" />
<span>Sono stato spento per @Anubis.Trading.DiaryText.Duration(away.Length), fino a @Format.ShortDate(away.EndedAt, Shell.Now, Shell.Zone, withTime: false).ToLowerInvariant() alle @TimeZoneInfo.ConvertTime(away.EndedAt, Shell.Zone).ToString("HH:mm", System.Globalization.CultureInfo.InvariantCulture). Ho ripreso i lavori saltati con le regole del recupero: cosa ho fatto e cosa ho lasciato perdere è nel <a href="/diario?tipo=sistema">diario</a>.</span>
<form method="post" action="/azioni/assenza/@away.Id/vista"><AntiforgeryToken /><button class="btn btn-small" type="submit">Ho visto</button></form>
</div>
}
@if (!o.CapitalSet && !Shell.IsDemo)
{
@@ -42,7 +58,9 @@
<span class="big-number">@Shell.Money(o.Result.Amount, c, withSign: true)</span>
<span class="badge @(gain ? "badge-gain" : "badge-loss")"><Arrow Up="gain" />@Format.Percent(o.Result.Percent, 2, withSign: true)</span>
</div>
<p class="note">Dal @Format.LongDate(o.From), partendo da @Shell.Money(o.FromEquity, c). Commissioni già tolte.</p>
var from = Format.LongDate(o.From);
var year = from.LastIndexOf(' ');
<p class="note">Dal @from[..year]<span class="desk-only">@from[year..]</span>, partendo da @Shell.Money(o.FromEquity, c).<span class="desk-only"> Commissioni già tolte.</span></p>
<div class="cells">
@foreach (var (cell, i) in o.Cells.Select((x, i) => (x, i)))
{
@@ -63,47 +81,322 @@
<h2><Icon Name="next" Size="18" />Adesso e dopo</h2>
@if (_runningDays is { } days)
{
<span class="note">Attivo da @days giorni senza interruzioni</span>
<span class="note desk-only">Attivo da @days giorni senza interruzioni</span>
}
</div>
<p class="statement">@Statement(o)</p>
<h3 class="list-title">Prossime mosse</h3>
@if (o.Next.Count == 0)
@if (p.NextDecision is { } decision)
{
<p class="note">Niente in programma.</p>
<p class="next-decision desk-only"><Icon Name="clock" Size="16" />@decision</p>
}
<h3 class="list-title desk-only">Prossime mosse</h3>
@if (moves.Count == 0)
{
<p class="note">Niente in programma: le mosse compaiono quando il motore di trading lavora con almeno una strategia.</p>
}
else
{
<ul class="moves">
@foreach (var move in o.Next)
@for (var i = 0; i < moves.Count; i++)
{
<li><span class="move-time">@move.Time</span><span>@move.Text</span></li>
var move = moves[i];
<li class="@(phoneMoves.Contains(i) ? null : "desk-only")"><span class="move-time">@move.Time</span><span>@move.Text@if (move.Where is { } where) {<span class="muted desk-only"> (@where)</span>}</span></li>
}
</ul>
}
<h3 class="list-title">Ultime azioni</h3>
@if (o.Last.Count == 0)
<div class="desk-only">
<h3 class="list-title">Ultime azioni</h3>
@if (last.Count == 0)
{
<p class="note">Ancora nessuna voce nel diario.</p>
}
else
{
<ul class="moves moves-past">
@foreach (var action in last)
{
<li><span class="move-time">@action.When</span><span>@action.Text</span></li>
}
</ul>
}
<a class="more" href="/diario">Apri il diario completo</a>
</div>
<a class="more phone-only" href="/diario">Tutte le mosse e il diario</a>
</div>
</section>
@* Accounts and the four limits (mockup 01); on the phone a compact list of the accounts (mockup 09). *@
<div class="overview-row">
@if (!v.HasAccounts)
{
<section class="panel overview-empty desk-only">
<p class="empty-title">Nessun conto ancora.</p>
<p>Il bot crea da solo i due simulatori interni, OKX e Alpaca, al primo avvio del motore di trading. I conti di prova compaiono quando ne aggiungi le chiavi in <a href="/impostazioni#conti">Impostazioni</a>.</p>
</section>
}
@foreach (var a in v.Accounts)
{
var up = a.Result >= 0;
var held = v.Positions.Where(x => x.AccountId == a.Id).ToList();
var slices = held.GroupBy(x => x.Strategy).OrderBy(g => StrategyNames.Base.ToList().FindIndex(b => b.Name == g.Key))
.Select(g => (Strategy: g.Key, Eur: g.Sum(x => x.ValueEur))).ToList();
var shares = Format.Shares([.. slices.Select(s => s.Eur), a.Eur(a.Cash)]);
var trend = p.Trends.GetValueOrDefault(a.Id);
<section class="panel overview-account desk-only" aria-label="@a.Name">
<div class="account-head">
<h2>@a.Name <span class="tag">@a.Kind</span></h2>
<span class="link-state @(Down(a) ? "down" : null)"><span class="dot"></span>@(Down(a) ? "Non risponde" : "Connesso")</span>
</div>
<div class="account-value">
<span class="account-equity">@Shell.Money(a.Eur(a.Equity), c)</span>
<span class="cell-change @(up ? "gain" : "loss")"><Arrow Up="up" />@Shell.Money(a.Eur(a.Result), c, withSign: true) (@Format.Percent(a.Percent, 2, withSign: true))</span>
</div>
<div class="split-bar thin" role="img" aria-label="Investito @Shell.Money(a.InvestedEur, c), liquidità @Shell.Money(a.Eur(a.Cash), c)">
@for (var i = 0; i < slices.Count; i++)
{
if (shares[i] > 0)
{
<span style="width: @Css(shares[i])%; background: var(--s-@slices[i].Strategy.Replace('_', '-'))"></span>
}
}
@if (shares[^1] > 0)
{
<span style="width: @Css(shares[^1])%; background: var(--cash)"></span>
}
</div>
<div class="split-key">
<span>@foreach (var s in slices) {<span class="swatch" style="background: var(--s-@s.Strategy.Replace('_', '-'))"></span>}Investito <strong>@Shell.Money(a.InvestedEur, c)</strong></span>
<span><span class="swatch" style="background: var(--cash)"></span>Liquidità <strong>@Shell.Money(a.Eur(a.Cash), c)</strong></span>
</div>
<div class="month">
<div>
<span class="cell-label">Ultimi 30 giorni</span>
<strong>@(trend is null ? "—" : Shell.Money(trend.Month, c, withSign: true))</strong>
</div>
@if (trend is { Points.Count: > 1 } t)
{
<svg class="spark-mini" viewBox="0 0 120 36" role="img" aria-label="Andamento degli ultimi 30 giorni">
<polyline points="@Spark(t.Points)" />
<circle cx="@SparkEnd(t.Points).X" cy="@SparkEnd(t.Points).Y" r="2.5" />
</svg>
}
</div>
<p class="note">@AccountNote(a, held)</p>
</section>
}
<section class="panel overview-risk desk-only" aria-labelledby="rischio-sintesi">
<div class="panel-head">
<h2 id="rischio-sintesi">Rischio</h2>
<span class="status-pill @(p.LimitsKept ? "gain" : "attention")"><Icon Name="@(p.LimitsKept ? "check" : "alert")" Size="16" />@(p.LimitsKept ? "Tutti i limiti rispettati" : "Un limite chiede attenzione")</span>
</div>
<div class="limit-list">
@foreach (var l in p.Limits)
{
<div class="limit">
<div class="limit-line"><span>@l.Label</span><span><strong>@l.Value</strong> <span class="muted">su @l.Threshold</span></span></div>
<span class="usage @(l.Worst is null ? null : "notched") @(l.Share >= 0.8m ? "high" : null)">
<span style="width: @Css(l.Share * 100m)%"></span>
@if (l.Worst is { } w)
{
<i class="notch" style="left: @Css(w * 100m)%"></i>
}
</span>
</div>
}
</div>
@if (p.Limits[1].Worst is not null && _worst is { } worst)
{
<p class="note">Ancora nessuna voce nel diario.</p>
<p class="note">La tacca segna il calo peggiore finora: @Format.Percent(worst * 100m, 2).</p>
}
<a class="more" href="/rischio">Vedi tutti i limiti</a>
</section>
</div>
@if (v.HasAccounts)
{
<section class="panel phone-only" aria-labelledby="conti-telefono">
<h2 id="conti-telefono">Conti</h2>
<ul class="phone-list">
@foreach (var a in v.Accounts)
{
var up = a.Result >= 0;
<li>
<span><strong>@a.Name</strong><span class="sub"><span class="dot @(Down(a) ? "down" : null)"></span>@a.Kind, @(Down(a) ? "non risponde" : "connesso")</span></span>
<span class="num"><strong>@Shell.Money(a.Eur(a.Equity), c)</strong><span class="sub @(up ? "gain" : "loss")"><Arrow Up="up" /><strong>@Shell.Money(a.Eur(a.Result), c, withSign: true)</strong></span></span>
</li>
}
</ul>
</section>
}
@* Positions by value and the strategies (mockup 01); on the phone the first four positions (mockup 09). *@
<div class="overview-row overview-row-wide">
<section class="panel overview-positions" aria-labelledby="posizioni-sintesi">
<div class="panel-head">
<h2 id="posizioni-sintesi">Posizioni aperte
@if (v.Positions.Count > 0)
{
<span class="note desk-only">@v.Positions.Count, per @Shell.Money(v.InvestedEur, c). Risultato aperto @Shell.Money(v.OpenResultEur, c, withSign: true)</span>
}
</h2>
@if (v.Positions.Count > 0)
{
<span class="note phone-only">@v.Positions.Count, risultato @Shell.Money(v.OpenResultEur, c, withSign: true)</span>
}
<a class="desk-only" href="/conti">Dettaglio</a>
</div>
@if (v.Positions.Count == 0)
{
<p class="note">Nessuna posizione aperta. Quando una strategia compra, qui vedi ogni posizione con il suo valore, il risultato e lo stop che la protegge.</p>
}
else
{
<ul class="moves moves-past">
@foreach (var action in o.Last)
<div class="table-wrap desk-only">
<table class="table positions">
<thead>
<tr><th scope="col">Strumento</th><th scope="col">Strategia</th><th scope="col" class="num">Valore</th><th scope="col" class="num">Risultato</th><th scope="col" class="num">Stop</th></tr>
</thead>
<tbody>
@foreach (var x in v.Positions)
{
var gain = x.ResultEur >= 0;
<tr>
<th scope="row"><strong>@x.Symbol</strong> <span class="muted">@x.Name</span></th>
<td class="nowrap"><span class="swatch round" style="background: var(--s-@x.Strategy.Replace('_', '-'))"></span>@StrategyNames.Label(x.Strategy)</td>
<td class="num">@Shell.Money(x.ValueEur, c)</td>
<td class="num nowrap @(gain ? "gain" : "loss")"><strong>@Shell.Money(x.ResultEur, c, withSign: true) (@Format.Percent(x.Percent, 2, withSign: true))</strong></td>
<td class="num nowrap">@if (x.Stop is { } stop) {@Format.Money(stop, x.Currency, decimals: AccountsView.StopDecimals(stop))<span class="muted"> (@Format.Percent(x.Last == 0 ? 0 : (stop / x.Last - 1m) * 100m, 1))</span>} else {<span class="muted">nessuno</span>}</td>
</tr>
}
</tbody>
</table>
</div>
<ul class="phone-list phone-only">
@foreach (var x in v.Positions.Take(4))
{
<li><span class="move-time">@action.When</span><span>@action.Text</span></li>
var gain = x.ResultEur >= 0;
<li>
<span><strong>@x.Symbol</strong><span class="sub"><span class="swatch round" style="background: var(--s-@x.Strategy.Replace('_', '-'))"></span>@StrategyNames.Label(x.Strategy)</span></span>
<span class="num">@Shell.Money(x.ValueEur, c)<strong class="sub @(gain ? "gain" : "loss")">@Shell.Money(x.ResultEur, c, withSign: true)</strong></span>
</li>
}
</ul>
<a class="more" href="/diario">Tutto il diario</a>
<a class="more phone-only" href="/conti">@(v.Positions.Count > 4 ? $"Vedi le altre {v.Positions.Count - 4}" : "Vedi i conti")</a>
}
</section>
<section class="panel overview-strategies desk-only" aria-labelledby="strategie-sintesi">
<div class="panel-head">
<h2 id="strategie-sintesi">Strategie</h2>
<a href="/strategie">Dettaglio</a>
</div>
@if (p.Strategies.Count == 0)
{
<p class="note">Le strategie compaiono quando parte il motore di trading.</p>
}
else
{
<ul class="strategy-list">
@foreach (var s in p.Strategies)
{
<li>
<div class="strategy-line"><strong><span class="swatch round" style="background: @s.Color"></span>@s.Name</strong><strong>@Shell.Money(s.Result, c, withSign: true)</strong></div>
<span class="strategy-bar"><span style="width: @Css(Math.Max(s.Bar * 100m, 2m))%; background: @s.Color"></span></span>
<span class="note">@s.Stage</span>
<span class="note">@s.Trades</span>
</li>
}
</ul>
}
@if (!p.AnyLive)
{
<p class="info-note">Nessuna strategia usa soldi veri. Passano al conto reale solo dopo aver superato tutti i requisiti, e solo se lo confermi tu.</p>
}
</section>
</div>
<section class="panel phone-only" aria-labelledby="rischio-telefono">
<div class="panel-head">
<h2 id="rischio-telefono">Rischio</h2>
<span class="status-pill @(p.LimitsKept ? "gain" : "attention")"><Icon Name="@(p.LimitsKept ? "check" : "alert")" Size="16" />@(p.LimitsKept ? "Limiti rispettati" : "Da controllare")</span>
</div>
<div class="limit-list">
@foreach (var l in new[] { p.Limits[2], p.Limits[1] })
{
<div class="limit">
<div class="limit-line"><span>@l.Label</span><span><strong>@l.Value</strong> <span class="muted">su @l.Threshold.Replace(",00%", "%", StringComparison.Ordinal)</span></span></div>
<span class="usage @(l.Worst is null ? null : "notched")">
<span style="width: @Css(l.Share * 100m)%"></span>
@if (l.Worst is { } w)
{
<i class="notch" style="left: @Css(w * 100m)%"></i>
}
</span>
</div>
}
</div>
</section>
@* News, model and system (mockup 01): sections without a frame, under a dark rule (SPEC 13.2). *@
<div class="overview-tiles desk-only">
<section class="plain tile" aria-labelledby="notizie-sintesi">
<div class="panel-head"><h2 id="notizie-sintesi">Notizie che contano</h2><a href="/notizie">Tutte</a></div>
@if (p.News is null)
{
<p class="note">La lettura delle notizie non è ancora attiva. Quando lo sarà, qui vedrai le tre che contano di più per le tue posizioni e cosa ne ho fatto.</p>
}
else
{
<ul class="news-list">
@foreach (var n in p.News)
{
<li>
<span class="note">@n.When, @n.About</span>
<strong>@n.Title</strong>
<span class="@(n.Active ? "attention" : "muted")">@if (n.Active) {<Icon Name="alert" Size="16" />}@n.Effect</span>
</li>
}
</ul>
}
</section>
<section class="plain tile" aria-labelledby="apprendimento-sintesi">
<div class="panel-head"><h2 id="apprendimento-sintesi">Apprendimento</h2><a href="/apprendimento">Dettaglio</a></div>
@if (p.Learning is not { } m)
{
<p class="note">L'apprendimento non è ancora attivo. Quando lo sarà, il modello osserverà i segnali delle strategie e qui vedrai quanto ci azzecca, prima che possa decidere qualcosa.</p>
}
else
{
<p><strong>Il modello osserva e impara, ma non decide ancora.</strong></p>
<div class="limit">
<div class="limit-line"><span>Segnali già valutati</span><span><strong>@m.Signals</strong> <span class="muted">su @m.Needed necessari</span></span></div>
<span class="usage"><span style="width: @Css(Math.Min(100m, m.Signals * 100m / Math.Max(1, m.Needed)))%"></span></span>
</div>
<dl class="tile-facts">
<dt>Previsioni azzeccate</dt><dd>@m.Precision% su dati mai visti</dd>
<dt>Ultimo addestramento</dt><dd>@m.Trained</dd>
<dt>Prossima verifica</dt><dd>@m.NextCheck</dd>
</dl>
}
</section>
<section class="plain tile" aria-labelledby="sistema-sintesi">
<div class="panel-head"><h2 id="sistema-sintesi">Sistema</h2><a href="/diario">Dettaglio</a></div>
<ul class="system-tiles">
@foreach (var part in p.System.Tiles)
{
<li class="@PartClass(part.State)"><Icon Name="@PartIcon(part.State)" Size="16" /><span>@part.Name</span><span class="muted">@part.Detail</span></li>
}
</ul>
</section>
</div>
@code {
private Overview _overview = null!;
private PanoramaView _panorama = null!;
private string _currency = "EUR";
private string _period = "avvio";
private int? _runningDays;
private decimal? _worst;
private Anubis.Trading.Absence? _absence;
[CascadingParameter]
public HttpContext? HttpContext { get; set; }
@@ -118,8 +411,13 @@
var services = Host.Services!;
_currency = Shell.Effective((await services.Settings.GetAsync(SettingsStore.Currency.Key, ct)).Value);
_overview = await Overview.LoadAsync(services, _period, Shell.Now, Shell.Zone, ct);
_panorama = await PanoramaView.LoadAsync(Host, Shell, _overview, ct);
_worst = _panorama.Limits[1].Worst is { } share ? -share * (await Anubis.Risk.RiskSettings.LoadAsync(services.Settings, ct)).DrawdownStop : null;
using var aggregates = Shell.Aggregates();
_runningDays = aggregates?.RootElement.GetProperty("giorno").GetInt32();
// SPEC 5.6: an absence longer than a day stays at the top until the user says they have seen it.
_absence = Shell.IsDemo ? null : await new Anubis.Trading.AbsenceBook(services.Db, Host.Clock).DeclaredAsync(ct);
}
private static string Statement(Overview o)
@@ -135,4 +433,60 @@
: $"{o.PositionsWithoutStop} senza stop";
return o.Positions == 1 ? $"Sorveglio una posizione, {stops}. {orders}" : $"Sorveglio {o.Positions} posizioni, {stops}. {orders}";
}
private static bool Down(AccountCard a) => a.State.LastOutcome == ReconcileOutcomes.Unreachable;
private string AccountNote(AccountCard a, IReadOnlyList<PositionRow> held)
{
var count = held.Count == 1 ? "1 posizione" : $"{held.Count} posizioni";
if (held.Count == 0)
{
count = "Nessuna posizione aperta";
}
if (a.Broker != Venues.Alpaca)
{
var stops = held.Count(x => x.Stop is not null);
var stopText = held.Count == 0 ? "" : stops == 1 ? ", 1 stop attivo" : $", {stops} stop attivi";
return $"{count}{stopText}. Commissioni pagate finora: {Shell.Money(a.Eur(a.Fees), _currency)}.";
}
var fractions = held.Any(x => x.Quantity != decimal.Truncate(x.Quantity));
var stopsText = held.Count == 0 ? ""
: " Stop permanenti sulle azioni intere" + (!fractions ? "." : _panorama.FractionStops is { } back
? $"; sulle frazioni tornano alle {TimeZoneInfo.ConvertTime(back, Shell.Zone).ToString("HH:mm", CultureInfo.InvariantCulture)}."
: "; sulle frazioni valgono fino alla chiusura.");
var dollars = a.Currency == "USD" && a.Rate > 0m ? $" Conto in dollari: {Format.Money(a.Equity, "USD")} al cambio {Format.Quantity(a.Rate)}." : "";
return count + "." + stopsText + dollars;
}
private static string PartClass(PartState state) => state switch
{
PartState.Ok => "ok",
PartState.Warn => "warn",
_ => "off",
};
private static string PartIcon(PartState state) => state switch
{
PartState.Ok => "tick",
PartState.Warn => "alert",
_ => "clock",
};
private static string Css(decimal value) => Math.Round(value, 3).ToString("0.###", CultureInfo.InvariantCulture);
private static string Spark(IReadOnlyList<decimal> points)
{
var (min, max) = (points.Min(), points.Max());
var range = Math.Max(0.01m, max - min);
return string.Join(' ', points.Select((x, i) => Css(i * 116m / (points.Count - 1) + 2m) + "," + Css(32m - (x - min) / range * 28m)));
}
private static (string X, string Y) SparkEnd(IReadOnlyList<decimal> points)
{
var (min, max) = (points.Min(), points.Max());
var range = Math.Max(0.01m, max - min);
return ("118", Css(32m - (points[^1] - min) / range * 28m));
}
}
@@ -12,7 +12,11 @@
}
else
{
<p>Due passi: dove tenere i dati e la password per entrare. Conti, capitale e Telegram si aggiungono dopo, dalle Impostazioni.</p>
<p>Due passi: dove tenere i dati e la password per entrare. Conti, capitale, Telegram e segnale di vita si aggiungono dopo, dalle Impostazioni: Telegram ha un pulsante che manda un messaggio di prova.</p>
@if (Host.Config.Platform == Anubis.Platform.PlatformKind.Desktop)
{
<p class="hint">Su un computer di casa ti consiglio il segnale di vita (Impostazioni, sezione Telegram): un servizio esterno che ti avvisa quando il bot smette di chiamarlo. È l'unico modo per sapere che il bot è spento quando il computer è spento.</p>
}
@if (Message is { } message)
{
<p class="notice notice-loss" role="alert"><Icon Name="alert" Size="16" />@message</p>
+296 -19
View File
@@ -1,30 +1,307 @@
@inherits LoadedComponent
@page "/rischio"
@inject AnubisHost Host
@inject ShellData Shell
@inject IClock Clock
@using Anubis.Trading
@using Anubis.Risk
<PageTitle>Rischio · Anubis</PageTitle>
<section class="panel">
<h2><Icon Name="shield" Size="18" />Interruttore</h2>
@if (_bot == "stopped")
{
<p class="notice notice-attention" role="status"><Icon Name="alert" Size="16" />"Ferma tutto" è acceso: il bot non apre e non gestisce operazioni finché non lo riattivi.</p>
<form method="post" action="/azioni/bot/riattiva" class="form-inline">
<AntiforgeryToken />
<input type="hidden" name="ritorno" value="/rischio" />
<button class="btn" type="submit" data-confirm="Riattivare il bot?">Riattiva il bot</button>
</form>
}
else
{
<p>"Ferma tutto", in alto, blocca il bot. Dopo averlo usato il bot si riattiva solo da qui, mai da solo e mai da Telegram.</p>
}
@{
var v = _view;
var c = _currency;
}
@if (_message is { } message)
{
<p class="notice notice-loss" role="status"><Icon Name="alert" Size="16" />@message</p>
}
<div class="risk-top">
<section class="panel" aria-labelledby="interruttore">
<div class="panel-head">
<h2 id="interruttore">Interruttore d'emergenza</h2>
<span class="link-state @(v.BotState == "active" ? null : "down")"><span class="dot"></span>@BotText(v.BotState)</span>
</div>
@if (v.WaitsForRestart)
{
<p class="notice notice-attention" role="status"><Icon Name="alert" Size="16" /><span>
@if (v.Tripped.Count > 0)
{
<text>L'interruttore è scattato @string.Join(" e ", v.Tripped.Select(t => Where(t.World))): il calo dal massimo ha superato il @Format.Percent(v.Limits.DrawdownStop * 100m, 0). </text>
}
@if (v.BotState == "stopped")
{
<text>Il bot non apre posizioni e non vende per le strategie; gli stop di protezione restano attivi.</text>
}
else
{
<text>Lì non apro posizioni e non vendo per le strategie; gli stop di protezione restano attivi. Il resto continua a lavorare.</text>
}
@if (v.CloseAllPending)
{
<text> Sta vendendo tutte le posizioni: ciò che è su un mercato chiuso parte all'apertura.</text>
}
</span></p>
@if (v.StopFile)
{
<p class="note">C'è il file <code>STOP</code> nella cartella dei dati: toglilo, poi conferma qui la ripartenza.</p>
}
@if (!Shell.IsDemo)
{
<form method="post" action="/azioni/bot/riattiva" class="form-inline">
<AntiforgeryToken />
<input type="hidden" name="ritorno" value="/rischio" />
<button class="btn" type="submit" disabled="@v.StopFile" data-confirm="@RestartText(v)">Riparti</button>
</form>
}
}
@if (v.BotState != "stopped")
{
<p>Con <strong>Ferma tutto</strong> annullo gli ordini in attesa e non apro nuove posizioni. Gli stop di protezione restano attivi, così quello che è aperto non rimane scoperto. Per ripartire serve la tua conferma, da qui.</p>
@if (!Shell.IsDemo)
{
<div class="choices">
<form method="post" action="/azioni/bot/ferma"><AntiforgeryToken /><input type="hidden" name="ritorno" value="/rischio" /><button class="btn btn-danger-solid" type="submit" data-confirm="Ferma tutto: annullo gli ordini in attesa e non apro posizioni finché non riparti da qui. Gli stop restano. Confermi?"><Icon Name="stop" Size="18" />Ferma tutto</button></form>
<form method="post" action="/azioni/bot/chiudi"><AntiforgeryToken /><input type="hidden" name="ritorno" value="/rischio" /><button class="btn btn-danger" type="submit" data-confirm="Ferma e chiudi: fermo tutto e vendo a mercato ogni posizione del bot. Ciò che è su un mercato chiuso parte all'apertura e intanto tiene il suo stop. Non si annulla. Confermi?">Ferma e chiudi tutte le posizioni</button></form>
</div>
}
<p class="note">Anche un file chiamato <code>STOP</code> nella cartella dei dati ferma tutto entro un minuto.</p>
}
</section>
<section class="panel" aria-labelledby="blocchi">
<h2 id="blocchi">Blocchi automatici</h2>
@if (!v.AnyBlock)
{
<p class="gain status-line"><Icon Name="check" Size="16" />Nessun blocco attivo</p>
}
else
{
<ul class="blocks">
@foreach (var t in v.Tripped)
{
<li class="loss"><Icon Name="alert" Size="16" />Interruttore scattato @Where(t.World) @(t.State.TrippedAt is { } at ? "il " + Moment(at) : ""): aspetta la tua conferma.</li>
}
@if (v.Risk.State.DailyBlockUntil is { } until && until > Clock.UtcNow)
{
<li><Icon Name="clock" Size="16" />Perdita del giorno oltre il limite: niente nuovi ingressi fino a @Moment(until).</li>
}
@if (v.Risk.State.WeeklyHalvingUntil is { } half && half > Clock.UtcNow)
{
<li><Icon Name="clock" Size="16" />Perdita della settimana oltre il limite: rischio dimezzato fino a @Moment(half).</li>
}
@if (v.Risk.Measures.Drawdown <= -v.Limits.DrawdownHalf && !v.Risk.State.Tripped)
{
<li><Icon Name="alert" Size="16" />Calo dal massimo oltre il @Format.Percent(v.Limits.DrawdownHalf * 100m, 0): rischio dimezzato.</li>
}
@if (v.BotState == "paused")
{
<li><Icon Name="pause" Size="16" />Bot in pausa: niente nuovi ingressi, stop e uscite proseguono.</li>
}
@if (v.BotState == "stopped" && v.Tripped.Count == 0)
{
<li class="loss"><Icon Name="stop" Size="16" />Ferma tutto acceso.</li>
}
</ul>
}
<dl class="block-list">
<dt>Prossimo blocco in programma</dt>
@if (v.NextCalendarBlock is { } next)
{
<dd><strong>@Capital(Local(next.From, "dddd d MMMM")), dalle @Local(next.From, "HH:mm") alle @Local(next.To, "HH:mm")</strong><span>@next.Title: in quell'ora non apro posizioni.</span></dd>
}
else
{
<dd><span>Nessun dato economico importante nei prossimi 14 giorni.</span></dd>
}
<dt>Ultimo blocco scattato</dt>
@if (v.LastCalendarBlock is { } last)
{
<dd><strong>@Capital(Local(last.From, "dddd d MMMM")), dalle @Local(last.From, "HH:mm") alle @Local(last.To, "HH:mm")</strong><span>@last.Title.</span></dd>
}
else
{
<dd><span>Nessuno negli ultimi 14 giorni.</span></dd>
}
<dt>Protezione day trade (azioni)</dt>
<dd><span>@(v.PdtGuard ? "Accesa: al massimo 3 operazioni in giornata ogni 5 sedute." : "Spenta. Si accende in Impostazioni se il conto Alpaca vale meno di 25.000 $.")</span></dd>
</dl>
</section>
</div>
<section class="panel" aria-labelledby="limiti">
<div class="panel-head">
<h2 id="limiti">I limiti <span class="note">Valgono per tutte le strategie insieme@(v.World == Worlds.Real ? ", sui conti veri" : ", nel mondo simulato"). Si cambiano dalle <a href="/impostazioni#rischio">impostazioni</a>, mai da soli.</span></h2>
</div>
<div class="table-wrap">
<table class="table limits">
<thead>
<tr><th scope="col">Limite</th><th scope="col" class="num">Adesso</th><th scope="col" class="wide">Quanto ne sto usando</th><th scope="col" class="num">Soglia</th><th scope="col" class="wide">Se lo supero</th></tr>
</thead>
<tbody>
@foreach (var row in v.Rows)
{
<tr>
<th scope="row">@row.Label</th>
<td class="num nowrap">@row.Now@if (row.Detail is { } d) {<text>&nbsp;</text><span class="muted">(@d)</span>}</td>
<td class="wide">
@if (row.Usage is { } u)
{
<span class="usage @(u >= 0.8m ? "high" : null)" role="img" aria-label="@Format.Percent(u * 100m, 0) del limite"><span style="width: @Css(u * 100m)%"></span></span>
}
</td>
<td class="num nowrap">@row.Threshold</td>
<td class="wide">@row.Beyond</td>
</tr>
}
</tbody>
</table>
</div>
</section>
<Pending Title="Limiti e scenari non sono ancora attivi.">Limiti di perdita, calo dal massimo, esposizione e scenari arrivano con le strategie e il controllo del rischio.</Pending>
<div class="columns">
<section class="panel" aria-labelledby="calo">
<h2 id="calo">Quanto è sceso dal massimo</h2>
<p class="note">Ogni fotografia del patrimonio@(v.Risk.State.PeakFrom is { } from ? " dal " + Local(from, "d MMMM") : " dall'avvio"). A zero vuol dire che il portafoglio era al suo punto più alto.</p>
@if (v.Drawdown.Count < 2)
{
<p class="note">Ancora troppo poche fotografie per disegnare la curva.</p>
}
else
{
var worst = v.Drawdown.MinBy(p => p.Drawdown)!;
var floor = Math.Min(-0.01m, worst.Drawdown * 1.2m);
var first = v.Drawdown[0].At;
var span = Math.Max(1d, (v.Drawdown[^1].At - first).TotalSeconds);
string X(DateTimeOffset at) => Css((decimal)((at - first).TotalSeconds / span) * 560m + 30m);
string Y(decimal d) => Css(10m + d / floor * 150m);
var points = string.Join(' ', v.Drawdown.Select(p => X(p.At) + "," + Y(p.Drawdown)));
<svg class="drawdown" viewBox="0 0 600 190" role="img" aria-label="Calo dal massimo: il peggiore @Format.Percent(worst.Drawdown * 100m, 2) il @Local(worst.At, "d MMMM")">
<line x1="30" x2="590" y1="10" y2="10" class="axis" />
<text x="0" y="14" class="tick">0%</text>
<text x="0" y="164" class="tick">@Format.Percent(floor * 100m, 1)</text>
<polygon points="30,10 @points @X(v.Drawdown[^1].At),10" class="area" />
<polyline points="@points" class="line" />
<circle cx="@X(worst.At)" cy="@Y(worst.Drawdown)" r="4" class="dot" />
<text x="@X(worst.At)" y="@(Css(Math.Min(180m, 26m + worst.Drawdown / floor * 150m)))" text-anchor="middle" class="label">@Format.Percent(worst.Drawdown * 100m, 2) il @Local(worst.At, "d MMM")</text>
</svg>
<p class="note">
@if (-worst.Drawdown < v.Limits.DrawdownHalf / 5m)
{
<text>Le soglie del @Format.Percent(v.Limits.DrawdownHalf * 100m, 0) e del @Format.Percent(v.Limits.DrawdownStop * 100m, 0) sono fuori scala: il calo peggiore è rimasto sotto un quinto del limite.</text>
}
else
{
<text>A @Format.Percent(-v.Limits.DrawdownHalf * 100m, 0) il rischio si dimezza, a @Format.Percent(-v.Limits.DrawdownStop * 100m, 0) il bot si ferma e aspetta te.</text>
}
</p>
}
</section>
<section class="panel" aria-labelledby="scenari">
<h2 id="scenari">Se succedesse domani</h2>
<p class="note">Quanto perderei con le posizioni di adesso. Gli scenari si cambiano in <a href="/impostazioni#rischio">Impostazioni</a>.</p>
@if (!v.HasPositions)
{
<p class="note">Nessuna posizione aperta: oggi nessuno scenario costerebbe nulla.</p>
}
else
{
<div class="table-wrap">
<table class="table">
<thead><tr><th scope="col">Scenario</th><th scope="col" class="num">Senza stop</th><th scope="col" class="num">Con gli stop</th></tr></thead>
<tbody>
@foreach (var s in v.Scenarios)
{
<tr>
<th scope="row">@s.Scenario.Label</th>
<td class="num nowrap"><strong>@Shell.Money(s.WithoutStops, c, decimals: 0)</strong> <span class="muted">(@Format.Percent(s.WithoutStopsShare * 100m, 1, withSign: true))</span></td>
<td class="num nowrap muted">@(s.SameWithStops ? "uguale" : "circa " + Shell.Money(s.WithStops ?? 0m, c, decimals: 0))</td>
</tr>
}
</tbody>
</table>
</div>
<p class="notice notice-attention"><Icon Name="alert" Size="16" />Gli stop non sono una garanzia: in un crollo improvviso il prezzo può saltarli. E nessuno stop protegge dal cambio.</p>
}
</section>
</div>
<div class="columns">
<section aria-labelledby="mai" class="plain">
<h2 id="mai">Cosa il bot non può fare, mai</h2>
<ul class="never">
<li><Icon Name="ban" Size="16" />Usare soldi presi in prestito o vendere allo scoperto</li>
<li><Icon Name="ban" Size="16" />Prelevare o spostare denaro: le chiavi dei conti non lo permettono</li>
<li><Icon Name="ban" Size="16" />Toccare posizioni che non ha aperto lui</li>
<li><Icon Name="ban" Size="16" />Passare ai soldi veri senza la tua conferma</li>
<li><Icon Name="ban" Size="16" />Rischiare di più per recuperare una perdita</li>
<li><Icon Name="ban" Size="16" />Comprare di più perché lo suggerisce una notizia o il modello</li>
</ul>
</section>
<section aria-labelledby="eventi" class="plain">
<div class="panel-head">
<h2 id="eventi">Eventi di rischio</h2>
@if (v.Events.Count == 0)
{
<span class="note">Nessun limite superato dall'avvio</span>
}
</div>
@if (v.Events.Count > 0)
{
<dl class="events">
@foreach (var e in v.Events)
{
<dt>@Local(e.At, "d MMMM")</dt>
<dd class="@(e.Severity is RiskSeverities.Critical or RiskSeverities.High ? "loss" : null)">@e.Message</dd>
}
</dl>
<a href="/diario">Vedi tutto nel diario</a>
}
</section>
</div>
@code {
private string _bot = "active";
private RiskView _view = null!;
private string _currency = "EUR";
private string? _message;
protected override async Task LoadAsync() =>
_bot = (await Host.Services!.Settings.GetAsync(SettingsStore.BotState.Key)).Value;
[CascadingParameter]
public HttpContext? HttpContext { get; set; }
[SupplyParameterFromQuery(Name = "errore")]
public string? Errore { get; set; }
protected override async Task LoadAsync()
{
var ct = HttpContext?.RequestAborted ?? CancellationToken.None;
var services = Host.Services!;
_currency = Shell.Effective((await services.Settings.GetAsync(SettingsStore.Currency.Key, ct)).Value);
_view = await RiskView.LoadAsync(services, Clock, Shell.Zone, ct);
_message = Errore switch
{
"file-stop" => "Il file STOP è ancora nella cartella dei dati: toglilo prima di ripartire.",
"bot" => "Comando non ammesso nello stato attuale del bot.",
_ => null,
};
}
private static string BotText(string state) => state switch
{
"active" => "Il bot sta operando",
"paused" => "In pausa",
_ => "Fermo",
};
private static string Where(string world) => world == Worlds.Real ? "sui conti veri" : "nel mondo simulato";
private static string RestartText(RiskView v) => v.Tripped.Count > 0
? "Riparti: il massimo di riferimento riparte dal patrimonio di adesso e il bot torna a operare. Confermi?"
: "Riparti: il bot torna a operare. Confermi?";
private static string Css(decimal value) => value.ToString("0.##", CultureInfo.InvariantCulture);
private string Local(DateTimeOffset at, string format) => TimeZoneInfo.ConvertTime(at, Shell.Zone).ToString(format, CultureInfo.GetCultureInfo("it-IT"));
private string Moment(DateTimeOffset at) => Local(at, "d MMMM 'alle' HH:mm");
private static string Capital(string text) => text.Length == 0 ? text : char.ToUpper(text[0], CultureInfo.GetCultureInfo("it-IT")) + text[1..];
}
@@ -1,5 +1,246 @@
@inherits LoadedComponent
@page "/strategie"
@inject AnubisHost Host
@inject ShellData Shell
@inject IClock Clock
@using Anubis.Trading
<PageTitle>Strategie · Anubis</PageTitle>
<Pending Title="Le strategie non sono ancora attive.">Arrivano dopo il laboratorio: ognuna con la sua descrizione, i requisiti per passare allo stadio successivo e i risultati.</Pending>
@{
var v = _view;
var c = _currency;
var working = v.Cards.Where(x => x.Stage.Stage != StrategyStages.Observation).ToList();
var watching = v.Cards.Where(x => x.Stage.Stage == StrategyStages.Observation).ToList();
}
@if (_message is { } message)
{
<p class="notice @(_ok ? "notice-gain" : "notice-loss")" role="status"><Icon Name="@(_ok ? "check" : "alert")" Size="16" />@message</p>
}
<section class="panel" aria-labelledby="percorso">
<h2 id="percorso">Il percorso verso i soldi veri <span class="note">Una strategia avanza solo se supera tutti i requisiti e se lo confermi tu. Se peggiora, torna indietro da sola.</span></h2>
<ol class="stages">
@foreach (var (stage, i) in new[] { StrategyStages.Observation, StrategyStages.Simulation, StrategyStages.Reduced, StrategyStages.Real }.Select((s, i) => (s, i)))
{
var here = v.Cards.Where(x => x.Stage.Stage == stage).ToList();
<li class="@(here.Count > 0 ? "here" : null)">
<span class="step">@(i + 1)</span>
<strong>@Capital(StrategyStages.Label(stage))</strong>
<p>@StageText(stage)</p>
<p class="stage-count">
@if (here.Count == 0)
{
<span class="muted">Nessuna strategia</span>
}
else
{
@foreach (var x in here)
{
<span class="swatch round" style="background: @x.Color"></span>
}
<strong>@(here.Count == 1 ? "1 strategia" : $"{here.Count} strategie")</strong>
}
</p>
</li>
}
</ol>
</section>
@foreach (var s in working)
{
var up = s.ResultEur >= 0;
<section class="panel strategy" aria-label="@s.Name">
<div class="strategy-about">
<h2><span class="swatch round" style="background: @s.Color"></span>@s.Name <span class="tag">@(s.Stage.ExitOnly ? "Sola uscita" : "In " + StrategyStages.Label(s.Stage.Stage).ToLower(CultureInfo.GetCultureInfo("it-IT")))</span></h2>
<p>@s.Description</p>
<ul>
@foreach (var rule in s.Rules)
{
<li>@rule</li>
}
</ul>
<p class="note">Conto @s.Account. Capitale assegnato: @Shell.Money(s.CapitalEur, c, decimals: 0).</p>
</div>
<div class="strategy-result">
<p class="strategy-number">@Shell.Money(s.ResultEur, c, withSign: true) <span class="cell-change @(up ? "gain" : "loss")"><Arrow Up="up" />@Format.Percent(s.ResultPercent, 2, withSign: true)</span></p>
@if (s.Curve.Count > 1)
{
var min = s.Curve.Min(p => p.Index);
var max = s.Curve.Max(p => p.Index);
var range = Math.Max(0.0001m, max - min);
var points = string.Join(' ', s.Curve.Select((p, i) => Css(i * 300m / (s.Curve.Count - 1)) + "," + Css(70m - (p.Index - min) / range * 60m)));
<svg class="spark" viewBox="0 0 300 80" role="img" aria-label="Andamento della strategia"><polyline points="@points" style="stroke: @s.Color" /></svg>
}
<dl class="account-lines">
<dt>Calo peggiore</dt><dd><strong>@Format.Percent(s.WorstFall * 100m, 1)</strong></dd>
<dt>Operazioni chiuse</dt><dd><strong>@s.Closed@(s.Closed > 0 ? $", di cui {s.Winners} in guadagno" : "")</strong></dd>
<dt>Posizioni aperte</dt><dd><strong>@(s.Open.Count == 0 ? "nessuna" : string.Join(", ", s.Open.Distinct()))</strong></dd>
@if (s.TwinGap is { } gap)
{
<dt>Rispetto al gemello simulato</dt><dd><strong>@Format.Percent(gap, 2, withSign: true)</strong> <span class="muted">del capitale usato</span></dd>
}
</dl>
</div>
<div class="strategy-steps">
<h3>Requisiti per @(s.Stage.Stage == StrategyStages.Reduced ? "il reale pieno" : StrategyStages.IsLive(s.Stage.Stage) ? "restare con soldi veri" : "usare soldi veri")</h3>
<ul class="checks">
@foreach (var r in s.Requirements)
{
<li class="@(r.Met ? "ok" : StrategiesView.IsPending(r) ? "wait" : "warn")">
<Icon Name="@(r.Met ? "tick" : StrategiesView.IsPending(r) ? "clock" : "alert")" Size="16" />
<span><strong>@r.Label</strong>: @r.Value <span class="muted">(@r.Target)</span></span>
</li>
}
</ul>
@if (s.Stage.ExitOnly)
{
<p class="notice notice-attention"><Icon Name="alert" Size="16" />Tornata indietro con posizioni vere ancora aperte: le gestisce fino a zero, con i loro stop, senza nuovi acquisti.</p>
}
@if (StrategyStages.IsLive(s.Stage.Stage) && !Shell.IsDemo)
{
<form method="post" action="/azioni/strategie/@s.Stage.Strategy/osservazione" class="form-inline">
<AntiforgeryToken />
<button class="btn btn-danger" type="submit" data-confirm="Togli i soldi veri a questa strategia: torna in simulazione e le posizioni vere restano in sola uscita fino a zero. Confermi?">Togli i soldi veri</button>
</form>
}
@if (s.Stage.Stage is StrategyStages.Simulation or StrategyStages.Reduced)
{
var target = s.Stage.Stage == StrategyStages.Reduced ? "Passa al reale" : "Passa al reale ridotto";
var broker = StrategyNames.BrokerOf(s.Stage.Strategy);
var accounts = v.LiveAccounts.Where(a => a.Broker == broker).ToList();
if (!s.AllMet)
{
<p class="form-inline"><button class="btn" type="button" disabled>@target</button> <span class="note">@(s.DaysToGo is { } d ? $"Disponibile tra {d} {(d == 1 ? "giorno" : "giorni")}, se gli altri requisiti reggono" : "Non ancora: mancano dei requisiti")</span></p>
@if (s.Requirements.Any(r => !r.Met && !StrategiesView.IsPending(r)))
{
<p class="notice notice-attention"><Icon Name="alert" Size="16" />Per ora non merita soldi veri. Resta in simulazione finché non migliora.</p>
}
}
else if (!v.LiveAllowed || accounts.Count == 0 || Shell.IsDemo)
{
<p class="form-inline"><button class="btn" type="button" disabled>@target</button> <span class="note">@(v.LiveAllowed ? "Serve un conto reale di " + (broker == "okx" ? "OKX" : "Alpaca") + " nelle Impostazioni." : "La configurazione d'avvio non consente i soldi veri.")</span></p>
}
else
{
<details class="confirm-live">
<summary class="btn">@target</summary>
<form method="post" action="/azioni/strategie/@s.Stage.Strategy/reale" class="form-grid">
<AntiforgeryToken />
<p>Stai per affidare soldi veri a <strong>@s.Name</strong>@(s.Stage.Stage == StrategyStages.Simulation ? ", con un quarto del capitale" : ""). Ecco cosa cambia:</p>
<ul class="confirm-summary">
@foreach (var line in s.Summary)
{
<li>@line</li>
}
</ul>
<p>Per confermare scrivi la frase qui sotto e la tua password.</p>
<label>Conto reale
<select name="conto" required>
@foreach (var a in accounts)
{
<option value="@a.Id">@a.Name</option>
}
</select>
</label>
<label>Scrivi: <code>@s.Phrase</code><input name="frase" required autocomplete="off" /></label>
<label>Password<input name="password" type="password" required autocomplete="current-password" /></label>
<label>Codice a sei cifre (se attivo)<input name="codice" inputmode="numeric" autocomplete="one-time-code" /></label>
<button class="btn btn-danger" type="submit">Conferma soldi veri</button>
</form>
</details>
}
}
@if (s.Stage.Stage == StrategyStages.Simulation && !Shell.IsDemo)
{
<form method="post" action="/azioni/strategie/@s.Stage.Strategy/osservazione" class="form-inline">
<AntiforgeryToken />
<button class="btn btn-small" type="submit" data-confirm="Riporta la strategia in osservazione: smette di operare nel simulatore. Confermi?">Riporta in osservazione</button>
</form>
}
</div>
</section>
}
<section class="plain" aria-labelledby="osservazione">
<div class="panel-head">
<h2 id="osservazione">In osservazione</h2>
<a href="/laboratorio">Prova una nuova strategia nel laboratorio</a>
</div>
@if (watching.Count == 0)
{
<p class="note">Nessuna strategia di base in osservazione.</p>
}
else
{
<dl class="observing">
@foreach (var s in watching)
{
var report = StrategiesView.ReportOf(s.Stage);
var passed = s.Stage.Qualification?.Passed == true;
<dt>@s.Name</dt>
<dd>
<span>@StrategiesView.ObservationReason(s.Stage)@if (report is not null) {<text> Rapporto: </text><code>@report</code>}</span>
@if (passed && !Shell.IsDemo)
{
<form method="post" action="/azioni/strategie/@s.Stage.Strategy/simulazione"><AntiforgeryToken /><button class="btn btn-small" type="submit">Porta in simulazione</button></form>
}
else if (!passed)
{
<span class="attention nowrap"><Icon Name="alert" Size="16" />@(s.Stage.Qualification?.Verdict.Contains("Alpaca", StringComparison.Ordinal) == true ? "In attesa dei dati" : "Non supera i test")</span>
}
</dd>
}
</dl>
}
</section>
@code {
private StrategiesView _view = null!;
private string _currency = "EUR";
private string? _message;
private bool _ok;
[CascadingParameter]
public HttpContext? HttpContext { get; set; }
[SupplyParameterFromQuery(Name = "ok")]
public string? Ok { get; set; }
[SupplyParameterFromQuery(Name = "errore")]
public string? Errore { get; set; }
protected override async Task LoadAsync()
{
var ct = HttpContext?.RequestAborted ?? CancellationToken.None;
var services = Host.Services!;
_currency = Shell.Effective((await services.Settings.GetAsync(SettingsStore.Currency.Key, ct)).Value);
_view = await StrategiesView.LoadAsync(services, Clock, TradingConsent.From(Host.Config).AllowLive, ct);
(_message, _ok) = (Ok, Errore) switch
{
("simulazione", _) => ("Fatto: la strategia è in simulazione, con ordini finti su prezzi veri.", true),
("osservazione", _) => ("Fatto: la strategia è in osservazione.", true),
("reale", _) => ("Fatto: la strategia usa soldi veri da adesso. Se peggiora torna indietro da sola.", true),
(_, "backtest") => ("Solo una strategia che ha superato il backtest può andare in simulazione.", false),
(_, "osservazione-giorni") => ($"Prima della simulazione servono almeno {StageBook.ObservationDays} giorni di segnali raccolti dal vivo in osservazione.", false),
(_, "frase") => ("La frase non corrisponde: scrivila esattamente come è mostrata.", false),
(_, "password") => ("Password o codice non corretti. Dopo cinque tentativi sbagliati devi aspettare 15 minuti.", false),
(_, "requisiti") => ("Non tutti i requisiti sono soddisfatti.", false),
(_, "conto") => ("Il conto scelto non è il conto reale del broker di questa strategia.", false),
(_, "consenso") => ("La configurazione d'avvio non consente i soldi veri.", false),
(_, { }) => ("Non è stato possibile completare la scelta: riprova.", false),
_ => (null, false),
};
}
private static string StageText(string stage) => stage switch
{
StrategyStages.Observation => "Produce solo segnali, nessun ordine.",
StrategyStages.Simulation => $"Ordini finti su prezzi veri, per almeno {StageBook.SimulationDays} giorni.",
StrategyStages.Reduced => "Soldi veri, con un quarto del capitale.",
_ => "Soldi veri, rischio pieno.",
};
private static string Capital(string text) => text.Length == 0 ? text : char.ToUpper(text[0], CultureInfo.GetCultureInfo("it-IT")) + text[1..];
private static string Css(decimal value) => value.ToString("0.##", CultureInfo.InvariantCulture);
}
+139
View File
@@ -0,0 +1,139 @@
using System.Collections;
using System.Globalization;
using Anubis.Core;
using Anubis.Db;
using Anubis.Diagnostics;
using Anubis.Platform;
namespace Anubis.Server;
/// <summary>The diagnostics report (issue #95, ADR 0013) as the command, the download and the Diario button make it.</summary>
internal static class DiagnosticsExport
{
/// <summary>Thirty days: as long as the samples are kept.</summary>
public const double MaxHours = 720;
/// <summary>
/// The report over the last <paramref name="hours"/> hours up to <paramref name="to"/>, or from
/// <paramref name="from"/>; without either, from the first sample kept, so a run of a release is covered whole.
/// </summary>
public static async Task<string> BuildAsync(Database db, StartupConfig config, IReadOnlyDictionary<string, string?> environment, DateTimeOffset now,
double? hours, DateTimeOffset? from, DateTimeOffset? to, CancellationToken cancellationToken)
{
var end = to ?? now;
var start = from ?? end - TimeSpan.FromHours(Math.Clamp(hours ?? MaxHours, 0.25, MaxHours));
if (from is null && hours is null)
{
var first = await db.ReadAsync(s => s.FirstOrDefaultAsync("SELECT MIN(taken_at) AS t FROM diagnostic_samples", r => r.MomentOrNull("t")), cancellationToken)
.ConfigureAwait(false);
if (first is { } f && f > start && f < end)
{
start = f;
}
}
var request = new DiagnosticsRequest(start, end, config.TimeZone)
{
LogsFolder = config.Data.Logs,
Setup = Setup(config, environment),
GeneratedAt = now,
};
return await DiagnosticsReporter.BuildAsync(db, request, cancellationToken).ConfigureAwait(false);
}
/// <summary>The startup settings, and which ANUBIS_ variables are set: values only for those that cannot hold a secret.</summary>
internal static IReadOnlyList<KeyValuePair<string, string>> Setup(StartupConfig config, IReadOnlyDictionary<string, string?> environment)
{
var lines = new List<KeyValuePair<string, string>>
{
new("Piattaforma", config.Platform.ToString().ToLowerInvariant()),
new("Database", config.Db?.Describe() ?? "non configurato"),
new("Cartella dei dati", config.Data.Root),
new("Raggiungibile da altri computer", config.AllowNetwork ? "sì" : "no"),
new("Ordini reali consentiti", config.AllowLive ? "sì" : "no"),
};
var variables = environment
.Where(e => e.Key.StartsWith(AnubisInfo.EnvironmentPrefix, StringComparison.OrdinalIgnoreCase) && !string.IsNullOrEmpty(e.Value))
.OrderBy(e => e.Key, StringComparer.Ordinal)
.Select(e => e.Key + (Secretive(e.Key) ? "" : "=" + e.Value))
.ToList();
lines.Add(new("Variabili impostate", variables.Count == 0 ? "nessuna" : string.Join(", ", variables)));
return lines;
}
private static readonly string[] _secretWords = ["KEY", "SECRET", "PASS", "TOKEN", "URL"];
private static bool Secretive(string name) => _secretWords.Any(word => name.Contains(word, StringComparison.OrdinalIgnoreCase));
public static IReadOnlyDictionary<string, string?> ProcessEnvironment() =>
Environment.GetEnvironmentVariables().Cast<DictionaryEntry>().ToDictionary(e => (string)e.Key, e => e.Value as string, StringComparer.Ordinal);
}
/// <summary>
/// "anubis diagnostics": the report to paste back after a long run of a release (issue #95, ADR 0013). It only reads,
/// so it runs while the bot works: in Docker with <c>docker exec</c>, on desktop from a second terminal.
/// </summary>
internal static class DiagnosticsCommand
{
private const string Usage =
"Uso: anubis diagnostics [--hours 72] [--from AAAA-MM-GGTHH:MMZ] [--to AAAA-MM-GGTHH:MMZ] [--out file.md]\n" +
" senza --hours né --from copre tutto ciò che è stato raccolto (al massimo 30 giorni).\n" +
"Opzioni comuni: [--platform docker|desktop] [--data-dir <cartella>]";
/// <summary>0 when the report was made, 1 when it could not be, 2 for the usage.</summary>
public static async Task<int> RunAsync(string[] args, TextWriter? output = null, IReadOnlyDictionary<string, string?>? environment = null, IClock? clock = null)
{
output ??= Console.Out;
clock ??= new SystemClock();
var own = new Dictionary<string, string>(StringComparer.Ordinal);
var rest = new List<string>();
for (var i = 0; i < args.Length; i++)
{
if (args[i] is "--from" or "--to" or "--hours" or "--out" && i + 1 < args.Length)
{
own[args[i]] = args[++i];
}
else if (args[i] is "--help" or "-h")
{
await output.WriteLineAsync(Usage).ConfigureAwait(false);
return 2;
}
else
{
rest.Add(args[i]);
}
}
environment ??= DiagnosticsExport.ProcessEnvironment();
try
{
double? hours = own.TryGetValue("--hours", out var h) ? double.Parse(h, CultureInfo.InvariantCulture) : null;
DateTimeOffset? to = own.TryGetValue("--to", out var t) ? DateTimeOffset.Parse(t, CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal) : null;
DateTimeOffset? from = own.TryGetValue("--from", out var f) ? DateTimeOffset.Parse(f, CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal) : null;
var options = ServerOptions.Parse([.. rest], environment);
var config = options.LoadConfig(environment);
if (!config.IsComplete)
{
await output.WriteLineAsync("La configurazione d'avvio non è completa: avvia prima il server una volta.").ConfigureAwait(false);
return 1;
}
Redactor.Register(config.Db!.Password);
await using var db = Database.Open(config.Db!, allowCreate: false);
var text = await DiagnosticsExport.BuildAsync(db, config, environment, clock.UtcNow, hours, from, to, CancellationToken.None).ConfigureAwait(false);
if (own.TryGetValue("--out", out var file))
{
await File.WriteAllTextAsync(Path.GetFullPath(file), text).ConfigureAwait(false);
}
await output.WriteAsync(text).ConfigureAwait(false);
return 0;
}
catch (Exception ex) when (ex is IOException or FormatException or ArgumentException or InvalidOperationException or SchemaException
or System.Data.Common.DbException or UnauthorizedAccessException or TimeZoneNotFoundException)
{
await output.WriteLineAsync("Non riuscito: " + Redactor.Redact(ex.Message)).ConfigureAwait(false);
return 1;
}
}
}
+243
View File
@@ -0,0 +1,243 @@
using System.Globalization;
using System.Text.RegularExpressions;
using Anubis.Core;
using Anubis.Db;
using Anubis.Trading;
namespace Anubis.Server;
/// <summary>A piece of a diary line: plain text, or an amount coloured as a gain or a loss (always with its sign).</summary>
internal sealed record DiaryPiece(string Text, string? Class = null);
/// <summary>
/// A diary entry as the page draws it (mockup 08): the first sentence as the main line, the rest underneath; a rule
/// the bot changed shows its lead in bold; a sale that closed a trade links to its "why".
/// </summary>
internal sealed record DiaryEntry(DateTimeOffset At, string Kind, string Icon, string? Lead, IReadOnlyList<DiaryPiece> Main, string? Rest, bool Attention, string? Trade);
internal sealed record DiaryDay(DateOnly Day, string Heading, IReadOnlyList<DiaryEntry> Entries);
/// <summary>
/// The diary of "Diario e sistema" (SPEC 13.6): in order of time, grouped by local day, with filters by kind and a
/// search. Static pages: the filters and "Mostra i giorni precedenti" are links with query values.
/// </summary>
internal sealed partial record DiaryView(IReadOnlyList<DiaryDay> Days, string Filter, string? Search, int Range, bool HasOlder, bool Truncated)
{
/// <summary>The risk kind of the filters: not a kind of its own in the journal, but the codes of the risk engine and its relatives.</summary>
public const string RiskKind = "Rischio";
public const int DefaultRange = 2;
public const int MaxSearchResults = 200;
/// <summary>The filter chips, in the order of the mockup: query value, label, kind.</summary>
public static readonly IReadOnlyList<(string Key, string Label, string? Kind)> Filters =
[
("tutto", "Tutto", null),
("decisioni", "Decisioni", DiaryKinds.Decisions),
("ordini", "Ordini", DiaryKinds.Orders),
("notizie", "Notizie", DiaryKinds.News),
("rischio", "Rischio", RiskKind),
("apprendimento", "Apprendimento", DiaryKinds.Learning),
("sistema", "Sistema", DiaryKinds.System),
];
/// <summary>Journal codes that speak of risk without passing through risk_events: refused orders, keys, prudent accounts, the guard.</summary>
private static readonly string[] _riskCodes = ["order_rejected", "stop_rejected", "key_can_withdraw", "account_prudent", "guard_intervention", "clock_gap", "sell_not_filled"];
private const string RiskCondition = "(code IN (SELECT kind FROM risk_events) OR code IN ('order_rejected', 'stop_rejected', 'key_can_withdraw', 'account_prudent', 'guard_intervention', 'clock_gap', 'sell_not_filled'))";
private static readonly string[] _daysLong = ["domenica", "lunedì", "martedì", "mercoledì", "giovedì", "venerdì", "sabato"];
private static readonly string[] _monthsLong = ["gennaio", "febbraio", "marzo", "aprile", "maggio", "giugno", "luglio", "agosto", "settembre", "ottobre", "novembre", "dicembre"];
private static readonly CompareInfo _compare = CultureInfo.GetCultureInfo("it-IT").CompareInfo;
public bool IsEmpty => Days.Count == 0;
/// <summary>The query string of this view with one value changed, for the links of the page.</summary>
public string Link(string? filter = null, int? range = null)
{
var parts = new List<string>();
var f = filter ?? Filter;
if (f != "tutto")
{
parts.Add("tipo=" + Uri.EscapeDataString(f));
}
if (!string.IsNullOrEmpty(Search))
{
parts.Add("cerca=" + Uri.EscapeDataString(Search));
}
var r = range ?? Range;
if (r != DefaultRange)
{
parts.Add("giorni=" + r.ToString(CultureInfo.InvariantCulture));
}
return "/diario" + (parts.Count == 0 ? "" : "?" + string.Join('&', parts));
}
public static async Task<DiaryView> LoadAsync(Database db, string? filter, string? search, string? days, DateTimeOffset now, TimeZoneInfo zone, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(db);
var key = Filters.Any(f => f.Key == filter) ? filter! : "tutto";
var kind = Filters.First(f => f.Key == key).Kind;
var text = string.IsNullOrWhiteSpace(search) ? null : search.Trim();
if (text is { Length: > 100 })
{
text = text[..100];
}
var range = int.TryParse(days, NumberStyles.None, CultureInfo.InvariantCulture, out var n) ? Math.Clamp(n, 1, 3650) : DefaultRange;
var (where, parameters) = kind switch
{
null => ("1 = 1", Array.Empty<(string, object?)>()),
RiskKind => (RiskCondition, []),
_ => ("kind = @k AND NOT " + RiskCondition, new (string, object?)[] { ("k", kind) }),
};
var (rows, older, truncated) = await db.ReadAsync(async s =>
{
const string columns = "SELECT occurred_at, kind, code, message FROM journal WHERE ";
if (text is not null)
{
// The search reads the whole diary, newest first; the comparison is done here, the same on both engines,
// ignoring case and accents.
var all = await s.QueryAsync(columns + where + " ORDER BY occurred_at DESC LIMIT 5000", Row, parameters).ConfigureAwait(false);
var found = all.Where(r => _compare.IndexOf(r.Message, text, CompareOptions.IgnoreCase | CompareOptions.IgnoreNonSpace) >= 0).ToList();
return (found.Take(MaxSearchResults).ToList(), false, found.Count > MaxSearchResults);
}
// The range ends on the newest day that has an entry, so a quiet bot never shows an empty page.
var latest = await s.FirstOrDefaultAsync("SELECT MAX(occurred_at) AS at FROM journal WHERE " + where, r => r.MomentOrNull("at"), parameters).ConfigureAwait(false);
if (latest is null)
{
return ([], false, false);
}
var today = LocalDay(now, zone);
var anchor = LocalDay(latest.Value, zone) is var last && last < today ? last : today;
var from = Start(anchor.AddDays(1 - range), zone);
var withFrom = parameters.Append(("from", from)).ToArray();
var list = await s.QueryAsync(columns + where + " AND occurred_at >= @from ORDER BY occurred_at DESC LIMIT 2000", Row, withFrom).ConfigureAwait(false);
var before = await s.CountAsync("SELECT COUNT(*) FROM journal WHERE " + where + " AND occurred_at < @from", withFrom).ConfigureAwait(false);
return (list.ToList(), before > 0, list.Count >= 2000);
}, cancellationToken).ConfigureAwait(false);
var risky = rows.Count == 0 ? new HashSet<string>(StringComparer.Ordinal)
: (await db.ReadAsync(s => s.QueryAsync("SELECT DISTINCT kind FROM risk_events", r => r.Text("kind")), cancellationToken).ConfigureAwait(false))
.Concat(_riskCodes).ToHashSet(StringComparer.Ordinal);
var day0 = LocalDay(now, zone);
var grouped = rows
.GroupBy(r => LocalDay(r.At, zone))
.Select(g => new DiaryDay(g.Key, Heading(g.Key, day0), [.. g.Select(r => Entry(r.At, risky.Contains(r.Code) ? RiskKind : r.Kind, r.Code, r.Message))]))
.ToList();
return new DiaryView(grouped, key, text, range, older, truncated);
static (DateTimeOffset At, string Kind, string Code, string Message) Row(DbRow r) => (r.Moment("occurred_at"), r.Text("kind"), r.Text("code"), r.Text("message"));
}
/// <summary>"Oggi, venerdì 9 ottobre", "Ieri, giovedì 8 ottobre", "Mercoledì 7 ottobre"; the year when it is not this one.</summary>
public static string Heading(DateOnly day, DateOnly today)
{
var name = $"{_daysLong[(int)day.DayOfWeek]} {day.Day.ToString(CultureInfo.InvariantCulture)} {_monthsLong[day.Month - 1]}" +
(day.Year == today.Year ? "" : " " + day.Year.ToString(CultureInfo.InvariantCulture));
return day == today ? "Oggi, " + name : day == today.AddDays(-1) ? "Ieri, " + name : char.ToUpperInvariant(name[0]) + name[1..];
}
/// <summary>Splits a diary sentence the way the mockup draws it.</summary>
public static DiaryEntry Entry(DateTimeOffset at, string kind, string code, string message)
{
ArgumentNullException.ThrowIfNull(message);
// The demo's row of appendix F says in words where the link goes; the link is drawn, the words are not.
var text = DemoLink().Replace(message, "").Trim();
string main;
string? rest = null;
string? lead = null;
var reason = text.IndexOf(" Motivo: ", StringComparison.Ordinal);
var attention = reason > 0;
if (attention)
{
// A rule changed by the bot: what it decided, in bold up to the colon, then why.
main = text[..reason];
rest = text[(reason + 1)..];
var colon = main.IndexOf(':', StringComparison.Ordinal);
if (colon > 0)
{
lead = main[..(colon + 1)];
main = main[(colon + 1)..];
}
}
else if (text.Length > 80 && SentenceEnd().Match(text) is { Success: true } end)
{
// Long entries: the first sentence on the main line, the explanation underneath. Short ones stay on one line.
main = text[..end.Index];
rest = text[(end.Index + end.Length)..];
}
else
{
main = text;
}
string? trade = null;
if (kind == DiaryKinds.Orders && Sale().Match(text) is { Success: true } sale)
{
// A sale that closed a trade: its "why" is on the Operazioni page, found by the instrument.
trade = sale.Groups["symbol"].Value.Split('/')[0];
}
return new DiaryEntry(at, kind, IconOf(kind), lead, Pieces(main), rest, attention, trade);
}
private static List<DiaryPiece> Pieces(string text)
{
var pieces = new List<DiaryPiece>();
var position = 0;
foreach (Match m in SignedAmount().Matches(text))
{
if (m.Index > position)
{
pieces.Add(new DiaryPiece(text[position..m.Index]));
}
pieces.Add(new DiaryPiece(m.Value, m.Value[0] == '+' ? "gain" : "loss"));
position = m.Index + m.Length;
}
if (position < text.Length)
{
pieces.Add(new DiaryPiece(text[position..]));
}
return pieces;
}
private static string IconOf(string kind) => kind switch
{
DiaryKinds.Orders => "arrows",
DiaryKinds.Decisions => "layers",
DiaryKinds.News => "news",
DiaryKinds.Learning => "network",
RiskKind => "shield",
_ => "sliders",
};
private static DateOnly LocalDay(DateTimeOffset moment, TimeZoneInfo zone) => DateOnly.FromDateTime(TimeZoneInfo.ConvertTime(moment, zone).DateTime);
private static DateTimeOffset Start(DateOnly day, TimeZoneInfo zone)
{
var local = day.ToDateTime(TimeOnly.MinValue);
return new DateTimeOffset(local, zone.GetUtcOffset(local)).ToUniversalTime();
}
[GeneratedRegex(@"\s*\(con il collegamento ""[^""]*""\)")]
private static partial Regex DemoLink();
[GeneratedRegex(@"(?<=[.!?])\s+(?=[A-ZÀ-Ý0-9])")]
private static partial Regex SentenceEnd();
[GeneratedRegex(@"^Vendut[oaie] \S+ (?<symbol>\S+) a [^:]+: [+−]")]
private static partial Regex Sale();
[GeneratedRegex(@"(?<![\w])[+−]\d[\d.,]*\s?[€$]")]
private static partial Regex SignedAmount();
}
+112 -3
View File
@@ -133,6 +133,13 @@ internal static partial class Endpoints
return Error(StatusCodes.Status400BadRequest, "usa_i_comandi", "Lo stato del bot si cambia con pausa, ripresa e Ferma tutto.");
}
// Risk limits go only through the settings page, which checks their order and asks for the confirmation of
// SPEC 12.4 when a limit goes up while a strategy uses real money: the check holds on the API too.
if (Anubis.Risk.RiskSettings.All.Any(d => d.Key == key))
{
return Error(StatusCodes.Status400BadRequest, "usa_le_impostazioni", "I limiti di rischio si cambiano da Impostazioni, con la conferma quando servono soldi veri.");
}
return await SetSettingAsync(host, key, body.Value, context, ct).ConfigureAwait(false);
});
@@ -156,6 +163,35 @@ internal static partial class Endpoints
"text/csv; charset=utf-8", $"anubis-posizioni-{day}.csv");
});
// The closed trades of the Operazioni screen as CSV (SPEC 13.6), with the same filters as the page.
api.MapGet("/operazioni.csv", async (AnubisHost host, ShellData shell, IClock clock, string? cerca, string? conto, string? strategia, string? esito, string? periodo,
CancellationToken ct) =>
{
var filter = TradeFilter.Parse(cerca, conto, strategia, esito, periodo, null);
var view = await TradesView.LoadAsync(host.Services!, clock, filter, int.MaxValue, ct).ConfigureAwait(false);
var day = TimeZoneInfo.ConvertTime(clock.UtcNow, shell.Zone).ToString("yyyy-MM-dd", System.Globalization.CultureInfo.InvariantCulture);
return Results.File(System.Text.Encoding.UTF8.GetPreamble().Concat(System.Text.Encoding.UTF8.GetBytes(view.Csv())).ToArray(),
"text/csv; charset=utf-8", $"anubis-operazioni-{day}.csv");
});
// The technical logs of the data folder as one zip (SPEC 13.6, "Diario e sistema"). They are redacted when
// written (JsonLog); only the files the log writer names, read while it keeps writing the newest one.
api.MapGet("/registri.zip", (AnubisHost host, ShellData shell, IClock clock) =>
{
var folder = host.Config.Data.Logs;
var day = TimeZoneInfo.ConvertTime(clock.UtcNow, shell.Zone).ToString("yyyy-MM-dd", System.Globalization.CultureInfo.InvariantCulture);
return Results.File(LogArchive(folder), "application/zip", $"anubis-registri-{day}.zip");
});
// The data collected for the analysis of a long run, as one Markdown report to paste (issue #95, ADR 0013).
api.MapGet("/diagnostica.md", async (AnubisHost host, ShellData shell, IClock clock, double? ore, CancellationToken ct) =>
{
var now = clock.UtcNow;
var text = await DiagnosticsExport.BuildAsync(host.Services!.Db, host.Config, DiagnosticsExport.ProcessEnvironment(), now, ore, null, null, ct).ConfigureAwait(false);
var day = TimeZoneInfo.ConvertTime(now, shell.Zone).ToString("yyyy-MM-dd-HHmm", System.Globalization.CultureInfo.InvariantCulture);
return Results.File(System.Text.Encoding.UTF8.GetBytes(text), "text/markdown; charset=utf-8", $"anubis-diagnostica-{day}.md");
});
// The external sources and the live feed: what is on, what is off and why (SPEC 4.4).
api.MapGet("/data/sources", async (AnubisHost host, IClock clock, CancellationToken ct) =>
{
@@ -177,7 +213,7 @@ internal static partial class Endpoints
api.MapPost("/bot/{command}", async (string command, AnubisHost host, HttpContext context, CancellationToken ct) =>
{
var (ok, message) = await BotCommandAsync(host, command, context, ct).ConfigureAwait(false);
var (ok, message) = await BotCommandAsync(host, command, context.User.Identity?.Name ?? "utente", ct).ConfigureAwait(false);
return ok ? Results.Json(new { state = message }, _json) : Error(StatusCodes.Status409Conflict, "comando_non_ammesso", message);
});
@@ -201,6 +237,32 @@ internal static partial class Endpoints
internal static IResult Error(int status, string code, string message) => Results.Json(new ApiError(code, message), _json, statusCode: status);
/// <summary>The daily log files (anubis-*.log) of a folder in a zip, newest first; an empty zip when there are none yet.</summary>
internal static byte[] LogArchive(string folder)
{
using var buffer = new MemoryStream();
using (var zip = new System.IO.Compression.ZipArchive(buffer, System.IO.Compression.ZipArchiveMode.Create, leaveOpen: true))
{
var files = Directory.Exists(folder) ? Directory.EnumerateFiles(folder, "anubis-*.log").OrderDescending(StringComparer.Ordinal).ToList() : [];
foreach (var file in files)
{
try
{
using var source = new FileStream(file, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete);
var entry = zip.CreateEntry(Path.GetFileName(file), System.IO.Compression.CompressionLevel.Optimal);
using var target = entry.Open();
source.CopyTo(target);
}
catch (IOException)
{
// A file removed by the rotation in the meantime: the others still go.
}
}
}
return buffer.ToArray();
}
private static async ValueTask<object?> CsrfFilterAsync(EndpointFilterInvocationContext context, EndpointFilterDelegate next)
{
var http = context.HttpContext;
@@ -237,16 +299,22 @@ internal static partial class Endpoints
/// Pause blocks new entries and keeps stops and exits; "Ferma tutto" is the switch of SPEC 9.3 (its full effect
/// arrives with the engine). Resume works only from a pause, never after "Ferma tutto" (SPEC 14).
/// </summary>
internal static async Task<(bool Ok, string Message)> BotCommandAsync(AnubisHost host, string command, HttpContext context, CancellationToken ct)
internal static async Task<(bool Ok, string Message)> BotCommandAsync(AnubisHost host, string command, string by, CancellationToken ct)
{
var settings = host.Services!.Settings;
var current = (await settings.GetAsync(SettingsStore.BotState.Key, ct).ConfigureAwait(false)).Value;
// A switch tripped by the fall from the high stops only its world and leaves the bot "active": the restart is
// the user's confirmation for that world too (SPEC 9.2, 9.3).
var tripped = host.Services.Trading?.Risk is { } engine && await engine.AnyTrippedAsync(ct).ConfigureAwait(false);
var target = command switch
{
"pause" or "pausa" when current == "active" => "paused",
"resume" or "riprendi" when current == "paused" => "active",
"stop" or "ferma" => "stopped",
"close" or "chiudi" => "stopped",
"restart" or "riattiva" when current == "stopped" => "active",
"restart" or "riattiva" when tripped => current,
_ => null,
};
if (target is null)
@@ -256,10 +324,51 @@ internal static partial class Endpoints
: "Comando non ammesso nello stato attuale del bot.");
}
await settings.SetAsync(SettingsStore.BotState.Key, target, context.User.Identity?.Name ?? "utente", ct).ConfigureAwait(false);
if (host.Services.Trading is { Risk: { } risk } trading)
{
// The switch of SPEC 9.3 goes through the risk engine: it cancels the orders under way and keeps the stops.
switch (command)
{
case "stop" or "ferma":
await risk.StopAllAsync(trading.Managers, by, "manual_stop", "Ferma tutto dall'interfaccia: annullo gli ordini in attesa e non apro posizioni. Gli stop restano.", ct)
.ConfigureAwait(false);
return (true, target);
case "close" or "chiudi":
await risk.CloseAllAsync(trading.Managers, by, ct).ConfigureAwait(false);
return (true, target);
case "restart" or "riattiva":
try
{
await risk.RestartAsync(by, ct).ConfigureAwait(false);
await NotifyBotAsync(host, "bot_restarted", $"Il bot è ripartito dopo «Ferma tutto» (da {by}).", ct).ConfigureAwait(false);
return (true, target);
}
catch (InvalidOperationException ex)
{
return (false, ex.Message);
}
}
}
await settings.SetAsync(SettingsStore.BotState.Key, target, by, ct).ConfigureAwait(false);
if (target != current)
{
// Blocks and unblocks reach the phone (SPEC 14), whoever gave the command.
await NotifyBotAsync(host, "bot_" + target, target switch
{
"paused" => $"Bot in pausa (da {by}): niente nuovi acquisti, stop e uscite proseguono.",
"active" => $"Bot ripartito dalla pausa (da {by}).",
"stopped" => $"«Ferma tutto» (da {by}): niente nuove posizioni, gli stop restano. Si riparte solo dall'interfaccia.",
_ => $"Stato del bot: {target} (da {by}).",
}, ct).ConfigureAwait(false);
}
return (true, target);
}
private static Task NotifyBotAsync(AnubisHost host, string code, string message, CancellationToken ct) =>
host.Services!.Db.WriteAsync(s => Anubis.Trading.Diary.NotifyAsync(s, host.Clock, code, message, ct), ct);
// Live status (SPEC 13.4): one message on connect, one at every change and a heartbeat every few seconds, so the
// page can tell how old its data are and warn when the link drops.
private static async Task StatusSocketAsync(HttpContext context, AnubisHost host, ShellData shell, EventBus bus)
+376 -4
View File
@@ -34,6 +34,27 @@ internal static partial class Forms
signed.MapPost("/chiavi/{broker}", SaveKeysAsync);
signed.MapPost("/chiavi/{broker}/togli", RemoveKeysAsync);
signed.MapPost("/capitale/{broker}", SetCapitalAsync);
signed.MapPost("/telegram", SaveTelegramAsync);
signed.MapPost("/telegram/prova", TestTelegramAsync);
signed.MapPost("/telegram/togli", RemoveTelegramAsync);
signed.MapPost("/segnale-di-vita", SaveHeartbeatAsync);
signed.MapPost("/segnale-di-vita/togli", RemoveHeartbeatAsync);
signed.MapPost("/assenza/{id:guid}/vista", AbsenceSeenAsync);
signed.MapPost("/laboratorio", PlanTrialAsync);
signed.MapPost("/strategie/{strategy}/{choice}", StrategyStageAsync);
signed.MapPost("/rischio/limiti", SetLimitsAsync);
signed.MapPost("/rischio/{key}", SetRiskSettingAsync);
}
/// <summary>The notice of a long absence (SPEC 5.6) goes away once the user has seen it.</summary>
private static async Task<IResult> AbsenceSeenAsync(Guid id, AnubisHost host, CancellationToken ct)
{
if (!host.IsDemo && host.Services is { } services)
{
await new Anubis.Trading.AbsenceBook(services.Db, host.Clock).AcknowledgeAsync(id, ct).ConfigureAwait(false);
}
return Results.Redirect("/");
}
private static string Back(IFormCollection form, string fallback)
@@ -155,7 +176,7 @@ internal static partial class Forms
private static async Task<IResult> SetSettingAsync(string key, IFormCollection form, AnubisHost host, HttpContext context, CancellationToken ct)
{
var back = Back(form, "/impostazioni");
if (key != SettingsStore.Theme.Key && key != SettingsStore.Currency.Key && key != SettingsStore.CentralBankMeetings.Key)
if (key != SettingsStore.Theme.Key && key != SettingsStore.Currency.Key && key != SettingsStore.CentralBankMeetings.Key && key != SettingsStore.TelegramStatus.Key)
{
return Results.Redirect(With(back, "errore", "impostazione"));
}
@@ -172,8 +193,8 @@ internal static partial class Forms
private static async Task<IResult> BotAsync(string command, IFormCollection form, AnubisHost host, HttpContext context, CancellationToken ct)
{
var back = Back(form, "/");
var (ok, _) = await Endpoints.BotCommandAsync(host, command, context, ct).ConfigureAwait(false);
return Results.Redirect(ok ? back : With(back, "errore", "bot"));
var (ok, message) = await Endpoints.BotCommandAsync(host, command, context.User.Identity?.Name ?? "utente", ct).ConfigureAwait(false);
return Results.Redirect(ok ? back : With(back, "errore", message.Contains("STOP", StringComparison.Ordinal) ? "file-stop" : "bot"));
}
/// <summary>The user's choice for an account in prudent mode after the 15 minutes (SPEC 7.4): align or leave out.</summary>
@@ -270,6 +291,106 @@ internal static partial class Forms
return Results.Redirect(back + "?ok=chiavi#conti");
}
/// <summary>
/// The bot's Telegram (SPEC 14): a test message first, sent at once; token and chat are saved only when it left.
/// </summary>
private static async Task<IResult> SaveTelegramAsync(IFormCollection form, AnubisHost host, HttpContext context, CancellationToken ct)
{
const string back = "/impostazioni";
if (host.IsDemo || host.Services is not { Telegram: { } telegram } services)
{
return Results.Redirect(back + "?errore=motore#telegram");
}
if ((await services.Secrets.StatusAsync(Anubis.Notifications.TelegramSettings.TokenSecret, ct).ConfigureAwait(false)).FromEnvironment
|| (await services.Secrets.StatusAsync(Anubis.Notifications.TelegramSettings.ChatSecret, ct).ConfigureAwait(false)).FromEnvironment)
{
return Results.Redirect(back + "?errore=telegram-ambiente#telegram");
}
var token = form["token"].ToString().Trim();
var chat = form["chat"].ToString().Trim();
if (token.Length is 0 or > 200 || chat.Length is 0 or > 40)
{
return Results.Redirect(back + "?errore=telegram-vuoto#telegram");
}
if (await telegram.TestAsync(new Anubis.Notifications.TelegramSettings(token, chat), ct).ConfigureAwait(false) is not null)
{
return Results.Redirect(back + "?errore=telegram-rifiutato#telegram");
}
var who = context.User.Identity?.Name ?? "utente";
await services.Secrets.SetAsync(Anubis.Notifications.TelegramSettings.TokenSecret, token, who, ct).ConfigureAwait(false);
await services.Secrets.SetAsync(Anubis.Notifications.TelegramSettings.ChatSecret, chat, who, ct).ConfigureAwait(false);
return Results.Redirect(back + "?ok=telegram#telegram");
}
// The form is bound even when unused: binding it is what makes the antiforgery check run on the route.
private static async Task<IResult> TestTelegramAsync(IFormCollection form, AnubisHost host, CancellationToken ct)
{
const string back = "/impostazioni";
if (host.IsDemo || host.Services is not { Telegram: { } telegram }
|| await telegram.ReadAsync(ct).ConfigureAwait(false) is not { } settings)
{
return Results.Redirect(back + "?errore=telegram-vuoto#telegram");
}
return Results.Redirect(await telegram.TestAsync(settings, ct).ConfigureAwait(false) is null ? back + "?ok=telegram-prova#telegram" : back + "?errore=telegram-prova#telegram");
}
private static async Task<IResult> RemoveTelegramAsync(IFormCollection form, AnubisHost host, HttpContext context, CancellationToken ct)
{
const string back = "/impostazioni";
if (host.IsDemo || host.Services is not { } services)
{
return Results.Redirect(back + "?errore=motore#telegram");
}
var who = context.User.Identity?.Name ?? "utente";
await services.Secrets.RemoveAsync(Anubis.Notifications.TelegramSettings.TokenSecret, who, ct).ConfigureAwait(false);
await services.Secrets.RemoveAsync(Anubis.Notifications.TelegramSettings.ChatSecret, who, ct).ConfigureAwait(false);
return Results.Redirect(back + "?ok=telegram-tolto#telegram");
}
/// <summary>The address of the checking service (SPEC 14): called once as a test, saved encrypted only when it answered.</summary>
private static async Task<IResult> SaveHeartbeatAsync(IFormCollection form, AnubisHost host, HttpContext context, CancellationToken ct)
{
const string back = "/impostazioni";
if (host.IsDemo || host.Services is not { Telegram: { ConfiguredHeartbeat: null } telegram } services)
{
return Results.Redirect(back + "?errore=motore#telegram");
}
var address = form["indirizzo"].ToString().Trim();
if (address.Length is 0 or > 500 || Anubis.Notifications.Heartbeat.Check(address) is not null)
{
return Results.Redirect(back + "?errore=segnale#telegram");
}
var probe = new Anubis.Notifications.Heartbeat(telegram.Http, host.Clock, _ => Task.FromResult<Uri?>(new Uri(address)), Microsoft.Extensions.Logging.Abstractions.NullLogger.Instance);
if (!await probe.BeatAsync(ct).ConfigureAwait(false))
{
return Results.Redirect(back + "?errore=segnale-fallito#telegram");
}
await services.Secrets.SetAsync(TelegramServices.HeartbeatSecret, address, context.User.Identity?.Name ?? "utente", ct).ConfigureAwait(false);
await telegram.Heartbeat.BeatAsync(ct).ConfigureAwait(false);
return Results.Redirect(back + "?ok=segnale#telegram");
}
private static async Task<IResult> RemoveHeartbeatAsync(IFormCollection form, AnubisHost host, HttpContext context, CancellationToken ct)
{
const string back = "/impostazioni";
if (host.IsDemo || host.Services is not { } services)
{
return Results.Redirect(back + "?errore=motore#telegram");
}
await services.Secrets.RemoveAsync(TelegramServices.HeartbeatSecret, context.User.Identity?.Name ?? "utente", ct).ConfigureAwait(false);
return Results.Redirect(back + "?ok=segnale-tolto#telegram");
}
private static async Task<IResult> RemoveKeysAsync(string broker, IFormCollection form, AnubisHost host, HttpContext context, CancellationToken ct)
{
const string back = "/impostazioni";
@@ -290,7 +411,7 @@ internal static partial class Forms
}
/// <summary>The capital entrusted to a broker, in euro (SPEC 7.1, 8.6): only the user sets it.</summary>
private static async Task<IResult> SetCapitalAsync(string broker, IFormCollection form, AnubisHost host, HttpContext context, CancellationToken ct)
private static async Task<IResult> SetCapitalAsync(string broker, IFormCollection form, AnubisHost host, IClock clock, HttpContext context, CancellationToken ct)
{
const string back = "/impostazioni";
if (broker is not (Anubis.Data.Venues.Okx or Anubis.Data.Venues.Alpaca))
@@ -308,6 +429,13 @@ internal static partial class Forms
return Results.Redirect(back + "?errore=importo#conti");
}
// More capital while a strategy of this broker has real money is more risk: it needs the procedure (SPEC 12.4).
if ((await trading.Capital.EntrustedAsync(broker, ct).ConfigureAwait(false))?.AmountEur is { } before && amount > before &&
await RaiseRefusalAsync(host.Services!, clock, broker, form, ct).ConfigureAwait(false) is { } refusal)
{
return Results.Redirect(back + "?errore=" + refusal + "#conti");
}
try
{
await trading.SetCapitalAsync(broker, amount, context.User.Identity?.Name ?? "utente", ct).ConfigureAwait(false);
@@ -324,6 +452,250 @@ internal static partial class Forms
return Results.Redirect(back + "?ok=capitale#conti");
}
/// <summary>
/// The user's choices on the Strategie screen (SPEC 12.3, 12.4): observation and simulation from a button, real
/// money only through the procedure, with the phrase, the password and the second factor checked by the stage book.
/// </summary>
private static async Task<IResult> StrategyStageAsync(string strategy, string choice, IFormCollection form, AnubisHost host, HttpContext context, CancellationToken ct)
{
if (host.IsDemo || host.Services is not { } services || !Anubis.Trading.StrategyNames.Base.Any(b => b.Name == strategy))
{
return Results.Redirect("/strategie?errore=scelta");
}
if (services.Trading?.Stages is not { } stages)
{
return Results.Redirect("/strategie?errore=motore");
}
var by = context.User.Identity?.Name ?? "utente";
try
{
switch (choice)
{
case "simulazione":
var stage = await stages.GetAsync(strategy, ct).ConfigureAwait(false);
if (stage?.Qualification is not { Passed: true })
{
// A strategy that did not pass its backtest stays in observation (SPEC 12.3).
return Results.Redirect("/strategie?errore=backtest");
}
await stages.MoveAsync(strategy, Anubis.Trading.StrategyStages.Simulation, "user", by, ct).ConfigureAwait(false);
return Results.Redirect("/strategie?ok=simulazione");
case "osservazione":
var current = await stages.GetAsync(strategy, ct).ConfigureAwait(false);
if (current is not null && Anubis.Trading.StrategyStages.IsLive(current.Stage))
{
await stages.RevokeAsync(strategy, by, "user", ct).ConfigureAwait(false);
return Results.Redirect("/strategie?ok=simulazione");
}
await stages.MoveAsync(strategy, Anubis.Trading.StrategyStages.Observation, "user", by, ct).ConfigureAwait(false);
return Results.Redirect("/strategie?ok=osservazione");
case "reale" when Guid.TryParse(form["conto"].ToString(), out var account):
var code = form["codice"].ToString();
await stages.ConfirmLiveAsync(new Anubis.Trading.LiveConfirmation(strategy, account, form["frase"].ToString(), form["password"].ToString(),
string.IsNullOrWhiteSpace(code) ? null : code.Trim(), by), ct).ConfigureAwait(false);
return Results.Redirect("/strategie?ok=reale");
default:
return Results.Redirect("/strategie?errore=scelta");
}
}
catch (Anubis.Trading.LiveConfirmationException ex)
{
var error = ex.Code switch
{
"phrase" => "frase",
"password" => "password",
"requirements" => "requisiti",
"backtest" => "backtest",
"observation" => "osservazione-giorni",
"account" => "conto",
Anubis.Risk.RiskReasons.NoConsent => "consenso",
_ => "scelta",
};
return Results.Redirect("/strategie?errore=" + error);
}
}
/// <summary>
/// The risk limits (SPEC 9.2, 13.5), all at once. Lowering is always allowed; raising any of them while a strategy
/// has real money needs the phrase, the password and the second factor (SPEC 12.4).
/// </summary>
private static async Task<IResult> SetLimitsAsync(IFormCollection form, AnubisHost host, IClock clock, HttpContext context, CancellationToken ct)
{
const string back = "/impostazioni";
if (host.IsDemo || host.Services is not { } services)
{
return Results.Redirect(back + "?errore=motore#rischio");
}
var current = new Dictionary<string, string>(StringComparer.Ordinal);
var next = new Dictionary<string, string>(StringComparer.Ordinal);
foreach (var definition in Anubis.Risk.RiskSettings.Limits)
{
current[definition.Key] = (await services.Settings.GetAsync(definition.Key, ct).ConfigureAwait(false)).Value;
var typed = form[definition.Key].ToString().Trim().Replace(',', '.');
if (typed.Length == 0)
{
next[definition.Key] = current[definition.Key];
continue;
}
if (definition.Check?.Invoke(typed) is not null)
{
return Results.Redirect(back + "?errore=limite#rischio");
}
next[definition.Key] = typed;
}
var limits = Anubis.Risk.RiskSettings.From(k => next[k]);
if (limits.DrawdownHalf >= limits.DrawdownStop || limits.DailyLoss > limits.WeeklyLoss)
{
return Results.Redirect(back + "?errore=ordine-limiti#rischio");
}
if (Anubis.Risk.RiskSettings.From(k => current[k]).RaisedBy(limits))
{
if (await RaiseRefusalAsync(services, clock, null, form, ct).ConfigureAwait(false) is { } refusal)
{
return Results.Redirect(back + "?errore=" + refusal + "#rischio");
}
}
var by = context.User.Identity?.Name ?? "utente";
foreach (var (key, value) in next.Where(n => n.Value != current[n.Key]))
{
await services.Settings.SetAsync(key, value, by, ct).ConfigureAwait(false);
}
return Results.Redirect(back + "?ok=limiti#rischio");
}
/// <summary>The day-trade guard and the scenarios of the Rischio screen: they change no limit, so no procedure.</summary>
private static async Task<IResult> SetRiskSettingAsync(string key, IFormCollection form, AnubisHost host, HttpContext context, CancellationToken ct)
{
const string back = "/impostazioni";
if (key != Anubis.Risk.RiskSettings.PdtGuard.Key && key != Anubis.Risk.Scenarios.Setting.Key)
{
return Results.Redirect(back + "?errore=impostazione#rischio");
}
var result = await Endpoints.SetSettingAsync(host, key, form["valore"].ToString(), context, ct).ConfigureAwait(false);
return Results.Redirect(result is IStatusCodeHttpResult { StatusCode: StatusCodes.Status204NoContent } ? back + "?ok=rischio#rischio" : back + "?errore=scenari#rischio");
}
/// <summary>Null when more risk may go ahead; otherwise the error code for the page.</summary>
private static async Task<string?> RaiseRefusalAsync(AppServices services, IClock clock, string? broker, IFormCollection form, CancellationToken ct)
{
// Without the trading engine no strategy can hold real money, so the stage book only has to say so.
var stages = services.Trading?.Stages ?? new Anubis.Trading.StageBook(services.Db, clock, Anubis.Trading.TradingConsent.None, (_, _, _) => Task.FromResult(false));
try
{
var code = form["codice"].ToString();
await stages.ConfirmRaiseAsync(broker, form["frase"].ToString(), form["password"].ToString(), string.IsNullOrWhiteSpace(code) ? null : code.Trim(), ct).ConfigureAwait(false);
return null;
}
catch (Anubis.Trading.LiveConfirmationException ex)
{
return ex.Code switch
{
"phrase" => "frase-rischio",
"demotion" => "retrocessione-rischio",
_ => "password-rischio",
};
}
}
/// <summary>
/// A new backtest trial (SPEC 12.2): the plan is written and queued at once, then the page of the trial follows it.
/// A field that cannot be read goes back as a code; the chosen strategy is kept, nothing typed travels in the address.
/// </summary>
private static async Task<IResult> PlanTrialAsync(IFormCollection form, AnubisHost host, CancellationToken ct)
{
var family = Anubis.Strategies.StrategyCatalog.Find(form["strategia"].ToString());
string Back(string code) => "/laboratorio?errore=" + code + (family is null ? "" : "&strategia=" + Uri.EscapeDataString(family.Id)) + "#nuovo";
if (host.IsDemo || host.Services?.Lab is not { } lab)
{
return Results.Redirect(Back("motore"));
}
if (family is null)
{
return Results.Redirect(Back(Anubis.Backtest.TrialRequestErrors.Strategy));
}
var source = form["origine"].ToString();
var venue = source == Anubis.Backtest.TrialSources.Research ? Anubis.Data.Venues.Binance : form["mercato"].ToString();
if (source == Anubis.Backtest.TrialSources.Database && venue is not (Anubis.Data.Venues.Okx or Anubis.Data.Venues.Alpaca))
{
return Results.Redirect(Back(Anubis.Backtest.TrialRequestErrors.Source));
}
if (ParseDay(form["dal"].ToString()) is not { } from || ParseDay(form["al"].ToString()) is not { } to)
{
return Results.Redirect(Back(Anubis.Backtest.TrialRequestErrors.Dates));
}
if (ParseEuro(form["capitale"].ToString()) is not { } capital)
{
return Results.Redirect(Back(Anubis.Backtest.TrialRequestErrors.Capital));
}
var grid = new Dictionary<string, IReadOnlyList<decimal>>(StringComparer.Ordinal);
foreach (var p in family.Parameters)
{
var text = form["griglia_" + p.Name].ToString();
if (string.IsNullOrWhiteSpace(text))
{
continue;
}
if (ParseList(text) is not { } values)
{
return Results.Redirect(Back(Anubis.Backtest.TrialRequestErrors.Grid));
}
grid[p.Name] = values;
}
var symbols = form["strumenti"].ToString()
.Split([',', ';', ' ', '\n', '\r', '\t'], StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries)
.Select(x => x.ToUpperInvariant())
.ToList();
try
{
var trial = await lab.PlanAsync(new Anubis.Backtest.TrialRequest(family.Id, source, venue, symbols, from, to, capital, grid), ct).ConfigureAwait(false);
return Results.Redirect("/laboratorio/" + trial.Id.ToString("D"));
}
catch (Anubis.Backtest.TrialRequestException ex)
{
return Results.Redirect(Back(ex.Code));
}
}
private static DateOnly? ParseDay(string text) =>
DateOnly.TryParseExact(text, "yyyy-MM-dd", System.Globalization.CultureInfo.InvariantCulture, System.Globalization.DateTimeStyles.None, out var day) ? day : null;
/// <summary>Values of a grid as people type them: "10, 20, 30" or "10 20 30"; decimals with a point.</summary>
internal static IReadOnlyList<decimal>? ParseList(string text)
{
var result = new List<decimal>();
foreach (var item in (text ?? "").Split([',', ';', ' '], StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries))
{
if (!decimal.TryParse(item, System.Globalization.NumberStyles.AllowDecimalPoint, System.Globalization.CultureInfo.InvariantCulture, out var value))
{
return null;
}
result.Add(value);
}
return result.Count is > 0 and <= 20 ? result : null;
}
/// <summary>An amount as people type it: "1500", "1.500", "1500,50", "1.500,50", "1500.50".</summary>
internal static decimal? ParseEuro(string text)
{
+2
View File
@@ -3,6 +3,7 @@ using System.Globalization;
using System.Text;
using System.Text.Json;
using Anubis.Core;
using Anubis.Diagnostics;
namespace Anubis.Server;
@@ -33,6 +34,7 @@ internal sealed class JsonLogProvider : ILoggerProvider
internal void Write(string category, LogLevel level, EventId eventId, string message, Exception? exception)
{
LogCounter.Record(level);
var now = _clock.UtcNow;
using var buffer = new MemoryStream();
using (var json = new Utf8JsonWriter(buffer))
+90
View File
@@ -0,0 +1,90 @@
using Anubis.Backtest;
using Anubis.Core;
namespace Anubis.Server;
/// <summary>How the laboratory pages write the numbers of a backtest, in plain Italian (SPEC 13.3).</summary>
internal static class LabText
{
public const string Missing = "—";
/// <summary>A fraction as a percentage: 0,1234 is "12,3%".</summary>
public static string Percent(double? fraction, int decimals = 1, bool withSign = false) =>
fraction is { } x && double.IsFinite(x) ? Format.Percent((decimal)(x * 100), decimals, withSign) : Missing;
public static string Number(double? value, int decimals = 2) =>
value is { } x && double.IsFinite(x) ? Format.Number((decimal)x, decimals) : Missing;
public static string Money(decimal value, string currency, bool withSign = false) => Format.Money(value, currency, withSign);
public static string Day(DateOnly? day) => day is { } d ? d.ToString("dd/MM/yyyy", System.Globalization.CultureInfo.InvariantCulture) : Missing;
public static string Period(DateOnly from, DateOnly to) => $"{Day(from)} – {Day(to)}";
/// <summary>Parameters as the user reads them: "fast=20;slow=100" becomes "fast 20, slow 100".</summary>
public static string Parameters(string text) =>
string.IsNullOrEmpty(text) ? "nessun parametro" : string.Join(", ", text.Split(';').Select(p => p.Replace('=', ' ').Replace('.', ',')));
public static string Status(Trial trial, string? progress) => trial.Status switch
{
TrialStatuses.Planned => "In coda",
TrialStatuses.Running => progress is null ? "In corso" : "In corso: " + progress,
TrialStatuses.Failed => "Non riuscito",
_ => "Completato",
};
public static string Verdict(string? verdict) => verdict switch
{
TrialVerdicts.Passed => "Superato",
TrialVerdicts.Rejected => "Non superato",
_ => Missing,
};
public static string Source(TrialData data) => data.Source == TrialSources.Research
? "storico di ricerca " + Venue(data.Venue)
: "barre raccolte di " + Venue(data.Venue);
public static string Venue(string venue) => venue switch
{
"okx" => "OKX",
"alpaca" => "Alpaca",
"binance" => "Binance",
_ => venue,
};
public static string Symbols(IReadOnlyList<string> symbols) =>
symbols.Count <= 4 ? string.Join(", ", symbols) : string.Join(", ", symbols.Take(3)) + $" e altri {symbols.Count - 3}";
/// <summary>The value of a check, written in its unit.</summary>
public static string CheckValue(ReportCheck check, string currency) => check.Value is not { } v
? "non si applica"
: check.Unit switch
{
"money" => Format.Money((decimal)v, currency, withSign: true),
"percent" => Percent(v),
"count" => Format.Number((decimal)v),
_ => Number(v),
};
public static string CheckThreshold(ReportCheck check, string currency) => check.Comparison + " " + check.Unit switch
{
"money" => Format.Money((decimal)check.Threshold, currency),
"percent" => Percent(check.Threshold, 0),
"count" => Format.Number((decimal)check.Threshold),
_ => Number(check.Threshold),
};
/// <summary>What the engine did not do as asked, in words.</summary>
public static string Note(string reason) => reason switch
{
BacktestReasons.NoCash => "Acquisti ridotti per mancanza di liquidità",
BacktestReasons.NoPrice => "Ordini senza prezzo di apertura",
BacktestReasons.UnknownSymbol => "Strumenti chiesti dalla strategia ma assenti dai dati",
BacktestReasons.NoShortSelling => "Vendite allo scoperto rifiutate",
BacktestReasons.WrittenOff => "Residui sotto il minimo lasciati dopo un'uscita",
BacktestReasons.Dividend => "Dividendi incassati",
_ when reason.StartsWith(BacktestReasons.AdjustedPrefix, StringComparison.Ordinal) =>
"Decisioni cambiate da correttivi e rischio (" + ReasonTexts.Text(reason[BacktestReasons.AdjustedPrefix.Length..]).TrimEnd('.') + ")",
_ => "Ordini sotto il minimo dello strumento, non inviati",
};
}
+21 -9
View File
@@ -9,7 +9,8 @@ public sealed record EquityPoint(DateTimeOffset At, DateOnly Day, decimal Equity
internal sealed record PeriodResult(string Label, decimal Amount, decimal Percent);
internal sealed record NextMove(string Time, string Text);
/// <summary>A move of the bot with its time; <paramref name="Where"/> is the account, <paramref name="Calendar"/> marks an economic event.</summary>
internal sealed record NextMove(string Time, string Text, string? Where = null, bool Calendar = false);
internal sealed record LastAction(string When, string Text);
@@ -30,7 +31,8 @@ internal sealed record Overview(
int OpenOrders,
IReadOnlyList<NextMove> Next,
IReadOnlyList<LastAction> Last,
bool CapitalSet = true)
bool CapitalSet = true,
DateTimeOffset? NextDecision = null)
{
public static readonly IReadOnlyList<(string Key, string Label)> Periods = [("avvio", "Dall'avvio"), ("30", "30 giorni"), ("7", "7 giorni"), ("oggi", "Oggi")];
@@ -52,7 +54,7 @@ internal sealed record Overview(
var open = await s.CountAsync(
$"SELECT COUNT(*) FROM orders o JOIN accounts a ON a.id = o.account_id WHERE a.environment IN ({mine}) " +
"AND o.status IN ('created', 'submitted', 'partially_filled') AND (o.purpose IS NULL OR o.purpose <> 'stop')").ConfigureAwait(false);
var j = await s.QueryAsync("SELECT occurred_at, message FROM journal ORDER BY occurred_at DESC LIMIT 4",
var j = await s.QueryAsync("SELECT occurred_at, message FROM journal ORDER BY occurred_at DESC LIMIT 3",
r => (At: r.Moment("occurred_at"), Text: r.Text("message"))).ConfigureAwait(false);
var capital = await s.CountAsync("SELECT COUNT(*) FROM capital_changes").ConfigureAwait(false);
return (p, held.Count, held.Count(x => x.Stop is null), open, j, capital > 0);
@@ -65,14 +67,18 @@ internal sealed record Overview(
.Select(g => g.Last())
.ToList();
var next = services.Scheduler.Upcoming(5)
// The moves are the jobs that decide or send orders (SPEC 13.2), not the background work that runs every minute.
var moves = services.Scheduler.Upcoming(200).Where(u => u.IsMove).Take(5).ToList();
var next = moves
.Select(u => new NextMove(TimeZoneInfo.ConvertTime(u.DueUtc, zone).ToString("HH:mm", System.Globalization.CultureInfo.InvariantCulture), u.Description))
.ToList();
var last = journal.Select(j => new LastAction(Format.ShortDate(j.At, now, zone), j.Text)).ToList();
DateTimeOffset? decision = moves.Count > 0 ? moves[0].DueUtc : null;
// Today's actions show the time alone, as in the mockup: "02:06", "Ieri 21:58".
var last = journal.Select(j => new LastAction(Today(Format.ShortDate(j.At, now, zone)), j.Text)).ToList();
if (daily.Count == 0)
{
return new Overview([], [], new PeriodResult("", 0, 0), default, 0, [], (int)positions, (int)withoutStop, (int)orders, next, last, capitalSet);
return new Overview([], [], new PeriodResult("", 0, 0), default, 0, [], (int)positions, (int)withoutStop, (int)orders, next, last, capitalSet, decision);
}
var latest = daily[^1];
@@ -95,11 +101,17 @@ internal sealed record Overview(
};
var series = daily.Where(p => p.Day >= start.Day).ToList();
var cells = new List<PeriodResult> { Between("Oggi", Since(1)), Between("Ultimi 7 giorni", Since(7)), Between("Ultimi 30 giorni", Since(30)) };
if (daily[0].Benchmark is { } firstBenchmark && latest.Benchmark is { } lastBenchmark && firstBenchmark != 0)
// "Senza bot" since the first day it is known, with the same flows taken out as for the bot (SPEC 7.5).
if (daily.FirstOrDefault(p => p.Benchmark is not null) is { Benchmark: { } firstBenchmark } first && latest.Benchmark is { } lastBenchmark)
{
cells.Add(new PeriodResult("Senza bot", lastBenchmark - firstBenchmark, (lastBenchmark - firstBenchmark) / firstBenchmark * 100m));
var flows = latest.Flows - first.Flows;
var amount = lastBenchmark - firstBenchmark - flows;
var basis = firstBenchmark + flows;
cells.Add(new PeriodResult("Senza bot", amount, basis == 0m ? 0m : amount / basis * 100m));
}
return new Overview(daily, series, Between("", start), start.Day, start.Equity, cells, (int)positions, (int)withoutStop, (int)orders, next, last, capitalSet);
return new Overview(daily, series, Between("", start), start.Day, start.Equity, cells, (int)positions, (int)withoutStop, (int)orders, next, last, capitalSet, decision);
}
private static string Today(string shortDate) => shortDate.StartsWith("Oggi ", StringComparison.Ordinal) ? shortDate["Oggi ".Length..] : shortDate;
}
+214
View File
@@ -0,0 +1,214 @@
using System.Globalization;
using System.Text.Json;
using Anubis.Core;
using Anubis.Data;
using Anubis.Risk;
using Anubis.Trading;
namespace Anubis.Server;
/// <summary>The last 30 days of an account: the result net of flows and the daily values in euro for the small line.</summary>
internal sealed record AccountTrend(decimal Month, IReadOnlyList<decimal> Points);
/// <summary>A news item of the "Notizie che contano" tile; <paramref name="Active"/> when its effect is still on.</summary>
internal sealed record PanoramaNews(string When, string About, string Title, string Effect, bool Active);
/// <summary>The model at a glance: signals counted toward the 300, precision on unseen cases, training and next check.</summary>
internal sealed record PanoramaLearning(int Signals, int Needed, int Precision, string Trained, string NextCheck);
/// <summary>One of the four limits of the Panoramica (SPEC 13.6): value, threshold, share used and, for the fall, the worst so far.</summary>
internal sealed record PanoramaLimit(string Label, string Value, string Threshold, decimal Share, decimal? Worst = null);
/// <summary>A strategy at a glance: result, the bar against the best one, its stage in words and its trades.</summary>
internal sealed record PanoramaStrategy(string Name, string Color, decimal Result, decimal Bar, string Stage, string Trades);
/// <summary>
/// Everything under the top block of the Panoramica (SPEC 13.6, mockup 01): accounts with their last 30 days, the
/// four main limits, positions, strategies, news, model and system. In demo mode the values of appendix F that
/// exist only as totals come from the aggregates (SPEC 13.1); outside it news and the model are not active yet.
/// </summary>
internal sealed record PanoramaView(
AccountsView Accounts,
IReadOnlyDictionary<Guid, AccountTrend> Trends,
IReadOnlyList<PanoramaLimit> Limits,
bool LimitsKept,
IReadOnlyList<PanoramaStrategy> Strategies,
bool AnyLive,
SystemView System,
IReadOnlyList<NextMove>? DemoMoves,
IReadOnlyList<LastAction>? DemoLast,
string? NextDecision,
IReadOnlyList<PanoramaNews>? News,
PanoramaLearning? Learning,
DateTimeOffset? FractionStops)
{
private static readonly CultureInfo _it = CultureInfo.InvariantCulture;
public static async Task<PanoramaView> LoadAsync(AnubisHost host, ShellData shell, Overview overview, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(host);
ArgumentNullException.ThrowIfNull(shell);
ArgumentNullException.ThrowIfNull(overview);
var services = host.Services!;
var clock = host.Clock;
var risk = await RiskView.LoadAsync(services, clock, shell.Zone, cancellationToken).ConfigureAwait(false);
var accounts = await AccountsView.LoadAsync(services, clock, shell.Zone, cancellationToken).ConfigureAwait(false);
var strategies = await StrategiesView.LoadAsync(services, clock, false, cancellationToken).ConfigureAwait(false);
var system = await SystemView.LoadAsync(host, shell, cancellationToken).ConfigureAwait(false);
var trends = await TrendsAsync(services, accounts, shell.Now, shell.Zone, cancellationToken).ConfigureAwait(false);
// The four limits of the Panoramica, read from the same rows as the Rischio page.
var rows = risk.Rows.ToDictionary(r => r.Key);
var day = rows[RiskSettings.DailyLoss.Key];
var invested = rows[RiskSettings.MaxInvested.Key];
var biggest = rows[RiskSettings.MaxPosition.Key];
var fall = risk.Risk.Measures.Drawdown;
decimal? worst = risk.Drawdown.Count > 0 ? risk.Drawdown.Min(d => d.Drawdown) : null;
var stop = risk.Limits.DrawdownStop;
var limits = new List<PanoramaLimit>
{
new("Perdita di oggi", day.Now, day.Threshold, day.Usage ?? 0m),
new("Calo dal massimo", Format.Percent(fall * 100m, 2), Format.Percent(-stop * 100m, 2), stop <= 0m ? 0m : Math.Clamp(-fall / stop, 0m, 1m),
worst is { } w && stop > 0m ? Math.Clamp(-w / stop, 0m, 1m) : null),
new("Capitale investito", invested.Now, invested.Threshold, invested.Usage ?? 0m),
new("Posizione più grande" + (biggest.Detail is { } symbol ? $" ({symbol.Split('/')[0]})" : ""), biggest.Now, biggest.Threshold, biggest.Usage ?? 0m),
};
var kept = !risk.AnyBlock && risk.Rows.All(r => r.Usage is null or < 1m);
// Strategies: the bar compares each result with the best one; the stage says what is still missing.
var best = strategies.Cards.Select(c => Math.Abs(c.ResultEur)).DefaultIfEmpty(0m).Max();
var cards = strategies.Cards.Select(c => new PanoramaStrategy(c.Name, c.Color, c.ResultEur, best == 0m ? 0m : Math.Abs(c.ResultEur) / best, Stage(c), Trades(c))).ToList();
var anyLive = strategies.Cards.Any(c => StrategyStages.IsLive(c.Stage.Stage));
// The daily stops of the fractions come back with the orders after the open (SPEC 7.2).
DateTimeOffset? fractions = accounts.MarketOpen ? null : accounts.MarketNext + (services.Trading?.Engine.Options.OrdersAfterOpen ?? new LiveEngineOptions().OrdersAfterOpen);
using var aggregates = shell.Aggregates();
if (aggregates is null)
{
return new PanoramaView(accounts, trends, limits, kept, cards, anyLive, system, null, null, Decision(overview.NextDecision, shell.Now, shell.Zone), null, null, fractions);
}
var root = aggregates.RootElement;
var demo = root.GetProperty("panoramica");
var moves = demo.GetProperty("mosse").EnumerateArray().Select(m => new NextMove(m.GetProperty("ora").GetString()!, m.GetProperty("testo").GetString()!,
m.TryGetProperty("dove", out var where) ? where.GetString() : null, m.GetProperty("calendario").GetBoolean())).ToList();
var last = demo.GetProperty("ultime_azioni").EnumerateArray().Select(a => new LastAction(a.GetProperty("quando").GetString()!, a.GetProperty("testo").GetString()!)).ToList();
var news = root.GetProperty("notizie").GetProperty("panoramica").EnumerateArray().Select(n => new PanoramaNews(n.GetProperty("quando").GetString()!,
n.GetProperty("riguarda").GetString()!, n.GetProperty("titolo").GetString()!, n.GetProperty("effetto").GetString()!, n.GetProperty("in_corso").GetBoolean())).ToList();
var model = root.GetProperty("apprendimento");
var learning = new PanoramaLearning(model.GetProperty("segnali").GetInt32(), model.GetProperty("necessari").GetInt32(), model.GetProperty("precisione_pct").GetInt32(),
model.GetProperty("ultimo_addestramento").GetString()!, model.GetProperty("prossima_verifica").GetString()!);
// The 30-day results of appendix F.2: the generated daily series of each account only ends on its total.
var month = root.GetProperty("conti").EnumerateArray().ToDictionary(c => c.GetProperty("nome").GetString()!,
c => decimal.Parse(c.GetProperty("trenta_giorni").GetString()!, _it), StringComparer.Ordinal);
var demoTrends = accounts.Accounts.ToDictionary(a => a.Id,
a => new AccountTrend(month.GetValueOrDefault(a.Name, trends.GetValueOrDefault(a.Id)?.Month ?? 0m), trends.GetValueOrDefault(a.Id)?.Points ?? []));
return new PanoramaView(accounts, demoTrends, limits, kept, cards, anyLive, system, moves, last, demo.GetProperty("prossima_decisione").GetString(), news, learning, fractions);
}
private static async Task<Dictionary<Guid, AccountTrend>> TrendsAsync(AppServices services, AccountsView accounts, DateTimeOffset now, TimeZoneInfo zone,
CancellationToken cancellationToken)
{
var shown = accounts.Accounts.ToDictionary(a => a.Id);
var world = accounts.Accounts.Any(a => TradingEnvironments.World(a.Environment) == Worlds.Real) ? Worlds.Real : Worlds.Simulated;
var rows = await services.Db.ReadAsync(s => s.QueryAsync(
"SELECT taken_at, account_id, currency, equity, flows FROM equity_snapshots WHERE scope = 'account' AND world = @w AND taken_at >= @f ORDER BY taken_at, id",
r => (At: r.Moment("taken_at"), Account: r.IdOrNull("account_id"), Currency: r.TextOrNull("currency"), Equity: r.Money("equity"), Flows: r.MoneyOrNull("flows") ?? 0m),
("w", world), ("f", now.AddDays(-32))), cancellationToken).ConfigureAwait(false);
var result = new Dictionary<Guid, AccountTrend>();
foreach (var group in rows.Where(r => r.Account is { } id && shown.ContainsKey(id)).GroupBy(r => r.Account!.Value))
{
var account = shown[group.Key];
decimal Eur(decimal amount, string? currency) => currency == "USD" ? account.Eur(amount) : amount;
var daily = group.GroupBy(r => DateOnly.FromDateTime(TimeZoneInfo.ConvertTime(r.At, zone).DateTime)).Select(g => g.Last()).ToList();
var latest = daily[^1];
var latestDay = DateOnly.FromDateTime(TimeZoneInfo.ConvertTime(latest.At, zone).DateTime);
var start = daily.LastOrDefault(p => DateOnly.FromDateTime(TimeZoneInfo.ConvertTime(p.At, zone).DateTime) <= latestDay.AddDays(-30));
start = start == default ? daily[0] : start;
var month = Eur(latest.Equity, latest.Currency) - Eur(start.Equity, start.Currency) - (Eur(latest.Flows, latest.Currency) - Eur(start.Flows, start.Currency));
var points = daily.Where(p => p.At >= start.At).Select(p => Eur(p.Equity - p.Flows, p.Currency)).ToList();
result[group.Key] = new AccountTrend(month, points);
}
return result;
}
/// <summary>Where a strategy is on its path, with the requirement that is furthest away: "In simulazione, giorno 58 di 60".</summary>
private static string Stage(StrategyCard card)
{
var stage = card.Stage.Stage;
if (stage == StrategyStages.Observation)
{
return "In osservazione";
}
if (stage == StrategyStages.Real)
{
return "Con soldi veri";
}
var where = stage == StrategyStages.Reduced ? "In reale ridotto" : "In simulazione";
var missing = card.Requirements
.Where(r => !r.Met && r.Key is "days" or "trades")
.Select(r => (r.Key, Value: Whole(r.Value), Target: Whole(r.Target)))
.Where(r => r.Value is not null && r.Target is > 0)
.OrderBy(r => (decimal)r.Value!.Value / r.Target!.Value)
.FirstOrDefault();
return missing.Key switch
{
"days" => $"{where}, giorno {missing.Value} di {missing.Target}",
"trades" => $"{where}, {missing.Value} operazioni su {missing.Target}",
_ => where,
};
static int? Whole(string text)
{
var digits = new string(text.Where(char.IsAsciiDigit).ToArray());
return int.TryParse(digits, NumberStyles.None, _it, out var n) ? n : null;
}
}
private static string Trades(StrategyCard card)
{
var open = card.Open.Distinct().Count();
var positions = open switch
{
0 => "Nessuna posizione aperta",
1 => "1 posizione aperta",
_ => $"{open} posizioni aperte",
};
var closed = card.Closed switch
{
0 => "nessuna operazione chiusa",
1 => $"1 operazione chiusa ({(card.Winners == 1 ? "in guadagno" : "in perdita")})",
_ => $"{card.Closed} operazioni chiuse ({card.Winners} in guadagno)",
};
return positions + ", " + closed;
}
/// <summary>"Prossima decisione tra 7 ore e 14 minuti, alle 15:35."</summary>
private static string? Decision(DateTimeOffset? due, DateTimeOffset now, TimeZoneInfo zone)
{
if (due is not { } at || at <= now)
{
return null;
}
var wait = at - now;
var hours = (int)wait.TotalHours;
var minutes = wait.Minutes;
var span = (hours, minutes) switch
{
(0, <= 1) => "un minuto",
(0, _) => $"{minutes} minuti",
(1, 0) => "un'ora",
(_, 0) => $"{hours} ore",
(1, _) => $"un'ora e {minutes} minuti",
_ => $"{hours} ore e {minutes} minuti",
};
var time = TimeZoneInfo.ConvertTime(at, zone).ToString("HH:mm", _it);
return wait < TimeSpan.FromHours(24) ? $"Prossima decisione tra {span}, alle {time}." : $"Prossima decisione {Format.ShortDate(at, now, zone).ToLowerInvariant()}.";
}
}
+18
View File
@@ -12,6 +12,24 @@ if (args.Length >= 1 && args[0] == "data")
return await DataCommand.RunAsync(args[1..]).ConfigureAwait(false);
}
// "anubis backtest base": the honest backtests of the base strategies, written to docs/backtests (Phase 5).
if (args.Length >= 1 && args[0] == "backtest")
{
return await BacktestCommand.RunAsync(args[1..]).ConfigureAwait(false);
}
// "anubis soak": the report of the endurance run in real time, read from the bot's records while it works (Phase 6).
if (args.Length >= 1 && args[0] == "soak")
{
return await SoakCommand.RunAsync(args[1..]).ConfigureAwait(false);
}
// "anubis diagnostics": the data collected for the analysis of a long run, as one report to paste (issue #95).
if (args.Length >= 1 && args[0] == "diagnostics")
{
return await DiagnosticsCommand.RunAsync(args[1..]).ConfigureAwait(false);
}
var app = ServerApp.Build(args);
await app.RunAsync().ConfigureAwait(false);
return 0;
+67
View File
@@ -0,0 +1,67 @@
using System.Globalization;
using System.Text.Json;
using Anubis.Core;
using Anubis.Trading;
namespace Anubis.Server;
/// <summary>
/// The fixed templates that turn the reason codes of the decisions table into plain Italian (SPEC 5.2): the "why"
/// card of a trade and the diary use these, never free text written at decision time. An unknown code is shown as
/// it is, so nothing is ever hidden.
/// </summary>
internal static class ReasonTexts
{
// The exits in the few words of a table column; the full sentence stays for the "why" card.
private static readonly Dictionary<string, string> ShortExits = new(StringComparer.Ordinal)
{
["trend_exit"] = "La tendenza si è interrotta: prezzo sotto lo stop mobile",
["momentum_exit_rank"] = "Non era più tra i 30 titoli più forti",
["momentum_exit_trend"] = "Il mercato è girato al ribasso",
["meanrev_exit_average"] = "Tornata sopra la media a 5 giorni",
["meanrev_exit_rsi"] = "Rimbalzo arrivato: forza tornata alta",
["meanrev_exit_earnings"] = "Uscita prima della trimestrale",
["meanrev_exit_time"] = "Tempo scaduto: nessun rimbalzo in 5 sedute",
["stop"] = "Stop di protezione raggiunto",
};
/// <summary>Why a trade was closed, in a short sentence.</summary>
public static string Exit(string? code) => code is null ? "Chiusa" : ShortExits.TryGetValue(code, out var text) ? text : Text(code).TrimEnd('.');
public static string Text(string code) => ReasonCatalog.Text(code);
/// <summary>The lines of one stage of a decision. <paramref name="data"/> is the JSON of the numbers the stage used.</summary>
public static IEnumerable<WhyLine> Stage(string stage, string outcome, IReadOnlyList<string> reasons, string? data)
{
ArgumentNullException.ThrowIfNull(reasons);
var good = outcome is DecisionOutcomes.Passed or DecisionOutcomes.Proposed or DecisionOutcomes.Same;
if (stage == DecisionStages.Risk && outcome == DecisionOutcomes.Passed && reasons.Count == 0)
{
yield return new WhyLine("Tutti i limiti di rischio rispettati" + Asked(data), true);
yield break;
}
foreach (var reason in reasons)
{
yield return new WhyLine(Text(reason).TrimEnd('.') + (stage == DecisionStages.Risk ? Asked(data) : ""), good || stage == DecisionStages.Strategy);
}
}
// "(chiesti 0,05, passati 0,03)" when the stage shrank the order.
private static string Asked(string? data)
{
if (data is null)
{
return "";
}
using var json = JsonDocument.Parse(data);
if (json.RootElement.TryGetProperty("asked", out var asked) && json.RootElement.TryGetProperty("allowed", out var allowed) &&
asked.TryGetDecimal(out var a) && allowed.TryGetDecimal(out var b) && b < a)
{
return $" (chiesti {Format.Quantity(a)}, passati {Format.Quantity(b)})";
}
return "";
}
}
+152
View File
@@ -0,0 +1,152 @@
using Anubis.Core;
using Anubis.Data;
using Anubis.Risk;
using Anubis.Strategies;
using Anubis.Trading;
namespace Anubis.Server;
/// <summary>A row of the limits table: what is used now against the threshold, and what happens beyond it.</summary>
internal sealed record LimitRow(string Label, string Now, string? Detail, decimal? Usage, string Threshold, string Beyond, string Key);
/// <summary>A point of the fall from the high, as a fraction: 0 at the high, −0,03 three percent below.</summary>
internal sealed record DrawdownPoint(DateTimeOffset At, decimal Drawdown);
/// <summary>
/// The Rischio screen (SPEC 9, 13.6, mockup 07): the emergency switch, the automatic blocks, the limits with what
/// they use now, the fall from the high, the scenarios and the risk events. One world, as on Conti.
/// </summary>
internal sealed record RiskView(
string BotState,
bool StopFile,
bool CloseAllPending,
string World,
WorldRisk Risk,
RiskLimits Limits,
IReadOnlyList<LimitRow> Rows,
IReadOnlyList<DrawdownPoint> Drawdown,
IReadOnlyList<ScenarioResult> Scenarios,
IReadOnlyList<(DateTimeOffset At, string Kind, string Severity, string Message, string? World)> Events,
(DateTimeOffset From, DateTimeOffset To, string Title)? NextCalendarBlock,
(DateTimeOffset From, DateTimeOffset To, string Title)? LastCalendarBlock,
bool PdtGuard,
bool HasPositions,
IReadOnlyList<WorldRisk> Tripped)
{
public bool AnyBlock => Tripped.Count > 0 || Risk.EntriesBlocked || Risk.Multiplier < 1m || BotState != "active";
/// <summary>The switch tripped by itself in a world (SPEC 9.2): it waits for the user's restart, like "Ferma tutto".</summary>
public bool WaitsForRestart => BotState == "stopped" || Tripped.Count > 0;
public static async Task<RiskView> LoadAsync(AppServices services, IClock clock, TimeZoneInfo zone, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(services);
var engine = services.Trading?.Risk ?? new RiskEngine(services.Db, clock, new Ledger(services.Db, clock),
new StageBook(services.Db, clock, TradingConsent.None, (_, _, _) => Task.FromResult(false)), services.Settings, new FxStore(services.Db, clock),
(_, _) => Task.FromResult<Instrument?>(null), (_, _) => Task.FromResult<Quote?>(null));
var limits = await RiskSettings.LoadAsync(services.Settings, cancellationToken).ConfigureAwait(false);
var accounts = await AccountsView.LoadAsync(services, clock, zone, cancellationToken).ConfigureAwait(false);
var world = accounts.Accounts.Any(a => TradingEnvironments.World(a.Environment) == Worlds.Real) ? Worlds.Real : Worlds.Simulated;
var risk = await engine.WorldAsync(world, cancellationToken).ConfigureAwait(false);
// Both worlds: a switch tripped in the simulation must show even when the page measures the real one.
var tripped = new List<WorldRisk>();
foreach (var w in new[] { Worlds.Real, Worlds.Simulated })
{
var each = w == world ? risk : await engine.WorldAsync(w, cancellationToken).ConfigureAwait(false);
if (each.State.Tripped)
{
tripped.Add(each);
}
}
// Weights, crypto and the risk of a trade are measured on the reference equity, as the risk engine does (SPEC 9.2).
var reference = await engine.ReferenceAsync(cancellationToken).ConfigureAwait(false);
var bot = (await services.Settings.GetAsync(SettingsStore.BotState.Key, cancellationToken).ConfigureAwait(false)).Value;
var now = clock.UtcNow;
var (index, sent) = await services.Db.ReadAsync(async s =>
{
var i = await s.QueryAsync("SELECT taken_at, twr_index FROM equity_snapshots WHERE scope = 'total' AND world = @w AND twr_index IS NOT NULL ORDER BY taken_at, id",
r => new IndexPoint(r.Moment("taken_at"), r.Money("twr_index")), ("w", world)).ConfigureAwait(false);
var o = await s.QueryAsync(
"SELECT o.account_id, COUNT(*) AS n FROM orders o WHERE o.submitted_at >= @f AND o.purpose IN ('entry', 'exit') GROUP BY o.account_id",
r => (Account: r.Id("account_id"), Count: r.Number("n")), ("f", now - TimeSpan.FromMinutes(1))).ConfigureAwait(false);
return (i, o);
}, cancellationToken).ConfigureAwait(false);
var shown = accounts.Accounts.Select(a => a.Id).ToHashSet();
var equity = accounts.TotalEur;
var positions = accounts.Positions;
var biggest = positions.MaxBy(p => p.ValueEur);
var crypto = positions.Where(p => p.Broker == Venues.Okx).Sum(p => p.ValueEur);
decimal Rate(PositionRow p) => p.Last == 0m ? 0m : p.ValueEur / (p.Quantity * p.Last);
var atRisk = positions.Select(p => p.Stop is { } stop ? Math.Max(0m, p.ValueEur - (p.Quantity * stop * Rate(p))) : p.ValueEur).ToList();
var perMinute = sent.Where(x => shown.Contains(x.Account)).Select(x => x.Count).DefaultIfEmpty(0).Max();
string Pct(decimal fraction, int decimals = 2, bool sign = false) => Format.Percent(fraction * 100m, decimals, sign);
decimal? Use(decimal value, decimal limit) => limit <= 0m ? null : Math.Clamp(value / limit, 0m, 1m);
var m = risk.Measures;
var rows = new List<LimitRow>
{
new("Perdita in un giorno", m.DayChange >= 0m ? "nessuna" : Pct(m.DayChange), m.DayChange >= 0m ? $"oggi {Pct(m.DayChange, 2, true)}" : null,
Use(-m.DayChange, limits.DailyLoss), Pct(-limits.DailyLoss), "Niente nuovi ingressi per 24 ore", RiskSettings.DailyLoss.Key),
new("Perdita in una settimana", m.WeekChange >= 0m ? "nessuna" : Pct(m.WeekChange), m.WeekChange >= 0m ? $"{Pct(m.WeekChange, 2, true)}" : null,
Use(-m.WeekChange, limits.WeeklyLoss), Pct(-limits.WeeklyLoss), "Rischio dimezzato per sette giorni", RiskSettings.WeeklyLoss.Key),
new("Calo dal massimo", Pct(m.Drawdown), Worst(index, risk.State.PeakFrom) is { } worst && worst < m.Drawdown ? $"peggiore {Pct(worst)}" : null,
Use(-m.Drawdown, limits.DrawdownStop), $"{Pct(-limits.DrawdownHalf, 0)} e {Pct(-limits.DrawdownStop, 0)}",
$"A {Pct(-limits.DrawdownHalf, 0)} dimezzo il rischio. A {Pct(-limits.DrawdownStop, 0)} fermo tutto e aspetto te", RiskSettings.DrawdownStop.Key),
new("Capitale investito", equity > 0m ? Pct(accounts.InvestedEur / equity, 1) : "—", null, equity > 0m ? Use(accounts.InvestedEur / equity, limits.MaxInvested) : null,
Pct(limits.MaxInvested, 0), "Non compro altro: non uso mai soldi presi in prestito", RiskSettings.MaxInvested.Key),
new("Peso di una sola posizione", biggest is null || reference <= 0m ? "—" : Pct(biggest.ValueEur / reference, 1), biggest?.Symbol,
biggest is null || reference <= 0m ? 0m : Use(biggest.ValueEur / reference, limits.MaxPosition), Pct(limits.MaxPosition, 0), "Riduco la posizione fino alla soglia",
RiskSettings.MaxPosition.Key),
new("Crypto sul totale", reference > 0m ? Pct(crypto / reference, 1) : "—", null, reference > 0m ? Use(crypto / reference, limits.MaxCrypto) : null, Pct(limits.MaxCrypto, 0),
"Niente nuovi acquisti di crypto", RiskSettings.MaxCrypto.Key),
new("Rischio di una singola operazione", atRisk.Count == 0 || reference <= 0m ? "—" : Pct(atRisk.Average() / reference), atRisk.Count == 0 ? null : "in media, oggi fino allo stop",
atRisk.Count == 0 || reference <= 0m ? 0m : Use(atRisk.Average() / reference, limits.MaxTradeRisk), Pct(limits.MaxTradeRisk), "All'acquisto rimpicciolisco l'ordine prima di inviarlo",
RiskSettings.MaxTradeRisk.Key),
new("Ordini al minuto", perMinute.ToString(System.Globalization.CultureInfo.InvariantCulture), null, Use(perMinute, limits.OrdersPerMinute),
limits.OrdersPerMinute.ToString(System.Globalization.CultureInfo.InvariantCulture), "Mi fermo e ti avviso: può essere un guasto", RiskSettings.OrdersPerMinute.Key),
new("Età dei prezzi", "controllata a ogni ordine", null, null, $"{(int)limits.MaxPriceAge.TotalSeconds} secondi", "Non apro posizioni finché i dati non tornano freschi",
RiskSettings.MaxPriceAge.Key),
};
var drawdown = new List<DrawdownPoint>();
var peak = 0m;
foreach (var p in index.Where(p => risk.State.PeakFrom is null || p.At >= risk.State.PeakFrom))
{
peak = Math.Max(peak, p.Index);
drawdown.Add(new DrawdownPoint(p.At, peak > 0m ? (p.Index / peak) - 1m : 0m));
}
var scenarioText = (await services.Settings.GetAsync(Anubis.Risk.Scenarios.Setting.Key, cancellationToken).ConfigureAwait(false)).Value;
var dollars = accounts.Accounts.Where(a => a.Currency == "USD").Sum(a => a.Eur(a.Equity));
var scenarios = Anubis.Risk.Scenarios.Compute(Anubis.Risk.Scenarios.Parse(scenarioText),
[.. positions.Select(p => new ScenarioPosition(p.Broker == Venues.Okx, p.ValueEur, p.Stop is { } stop ? p.Quantity * stop * Rate(p) : null))], dollars, equity);
var calendar = await new CalendarEventStore(services.Db, clock).BetweenAsync(now.AddDays(-14), now.AddDays(14), cancellationToken).ConfigureAwait(false);
var windows = calendar.Select(e => (Event: e, Window: CalendarBlocks.Window(e))).Where(x => x.Window is not null)
.Select(x => (x.Window!.Value.From, x.Window.Value.To, x.Event.Title)).OrderBy(x => x.From).ToList();
(DateTimeOffset, DateTimeOffset, string)? next = windows.FirstOrDefault(w => w.To > now) is { Title: not null } n ? n : null;
(DateTimeOffset, DateTimeOffset, string)? last = windows.LastOrDefault(w => w.To <= now) is { Title: not null } l ? l : null;
return new RiskView(bot, engine.StopRequested, (await engine.BotAsync(cancellationToken).ConfigureAwait(false)).CloseAll, world, risk, limits, rows, drawdown, scenarios,
await engine.EventsAsync(8, cancellationToken).ConfigureAwait(false), next, last, await RiskSettings.PdtGuardAsync(services.Settings, cancellationToken).ConfigureAwait(false),
positions.Count > 0, tripped);
}
private static decimal? Worst(IReadOnlyList<IndexPoint> index, DateTimeOffset? from)
{
decimal? worst = null;
var peak = 0m;
foreach (var p in index.Where(p => from is null || p.At >= from))
{
peak = Math.Max(peak, p.Index);
var d = peak > 0m ? (p.Index / peak) - 1m : 0m;
worst = worst is null ? d : Math.Min(worst.Value, d);
}
return worst;
}
}
+13 -1
View File
@@ -4,6 +4,7 @@ using Anubis.Core;
using Anubis.Demo;
using Anubis.Platform;
using Anubis.Server.Components;
using Anubis.Supervision;
using Microsoft.AspNetCore.Authentication;
using Microsoft.AspNetCore.DataProtection;
@@ -52,8 +53,12 @@ internal static class ServerApp
services.AddSingleton(clock);
services.AddSingleton(overrides ?? new ServerOverrides());
services.AddSingleton<EventBus>();
// Started by the guard (SPEC 9.3): the channel to it. Never in demo mode, which protects nothing.
services.AddSingleton(sp => new AnubisHost(options, config, environment, clock, sp.GetRequiredService<EventBus>(),
sp.GetRequiredService<ILogger<AnubisHost>>(), overrides));
sp.GetRequiredService<ILogger<AnubisHost>>(), overrides)
{
Guard = options.Demo ? null : GuardLink.FromEnvironment(environment, sp.GetRequiredService<ILogger<GuardLink>>()),
});
services.AddHostedService(sp => sp.GetRequiredService<AnubisHost>());
services.AddSingleton(new LoginLimiter(clock));
services.AddSingleton<ShellData>();
@@ -82,6 +87,13 @@ internal static class ServerApp
var app = builder.Build();
var host = app.Services.GetRequiredService<AnubisHost>();
if (host.Guard is { } guard)
{
// The guard asks for an orderly stop when the container stops; every stop of the engine is announced to it.
guard.StopRequested = app.Lifetime.StopApplication;
app.Lifetime.ApplicationStopping.Register(() => guard.AnnounceAsync("uscita").GetAwaiter().GetResult());
app.Lifetime.ApplicationStopped.Register(() => guard.DisposeAsync().AsTask().GetAwaiter().GetResult());
}
app.Use(SecurityHeaders);
app.Use((context, next) => LocalOnly(context, next, options, config));
+3
View File
@@ -76,6 +76,9 @@ internal sealed record ServerOverrides
public Func<Anubis.Brokers.IWebSocketTransport>? MarketSockets { get; init; }
/// <summary>Answers the Telegram calls and the sign of life instead of the network; tests without it have no Telegram.</summary>
public HttpMessageHandler? TelegramHttp { get; init; }
/// <summary>True starts the accounts and the order managers in a test; off by default in tests.</summary>
public bool? Trading { get; init; }
}
+77
View File
@@ -0,0 +1,77 @@
using System.Collections;
using System.Globalization;
using Anubis.Core;
using Anubis.Db;
using Anubis.Supervision;
namespace Anubis.Server;
/// <summary>
/// "anubis soak": the report of an endurance run in real time (SPEC 17, phase 6) from the bot's own records. It only
/// reads, so it can run while the bot works: in Docker with <c>docker exec</c>, on desktop from a second terminal.
/// </summary>
internal static class SoakCommand
{
private const string Usage =
"Uso: anubis soak [--from AAAA-MM-GGTHH:MMZ] [--to AAAA-MM-GGTHH:MMZ] [--hours 48] [--out file.md]\n" +
" senza --from la prova è delle ultime --hours ore fino a --to (o adesso).\n" +
"Opzioni comuni: [--platform docker|desktop] [--data-dir <cartella>]";
/// <summary>0 when the run is in order, 4 when something must be looked at, 1 when the report could not be made.</summary>
public static async Task<int> RunAsync(string[] args, TextWriter? output = null, IReadOnlyDictionary<string, string?>? environment = null, IClock? clock = null)
{
output ??= Console.Out;
clock ??= new SystemClock();
var own = new Dictionary<string, string>(StringComparer.Ordinal);
var rest = new List<string>();
for (var i = 0; i < args.Length; i++)
{
if (args[i] is "--from" or "--to" or "--hours" or "--out" && i + 1 < args.Length)
{
own[args[i]] = args[++i];
}
else if (args[i] is "--help" or "-h")
{
await output.WriteLineAsync(Usage).ConfigureAwait(false);
return 2;
}
else
{
rest.Add(args[i]);
}
}
environment ??= Environment.GetEnvironmentVariables().Cast<DictionaryEntry>().ToDictionary(e => (string)e.Key, e => e.Value as string, StringComparer.Ordinal);
try
{
var hours = own.TryGetValue("--hours", out var h) ? double.Parse(h, CultureInfo.InvariantCulture) : 48;
var to = own.TryGetValue("--to", out var t) ? DateTimeOffset.Parse(t, CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal) : clock.UtcNow;
var from = own.TryGetValue("--from", out var f) ? DateTimeOffset.Parse(f, CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal) : to - TimeSpan.FromHours(hours);
var options = ServerOptions.Parse([.. rest], environment);
var config = options.LoadConfig(environment);
if (!config.IsComplete)
{
await output.WriteLineAsync("La configurazione d'avvio non è completa: avvia prima il server una volta.").ConfigureAwait(false);
return 1;
}
Redactor.Register(config.Db!.Password);
await using var db = Database.Open(config.Db!, allowCreate: false);
var report = await SoakReporter.BuildAsync(db, from, to, TimeSpan.FromHours(hours), config.TimeZone, CancellationToken.None).ConfigureAwait(false);
var text = report.Markdown(config.TimeZone);
if (own.TryGetValue("--out", out var file))
{
await File.WriteAllTextAsync(Path.GetFullPath(file), text).ConfigureAwait(false);
}
await output.WriteAsync(text).ConfigureAwait(false);
return report.Passed ? 0 : 4;
}
catch (Exception ex) when (ex is IOException or FormatException or ArgumentException or InvalidOperationException or SchemaException
or System.Data.Common.DbException or UnauthorizedAccessException or TimeZoneNotFoundException)
{
await output.WriteLineAsync("Non riuscito: " + Redactor.Redact(ex.Message)).ConfigureAwait(false);
return 1;
}
}
}
+210
View File
@@ -0,0 +1,210 @@
using Anubis.Backtest;
using Anubis.Core;
using Anubis.Data;
using Anubis.Strategies;
using Anubis.Trading;
namespace Anubis.Server;
/// <summary>A strategy of the Strategie screen: its stage, results, requirements and what the user can do next.</summary>
internal sealed record StrategyCard(
StrategyStage Stage,
string Name,
string Description,
IReadOnlyList<string> Rules,
string Account,
string Currency,
decimal CapitalEur,
decimal ResultEur,
decimal ResultPercent,
decimal WorstFall,
int Closed,
int Winners,
IReadOnlyList<string> Open,
IReadOnlyList<(DateTimeOffset At, decimal Index)> Curve,
IReadOnlyList<Requirement> Requirements,
string Phrase,
int? DaysToGo,
IReadOnlyList<string> Summary,
decimal? TwinGap)
{
public string Color => "var(--s-" + Stage.Strategy.Replace('_', '-') + ")";
public bool AllMet => Requirements.All(r => r.Met);
}
/// <summary>
/// The Strategie screen (SPEC 12.3, 13.6, mockup 04): the path to real money in four stages, a card for each base
/// strategy with its requirements, and the ones in observation with the reason.
/// </summary>
internal sealed record StrategiesView(IReadOnlyList<StrategyCard> Cards, bool LiveAllowed, IReadOnlyList<(Guid Id, string Name, string Broker)> LiveAccounts)
{
/// <summary>What each base strategy does, in four lines (SPEC 8).</summary>
public static readonly IReadOnlyDictionary<string, IReadOnlyList<string>> Rules = new Dictionary<string, IReadOnlyList<string>>(StringComparer.Ordinal)
{
[StrategyNames.CryptoTrend] =
[
"Guarda le 10 monete in euro più scambiate una volta al giorno, dopo la chiusura delle 00:00 UTC",
"Entra quando il prezzo segna nuovi massimi su più periodi, da 5 a 360 giorni",
"Più una moneta è agitata, meno ne compra",
"Mai a leva, mai al ribasso",
],
[StrategyNames.EquityMomentum] =
[
"Rifà la classifica ogni lunedì e tiene fino a 12 titoli",
"Vende chi esce dai primi 30 o tocca lo stop",
"Compra solo quando il mercato nel suo insieme sale",
"Solo azioni singole, nessun ETF americano",
],
[StrategyNames.EquityMeanrev] =
[
"Solo titoli sopra la loro media a 200 giorni",
"Esce al rimbalzo, oppure dopo 5 sedute",
"Salta i giorni prima delle trimestrali",
"Mai uscita nello stesso giorno dell'acquisto",
],
};
public static async Task<StrategiesView> LoadAsync(AppServices services, IClock clock, bool liveAllowed, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(services);
var stages = services.Trading?.Stages ?? new StageBook(services.Db, clock, TradingConsent.None, (_, _, _) => Task.FromResult(false));
var ledger = services.Trading?.Ledger ?? new Ledger(services.Db, clock);
var rate = (await new FxStore(services.Db, clock).LatestAsync("EUR", "USD", cancellationToken).ConfigureAwait(false))?.Rate ?? 0m;
var accounts = await services.Db.ReadAsync(s => s.QueryAsync("SELECT id, name, broker, environment, base_currency FROM accounts",
r => (Id: r.Id("id"), Name: r.Text("name"), Broker: r.Text("broker"), Environment: r.Text("environment"), Currency: r.Text("base_currency"))), cancellationToken)
.ConfigureAwait(false);
var equities = await ledger.EquitiesAsync(cancellationToken).ConfigureAwait(false);
var positions = await ledger.PositionsAsync(null, cancellationToken).ConfigureAwait(false);
var limits = await Anubis.Risk.RiskSettings.LoadAsync(services.Settings, cancellationToken).ConfigureAwait(false);
decimal? reference = services.Trading?.Risk is { } risk ? await risk.ReferenceAsync(cancellationToken).ConfigureAwait(false) : null;
async Task<string> SymbolAsync(Guid instrument) =>
await services.Db.ReadAsync(s => s.FirstOrDefaultAsync("SELECT symbol FROM instruments WHERE id = @id", r => r.Text("symbol"), ("id", instrument)), cancellationToken)
.ConfigureAwait(false) ?? "?";
var cards = new List<StrategyCard>();
foreach (var stage in await stages.AllAsync(cancellationToken).ConfigureAwait(false))
{
if (!StrategyNames.Base.Any(b => b.Name == stage.Strategy))
{
continue;
}
var family = StrategyCatalog.Find(stage.Strategy);
var ledgerAccount = stage.OrdersAccount ?? stage.AccountId;
var account = accounts.FirstOrDefault(a => a.Id == ledgerAccount);
var usd = account.Currency == "USD";
decimal Eur(decimal amount) => usd ? (rate > 0m ? amount / rate : 0m) : amount;
var equity = equities.FirstOrDefault(e => e.AccountId == ledgerAccount && e.Strategy == stage.Strategy);
// As on Conti: each position is valued to the cent before the sum, so both pages show the same figure.
var held = positions.Where(p => p.AccountId == ledgerAccount && p.Strategy == stage.Strategy).Sum(p => Math.Round(p.Value, 2, MidpointRounding.AwayFromZero));
var result = equity is null ? 0m : equity.Cash + held - equity.Flows;
var since = stage.StageSince ?? DateTimeOffset.MinValue;
var (curve, trades) = await services.Db.ReadAsync(async s =>
{
var c = await s.QueryAsync(
"SELECT taken_at, twr_index FROM equity_snapshots WHERE scope = 'strategy' AND account_id = @a AND strategy = @s AND twr_index IS NOT NULL ORDER BY taken_at, id",
r => (At: r.Moment("taken_at"), Index: r.Money("twr_index")), ("a", ledgerAccount), ("s", stage.Strategy)).ConfigureAwait(false);
var t = await s.QueryAsync("SELECT realized FROM trades WHERE account_id = @a AND strategy = @s AND status = 'closed'", r => r.Money("realized"),
("a", ledgerAccount), ("s", stage.Strategy)).ConfigureAwait(false);
return (c, t);
}, cancellationToken).ConfigureAwait(false);
var peak = 0m;
var worst = 0m;
foreach (var p in curve)
{
peak = Math.Max(peak, p.Index);
worst = Math.Min(worst, peak > 0m ? (p.Index / peak) - 1m : 0m);
}
var symbols = new List<string>();
foreach (var p in positions.Where(p => p.AccountId == ledgerAccount && p.Strategy == stage.Strategy))
{
symbols.Add(await SymbolAsync(p.InstrumentId).ConfigureAwait(false));
}
// What the confirmation of SPEC 12.4 changes: the capital used, the purchases of the first cycle (estimated
// from what the simulation holds today) and the limits with their base.
var summary = new List<string>();
if (stage.Stage is StrategyStages.Simulation or StrategyStages.Reduced)
{
var toReduced = stage.Stage == StrategyStages.Simulation;
var factor = toReduced ? StrategyStages.ReducedShare : 1m;
var simEquity = equities.FirstOrDefault(e => e.AccountId == stage.AccountId && e.Strategy == stage.Strategy);
var simUsd = accounts.FirstOrDefault(a => a.Id == stage.AccountId).Currency == "USD";
decimal SimEur(decimal amount) => simUsd ? (rate > 0m ? amount / rate : 0m) : amount;
summary.Add(toReduced
? $"Capitale usato: un quarto di quello assegnato, {Format.Money(SimEur(simEquity?.Flows ?? 0m) * factor, decimals: 0)}."
: $"Capitale usato: tutto quello assegnato, {Format.Money(SimEur(simEquity?.Flows ?? 0m), decimals: 0)}.");
var buys = new List<string>();
foreach (var p in positions.Where(p => p.AccountId == stage.AccountId && p.Strategy == stage.Strategy && p.Quantity > 0m))
{
buys.Add($"{await SymbolAsync(p.InstrumentId).ConfigureAwait(false)} per circa {Format.Money(SimEur(p.Value) * factor, decimals: 0)}");
}
summary.Add(buys.Count == 0
? "Acquisti al primo ciclo: nessuno per ora, la simulazione non tiene posizioni."
: (toReduced ? "Acquisti al primo ciclo, stimati da ciò che la simulazione tiene oggi: " : "Al primo ciclo le posizioni vere crescono fino alla misura piena: ") +
string.Join(", ", buys) + ".");
summary.Add("Limiti che valgono: perdita in un giorno " + Format.Percent(limits.DailyLoss * 100m, 1) + ", calo dal massimo " +
Format.Percent(limits.DrawdownHalf * 100m, 0) + " e " + Format.Percent(limits.DrawdownStop * 100m, 0) + ", una posizione " +
Format.Percent(limits.MaxPosition * 100m, 0) + ", rischio di un'operazione " + Format.Percent(limits.MaxTradeRisk * 100m, 1) +
", misurati sul patrimonio di riferimento" + (reference is { } r ? $" (oggi {Format.Money(r, decimals: 0)})" : "") +
(toReduced ? ", e in reale ridotto su un quarto di esso." : "."));
}
// The simulated twin against the real account (SPEC 12.3): the gap of results, as a share of the capital used.
decimal? twinGap = null;
if (stage.HasTwin && stage.AccountId is { } twinAccount &&
equities.FirstOrDefault(e => e.AccountId == twinAccount && e.Strategy == stage.Strategy) is { Flows: > 0m } twin && equity is { Flows: > 0m } real)
{
twinGap = ((real.Equity - real.Flows) / real.Flows * 100m) - ((twin.Equity - twin.Flows) / twin.Flows * 100m);
}
var requirements = await stages.RequirementsAsync(stage.Strategy, cancellationToken).ConfigureAwait(false);
var next = stage.Stage == StrategyStages.Reduced ? StrategyStages.Real : StrategyStages.Reduced;
var days = requirements.FirstOrDefault(r => r.Key == "days");
int? toGo = days is { Met: false } && stage.StageSince is { } from
? Math.Max(1, (int)Math.Ceiling(((stage.Stage == StrategyStages.Reduced ? StageBook.ReducedDays : StageBook.SimulationDays) - (clock.UtcNow - from).TotalDays)))
: null;
var flows = equity?.Flows ?? 0m;
cards.Add(new StrategyCard(stage, StrategyNames.Label(stage.Strategy, withMarket: true) + (stage.Strategy == StrategyNames.CryptoTrend ? "" : " USA"),
family?.Description ?? "", Rules.GetValueOrDefault(stage.Strategy) ?? [], account.Name ?? "—", account.Currency ?? "EUR", Eur(flows), Eur(result),
flows > 0m ? result / flows * 100m : 0m, worst, trades.Count, trades.Count(r => r >= 0m), symbols, [.. curve.Where(p => p.At >= since || stage.StageSince is null)],
requirements, StageBook.Phrase(stage.Strategy, next), toGo, summary, twinGap));
}
var live = accounts.Where(a => a.Environment == TradingEnvironments.Live).Select(a => (a.Id, a.Name, a.Broker)).ToList();
return new StrategiesView([.. cards.OrderBy(c => StrategyNames.Base.ToList().FindIndex(b => b.Name == c.Stage.Strategy))], liveAllowed, live);
}
/// <summary>A requirement not met yet only because time has to pass or nothing could be measured so far.</summary>
public static bool IsPending(Requirement requirement)
{
ArgumentNullException.ThrowIfNull(requirement);
return !requirement.Met && (requirement.Key is "days" or "trades" || requirement.Value is "da misurare" or "nessuna esecuzione");
}
/// <summary>Why a strategy is in observation, in words: the outcome of its backtest, or the user's choice.</summary>
public static string ObservationReason(StrategyStage stage)
{
ArgumentNullException.ThrowIfNull(stage);
if (stage.Qualification is { Passed: false } q)
{
return q.Verdict;
}
if (stage.Qualification is null)
{
return "Il backtest non è ancora stato fatto: finché manca resta in osservazione.";
}
return stage.StageReason switch
{
"user" or null => "Superato il backtest: puoi portarla in simulazione quando vuoi.",
var r => "Tornata in osservazione: " + ReasonTexts.Text(r) + ".",
};
}
/// <summary>The report of a strategy's backtest in the repository, for the link.</summary>
public static string? ReportOf(StrategyStage stage) => stage.Qualification?.Report ?? BacktestOutcomes.Find(stage.Strategy)?.Report;
}
+332
View File
@@ -0,0 +1,332 @@
using System.Diagnostics;
using System.Globalization;
using System.Text.Json;
using Anubis.Core;
using Anubis.Data;
using Anubis.Db;
using Anubis.Platform;
namespace Anubis.Server;
/// <summary>How a component is doing: working, off by choice or not there yet, or in need of a look.</summary>
internal enum PartState
{
Ok,
Off,
Warn,
}
/// <summary>A component of the system with a line on how it is doing.</summary>
internal sealed record SystemPart(string Name, string Detail, PartState State);
/// <summary>A resource of the computer: what is used, out of how much, as a share for the bar.</summary>
internal sealed record HostResource(string Label, string Value, string? Of, decimal Share);
/// <summary>A job in the scheduler, with its time as people say it ("15:35", "Dom 03:00").</summary>
internal sealed record PlannedJob(string Time, string Text);
/// <summary>
/// The "Diario e sistema" right column and the Sistema tile of the Panoramica (SPEC 13.6, mockups 01 and 08): the
/// components with their state, the jobs in the scheduler, the resources of the computer, version and platform.
/// In demo mode the values of appendix F.10 (SPEC 13.1); otherwise the real services, process and disk.
/// </summary>
internal sealed record SystemView(
IReadOnlyList<SystemPart> Parts,
IReadOnlyList<SystemPart> Tiles,
IReadOnlyList<PlannedJob> Jobs,
IReadOnlyList<HostResource> Resources,
string Running)
{
private static readonly CultureInfo _it = CultureInfo.InvariantCulture;
private static readonly string[] _days = ["Dom", "Lun", "Mar", "Mer", "Gio", "Ven", "Sab"];
private static readonly string[] _months = ["gennaio", "febbraio", "marzo", "aprile", "maggio", "giugno", "luglio", "agosto", "settembre", "ottobre", "novembre", "dicembre"];
/// <summary>Nothing needs a look; components that are off by choice or not active yet do not count.</summary>
public bool AllWell => Parts.All(p => p.State != PartState.Warn);
public static async Task<SystemView> LoadAsync(AnubisHost host, ShellData shell, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(host);
ArgumentNullException.ThrowIfNull(shell);
using var aggregates = shell.Aggregates();
return aggregates is not null ? Demo(aggregates.RootElement, shell) : await RealAsync(host, shell, cancellationToken).ConfigureAwait(false);
}
private static SystemView Demo(JsonElement root, ShellData shell)
{
var system = root.GetProperty("sistema");
var accounts = root.GetProperty("conti").EnumerateArray().ToList();
var okxMs = accounts[0].GetProperty("risposta_ms").GetInt32();
var alpacaMs = accounts[1].GetProperty("risposta_ms").GetInt32();
var database = root.GetProperty("database").GetString()!;
var size = Format.Number(Dec(system, "database_gb"), 1) + " GB";
var telegram = system.GetProperty("telegram_ultimo").GetString();
var parts = new List<SystemPart>
{
new("Motore di trading", $"attivo, un controllo ogni {system.GetProperty("controllo_secondi").GetInt32()} secondi", PartState.Ok),
new("Prezzi da OKX Europe", $"in diretta, risposta in {okxMs} ms", PartState.Ok),
new("Prezzi da Alpaca", "collegato, " + Market(shell), PartState.Ok),
new("Lettura delle notizie", $"{system.GetProperty("notizie_fonti").GetInt32()} fonti, ultima lettura alle {system.GetProperty("notizie_ultima").GetString()}", PartState.Ok),
new("Database " + database, $"{size}, ultimo salvataggio alle {system.GetProperty("ultimo_salvataggio").GetString()}", PartState.Ok),
new("Telegram", $"collegato, ultimo messaggio alle {telegram}", PartState.Ok),
};
var tiles = new List<SystemPart>
{
new("OKX Europe", $"connesso, {okxMs} ms", PartState.Ok),
new("Alpaca", $"connesso, {alpacaMs} ms", PartState.Ok),
new("Prezzi di mercato", $"aggiornati {root.GetProperty("dati_aggiornati_secondi").GetInt32()} s fa", PartState.Ok),
new("Notizie", $"{root.GetProperty("notizie").GetProperty("lette_24h").GetInt32()} lette in 24 ore", PartState.Ok),
new("Database", $"{database}, {size}", PartState.Ok),
new("Telegram", $"ultimo messaggio alle {telegram}", PartState.Ok),
};
var jobs = system.GetProperty("lavori").EnumerateArray()
.Select(j => new PlannedJob(j.GetProperty("ora").GetString()!, j.GetProperty("testo").GetString()!)).ToList();
var memory = Dec(system, "memoria_gb");
var memoryTotal = system.GetProperty("memoria_totale_gb").GetDecimal();
var disk = Dec(system, "disco_gb");
var diskTotal = system.GetProperty("disco_totale_gb").GetDecimal();
var resources = new List<HostResource>
{
new("Processore", system.GetProperty("cpu_pct").GetInt32().ToString(_it) + "%", null, system.GetProperty("cpu_pct").GetDecimal() / 100m),
new("Memoria", Format.Number(memory, 1) + " GB", Format.Number(memoryTotal), memory / memoryTotal),
new("Spazio per i dati", Format.Number(disk, 1) + " GB", Format.Number(diskTotal), disk / diskTotal),
};
var started = DateOnly.ParseExact(root.GetProperty("avvio").GetString()!, "yyyy-MM-dd", _it);
var running = $"Anubis {root.GetProperty("versione").GetString()} in {root.GetProperty("piattaforma").GetString()}, acceso dal {started.Day} {_months[started.Month - 1]}: " +
$"{root.GetProperty("giorno").GetInt32()} giorni senza interruzioni.";
return new SystemView(parts, tiles, jobs, resources, running);
}
private static async Task<SystemView> RealAsync(AnubisHost host, ShellData shell, CancellationToken cancellationToken)
{
var services = host.Services!;
var now = shell.Now;
var zone = shell.Zone;
var parts = new List<SystemPart>();
var tiles = new List<SystemPart>();
// The trading engine, and the state the user chose for the bot.
var bot = (await services.Settings.GetAsync(SettingsStore.BotState.Key, cancellationToken).ConfigureAwait(false)).Value;
var trading = services.Trading;
parts.Add(trading switch
{
null => new SystemPart("Motore di trading", "spento: le strategie non lavorano", PartState.Off),
{ IsStarted: false } => new SystemPart("Motore di trading", "non è ancora partito: se non parte entro qualche minuto, guarda i registri tecnici", PartState.Warn),
_ when bot == "stopped" => new SystemPart("Motore di trading", "fermo con Ferma tutto: riparte solo con la tua conferma, dalla pagina Rischio", PartState.Warn),
_ => new SystemPart("Motore di trading",
$"{(bot == "paused" ? "in pausa" : "attivo")}, un controllo ogni {(int)trading.TickInterval.TotalSeconds} secondi", PartState.Ok),
});
// The two price feeds: the live OKX connection, the state of the Alpaca source.
var market = services.MarketData;
var sources = market is null ? [] : await new SourceStateStore(services.Db, host.Clock).ListAsync(cancellationToken).ConfigureAwait(false);
var okx = sources.FirstOrDefault(s => s.Source == MarketDataService.OkxSource);
var (okxPart, okxTile) = market switch
{
null => (new SystemPart("Prezzi da OKX Europe", "spenti", PartState.Off), new SystemPart("OKX Europe", "spento", PartState.Off)),
{ LiveConnected: true } => (new SystemPart("Prezzi da OKX Europe", "in diretta", PartState.Ok), new SystemPart("OKX Europe", "connesso", PartState.Ok)),
_ when Failing(okx) => (new SystemPart("Prezzi da OKX Europe", "non rispondono: riprovo da solo", PartState.Warn), new SystemPart("OKX Europe", "non risponde", PartState.Warn)),
_ => (new SystemPart("Prezzi da OKX Europe", "in collegamento", PartState.Off), new SystemPart("OKX Europe", "in collegamento", PartState.Off)),
};
parts.Add(okxPart);
var alpaca = sources.FirstOrDefault(s => s.Source == MarketDataService.AlpacaSource);
var (alpacaPart, alpacaTile) = (market, alpaca) switch
{
(null, _) => (new SystemPart("Prezzi da Alpaca", "spenti", PartState.Off), new SystemPart("Alpaca", "spento", PartState.Off)),
(_, null) => (new SystemPart("Prezzi da Alpaca", "in attesa del primo collegamento", PartState.Off), new SystemPart("Alpaca", "in attesa", PartState.Off)),
(_, { Enabled: false }) => (new SystemPart("Prezzi da Alpaca", "spenti: mancano le chiavi del conto di prova, in Impostazioni", PartState.Off),
new SystemPart("Alpaca", "senza chiavi", PartState.Off)),
_ when Failing(alpaca) => (new SystemPart("Prezzi da Alpaca", "non rispondono: riprovo da solo", PartState.Warn), new SystemPart("Alpaca", "non risponde", PartState.Warn)),
_ => (new SystemPart("Prezzi da Alpaca", "collegato, " + Market(shell, market.UsCalendar), PartState.Ok), new SystemPart("Alpaca", "connesso", PartState.Ok)),
};
parts.Add(alpacaPart);
tiles.Add(okxTile);
tiles.Add(alpacaTile);
var newest = market?.Freshness.Snapshot().Select(s => s.LastDataAt).Max();
tiles.Add(newest is { } at
? new SystemPart("Prezzi di mercato", "aggiornati " + Ago(now - at), now - at > TimeSpan.FromMinutes(5) ? PartState.Warn : PartState.Ok)
: new SystemPart("Prezzi di mercato", "in attesa dei primi prezzi", PartState.Off));
// News arrive with phase 7 (SPEC 10).
parts.Add(new SystemPart("Lettura delle notizie", "non ancora attiva", PartState.Off));
tiles.Add(new SystemPart("Notizie", "non ancora attive", PartState.Off));
// The database: engine, size, last backup.
var engine = services.Db.Engine == DbEngine.Postgres ? "PostgreSQL" : "SQLite";
var bytes = await SizeAsync(services.Db, host.Config.Db, cancellationToken).ConfigureAwait(false);
var backups = services.Backups.List();
var lastBackup = backups.Count == 0 ? null : (DateTimeOffset?)backups.Max(b => b.TakenAt);
var size = bytes is { } b ? Bytes(b) : null;
var saved = lastBackup is { } l ? "ultimo salvataggio " + When(l, now, zone) : "nessun salvataggio ancora";
parts.Add(new SystemPart("Database " + engine, (size is null ? "" : size + ", ") + saved, PartState.Ok));
tiles.Add(new SystemPart("Database", engine + (size is null ? "" : ", " + size), PartState.Ok));
// Telegram: the last message that really left (SPEC 14), never one only queued.
var telegram = services.Telegram;
var settings = telegram is null ? null : await telegram.ReadAsync(cancellationToken).ConfigureAwait(false);
if (telegram is null || settings is null)
{
parts.Add(new SystemPart("Telegram", "non configurato: si imposta in Impostazioni", PartState.Off));
tiles.Add(new SystemPart("Telegram", "non configurato", PartState.Off));
}
else
{
var summary = await telegram.Sender.SummaryAsync(cancellationToken).ConfigureAwait(false);
var last = summary.LastSentAt is { } sent ? "ultimo messaggio " + When(sent, now, zone) : "nessun messaggio inviato ancora";
var stuck = summary.Retrying + summary.GaveUp > 0;
parts.Add(new SystemPart("Telegram", stuck ? "alcuni messaggi non sono partiti: riprovo da solo; " + last : "collegato, " + last, stuck ? PartState.Warn : PartState.Ok));
tiles.Add(new SystemPart("Telegram", last, stuck ? PartState.Warn : PartState.Ok));
}
// Appointments only: the jobs that run every minute or every hour are background work.
var jobs = services.Scheduler.Upcoming(200).Where(u => !u.Periodic).Take(7)
.Select(u => new PlannedJob(JobTime(u.DueUtc, now, zone), u.Description)).ToList();
var since = host.Status.Since;
var platform = host.Config.Platform == PlatformKind.Docker ? "in Docker" : "come applicazione desktop";
var localSince = TimeZoneInfo.ConvertTime(since, zone);
var days = (int)Math.Floor((now - since).TotalDays);
var running = days >= 1
? $"Anubis {AnubisInfo.Version} {platform}, acceso dal {localSince.Day} {_months[localSince.Month - 1]}: {days} {(days == 1 ? "giorno" : "giorni")} senza interruzioni."
: $"Anubis {AnubisInfo.Version} {platform}, acceso oggi alle {localSince.ToString("HH:mm", _it)}.";
return new SystemView(parts, tiles, jobs, MeasureHost(host.Config.Data), running);
}
private static bool Failing(SourceState? state) =>
state is { LastError: not null } && (state.LastSuccessAt is null || state.LastRequestAt > state.LastSuccessAt);
private static string Market(ShellData shell, IMarketCalendar? calendar = null)
{
var (open, next) = AccountsView.MarketState(calendar, shell.Now);
var time = TimeZoneInfo.ConvertTime(next, shell.Zone).ToString("HH:mm", _it);
return open ? "mercato aperto fino alle " + time : "mercato chiuso fino alle " + time;
}
private static async Task<long?> SizeAsync(Database db, DbSettings? settings, CancellationToken cancellationToken)
{
try
{
if (db.Engine == DbEngine.Postgres)
{
return await db.ReadAsync(s => s.FirstOrDefaultAsync("SELECT pg_database_size(current_database()) AS n", r => (long?)r.Number("n")), cancellationToken)
.ConfigureAwait(false);
}
if (settings?.SqlitePath is { } path && File.Exists(path))
{
var wal = path + "-wal";
return new FileInfo(path).Length + (File.Exists(wal) ? new FileInfo(wal).Length : 0);
}
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or System.Data.Common.DbException)
{
// The size is a detail: without it the line still says engine and backups.
}
return null;
}
/// <summary>
/// Processor of this process (share of all cores since the last look, or since it started), its memory against
/// what the container or computer gives it, and the disk that holds the data folder.
/// </summary>
private static List<HostResource> MeasureHost(DataDirectory data)
{
var list = new List<HostResource>();
var cpu = CpuSampler.Share();
list.Add(new HostResource("Processore", Format.Number(cpu * 100m) + "%", null, cpu));
using var process = Process.GetCurrentProcess();
var used = process.WorkingSet64;
var total = GC.GetGCMemoryInfo().TotalAvailableMemoryBytes;
if (total > 0)
{
list.Add(new HostResource("Memoria", Bytes(used), Bytes(total, unit: false), Math.Clamp((decimal)used / total, 0m, 1m)));
}
try
{
var drive = new DriveInfo(Path.GetPathRoot(Path.GetFullPath(data.Root))!);
if (drive.IsReady && drive.TotalSize > 0)
{
var taken = drive.TotalSize - drive.AvailableFreeSpace;
list.Add(new HostResource("Spazio per i dati", Bytes(taken), Bytes(drive.TotalSize, unit: false), Math.Clamp((decimal)taken / drive.TotalSize, 0m, 1m)));
}
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or ArgumentException)
{
// A disk that cannot be measured is left out rather than shown wrong.
}
return list;
}
/// <summary>A size as people read it: "780 MB", "1,8 GB"; without the unit for the "su 50" after the value.</summary>
public static string Bytes(long bytes, bool unit = true)
{
const decimal giga = 1024m * 1024m * 1024m;
var gb = bytes / giga;
if (!unit)
{
return Format.Number(gb, gb >= 10m ? 0 : 1);
}
return gb >= 1m ? Format.Number(gb, 1) + " GB" : Format.Number(Math.Max(1m, bytes / (1024m * 1024m))) + " MB";
}
private static string When(DateTimeOffset moment, DateTimeOffset now, TimeZoneInfo zone)
{
var text = Format.ShortDate(moment, now, zone);
return text.StartsWith("Oggi ", StringComparison.Ordinal) ? "alle " + text["Oggi ".Length..] : text.StartsWith("Ieri ", StringComparison.Ordinal) ? "ieri alle " + text["Ieri ".Length..] : "del " + text;
}
private static string Ago(TimeSpan elapsed) => elapsed.TotalSeconds < 60 ? $"{Math.Max(0, (int)elapsed.TotalSeconds)} s fa" : Format.Ago(elapsed);
/// <summary>The time alone within a day, otherwise the day too: "15:35", "Dom 03:00".</summary>
public static string JobTime(DateTimeOffset due, DateTimeOffset now, TimeZoneInfo zone)
{
var local = TimeZoneInfo.ConvertTime(due, zone);
var time = local.ToString("HH:mm", _it);
return due - now < TimeSpan.FromHours(24) ? time : _days[(int)local.DayOfWeek] + " " + time;
}
private static decimal Dec(JsonElement element, string name) => decimal.Parse(element.GetProperty(name).GetString()!, _it);
/// <summary>The processor used by this process between two looks, as a share of all cores.</summary>
private static class CpuSampler
{
private static readonly Lock _gate = new();
private static (long Ticks, TimeSpan Cpu)? _last;
public static decimal Share()
{
using var process = Process.GetCurrentProcess();
var cpu = process.TotalProcessorTime;
var ticks = Stopwatch.GetTimestamp();
lock (_gate)
{
TimeSpan wall;
TimeSpan used;
if (_last is { } last && Stopwatch.GetElapsedTime(last.Ticks, ticks) >= TimeSpan.FromSeconds(1))
{
wall = Stopwatch.GetElapsedTime(last.Ticks, ticks);
used = cpu - last.Cpu;
}
else
{
// First look, or two looks too close together: the average since the process started.
wall = Uptime(process);
used = cpu;
}
_last = (ticks, cpu);
var share = wall <= TimeSpan.Zero ? 0d : used.TotalMilliseconds / (wall.TotalMilliseconds * Environment.ProcessorCount);
return Math.Clamp((decimal)share, 0m, 1m);
}
}
// A measure of the process, not a business time: the system clock against the process's own start, since the
// injected clock may stand still (demo) or run in another world (tests).
private static TimeSpan Uptime(Process process) => DateTime.UtcNow - process.StartTime.ToUniversalTime();
}
}
+176
View File
@@ -0,0 +1,176 @@
using System.Text;
using Anubis.Core;
using Anubis.Data;
using Anubis.Notifications;
using Anubis.Trading;
namespace Anubis.Server;
/// <summary>
/// The answers of the Telegram commands and the messages the bot writes by itself (SPEC 14), from the same data as
/// the Panoramica: one world, the simulated one until there is a real account.
/// </summary>
internal sealed class TelegramDesk(AnubisHost host) : ITelegramDesk
{
private const string By = "Telegram";
private AppServices Services => host.Services ?? throw new InvalidOperationException("Il bot sta ancora partendo: riprova tra poco.");
private TimeZoneInfo Zone => host.Config.TimeZone;
public async Task<string> StatusAsync(CancellationToken cancellationToken)
{
var services = Services;
var now = host.Clock.UtcNow;
var bot = (await services.Settings.GetAsync(SettingsStore.BotState.Key, cancellationToken).ConfigureAwait(false)).Value;
var overview = await Overview.LoadAsync(services, "oggi", now, Zone, cancellationToken).ConfigureAwait(false);
var text = new StringBuilder();
text.Append(bot switch { "paused" => "Bot in pausa", "stopped" => "Bot fermo («Ferma tutto»)", _ => "Bot attivo" })
.Append(" · solo denaro finto").AppendLine(".");
if (overview.HasData)
{
var today = overview.Cells[0];
text.Append("Patrimonio ").Append(Format.Money(overview.Points[^1].Equity)).Append(", oggi ").Append(Format.Money(today.Amount, withSign: true))
.Append(" (").Append(Format.Percent(today.Percent, 2, withSign: true)).AppendLine(").");
}
text.Append(Positions(overview.Positions, overview.PositionsWithoutStop));
if (overview.OpenOrders > 0)
{
text.Append(' ').Append(overview.OpenOrders == 1 ? "1 ordine in corso." : $"{overview.OpenOrders} ordini in corso.");
}
text.AppendLine();
if (overview.Next.Count > 0 && overview.Next[0] is var next)
{
text.Append("Prossima mossa: ").Append(next.Time).Append(' ').Append(next.Text).AppendLine(".");
}
return text.ToString().TrimEnd();
}
public async Task<string> PositionsAsync(CancellationToken cancellationToken)
{
var services = Services;
var rows = await services.Db.ReadAsync(s => s.QueryAsync(
"SELECT a.name, a.environment, a.base_currency, i.symbol, p.strategy, p.quantity, p.average_price, p.last_price, p.stop_price FROM positions p " +
"JOIN accounts a ON a.id = p.account_id JOIN instruments i ON i.id = p.instrument_id WHERE p.quantity > 0 ORDER BY a.name, i.symbol",
r => (Account: r.Text("name"), Environment: r.Text("environment"), Currency: r.Text("base_currency"), Symbol: r.Text("symbol"), Strategy: r.Text("strategy"),
Quantity: r.Money("quantity"), Average: r.Money("average_price"), Last: r.MoneyOrNull("last_price"), Stop: r.MoneyOrNull("stop_price"))), cancellationToken)
.ConfigureAwait(false);
var world = rows.Any(r => TradingEnvironments.World(r.Environment) == Worlds.Real) ? Worlds.Real : Worlds.Simulated;
rows = [.. rows.Where(r => TradingEnvironments.World(r.Environment) == world)];
if (rows.Count == 0)
{
return "Nessuna posizione aperta.";
}
var text = new StringBuilder();
foreach (var r in rows)
{
var last = r.Last ?? r.Average;
var gain = (last - r.Average) * r.Quantity;
text.Append(DiaryText.Shown(r.Symbol)).Append(": ").Append(Format.Quantity(r.Quantity)).Append(" a ").Append(Format.Money(r.Average, r.Currency))
.Append(", ora ").Append(Format.Money(last, r.Currency)).Append(" (").Append(Format.Money(gain, r.Currency, withSign: true)).Append("), ")
.Append(r.Stop is { } stop ? "stop a " + Format.Money(stop, r.Currency) : "SENZA STOP")
.Append(" · ").Append(StrategyNames.Label(r.Strategy)).Append(", ").Append(r.Account).AppendLine();
}
return text.ToString().TrimEnd();
}
public async Task<string> TodayAsync(CancellationToken cancellationToken)
{
var services = Services;
var now = host.Clock.UtcNow;
var local = TimeZoneInfo.ConvertTime(now, Zone);
var midnight = new DateTimeOffset(local.Date, local.Offset).ToUniversalTime();
var entries = await services.Db.ReadAsync(s => s.QueryAsync(
"SELECT occurred_at, message FROM journal WHERE kind = @k AND occurred_at >= @from ORDER BY occurred_at",
r => (At: r.Moment("occurred_at"), Text: r.Text("message")), ("k", DiaryKinds.Orders), ("from", midnight)), cancellationToken).ConfigureAwait(false);
var overview = await Overview.LoadAsync(services, "oggi", now, Zone, cancellationToken).ConfigureAwait(false);
var text = new StringBuilder();
if (overview.HasData)
{
text.Append("Oggi ").Append(Format.Money(overview.Cells[0].Amount, withSign: true)).Append(" (").Append(Format.Percent(overview.Cells[0].Percent, 2, withSign: true))
.AppendLine(").");
}
if (entries.Count == 0)
{
text.Append("Nessun ordine oggi.");
return text.ToString();
}
const int Shown = 15;
foreach (var e in entries.TakeLast(Shown))
{
text.Append(TimeZoneInfo.ConvertTime(e.At, Zone).ToString("HH:mm", System.Globalization.CultureInfo.InvariantCulture)).Append(' ').AppendLine(e.Text);
}
if (entries.Count > Shown)
{
text.Append("…e altre ").Append(entries.Count - Shown).Append(" righe nel Diario.");
}
return text.ToString().TrimEnd();
}
public Task<string> PauseAsync(CancellationToken cancellationToken) =>
CommandAsync("pausa", "In pausa: niente nuovi acquisti; stop e uscite proseguono. /riprendi per ripartire.", cancellationToken);
public Task<string> ResumeAsync(CancellationToken cancellationToken) => CommandAsync("riprendi", "Ripartito dalla pausa.", cancellationToken);
public Task<string> StopAllAsync(CancellationToken cancellationToken) =>
CommandAsync("ferma", "«Ferma tutto» fatto: ordini in attesa annullati, niente nuove posizioni, gli stop restano. Si riparte solo dall'interfaccia, pagina Rischio.",
cancellationToken);
/// <summary>The morning summary (SPEC 14): the night, the positions, what comes today.</summary>
public async Task<string> MorningAsync(CancellationToken cancellationToken)
{
var services = Services;
var now = host.Clock.UtcNow;
var overview = await Overview.LoadAsync(services, "oggi", now, Zone, cancellationToken).ConfigureAwait(false);
var text = new StringBuilder("Buongiorno.").AppendLine();
if (overview.HasData)
{
text.Append("Patrimonio ").Append(Format.Money(overview.Points[^1].Equity)).Append("; ultimi 7 giorni ").Append(Format.Money(overview.Cells[1].Amount, withSign: true))
.AppendLine(".");
}
text.AppendLine(Positions(overview.Positions, overview.PositionsWithoutStop));
var local = TimeZoneInfo.ConvertTime(now, Zone).Date;
var today = overview.Next.Count == 0 ? [] : services.Scheduler.Upcoming(200)
.Where(u => u.IsMove && TimeZoneInfo.ConvertTime(u.DueUtc, Zone).Date == local)
.Select(u => TimeZoneInfo.ConvertTime(u.DueUtc, Zone).ToString("HH:mm", System.Globalization.CultureInfo.InvariantCulture) + " " + u.Description)
.ToList();
if (today.Count > 0)
{
text.AppendLine("Oggi:");
foreach (var line in today)
{
text.AppendLine(line);
}
}
return text.ToString().TrimEnd();
}
private static string Positions(int count, int withoutStop) => count switch
{
0 => "Nessuna posizione aperta.",
1 => withoutStop == 0 ? "1 posizione, con il suo stop." : "1 posizione, SENZA STOP.",
_ => withoutStop == 0 ? $"{count} posizioni, tutte con lo stop." : $"{count} posizioni, {withoutStop} SENZA STOP.",
};
private async Task<string> CommandAsync(string command, string done, CancellationToken cancellationToken)
{
if (host.IsDemo)
{
return "Nella modalità dimostrativa il bot non si comanda.";
}
var (ok, message) = await Endpoints.BotCommandAsync(host, command, By, cancellationToken).ConfigureAwait(false);
return ok ? done : message;
}
}
+173
View File
@@ -0,0 +1,173 @@
using Anubis.Core;
using Anubis.Db;
using Anubis.Notifications;
namespace Anubis.Server;
/// <summary>
/// Telegram and the sign of life (SPEC 14): the queue sent every few seconds, the commands read as they arrive, the
/// call to the checking service every minute, the morning summary and the optional periodic status.
/// </summary>
internal sealed partial class TelegramServices : IDisposable
{
/// <summary>The secret that keeps the address of the checking service when the configuration does not give one.</summary>
public const string HeartbeatSecret = "HEARTBEAT_URL";
public static readonly TimeOnly MorningAt = new(8, 0);
public static readonly TimeOnly EveningAt = new(20, 0);
private readonly HttpClient _http;
private readonly Database _db;
private readonly IClock _clock;
private readonly SettingsStore _settings;
private readonly SecretStore _secrets;
private readonly TimeZoneInfo _zone;
private readonly Microsoft.Extensions.Logging.ILogger _logger;
public TelegramServices(AnubisHost host, Database db, IClock clock, SettingsStore settings, SecretStore secrets, string? configuredHeartbeat, HttpMessageHandler? handler,
Microsoft.Extensions.Logging.ILogger logger)
{
ArgumentNullException.ThrowIfNull(host);
_db = db;
_clock = clock;
_settings = settings;
_secrets = secrets;
_zone = host.Config.TimeZone;
_logger = logger;
_http = handler is null ? new HttpClient() : new HttpClient(handler, disposeHandler: false);
// Longer than the wait of the commands' long polling.
_http.Timeout = TimeSpan.FromSeconds(60);
Desk = new TelegramDesk(host);
Sender = new NotificationSender(db, clock, ReadAsync, Client);
Bot = new TelegramBot(clock, ReadAsync, Client, new TelegramCommands(Desk, clock), logger);
ConfiguredHeartbeat = configuredHeartbeat;
Heartbeat = new Heartbeat(_http, clock, HeartbeatAddressAsync, logger);
}
public TelegramDesk Desk { get; }
public NotificationSender Sender { get; }
public TelegramBot Bot { get; }
public Heartbeat Heartbeat { get; }
/// <summary>The address from the startup configuration (ANUBIS_HEARTBEAT_URL): it wins over the one saved here.</summary>
public string? ConfiguredHeartbeat { get; }
/// <summary>How often the queue is looked at.</summary>
public TimeSpan SendEvery { get; init; } = TimeSpan.FromSeconds(10);
/// <summary>The client of Telegram and of the checking service.</summary>
public HttpClient Http => _http;
public TelegramClient Client(string token) => new(_http, token);
public Task<TelegramSettings?> ReadAsync(CancellationToken cancellationToken) => TelegramSettings.ReadAsync(_secrets, cancellationToken);
public async Task<Uri?> HeartbeatAddressAsync(CancellationToken cancellationToken)
{
var text = ConfiguredHeartbeat ?? await _secrets.RevealAsync(HeartbeatSecret, cancellationToken).ConfigureAwait(false);
return string.IsNullOrWhiteSpace(text) || Heartbeat.Check(text) is not null ? null : new Uri(text.Trim());
}
public IReadOnlyList<ScheduledJob> Jobs() =>
[
new("segnale-di-vita", "Segnale di vita al servizio di controllo", Schedule.Every(TimeSpan.FromMinutes(1)), async ct => await Heartbeat.BeatAsync(ct).ConfigureAwait(false))
{
MaxLateness = TimeSpan.FromMinutes(2),
},
// The summary is still useful a little late; after four hours the day has moved on (SPEC 5.6).
new("riepilogo-mattino", "Riepilogo del mattino su Telegram", Schedule.Daily(MorningAt, _zone), MorningAsync)
{
MaxLateness = TimeSpan.FromHours(4),
CatchUpWithin = TimeSpan.FromHours(4),
},
new("stato-ogni-ora", "Stato periodico su Telegram, se scelto ogni ora", Schedule.Every(TimeSpan.FromHours(1)), ct => PeriodicAsync("hourly", ct)),
new("stato-ogni-giorno", "Stato periodico su Telegram, se scelto ogni giorno", Schedule.Daily(EveningAt, _zone), ct => PeriodicAsync("daily", ct)),
];
/// <summary>Sends the queue and reads the commands until the services stop.</summary>
public async Task RunAsync(CancellationToken cancellationToken)
{
var commands = Bot.RunAsync(cancellationToken);
try
{
while (!cancellationToken.IsCancellationRequested)
{
try
{
await Sender.SendDueAsync(cancellationToken).ConfigureAwait(false);
}
catch (Exception ex) when (!cancellationToken.IsCancellationRequested)
{
// A busy or unreachable database, or anything else: the queue keeps its messages for the next pass.
LogSendFailed(_logger, ex);
}
await Task.Delay(SendEvery, cancellationToken).ConfigureAwait(false);
}
}
finally
{
try
{
await commands.ConfigureAwait(false);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
}
}
}
/// <summary>
/// The test of Impostazioni: a message to the chat, sent at once and not through the queue, so the answer says
/// whether Telegram took it. Null when it went; otherwise the reason, in the user's words.
/// </summary>
public async Task<string?> TestAsync(TelegramSettings telegram, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(telegram);
try
{
var client = Client(telegram.Token);
var name = await client.GetMeAsync(cancellationToken).ConfigureAwait(false);
await client.SendAsync(telegram.ChatId, $"Anubis: messaggio di prova. Da qui ti scrivo{(name.Length > 0 ? " come @" + name : "")}; scrivi /aiuto per i comandi.",
cancellationToken).ConfigureAwait(false);
return null;
}
catch (TelegramException ex)
{
return ex.Message;
}
}
public void Dispose() => _http.Dispose();
private async Task MorningAsync(CancellationToken cancellationToken)
{
if (await ReadAsync(cancellationToken).ConfigureAwait(false) is null)
{
return;
}
var text = await Desk.MorningAsync(cancellationToken).ConfigureAwait(false);
await _db.WriteAsync(s => NotificationSender.EnqueueAsync(s, _clock, "morning_summary", text, cancellationToken), cancellationToken).ConfigureAwait(false);
}
private async Task PeriodicAsync(string every, CancellationToken cancellationToken)
{
if ((await _settings.GetAsync(SettingsStore.TelegramStatus.Key, cancellationToken).ConfigureAwait(false)).Value != every
|| await ReadAsync(cancellationToken).ConfigureAwait(false) is null)
{
return;
}
var text = await Desk.StatusAsync(cancellationToken).ConfigureAwait(false);
await _db.WriteAsync(s => NotificationSender.EnqueueAsync(s, _clock, "periodic_status", text, cancellationToken), cancellationToken).ConfigureAwait(false);
}
[Microsoft.Extensions.Logging.LoggerMessage(Level = Microsoft.Extensions.Logging.LogLevel.Error, Message = "Sending the Telegram queue failed")]
private static partial void LogSendFailed(Microsoft.Extensions.Logging.ILogger logger, Exception exception);
}
+198
View File
@@ -0,0 +1,198 @@
using System.Globalization;
using System.Text;
using Anubis.Core;
using Anubis.Data;
using Anubis.Trading;
namespace Anubis.Server;
/// <summary>The filters of the Operazioni screen, read from the address so a filtered page can be bookmarked.</summary>
internal sealed record TradeFilter(string? Search, Guid? Account, string? Strategy, string? Result, string Period, int Page)
{
public const int PageSize = 20;
public static readonly IReadOnlyList<(string Key, string Label)> Periods = [("avvio", "Dall'avvio"), ("7", "Ultimi 7 giorni"), ("30", "Ultimi 30 giorni"), ("90", "Ultimi 90 giorni")];
public static readonly IReadOnlyList<(string Key, string Label)> Results = [("", "Guadagni e perdite"), ("guadagni", "Solo guadagni"), ("perdite", "Solo perdite")];
public static TradeFilter Parse(string? search, string? account, string? strategy, string? result, string? period, string? page) => new(
string.IsNullOrWhiteSpace(search) ? null : search.Trim(),
Guid.TryParse(account, out var a) ? a : null,
string.IsNullOrWhiteSpace(strategy) ? null : strategy,
result is "guadagni" or "perdite" ? result : null,
Periods.Any(p => p.Key == period) ? period! : "avvio",
int.TryParse(page, NumberStyles.None, CultureInfo.InvariantCulture, out var p) && p > 0 ? p : 1);
/// <summary>The query string of this filter with one value changed.</summary>
public string Query(int? page = null)
{
var parts = new List<string>();
void Add(string key, string? value)
{
if (!string.IsNullOrEmpty(value))
{
parts.Add(key + "=" + Uri.EscapeDataString(value));
}
}
Add("cerca", Search);
Add("conto", Account?.ToString());
Add("strategia", Strategy);
Add("esito", Result);
Add("periodo", Period == "avvio" ? null : Period);
var p = page ?? Page;
Add("pagina", p > 1 ? p.ToString(CultureInfo.InvariantCulture) : null);
return parts.Count == 0 ? "" : "?" + string.Join('&', parts);
}
}
/// <summary>A step of the "why" card: a stage of a decision in plain Italian.</summary>
internal sealed record WhyLine(string Text, bool Good);
/// <summary>A closed trade of the Operazioni screen, in its own currency and in euro.</summary>
internal sealed record TradeRow(Guid Id, string Symbol, string Name, string Strategy, string Account, string Currency, DateTimeOffset OpenedAt, DateTimeOffset ClosedAt,
decimal Quantity, decimal EntryPrice, decimal ExitPrice, decimal Realized, decimal RealizedEur, decimal Fees, string ExitReason, string? ExitCode,
IReadOnlyList<WhyLine> Bought, IReadOnlyList<WhyLine> Sold, decimal? EntrySlipBp, decimal? ExitSlipBp)
{
public decimal Percent => EntryPrice * Quantity == 0m ? 0m : Realized / (EntryPrice * Quantity) * 100m;
}
/// <summary>
/// The Operazioni screen (SPEC 13.6, mockup 03): closed trades with their figures, filters, the result of each
/// trade as bars, and for each one the "why" card built from the rows of the decisions table (SPEC 5.2).
/// </summary>
internal sealed record TradesView(
IReadOnlyList<TradeRow> Rows,
int Matching,
int Total,
DateTimeOffset? Since,
decimal Gains,
decimal Losses,
int Winners,
decimal Fees,
IReadOnlyList<(DateTimeOffset At, decimal Eur, string Symbol)> Bars,
IReadOnlyList<(Guid Id, string Name)> Accounts,
IReadOnlyList<string> Strategies)
{
public int Losers => Matching - Winners;
/// <summary>Euro gained for each euro lost: null while nothing was lost.</summary>
public decimal? PerEuroLost => Losses < 0m ? Gains / -Losses : null;
public decimal Net => Gains + Losses;
public static async Task<TradesView> LoadAsync(AppServices services, IClock clock, TradeFilter filter, int? limit, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(services);
ArgumentNullException.ThrowIfNull(filter);
var rate = (await new FxStore(services.Db, clock).LatestAsync("EUR", "USD", cancellationToken).ConfigureAwait(false))?.Rate ?? 0m;
var (accounts, trades) = await services.Db.ReadAsync(async s =>
{
var a = await s.QueryAsync("SELECT id, name, environment, base_currency FROM accounts ORDER BY broker DESC, created_at",
r => (Id: r.Id("id"), Name: r.Text("name"), Environment: r.Text("environment"), Currency: r.Text("base_currency"))).ConfigureAwait(false);
var t = await s.QueryAsync(
"SELECT t.id, t.account_id, t.instrument_id, t.strategy, t.opened_at, t.closed_at, t.bought_quantity, t.bought_value, t.sold_quantity, t.sold_value, t.fees, t.realized, " +
"t.exit_reason, i.symbol, i.name FROM trades t JOIN instruments i ON i.id = t.instrument_id WHERE t.status = 'closed' ORDER BY t.closed_at DESC, t.id",
r => new Closed(r.Id("id"), r.Id("account_id"), r.Id("instrument_id"), r.Text("strategy"), r.Moment("opened_at"), r.Moment("closed_at"), r.Money("bought_quantity"),
r.Money("bought_value"), r.Money("sold_quantity"), r.Money("sold_value"), r.Money("fees"), r.Money("realized"), r.TextOrNull("exit_reason"), r.Text("symbol"),
r.TextOrNull("name") ?? "")).ConfigureAwait(false);
return (a, t);
}, cancellationToken).ConfigureAwait(false);
// One world only, as on the Conti screen: real accounts when there are any, so finto and vero never add up.
var world = accounts.Any(a => TradingEnvironments.World(a.Environment) == Worlds.Real) ? Worlds.Real : Worlds.Simulated;
var shown = accounts.Where(a => TradingEnvironments.World(a.Environment) == world).ToDictionary(a => a.Id);
decimal Eur(Guid account, decimal amount) => shown[account].Currency == "USD" ? (rate > 0m ? amount / rate : 0m) : amount;
var all = trades.Where(t => shown.ContainsKey(t.Account)).ToList();
var from = filter.Period switch
{
"7" => clock.UtcNow.AddDays(-7),
"30" => clock.UtcNow.AddDays(-30),
"90" => clock.UtcNow.AddDays(-90),
_ => (DateTimeOffset?)null,
};
var matching = all.Where(t =>
(filter.Account is null || t.Account == filter.Account) &&
(filter.Strategy is null || t.Strategy == filter.Strategy) &&
(from is null || t.ClosedAt >= from) &&
(filter.Result is null || (filter.Result == "guadagni" ? t.Realized >= 0m : t.Realized < 0m)) &&
(filter.Search is null || t.Symbol.Contains(filter.Search, StringComparison.OrdinalIgnoreCase) || t.Name.Contains(filter.Search, StringComparison.OrdinalIgnoreCase)))
.ToList();
var page = limit is { } n ? matching.Take(n).ToList() : matching.Skip((filter.Page - 1) * TradeFilter.PageSize).Take(TradeFilter.PageSize).ToList();
var rows = new List<TradeRow>();
foreach (var t in page)
{
var (bought, sold, entrySlip, exitSlip, code) = await WhyAsync(services, t, cancellationToken).ConfigureAwait(false);
var quantity = t.BoughtQuantity;
rows.Add(new TradeRow(t.Id, t.Symbol, t.Name, t.Strategy, shown[t.Account].Name, shown[t.Account].Currency, t.OpenedAt, t.ClosedAt, quantity,
quantity == 0m ? 0m : t.BoughtValue / quantity, t.SoldQuantity == 0m ? 0m : t.SoldValue / t.SoldQuantity, t.Realized,
Math.Round(Eur(t.Account, t.Realized), 2, MidpointRounding.AwayFromZero), t.Fees, ReasonTexts.Exit(code ?? t.ExitReason), code ?? t.ExitReason, bought, sold, entrySlip,
exitSlip));
}
var bars = matching.OrderBy(t => t.ClosedAt).Select(t => (t.ClosedAt, Math.Round(Eur(t.Account, t.Realized), 2, MidpointRounding.AwayFromZero), t.Symbol)).ToList();
return new TradesView(rows, matching.Count, all.Count, all.Count > 0 ? all.Min(t => t.ClosedAt) : null,
bars.Where(b => b.Item2 >= 0m).Sum(b => b.Item2), bars.Where(b => b.Item2 < 0m).Sum(b => b.Item2), bars.Count(b => b.Item2 >= 0m),
matching.Sum(t => Math.Round(Eur(t.Account, t.Fees), 2, MidpointRounding.AwayFromZero)), bars,
[.. shown.Values.Select(a => (a.Id, a.Name))], [.. all.Select(t => t.Strategy).Distinct().Order(StringComparer.Ordinal)]);
}
/// <summary>The trades as CSV for an Italian spreadsheet: semicolons, decimal commas, one row per closed trade.</summary>
public string Csv()
{
var text = new StringBuilder("chiusa_il;aperta_il;strumento;nome;strategia;conto;quantita;prezzo_entrata;prezzo_uscita;valuta;risultato;risultato_eur;commissioni;motivo\n");
foreach (var r in Rows)
{
text.Append(string.Join(';', Moment(r.ClosedAt), Moment(r.OpenedAt), Clean(r.Symbol), Clean(r.Name), r.Strategy, Clean(r.Account), Number(r.Quantity), Number(r.EntryPrice),
Number(r.ExitPrice), r.Currency, Number(r.Realized), Number(r.RealizedEur), Number(r.Fees), Clean(r.ExitReason))).Append('\n');
}
return text.ToString();
static string Moment(DateTimeOffset at) => at.UtcDateTime.ToString("yyyy-MM-dd'T'HH:mm:ss'Z'", CultureInfo.InvariantCulture);
static string Number(decimal value) => (Math.Round(value, 8) / 1.000000000000000000000000000000000m).ToString(CultureInfo.InvariantCulture).Replace('.', ',');
static string Clean(string value) => value.Replace(';', ',').Replace('\n', ' ');
}
private sealed record Closed(Guid Id, Guid Account, Guid Instrument, string Strategy, DateTimeOffset OpenedAt, DateTimeOffset ClosedAt, decimal BoughtQuantity, decimal BoughtValue,
decimal SoldQuantity, decimal SoldValue, decimal Fees, decimal Realized, string? ExitReason, string Symbol, string Name);
/// <summary>
/// The decisions behind a trade: the first purchase and the last sale of its executions, matched to their decision
/// through the order key (the hash of the decision id), and every stage of those decisions in words.
/// </summary>
private static async Task<(List<WhyLine> Bought, List<WhyLine> Sold, decimal? EntrySlip, decimal? ExitSlip, string? ExitCode)> WhyAsync(AppServices services, Closed t,
CancellationToken cancellationToken)
{
var (executions, decisions) = await services.Db.ReadAsync(async s =>
{
var e = await s.QueryAsync(
"SELECT side, order_key, reference_price, purpose, started_at FROM executions WHERE account_id = @a AND instrument_id = @i AND strategy = @s AND started_at >= @f AND started_at <= @t " +
"ORDER BY started_at, id",
r => (Side: r.Text("side"), Key: r.Text("order_key"), Reference: r.MoneyOrNull("reference_price"), Purpose: r.Text("purpose")),
("a", t.Account), ("i", t.Instrument), ("s", t.Strategy), ("f", t.OpenedAt.AddMinutes(-5)), ("t", t.ClosedAt)).ConfigureAwait(false);
var d = await s.QueryAsync(
"SELECT decision_id, stage, outcome, reasons, data FROM decisions WHERE strategy = @s AND symbol = @sym AND created_at >= @f AND created_at <= @t ORDER BY created_at, id",
r => (Id: r.Id("decision_id"), Stage: r.Text("stage"), Outcome: r.Text("outcome"), Reasons: r.Text("reasons"), Data: r.TextOrNull("data")),
("s", t.Strategy), ("sym", t.Symbol), ("f", t.OpenedAt.AddDays(-1)), ("t", t.ClosedAt.AddMinutes(5))).ConfigureAwait(false);
return (e, d);
}, cancellationToken).ConfigureAwait(false);
var byKey = decisions.GroupBy(d => d.Id).ToDictionary(g => ClientOrderIds.Key(g.Key), g => g.ToList(), StringComparer.Ordinal);
List<WhyLine> Lines(string? key) => key is not null && byKey.TryGetValue(key, out var rows)
? [.. rows.SelectMany(r => ReasonTexts.Stage(r.Stage, r.Outcome, System.Text.Json.JsonSerializer.Deserialize<List<string>>(r.Reasons) ?? [], r.Data))]
: [];
var buy = executions.FirstOrDefault(e => e.Side == "buy");
var sell = executions.LastOrDefault(e => e.Side == "sell");
var sold = Lines(sell.Key);
var code = sell.Key is not null && byKey.TryGetValue(sell.Key, out var exitRows)
? exitRows.Where(r => r.Stage == DecisionStages.Strategy).SelectMany(r => System.Text.Json.JsonSerializer.Deserialize<List<string>>(r.Reasons) ?? []).FirstOrDefault()
: sell.Purpose is "stop" or "protective" ? "stop" : null;
decimal? entrySlip = buy.Reference is > 0m && t.BoughtQuantity > 0m ? Math.Round(((t.BoughtValue / t.BoughtQuantity) / buy.Reference.Value - 1m) * 10_000m, 1) : null;
decimal? exitSlip = sell.Reference is > 0m && t.SoldQuantity > 0m ? Math.Round((1m - (t.SoldValue / t.SoldQuantity) / sell.Reference.Value) * 10_000m, 1) : null;
return (Lines(buy.Key), sold, entrySlip, exitSlip, code);
}
}
+32
View File
@@ -142,6 +142,10 @@ public sealed class AccessStore(Database db, IClock clock, SecretStore secrets,
public static readonly TimeSpan IdleTimeout = TimeSpan.FromHours(12);
public static readonly TimeSpan MaxLifetime = TimeSpan.FromDays(7);
// Wrong fresh re-authentications (SPEC 12.4) are limited like sign-ins, so a stolen session cannot try passwords.
private const string FreshAttempts = "reauthentication";
private readonly LoginLimiter _fresh = new(clock);
// Each second-factor code works once (RFC 6238, 5.2): the last time step used, in memory.
private readonly Lock _stepGate = new();
private long _lastStep = long.MinValue;
@@ -306,6 +310,34 @@ public sealed class AccessStore(Database db, IClock clock, SecretStore secrets,
return true;
}
/// <summary>
/// The fresh re-authentication of SPEC 12.4: the password and, when it is on, a code from the app, checked right
/// now whatever the session. Nothing is written. After five wrong attempts in 15 minutes every attempt fails,
/// even with the right password, until the window has passed.
/// </summary>
public async Task<bool> ReauthenticateAsync(string password, string? code, CancellationToken cancellationToken = default)
{
if (_fresh.Begin(FreshAttempts) > TimeSpan.Zero)
{
return false;
}
var user = await db.ReadAsync(s => s.FirstOrDefaultAsync("SELECT id, password_hash, totp_secret FROM users WHERE username = @u",
r => (Id: r.Id("id"), Hash: r.Text("password_hash"), Totp: r.BytesOrNull("totp_secret")), ("u", AdminName)), cancellationToken).ConfigureAwait(false);
if (user == default || string.IsNullOrEmpty(password) || !Passwords.Verify(password, user.Hash))
{
return false;
}
if (user.Totp is null || AcceptCode(Convert.FromBase64String(secrets.Open("totp:" + user.Id.ToString("D"), user.Totp)), code))
{
_fresh.Succeeded(FreshAttempts);
return true;
}
return false;
}
/// <summary>Turning the second factor off asks for both: the password and a code from the app.</summary>
public async Task<bool> DisableSecondFactorAsync(Guid userId, string password, string? code, CancellationToken cancellationToken = default)
{
+2
View File
@@ -10,6 +10,8 @@
<ItemGroup>
<EmbeddedResource Include="Db\Migrations\*.sql" LogicalName="Anubis.Db.Migrations.%(Filename)%(Extension)" />
<EmbeddedResource Include="Demo\Data\*" LogicalName="Anubis.Demo.Data.%(Filename)%(Extension)" />
<!-- The backtest outcomes travel with the code (SPEC 12.3): read at the first start. -->
<EmbeddedResource Include="..\..\docs\backtests\*.outcome.json" LogicalName="Anubis.Backtests.%(Filename)%(Extension)" />
</ItemGroup>
<ItemGroup>
<InternalsVisibleTo Include="Anubis.Tests" />
+8
View File
@@ -1,3 +1,5 @@
using System.Reflection;
namespace Anubis;
/// <summary>Names fixed by ADR 0001, shared by every part of the bot.</summary>
@@ -15,4 +17,10 @@ public static class AnubisInfo
/// <summary>Version of the running build, for example "0.0.1".</summary>
public static string Version { get; } =
typeof(AnubisInfo).Assembly.GetName().Version?.ToString(3) ?? "0.0.0";
/// <summary>The commit the build came from (the SDK appends it to the informational version), when known.</summary>
public static string? Commit { get; } =
typeof(AnubisInfo).Assembly.GetCustomAttribute<AssemblyInformationalVersionAttribute>()?.InformationalVersion is { } full && full.IndexOf('+', StringComparison.Ordinal) is var plus and >= 0
? full[(plus + 1)..][..Math.Min(12, full.Length - plus - 1)]
: null;
}
+94
View File
@@ -0,0 +1,94 @@
using Anubis.Data;
using Anubis.Db;
using Anubis.Trading;
namespace Anubis.Backtest;
/// <summary>A fill of the backtest: quantity received or given, cash moved and fee, all in the dataset currency.</summary>
public sealed record CostedFill(decimal Quantity, decimal Price, decimal Gross, decimal CashFee, decimal CoinFee, decimal CashDelta)
{
/// <summary>Every fee valued in the dataset currency: cash fees plus the coin kept by the venue at the fill price.</summary>
public decimal FeeValue => CashFee + DbValues.Money(CoinFee * Price);
}
/// <summary>
/// The costs of SPEC 12.1 for a market order: the same price lists and slippage tiers as the internal simulator
/// (SPEC 7.3, ADR 0009 point 5), rounded the cautious way. <see cref="Multiplier"/> scales fees and slippage
/// together, for the runs at double and triple costs of SPEC 12.2, step 3.
/// </summary>
public sealed record CostModel(decimal Multiplier = 1m)
{
public static CostModel Standard { get; } = new();
/// <summary>The price list of the venue where the instrument would trade. Research archives trade like OKX.</summary>
public static PaperOptions PriceList(InstrumentSpec spec)
{
ArgumentNullException.ThrowIfNull(spec);
return spec.Venue == Venues.Alpaca ? PaperOptions.Alpaca() : PaperOptions.Okx(spec.QuoteCurrency);
}
public decimal SlippageRate(InstrumentSpec spec) => PaperBroker.SlippageBp(spec) / 10_000m * Multiplier;
/// <summary>The price paid or received: the reference moved by the slippage, rounded to the tick against us.</summary>
public decimal ExecutionPrice(InstrumentSpec spec, OrderSide side, decimal reference)
{
ArgumentNullException.ThrowIfNull(spec);
var slip = SlippageRate(spec);
var price = side == OrderSide.Buy ? reference * (1m + slip) : reference * (1m - slip);
var tick = OrderRules.TickOf(spec, price);
price = tick is > 0m ? (side == OrderSide.Buy ? OrderRules.RoundUp(price, tick.Value) : OrderRules.RoundDown(price, tick.Value)) : price;
return DbValues.Money(price);
}
/// <summary>The cost of a market order of <paramref name="quantity"/> at <paramref name="price"/>, as the venue would charge it.</summary>
public CostedFill Fill(InstrumentSpec spec, OrderSide side, decimal quantity, decimal price)
{
var list = PriceList(spec);
var taker = list.TakerFee * Multiplier;
var gross = DbValues.Money(quantity * price);
var share = list.ShareFee > 0m ? OrderRules.RoundUp(quantity * list.ShareFee * Multiplier, 0.01m) : 0m;
if (side == OrderSide.Buy)
{
if (list.FeeInKind)
{
var coin = DbValues.Money(quantity * taker);
return new CostedFill(quantity - coin, price, gross, 0m, coin, -gross);
}
var fee = DbValues.Money(gross * taker) + share;
return new CostedFill(quantity, price, gross, fee, 0m, -(gross + fee));
}
var sellFee = DbValues.Money(gross * taker) + share
+ (list.SellValueFee > 0m ? OrderRules.RoundUp(gross * list.SellValueFee * Multiplier, 0.01m) : 0m);
return new CostedFill(quantity, price, gross, sellFee, 0m, gross - sellFee);
}
/// <summary>The largest quantity whose purchase, fees included, fits in <paramref name="cash"/>: no leverage (SPEC 9.1).</summary>
public decimal Affordable(InstrumentSpec spec, decimal cash, decimal price)
{
ArgumentNullException.ThrowIfNull(spec);
if (cash <= 0m || price <= 0m)
{
return 0m;
}
var list = PriceList(spec);
var perUnit = price * (1m + (list.FeeInKind ? 0m : list.TakerFee * Multiplier)) + (list.ShareFee * Multiplier);
var lot = OrderRules.LotOf(spec);
var quantity = OrderRules.RoundDown(cash / perUnit, lot);
// Fees rounded up to the cent can push the total a little past the cash: step back by the excess.
for (var guard = 0; guard < 8 && quantity > 0m; guard++)
{
var excess = -Fill(spec, OrderSide.Buy, quantity, price).CashDelta - cash;
if (excess <= 0m)
{
break;
}
quantity = OrderRules.RoundDown(quantity - (excess / perUnit) - lot, lot);
}
return Math.Max(quantity, 0m);
}
}
+236
View File
@@ -0,0 +1,236 @@
using System.Globalization;
using System.Security.Cryptography;
using System.Text;
using Anubis.Data;
using Anubis.Strategies;
namespace Anubis.Backtest;
/// <summary>
/// One instrument of a backtest: its rules, its real daily bars (oldest first, one per day) and its corporate
/// actions. Signals read the bars adjusted and anchored to the decision day; executions use the real prices.
/// </summary>
public sealed class BacktestInstrument
{
private readonly Dictionary<DateOnly, int> _index;
public BacktestInstrument(InstrumentSpec spec, IReadOnlyList<DailyBar> bars, IReadOnlyList<CorporateAction>? actions = null, string source = "")
{
ArgumentNullException.ThrowIfNull(spec);
ArgumentNullException.ThrowIfNull(bars);
Spec = spec;
Bars = [.. bars.OrderBy(b => b.Day)];
Actions = [.. (actions ?? []).Where(a => a.Symbol == spec.Symbol).OrderBy(a => a.ExDate)];
Source = source;
_index = new Dictionary<DateOnly, int>(Bars.Count);
for (var i = 0; i < Bars.Count; i++)
{
if (!_index.TryAdd(Bars[i].Day, i))
{
throw new ArgumentException("Two bars on the same day for " + spec.Symbol, nameof(bars));
}
}
// Cumulative price factor of every bar: the product of the factors of the actions that come after it. A bar
// anchored to day A is the real bar times Factors[i] / Factors[A] (SPEC 6.10).
Factors = new double[Bars.Count];
var running = 1d;
var next = Actions.Count - 1;
while (next >= 0 && Bars.Count > 0 && Actions[next].ExDate > Bars[^1].Day)
{
next--;
}
for (var i = Bars.Count - 1; i >= 0; i--)
{
Factors[i] = running;
while (next >= 0 && Actions[next].ExDate > (i > 0 ? Bars[i - 1].Day : DateOnly.MinValue) && Actions[next].ExDate <= Bars[i].Day)
{
running *= Adjustments.PriceFactor(Actions[next], i > 0 ? Bars[i - 1].Close : null);
next--;
}
}
}
public InstrumentSpec Spec { get; }
public string Symbol => Spec.Symbol;
public IReadOnlyList<DailyBar> Bars { get; }
public IReadOnlyList<CorporateAction> Actions { get; }
/// <summary>Where the bars come from, in words for the report.</summary>
public string Source { get; }
internal double[] Factors { get; }
public int IndexOf(DateOnly day) => _index.TryGetValue(day, out var i) ? i : -1;
/// <summary>Number of bars up to and including <paramref name="day"/>.</summary>
public int CountThrough(DateOnly day)
{
var lo = 0;
var hi = Bars.Count;
while (lo < hi)
{
var mid = (lo + hi) / 2;
if (Bars[mid].Day <= day)
{
lo = mid + 1;
}
else
{
hi = mid;
}
}
return lo;
}
public BarSeries SeriesThrough(DateOnly day) => new(Bars, Factors, CountThrough(day));
internal BacktestInstrument Through(DateOnly day) =>
new(Spec, [.. Bars.Where(b => b.Day <= day)], [.. Actions.Where(a => a.ExDate <= day)], Source);
}
/// <summary>
/// The data of a backtest: instruments in one currency, and the sessions (every day with at least one bar). Built
/// once and never changed, so the same dataset and parameters always give the same result (SPEC 12.1).
/// </summary>
public sealed class BacktestDataset
{
private readonly Dictionary<string, BacktestInstrument> _bySymbol;
/// <param name="instruments">Every instrument, the tradable ones and those used only as data.</param>
/// <param name="currency">The one currency of the backtest.</param>
/// <param name="calendar">The market's sessions, which may go beyond the bars; null means the days with bars.</param>
/// <param name="dataOnly">Instruments the strategy reads but never trades, such as SPY for the market filter.</param>
public BacktestDataset(IEnumerable<BacktestInstrument> instruments, string currency, IEnumerable<DateOnly>? calendar = null, IEnumerable<string>? dataOnly = null)
{
ArgumentNullException.ThrowIfNull(instruments);
Instruments = [.. instruments.OrderBy(i => i.Symbol, StringComparer.Ordinal)];
Currency = currency;
_bySymbol = Instruments.ToDictionary(i => i.Symbol, StringComparer.Ordinal);
Sessions = [.. Instruments.SelectMany(i => i.Bars.Select(b => b.Day)).Distinct().Order()];
Calendar = [.. (calendar ?? []).Concat(Sessions).Distinct().Order()];
DataOnly = new SortedSet<string>(dataOnly ?? [], StringComparer.Ordinal);
var wrong = Instruments.FirstOrDefault(i => i.Spec.QuoteCurrency != currency && !DataOnly.Contains(i.Symbol));
if (wrong is not null)
{
throw new ArgumentException($"{wrong.Symbol} is in {wrong.Spec.QuoteCurrency}, the backtest is in {currency}", nameof(instruments));
}
}
public IReadOnlyList<BacktestInstrument> Instruments { get; }
/// <summary>The instruments the strategy may trade: every one but those used only as data.</summary>
public IEnumerable<BacktestInstrument> Tradable => Instruments.Where(i => !DataOnly.Contains(i.Symbol));
public IReadOnlySet<string> DataOnly { get; }
public string Currency { get; }
public IReadOnlyList<DateOnly> Sessions { get; }
/// <summary>
/// The market calendar: the sessions with bars and, when given, the later ones. Cutting the data keeps it whole,
/// because a calendar is known in advance.
/// </summary>
public IReadOnlyList<DateOnly> Calendar { get; }
/// <summary>The next session after <paramref name="day"/>: every day for crypto, the calendar for stocks.</summary>
public DateOnly? NextSession(DateOnly day)
{
if (IsCrypto)
{
return day.AddDays(1);
}
int lo = 0, hi = Calendar.Count;
while (lo < hi)
{
var mid = (lo + hi) / 2;
if (Calendar[mid] <= day)
{
lo = mid + 1;
}
else
{
hi = mid;
}
}
return lo < Calendar.Count ? Calendar[lo] : null;
}
public bool IsCrypto => Instruments.Count > 0 && Tradable.All(i => i.Spec.AssetClass == AssetClasses.Crypto);
/// <summary>Sessions in a year, to annualise: crypto trades every day, stocks about 252 days.</summary>
public int PeriodsPerYear => IsCrypto ? 365 : 252;
public BacktestInstrument? this[string symbol] => _bySymbol.GetValueOrDefault(symbol);
/// <summary>The same data cut after <paramref name="day"/>: what was known on that day (SPEC 12.2, step 2).</summary>
public BacktestDataset Through(DateOnly day) => new(Instruments.Select(i => i.Through(day)), Currency, Calendar, DataOnly);
/// <summary>The same data between two days, both included.</summary>
public BacktestDataset Between(DateOnly from, DateOnly to) => new(
Instruments.Select(i => new BacktestInstrument(i.Spec, [.. i.Bars.Where(b => b.Day >= from && b.Day <= to)], [.. i.Actions.Where(a => a.ExDate > from && a.ExDate <= to)], i.Source))
.Where(i => i.Bars.Count > 0),
Currency,
Calendar.Where(d => d >= from),
DataOnly);
/// <summary>
/// A SHA-256 of every bar, action and rule used: two reports with the same fingerprint and the same plan were
/// computed on exactly the same data.
/// </summary>
public string Fingerprint()
{
var text = new StringBuilder();
text.Append(Currency).Append('\n');
if (Calendar.Count > Sessions.Count)
{
text.Append(CultureInfo.InvariantCulture, $"calendar {Calendar.Count} {Calendar[^1]:yyyy-MM-dd}\n");
}
foreach (var i in Instruments)
{
var s = i.Spec;
text.Append(CultureInfo.InvariantCulture, $"{s.Venue}|{s.Symbol}|{s.AssetClass}|{s.QuoteCurrency}|{s.TickSize}|{s.LotSize}|{s.MinSize}|{s.MinNotional}|{s.Fractionable}");
text.Append(DataOnly.Contains(i.Symbol) ? "|data\n" : "\n");
foreach (var b in i.Bars)
{
text.Append(CultureInfo.InvariantCulture, $"{b.Day:yyyy-MM-dd},{b.Open:R},{b.High:R},{b.Low:R},{b.Close:R},{b.Volume:R}\n");
}
foreach (var a in i.Actions)
{
text.Append(CultureInfo.InvariantCulture, $"{a.Kind},{a.ExDate:yyyy-MM-dd},{a.Ratio},{a.CashAmount}\n");
}
}
return Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(text.ToString())));
}
/// <summary>When a session opens and closes: crypto at midnight UTC (trap 23), stocks 09:30–16:00 in New York.</summary>
public (DateTimeOffset Opens, DateTimeOffset Closes) SessionTimes(DateOnly day)
{
if (IsCrypto)
{
var open = new DateTimeOffset(day.ToDateTime(TimeOnly.MinValue), TimeSpan.Zero);
return (open, open.AddDays(1));
}
return (NewYork(day, new TimeOnly(9, 30)), NewYork(day, new TimeOnly(16, 0)));
}
private static readonly TimeZoneInfo NewYorkZone = TimeZoneInfo.FindSystemTimeZoneById("America/New_York");
private static DateTimeOffset NewYork(DateOnly day, TimeOnly time)
{
var local = day.ToDateTime(time);
return new DateTimeOffset(local, NewYorkZone.GetUtcOffset(local)).ToUniversalTime();
}
}
+615
View File
@@ -0,0 +1,615 @@
using Anubis.Data;
using Anubis.Db;
using Anubis.Strategies;
using Anubis.Trading;
namespace Anubis.Backtest;
/// <summary>How a backtest runs (SPEC 12.1).</summary>
public sealed record BacktestOptions
{
public decimal InitialCapital { get; init; } = 10_000m;
public CostModel Costs { get; init; } = CostModel.Standard;
/// <summary>First and last session simulated; null means the whole dataset.</summary>
public DateOnly? From { get; init; }
public DateOnly? To { get; init; }
/// <summary>The correctives of SPEC 8.5. Until Phase 5 they let everything through.</summary>
public IReadOnlyList<IIntentStage> Correctives { get; init; } = [PassThroughStage.Correctives];
/// <summary>The risk engine of SPEC 9. Until Phase 5 it lets everything through.</summary>
public IIntentStage Risk { get; init; } = PassThroughStage.Risk;
/// <summary>
/// The strategy's share of the bot's total capital (SPEC 8.6, 9.2): the reference equity is the strategy's equity
/// divided by it. Null means the strategy is the whole bot.
/// </summary>
public decimal? ReferenceShare { get; init; }
}
/// <summary>A book to start from: a state recorded live, for the one-step comparison of SPEC 12.3.</summary>
public sealed record PortfolioState(decimal Cash, IReadOnlyDictionary<string, Holding> Holdings)
{
public static PortfolioState OnlyCash(decimal cash) => new(cash, new Dictionary<string, Holding>());
}
public sealed record EquityPoint(DateOnly Day, decimal Equity, decimal Cash, decimal Invested);
/// <summary>An execution. <see cref="Fee"/> is every fee valued in the dataset currency.</summary>
public sealed record BacktestFill(DateOnly Day, string Symbol, OrderSide Side, decimal Quantity, decimal Price, decimal Gross, decimal Fee, string Reason);
/// <summary>An operation from zero to zero (SPEC 7.5). <see cref="Result"/> is net of every cost; an open one is valued at the last close.</summary>
public sealed record BacktestTrade(string Symbol, DateOnly Opened, DateOnly? Closed, decimal Invested, decimal Result, decimal Fees)
{
public bool IsClosed => Closed is not null;
}
/// <summary>What the strategy proposed on a day, and what was left after correctives and risk.</summary>
public sealed record DecisionRecord(DateOnly Day, IReadOnlyList<Intent> Proposed, IReadOnlyList<Intent> Final);
/// <summary>Something the engine did not do as asked, and why: an order below the minimum, no cash, no price.</summary>
public sealed record BacktestNote(DateOnly Day, string Symbol, string Reason);
/// <summary>The outcome of one run of one strategy on one dataset.</summary>
public sealed record BacktestRun(
string Currency,
decimal InitialCapital,
IReadOnlyList<EquityPoint> Equity,
IReadOnlyList<BacktestFill> Fills,
IReadOnlyList<BacktestTrade> Trades,
IReadOnlyList<DecisionRecord> Decisions,
IReadOnlyList<BacktestNote> Notes)
{
public decimal FinalEquity => Equity.Count > 0 ? Equity[^1].Equity : InitialCapital;
public decimal NetResult => FinalEquity - InitialCapital;
public decimal Fees => Fills.Sum(f => f.Fee);
public decimal Dividends { get; init; }
/// <summary>Value of the leftovers below the minimum written off after an exit (SPEC 7.2).</summary>
public decimal WrittenOff { get; init; }
/// <summary>Daily returns of the equity, one per session after the first.</summary>
public double[] DailyReturns()
{
var result = new double[Math.Max(Equity.Count - 1, 0)];
for (var i = 1; i < Equity.Count; i++)
{
var before = Equity[i - 1].Equity;
result[i - 1] = before > 0m ? (double)(Equity[i].Equity / before) - 1d : 0d;
}
return result;
}
}
/// <summary>Reasons the engine writes in fills and notes.</summary>
public static class BacktestReasons
{
public const string Stop = "stop";
public const string StopGap = "stop_gap";
public const string WrittenOff = "residue_written_off";
public const string NoCash = "no_cash";
public const string NoPrice = "no_price";
public const string UnknownSymbol = "unknown_symbol";
public const string NoShortSelling = "no_short_selling";
public const string Dividend = "dividend";
/// <summary>
/// Prefix of the notes that count the codes of the correctives and of the risk engine on the intents of each close
/// (<c>adjusted:trade_risk_limit</c>): how often a rule changed what the strategy asked for.
/// </summary>
public const string AdjustedPrefix = "adjusted:";
}
/// <summary>
/// The event-driven backtest engine (SPEC 12.1). Each session is a sequence of events in time order: corporate
/// actions before the open, the open (stops that open below their level, then the orders decided at the previous
/// close, sales first), the session (stops touched by the low), the close (valuation and decision). A decision sees
/// the data up to its close and its orders go at the next open: the delay between deciding and executing. The same
/// <see cref="IStrategy"/> classes run live: the strategy cannot tell the difference.
/// </summary>
public static class BacktestEngine
{
/// <summary>Changes when the engine computes differently: plans record it, so a report says which engine made it.</summary>
public const string Version = "2";
public static BacktestRun Run(IStrategy strategy, BacktestDataset data, BacktestOptions? options = null, PortfolioState? start = null)
{
ArgumentNullException.ThrowIfNull(strategy);
ArgumentNullException.ThrowIfNull(data);
options ??= new BacktestOptions();
CheckSchedule(strategy);
var simulation = new Simulation(strategy, data, options, start ?? PortfolioState.OnlyCash(options.InitialCapital));
// Ordered by time, then by day: a crypto close and the next open fall on the same midnight, and the close of
// the earlier day comes first.
var events = new PriorityQueue<(SessionEvent Kind, DateOnly Day), (DateTimeOffset At, int Day, int Order)>();
foreach (var day in data.Sessions.Where(d => (options.From is null || d >= options.From) && (options.To is null || d <= options.To)))
{
var (opens, closes) = data.SessionTimes(day);
events.Enqueue((SessionEvent.CorporateActions, day), (opens, day.DayNumber, 0));
events.Enqueue((SessionEvent.Open, day), (opens, day.DayNumber, 1));
events.Enqueue((SessionEvent.Session, day), (opens, day.DayNumber, 2));
events.Enqueue((SessionEvent.Close, day), (closes, day.DayNumber, 3));
}
while (events.TryDequeue(out var e, out var when))
{
switch (e.Kind)
{
case SessionEvent.CorporateActions:
simulation.CorporateActions(e.Day);
break;
case SessionEvent.Open:
simulation.Open(e.Day);
break;
case SessionEvent.Session:
simulation.Session(e.Day);
break;
default:
simulation.Close(e.Day, when.At);
break;
}
}
return simulation.Finish();
}
/// <summary>
/// One decision cycle from a recorded book (SPEC 12.1, 12.3): the strategy decides on the close of
/// <paramref name="day"/> with the data known then and the book given, and nothing else runs. The live engine
/// decides with this same call (SPEC 5.2): <paramref name="history"/> is the strategy's equity at each close up to
/// <paramref name="day"/>, which the risk stage walks as it does in a whole run.
/// </summary>
public static DecisionRecord DecideOnce(IStrategy strategy, BacktestDataset data, PortfolioState state, DateOnly day, BacktestOptions? options = null,
IReadOnlyList<EquityMark>? history = null)
{
ArgumentNullException.ThrowIfNull(strategy);
ArgumentNullException.ThrowIfNull(data);
ArgumentNullException.ThrowIfNull(state);
CheckSchedule(strategy);
var simulation = new Simulation(strategy, data.Through(day), options ?? new BacktestOptions(), state);
simulation.MarkThrough(day);
simulation.SeedHistory(history ?? []);
return simulation.Decide(day, data.SessionTimes(day).Closes);
}
private static void CheckSchedule(IStrategy strategy)
{
if (strategy.Schedule().Any(t => t.Anchor != DecisionAnchor.Close))
{
throw new NotSupportedException($"{strategy.Id}: the daily backtest decides only on the close of a session");
}
}
private enum SessionEvent
{
CorporateActions,
Open,
Session,
Close,
}
private sealed class Position
{
public decimal Quantity { get; set; }
public decimal AveragePrice { get; set; }
public decimal? Stop { get; set; }
public DateOnly EnteredOn { get; init; }
public required OpenTrade Trade { get; init; }
}
private sealed class OpenTrade
{
public DateOnly Opened { get; init; }
public decimal Bought { get; set; }
public decimal CashFlow { get; set; }
public decimal Fees { get; set; }
}
private sealed record Pending(string Symbol, OrderSide Side, decimal Quantity, bool Exit, decimal? Stop, string Reason)
{
public decimal? StopOffset { get; init; }
}
private sealed class Simulation
{
private readonly IStrategy _strategy;
private readonly BacktestDataset _data;
private readonly BacktestOptions _options;
private readonly CostModel _costs;
private readonly SortedDictionary<string, Position> _positions = new(StringComparer.Ordinal);
private readonly SortedDictionary<string, Pending> _pending = new(StringComparer.Ordinal);
private readonly Dictionary<string, decimal> _lastClose = new(StringComparer.Ordinal);
private readonly Dictionary<string, DateOnly> _actionsDoneThrough = new(StringComparer.Ordinal);
private readonly List<EquityPoint> _equity = [];
private readonly List<EquityMark> _history = [];
private readonly List<BacktestFill> _fills = [];
private readonly List<BacktestTrade> _trades = [];
private readonly List<DecisionRecord> _decisions = [];
private readonly List<BacktestNote> _notes = [];
private readonly decimal _initial;
private decimal _cash;
private decimal _dividends;
private decimal _writtenOff;
public Simulation(IStrategy strategy, BacktestDataset data, BacktestOptions options, PortfolioState start)
{
_strategy = strategy;
_data = data;
_options = options;
_costs = options.Costs;
_cash = start.Cash;
var first = data.Sessions.FirstOrDefault(d => options.From is null || d >= options.From);
foreach (var (symbol, h) in start.Holdings.Where(h => h.Value.Quantity > 0m))
{
var trade = new OpenTrade { Opened = h.EnteredOn, Bought = DbValues.Money(h.Quantity * h.AveragePrice), CashFlow = -DbValues.Money(h.Quantity * h.AveragePrice) };
_positions[symbol] = new Position { Quantity = h.Quantity, AveragePrice = h.AveragePrice, Stop = h.Stop, EnteredOn = h.EnteredOn, Trade = trade };
_lastClose[symbol] = h.AveragePrice;
}
_initial = start.Cash + _positions.Values.Sum(p => p.Trade.Bought);
foreach (var i in data.Instruments)
{
_actionsDoneThrough[i.Symbol] = first == default ? DateOnly.MinValue : first.AddDays(-1);
}
}
public void SeedHistory(IReadOnlyList<EquityMark> history) => _history.AddRange(history);
public void MarkThrough(DateOnly day)
{
foreach (var i in _data.Instruments)
{
var n = i.CountThrough(day);
if (n > 0)
{
_lastClose[i.Symbol] = Price(i.Bars[n - 1].Close);
}
}
}
public void CorporateActions(DateOnly day)
{
foreach (var i in _data.Instruments)
{
var done = _actionsDoneThrough[i.Symbol];
foreach (var action in i.Actions.Where(a => a.ExDate > done && a.ExDate <= day))
{
Apply(i, action);
}
_actionsDoneThrough[i.Symbol] = day;
}
}
private void Apply(BacktestInstrument instrument, CorporateAction action)
{
var symbol = instrument.Symbol;
if (action.ChangesShares)
{
if (_positions.TryGetValue(symbol, out var p))
{
p.Quantity = Adjustments.ConvertQuantity(p.Quantity, action);
p.AveragePrice = DbValues.Money(Adjustments.ConvertLevel(p.AveragePrice, action, null));
p.Stop = p.Stop is { } s ? DbValues.Money(Adjustments.ConvertLevel(s, action, null)) : null;
}
if (_pending.TryGetValue(symbol, out var order))
{
_pending[symbol] = order with
{
Quantity = Adjustments.ConvertQuantity(order.Quantity, action),
Stop = order.Stop is { } s ? DbValues.Money(Adjustments.ConvertLevel(s, action, null)) : null,
};
}
if (_lastClose.TryGetValue(symbol, out var close))
{
_lastClose[symbol] = DbValues.Money(Adjustments.ConvertLevel(close, action, null));
}
}
else if (action.Kind == CorporateActionKinds.CashDividend && action.CashAmount is > 0m && _positions.TryGetValue(symbol, out var held)
&& (action.Currency is null || action.Currency == _data.Currency))
{
// To whoever held the shares at the close before the ex-date: that is the book now, before the open.
var amount = DbValues.Money(held.Quantity * action.CashAmount.Value);
_cash += amount;
_dividends += amount;
held.Trade.CashFlow += amount;
}
}
public void Open(DateOnly day)
{
// Stops resting on the broker fill first when the price opens below them (SPEC 7.2, trap 24).
foreach (var (symbol, p) in _positions.ToList())
{
if (Bar(symbol, day) is { } bar && p.Stop is { } stop && Price(bar.Open) <= stop)
{
Sell(day, symbol, p.Quantity, Price(bar.Open), BacktestReasons.StopGap, exit: true);
_pending.Remove(symbol);
}
}
var due = _pending.Values.Where(o => Bar(o.Symbol, day) is not null).OrderBy(o => o.Side == OrderSide.Buy).ToList();
foreach (var order in due)
{
_pending.Remove(order.Symbol);
var open = Price(Bar(order.Symbol, day)!.Open);
if (order.Side == OrderSide.Sell)
{
if (_positions.TryGetValue(order.Symbol, out var p))
{
Sell(day, order.Symbol, order.Exit ? p.Quantity : Math.Min(order.Quantity, p.Quantity), open, order.Reason, order.Exit);
}
}
else
{
Buy(day, order, open);
}
}
}
public void Session(DateOnly day)
{
foreach (var (symbol, p) in _positions.ToList())
{
if (Bar(symbol, day) is { } bar && p.Stop is { } stop && Price(bar.Low) <= stop)
{
Sell(day, symbol, p.Quantity, Math.Min(stop, Price(bar.Open)), BacktestReasons.Stop, exit: true);
_pending.Remove(symbol);
}
}
}
public void Close(DateOnly day, DateTimeOffset at)
{
MarkOn(day);
var invested = Invested();
_equity.Add(new EquityPoint(day, _cash + invested, _cash, invested));
_history.Add(new EquityMark(day, _cash + invested, invested));
var record = Decide(day, at);
_decisions.Add(record);
foreach (var intent in record.Final)
{
foreach (var code in intent.Adjustments)
{
_notes.Add(new BacktestNote(day, intent.Symbol, BacktestReasons.AdjustedPrefix + code));
}
}
// Exits asked for at this close go now (equity_meanrev's 15:55); the rest waits for the next open.
foreach (var exit in record.Final.Where(i => i.AtClose && i.TargetWeight == 0m))
{
if (_positions.TryGetValue(exit.Symbol, out var p) && _lastClose.TryGetValue(exit.Symbol, out var close))
{
Sell(day, exit.Symbol, p.Quantity, close, exit.Reason, exit: true);
_pending.Remove(exit.Symbol);
}
}
Plan(day, [.. record.Final.Where(i => !(i.AtClose && i.TargetWeight == 0m))]);
}
public DecisionRecord Decide(DateOnly day, DateTimeOffset at)
{
var available = _data.Tradable.Where(i => i.CountThrough(day) > 0).Select(i => i.Symbol).ToList();
BarSeries Bars(string s) => _data[s]?.SeriesThrough(day) ?? BarSeries.Empty;
var universe = _strategy.Universe(new UniverseContext(day, available) { Bars = Bars, Spec = s => _data[s]?.Spec });
var equity = _cash + Invested();
var holdings = _positions.ToDictionary(p => p.Key, p => new Holding(p.Value.Quantity, p.Value.AveragePrice, p.Value.Stop, p.Value.EnteredOn), StringComparer.Ordinal);
var context = new StrategyContext(day, at, universe, new PortfolioView(_cash, equity, holdings), Bars, s => _data[s]?.Spec)
{
NextSession = _data.NextSession(day),
ReferenceEquity = _options.ReferenceShare is > 0m and var share ? equity / share : equity,
PeriodsPerYear = _data.PeriodsPerYear,
History = _history,
};
var proposed = _strategy.Decide(context);
var final = proposed;
foreach (var stage in _options.Correctives)
{
final = stage.Apply(context, final);
}
final = _options.Risk.Apply(context, final);
return new DecisionRecord(day, proposed, final);
}
/// <summary>Turns the intents of a close into orders for the next open, sized on the equity of that close.</summary>
private void Plan(DateOnly day, IReadOnlyList<Intent> intents)
{
var equity = _cash + Invested();
foreach (var intent in intents.GroupBy(i => i.Symbol, StringComparer.Ordinal).Select(g => g.Last()))
{
var instrument = _data[intent.Symbol];
if (instrument is null)
{
_notes.Add(new BacktestNote(day, intent.Symbol, BacktestReasons.UnknownSymbol));
continue;
}
_positions.TryGetValue(intent.Symbol, out var p);
var stop = intent.Stop is { } level ? OrderRules.StopPrice(level, OrderRules.TickOf(instrument.Spec, level)) : (decimal?)null;
if (p is not null && stop is not null)
{
p.Stop = stop;
}
if (intent.TargetWeight is not { } weight)
{
continue;
}
if (weight < 0m)
{
_notes.Add(new BacktestNote(day, intent.Symbol, BacktestReasons.NoShortSelling));
continue;
}
if (!_lastClose.TryGetValue(intent.Symbol, out var close) || close <= 0m)
{
_notes.Add(new BacktestNote(day, intent.Symbol, BacktestReasons.NoPrice));
continue;
}
var current = p?.Quantity ?? 0m;
if (weight == 0m)
{
if (current > 0m)
{
_pending[intent.Symbol] = new Pending(intent.Symbol, OrderSide.Sell, current, Exit: true, null, intent.Reason);
}
else
{
_pending.Remove(intent.Symbol);
}
continue;
}
var target = weight * equity / close;
var difference = target - current;
if (difference > 0m)
{
_pending[intent.Symbol] = new Pending(intent.Symbol, OrderSide.Buy, difference, Exit: false, stop, intent.Reason) { StopOffset = intent.StopOffset };
}
else if (difference < 0m)
{
_pending[intent.Symbol] = new Pending(intent.Symbol, OrderSide.Sell, -difference, Exit: false, null, intent.Reason);
}
}
}
private void Buy(DateOnly day, Pending order, decimal reference)
{
var spec = _data[order.Symbol]!.Spec;
var price = _costs.ExecutionPrice(spec, OrderSide.Buy, reference);
var quantity = Math.Min(order.Quantity, _costs.Affordable(spec, _cash, price));
if (quantity < order.Quantity)
{
// Less cash than the decision assumed (the open moved): the purchase shrinks, never borrows (SPEC 9.1).
_notes.Add(new BacktestNote(day, order.Symbol, BacktestReasons.NoCash));
}
var rounded = OrderRules.Round(spec, OrderSide.Buy, OrderKind.Market, quantity, price);
if (!rounded.IsValid)
{
if (quantity >= order.Quantity)
{
_notes.Add(new BacktestNote(day, order.Symbol, rounded.Rejection!));
}
return;
}
var fill = _costs.Fill(spec, OrderSide.Buy, rounded.Quantity, price);
_cash += fill.CashDelta;
if (!_positions.TryGetValue(order.Symbol, out var p))
{
p = new Position { EnteredOn = day, Trade = new OpenTrade { Opened = day } };
_positions[order.Symbol] = p;
}
p.AveragePrice = DbValues.Money(((p.Quantity * p.AveragePrice) + fill.Gross) / (p.Quantity + fill.Quantity));
p.Quantity += fill.Quantity;
p.Stop = order.Stop ?? p.Stop;
if (p.Stop is null && order.StopOffset is > 0m and var offset && price - offset > 0m)
{
// A distance from the price obtained: the stop of a new position (SPEC 8.2, 8.3).
p.Stop = OrderRules.StopPrice(price - offset, OrderRules.TickOf(spec, price - offset));
}
p.Trade.Bought += fill.Gross;
p.Trade.CashFlow += fill.CashDelta;
p.Trade.Fees += fill.FeeValue;
_fills.Add(new BacktestFill(day, order.Symbol, OrderSide.Buy, rounded.Quantity, price, fill.Gross, fill.FeeValue, order.Reason));
}
private void Sell(DateOnly day, string symbol, decimal quantity, decimal reference, string reason, bool exit)
{
var p = _positions[symbol];
var spec = _data[symbol]!.Spec;
var price = _costs.ExecutionPrice(spec, OrderSide.Sell, reference);
var rounded = OrderRules.Round(spec, OrderSide.Sell, OrderKind.Market, Math.Min(quantity, p.Quantity), price);
if (rounded.IsValid)
{
var fill = _costs.Fill(spec, OrderSide.Sell, rounded.Quantity, price);
_cash += fill.CashDelta;
p.Quantity -= rounded.Quantity;
p.Trade.CashFlow += fill.CashDelta;
p.Trade.Fees += fill.FeeValue;
_fills.Add(new BacktestFill(day, symbol, OrderSide.Sell, rounded.Quantity, price, fill.Gross, fill.FeeValue, reason));
}
else if (!exit)
{
_notes.Add(new BacktestNote(day, symbol, rounded.Rejection!));
return;
}
// After an exit, what is below the minimum cannot be sold on its own: it is written off (SPEC 7.2).
if (p.Quantity > 0m && (exit || OrderRules.IsUnsellable(spec, p.Quantity, price)))
{
_writtenOff += DbValues.Money(p.Quantity * price);
_notes.Add(new BacktestNote(day, symbol, BacktestReasons.WrittenOff));
p.Quantity = 0m;
}
if (p.Quantity <= 0m)
{
_positions.Remove(symbol);
_trades.Add(new BacktestTrade(symbol, p.Trade.Opened, day, p.Trade.Bought, p.Trade.CashFlow, p.Trade.Fees));
}
}
public BacktestRun Finish()
{
foreach (var (symbol, p) in _positions)
{
var value = DbValues.Money(p.Quantity * _lastClose.GetValueOrDefault(symbol));
_trades.Add(new BacktestTrade(symbol, p.Trade.Opened, null, p.Trade.Bought, p.Trade.CashFlow + value, p.Trade.Fees));
}
return new BacktestRun(_data.Currency, _initial, _equity, _fills, [.. _trades.OrderBy(t => t.Opened).ThenBy(t => t.Symbol, StringComparer.Ordinal)], _decisions, _notes)
{
Dividends = _dividends,
WrittenOff = _writtenOff,
};
}
private void MarkOn(DateOnly day)
{
foreach (var i in _data.Instruments)
{
if (Bar(i.Symbol, day) is { } bar)
{
_lastClose[i.Symbol] = Price(bar.Close);
}
}
}
private decimal Invested() => _positions.Sum(p => DbValues.Money(p.Value.Quantity * _lastClose.GetValueOrDefault(p.Key)));
private DailyBar? Bar(string symbol, DateOnly day)
{
var instrument = _data[symbol];
var index = instrument?.IndexOf(day) ?? -1;
return index >= 0 ? instrument!.Bars[index] : null;
}
/// <summary>Bars are analysis data in floating point; prices used for money become decimal here, once.</summary>
private static decimal Price(double value) => DbValues.Money((decimal)value);
}
}
+400
View File
@@ -0,0 +1,400 @@
using System.Collections.Concurrent;
using System.Globalization;
using System.Threading.Channels;
using Anubis.Core;
using Anubis.Data;
using Anubis.Db;
using Anubis.Strategies;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
namespace Anubis.Backtest;
/// <summary>What the user asks for in the new-trial form of the laboratory.</summary>
public sealed record TrialRequest(
string Strategy,
string Source,
string Venue,
IReadOnlyList<string> Symbols,
DateOnly From,
DateOnly To,
decimal InitialCapital,
IReadOnlyDictionary<string, IReadOnlyList<decimal>>? Grid = null);
/// <summary>A request the laboratory refuses: a short code for the address of the form, and the reason in Italian.</summary>
public sealed class TrialRequestException(string code, string message) : Exception(message)
{
public string Code { get; } = code;
}
/// <summary>Codes of <see cref="TrialRequestException"/>.</summary>
public static class TrialRequestErrors
{
public const string Strategy = "strategia";
public const string Source = "origine";
public const string Symbols = "strumenti";
public const string Dates = "date";
public const string Capital = "capitale";
public const string Grid = "griglia";
public const string Combinations = "combinazioni";
}
/// <summary>
/// The laboratory (SPEC 12.2, 13.4): it writes the plan of a trial before running it, runs one trial at a time on a
/// dedicated thread with low priority, so trading and the interface never wait for a backtest, and writes the report
/// back. Only the asynchronous side touches the database. After a restart, trials left halfway start again.
/// </summary>
public sealed partial class Laboratory : IDisposable
{
public const int MaxSymbols = 60;
public const int MaxVariants = 200;
/// <summary>The pause after an error of the queue itself (the database out of reach), before trying again.</summary>
public static readonly TimeSpan RetryDelay = TimeSpan.FromSeconds(30);
private readonly Database _db;
private readonly IClock _clock;
private readonly string _researchRoot;
private readonly ILogger _logger;
private readonly Channel<Guid> _queue = Channel.CreateUnbounded<Guid>(new UnboundedChannelOptions { SingleReader = true });
private readonly ConcurrentDictionary<Guid, string> _progress = new();
private readonly LowPriorityWorker _worker = new("anubis-laboratory");
public Laboratory(Database db, IClock clock, string researchRoot, ILogger? logger = null)
{
_db = db;
_clock = clock;
_researchRoot = researchRoot;
_logger = logger ?? NullLogger.Instance;
Trials = new TrialStore(db, clock);
}
public TrialStore Trials { get; }
/// <summary>What the running trial is doing, in words; null when it is not running here.</summary>
public string? Progress(Guid id) => _progress.GetValueOrDefault(id);
/// <summary>Checks the request, writes the plan and queues the trial. The plan is final from here on.</summary>
public async Task<Trial> PlanAsync(TrialRequest request, CancellationToken cancellationToken)
{
var (plan, variants) = BuildPlan(request, _clock.UtcNow);
var trial = await Trials.PlanAsync(plan, variants, cancellationToken).ConfigureAwait(false);
await _queue.Writer.WriteAsync(trial.Id, cancellationToken).ConfigureAwait(false);
return trial;
}
/// <summary>The plan of a request: the family's own grid where the request gives none, the default costs, checks and thresholds.</summary>
public static (TrialPlan Plan, int Variants) BuildPlan(TrialRequest request, DateTimeOffset now)
{
ArgumentNullException.ThrowIfNull(request);
var family = StrategyCatalog.Find(request.Strategy) ?? throw new TrialRequestException(TrialRequestErrors.Strategy, "Strategia sconosciuta.");
if (request.Source is not (TrialSources.Database or TrialSources.Research))
{
throw new TrialRequestException(TrialRequestErrors.Source, "Origine dei dati sconosciuta.");
}
var symbols = request.Symbols.Select(s => s.Trim()).Where(s => s.Length > 0).Distinct(StringComparer.Ordinal).ToList();
if (symbols.Count == 0)
{
throw new TrialRequestException(TrialRequestErrors.Symbols, "Indica almeno uno strumento.");
}
if (symbols.Count > MaxSymbols)
{
throw new TrialRequestException(TrialRequestErrors.Symbols, $"Al massimo {MaxSymbols} strumenti per prova.");
}
if (request.From >= request.To)
{
throw new TrialRequestException(TrialRequestErrors.Dates, "La data di inizio deve venire prima di quella di fine.");
}
if (request.InitialCapital is < 1m or > 10_000_000m)
{
throw new TrialRequestException(TrialRequestErrors.Capital, "Il capitale iniziale deve essere tra 1 e 10.000.000.");
}
var grid = new SortedDictionary<string, IReadOnlyList<decimal>>(StringComparer.Ordinal);
foreach (var p in family.Parameters)
{
var values = request.Grid?.GetValueOrDefault(p.Name) is { Count: > 0 } given ? given : p.Grid;
if (values.Any(x => x < p.Min || x > p.Max))
{
throw new TrialRequestException(TrialRequestErrors.Grid, string.Create(CultureInfo.GetCultureInfo("it-IT"), $"{p.Label}: i valori vanno da {p.Min} a {p.Max}."));
}
// A whole parameter given 10.5 would run as 10: two variants that are the same would count twice in the DSR.
if (p.Whole && values.Any(x => x != decimal.Truncate(x)))
{
throw new TrialRequestException(TrialRequestErrors.Grid, p.Label + ": servono numeri interi.");
}
grid[p.Name] = [.. values.Distinct().Order()];
}
var plan = new TrialPlan(
family.Id,
family.Version,
grid,
new TrialData(request.Source, request.Venue, symbols, request.From, request.To),
request.InitialCapital,
[1m, 2m, 3m],
TrialValidation.Default,
TrialThresholds.Default,
BacktestEngine.Version,
now)
{
ReferenceShare = StrategyPipeline.DefaultShare(family.Id),
};
var variants = ParameterGrid.Of(family, plan.Grid).Count;
if (variants == 0)
{
throw new TrialRequestException(TrialRequestErrors.Combinations, "Nessuna combinazione dei parametri è valida.");
}
if (variants > MaxVariants)
{
throw new TrialRequestException(TrialRequestErrors.Combinations, $"Troppe combinazioni ({variants}): al massimo {MaxVariants}.");
}
return (plan, variants);
}
/// <summary>Runs the queue until stopped: first the trials left planned or halfway by the last run, then the new ones.</summary>
/// <remarks>
/// An error of the queue itself (the database out of reach) never stops it: it waits <see cref="RetryDelay"/> and
/// tries the same trial again. A trial left running by such an error is planned again at the next start.
/// </remarks>
public async Task RunAsync(CancellationToken cancellationToken)
{
while (true)
{
try
{
foreach (var id in await Trials.ResumeAsync(cancellationToken).ConfigureAwait(false))
{
await _queue.Writer.WriteAsync(id, cancellationToken).ConfigureAwait(false);
}
break;
}
catch (Exception ex) when (!cancellationToken.IsCancellationRequested)
{
LogQueueFailed(_logger, ex);
await _clock.WaitUntilAsync(_clock.UtcNow + RetryDelay, cancellationToken).ConfigureAwait(false);
}
}
await foreach (var id in _queue.Reader.ReadAllAsync(cancellationToken).ConfigureAwait(false))
{
try
{
await ProcessAsync(id, cancellationToken).ConfigureAwait(false);
}
catch (Exception ex) when (!cancellationToken.IsCancellationRequested)
{
LogQueueFailed(_logger, ex);
await _clock.WaitUntilAsync(_clock.UtcNow + RetryDelay, cancellationToken).ConfigureAwait(false);
_queue.Writer.TryWrite(id);
}
}
}
/// <summary>One trial: load its data, compute on the worker thread, write the report. A failure is recorded, not thrown.</summary>
public async Task ProcessAsync(Guid id, CancellationToken cancellationToken)
{
if (!await Trials.StartAsync(id, cancellationToken).ConfigureAwait(false))
{
return;
}
_progress[id] = "lettura dei dati";
try
{
var trial = await Trials.GetAsync(id, cancellationToken).ConfigureAwait(false) ?? throw new InvalidOperationException("Prova non trovata.");
if (!trial.PlanIntact)
{
throw new InvalidOperationException("Il piano registrato non corrisponde più alla sua impronta: la prova non parte.");
}
var family = StrategyCatalog.Find(trial.Strategy) ?? throw new InvalidOperationException("Strategia non più disponibile: " + trial.Strategy);
if (family.Version != trial.StrategyVersion)
{
throw new InvalidOperationException($"La strategia è cambiata dalla versione {trial.StrategyVersion} alla {family.Version}: pianifica una prova nuova.");
}
var data = await LoadAsync(_db, _clock, _researchRoot, trial.Plan.Data, cancellationToken).ConfigureAwait(false);
var prior = await Trials.PriorAsync(trial.Strategy, id, cancellationToken).ConfigureAwait(false);
var now = _clock.UtcNow;
var (report, summary) = await _worker.RunAsync(
() => TrialRunner.Run(id, trial.Plan, trial.PlanHash, family, data, prior, now, text => _progress[id] = text, cancellationToken)).ConfigureAwait(false);
await Trials.FinishAsync(id, report.DataFingerprint, summary, report.ToJson(), report.Passed, cancellationToken).ConfigureAwait(false);
LogFinished(_logger, id, trial.Strategy, report.Passed);
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
throw; // Left running: the next start plans it again.
}
catch (Exception ex)
{
LogFailed(_logger, id, ex);
await Trials.FailAsync(id, Redactor.Redact(ex.Message), CancellationToken.None).ConfigureAwait(false);
}
finally
{
_progress.TryRemove(id, out _);
}
}
/// <summary>
/// The bars of a trial. From the database: daily bars and, for stocks, corporate actions. From the research history:
/// Binance daily archives, with estimated order rules (the archive does not carry them).
/// </summary>
public static async Task<BacktestDataset> LoadAsync(Database db, IClock clock, string researchRoot, TrialData data, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(data);
var instruments = new List<BacktestInstrument>();
if (data.Source == TrialSources.Database)
{
var catalog = new InstrumentCatalog(db, clock);
var bars = new BarStore(db, clock);
var actions = new CorporateActionStore(db, clock);
foreach (var symbol in data.Symbols)
{
var instrument = await catalog.FindAsync(data.Venue, symbol, cancellationToken).ConfigureAwait(false)
?? throw new InvalidOperationException($"Strumento sconosciuto: {symbol} su {data.Venue}.");
var daily = await bars.DailyAsync(instrument.Id, data.HistoryStart(), data.To, cancellationToken).ConfigureAwait(false);
var events = instrument.IsCrypto ? [] : await actions.ForSymbolAsync(symbol, cancellationToken).ConfigureAwait(false);
instruments.Add(new BacktestInstrument(instrument.Spec, daily, [.. events.Where(a => a.ExDate > data.HistoryStart() && a.ExDate <= data.To)], "barre raccolte"));
}
}
else
{
return await LoadResearchAsync(researchRoot, data, cancellationToken).ConfigureAwait(false);
}
return Checked(instruments, data);
}
/// <summary>The bars of a trial on the research history: no database needed (the protocol runs from the command line).</summary>
public static async Task<BacktestDataset> LoadResearchAsync(string researchRoot, TrialData data, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(data);
var instruments = new List<BacktestInstrument>();
var store = new ResearchStore(researchRoot);
var from = new DateTimeOffset(data.HistoryStart().ToDateTime(TimeOnly.MinValue), TimeSpan.Zero);
var to = new DateTimeOffset(data.To.AddDays(1).ToDateTime(TimeOnly.MinValue), TimeSpan.Zero);
var source = data.Venue == EuroHistory.Venue ? "storico Binance convertito in euro" : "storico di ricerca";
foreach (var symbol in data.Symbols)
{
var daily = new List<DailyBar>();
await foreach (var b in store.ReadAsync(data.Venue, symbol, Timeframes.Day, from, to, cancellationToken).ConfigureAwait(false))
{
daily.Add(new DailyBar(DateOnly.FromDateTime(b.OpensAt.UtcDateTime), (double)b.Open, (double)b.High, (double)b.Low, (double)b.Close, (double)b.Volume));
}
instruments.Add(new BacktestInstrument(ResearchSpec(data.Venue, symbol), daily, null, source));
}
return Checked(instruments, data);
}
private static BacktestDataset Checked(List<BacktestInstrument> instruments, TrialData data)
{
var empty = instruments.FirstOrDefault(i => !i.Bars.Any(b => b.Day >= data.From && b.Day <= data.To));
if (empty is not null)
{
throw new InvalidOperationException($"Nessuna barra giornaliera per {empty.Symbol} tra il {data.From:dd/MM/yyyy} e il {data.To:dd/MM/yyyy}.");
}
var currencies = instruments.Select(i => i.Spec.QuoteCurrency).Distinct(StringComparer.Ordinal).ToList();
if (currencies.Count != 1)
{
throw new InvalidOperationException("Gli strumenti di una prova devono essere quotati nella stessa valuta: " + string.Join(", ", currencies) + ".");
}
return new BacktestDataset(instruments, currencies[0]);
}
/// <summary>Order rules for a research series: the archive has none, so small steps that never bind are assumed.</summary>
public static InstrumentSpec ResearchSpec(string venue, string symbol)
{
ArgumentNullException.ThrowIfNull(symbol);
string[] quotes = ["USDT", "USDC", "EUR", "USD"];
var quote = quotes.FirstOrDefault(q => symbol.EndsWith(q, StringComparison.Ordinal) && symbol.Length > q.Length)
?? throw new InvalidOperationException("Valuta di quotazione non riconosciuta per " + symbol + ".");
return new InstrumentSpec(venue, symbol, symbol, AssetClasses.Crypto, quote)
{
BaseCurrency = symbol[..^quote.Length],
TickSize = 0.00000001m,
LotSize = 0.00000001m,
MinSize = 0.00000001m,
MinNotional = 5m,
Status = "live",
};
}
public void Dispose()
{
_queue.Writer.TryComplete();
_worker.Dispose();
}
[LoggerMessage(Level = LogLevel.Information, Message = "Backtest trial {Id} of {Strategy} finished, passed: {Passed}")]
private static partial void LogFinished(ILogger logger, Guid id, string strategy, bool passed);
[LoggerMessage(Level = LogLevel.Warning, Message = "Laboratory queue error, trying again shortly")]
private static partial void LogQueueFailed(ILogger logger, Exception exception);
[LoggerMessage(Level = LogLevel.Warning, Message = "Backtest trial {Id} failed")]
private static partial void LogFailed(ILogger logger, Guid id, Exception exception);
}
/// <summary>
/// One thread with low priority that runs heavy work in order. Its callers await the result on their own side: the
/// thread never touches the database.
/// </summary>
internal sealed class LowPriorityWorker : IDisposable
{
private readonly BlockingCollection<Action> _work = [];
private readonly Thread _thread;
public LowPriorityWorker(string name)
{
_thread = new Thread(Loop) { IsBackground = true, Name = name, Priority = ThreadPriority.BelowNormal };
_thread.Start();
}
public Task<T> RunAsync<T>(Func<T> work)
{
var done = new TaskCompletionSource<T>(TaskCreationOptions.RunContinuationsAsynchronously);
_work.Add(() =>
{
try
{
done.SetResult(work());
}
catch (Exception ex)
{
done.SetException(ex);
}
});
return done.Task;
}
private void Loop()
{
foreach (var item in _work.GetConsumingEnumerable())
{
item();
}
}
public void Dispose()
{
_work.CompleteAdding();
_thread.Join(TimeSpan.FromSeconds(10));
_work.Dispose();
}
}
+127
View File
@@ -0,0 +1,127 @@
using System.Globalization;
using Anubis.Strategies;
namespace Anubis.Backtest;
/// <summary>A decision that changed when data after it was added: the strategy looked ahead.</summary>
public sealed record LookAheadMismatch(DateOnly Cutoff, DateOnly Day, string Full, string Cut);
public sealed record LookAheadResult(IReadOnlyList<DateOnly> Cutoffs, int Decisions, IReadOnlyList<LookAheadMismatch> Mismatches)
{
public bool Passed => Mismatches.Count == 0;
}
/// <summary>
/// The automatic test of SPEC 12.2 point 2: the signal of day t must not change when data after t is added. The
/// strategy runs on the full data and again on the data cut at a few days; every decision up to each cut must be the
/// same in both runs. The strategy is built afresh for each run from the data it receives, so one that keeps a
/// reference to the data and reads tomorrow from it is caught.
/// </summary>
public static class LookAheadCheck
{
public static LookAheadResult Run(Func<BacktestDataset, IStrategy> create, BacktestDataset data, BacktestOptions? options = null, int cuts = 6)
{
ArgumentNullException.ThrowIfNull(create);
ArgumentNullException.ThrowIfNull(data);
options ??= new BacktestOptions();
var sessions = data.Sessions
.Where(d => (options.From is null || d >= options.From) && (options.To is null || d <= options.To))
.ToList();
// Cut days spread over the run, never the last day (cutting there changes nothing).
var cutoffs = Enumerable.Range(1, cuts)
.Select(k => (int)((long)(sessions.Count - 1) * k / (cuts + 1)))
.Where(i => i > 0)
.Distinct()
.Select(i => sessions[i])
.ToList();
var full = BacktestEngine.Run(create(data), data, options);
var mismatches = new List<LookAheadMismatch>();
var compared = 0;
foreach (var cutoff in cutoffs)
{
var cutData = data.Through(cutoff);
var cut = BacktestEngine.Run(create(cutData), cutData, options);
var before = full.Decisions.Where(d => d.Day <= cutoff).ToDictionary(d => d.Day);
var after = cut.Decisions.ToDictionary(d => d.Day);
foreach (var day in before.Keys.Union(after.Keys).Order())
{
compared++;
var a = before.TryGetValue(day, out var x) ? Describe(x.Proposed) : "-";
var b = after.TryGetValue(day, out var y) ? Describe(y.Proposed) : "-";
if (a != b)
{
mismatches.Add(new LookAheadMismatch(cutoff, day, a, b));
}
}
}
return new LookAheadResult(cutoffs, compared, mismatches);
}
/// <summary>Intents as text, with stops to nine significant digits: adjusted prices may differ in the last bit.</summary>
private static string Describe(IReadOnlyList<Intent> intents) => string.Join(" | ", intents.Select(i => string.Create(
CultureInfo.InvariantCulture,
$"{i.Symbol} {(i.TargetWeight is { } w ? ((double)w).ToString("G9", CultureInfo.InvariantCulture) : "keep")} {(i.Stop is { } s ? ((double)s).ToString("G9", CultureInfo.InvariantCulture) : "-")} {(i.StopOffset is { } o ? ((double)o).ToString("G9", CultureInfo.InvariantCulture) : "-")}{(i.AtClose ? " close" : "")} {i.Reason}")));
}
/// <summary>How the live intent and the backtest intent for one instrument compare (SPEC 12.3).</summary>
public sealed record IntentComparison(string Symbol, IntentAction Live, IntentAction Backtest, decimal? LiveWeight, decimal? BacktestWeight, bool Differs);
/// <summary>
/// The one-step comparison of SPEC 12.3: the intents of the strategy (before correctives and risk) for the instruments
/// held and those either side wants to enter. Two intents differ if the action differs or the size differs by more
/// than 10%.
/// </summary>
public static class OneStepComparison
{
public const decimal SizeTolerance = 0.10m;
public static IntentAction ActionOf(Intent? intent, bool held) => intent?.TargetWeight switch
{
null => IntentAction.Keep,
<= 0m => held ? IntentAction.Exit : IntentAction.Keep,
_ => held ? IntentAction.Resize : IntentAction.Enter,
};
public static IReadOnlyList<IntentComparison> Compare(IReadOnlyList<Intent> live, IReadOnlyList<Intent> backtest, IReadOnlySet<string> held)
{
ArgumentNullException.ThrowIfNull(live);
ArgumentNullException.ThrowIfNull(backtest);
ArgumentNullException.ThrowIfNull(held);
var a = live.GroupBy(i => i.Symbol).ToDictionary(g => g.Key, g => g.Last(), StringComparer.Ordinal);
var b = backtest.GroupBy(i => i.Symbol).ToDictionary(g => g.Key, g => g.Last(), StringComparer.Ordinal);
var symbols = new SortedSet<string>(held, StringComparer.Ordinal);
foreach (var (symbol, intent) in a.Concat(b))
{
if (ActionOf(intent, held.Contains(symbol)) == IntentAction.Enter)
{
symbols.Add(symbol);
}
}
var result = new List<IntentComparison>();
foreach (var symbol in symbols)
{
var isHeld = held.Contains(symbol);
var x = a.GetValueOrDefault(symbol);
var y = b.GetValueOrDefault(symbol);
var actionX = ActionOf(x, isHeld);
var actionY = ActionOf(y, isHeld);
var differs = actionX != actionY || SizeDiffers(x?.TargetWeight, y?.TargetWeight, actionX);
result.Add(new IntentComparison(symbol, actionX, actionY, x?.TargetWeight, y?.TargetWeight, differs));
}
return result;
}
private static bool SizeDiffers(decimal? x, decimal? y, IntentAction action)
{
if (action is not (IntentAction.Enter or IntentAction.Resize) || x is not { } p || y is not { } q)
{
return false;
}
return Math.Abs(p - q) > SizeTolerance * Math.Max(Math.Abs(p), Math.Abs(q));
}
}
+140
View File
@@ -0,0 +1,140 @@
namespace Anubis.Backtest;
/// <summary>The result of one calendar year: equity at the end of the year before (or the start) and at the end of this one.</summary>
public sealed record YearResult(int Year, decimal Start, decimal End)
{
public decimal Result => End - Start;
public double Return => Start > 0m ? (double)(End / Start) - 1d : 0d;
}
/// <summary>
/// The figures of a run that the report of SPEC 12.2 point 6 shows. Returns and drawdowns are fractions (0,12 is
/// 12%); Sharpe and volatility are annualised with the sessions of a year (365 for crypto, 252 for stocks), without
/// a risk-free rate.
/// </summary>
public sealed record BacktestMetrics(
DateOnly? From,
DateOnly? To,
decimal InitialCapital,
decimal FinalEquity,
double TotalReturn,
double AnnualReturn,
double Volatility,
double Sharpe,
double DailySharpe,
double MaxDrawdown,
DateOnly? DrawdownPeak,
DateOnly? DrawdownLow,
int Trades,
int Winners,
double? ProfitFactor,
decimal AverageWin,
decimal AverageLoss,
decimal Fees,
double Exposure,
IReadOnlyList<YearResult> Years)
{
public decimal NetResult => FinalEquity - InitialCapital;
public double WinRate => Trades > 0 ? (double)Winners / Trades : 0d;
public static BacktestMetrics Of(BacktestRun run, int periodsPerYear)
{
ArgumentNullException.ThrowIfNull(run);
var equity = run.Equity;
var returns = run.DailyReturns();
var moments = Moments.Of(returns);
var annualisation = Math.Sqrt(periodsPerYear);
var total = run.InitialCapital > 0m ? (double)(run.FinalEquity / run.InitialCapital) - 1d : 0d;
var annual = 0d;
if (equity.Count > 1 && run.InitialCapital > 0m && run.FinalEquity > 0m)
{
// Calendar time, the same for markets that trade every day and markets that rest at the weekend.
var years = (equity[^1].Day.DayNumber - equity[0].Day.DayNumber) / 365.25;
annual = years > 0 ? Math.Pow(1 + total, 1 / years) - 1 : 0d;
}
var (drawdown, peak, low) = Drawdown(equity);
var closed = run.Trades.Where(t => t.IsClosed).ToList();
var wins = closed.Where(t => t.Result > 0m).ToList();
var losses = closed.Where(t => t.Result <= 0m).ToList();
var gains = wins.Sum(t => t.Result);
var lost = -losses.Sum(t => t.Result);
// Gains over losses of the closed trades; no value without a losing trade (nothing to divide by).
double? profitFactor = lost > 0m ? (double)(gains / lost) : null;
var exposure = equity.Count > 0
? equity.Average(p => p.Equity > 0m ? (double)(p.Invested / p.Equity) : 0d)
: 0d;
return new BacktestMetrics(
equity.Count > 0 ? equity[0].Day : null,
equity.Count > 0 ? equity[^1].Day : null,
run.InitialCapital,
run.FinalEquity,
total,
annual,
moments.StandardDeviation * annualisation,
moments.Sharpe * annualisation,
moments.Sharpe,
drawdown,
peak,
low,
closed.Count,
wins.Count,
profitFactor,
wins.Count > 0 ? gains / wins.Count : 0m,
losses.Count > 0 ? -lost / losses.Count : 0m,
run.Fees,
exposure,
Yearly(run));
}
/// <summary>The deepest fall of the equity from a previous high, as a fraction of that high, with its two days.</summary>
public static (double Depth, DateOnly? Peak, DateOnly? Low) Drawdown(IReadOnlyList<EquityPoint> equity)
{
ArgumentNullException.ThrowIfNull(equity);
var depth = 0d;
DateOnly? peakDay = null, lowDay = null;
var high = decimal.MinValue;
DateOnly highDay = default;
foreach (var p in equity)
{
if (p.Equity > high)
{
high = p.Equity;
highDay = p.Day;
}
if (high > 0m)
{
var fall = (double)((high - p.Equity) / high);
if (fall > depth)
{
depth = fall;
peakDay = highDay;
lowDay = p.Day;
}
}
}
return (depth, peakDay, lowDay);
}
private static List<YearResult> Yearly(BacktestRun run)
{
var result = new List<YearResult>();
var start = run.InitialCapital;
foreach (var year in run.Equity.GroupBy(p => p.Day.Year).OrderBy(g => g.Key))
{
var end = year.Last().Equity;
result.Add(new YearResult(year.Key, start, end));
start = end;
}
return result;
}
}
+378
View File
@@ -0,0 +1,378 @@
using System.Globalization;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using Anubis.Core;
using Anubis.Data;
using Anubis.Strategies;
using Anubis.Trading;
namespace Anubis.Backtest;
/// <summary>
/// The Binance USDT archives turned into euro (SPEC 8.1): the EUR pairs of OKX start in November 2023, so the years
/// before use the USDT history divided by the ECB EUR/USD rate of the day, one USDT counted as one dollar. The
/// converted series live in the research history under their own venue, so a trial on them can be planned again.
/// </summary>
public static class EuroHistory
{
public const string Venue = "binance_eur";
/// <summary>A rate older than this is not used: the ECB never skips more than a long holiday weekend.</summary>
public static readonly int MaxRateAgeDays = 7;
/// <summary>
/// A hole longer than this in a coin's daily series ends it: what comes after is a relaunch under the same ticker
/// (LUNA after May 2022), not the coin the earlier bars describe.
/// </summary>
public static readonly int MaxGapDays = 7;
/// <summary><c>BTCUSDT</c> becomes <c>BTCEUR</c>.</summary>
public static string SymbolOf(string usdtSymbol)
{
ArgumentNullException.ThrowIfNull(usdtSymbol);
return usdtSymbol.EndsWith("USDT", StringComparison.Ordinal) && usdtSymbol.Length > 4
? usdtSymbol[..^4] + "EUR"
: throw new ArgumentException("Not a USDT pair: " + usdtSymbol, nameof(usdtSymbol));
}
/// <summary>
/// Divides prices and traded value by the last EUR/USD rate fixed before the bar opens: the rate of the day before,
/// or of the last day before that with a rate. The rate of day d is fixed at 16:00 CET of d, after the bar of d
/// opens at 00:00 UTC, so using it for that bar's open (where the backtest fills) would read the future. A bar is
/// dropped when no rate of the last <see cref="MaxRateAgeDays"/> days exists. The series stops at the first hole
/// longer than <see cref="MaxGapDays"/> days.
/// </summary>
public static IReadOnlyList<ResearchBar> Convert(IEnumerable<ResearchBar> bars, IReadOnlyList<FxRate> eurUsd)
{
ArgumentNullException.ThrowIfNull(bars);
ArgumentNullException.ThrowIfNull(eurUsd);
var rates = eurUsd.Where(r => r.Base == "EUR" && r.Quote == "USD" && r.Rate > 0m).OrderBy(r => r.Date).ToList();
var result = new List<ResearchBar>();
var k = -1;
DateTimeOffset? previous = null;
foreach (var bar in bars.OrderBy(b => b.OpensAt))
{
if (previous is { } p && bar.OpensAt - p > TimeSpan.FromDays(MaxGapDays))
{
break;
}
previous = bar.OpensAt;
var day = DateOnly.FromDateTime(bar.OpensAt.UtcDateTime);
while (k + 1 < rates.Count && rates[k + 1].Date < day)
{
k++;
}
if (k < 0 || rates[k].Date < day.AddDays(-MaxRateAgeDays))
{
continue;
}
var rate = rates[k].Rate;
result.Add(new ResearchBar(bar.OpensAt, Round(bar.Open / rate), Round(bar.High / rate), Round(bar.Low / rate), Round(bar.Close / rate), bar.Volume)
{
QuoteVolume = bar.QuoteVolume is { } q ? Round(q / rate) : null,
Trades = bar.Trades,
});
}
return result;
}
/// <summary>Converts every daily USDT series of the Binance archives in the store. Returns the symbols written.</summary>
public static async Task<IReadOnlyList<string>> ConvertAllAsync(ResearchStore store, IReadOnlyList<FxRate> eurUsd, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(store);
var written = new List<string>();
foreach (var (_, symbol, _) in store.Series().Where(s => s.Venue == Venues.Binance && s.Timeframe == Timeframes.Day && s.Symbol.EndsWith("USDT", StringComparison.Ordinal)))
{
var bars = new List<ResearchBar>();
await foreach (var bar in store.ReadAsync(Venues.Binance, symbol, Timeframes.Day, Ever.From, Ever.To, cancellationToken).ConfigureAwait(false))
{
bars.Add(bar);
}
var converted = Convert(bars, eurUsd);
if (converted.Count > 0)
{
await store.WriteAsync(Venue, SymbolOf(symbol), Timeframes.Day, converted, cancellationToken).ConfigureAwait(false);
written.Add(SymbolOf(symbol));
}
}
return written;
}
// Twelve decimals keep seven significant digits even for coins priced at a hundred-thousandth of a euro.
private static decimal Round(decimal value) => Math.Round(value, 12, MidpointRounding.ToEven);
private static class Ever
{
public static readonly DateTimeOffset From = new(2009, 1, 1, 0, 0, 0, TimeSpan.Zero);
public static readonly DateTimeOffset To = new(2100, 1, 1, 0, 0, 0, TimeSpan.Zero);
}
}
/// <summary>A trial of the protocol: which universe, its report and the daily returns of the chosen variant.</summary>
public sealed record ProtocolRun(string Name, string Description, TrialPlan Plan, BacktestReport Report, IReadOnlyList<double> DailyReturns, decimal ExpectedSlippageBp);
/// <summary>What the protocol of a strategy concluded: the outcome file and the trials behind it.</summary>
public sealed record ProtocolOutcome(Qualification Qualification, IReadOnlyList<ProtocolRun> Runs);
/// <summary>
/// The anti-illusion protocol of SPEC 12.2 for <c>crypto_trend</c>, run from the command line on the research
/// history: it must pass twice (SPEC 8.1), on the broad universe and on BTC, ETH and SOL alone. Everything is fixed
/// before the run: dates, capital, grid, costs, thresholds. The result is written whatever it is.
/// </summary>
public static class CryptoTrendProtocol
{
public const string Strategy = StrategyNames.CryptoTrend;
public const string ReportPath = "docs/backtests/crypto_trend.md";
public const decimal Capital = 10_000m;
/// <summary>Two years of Binance history before this day are enough for the longest period (360) and the 400 days of age.</summary>
public static readonly DateOnly From = new(2019, 1, 1);
public static readonly IReadOnlyList<string> Narrow = ["BTCEUR", "ETHEUR", "SOLEUR"];
/// <summary>
/// The USDT pairs downloaded from the Binance archives: every coin that was among the ten most traded at some time
/// since 2018, including those later delisted (LUNA, FTT, BCC, VEN, XMR...), so the past universe is not chosen
/// with hindsight. Stablecoins are left out: the strategy excludes them anyway. A pair without archives is skipped.
/// </summary>
public static readonly IReadOnlyList<string> Archives =
[
"BTCUSDT", "ETHUSDT", "BNBUSDT", "XRPUSDT", "ADAUSDT", "SOLUSDT", "DOGEUSDT", "LTCUSDT", "BCCUSDT", "BCHABCUSDT", "BCHUSDT", "BCHSVUSDT",
"EOSUSDT", "TRXUSDT", "ETCUSDT", "LINKUSDT", "DOTUSDT", "MATICUSDT", "POLUSDT", "AVAXUSDT", "LUNAUSDT", "FTTUSDT", "SHIBUSDT", "FILUSDT",
"ATOMUSDT", "UNIUSDT", "NEOUSDT", "XLMUSDT", "VENUSDT", "VETUSDT", "SANDUSDT", "APEUSDT", "GMTUSDT", "NEARUSDT", "PEPEUSDT", "WIFUSDT",
"SUIUSDT", "ARBUSDT", "OPUSDT", "ORDIUSDT", "ONTUSDT", "IOTAUSDT", "QTUMUSDT", "ICXUSDT", "XMRUSDT", "DASHUSDT", "ZECUSDT", "TONUSDT", "TRUMPUSDT",
];
/// <summary>Length of the expected path of the outcome: one year of daily closes.</summary>
public const int PathDays = 365;
public static async Task<ProtocolOutcome> RunAsync(string researchRoot, DateOnly to, DateTimeOffset now, Action<string>? progress, CancellationToken cancellationToken)
{
var store = new ResearchStore(researchRoot);
var broad = new List<string>();
foreach (var (_, symbol, _) in store.Series().Where(s => s.Venue == EuroHistory.Venue && s.Timeframe == Timeframes.Day))
{
// Every converted pair goes in: the strategy picks its top 10 month by month, among coins that died too.
if (await HasBarsAsync(store, symbol, to, cancellationToken).ConfigureAwait(false))
{
broad.Add(symbol);
}
}
if (Narrow.Any(s => !broad.Contains(s)))
{
throw new InvalidOperationException("Mancano gli storici di BTC, ETH o SOL in euro: scarica prima gli archivi Binance e il cambio BCE.");
}
var family = StrategyCatalog.Find(Strategy) ?? throw new InvalidOperationException("crypto_trend non è nel catalogo.");
var first = await RunOneAsync(family, "Universo ampio", $"Le 10 monete con più scambi, scelte ogni mese tra {broad.Count} coppie (anche quelle poi chiuse).",
broad, to, now, researchRoot, PriorTrials.None, progress, cancellationToken).ConfigureAwait(false);
var prior = new PriorTrials(1, first.Report.Variants.Count, [.. first.Report.Variants.Select(v => v.Sharpe)]);
var second = await RunOneAsync(family, "BTC, ETH e SOL", "Solo le tre coppie in euro con la liquidità migliore.",
Narrow, to, now, researchRoot, prior, progress, cancellationToken).ConfigureAwait(false);
List<ProtocolRun> runs = [first, second];
return new ProtocolOutcome(QualificationOf(runs), runs);
}
/// <summary>The outcome file: passed only when every trial passed; measures from the broad universe, the one traded live.</summary>
public static Qualification QualificationOf(IReadOnlyList<ProtocolRun> runs)
{
ArgumentNullException.ThrowIfNull(runs);
var main = runs[0];
var passed = runs.All(r => r.Report.Passed);
var it = CultureInfo.GetCultureInfo("it-IT");
string verdict;
if (passed)
{
verdict = "Protocollo superato su entrambi gli universi: parte in simulazione.";
}
else
{
var failures = runs.Where(r => !r.Report.Passed)
.Select(r => r.Name + ": " + string.Join("; ", r.Report.Checks.Where(c => c.Passed == false).Select(c => Describe(c, it))));
verdict = "Protocollo non superato. " + string.Join(". ", failures) + ". Resta in osservazione.";
}
var taker = PaperOptions.Okx().TakerFee;
return new Qualification
{
Passed = passed,
Verdict = verdict,
Report = ReportPath,
PlanHash = TrialPlan.Hash(main.Plan.ToJson()),
Parameters = main.Report.Variants.FirstOrDefault(v => v.Chosen)?.Parameters,
MaxDrawdown = Math.Round((decimal)main.Report.Metrics.MaxDrawdown, 4),
ExpectedSlippageBp = main.ExpectedSlippageBp,
RoundTripCost = Math.Round(2m * (taker + (main.ExpectedSlippageBp / 10_000m)), 6),
ExpectedPathP5 = [.. PathBootstrap.Percentile(main.DailyReturns, PathDays, 0.05, PathBootstrap.Paths, PathBootstrap.Block, PathBootstrap.Seed)
.Select(x => Math.Round(x, 4))],
};
}
/// <summary>A failed check in plain Italian, e.g. "calo massimo 38,2%, oltre il 25%".</summary>
public static string Describe(ReportCheck check, IFormatProvider culture)
{
ArgumentNullException.ThrowIfNull(check);
var value = check.Value is not { } v ? "non calcolabile"
: check.Unit switch
{
"percent" => string.Create(culture, $"{v * 100:0.0}%"),
"money" => string.Create(culture, $"{v:#,0.00} €"),
"count" => string.Create(culture, $"{v:0}"),
_ => string.Create(culture, $"{v:0.000}"),
};
var threshold = check.Unit == "percent" ? string.Create(culture, $"{check.Threshold * 100:0}%") : string.Create(culture, $"{check.Threshold:0.00}");
return $"{check.Name.ToLower(CultureInfo.GetCultureInfo("it-IT"))} {value}, serve {check.Comparison} {threshold}";
}
private static async Task<ProtocolRun> RunOneAsync(IStrategyFamily family, string name, string description, IReadOnlyList<string> symbols, DateOnly to,
DateTimeOffset now, string researchRoot, PriorTrials prior, Action<string>? progress, CancellationToken cancellationToken)
{
var request = new TrialRequest(family.Id, TrialSources.Research, EuroHistory.Venue, symbols, From, to, Capital, null);
var (plan, _) = Laboratory.BuildPlan(request, now);
var json = plan.ToJson();
var hash = TrialPlan.Hash(json);
var data = await Laboratory.LoadResearchAsync(researchRoot, plan.Data, cancellationToken).ConfigureAwait(false);
// The id comes from the plan, so the same plan on the same data writes the same report.
var id = new Guid(SHA256.HashData(Encoding.UTF8.GetBytes(hash)).AsSpan(0, 16));
var (report, _) = TrialRunner.Run(id, plan, hash, family, data, prior, now, text => progress?.Invoke(name + ": " + text), cancellationToken);
// The chosen variant again, for its daily returns and the slippage of what it traded.
var variant = ParameterGrid.Of(family, plan.Grid).First(v => v.ToString() == report.Chosen);
var options = StrategyPipeline.For(family.Id, data, new BacktestOptions { InitialCapital = plan.InitialCapital, From = plan.Data.From, To = plan.Data.To }, plan.ReferenceShare);
var run = BacktestEngine.Run(family.Create(variant.Parameters), data, options);
var specs = data.Instruments.ToDictionary(i => i.Symbol, i => i.Spec, StringComparer.Ordinal);
var traded = run.Trades.Sum(t => t.Invested);
var slippage = traded > 0m
? Math.Round(run.Trades.Sum(t => t.Invested * PaperBroker.SlippageBp(specs.GetValueOrDefault(t.Symbol))) / traded, 2)
: PaperBroker.SlippageBp(null);
return new ProtocolRun(name, description, plan, report, run.DailyReturns(), slippage);
}
private static async Task<bool> HasBarsAsync(ResearchStore store, string symbol, DateOnly to, CancellationToken cancellationToken)
{
var from = new DateTimeOffset(From.ToDateTime(TimeOnly.MinValue), TimeSpan.Zero);
var end = new DateTimeOffset(to.AddDays(1).ToDateTime(TimeOnly.MinValue), TimeSpan.Zero);
await foreach (var _ in store.ReadAsync(EuroHistory.Venue, symbol, Timeframes.Day, from, end, cancellationToken).ConfigureAwait(false))
{
return true;
}
return false;
}
}
/// <summary>
/// The expected path of SPEC 12.3: a stationary block bootstrap of the backtest's daily returns (blocks keep the
/// clustering of volatility), and for each day from the start the given percentile of the cumulative result. The seed
/// is fixed, so the same returns always give the same path.
/// </summary>
public static class PathBootstrap
{
public const int Paths = 2000;
public const int Block = 20;
public const int Seed = 20_260_101;
public static IReadOnlyList<double> Percentile(IReadOnlyList<double> returns, int days, double percentile, int paths, int block, int seed)
{
ArgumentNullException.ThrowIfNull(returns);
ArgumentOutOfRangeException.ThrowIfLessThan(days, 1);
ArgumentOutOfRangeException.ThrowIfLessThan(paths, 1);
ArgumentOutOfRangeException.ThrowIfLessThan(block, 1);
if (returns.Count == 0)
{
return [.. Enumerable.Repeat(0d, days)];
}
var random = new Random(seed);
var cumulative = new double[days][];
for (var d = 0; d < days; d++)
{
cumulative[d] = new double[paths];
}
for (var p = 0; p < paths; p++)
{
var value = 1d;
var index = random.Next(returns.Count);
for (var d = 0; d < days; d++)
{
// A new block starts with probability 1/block: blocks of geometric length, mean "block" days.
if (d > 0 && random.NextDouble() < 1d / block)
{
index = random.Next(returns.Count);
}
else if (d > 0)
{
index = (index + 1) % returns.Count;
}
value *= 1d + returns[index];
cumulative[d][p] = value - 1d;
}
}
var result = new double[days];
for (var d = 0; d < days; d++)
{
Array.Sort(cumulative[d]);
var rank = Math.Clamp((int)Math.Floor(percentile * (paths - 1)), 0, paths - 1);
result[d] = cumulative[d][rank];
}
return result;
}
}
/// <summary>
/// The outcome files of <c>docs/backtests/</c>, built into the program: loaded at the first start, they decide
/// whether a base strategy starts in simulation or stays in observation (SPEC 12.3).
/// </summary>
public static class BacktestOutcomes
{
public const string ResourcePrefix = "Anubis.Backtests.";
public static Qualification? Find(string strategy)
{
using var stream = typeof(BacktestOutcomes).Assembly.GetManifestResourceStream(ResourcePrefix + strategy + ".outcome.json");
return stream is null ? null : JsonSerializer.Deserialize<Qualification>(stream, Qualification.Json);
}
private static readonly JsonSerializerOptions Indented = new(Qualification.Json) { WriteIndented = true };
public static string ToJson(Qualification qualification) => JsonSerializer.Serialize(qualification, Indented) + "\n";
}
/// <summary>
/// The stock strategies wait for the Alpaca history (SPEC 8.2, 8.3): without the keys there are no adjusted daily
/// bars of the S&amp;P 500 members, so the protocol cannot run and they stay in observation. The outcome file says so.
/// </summary>
public static class EquityProtocol
{
public static readonly IReadOnlyList<string> Explanation =
[
"Per provare le strategie sulle azioni servono gli storici giornalieri di Alpaca, con frazionamenti e dividendi, che si scaricano solo con le chiavi di un conto Alpaca (anche dimostrativo). Le chiavi non sono ancora state inserite.",
"Finché manca il backtest la strategia resta in osservazione: guarda il mercato e scrive le sue decisioni nel diario, ma non opera, nemmeno in simulazione.",
"Quando avrai inserito le chiavi nelle Impostazioni, il bot raccoglie gli storici. Poi il backtest si rifà con gli stessi controlli di crypto_trend e il nuovo esito arriva con un aggiornamento del programma: se supera le soglie, la strategia parte in simulazione da sola al primo avvio dopo l'aggiornamento.",
];
public static Qualification Waiting(string strategy) => new()
{
Passed = false,
Verdict = "In attesa delle chiavi Alpaca: senza lo storico delle azioni il backtest non si può fare, quindi la strategia resta in osservazione.",
Report = "docs/backtests/" + strategy + ".md",
};
}
+220
View File
@@ -0,0 +1,220 @@
using System.Globalization;
using System.Text;
using Anubis.Trading;
namespace Anubis.Backtest;
/// <summary>
/// The report of <c>docs/backtests/</c> in plain Italian (SPEC 12.2 point 6): for each trial the figures of the chosen
/// variant against buy and hold, by year, at single, double and triple costs, the validations, the checks and what
/// the backtest does not model. It is written whatever the outcome.
/// </summary>
public static class ProtocolReport
{
private static readonly CultureInfo It = CultureInfo.GetCultureInfo("it-IT");
public static string Markdown(ProtocolOutcome outcome, string title, IReadOnlyList<string> dataNotes, string command)
{
ArgumentNullException.ThrowIfNull(outcome);
ArgumentNullException.ThrowIfNull(dataNotes);
var q = outcome.Qualification;
var first = outcome.Runs[0].Report;
var b = new StringBuilder();
b.Append("# Backtest di `").Append(first.Strategy).Append("` (").Append(first.StrategyName).AppendLine(")").AppendLine();
b.Append("**Esito: ").Append(q.Passed ? "superato" : "non superato").Append(".** ").AppendLine(q.Verdict).AppendLine();
b.AppendLine(Text($"Calcolato il {first.ComputedAt:yyyy-MM-dd} con la versione {first.StrategyVersion} della strategia. Soglie, griglia dei parametri, costi e date erano fissati prima del calcolo (SPEC 12.2) e non sono stati cambiati dopo."))
.AppendLine();
b.Append("Per rifarlo: `").Append(command).AppendLine("`. Il file dell'esito, letto dal bot al primo avvio, è `docs/backtests/" + first.Strategy + ".outcome.json`.").AppendLine();
b.AppendLine("## Dati").AppendLine();
foreach (var note in dataNotes)
{
b.Append("- ").AppendLine(note);
}
b.AppendLine();
b.AppendLine("## Riepilogo").AppendLine();
b.AppendLine("| Prova | Variante scelta | Rendimento annuo | Calo massimo | Netto a costi doppi | DSR | PBO | Esito |");
b.AppendLine("|---|---|---:|---:|---:|---:|---:|---|");
foreach (var run in outcome.Runs)
{
var r = run.Report;
var doubled = r.Costs.FirstOrDefault(c => c.Multiplier == 2m);
b.Append("| ").Append(run.Name).Append(" | `").Append(r.Chosen).Append("` | ").Append(Pct(r.Metrics.AnnualReturn)).Append(" | ").Append(Pct(-r.Metrics.MaxDrawdown))
.Append(" | ").Append(doubled is null ? "—" : Money(doubled.NetResult, r.Currency)).Append(" | ").Append(r.DeflatedSharpe.Value.ToString("0.000", It))
.Append(" | ").Append(r.Overfitting is { } o ? o.Probability.ToString("0.000", It) : "—").Append(" | ").Append(r.Passed ? "superata" : "non superata").AppendLine(" |");
}
b.AppendLine();
b.AppendLine("## Cosa conserva il bot").AppendLine();
b.AppendLine(Text($"- Calo massimo del backtest: {Pct(-(double)q.MaxDrawdown)}. Dal vivo, un calo oltre 1,5 volte questo riporta la strategia indietro (SPEC 12.3)."));
b.AppendLine(Text($"- Scostamento di prezzo previsto: {q.ExpectedSlippageBp.ToString("0.##", It)} punti base; costo stimato di andata e ritorno {Pct((double)q.RoundTripCost, 2)}."));
if (q.ExpectedPathP5.Count > 0)
{
b.AppendLine(Text($"- Percorso atteso al 5° percentile (ricampionamento a blocchi dei rendimenti giornalieri): {Pct(q.ExpectedPathP5[Math.Min(29, q.ExpectedPathP5.Count - 1)])} dopo 30 giorni, {Pct(q.ExpectedPathP5[Math.Min(89, q.ExpectedPathP5.Count - 1)])} dopo 90, {Pct(q.ExpectedPathP5[^1])} dopo {q.ExpectedPathP5.Count}. Venti sedute di fila sotto questo percorso riportano la strategia indietro."));
}
b.AppendLine();
foreach (var run in outcome.Runs)
{
Trial(b, run);
}
b.AppendLine("## Cosa il backtest non rappresenta").AppendLine();
foreach (var line in first.Limitations)
{
b.Append("- ").AppendLine(line);
}
return b.ToString();
}
/// <summary>The report of a strategy whose backtest cannot run yet, with what it waits for.</summary>
public static string Waiting(string strategy, string name, Qualification qualification, IReadOnlyList<string> lines)
{
ArgumentNullException.ThrowIfNull(qualification);
ArgumentNullException.ThrowIfNull(lines);
var b = new StringBuilder();
b.Append("# Backtest di `").Append(strategy).Append("` (").Append(name).AppendLine(")").AppendLine();
b.Append("**Esito: in attesa.** ").AppendLine(qualification.Verdict).AppendLine();
foreach (var line in lines)
{
b.AppendLine(line).AppendLine();
}
return b.ToString();
}
private static void Trial(StringBuilder b, ProtocolRun run)
{
var r = run.Report;
var m = r.Metrics;
var h = r.Benchmark;
// Validations and variants keep the Sharpe of one session: shown per year, like the metrics above.
var annual = Math.Sqrt(r.PeriodsPerYear);
b.Append("## ").AppendLine(run.Name).AppendLine();
b.AppendLine(run.Description).AppendLine();
b.AppendLine(Text($"Dal {m.From:dd/MM/yyyy} al {m.To:dd/MM/yyyy}, {r.Sessions} chiusure giornaliere, capitale iniziale {Money(m.InitialCapital, r.Currency)}. Variante scelta: `{r.Chosen}` (la zona della griglia dove il risultato cambia meno, non la migliore). Impronta del piano `{r.PlanHash[..12]}`, dei dati `{r.DataFingerprint[..12]}`."))
.AppendLine();
b.AppendLine("| | Strategia | Comprare e tenere |").AppendLine("|---|---:|---:|");
Row(b, "Risultato netto", Money(m.NetResult, r.Currency), Money(h.NetResult, r.Currency));
Row(b, "Rendimento totale", Pct(m.TotalReturn), Pct(h.TotalReturn));
Row(b, "Rendimento annuo", Pct(m.AnnualReturn), Pct(h.AnnualReturn));
Row(b, "Volatilità annua", Pct(m.Volatility), Pct(h.Volatility));
Row(b, "Calo massimo", Pct(-m.MaxDrawdown) + Text($" ({m.DrawdownPeak:MM/yyyy} → {m.DrawdownLow:MM/yyyy})"), Pct(-h.MaxDrawdown));
Row(b, "Sharpe annuo", Num(m.Sharpe), Num(h.Sharpe));
Row(b, "Operazioni chiuse", m.Trades.ToString(It), h.Trades.ToString(It));
Row(b, "Operazioni in guadagno", Pct(m.WinRate), "—");
Row(b, "Rapporto guadagni/perdite", m.ProfitFactor is { } pf ? Num(pf) : "—", "—");
Row(b, "Commissioni pagate", Money(m.Fees, r.Currency), Money(h.Fees, r.Currency));
Row(b, "Tempo investito", Pct(m.Exposure), Pct(h.Exposure));
b.AppendLine();
b.AppendLine("**Per anno**").AppendLine();
b.AppendLine("| Anno | Strategia | Comprare e tenere |").AppendLine("|---|---:|---:|");
var held = h.Years.ToDictionary(y => y.Year);
foreach (var y in m.Years)
{
Row(b, y.Year.ToString(CultureInfo.InvariantCulture), Pct(y.Return), held.TryGetValue(y.Year, out var hy) ? Pct(hy.Return) : "—");
}
b.AppendLine();
b.AppendLine("**Sensibilità ai costi** (commissioni e scostamento moltiplicati insieme)").AppendLine();
b.AppendLine("| Costi | Risultato netto | Rendimento totale | Sharpe annuo | Calo massimo | Commissioni |").AppendLine("|---|---:|---:|---:|---:|---:|");
foreach (var c in r.Costs)
{
b.AppendLine(Text($"| ×{c.Multiplier:0} | {Money(c.NetResult, r.Currency)} | {Pct(c.TotalReturn)} | {Num(c.Sharpe)} | {Pct(-c.MaxDrawdown)} | {Money(c.Fees, r.Currency)} |"));
}
b.AppendLine();
b.AppendLine("**Convalide**").AppendLine();
if (r.WalkForward is { } wf)
{
b.AppendLine(Text($"- Finestre che avanzano ({wf.Windows.Count}, con {wf.Purge} giorni di stacco): rendimento fuori campione {Pct(wf.Return)}, Sharpe annuo {Num(wf.Sharpe * annual)}."));
}
if (r.Combinatorial is { } cv)
{
b.AppendLine(Text($"- K-fold purgato combinatorio ({cv.Groups} gruppi, {cv.TestGroups} di prova, {cv.Splits} divisioni): Sharpe annuo medio dei percorsi {Num(cv.MeanSharpe * annual)}, peggiore {Num(cv.WorstSharpe * annual)}, percorsi positivi {Pct(cv.PositiveShare)}."));
}
b.AppendLine(Text($"- Deflated Sharpe Ratio {Num(r.DeflatedSharpe.Value)}, contando {r.TrialsCounted} varianti provate in tutto."));
b.AppendLine(r.Overfitting is { } pbo
? Text($"- Probabilità di sovra-adattamento (PBO) {Num(pbo.Probability)}, su {pbo.Combinations} combinazioni di {pbo.Variants} varianti.")
: "- Probabilità di sovra-adattamento: non calcolabile con una sola variante.");
b.AppendLine(Text($"- Sguardo in avanti: {r.LookAhead.Decisions} decisioni rifatte tagliando i dati in {r.LookAhead.Cutoffs.Count} punti, {r.LookAhead.Changed} cambiate."));
b.AppendLine();
b.AppendLine("**Soglie**").AppendLine();
b.AppendLine("| Controllo | Valore | Soglia | Esito |").AppendLine("|---|---:|---:|---|");
foreach (var c in r.Checks)
{
var value = c.Value is not { } v ? "—" : c.Unit switch
{
"percent" => Pct(v),
"money" => Money((decimal)v, r.Currency),
"count" => v.ToString("0", It),
_ => v.ToString("0.000", It),
};
var threshold = c.Unit == "percent" ? Pct(c.Threshold, 0) : c.Unit == "money" ? Money((decimal)c.Threshold, r.Currency) : c.Threshold.ToString("0.##", It);
b.AppendLine(Text($"| {c.Name} | {value} | {c.Comparison} {threshold} | {(c.Passed is null ? "non si applica" : c.Passed.Value ? "superato" : "**non superato**")} |"));
}
b.AppendLine();
b.AppendLine("**Varianti della griglia**").AppendLine();
b.AppendLine("| Parametri | Sharpe annuo | Stabilità della zona | Risultato netto | Calo massimo | Operazioni |").AppendLine("|---|---:|---:|---:|---:|---:|");
foreach (var v in r.Variants)
{
b.AppendLine(Text($"| `{v.Parameters}`{(v.Chosen ? " (scelta)" : "")} | {Num(v.Sharpe * annual)} | {Num(v.PlateauScore * annual)} | {Money(v.NetResult, r.Currency)} | {Pct(-v.MaxDrawdown)} | {v.Trades} |"));
}
var adjusted = r.Notes.Where(n => n.Key.StartsWith(BacktestReasons.AdjustedPrefix, StringComparison.Ordinal))
.ToDictionary(n => n.Key[BacktestReasons.AdjustedPrefix.Length..], n => n.Value, StringComparer.Ordinal);
var engine = r.Notes.Where(n => !n.Key.StartsWith(BacktestReasons.AdjustedPrefix, StringComparison.Ordinal)).ToDictionary(n => n.Key, n => n.Value, StringComparer.Ordinal);
void Counts(string title, string what, IReadOnlyDictionary<string, int> counts)
{
b.AppendLine();
b.AppendLine(title).AppendLine();
if (counts.Count == 0)
{
b.AppendLine("Nessuno.");
return;
}
b.AppendLine(what).AppendLine();
b.AppendLine("| Motivo | Volte |").AppendLine("|---|---:|");
foreach (var (reason, count) in counts.OrderByDescending(n => n.Value).ThenBy(n => n.Key, StringComparer.Ordinal))
{
b.AppendLine(Text($"| `{reason}` | {count} |"));
}
}
Counts("**Interventi di correttivi e rischio sulla variante scelta**",
"Quante volte, per strumento e giorno, un correttivo o una regola di rischio ha cambiato ciò che la strategia chiedeva.", adjusted);
Counts("**Note del motore di prova**", "Ordini non eseguiti come chiesto: sotto il minimo, residui lasciati dopo un'uscita, liquidità.", engine);
b.AppendLine();
b.AppendLine("**Strumenti**").AppendLine();
b.AppendLine("| Coppia | Prima barra | Ultima barra | Barre |").AppendLine("|---|---|---|---:|");
foreach (var i in r.Instruments)
{
b.AppendLine(Text($"| {i.Symbol} | {i.First:yyyy-MM-dd} | {i.Last:yyyy-MM-dd} | {i.Bars} |"));
}
b.AppendLine();
}
private static void Row(StringBuilder b, string label, string strategy, string benchmark) =>
b.Append("| ").Append(label).Append(" | ").Append(strategy).Append(" | ").Append(benchmark).AppendLine(" |");
private static string Pct(double fraction, int decimals = 1) =>
(fraction < 0 ? "−" : "") + Math.Abs(fraction * 100).ToString("N" + decimals.ToString(CultureInfo.InvariantCulture), It) + "%";
private static string Num(double value) => (value < 0 ? "−" : "") + Math.Abs(value).ToString("0.00", It);
private static string Money(decimal value, string currency) =>
(value < 0 ? "−" : "") + Math.Abs(value).ToString("#,0.00", It) + " " + (currency == "EUR" ? "€" : currency);
private static string Text(FormattableString text) => text.ToString(It);
}
+284
View File
@@ -0,0 +1,284 @@
using System.Diagnostics;
using System.Globalization;
using System.Text.Json;
using Anubis.Data;
using Anubis.Strategies;
namespace Anubis.Backtest;
/// <summary>An instrument of the report: where its bars come from and what period they cover.</summary>
public sealed record ReportInstrument(string Symbol, string Venue, DateOnly? First, DateOnly? Last, int Bars, int Actions, string Source);
/// <summary>A point of the equity curve of the chosen variant and of buy and hold, on the same day.</summary>
public sealed record ReportPoint(DateOnly Day, decimal Equity, decimal Benchmark);
/// <summary>A variant of the grid with its full-period figures.</summary>
public sealed record ReportVariant(string Parameters, double Sharpe, double PlateauScore, decimal NetResult, double MaxDrawdown, int Trades, bool Chosen);
/// <summary>The chosen variant with costs multiplied (SPEC 12.2 point 3).</summary>
public sealed record ReportCosts(decimal Multiplier, decimal NetResult, double TotalReturn, double Sharpe, double MaxDrawdown, decimal Fees);
public sealed record ReportWindow(int Window, DateOnly TrainFrom, DateOnly TrainTo, DateOnly TestFrom, DateOnly TestTo, string Chosen, double Return, double Sharpe);
public sealed record ReportWalkForward(IReadOnlyList<ReportWindow> Windows, int Purge, double Return, double Sharpe);
public sealed record ReportCombinatorial(int Groups, int TestGroups, int Splits, int Purge, int Embargo, IReadOnlyList<double> PathSharpes, double MeanSharpe, double WorstSharpe, double PositiveShare);
public sealed record ReportOverfitting(int Groups, int Combinations, int Variants, double Probability);
public sealed record ReportLookAhead(IReadOnlyList<DateOnly> Cutoffs, int Decisions, int Changed, IReadOnlyList<string> Examples)
{
public bool Passed => Changed == 0;
}
/// <summary>
/// A threshold of SPEC 12.2 checked on the result. <see cref="Passed"/> is null when the check does not apply (PBO with
/// a single variant).
/// </summary>
public sealed record ReportCheck(string Key, string Name, double? Value, string Comparison, double Threshold, string Unit, bool? Passed);
public static class ReportCheckKeys
{
public const string DoubleCosts = "double_costs";
public const string DeflatedSharpe = "deflated_sharpe";
public const string Overfitting = "overfitting";
public const string Drawdown = "drawdown";
public const string LookAhead = "look_ahead";
}
/// <summary>
/// The final report of a trial (SPEC 12.2 point 6): figures of the chosen variant, buy and hold, cost sensitivity, the
/// grid, the validations, the checks and what the backtest does not model.
/// </summary>
public sealed record BacktestReport(
Guid TrialId,
string Strategy,
string StrategyName,
string StrategyVersion,
string PlanHash,
string DataFingerprint,
string Currency,
int PeriodsPerYear,
int Sessions,
IReadOnlyList<ReportInstrument> Instruments,
string Chosen,
BacktestMetrics Metrics,
BacktestMetrics Benchmark,
IReadOnlyList<ReportCosts> Costs,
IReadOnlyList<ReportVariant> Variants,
ReportWalkForward? WalkForward,
ReportCombinatorial? Combinatorial,
ReportOverfitting? Overfitting,
DeflatedSharpe DeflatedSharpe,
int TrialsCounted,
ReportLookAhead LookAhead,
IReadOnlyList<ReportCheck> Checks,
bool Passed,
IReadOnlyList<string> Limitations,
IReadOnlyList<ReportPoint> Curve,
IReadOnlyList<BacktestTrade> Trades,
IReadOnlyDictionary<string, int> Notes,
DateTimeOffset ComputedAt,
double Seconds)
{
public string ToJson() => JsonSerializer.Serialize(this, TrialPlan.Json);
public static BacktestReport FromJson(string text) => JsonSerializer.Deserialize<BacktestReport>(text, TrialPlan.Json) ?? throw new JsonException("Empty report");
}
/// <summary>
/// Runs a trial from its plan, on data already loaded (pure computation, no database: the laboratory runs it on its own
/// low-priority thread). Every variant of the grid runs once on the whole period; the validations work on their daily
/// returns; the chosen variant (plateau rule on the whole period) is then run again with doubled and tripled costs.
/// </summary>
public static class TrialRunner
{
public const int MaxCurvePoints = 500;
public const int MaxTrades = 300;
public static (BacktestReport Report, TrialSummary Summary) Run(
Guid trialId, TrialPlan plan, string planHash, IStrategyFamily family, BacktestDataset data, PriorTrials prior, DateTimeOffset now,
Action<string>? progress = null, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(plan);
ArgumentNullException.ThrowIfNull(family);
ArgumentNullException.ThrowIfNull(data);
ArgumentNullException.ThrowIfNull(prior);
var watch = Stopwatch.StartNew();
var sessions = data.Sessions.Count(d => d >= plan.Data.From && d <= plan.Data.To);
if (sessions < 30)
{
throw new InvalidOperationException($"Dati insufficienti: {sessions} sedute, ne servono almeno 30.");
}
var variants = ParameterGrid.Of(family, plan.Grid);
if (variants.Count == 0)
{
throw new InvalidOperationException("Nessuna combinazione valida nella griglia dei parametri.");
}
// The strategy goes through its correctives and the risk engine; buy and hold, the comparison, never does.
var plain = new BacktestOptions { InitialCapital = plan.InitialCapital, From = plan.Data.From, To = plan.Data.To };
var options = StrategyPipeline.For(family.Id, data, plain, plan.ReferenceShare);
var runs = new BacktestRun[variants.Count];
for (var i = 0; i < variants.Count; i++)
{
cancellationToken.ThrowIfCancellationRequested();
progress?.Invoke($"variante {i + 1} di {variants.Count}");
runs[i] = BacktestEngine.Run(family.Create(variants[i].Parameters), data, options);
}
var returns = runs.Select(r => r.DailyReturns()).ToArray();
var sharpes = returns.Select(r => Moments.Of(r).Sharpe).ToArray();
var scores = Validation.PlateauScores(variants, sharpes);
var chosenIndex = Validation.ChoosePlateau(variants, sharpes);
var chosen = variants[chosenIndex];
var run = runs[chosenIndex];
var metrics = BacktestMetrics.Of(run, data.PeriodsPerYear);
progress?.Invoke("costi doppi e tripli, comprare e tenere");
var costs = new List<ReportCosts>();
foreach (var multiplier in plan.CostMultipliers.Distinct().Order())
{
var costed = multiplier == 1m ? run : BacktestEngine.Run(family.Create(chosen.Parameters), data, options with { Costs = new CostModel(multiplier) });
var m = multiplier == 1m ? metrics : BacktestMetrics.Of(costed, data.PeriodsPerYear);
costs.Add(new ReportCosts(multiplier, costed.NetResult, m.TotalReturn, m.Sharpe, m.MaxDrawdown, costed.Fees));
}
var benchmarkRun = BacktestEngine.Run(new BuyAndHold(data.Tradable.Count()), data, plain);
var benchmark = BacktestMetrics.Of(benchmarkRun, data.PeriodsPerYear);
cancellationToken.ThrowIfCancellationRequested();
progress?.Invoke("convalide");
var days = run.Equity.Select(p => p.Day).ToArray();
DateOnly DayOf(int returnIndex) => days[Math.Clamp(returnIndex + 1, 0, days.Length - 1)];
var v = plan.Validation;
var wf = Validation.WalkForward(variants, returns, v.WalkForwardWindows);
var walkForward = wf is null ? null : new ReportWalkForward(
[.. wf.Windows.Select(w => new ReportWindow(w.Window, DayOf(w.TrainFrom), DayOf(w.TrainTo - 1), DayOf(w.TestFrom), DayOf(w.TestTo - 1),
variants[w.Chosen].ToString(), w.TestReturn, w.TestSharpe))],
wf.Purge, wf.Return, wf.Sharpe);
var cv = Validation.Combinatorial(variants, returns, v.CombinatorialGroups, v.CombinatorialTestGroups);
var combinatorial = cv is null ? null : new ReportCombinatorial(cv.Groups, cv.TestGroups, cv.Splits, cv.Purge, cv.Embargo, cv.PathSharpes,
cv.MeanSharpe, cv.WorstSharpe, cv.PositiveShare);
var pbo = Validation.Overfitting(returns, v.OverfittingGroups);
var overfitting = pbo is null ? null : new ReportOverfitting(pbo.Groups, pbo.Combinations, pbo.Variants, pbo.Probability);
// Every variant of every finished trial of this strategy counts as a try (SPEC 12.2 point 1).
var trials = prior.Variants + variants.Count;
var spread = Moments.Of([.. prior.Sharpes, .. sharpes]).StandardDeviation;
var dsr = SharpeTests.Deflated(returns[chosenIndex], trials, spread * spread);
progress?.Invoke("test anti sguardo in avanti");
var look = LookAheadCheck.Run(_ => family.Create(chosen.Parameters), data, options, v.LookAheadCuts);
var lookAhead = new ReportLookAhead(look.Cutoffs, look.Decisions, look.Mismatches.Count,
[.. look.Mismatches.Take(5).Select(m => $"{m.Day:yyyy-MM-dd}: {m.Full} / {m.Cut}")]);
var t = plan.Thresholds;
var doubled = costs.FirstOrDefault(c => c.Multiplier == 2m);
List<ReportCheck> checks =
[
new(ReportCheckKeys.DoubleCosts, "Risultato netto con costi doppi", doubled is null ? null : (double)doubled.NetResult, ">", (double)t.MinNetAtDoubleCosts, "money",
doubled is null ? false : doubled.NetResult > t.MinNetAtDoubleCosts),
new(ReportCheckKeys.DeflatedSharpe, "Deflated Sharpe Ratio", dsr.Value, "≥", t.MinDeflatedSharpe, "ratio", dsr.Value >= t.MinDeflatedSharpe),
new(ReportCheckKeys.Overfitting, "Probabilità di sovra-adattamento (PBO)", pbo?.Probability, "≤", t.MaxOverfitting, "ratio",
pbo is null ? null : pbo.Probability <= t.MaxOverfitting),
new(ReportCheckKeys.Drawdown, "Calo massimo", metrics.MaxDrawdown, "≤", t.MaxDrawdown, "percent", metrics.MaxDrawdown <= t.MaxDrawdown),
new(ReportCheckKeys.LookAhead, "Decisioni cambiate aggiungendo dati successivi", look.Mismatches.Count, "=", 0, "count", look.Passed),
];
var passed = checks.All(c => c.Passed != false);
var benchmarkByDay = benchmarkRun.Equity.ToDictionary(p => p.Day, p => p.Equity);
var step = Math.Max(1, (int)Math.Ceiling(run.Equity.Count / (double)MaxCurvePoints));
var curve = run.Equity
.Where((_, i) => i % step == 0 || i == run.Equity.Count - 1)
.Select(p => new ReportPoint(p.Day, p.Equity, benchmarkByDay.GetValueOrDefault(p.Day, plan.InitialCapital)))
.ToList();
var report = new BacktestReport(
trialId,
family.Id,
family.Name,
family.Version,
planHash,
data.Fingerprint(),
data.Currency,
data.PeriodsPerYear,
sessions,
[.. data.Instruments.Select(i => new ReportInstrument(i.Symbol, i.Spec.Venue, i.Bars.Count > 0 ? i.Bars[0].Day : null,
i.Bars.Count > 0 ? i.Bars[^1].Day : null, i.Bars.Count, i.Actions.Count, i.Source))],
chosen.ToString(),
metrics,
benchmark,
costs,
[.. variants.Select((x, i) => new ReportVariant(x.ToString(), sharpes[i], scores[i], runs[i].NetResult,
Metrics(runs[i], data).MaxDrawdown, runs[i].Trades.Count(tr => tr.IsClosed), i == chosenIndex))],
walkForward,
combinatorial,
overfitting,
dsr,
trials,
lookAhead,
checks,
passed,
Limitations(plan, family, data),
curve,
[.. run.Trades.TakeLast(MaxTrades)],
run.Notes.GroupBy(n => n.Reason).ToDictionary(g => g.Key, g => g.Count(), StringComparer.Ordinal),
now,
watch.Elapsed.TotalSeconds);
var summary = new TrialSummary(data.Currency, chosen.ToString(), run.NetResult, metrics.TotalReturn, metrics.Sharpe, metrics.MaxDrawdown, metrics.Trades,
dsr.Value, pbo?.Probability, sharpes);
return (report, summary);
}
private static BacktestMetrics Metrics(BacktestRun run, BacktestDataset data) => BacktestMetrics.Of(run, data.PeriodsPerYear);
/// <summary>What this backtest does not model, in plain Italian (SPEC 12.1: the report declares it).</summary>
public static IReadOnlyList<string> Limitations(TrialPlan plan, IStrategyFamily family, BacktestDataset data)
{
ArgumentNullException.ThrowIfNull(plan);
ArgumentNullException.ThrowIfNull(family);
ArgumentNullException.ThrowIfNull(data);
var result = new List<string>
{
"I correttivi senza una storia ricostruibile (notizie, calendario economico, trimestrali, modello) non ricevono dati e non intervengono.",
"Le decisioni si prendono sulla chiusura e si eseguono all'apertura della seduta dopo, con lo scostamento stimato: nessun inseguimento con ordini limite.",
"Sharpe e volatilità sono calcolati senza tasso privo di rischio.",
};
if (StrategyPipeline.IsBase(family.Id))
{
var share = plan.ReferenceShare ?? StrategyPipeline.DefaultShare(family.Id) ?? 1m;
result.Add(string.Create(CultureInfo.GetCultureInfo("it-IT"),
$"Limiti di rischio e regime crypto applicati come dal vivo. La strategia è il {share * 100m:0.#}% del bot: il resto conta come liquidità ferma. Ogni chiusura vale come un giorno; se scatta l'interruttore, supponiamo che tu riparta dopo sette giorni. Età dei prezzi, prezzo dell'ordine e orologio esistono solo dal vivo."));
}
else
{
result.Add("Strategia d'esempio: correttivi e limiti di rischio non si applicano, ogni intenzione passa così com'è.");
}
if (plan.Data.Source == TrialSources.Research && plan.Data.Venue == EuroHistory.Venue)
{
result.Add("Prezzi dello storico Binance in USDT convertiti in euro con il cambio BCE del giorno, contando un USDT come un dollaro: è un'approssimazione delle coppie in euro di OKX. " +
"Su questo storico lo spread non si può misurare: l'universo sono le monete con più scambi, passo di prezzo e quantità minima sono stime, i costi sono quelli di OKX.");
}
else if (plan.Data.Source == TrialSources.Research)
{
result.Add("Prezzi dello storico Binance in USDT: passo di prezzo e quantità minima sono stime, i costi sono quelli di OKX.");
}
if (data.Instruments.Any(i => i.Spec.AssetClass == AssetClasses.Equity && i.Actions.Count == 0))
{
result.Add("Per alcuni titoli non risultano frazionamenti né dividendi nel periodo: se ce ne sono stati, mancano dal database.");
}
if (family.Id == "sma_cross")
{
result.Add("L'incrocio di medie è una strategia d'esempio per provare il laboratorio, non una delle strategie di base.");
}
return result;
}
}
+247
View File
@@ -0,0 +1,247 @@
using System.Runtime.CompilerServices;
using Anubis.Data;
using Anubis.Risk;
using Anubis.Strategies;
using Anubis.Trading;
namespace Anubis.Backtest;
/// <summary>
/// The risk engine of SPEC 9 in the backtest of one strategy (SPEC 12.1): the same loss monitor and size rules as
/// live, measured on the strategy's closes. The strategy is one part of the bot; the rest is counted as cash that
/// does not move, so a loss of the strategy weighs on the world as its share does. Two choices of the backtest
/// (ADR 0011): every close is its own Italian day, and after the switch trips the user is assumed to restart a week
/// later, the high starting again from there. Price age, order price and clock exist only live.
/// </summary>
public sealed class BacktestRisk(RiskLimits? limits = null) : IIntentStage
{
/// <summary>When the user is assumed to confirm the restart after the switch tripped.</summary>
public static readonly TimeSpan AssumedRestart = TimeSpan.FromDays(7);
private readonly RiskLimits _limits = limits ?? RiskLimits.Default;
private readonly ConditionalWeakTable<IReadOnlyList<EquityMark>, Walker> _walkers = [];
public string Name => "risk";
public RiskLimits Limits => _limits;
/// <summary>Each close as a moment: noon UTC, inside its own Italian day.</summary>
public static DateTimeOffset At(DateOnly day) => new(day.ToDateTime(new TimeOnly(12, 0)), TimeSpan.Zero);
public IReadOnlyList<Intent> Apply(StrategyContext context, IReadOnlyList<Intent> intents)
{
ArgumentNullException.ThrowIfNull(context);
ArgumentNullException.ThrowIfNull(intents);
var walker = _walkers.GetValue(context.History, _ => new Walker(_limits));
var (state, measures, other) = walker.Advance(context);
var now = At(context.Day);
if (state.Tripped)
{
// "Ferma tutto": nothing is bought or sold on the strategy's initiative; stops still follow it (SPEC 9.3).
return [.. intents.Select(i => i.TargetWeight is null ? i : (i with { TargetWeight = null, StopOffset = null }).Adjusted(RiskReasons.SwitchTripped))];
}
var blocked = RiskMonitor.EntriesBlocked(state, now);
var multiplier = RiskMonitor.Multiplier(state, measures, now, _limits);
var equity = context.Portfolio.Equity;
var invested = 0m;
var crypto = 0m;
foreach (var (symbol, h) in context.Portfolio.Holdings.Where(h => h.Value.Quantity > 0m))
{
var value = h.Quantity * BookWeights.Price(context, symbol);
invested += value;
if (context.Spec(symbol)?.AssetClass == AssetClasses.Crypto)
{
crypto += value;
}
}
var result = new List<Intent>(intents.Count);
foreach (var intent in intents)
{
if (intent.TargetWeight is not { } target || equity <= 0m)
{
result.Add(intent);
continue;
}
var current = BookWeights.Current(context, intent.Symbol);
if (target <= current)
{
result.Add(intent);
continue;
}
if (blocked)
{
result.Add(BookWeights.Reduce(context, intent, 0m, RiskReasons.DailyLossBlock));
continue;
}
var price = BookWeights.Price(context, intent.Symbol);
var holding = context.Portfolio.Of(intent.Symbol);
var held = holding?.Quantity ?? 0m;
var stop = intent.Stop ?? holding?.Stop ?? (intent.StopOffset is { } offset ? price - offset : null);
var isCrypto = context.Spec(intent.Symbol)?.AssetClass == AssetClasses.Crypto;
var wanted = (target * equity / price) - held;
var size = RiskRules.SizePurchase(new PurchaseRequest
{
Price = price,
Held = held,
Wanted = wanted,
Stop = stop,
Multiplier = multiplier,
WorldEquity = equity + other,
WorldInvested = invested,
Reference = context.ReferenceEquity,
IsCrypto = isCrypto,
CryptoValue = crypto,
}, _limits);
var bought = size.Reduced ? size.Quantity : wanted;
invested += bought * price;
crypto += isCrypto ? bought * price : 0m;
if (!size.Reduced)
{
result.Add(intent);
continue;
}
var weight = Math.Round((held + size.Quantity) * price / equity, 6, MidpointRounding.ToZero);
var adjusted = intent;
foreach (var reason in size.Reasons)
{
adjusted = adjusted.Adjusted(reason);
}
result.Add(weight > current
? adjusted with { TargetWeight = weight }
: held > 0m ? adjusted with { TargetWeight = null } : adjusted with { TargetWeight = null, Stop = null, StopOffset = null });
}
// A position heavier than the threshold comes back to it at the first chance (SPEC 9.2).
foreach (var (symbol, h) in context.Portfolio.Holdings.Where(h => h.Value.Quantity > 0m))
{
var price = BookWeights.Price(context, symbol);
if (RiskRules.Excess(h.Quantity, price, context.ReferenceEquity, _limits) <= 0m)
{
continue;
}
var most = Math.Round(_limits.MaxPosition * context.ReferenceEquity / equity, 6, MidpointRounding.ToZero);
var index = result.FindLastIndex(i => i.Symbol == symbol);
if (index >= 0 && result[index].TargetWeight is { } w && w <= most)
{
continue;
}
if (index >= 0)
{
result[index] = (result[index] with { TargetWeight = most }).Adjusted(RiskReasons.PositionLimit);
}
else
{
result.Add(new Intent(symbol, most, RiskReasons.PositionLimit) { Adjustments = [RiskReasons.PositionLimit] });
}
}
return result;
}
/// <summary>The loss monitor walked along one run's closes, one step for each new close.</summary>
private sealed class Walker(RiskLimits limits)
{
private readonly Lock _lock = new();
private readonly List<IndexPoint> _points = [];
private RiskMonitorState _state = RiskMonitorState.Initial;
private RiskMeasures _measures = RiskMeasures.None;
private decimal _peak;
private decimal? _other;
public (RiskMonitorState State, RiskMeasures Measures, decimal Other) Advance(StrategyContext context)
{
lock (_lock)
{
var history = context.History;
// The rest of the bot, as cash that does not move: the strategy's first equity times (1 / share − 1).
_other ??= history.Count > 0 && context.Portfolio.Equity > 0m
? Math.Max(0m, history[0].Equity * ((context.ReferenceEquity / context.Portfolio.Equity) - 1m))
: 0m;
for (var i = _points.Count; i < history.Count; i++)
{
var at = At(history[i].Day);
var index = history[i].Equity + _other.Value;
if (_state.Tripped && _state.TrippedAt is { } tripped && at >= tripped + AssumedRestart)
{
_state = RiskMonitor.Restart(_state, at);
_peak = 0m;
}
_points.Add(new IndexPoint(at, index));
_peak = Math.Max(_peak, index);
var dayBase = i > 0 ? _points[i - 1].Index : index;
var weekBase = _points[WeekBase(at - TimeSpan.FromDays(7))].Index;
_measures = new RiskMeasures(RiskMeasures.Change(index, dayBase), RiskMeasures.Change(index, weekBase), RiskMeasures.Change(index, _peak));
_state = RiskMonitor.Step(_state, _measures, at, limits).State;
}
return (_state, _measures, _other.Value);
}
}
private int WeekBase(DateTimeOffset at)
{
int lo = 0, hi = _points.Count - 1, found = 0;
while (lo <= hi)
{
var mid = (lo + hi) / 2;
if (_points[mid].At <= at)
{
found = mid;
lo = mid + 1;
}
else
{
hi = mid - 1;
}
}
return found;
}
}
}
/// <summary>
/// The full path of a decision in the backtest (SPEC 5.2): the base strategies go through their correctives and the
/// risk engine, with the share of the bot each one has by default (SPEC 8.6: 40%, 45% and 15% with the example
/// capital). The examples of the laboratory and the buy-and-hold comparison go through as they are.
/// </summary>
public static class StrategyPipeline
{
public static decimal? DefaultShare(string strategy) => strategy switch
{
StrategyNames.CryptoTrend => 0.40m,
StrategyNames.EquityMomentum => 0.45m,
StrategyNames.EquityMeanrev => 0.15m,
_ => null,
};
public static bool IsBase(string strategy) => DefaultShare(strategy) is not null;
public static BacktestOptions For(string strategy, BacktestDataset data, BacktestOptions options, decimal? share = null)
{
ArgumentNullException.ThrowIfNull(data);
ArgumentNullException.ThrowIfNull(options);
if (!IsBase(strategy))
{
return options;
}
IReadOnlyList<IIntentStage> correctives = [];
if (strategy == StrategyNames.CryptoTrend && data.Tradable.FirstOrDefault(i => Symbols.BaseOf(i.Spec) == "BTC") is { } bitcoin)
{
correctives = [new CryptoRegimeStage(bitcoin.Symbol)];
}
return options with { Correctives = correctives, Risk = new BacktestRisk(), ReferenceShare = share ?? options.ReferenceShare ?? DefaultShare(strategy) };
}
}
+184
View File
@@ -0,0 +1,184 @@
namespace Anubis.Backtest;
/// <summary>The standard normal distribution, written in house (ADR 0003: no statistics package).</summary>
public static class Normal
{
/// <summary>Φ(x) with Hart's algorithm 5666 as given by West (2004), accurate to double precision.</summary>
public static double Cdf(double x)
{
var z = Math.Abs(x);
double tail;
if (z > 37)
{
tail = 0;
}
else if (z < 7.07106781186547)
{
double[] p = [3.52624965998911E-02, 0.700383064443688, 6.37396220353165, 33.912866078383, 112.079291497871, 221.213596169931, 220.206867912376];
double[] q = [8.83883476483184E-02, 1.75566716318264, 16.064177579207, 86.7807322029461, 296.564248779674, 637.333633378831, 793.826512519948, 440.413735824752];
tail = Math.Exp(-z * z / 2) * Horner(p, z) / Horner(q, z);
}
else
{
var b = z + 0.65;
b = z + (4 / b);
b = z + (3 / b);
b = z + (2 / b);
b = z + (1 / b);
tail = Math.Exp(-z * z / 2) / b / 2.506628274631;
}
return x > 0 ? 1 - tail : tail;
}
/// <summary>Φ⁻¹(p) with Acklam's rational approximation, refined with one Halley step on <see cref="Cdf"/>.</summary>
public static double Quantile(double p)
{
if (p <= 0.0)
{
return double.NegativeInfinity;
}
if (p >= 1.0)
{
return double.PositiveInfinity;
}
double[] a = [-3.969683028665376e+01, 2.209460984245205e+02, -2.759285104469687e+02, 1.383577518672690e+02, -3.066479806614716e+01, 2.506628277459239e+00];
double[] b = [-5.447609879822406e+01, 1.615858368580409e+02, -1.556989798598866e+02, 6.680131188771972e+01, -1.328068155288572e+01, 1.0];
double[] c = [-7.784894002430293e-03, -3.223964580411365e-01, -2.400758277161838e+00, -2.549732539343734e+00, 4.374664141464968e+00, 2.938163982698783e+00];
double[] d = [7.784695709041462e-03, 3.224671290700398e-01, 2.445134137142996e+00, 3.754408661907416e+00, 1.0];
const double low = 0.02425;
double x;
if (p < low)
{
var q = Math.Sqrt(-2 * Math.Log(p));
x = Horner(c, q) / Horner(d, q);
}
else if (p <= 1 - low)
{
var q = p - 0.5;
var r = q * q;
x = Horner(a, r) * q / Horner(b, r);
}
else
{
var q = Math.Sqrt(-2 * Math.Log(1 - p));
x = -Horner(c, q) / Horner(d, q);
}
var e = Cdf(x) - p;
var u = e * Math.Sqrt(2 * Math.PI) * Math.Exp(x * x / 2);
return x - (u / (1 + (x * u / 2)));
}
/// <summary>A polynomial with coefficients from the highest power down, evaluated at <paramref name="x"/>.</summary>
private static double Horner(double[] coefficients, double x)
{
var result = 0.0;
foreach (var k in coefficients)
{
result = (result * x) + k;
}
return result;
}
}
/// <summary>Moments of a series of returns, as the formulas of Bailey and López de Prado use them.</summary>
public readonly record struct Moments(int Count, double Mean, double StandardDeviation, double Skewness, double Kurtosis)
{
/// <summary>Sharpe ratio per period, without a risk-free rate: mean over standard deviation.</summary>
public double Sharpe => StandardDeviation > 0 ? Mean / StandardDeviation : 0;
/// <summary>Mean, sample standard deviation, population skewness and (non-excess) kurtosis: 3 for a normal series.</summary>
public static Moments Of(ReadOnlySpan<double> values)
{
var n = values.Length;
if (n == 0)
{
return new Moments(0, 0, 0, 0, 3);
}
var mean = 0.0;
foreach (var v in values)
{
mean += v;
}
mean /= n;
double m2 = 0, m3 = 0, m4 = 0;
foreach (var v in values)
{
var d = v - mean;
var d2 = d * d;
m2 += d2;
m3 += d2 * d;
m4 += d2 * d2;
}
var sample = n > 1 ? Math.Sqrt(m2 / (n - 1)) : 0;
var population = m2 / n;
var skew = population > 0 ? m3 / n / Math.Pow(population, 1.5) : 0;
var kurt = population > 0 ? m4 / n / (population * population) : 3;
return new Moments(n, mean, sample, skew, kurt);
}
}
/// <summary>The outcome of the Deflated Sharpe Ratio: every number that went into it, for the report.</summary>
public sealed record DeflatedSharpe(double Sharpe, double Benchmark, int Observations, double Skewness, double Kurtosis, int Trials, double SharpeVariance, double Value);
/// <summary>
/// Probabilistic and Deflated Sharpe Ratio (Bailey and López de Prado, "The Sharpe Ratio Efficient Frontier",
/// 2012; "The Deflated Sharpe Ratio", 2014), written in house as SPEC 12.2 asks. Sharpe ratios are per period
/// (daily), not annualised.
/// </summary>
public static class SharpeTests
{
private const double EulerGamma = 0.5772156649015329;
/// <summary>Probability that the true Sharpe ratio exceeds <paramref name="benchmark"/>, given the sample's length and shape.</summary>
public static double Probabilistic(double sharpe, double benchmark, int observations, double skewness, double kurtosis)
{
if (observations < 2)
{
return 0.5;
}
var denominator = 1 - (skewness * sharpe) + ((kurtosis - 1) / 4 * sharpe * sharpe);
if (denominator <= 0)
{
return sharpe > benchmark ? 1 : 0;
}
return Normal.Cdf((sharpe - benchmark) * Math.Sqrt(observations - 1) / Math.Sqrt(denominator));
}
/// <summary>
/// The Sharpe ratio the best of <paramref name="trials"/> independent tries would show by chance alone, when
/// their Sharpe ratios vary by <paramref name="variance"/> around zero. One trial expects zero.
/// </summary>
public static double ExpectedMaximum(int trials, double variance)
{
if (trials <= 1 || variance <= 0)
{
return 0;
}
return Math.Sqrt(variance) * (((1 - EulerGamma) * Normal.Quantile(1 - (1.0 / trials))) + (EulerGamma * Normal.Quantile(1 - (1.0 / (trials * Math.E)))));
}
/// <summary>The Deflated Sharpe Ratio of a series, after <paramref name="trials"/> tries whose Sharpe ratios had variance <paramref name="variance"/>.</summary>
public static DeflatedSharpe Deflated(ReadOnlySpan<double> returns, int trials, double variance)
{
var m = Moments.Of(returns);
return Deflated(m.Sharpe, m.Count, m.Skewness, m.Kurtosis, trials, variance);
}
public static DeflatedSharpe Deflated(double sharpe, int observations, double skewness, double kurtosis, int trials, double variance)
{
var benchmark = ExpectedMaximum(trials, variance);
return new DeflatedSharpe(sharpe, benchmark, observations, skewness, kurtosis, trials, variance,
Probabilistic(sharpe, benchmark, observations, skewness, kurtosis));
}
}
+238
View File
@@ -0,0 +1,238 @@
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using System.Text.Json.Serialization;
using Anubis.Core;
using Anubis.Db;
namespace Anubis.Backtest;
/// <summary>Where the bars of a trial come from.</summary>
public static class TrialSources
{
/// <summary>Daily bars and corporate actions collected in the database (SPEC 6.2).</summary>
public const string Database = "database";
/// <summary>The research history on disk (SPEC 6.6): Binance daily archives from 2017, in USDT.</summary>
public const string Research = "research";
}
/// <summary>
/// The data a trial reads: one venue, a few instruments, a period (both days included). The bars of the
/// <see cref="WarmupDays"/> before <see cref="From"/> are read too, only so the indicators and the listing age are ready on
/// the first day: the simulation, the measures and buy and hold start at <see cref="From"/>.
/// </summary>
public sealed record TrialData(string Source, string Venue, IReadOnlyList<string> Symbols, DateOnly From, DateOnly To)
{
/// <summary>Calendar days of history before the start: a year of 252 sessions, a 360-day lookback, 400 days of listing.</summary>
public const int WarmupDays = 450;
/// <summary>The first day read from the history.</summary>
public DateOnly HistoryStart() => From.AddDays(-WarmupDays);
}
/// <summary>
/// The thresholds of SPEC 12.2, fixed in the plan before the trial runs: changing them after seeing the results needs
/// an ADR.
/// </summary>
public sealed record TrialThresholds(decimal MinNetAtDoubleCosts, double MinDeflatedSharpe, double MaxOverfitting, double MaxDrawdown)
{
public static TrialThresholds Default { get; } = new(0m, 0.90, 0.40, 0.25);
}
/// <summary>How the trial validates: walk-forward windows, purged cross-validation, PBO blocks, look-ahead cuts.</summary>
public sealed record TrialValidation(int WalkForwardWindows, int CombinatorialGroups, int CombinatorialTestGroups, int OverfittingGroups, int LookAheadCuts)
{
public static TrialValidation Default { get; } = new(4, 6, 2, 16, 6);
}
/// <summary>
/// The plan of a trial (SPEC 12.2 point 1): rules, grid, data, costs, validations and thresholds, written before any
/// result and never changed. Its hash goes with the report.
/// </summary>
public sealed record TrialPlan(
string Strategy,
string StrategyVersion,
IReadOnlyDictionary<string, IReadOnlyList<decimal>> Grid,
TrialData Data,
decimal InitialCapital,
IReadOnlyList<decimal> CostMultipliers,
TrialValidation Validation,
TrialThresholds Thresholds,
string Engine,
DateTimeOffset WrittenAt)
{
/// <summary>
/// The strategy's share of the bot (SPEC 9.2): the reference equity of the backtest is its capital divided by it.
/// Null for the examples, which are not part of the bot.
/// </summary>
public decimal? ReferenceShare { get; init; }
public static readonly JsonSerializerOptions Json = new()
{
PropertyNamingPolicy = JsonNamingPolicy.SnakeCaseLower,
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
NumberHandling = JsonNumberHandling.AllowNamedFloatingPointLiterals,
WriteIndented = false,
};
public string ToJson() => JsonSerializer.Serialize(this, Json);
public static TrialPlan FromJson(string text) => JsonSerializer.Deserialize<TrialPlan>(text, Json) ?? throw new JsonException("Empty plan");
/// <summary>SHA-256 of the plan's text as stored.</summary>
public static string Hash(string json) => Convert.ToHexStringLower(SHA256.HashData(Encoding.UTF8.GetBytes(json)));
}
public static class TrialStatuses
{
public const string Planned = "planned";
public const string Running = "running";
public const string Done = "done";
public const string Failed = "failed";
}
public static class TrialVerdicts
{
public const string Passed = "passed";
public const string Rejected = "rejected";
}
/// <summary>The few numbers of a finished trial the list shows, and the Sharpe ratios the next trials' DSR counts.</summary>
public sealed record TrialSummary(
string Currency,
string Chosen,
decimal NetResult,
double TotalReturn,
double Sharpe,
double MaxDrawdown,
int Trades,
double? DeflatedSharpe,
double? Overfitting,
IReadOnlyList<double> VariantSharpes);
/// <summary>A row of the trial register.</summary>
public sealed record Trial(
Guid Id,
string Strategy,
string StrategyVersion,
TrialPlan Plan,
string PlanHash,
int Variants,
string Status,
string? Verdict,
string? DataFingerprint,
TrialSummary? Summary,
string? Error,
DateTimeOffset CreatedAt,
DateTimeOffset? StartedAt,
DateTimeOffset? FinishedAt)
{
public bool IsFinished => Status is TrialStatuses.Done or TrialStatuses.Failed;
/// <summary>The stored plan text still gives the stored hash: nobody changed it after it was written.</summary>
public bool PlanIntact { get; init; } = true;
}
/// <summary>The trials before this one of the same strategy: how many variants they tried and their Sharpe ratios.</summary>
public sealed record PriorTrials(int Trials, int Variants, IReadOnlyList<double> Sharpes)
{
public static PriorTrials None { get; } = new(0, 0, []);
}
/// <summary>
/// The trial register (SPEC 12.2 point 1). A plan is inserted once and never updated: the store has no method that
/// changes it, and every later write touches only status, results and times.
/// </summary>
public sealed class TrialStore(Database db, IClock clock)
{
private const string Columns = "id, strategy, strategy_version, plan, plan_hash, variants, status, verdict, data_fingerprint, summary, error, created_at, started_at, finished_at";
public async Task<Trial> PlanAsync(TrialPlan plan, int variants, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(plan);
var id = Ids.NewId(clock);
var json = plan.ToJson();
var now = clock.UtcNow;
await db.WriteAsync(s => s.ExecuteAsync(
"INSERT INTO backtest_trials (id, strategy, strategy_version, plan, plan_hash, variants, status, created_at) " +
"VALUES (@id, @s, @v, @p, @h, @n, @st, @now)",
cancellationToken, ("id", id), ("s", plan.Strategy), ("v", plan.StrategyVersion), ("p", new DbJson(json)), ("h", TrialPlan.Hash(json)),
("n", variants), ("st", TrialStatuses.Planned), ("now", now)), cancellationToken).ConfigureAwait(false);
return (await GetAsync(id, cancellationToken).ConfigureAwait(false))!;
}
public Task<Trial?> GetAsync(Guid id, CancellationToken cancellationToken) =>
db.ReadAsync(s => s.FirstOrDefaultAsync($"SELECT {Columns} FROM backtest_trials WHERE id = @id", Read, ("id", id)), cancellationToken);
public async Task<IReadOnlyList<Trial>> ListAsync(int limit, CancellationToken cancellationToken) =>
await db.ReadAsync(s => s.QueryAsync($"SELECT {Columns} FROM backtest_trials ORDER BY created_at DESC, id DESC LIMIT @n", Read, ("n", limit)),
cancellationToken).ConfigureAwait(false);
public Task<string?> ReportJsonAsync(Guid id, CancellationToken cancellationToken) =>
db.ReadAsync(s => s.FirstOrDefaultAsync("SELECT report FROM backtest_trials WHERE id = @id AND report IS NOT NULL", r => r.Json("report").Text, ("id", id)),
cancellationToken);
/// <summary>Marks a planned trial as running. False when it was not planned (another worker took it, or it is finished).</summary>
public async Task<bool> StartAsync(Guid id, CancellationToken cancellationToken) =>
await db.WriteAsync(s => s.ExecuteAsync("UPDATE backtest_trials SET status = @r, started_at = @now WHERE id = @id AND status = @p",
cancellationToken, ("r", TrialStatuses.Running), ("now", clock.UtcNow), ("id", id), ("p", TrialStatuses.Planned)), cancellationToken).ConfigureAwait(false) == 1;
public Task FinishAsync(Guid id, string fingerprint, TrialSummary summary, string reportJson, bool passed, CancellationToken cancellationToken) =>
db.WriteAsync(s => s.ExecuteAsync(
"UPDATE backtest_trials SET status = @d, verdict = @v, data_fingerprint = @f, summary = @s, report = @rep, finished_at = @now WHERE id = @id AND status = @r",
cancellationToken, ("d", TrialStatuses.Done), ("v", passed ? TrialVerdicts.Passed : TrialVerdicts.Rejected), ("f", fingerprint),
("s", new DbJson(JsonSerializer.Serialize(summary, TrialPlan.Json))), ("rep", new DbJson(reportJson)), ("now", clock.UtcNow), ("id", id),
("r", TrialStatuses.Running)), cancellationToken);
public Task FailAsync(Guid id, string error, CancellationToken cancellationToken) =>
db.WriteAsync(s => s.ExecuteAsync("UPDATE backtest_trials SET status = @f, error = @e, finished_at = @now WHERE id = @id AND status IN (@p, @r)",
cancellationToken, ("f", TrialStatuses.Failed), ("e", error), ("now", clock.UtcNow), ("id", id), ("p", TrialStatuses.Planned),
("r", TrialStatuses.Running)), cancellationToken);
/// <summary>
/// After a restart: trials left running go back to planned (their work was lost with the process), and every planned
/// trial is returned oldest first, to run again from the start with the same plan.
/// </summary>
public async Task<IReadOnlyList<Guid>> ResumeAsync(CancellationToken cancellationToken) =>
await db.WriteAsync(async s =>
{
await s.ExecuteAsync("UPDATE backtest_trials SET status = @p, started_at = NULL WHERE status = @r", cancellationToken,
("p", TrialStatuses.Planned), ("r", TrialStatuses.Running)).ConfigureAwait(false);
return await s.QueryAsync("SELECT id FROM backtest_trials WHERE status = @p ORDER BY created_at, id", r => r.Id("id"), ("p", TrialStatuses.Planned))
.ConfigureAwait(false);
}, cancellationToken).ConfigureAwait(false);
/// <summary>The finished trials of <paramref name="strategy"/> other than <paramref name="except"/>: what the DSR deflates for.</summary>
public async Task<PriorTrials> PriorAsync(string strategy, Guid except, CancellationToken cancellationToken)
{
var rows = await db.ReadAsync(s => s.QueryAsync(
"SELECT variants, summary FROM backtest_trials WHERE strategy = @s AND id <> @id AND status = @d",
r => (Variants: (int)r.Number("variants"), Summary: r.IsNull("summary") ? null : JsonSerializer.Deserialize<TrialSummary>(r.Json("summary").Text, TrialPlan.Json)),
("s", strategy), ("id", except), ("d", TrialStatuses.Done)), cancellationToken).ConfigureAwait(false);
return new PriorTrials(rows.Count, rows.Sum(r => r.Variants), [.. rows.SelectMany(r => r.Summary?.VariantSharpes ?? [])]);
}
private static Trial Read(DbRow r)
{
var plan = r.Json("plan").Text;
return new Trial(
r.Id("id"),
r.Text("strategy"),
r.Text("strategy_version"),
TrialPlan.FromJson(plan),
r.Text("plan_hash"),
(int)r.Number("variants"),
r.Text("status"),
r.TextOrNull("verdict"),
r.TextOrNull("data_fingerprint"),
r.IsNull("summary") ? null : JsonSerializer.Deserialize<TrialSummary>(r.Json("summary").Text, TrialPlan.Json),
r.TextOrNull("error"),
r.Moment("created_at"),
r.MomentOrNull("started_at"),
r.MomentOrNull("finished_at"))
{
PlanIntact = TrialPlan.Hash(plan) == r.Text("plan_hash"),
};
}
}
+469
View File
@@ -0,0 +1,469 @@
using Anubis.Strategies;
namespace Anubis.Backtest;
/// <summary>One point of a parameter grid: its parameters and its place on each parameter's grid (for neighbours).</summary>
public sealed record Variant(int Index, StrategyParameters Parameters, IReadOnlyList<int> Position)
{
public override string ToString() => Parameters.ToString();
}
/// <summary>The variants of a strategy family: every point of the grid the family accepts, in a fixed order.</summary>
public static class ParameterGrid
{
/// <summary>
/// The Cartesian product of the grids (the family's own, or <paramref name="grids"/> where given), without the
/// points the family rejects. A family without parameters has one variant.
/// </summary>
public static IReadOnlyList<Variant> Of(IStrategyFamily family, IReadOnlyDictionary<string, IReadOnlyList<decimal>>? grids = null)
{
ArgumentNullException.ThrowIfNull(family);
var axes = family.Parameters
.Select(p => (p.Name, Values: (IReadOnlyList<decimal>)[.. (grids?.GetValueOrDefault(p.Name) ?? p.Grid).Distinct().Order()]))
.ToList();
if (axes.Any(a => a.Values.Count == 0))
{
throw new ArgumentException("Every parameter needs at least one value on its grid", nameof(grids));
}
var result = new List<Variant>();
var position = new int[axes.Count];
while (true)
{
var parameters = new StrategyParameters(axes.Select((a, i) => KeyValuePair.Create(a.Name, a.Values[position[i]])));
if (family.Accepts(parameters))
{
result.Add(new Variant(result.Count, parameters, [.. position]));
}
// Next point: the last parameter moves fastest.
var k = axes.Count - 1;
while (k >= 0 && ++position[k] == axes[k].Values.Count)
{
position[k] = 0;
k--;
}
if (k < 0)
{
return result;
}
}
}
}
/// <summary>Count, sum and sum of squares of some daily returns: enough for a mean and a Sharpe ratio.</summary>
internal readonly record struct ReturnSums(int Count, double Sum, double Squares)
{
public static ReturnSums operator +(ReturnSums a, ReturnSums b) => new(a.Count + b.Count, a.Sum + b.Sum, a.Squares + b.Squares);
public static ReturnSums Of(double[] returns, int from, int to)
{
double sum = 0, squares = 0;
for (var t = from; t < to; t++)
{
sum += returns[t];
squares += returns[t] * returns[t];
}
return new ReturnSums(Math.Max(to - from, 0), sum, squares);
}
/// <summary>Daily Sharpe ratio: mean over sample standard deviation, 0 when it cannot be computed.</summary>
public double Sharpe
{
get
{
if (Count < 2)
{
return 0;
}
var mean = Sum / Count;
var variance = (Squares - (Count * mean * mean)) / (Count - 1);
return variance > 1e-30 ? mean / Math.Sqrt(variance) : 0;
}
}
}
/// <summary>A choice of variant on some data and how it then did on other data.</summary>
public sealed record WalkForwardWindow(int Window, int TrainFrom, int TrainTo, int TestFrom, int TestTo, int Chosen, double TestReturn, double TestSharpe);
/// <summary>Walk-forward validation: each window chooses on the past and is judged on what follows.</summary>
public sealed record WalkForwardResult(IReadOnlyList<WalkForwardWindow> Windows, int Purge, double Return, double Sharpe);
/// <summary>Combinatorial purged cross-validation: every path of out-of-sample choices, with its Sharpe ratio.</summary>
public sealed record CombinatorialResult(int Groups, int TestGroups, int Splits, int Purge, int Embargo, IReadOnlyList<int> Chosen, IReadOnlyList<double> PathSharpes)
{
public int Paths => PathSharpes.Count;
public double MeanSharpe => PathSharpes.Count > 0 ? PathSharpes.Average() : 0;
public double WorstSharpe => PathSharpes.Count > 0 ? PathSharpes.Min() : 0;
public double PositiveShare => PathSharpes.Count > 0 ? (double)PathSharpes.Count(s => s > 0) / PathSharpes.Count : 0;
}
/// <summary>Probability of backtest overfitting by CSCV, with the logits it counts.</summary>
public sealed record OverfittingResult(int Groups, int Combinations, int Variants, double Probability, IReadOnlyList<double> Logits);
/// <summary>
/// The validations of SPEC 12.2 points 4 and 5 on the daily returns of every variant (one array per variant, all the
/// same length and on the same sessions). Choices follow the plateau rule of point 5: a variant scores its own Sharpe
/// ratio, but no more than the mean of its zone (itself and its neighbours on the grid), so a lonely peak loses to a
/// broad hill and a poor variant does not win by standing next to a good one.
/// </summary>
public static class Validation
{
/// <summary>
/// Each variant's value, capped at the mean of its zone: itself and the variants one step away on every axis.
/// </summary>
public static double[] PlateauScores(IReadOnlyList<Variant> variants, IReadOnlyList<double> values)
{
ArgumentNullException.ThrowIfNull(variants);
ArgumentNullException.ThrowIfNull(values);
var scores = new double[variants.Count];
for (var i = 0; i < variants.Count; i++)
{
double sum = 0;
var count = 0;
for (var j = 0; j < variants.Count; j++)
{
if (Neighbours(variants[i].Position, variants[j].Position))
{
sum += values[j];
count++;
}
}
scores[i] = Math.Min(values[i], sum / count);
}
return scores;
}
/// <summary>The variant with the best plateau score; ties go to the first in grid order.</summary>
public static int ChoosePlateau(IReadOnlyList<Variant> variants, IReadOnlyList<double> values)
{
var scores = PlateauScores(variants, values);
var best = 0;
for (var i = 1; i < scores.Length; i++)
{
if (scores[i] > scores[best])
{
best = i;
}
}
return best;
}
/// <summary>Sessions to drop around a test block: 1% of the series, at least one (López de Prado, AFML ch. 7).</summary>
public static int DefaultGap(int observations) => Math.Max(1, (int)Math.Ceiling(observations * 0.01));
/// <summary>
/// Walk-forward with a growing training window: the series is cut into <paramref name="windows"/> + 1 equal blocks;
/// window k chooses on blocks 0..k (less the last <paramref name="purge"/> sessions, which touch the test) and is
/// judged on block k + 1. Null when the series is too short for the blocks.
/// </summary>
public static WalkForwardResult? WalkForward(IReadOnlyList<Variant> variants, IReadOnlyList<double[]> returns, int windows = 4, int? purge = null)
{
var length = CheckShape(variants, returns);
var gap = purge ?? DefaultGap(length);
var block = length / (windows + 1);
if (windows < 1 || block < 2 + gap)
{
return null;
}
var result = new List<WalkForwardWindow>();
var outOfSample = new List<double>();
for (var k = 0; k < windows; k++)
{
var testFrom = (k + 1) * block;
var testTo = k == windows - 1 ? length : testFrom + block;
var trainTo = testFrom - gap;
var sharpes = returns.Select(r => ReturnSums.Of(r, 0, trainTo).Sharpe).ToArray();
var chosen = ChoosePlateau(variants, sharpes);
var test = returns[chosen][testFrom..testTo];
outOfSample.AddRange(test);
result.Add(new WalkForwardWindow(k + 1, 0, trainTo, testFrom, testTo, chosen, Compound(test), ReturnSums.Of(test, 0, test.Length).Sharpe));
}
var all = outOfSample.ToArray();
return new WalkForwardResult(result, gap, Compound(all), ReturnSums.Of(all, 0, all.Length).Sharpe);
}
/// <summary>
/// Combinatorial purged cross-validation (López de Prado, AFML ch. 12): the series is cut into
/// <paramref name="groups"/> blocks; every choice of <paramref name="testGroups"/> of them is a test set, the rest
/// trains, less <paramref name="purge"/> sessions before each test block and <paramref name="embargo"/> after it.
/// Each block is tested in several splits; taking them in order builds the paths, each a full out-of-sample series.
/// </summary>
public static CombinatorialResult? Combinatorial(IReadOnlyList<Variant> variants, IReadOnlyList<double[]> returns, int groups = 6, int testGroups = 2, int? purge = null, int? embargo = null)
{
var length = CheckShape(variants, returns);
ArgumentOutOfRangeException.ThrowIfLessThan(groups, 2);
ArgumentOutOfRangeException.ThrowIfLessThan(testGroups, 1);
ArgumentOutOfRangeException.ThrowIfGreaterThanOrEqual(testGroups, groups);
var before = purge ?? DefaultGap(length);
var after = embargo ?? DefaultGap(length);
var bounds = Bounds(length, groups);
if (bounds.Any(b => b.To - b.From < 2 + before + after))
{
return null;
}
var splits = Combinations(groups, testGroups);
var chosen = new int[splits.Count];
// For each group, the out-of-sample returns of every split that tests it, in split order.
var tested = Enumerable.Range(0, groups).Select(_ => new List<double[]>()).ToArray();
for (var s = 0; s < splits.Count; s++)
{
var test = splits[s];
var train = new bool[length];
for (var g = 0; g < groups; g++)
{
if (!test.Contains(g))
{
for (var t = bounds[g].From; t < bounds[g].To; t++)
{
train[t] = true;
}
}
}
foreach (var g in test)
{
for (var t = Math.Max(0, bounds[g].From - before); t < bounds[g].From; t++)
{
train[t] = false;
}
for (var t = bounds[g].To; t < Math.Min(length, bounds[g].To + after); t++)
{
train[t] = false;
}
}
var sharpes = returns.Select(r => Masked(r, train).Sharpe).ToArray();
chosen[s] = ChoosePlateau(variants, sharpes);
foreach (var g in test)
{
tested[g].Add(returns[chosen[s]][bounds[g].From..bounds[g].To]);
}
}
var paths = tested[0].Count;
var pathSharpes = new double[paths];
for (var p = 0; p < paths; p++)
{
var path = tested.SelectMany(g => g[p]).ToArray();
pathSharpes[p] = ReturnSums.Of(path, 0, path.Length).Sharpe;
}
return new CombinatorialResult(groups, testGroups, splits.Count, before, after, chosen, pathSharpes);
}
/// <summary>
/// Probability of backtest overfitting by combinatorially symmetric cross-validation (Bailey, Borwein, López de
/// Prado and Zhu, 2015). The series is cut into <paramref name="groups"/> blocks; for every half chosen as
/// in-sample, the variant with the best in-sample Sharpe is ranked among all variants on the other half. The logit
/// of its relative rank is negative when it lands in the worse half; PBO is the share of those cases. Null with
/// fewer than two variants or too short a series.
/// </summary>
public static OverfittingResult? Overfitting(IReadOnlyList<double[]> returns, int groups = 16)
{
ArgumentNullException.ThrowIfNull(returns);
if (returns.Count < 2)
{
return null;
}
var length = returns[0].Length;
if (returns.Any(r => r.Length != length))
{
throw new ArgumentException("Every variant needs returns on the same sessions", nameof(returns));
}
// Even, and at least 4 sessions per block.
groups = Math.Min(groups, length / 4) / 2 * 2;
if (groups < 4)
{
return null;
}
var bounds = Bounds(length, groups);
var n = returns.Count;
var sums = new ReturnSums[n, groups];
for (var v = 0; v < n; v++)
{
for (var g = 0; g < groups; g++)
{
sums[v, g] = ReturnSums.Of(returns[v], bounds[g].From, bounds[g].To);
}
}
var logits = new List<double>();
var outOfSample = new double[n];
foreach (var inSample in Combinations(groups, groups / 2))
{
var isIn = new bool[groups];
foreach (var g in inSample)
{
isIn[g] = true;
}
var best = -1;
var bestSharpe = double.NegativeInfinity;
for (var v = 0; v < n; v++)
{
ReturnSums a = default, b = default;
for (var g = 0; g < groups; g++)
{
if (isIn[g])
{
a += sums[v, g];
}
else
{
b += sums[v, g];
}
}
if (a.Sharpe > bestSharpe)
{
bestSharpe = a.Sharpe;
best = v;
}
outOfSample[v] = b.Sharpe;
}
// Rank 1 is the worst out of sample, n the best; ties share the average rank.
double below = 0, equal = 0;
for (var v = 0; v < n; v++)
{
if (v == best)
{
continue;
}
if (outOfSample[v] < outOfSample[best])
{
below++;
}
else if (outOfSample[v] == outOfSample[best])
{
equal++;
}
}
var omega = (1 + below + (equal / 2)) / (n + 1);
logits.Add(Math.Log(omega / (1 - omega)));
}
return new OverfittingResult(groups, logits.Count, n, (double)logits.Count(l => l <= 0) / logits.Count, logits);
}
/// <summary>Growth of 1 over the returns, less 1.</summary>
public static double Compound(IEnumerable<double> returns)
{
ArgumentNullException.ThrowIfNull(returns);
var value = 1d;
foreach (var r in returns)
{
value *= 1 + r;
}
return value - 1;
}
/// <summary>Every way of choosing <paramref name="k"/> of <paramref name="n"/> items, in lexicographic order.</summary>
internal static List<int[]> Combinations(int n, int k)
{
var result = new List<int[]>();
var current = new int[k];
for (var i = 0; i < k; i++)
{
current[i] = i;
}
while (true)
{
result.Add([.. current]);
var j = k - 1;
while (j >= 0 && current[j] == n - k + j)
{
j--;
}
if (j < 0)
{
return result;
}
current[j]++;
for (var i = j + 1; i < k; i++)
{
current[i] = current[i - 1] + 1;
}
}
}
private static (int From, int To)[] Bounds(int length, int groups)
{
var result = new (int, int)[groups];
for (var g = 0; g < groups; g++)
{
result[g] = ((int)((long)length * g / groups), (int)((long)length * (g + 1) / groups));
}
return result;
}
private static ReturnSums Masked(double[] returns, bool[] mask)
{
ReturnSums sums = default;
for (var t = 0; t < returns.Length; t++)
{
if (mask[t])
{
sums += new ReturnSums(1, returns[t], returns[t] * returns[t]);
}
}
return sums;
}
private static bool Neighbours(IReadOnlyList<int> a, IReadOnlyList<int> b)
{
for (var k = 0; k < a.Count; k++)
{
if (Math.Abs(a[k] - b[k]) > 1)
{
return false;
}
}
return true;
}
private static int CheckShape(IReadOnlyList<Variant> variants, IReadOnlyList<double[]> returns)
{
ArgumentNullException.ThrowIfNull(variants);
ArgumentNullException.ThrowIfNull(returns);
if (variants.Count == 0 || variants.Count != returns.Count)
{
throw new ArgumentException("One return series per variant", nameof(returns));
}
var length = returns[0].Length;
if (returns.Any(r => r.Length != length))
{
throw new ArgumentException("Every variant needs returns on the same sessions", nameof(returns));
}
return length;
}
}
+105 -6
View File
@@ -118,6 +118,15 @@ public sealed record ScheduledJob(string Name, string Description, Schedule Sche
/// skipped and reported instead of executed blindly (trap 21, SPEC 5.6).
/// </summary>
public TimeSpan MaxLateness { get; init; } = TimeSpan.FromMinutes(15);
/// <summary>A move of the bot the user reads in "Prossime mosse" (SPEC 13.6), not a housekeeping job.</summary>
public bool IsMove { get; init; }
/// <summary>
/// A run missed while the bot was off is made once at the restart, queued, when the restart comes within this
/// time of the missed run; later it is skipped and reported (SPEC 5.6). Null: a missed run is simply not made.
/// </summary>
public TimeSpan? CatchUpWithin { get; init; }
}
public enum JobOutcome
@@ -128,10 +137,22 @@ public enum JobOutcome
}
/// <summary>Published on the bus after every run, skip or failure.</summary>
public sealed record JobRun(string Name, DateTimeOffset DueUtc, DateTimeOffset StartedUtc, DateTimeOffset FinishedUtc, JobOutcome Outcome, string? Error);
public sealed record JobRun(string Name, DateTimeOffset DueUtc, DateTimeOffset StartedUtc, DateTimeOffset FinishedUtc, JobOutcome Outcome, string? Error)
{
/// <summary>The run was missed while the bot was off and made, or skipped, at the restart (SPEC 5.6).</summary>
public bool Missed { get; init; }
public string? Description { get; init; }
}
/// <summary>A run coming up, for "Prossime mosse" and the jobs list.</summary>
public sealed record UpcomingRun(string Name, string Description, DateTimeOffset DueUtc);
public sealed record UpcomingRun(string Name, string Description, DateTimeOffset DueUtc)
{
public bool IsMove { get; init; }
/// <summary>A job that comes back at a fixed interval (every minute, every hour): background work, not an appointment.</summary>
public bool Periodic { get; init; }
}
/// <summary>
/// Runs jobs at fixed times, at intervals, or relative to market open and close (SPEC 5.5).
@@ -142,21 +163,60 @@ public sealed partial class Scheduler(IClock clock, EventBus bus, ILogger<Schedu
private readonly ILogger _logger = logger ?? NullLogger<Scheduler>.Instance;
private readonly Lock _gate = new();
private readonly Dictionary<string, (ScheduledJob Job, DateTimeOffset? Due)> _jobs = new(StringComparer.Ordinal);
private readonly Dictionary<string, DateTimeOffset> _recovering = new(StringComparer.Ordinal);
private readonly List<JobRun> _missed = [];
private TaskCompletionSource _changed = new(TaskCreationOptions.RunContinuationsAsynchronously);
/// <summary>Moment of the last loop iteration; the health check uses it to see the loop is alive.</summary>
public DateTimeOffset? LastBeatUtc { get; private set; }
public void Add(ScheduledJob job)
public void Add(ScheduledJob job) => Add(job, null);
/// <summary>
/// Adds a job whose last run, from an earlier life of the process, was due at <paramref name="lastDue"/>. A job with
/// <see cref="ScheduledJob.CatchUpWithin"/> that missed a run since then makes it once at the start, or reports it
/// skipped when the restart is too late (SPEC 5.6).
/// </summary>
public void Add(ScheduledJob job, DateTimeOffset? lastDue)
{
ArgumentNullException.ThrowIfNull(job);
var now = clock.UtcNow;
var missed = job.CatchUpWithin is not null && lastDue is { } last ? LastMissed(job.Schedule, last, now) : null;
lock (_gate)
{
_jobs[job.Name] = (job, job.Schedule.NextAfter(clock.UtcNow));
if (missed is { } due && now - due <= job.CatchUpWithin)
{
_jobs[job.Name] = (job, now);
_recovering[job.Name] = due;
}
else
{
_jobs[job.Name] = (job, job.Schedule.NextAfter(now));
if (missed is { } skipped)
{
_missed.Add(new JobRun(job.Name, skipped, now, now, JobOutcome.Skipped, "Saltato al riavvio: oltre l'orario utile.")
{ Missed = true, Description = job.Description });
}
}
_changed.TrySetResult();
}
}
/// <summary>The last due time after <paramref name="last"/> and not after <paramref name="now"/>: the run the absence took.</summary>
private static DateTimeOffset? LastMissed(Schedule schedule, DateTimeOffset last, DateTimeOffset now)
{
DateTimeOffset? missed = null;
var at = schedule.NextAfter(last);
for (var i = 0; i < 100_000 && at is { } due && due <= now; i++)
{
missed = due;
at = schedule.NextAfter(due);
}
return missed;
}
public IReadOnlyList<UpcomingRun> Upcoming(int count = 10)
{
lock (_gate)
@@ -164,7 +224,7 @@ public sealed partial class Scheduler(IClock clock, EventBus bus, ILogger<Schedu
return _jobs.Values.Where(j => j.Due is not null)
.OrderBy(j => j.Due)
.Take(count)
.Select(j => new UpcomingRun(j.Job.Name, j.Job.Description, j.Due!.Value))
.Select(j => new UpcomingRun(j.Job.Name, j.Job.Description, j.Due!.Value) { IsMove = j.Job.IsMove, Periodic = j.Job.Schedule is IntervalSchedule })
.ToList();
}
}
@@ -218,8 +278,13 @@ public sealed partial class Scheduler(IClock clock, EventBus bus, ILogger<Schedu
{
var now = clock.UtcNow;
List<(ScheduledJob Job, DateTimeOffset Due)> due;
List<JobRun> missed;
lock (_gate)
{
// The runs an absence took and the restart came too late for: reported once, at the first round.
missed = [.. _missed];
_missed.Clear();
due = _jobs.Values.Where(j => j.Due is { } d && d <= now).Select(j => (j.Job, j.Due!.Value)).OrderBy(j => j.Value).ToList();
foreach (var (job, _) in due)
{
@@ -227,9 +292,28 @@ public sealed partial class Scheduler(IClock clock, EventBus bus, ILogger<Schedu
}
}
foreach (var (job, dueAt) in due)
foreach (var run in missed)
{
bus.Publish(run);
}
foreach (var (job, queuedAt) in due)
{
var started = clock.UtcNow;
bool recovering;
DateTimeOffset dueAt;
lock (_gate)
{
recovering = _recovering.Remove(job.Name, out var missedDue);
dueAt = recovering ? missedDue : queuedAt;
}
if (recovering)
{
await RecoverAsync(job, dueAt, started, cancellationToken).ConfigureAwait(false);
continue;
}
if (started - dueAt > job.MaxLateness)
{
LogSkipped(_logger, job.Name, dueAt);
@@ -250,6 +334,21 @@ public sealed partial class Scheduler(IClock clock, EventBus bus, ILogger<Schedu
}
}
/// <summary>A run missed while the bot was off, made once now: never skipped for lateness, reported as recovered.</summary>
private async Task RecoverAsync(ScheduledJob job, DateTimeOffset dueAt, DateTimeOffset started, CancellationToken cancellationToken)
{
try
{
await job.Run(cancellationToken).ConfigureAwait(false);
bus.Publish(new JobRun(job.Name, dueAt, started, clock.UtcNow, JobOutcome.Completed, null) { Missed = true, Description = job.Description });
}
catch (Exception ex) when (ex is not OperationCanceledException || !cancellationToken.IsCancellationRequested)
{
LogFailed(_logger, ex, job.Name);
bus.Publish(new JobRun(job.Name, dueAt, started, clock.UtcNow, JobOutcome.Failed, ex.Message) { Missed = true, Description = job.Description });
}
}
[LoggerMessage(Level = LogLevel.Warning, Message = "Job {Job} skipped: it was due at {Due} and is too late to run")]
private static partial void LogSkipped(ILogger logger, string job, DateTimeOffset due);
+5 -1
View File
@@ -41,7 +41,11 @@ public sealed class SettingsStore(Database db, IClock clock, EventBus bus, IRead
public static readonly SettingDefinition CentralBankMeetings = new("calendar.central_banks", "Riunioni di Fed e BCE",
Data.CentralBanks.DefaultList, Check: Data.CentralBanks.Check);
public static readonly IReadOnlyList<SettingDefinition> All = [Theme, Currency, BotState, CentralBankMeetings];
/// <summary>The optional periodic status on Telegram (SPEC 14): never, every hour or every evening.</summary>
public static readonly SettingDefinition TelegramStatus = new("telegram.status", "Stato periodico su Telegram", "never", ["never", "hourly", "daily"]);
public static readonly IReadOnlyList<SettingDefinition> All =
[Theme, Currency, BotState, CentralBankMeetings, TelegramStatus, .. Risk.RiskSettings.All, Risk.Scenarios.Setting];
private readonly IReadOnlyDictionary<string, StartupOverride> _overrides = overrides ?? new Dictionary<string, StartupOverride>();
+95 -8
View File
@@ -54,7 +54,7 @@ public sealed partial class MarketDataService : IDisposable
private (string Key, string Secret, AlpacaClient Client)? _alpaca;
public MarketDataService(Database db, IClock clock, EventBus bus, SettingsStore settings, SecretStore secrets, DataDirectory? data,
HttpMessageHandler? handler = null, Func<IWebSocketTransport>? sockets = null, ILogger? logger = null)
HttpMessageHandler? handler = null, Func<IWebSocketTransport>? sockets = null, ILogger? logger = null, OkxOptions? okx = null)
{
_db = db;
_clock = clock;
@@ -73,7 +73,7 @@ public sealed partial class MarketDataService : IDisposable
_http.Timeout = TimeSpan.FromSeconds(60);
_http.DefaultRequestHeaders.UserAgent.ParseAdd($"{AnubisInfo.Name}/{AnubisInfo.Version}");
Okx = new OkxClient(_http, clock, null, sockets);
Okx = new OkxClient(_http, clock, okx, sockets);
Freshness = new FreshnessMonitor(clock, bus);
References = new ReferenceSources(new ApiHttp(_http, clock, "reference"), db, clock, settings, secrets.RevealAsync, Okx);
UsCalendar = new StoredMarketCalendar(db, clock, "us_equity");
@@ -105,21 +105,22 @@ public sealed partial class MarketDataService : IDisposable
{ MaxLateness = TimeSpan.FromHours(6) },
new("dati-azioni-chiusura", "Barre giornaliere delle azioni dopo la chiusura", Schedule.Market(MarketAnchor.Close, TimeSpan.FromMinutes(30), UsCalendar),
_ => WakeWorker()) { MaxLateness = TimeSpan.FromHours(6) },
// SPEC 5.6: after an absence the last rate available is downloaded at once.
new("dati-cambi-mattina", "Cambi di riferimento della BCE", Schedule.Daily(new TimeOnly(8, 0), Zones.Rome), References.RefreshFxAsync)
{ MaxLateness = TimeSpan.FromHours(4) },
{ MaxLateness = TimeSpan.FromHours(4), CatchUpWithin = TimeSpan.FromDays(3650) },
new("dati-cambi-pomeriggio", "Cambi di riferimento della BCE", Schedule.Daily(new TimeOnly(16, 30), Zones.Rome), References.RefreshFxAsync)
{ MaxLateness = TimeSpan.FromHours(4) },
{ MaxLateness = TimeSpan.FromHours(4), CatchUpWithin = TimeSpan.FromDays(3650) },
new("dati-calendario", "Calendario economico, riunioni e trimestrali", Schedule.Every(TimeSpan.FromHours(1)), RefreshCalendarAsync)
{ MaxLateness = TimeSpan.FromMinutes(50) },
new("dati-umore", "Umore del mercato", Schedule.Every(TimeSpan.FromHours(1)), References.RefreshMoodAsync) { MaxLateness = TimeSpan.FromMinutes(50) },
new("dati-spread", "Campione orario degli spread delle crypto", Schedule.Every(TimeSpan.FromHours(1)), SampleSpreadsAsync)
{ MaxLateness = TimeSpan.FromMinutes(50) },
new("dati-calendario-borsa", "Calendario di borsa di Alpaca", Schedule.Daily(new TimeOnly(4, 0), Zones.Rome), RefreshUsCalendarAsync)
{ MaxLateness = TimeSpan.FromHours(12) },
{ MaxLateness = TimeSpan.FromHours(12), CatchUpWithin = TimeSpan.FromDays(3650) },
new("dati-azioni-societarie", "Frazionamenti e dividendi prima dell'apertura", Schedule.Market(MarketAnchor.Open, TimeSpan.FromHours(-1), UsCalendar),
RefreshCorporateActionsAsync) { MaxLateness = TimeSpan.FromMinutes(50) },
new("dati-pulizia", "Pulizia delle barre al minuto e degli spread vecchi", Schedule.Daily(new TimeOnly(3, 45), Zones.Rome), PruneAsync)
{ MaxLateness = TimeSpan.FromHours(12) },
{ MaxLateness = TimeSpan.FromHours(12), CatchUpWithin = TimeSpan.FromDays(3650) },
];
/// <summary>
@@ -130,7 +131,8 @@ public sealed partial class MarketDataService : IDisposable
{
await UsCalendar.LoadAsync(cancellationToken).ConfigureAwait(false);
_work.Release();
await Task.WhenAll(WorkerAsync(cancellationToken), LiveAsync(cancellationToken), FreshnessAsync(cancellationToken)).ConfigureAwait(false);
await Task.WhenAll(WorkerAsync(cancellationToken), LiveAsync(cancellationToken), StockLiveAsync(cancellationToken), FreshnessAsync(cancellationToken))
.ConfigureAwait(false);
}
public Task WakeWorker()
@@ -419,7 +421,8 @@ public sealed partial class MarketDataService : IDisposable
await UsCalendar.SaveAsync(sessions, today.AddDays(-30), today.AddDays(90), cancellationToken).ConfigureAwait(false);
}
private async Task RefreshCorporateActionsAsync(CancellationToken cancellationToken)
/// <summary>Splits and dividends of the stock universe from Alpaca; read first at a restart, before any stop is touched (SPEC 5.6, 6.10).</summary>
public async Task RefreshCorporateActionsAsync(CancellationToken cancellationToken)
{
if (await AlpacaAsync(cancellationToken).ConfigureAwait(false) is not { } alpaca)
{
@@ -491,6 +494,90 @@ public sealed partial class MarketDataService : IDisposable
}
}
/// <summary>
/// The stock feed (IEX, issue #47): during the US session only, the stocks held, those with an order planned and the
/// market index, at most <see cref="StockStreamLimit"/> symbols as the free feed allows. Outside the session the
/// quotes come by REST when asked, and no stock is followed for freshness.
/// </summary>
private async Task StockLiveAsync(CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
try
{
var now = _clock.UtcNow;
var today = DateOnly.FromDateTime(now.UtcDateTime);
var session = UsCalendar.Sessions(today.AddDays(-1), today.AddDays(10)).FirstOrDefault(s => s.CloseUtc + StockStreamMargin > now);
if (session is null || session.OpenUtc - StockStreamMargin > now)
{
var next = session is null ? now.AddHours(1) : Min(session.OpenUtc - StockStreamMargin, now.AddHours(1));
await _clock.WaitUntilAsync(next, cancellationToken).ConfigureAwait(false);
continue;
}
var alpaca = await AlpacaAsync(cancellationToken).ConfigureAwait(false);
var instruments = alpaca is null ? [] : await StockLiveInstrumentsAsync(cancellationToken).ConfigureAwait(false);
if (alpaca is null || instruments.Count == 0)
{
await _clock.WaitUntilAsync(now.AddMinutes(5), cancellationToken).ConfigureAwait(false);
continue;
}
var end = session.CloseUtc + StockStreamMargin;
using var round = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
var collector = new MarketCollector(alpaca, _db, _clock, _bus, Freshness, logger: _logger);
var run = collector.RunLiveAsync(instruments, round.Token);
var ids = instruments.Select(i => i.Id).ToHashSet();
while (!run.IsCompleted)
{
using var nap = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
var wait = _clock.WaitUntilAsync(Min(_clock.UtcNow.AddMinutes(5), end), nap.Token);
await Task.WhenAny(run, wait).ConfigureAwait(false);
await nap.CancelAsync().ConfigureAwait(false);
if (!run.IsCompleted && (_clock.UtcNow >= end
|| !(await StockLiveInstrumentsAsync(cancellationToken).ConfigureAwait(false)).Select(i => i.Id).ToHashSet().SetEquals(ids)))
{
await round.CancelAsync().ConfigureAwait(false);
}
}
await run.ConfigureAwait(false);
foreach (var id in ids)
{
Freshness.Unfollow(id);
}
}
catch (OperationCanceledException) when (cancellationToken.IsCancellationRequested)
{
break;
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
LogStepFailed(_logger, ex, "live azioni");
await _clock.WaitUntilAsync(_clock.UtcNow.AddMinutes(1), cancellationToken).ConfigureAwait(false);
}
}
}
/// <summary>The free IEX feed follows at most 30 symbols.</summary>
public const int StockStreamLimit = 30;
private static readonly TimeSpan StockStreamMargin = TimeSpan.FromMinutes(5);
private static DateTimeOffset Min(DateTimeOffset a, DateTimeOffset b) => a <= b ? a : b;
private async Task<IReadOnlyList<Instrument>> StockLiveInstrumentsAsync(CancellationToken cancellationToken)
{
var symbols = await _db.ReadAsync(s => s.QueryAsync(
"SELECT DISTINCT i.symbol FROM instruments i WHERE i.venue = @v AND (i.symbol = @m"
+ " OR i.id IN (SELECT instrument_id FROM positions WHERE quantity <> 0)"
+ " OR i.id IN (SELECT instrument_id FROM planned_orders WHERE status = @p)) ORDER BY i.symbol",
r => r.Text("symbol"), ("v", Venues.Alpaca), ("m", Trading.StrategyNames.MarketSymbol), ("p", "pending")), cancellationToken).ConfigureAwait(false);
var wanted = symbols.OrderBy(s => s == Trading.StrategyNames.MarketSymbol ? 0 : 1).ThenBy(s => s, StringComparer.Ordinal).Take(StockStreamLimit).ToHashSet(StringComparer.Ordinal);
var all = await _catalog.ListAsync(Venues.Alpaca, cancellationToken).ConfigureAwait(false);
return all.Where(i => wanted.Contains(i.Symbol) && i.IsLive).ToList();
}
private async Task<IReadOnlyList<Instrument>> LiveInstrumentsAsync(CancellationToken cancellationToken)
{
var today = DateOnly.FromDateTime(_clock.UtcNow.UtcDateTime);
+27
View File
@@ -117,8 +117,16 @@ public sealed class Database : IAsyncDisposable
return builder.ToString();
}
/// <summary>Fault tests only (issue #47): while true every new connection fails as if the server could not be reached.</summary>
internal bool Unreachable { get; set; }
public async Task<DbConnection> OpenConnectionAsync(CancellationToken cancellationToken)
{
if (Unreachable)
{
throw new DatabaseUnreachableException();
}
if (_postgres is not null)
{
return await _postgres.OpenConnectionAsync(cancellationToken).ConfigureAwait(false);
@@ -239,6 +247,25 @@ public sealed class Database : IAsyncDisposable
}
/// <summary>One open transaction. Parameters are named <c>@name</c> on both engines.</summary>
/// <summary>The database could not be reached (fault tests).</summary>
public sealed class DatabaseUnreachableException : DbException
{
public DatabaseUnreachableException()
: base("Database non raggiungibile.")
{
}
public DatabaseUnreachableException(string message)
: base(message)
{
}
public DatabaseUnreachableException(string message, Exception innerException)
: base(message, innerException)
{
}
}
public sealed class DbSession(DbEngine engine, DbConnection connection, DbTransaction transaction)
{
public DbEngine Engine => engine;
+3
View File
@@ -91,6 +91,9 @@ public static class DbValues
("bool", DbEngine.Sqlite) => "INTEGER",
("bytes", DbEngine.Postgres) => "BYTEA",
("bytes", DbEngine.Sqlite) => "BLOB",
// A whole number beyond 2^31 (bytes of memory, counters of a long run): SQLite's INTEGER is already 64 bits.
("long", DbEngine.Postgres) => "BIGINT",
("long", DbEngine.Sqlite) => "INTEGER",
("float", DbEngine.Postgres) => "DOUBLE PRECISION",
("float", DbEngine.Sqlite) => "REAL",
("strict", DbEngine.Postgres) => "",
+25
View File
@@ -0,0 +1,25 @@
-- Phase 4 schema (SPEC 12.2, ADR 0010): the register of backtest trials of the laboratory.
-- One row per trial. The plan (strategy, grid, data, costs, validations, thresholds) is written before any result
-- and never changed afterwards: plan_hash is the SHA-256 of its text, so a report always shows the plan it answers.
-- Every trial stays, failed and passed alike: the Deflated Sharpe Ratio counts them all (SPEC 12.2 point 1).
CREATE TABLE backtest_trials (
id {uuid} PRIMARY KEY,
strategy TEXT NOT NULL,
strategy_version TEXT NOT NULL,
plan {json} NOT NULL,
plan_hash TEXT NOT NULL,
variants INTEGER NOT NULL,
status TEXT NOT NULL,
verdict TEXT,
data_fingerprint TEXT,
summary {json},
report {json},
error TEXT,
created_at {timestamp} NOT NULL,
started_at {timestamp},
finished_at {timestamp}
){strict};
CREATE INDEX backtest_trials_strategy ON backtest_trials (strategy, created_at);
CREATE INDEX backtest_trials_status ON backtest_trials (status);
+49
View File
@@ -0,0 +1,49 @@
-- Phase 5 schema (SPEC 5.2, 9, 12.3, 12.4, ADR 0011): the record of every stage of a decision, the state of the
-- risk engine for each world, and what a strategy needs to reach real money and to leave it.
-- One row for each stage a decision goes through (SPEC 5.2): strategy, correctives, sizing, risk, order. Reason
-- codes and the numbers used, never free text: the diary and the "why" card are written from these rows with fixed
-- templates. decision_id groups the rows of one decision.
CREATE TABLE decisions (
id {uuid} PRIMARY KEY,
decision_id {uuid} NOT NULL,
strategy TEXT NOT NULL,
account_id {uuid},
symbol TEXT,
stage TEXT NOT NULL,
outcome TEXT NOT NULL,
reasons {json} NOT NULL,
data {json},
created_at {timestamp} NOT NULL
){strict};
CREATE INDEX decisions_decision ON decisions (decision_id);
CREATE INDEX decisions_strategy ON decisions (strategy, created_at);
-- What the risk engine remembers for each world (SPEC 9.2, 9.3): blocks with their end, the moment the high is
-- measured from, the switch tripped by a limit and the warnings already sent. One row per world.
CREATE TABLE risk_state (
id {uuid} PRIMARY KEY,
world TEXT NOT NULL UNIQUE,
state {json} NOT NULL,
created_at {timestamp} NOT NULL,
updated_at {timestamp} NOT NULL
){strict};
-- Risk events belong to a world (SPEC 9.2: a limit stops only the world where it was crossed).
ALTER TABLE risk_events ADD COLUMN world TEXT;
CREATE INDEX risk_events_time ON risk_events (occurred_at);
-- Stages (SPEC 12.3, 12.4, 7.1). account_id stays the account of the simulation (internal simulator or demo);
-- live_account_id is the real account, once the strategy has one. live_confirmed_at is set only by the procedure
-- of 12.4 and cleared by a revocation or a demotion; exit_only keeps a demoted strategy managing its real
-- positions down to zero. stage_since and stage_reason say since when and why the strategy is where it is;
-- qualification is the backtest outcome it qualified with, which the requirements and demotions compare against.
ALTER TABLE strategy_state ADD COLUMN live_account_id {uuid};
ALTER TABLE strategy_state ADD COLUMN live_confirmed_at {timestamp};
ALTER TABLE strategy_state ADD COLUMN live_confirmed_by TEXT;
ALTER TABLE strategy_state ADD COLUMN exit_only {bool};
ALTER TABLE strategy_state ADD COLUMN stage_since {timestamp};
ALTER TABLE strategy_state ADD COLUMN stage_reason TEXT;
ALTER TABLE strategy_state ADD COLUMN qualification {json};
+65
View File
@@ -0,0 +1,65 @@
-- Phase 6 schema (SPEC 5.5, 5.6, 9.3, 9.4, 14, ADR 0012): the live engine, recovery after an absence, Telegram.
-- One row for each decision the live engine makes: run_key names the strategy and the close it decides on, so a
-- decision is made once even across restarts. The row is written in the same transaction as its planned orders,
-- its decision rows and its diary lines: either all of it exists or none. state is the book the strategy decided
-- from, for the one-step comparison of SPEC 12.3 the next day.
CREATE TABLE engine_runs (
id {uuid} PRIMARY KEY,
strategy TEXT NOT NULL,
run_key TEXT NOT NULL UNIQUE,
decision_day TEXT NOT NULL,
decision_id {uuid} NOT NULL,
mode TEXT NOT NULL,
state {json},
proposed {json},
created_at {timestamp} NOT NULL
){strict};
CREATE INDEX engine_runs_strategy ON engine_runs (strategy, decision_day);
-- Orders decided and not yet handed to the order manager (SPEC 5.5: the stock orders decided after the close go at
-- the next open; SPEC 5.6: an entry has a deadline and a reference price it must not drift from). The planned
-- order's id is the decision id of the trade intent: the order key derives from it, so a restart that finds the
-- execution already there never sends it twice.
CREATE TABLE planned_orders (
id {uuid} PRIMARY KEY,
decision_id {uuid} NOT NULL,
strategy TEXT NOT NULL,
account_id {uuid} NOT NULL REFERENCES accounts (id),
instrument_id {uuid} NOT NULL REFERENCES instruments (id),
side TEXT NOT NULL,
purpose TEXT NOT NULL,
quantity {money} NOT NULL,
exit_all {bool} NOT NULL,
stop_level {money},
stop_offset {money},
reference_price {money} NOT NULL,
reason TEXT NOT NULL,
due_at {timestamp} NOT NULL,
valid_until {timestamp},
status TEXT NOT NULL,
outcome TEXT,
execution_id {uuid},
created_at {timestamp} NOT NULL,
updated_at {timestamp} NOT NULL
){strict};
CREATE INDEX planned_orders_due ON planned_orders (status, due_at);
-- Absences (SPEC 5.6): measured at the restart from the last heartbeat. One longer than 24 hours is declared on the
-- Panoramica until the user acknowledges it.
CREATE TABLE absences (
id {uuid} PRIMARY KEY,
started_at {timestamp} NOT NULL,
ended_at {timestamp} NOT NULL,
acknowledged_at {timestamp},
created_at {timestamp} NOT NULL
){strict};
-- Every notification has its state (SPEC 14): pending, sent or failed, with the attempts and the last error.
ALTER TABLE notifications ADD COLUMN attempts INTEGER;
ALTER TABLE notifications ADD COLUMN next_attempt_at {timestamp};
ALTER TABLE notifications ADD COLUMN last_error TEXT;
CREATE INDEX notifications_status ON notifications (status, created_at);
@@ -0,0 +1,50 @@
-- Data collected for the analysis of a long run (issue #95, ADR 0013). Both tables are kept for 30 days.
-- One sample of the process every 5 minutes, and one at every start. The counters (collections, pauses, processor
-- time, lock contentions, exceptions) are totals since the process started: the report takes the differences
-- between two samples of the same process. warnings and errors are the log lines since the previous sample.
-- calls holds the outbound HTTP calls since the previous sample, by destination, with their duration buckets.
CREATE TABLE diagnostic_samples (
id {uuid} PRIMARY KEY,
taken_at {timestamp} NOT NULL,
process_started_at {timestamp} NOT NULL,
kind TEXT NOT NULL,
version TEXT NOT NULL,
working_set {long} NOT NULL,
private_bytes {long} NOT NULL,
managed_heap {long} NOT NULL,
gc_committed {long} NOT NULL,
gc_gen0 {long} NOT NULL,
gc_gen1 {long} NOT NULL,
gc_gen2 {long} NOT NULL,
gc_pause_ms {float} NOT NULL,
threads INTEGER NOT NULL,
handles INTEGER NOT NULL,
cpu_ms {float} NOT NULL,
pool_threads INTEGER NOT NULL,
pool_pending {long} NOT NULL,
lock_contentions {long} NOT NULL,
exceptions {long} NOT NULL,
db_bytes {long},
warnings INTEGER NOT NULL,
errors INTEGER NOT NULL,
calls {json},
created_at {timestamp} NOT NULL
){strict};
CREATE INDEX diagnostic_samples_taken ON diagnostic_samples (taken_at);
-- Every run of a scheduled job, with its lateness and duration. jobs keeps only the last one.
CREATE TABLE job_runs (
id {uuid} PRIMARY KEY,
job TEXT NOT NULL,
due_at {timestamp} NOT NULL,
started_at {timestamp} NOT NULL,
finished_at {timestamp} NOT NULL,
outcome TEXT NOT NULL,
missed {bool} NOT NULL,
error TEXT,
created_at {timestamp} NOT NULL
){strict};
CREATE INDEX job_runs_started ON job_runs (started_at);
+1 -1
View File
@@ -32,7 +32,7 @@ public static partial class Migrations
{
public const string VersionTable = "schema_versions";
[GeneratedRegex(@"\{(money|uuid|timestamp|json|bool|bytes|float|strict)\}")]
[GeneratedRegex(@"\{(money|uuid|timestamp|json|bool|bytes|float|long|strict)\}")]
internal static partial Regex Tokens();
[GeneratedRegex(@"^(\d{4})_([a-z0-9_]+)\.sql$")]
+3 -2
View File
@@ -83,9 +83,10 @@ public static partial class Schema
"{BOOL}" => ColumnKind.Bool,
"{BYTES}" => ColumnKind.Bytes,
"{FLOAT}" => ColumnKind.Measure,
"{LONG}" => ColumnKind.Number,
"TEXT" => ColumnKind.Text,
"INTEGER" or "BIGINT" => ColumnKind.Number,
var other => throw new InvalidOperationException($"Column type {other} is not portable: use TEXT, INTEGER, BIGINT or a token."),
"INTEGER" => ColumnKind.Number,
var other => throw new InvalidOperationException($"Column type {other} is not portable: use TEXT, INTEGER or a token."),
};
var upper = definition.ToUpperInvariant();
var nullable = !upper.Contains("NOT NULL", StringComparison.Ordinal) && !upper.Contains("PRIMARY KEY", StringComparison.Ordinal);
+38 -3
View File
@@ -50,10 +50,45 @@
{ "nome": "equity_meanrev", "etichetta": "Mean reversion azioni USA", "conto": "Alpaca", "giorno": 58, "di": 60, "quota_pct": 15, "capitale": "1500", "risultato": "11.36", "risultato_pct": "0.76", "calo_peggiore_pct": "-1.1", "chiuse": 21, "in_guadagno": 10 }
],
"operazioni": { "chiuse": 41, "in_guadagno": 22, "guadagno_medio": "28.42", "perdita_media": "-20.31", "rapporto": "1.62", "incassati": "239.28", "commissioni": "14.82" },
"notizie": { "lette_24h": 312, "sulle_posizioni": 9, "con_effetto": 2, "spesa_oggi": "0.04", "tetto": "0.50" },
"apprendimento": { "stato": "observation", "segnali": 212, "necessari": 300, "precisione_pct": 56 },
"notizie": {
"lette_24h": 312, "sulle_posizioni": 9, "con_effetto": 2, "spesa_oggi": "0.04", "tetto": "0.50",
"panoramica": [
{ "quando": "06:10", "riguarda": "su SOL", "titolo": "Rinviato di una settimana l'aggiornamento della rete Solana", "effetto": "Negativa: per 24 ore non aumento SOL", "in_corso": true },
{ "quando": "Ieri 22:41", "riguarda": "su JPM", "titolo": "JPMorgan pubblica i conti martedì prima dell'apertura", "effetto": "Evento in arrivo: niente mean reversion su JPM fino a martedì", "in_corso": true },
{ "quando": "07:52", "riguarda": "sul mercato in generale", "titolo": "I verbali della banca centrale USA non cambiano le attese sui tassi", "effetto": "Nessun effetto: non cambio nulla", "in_corso": false }
]
},
"panoramica": {
"prossima_decisione": "Prossima decisione tra 7 ore e 14 minuti, all'apertura di Wall Street.",
"mosse": [
{ "ora": "15:35", "testo": "Rimetto gli stop del giorno sulle frazioni di 6 azioni", "dove": "Alpaca", "calendario": false },
{ "ora": "16:00", "testo": "Esce la fiducia dei consumatori USA: impatto medio, non blocco gli ingressi", "calendario": true },
{ "ora": "21:55", "testo": "Controllo AAPL: vendo se è sopra la media a 5 giorni", "calendario": false },
{ "ora": "22:20", "testo": "Wall Street ha chiuso: calcolo i segnali sulle azioni", "calendario": false },
{ "ora": "02:05", "testo": "Chiusura giornaliera crypto: ricalcolo i trend di 10 monete", "calendario": false }
],
"ultime_azioni": [
{ "quando": "02:06", "testo": "Alzato lo stop di BTC a 66.180 € (era 65.400 €)" },
{ "quando": "Ieri 21:58", "testo": "Venduto MSFT: +9,84 $ in 3 giorni" },
{ "quando": "Ieri 15:36", "testo": "Comprate 2 AAPL a 259,80 $" }
]
},
"apprendimento": {
"stato": "observation", "segnali": 212, "necessari": 300, "precisione_pct": 56,
"ultimo_addestramento": "domenica 4 ottobre", "prossima_verifica": "domenica 11 ottobre, 03:00"
},
"sistema": {
"controllo_secondi": 5, "notizie_fonti": 16, "database_gb": "1.8", "ultimo_salvataggio": "03:30", "telegram_ultimo": "08:00",
"cpu_pct": 6, "memoria_gb": "1.1", "memoria_totale_gb": 4, "disco_gb": "2.3", "disco_totale_gb": 50
"cpu_pct": 6, "memoria_gb": "1.1", "memoria_totale_gb": 4, "disco_gb": "2.3", "disco_totale_gb": 50,
"notizie_ultima": "08:20",
"lavori": [
{ "ora": "15:35", "testo": "Stop del giorno sulle azioni" },
{ "ora": "21:55", "testo": "Uscite della mean reversion" },
{ "ora": "22:20", "testo": "Segnali sulle azioni" },
{ "ora": "02:05", "testo": "Segnali sulle crypto" },
{ "ora": "03:00", "testo": "Controllo del modello" },
{ "ora": "03:30", "testo": "Salvataggio del database" },
{ "ora": "Dom 03:00", "testo": "Verifica settimanale del modello" }
]
}
}
+70 -9
View File
@@ -120,10 +120,25 @@ public static class DemoMode
["Mean reversion"] = ("equity_meanrev", alpaca, 1740.15m, 1236.71m, 0.25m),
};
var currencyOf = new Dictionary<Guid, string> { [okx] = "EUR", [alpaca] = "USD" };
// F.4: 58 days of 60 in simulation, after a passed backtest (an example: the real outcomes are in docs/backtests).
foreach (var (name, account, capital, _, share) in strategies.Values)
{
await s.ExecuteAsync("INSERT INTO strategy_state (id, strategy, account_id, stage, capital, share, state, created_at, updated_at) VALUES (@id, @n, @a, 'simulation', @c, @sh, @st, @t, @t)",
cancellationToken, ("id", ids.Next()), ("n", name), ("a", account), ("c", capital), ("sh", share), ("st", new DbJson("{\"day\":58,\"of\":60}")), ("t", now))
var example = new Trading.Qualification
{
Passed = true,
Verdict = "Superato (dato di esempio).",
Report = "docs/backtests/" + name + ".md",
MaxDrawdown = 0.18m,
ExpectedSlippageBp = 8m,
RoundTripCost = 0.0056m,
ExpectedPathP5 = [.. Enumerable.Range(1, 365).Select(d => Math.Round(-0.012 * Math.Sqrt(d), 4))],
};
await s.ExecuteAsync(
"INSERT INTO strategy_state (id, strategy, account_id, stage, stage_since, stage_reason, qualification, capital, share, state, created_at, updated_at) " +
"VALUES (@id, @n, @a, 'simulation', @since, 'user', @q, @c, @sh, @st, @t, @t)",
cancellationToken, ("id", ids.Next()), ("n", name), ("a", account), ("since", now.AddDays(-58)),
("q", new DbJson(JsonSerializer.Serialize(example, Trading.Qualification.Json))), ("c", capital), ("sh", share),
("st", new DbJson("{\"day\":58,\"of\":60}")), ("t", now))
.ConfigureAwait(false);
}
@@ -177,20 +192,22 @@ public static class DemoMode
[
("account", okx, null, 4000m, 231.60m),
("account", alpaca, null, 6000m, 180.86m),
("strategy", null, "crypto_trend", 4000m, 231.60m),
("strategy", null, "equity_momentum", 4500m, 169.50m),
("strategy", null, "equity_meanrev", 1500m, 11.36m),
("strategy", okx, "crypto_trend", 4000m, 231.60m),
("strategy", alpaca, "equity_momentum", 4500m, 169.50m),
("strategy", alpaca, "equity_meanrev", 1500m, 11.36m),
];
foreach (var (day, equity, benchmark, drawdown) in daily)
{
var at = day == DateOnly.FromDateTime(Now.DateTime) ? now : new DateTimeOffset(day.ToDateTime(new TimeOnly(22, 0)), rome.GetUtcOffset(day.ToDateTime(new TimeOnly(22, 0))));
await s.ExecuteAsync("INSERT INTO equity_snapshots (id, taken_at, scope, world, currency, equity, benchmark, drawdown_pct, created_at) VALUES (@id, @at, 'total', 'simulated', 'EUR', @e, @b, @d, @t)",
cancellationToken, ("id", ids.Next()), ("at", at), ("e", equity), ("b", benchmark), ("d", drawdown), ("t", now)).ConfigureAwait(false);
await s.ExecuteAsync("INSERT INTO equity_snapshots (id, taken_at, scope, world, currency, equity, benchmark, drawdown_pct, twr_index, created_at) VALUES (@id, @at, 'total', 'simulated', 'EUR', @e, @b, @d, @x, @t)",
cancellationToken, ("id", ids.Next()), ("at", at), ("e", equity), ("b", benchmark), ("d", drawdown), ("x", Math.Round(equity / 10000m, 6)), ("t", now)).ConfigureAwait(false);
foreach (var (scope, account, strategy, start, result) in series)
{
var value = Math.Round(start + (equity - 10000m) * result / total, 2, MidpointRounding.AwayFromZero);
await s.ExecuteAsync("INSERT INTO equity_snapshots (id, taken_at, scope, world, currency, account_id, strategy, equity, created_at) VALUES (@id, @at, @sc, 'simulated', 'EUR', @a, @st, @e, @t)",
cancellationToken, ("id", ids.Next()), ("at", at), ("sc", scope), ("a", account), ("st", strategy), ("e", value), ("t", now)).ConfigureAwait(false);
// Without deposits along the way the index is the value over the start.
await s.ExecuteAsync("INSERT INTO equity_snapshots (id, taken_at, scope, world, currency, account_id, strategy, equity, twr_index, created_at) VALUES (@id, @at, @sc, 'simulated', 'EUR', @a, @st, @e, @x, @t)",
cancellationToken, ("id", ids.Next()), ("at", at), ("sc", scope), ("a", account), ("st", strategy), ("e", value), ("x", Math.Round(value / start, 6)), ("t", now))
.ConfigureAwait(false);
}
}
@@ -269,6 +286,50 @@ public static class DemoMode
"VALUES (@id, @a, @i, @s, 'closed', @o, @c, @q, @b, @q, @sv, @f, @r, @e, @t, @t)",
cancellationToken, ("id", ids.Next()), ("a", account), ("i", instrument), ("s", strategy), ("o", closed.AddDays(-5)), ("c", closed), ("q", quantity), ("b", bought),
("sv", bought + realized + fee), ("f", fee), ("r", realized), ("e", realized >= 0m ? "signal" : "stop"), ("t", now)).ConfigureAwait(false);
await SeedWhyAsync(s, ids, account, instrument, strategy, symbol, quantity, bought, bought + realized + fee, closed.AddDays(-5), closed, realized < 0m, now, cancellationToken)
.ConfigureAwait(false);
}
}
// F.6: the decisions behind each trade, so its "why" card has lines: the strategy's reason, correctives and risk on
// the purchase; the strategy's exit on a sale it decided, or the protective stop that sold at a loss.
private static async Task SeedWhyAsync(DbSession s, IdSource ids, Guid account, Guid instrument, string strategy, string symbol, decimal quantity, decimal bought,
decimal sold, DateTimeOffset opened, DateTimeOffset closed, bool stopped, DateTimeOffset now, CancellationToken cancellationToken)
{
var (entry, exit) = strategy switch
{
"crypto_trend" => ("trend_entry", "trend_exit"),
"equity_momentum" => ("momentum_entry", "momentum_exit_rank"),
_ => ("meanrev_entry", "meanrev_exit_average"),
};
async Task DecideAsync(Guid decision, string stage, string outcome, string[] reasons, DateTimeOffset at) =>
await s.ExecuteAsync(
"INSERT INTO decisions (id, decision_id, strategy, account_id, symbol, stage, outcome, reasons, data, created_at) VALUES (@id, @d, @s, @a, @sym, @st, @o, @r, NULL, @t)",
cancellationToken, ("id", ids.Next()), ("d", decision), ("s", strategy), ("a", account), ("sym", symbol), ("st", stage), ("o", outcome),
("r", new DbJson(JsonSerializer.Serialize(reasons))), ("t", at)).ConfigureAwait(false);
async Task ExecuteAsync(string side, string purpose, Guid decision, decimal reference, DateTimeOffset at) =>
await s.ExecuteAsync(
"INSERT INTO executions (id, account_id, instrument_id, strategy, side, purpose, quantity, filled, order_key, sequence, step, reference_price, status, started_at, created_at, updated_at) " +
"VALUES (@id, @a, @i, @s, @side, @p, @q, @q, @k, 1, 0, @r, @st, @at, @t, @t)",
cancellationToken, ("id", ids.Next()), ("a", account), ("i", instrument), ("s", strategy), ("side", side), ("p", purpose), ("q", quantity),
("k", Trading.ClientOrderIds.Key(decision)), ("r", Math.Round(reference, 6)), ("st", Trading.ExecutionStates.Done), ("at", at), ("t", now)).ConfigureAwait(false);
var buy = ids.Next();
await DecideAsync(buy, Trading.DecisionStages.Strategy, Trading.DecisionOutcomes.Proposed, [entry], opened.AddMinutes(-1)).ConfigureAwait(false);
await DecideAsync(buy, Trading.DecisionStages.Correctives, Trading.DecisionOutcomes.Passed, [], opened.AddMinutes(-1)).ConfigureAwait(false);
await DecideAsync(buy, Trading.DecisionStages.Risk, Trading.DecisionOutcomes.Passed, [], opened.AddMinutes(-1)).ConfigureAwait(false);
await ExecuteAsync("buy", "entry", buy, bought / quantity / 1.0002m, opened).ConfigureAwait(false);
if (stopped)
{
await ExecuteAsync("sell", "protective", ids.Next(), sold / quantity * 1.0003m, closed).ConfigureAwait(false);
}
else
{
var sale = ids.Next();
await DecideAsync(sale, Trading.DecisionStages.Strategy, Trading.DecisionOutcomes.Proposed, [exit], closed.AddMinutes(-1)).ConfigureAwait(false);
await ExecuteAsync("sell", "exit", sale, sold / quantity * 1.0001m, closed).ConfigureAwait(false);
}
}
+49
View File
@@ -0,0 +1,49 @@
using Microsoft.Extensions.Logging;
namespace Anubis.Diagnostics;
/// <summary>
/// The warning and error lines written by the logs since the last look (issue #95). The log writer counts each line;
/// the sampler takes the counts every 5 minutes, so the report shows when the troubles came, not only how many.
/// </summary>
public static class LogCounter
{
private static long _warnings;
private static long _errors;
public static void Record(LogLevel level)
{
if (level == LogLevel.Warning)
{
Interlocked.Increment(ref _warnings);
}
else if (level >= LogLevel.Error && level != LogLevel.None)
{
Interlocked.Increment(ref _errors);
}
}
/// <summary>The lines since the previous call, and the counts start again from zero.</summary>
public static (int Warnings, int Errors) Take() =>
((int)Math.Min(int.MaxValue, Interlocked.Exchange(ref _warnings, 0)), (int)Math.Min(int.MaxValue, Interlocked.Exchange(ref _errors, 0)));
}
/// <summary>
/// Every exception thrown in the process, caught or not, since the counting started. Many are normal (a cancelled
/// wait, a lost connection retried), but a count that keeps climbing points to a loop that fails over and over.
/// </summary>
public static class ExceptionCounter
{
private static long _count;
private static int _started;
public static long Count => Interlocked.Read(ref _count);
public static void Start()
{
if (Interlocked.Exchange(ref _started, 1) == 0)
{
AppDomain.CurrentDomain.FirstChanceException += (_, _) => Interlocked.Increment(ref _count);
}
}
}
+181
View File
@@ -0,0 +1,181 @@
using System.Diagnostics;
using System.Text.Json;
using Anubis.Core;
using Anubis.Db;
using Anubis.Platform;
namespace Anubis.Diagnostics;
public static class SampleKinds
{
/// <summary>The first sample of a process, written as soon as the bot is ready.</summary>
public const string Start = "start";
public const string Periodic = "periodic";
}
/// <summary>One look at the process (issue #95). Counters are totals since the process started.</summary>
public sealed record ProcessSample
{
public required DateTimeOffset TakenAt { get; init; }
public required DateTimeOffset ProcessStartedAt { get; init; }
public required string Kind { get; init; }
public string Version { get; init; } = AnubisInfo.Version;
public long WorkingSet { get; init; }
public long PrivateBytes { get; init; }
public long ManagedHeap { get; init; }
public long GcCommitted { get; init; }
public long Gen0 { get; init; }
public long Gen1 { get; init; }
public long Gen2 { get; init; }
public double GcPauseMs { get; init; }
public int Threads { get; init; }
public int Handles { get; init; }
public double CpuMs { get; init; }
public int PoolThreads { get; init; }
public long PoolPending { get; init; }
public long LockContentions { get; init; }
public long Exceptions { get; init; }
public long? DbBytes { get; init; }
public int Warnings { get; init; }
public int Errors { get; init; }
public IReadOnlyList<HttpCallStats> Calls { get; init; } = [];
}
/// <summary>
/// Every 5 minutes, and once at each start, a sample of the process goes to <c>diagnostic_samples</c>, with the log
/// warnings and errors and the outbound calls since the previous one (issue #95, ADR 0013). Samples and job runs older
/// than 30 days are deleted once a day. It only measures: nothing here changes what the bot does.
/// </summary>
public sealed class DiagnosticSampler(Database db, IClock clock, HttpCallMeter meter)
{
public const string JobName = "diagnostica";
public static readonly TimeSpan Interval = TimeSpan.FromMinutes(5);
public static readonly TimeSpan Keep = TimeSpan.FromDays(30);
internal static readonly JsonSerializerOptions Json = new(JsonSerializerDefaults.Web);
private DateTimeOffset? _lastPrune;
public ScheduledJob Job() => new(JobName, "Raccolta dei dati per l'analisi", Schedule.Every(Interval), ct => TakeAsync(SampleKinds.Periodic, ct))
{
MaxLateness = Interval,
};
public async Task<ProcessSample> TakeAsync(string kind, CancellationToken cancellationToken)
{
var (warnings, errors) = LogCounter.Take();
var sample = Probe(clock.UtcNow, kind) with
{
DbBytes = await DatabaseBytesAsync(db, cancellationToken).ConfigureAwait(false),
Warnings = warnings,
Errors = errors,
Calls = meter.Take(),
};
var prune = _lastPrune is not { } last || sample.TakenAt - last >= TimeSpan.FromDays(1);
await db.WriteAsync(async s =>
{
await WriteAsync(s, clock, sample, cancellationToken).ConfigureAwait(false);
if (prune)
{
var before = sample.TakenAt - Keep;
await s.ExecuteAsync("DELETE FROM diagnostic_samples WHERE taken_at < @t", cancellationToken, ("t", before)).ConfigureAwait(false);
await s.ExecuteAsync("DELETE FROM job_runs WHERE started_at < @t", cancellationToken, ("t", before)).ConfigureAwait(false);
}
}, cancellationToken).ConfigureAwait(false);
if (prune)
{
_lastPrune = sample.TakenAt;
}
return sample;
}
/// <summary>The process as it is now. Process times come from the system clock: they measure the machine, not the market.</summary>
public static ProcessSample Probe(DateTimeOffset now, string kind)
{
using var process = Process.GetCurrentProcess();
var gc = GC.GetGCMemoryInfo();
return new ProcessSample
{
TakenAt = now,
ProcessStartedAt = new DateTimeOffset(process.StartTime.ToUniversalTime(), TimeSpan.Zero),
Kind = kind,
WorkingSet = process.WorkingSet64,
PrivateBytes = process.PrivateMemorySize64,
ManagedHeap = GC.GetTotalMemory(forceFullCollection: false),
GcCommitted = gc.TotalCommittedBytes,
Gen0 = GC.CollectionCount(0),
Gen1 = GC.CollectionCount(1),
Gen2 = GC.CollectionCount(2),
GcPauseMs = Math.Round(GC.GetTotalPauseDuration().TotalMilliseconds, 1),
Threads = process.Threads.Count,
Handles = process.HandleCount,
CpuMs = Math.Round(process.TotalProcessorTime.TotalMilliseconds, 1),
PoolThreads = ThreadPool.ThreadCount,
PoolPending = ThreadPool.PendingWorkItemCount,
LockContentions = Monitor.LockContentionCount,
Exceptions = ExceptionCounter.Count,
};
}
public static async Task WriteAsync(DbSession s, IClock clock, ProcessSample sample, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(s);
ArgumentNullException.ThrowIfNull(sample);
await s.ExecuteAsync(
"INSERT INTO diagnostic_samples (id, taken_at, process_started_at, kind, version, working_set, private_bytes, managed_heap, gc_committed, gc_gen0, gc_gen1, gc_gen2, " +
"gc_pause_ms, threads, handles, cpu_ms, pool_threads, pool_pending, lock_contentions, exceptions, db_bytes, warnings, errors, calls, created_at) VALUES " +
"(@id, @t, @ps, @k, @v, @ws, @pb, @mh, @gcc, @g0, @g1, @g2, @gp, @th, @h, @cpu, @pt, @pp, @lc, @ex, @db, @w, @e, @calls, @now)",
cancellationToken,
("id", Ids.NewId(clock)), ("t", sample.TakenAt), ("ps", sample.ProcessStartedAt), ("k", sample.Kind), ("v", sample.Version), ("ws", sample.WorkingSet),
("pb", sample.PrivateBytes), ("mh", sample.ManagedHeap), ("gcc", sample.GcCommitted), ("g0", sample.Gen0), ("g1", sample.Gen1), ("g2", sample.Gen2),
("gp", sample.GcPauseMs), ("th", sample.Threads), ("h", sample.Handles), ("cpu", sample.CpuMs), ("pt", sample.PoolThreads), ("pp", sample.PoolPending),
("lc", sample.LockContentions), ("ex", sample.Exceptions), ("db", sample.DbBytes), ("w", sample.Warnings), ("e", sample.Errors),
("calls", sample.Calls.Count == 0 ? null : new DbJson(JsonSerializer.Serialize(sample.Calls, Json))), ("now", clock.UtcNow)).ConfigureAwait(false);
}
/// <summary>The size of the database: its files on SQLite (with the journal), what PostgreSQL reports for it.</summary>
public static async Task<long?> DatabaseBytesAsync(Database db, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(db);
try
{
if (db.Engine == DbEngine.Sqlite)
{
return db.SqlitePath is not { } path ? null
: new[] { path, path + "-wal", path + "-shm" }.Where(File.Exists).Sum(f => new FileInfo(f).Length);
}
return await db.ReadAsync(s => s.CountAsync("SELECT pg_database_size(current_database())"), cancellationToken).ConfigureAwait(false);
}
catch (Exception ex) when (ex is IOException or UnauthorizedAccessException or System.Data.Common.DbException)
{
return null;
}
}
}
+595
View File
@@ -0,0 +1,595 @@
using System.Globalization;
using System.Runtime.InteropServices;
using System.Text;
using System.Text.Json;
using System.Text.RegularExpressions;
using Anubis.Core;
using Anubis.Db;
using Anubis.Platform;
using Anubis.Supervision;
namespace Anubis.Diagnostics;
/// <summary>What the report covers and the facts about the installation that only the caller knows.</summary>
public sealed record DiagnosticsRequest(DateTimeOffset From, DateTimeOffset To, TimeZoneInfo Zone)
{
/// <summary>The folder of the technical logs (<c>anubis-yyyyMMdd.log</c>); without it the report skips them.</summary>
public string? LogsFolder { get; init; }
/// <summary>Startup settings with nothing secret in them, shown as they are.</summary>
public IReadOnlyList<KeyValuePair<string, string>> Setup { get; init; } = [];
public DateTimeOffset GeneratedAt { get; init; }
}
/// <summary>
/// The report the user pastes back after a long run of a release (issue #95, ADR 0013): in Markdown, at most 60 KB,
/// with everything passed through <see cref="Redactor"/> and the user's folders masked. It only reads.
/// </summary>
public static partial class DiagnosticsReporter
{
public const int MaxBytes = 60 * 1024;
private const int MaxLogGroups = 40;
private const long MaxLogBytes = 256L * 1024 * 1024;
/// <summary>Two missed samples in a row: the process lived but did not sample (sleep, a stuck scheduler).</summary>
private static readonly TimeSpan Silence = DiagnosticSampler.Interval * 2 + TimeSpan.FromMinutes(1);
private static readonly TimeSpan[] Steps =
[TimeSpan.FromMinutes(15), TimeSpan.FromMinutes(30), TimeSpan.FromHours(1), TimeSpan.FromHours(2), TimeSpan.FromHours(3),
TimeSpan.FromHours(6), TimeSpan.FromHours(12), TimeSpan.FromDays(1)];
public static async Task<string> BuildAsync(Database db, DiagnosticsRequest request, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(db);
ArgumentNullException.ThrowIfNull(request);
if (request.To <= request.From)
{
throw new ArgumentException("La fine del periodo deve venire dopo l'inizio.", nameof(request));
}
var zone = request.Zone;
var findings = new List<string>();
var soak = await SoakReporter.BuildAsync(db, request.From, request.To, request.To - request.From, zone, cancellationToken).ConfigureAwait(false);
var data = await db.ReadAsync(async s =>
{
(string Name, object? Value)[] window = [("from", request.From), ("to", request.To)];
var engine = await s.FirstOrDefaultAsync(s.Engine == DbEngine.Sqlite ? "SELECT 'SQLite ' || sqlite_version() AS v" : "SELECT version() AS v", r => r.Text("v")).ConfigureAwait(false);
var samples = await s.QueryAsync("SELECT * FROM diagnostic_samples WHERE taken_at >= @from AND taken_at <= @to ORDER BY taken_at", ReadSample, window).ConfigureAwait(false);
var runs = await s.QueryAsync("SELECT job, due_at, started_at, finished_at, outcome, missed, error FROM job_runs WHERE started_at >= @from AND started_at <= @to",
r => new Run(r.Text("job"), r.Moment("due_at"), r.Moment("started_at"), r.Moment("finished_at"), r.Text("outcome"), r.Bool("missed"), r.TextOrNull("error")), window)
.ConfigureAwait(false);
var absences = await s.QueryAsync("SELECT started_at, ended_at FROM absences WHERE ended_at >= @from AND started_at <= @to ORDER BY started_at",
r => (Start: r.Moment("started_at"), End: r.Moment("ended_at")), window).ConfigureAwait(false);
return (Engine: engine, Samples: samples, Runs: runs, Absences: absences);
}, cancellationToken).ConfigureAwait(false);
var logs = request.LogsFolder is { } folder ? ReadLogs(folder, request.From, request.To) : null;
var sections = new List<string>
{
Setup(request, data.Engine, data.Samples),
Soak(soak, findings),
Starts(data.Samples, data.Absences, zone, findings),
Jobs(data.Runs, findings),
Calls(data.Samples, findings),
Logs(logs, zone, findings),
Resources(data.Samples, request, findings),
};
var text = new StringBuilder()
.AppendLine("# Anubis · dati per l'analisi")
.AppendLine()
.AppendLine(CultureInfo.InvariantCulture, $"Versione {AnubisInfo.Version} (commit {AnubisInfo.Commit ?? "non noto"}). Dal {SoakReport.Moment(request.From, zone)} al " +
$"{SoakReport.Moment(request.To, zone)} ({SoakReport.Hours(request.To - request.From)}), ora di {zone.Id}. Fatto il {SoakReport.Moment(request.GeneratedAt, zone)}.")
.AppendLine()
.AppendLine("## In breve")
.AppendLine();
foreach (var finding in findings.Count == 0 ? ["Nessun segnale di problema nei dati raccolti."] : findings)
{
text.AppendLine("- " + finding);
}
foreach (var section in sections)
{
text.AppendLine().Append(section);
}
return Fit(Mask(text.ToString().ReplaceLineEndings("\n")));
}
/// <summary>The user's folders and name out, the secrets out, and no more than <see cref="MaxBytes"/>.</summary>
internal static string Mask(string text)
{
var profile = Environment.GetFolderPath(Environment.SpecialFolder.UserProfile);
if (profile.Length > 3)
{
text = text.Replace(profile, "%USERPROFILE%", StringComparison.OrdinalIgnoreCase);
}
var user = Environment.UserName;
if (user.Length >= 3)
{
text = text.Replace("\\" + user + "\\", "\\‹utente›\\", StringComparison.OrdinalIgnoreCase)
.Replace("/" + user + "/", "/‹utente›/", StringComparison.OrdinalIgnoreCase);
}
return Redactor.Redact(text);
}
internal static string Fit(string text)
{
if (Encoding.UTF8.GetByteCount(text) <= MaxBytes)
{
return text;
}
const string note = "\n\n_Rapporto tagliato a 60 KB: le righe più vecchie delle risorse mancano._\n";
var limit = MaxBytes - Encoding.UTF8.GetByteCount(note);
var cut = text.Length;
while (cut > 0 && Encoding.UTF8.GetByteCount(text.AsSpan(0, cut)) > limit)
{
cut = text.LastIndexOf('\n', cut - 1);
}
return text[..Math.Max(0, cut)] + note;
}
private static string Setup(DiagnosticsRequest request, string? engine, List<ProcessSample> samples)
{
var text = new StringBuilder("## Installazione\n\n");
text.AppendLine(CultureInfo.InvariantCulture, $"- Sistema: {RuntimeInformation.OSDescription} ({RuntimeInformation.OSArchitecture}), {Environment.ProcessorCount} processori")
.AppendLine(CultureInfo.InvariantCulture, $"- Runtime: {RuntimeInformation.FrameworkDescription}")
.AppendLine(CultureInfo.InvariantCulture, $"- Motore del database: {engine ?? "non noto"}");
if (samples.LastOrDefault(s => s.DbBytes is not null) is { DbBytes: { } bytes })
{
text.AppendLine(CultureInfo.InvariantCulture, $"- Dimensione del database: {Mb(bytes)} MB");
}
foreach (var (name, value) in request.Setup)
{
text.AppendLine(CultureInfo.InvariantCulture, $"- {name}: {value}");
}
return text.ToString();
}
private static string Soak(SoakReport soak, List<string> findings)
{
var failed = soak.Checks.Where(c => !c.Ok).ToList();
if (failed.Count > 0)
{
findings.Add($"Controlli del lavoro da guardare: {failed.Count} (sezione «Controlli del lavoro»).");
}
var text = new StringBuilder("## Controlli del lavoro\n\n");
foreach (var check in soak.Checks)
{
text.AppendLine(CultureInfo.InvariantCulture, $"- {(check.Ok ? "✓" : "✗")} {check.Text}");
}
foreach (var fact in soak.Facts)
{
text.AppendLine("- " + fact);
}
return text.ToString();
}
private static string Starts(List<ProcessSample> samples, List<(DateTimeOffset Start, DateTimeOffset End)> absences, TimeZoneInfo zone, List<string> findings)
{
var text = new StringBuilder("## Avvii, spegnimenti e silenzi\n\n");
var processes = Processes(samples);
if (processes.Count == 0)
{
findings.Add("Nessun campione nel periodo: la raccolta dei dati non ha girato (versione senza raccolta, o bot spento).");
return text.AppendLine("Nessun campione nel periodo.").ToString();
}
text.AppendLine("| Avviato | Versione | Primo campione | Ultimo campione | Campioni |").AppendLine("|---|---|---|---|---|");
foreach (var p in processes)
{
text.AppendLine(CultureInfo.InvariantCulture,
$"| {SoakReport.Moment(p[0].ProcessStartedAt, zone)} | {p[0].Version} | {SoakReport.Moment(p[0].TakenAt, zone)} | {SoakReport.Moment(p[^1].TakenAt, zone)} | {p.Count} |");
}
if (processes.Count > 1)
{
findings.Add($"Il bot è ripartito {processes.Count - 1} volte nel periodo.");
}
text.AppendLine();
if (absences.Count > 0)
{
var total = absences.Aggregate(TimeSpan.Zero, (sum, a) => sum + (a.End - a.Start));
findings.Add($"Spento {absences.Count} volte, in tutto {Duration(total)}.");
text.AppendLine("Spento: " + string.Join(", ", absences.Select(a => $"dal {SoakReport.Moment(a.Start, zone)} per {Duration(a.End - a.Start)}")) + ".");
}
else
{
text.AppendLine("Nessuno spegnimento registrato.");
}
var silences = processes.SelectMany(p => p.Zip(p.Skip(1)).Where(x => x.Second.TakenAt - x.First.TakenAt > Silence)).ToList();
if (silences.Count > 0)
{
var total = silences.Aggregate(TimeSpan.Zero, (sum, x) => sum + (x.Second.TakenAt - x.First.TakenAt));
findings.Add($"Silenzi a processo acceso: {silences.Count}, in tutto {Duration(total)} (computer in sospensione, o lavori bloccati).");
text.AppendLine().AppendLine("Silenzi a processo acceso (nessun campione per più di 11 minuti): " +
string.Join(", ", silences.Take(30).Select(x => $"dal {SoakReport.Moment(x.First.TakenAt, zone)} per {Duration(x.Second.TakenAt - x.First.TakenAt)}")) +
(silences.Count > 30 ? $" e altri {silences.Count - 30}" : "") + ".");
}
return text.ToString();
}
private static string Jobs(List<Run> runs, List<string> findings)
{
var text = new StringBuilder("## Lavori pianificati\n\n");
if (runs.Count == 0)
{
return text.AppendLine("Nessuna esecuzione registrata nel periodo.").ToString();
}
text.AppendLine("| Lavoro | Esecuzioni | Fallite | Saltate | Recuperate | Ritardo medio · massimo | Durata mediana · 95% · massima | Ultimo errore |")
.AppendLine("|---|---|---|---|---|---|---|---|");
foreach (var job in runs.GroupBy(r => r.Job, StringComparer.Ordinal).OrderBy(g => g.Key, StringComparer.Ordinal))
{
var list = job.OrderBy(r => r.Started).ToList();
var failed = list.Count(r => r.Outcome == "failed");
var skipped = list.Count(r => r.Outcome == "skipped");
var late = list.Where(r => r.Outcome != "skipped" && !r.Missed).Select(r => (r.Started - r.Due).TotalSeconds).DefaultIfEmpty(0).ToList();
var took = list.Where(r => r.Outcome != "skipped").Select(r => (r.Finished - r.Started).TotalSeconds).Order().ToList();
var error = list.LastOrDefault(r => r.Error is not null)?.Error;
text.AppendLine(CultureInfo.InvariantCulture,
$"| {job.Key} | {list.Count} | {failed} | {skipped} | {list.Count(r => r.Missed)} | {Seconds(late.Average())} · {Seconds(late.Max())} | " +
$"{Seconds(Rank(took, 0.5))} · {Seconds(Rank(took, 0.95))} · {Seconds(took.LastOrDefault())} | {Cell(error, 160)} |");
if (failed > 0)
{
findings.Add($"Il lavoro «{job.Key}» è fallito {failed} volte su {list.Count}.");
}
if (skipped > 0)
{
findings.Add($"Il lavoro «{job.Key}» è stato saltato {skipped} volte (troppo in ritardo).");
}
}
return text.ToString();
}
private static string Calls(List<ProcessSample> samples, List<string> findings)
{
var text = new StringBuilder("## Chiamate esterne\n\n");
var calls = HttpCallStats.Merge(samples.SelectMany(s => s.Calls));
if (calls.Count == 0)
{
return text.AppendLine("Nessuna chiamata registrata nel periodo.").ToString();
}
text.AppendLine("| Destinazione | Chiamate | Fallite | Esiti | Media ms | 50% ms | 95% ms | Massimo ms |").AppendLine("|---|---|---|---|---|---|---|---|");
foreach (var c in calls)
{
text.AppendLine(CultureInfo.InvariantCulture,
$"| {Cell(c.Host, 80)} | {c.Calls} | {c.Failures} | {Cell(string.Join(", ", c.Outcomes.Select(o => o.Key + "×" + o.Value)), 160)} | {Ms(c.AverageMs)} | " +
$"{Ms(c.Percentile(0.5))} | {Ms(c.Percentile(0.95))} | {Ms(c.MaxMs)} |");
if (c.Calls >= 20 && c.Failures * 20 > c.Calls)
{
findings.Add($"Le chiamate a {c.Host} falliscono spesso: {c.Failures} su {c.Calls}.");
}
if (c.Calls >= 20 && c.Percentile(0.95) >= 5000)
{
findings.Add($"Le chiamate a {c.Host} sono lente: il 95% sta sotto {Ms(c.Percentile(0.95))} ms.");
}
}
return text.ToString();
}
private static string Logs(LogSummary? logs, TimeZoneInfo zone, List<string> findings)
{
var text = new StringBuilder("## Registri tecnici: avvisi ed errori\n\n");
if (logs is null)
{
return text.AppendLine("Cartella dei registri non disponibile.").ToString();
}
text.AppendLine(CultureInfo.InvariantCulture,
$"Letti {logs.Files} file, {logs.Lines} righe nel periodo: {logs.Warnings} avvisi e {logs.Errors} errori{(logs.Truncated ? " (lettura fermata a 256 MB)" : "")}.").AppendLine();
if (logs.Errors > 0)
{
findings.Add($"Errori nei registri tecnici: {logs.Errors}, di {logs.Groups.Count(g => g.Level != "warning")} tipi.");
}
if (logs.Groups.Count == 0)
{
return text.ToString();
}
text.AppendLine("| Livello | Volte | Prima | Ultima | Origine | Messaggio |").AppendLine("|---|---|---|---|---|---|");
foreach (var g in logs.Groups.OrderBy(g => g.Level == "warning" ? 1 : 0).ThenByDescending(g => g.Count).Take(MaxLogGroups))
{
var message = g.Error is null ? g.Sample : g.Sample + " — " + g.Error;
text.AppendLine(CultureInfo.InvariantCulture,
$"| {g.Level} | {g.Count} | {SoakReport.Moment(g.First, zone)} | {SoakReport.Moment(g.Last, zone)} | {Cell(g.Category, 60)} | {Cell(message, 300)} |");
}
if (logs.Groups.Count > MaxLogGroups)
{
text.AppendLine().AppendLine(CultureInfo.InvariantCulture, $"Altri {logs.Groups.Count - MaxLogGroups} tipi di messaggio non mostrati.");
}
return text.ToString();
}
private static string Resources(List<ProcessSample> samples, DiagnosticsRequest request, List<string> findings)
{
var text = new StringBuilder("## Risorse nel tempo\n\n");
var processes = Processes(samples);
if (processes.Count == 0)
{
return text.AppendLine("Nessun campione.").ToString();
}
foreach (var p in processes)
{
var settled = p.Where(s => s.TakenAt - s.ProcessStartedAt >= TimeSpan.FromMinutes(30)).ToList();
var line = $"Processo avviato il {SoakReport.Moment(p[0].ProcessStartedAt, request.Zone)}: memoria da {Mb(p[0].WorkingSet)} a {Mb(p[^1].WorkingSet)} MB, " +
$"memoria gestita da {Mb(p[0].ManagedHeap)} a {Mb(p[^1].ManagedHeap)} MB, handle da {p[0].Handles} a {p[^1].Handles}, thread da {p[0].Threads} a {p[^1].Threads}";
if (settled.Count >= 12 && settled[^1].TakenAt - settled[0].TakenAt >= TimeSpan.FromHours(3))
{
var ws = Slope(settled, s => s.WorkingSet / 1048576.0);
var heap = Slope(settled, s => s.GcCommitted / 1048576.0);
var handles = Slope(settled, s => s.Handles);
var threads = Slope(settled, s => s.Threads);
line += $"; pendenza dopo la prima mezz'ora: memoria {Signed(ws)} MB/ora, memoria del GC {Signed(heap)} MB/ora, handle {Signed(handles)}/ora, thread {Signed(threads)}/ora";
var hours = (settled[^1].TakenAt - settled[0].TakenAt).TotalHours;
if (ws > 2 && ws * hours > settled[0].WorkingSet / 1048576.0 * 0.2)
{
findings.Add($"La memoria cresce di {Signed(ws)} MB l'ora da {Duration(TimeSpan.FromHours(hours))}: possibile perdita.");
}
if (handles > 10 && handles * hours > settled[0].Handles * 0.2)
{
findings.Add($"Gli handle crescono di {Signed(handles)} l'ora: possibile perdita di connessioni o file.");
}
if (threads > 1 && threads * hours > settled[0].Threads * 0.3)
{
findings.Add($"I thread crescono di {Signed(threads)} l'ora.");
}
}
var hoursAlive = (p[^1].TakenAt - p[0].ProcessStartedAt).TotalHours;
if (hoursAlive >= 1 && (p[^1].Exceptions / hoursAlive) > 2000)
{
findings.Add($"Molte eccezioni: {Format.Number((decimal)(p[^1].Exceptions / hoursAlive))} l'ora nel processo avviato il {SoakReport.Moment(p[0].ProcessStartedAt, request.Zone)}.");
}
text.AppendLine(line + ".");
}
var dbFirst = samples.FirstOrDefault(s => s.DbBytes is not null);
var dbLast = samples.LastOrDefault(s => s.DbBytes is not null);
if (dbFirst is { DbBytes: { } a } && dbLast is { DbBytes: { } b } && dbLast.TakenAt - dbFirst.TakenAt >= TimeSpan.FromHours(6))
{
var perDay = (b - a) / (dbLast.TakenAt - dbFirst.TakenAt).TotalDays;
text.AppendLine(CultureInfo.InvariantCulture, $"Il database è passato da {Mb(a)} a {Mb(b)} MB ({Signed(perDay / 1048576.0)} MB al giorno).");
}
var span = request.To - request.From;
var step = Steps.FirstOrDefault(s => span / s <= 48, Steps[^1]);
text.AppendLine()
.AppendLine(CultureInfo.InvariantCulture, $"Un campione ogni {Duration(step)}; i conteggi sono dal campione della riga prima. Memorie in MB, CPU in % di un processore.")
.AppendLine()
.AppendLine("| Ora | Memoria | Gestita | GC | Thread | Handle | Coda | CPU % | GC2 | Pause ms | Eccezioni | Avvisi | Errori | DB | |")
.AppendLine("|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|");
ProcessSample? previous = null;
foreach (var bucket in samples.GroupBy(s => s.TakenAt.UtcTicks / step.Ticks).OrderBy(g => g.Key))
{
var last = bucket.Last();
var same = previous is not null && previous.ProcessStartedAt == last.ProcessStartedAt;
var basis = same ? previous! : null;
var wall = (last.TakenAt - (basis?.TakenAt ?? last.ProcessStartedAt)).TotalMilliseconds;
var cpu = wall <= 0 ? 0 : (last.CpuMs - (basis?.CpuMs ?? 0)) / wall * 100;
text.AppendLine(CultureInfo.InvariantCulture,
$"| {SoakReport.Moment(last.TakenAt, request.Zone)} | {Mb(last.WorkingSet)} | {Mb(last.ManagedHeap)} | {Mb(last.GcCommitted)} | {last.Threads} | {last.Handles} | " +
$"{last.PoolPending} | {Format.Number((decimal)cpu, 1)} | {last.Gen2 - (basis?.Gen2 ?? 0)} | {Format.Number((decimal)(last.GcPauseMs - (basis?.GcPauseMs ?? 0)))} | " +
$"{last.Exceptions - (basis?.Exceptions ?? 0)} | {bucket.Sum(s => s.Warnings)} | {bucket.Sum(s => s.Errors)} | {(last.DbBytes is { } db ? Mb(db) : "")} | " +
$"{(previous is not null && !same ? "riavvio" : "")} |");
previous = last;
}
return text.ToString();
}
private static List<List<ProcessSample>> Processes(List<ProcessSample> samples) =>
samples.GroupBy(s => s.ProcessStartedAt).Select(g => g.OrderBy(s => s.TakenAt).ToList()).OrderBy(p => p[0].TakenAt).ToList();
private static ProcessSample ReadSample(DbRow r) => new()
{
TakenAt = r.Moment("taken_at"),
ProcessStartedAt = r.Moment("process_started_at"),
Kind = r.Text("kind"),
Version = r.Text("version"),
WorkingSet = r.Number("working_set"),
PrivateBytes = r.Number("private_bytes"),
ManagedHeap = r.Number("managed_heap"),
GcCommitted = r.Number("gc_committed"),
Gen0 = r.Number("gc_gen0"),
Gen1 = r.Number("gc_gen1"),
Gen2 = r.Number("gc_gen2"),
GcPauseMs = r.Measure("gc_pause_ms"),
Threads = (int)r.Number("threads"),
Handles = (int)r.Number("handles"),
CpuMs = r.Measure("cpu_ms"),
PoolThreads = (int)r.Number("pool_threads"),
PoolPending = r.Number("pool_pending"),
LockContentions = r.Number("lock_contentions"),
Exceptions = r.Number("exceptions"),
DbBytes = r.IsNull("db_bytes") ? null : r.Number("db_bytes"),
Warnings = (int)r.Number("warnings"),
Errors = (int)r.Number("errors"),
Calls = r.TextOrNull("calls") is { } json ? JsonSerializer.Deserialize<List<HttpCallStats>>(json, DiagnosticSampler.Json) ?? [] : [],
};
/// <summary>The warning and error lines of the technical logs in the window, grouped by what they say.</summary>
internal static LogSummary ReadLogs(string folder, DateTimeOffset from, DateTimeOffset to)
{
var groups = new Dictionary<string, LogGroup>(StringComparer.Ordinal);
int files = 0, warnings = 0, errors = 0;
long lines = 0, read = 0;
var truncated = false;
if (!Directory.Exists(folder))
{
return new LogSummary(0, 0, 0, 0, false, []);
}
var first = DateOnly.FromDateTime(from.UtcDateTime).AddDays(-1);
var last = DateOnly.FromDateTime(to.UtcDateTime).AddDays(1);
foreach (var file in Directory.EnumerateFiles(folder, "anubis-*.log").Order(StringComparer.Ordinal))
{
var name = Path.GetFileNameWithoutExtension(file);
if (!DateOnly.TryParseExact(name["anubis-".Length..], "yyyyMMdd", CultureInfo.InvariantCulture, DateTimeStyles.None, out var day) || day < first || day > last)
{
continue;
}
if (read >= MaxLogBytes)
{
truncated = true;
break;
}
files++;
try
{
using var stream = new FileStream(file, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete);
using var reader = new StreamReader(stream, Encoding.UTF8);
while (reader.ReadLine() is { } line)
{
read += line.Length + 1;
if (read > MaxLogBytes)
{
truncated = true;
break;
}
if (!TryParse(line, out var entry) || entry.At < from || entry.At > to)
{
continue;
}
lines++;
if (entry.Level is not ("warning" or "error" or "critical"))
{
continue;
}
if (entry.Level == "warning")
{
warnings++;
}
else
{
errors++;
}
var category = entry.Category[(entry.Category.LastIndexOf('.') + 1)..];
var key = entry.Level + "|" + category + "|" + Normalize(entry.Message);
if (groups.TryGetValue(key, out var group))
{
groups[key] = group with { Count = group.Count + 1, Last = entry.At > group.Last ? entry.At : group.Last, Error = group.Error ?? entry.Error };
}
else
{
groups[key] = new LogGroup(entry.Level, category, entry.Message, entry.Error, 1, entry.At, entry.At);
}
}
}
catch (IOException)
{
}
catch (UnauthorizedAccessException)
{
}
}
return new LogSummary(files, lines, warnings, errors, truncated, [.. groups.Values]);
}
private static bool TryParse(string line, out (DateTimeOffset At, string Level, string Category, string Message, string? Error) entry)
{
entry = default;
try
{
using var json = JsonDocument.Parse(line);
var root = json.RootElement;
if (!root.TryGetProperty("at", out var at) || !DateTimeOffset.TryParse(at.GetString(), CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal, out var moment))
{
return false;
}
entry = (moment, root.TryGetProperty("level", out var l) ? l.GetString() ?? "" : "", root.TryGetProperty("category", out var c) ? c.GetString() ?? "" : "",
root.TryGetProperty("message", out var m) ? m.GetString() ?? "" : "", root.TryGetProperty("error", out var e) ? e.GetString() : null);
return true;
}
catch (JsonException)
{
return false;
}
}
/// <summary>The message with ids and numbers taken out, so the same trouble with other values counts once.</summary>
internal static string Normalize(string message) => Numbers().Replace(Guids().Replace(message, "‹id›"), "#");
private static double Slope(List<ProcessSample> samples, Func<ProcessSample, double> value)
{
var t0 = samples[0].TakenAt;
var xs = samples.Select(s => (s.TakenAt - t0).TotalHours).ToList();
var ys = samples.Select(value).ToList();
var mx = xs.Average();
var my = ys.Average();
var den = xs.Sum(x => (x - mx) * (x - mx));
return den == 0 ? 0 : xs.Zip(ys).Sum(p => (p.First - mx) * (p.Second - my)) / den;
}
private static double Rank(List<double> sorted, double q) => sorted.Count == 0 ? 0 : sorted[Math.Clamp((int)Math.Ceiling(q * sorted.Count) - 1, 0, sorted.Count - 1)];
private static string Mb(long bytes) => Format.Number((decimal)Math.Round(bytes / 1048576.0, 1), 1);
private static string Ms(double ms) => Format.Number((decimal)Math.Round(ms));
private static string Seconds(double seconds) => seconds < 10 ? Format.Number((decimal)Math.Round(seconds, 2), 2) + " s" : Format.Number((decimal)Math.Round(seconds)) + " s";
private static string Signed(double value) => Format.Number((decimal)Math.Round(value, 2), 2, withSign: true);
private static string Duration(TimeSpan span) => span.TotalMinutes < 90
? Format.Number((decimal)Math.Round(span.TotalMinutes)) + " minuti"
: span.TotalHours < 48 ? SoakReport.Hours(TimeSpan.FromHours(Math.Round(span.TotalHours, 1))) : Format.Number((decimal)Math.Round(span.TotalDays, 1), 1) + " giorni";
private static string Cell(string? text, int max)
{
if (string.IsNullOrEmpty(text))
{
return "";
}
var flat = text.ReplaceLineEndings(" ").Replace("|", "\\|", StringComparison.Ordinal);
return flat.Length <= max ? flat : flat[..max] + "…";
}
[GeneratedRegex(@"\b[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\b")]
private static partial Regex Guids();
[GeneratedRegex(@"\d+(?:[.,:]\d+)*")]
private static partial Regex Numbers();
private sealed record Run(string Job, DateTimeOffset Due, DateTimeOffset Started, DateTimeOffset Finished, string Outcome, bool Missed, string? Error);
internal sealed record LogGroup(string Level, string Category, string Sample, string? Error, int Count, DateTimeOffset First, DateTimeOffset Last);
internal sealed record LogSummary(int Files, long Lines, int Warnings, int Errors, bool Truncated, IReadOnlyList<LogGroup> Groups);
}
+196
View File
@@ -0,0 +1,196 @@
using System.Diagnostics.Metrics;
using System.Globalization;
namespace Anubis.Diagnostics;
/// <summary>
/// The outbound HTTP calls of one destination over a stretch of time (issue #95): how many, how many failed and how,
/// and how long they took, in fixed buckets so that stretches can be added together without keeping every call.
/// </summary>
public sealed record HttpCallStats(string Host, long Calls, long Failures, IReadOnlyDictionary<string, long> Outcomes, long[] Buckets, double MaxMs, double TotalMs)
{
/// <summary>Upper bounds in milliseconds of the duration buckets; one more bucket holds everything slower.</summary>
public static IReadOnlyList<double> Bounds { get; } = [5, 10, 20, 30, 50, 75, 100, 150, 200, 250, 300, 400, 500, 750, 1000, 1500, 2000, 3000, 5000, 10000, 20000, 30000, 60000];
public double AverageMs => Calls == 0 ? 0 : TotalMs / Calls;
/// <summary>
/// The duration under which a share <paramref name="q"/> of the calls finished, read from the buckets: the upper
/// bound of the bucket where that share is reached, never more than the slowest call.
/// </summary>
public double Percentile(double q)
{
var total = Buckets.Sum();
if (total == 0)
{
return 0;
}
var rank = Math.Max(1, (long)Math.Ceiling(q * total));
long seen = 0;
for (var i = 0; i < Buckets.Length; i++)
{
seen += Buckets[i];
if (seen >= rank)
{
return i < Bounds.Count ? Math.Min(Bounds[i], MaxMs) : MaxMs;
}
}
return MaxMs;
}
public static IReadOnlyList<HttpCallStats> Merge(IEnumerable<HttpCallStats> stats)
{
ArgumentNullException.ThrowIfNull(stats);
return stats.GroupBy(s => s.Host, StringComparer.Ordinal).Select(g =>
{
var buckets = new long[Bounds.Count + 1];
var outcomes = new SortedDictionary<string, long>(StringComparer.Ordinal);
foreach (var s in g)
{
for (var i = 0; i < Math.Min(buckets.Length, s.Buckets.Length); i++)
{
buckets[i] += s.Buckets[i];
}
foreach (var (outcome, count) in s.Outcomes)
{
outcomes[outcome] = outcomes.GetValueOrDefault(outcome) + count;
}
}
return new HttpCallStats(g.Key, g.Sum(s => s.Calls), g.Sum(s => s.Failures), outcomes, buckets, g.Max(s => s.MaxMs), g.Sum(s => s.TotalMs));
}).OrderByDescending(s => s.Calls).ThenBy(s => s.Host, StringComparer.Ordinal).ToList();
}
internal static int Bucket(double milliseconds)
{
for (var i = 0; i < Bounds.Count; i++)
{
if (milliseconds <= Bounds[i])
{
return i;
}
}
return Bounds.Count;
}
}
/// <summary>
/// Listens to the duration that .NET records for every outbound HTTP request (meter <c>System.Net.Http</c>,
/// instrument <c>http.client.request.duration</c>) and adds it up by destination until <see cref="Take"/>. No
/// package and no change to the brokers' clients: every <see cref="HttpClient"/> of the process is measured.
/// </summary>
public sealed class HttpCallMeter : IDisposable
{
public const string MeterName = "System.Net.Http";
public const string InstrumentName = "http.client.request.duration";
private static readonly Lazy<HttpCallMeter> _shared = new(() => new HttpCallMeter());
private readonly MeterListener _listener = new();
private readonly Lock _gate = new();
private Dictionary<string, Tally> _tallies = new(StringComparer.Ordinal);
public HttpCallMeter()
{
_listener.InstrumentPublished = (instrument, listener) =>
{
if (instrument.Meter.Name == MeterName && instrument.Name == InstrumentName)
{
listener.EnableMeasurementEvents(instrument);
}
};
_listener.SetMeasurementEventCallback<double>(OnMeasurement);
_listener.Start();
}
/// <summary>The one meter of the running bot, started at the first use.</summary>
public static HttpCallMeter Shared => _shared.Value;
/// <summary>The calls since the previous call to this method, by destination; the counts start again from zero.</summary>
public IReadOnlyList<HttpCallStats> Take()
{
Dictionary<string, Tally> taken;
lock (_gate)
{
taken = _tallies;
_tallies = new Dictionary<string, Tally>(StringComparer.Ordinal);
}
return taken.Select(t => t.Value.Stats(t.Key)).OrderBy(s => s.Host, StringComparer.Ordinal).ToList();
}
public void Dispose() => _listener.Dispose();
private void OnMeasurement(Instrument instrument, double seconds, ReadOnlySpan<KeyValuePair<string, object?>> tags, object? state)
{
string? host = null;
int? port = null;
int? status = null;
string? error = null;
foreach (var tag in tags)
{
switch (tag.Key)
{
case "server.address":
host = tag.Value?.ToString();
break;
case "server.port":
port = tag.Value is null ? null : Convert.ToInt32(tag.Value, CultureInfo.InvariantCulture);
break;
case "http.response.status_code":
status = tag.Value is null ? null : Convert.ToInt32(tag.Value, CultureInfo.InvariantCulture);
break;
case "error.type":
error = tag.Value?.ToString();
break;
}
}
var key = (host ?? "?") + (port is { } p and not (80 or 443) ? ":" + p.ToString(CultureInfo.InvariantCulture) : "");
var outcome = status?.ToString(CultureInfo.InvariantCulture) ?? error ?? "?";
// 101 is the answer that opens a WebSocket: a success like 2xx and 3xx.
var failed = status is not (>= 100 and < 400);
var milliseconds = Math.Max(0, seconds * 1000);
lock (_gate)
{
if (!_tallies.TryGetValue(key, out var tally))
{
tally = new Tally();
_tallies[key] = tally;
}
tally.Add(outcome, failed, milliseconds);
}
}
private sealed class Tally
{
private readonly long[] _buckets = new long[HttpCallStats.Bounds.Count + 1];
private readonly SortedDictionary<string, long> _outcomes = new(StringComparer.Ordinal);
private long _calls;
private long _failures;
private double _max;
private double _total;
public void Add(string outcome, bool failed, double milliseconds)
{
_calls++;
if (failed)
{
_failures++;
}
_outcomes[outcome] = _outcomes.GetValueOrDefault(outcome) + 1;
_buckets[HttpCallStats.Bucket(milliseconds)]++;
_max = Math.Max(_max, milliseconds);
_total += milliseconds;
}
public HttpCallStats Stats(string host) => new(host, _calls, _failures, new SortedDictionary<string, long>(_outcomes, StringComparer.Ordinal),
(long[])_buckets.Clone(), Math.Round(_max, 1), Math.Round(_total, 1));
}
}
+133
View File
@@ -0,0 +1,133 @@
using Anubis.Core;
using Anubis.Db;
namespace Anubis.Notifications;
/// <summary>The state of a notification (SPEC 14): never "sent" before Telegram said ok.</summary>
public static class NotificationStates
{
/// <summary>In the queue: not tried yet, or Telegram is not configured.</summary>
public const string Pending = "pending";
public const string Sent = "sent";
/// <summary>Tried and refused; tried again at <c>next_attempt_at</c>, or never again when that is empty.</summary>
public const string Failed = "failed";
}
/// <summary>The bot's Telegram: the token and the one chat it writes to and takes commands from.</summary>
public sealed record TelegramSettings(string Token, string ChatId)
{
public const string TokenSecret = "TELEGRAM_BOT_TOKEN";
public const string ChatSecret = "TELEGRAM_CHAT_ID";
/// <summary>Both secrets, or null while either is missing.</summary>
public static async Task<TelegramSettings?> ReadAsync(SecretStore secrets, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(secrets);
var token = await secrets.RevealAsync(TokenSecret, cancellationToken).ConfigureAwait(false);
var chat = await secrets.RevealAsync(ChatSecret, cancellationToken).ConfigureAwait(false);
return string.IsNullOrWhiteSpace(token) || string.IsNullOrWhiteSpace(chat) ? null : new TelegramSettings(token.Trim(), chat.Trim());
}
}
/// <summary>What Impostazioni shows of the queue: the last message that left, what waits, what failed and why.</summary>
public sealed record NotificationSummary(DateTimeOffset? LastSentAt, int Pending, int Retrying, int GaveUp, string? LastError);
/// <summary>
/// The Telegram queue (SPEC 14). Every message is written first, then sent: a refusal or a network failure is
/// recorded and tried again with growing pauses; a message that could not leave within a day is given up, with the
/// reason, rather than arriving late as if it were news.
/// </summary>
public sealed class NotificationSender(Database db, IClock clock, Func<CancellationToken, Task<TelegramSettings?>> settings, Func<string, TelegramClient> client)
{
/// <summary>Older than this, a message is no longer worth sending.</summary>
public static readonly TimeSpan MaxAge = TimeSpan.FromHours(24);
public const int MaxAttempts = 8;
/// <summary>At most this many messages per pass: Telegram limits the messages to one chat.</summary>
public const int Batch = 20;
/// <summary>Writes a message in the queue, in the same transaction as what it tells.</summary>
public static Task EnqueueAsync(DbSession s, IClock clock, string kind, string message, CancellationToken cancellationToken = default)
{
ArgumentNullException.ThrowIfNull(s);
ArgumentNullException.ThrowIfNull(clock);
return s.ExecuteAsync(
"INSERT INTO notifications (id, channel, kind, message, status, attempts, created_at) VALUES (@id, 'telegram', @k, @m, @st, 0, @at)", cancellationToken,
("id", Ids.NewId(clock)), ("k", kind), ("m", message), ("st", NotificationStates.Pending), ("at", clock.UtcNow));
}
/// <summary>The pause before attempt <paramref name="attempts"/> + 1: 1, 2, 4, 8, 16, 32, then 60 minutes.</summary>
public static TimeSpan Pause(int attempts) => TimeSpan.FromMinutes(Math.Min(60, 1 << Math.Clamp(attempts - 1, 0, 6)));
/// <summary>Sends what is due; the answer is how many left. Without Telegram configured everything stays in the queue.</summary>
public async Task<int> SendDueAsync(CancellationToken cancellationToken)
{
if (await settings(cancellationToken).ConfigureAwait(false) is not { } telegram)
{
return 0;
}
var now = clock.UtcNow;
var due = await db.ReadAsync(s => s.QueryAsync(
"SELECT id, message, attempts, created_at FROM notifications WHERE channel = 'telegram' AND (status = @p OR (status = @f AND next_attempt_at IS NOT NULL AND next_attempt_at <= @now)) " +
"ORDER BY created_at, id LIMIT " + Batch.ToString(System.Globalization.CultureInfo.InvariantCulture),
r => (Id: r.Id("id"), Message: r.Text("message"), Attempts: r.IsNull("attempts") ? 0 : (int)r.Number("attempts"), Created: r.Moment("created_at")),
("p", NotificationStates.Pending), ("f", NotificationStates.Failed), ("now", now)), cancellationToken).ConfigureAwait(false);
var bot = client(telegram.Token);
var sent = 0;
foreach (var item in due)
{
if (now - item.Created > MaxAge)
{
await MarkAsync(item.Id, NotificationStates.Failed, item.Attempts, null, "Non partito entro un giorno: non lo mando più.", cancellationToken).ConfigureAwait(false);
continue;
}
try
{
await bot.SendAsync(telegram.ChatId, item.Message, cancellationToken).ConfigureAwait(false);
await MarkAsync(item.Id, NotificationStates.Sent, item.Attempts + 1, null, null, cancellationToken).ConfigureAwait(false);
sent++;
}
catch (TelegramException ex)
{
var attempts = item.Attempts + 1;
DateTimeOffset? next = ex.Retryable && attempts < MaxAttempts
? clock.UtcNow + (ex.RetryAfter is { } wait && wait > Pause(attempts) ? wait : Pause(attempts))
: null;
await MarkAsync(item.Id, NotificationStates.Failed, attempts, next, ex.Message, cancellationToken).ConfigureAwait(false);
if (ex.Retryable)
{
// Telegram unreachable or asking to slow down: the others wait for the next pass.
break;
}
}
}
return sent;
}
public Task<NotificationSummary> SummaryAsync(CancellationToken cancellationToken) =>
db.ReadAsync(async s =>
{
var last = await s.FirstOrDefaultAsync("SELECT MAX(sent_at) AS at FROM notifications WHERE channel = 'telegram' AND status = @s",
r => r.MomentOrNull("at"), ("s", NotificationStates.Sent)).ConfigureAwait(false);
var pending = await s.CountAsync("SELECT COUNT(*) FROM notifications WHERE channel = 'telegram' AND status = @p", ("p", NotificationStates.Pending)).ConfigureAwait(false);
var retrying = await s.CountAsync("SELECT COUNT(*) FROM notifications WHERE channel = 'telegram' AND status = @f AND next_attempt_at IS NOT NULL",
("f", NotificationStates.Failed)).ConfigureAwait(false);
var gaveUp = await s.CountAsync("SELECT COUNT(*) FROM notifications WHERE channel = 'telegram' AND status = @f AND next_attempt_at IS NULL",
("f", NotificationStates.Failed)).ConfigureAwait(false);
var error = await s.FirstOrDefaultAsync(
"SELECT last_error FROM notifications WHERE channel = 'telegram' AND status = @f ORDER BY created_at DESC, id DESC LIMIT 1",
r => r.TextOrNull("last_error"), ("f", NotificationStates.Failed)).ConfigureAwait(false);
return new NotificationSummary(last, (int)pending, (int)retrying, (int)gaveUp, error);
}, cancellationToken);
private Task<int> MarkAsync(Guid id, string status, int attempts, DateTimeOffset? next, string? error, CancellationToken cancellationToken) =>
db.WriteAsync(s => s.ExecuteAsync(
"UPDATE notifications SET status = @s, attempts = @a, next_attempt_at = @n, last_error = @e, sent_at = @sent WHERE id = @id", cancellationToken,
("s", status), ("a", attempts), ("n", next), ("e", error), ("sent", status == NotificationStates.Sent ? clock.UtcNow : (DateTimeOffset?)null), ("id", id)), cancellationToken);
}
+289
View File
@@ -0,0 +1,289 @@
using Anubis.Core;
using Microsoft.Extensions.Logging;
namespace Anubis.Notifications;
/// <summary>
/// What the Telegram commands read and do (SPEC 14), answered in plain Italian. Nothing here turns on real money,
/// changes a limit or shows a secret: the interface does those, never the phone.
/// </summary>
public interface ITelegramDesk
{
Task<string> StatusAsync(CancellationToken cancellationToken);
Task<string> PositionsAsync(CancellationToken cancellationToken);
Task<string> TodayAsync(CancellationToken cancellationToken);
Task<string> PauseAsync(CancellationToken cancellationToken);
/// <summary>Only from a pause, never after "Ferma tutto" (SPEC 9.3, 14).</summary>
Task<string> ResumeAsync(CancellationToken cancellationToken);
/// <summary>"Ferma tutto", already confirmed.</summary>
Task<string> StopAllAsync(CancellationToken cancellationToken);
}
/// <summary>The commands of SPEC 14 and the confirmation of /ferma: two minutes to send /conferma.</summary>
public sealed class TelegramCommands(ITelegramDesk desk, IClock clock)
{
public static readonly TimeSpan ConfirmWithin = TimeSpan.FromMinutes(2);
public const string Help =
"Comandi:\n" +
"/stato – come sta il bot\n" +
"/posizioni – cosa ho in mano, con gli stop\n" +
"/oggi – cosa ho fatto oggi\n" +
"/pausa – niente nuovi acquisti, stop e uscite proseguono\n" +
"/riprendi – riparto dalla pausa\n" +
"/ferma – «Ferma tutto», con conferma\n" +
"/aiuto – questo elenco\n" +
"Soldi veri, limiti e chiavi si cambiano solo dall'interfaccia.";
private DateTimeOffset? _stopAskedAt;
/// <summary>The answer to one message; null for a message that is not a command.</summary>
public async Task<string?> AnswerAsync(string text, CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(text);
var word = text.Trim().Split((char[])[' ', '\n', '\t'], 2)[0];
if (!word.StartsWith('/'))
{
return null;
}
// In a group Telegram adds the bot's name: "/stato@AnubisBot".
var command = word[1..].Split('@')[0].ToLowerInvariant();
if (command != "conferma")
{
_stopAskedAt = null;
}
switch (command)
{
case "start" or "aiuto" or "help":
return Help;
case "stato":
return await desk.StatusAsync(cancellationToken).ConfigureAwait(false);
case "posizioni":
return await desk.PositionsAsync(cancellationToken).ConfigureAwait(false);
case "oggi":
return await desk.TodayAsync(cancellationToken).ConfigureAwait(false);
case "pausa":
return await desk.PauseAsync(cancellationToken).ConfigureAwait(false);
case "riprendi":
return await desk.ResumeAsync(cancellationToken).ConfigureAwait(false);
case "ferma":
_stopAskedAt = clock.UtcNow;
return "Vuoi davvero «Ferma tutto»? Annullo gli ordini in attesa e non apro nuove posizioni; gli stop restano. " +
"Da qui non si riparte: si riparte solo dall'interfaccia, pagina Rischio.\nPer confermare scrivi /conferma entro 2 minuti.";
case "conferma":
if (_stopAskedAt is { } asked && clock.UtcNow - asked <= ConfirmWithin)
{
_stopAskedAt = null;
return await desk.StopAllAsync(cancellationToken).ConfigureAwait(false);
}
_stopAskedAt = null;
return "Non c'è niente da confermare: scrivi prima /ferma.";
default:
return "Non conosco questo comando.\n" + Help;
}
}
}
/// <summary>
/// Reads the messages sent to the bot and answers the commands (SPEC 14). Only the chat of the settings is obeyed;
/// the others get no answer at all. A message older than five minutes, found after a stop of the bot, is not
/// obeyed: a /pausa of yesterday must not pause today.
/// </summary>
public sealed partial class TelegramBot(IClock clock, Func<CancellationToken, Task<TelegramSettings?>> settings, Func<string, TelegramClient> client,
TelegramCommands commands, ILogger logger)
{
public static readonly TimeSpan StaleAfter = TimeSpan.FromMinutes(5);
/// <summary>How long one call waits for a message on Telegram's side.</summary>
public int WaitSeconds { get; init; } = 25;
/// <summary>The pause after a failure, or while Telegram is not configured.</summary>
public TimeSpan Retry { get; init; } = TimeSpan.FromSeconds(30);
private long _offset;
/// <summary>One read of the messages; the answer is how many commands were obeyed.</summary>
public async Task<int> PollOnceAsync(CancellationToken cancellationToken)
{
if (await settings(cancellationToken).ConfigureAwait(false) is not { } telegram)
{
return -1;
}
var bot = client(telegram.Token);
var updates = await bot.GetUpdatesAsync(_offset, WaitSeconds, cancellationToken).ConfigureAwait(false);
var obeyed = 0;
foreach (var update in updates)
{
_offset = Math.Max(_offset, update.UpdateId + 1);
if (update.ChatId != telegram.ChatId)
{
LogStranger(logger, update.ChatId ?? "?");
continue;
}
if (update.Text is not { } text || update.Date is not { } date || clock.UtcNow - date > StaleAfter)
{
continue;
}
string? answer;
try
{
answer = await commands.AnswerAsync(text, cancellationToken).ConfigureAwait(false);
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
LogCommandFailed(logger, ex);
answer = "Non ci sono riuscito: " + Redactor.Redact(ex.Message);
}
if (answer is not null)
{
obeyed++;
await bot.SendAsync(telegram.ChatId, answer, cancellationToken).ConfigureAwait(false);
}
}
return obeyed;
}
/// <summary>The loop: reads, answers, and after a failure waits and tries again.</summary>
public async Task RunAsync(CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
TimeSpan? pause = null;
try
{
if (await PollOnceAsync(cancellationToken).ConfigureAwait(false) < 0)
{
pause = Retry;
}
}
catch (TelegramException ex)
{
LogPollFailed(logger, ex.Message);
pause = ex.RetryAfter is { } after && after > Retry ? after : Retry;
}
catch (Exception ex) when (!cancellationToken.IsCancellationRequested)
{
// The database or the network failed in a way Telegram did not report: the loop goes on after a pause.
LogLoopFailed(logger, ex);
pause = Retry;
}
if (pause is { } wait)
{
await Task.Delay(wait, cancellationToken).ConfigureAwait(false);
}
}
}
[LoggerMessage(Level = LogLevel.Warning, Message = "Telegram message from another chat ignored: {Chat}")]
private static partial void LogStranger(ILogger logger, string chat);
[LoggerMessage(Level = LogLevel.Warning, Message = "Telegram read failed: {Error}")]
private static partial void LogPollFailed(ILogger logger, string error);
[LoggerMessage(Level = LogLevel.Error, Message = "Reading the Telegram commands failed")]
private static partial void LogLoopFailed(ILogger logger, Exception ex);
[LoggerMessage(Level = LogLevel.Error, Message = "Telegram command failed")]
private static partial void LogCommandFailed(ILogger logger, Exception ex);
}
/// <summary>
/// The sign of life (SPEC 14): a call to an outside checking service every minute. When the calls stop, that
/// service warns the user: the only way to know the bot is off while it is off.
/// </summary>
public sealed partial class Heartbeat(HttpClient http, IClock clock, Func<CancellationToken, Task<Uri?>> address, ILogger logger)
{
public DateTimeOffset? LastOk { get; private set; }
public string? LastError { get; private set; }
/// <summary>True when the call went through; false when there is no address or it failed.</summary>
public async Task<bool> BeatAsync(CancellationToken cancellationToken)
{
if (await address(cancellationToken).ConfigureAwait(false) is not { } url)
{
return false;
}
try
{
using var response = await http.GetAsync(url, cancellationToken).ConfigureAwait(false);
if (response.IsSuccessStatusCode)
{
LastOk = clock.UtcNow;
LastError = null;
return true;
}
LastError = $"Il servizio ha risposto {(int)response.StatusCode}.";
}
catch (HttpRequestException ex)
{
LastError = "Il servizio non risponde: " + Redactor.Redact(ex.Message);
}
catch (TaskCanceledException) when (!cancellationToken.IsCancellationRequested)
{
LastError = "Il servizio non ha risposto in tempo.";
}
LogFailed(logger, LastError);
return false;
}
/// <summary>An address the interface accepts: https, or http only on this computer or the local network.</summary>
public static string? Check(string value)
{
if (string.IsNullOrWhiteSpace(value))
{
return null;
}
if (!Uri.TryCreate(value.Trim(), UriKind.Absolute, out var uri) || uri.Scheme is not ("https" or "http"))
{
return "Serve un indirizzo che inizi con https://.";
}
return uri.Scheme == "http" && !(uri.IsLoopback || uri.Host.EndsWith(".local", StringComparison.OrdinalIgnoreCase) || IsPrivate(uri.Host))
? "Fuori dalla rete di casa serve https://."
: null;
}
/// <summary>An address of a home network: 10.x, 172.16-31.x, 192.168.x, link-local, or their IPv6 equivalents.</summary>
private static bool IsPrivate(string host)
{
if (!System.Net.IPAddress.TryParse(host.Trim('[', ']'), out var ip))
{
return false;
}
if (ip.IsIPv4MappedToIPv6)
{
ip = ip.MapToIPv4();
}
if (ip.AddressFamily == System.Net.Sockets.AddressFamily.InterNetworkV6)
{
return System.Net.IPAddress.IsLoopback(ip) || ip.IsIPv6LinkLocal || ip.IsIPv6SiteLocal || ip.IsIPv6UniqueLocal;
}
var b = ip.GetAddressBytes();
return b[0] == 10 || b[0] == 127 || (b[0] == 172 && b[1] is >= 16 and <= 31) || (b[0] == 192 && b[1] == 168) || (b[0] == 169 && b[1] == 254);
}
[LoggerMessage(Level = LogLevel.Warning, Message = "Heartbeat failed: {Error}")]
private static partial void LogFailed(ILogger logger, string error);
}
Loaded 100 of 166 files, more files were not shown because too many files have changed in this diff. Show more