- Gaps between the last stored bar and the first new one were never found,
so a minute the live feed skipped, or a day missing right after the stored
history, was lost for good. A gap is now new when it ends after the last
bar held, and is downloaded again like any other.
- When one OKX connection dropped while the other kept writing, the write to
the closed feed threw out of the live loop and left it marked connected.
Writes now stop quietly, and the loop treats any failure (database
included) as a drop: marked down, back off, reconnect.
- The background worker now survives a failed round instead of stopping for
the life of the process.
- Alpaca gives adjusted prices on the basis of the download day, so a later
split looked like a crash. Each download re-reads the last stored day and,
if its adjusted/real ratio changed, rescales all older adjusted prices.
- Stock days are fetched up to the last closed New York session, so the job
after the close collects that day; missing stock days are checked against
the exchange calendar; delisted stocks with history are not asked again.
- Smaller: a lock in the freshness monitor, no NaN mood readings (SQLite
stored them as NULL), and the network proof runs only with
ANUBIS_LIVE_PROOF=1. The test wait is 30 s for busy machines.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ADR 0008 collects the market data decisions the spec leaves open: research
history in csv.gz instead of Parquet (no extra package), the immediate second
reading of suspect data, EUR pairs only, Alpaca only with the paper keys,
name-based fund exclusion and the background worker. API-NOTES gets the facts
checked while writing the adapters and the biases that remain in past
universes. ROADMAP and STATE mark the phase done, with the stock history still
waiting for the user's Alpaca keys.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A LiveDemo test, outside the normal suite, that downloads the whole OKX EUR
daily history of BTC, ETH and SOL and the Binance BTCUSDT archives since 2017,
then writes the quality report. It is the evidence for the phase criteria that
a fake network cannot give.
The detached database test now retries the file move for a few seconds:
Windows refuses to move the SQLite file while a request of the running server
holds it, which made the test fail now and then.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A "Dati" section lists every external source with its state, last answer
and the reason it is off (Alpaca without the paper keys, FRED and Finnhub
without theirs), links the 30-day quality report, and edits the Fed/ECB
meeting list; a list that does not parse is refused with an example.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MarketDataService wires the collector, the reference sources, the
universes and the corporate actions into the scheduler. Short jobs (rates,
calendar, mood, spread samples, corporate actions) run on their schedule;
long downloads (catalogues, the first history, thousands of stocks, the
Binance archives) run on a background worker the jobs only wake, so they
never delay the backup or each other. The live crypto feed follows the
crypto universe, or BTC, ETH and SOL until one exists.
Alpaca uses only the paper account keys; without them the stock part is off
and the sources list says why. Market data is off in demo mode, with
ANUBIS_MARKET_DATA=off, and in tests that bring no fake network.
New: GET /api/data/quality (JSON or Markdown) and /api/data/sources, and
"anubis data report|binance|csv" on the command line.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MarketCollector downloads what is missing at startup and after every drop
of the live feed, checks every bar and never corrects in silence: suspect
bars are stored as suspect and read a second time (same values confirm,
different ones replace, impossible ones are dropped), gaps are downloaded
again once and what the source still lacks is recorded as unrecoverable.
A test detaches the feed for ten minutes and finds every minute recovered.
The stock and crypto universes are chosen "as of a date" from the bars up to
that date only: funds, ETFs and ETNs are excluded through nasdaqtraded.txt
and the security name, stocks delisted later still count when they traded.
Corporate actions are kept once per source key and found by ex-date.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Readers for the ECB rates, the Forex Factory calendar, FRED release dates,
Finnhub earnings, the Fed/ECB meeting list, fear and greed, the Cboe VIX
history, nasdaqtraded.txt and the Binance kline archives, each tested on
recorded responses. ReferenceSources downloads them within their limits
and records every attempt in source_state, so limits hold across restarts
and the interface can say which source is off and why (FRED and Finnhub
without a key). The central-bank list is an editable, validated setting.
BinanceImporter checks every monthly zip against its SHA-256 file before
writing it to the research history; CsvImporter reads user files with a
configurable column mapping and reports unreadable lines and gaps instead
of fixing them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both clients are written in house on HttpClient and ClientWebSocket, behind
IMarketDataSource, with a rate limiter on the injected clock and retries
that wait on it too.
OKX: EUR pairs only (trap 14), rules from public/instruments, daily
candles with bar=1Dutc and a refusal of any candle not opening at 00:00
UTC (trap 23), backwards paging of history-candles, errors read from the
code field because OKX answers them with HTTP 200, tickers on the public
stream and candles on the business stream with ping after silence. The
HMAC signer and the demo header are ready for Phase 3.
Alpaca: consolidated daily bars, real and adjusted, dated in New York and
never newer than 15 minutes (trap 8), an explicit feed on every request,
IEX for recent minutes, quotes and the stream (at most 30 symbols),
calendar and clock converted to UTC, corporate actions. Without keys
nothing is sent and the reason is stated. Tests use the documentation's
examples and synthetic answers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Migration 0002 adds the market data tables on both engines: daily and
one-minute bars, corporate actions, FX rates, the exchange calendar,
calendar events, mood indicators, spread samples, data issues, source
state and universe members, plus the trading rules on instruments.
The data layer is format-independent so the venue clients can build on
it: the instrument catalogue (rules from the venue, delisted kept), bar
upserts with ON CONFLICT, the SPEC 6.4 checks with an issue register
that blocks entries until a second reading, freshness by age of the last
datum (trap 10), research history in .csv.gz with exact decimals, and
adjusted series anchored at the decision day with level and quantity
conversion across splits and dividends.
Recorded API responses from OKX, the Alpaca docs and the free sources
are kept byte for byte as test fixtures; the secret scan now ignores
pagination cursors such as Alpaca's next_page_token.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The user allowed the download, so the interface now uses the typeface
of the mockups instead of the system one. The variable file (weights
400-700) and its SIL OFL licence are embedded and served by the app
itself, never fetched from the internet; source and SHA-256 are in
API-NOTES. The typeface is slightly wider, so top-bar labels and
buttons move to 13 px as in the mockup, keeping the date line whole at
1440 px.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An error-page choice released the startup loop before resetting the
services, so the loop could start the application and then lose its
services while reporting ready. The reset now happens while the loop is
still held, and moves the status back to Starting.
A SQLite file could vanish between the existence check and the open;
with create allowed, the open would put an empty database in its place.
Once the file has been seen, connections open read-write only and fail
on a missing file.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 1 completed on top of the library foundations:
- Server: startup state machine (config, first run, database down, schema,
identity, ready) with an error page that retries by itself; the configured
database is the only one ever opened, and a vanished SQLite file is never
recreated. REST API and status WebSocket; session cookie auth, CSRF on
every form and write call; desktop and demo answer only the computer itself.
- Interface: static Blazor rendering with plain forms (no interactive
runtime), menu, top bar and phone bottom bar as in the mockups, light and
dark themes with appendix C tokens and a contrast test, Panoramica, Diario,
Rischio, Impostazioni; service pages for sign-in, first access, desktop
first run and the database error page with its choices.
- Demo mode (tasks seed-demo): appendix F in a database of its own under the
temp folder, frozen clock, never touches the configured installation.
- Docker: entrypoint gives /data to PUID:PGID and drops root with setpriv,
HEALTHCHECK through bash /dev/tcp; proven with SQLite, PostgreSQL, a wrong
configuration and a database that goes away and comes back.
- tasks screenshots drives the installed Edge/Chrome through DevTools for
exact 1440 and 390 pixel shots; anubis db export|import on the command line.
Fixes from the independent review: return paths without control characters,
Host check against DNS rebinding, attempt limit counted before the check and
also on the error-page password, one-use TOTP codes and a code to switch the
second factor off, error-page choices serialized with the startup loop,
identity checked before migrating, restore restarts with "Ferma tutto" on,
damaged archives reported instead of crashing. Running the server tests on
PostgreSQL exposed pages rendered before their data arrived; data-loading
components now load first (LoadedComponent).
Decisions in ADR 0007. Tests: 100 on SQLite and on PostgreSQL.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Phase 1 foundations in the Anubis library:
- Platform: startup configuration from environment variables (Docker) or
config.toml (desktop, in-house TOML subset), data folder layout, secret key
custody (ANUBIS_SECRET_KEY, DPAPI on Windows, restricted file elsewhere),
installation identity and database fingerprint.
- Db: ADO.NET with Npgsql and Microsoft.Data.Sqlite, money as NUMERIC(38,18)
or canonical text rounded to 18 decimals, UTC moments, UUIDv7 ids; numbered
SQL migrations with type tokens and a version table; engine-independent
export/import with count and accounting-total checks; backups keeping 14.
- Core: injectable clock, channel event bus, scheduler with fixed, interval
and market-relative times, settings with change log, AES-GCM secrets, log
redaction, Italian formats.
- Access: PBKDF2-SHA512 hashes, RFC 6238 TOTP, hashed session tokens,
sign-in attempt limit.
SQLitePCLRaw (Apache-2.0) arrives with Microsoft.Data.Sqlite and is declared
in DependencyPolicyTests and API-NOTES.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The independent review found that the policy tests could be bypassed:
Microsoft-named direct packages and GlobalPackageReference slipped past
the dependency check, the live-money scan missed the JSON, Dockerfile
and Unraid template forms, the secret scan missed short project
secrets but would have failed on the user's own ignored .env. The
scans now read only the files git would commit and cover those cases.
Also adds the fixed-seed generator for property tests, the REGISTRY
variables reserved for phase 9 in the CI, a git pre-commit hook in
place of the pre-commit tool (tasks hooks), the missing SPEC 4 facts
in API-NOTES (Fed/ECB/BLS times, USDC depth on OKX) and the full list
of today's indirect packages.
refs #3 refs #5 refs #6
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Empty but complete solution so every later phase starts from a working
toolchain: analyzers as errors, locked restore, format check, policy tests
that keep packages within ADR 0004 and keep live trading switched off,
a /health endpoint served from Docker, and a Gitea workflow with only
run steps. API facts of SPEC section 4 re-checked in docs/API-NOTES.md;
ADR 0006 records what changed (MSTest base packages, Ubuntu images,
OKX WebSocket on 443, Alpaca margin-only accounts).
refs #1 refs #2 refs #3 refs #4 refs #5
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The user prefers C# and asked for no SDKs or extra components without
permission. Speed is not a factor at this trading frequency and Python's
advantages were exactly the SDKs now excluded, so ADR 0003 adopts the
appendix D variant with Blazor instead of React, in-house broker clients,
ADO.NET, .csv.gz research storage, a C# task runner instead of make and an
in-house Windows installer instead of Inno Setup.
ADR 0004 lists the only packages the user authorized (Microsoft.Data.Sqlite,
ProtectedData, WebView2, MSTest, Npgsql, ML.NET with LightGBM/FastTree,
ONNX Runtime, Tokenizers) and the ask-first rule for anything else.
ADR 0005 records why the bot is called Anubis (he weighs and protects) and
replaces the two-semicircle mark with the scale of the weighing of the
heart: SVG, PNG, ICO and Unraid icon, applied to all nine screens.
CLAUDE.md, SVILUPPO, STATE, README, ROADMAP and templates follow; the 84
Gitea issues were rewritten to match and verified against the source.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Load the full project specification renamed from the provisional "Giano"
to "Anubis" (ADR 0001), the nine reference screens with the new name,
the Unraid template draft pointing at the Gitea container registry, and
the working documents the development phases rely on: CLAUDE.md,
ROADMAP (linked to the 84 Gitea issues and 12 milestones), STATE,
development rules (SVILUPPO), ADR 0002 on Gitea issues, packages and
installer, and Gitea issue/PR templates. No application code yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>