From 0f8cfd4755816a7aed98f8044c7fbaca2357f099 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 2 Oct 2026 22:38:50 +0000 Subject: [PATCH] docs: architecture map, phase status, changelog and the development README ARCHITECTURE gains the module map, hosts, job kinds with their handlers, the data flow and the extension points; STATUS records the measured performance gates and the Phase 8 next steps; CHANGELOG summarises everything built under Unreleased; README covers build, tests, CI, the release process, the secrets and how to bootstrap the public releases repository. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_012iLMBNwBE2GnphAsu4cjaC --- README.md | 119 ++++++++++++++++++++++++- docs/ARCHITECTURE.md | 205 ++++++++++++++++++++++++++++++++----------- docs/CHANGELOG.md | 85 +++++++++++++++--- docs/STATUS.md | 32 +++++-- 4 files changed, 371 insertions(+), 70 deletions(-) diff --git a/README.md b/README.md index 39227d8e..963b8d80 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,117 @@ -# TimeLapseCreator -Desktop application for creating time lapse starting from a sequence +# Titano (development repository) + +Time-lapse ecosystem: **Titano** (Windows desktop client), **Titano Server** (Linux/Docker and Windows) and +**Titano Uploader** (Windows), sharing one C#/.NET 10 engine with zero third-party packages. This repository is +private; users get releases, the Docker image and the Unraid template from the public releases repository. + +- Specification: [`docs/BRIEF.md`](docs/BRIEF.md) · working rules: [`CLAUDE.md`](CLAUDE.md) +- Architecture and conventions: [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) · decisions: + [`docs/DECISIONS.md`](docs/DECISIONS.md) · status: [`docs/STATUS.md`](docs/STATUS.md) · changes: + [`docs/CHANGELOG.md`](docs/CHANGELOG.md) +- Visual design: [`docs/ui/DESIGN.md`](docs/ui/DESIGN.md) and `docs/ui/mockups/` + +## Build, test, run + +Requirements: the .NET SDK pinned in `global.json`. Windows builds and runs everything; Linux builds everything +(the Windows projects for compile checking) and runs Core, Data, Protocol, Server and Tools. + +```sh +dotnet build Titano.slnx -c Release + +# Tests (one program per area; --filter text|category:name, --junit file, --verbose) +dotnet run --project tests/Titano.Core.Tests -c Release -- --junit out/core.xml +dotnet run --project tests/Titano.Data.Tests -c Release +dotnet run --project tests/Titano.Server.Tests -c Release +dotnet run --project tests/Titano.Core.Windows.Tests -c Release # Windows +dotnet run --project tests/Titano.App.Tests -c Release # Windows + +# PostgreSQL integration tests (16 or 17) +TITANO_TEST_PG="Host=127.0.0.1;Port=5432;Database=titano_test;Username=titano;Password=…;SslMode=disable" \ + dotnet run --project tests/Titano.Data.Tests -c Release + +# Performance gates (500,000-photo library, 100,000-frame rebuild; minutes) +TITANO_PERF=1 dotnet run --project tests/Titano.Data.Tests -c Release -- --filter category:perf --verbose + +# Checks run by CI +dotnet run --project src/Titano.Tools -- package-guard +dotnet run --project src/Titano.Tools -- i18n check +dotnet run --project src/Titano.Tools -- xaml-lint +dotnet format whitespace Titano.slnx --verify-no-changes + +# CC0 RAW samples for the decoder tests (tests/fixtures/manifest.json explains how to fill in URLs) +dotnet run --project src/Titano.Tools -- fixtures +``` + +Run: + +```sh +dotnet run --project src/Titano.Server # admin panel https://localhost:8443/admin +src/Titano.App/bin/Release/net10.0-windows/Titano.exe [--capture ] +src/Titano.Uploader/bin/Release/net10.0-windows/Titano.Uploader.exe +docker compose -f deploy/docker/docker-compose.yml up -d +``` + +The server reads `/data/server.json` (`TITANO_DATA_PATH`) and `TITANO_*` environment variables; +`Titano.Server --help` lists them. On Linux it needs PostgreSQL; on Windows it defaults to SQLite. + +## CI + +| Workflow | When | What | +|---|---|---| +| `.github/workflows/ci.yml` | push to `main`, pull requests | Windows + Linux build (warnings are errors), all tests, package guard, i18n check, XAML lint, format check, CC0 fixtures (a warning per missing sample), screenshots (`--capture`, also pushed to the `ci-screenshots` branch for same-repository PRs), Windows program artifacts, PostgreSQL 16/17 integration, Docker image smoke test | +| `.github/workflows/docker.yml` | nightly | `ghcr.io//titano-server:nightly` (multi-arch) | +| `.github/workflows/perf.yml` | weekly, manual | Performance gates on SQLite and PostgreSQL 17; timings in the job summary | +| `.github/workflows/release.yml` | tag `v*`, manual re-publish | The release (below) | + +## Release process + +1. Make sure `main` is green and `docs/CHANGELOG.md` / `docs/STATUS.md` are up to date. +2. Tag and push: `git tag v0.2.0 && git push origin v0.2.0` (pre-releases: `v0.2.0-rc.1`; they are published as + GitHub pre-releases, get no `:latest` image and are ignored by the in-app update check). +3. `release.yml` then: + - validates the tag, builds with `TitanoVersion` from the tag and runs every test on Windows; + - publishes self-contained single-file programs, signs them when signing secrets exist, packs + `Titano-Setup-.exe` and `Titano-Uploader-Setup-.exe` (`Titano.Tools pack-installer`), signs the setups, + zips `Titano-Server-win-x64-.zip` and `Titano-win-x64-portable-.zip`; + - writes release notes from the Conventional Commits since the previous tag (`Titano.Tools changelog`); + - builds and pushes `ghcr.io//titano-server:` (and `:latest` for stable tags) for amd64 and arm64; + - computes `SHA256SUMS`, updates the public releases repository and creates (or updates) the GitHub release there. +4. To publish an existing tag again (for example after adding a secret), run *Release* manually with the tag. + +### Secrets and variables + +| Name | Kind | Purpose | +|---|---|---| +| `RELEASES_TOKEN` | secret | Fine-grained PAT with *Contents: read and write* on the public releases repository. Without it the release is created in this (private) repository, which clients cannot see. | +| `RELEASES_REPO` | variable (optional) | `owner/name` of the public repository; default `/titano-releases`. Builds embed it through the `TitanoReleasesRepo` MSBuild property (default `alby96/titano-releases`), so keep the two in line. | +| `SIGN_PFX_BASE64` | secret (optional) | Base64 of the Authenticode code-signing certificate (`.pfx`). Without it builds are unsigned. | +| `SIGN_PASSWORD` | secret (optional) | Password of that `.pfx`. | + +`GITHUB_TOKEN` pushes the image to GHCR; no other secret is used. + +### Bootstrapping the public releases repository + +1. Create a **public** repository named `titano-releases` under the same owner (or set `RELEASES_REPO`), with a + README so it has a default branch. Enable *Issues* and *Private vulnerability reporting* (Settings › Security). +2. Create the fine-grained PAT (repository access: only that repository; permissions: Contents read/write) and store + it as the `RELEASES_TOKEN` secret of this repository. +3. Push a pre-release tag (`v0.1.0-rc.1`). The release job copies `deploy/releases-repo/` (README, EULA draft, + SECURITY policy, issue forms), the Unraid template and its icon, the compose file and `.env.example`, the dark + screenshots, `i18n/` and the latest `SHA256SUMS` into the public repository, rewriting links to its name, then + creates the release with the installers. +4. After the first image push, open the `titano-server` package on GHCR › *Package settings* and set its + visibility to **Public** (GitHub offers no API for this; the release summary reminds you). +5. Have the EULA draft (`deploy/releases-repo/EULA.md`) reviewed before the first stable release. +6. In Unraid, add `https://github.com//titano-releases` under Docker › Template repositories to validate the + template. + +## Repository layout + +``` +src/ Titano.Core, Core.Windows, Data, Protocol, Server, Ui, App, Uploader, Setup, Tools +tests/ Titano.Testing and one test program per area; fixtures/ (manifest of CC0 samples) +i18n/ /.resx string catalogs, GLOSSARY.md +deploy/ docker/ (Dockerfile, compose), unraid/ (template), releases-repo/ (public repository content) +branding/ SVG masters of the icons +docs/ BRIEF, ARCHITECTURE, DECISIONS, STATUS, CHANGELOG, ui/ +``` diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 8a51336d..e790a519 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,83 +1,182 @@ # Titano — Architecture and code conventions -Companion to `docs/BRIEF.md` (scope, behaviour) and `docs/ui/DESIGN.md` (looks). This file says -*where code goes* and *how it is written*. +Companion to `docs/BRIEF.md` (scope, behaviour) and `docs/ui/DESIGN.md` (looks). This file says *where code goes*, +*how data moves* and *how it is written*. Decisions behind the shapes described here are in `docs/DECISIONS.md`. ## 1. Projects | Project | TFM | Purpose | |---|---|---| -| `src/Titano.Core` | net10.0 | Engine shared by every program: metadata, codecs, imaging, analysis, motion, render, video containers, archive model, naming, import planning, localization runtime, brand mark | -| `src/Titano.Core.Windows` | net10.0-windows | WIC decoder, Media Foundation encoder, DXGI, device APIs, shell helpers (P/Invoke/COM only) | -| `src/Titano.Data` | net10.0 | `IDatabase`, SQLite binding (winsqlite3 on Windows, system libsqlite3 elsewhere for tests/dev), PostgreSQL wire client, migrations, repositories | -| `src/Titano.Protocol` | net10.0 | API DTOs, source-generated JSON contexts, upload/sync/event schemas | -| `src/Titano.Server` | net10.0 (Web SDK) | ASP.NET Core minimal API, auth, jobs, storage, admin web panel (embedded) | -| `src/Titano.Ui` | net10.0-windows (WPF) | Theme (DESIGN tokens), icons, custom controls, `Loc` markup extension, MVVM helpers | +| `src/Titano.Core` | net10.0 | Engine shared by every program: metadata, codecs, imaging, analysis, advisor, motion, render, video containers and managed encoders, archive format, naming, import planning, packaging, localization runtime, diagnostics, brand mark | +| `src/Titano.Core.Windows` | net10.0-windows | WIC decoders and codec inventory, Media Foundation encoders, DXGI adapters, removable devices, DPAPI, shell launcher, window chrome, power (P/Invoke/COM only) | +| `src/Titano.Data` | net10.0 | `IDatabase`, SQLite binding, PostgreSQL wire client, migrations, repositories, the job engine, the archive engine (`Archiving`) and the analysis/render engine (`Processing`) | +| `src/Titano.Protocol` | net10.0 | API v1 DTOs, source-generated JSON context, typed `TitanoClient` with certificate pinning and bandwidth limiting | +| `src/Titano.Server` | net10.0 (Web SDK) | ASP.NET Core minimal API, auth, uploads, sync, jobs, render nodes, events, embedded admin panel, Windows service host | +| `src/Titano.Ui` | net10.0-windows (WPF) | Theme (DESIGN tokens), icons, standard and custom controls, visuals (charts, filmstrip, preview, curve editor, virtualizing grid), `Loc` markup extension, MVVM helpers | | `src/Titano.App` | WPF exe `Titano.exe` | Desktop client | -| `src/Titano.Uploader` | WPF exe | Uploader | -| `src/Titano.Setup` | WPF exe | Installer/uninstaller | -| `src/Titano.Tools` | net10.0 exe | CLI: make-icon, i18n, xaml-lint, package-guard, selftest, diagnose, fixtures, changelog | -| `tests/Titano.Testing` | net10.0 | In-house test framework (`[Test]`, `[Theory]`+`[Case]`, `Assert`, `TestContext`) | +| `src/Titano.Uploader` | WPF exe | Uploader (tray, watch folders, resumable queue) | +| `src/Titano.Setup` | WPF exe | In-house installer/uninstaller (TPK1 payload) | +| `src/Titano.Tools` | net10.0 exe | CLI: `make-icon`, `i18n` (check, export, missing, import), `xaml-lint`, `package-guard`, `changelog`, `pack-installer`, `selftest`, `diagnose`, `fixtures` | +| `tests/Titano.Testing` | net10.0 | In-house test framework (`[Test]`, `[Theory]`+`[Case]`, `[Category]`, `[RequiresEnvironment]`, `[WindowsOnly]`, `[Serial]`, `Assert`, `TestContext`, JUnit) | | `tests/Titano.*.Tests` | exe | One test program per area; `dotnet run --project tests/X -- --junit out.xml` | -## 2. Namespaces in Titano.Core +Dependencies point one way: `Core` ← `Core.Windows`, `Data`, `Protocol` ← `Server`; `Ui` ← `App`, `Uploader`, `Setup`. +`Core`, `Data`, `Protocol` and `Server` never reference Windows-only code (Linux CI builds and tests them). + +## 2. Module map + +### 2.1 Titano.Core namespaces | Folder / namespace | Content | |---|---| -| `Imaging` | `ImageBuffer` (planar float, linear light), `ImageFileFormat`, `IImageDecoderProvider`, `ImageDecoder`, colour/geometry helpers, `FrameWindow` | -| `Metadata` (+ `Metadata.Tiff`, `Metadata.Bmff`, …) | Signature sniffing, TIFF/IFD (`TiffReader`), Exif/GPS/XMP/MakerNotes, ISO-BMFF, RIFF, RAF, PNG chunks, `MetadataReader`, `ExifWriter` | -| `Codecs` (`Codecs.Jpeg`, `Codecs.Png`, `Codecs.Tiff`, `Codecs.Dng`, `Codecs.Gif`, `Codecs.Ico`) | Pure managed image codecs | -| `Video` | `Mp4Muxer`, box writer, Annex-B/OBU parsing, MP4 inspector, AVI-MJPEG writer, managed encoder provider, encoder contracts | -| `Analysis` | Luminance statistics, deflicker, regions, holy grail, white balance, colour temperature, light/deep analysis | -| `Motion` | FFT, phase correlation, pyramids, optical flow, motion blur, stabilisation, virtual camera, interpolation, stacking | -| `Math` (namespace `Titano.Core.Mathematics`, so it never shadows `System.Math`) | Splines (Fritsch–Carlson, Catmull-Rom, Bézier), robust local regression, statistics, solar position, 3×3 colour matrices | -| `Render` | Render plan, pipeline, export presets | -| `Advisor` | Recommendation rules with localisable reasons, sequence presets | -| `Archive` | Archive layout, sidecars (`sequence.json`, `project.json`), scanner, rebuild, verify, duplicates, orphans, adopt | +| `Imaging` (+ `Decoders`, `Geometry`, `Thumbnails`) | `ImageBuffer` (planar float, linear light), `IImageDecoderProvider`, `DecoderFactory` (decode tiers: native → embedded preview), `ManagedDecoderProvider`, `FrameWindow` (read-ahead, memory budget, float32 spill), resampler, thumbnail cache | +| `Metadata` (+ `Tiff`, `Jpeg`, `Bmff`, `Raf`, `Riff`, `Png`, `Xmp`, `MakerNotes`) | Signature sniffing, Exif/GPS/XMP/MakerNotes, ISO-BMFF (HEIF, CR3), `MetadataReader`, `MetadataBuilder` (all fallback chains), `ExifWriter` | +| `Codecs` (`Jpeg`, `Png`, `Tiff`, `Dng`, `Raw`, `Gif`, `Apng`, `Ico`) | Pure managed codecs; native RAW development (DNG, CR2, NEF, ARW, PEF, RAF) | +| `Analysis` (+ `Deep`) | Luminance statistics, deflicker, regions, holy grail, white balance, colour temperature, light analysis, streaming deep analysis with checkpoints | +| `Advice` | `Advisor`: recommendations with localisable reasons, apply/optimise, sequence presets | +| `Motion` | FFT, phase correlation, pyramids, optical flow, motion blur, stabiliser, virtual camera, interpolation, stacking | +| `Math` (namespace `Titano.Core.Mathematics`) | Splines, LOWESS, statistics, solar position, 3×3 colour matrices | +| `Project` | `ProjectSettings` (Auto/manual values), export presets | +| `Render` | Render plan (timing modes), preparation pass, geometry, `RenderPipeline`, self-test suite | +| `Export` | Preset resolution, encoder selection with fallbacks, output naming | +| `Video` (+ `Mp4`, `Avi`, `Bitstream`, `Yuv`, `Managed`) | `IVideoEncoderProvider`, MP4 muxer and inspector, AVI-MJPEG, NV12/P010, managed encoders (MJPEG-AVI, GIF, APNG, image sequences) | +| `Archiving` | Archive layout and paths, sidecars (`sequence.json`, `project.json`), scanner, index snapshot, leases, thumbnail packs, verify, adopt, rename plans | +| `Import` | Card scan, session detection, import plan, hash-verified copy, card eraser (re-hash before delete) | | `Naming` | Folder/file templates and tokens | -| `Import` | Card/folder scan, session detection, import plan, hash-verified copy | -| `Model` | Domain entities (§5) shared by Data, Server and clients | -| `Localization`, `Diagnostics`, `IO`, `Hashing`, `Branding` | Infrastructure | +| `Model` | Domain entities and enums (§5) shared by Data, Server and clients | +| `Packaging` | TPK1 installer payload writer/reader, staged install transaction | +| `Diagnostics` | `Log`, file sink, synthetic sequences, `SecretScrubber`, `DiagnosticsArchive` (scrubbed diagnostics zip) | +| `Localization`, `IO`, `Hashing`, `Branding` | `Loc`/plural rules, `AtomicFile`, CRC-32/SHA helpers, brand mark geometry | -## 3. Conventions +### 2.2 Titano.Data + +| Folder | Content | +|---|---| +| `Sqlite`, `Postgres` | SQLite P/Invoke binding (winsqlite3 / libsqlite3), in-house PostgreSQL wire protocol v3 client with pool and LISTEN | +| `Migrations/{sqlite,postgres}/NNNN_name.sql` | Embedded, applied in one transaction at open (`MigrationRunner`) | +| `Repositories` | `TitanoDatabase` facade: archives, sequences, frames, tags, renders, jobs, cards, users, tokens, uploads, render nodes, settings, audit | +| `Jobs` | `JobEngine`, `IJobHandler`, `IJobHandlerProvider`, `JobContext` (progress, checkpoints, follow-up jobs) | +| `Archiving` | `ArchiveService` (create/adopt/open), `ArchiveIndexer` (scan, rebuild), `ArchiveIngest` (uploads and imports), `ImportService`, `ArchiveVerifier`, `ArchiveImages`, `ArchiveMaintenance`, `MassRenamer`, job handlers | +| `Processing` | `ProcessingService`, `SequenceAnalyser`, `SequenceRender`, `ProjectStore`, `RenderNodeClient`, job handlers | + +### 2.3 Hosts + +| Host | Composition root | What it runs | +|---|---|---| +| Titano (desktop) | `App/Infrastructure/AppServices` | SQLite index in `%LOCALAPPDATA%\Titano` (or `.\data` when portable); `ArchiveService` with WIC first, then managed decoders; `ProcessingService` with Media Foundation, then managed encoders; one `JobEngine` with the handlers of both; thumbnails; `ServerSession`; the render node while opted in and signed in; update check and download (`About/UpdateCheck`, `About/UpdateDownload`); diagnostics bundle | +| Titano Server | `Server/Hosting/ServerHost.Build` | Options from `server.json` + `TITANO_*`; PostgreSQL (or SQLite on Windows); `ArchiveModule` then `ProcessingModule` register job handler providers; `JobEngineHost` (never claims renders); `RenderNodeService`/`RenderNodeMonitor`; `UploadService`; `EventHub` (WebSocket); embedded admin panel; LAN discovery beacon; Windows service host | +| Titano Uploader | `Uploader/App.xaml.cs` | `TitanoClient`, persisted queue, watch folders; no database | +| Titano Setup | `Setup/Program` | TPK1 payload extraction, registry, shortcuts, silent mode, uninstaller | + +### 2.4 Job kinds and their handlers + +| `JobKind` | Handler (Titano.Data) | Desktop | Server | +|---|---|---|---| +| `Scan`, `Rebuild` | `Archiving/Jobs/ScanJobHandler` (via `ArchiveService.CreateJobHandlers`) | yes | yes | +| `Verify` | `Archiving/Jobs/VerifyJobHandler` | yes | yes | +| `Import` | `Archiving/Jobs/ImportJobHandler` | yes | yes | +| `Thumbs`, `Proxies` | `Archiving/Jobs/ImageJobHandler` | yes | yes | +| `Maintenance` | `Archiving/Jobs/MaintenanceJobHandler` | yes | yes | +| `LightAnalyse`, `DeepAnalyse` | `Processing/Jobs/AnalyseJobHandler` (via `ProcessingService.CreateJobHandlers`) | yes | yes | +| `Render` | `Processing/Jobs/RenderJobHandler` | yes | no: render nodes claim them (`Server/Jobs/RenderNodeService`) | +| `UploadCommit`, `Duplicates`, `Orphans`, `Adopt`, `Move` | none: done synchronously (upload completion, `ArchiveService` queries and actions) | — | — | + +A job engine claims only kinds it has handlers for. Light analysis is queued by the engine (`JobContext` follow-up) +after an import or a scan that indexed something; screens never queue it again. + +## 3. Data flow + +``` + card / folder ──► Core.Import (plan, sessions) ──► ImportService ─┐ + Uploader / client ──► POST /uploads, chunks ──► UploadService ────┤ hash verified + ▼ + ArchiveIngest: temp → fsync → rename into originals/, + sidecar (sequence.json, journalled) → index rows + │ + ┌────────────────────────────────────────────────────┼─────────────────────────┐ + ▼ ▼ ▼ + LightAnalyse (metadata) Thumbs / Proxies (thumbs.pack) events: sequenceChanged + DeepAnalyse (pixels, checkpoints) + │ + ▼ + Advice.Advisor → ProjectSettings (Auto values with reasons) → project.json + │ + ▼ + Render job: RenderPlan → RenderPipeline (FrameWindow, motion, deflicker…) → IVideoEncoderProvider + │ desktop: Media Foundation → managed; server: render node (GPU PC) + ▼ + output file (temp → rename) + renders row +``` + +The sidecars are the truth; the index is derived and can always be rebuilt (`ScanMode.Rebuild`, using the index +snapshot when fresh). Writers coordinate through lease files, so a desktop and a server may share an archive. + +## 4. Extension points + +| To add | Implement / register | +|---|---| +| An image format or decoder | `IImageDecoderProvider` (managed in `Core/Imaging/Decoders` or `Codecs`, WIC in `Core.Windows`); chained by `DecoderFactory` | +| A video encoder | `IVideoEncoderProvider` (`Core/Video/Managed`, `Core.Windows/MediaFoundation`); list it in the host's `ProcessingOptions.Encoders` | +| A background operation | a `JobKind`, an `IJobHandler` (in `Data/Archiving/Jobs` or `Data/Processing/Jobs`) returned by the service's `CreateJobHandlers`; the job engine, Jobs screen, admin and API pick it up | +| A server feature | a static `XxxEndpoints.Map(api)` in `Server/Api` with `.Requires(Permissions…)`, called from `ServerHost.ConfigurePipeline`; DTOs in `Titano.Protocol` with the JSON context | +| A server storage backend | `IArchiveIngest`, `IFrameAssetLocator` through the `ServerHost.Build` registration callback | +| A schema change | `Data/Migrations/{sqlite,postgres}/NNNN_topic.sql` (never edit an applied one) | +| An advisor rule | `Core/Advice` with a localised reason key in `Advisor` | +| A UI string | `i18n/en/.resx` + `i18n/it/.it.resx` (other languages via `i18n missing`/`import`) | +| An admin page | ES module in `Server/Admin/web/pages`, strings in the `Admin` area | +| A CLI command | `ICommand` in `Tools/Commands`, listed in `Tools/Program.cs` | + +## 5. Conventions - File-scoped namespaces, `sealed` by default, nullable on, warnings are errors. No `#pragma warning disable` except narrowly around interop with a reason. - No third-party packages, ever. If the BCL lacks it (CRC-32, Adler, JPEG, PNG decode…), write it. -- Parsers never throw on malformed input: they return partial results plus warnings. Every offset is - bounds-checked against the stream length; recursion and loops are capped. +- Never name a namespace `Archive` (it would shadow the entity): the archive code lives in `Archiving`. Inside + Titano.Core never write an unqualified `Math.`: use `System.Math` or `double.Max`. +- Parsers never throw on malformed input: they return partial results plus warnings. Every offset is bounds-checked + against the stream length; recursion and loops are capped. - Never load a whole RAW to read metadata: windowed reads through `Stream` + `Seek`. -- Hot loops: `Span`, `System.Numerics.Vector`/`Vector256` where it measurably pays, no LINQ. - Parallelise per row/tile with `Parallel.For` only when work per item is large. +- Hot loops: `Span`, `System.Numerics.Vector`/`Vector256` where it measurably pays, no LINQ. Parallelise per + row/tile with `Parallel.For` only when work per item is large. - Pixel math in linear light, `float`, Rec.709 primaries. Convert to sRGB/8-bit only at the edges. -- Every long operation takes a `CancellationToken` and reports progress through `IProgress`. -- Libraries use `ConfigureAwait(false)`. No `async void` except WPF event handlers. -- Logging through `Titano.Core.Diagnostics.Log` in Core; `ILogger` on the server. +- Every long operation takes a `CancellationToken` and reports progress through `IProgress` or the `JobContext`. +- Libraries use `ConfigureAwait(false)`. No `async void` except WPF event handlers. Heavy work and I/O never run on + the UI thread. +- Logging through `Titano.Core.Diagnostics.Log` in Core and Data; `ILogger` on the server. Anything that leaves the + machine (diagnostics bundles) passes through `SecretScrubber`. - Public types and non-obvious members get a one-line `///` summary. Comments explain *why*, not *what*. -- Disk writes that must survive crashes use `AtomicFile` (temp → fsync → rename). -- IDs are UUIDv7 (`Guid.CreateVersion7()`); timestamps UTC; capture times are camera wall-clock with an - optional offset. +- Disk writes that must survive crashes use `AtomicFile` (temp → fsync → rename); originals are never modified. +- IDs are UUIDv7 (`Guid.CreateVersion7()`); timestamps UTC; capture times are camera wall-clock with an optional + offset. +- The only OS actions the programs perform are those of BRIEF §8.6: open a URL and show a folder or file in Explorer + (`ShellActions` in the App), the format-volume dialog (`Core.Windows/Devices`). Downloaded updates are shown in + Explorer, never started. -## 4. Tests +## 6. Tests -- Framework: `Titano.Testing`. Test classes are `public sealed`; constructor may take `TestContext` - (temp directories, cancellation, log). Use `[WindowsOnly]` for OS-specific tests and - `[RequiresEnvironment("TITANO_TEST_PG")]` for PostgreSQL integration tests. +- Framework: `Titano.Testing`. Test classes are `public sealed`; constructor may take `TestContext` (temp + directories, cancellation, log). `[WindowsOnly]` for OS-specific tests, `[RequiresEnvironment("TITANO_TEST_PG")]` + for PostgreSQL integration tests. - Put tests under `tests/Titano..Tests//Tests.cs`, mirroring the source folder. -- Fixtures are synthetic and generated in code wherever possible; tiny binary fixtures may go in - `tests/fixtures/` (< 50 KB each). Larger CC0 samples are fetched by `Titano.Tools fixtures` into - `tests/fixtures/external/` (git-ignored) and tests depending on them skip when absent. +- `Titano.App.Tests` targets net10.0-windows and runs on Windows CI only: keep logic in plain classes. +- Fixtures are synthetic and generated in code wherever possible; tiny binary fixtures may go in `tests/fixtures/` + (< 50 KB each). CC0 RAW samples are fetched by `Titano.Tools fixtures` into `tests/fixtures/external/` (not + committed); tests depending on them skip when absent and CI shows a warning per missing sample. - Fuzz tests: truncate and corrupt valid inputs at many offsets; assert no exception escapes. +- Performance gates (`tests/Titano.Data.Tests/Perf`, `[Category("perf")]`, run only with `TITANO_PERF=1`): the + 500,000-photo library (Library open < 1 s, filters < 200 ms) on SQLite and PostgreSQL, and the 100,000-frame + rebuild from sidecars. `.github/workflows/perf.yml` runs them weekly against PostgreSQL 17. -## 5. Localization +## 7. Localization -- Keys: `Area.Group.Name`; area = file name. Areas: Common, Shell, Library, Editor, Import, Archive, - Jobs, Server, Settings, Tooltips, Errors, Advisor, Admin, Uploader, Setup. -- `en` is the source (`i18n/en/.resx`), `it` is mandatory (`i18n/it/.it.resx`). +- Keys: `Area.Group.Name`; area = file name. Areas: About, Admin, Advisor, Archive, Common, Editor, Errors, Export, + Import, Jobs, Library, Server, Settings, Setup, Shell, Tooltips, Uploader. +- `en` is the source (`i18n/en/.resx`), `it` is mandatory (`i18n/it/.it.resx`); other languages are + machine-translated through `i18n missing` / `i18n import` with `i18n/GLOSSARY.md` and a per-language glossary. - Labels ≤ 30 chars, tooltips/errors/advisor reasons ≤ 200 chars (one or two sentences). - Plurals: `Key.one` / `Key.few` / `Key.many` / `Key.other`, resolved by `Loc.Plural("Key", n)`. -## 6. Build and checks +## 8. Build, checks and CI ``` dotnet build Titano.slnx -c Release @@ -85,4 +184,12 @@ dotnet run --project tests/Titano.Core.Tests -c Release -- --junit out/core.xml dotnet run --project src/Titano.Tools -- package-guard dotnet run --project src/Titano.Tools -- i18n check dotnet run --project src/Titano.Tools -- xaml-lint +dotnet format whitespace Titano.slnx --verify-no-changes ``` + +| Workflow | Trigger | Does | +|---|---|---| +| `ci.yml` | push to `main`, pull requests | Windows + Linux build and tests, guards, i18n, XAML lint, format, fixtures (warnings when missing), screenshots, PostgreSQL 16/17, Docker smoke test | +| `release.yml` | tag `v*` (or manual re-publish) | Tests, self-contained publish, optional signing, installers, portable and server zips, SHA256SUMS, release notes, multi-arch GHCR image, public releases repository update and GitHub release | +| `docker.yml` | nightly | `ghcr.io//titano-server:nightly` | +| `perf.yml` | weekly, manual | Performance gates (SQLite, PostgreSQL 17) with timings in the job summary | diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 73358d86..e7898c29 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -1,17 +1,82 @@ # Changelog -All notable changes. Format: Keep a Changelog; versions follow SemVer; commits follow Conventional Commits. +All notable changes. Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow SemVer; +commits follow Conventional Commits (release notes are generated from them by `Titano.Tools changelog`). ## [Unreleased] ### Added -- Repository foundation: solution, build rules (warnings as errors, zero-package guard), .NET 10 SDK pin. -- `Titano.Testing`: in-house test framework and runner with JUnit XML output and GitHub annotations. -- Localization runtime (`Loc`, CLDR plural rules, 38 shipped languages), `en` + `it` catalogs, `i18n check/export/missing/import`, translation glossary. -- Brand mark geometry, SDF rasteriser, PNG encoder, ICO writer and `make-icon` (program icons generated at build time). -- Core contracts: image buffers and decoder providers, video encoder providers, domain model, TIFF/IFD reader, sRGB helpers, atomic file writes, logging. -- `Titano.Ui` design system: DESIGN.md tokens (dark/light), 100+ stroke icons, styles for every standard control, custom chrome window with snap layouts, navigation rail, job bar, Auto/manual slider, chips, status glyphs, toasts, empty states. -- Titano desktop shell: rail, job bar, status strip, sections, Settings (language, theme) and About; `--capture` screenshot mode. -- Titano Server skeleton: configuration from environment/`server.json`, self-signed TLS with fingerprint, `/health`, `/api/v1/server/info`, `/api/v1/i18n/{culture}`, embedded admin panel (login, dashboard). + +#### Foundation +- Solution and build rules: warnings as errors, nullable, .NET 10 SDK pin, zero-package guard (build target and + `Titano.Tools package-guard`). +- `Titano.Testing`: in-house test framework and runner (theories, categories, timeouts, environment and Windows + gates, serial classes, parallel runs, JUnit XML, GitHub annotations). +- Localization runtime (`Loc`, CLDR plural rules with few/many forms, runtime culture switch, RTL), `en` + `it` + catalogs, machine translations with per-language glossaries, `i18n check/export/missing/import`. +- Brand mark geometry, SDF rasteriser, PNG encoder, ICO writer and `make-icon` (icons generated at build time). + +#### Engine (Titano.Core) +- Metadata: signature sniffing, TIFF/Exif/GPS/XMP/MakerNotes, JPEG segments, ISO-BMFF (HEIF, CR3), RAF, PNG, + WebP and RIFF parsers with fuzz-tested partial results; `MetadataBuilder` fallback chains; `ExifWriter`. +- Codecs: JPEG (baseline, progressive, lossless; libjpeg-exact integer IDCT), PNG, TIFF, linear DNG writer, GIF and + APNG encoders, ICO; native RAW development for DNG (opcodes, AHD demosaic, DNG colour), CR2, NEF, ARW, PEF, RAF. +- Imaging: planar linear-light buffers, decoder providers and decode tiers, separable resampler, frame window with + read-ahead, memory budget and float32 spill, `thumbs.pack`, thumbnail cache, proxy builder. +- Analysis and advisor: frame statistics, colour temperature, deflicker (global and regional), holy-grail step + redistribution, white-balance smoothing, light analysis with findings, streaming deep analysis with checkpoints; + `Advisor` recommendations with localised reasons, apply/optimise, sequence presets. +- Motion: FFT and phase correlation, pyramidal Lucas–Kanade flow, motion blur, nine-patch stabiliser, virtual + camera, frame interpolation, stacking and star trails. +- Render and export: render plan (constant, adaptive, interpolated, remapped timing), preparation pass, geometry, + pipeline, self-test suite (`Titano.Tools selftest`); preset resolution, encoder selection with hardware preference + and fallbacks, output naming. +- Video: streaming MP4 muxer (H.264/HEVC/AV1) with inspector, AVI-MJPEG with OpenDML indexes, NV12/P010 + conversion, level tables, managed encoder provider (MJPEG-AVI, image sequences, GIF, APNG). +- Archive format: layout and naming templates, sidecars, leases, index snapshot, scanner, verify, adopt, orphans, + rename plans; card import planning, sessions, hash-verified copy and verified erase; TPK1 installer payload. +- Diagnostics: `SecretScrubber` and `DiagnosticsArchive` for diagnostics bundles that never carry passwords, + tokens, keys or photos. + +#### Data (Titano.Data) +- SQLite binding (winsqlite3 / libsqlite3) and in-house PostgreSQL wire protocol v3 client (pool, prepared + statements, COPY, LISTEN, cancellation). +- Migrations, repositories and the `TitanoDatabase` facade; one persistent job engine with checkpoints, + pause/resume, priorities and crash recovery for client and server. +- Archive engine: create/adopt/open, incremental and full scans, rebuild from sidecars, ingest of uploads and card + imports, verify, thumbnails and proxies, maintenance, mass rename, job handlers. +- Analysis and render engine: light/deep analysis and render job handlers, render node client. + +#### Programs +- Titano desktop: shell with rail, job bar and toasts; Welcome, Library (virtualised grid and list, filters, sort, + hover scrubbing), Jobs, Server (trust-on-first-use sign-in, DPAPI token), Settings (general, archive, rendering + with encoder inventory and self-test, storage, server, updates, advanced) and About; `--capture` screenshots. +- Update check against the public releases repository and **Download**: the setup is fetched into + `%TEMP%\Titano\updates`, verified against the release's `SHA256SUMS` and shown in Explorer. +- Diagnostics bundle (Settings › Advanced, About): environment, codec and encoder inventory, scrubbed settings, + recent failed jobs and logs. +- Titano Server: configuration from `server.json` and `TITANO_*`, TLS (auto, PEM, proxy), API v1 (auth, users, + sequences, frames, uploads, sync, jobs, render nodes, events over WebSocket, archive), admin panel (dashboard, + users, archive, jobs, settings, logs, about), audit log, LAN discovery, Windows service. +- Server diagnostics: `GET /api/v1/server/diagnostics` (ManageArchive, audited) and a *Diagnostics* button in admin › + Logs. +- Titano Uploader: tray program with resumable persisted queue, session grouping, watch folders, bandwidth limit, + verified-only source deletion. +- Titano Setup: one-screen installer, per-user or all-users, components, shortcuts, file types, upgrade in place, + silent mode, uninstaller that keeps user data unless asked. + +#### Distribution and CI - Docker image (multi-arch, PUID/PGID, health check), compose file with PostgreSQL 17, Unraid template. -- CI (Windows + Linux build/test, PostgreSQL 16/17, Docker smoke test, screenshots, Windows program artifacts), release and nightly image workflows. +- CI: Windows + Linux build and tests, PostgreSQL 16/17, Docker smoke test, screenshots branch, i18n, XAML lint, + format and package guards. +- Release workflow: tests, self-contained publish, optional Authenticode signing, installers, portable and server + zips, `SHA256SUMS`, release notes from Conventional Commits, multi-arch GHCR image, and the public releases + repository (README, EULA draft, SECURITY policy, issue forms, Unraid template, compose file, screenshots, `i18n/`). +- Performance gates (`perf.yml`, weekly): 500,000-photo library on SQLite and PostgreSQL 17, 100,000-frame rebuild + from sidecars. + +### Fixed +- Release workflow: signing and the public-repository publish never ran (step conditions cannot read the step's own + secrets); the publish job lacked the .NET SDK for `make-icon`; re-publishing an existing tag failed; the portable + zip lacked `portable.txt`. +- CI fixtures step reported missing RAW samples as a silent pass; it now emits a warning per missing sample. diff --git a/docs/STATUS.md b/docs/STATUS.md index 8ae31a7c..d9464e4b 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -4,15 +4,15 @@ Phase plan: docs/BRIEF.md §22. Windows behaviour is verified on CI (`windows-la | Phase | State | |---|---| -| 0 Bootstrap | Done: solution, rules, Titano.Testing, zero-package guard, i18n (en, it; 36 cultures scaffolded), WPF shell, server skeleton, Docker, icons, CI. | -| 1 Metadata and imaging | Done: metadata parsers, managed codecs (JPEG, PNG, TIFF, GIF, APNG, ICO, linear DNG writer), WIC provider, colour/resampling. RAW/DNG decode pipeline in progress. | -| 2 Data layer and archive | Data layer done (SQLite binding, PostgreSQL wire client, migrations, repositories, job engine). Archive layout, sidecars, scan, rebuild, verify, duplicates, orphans, adopt: in progress. | -| 3 Library and Archive manager | Visual controls done (virtualised card grid, charts, filmstrip, preview, curve editor). Library, Jobs, Settings, Welcome, Server and About screens: in progress. Archive manager screen: next. | -| 4 Card import | Engine in progress with the archive workstream; screen next. | -| 5 Analysis advisor | In progress. | -| 6 Engine, encoding, editor | Video containers and encoders done (MP4, AVI, GIF/APNG, Media Foundation H.264/HEVC/AV1); render pipeline in progress; editor screen next. | -| 7 Server, sync, uploader | Uploader done. Full server (API, auth, uploads, sync, jobs, render nodes, events, admin panel, Windows service) in progress. | -| 8 Release readiness | Installer, release/docker workflows and Unraid template in place; translations, accessibility and performance passes pending. | +| 0 Bootstrap | Done: solution, rules, Titano.Testing, zero-package guard, i18n, WPF shell, server skeleton, Docker, icons, CI. | +| 1 Metadata and imaging | Done: metadata parsers, managed codecs, native RAW development (DNG, CR2, NEF, ARW, PEF, RAF), WIC provider and codec inventory, decode tiers, thumbnails and proxies, `diagnose`. Real-camera fixtures wait for verified raw.pixls.us URLs (CI warns per missing sample). | +| 2 Data layer and archive | Done: SQLite binding, PostgreSQL wire client, migrations, repositories, job engine, archive layout, sidecars, scan, rebuild, verify, duplicates, orphans, adopt, mass rename. Gate measured: a 100,000-frame archive rebuilds from sidecars into the identical index in 1.1 s (budget 120 s). | +| 3 Library and Archive manager | Library, Jobs, Settings, Welcome, Server and About screens done; Archive manager screen in progress. Gate measured on 500,000 photos: Library open 313 ms cold on SQLite, 162 ms on PostgreSQL (budget 1 s); every filter, search, sort and next page ≤ 8.4 ms (budget 200 ms). | +| 4 Card import | Engine done (plan, sessions, hash-verified copy, verified erase, import jobs); screen in progress. | +| 5 Analysis advisor | Engine done (light and deep analysis, advisor, presets, analysis jobs); editor integration in progress. | +| 6 Engine, encoding, editor | Render pipeline, MP4/AVI/GIF/APNG/image sequences, Media Foundation H.264/HEVC/AV1 done; sequence editor in progress. | +| 7 Server, sync, uploader | Server (API, auth, uploads, jobs, render nodes, events, admin panel, Windows service) and Uploader done; client sync in progress. | +| 8 Release readiness | Release pipeline end to end (installers, portable and server zips, SHA256SUMS, release notes, optional signing, multi-arch GHCR image, public releases repository content), update check with verified download, scrubbed diagnostics bundles (desktop and server), performance gates, machine translations. Pending: remaining languages, RTL and accessibility passes, a tagged pre-release published from CI, install/upgrade/uninstall test on Windows, Unraid template validation on a real server. | | 9 Monetization | Not started: requires explicit approval (§22). | ## How to try it @@ -23,4 +23,18 @@ dotnet run --project src/Titano.Server # https://localhost:8443/admin src/Titano.App/bin/Release/net10.0-windows/Titano.exe # Windows src/Titano.Uploader/bin/Release/net10.0-windows/Titano.Uploader.exe # Windows docker compose -f deploy/docker/docker-compose.yml up -d # server + PostgreSQL 17 +TITANO_PERF=1 dotnet run --project tests/Titano.Data.Tests -c Release -- --filter category:perf --verbose # performance gates ``` + +- Diagnostics: Settings › Advanced › *Export bundle* (desktop) or admin panel › Logs › *Diagnostics* (server, + `GET /api/v1/server/diagnostics`). Open the zip: logs, reports, settings and failed jobs, no passwords or tokens. +- Update download: About › *Check for updates*, then *Download* when a newer release exists; the verified setup + appears in Explorer. + +## Next steps (Phase 8 gate) + +1. Create the public repository (`/titano-releases`), add the `RELEASES_TOKEN` secret, push a `v0.x.y-rc.1` + tag and check the release, the public repository content and the GHCR image (set it public once). +2. On Windows: install, upgrade over the previous version, uninstall (with and without user data), silent mode. +3. Unraid: add the template repository, install with the official PostgreSQL template, reach the WebUI. +4. Remaining languages, RTL pass (`ar`, `he`, `fa`, `ur`), accessibility pass.