c5ed31bcedbc37c3e117dfe8b5c8665eca66d880
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
67a6d51cbe |
fix(archive): convert job applies exactly the confirmed plan, resumes safely and watches its space
The convert job planned every sequence again when it ran and checked only the frames version, so a verify, a repaired upload or a tier change between the dry run and the job let it replace photos the confirmed plan had listed as staying unchanged. - Each sequence of a convert job now carries the digest of the photos it converts, each with its SHA-256 (ConvertSequencePlan.Digest, also in the plan's fingerprint). Under the sequence's lease, its sidecar settled, the job plans it again and goes on only while the digest matches; it then pins those photos (convert-<job>.pins), so photos that come in or are verified later stay as they are, a retry included. Stage B checks under the lease that every pinned photo still holds the content converted instead of holding the frames version, so a sequence still receiving uploads converts what was confirmed. - A resumed job settles the journals a crash left in a sequence (a half-done replacement among them) before it plans, and reports a sequence its replacement completed as Converted, never Nothing or a partial swap. - The passes a converted or new sequence needs are queued as soon as its outcome is journaled; the journal records the statuses they follow from and whether they were queued, so a pause, yield, stop or crash no longer loses them. - Staged conversions of a blocked or failed sequence are kept for the retry, which re-verifies and takes them; maintenance keeps a convert job's staging while the job is queued, running or paused, and removes its journal and staging once the job is done or gone from the history. - A dry run counts the sequences of a queued conversion as busy, so the same conversion cannot be confirmed twice. - New sequence copies are owned by the source's owner, else by the requesting user's id (never the user name); a staged copy that vanished fails the sequence instead of leaving its photo out; the job log states how many photos stayed in the source alone. - Free space is measured where the conversion writes: the server takes the tighter of the space path's volume and the archive's own (Unraid cache), the desktop asks Windows (GetDiskFreeSpaceEx), which also measures UNC archives; the job checks again before each sequence and each photo, scaling estimates by the files staged so far, and stops before the volume fills with what it staged kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 2d7d461c33b9f257bf57793cc7cf331d0425c14e) |
||
|
|
f86a151a5d |
fix(convert): keep what the original holds and plan only what converts
Phase 4b review, codec findings. A Replace makes the converted file the photo, so a conversion must not lose what the original held, and the dry run must promise only what the job will do. - Linear DNG from a conversion is scene-referred: the pixels are stored in linear ProPhoto primaries (colours outside sRGB stay positive) and divided by the power of two that brings their peak under the white level (at most 4 stops), with BaselineExposure giving it back and ColorMatrix1/ForwardMatrix1 describing that space. Highlights a RAW develops above 1 and its wide-gamut colours now read back as written. Exports keep the clipped linear-sRGB DNG. - A conversion reads with the managed decoders first for every format they read natively (RAWs developed at float precision, as on the server) and with WIC only for what they cannot. A RAW only WIC's Raw Image Extension develops (8 bits) is refused for TIFF-16 and linear DNG and planned out as "eight_bit"; JPEG still converts it. Providers say so through IImageDecoderProvider.DevelopsRawAt8Bits. - The original's colour space is kept: a JPEG (APP2) or TIFF (tag 34675) carries its ICC profile, with the values unchanged; a linear DNG gets the values converted through a matrix/TRC profile; a gray profile converts to sRGB. Exif Adobe RGB (uncalibrated + R03) and HEIF nclx P3 get an in-house profile. A space nothing here can keep (CMYK, a lookup-table profile into a DNG, uncalibrated without R03, HDR nclx) is planned out as "color_profile". New in-house ICC parser/writer and header reader for JPEG, PNG, TIFF, HEIF/AVIF and WebP. - The planner asks every file, whatever tier was recorded (unknown before the light analysis, or by another host), with the conversion's decoders, headers only and eight at a time; a RAW the server cannot develop is "convert_on_pc" in the plan instead of failing its sequence. Photos a target cannot hold (a JPEG side above 65535, more samples than one buffer) are planned out as "too_large", and an overflow fails its photo rather than the job. - A photo that fails reads in the job log as one catalog sentence by cause (Archive.Convert.Failed.*), the file named once; the exception's English text goes to the diagnostics log only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 0ca1e91ea09946d7a55f45695ee58e1a0ff5f779) |
||
|
|
0e52929274 |
feat(archive): convert sequences to JPEG, TIFF-16 or linear DNG after import
Owner review 2, R7 asks for mass conversion after import on the web panel and the desktop, with a choice at every run between Replace and New sequence. This is the engine and the server API both UIs build on. - Core/Conversion: ConvertOptions (JPEG 4:4:4 at quality 1-100, default 95; TIFF-16 sRGB Deflate with the predictor; linear DNG, the archival choice; converted names keep the stem and take the target's extension in the old one's case). FrameConverter decodes at the native tier only, encodes with the in-house writers, writes Exif from the sidecar (capture time to the tick, offset, exposure) and the original (camera, serial, lens, modes, GPS), hashes the bytes as made, writes atomically and verifies: read back, the metadata again, a decode by the server's decoders. ExifWriter now writes the exposure program, metering mode, flash and white balance. ConvertPlanner: convertible frames, skips with reasons, size estimates per pixel, the plan's space and fingerprint. - FrameTrash.ReplaceAsync swaps the files of frames in place: same ids and indices, the old files and their XMP companions into one Replaced trash entry (records, replacement hashes, a copy of analysis.json), tombstones aliasing old content to new, proxies of the swapped frames dropped, under .chronos/replace.pending.json, which the next writer undoes or completes. Restoring a Replaced entry swaps back while every converted frame is unchanged (the converted files become a Replaced entry in turn), else the originals come back as a new sequence. - MassConverter and JobKind.Convert (19): the dry run (busy, offline, unreadable, space against ArchiveServiceOptions.FreeSpace), stage A converting into staging/convert-<job>/ in parallel with each source re-hashed and each staged file appended to journal/convert-<job>.staged, stage B per sequence all or nothing under the lease at the planned frames version: Replace, or a new sequence through the ingest with the source's tags, rating, notes and project. Resumes keep verified staged files; a sequence changed, busy or with a failed photo is left as it was and fails the job at the end, the journal kept for a retry. - POST /api/v1/archive/convert: dry run with the plan, its space line and fingerprint, or a 202 convert job; formats the server cannot read at full resolution are skipped as convert_on_pc; validated, audited (archive_converted), sequenceChanged converted/created. The server measures the space volume for the check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 809210d924cfc3f597ecdea26106fb3ff62d3762) (cherry picked from commit e13d4c0bb1bd9441933b6f3e5b6220e865a7228c) |