fix(admin): revalidate panel files, count bulk jobs, refresh every loaded row
The phase 3 packages land together, so the panel's pieces must stay in step:
- Admin files are served with Cache-Control: no-cache and their content
ETag (If-None-Match lists, W/ tags and * honoured). With max-age=3600 a
browser that loaded the panel before an upgrade could run new modules
next to cached old ones for up to an hour; now it revalidates each file
and only what the upgrade changed is sent again.
- The Library's Upload and a sequence's Add photos open upload.js, which
now ships with the panel: the test requires the module (no 404), the
dynamic import and the openUpload({ api, sequence, onDone }) calls.
- Bulk jobs over more than 1000 sequences queue one job per 1000; the
toast now says how many were queued, also when a later chunk failed.
- The live refresh re-reads every loaded row, a thousand per request,
instead of dropping rows beyond the first 1000 back to Load more.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
0e52bf0e1b
commit
6ea611d949
6 files changed
+104
-24
No files matched your search
@@ -88,7 +88,7 @@ Dependencies point one way: `Core` ← `Core.Windows`, `Data`, `Protocol` ← `S
|
||||
| `Server/Api`, `Server/Events` | Endpoint groups (`XxxEndpoints.Map`; `ScheduleEndpoints`: `/api/v1/schedules`; `LibraryEndpoints`: the Library filters' facets, sequence thumbnail, photo delete, bulk delete/retag/jobs; `TrashEndpoints`: list, restore, empty), API errors, the WebSocket `EventHub` |
|
||||
| `Server/Security` | Accounts, password hashing, tokens, session cookies, login guard, access rules, audit log |
|
||||
| `Server/Storage`, `Server/Jobs` | Archive and processing modules, uploads (`UploadMaintenance`: recovery at start, and the sweep that is the `uploads` clean-up step), render store; `JobEngineHost`, `SchedulerHost` (the `Scheduler` on the engine, started after it; archive.json read every 10 minutes whatever the scan schedule does; every schedule change and run published as a `scheduleChanged` event) and `ServerScheduleHost` (time zone, archive refreshed from archive.json, system schedules), render node service, monitor and availability |
|
||||
| `Server/Hosting`, `Server/Admin/web` | `ServerHost`, `ServerLogging` (console and file levels the logging filters read at every line, the file sink's limits, the engine's level; applied again when Settings › Logs is saved), certificates, discovery beacon, embedded static files, Windows service (start and stop failures in the Application event log); the admin panel (ES modules, one per page; `schedules.js` is the Jobs page's Scheduled tab; the Library's views in `pages/library/`: `sequence.js` a sequence and its photo viewer, `trash.js` the archive's trash, `shared.js` cards, selection and grid keyboard; `upload.js` the upload dialog the Library opens on demand, hashing in the module worker `hash-worker.js` with the in-house incremental SHA-256 of `sha256.js`, self-tested before its first use) |
|
||||
| `Server/Hosting`, `Server/Admin/web` | `ServerHost`, `ServerLogging` (console and file levels the logging filters read at every line, the file sink's limits, the engine's level; applied again when Settings › Logs is saved), certificates, discovery beacon, embedded static files (`EmbeddedStaticFiles`: `no-cache` and a content ETag, so an upgrade never mixes old and new modules), Windows service (start and stop failures in the Application event log); the admin panel (ES modules, one per page; `schedules.js` is the Jobs page's Scheduled tab; the Library's views in `pages/library/`: `sequence.js` a sequence and its photo viewer, `trash.js` the archive's trash, `shared.js` cards, selection and grid keyboard; `upload.js` the upload dialog the Library opens on demand, hashing in the module worker `hash-worker.js` with the in-house incremental SHA-256 of `sha256.js`, self-tested before its first use) |
|
||||
|
||||
### 2.5 Hosts
|
||||
|
||||
|
||||
+2
-1
@@ -272,7 +272,8 @@
|
||||
<data name="Admin.Library.Job.Thumbnails.Tip" xml:space="preserve"><value>Queue drawing the thumbnails of the selected sequences again</value></data>
|
||||
<data name="Admin.Library.Job.Verify" xml:space="preserve"><value>Verify</value></data>
|
||||
<data name="Admin.Library.Job.Verify.Tip" xml:space="preserve"><value>Queue a verify: every original of the selected sequences is read again and its SHA-256 compared</value></data>
|
||||
<data name="Admin.Library.JobQueued" xml:space="preserve"><value>Job queued.</value></data>
|
||||
<data name="Admin.Library.JobsQueued.one" xml:space="preserve"><value>{0} job queued.</value></data>
|
||||
<data name="Admin.Library.JobsQueued.other" xml:space="preserve"><value>{0} jobs queued.</value></data>
|
||||
<data name="Admin.Library.NoMatch" xml:space="preserve"><value>No sequences match the filters</value></data>
|
||||
<data name="Admin.Library.NoTags" xml:space="preserve"><value>Type at least one tag to add or remove.</value><comment>long</comment></data>
|
||||
<data name="Admin.Library.NothingQueued" xml:space="preserve"><value>Nothing queued.</value></data>
|
||||
|
||||
@@ -272,7 +272,8 @@
|
||||
<data name="Admin.Library.Job.Thumbnails.Tip" xml:space="preserve"><value>Metti in coda il ridisegno delle miniature delle sequenze selezionate</value></data>
|
||||
<data name="Admin.Library.Job.Verify" xml:space="preserve"><value>Verifica</value></data>
|
||||
<data name="Admin.Library.Job.Verify.Tip" xml:space="preserve"><value>Metti in coda una verifica: ogni originale delle sequenze selezionate viene riletto e il suo SHA-256 confrontato</value></data>
|
||||
<data name="Admin.Library.JobQueued" xml:space="preserve"><value>Job in coda.</value></data>
|
||||
<data name="Admin.Library.JobsQueued.one" xml:space="preserve"><value>{0} job in coda.</value></data>
|
||||
<data name="Admin.Library.JobsQueued.other" xml:space="preserve"><value>{0} job in coda.</value></data>
|
||||
<data name="Admin.Library.NoMatch" xml:space="preserve"><value>Nessuna sequenza corrisponde ai filtri</value></data>
|
||||
<data name="Admin.Library.NoTags" xml:space="preserve"><value>Digita almeno un tag da aggiungere o togliere.</value><comment>long</comment></data>
|
||||
<data name="Admin.Library.NothingQueued" xml:space="preserve"><value>Nulla in coda.</value></data>
|
||||
|
||||
@@ -187,20 +187,31 @@ async function renderList(root, { api, setMeta, actions, onEvent }, params) {
|
||||
drawFoot();
|
||||
}
|
||||
|
||||
/** The loaded rows again, as many as are shown (live updates); the selection and the keyboard place stay. */
|
||||
/**
|
||||
* The loaded rows again, as many as are shown (live updates), a thousand per request; the selection and the keyboard
|
||||
* place stay.
|
||||
*/
|
||||
async function refreshVisible() {
|
||||
const mine = ++generation;
|
||||
const wanted = Math.max(rows.length, PAGE);
|
||||
try {
|
||||
const [page, f] = await Promise.all([
|
||||
api.get(listQuery(Math.min(Math.max(rows.length, PAGE), BULK_LIMIT), null)),
|
||||
api.get("/api/v1/sequences/facets").catch(() => facets),
|
||||
]);
|
||||
const facetsRead = api.get("/api/v1/sequences/facets").catch(() => facets);
|
||||
const first = await api.get(listQuery(Math.min(wanted, BULK_LIMIT), null));
|
||||
let items = first.items;
|
||||
let after = first.nextCursor;
|
||||
while (after && items.length < wanted) {
|
||||
if (mine !== generation || !root.isConnected) return;
|
||||
const page = await api.get(listQuery(Math.min(wanted - items.length, BULK_LIMIT), after));
|
||||
items = items.concat(page.items);
|
||||
after = page.nextCursor;
|
||||
}
|
||||
const f = await facetsRead;
|
||||
if (mine !== generation || !root.isConnected) return;
|
||||
total = page.total ?? total;
|
||||
total = first.total ?? total;
|
||||
facets = f;
|
||||
drawFilters();
|
||||
rows = page.items;
|
||||
cursor = page.nextCursor;
|
||||
rows = items;
|
||||
cursor = after;
|
||||
const known = new Set(rows.map((r) => r.id));
|
||||
if (!matching) selection.set([...selection.ids].filter((id) => known.has(id)));
|
||||
const focused = itemsHost.contains(document.activeElement);
|
||||
@@ -378,9 +389,11 @@ async function renderList(root, { api, setMeta, actions, onEvent }, params) {
|
||||
return [...selection.ids].map((id) => known.get(id)).filter(Boolean);
|
||||
}
|
||||
|
||||
/** One job per thousand sequences (a bulk request's limit); the toast counts them, and says those queued before a failure. */
|
||||
async function queueJobs(kind) {
|
||||
let queued = 0;
|
||||
let skipped = 0;
|
||||
let failure = null;
|
||||
try {
|
||||
for (const ids of chunks([...selection.ids])) {
|
||||
const result = await api.post("/api/v1/sequences/bulk/jobs", { ids, kind });
|
||||
@@ -388,11 +401,10 @@ async function renderList(root, { api, setMeta, actions, onEvent }, params) {
|
||||
skipped += result.skipped;
|
||||
}
|
||||
} catch (err) {
|
||||
toast(errorText(err), true);
|
||||
return;
|
||||
failure = errorText(err);
|
||||
}
|
||||
toast([queued > 0 ? t("Admin.Library.JobQueued") : t("Admin.Library.NothingQueued"), skipped > 0 ? plural("Admin.Library.Skipped", skipped) : ""]
|
||||
.filter(Boolean).join(" · "));
|
||||
const done = queued > 0 ? plural("Admin.Library.JobsQueued", queued) : failure ? "" : t("Admin.Library.NothingQueued");
|
||||
toast([done, skipped > 0 ? plural("Admin.Library.Skipped", skipped) : "", failure ?? ""].filter(Boolean).join(" · "), failure !== null);
|
||||
}
|
||||
|
||||
async function retag() {
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
using System.Collections.Frozen;
|
||||
using System.Reflection;
|
||||
using System.Security.Cryptography;
|
||||
using Microsoft.Extensions.Primitives;
|
||||
|
||||
namespace Chronos.Server.Hosting;
|
||||
|
||||
@@ -8,6 +9,12 @@ namespace Chronos.Server.Hosting;
|
||||
/// Serves the admin panel from embedded resources named <c>admin/…</c> (no build step, single-file friendly),
|
||||
/// with strong ETags, conditional GET and SPA fallback to <c>index.html</c>.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Every file, the modules included, is sent with <c>Cache-Control: no-cache</c>: the browser keeps it but asks again
|
||||
/// before each use, and an unchanged file costs a 304 without a body. The ETag is a hash of the content, so an upgrade
|
||||
/// changes the ETag of exactly the files it changed and a browser never runs new modules next to cached old ones (with
|
||||
/// <c>max-age</c> it could, until the cache expired).
|
||||
/// </remarks>
|
||||
public sealed class EmbeddedStaticFiles
|
||||
{
|
||||
private static readonly FrozenDictionary<string, string> ContentTypes = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase)
|
||||
@@ -55,9 +62,9 @@ public sealed class EmbeddedStaticFiles
|
||||
}
|
||||
var response = context.Response;
|
||||
response.Headers.ETag = file.ETag;
|
||||
response.Headers.CacheControl = path.EndsWith(".html", StringComparison.OrdinalIgnoreCase) ? "no-cache" : "public, max-age=3600";
|
||||
response.Headers.CacheControl = "no-cache";
|
||||
response.Headers.XContentTypeOptions = "nosniff";
|
||||
if (context.Request.Headers.IfNoneMatch == file.ETag)
|
||||
if (NoneMatchHolds(context.Request.Headers.IfNoneMatch, file.ETag))
|
||||
{
|
||||
response.StatusCode = StatusCodes.Status304NotModified;
|
||||
return true;
|
||||
@@ -67,4 +74,23 @@ public sealed class EmbeddedStaticFiles
|
||||
if (!HttpMethods.IsHead(context.Request.Method)) await response.Body.WriteAsync(file.Content, context.RequestAborted);
|
||||
return true;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Whether an <c>If-None-Match</c> header names <paramref name="etag"/>: <c>*</c>, or one of its entity tags under the
|
||||
/// weak comparison GET uses (RFC 9110 §13.1.2), so a tag a proxy marked <c>W/</c> still matches.
|
||||
/// </summary>
|
||||
internal static bool NoneMatchHolds(StringValues header, string etag)
|
||||
{
|
||||
foreach (var value in header)
|
||||
{
|
||||
if (string.IsNullOrEmpty(value)) continue;
|
||||
foreach (var part in value.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries))
|
||||
{
|
||||
if (part == "*") return true;
|
||||
var tag = part.StartsWith("W/", StringComparison.Ordinal) ? part[2..] : part;
|
||||
if (string.Equals(tag, etag, StringComparison.Ordinal)) return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,7 @@
|
||||
using System.Net;
|
||||
using System.Net.Http.Headers;
|
||||
using System.Net.Http.Json;
|
||||
using System.Security.Cryptography;
|
||||
using System.Security.Cryptography.X509Certificates;
|
||||
using System.Text.RegularExpressions;
|
||||
using Microsoft.AspNetCore.Hosting.Server;
|
||||
@@ -93,6 +95,44 @@ public sealed partial class SystemEndpointTests(TestContext context)
|
||||
Assert.Equal(HttpStatusCode.NotModified, cached.StatusCode);
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task AdminPanelFilesAreRevalidatedSoAnUpgradeNeverMixesOldAndNewModules()
|
||||
{
|
||||
// A browser that kept app.js from before an upgrade must not run it next to a new ui.js: every file is kept but
|
||||
// asked for again before each use, and its ETag follows its content, so only what the upgrade changed is sent.
|
||||
await using var server = await TestServer.StartAsync(context);
|
||||
foreach (var path in new[] { "/admin/", "/admin/app.js", "/admin/ui.js", "/admin/admin.css", "/admin/icons.js", "/admin/pages/library.js", "/admin/upload.js" })
|
||||
{
|
||||
using var response = await server.Client.GetAsync(path);
|
||||
Assert.Equal(HttpStatusCode.OK, response.StatusCode, path);
|
||||
var cache = Assert.NotNull(response.Headers.CacheControl);
|
||||
Assert.True(cache.NoCache && cache.MaxAge is null && !cache.Public, $"{path}: {cache}");
|
||||
var etag = Assert.NotNull(response.Headers.ETag);
|
||||
Assert.False(etag.IsWeak, path);
|
||||
var bytes = await response.Content.ReadAsByteArrayAsync();
|
||||
Assert.Equal($"\"{Convert.ToHexString(SHA256.HashData(bytes).AsSpan(0, 12))}\"", etag.Tag, path);
|
||||
|
||||
foreach (var sent in new[] { etag, new EntityTagHeaderValue(etag.Tag, isWeak: true), EntityTagHeaderValue.Any })
|
||||
{
|
||||
using var again = new HttpRequestMessage(HttpMethod.Get, path);
|
||||
again.Headers.IfNoneMatch.Add(new EntityTagHeaderValue("\"0123\""));
|
||||
again.Headers.IfNoneMatch.Add(sent);
|
||||
using var notModified = await server.Client.SendAsync(again);
|
||||
Assert.Equal(HttpStatusCode.NotModified, notModified.StatusCode, $"{path} with {sent}");
|
||||
Assert.Equal(etag, notModified.Headers.ETag, path);
|
||||
Assert.True(Assert.NotNull(notModified.Headers.CacheControl).NoCache, path);
|
||||
}
|
||||
|
||||
// A file the browser holds from another build has another ETag: the new content comes back.
|
||||
using var stale = new HttpRequestMessage(HttpMethod.Get, path);
|
||||
stale.Headers.IfNoneMatch.Add(new EntityTagHeaderValue("\"0123456789ABCDEF01234567\""));
|
||||
using var fresh = await server.Client.SendAsync(stale);
|
||||
Assert.Equal(HttpStatusCode.OK, fresh.StatusCode, path);
|
||||
var resent = await fresh.Content.ReadAsByteArrayAsync();
|
||||
Assert.BytesEqual(bytes, resent, path);
|
||||
}
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task PlainHttpFromAnotherMachineGoesToThePublicHttpsPort()
|
||||
{
|
||||
@@ -150,10 +190,10 @@ public sealed partial class SystemEndpointTests(TestContext context)
|
||||
}
|
||||
|
||||
[Test]
|
||||
public async Task AdminPanelListsTheLibraryAndLoadsTheWebUploadOnlyWhenItIsThere()
|
||||
public async Task AdminPanelListsTheLibraryAndLoadsTheWebUploadOnDemand()
|
||||
{
|
||||
// Owner review 2, R6: the Library comes right after the Dashboard. The web upload (R3) is a module of its own that the
|
||||
// Library imports on demand, so the panel works with or without it.
|
||||
// Library imports on demand (Upload, and a sequence's Add photos), so a page that never uploads never loads it.
|
||||
await using var server = await TestServer.StartAsync(context);
|
||||
var app = await server.Client.GetStringAsync("/admin/app.js");
|
||||
var top = MenuPages(app)[0];
|
||||
@@ -175,11 +215,11 @@ public sealed partial class SystemEndpointTests(TestContext context)
|
||||
foreach (Match import in StaticImport().Matches(source))
|
||||
Assert.True(Resolve(folder, import.Groups[1].Value) != "upload.js", $"{module} imports upload.js statically");
|
||||
}
|
||||
using var upload = await server.Client.GetAsync("/admin/upload.js");
|
||||
if (upload.StatusCode == HttpStatusCode.OK)
|
||||
Assert.True(Regex.IsMatch(await upload.Content.ReadAsStringAsync(), @"export (?:async )?function openUpload\("), "upload.js exports openUpload");
|
||||
else
|
||||
Assert.Equal(HttpStatusCode.NotFound, upload.StatusCode);
|
||||
// The module the dynamic import names is there (no 404 in the browser) and exports what the Library calls.
|
||||
Assert.Equal("upload.js", Resolve("pages/", "../upload.js"));
|
||||
Assert.True(Regex.IsMatch(modules["upload.js"], @"export (?:async )?function openUpload\(\{ api, sequence, onDone \}"), "upload.js exports openUpload({ api, sequence, onDone })");
|
||||
Assert.Contains("openUpload({ api, onDone: ", modules["pages/library.js"]);
|
||||
Assert.Contains("openUpload({ api, sequence: seq, onDone: ", modules["pages/library/sequence.js"]);
|
||||
}
|
||||
|
||||
/// <summary>The admin panel's modules from app.js and the menu's pages along their static imports, with their source.</summary>
|
||||
|
||||
Reference in new issue
Block a user