Prove che cadevano con il runner carico (#46) #49

Merged
Alby96 merged 1 commits from correzione/prove-instabili into main 2026-11-13 21:05:18 +01:00
34 changed files with 875 additions and 245 deletions

No files matched your search

+5
View File
@@ -323,3 +323,8 @@ Ogni scelta tecnica presa strada facendo, con data e motivo (specifica §15).
| 2026-11-13 | Rapporto di diagnostica della raccolta (#39): un registratore unico (`CollectionDiagnostics`) tiene per ogni sessione del motore, e per ogni giorno fuori dalle sessioni, solo conteggi, misure, orari, valori di elenchi chiusi e forme degli indirizzi; le tracce finite vanno in `Dati/diagnostica/sessioni-aaaa-mm-gg.jsonl` (JSON per riga, 14 giorni, una riga rovinata si salta) e «Esporta il rapporto di diagnostica» in Attività (`GET /api/engine/diagnostica`, ammesso anche con un problema di configurazione) le scarica in un file JSON. La struttura delle chiavi delle risposte sconosciute è quella **severa** (`KeyStructure.DescribeStrict`): resta solo una chiave fatta tutta di parole di un **elenco chiuso** (`StructureWords`: le parole delle chiavi dei campioni ripuliti di `tests/Febe.Tests/campioni`, dei vocabolari dei lettori incorporati e degli indirizzi del sito; alcune parole che sono anche nomi, come «max», solo dentro una chiave composta) e non uguale a un valore stringa dello stesso corpo; ogni altra chiave diventa `#`, anche un nome utente senza punto («marta_92») o un codice di post. Le forme degli indirizzi (`PathPatterns`) e la regola delle chiavi-dato (`KeyStructure.IsDataKey`) passano da `tools/Febe.Sonda` a `Febe.Core`, e la sonda le richiama. Gli eventi «formato non riconosciuto» scrivono ancora la struttura di prima: passarli a quella severa è un seguito possibile | Il file deve poter uscire dalla macchina di Alberto senza ripulitura, quindi niente testo libero per costruzione. Non nel database: nessuna migrazione, dimensione limitata dai tetti delle tracce e dai 14 giorni, cartella che si può cancellare a mano senza toccare nulla. Elenco chiuso e non regole su che aspetto ha un nome: un nome utente fatto di lettere e cifre non si distingue da una chiave, e la revisione l'ha trovato nel rapporto. Ammesso con un problema di configurazione perché l'indirizzo non risponda mai con un errore, ma lì il motore non è partito e il rapporto è vuoto |
| 2026-11-13 | Rapporto di diagnostica (#39), seconda revisione: nella struttura severa una chiave fatta di sole parole «solo composte» (`max_mark`) non passa più, e le **voci di una mappa** (chiavi sorelle i cui valori hanno la stessa forma) diventano `#` anche se fatte di parole dell'elenco, salvo le chiavi intere dei vocabolari dei lettori (`user`, `owner`). Una sessione scrive nel registro una riga «in corso» quando parte e al più ogni 5 minuti a fine ricetta; leggendo resta la riga finita, altrimenti l'ultima, e la testa del rapporto conta `fileIlleggibili` e `sessioniSenzaFine`. Il registratore ascolta anche i cambi dell'interruttore (da `EngineHost`), contando una volta sola lo stesso segnale detto anche dal rilevatore. La prova sulla riservatezza confronta le chiavi della struttura con i campi veri dei campioni e del corpo, non con `StructureWords` | Un nome fatto di parole comuni («the_artist») passava la sola lista; una mappa di nomi si riconosce dalla forma dei valori, a costo di qualche dettaglio (`first_frame` e `igtv_first_frame` diventano `#`). Senza righe intermedie una chiusura brusca cancellava proprio la sessione andata storta e la testa diceva 0; il cookie di sessione perso arriva all'interruttore senza passare dal rilevatore e il rapporto non lo diceva. Una prova che usa la funzione sotto prova come oracolo non trova i suoi errori. Resta il rischio di un solo nome di parole comuni usato come chiave, senza sorelle della stessa forma e mai ripetuto come valore |
| 2026-11-13 | Rapporto di diagnostica (#39), terza revisione: nella struttura severa una chiave con un **valore semplice** (testo, numero, vero o falso, null) resta solo se è un **campo dell'elenco chiuso** (`StructureWords.IsFieldKey`: chiavi intere dei vocabolari dei lettori e delle chiavi dei campioni ripuliti, congelate nel codice perché i campioni non viaggiano con Febe), altrimenti `#`. Le voci di una mappa si riconoscono anche quando i valori non hanno la stessa forma esatta: due valori sorelli entrambi oggetti o entrambi elenchi, di cui uno vuoto, o entrambi elenchi di valori semplici, o con almeno metà delle chiavi in comune, o con un campo d'identità in comune (`pk`, `id`, `username`…); in un elenco di oggetti le chiavi che ha solo il primo diventano `#`; fra le voci restano solo i campi dell'elenco chiuso. Una traccia finita la cui scrittura fallisce si riscrive con la riga successiva e quando si raccoglie il rapporto (al più 16 in attesa); le righe «in corso» non si riscrivono | La regola della stessa forma esatta lasciava passare i nomi in mappe di conteggi o di valori vuoti, in voci con campi facoltativi diversi (il caso normale degli utenti di Instagram) e in elenchi di mappe a una voce. Il prezzo è il dettaglio: un campo nuovo con un valore semplice diventa `#`. Senza la riscrittura, un errore momentaneo proprio sulla riga finale lasciava nel registro solo le righe «in corso» e dopo un riavvio la sessione contava fra quelle senza fine |
| 2026-11-13 | Prova di scala della libreria (§4.6, 200 000 elementi): i limiti valgono **per classe** di interrogazione, circa 5-8 volte i tempi misurati a macchina scarica: 250 ms per finestre e ricerche su indice (prima finestra, salto a una data, filtri, «è già noto», SHA-256, coda), 1500 ms per le aggregazioni sull'intera libreria o su un account (ricerche, totali, conteggi, caricamento delle impronte, elenco degli account con i conteggi, scheda dell'account). Ogni interrogazione si misura come il **migliore** di almeno tre giri dopo uno di riscaldamento, e se resta sopra il limite si riprova (al più 15 giri o 20 s, con una pausa di 100 ms) finché scende sotto. La classe gira `[DoNotParallelize]`, dopo le prove parallele; il riempimento resta fuori dalla misura e ha solo un tetto contro un blocco (20 minuti) | #46: la prova cadeva sotto carico senza nessun cambio di codice («scheda account» 1770 ms nella CI, 251,7 ms in locale). Due aggregazioni (140-200 ms a vuoto) stavano nella classe da 250 ms, con un margine di 1,2-1,8 volte; e tre giri consecutivi non superano un picco di carico, mentre le altre 11 prove parallele misuravano nello stesso momento. Il minimo è la stima giusta del costo: il carico aggiunge solo tempo, e un'interrogazione davvero più lenta resta sopra il limite in ogni giro |
| 2026-11-13 | Scheda dell'account: byte in libreria e file in coda in **una sola passata** sui file dell'account, partendo dai suoi elementi (`entry CROSS JOIN media`), non più due interrogazioni separate | #46: senza le statistiche di ANALYZE SQLite partiva dall'indice dello stato dei file e leggeva i file di tutta la libreria per la scheda di un account; sotto carico era l'interrogazione che rallentava di più (1770 ms nella CI). Ora il piano è lo stesso con e senza statistiche e proporzionale all'account, come per `LoadPerceptualHashesAsync` |
| 2026-11-13 | Avvio di Chromium (`ChromiumLaunchOptions.StartupTimeout`): **120 s** invece di 30 per aprire la porta di debug, e la prima scheda si aspetta entro la **stessa scadenza**, non più 50 tentativi da 100 ms; finisce subito se il processo si chiude, e una risposta lenta o mancata di `/json/list` vale «non ancora» come già per `/json/version`. Il ricollegamento dopo una connessione caduta usa la stessa scadenza ma smette appena il processo si chiude o il supervisore lo sostituisce, così non tiene il blocco del collegamento a un riavvio. Nessun comando CDP nuovo e nessun cambio della riga di comando (§6.1) | #46: la scadenza serve solo contro un Chromium vivo e bloccato, perché uno che si chiude fa già fallire l'avvio all'istante. Su una macchina carica (la CI era 4-6 volte più lenta del solito, un NAS durante un controllo di parità) Febe uccideva un Chromium lento ma sano e ricominciava da capo sotto lo stesso carico, anche per sempre; i 5 s per la prima scheda erano il punto più fragile |
| 2026-11-13 | Nelle prove i limiti di tempo sono **protezioni contro un blocco, non misure**: ogni attesa finisce appena la sua condizione è vera e il limite è largo (il browser delle prove dell'interfaccia: 2 minuti per avvio, comandi DevTools e condizioni, e si ferma subito se Chromium si chiude, con le sue ultime righe di stderr; l'avvio del server di prova: 4 minuti, con l'ultimo errore del browser); niente attese fisse né conteggi di tentativi al posto di una scadenza. Ciò che una prova deve far accadere a un certo momento (il rilascio di un blocco, una lettura che non deve finire mai) gira su un thread suo, non sul pool; il pool delle prove parte con almeno 64 thread. Il browser delle prove dell'interfaccia usa la porta di debug 0 e legge quella scelta da Chromium; il browser di Febe resta sulla porta fissa (§6.1). Nelle prove di WebView2 su Windows la seconda «Scollega» e l'arresto non hanno più un limite di 10 s: la prova chiede che il processo del browser esca davvero (atteso per condizione) e che il profilo sparisca, con 90 s solo contro un blocco | #46: le prove cadevano a ogni carico del runner, ogni volta prove diverse, mai sul codice cambiato. Un limite stretto su un'attesa non misura nulla: dice solo quanto era carica la macchina. Con il parallelismo per metodo diverse prove bloccano thread del pool, e un pool di 2-4 thread ne aggiunge altri lentamente: il rilascio di un blocco dopo 300 ms arrivava dopo più di 10 s. Fra la porta libera trovata e il bind di Chromium un altro processo poteva prenderla. Con tutti i core occupati il processo di WebView2 (a priorità bassa) segnala l'uscita dopo i 15 s di attesa dell'host: la prova di «Scollega» cadeva 5 volte su 5 a 15,7-21 s, mentre ciò che deve escludere, una WebView orfana, tiene vivo il processo per sempre e lascia il profilo |
| 2026-11-13 | `FebeDatabase.DisposeAsync` chiude davvero i file prima di tornare: ogni comando SQLite si chiude appena usato (gli aiuti `Execute` e `Scalar` sulla connessione dello scrittore, e un ambito che chiude i comandi di ogni lettura), le connessioni di lettura del pool si chiudono **prima** dello scrittore, così è lo scrittore, ultimo, a togliere `febe.sqlite-wal` e `-shm`. All'arresto `FebeApp` aspetta anche i lavori in sottofondo che un avvio ancora in corso aggiunge mentre si ferma | #46: un comando lasciato al garbage collector tiene viva la sua istruzione finché il finalizzatore non passa; la connessione chiusa nel frattempo si chiudeva davvero solo dopo, e il WAL spariva durante lo spostamento della cartella Dati (`Could not find file …febe.sqlite-wal`). Riprodotto tenendo occupato il thread del finalizzatore, ora con una prova apposita. Una connessione di sola lettura chiusa per ultima lascia il WAL: per questo il pool si svuota prima |
+23
View File
@@ -7,6 +7,29 @@ fra le sessioni; ciò che è da fare si legge nelle issue aperte.
## In breve
**13 novembre 2026, sera: le prove che cadevano con il runner carico** (#46).
- Cause trovate, non solo tempi allungati. Tre erano difetti del prodotto:
- `FebeDatabase.DisposeAsync` lasciava i file aperti: i comandi SQLite non chiusi tenevano viva la connessione
fino al finalizzatore, e il WAL spariva durante lo spostamento della cartella Dati (`FoldersAutostart…`). Ora
ogni comando si chiude (`Sql.Execute`/`Scalar`, `CommandScope` per le letture) e il pool di lettura si svuota prima
dello scrittore; prova nuova `DatabaseCloseTests`, che cade senza la correzione.
- La scheda dell'account, senza le statistiche di ANALYZE, leggeva i file di tutta la libreria: ora una passata sola
sui file dell'account (`AccountStore.GetStatsAsync`).
- All'arresto `FebeApp` non aspettava i lavori in sottofondo aggiunti da un avvio ancora in corso.
- Avvio di Chromium: 120 s invece di 30 per un browser vivo ma lento, e la prima scheda si aspetta entro la stessa
scadenza (prima 5 s); un Chromium che si chiude fa fallire subito come prima. Il ricollegamento dopo una connessione
caduta smette appena il processo cambia.
- Prove: i limiti di tempo sono protezioni contro un blocco, non misure (aiuto comune `TestWait`); la prova di scala
ha limiti per classe di interrogazione, prende il tempo migliore e gira da sola dopo le prove parallele; il browser
delle prove dell'interfaccia usa la porta 0 e mostra lo stderr di Chromium quando fallisce; la prova di «Scollega»
di WebView2 chiede che il processo esca davvero, non 10 s. Cinque righe in `docs/decisioni.md`.
- Verificato su Windows con tutti i core occupati: le classi corrette 10 volte su 10, la prova di scala 5 su 5, le
prove di WebView2 3 su 3. **Non verificato qui**: le prove con Chromium e Xvfb girano solo nella CI Linux; il
contenitore Linux con 2 CPU non è partito perché C: era pieno (0,2 GB liberi) e Docker Desktop si è fermato.
- **Per Alberto**: liberare spazio su C: (il disco di Docker, `docker_data.vhdx`, è di circa 45 GB) e riavviare
Docker Desktop; poi si possono togliere le immagini `febe-ui46:base`, `febe-ui46:locale` e `febe-ci46-locale`.
**13 novembre 2026, pomeriggio: il rapporto di diagnostica della raccolta** (#39).
- Attività ha un terzo pulsante, «Diagnostica», che scarica `febe-diagnostica-AAAA-MM-GG.json`
+78 -14
View File
@@ -37,6 +37,8 @@ public sealed class FebeApp : IAsyncDisposable
private readonly FebeAppOptions _options;
private readonly IFebePlatform _platform;
private readonly CancellationTokenSource _lifetime = new();
// The start adds to these lists from its background tasks while a stop may already be running (#46): one lock.
private readonly Lock _gate = new();
private readonly List<Task> _background = [];
private readonly List<IDisposable> _disposables = [];
private readonly List<Action> _unsubscribe = [];
@@ -261,14 +263,14 @@ public sealed class FebeApp : IAsyncDisposable
services.Diagnostics = diagnostics;
EventHandler<Permit> permitDenied = (_, permit) => diagnostics.PermitDenied(null, permit.Kind, permit.Reason);
services.Governor.Denied += permitDenied;
_unsubscribe.Add(() => services.Governor.Denied -= permitDenied);
OnStop(() => services.Governor.Denied -= permitDenied);
var runner = new DeferredRecipeRunner();
var engine = new EngineHost(services, runner);
Engine = engine;
connection.AttachEngine(engine);
EventHandler<bool> drivingChanged = (_, _) => engine.NotifyUserDrivingChanged();
browser.UserDriving.Changed += drivingChanged;
_unsubscribe.Add(() => browser.UserDriving.Changed -= drivingChanged);
OnStop(() => browser.UserDriving.Changed -= drivingChanged);
_telegram = new TelegramClient(hooks.TelegramHandler);
var telegramChannel = new TelegramAlertChannel(_telegram);
@@ -320,10 +322,43 @@ public sealed class FebeApp : IAsyncDisposable
// Housekeeping in the background: temporary files of an interrupted run, the daily copy of the database.
var lifetime = _lifetime.Token;
_background.Add(Task.Run(() => CleanTemporaryFilesAsync(staging, layout, lifetime), CancellationToken.None));
_background.Add(Task.Run(() => BackupLoopAsync(new DailyBackup(Database, Clock, data.Backups), lifetime), CancellationToken.None));
_background.Add(Task.Run(() => BringUpBrowserAsync(ingest, staging, services, engine, runner, lifetime), CancellationToken.None));
_background.Add(Task.Run(() => Alerts.WatchDiskSpaceAsync(lifetime), CancellationToken.None));
InBackground(() => CleanTemporaryFilesAsync(staging, layout, lifetime));
InBackground(() => BackupLoopAsync(new DailyBackup(Database, Clock, data.Backups), lifetime));
InBackground(() => BringUpBrowserAsync(ingest, staging, services, engine, runner, lifetime));
InBackground(() => Alerts.WatchDiskSpaceAsync(lifetime));
}
/// <summary>Runs <paramref name="work"/> in the background; the stop waits for it.</summary>
private void InBackground(Func<Task> work)
{
lock (_gate)
{
_background.Add(Task.Run(work, CancellationToken.None));
}
}
/// <summary>Keeps <paramref name="disposable"/> for the stop; disposes it at once when the stop already took its list.</summary>
private void DisposeOnStop(IDisposable disposable)
{
lock (_gate)
{
if (Volatile.Read(ref _disposed) == 0)
{
_disposables.Add(disposable);
return;
}
}
disposable.Dispose();
}
/// <summary>Keeps <paramref name="unsubscribe"/> for the stop.</summary>
private void OnStop(Action unsubscribe)
{
lock (_gate)
{
_unsubscribe.Add(unsubscribe);
}
}
private async Task StartWebAsync(FebeAppHooks hooks, Action<IServiceCollection> register)
@@ -392,8 +427,8 @@ public sealed class FebeApp : IAsyncDisposable
runner.Target = Pipeline.Runner;
await CheckCollectionConfigurationAsync(Pipeline, engine, ct).ConfigureAwait(false);
var changes = Web.Services.GetRequiredService<LibraryChanges>();
_disposables.Add(changes.Attach(Pipeline.Downloads, Clock));
_background.Add(Task.Run(() => DownloadLoopAsync(Pipeline.Downloads, ct), CancellationToken.None));
DisposeOnStop(changes.Attach(Pipeline.Downloads, Clock));
InBackground(() => DownloadLoopAsync(Pipeline.Downloads, ct));
// Every finished recipe may have queued files: the queue looks at once instead of at its next idle poll.
var downloads = Pipeline.Downloads;
@@ -406,7 +441,7 @@ public sealed class FebeApp : IAsyncDisposable
}
services.Events.Notified += WakeDownloads;
_unsubscribe.Add(() => services.Events.Notified -= WakeDownloads);
OnStop(() => services.Events.Notified -= WakeDownloads);
await Connection!.RefreshAsync(ct).ConfigureAwait(false);
if (o.TestSiteAutoLogin && Connection.IsConnected != true)
@@ -414,7 +449,7 @@ public sealed class FebeApp : IAsyncDisposable
await TestSiteLoginAsync(ct).ConfigureAwait(false);
}
_background.Add(Task.Run(() => Connection.RunAsync(ct), CancellationToken.None));
InBackground(() => Connection.RunAsync(ct));
await engine.StartAsync(ct).ConfigureAwait(false);
Log.Info("motore", Connection.IsConnected == true
? "Motore avviato: il browser è collegato."
@@ -634,17 +669,46 @@ public sealed class FebeApp : IAsyncDisposable
}
await _lifetime.CancelAsync().ConfigureAwait(false);
foreach (var task in _background.ToArray())
// A start still in progress adds tasks while the first ones are awaited: look again until nothing new appears,
// so nothing still reads the database once it is closed (#46).
var awaited = 0;
while (true)
{
await Quietly(() => task).ConfigureAwait(false);
Task[] pending;
lock (_gate)
{
pending = _background.Skip(awaited).ToArray();
}
if (pending.Length == 0)
{
break;
}
foreach (var task in pending)
{
await Quietly(() => task).ConfigureAwait(false);
}
awaited += pending.Length;
}
foreach (var disposable in _disposables)
IDisposable[] disposables;
Action[] unsubscribes;
lock (_gate)
{
disposables = [.. _disposables];
unsubscribes = [.. _unsubscribe];
_disposables.Clear();
}
foreach (var disposable in disposables)
{
disposable.Dispose();
}
foreach (var unsubscribe in _unsubscribe)
foreach (var unsubscribe in unsubscribes)
{
unsubscribe();
}
@@ -29,7 +29,12 @@ public sealed record ChromiumLaunchOptions
/// <summary>Start page: always the rest page.</summary>
public string StartUrl { get; init; } = "about:blank";
public TimeSpan StartupTimeout { get; init; } = TimeSpan.FromSeconds(30);
/// <summary>
/// How long a Chromium that is alive may take to open its debugging port and its first tab. A crash fails the start
/// at once; this only bounds a browser that hangs, so it is generous for a loaded machine (docs/decisioni.md, #46).
/// </summary>
public TimeSpan StartupTimeout { get; init; } = TimeSpan.FromSeconds(120);
public RestartPolicy Restart { get; init; } = RestartPolicy.Default;
+53 -13
View File
@@ -255,27 +255,37 @@ public sealed class ChromiumProcessHost : IBrowserHost
$"La porta di debug {port} non è limitata a 127.0.0.1 ({listeners.Count} punti di ascolto): avvio annullato (specifica §6.1).");
}
await ConnectAsync(port, ct).ConfigureAwait(false);
await ConnectAsync(
port,
deadline,
() => process.HasExited ? $"Chromium si è chiuso all'avvio (codice {process.ExitCode}). {_supervisor?.StderrTail()}".TrimEnd() : null,
ct).ConfigureAwait(false);
}
private async Task ConnectAsync(int port, CancellationToken ct)
/// <summary>
/// Connects to the first page target. The first tab may appear a while after the port opens on a loaded machine:
/// the wait shares the start deadline, and ends at once when <paramref name="gone"/> says the process is no longer the
/// one to connect to (it exited, or the supervisor replaced it): nobody waits on the connect lock for a dead browser.
/// </summary>
private async Task ConnectAsync(int port, DateTimeOffset deadline, Func<string?> gone, CancellationToken ct)
{
await _connectLock.WaitAsync(ct).ConfigureAwait(false);
try
{
CdpTarget? page = null;
for (var attempt = 0; attempt < 50 && page is null; attempt++)
CdpTarget? page;
while ((page = await TryFindPageAsync(port, ct).ConfigureAwait(false)) is null)
{
page = await CdpTargetDiscovery.FindPageAsync(port, ct).ConfigureAwait(false);
if (page is null)
if (gone() is { } reason)
{
await _clock.Delay(TimeSpan.FromMilliseconds(100), ct).ConfigureAwait(false);
throw new InvalidOperationException(reason);
}
}
if (page is null)
{
throw new InvalidOperationException("Chromium non ha nessuna scheda aperta.");
if (_clock.Now >= deadline)
{
throw new InvalidOperationException("Chromium non ha nessuna scheda aperta.");
}
await _clock.Delay(TimeSpan.FromMilliseconds(100), ct).ConfigureAwait(false);
}
var client = await CdpClient.ConnectAsync(page.WebSocketDebuggerUrl, _log, null, ct).ConfigureAwait(false);
@@ -304,6 +314,23 @@ public sealed class ChromiumProcessHost : IBrowserHost
}
}
/// <summary>The page target, or null when there is none yet or the endpoint did not answer this time.</summary>
private static async Task<CdpTarget?> TryFindPageAsync(int port, CancellationToken ct)
{
try
{
return await CdpTargetDiscovery.FindPageAsync(port, ct).ConfigureAwait(false);
}
catch (HttpRequestException)
{
return null;
}
catch (TaskCanceledException) when (!ct.IsCancellationRequested)
{
return null; // a slow answer of /json/list, not a missing one
}
}
private void OnClientDisconnected(object? sender, Exception? reason)
{
if (Volatile.Read(ref _stopping) == 1 || Volatile.Read(ref _restarting) == 1 || !ReferenceEquals(sender, Volatile.Read(ref _client)))
@@ -330,11 +357,19 @@ public sealed class ChromiumProcessHost : IBrowserHost
return;
}
for (var attempt = 0; attempt < 3; attempt++)
// The process that dropped the connection: if it exits or the supervisor restarts it, stop at once and leave
// the connect lock to the restart, which connects by itself.
var processId = supervisor.ProcessId;
string? Gone() => supervisor.State != SupervisedState.Running || supervisor.ProcessId != processId
? "Chromium è stato riavviato o si è chiuso durante il ricollegamento."
: null;
var deadline = _clock.Now + _launch.StartupTimeout;
for (var attempt = 0; attempt < 3 && Gone() is null; attempt++)
{
try
{
await ConnectAsync(_options.DebugPort, ct).ConfigureAwait(false);
await ConnectAsync(_options.DebugPort, deadline, Gone, ct).ConfigureAwait(false);
_log.Info("Collegamento con Chromium ripristinato.");
return;
}
@@ -344,6 +379,11 @@ public sealed class ChromiumProcessHost : IBrowserHost
}
}
if (Gone() is not null)
{
return;
}
_log.Warn("Collegamento con Chromium perso: riavvio del browser.");
supervisor.KillCurrent();
}
+37
View File
@@ -0,0 +1,37 @@
using Microsoft.Data.Sqlite;
namespace Febe.Data;
/// <summary>
/// Keeps the commands that <see cref="Sql.Command"/> creates on this thread while a read runs, and disposes them when the
/// read ends. Reads build their commands inline (<c>c.Command(...).ExecuteReader()</c>); left to the garbage collector,
/// a command keeps its SQLite statement alive until the finalizer runs, so the connection closed by
/// <see cref="FebeDatabase.DisposeAsync"/> would only really close then, keeping the files open (#46).
/// </summary>
internal sealed class CommandScope : IDisposable
{
[ThreadStatic]
private static CommandScope? t_current;
private readonly CommandScope? _outer;
private readonly List<SqliteCommand> _commands = [];
private CommandScope(CommandScope? outer) => _outer = outer;
/// <summary>Starts tracking the commands created on this thread until the scope is disposed.</summary>
public static CommandScope Begin() => t_current = new CommandScope(t_current);
/// <summary>Adds <paramref name="command"/> to the open scope of this thread, if any.</summary>
public static void Track(SqliteCommand command) => t_current?._commands.Add(command);
public void Dispose()
{
t_current = _outer;
foreach (var command in _commands)
{
command.Dispose();
}
_commands.Clear();
}
}
+16 -11
View File
@@ -118,16 +118,16 @@ public sealed class FebeDatabase : IFebeStorage, IDatabaseMaintenance
try
{
connection.Open();
var mode = connection.Command("PRAGMA journal_mode = WAL").ExecuteScalar() as string;
var mode = connection.Scalar("PRAGMA journal_mode = WAL") as string;
if (!string.Equals(mode, "wal", StringComparison.OrdinalIgnoreCase))
{
throw new DatabaseMigrationException(
$"Febe non riesce a mettere il database in modalità WAL (modalità attuale: {mode}). La cartella dei dati deve stare su un disco locale.");
}
connection.Command("PRAGMA synchronous = NORMAL").ExecuteNonQuery();
connection.Command($"PRAGMA busy_timeout = {BusyTimeoutMs}").ExecuteNonQuery();
connection.Command("PRAGMA foreign_keys = ON").ExecuteNonQuery();
connection.Execute("PRAGMA synchronous = NORMAL");
connection.Execute($"PRAGMA busy_timeout = {BusyTimeoutMs}");
connection.Execute("PRAGMA foreign_keys = ON");
var version = MigrationRunner.Apply(connection, migrations);
return new FebeDatabase(path, connection, readConnectionString, version);
}
@@ -155,6 +155,7 @@ public sealed class FebeDatabase : IFebeStorage, IDatabaseMaintenance
try
{
using var connection = OpenReadConnection();
using var commands = CommandScope.Begin(); // disposed first: the connection then really closes
return Task.FromResult(work(connection));
}
catch (Exception ex) when (ex is not OutOfMemoryException)
@@ -184,7 +185,8 @@ public sealed class FebeDatabase : IFebeStorage, IDatabaseMaintenance
connection =>
{
// VACUUM INTO reads a consistent snapshot; it works on a read-only connection and does not block the writer.
connection.Command("VACUUM INTO @destination").With("@destination", destination).ExecuteNonQuery();
using var vacuum = connection.Command("VACUUM INTO @destination").With("@destination", destination);
vacuum.ExecuteNonQuery();
return true;
},
ct);
@@ -210,9 +212,17 @@ public sealed class FebeDatabase : IFebeStorage, IDatabaseMaintenance
}
Writer.Dispose();
// The idle read connections close first, so the writer is the last connection: only the writer can checkpoint
// and delete febe.sqlite-wal and -shm when it closes (a read-only connection leaves them behind).
using (var pooled = new SqliteConnection(_readConnectionString))
{
SqliteConnection.ClearPool(pooled);
}
try
{
_writerConnection.Command("PRAGMA wal_checkpoint(TRUNCATE)").ExecuteNonQuery();
_writerConnection.Execute("PRAGMA wal_checkpoint(TRUNCATE)");
}
catch (SqliteException)
{
@@ -220,11 +230,6 @@ public sealed class FebeDatabase : IFebeStorage, IDatabaseMaintenance
}
_writerConnection.Dispose();
using (var pooled = new SqliteConnection(_readConnectionString))
{
SqliteConnection.ClearPool(pooled);
}
return ValueTask.CompletedTask;
}
}
+7 -7
View File
@@ -77,7 +77,7 @@ internal static partial class MigrationRunner
}
public static int GetVersion(SqliteConnection connection) =>
(int)connection.Command("PRAGMA user_version").ScalarInt64();
Convert.ToInt32(connection.Scalar("PRAGMA user_version"), CultureInfo.InvariantCulture);
/// <summary>Applies the pending migrations. Returns the resulting version.</summary>
public static int Apply(SqliteConnection connection, IReadOnlyList<Migration> migrations)
@@ -97,7 +97,7 @@ internal static partial class MigrationRunner
}
// Table rebuilds in later migrations need foreign keys off; they are checked before each commit instead.
connection.Command("PRAGMA foreign_keys = OFF").ExecuteNonQuery();
connection.Execute("PRAGMA foreign_keys = OFF");
try
{
foreach (var migration in migrations.Where(m => m.Version > current))
@@ -105,8 +105,9 @@ internal static partial class MigrationRunner
using var transaction = connection.BeginTransaction(deferred: false);
try
{
connection.Command(migration.Sql, transaction).ExecuteNonQuery();
using (var check = connection.Command("PRAGMA foreign_key_check", transaction).ExecuteReader())
connection.Execute(migration.Sql, transaction);
using (var checkCommand = connection.Command("PRAGMA foreign_key_check", transaction))
using (var check = checkCommand.ExecuteReader())
{
if (check.Read())
{
@@ -115,8 +116,7 @@ internal static partial class MigrationRunner
}
}
connection.Command(string.Create(CultureInfo.InvariantCulture, $"PRAGMA user_version = {migration.Version}"), transaction)
.ExecuteNonQuery();
connection.Execute(string.Create(CultureInfo.InvariantCulture, $"PRAGMA user_version = {migration.Version}"), transaction);
transaction.Commit();
}
catch (Exception ex) when (ex is SqliteException or DatabaseMigrationException)
@@ -130,7 +130,7 @@ internal static partial class MigrationRunner
}
finally
{
connection.Command("PRAGMA foreign_keys = ON").ExecuteNonQuery();
connection.Execute("PRAGMA foreign_keys = ON");
}
return GetVersion(connection);
+19
View File
@@ -75,9 +75,28 @@ internal static class Sql
var command = connection.CreateCommand();
command.CommandText = sql;
command.Transaction = transaction;
CommandScope.Track(command);
return command;
}
/// <summary>
/// Runs a statement and disposes its command at once. A command left to the garbage collector keeps its SQLite
/// statement alive until the finalizer runs, and a connection closed meanwhile only closes then: the WAL and its
/// handles outlive <see cref="FebeDatabase.DisposeAsync"/> (#46).
/// </summary>
public static int Execute(this SqliteConnection connection, string sql, SqliteTransaction? transaction = null)
{
using var command = connection.Command(sql, transaction);
return command.ExecuteNonQuery();
}
/// <summary>Runs a query for one value and disposes its command at once (see <see cref="Execute"/>).</summary>
public static object? Scalar(this SqliteConnection connection, string sql)
{
using var command = connection.Command(sql);
return command.ExecuteScalar();
}
/// <summary>Sets a parameter, adding it the first time: a prepared command can be run again with new values.</summary>
public static SqliteCommand With(this SqliteCommand command, string name, object? value)
{
+15 -8
View File
@@ -221,14 +221,21 @@ internal sealed class AccountStore(FebeDatabase db) : IAccountStore
}
}
var bytes = c.Command("""
SELECT COALESCE(SUM(m.bytes), 0) FROM media m JOIN entry e ON e.id = m.entry_id
WHERE e.account_id = @a AND m.state IN ('Downloaded', 'Imported')
""").With("@a", accountId).ScalarInt64();
var pending = c.Command("""
SELECT COUNT(*) FROM media m JOIN entry e ON e.id = m.entry_id
WHERE e.account_id = @a AND m.state IN ('ToDownload', 'Expired')
""").With("@a", accountId).ScalarInt64();
// One pass over the account's files, driven from its entries (CROSS JOIN keeps entry as the outer loop):
// without ANALYZE statistics SQLite otherwise starts from the media state index and reads the files of
// the whole library for one account's card (#46).
long bytes, pending;
using (var r = c.Command("""
SELECT COALESCE(SUM(CASE WHEN m.state IN ('Downloaded', 'Imported') THEN m.bytes END), 0),
COALESCE(SUM(m.state IN ('ToDownload', 'Expired')), 0)
FROM entry e CROSS JOIN media m ON m.entry_id = e.id
WHERE e.account_id = @a
""").With("@a", accountId).ExecuteReader())
{
r.Read();
bytes = r.GetInt64(0);
pending = r.GetInt64(1);
}
var collections = c.Command("SELECT COUNT(*) FROM highlight WHERE account_id = @a AND removed_at IS NULL")
.With("@a", accountId).ScalarInt64();
+9 -2
View File
@@ -83,8 +83,15 @@ public sealed class WebView2CdpTests
var gestures = new HumanGestures(cdp, clock, new SeededRandom(20261020));
await gestures.MoveToAsync(new PagePoint(640, 450), ct);
var requested = await gestures.ScrollBurstAsync(ct: ct);
await Task.Delay(800, ct);
var scrolled = (await cdp.EvaluateAsync("window.scrollY", ct)).GetDouble();
// The smooth scroll runs after the events: by condition, not after a fixed time (#46).
var scrolled = 0.0;
var watch = System.Diagnostics.Stopwatch.StartNew();
while ((scrolled = (await cdp.EvaluateAsync("window.scrollY", ct)).GetDouble()) <= 100 && watch.Elapsed < TimeSpan.FromSeconds(30))
{
await Task.Delay(50, ct);
}
TestContext.WriteLine($"Scorrimento: chiesti {requested} pixel, pagina scesa di {scrolled}.");
Assert.IsGreaterThan(100.0, scrolled, "the wheel really scrolled the page out of view");
@@ -137,9 +137,12 @@ public sealed class WebView2LifecycleTests
/// browser, then the waiting attempt started a second WebView in the same window without closing the first. The
/// orphan kept its browser process alive on the profile: the next «Scollega» waited 15 s for nothing and could not
/// delete the profile (the Instagram session stayed on disk), and exiting waited 15 s more.
/// The proof is what happens, not how long it takes (#46): an orphan keeps the browser process alive, so that process
/// never exits and the profile stays. Under full CPU load a healthy process may report its exit after ExitWait
/// (15 s), which a time limit could not tell from the orphan; the limits here only stop a hang.
/// </summary>
[TestMethod]
[Timeout(180_000, CooperativeCancellation = true)]
[Timeout(420_000, CooperativeCancellation = true)]
public async Task ScollegaWhileTheFirstStartWaitsToRetryLeavesOneWebViewAndTheNextScollegaWorks()
{
var ct = TestContext.CancellationToken;
@@ -150,7 +153,7 @@ public sealed class WebView2LifecycleTests
Directory.CreateDirectory(Path.GetDirectoryName(profile)!);
await File.WriteAllTextAsync(profile, "occupa il posto della cartella del profilo", ct);
var run = Task.Run(() => bed.Browser.RunAsync(ct), ct);
await bed.WaitUntilAsync(() => bed.Lines.Any("Nuovo tentativo fra 15 s."), TimeSpan.FromSeconds(40), "il secondo avvio non riuscito", ct);
await bed.WaitUntilAsync(() => bed.Lines.Any("Nuovo tentativo fra 15 s."), TimeSpan.FromSeconds(90), "il secondo avvio non riuscito", ct);
File.Delete(profile);
// «Scollega» while RunAsync waits 15 s for its third attempt: the browser starts with it.
@@ -159,30 +162,34 @@ public sealed class WebView2LifecycleTests
var pid = bed.Browser.Host.BrowserProcessId;
// The waiting attempt finds the browser running: no second WebView, the one of «Scollega» untouched.
await run.WaitAsync(TimeSpan.FromSeconds(40), ct);
await run.WaitAsync(TimeSpan.FromSeconds(90), ct);
var children = await bed.ChildWindowsAsync();
Assert.HasCount(1, children, "one WebView in the window: " + string.Join(", ", children));
Assert.AreEqual(pid, bed.Browser.Host.BrowserProcessId, "the browser process of «Scollega» is still the one in use");
Assert.IsTrue(bed.Lines.Any("Browser di Febe pronto: WebView2 "), bed.Lines.Text);
Assert.IsTrue(bed.Browser.IsAvailable);
// The next «Scollega»: the only browser process exits at once, the profile goes, the browser comes back.
// The next «Scollega»: the only browser process exits, the profile goes, the browser comes back.
var marker = Path.Combine(profile, "segno-della-prova");
await File.WriteAllTextAsync(marker, "x", ct);
using var first = Process.GetProcessById(pid);
var watch = Stopwatch.StartNew();
await Task.Run(() => bed.Browser.RestartWithEmptyProfileAsync(ct), ct);
await Task.Run(() => bed.Browser.RestartWithEmptyProfileAsync(ct), ct).WaitAsync(HangGuard, ct);
TestContext.WriteLine($"Secondo «Scollega»: {watch.Elapsed.TotalSeconds:0.0} s.");
Assert.IsLessThan(10, watch.Elapsed.TotalSeconds, "no wait for a browser process that never exits");
Assert.IsTrue(await HasExitedAsync(first, ct), "the browser process of the first «Scollega» exited: no orphan kept it alive");
Assert.IsFalse(File.Exists(marker), "the profile with the session was deleted");
Assert.HasCount(1, await bed.ChildWindowsAsync());
Assert.IsTrue(bed.Browser.IsAvailable);
var second = bed.Browser.Host.BrowserProcessId;
Assert.AreNotEqual(pid, second, "a new browser process");
using var last = Process.GetProcessById(second);
// Exiting closes the only WebView and its process exits: no wait in vain.
// Exiting closes the only WebView and its process exits: nothing is left running.
watch.Restart();
await bed.Browser.DisposeAsync();
await bed.Browser.DisposeAsync().AsTask().WaitAsync(HangGuard, ct);
TestContext.WriteLine($"Arresto: {watch.Elapsed.TotalSeconds:0.0} s.");
TestContext.WriteLine(bed.Lines.Text);
Assert.IsLessThan(10, watch.Elapsed.TotalSeconds, "exiting waits for nothing");
Assert.IsTrue(await HasExitedAsync(last, ct), "exiting leaves no browser process behind");
Assert.HasCount(2, bed.Lines.Lines.Where(l => l.Contains("ERRORE", StringComparison.Ordinal)), "only the two failed starts: " + bed.Lines.Text);
}
@@ -214,11 +221,15 @@ public sealed class WebView2LifecycleTests
Assert.IsFalse(run.IsCompleted, "the start waits for the UI thread");
// «Esci»: FebeApp cancels its lifetime, then waits for RunAsync.
// The proof is the order, not a time (#46): RunAsync returns while the UI thread is still busy, which it stays
// until released below (or 60 s). The 30 s only bound a RunAsync that does wait for it.
var watch = Stopwatch.StartNew();
await lifetime.CancelAsync();
bool uiStillBusy;
try
{
await run.WaitAsync(TimeSpan.FromSeconds(10), ct);
await run.WaitAsync(TimeSpan.FromSeconds(30), ct);
uiStillBusy = !busy.Task.IsCompleted;
}
finally
{
@@ -227,7 +238,7 @@ public sealed class WebView2LifecycleTests
}
TestContext.WriteLine($"RunAsync è tornato {watch.Elapsed.TotalSeconds:0.0} s dopo la cancellazione.");
Assert.IsLessThan(3, watch.Elapsed.TotalSeconds, "RunAsync stops waiting at once");
Assert.IsTrue(uiStillBusy, "RunAsync stops waiting at once, while the UI thread is still busy");
Assert.IsFalse(bed.Browser.IsAvailable);
// The start that was waiting for the UI thread does not begin after the exit, and stopping is quick.
@@ -240,6 +251,31 @@ public sealed class WebView2LifecycleTests
Assert.IsFalse(bed.Lines.Any("Browser di Febe pronto"), bed.Lines.Text);
}
/// <summary>
/// Only against a hang: well above ExitWait of WebView2Host (15 s) and the retries of the profile deletion (5 s),
/// so a slow but healthy machine never reaches it (#46).
/// </summary>
private static readonly TimeSpan HangGuard = TimeSpan.FromSeconds(90);
/// <summary>
/// Whether a browser process exits, waiting for it by condition: a process that exits late under load is fine, one
/// kept alive by an orphan WebView never exits and the guard ends the wait.
/// </summary>
private static async Task<bool> HasExitedAsync(Process process, CancellationToken ct)
{
using var guard = CancellationTokenSource.CreateLinkedTokenSource(ct);
guard.CancelAfter(HangGuard);
try
{
await process.WaitForExitAsync(guard.Token);
return true;
}
catch (OperationCanceledException) when (!ct.IsCancellationRequested)
{
return false;
}
}
[TestMethod]
[Timeout(180_000, CooperativeCancellation = true)]
public async Task WhenItsBrowserProcessDiesTheBrowserStartsAgainByItself()
@@ -13,6 +13,7 @@ using Febe.Browser.Targets;
using Febe.Core.Abstractions;
using Febe.Core.Collection;
using Febe.Tests.Browser.TestSupport;
using Febe.Tests.Support;
namespace Febe.Tests.Browser;
@@ -139,7 +140,11 @@ public sealed class BrowserIntegrationTests
var requested = await gestures.ScrollBurstAsync();
requested += await gestures.ScrollBurstAsync();
await Task.Delay(600);
// The scroll runs in the compositor after the events: by condition, not after a fixed time (#46).
await TestWait.UntilAsync(
async () => await Browser.EvalAsync<double>("window.scrollY") > 200 && await Browser.EvalAsync<int>("(window.__wheel && window.__wheel.count) || 0") >= 6,
"la pagina scesa con la rotella");
var scrolled = await Browser.EvalAsync<double>("window.scrollY");
var wheel = await Browser.Cdp.EvaluateAsync("window.__wheel");
@@ -149,7 +154,7 @@ public sealed class BrowserIntegrationTests
Assert.IsTrue(wheel.GetProperty("trusted").GetBoolean(), "Wheel events are trusted, as real ones.");
await gestures.ScrollBackAsync();
await Task.Delay(400);
await TestWait.UntilAsync(async () => await Browser.EvalAsync<double>("window.scrollY") < scrolled, "la pagina risalita");
Assert.IsTrue(await Browser.EvalAsync<double>("window.scrollY") < scrolled, "A scroll back goes up.");
}
@@ -225,7 +230,8 @@ public sealed class BrowserIntegrationTests
// S4 locally: click into a field through the remote view and type.
await view.NavigateAsync(Browser.Site.Url("/modulo"));
await Browser.Cdp.WaitForLoadAsync(Browser.Clock, TimeSpan.FromSeconds(10));
Assert.IsTrue(await Browser.Cdp.WaitForLoadAsync(Browser.Clock, TimeSpan.FromSeconds(60)), "La pagina del modulo non ha finito di caricare.");
await Browser.WaitForPaintAsync();
await Task.Delay(RealBrowser.SettleAfterLoad);
var field = await new TargetResolver(Browser.Cdp, TargetMap.Parse(Targets)).ResolveAsync("campo");
Assert.IsTrue(field.Found, field.Message);
+3 -9
View File
@@ -11,6 +11,7 @@ using Febe.Core.Collection;
using Febe.Core.Collection.Readers;
using Febe.Core.Collection.Recipes;
using Febe.Tests.Browser.TestSupport;
using Febe.Tests.Support;
namespace Febe.Tests.Browser;
@@ -418,15 +419,8 @@ public sealed class CaptureUnitTests
["response"] = new JsonObject { ["url"] = url, ["status"] = 200, ["mimeType"] = "text/html; charset=utf-8" },
};
private static async Task WaitUntilAsync(Func<bool> condition)
{
for (var i = 0; i < 300 && !condition(); i++)
{
await Task.Delay(10);
}
Assert.IsTrue(condition(), "Condition not reached in time.");
}
// By condition until a deadline, not a count of polls: bodies and acks run on pool continuations (#46).
private static Task WaitUntilAsync(Func<bool> condition) => TestWait.UntilAsync(condition, "condizione della prova");
/// <summary>The body of a cleaned Instagram sample (<c>tests/campioni/instagram</c>), as JSON text.</summary>
private static string SampleBody(string name)
+4 -9
View File
@@ -7,6 +7,7 @@ using System.Text.Json;
using System.Text.Json.Nodes;
using Febe.Browser.Cdp;
using Febe.Browser.Diagnostics;
using Febe.Tests.Support;
namespace Febe.Tests.Browser;
@@ -161,15 +162,9 @@ public sealed class CdpClientTests
Assert.IsFalse(all.Contains("segret", StringComparison.Ordinal), "Parameters, events and results never reach the log.");
}
private static async Task WaitUntilAsync(Func<bool> condition)
{
for (var i = 0; i < 200 && !condition(); i++)
{
await Task.Delay(10);
}
Assert.IsTrue(condition());
}
// By condition until a deadline, not a count of polls: the receive loop runs on the pool, which the parallel tests may
// keep busy for seconds (#46).
private static Task WaitUntilAsync(Func<bool> condition) => TestWait.UntilAsync(condition, "condizione della prova");
/// <summary>A CdpClient connected to a server-side WebSocket over a loopback TCP connection.</summary>
private sealed class SocketPair : IAsyncDisposable
+13 -7
View File
@@ -106,20 +106,25 @@ public sealed class HostingTests
}
var restarted = 0;
var twice = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
await using var supervisor = new SupervisedProcess(
"prova",
() => new ProcessStartInfo("/bin/sleep", "0.2"),
(_, _) => Task.CompletedTask,
new SystemClock(),
policy: new RestartPolicy(TimeSpan.FromMilliseconds(50), TimeSpan.FromMilliseconds(200), TimeSpan.FromMinutes(1)));
supervisor.Restarted += (_, _) => Interlocked.Increment(ref restarted);
supervisor.Restarted += (_, _) =>
{
if (Interlocked.Increment(ref restarted) >= 2)
{
twice.TrySetResult();
}
};
await supervisor.StartAsync(CancellationToken.None);
for (var i = 0; i < 100 && Volatile.Read(ref restarted) < 2; i++)
{
await Task.Delay(50);
}
// By condition; the limit only bounds a supervisor that never restarts (process starts are slow under load, #46).
await twice.Task.WaitAsync(TimeSpan.FromSeconds(30));
Assert.IsTrue(Volatile.Read(ref restarted) >= 2, "The process was restarted after exiting.");
await supervisor.StopAsync(TimeSpan.FromSeconds(2));
Assert.AreEqual(SupervisedState.Stopped, supervisor.State);
@@ -145,9 +150,10 @@ public sealed class HostingTests
var pid = supervisor.ProcessId;
Assert.IsNotNull(pid);
// The kill would come only after 120 s: any stop well before that was SIGTERM, with room for a loaded runner (#46).
var watch = Stopwatch.StartNew();
await supervisor.StopAsync(TimeSpan.FromSeconds(5));
Assert.IsTrue(watch.Elapsed < TimeSpan.FromSeconds(4), "SIGTERM ends sleep at once; no need to wait for the kill.");
await supervisor.StopAsync(TimeSpan.FromSeconds(120));
Assert.IsLessThan(TimeSpan.FromSeconds(30), watch.Elapsed, "SIGTERM ends sleep at once; no need to wait for the kill.");
Assert.IsFalse(Process.GetProcesses().Any(p => p.Id == pid && !p.HasExited && p.ProcessName == "sleep"));
}
}
@@ -145,16 +145,38 @@ public sealed class RealBrowser : IAsyncDisposable
throw new InvalidOperationException($"Navigazione fallita: {result.ErrorText}");
}
if (!await Cdp.WaitForLoadAsync(Clock, TimeSpan.FromSeconds(15)))
// The limit only bounds a hang: the wait ends with the load (#46).
if (!await Cdp.WaitForLoadAsync(Clock, TimeSpan.FromSeconds(60)))
{
throw new TimeoutException("La pagina non ha finito di caricare.");
}
// Like the post-load wait of the recipes (1–6 s, shortened here): Chromium ignores input that arrives
// right after a navigation, before the page has painted.
// right after a navigation, before the page has painted. On a loaded runner the first paint comes later.
await WaitForPaintAsync();
await Task.Delay(SettleAfterLoad);
}
/// <summary>
/// Waits until the page reports its first paint (a read-only look at the paint entries), up to 15 s. Not reported
/// in time is not an error: the settle after it stays as before.
/// </summary>
public async Task<bool> WaitForPaintAsync()
{
var watch = System.Diagnostics.Stopwatch.StartNew();
while (watch.Elapsed < TimeSpan.FromSeconds(15))
{
if (await EvalAsync<bool>("performance.getEntriesByType('paint').length > 0"))
{
return true;
}
await Task.Delay(50);
}
return false;
}
public static TimeSpan SettleAfterLoad { get; } = TimeSpan.FromMilliseconds(500);
public async Task<T> EvalAsync<T>(string expression) =>
@@ -22,6 +22,8 @@ namespace Febe.Tests.Collection;
[TestClass]
public sealed class DownloadQueueTests
{
public TestContext TestContext { get; set; } = null!;
[TestMethod]
public async Task A429OfTheCdnIsAThrottleSignalThatStopsTheQueueWithoutCountingAnAttempt()
{
@@ -141,7 +143,7 @@ public sealed class DownloadQueueTests
}
[TestMethod]
[Timeout(30_000, CooperativeCancellation = true)]
[Timeout(60_000, CooperativeCancellation = true)]
public async Task ManyWakeCallsDuringAPassGiveOneMorePassNotOneEach()
{
using var temp = new TempFolder("coda-sveglia");
@@ -151,19 +153,24 @@ public sealed class DownloadQueueTests
// Every pass tries the file again (no wait between attempts here): the passes are the attempts.
var fetcher = ScriptedFetcher.Status(503);
var queue = new DownloadQueue(library.Db, library.Ingest, fetcher, new FakeDataBudget(), new IdleClock(), new SeededRandom(1),
var clock = new IdleClock();
var queue = new DownloadQueue(library.Db, library.Ingest, fetcher, new FakeDataBudget(), clock, new SeededRandom(1),
new DownloadOptions { StagingFolder = temp.Combine("dati", "temporanei"), RetryDelays = [], MaxAttempts = 1000 });
for (var i = 0; i < 50; i++)
{
queue.Wake(); // e.g. one per «ricetta.*» event of a long session
}
// Every pass ends by starting an idle wait (a wake cuts it short): the second one means two passes are done.
// By condition, not by a time window two passes must fit in (#46).
using var stop = new CancellationTokenSource();
var running = queue.RunAsync(stop.Token);
await Task.Delay(TimeSpan.FromMilliseconds(500));
await clock.SecondIdle.Task.WaitAsync(TimeSpan.FromSeconds(20), TestContext.CancellationToken);
await Task.Delay(TimeSpan.FromMilliseconds(300), TestContext.CancellationToken); // negative check: no third pass
await stop.CancelAsync();
await Assert.ThrowsAsync<OperationCanceledException>(() => running);
Assert.AreEqual(2, fetcher.Calls, "The pass, then one more for all the calls: then it waits.");
Assert.AreEqual(2, clock.IdleWaits, "After the second pass the queue waits: nothing wakes it again.");
}
[TestMethod]
@@ -450,12 +457,32 @@ public sealed class DownloadQueueTests
/// <summary>A clock whose short waits (pauses between files) pass at once and whose long ones (idle) never end.</summary>
private sealed class IdleClock : IClock
{
private int _idleWaits;
public DateTimeOffset Now => Make.T0;
public TimeZoneInfo LocalZone => TestZones.Rome;
public Task Delay(TimeSpan time, CancellationToken ct) =>
time < TimeSpan.FromSeconds(10) ? Task.CompletedTask : Task.Delay(Timeout.Infinite, ct);
/// <summary>How many idle waits started.</summary>
public int IdleWaits => Volatile.Read(ref _idleWaits);
/// <summary>Completed when the second idle wait starts.</summary>
public TaskCompletionSource SecondIdle { get; } = new(TaskCreationOptions.RunContinuationsAsynchronously);
public Task Delay(TimeSpan time, CancellationToken ct)
{
if (time < TimeSpan.FromSeconds(10))
{
return Task.CompletedTask;
}
if (Interlocked.Increment(ref _idleWaits) == 2)
{
SecondIdle.TrySetResult();
}
return Task.Delay(Timeout.Infinite, ct);
}
}
/// <summary>A content stream that runs <paramref name="midway"/> once, half way through.</summary>
@@ -45,7 +45,7 @@ public sealed class ProbeIntegrationTests
}
[TestMethod]
[Timeout(60_000, CooperativeCancellation = true)]
[Timeout(180_000, CooperativeCancellation = true)]
public async Task TheProbeRecordsResponsesAndTargetsWithoutGesturesAndItsSamplesCleanUp()
{
if (s_unavailable is not null)
@@ -65,12 +65,18 @@ public sealed class ProbeIntegrationTests
// Alberto opens a profile and points at the «Reel» tab.
var cdp = s_chromium.Cdp;
await cdp.NavigateAsync(site.Url("/profilo/marta.prova/"));
await cdp.WaitForLoadAsync(new SystemClock(), TimeSpan.FromSeconds(10));
await Task.Delay(800);
Assert.IsTrue(await cdp.WaitForLoadAsync(new SystemClock(), TimeSpan.FromSeconds(60)), "Il profilo non ha finito di caricare.");
// By condition, not after fixed pauses (#46): the data answer recorded and the tab drawn.
await TestWait.UntilAsync(
async () => probe.Stats.Responses >= 1
&& (await cdp.EvaluateAsync("[...document.querySelectorAll('a[role=tab]')].some((a) => a.textContent === 'Reel')")).ValueKind == JsonValueKind.True,
"i dati del profilo registrati e la scheda «Reel» disegnata",
ct: TestContext.CancellationToken);
var tab = await cdp.EvaluateAsync("(() => { const r = [...document.querySelectorAll('a[role=tab]')].find((a) => a.textContent === 'Reel').getBoundingClientRect(); return { x: r.left + r.width / 2, y: r.top + r.height / 2 }; })()");
var alberto = new HumanGestures(cdp, new SystemClock(), new SeededRandom(5));
await alberto.MoveToAsync(new PagePoint(tab.GetProperty("x").GetDouble(), tab.GetProperty("y").GetDouble()));
await Task.Delay(600);
await TestWait.UntilAsync(() => HoverRecorded(raw), "il passaggio sulla scheda «Reel» registrato", ct: TestContext.CancellationToken);
await probe.StopAsync();
Assert.IsTrue(probe.Stats.Responses >= 1, "The profile data was recorded.");
}
@@ -90,9 +96,7 @@ public sealed class ProbeIntegrationTests
}
var interactions = File.ReadAllLines(Path.Combine(raw, "interazioni.jsonl"));
Assert.IsTrue(
interactions.Any(l => l.Contains("\"sopra\"", StringComparison.Ordinal) && l.Contains("\"role\":\"tab\"", StringComparison.Ordinal) && l.Contains("Reel", StringComparison.Ordinal)),
"The hovered tab is described with role and accessible name: " + string.Join(" | ", interactions));
Assert.IsTrue(interactions.Any(IsReelHover), "The hovered tab is described with role and accessible name: " + string.Join(" | ", interactions));
Assert.IsTrue(File.ReadAllText(Path.Combine(raw, "navigazioni.jsonl")).Contains("/profilo/marta.prova/", StringComparison.Ordinal));
// M0 path: clean, then verify.
@@ -118,4 +122,21 @@ public sealed class ProbeIntegrationTests
public IDisposable On(string eventName, Action<JsonElement> handler) => inner.On(eventName, handler);
}
private static bool IsReelHover(string line) =>
line.Contains("\"sopra\"", StringComparison.Ordinal) && line.Contains("\"role\":\"tab\"", StringComparison.Ordinal) && line.Contains("Reel", StringComparison.Ordinal);
/// <summary>True once the probe has written the hover on «Reel» (the file may be in the middle of a write).</summary>
private static bool HoverRecorded(string raw)
{
try
{
var path = Path.Combine(raw, "interazioni.jsonl");
return File.Exists(path) && File.ReadAllLines(path).Any(IsReelHover);
}
catch (IOException)
{
return false;
}
}
}
@@ -146,7 +146,9 @@ public sealed class StopAndDownloadIntegrationTests
[Timeout(60_000, CooperativeCancellation = true)]
public async Task AnUnknownResponseShapeStopsTheRecipeWithTheKeyStructureOnly()
{
await using var h = await CollectionHarness.CreateAsync(Chromium, new Febe.Core.Collection.Recipes.RecipeOptions { DataTimeout = TimeSpan.FromSeconds(8) });
// The recipe waits for a recognized answer that never comes, then judges what it captured: the wait must leave the
// page time to load and its body to be read. 8 s of real time, on the clock compressed 10× (#46).
await using var h = await CollectionHarness.CreateAsync(Chromium, new Febe.Core.Collection.Recipes.RecipeOptions { DataTimeout = TimeSpan.FromSeconds(80) });
var marta = await h.AddAccountAsync("marta.prova");
h.Site.Control.Set(StopMode.UnknownFormat);
@@ -0,0 +1,65 @@
using System.Runtime.CompilerServices;
using Febe.Data;
using Febe.Tests.Support;
namespace Febe.Tests.Data;
/// <summary>
/// #46: <see cref="FebeDatabase.DisposeAsync"/> must really close the database, even when the finalizer thread is behind.
/// A command left to the garbage collector keeps its SQLite statement until it is finalized; a connection closed
/// meanwhile only closes then, and its WAL disappeared under the move of the Data folder. The class runs alone because
/// it holds the process-wide finalizer thread for a moment.
/// </summary>
[TestClass]
[DoNotParallelize]
public sealed class DatabaseCloseTests
{
public TestContext TestContext { get; set; } = null!;
[TestMethod]
[Timeout(120_000, CooperativeCancellation = true)]
public async Task DisposeClosesTheFilesEvenWhenTheFinalizerIsBehind()
{
using var temp = new TempFolder();
using var finalizing = new ManualResetEventSlim();
using var release = new ManualResetEventSlim();
var folder = temp.Combine("dati");
try
{
// Holds the finalizer thread: whatever the next collections queue for finalization waits behind it.
BlockTheFinalizer(finalizing, release);
GC.Collect();
Assert.IsTrue(finalizing.Wait(TimeSpan.FromSeconds(30), TestContext.CancellationToken), "Il finalizzatore non è partito.");
var db = await FebeDatabase.OpenAsync(folder, TestContext.CancellationToken);
await db.Settings.SetAsync("prova", "1", TestContext.CancellationToken);
Assert.AreEqual("1", await db.Settings.GetAsync("prova", TestContext.CancellationToken));
GC.Collect(); // commands nobody disposed are collected now and wait for the finalizer
await db.DisposeAsync();
Assert.IsFalse(File.Exists(Path.Combine(folder, "febe.sqlite-wal")), "Il WAL resta dopo la chiusura.");
Assert.IsFalse(File.Exists(Path.Combine(folder, "febe.sqlite-shm")), "Il file -shm resta dopo la chiusura.");
await using var again = await FebeDatabase.OpenAsync(folder, TestContext.CancellationToken);
Assert.AreEqual("1", await again.Settings.GetAsync("prova", TestContext.CancellationToken));
}
finally
{
release.Set();
GC.Collect();
GC.WaitForPendingFinalizers();
}
}
[MethodImpl(MethodImplOptions.NoInlining)]
private static void BlockTheFinalizer(ManualResetEventSlim finalizing, ManualResetEventSlim release) =>
_ = new FinalizerBlocker(finalizing, release);
private sealed class FinalizerBlocker(ManualResetEventSlim finalizing, ManualResetEventSlim release)
{
~FinalizerBlocker()
{
finalizing.Set();
release.Wait(TimeSpan.FromSeconds(30)); // bounded: the finalizer thread never stays blocked for good
}
}
}
+48 -48
View File
@@ -6,24 +6,35 @@ using Febe.Tests.Support;
namespace Febe.Tests.Data;
/// <summary>Spec §4.6: the library must stay fluid with 200,000 entries.</summary>
/// <summary>
/// Spec §4.6: the library must stay fluid with 200,000 entries. The class runs alone, after the parallel tests, so the
/// timings do not compete with the other workers and the long fill does not load them (#46).
/// </summary>
[TestClass]
[DoNotParallelize]
public sealed class LibraryScaleTests
{
private const int Entries = 200_000;
private const int Accounts = 40;
// Bounds per class of query, about 5-8× the values measured on an unloaded machine (docs/decisioni.md, #46).
// Index seeks and windows measure 2-30 ms; aggregates over the whole library or an account 60-220 ms.
private const double IndexBoundMs = 250;
private const double AggregateBoundMs = 1500;
public TestContext TestContext { get; set; } = null!;
[TestMethod]
[Timeout(1_200_000, CooperativeCancellation = true)] // only a hang guard: under heavy load the fill took 7.5 minutes
public async Task LibraryQueriesStayFastWith200000Entries()
{
var ct = TestContext.CancellationToken;
using var temp = new TempFolder();
await using var db = await TestDatabase.OpenAsync(temp);
var accountIds = new List<long>();
for (var a = 0; a < Accounts; a++)
{
accountIds.Add((await db.Accounts.AddAsync(Make.Account($"account{a:00}.demo", $"{1000 + a}"))).Id);
accountIds.Add((await db.Accounts.AddAsync(Make.Account($"account{a:00}.demo", $"{1000 + a}"), ct)).Id);
}
var words = new[] { "mare", "montagna", "città", "tramonto", "cena", "amici", "viaggio", "festa", "libro", "caffè" };
@@ -50,82 +61,71 @@ public sealed class LibraryScaleTests
media));
}
await db.Entries.AddManyAsync(items);
await db.Entries.AddManyAsync(items, ct);
}
fill.Stop();
TestContext.WriteLine($"Riempimento: {Entries:N0} elementi in {fill.Elapsed.TotalSeconds:F1} s");
// Plans differ with and without ANALYZE statistics: both must be fast.
await MeasureAll(db, accountIds, "senza statistiche");
await db.WriteAsync(w => w.Command("ANALYZE").ExecuteNonQuery(), default);
await MeasureAll(db, accountIds, "con ANALYZE");
// The fill stays outside the timings.
await MeasureAll(db, accountIds, "senza statistiche", ct);
await db.WriteAsync(w => w.Command("ANALYZE").ExecuteNonQuery(), ct);
await MeasureAll(db, accountIds, "con ANALYZE", ct);
}
private async Task MeasureAll(FebeDatabase db, List<long> accountIds, string label)
private async Task MeasureAll(FebeDatabase db, List<long> accountIds, string label, CancellationToken ct)
{
var times = new Dictionary<string, double>();
async Task<T> Measure<T>(string name, Func<Task<T>> query)
var times = new List<(string Name, double Bound, Timing<object?> Timing)>();
async Task<T> Measure<T>(string name, double bound, Func<Task<T>> query)
{
await query(); // warm the page cache
var best = double.MaxValue;
T result = default!;
for (var run = 0; run < 3; run++)
{
var watch = Stopwatch.StartNew();
result = await query();
best = Math.Min(best, watch.Elapsed.TotalMilliseconds);
}
times[name] = best;
return result;
var timing = await TestWait.BestTimeAsync(query, bound, ct: ct);
times.Add((name, bound, new Timing<object?>(timing.BestMs, timing.Runs, null)));
return timing.Result;
}
var first = await Measure("prima finestra", () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200 }));
var first = await Measure("prima finestra", IndexBoundMs, () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200 }));
Assert.HasCount(200, first.Tiles);
var middleCursor = LibraryCursor.At(new DateTimeOffset(2021, 6, 1, 0, 0, 0, TimeSpan.Zero));
var middle = await Measure("finestra a metà (salto a una data)", () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Cursor = middleCursor }));
var middle = await Measure("finestra a metà (salto a una data)", IndexBoundMs, () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Cursor = middleCursor }));
Assert.HasCount(200, middle.Tiles);
Assert.IsTrue(middle.Tiles.All(t => t.PublishedAt < middleCursor.PublishedAt));
await Measure("finestra più recente da metà", () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Cursor = middleCursor, Direction = LibraryDirection.Newer }));
await Measure("filtro Video", () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { Kind = LibraryKindFilter.Videos } }));
await Measure("filtro Storie", () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { Kind = LibraryKindFilter.Stories } }));
await Measure("filtro In evidenza", () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { Kind = LibraryKindFilter.Highlights } }));
await Measure("filtro un account", () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { AccountIds = [accountIds[3]] } }));
await Measure("filtro tre account", () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { AccountIds = [accountIds[3], accountIds[9], accountIds[30]] } }));
await Measure("filtro periodo e provenienza", () => db.Entries.QueryLibraryAsync(new LibraryQuery
await Measure("finestra più recente da metà", IndexBoundMs, () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Cursor = middleCursor, Direction = LibraryDirection.Newer }));
await Measure("filtro Video", IndexBoundMs, () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { Kind = LibraryKindFilter.Videos } }));
await Measure("filtro Storie", IndexBoundMs, () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { Kind = LibraryKindFilter.Stories } }));
await Measure("filtro In evidenza", IndexBoundMs, () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { Kind = LibraryKindFilter.Highlights } }));
await Measure("filtro un account", IndexBoundMs, () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { AccountIds = [accountIds[3]] } }));
await Measure("filtro tre account", IndexBoundMs, () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { AccountIds = [accountIds[3], accountIds[9], accountIds[30]] } }));
await Measure("filtro periodo e provenienza", IndexBoundMs, () => db.Entries.QueryLibraryAsync(new LibraryQuery
{
Limit = 200,
Filter = new LibraryFilter { From = new DateTimeOffset(2020, 1, 1, 0, 0, 0, TimeSpan.Zero), To = new DateTimeOffset(2020, 2, 1, 0, 0, 0, TimeSpan.Zero), Provenance = Provenance.Imported },
}));
var rare = await Measure("ricerca rara nella didascalia", () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { Search = "PAROLARARA" } }));
var rare = await Measure("ricerca rara nella didascalia", AggregateBoundMs, () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { Search = "PAROLARARA" } }));
Assert.HasCount(1, rare.Tiles);
await Measure("ricerca comune", () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { Search = "caffe" } }));
await Measure("ricerca per account", () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { Search = "account07" } }));
var totals = await Measure("totali", () => db.Entries.GetLibraryTotalsAsync(new LibraryFilter()));
await Measure("ricerca comune", AggregateBoundMs, () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { Search = "caffe" } }));
await Measure("ricerca per account", AggregateBoundMs, () => db.Entries.QueryLibraryAsync(new LibraryQuery { Limit = 200, Filter = new LibraryFilter { Search = "account07" } }));
var totals = await Measure("totali", AggregateBoundMs, () => db.Entries.GetLibraryTotalsAsync(new LibraryFilter()));
Assert.AreEqual(new LibraryTotals(Entries, Accounts), totals);
var days = await Measure("conteggi per giorno", () => db.Entries.CountLibraryByDayAsync(new LibraryFilter(), TestZones.Rome));
var days = await Measure("conteggi per giorno", AggregateBoundMs, () => db.Entries.CountLibraryByDayAsync(new LibraryFilter(), TestZones.Rome));
Assert.AreEqual(Entries, days.Sum(d => d.Count));
await Measure("è già noto? (pagina di 12)", async () => await db.Entries.GetKnownInstagramIdsAsync(accountIds[5], EntryKind.Post, Enumerable.Range(0, 12).Select(i => $"id{5 + (i * 40)}").ToList()));
await Measure("SHA-256 in tutta la libreria", () => db.Media.FindBySha256Async(Make.Sha(150_001)));
var summaries = await Measure("elenco account con conteggi", () => db.Accounts.ListSummariesAsync(new DateTimeOffset(2026, 10, 1, 0, 0, 0, TimeSpan.Zero)));
await Measure("è già noto? (pagina di 12)", IndexBoundMs, async () => await db.Entries.GetKnownInstagramIdsAsync(accountIds[5], EntryKind.Post, Enumerable.Range(0, 12).Select(i => $"id{5 + (i * 40)}").ToList()));
await Measure("SHA-256 in tutta la libreria", IndexBoundMs, () => db.Media.FindBySha256Async(Make.Sha(150_001)));
var summaries = await Measure("elenco account con conteggi", AggregateBoundMs, () => db.Accounts.ListSummariesAsync(new DateTimeOffset(2026, 10, 1, 0, 0, 0, TimeSpan.Zero)));
Assert.AreEqual(Entries, summaries.Sum(s => s.EntriesInLibrary));
await Measure("scheda account", () => db.Accounts.GetStatsAsync(accountIds[2]));
await Measure("coda degli scaricamenti", () => db.Media.ListPendingDownloadsAsync(50));
var hashes = await Measure("caricamento impronte", () => db.Media.LoadPerceptualHashesAsync());
await Measure("scheda account", AggregateBoundMs, () => db.Accounts.GetStatsAsync(accountIds[2]));
await Measure("coda degli scaricamenti", IndexBoundMs, () => db.Media.ListPendingDownloadsAsync(50));
var hashes = await Measure("caricamento impronte", AggregateBoundMs, () => db.Media.LoadPerceptualHashesAsync());
Assert.IsGreaterThan(150_000, hashes.Count);
foreach (var (name, ms) in times)
foreach (var (name, bound, timing) in times)
{
TestContext.WriteLine($"[{label}] {name,-40} {ms,8:F1} ms");
TestContext.WriteLine($"[{label}] {name,-40} {timing.BestMs,8:F1} ms (limite {bound:F0}, {timing.Runs} prove)");
}
// Bounds with room for a slower CI runner and for tests running in parallel; measured values are ~10× lower.
foreach (var (name, ms) in times)
foreach (var (name, bound, timing) in times)
{
var heavy = name.StartsWith("ricerca", StringComparison.Ordinal) || name.StartsWith("conteggi", StringComparison.Ordinal)
|| name.StartsWith("totali", StringComparison.Ordinal) || name.StartsWith("caricamento", StringComparison.Ordinal);
Assert.IsLessThan(heavy ? 1500.0 : 250.0, ms, $"[{label}] {name}");
Assert.IsLessThan(bound, timing.BestMs, $"[{label}] {name}");
}
}
}
+8 -5
View File
@@ -1,4 +1,3 @@
using System.Diagnostics;
using Febe.Core.Abstractions;
using Febe.Core.Domain;
using Febe.Data;
@@ -110,27 +109,31 @@ public sealed class SingleWriterTests
using var temp = new TempFolder();
await using var db = await TestDatabase.OpenAsync(temp);
await db.Settings.SetAsync("prima", "1");
using var inside = new ManualResetEventSlim(false);
using var gate = new ManualResetEventSlim(false);
// The write holds the writer and its transaction until the gate opens (at most 60 s, only a hang guard).
var longWrite = db.WriteAsync(
w =>
{
w.Command("INSERT INTO setting (key, value) VALUES ('durante', '1')").ExecuteNonQuery();
gate.Wait(TimeSpan.FromSeconds(10));
inside.Set();
gate.Wait(TimeSpan.FromSeconds(60));
},
default);
Assert.IsTrue(inside.Wait(TimeSpan.FromSeconds(60)), "La scrittura lunga non è partita.");
var watch = Stopwatch.StartNew();
// An ordering, not a time (#46): the reads end while the write still holds its uncommitted transaction.
var value = await db.Settings.GetAsync("prima");
var during = await db.Settings.GetAsync("durante");
watch.Stop();
Assert.IsFalse(longWrite.IsCompleted, "Le letture sono finite solo dopo la scrittura lunga.");
gate.Set();
await longWrite;
Assert.AreEqual("1", value);
Assert.IsNull(during, "Una lettura non vede una scrittura non ancora confermata.");
Assert.IsLessThan(2000L, watch.ElapsedMilliseconds);
Assert.AreEqual("1", await db.Settings.GetAsync("durante"));
}
[TestMethod]
+13 -6
View File
@@ -380,14 +380,21 @@ public sealed class DesktopLogicTests
Assert.AreEqual(DataFolderLockOutcome.HeldByAnother, DataFolderLock.TryAcquire(alias, out var second, TimeSpan.Zero));
Assert.IsNull(second);
// Held for a moment only (an antivirus looking at the file): the start waits instead of giving up.
var release = Task.Run(async () =>
// Held for a moment only (an antivirus looking at the file): the start waits instead of giving up. The release
// runs on a thread of its own, not on the thread pool, which the parallel tests may keep busy for seconds (#46);
// the patience only bounds a lock that is never released, and the wait ends as soon as the file is free.
var released = 0;
var releaser = new Thread(() =>
{
await Task.Delay(300, TestContext.CancellationToken);
Thread.Sleep(300);
Volatile.Write(ref released, 1);
first.Dispose();
}, TestContext.CancellationToken);
Assert.AreEqual(DataFolderLockOutcome.Acquired, DataFolderLock.TryAcquire(alias, out var later, TimeSpan.FromSeconds(10)));
release.Wait(TestContext.CancellationToken);
})
{ IsBackground = true };
releaser.Start();
Assert.AreEqual(DataFolderLockOutcome.Acquired, DataFolderLock.TryAcquire(alias, out var later, TimeSpan.FromSeconds(60)));
Assert.AreEqual(1, Volatile.Read(ref released), "The lock was taken only once the other one let it go.");
releaser.Join();
later!.Dispose();
// A Data folder where nothing can be written: the start-up checks explain it, the lock does not decide.
@@ -153,6 +153,7 @@ public sealed class WebView2CdpSessionTests
await using var session = new WebView2CdpSession(log);
session.Attach(devTools);
var bodies = new ConcurrentQueue<string>();
var bodyArrived = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
using var bad = session.On("Network.loadingFinished", _ => throw new InvalidOperationException(Secret));
// The pattern of ResponseTap: the handler queues the command, it never blocks.
@@ -160,13 +161,13 @@ public sealed class WebView2CdpSessionTests
{
var body = await session.SendAsync("Network.getResponseBody", new JsonObject { ["requestId"] = e.GetProperty("requestId").GetString() });
bodies.Enqueue(body.GetProperty("body").GetString()!);
bodyArrived.TrySetResult();
}));
devTools.Raise("Network.loadingFinished", """{"requestId":"7"}""");
await session.DrainAsync();
for (var i = 0; i < 50 && bodies.IsEmpty; i++)
{
await Task.Delay(20, TestContext.CancellationToken);
}
// The handler's Task.Run must get a pool thread: by condition, with a generous limit (#46).
await bodyArrived.Task.WaitAsync(TimeSpan.FromSeconds(30), TestContext.CancellationToken);
Assert.AreEqual("x", bodies.Single());
StringAssert.Contains(log.All, "Network.loadingFinished");
+18 -14
View File
@@ -99,22 +99,26 @@ public sealed class EngineMonitorTests
{
await using var bed = await EngineTestBed.CreateAsync();
var host = bed.Start(new ScriptedRunner(bed.Governor, bed.Clock));
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(20));
using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(60));
var received = new List<EngineNotification>();
var reading = Task.Run(async () =>
{
await foreach (var notification in host.Monitor.SubscribeAsync(cts.Token))
{
received.Add(notification);
if (notification.Kind == EngineNotificationKind.Event && notification.Event!.Code == EngineEventCodes.Paused)
{
break;
}
}
});
await Task.Delay(100);
// Subscribed on this thread before the pause: the first MoveNextAsync runs up to its wait and registers the
// subscriber, so no pool thread has to start in time for the event (#46).
await using var stream = host.Monitor.SubscribeAsync(cts.Token).GetAsyncEnumerator(cts.Token);
var next = stream.MoveNextAsync();
await host.PauseAsync();
await reading;
while (await next)
{
var notification = stream.Current;
received.Add(notification);
if (notification.Kind == EngineNotificationKind.Event && notification.Event!.Code == EngineEventCodes.Paused)
{
break;
}
next = stream.MoveNextAsync();
}
Assert.IsTrue(received.Any(n => n.Event?.Code == EngineEventCodes.Paused));
StringAssert.Contains(received.Last().Event!.Message, "Riprendi");
}
+4 -2
View File
@@ -110,8 +110,10 @@ public sealed class MediaReaderTests
foreach (var (name, bytes) in new[] { ("moov", insideMoov), ("primo livello", topLevel) })
{
var reading = Task.Run(() => Mp4Reader.Read(new MemoryStream(bytes), Now));
Assert.IsTrue(reading.Wait(TimeSpan.FromSeconds(10)), $"{name}: la lettura deve finire.");
// A thread of its own (not the pool, which the parallel tests may keep busy) and a generous limit: only an
// endless loop fails it (#46).
var reading = Task.Factory.StartNew(() => Mp4Reader.Read(new MemoryStream(bytes), Now), CancellationToken.None, TaskCreationOptions.LongRunning, TaskScheduler.Default);
Assert.IsTrue(reading.Wait(TimeSpan.FromSeconds(60)), $"{name}: la lettura deve finire.");
var info = reading.Result;
Assert.IsNull(info?.CreatedAt, name);
Assert.IsNull(info?.Width, name);
+15 -22
View File
@@ -37,7 +37,7 @@ public sealed partial class ServerEndToEndTests
}
[TestMethod]
[Timeout(360_000, CooperativeCancellation = true)]
[Timeout(540_000, CooperativeCancellation = true)]
public async Task TheRealServerCollectsFromTheTestSiteStopsAtTheVerificationAndShowsTheBrowser()
{
var ct = TestContext.CancellationToken;
@@ -46,7 +46,7 @@ public sealed partial class ServerEndToEndTests
await using var bed = await ServerTestBed.StartAsync(telegram: telegram, freeSpace: disk);
var server = bed.Server;
Assert.IsEmpty(server.Problems);
await server.Running.WaitAsync(TimeSpan.FromSeconds(60), ct);
await bed.WaitRunningAsync(ct);
var connection = server.Connection!;
// The health check answers without a session; everything else wants the password.
@@ -204,13 +204,13 @@ public sealed partial class ServerEndToEndTests
}
[TestMethod]
[Timeout(120_000, CooperativeCancellation = true)]
[Timeout(360_000, CooperativeCancellation = true)]
public async Task FinishWithoutSessionSaysSoAndDisconnectDeletesTheProfile()
{
var ct = TestContext.CancellationToken;
await using var bed = await ServerTestBed.StartAsync(autoLogin: true);
var server = bed.Server;
await server.Running.WaitAsync(TimeSpan.FromSeconds(60), ct);
await bed.WaitRunningAsync(ct);
var connection = server.Connection!;
Assert.AreEqual(ConnectionState.Connected, (await connection.GetStatusAsync(ct)).State, "the test site's automatic login");
@@ -222,7 +222,10 @@ public sealed partial class ServerEndToEndTests
Assert.IsTrue(server.Browser.IsAvailable, "the browser runs again, with an empty profile");
Assert.AreEqual(ConnectionState.NotConnected, (await connection.GetStatusAsync(ct)).State);
Assert.IsFalse(server.Browser.UserIsDriving);
Assert.IsTrue(server.Engine!.Services.Runtime.IsRunning || await WaitRunningAsync(server), "the engine loop runs again (and waits for a connection)");
await ServerTestBed.WaitUntilAsync(
() => Task.FromResult(server.Engine!.Services.Runtime.IsRunning),
TimeSpan.FromSeconds(60),
"il motore di nuovo in funzione, in attesa del collegamento");
// «Ho finito» without having logged in: Febe says it does not find the session.
await connection.BeginUserDrivingAsync(openLogin: false, ct);
@@ -243,13 +246,13 @@ public sealed partial class ServerEndToEndTests
}
[TestMethod]
[Timeout(120_000, CooperativeCancellation = true)]
[Timeout(420_000, CooperativeCancellation = true)]
public async Task TwoViewsOpenedAtTheSameMomentLeaveOneViewWithItsFramesAndOneLease()
{
var ct = TestContext.CancellationToken;
await using var bed = await ServerTestBed.StartAsync();
var server = bed.Server;
await server.Running.WaitAsync(TimeSpan.FromSeconds(60), ct);
await bed.WaitRunningAsync(ct);
var host = server.RemoteView!;
var gate = server.Browser!.UserDriving;
var released = 0;
@@ -268,7 +271,7 @@ public sealed partial class ServerEndToEndTests
// «about:blank» here means the engine's rest has committed.
await ServerTestBed.WaitUntilAsync(
async () => await Febe.Browser.Cdp.CdpPageExtensions.GetMainFrameUrlAsync(server.Browser.Cdp, ct) == "about:blank",
TimeSpan.FromSeconds(20),
TimeSpan.FromSeconds(60),
"la pagina di riposo");
// Two tabs restored together, or the phone and the PC at once: the newest wins, the other is ended — and the
@@ -302,13 +305,13 @@ public sealed partial class ServerEndToEndTests
}
[TestMethod]
[Timeout(180_000, CooperativeCancellation = true)]
[Timeout(480_000, CooperativeCancellation = true)]
public async Task AViewOpenedWhileThePageIsNavigatingOpensAndShowsItsFrames()
{
var ct = TestContext.CancellationToken;
await using var bed = await ServerTestBed.StartAsync();
var server = bed.Server;
await server.Running.WaitAsync(TimeSpan.FromSeconds(60), ct);
await bed.WaitRunningAsync(ct);
var host = server.RemoteView!;
var browser = server.Browser!;
@@ -318,7 +321,7 @@ public sealed partial class ServerEndToEndTests
// engine's rest after the last view has committed and nothing else navigates.
await ServerTestBed.WaitUntilAsync(
async () => await Febe.Browser.Cdp.CdpPageExtensions.GetMainFrameUrlAsync(browser.Cdp, ct) == "about:blank",
TimeSpan.FromSeconds(20),
TimeSpan.FromSeconds(60),
"la pagina di riposo");
// Even rounds: «Accedi a Instagram» on the Collegamento page navigates to the login page while the same page
@@ -353,7 +356,7 @@ public sealed partial class ServerEndToEndTests
private static async Task<bool> FrameArrivesAsync(IRemoteViewConnection view, CancellationToken ct)
{
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(ct);
timeout.CancelAfter(TimeSpan.FromSeconds(15));
timeout.CancelAfter(TimeSpan.FromSeconds(60)); // only a hang guard: it returns at the first frame
try
{
await foreach (var output in view.ReadAsync(timeout.Token))
@@ -371,16 +374,6 @@ public sealed partial class ServerEndToEndTests
return false;
}
private static async Task<bool> WaitRunningAsync(Febe.Server.FebeServer server)
{
for (var i = 0; i < 40 && !server.Engine!.Services.Runtime.IsRunning; i++)
{
await Task.Delay(100);
}
return server.Engine!.Services.Runtime.IsRunning;
}
private static async Task<List<(Entry Entry, Media Media)>> StoredAsync(Febe.Server.FebeServer server, long accountId, CancellationToken ct)
{
var list = new List<(Entry, Media)>();
@@ -9,8 +9,22 @@ namespace Febe.Tests.Server.Support;
internal sealed class FakeRemoteViewHost : IRemoteViewHost
{
private readonly Lock _gate = new();
private readonly List<FakeRemoteViewConnection> _opened = [];
public List<FakeRemoteViewConnection> Opened { get; } = [];
/// <summary>
/// The views opened so far, copied under the lock the server writes with: the tests read it from their own thread
/// while the server may be adding to it (#46).
/// </summary>
public IReadOnlyList<FakeRemoteViewConnection> Opened
{
get
{
lock (_gate)
{
return [.. _opened];
}
}
}
public bool Unavailable { get; set; }
@@ -24,12 +38,12 @@ internal sealed class FakeRemoteViewHost : IRemoteViewHost
var connection = new FakeRemoteViewConnection();
lock (_gate)
{
if (Opened.LastOrDefault() is { Disposed: false } previous)
if (_opened.LastOrDefault() is { Disposed: false } previous)
{
previous.End(RemoteViewEnd.Replaced);
}
Opened.Add(connection);
_opened.Add(connection);
}
return Task.FromResult<IRemoteViewConnection>(connection);
@@ -39,12 +53,24 @@ internal sealed class FakeRemoteViewHost : IRemoteViewHost
internal sealed class FakeRemoteViewConnection : IRemoteViewConnection
{
private readonly Channel<RemoteViewOutput> _output = Channel.CreateUnbounded<RemoteViewOutput>();
private volatile bool _disposed;
private volatile object? _ended;
public ConcurrentQueue<object> Inputs { get; } = new();
public bool Disposed { get; private set; }
/// <summary>Set by the server's thread, read by the test's: volatile.</summary>
public bool Disposed
{
get => _disposed;
private set => _disposed = value;
}
public RemoteViewEnd? Ended
{
get => (RemoteViewEnd?)_ended;
private set => _ended = value;
}
public RemoteViewEnd? Ended { get; private set; }
public void Write(RemoteViewOutput output) => _output.Writer.TryWrite(output);
@@ -163,7 +163,30 @@ internal sealed class ServerTestBed : IAsyncDisposable
public ThumbnailStore Thumbnails => new(Server.Data!.Thumbnails);
/// <summary>
/// The hang guard of <see cref="WaitRunningAsync"/>. A cold Chromium start on a loaded runner can take minutes,
/// with a retry after a failed attempt (#46); the wait ends as soon as Febe is up.
/// </summary>
public static readonly TimeSpan RunningPatience = TimeSpan.FromMinutes(4);
/// <summary>
/// Waits until Febe has started (Xvfb, Chromium, pipeline, connection, engine). A start that never ends fails with
/// the last error of the browser instead of a bare timeout.
/// </summary>
public async Task WaitRunningAsync(CancellationToken ct)
{
try
{
await Server.Running.WaitAsync(RunningPatience, ct);
}
catch (TimeoutException)
{
Assert.Fail($"Febe non è partito in {RunningPatience.TotalMinutes:0} minuti: {Server.Browser?.LastError ?? "nessun errore del browser"}.");
}
}
/// <summary>Waits (real time) until <paramref name="condition"/> holds; fails with <paramref name="what"/> otherwise.</summary>
public static async Task WaitUntilAsync(Func<Task<bool>> condition, TimeSpan timeout, string what)
{
var deadline = DateTime.UtcNow + timeout;
+19
View File
@@ -0,0 +1,19 @@
namespace Febe.Tests.Support;
/// <summary>Set-up of the whole test assembly.</summary>
[TestClass]
public static class TestAssembly
{
/// <summary>
/// The parallel tests (method level, see MSTestSettings.cs) block pool threads in several places: gates, barriers,
/// synchronous database work. On a runner with 2-4 cores the pool starts with as many threads as cores and adds more
/// only slowly, so a continuation another test waits for may start seconds late (#46). A larger minimum lets the
/// pool grow at once; it changes nothing in the code under test.
/// </summary>
[AssemblyInitialize]
public static void Initialize(TestContext context)
{
ThreadPool.GetMinThreads(out var workers, out var completion);
ThreadPool.SetMinThreads(Math.Max(workers, 64), Math.Max(completion, 64));
}
}
+67
View File
@@ -0,0 +1,67 @@
using System.Diagnostics;
namespace Febe.Tests.Support;
/// <summary>
/// Waits by condition and timings that a loaded runner does not fail by chance (#46). A wait returns as soon as its
/// condition holds; its limit only bounds a hang, so it is generous. A timing keeps the best of repeated runs: load only
/// adds time, so the minimum is the cost of the operation itself.
/// </summary>
internal static class TestWait
{
/// <summary>The default hang guard of a condition wait.</summary>
public static readonly TimeSpan Patience = TimeSpan.FromSeconds(30);
/// <summary>Polls <paramref name="condition"/> until it holds; fails with <paramref name="what"/> after the limit.</summary>
public static Task UntilAsync(Func<bool> condition, string what, TimeSpan? limit = null, CancellationToken ct = default) =>
UntilAsync(() => Task.FromResult(condition()), what, limit, ct);
/// <summary>Polls <paramref name="condition"/> until it holds; fails with <paramref name="what"/> after the limit.</summary>
public static async Task UntilAsync(Func<Task<bool>> condition, string what, TimeSpan? limit = null, CancellationToken ct = default)
{
var patience = limit ?? Patience;
var watch = Stopwatch.StartNew();
while (!await condition())
{
if (watch.Elapsed > patience)
{
Assert.Fail($"Tempo scaduto ({patience.TotalSeconds:0} s): {what}.");
}
await Task.Delay(20, ct);
}
}
/// <summary>
/// Times <paramref name="run"/> after one warm-up run, at least three times and then until the best time is under
/// <paramref name="boundMs"/> or the budget (<paramref name="maxRuns"/> runs, <paramref name="budget"/> of wall time)
/// runs out. A load spike only delays the result; an operation that is really slower stays over the bound.
/// </summary>
public static async Task<Timing<T>> BestTimeAsync<T>(
Func<Task<T>> run, double boundMs, int maxRuns = 15, TimeSpan? budget = null, CancellationToken ct = default)
{
var limit = budget ?? TimeSpan.FromSeconds(20);
var result = await run(); // warm the caches
var best = double.MaxValue;
var runs = 0;
var wall = Stopwatch.StartNew();
while (runs < 3 || (best >= boundMs && runs < maxRuns && wall.Elapsed < limit))
{
ct.ThrowIfCancellationRequested();
if (runs >= 3)
{
await Task.Delay(100, ct); // let a burst of load pass before trying again
}
var watch = Stopwatch.StartNew();
result = await run();
best = Math.Min(best, watch.Elapsed.TotalMilliseconds);
runs++;
}
return new Timing<T>(best, runs, result);
}
}
/// <summary>The best time of <see cref="TestWait.BestTimeAsync"/>, how many runs it took and the last result.</summary>
internal readonly record struct Timing<T>(double BestMs, int Runs, T Result);
+6 -1
View File
@@ -249,10 +249,15 @@ public sealed class InterfaceTests
await ui.OpenAsync(new Uri(server.Address, "/impostazioni/collegamento"));
await ui.WaitAsync("document.querySelector('.arresto') !== null", "striscia rossa della verifica");
await ui.WaitForAsync(() => connection.Snapshot().Contains("begin:False"), "verifica nella vista remota");
// The lease (a POST) and the view (a WebSocket on a new connection) start together; on a loaded machine the
// view comes later. «Ho finito» once both are there, as Alberto sees them, else it closes a socket still connecting.
await ui.WaitForAsync(() => connection.Snapshot().Contains("begin:False") && view.Opened.Count == 1, "verifica nella vista remota");
await ui.WaitAsync("window.__sockets.some((s) => s.url.endsWith('/api/vista') && s.readyState === WebSocket.OPEN)", "vista remota collegata");
await ui.EvalAsync("document.querySelector('.bottoni-collegamento .btn.primo').click()");
await ui.WaitAsync("document.querySelector('.riapri-browser') !== null", "in attesa di «Riprendi»");
// «Riprendi»: the engine works again and the session is there.
engine.Mode = EngineMode.Resting;
connection.Status = Connected;
+122 -28
View File
@@ -108,7 +108,15 @@ internal sealed class UiBrowser : IAsyncDisposable
})();
""";
/// <summary>
/// The hang guard of every wait of this browser: its start, each DevTools command, each condition. The waits end as
/// soon as their condition holds; the guard only bounds a hang, sized for a loaded runner (#46). A browser that dies
/// fails the wait at once.
/// </summary>
internal static readonly TimeSpan Patience = TimeSpan.FromMinutes(2);
private readonly Process _process;
private readonly ConcurrentQueue<string> _stderr;
private readonly ClientWebSocket _socket;
private readonly string _profile;
private readonly ConcurrentDictionary<int, TaskCompletionSource<JsonElement>> _pending = new();
@@ -117,9 +125,10 @@ internal sealed class UiBrowser : IAsyncDisposable
private readonly Task _reader;
private int _id;
private UiBrowser(Process process, ClientWebSocket socket, string profile)
private UiBrowser(Process process, ConcurrentQueue<string> stderr, ClientWebSocket socket, string profile)
{
_process = process;
_stderr = stderr;
_socket = socket;
_profile = profile;
_reader = Task.Run(ReadLoopAsync);
@@ -137,19 +146,20 @@ internal sealed class UiBrowser : IAsyncDisposable
}
var profile = Path.Combine(Path.GetTempPath(), "febe-interfaccia-" + Guid.NewGuid().ToString("N"));
var port = LocalTestSite.FreePort();
var info = new ProcessStartInfo(chromium)
{
UseShellExecute = false,
RedirectStandardOutput = true,
RedirectStandardError = true,
};
// Port 0: Chromium picks a free port and says which (stderr and DevToolsActivePort), so no other process can take
// it between a free-port probe and the bind. Only this test browser: Febe's own browser keeps a fixed port (§6.1).
foreach (var argument in new[]
{
"--headless", "--disable-gpu", "--hide-scrollbars", "--no-first-run", "--no-default-browser-check",
"--disable-extensions", "--disable-background-networking", "--disable-component-update", "--no-proxy-server",
"--mute-audio", $"--window-size={width},{height}", $"--user-data-dir={profile}",
$"--remote-debugging-port={port}", "--remote-debugging-address=127.0.0.1", "about:blank",
"--remote-debugging-port=0", "--remote-debugging-address=127.0.0.1", "about:blank",
})
{
info.ArgumentList.Add(argument);
@@ -173,29 +183,60 @@ internal sealed class UiBrowser : IAsyncDisposable
}
var process = Process.Start(info) ?? throw new InvalidOperationException("Chromium non parte.");
var stderr = new ConcurrentQueue<string>();
var listening = 0;
process.OutputDataReceived += (_, _) => { };
process.ErrorDataReceived += (_, _) => { };
process.ErrorDataReceived += (_, e) =>
{
if (e.Data is { } line)
{
if (ListeningPort(line) is > 0 and var found)
{
Interlocked.CompareExchange(ref listening, found, 0);
}
stderr.Enqueue(line);
while (stderr.Count > 20 && stderr.TryDequeue(out string? _))
{
}
}
};
process.BeginOutputReadLine();
process.BeginErrorReadLine();
try
{
using var http = new HttpClient(new HttpClientHandler { UseProxy = false }) { Timeout = TimeSpan.FromSeconds(2) };
// By condition: the port Chromium chose, then a page target. Fails at once if Chromium exits.
using var http = new HttpClient(new HttpClientHandler { UseProxy = false }) { Timeout = TimeSpan.FromSeconds(10) };
string? debugger = null;
var deadline = DateTime.UtcNow + TimeSpan.FromSeconds(20);
while (debugger is null && DateTime.UtcNow < deadline)
var watch = Stopwatch.StartNew();
while (debugger is null)
{
try
if (process.HasExited)
{
using var list = JsonDocument.Parse(await http.GetStringAsync(new Uri($"http://127.0.0.1:{port}/json/list")));
debugger = list.RootElement.EnumerateArray()
.Where(t => t.GetProperty("type").GetString() == "page")
.Select(t => t.GetProperty("webSocketDebuggerUrl").GetString())
.FirstOrDefault();
throw new InvalidOperationException($"Chromium si è chiuso all'avvio (codice {process.ExitCode}). {Tail(stderr)}");
}
catch (Exception ex) when (ex is HttpRequestException or TaskCanceledException or JsonException)
if (watch.Elapsed > Patience)
{
// Chromium is still starting.
throw new TimeoutException($"Chromium non risponde sulla porta di debug dopo {Patience.TotalSeconds:0} s. {Tail(stderr)}");
}
var port = Volatile.Read(ref listening) is > 0 and var known ? known : ReadActivePort(profile);
if (port > 0)
{
try
{
using var list = JsonDocument.Parse(await http.GetStringAsync(new Uri($"http://127.0.0.1:{port}/json/list")));
debugger = list.RootElement.EnumerateArray()
.Where(t => t.GetProperty("type").GetString() == "page")
.Select(t => t.GetProperty("webSocketDebuggerUrl").GetString())
.FirstOrDefault();
}
catch (Exception ex) when (ex is HttpRequestException or TaskCanceledException or JsonException)
{
// Chromium is still starting.
}
}
if (debugger is null)
@@ -204,15 +245,14 @@ internal sealed class UiBrowser : IAsyncDisposable
}
}
if (debugger is null)
{
throw new TimeoutException("Chromium non risponde sulla porta di debug.");
}
var socket = new ClientWebSocket();
socket.Options.Proxy = null;
await socket.ConnectAsync(new Uri(debugger), CancellationToken.None);
var browser = new UiBrowser(process, socket, profile);
using (var connect = new CancellationTokenSource(Patience))
{
await socket.ConnectAsync(new Uri(debugger), connect.Token);
}
var browser = new UiBrowser(process, stderr, socket, profile);
await browser.SendAsync("Runtime.enable");
await browser.SendAsync("Page.enable");
await browser.SendAsync("Log.enable");
@@ -243,7 +283,17 @@ internal sealed class UiBrowser : IAsyncDisposable
_send.Release();
}
return await answer.Task.WaitAsync(TimeSpan.FromSeconds(30));
// A browser that dies fails the answer at once (ReadLoopAsync); a slow one is waited for.
try
{
return await answer.Task.WaitAsync(Patience);
}
catch (TimeoutException)
{
_pending.TryRemove(id, out _);
throw new TimeoutException(
$"Chromium non ha risposto a {method} in {Patience.TotalSeconds:0} s{(_process.HasExited ? " (si è chiuso)" : string.Empty)}. {Tail(_stderr)}");
}
}
/// <summary>Runs a script in the page (promises awaited) and returns its value; an exception fails the test.</summary>
@@ -270,9 +320,9 @@ internal sealed class UiBrowser : IAsyncDisposable
/// Polls until a condition of the page holds (an expression, or a promise of one; errors while the page changes
/// count as «not yet»).
/// </summary>
public async Task WaitAsync(string condition, string what, int milliseconds = 30_000)
public async Task WaitAsync(string condition, string what, TimeSpan? limit = null)
{
var deadline = DateTime.UtcNow + TimeSpan.FromMilliseconds(milliseconds);
var deadline = DateTime.UtcNow + (limit ?? Patience);
string? last = null;
while (DateTime.UtcNow < deadline)
{
@@ -295,9 +345,9 @@ internal sealed class UiBrowser : IAsyncDisposable
}
/// <summary>Polls a condition of the test (the server side) while the page works.</summary>
public async Task WaitForAsync(Func<bool> condition, string what, int milliseconds = 30_000)
public async Task WaitForAsync(Func<bool> condition, string what, TimeSpan? limit = null)
{
var deadline = DateTime.UtcNow + TimeSpan.FromMilliseconds(milliseconds);
var deadline = DateTime.UtcNow + (limit ?? Patience);
while (DateTime.UtcNow < deadline)
{
if (condition())
@@ -333,7 +383,13 @@ internal sealed class UiBrowser : IAsyncDisposable
/// <summary>Loads an address of the interface and waits until the frame and the first page are drawn.</summary>
public async Task OpenAsync(Uri address)
{
await SendAsync("Page.navigate", new JsonObject { ["url"] = address.AbsoluteUri });
// A navigation that fails says so at once, instead of a wait for a page that never comes.
var navigation = await SendAsync("Page.navigate", new JsonObject { ["url"] = address.AbsoluteUri });
if (navigation.TryGetProperty("errorText", out var error) && error.GetString() is { Length: > 0 } text)
{
Assert.Fail($"apertura di {address}: {text}\n{Report()}");
}
await WaitAsync(
$"location.href.startsWith({JsonSerializer.Serialize(address.GetLeftPart(UriPartial.Authority))}) && document.readyState === 'complete' && !document.documentElement.hasAttribute('aria-busy') && document.querySelector('main.pagina .corpo') !== null",
$"apertura di {address}");
@@ -402,7 +458,45 @@ internal sealed class UiBrowser : IAsyncDisposable
RealBrowser.TryDelete(_profile);
}
/// <summary>The port of «DevTools listening on ws://127.0.0.1:&lt;port&gt;/devtools/browser/…», or 0.</summary>
private static int ListeningPort(string line)
{
const string Marker = "DevTools listening on ws://127.0.0.1:";
var at = line.IndexOf(Marker, StringComparison.Ordinal);
if (at < 0)
{
return 0;
}
var rest = line[(at + Marker.Length)..];
var end = rest.IndexOf('/', StringComparison.Ordinal);
return int.TryParse(end < 0 ? rest : rest[..end], out var port) ? port : 0;
}
/// <summary>The port Chromium wrote in DevToolsActivePort (first line), or 0 while it has not.</summary>
private static int ReadActivePort(string profile)
{
try
{
var lines = File.ReadAllLines(Path.Combine(profile, "DevToolsActivePort"));
return lines.Length > 0 && int.TryParse(lines[0], out var port) ? port : 0;
}
catch (IOException)
{
return 0;
}
catch (UnauthorizedAccessException)
{
return 0;
}
}
/// <summary>The last lines Chromium wrote on stderr, for failure messages.</summary>
private static string Tail(ConcurrentQueue<string> stderr) =>
stderr.IsEmpty ? "Chromium non ha scritto nulla su stderr." : "Ultime righe di Chromium: " + string.Join(" | ", stderr);
private static void Kill(Process process)
{
try
{