bitcrafts/vigil
1
0
Fork 0
Code Issues Pull requests Projects Releases 3 Packages Wiki Activity Actions

release/v1.0.0 #9

Merged
younes-benmoussa merged 38 commits from release/v1.0.0 into main 2026-06-12 22:37:45 +00:00
Conversation 0 Commits 38 Files changed 57 +6760 -1053
younes-benmoussa commented 2026-06-12 22:37:23 +00:00
Owner
Copy link
No description provided. 
A terminal verify/scan row write could fail (e.g. 'database is locked')
while the job still finished 'succeeded': job_recorder swallowed every
persistence exception by contract, and DbWriter dispatched on_done
unconditionally. For an integrity tool, showing 'no changes' for a run
that was never persisted is a correctness bug.

- job_recorder: row-write failures now propagate; only diff/ndjson
  sidecar writes keep degrading gracefully (the row stays the source
  of truth). record_hash_success raises on a missing hash_result
  instead of silently writing nothing.
- DbWriter: on_done now receives the command outcome (None on
  success, the raised exception otherwise).
- JobQueue._finalize: a failed write demotes a would-be 'succeeded'
  job to 'failed' with the cause in error_message (translated, fr
  catalog updated); already failed/cancelled jobs keep their state.

Covered by RED-first tests on both the writer path and the
synchronous fallback path, plus a DbWriter contract test.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
Spec-first for the 1.0 hardening: versioned, replayable, incremental
migrations on PRAGMA user_version with FLOOR_SCHEMA_VERSION = 40 as
the 1.x compatibility floor; one migration = one BEGIN IMMEDIATE
transaction with the version bump inside it; concurrency-safe across
the UI and DbWriter connections; distinct error model for pre-floor,
too-new and failed-migration databases; backup restore widened from
exact-match to the supported version range.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
Move the consolidated v40 schema (split into individual statements —
executescript commits implicitly and would break transactional
application) and the version errors out of db.py into a new
backend/migrations.py hosting the 1.x migration engine:

- PRAGMA user_version registry: FLOOR_SCHEMA_VERSION = 40 (the 1.0
  compatibility floor), MIGRATIONS born empty, CURRENT_SCHEMA_VERSION
  derived from the registry length so the two cannot drift.
- migrate_to_current(): fresh databases get the baseline in ONE
  BEGIN IMMEDIATE transaction (previously executescript, non-atomic);
  databases in [floor, current) replay one migration per transaction
  with the user_version bump inside it; pre-floor databases keep the
  reset-only error; databases newer than the build now get a distinct
  SchemaTooNewError raised before any write.
- Concurrency-safe across the UI and DbWriter connections: re-read of
  user_version under BEGIN IMMEDIATE, busy_timeout raised to 30 s and
  restored; foreign_keys toggled off outside transactions and
  compensated by foreign_key_check inside each one.

db.py re-exports the public names so main.py, preferences.py and the
existing tests keep working unchanged. Engine spec and slice plan in
docs/schema-migrations.md.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
Exercise the engine through its injection points (floor / migrations /
baseline) with synthetic registries: ordered round-trip replay,
fresh-vs-migrated schema equivalence, data preservation, atomic
rollback of a failing migration (earlier steps stay committed),
below-floor and too-new refusals (the latter proven write-free),
two-connection concurrent migration without double-application,
foreign_key_check enforcement, and pragma restoration.

Also pins the production schema: a frozen copy of the v40 baseline
('frozen at release 1.0 — never edit') replayed through the real
registry must equal a fresh database, which forbids editing
BASELINE_STATEMENTS without shipping a migration.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
A database created by a newer Vigil (SchemaTooNewError) now tells the
user to upgrade the application and explicitly NOT to delete the file;
a rolled-back migration (MigrationFailedError) reports the chained
cause and that the database was left unchanged. Both exit 1, like the
existing pre-floor reset message.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
stage_restore/restore_backup move from exact-match on schema_version
to a supported range [minimum_schema_version, expected]: a 1.0 backup
(v40) stays restorable by every later 1.x build — Database.reopen()
migrates the swapped-in file right after the swap. Backups outside
the range (pre-floor or from a newer build) are still rejected with
BackupSchemaVersionMismatch, now carrying the minimum.

PreferencesService passes FLOOR_SCHEMA_VERSION; at 1.0 floor ==
current, so behaviour is unchanged in practice. Default minimum=None
keeps exact-match for any other caller.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
A3 hardening: the unbounded history reads (list_verifies, list_scans,
list_other_verifies) move off the GTK main loop through the existing
DbWriter FIFO (read-your-writes for free), with lifecycle guards and
a minimal loading state. Decision recorded: R1 stays closed — the gi
facade of job_queue.py is its documented design, and none of A3's
problem lives there.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
_start_load fetched the entire verify history (list_verifies, no
LIMIT) on the GTK main loop to read one row's diff_path; replace with
the existing get_verify_row PK lookup (docs/async-history-reads.md,
slice T1).

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
submit_read(query, on_result, on_error) runs the query on the existing
writer thread and delivers its value (or the exception) on the main
loop through the injected dispatch. Built on submit(): no worker
changes. Sharing the single FIFO gives read-your-writes against job
persistence for free, and the worker's autocommit WAL connection sees
UI-connection commits (set-baseline / import flows).

JobQueue exposes the shared instance as a read-only db_writer property
for the history pages (docs/async-history-reads.md, slice T2).

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
_rebuild_rows splits into a submit half (captures target id and the
frozen HistoryFilterState by value) and a render half; with a reader
(the JobQueue DbWriter, resolved by the new
_utils.get_app_db_writer), list_verifies + history_filter.apply run
on the writer FIFO off the GTK loop and the result is delivered via
GLib.idle_add. 'Compare with…' offloads list_other_verifies the same
way. Without a reader the page keeps the synchronous path, so
existing standalone/test construction is unchanged.

Lifecycle guards: a destroy-flag plus a per-reload token drop results
arriving for a disposed page or superseded by a newer reload; row
detaching and _verify_data.clear() moved to render time so in-flight
reads never strand the kebab actions. Read errors toast and fall back
to the empty state. The stack gains a 'loading' page (Adw.Spinner,
a11y-labelled) shown on the initial async load only (anti-flash on
re-reads). fr catalog updated.

docs/async-history-reads.md, slice T3.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
Same contract as TargetHistoryPage (slice T3): reader kwarg resolved
through get_app_db_writer in the tool-page factories, _rebuild_rows
split into submit/render halves with destroy-flag and reload-token
guards, list_verifies / list_scans offloaded to the writer FIFO,
'loading' spinner page on initial async load, toast + empty state on
read errors, synchronous fallback without a reader.

docs/async-history-reads.md, slice T4.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
The V3 size cap used stat(), which follows symlinks and reports 0
bytes for a FIFO: a symlinked or non-regular source could bypass the
cap, and read_text on a FIFO with no writer blocks the import forever
(reproduced — the RED test hung). Check os.lstat first and refuse
anything but a regular file, before the size cap and the read.

Covered by tests: direct symlink, symlink to a FIFO, and a FIFO
source.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
- po/en.po (msgen: msgstr == msgid) + 'en' in LINGUAS: pins the source
  language explicitly instead of relying on the gettext fallback;
  msgfmt --check-format runs on it at build time like fr.
- _i18n._pick_localedir: accept a localedir shipping any language's
  catalog instead of probing hardcoded 'fr'.
- hash_page: three user-facing strings escaped gettext ('{first} (+{n}
  more)' — msgid already existed, 'never run', 'unknown'); fr
  translations added.
- job_queue_page: state labels move to the call-time gettext pattern
  (module-level dict froze them at import; same convention as
  integrity_history_page).
- POT + fr.po regenerated (line-number churn); fr stays 0 fuzzy /
  0 untranslated.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
A hash run against a nonexistent path exits 1, which HashRunner
classifies as partial success; the empty manifest then failed parsing
and the user saw the internal 'manifest missing summary' instead of
the tool's own diagnostic (reproduced against the real binary:
'bchash: stat: …: not found'). Drain stderr in that branch and emit
the same '[symbolic] exit code N: stderr' format as the
error-classified exits, falling back to the parser message only when
stderr is empty.

The analogous parse paths of duplicate/integrity are only reachable
on their partial-success exits and were not reproduced; left
unchanged.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
Audit of the A2 contract found two leftovers:

- DuplicateScanRunner swallowed archive_scan failures (the call that
  writes the scan row) and emitted 'finished' anyway, on both the
  normal and the empty-scan paths — the job ended succeeded with no
  row. The runner now emits 'error' with the cause, routing the job
  through the failed path (RED-first against the real bcduplicate).
- A DbWriter worker that failed to OPEN its Database died before the
  loop: queued commands never ran, no on_done fired, jobs froze in
  'running' and flush() burned its full timeout. The worker now drains
  the queue in a degraded loop, failing every command through on_done
  with the open error (the facade demotes the job), and the flush
  sentinel keeps working.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>

# Conflicts:
#	tests/test_db_writer.py
A fault between BEGIN IMMEDIATE and the per-migration try (e.g. the
version re-read under the write lock) leaked an open transaction: the
write lock stayed held and the finally-restored pragmas became silent
no-ops, leaving the connection with foreign_keys OFF. Wrap the whole
post-BEGIN body so any escaping exception rolls back first
(fault-injection test via a delegating flaky connection).

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
Manual QA on a whole-home hash target: the manifest only grows when a
file COMPLETES, so during a multi-gigabyte file the job row sat frozen
on 'Running · N minutes ago' with no way to tell a working runner from
a stuck one (the tool had read 84 GiB). The notifiable progress_bytes
property also had no consumer in the UI.

The 500 ms progress poller now reads the child's cumulative read
volume (/proc/<pid>/io rchar — Linux-only like the rest of the app)
and emits it as bytes_read alongside bytes_written; Job/JobItem gain
progress_read; the job-queue row shows 'Running · 84.0 GiB read · …'
bound to notify::progress-read. Works for all three tools, including
the duplicate scanner whose output file is only written at the end.
fr catalog updated (en refresh deferred to the final POT regen).

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
The previous commit accidentally swept in dist/*.deb (built during the
release validation) and a working-tree deletion of the 256x256 app
icon caused by packaging/build-deb.sh side effects. Restore the icon,
untrack dist/ and gitignore it.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
submit() read _closed and enqueued outside the lock: a racing close()
could slot the item behind the stop sentinel and its on_done would
never fire. submit now starts the thread and enqueues under the lock
(absorbing the dead _ensure_thread), and _stop_thread enqueues the
sentinel under the same lock so no item can interleave with the
shutdown handoff.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
- The three history pages gain an 'error' stack page (StatusPage,
  reuses the existing 'Could not load history.' string): falling back
  to 'empty' falsely claimed there was no history when the load
  failed.
- 'Compare with…' gets the same per-activation token guard as
  _rebuild_rows, so a rapid double click cannot present two dialogs.
- Remove JobQueue._record_terminal_verify: production persistence goes
  through _finalize/job_recorder; the two tests using it as a shim now
  call record_hash_success directly.
- _utils docstring no longer claims widget-free purity
  (get_app_db_writer/route_toast/attach_responsive_clamp).

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
New locale-independent kinds alongside hash/integrity/no-changes:
baseline {n}, imported {n}, empty, capture-failed, verify-failed,
compare-failed and kernel-gate, each rendered through existing msgids
(one new: 'Requires Linux 5.6 or newer'). is_kernel_gate_verdict
recognises the encoded form and keeps the legacy substring sentinel
for pre-encoding rows. Unknown kinds and legacy prose still pass
through unchanged. Nothing emits the new kinds yet.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
integrity history and changes pages delegate kernel-gate detection to
verdict.is_kernel_gate_verdict (encoded form + legacy substring
sentinel), and the changes page renders failure descriptions through
render_verdict so encoded kinds display translated while legacy prose
passes through. Readers understand the encoded form BEFORE the writers
start emitting it (next slice). Drops the unused _FAILURE_KINDS tuple.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
The remaining writers move from raw English prose or
translated-at-persistence strings to the encoded kinds: hash/integrity
baselines and the empty result (job_recorder), the imported baseline
(external_manifest), and the integrity failure verdicts including the
kernel gate (job_queue — the local 'verdict' variable shadowing the
module is renamed, which UnboundLocalError'd any encode call there).
record_integrity_verify_failure's parameter is renamed encoded_verdict
to state the contract. In-memory job.verdict_summary keeps the
translated rendering (A2 display pattern, unchanged).

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
The framing of tool failure messages (failed to spawn, exit code N,
cannot read scan output, malformed output, manifest missing summary,
integrity manifest missing trailer, scan completed but saving failed)
now goes through gettext; the tool's stderr fragments and the symbolic
exit-code names stay raw by design. A shared _exit_error_message
helper unifies the '[symbolic] exit code N: stderr' shape — the
_CommunicateRunner error path previously omitted the exit-code part
and now matches the other runners. tools.py joins POTFILES.in; fr
strings land with the final POT regeneration.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
The metainfo was installed verbatim: summary, description and release
notes stayed English in software centers even with a French session
(while 0.4.0's own notes announce the French translation). Move it to
.metainfo.xml.in + i18n.merge_file(type: 'xml') like the .desktop, add
it to POTFILES.in, and keep a configure_file copy fallback when msgfmt
is absent. test_version_consistency reads the .in source. French
strings land with the final POT regeneration.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
POT regenerated over the full hardening surface (metainfo, runner
error framings, job liveness, A2/A3 strings). The metainfo's inline
xml:lang=fr entries move into fr.po (merge_file injects them back at
build time, verified on the merged output). fr: 421 messages, zero
untranslated, zero fuzzy; en regenerated with msgen (msgstr == msgid).

Validation: ruff + mypy + 945 tests + 3 slow benchmarks green, fresh
meson builds in both trees, appstreamcli clean offline, real-tools QA
script re-run (17 pass, the known upstream-semantics expectation
aside).

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
Bump every declared version site (meson.build, pyproject.toml,
vigil.__version__, AppStream metainfo — test_version_consistency
stays green; the About dialog derives from __version__), add the
1.0.0 release entry summarising the hardening work, and introduce
CHANGELOG.md.

Also point the metainfo bugtracker/vcs-browser URLs at the Forgejo
repo — they referenced a nonexistent GitHub project, failing
appstreamcli validate (now clean offline; only a pedantic hint
remains).

The .deb builds as bitcrafts-vigil_1.0.0_amd64.deb (version assert
green). NOT tagged — the Forgejo tag stays the release trigger and
requires the hardening branches to land first.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
CHANGELOG and metainfo release notes gain the post-audit items (live
read progress, locale-independent persisted verdicts, translatable
runner errors and metainfo, error page for failed history loads);
catalogs resynced — fr 428 messages, zero untranslated/fuzzy, en
mirrored. .deb rebuilt green with the version assert. Still no tag.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
test_e2e_writer_reopens_after_reset called wipe_all_data() without the
tmp_xdg fixture: its database was isolated (tmp_db) but
manifest_paths.archive_dirs() resolved from the live environment, so
every full-suite run emptied the developer's real
~/.local/share/bitcrafts-vigil/{hash,duplicate,integrity} while the
real DB kept referencing the deleted manifests — surfacing later as
'cannot read manifest' verification failures in the app. The test is
skipif'd without the real bchash, so CI never saw it.

Defense in depth: an autouse fixture now redirects all XDG vars (and
XDG_RUNTIME_DIR) to a per-test temp dir for EVERY test, the leaking
test also takes tmp_xdg explicitly, and a sentinel module asserts
archive_dirs()/database_path() never resolve into the real home.
Verified: a marker file in the real archive dir survives the full
suite.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
feat(ui): full error diagnostic in the failed job row tooltip
All checks were successful
pr / lint (pull_request) Successful in 5s
Details
pr / build-test (pull_request) Successful in 11s
Details
258a65fa60
 
The status label ellipsizes and the job queue was the only surface
showing a failure's cause — a long tool diagnostic ('bcintegrity:
cannot read manifest /long/path…') was unreadable. Failed rows now
carry the complete error_message as the label tooltip (cleared on row
recycling). Persisting failure details for the history pages needs a
verifies.error_message column — first real 1.x schema migration,
deliberately not taken here.

Signed-off-by: Younes Benmoussa <younes.benmoussa@pm.me>
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
bitcrafts/vigil!9
No description provided.