Harden archives, snapshots, locks, and search against corruption and races #26
Assignees
Tyler (Tyler Westbrook)
Clear assignees
Tyler (Tyler Westbrook)
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: Tyler/StepForge#26
Reference in New Issue
Block a user
Phase 1/2 of the improvement plan, PR 6. Mirrors GitHub PR Twest2/StepForge#10.
Verified: 255 unit tests incl. new recovery suite; startup smoke + workflow E2E pass.
Phase 1/2 of the improvement plan (PR 6 of the sequence). Recovery and resource-limit hardening for the storage-adjacent modules. ZIP resource limits (core/zip.js): - unzipSync now enforces entry-count, total compressed, total inflated, and per-entry inflated budgets, and caps inflation with inflateRawSync maxOutputLength so a deflate bomb can't exhaust memory. Exact inflated-size match (not "at least") and CRC verification are kept. Import uses the default limits. Transactional archive import (core/archive.js): - The import validates the guide AND every step before writing anything, then stages the whole guide in a temp directory and publishes it with a single atomic rename. A corrupt step no longer leaves a partial guide in the library; a failure cleans up the staging directory. Atomic snapshot restore (core/snapshots.js): - Restore extracts and validates into a temp directory first; only then does it swap content in, moving live content aside so a mid-swap failure rolls back. A corrupt/truncated snapshot can no longer destroy the live guide (the old restore deleted live content before extracting). - Fixed snapshot filename collisions: names kept milliseconds so two backups in the same second no longer overwrite each other. Automatic backups (core/snapshots.js): - Implemented the previously-dead backups.automatic/everyNSaves/keepLast settings: autoSnapshotIfDue snapshots every N saves and prunes to keepLast, wired into the save choke point in main.js. Never throws — a backup failure cannot break the save that triggered it. Exclusive locks (core/locks.js): - acquireLock uses O_CREAT|O_EXCL (flag 'wx') so only one writer wins the race; the old read-then-write left a window where two writers both believed they held the lock. Added a per-acquisition token so release only removes the exact lock it took (never one a force-steal replaced). Same-process re-acquire still succeeds; cross-process fresh locks conflict. Search reconciliation (core/search.js): - New reconcile(store) rebuilds/repairs the index against the library at startup using per-guide fingerprints (updatedAt+revision): reindexes new/ changed guides, drops entries for deleted ones, and exposes a recovery status ('ok'|'reset'|'reconciled'). A missing/corrupt/version-mismatched index recovers instead of silently returning nothing. Wired into startup. Recovery surface: - New recovery:status IPC + preload method returns quarantined files (this session) and the search index status so the UI can surface data issues. Tests: ZIP bomb/limits, transactional import abort with no partial guide, atomic snapshot restore preserving the live guide on corruption, exclusive lock conflict/steal/release-by-token, same-process re-acquire, search reconcile (rebuild/drop/reindex/corrupt-reset), and automatic backup cadence/pruning. 255 unit tests pass; startup smoke and workflow E2E pass. Co-Authored-By: Claude Fable 5 <[email protected]>Merged into main as
3707930(same-SHA merge to both forges; GitHub mirror PR Twest2/StepForge#10 MERGED, CI green ubuntu+windows). Closing.Pull request closed