Harden archives, snapshots, locks, and search against corruption and races
Template tests / tests (pull_request) Failing after 33s

Phase 1/2 of the improvement plan (PR 6 of the sequence). Recovery and
resource-limit hardening for the storage-adjacent modules.

ZIP resource limits (core/zip.js):
- unzipSync now enforces entry-count, total compressed, total inflated, and
  per-entry inflated budgets, and caps inflation with inflateRawSync
  maxOutputLength so a deflate bomb can't exhaust memory. Exact inflated-size
  match (not "at least") and CRC verification are kept. Import uses the
  default limits.

Transactional archive import (core/archive.js):
- The import validates the guide AND every step before writing anything, then
  stages the whole guide in a temp directory and publishes it with a single
  atomic rename. A corrupt step no longer leaves a partial guide in the
  library; a failure cleans up the staging directory.

Atomic snapshot restore (core/snapshots.js):
- Restore extracts and validates into a temp directory first; only then does
  it swap content in, moving live content aside so a mid-swap failure rolls
  back. A corrupt/truncated snapshot can no longer destroy the live guide
  (the old restore deleted live content before extracting).
- Fixed snapshot filename collisions: names kept milliseconds so two backups
  in the same second no longer overwrite each other.

Automatic backups (core/snapshots.js):
- Implemented the previously-dead backups.automatic/everyNSaves/keepLast
  settings: autoSnapshotIfDue snapshots every N saves and prunes to keepLast,
  wired into the save choke point in main.js. Never throws — a backup failure
  cannot break the save that triggered it.

Exclusive locks (core/locks.js):
- acquireLock uses O_CREAT|O_EXCL (flag 'wx') so only one writer wins the
  race; the old read-then-write left a window where two writers both believed
  they held the lock. Added a per-acquisition token so release only removes
  the exact lock it took (never one a force-steal replaced). Same-process
  re-acquire still succeeds; cross-process fresh locks conflict.

Search reconciliation (core/search.js):
- New reconcile(store) rebuilds/repairs the index against the library at
  startup using per-guide fingerprints (updatedAt+revision): reindexes new/
  changed guides, drops entries for deleted ones, and exposes a recovery
  status ('ok'|'reset'|'reconciled'). A missing/corrupt/version-mismatched
  index recovers instead of silently returning nothing. Wired into startup.

Recovery surface:
- New recovery:status IPC + preload method returns quarantined files (this
  session) and the search index status so the UI can surface data issues.

Tests: ZIP bomb/limits, transactional import abort with no partial guide,
atomic snapshot restore preserving the live guide on corruption, exclusive
lock conflict/steal/release-by-token, same-process re-acquire, search
reconcile (rebuild/drop/reindex/corrupt-reset), and automatic backup
cadence/pruning. 255 unit tests pass; startup smoke and workflow E2E pass.

Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
2026-07-03 23:10:21 -05:00
co-authored by Claude Fable 5
parent f31f1407a5
commit 8aa9756b8a
8 changed files with 622 additions and 41 deletions
+22 -1
View File
@@ -17,7 +17,7 @@ const { buildRenderAst } = require('../core/renderast');
const { runExport, EXPORTERS } = require('../exporters');
const { runExportInWorker } = require('./export-runner');
const { exportGuideArchive, importGuideArchive, saveLinkedGuide } = require('../core/archive');
const { createSnapshot, listSnapshots, restoreSnapshot } = require('../core/snapshots');
const { createSnapshot, listSnapshots, restoreSnapshot, autoSnapshotIfDue } = require('../core/snapshots');
const { readLock } = require('../core/locks');
const CaptureService = require('./capture');
const { TextIntelService } = require('./text-intel');
@@ -72,6 +72,9 @@ function reindex(guideId) {
} catch {
// index failures must never block saves
}
// Automatic backup policy runs on the same save choke point. It is
// self-contained and never throws, so it can't affect the save either.
autoSnapshotIfDue(store, guideId, settings);
}
function orderedSteps(guideId) {
@@ -734,6 +737,13 @@ function setupIpc() {
return guide;
}, { validate: (a) => c.id(a.guideId) && c.fileName(a.name) });
// recovery status: corrupt files quarantined this session and search index
// health, so the UI can surface them instead of data silently vanishing.
h('recovery:status', () => ({
quarantined: store.getRecoveryReport(),
searchStatus: searchIndex.status,
}));
// templates
const validFormat = (v) => c.oneOf(v, FORMATS);
h('templates:list', ({ format }) => templates.list(format),
@@ -916,6 +926,17 @@ if (!gotLock) {
store = new GuideStore(dataDir);
settings = new Settings(store.settingsDir);
searchIndex = new SearchIndex(store.indexDir);
// Rebuild/reconcile the index against the library at startup so a missing,
// corrupt, or version-mismatched index recovers instead of silently
// returning nothing.
try {
const summary = searchIndex.reconcile(store);
if (summary.reindexed || summary.removed || summary.status !== 'ok') {
console.log(`[stepforge] search index reconciled: ${JSON.stringify(summary)}`);
}
} catch (err) {
console.error(`[stepforge] search reconcile failed: ${err && err.message}`);
}
templates = new TemplateManager(store.templatesDir);
textIntel = new TextIntelService({
store,