Phase 3 of the improvement plan (PR 10 of the sequence): honest Wayland
behavior and a least-privilege alternative to the broad `input` group the
audit flagged as a keylogging surface.
Least-privilege input access:
- packaging/linux/common/60-stepforge-input.rules grants the ACTIVE session
read access to MOUSE devices only (ID_INPUT_MOUSE, excluding
ID_INPUT_KEYBOARD) via a systemd uaccess ACL — session-scoped, device-
scoped, and never keyboards. This replaces `usermod -aG input`, which grants
every input device (keyboards included) to the user permanently.
- scripts/linux/enable-click-capture.sh installs it opt-in: it prints the exact
rule, requires confirmation, and documents the security tradeoff. It never
runs the broad-group command.
Honest trigger reporting:
- New chooseCaptureTrigger() (app/platform/linux/diagnostics.js, re-exported
from app/platform/index.js) maps a capability profile to the real trigger:
per-click with a marker on X11+xinput; per-click WITHOUT coordinates or a
marker on Wayland evdev (the platform exposes no pointer position); and an
honest hotkey/interval fallback otherwise. It never promises per-click
capture with coordinates on Wayland.
- platform:capabilities now includes the active trigger for the machine and
the user's fallback setting, for the diagnostics UI.
Docs:
- GETTING_STARTED_WITH_LINUX.md no longer instructs every Wayland user to join
the `input` group; it presents that as a warning, documents the least-
privilege script, corrects the capability table (per-click is opt-in, mice
only; hotkey/interval is the default Wayland trigger), and points at
Settings → Diagnostics.
Tests: trigger decisions for X11/xinput, Wayland evdev (no coordinates/marker),
Wayland fallback (interval/hotkey with an honest note), the platform facade
wiring, Windows; the udev rule (mouse-only, excludes keyboards, uaccess); the
opt-in enable script (confirms, installs the rule, never runs usermod -aG
input as a command); and docs guards. 289 unit tests pass; startup smoke and
click self-test unchanged (stream, markers 3/3, burst 8/8).
Co-Authored-By: Claude Fable 5 <[email protected]>
Phase 3 of the improvement plan (PR 9 of the sequence): the dnf/Fedora
packaging half, in separate Linux-specific files, mirroring the apt/.deb work.
- packaging/linux/common/stage-runtime.sh: shared runtime-only payload staging
extracted from the .deb builder so the two package formats never drift. It
copies app code + a fixed Electron runtime + production npm deps (npm ls
--omit=dev), guards against dev-dep leaks, and fails without node_modules.
The .deb builder now delegates to it.
- packaging/linux/fedora/stepforge.spec: runtime Requires (nss, nspr, gtk3,
…), Recommends (xinput, portal, pipewire), %license, and a %post that makes
chrome-sandbox setuid and refreshes desktop/MIME/icon caches. No compilation
— it packages the prebuilt BuildRoot.
- packaging/linux/fedora/package.sh: stages the shared payload, substitutes
version/maintainer into the spec, and runs rpmbuild against the prebuilt
BuildRoot with detected arch. Requires rpmbuild + node_modules; emits an
.rpm + sha256.
- scripts/linux/dnf/install-runtime-deps.sh and install-build-deps.sh:
separate runtime vs build dependency sets for dnf (runtime installs no build
tools).
- docs/linux/dnf.md: Fedora/RHEL install + build guide, distinct from apt.md.
Tests: packaging-linux.test.js gains Fedora/dnf structural checks (spec
Requires + MPL-2.0 + %license + sandbox setup, builder shares staging +
requires rpmbuild, dnf build/runtime dep separation, shared staging never
copies the whole dev tree). tests/integration/linux/package-rpm.test.sh builds
a real .rpm and asserts runtime-only contents (honest skip when rpmbuild is
absent, as on this apt host).
Verified: 13 packaging unit tests pass; the refactored .deb builder still
produces a valid runtime-only package (integration test green); the .rpm
integration test skips cleanly where rpmbuild is unavailable.
Co-Authored-By: Claude Fable 5 <[email protected]>
Phase 3 of the improvement plan (PR 8 of the sequence): the apt/X11 packaging
half of Linux support, in separate Linux-specific files. Replaces the old
scripts/package-linux.sh, which the audit flagged as "not production
packaging" (it copied the whole dev node_modules — including vulnerable build
deps — plus docs/prompts/examples/audit files, hardcoded amd64, declared only
xinput, lacked desktop/icon/MIME integration, and could build without
node_modules).
Production builder (packaging/linux/debian/package.sh):
- Stages ONLY runtime files: app code, a fixed Electron runtime, and the
production npm deps (enumerated via npm ls --omit=dev). Never copies the
development node_modules; guards against electron-builder/app-builder-lib
leaking in. Fails if node_modules is absent instead of shipping an unusable
artifact.
- Detects architecture (dpkg --print-architecture, x64/arm64) rather than
hardcoding amd64. Generates DEBIAN/control from control.in with proper
runtime Depends, real maintainer, and homepage.
- Installs a desktop entry, hicolor icons (16–512), .sfgz/.sfglt MIME
registration, the launcher, and the license. postinst makes chrome-sandbox
setuid and refreshes desktop/MIME/icon caches; postrm cleans them.
- Emits a .deb, a portable tarball that now INCLUDES /usr/bin/stepforge (the
old tarball omitted it), and a sha256 sums file.
Launcher (packaging/linux/common/launcher.sh):
- Runs sandboxed; prefers the user-namespace sandbox, accepts a root-owned
setuid helper, and otherwise refuses to launch with an actionable message.
--no-sandbox requires an explicit STEPFORGE_ALLOW_NO_SANDBOX opt-in. Never
installs anything at runtime.
Setup (separate build vs runtime, apt only):
- scripts/linux/apt/install-runtime-deps.sh (Chromium/Electron libs, X11
tools, portal/PipeWire) and install-build-deps.sh (dpkg-dev, fakeroot,
xvfb). Runtime script installs no build tools.
Assets: original StepForge icon — packaging/assets/stepforge.svg plus a
generator (scripts/make-icons.js) that renders the PNG set with the repo's own
rasterizer/PNG writer (no third-party art). npm run icons regenerates them.
Wiring: package.json gains package:linux:deb / package:linux:rpm / icons;
build-release.sh uses the production builder and requires node_modules; README
points at the apt/dnf guides.
Tests: tests/unit/packaging-linux.test.js (structural: files present in their
separate locations, old script gone, valid desktop entry, templated arch +
runtime Depends, launcher gates --no-sandbox, builder requires node_modules
and guards dev-dep leaks, apt build/runtime dep separation, original icon set
generates a valid PNG) runs in the normal suite;
tests/integration/linux/package-deb.test.sh builds a real .deb and asserts the
right files present and the dev tree / build tooling / app docs absent
(honest skip only when dpkg-deb/node_modules are genuinely missing).
Verified locally: 276 unit tests pass; the integration test builds and
validates stepforge_0.3.2_amd64.deb; build-release E2E passes with the new
production package.
Co-Authored-By: Claude Fable 5 <[email protected]>
Phase 1 of the improvement plan (PR 3 of the sequence). The docs claimed
"fully offline"/"never talks to the network," but text-intel makes HTTP
requests to a configurable Ollama host that could be remote, with no timeout,
no cancellation, and no size limit; the Windows hook logged raw keystrokes
into capture metadata that could then be sent to that host.
Privacy — raw keystroke capture:
- New capture.captureTypedText setting, default false. With it off, printable
characters are never buffered in JS and the Windows keyboard hook never even
emits them across the process boundary (the flag is threaded into the C#).
Shortcut/navigation detection (Ctrl+T, Enter, …) is unaffected.
AI network hardening (app/text-intel.js):
- Every Ollama call goes through fetchJson with an AbortController deadline
(ai.timeoutMs, default 60s): a dead endpoint fails fast instead of leaving
UI actions pending forever.
- Cancellation: in-flight requests are tracked and cancelInflight(guideId)
aborts them; new ai:cancel IPC + api.ai.cancel are called when the editor
closes, and shutdown cancels everything.
- Bounded concurrency (2) for AI network work.
- Screenshots are only attached when allowed (ai.attachScreenshots), the model
is vision-capable, and the image is within ai.maxImageBytes — no more
unbounded base64-expanded 4K bodies.
Local-first host policy (core/text-intel.js):
- New isLoopbackHost + validateOllamaHost. By default only a loopback Ollama
endpoint is contacted; a remote host is refused with a clear message unless
ai.allowRemoteHost is explicitly enabled. Blocked hosts are never contacted.
Honest documentation:
- README, package.json, and the welcome screen drop "fully offline"/"never
talks to the network"/"Electron is the only dependency" for an accurate
local-first contract that discloses the optional AI path and the bundled
Tesseract OCR dependency.
- New docs/PRIVACY.md details exactly what is collected locally and the one
outbound (opt-in, loopback-by-default) AI feature.
Tests: loopback/remote host matrix, remote-blocked-without-opt-in (and never
contacted), remote-allowed-with-opt-in, request timeout, explicit cancel vs
timeout, typed-text off-by-default vs opted-in, shortcut detection still works,
and a source guard that the C# CHAR emission stays behind the opt-in. 224 unit
tests pass; startup smoke and workflow E2E pass.
Co-Authored-By: Claude Fable 5 <[email protected]>
Phase 0 of the improvement plan (ai_prompts/prompt4.md): make the baseline
reproducible and stop the test runner from masking real failures.
- Pin Node >= 22.12 (engines + .nvmrc + engine-strict); every entry point
fails fast with clear guidance instead of dying late with ERR_REQUIRE_ESM
inside the packaging dependency graph.
- electron-launcher.js is diagnostics-only: all runtime npm install/rebuild/
repair paths are removed. npm ci on the pinned toolchain is the only
supported install path (README + GETTING_STARTED updated).
- Refuse to silently launch unsandboxed on Linux: --no-sandbox now requires
an explicit STEPFORGE_ALLOW_NO_SANDBOX/ELECTRON_DISABLE_SANDBOX opt-in and
is otherwise a hard error with actionable fixes; user-namespace sandboxing
is detected and preferred.
- Click-capture E2E no longer converts startup crashes into "SKIPPED": the
only allowed skip is the upfront absence of a display server. A missing
shared library or crash now fails with the startup log. Same guard added
to the startup smoke check.
- GitHub CI: run on pull_request, pin Node from .nvmrc, drop the macOS matrix
entry (not a support target), and audit production and full dependency
trees as separate signals. Gitea CI: pull_request trigger + pinned Node.
- Refresh package-lock on Node 22/npm 10 and remediate the form-data and
undici advisories (npm audit: 0 vulnerabilities, prod and full tree).
- Stop tracking generated machine-specific build reports
(build/build_report.md, build/artifacts_manifest.json).
Verified: 203 unit tests pass; repo-structure, startup-smoke, unit-workflows,
sample-artifacts, and build-release checks pass locally with a real Electron
launch. The click self-test now truthfully reports the pre-existing Linux
arm/debounce capture failures (also red on Gitea CI main run 177) instead of
hiding behind SKIPPED; that defect is scheduled for the capture-fix PR.
Co-Authored-By: Claude Fable 5 <[email protected]>
Add tests/checks/test_click_capture_selftest.sh: runs the real Electron
STEPFORGE_CLICK_SELFTEST session and asserts every scenario passes — 3/3
markers at 0.00% offset, 8/8 burst clicks kept on finish, the first armed
click captured (warmup click ignored), and the debounce (4/4). Picked up
automatically by tests/run_test.sh. Skips cleanly when the host has no
capture environment so it never falsely fails CI, but fails the suite on any
real regression in click->screenshot->step behavior.
Document the guard in ARCHITECTURE.md and CHANGELOG.md.
Co-Authored-By: Claude Fable 5 <[email protected]>
Root cause: warm-before-hide kept the window visible during backend warmup,
and on a restart that warmup could take several seconds (the stream backend
start waits up to 8s). During that visible window, clicks over the app were
skipped by the userIsInApp guard and clicks elsewhere were shot post-click,
so a restarted session looked like it stopped after one click.
- Recording is now 'armed' only after the window is hidden and the buffer is
primed. A new warmingUp flag makes onOsClick ignore clicks during warmup
(the window is covering the user's work anyway) instead of mishandling
them. Cleared on pause/finish.
- armRecording caps the warmup wait (WARMUP_MAX_MS=1500): the window hides
and the session arms even if the backend start hangs, so it can never sit
visible for seconds dropping clicks. The backend keeps coming up in the
background; the first click or two may take the fresh-shot fallback.
- A generation token invalidates an in-flight backend start whose session
has since finished, so a slow start can't install into a new session or
leave the starting-guard stuck and block the restart from starting one.
Tests: 4 new behavioral capture tests (warmup ignores clicks; pause/finish
clear it; armRecording warms-then-hides-then-arms; a hung start still arms
within the cap; a stale start is discarded and frees the guard) plus a new
end-to-end self-test scenario (warmup click ignored, first armed click
captured). 152 unit tests + all repo checks pass.
Co-Authored-By: Claude Fable 5 <[email protected]>
Per request: clicks of the same button closer together than
capture.clickDebounceMs (default 200ms) now collapse into a single step, so
accidental fast/double clicks don't each become a step. It is a leading-edge
debounce measured from the last *accepted* click, so a run of fast clicks
can't push the next deliberate click out — two clicks spaced beyond the
window (e.g. the reported 400-500ms apart) always register.
Replaces the prior 8ms duplicate-delivery suppression (subsumed by the
window). Configurable; 0 captures every click.
Tests (the point of this change is that it can't silently regress):
- 13 behavioral unit tests in capture.test.js that drive real onOsClick
calls with controlled timestamps and assert which clicks survive — the
reported 400/450/500ms cases, sub-window collapse, the 200ms boundary,
per-button independence, configurability, debounce=0, last-accepted (not
last-dropped) reference, session reset, and a full onOsClick -> queue ->
store integration check. No keyword/comment assertions.
- A fourth end-to-end self-test scenario (burst of 40ms clicks collapses to
1; three 300ms-apart clicks each register => 4 total). The marker/drain
scenarios set debounce to 0 so they keep stressing the frame pipeline.
147 unit tests + all repo checks pass.
Co-Authored-By: Claude Fable 5 <[email protected]>
The first screenshot of a session was late while every later one was fine.
Cause: on 'Start recording' the window hid first and the capture backend
started warming up after — creating worker, getUserMedia, first frame takes
~1s. A click in that gap found no buffered frame and took the post-click
fresh shot.
armRecording() now warms the recorder while the window is still visible and
only hides once frames are buffering (with a brief post-hide settle so the
first frame shows the user's screen, not the dismissed app window). Verified
end to end with a new self-test scenario that clicks 250ms after start: the
first click is now served a pre-click frame instead of a post-click shot.
Co-Authored-By: Claude Fable 5 <[email protected]>
The remaining 'captured slightly after the click' reports came from the
fresh-shot fallback, which grabs the screen when the click is processed
(after it). The previous lead change made that fallback *more* likely: a
frame now had to be >=120ms before the click to qualify, so on machines
where the capture stream can't always keep a frame that old buffered, more
clicks fell through to the post-click shot.
Make the click-lead a two-tier preference instead of a hard gate in
selectFrameForClick:
1. newest frame captured at least leadMs before the click (ideal margin), else
2. newest frame captured before the click at all.
Only when no pre-click frame exists does the caller fresh-shot. leadMs is
threaded through the stream backend to the worker so both selection paths
agree. Verified end to end: frames land ~120-170ms before each click,
markers stay at 0.00%, and the 8-click burst still saves all 8.
Co-Authored-By: Claude Fable 5 <[email protected]>
Real-world recording now saves every click with exact markers; the only
remaining nit was screenshots feeling a touch late. Add a configurable
click-lead (capture.clickLeadMs, default 120ms) that targets the screen
just before the hook timestamp, and tighten the stream sampling cadence to
50ms so a frame near that target always exists. Verified end to end: frames
now land ~120-160ms before the click (was 25-57ms), markers stay at 0.00%
offset, and the 8-click burst still saves all 8.
Also document the milestone in docs/CHANGELOG.md and remove an accidental
paste of Gitea commit-page text from it.
Co-Authored-By: Claude Fable 5 <[email protected]>
Root cause of 'I clicked many times but only got two screenshots':
finishing/pausing a session called backend.stop(), which cancelled every
in-flight frame request to null. Clicks whose PNG had not finished
encoding yet were then dropped — only the first few survived.
Fixes:
- Stream backend now *drains* on stop: it stops accepting new requests but
keeps the worker alive until frames already selected for queued clicks
finish encoding. stop({ immediate: true }) keeps the old abandon-now
behavior for an unhealthy worker.
- Two-stage worker reply: a fast 'frame-selected' ack pins the pairing and
proves liveness; the slow PNG payload follows. A slow encode (seconds on
software-rendered hosts) is no longer mistaken for a dead worker, which
had been forcing the post-click fresh-shot fallback (late screenshots).
- Queued clicks carry their guide id and are stored even if the session
ends while they wait in the queue.
- The tray gesture that stops a session is discarded by matching its
recorded screen position, not a time window — a fast workflow click near
the stop is no longer collateral damage. (Replaces the earlier grace
window, which dropped whole bursts.)
- A click on a display with no ready stream resolves null so the caller
fresh-shots the correct monitor instead of returning another screen.
- STEPFORGE_CAPTURE_LOG=1 prints one line per click decision; the
second-instance handler now surfaces the running window instead of
exiting silently.
- Self-test gains a fast-burst-then-finish scenario (8/8 saved) and the
marker/coordinate checks remain at 0.00% offset.
Tests: 133 unit + all repo checks passing.
Co-Authored-By: Claude Fable 5 <[email protected]>
Implements the architecture change from ai_prompts/prompt3.md:
- New app/click-frames.js: shared timestamped frame ring + strict
click-to-frame pairing (never a frame whose grab started after the
click); legacy slack behavior kept behind capture.strictClickFrames=false.
- New stream capture backend (app/stream-backend.js + hidden worker
window): per-display desktop media streams sampled into ring buffers
and PNG-encoded entirely off the main process, so click delivery is
never starved by capture work. Auto-degrades to the legacy in-process
frame loop when streams cannot start or the worker stops answering.
- Clicks are paired with their frame at event time (eager pairing in
enqueueClickCapture); only the storing is serialized, so slow encodes
cannot skew later clicks in a fast burst.
- Linux watcher: restored event-time root coordinates from
xinput test-xi2 and merge raw/regular twin events structurally.
- Replaced the 40ms time debounce with source-aware duplicate
suppression: fast legitimate clicks are never dropped.
- New app/coords.js: physical-to-DIP conversion with multi-monitor and
scale-factor handling; Windows keeps screenToDipPoint.
- STEPFORGE_CLICK_SELFTEST end-to-end hook: 3/3 clicks become steps via
the stream backend with 0.00% marker offset on this host.
- Tests rewritten/added: strict selection, coords, stream backend,
Linux coordinate parsing, twin merge, burst clicking (126 passing).
Co-Authored-By: Claude Fable 5 <[email protected]>
TylerandClaude Fable 5