**Keyboard hook (Windows)**
- Extends the existing C# WH_MOUSE_LL process to also install
WH_KEYBOARD_LL alongside it (keyboard hook is optional — failure
does not break mouse capture).
- Emits CHAR <code> <ts> for printable keystrokes, KEY <name> <ts>
for modifier combos (Ctrl+T) and special keys (Backspace, Enter).
**Text accumulation in capture session**
- CaptureService tracks _keyBuffer (typed chars since last step) and
_lastShortcut (last modifier combo) using the new onKeyboardEvent()
method.
- snapshotKeyContext() is called at enqueueClickCapture time so each
step's clickMeta.keyContext carries { recentTyped, recentShortcut }.
- Buffer resets after each snapshot; stale input (>8s gap) is dropped.
**UIAutomation element value**
- collectWindowsWindowContext now reads ValuePattern.Current.Value
from the clicked element — giving us what's actually typed in a
search box or text field without needing the keyboard buffer.
**Smart title generation (core/text-intel.js)**
- Priority chain: keyboard shortcut → element value → typed text
→ OCR → element label → page title → app name.
- SHORTCUT_TITLES maps 50+ common shortcuts (Ctrl+T, Ctrl+S, F5 …)
to natural language descriptions: "Open new tab", "Save", etc.
- qualifyTitleWithApp() appends "in Chrome / VS Code / Terminal / …"
to every title when the app is known: "Click Save in VS Code",
"Search for oracle in Chrome", "Open new tab in Chrome".
- APP_DISPLAY_NAMES covers browsers, editors, terminals, office apps.
Six new unit tests cover shortcuts, typed-text search, element value,
and app-qualified OCR titles. Capture test updated for keyContext.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
The title engine was falling back to "Screen capture" for all browser
captures because it treated the entire browser window title as noise.
Changes:
- stripBrowserNameSuffix: removes "- Google Chrome" / "| Firefox" etc.
from the end of window titles, leaving just the page title.
"oracle - Google Search - Google Chrome" → "oracle - Google Search"
"Oracle | Cloud Applications - Google Chrome" → page title only
- extractSearchQuery: detects "[query] - Google Search" / Bing / etc.
patterns after stripping the browser suffix, and formats the result
as "Search for oracle".
- buildCaptureTitle: uses stripped page title + search detection before
falling through to the "Screen capture" fallback.
- pickBestOcrPhrase: considers the full OCR line (≤80 chars) as a
candidate with a +35 completeness bonus before splitting on | or ·.
This preserves "Oracle | Cloud Applications and Cloud Platform" as a
single phrase instead of breaking it into fragments.
- candidateWords: filters out standalone punctuation tokens (|, ·, •)
so they don't inflate word-count penalties for compound brand names.
- verbForElementRole: hyperlinks and links now produce "Select" instead
of "Click"; search box / search field produces "Search for".
Five new unit tests cover: browser title stripping, search query
extraction, full pipe-separated link text, link and search box verbs.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- Remove the standalone "AI Rewrite" section from the editor panel.
The existing title and description AI buttons already flush the step
first, so they rewrite whatever the user has typed in those fields.
- Never pass a generic fallback title ("Screen capture", "Window
capture", "Region capture", "Capture") as the AI title candidate or
as step content. It is now treated as "(not set — generate a
specific action title)" so the AI always produces something real.
- hasRichCaptureContext now counts any non-trivial app name or window
title as sufficient context, instead of requiring non-browser noise.
- Prompt rules updated: "NEVER output Screen/Window/Region capture",
separate paths for improving a user draft vs generating from context,
and clearer guidance when context is limited (use app/window name).
- isPlaceholderTitle helper guards summarizeStepForAi so a
default-titled step presents itself as empty to the AI.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- Store captureMetadata (OCR text, window/app/element info) with each
step at capture time so AI always has the original rich context.
- Add buildCaptureContext() to TextIntelService; capture.js uses it
instead of buildCaptureTitle() so both title and metadata come from
one pass.
- generateStepPatch() prefers stored captureMetadata over re-running
OCR, giving the AI the best possible context when the user clicks
an AI button later.
- Add autoDoc setting: when enabled every capture (shoot, region, and
session hotkey/click) is automatically documented by AI. Manual
captures await AI before returning; session captures fire-and-forget
and push a step:updated event so the renderer reloads seamlessly.
- Add ai:rewriteText IPC and rewriteText() method for plain-text
polishing via a separate callOllamaText() that skips JSON mode.
- Add "AI Rewrite" section in the editor right panel: textarea + AI
button that rewrites whatever the user types in place.
- Improve buildAiPrompt() rules: action-focused title instructions,
explicit anti-junk rules (no "Capture the screen / OCR" blocks),
and a context-quality gate that suppresses blocks when context is
thin.
- Add autoDoc checkbox to AI settings dialog.
- Renderer handles step:updated to reload the selected step after
background auto-doc finishes.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Adds PdfBuilder.linkRect() for /Subtype /Link annotations with a
/Dest pointing at another page's destination. TOC entries record a
target placeholder that the per-step loop fills in once it knows
which page the step landed on, so clicking a Contents line jumps the
reader straight to that step.
Lets users record author/co-authors/organization for a guide via a new
"Guide information…" dialog; this metadata renders below the title on
the PDF cover (title now sits above the accent rule). PDF export also
paginates so each step fits its own page where possible, keeps a
step's title/image/lead-in together, and forces the next step onto a
fresh page after an oversized step overflows. Exports now run in a
forked helper process so large guides no longer freeze the UI.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
- The canvas Undo/Redo buttons now also undo/redo step deletion
(single or multi-select), restoring the step's data, images, and
position in the order. Backed by a new step:restore IPC call and
GuideStore.restoreStep().
- Remove "Finish capture session" from the capture menu. The
top-right recording bar in the guide editor is now the only place
to start/stop recording, with its toggle relabeled
"Start recording" / "Stop recording".
writeDescription/emitBlocks rendered each word as a separately positioned
Tj, advancing by our approximate average-glyph-width estimate — which is
much wider than real space/character widths, producing "This is bold"
style gaps. Render each line as one continuous text object instead
(PdfBuilder.textRun), so consecutive Tj operators advance using the
viewer's real font metrics.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
PDF exports now preserve bold/italic/links/lists/blockquotes from the
description editor (via a new HTML block/run parser) instead of flattening
to plain text. Literal "[text](url)" links inserted by the editor's Link
button are converted to real <a> tags before rendering. Tabs and common
"smart typography" characters (smart quotes, dashes, ellipsis, bullet) are
mapped to their WinAnsiEncoding glyphs instead of rendering as "?".
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
panX/panY now represent 0..1 fractions of the pannable range rather
than absolute image positions, so the slider's full travel always
covers edge-to-edge regardless of zoom level. Pan Y is inverted so
sliding right moves the view up. Updated both the editor canvas and
core/raster.js export so they stay in sync.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
Per request: clicks of the same button closer together than
capture.clickDebounceMs (default 200ms) now collapse into a single step, so
accidental fast/double clicks don't each become a step. It is a leading-edge
debounce measured from the last *accepted* click, so a run of fast clicks
can't push the next deliberate click out — two clicks spaced beyond the
window (e.g. the reported 400-500ms apart) always register.
Replaces the prior 8ms duplicate-delivery suppression (subsumed by the
window). Configurable; 0 captures every click.
Tests (the point of this change is that it can't silently regress):
- 13 behavioral unit tests in capture.test.js that drive real onOsClick
calls with controlled timestamps and assert which clicks survive — the
reported 400/450/500ms cases, sub-window collapse, the 200ms boundary,
per-button independence, configurability, debounce=0, last-accepted (not
last-dropped) reference, session reset, and a full onOsClick -> queue ->
store integration check. No keyword/comment assertions.
- A fourth end-to-end self-test scenario (burst of 40ms clicks collapses to
1; three 300ms-apart clicks each register => 4 total). The marker/drain
scenarios set debounce to 0 so they keep stressing the frame pipeline.
147 unit tests + all repo checks pass.
Co-Authored-By: Claude Fable 5 <[email protected]>
The first screenshot of a session was late while every later one was fine.
Cause: on 'Start recording' the window hid first and the capture backend
started warming up after — creating worker, getUserMedia, first frame takes
~1s. A click in that gap found no buffered frame and took the post-click
fresh shot.
armRecording() now warms the recorder while the window is still visible and
only hides once frames are buffering (with a brief post-hide settle so the
first frame shows the user's screen, not the dismissed app window). Verified
end to end with a new self-test scenario that clicks 250ms after start: the
first click is now served a pre-click frame instead of a post-click shot.
Co-Authored-By: Claude Fable 5 <[email protected]>
Real-world recording now saves every click with exact markers; the only
remaining nit was screenshots feeling a touch late. Add a configurable
click-lead (capture.clickLeadMs, default 120ms) that targets the screen
just before the hook timestamp, and tighten the stream sampling cadence to
50ms so a frame near that target always exists. Verified end to end: frames
now land ~120-160ms before the click (was 25-57ms), markers stay at 0.00%
offset, and the 8-click burst still saves all 8.
Also document the milestone in docs/CHANGELOG.md and remove an accidental
paste of Gitea commit-page text from it.
Co-Authored-By: Claude Fable 5 <[email protected]>
Implements the architecture change from ai_prompts/prompt3.md:
- New app/click-frames.js: shared timestamped frame ring + strict
click-to-frame pairing (never a frame whose grab started after the
click); legacy slack behavior kept behind capture.strictClickFrames=false.
- New stream capture backend (app/stream-backend.js + hidden worker
window): per-display desktop media streams sampled into ring buffers
and PNG-encoded entirely off the main process, so click delivery is
never starved by capture work. Auto-degrades to the legacy in-process
frame loop when streams cannot start or the worker stops answering.
- Clicks are paired with their frame at event time (eager pairing in
enqueueClickCapture); only the storing is serialized, so slow encodes
cannot skew later clicks in a fast burst.
- Linux watcher: restored event-time root coordinates from
xinput test-xi2 and merge raw/regular twin events structurally.
- Replaced the 40ms time debounce with source-aware duplicate
suppression: fast legitimate clicks are never dropped.
- New app/coords.js: physical-to-DIP conversion with multi-monitor and
scale-factor handling; Windows keeps screenToDipPoint.
- STEPFORGE_CLICK_SELFTEST end-to-end hook: 3/3 clicks become steps via
the stream backend with 0.00% marker offset on this host.
- Tests rewritten/added: strict selection, coords, stream backend,
Linux coordinate parsing, twin merge, burst clicking (126 passing).
Co-Authored-By: Claude Fable 5 <[email protected]>
The session previously only listened for the global hotkey, which is
unreliable under WSLg/Wayland — users got one screenshot and nothing
more. Sessions now layer three triggers:
- click-capture via OS adapters (xinput test-xi2 on X11, PowerShell
GetAsyncKeyState polling on Windows), debounced, ignoring clicks on
StepForge itself
- interval auto-capture (3/5/10 s) as the always-works fallback,
enabled by default when click detection is unavailable
- the existing global hotkey, plus a manual Shoot button
The REC bar now shows live count + active trigger with Shoot / Auto /
Pause / Finish. New captures and added steps are selected in the
editor (explicit reload(stepId) wins over a surviving selection).
Capture self-test hook (STEPFORGE_CAPTURE_SELFTEST) verifies 3x
hotkey-path captures and interval capture end-to-end.
Co-Authored-By: Claude Fable 5 <[email protected]>
- Hand-rolled ZIP writer/reader (node:zlib only) with CRC verification,
UTF-8 names, store/deflate, path-traversal validation; verified
interoperable with system unzip
- .sfgz guide archives: export, copy-import (fresh ids, remapped
substeps), linked-import with explicit write-back save
- Advisory .lock-sfgz sidecar locks with stale detection and force-steal
- Snapshot backups with pruning and undoable restore
- 7 more workflow tests (19 total)
Co-Authored-By: Claude Fable 5 <[email protected]>
- Folder-based GuideStore with atomic writes, trash/restore, duplicate,
substep reparenting, folders/favorites, working-image crop/reset
- Allowlist HTML sanitizer applied on every store write
- Placeholder scopes (guide > global > system) and collection
- Persisted app settings with deep default merge
- 16 workflow tests exercising real on-disk round-trips
Co-Authored-By: Claude Fable 5 <[email protected]>
TylerandClaude Sonnet 4.6