Author SHA1 Message Date
TylerandClaude Fable 5 50e445e7c3 Close the renderer privilege boundary: navigation, IPC, permissions, shell
Template tests / tests (pull_request) Failing after 33s
Phase 1 of the improvement plan (PR 2 of the sequence). A remote page could
previously inherit the privileged preload bridge: the main window had no
navigation guard or popup handler, IPC handlers accepted any sender, every
Electron permission was granted to everyone, and shell:openPath accepted an
arbitrary renderer-supplied target.

- New app/security.js (plain-Node testable) centralizes the policy:
  app-page identity, navigation/popup denial, deny-by-default permissions,
  external-URL validation, IPC sender guard, payload budgets, field
  validators, and a produced-files registry.
- Every window (main, region overlay, capture worker) is sandboxed, denies
  all navigation away from its own page, denies every popup, and refuses
  webview attachment.
- Every IPC channel now rejects events that are not from the main window's
  top frame on index.html, rejects non-plain/oversized argument bags, and
  channels with risky inputs validate ids, enums, names, and sizes
  (path-traversal and prototype-pollution guards included).
- Permissions are deny-by-default; the only grant in the app is display
  capture (+media) for the dedicated capture-worker page. The display-media
  handler also verifies the requesting frame.
- shell:openPath/showItemInFolder are gone. Replacements are intent-specific:
  openProduced (only files the main process produced this session),
  revealLinkedArchive (path read from the store, not the renderer), and
  openExternal (parsed, scheme-checked http(s)/mailto only).
- archive:peek removed: unused, and it let the renderer read arbitrary
  archives by path.
- export:run only accepts output directories that came from a dialog pick or
  remembered settings; anything else re-prompts.
- Renderer: description links never navigate; http(s)/mailto open externally
  via the validated handler.
- 12 new security regression tests: hostile navigation targets, permission
  matrix, sender spoofing (wrong window/subframe/navigated/disposed frames),
  oversized payloads, traversal/pollution attempts, produced-file registry,
  and source-level guards (no blanket grants, no generic shell channels,
  sandbox on every window).

Verified: 215 unit tests pass; startup smoke, unit-workflows, sample
artifacts, and build-release E2E pass; click self-test still reaches
source: stream with markers 3/3 and burst 8/8 under the deny-by-default
policy (arm/debounce remain the known pre-existing failures, untouched
here); UI screenshot confirms the sandboxed renderer boots.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-03 11:25:36 -07:00
Tyler 6a3005f24c Merge pull request #5 from Twest2/StepForge pr/01-toolchain-ci
Template tests / tests (push) Failing after 33s
Pin Node toolchain, remove runtime npm repair, make CI and E2E truthful (plan PR 1)
2026-07-03 11:12:29 -07:00
TylerandClaude Fable 5 0f966a5fd0 Pin the Node toolchain, remove runtime npm repair, make CI and E2E truthful
Template tests / tests (pull_request) Failing after 1m26s
Phase 0 of the improvement plan (ai_prompts/prompt4.md): make the baseline
reproducible and stop the test runner from masking real failures.

- Pin Node >= 22.12 (engines + .nvmrc + engine-strict); every entry point
  fails fast with clear guidance instead of dying late with ERR_REQUIRE_ESM
  inside the packaging dependency graph.
- electron-launcher.js is diagnostics-only: all runtime npm install/rebuild/
  repair paths are removed. npm ci on the pinned toolchain is the only
  supported install path (README + GETTING_STARTED updated).
- Refuse to silently launch unsandboxed on Linux: --no-sandbox now requires
  an explicit STEPFORGE_ALLOW_NO_SANDBOX/ELECTRON_DISABLE_SANDBOX opt-in and
  is otherwise a hard error with actionable fixes; user-namespace sandboxing
  is detected and preferred.
- Click-capture E2E no longer converts startup crashes into "SKIPPED": the
  only allowed skip is the upfront absence of a display server. A missing
  shared library or crash now fails with the startup log. Same guard added
  to the startup smoke check.
- GitHub CI: run on pull_request, pin Node from .nvmrc, drop the macOS matrix
  entry (not a support target), and audit production and full dependency
  trees as separate signals. Gitea CI: pull_request trigger + pinned Node.
- Refresh package-lock on Node 22/npm 10 and remediate the form-data and
  undici advisories (npm audit: 0 vulnerabilities, prod and full tree).
- Stop tracking generated machine-specific build reports
  (build/build_report.md, build/artifacts_manifest.json).

Verified: 203 unit tests pass; repo-structure, startup-smoke, unit-workflows,
sample-artifacts, and build-release checks pass locally with a real Electron
launch. The click self-test now truthfully reports the pre-existing Linux
arm/debounce capture failures (also red on Gitea CI main run 177) instead of
hiding behind SKIPPED; that defect is scheduled for the capture-fix PR.

Co-Authored-By: Claude Fable 5 <[email protected]>
2026-07-03 13:09:51 -07:00
Tyler 534a28ece8 Merge GitHub main and keep Linux WIP
Template tests / tests (push) Failing after 33s
2026-06-29 08:22:40 -05:00
Tyler c2e05c900f Remove linux support
Template tests / tests (push) Successful in 2m0s
2026-06-29 08:11:43 -05:00
Tyler 7c006a7bb7 Fix Windows installer and build versioning 2026-06-26 22:37:13 -05:00
Tyler 999f4a13b8 Minor fixes 2026-06-26 22:21:53 -05:00
Tyler 3356b935fc Minor fixes 2026-06-26 22:10:54 -05:00
Tyler 749f8d2d0d Add back red circle to windows 2026-06-26 21:38:22 -05:00
Tyler dd49e42290 Fix npm test glob on CI 2026-06-26 18:41:12 -05:00
Tyler 266a92fedb Fix windows breakage from linux dev 2026-06-26 18:38:03 -05:00
Tyler 0325b6efbc ubuntu not working. idk I'm just gonna use windows anyway 2026-06-26 18:00:22 -05:00
Tyler 3c5c520799 work on linux release
Linux release not fully working as well as windows, will investigate later
2026-06-26 17:02:35 -05:00
Tyler 962f929de2 fixed recording workflow on ubuntu
ai gen description: Root cause 1 — The mandatory constraint removal (most critical)

In Electron 29+ / Chromium 116+, the old getUserMedia format with a mandatory: {} wrapper was removed. The capture worker was still using it:

video: { mandatory: { chromeMediaSource: 'desktop', ... } }  // broke in Electron 29+
This caused getUserMedia to throw in the worker, which sent a stream-error back to the main process, which fell back to the frame loop. Fix: flat constraint format with no mandatory wrapper.

Root cause 2 — Frame loop spammed getSources() every 200ms

Once the stream backend failed, the fallback frame loop called desktopCapturer.getSources() 5× per second. On Linux via the XDG portal, each getSources() call is a new permission request → dialog every few seconds. Fix: on Linux without click capture, don't fall back to the frame loop at all — just let interval/hotkey captures take individual fresh shots.

Root cause 3 — Interval/hotkey captures bypassed the stream backend

Even when the stream backend is running, sessionCapture('interval') went straight to shoot() → grab() → getSources(). Fix: non-click triggers now pull a buffered frame from the stream backend's ring buffer (sampled every 100ms), completely avoiding getSources().

Root cause 4 — Stream backend never started on Wayland

The earlier fix made recorderWanted = false on Wayland, so the stream backend never even started. Fix: recorderWanted is now true whenever stream capture is enabled (the default), regardless of whether click detection is available.

Safety net — setPermissionCheckHandler

Electron 29+ requires an explicit permission grant for display-capture in renderer windows. Added a handler that grants all permissions (safe for a fully local/offline app like StepForge).

Windows is completely unaffected — the constraint fix is Electron-version level (affects both platforms the same), and all the Linux-specific frame-loop guards only fire when clickCaptureAvailable() returns false, which only happens on Wayland.
2026-06-26 16:11:16 -05:00
Tyler a5a8498e71 Keep recording bar in ubuntu 2026-06-26 15:49:49 -05:00
Tyler 32788fafdc fixed ubuntu (Wayland) recording workflow 2026-06-26 15:48:09 -05:00
TylerandGitHub 412a4f4820 Merge pull request #4 from Twest2/feature/ai-vision-hybrid
Feature/ai vision hybrid
2026-06-26 08:52:52 -05:00
Tyler 3915865029 Keep ai model saved when app closes. 2026-06-26 08:48:38 -05:00
Tyler 79edb6bb56 Change model
Ran into an issue with the previous model for having mllama archetecture so I changed it
2026-06-26 08:36:45 -05:00
Tyler a971ff870f Added feature to where local ai models can see the image in each step. 2026-06-26 08:05:18 -05:00
TylerandGitHub 9bd4a959c4 Semantic capture titles, AI documentation assist, and keyboard tracking (#3)
Template tests / tests (push) Successful in 1m58s
Semantic capture titles, AI documentation assist, and keyboard tracking
2026-06-24 12:11:11 -05:00
TylerandClaude Sonnet 4.6 ebeb6ac578 label Enable AI setting as experimental
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 12:04:40 -05:00
TylerandClaude Sonnet 4.6 f60cf1a688 label generate-all-steps menu item as experimental
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 12:02:59 -05:00
TylerandClaude Sonnet 4.6 2d1b474931 fix generate-all-steps: never overwrite existing content, right target per step
The previous version called target:'all' on every step unconditionally,
which caused the AI to rewrite steps that already had good titles and
descriptions — often making them worse or inaccurate.

New behaviour:
- Build a queue of only the steps that are actually missing content
  (placeholder title and/or empty description).
- Determine target per step: 'title', 'description', or 'all' — only
  fill what is blank; leave existing user-written text completely alone.
- If all steps already have titles and descriptions, show a toast and stop.
- Call reload() once at the end instead of patching this.steps mid-loop,
  so the editor and step list both update atomically from the store.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 11:27:24 -05:00
TylerandClaude Sonnet 4.6 c60b7247af generate all text fields with AI now fills every step, not just current
Previously the menu action only ran AI generation on the selected step.
Now it loops through all steps sequentially, showing progress ("AI:
filling step N of M…"), then reloads the visible step when done.
Partial failures are reported in the final toast.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 11:22:27 -05:00
TylerandClaude Sonnet 4.6 cf056d2b97 stop AI from generating text/code/table blocks on generate-all
The model was producing Text and Code blocks on every 'generate all
fields' action — uninstructed, generic, and never useful for the user.

Blocks are only appropriate when the user explicitly edits an existing
block (target === 'block'). For title/description/all generation:
- Schema collapses to { "title", "description" } — no blocks key
- targetText no longer mentions blocks
- allowedBlockNote is null (omitted from prompt)
- Rule explicitly says "Do NOT add any blocks array"

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 11:18:41 -05:00
TylerandClaude Sonnet 4.6 3fa366a0d0 revert UIAutomation background thread — it stalled click detection
TryLoadUia() blocked for several seconds loading reflection assemblies at
ClickProcessorLoop startup, during which no CLICK events were emitted.
Restored MouseHookCallback to emit CTX + CLICK synchronously (fast Win32
only) as before — no startup cost, clicks are never delayed.

The wider OCR crop and smarter search-results title fallback are kept.
UIAutomation element lookup can be revisited with a pre-warmed approach.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 11:14:49 -05:00
TylerandClaude Sonnet 4.6 d0b1816175 fix step-behind titles: UIAutomation element label from click watcher + smarter search fallback
**Root cause**
When OCR fails, the title fell back to the browser window title.  For a
click on a search-results page, the window title reflects the *previous*
search query ("oracle - Google Search"), producing "Search for Oracle"
even though the user is clicking a link *on* that page.

**Fix 1: UIAutomation element label from the click watcher**
The C# click-watcher hook now enriches each click in a background thread
(ClickProcessorLoop) rather than in the hook callback:
- MouseHookCallback captures window title synchronously (fast Win32),
  then queues a PendingClick and returns immediately.
- ClickProcessorLoop calls AutomationElement.FromPoint() via reflection
  (no compile-time assembly reference → no startup failure if UIA is
  absent).  Wrapped in a 300ms timeout thread so slow UIA calls don't
  delay the click event past the frame buffer window.
- Emits CTX + ELEM (label/role/value) + CLICK as an atomic batch.

Node.js:
- Parses ELEM events, merges element info into _pendingWindowContext.
- clickMeta.windowContext now carries elementLabel/elementRole/elementValue
  in addition to windowTitle/appName.
- buildCaptureTitle priority-5 (element label) now fires from click-watcher
  data, giving "Select Oracle | Cloud Applications…" without OCR.

**Fix 2: Wider OCR crop**
ocrAroundClick now uses a full-display-width × 100px horizontal strip at
the click height.  The previous 420 px crop cropped through long link text
(e.g. "Oracle | Cloud Applications and Cloud Platform"), causing fragments
to be scored lower than the complete text.

**Fix 3: Search-results window title fallback**
extractSearchQuery now only produces "Search for Oracle" when recentTyped
is non-empty (the user was actually typing a query).  For a pure click on
the search-results page (no recent typing), the fallback is "Select a
Oracle result in Chrome" — honest about what we know without implying the
user performed the search in this step.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 11:05:34 -05:00
TylerandClaude Sonnet 4.6 dfa65c894b ci: only run tests on push to main, not on PR commits
Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 10:53:30 -05:00
TylerandClaude Sonnet 4.6 162ae0ae05 improve AI rewrite UX and title quality
**AI button: dynamic tooltip hints**
- titleAiBtn and descAiBtn now show "Rewrite with AI" when the field
  already has user content, "Generate with AI" when empty.
- updateAiButtonHints() fires on every title/description input event
  and whenever syncStepFields() runs to keep hints current.

**Stronger rewrite prompt**
- When the user has typed a draft title or description, the prompt now
  shows it explicitly: "User's draft title (rewrite this): '...'"
- Rules changed from "improve its wording" to "Your only job is to
  polish its grammar and phrasing. Do NOT replace it with something
  different." — prevents the model from ignoring the user's text and
  generating fresh content from capture context.
- The suggested-title hint is suppressed when a draft title exists so
  the model doesn't silently swap the user's text for the auto-title.

**Title quality: generic window title filter**
- GENERIC_WINDOW_TITLES Set filters "New Tab", "Untitled", "Loading"
  etc. from the window-title path so they no longer produce titles
  like "Open New Tab in Chrome".

**Title quality: app name stripping for non-browser apps**
- stripBrowserNameSuffix now accepts an optional appName; it strips
  the app's display name and process name from the window title suffix
  using the same pattern as browser names.
- "Document1.docx - Word" with appName "winword" → "Document1.docx".
- buildCaptureTitle passes metadata.appName into the strip call.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 10:42:56 -05:00
TylerandClaude Sonnet 4.6 879434f14f fix: capture window title from click watcher instead of spawning PowerShell
Root cause of all-"Screen capture" titles: collectWindowsWindowContext
called execFileSync('powershell.exe') with a 1200 ms timeout, but
PowerShell cold-start on typical Windows systems takes 1-3 seconds.
Every capture timed out silently, returned empty metadata, and fell
back to "Screen capture".

Fix:
- The click watcher C# process (already running, already compiled)
  now emits a CTX event immediately before each CLICK event using
  GetForegroundWindow → GetWindowText + QueryFullProcessImageName.
  These are synchronous Win32 calls, sub-millisecond, no startup cost.
- CTX payload is base64-encoded to survive the line protocol safely:
    CTX <b64title> <b64app> <unixMs>
- processClickWatcherData parses CTX lines and stores the decoded
  strings in this._lastWindowContext.
- enqueueClickCapture attaches it as clickMeta.windowContext.
- buildCaptureContext uses it directly (Promise.resolve) when present,
  bypassing the PowerShell spawn entirely.

Fallback (manual captures without a session click watcher running):
- collectWindowsWindowContext is now async and uses execFile with a
  4 s timeout instead of execFileSync at 1200 ms, so it no longer
  blocks the event loop and has room to succeed on slower machines.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 10:16:11 -05:00
TylerandClaude Sonnet 4.6 13b5f67de8 add keyboard tracking and app-qualified smart titles
**Keyboard hook (Windows)**
- Extends the existing C# WH_MOUSE_LL process to also install
  WH_KEYBOARD_LL alongside it (keyboard hook is optional — failure
  does not break mouse capture).
- Emits CHAR <code> <ts> for printable keystrokes, KEY <name> <ts>
  for modifier combos (Ctrl+T) and special keys (Backspace, Enter).

**Text accumulation in capture session**
- CaptureService tracks _keyBuffer (typed chars since last step) and
  _lastShortcut (last modifier combo) using the new onKeyboardEvent()
  method.
- snapshotKeyContext() is called at enqueueClickCapture time so each
  step's clickMeta.keyContext carries { recentTyped, recentShortcut }.
- Buffer resets after each snapshot; stale input (>8s gap) is dropped.

**UIAutomation element value**
- collectWindowsWindowContext now reads ValuePattern.Current.Value
  from the clicked element — giving us what's actually typed in a
  search box or text field without needing the keyboard buffer.

**Smart title generation (core/text-intel.js)**
- Priority chain: keyboard shortcut → element value → typed text
  → OCR → element label → page title → app name.
- SHORTCUT_TITLES maps 50+ common shortcuts (Ctrl+T, Ctrl+S, F5 …)
  to natural language descriptions: "Open new tab", "Save", etc.
- qualifyTitleWithApp() appends "in Chrome / VS Code / Terminal / …"
  to every title when the app is known: "Click Save in VS Code",
  "Search for oracle in Chrome", "Open new tab in Chrome".
- APP_DISPLAY_NAMES covers browsers, editors, terminals, office apps.

Six new unit tests cover shortcuts, typed-text search, element value,
and app-qualified OCR titles. Capture test updated for keyContext.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 10:04:34 -05:00
TylerandClaude Sonnet 4.6 8d645c0aca fix smart title generation from browser window context
The title engine was falling back to "Screen capture" for all browser
captures because it treated the entire browser window title as noise.

Changes:
- stripBrowserNameSuffix: removes "- Google Chrome" / "| Firefox" etc.
  from the end of window titles, leaving just the page title.
  "oracle - Google Search - Google Chrome" → "oracle - Google Search"
  "Oracle | Cloud Applications - Google Chrome" → page title only

- extractSearchQuery: detects "[query] - Google Search" / Bing / etc.
  patterns after stripping the browser suffix, and formats the result
  as "Search for oracle".

- buildCaptureTitle: uses stripped page title + search detection before
  falling through to the "Screen capture" fallback.

- pickBestOcrPhrase: considers the full OCR line (≤80 chars) as a
  candidate with a +35 completeness bonus before splitting on | or ·.
  This preserves "Oracle | Cloud Applications and Cloud Platform" as a
  single phrase instead of breaking it into fragments.

- candidateWords: filters out standalone punctuation tokens (|, ·, •)
  so they don't inflate word-count penalties for compound brand names.

- verbForElementRole: hyperlinks and links now produce "Select" instead
  of "Click"; search box / search field produces "Search for".

Five new unit tests cover: browser title stripping, search query
extraction, full pipe-separated link text, link and search box verbs.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 09:44:36 -05:00
TylerandClaude Sonnet 4.6 68ceee83ef fix ai title generation and remove separate rewrite section
- Remove the standalone "AI Rewrite" section from the editor panel.
  The existing title and description AI buttons already flush the step
  first, so they rewrite whatever the user has typed in those fields.

- Never pass a generic fallback title ("Screen capture", "Window
  capture", "Region capture", "Capture") as the AI title candidate or
  as step content. It is now treated as "(not set — generate a
  specific action title)" so the AI always produces something real.

- hasRichCaptureContext now counts any non-trivial app name or window
  title as sufficient context, instead of requiring non-browser noise.

- Prompt rules updated: "NEVER output Screen/Window/Region capture",
  separate paths for improving a user draft vs generating from context,
  and clearer guidance when context is limited (use app/window name).

- isPlaceholderTitle helper guards summarizeStepForAi so a
  default-titled step presents itself as empty to the AI.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 09:27:23 -05:00
TylerandClaude Sonnet 4.6 d244626583 ai smart auto-documentation and text rewrite
- Store captureMetadata (OCR text, window/app/element info) with each
  step at capture time so AI always has the original rich context.
- Add buildCaptureContext() to TextIntelService; capture.js uses it
  instead of buildCaptureTitle() so both title and metadata come from
  one pass.
- generateStepPatch() prefers stored captureMetadata over re-running
  OCR, giving the AI the best possible context when the user clicks
  an AI button later.
- Add autoDoc setting: when enabled every capture (shoot, region, and
  session hotkey/click) is automatically documented by AI. Manual
  captures await AI before returning; session captures fire-and-forget
  and push a step:updated event so the renderer reloads seamlessly.
- Add ai:rewriteText IPC and rewriteText() method for plain-text
  polishing via a separate callOllamaText() that skips JSON mode.
- Add "AI Rewrite" section in the editor right panel: textarea + AI
  button that rewrites whatever the user types in place.
- Improve buildAiPrompt() rules: action-focused title instructions,
  explicit anti-junk rules (no "Capture the screen / OCR" blocks),
  and a context-quality gate that suppresses blocks when context is
  thin.
- Add autoDoc checkbox to AI settings dialog.
- Renderer handles step:updated to reload the selected step after
  background auto-doc finishes.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-24 09:17:31 -05:00
Tyler 375c14b028 prefer OCR for capture titles 2026-06-24 08:55:19 -05:00
Tyler 1a1dbb846b fix settings AI test connection wiring 2026-06-23 18:14:12 -05:00
Tyler 0bd326d8b2 harden Electron startup repair 2026-06-23 18:10:35 -05:00
Tyler bd10f7d147 lazy load ocr dependency 2026-06-23 18:02:24 -05:00
Tyler 33eb0f745e semantic titles ai assist 2026-06-23 17:46:33 -05:00
TylerandGitHub cb5e0c6837 Merge pull request #2 from Twest2/text_locations
Honor text block positions in exports
2026-06-23 16:57:06 -05:00
Tyler ffaa123893 Honor text block positions in exports 2026-06-23 16:44:28 -05:00
59 changed files with 5895 additions and 725 deletions
+6
View File
@@ -4,6 +4,7 @@ on:
push: push:
branches: branches:
- main - main
pull_request:
workflow_dispatch: workflow_dispatch:
jobs: jobs:
@@ -14,6 +15,11 @@ jobs:
- name: Checkout repository - name: Checkout repository
uses: https://gitea.com/actions/checkout@v4 uses: https://gitea.com/actions/checkout@v4
- name: Use pinned Node toolchain
uses: https://github.com/actions/setup-node@v4
with:
node-version-file: .nvmrc
- name: Install dependencies - name: Install dependencies
run: npm ci --cache ~/.npm --prefer-offline run: npm ci --cache ~/.npm --prefer-offline
+24 -3
View File
@@ -4,7 +4,6 @@ on:
push: push:
branches: [main] branches: [main]
pull_request: pull_request:
branches: [main]
# Cancel an in-progress run when newer commits are pushed to the same ref. # Cancel an in-progress run when newer commits are pushed to the same ref.
concurrency: concurrency:
@@ -17,14 +16,16 @@ jobs:
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
strategy: strategy:
fail-fast: false fail-fast: false
# Supported desktop targets are Windows and Linux. macOS is not a
# support target; do not imply it by testing on it.
matrix: matrix:
os: [ubuntu-latest, windows-latest, macos-latest] os: [ubuntu-latest, windows-latest]
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- uses: actions/setup-node@v4 - uses: actions/setup-node@v4
with: with:
node-version: 20 node-version-file: .nvmrc
cache: npm cache: npm
# The capture unit tests require app/capture.js, which require()s the # The capture unit tests require app/capture.js, which require()s the
@@ -35,3 +36,23 @@ jobs:
- name: Run unit tests - name: Run unit tests
run: npm test run: npm test
audit:
name: Dependency audit
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: .nvmrc
# Production dependencies must be clean: these ship inside packages.
- name: Audit production dependencies (blocking)
run: npm audit --omit=dev --package-lock-only --audit-level=high
# Full-tree audit including build/dev tooling is an explicit separate
# signal: it must be visible but does not block unrelated changes.
- name: Audit full dependency tree (informational)
run: npm audit --package-lock-only --audit-level=high
continue-on-error: true
+5 -38
View File
@@ -2,13 +2,13 @@ name: Release
# Manual release: from the GitHub "Actions" tab pick "Release", click # Manual release: from the GitHub "Actions" tab pick "Release", click
# "Run workflow", enter the version tag, and it builds the Windows installer # "Run workflow", enter the version tag, and it builds the Windows installer
# .exe and the Linux tarball + .deb, then publishes a GitHub Release with all # .exe, then publishes a GitHub Release with that artifact attached and
# artifacts attached and auto-generated notes. # auto-generated notes.
on: on:
workflow_dispatch: workflow_dispatch:
inputs: inputs:
version: version:
description: 'Release tag, e.g. v0.1.1 (the tag is created at the current commit on this branch)' description: 'Release tag, e.g. v0.3.2.1 (the tag is created at the current commit on this branch)'
required: true required: true
type: string type: string
prerelease: prerelease:
@@ -42,7 +42,7 @@ jobs:
shell: bash shell: bash
env: env:
VERSION: ${{ inputs.version }} VERSION: ${{ inputs.version }}
run: npm version "${VERSION#v}" --no-git-tag-version --allow-same-version run: node scripts/stamp-version.js "${VERSION#v}"
- name: Configure Windows code signing - name: Configure Windows code signing
shell: bash shell: bash
@@ -68,41 +68,9 @@ jobs:
path: releases/*.exe path: releases/*.exe
if-no-files-found: error if-no-files-found: error
build-linux:
name: Build Linux tarball + .deb
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
cache: npm
- name: Install dependencies
run: npm ci
- name: Set version from input
shell: bash
env:
VERSION: ${{ inputs.version }}
run: npm version "${VERSION#v}" --no-git-tag-version --allow-same-version
- name: Package Linux artifacts (tarball + .deb)
shell: bash
env:
STEPFORGE_PACKAGE_DIR: ${{ github.workspace }}/build/artifacts
run: bash scripts/package-linux.sh
- uses: actions/upload-artifact@v4
with:
name: linux-artifacts
path: build/artifacts/*
if-no-files-found: error
release: release:
name: Publish GitHub Release name: Publish GitHub Release
needs: [build-windows, build-linux] needs: [build-windows]
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
@@ -127,7 +95,6 @@ jobs:
fi fi
gh release create "$TAG" \ gh release create "$TAG" \
dist/windows-installer/*.exe \ dist/windows-installer/*.exe \
dist/linux-artifacts/* \
--title "StepForge $TAG" \ --title "StepForge $TAG" \
--target "${{ github.sha }}" \ --target "${{ github.sha }}" \
--generate-notes \ --generate-notes \
+2
View File
@@ -6,3 +6,5 @@ releases/
.tmp/ .tmp/
tests/.tmp/ tests/.tmp/
examples/.tmp/ examples/.tmp/
build/build_report.md
build/artifacts_manifest.json
+3
View File
@@ -0,0 +1,3 @@
# Refuse installs on Node versions outside package.json "engines".
# The locked dependency graph (electron-builder toolchain) needs Node >= 22.12.
engine-strict=true
+1
View File
@@ -0,0 +1 @@
22.17.0
+18 -9
View File
@@ -1,9 +1,11 @@
# StepForge # StepForge
StepForge is a **fully offline**, open-source desktop app for Windows and StepForge is a **fully offline**, open-source desktop app for Windows, with
Linux that captures step-by-step workflows as screenshots, lets you annotate Linux (WIP) builds. It captures step-by-step workflows as screenshots, lets
and describe each step in a focused three-pane editor, and exports the result you annotate and describe each step in a focused three-pane editor, and
to Markdown, DOCX, PPTX, PDF, HTML (WIP), GIF (WIP), confluence (WIP), Wiki.js (WIP), and image bundles (WIP). The current reconmendations for exporting is Markdown and PDF. exports the result to Markdown, DOCX, PPTX, PDF, HTML (WIP), GIF (WIP),
confluence (WIP), Wiki.js (WIP), and image bundles (WIP). The current
reconmendations for exporting is Markdown and PDF.
It is an independent offline desktop guide-capture tool inspired by publicly It is an independent offline desktop guide-capture tool inspired by publicly
documented workflow patterns of commercial documentation tools like Folge. It contains no documented workflow patterns of commercial documentation tools like Folge. It contains no
@@ -60,17 +62,24 @@ using only Node built-ins.
## Getting Started ## Getting Started
For a windows installation, see [docs/windows_installation](docs/windows_installation.md) or for a developer/more in depth walkthrough, see [docs/GETTING_STARTED.md](docs/GETTING_STARTED.md). For a Windows installation, see [docs/windows_installation](docs/windows_installation.md) or for a developer/more in depth walkthrough, see [docs/GETTING_STARTED.md](docs/GETTING_STARTED.md).
Requirements: Node.js 20+ and npm (Electron is the only dependency). On **Linux** (⚠️ work in progress — X11 vs Wayland, enabling per-click capture, the screen-share prompt), see [docs/GETTING_STARTED_WITH_LINUX.md](docs/GETTING_STARTED_WITH_LINUX.md).
Requirements: Node.js 22.12+ and npm (pinned in `.nvmrc`; installs are
refused on older Nodes because the packaging toolchain needs 22.12+).
```bash ```bash
npm install # one-time, fetches the Electron shell npm ci # one-time, installs the locked dependency tree
npm start # launch StepForge npm start # launch StepForge
``` ```
Dependencies are only ever installed by you, via `npm ci` — the app never
downloads or repairs packages at runtime.
First run creates the local data directory (`~/.local/share/stepforge` on First run creates the local data directory (`~/.local/share/stepforge` on
Linux, `%APPDATA%/stepforge` on Windows; override with `STEPFORGE_DATA_DIR`). Linux (WIP), `%APPDATA%/stepforge` on Windows; override with
`STEPFORGE_DATA_DIR`).
## Testing ## Testing
@@ -92,7 +101,7 @@ documents, and validating the bytes of the output, not string matching.
bash scripts/bootstrap-offline.sh # verify toolchain availability bash scripts/bootstrap-offline.sh # verify toolchain availability
bash scripts/verify.sh # full test suite + smoke checks bash scripts/verify.sh # full test suite + smoke checks
bash scripts/build-release.sh # assemble runnable app directory bash scripts/build-release.sh # assemble runnable app directory
bash scripts/package-linux.sh # portable tar.gz + .deb (+ AppDir spec) bash scripts/package-linux.sh # local Linux packaging (WIP; not part of release)
npm run package:windows # Windows installer .exe in releases/ npm run package:windows # Windows installer .exe in releases/
pwsh scripts/package-windows.ps1 # same Windows installer build via PowerShell pwsh scripts/package-windows.ps1 # same Windows installer build via PowerShell
``` ```
+1 -1
View File
@@ -1 +1 @@
In the future, planned updates include a re-vamp of the settings panel, text box's (important, tip, warning, and note) are not properly moving to the intended areas and I will look into locking the size (or being able to control the size) of images throughout varias export formats. In the future, planned updates include a re-vamp of the settings panel, text box's (important, tip, warning, and note) are not properly moving to the intended areas and I will look into locking the size (or being able to control the size) of images throughout varias export formats. OCR or local ai would be pretty cool....
+329
View File
@@ -0,0 +1,329 @@
# StepForge comprehensive improvement plan
This document is an implementation handoff for another coding agent. It is based on a repository-wide audit of commit `534a28e` on 2026-07-03. It is a plan, not authorization to make all changes in one unreviewable patch.
## Objective
Turn StepForge into a reliable, secure, maintainable Windows and Linux desktop application while preserving user data and the core capture/edit/export workflow. Work in small phases, add tests before or with each fix, and keep each pull request focused. Do not claim a capability until its acceptance test passes on the relevant operating system.
Linux is a platform rewrite, not a collection of `process.platform === "linux"` branches in Windows-oriented files. All Linux-only runtime, setup, packaging, and tests must live in separate Linux-specific files. Apt- and dnf-based distributions must also have separate setup/package files.
## Audit baseline
- Repository: Electron app with a dependency-light Node core.
- Approximate size: 10,949 lines in `app/`, 4,134 in `core/`, 2,658 in `exporters/`, 1,119 in `scripts/`, and 5,119 in tests.
- Largest production files are already too broad: `app/renderer/editor.js` (2,336 lines), `app/capture.js` (2,055), `app/renderer/dialogs.js` (1,039), `app/renderer/app.js` (944), and `app/main.js` (905).
- `node --check` succeeds for all checked JavaScript files.
- The full `bash tests/run_test.sh` run fails in `test_startup_smoke.sh`: Electron cannot load `libnspr4.so` on this Linux host. The earlier click self-test reports “SKIPPED,” masking that startup failure as a missing capture environment.
- Direct unit run: 205 tests, 200 passed, 1 failed, 4 skipped. The failure is `tests/unit/package-windows.test.js`, caused by the packaging dependency graph being loaded under Node 18 (`ERR_REQUIRE_ESM`). The four skipped tests are external renderer/codec validations.
- The host has Node 18.19.1. Documentation says Node 20+, but the lockfile contains packages requiring at least Node 20.19 and some packaging packages requiring Node 22.12. There is no `engines` field or hard prerequisite check.
- Sample generation and the current build-release workflow tests pass. They do not prove that the produced Linux package launches or that it has all system libraries.
- `npm audit --package-lock-only` reports two high-severity issues in build/dev dependencies (`form-data` and `undici`). `npm audit --omit=dev --package-lock-only` reports no production issue. This distinction disappears in the current Linux package because it copies all of `node_modules`, including dev/build dependencies.
- The launcher silently ran `npm install --package-lock=false` when dependencies were missing. That changed the ignored `node_modules` tree to versions different from `package-lock.json`. A desktop launcher must not repair itself by accessing npm at runtime.
- The worktree was clean before this plan; diagnostics only created ignored `node_modules` content.
## Confirmed high-priority findings
### Security and privacy
1. **A remote page can inherit the privileged preload API.** The main window has no `will-navigate` guard and no `setWindowOpenHandler`. Stored descriptions allow `https:` links. If a link navigates the main window, the preload still runs and exposes `window.stepforge` to the new page. IPC handlers do not validate sender URL/origin, and `shell:openPath` accepts an arbitrary renderer-provided target. This is a release-blocking privilege-boundary defect.
2. **The default session grants every Electron permission.** `app/main.js` uses permission handlers that return `true` for every permission and every requester. The comment that this is safe because content is local is not a security control. Grant only display capture, only to the dedicated capture worker, and reject everything else.
3. **Windows recording behaves like a keylogger.** The PowerShell/C# hook embedded in `app/capture.js` installs a global keyboard hook, reconstructs printable characters, buffers up to 200 characters, persists them in `captureMetadata`, and can send them with screenshots to an Ollama host. This can capture passwords or other sensitive text. It is not adequately disclosed or consented to. Disable raw character capture by default; ideally remove it. If retained, make it an explicit, separately consented feature with sensitive-field suppression, short in-memory lifetime, redaction, no persistence by default, and tests.
4. **“Fully offline” and “zero HTTP requests” are factually false.** `app/text-intel.js` performs configurable HTTP requests to an Ollama host and accepts a host that can be remote. The launcher can contact npm. The docs also say Electron is the only dependency, while Tesseract and its language data are production dependencies. Choose and document an accurate contract such as “local-first, no telemetry, optional user-configured Ollama,” then enforce it. If remote hosts are prohibited, validate loopback/unix-socket targets instead of accepting arbitrary HTTP endpoints.
5. **AI requests have no timeout, cancellation, size limit, or concurrency policy.** A dead Ollama endpoint can leave UI actions pending indefinitely. Full screenshots are base64-expanded into requests. Add `AbortController` deadlines, cancellation when a guide/step closes, request-size limits, bounded concurrency, and explicit data-disclosure UI.
6. **Archive import lacks resource limits and transactional extraction.** ZIP entry paths and CRCs are checked, but entry count, compressed size, inflated size, compression ratio, manifest size, and total extracted bytes are not bounded. A ZIP bomb can exhaust memory because the archive and inflated entries are handled synchronously in memory. Import writes `guide.json` before all steps validate, leaving partial guides after an error.
7. **Imported export templates can inject active HTML.** `customCss`, accent values, and other template values are interpolated into generated HTML without a typed schema. A malicious `.sfglt` can close the style element and add script. Validate every formats options, restrict colors/numbers/enums, and either remove arbitrary CSS or safely encode it with a clearly documented trusted-template boundary.
8. **The HTML sanitizer is regex-based and link navigation is not separated from rendering.** Replace ad hoc URL checks with URL parsing and a strict scheme/host policy. Add hostile HTML fixtures and ensure renderer links are intercepted rather than navigating the privileged window.
### Broken or misleading behavior
1. **Region capture returns the wrong shape.** `CaptureService.regionCapture()` stores the result of `storeFrameAsStep()` in `step` and then returns `{ ok: true, step }`. The actual step is therefore at `result.step.step`. Region capture selection and region auto-documentation expect `result.step.stepId`, so they break. Return the `storeFrameAsStep()` result directly and add an IPC-to-renderer workflow test.
2. **Cancelled region capture leaks an IPC listener.** `pickRegion()` removes `region:picked` only when an event arrives. Closing/cancelling the overlay leaves the listener and captured window references behind. Cleanup must be idempotent on pick, close, load failure, and app shutdown. Validate/clamp the received rectangle before cropping.
3. **Autosave can report clean after a failed write.** `flushStep()` and `flushGuide()` clear dirty flags before awaiting IPC. A rejected save can lose the visible dirty state and is often invoked through a debounce that does not handle rejected promises. Use a serialized save queue with states (`dirty`, `saving`, `saved`, `error`), only clear the matching revision after success, retry safely, show persistent failure UI, and flush on navigation/close.
4. **Concurrent whole-object saves can overwrite newer edits.** Editor saves, capture auto-documentation, AI generation, and background step updates all read and write full step objects with no revision check. An AI response based on stale data can overwrite user edits; `step:updated` can reload the editor while local changes are pending. Add per-guide/per-step revision numbers and compare-and-swap saves or field-level patches. Resolve conflicts explicitly.
5. **Configured automatic backups do not exist.** `backups.automatic` and `backups.everyNSaves` are defined but never used. Implement a save-count/time policy with pruning and failure reporting, or remove the settings and the documentation claim. Snapshot restore must extract into a temporary directory, validate fully, and atomically swap; the current restore deletes live content before extraction succeeds.
6. **Strict click timing still falls back to a post-click shot.** The selection logic rejects post-click frames, but `sessionCapture()` then takes a fresh shot after the click and stores it. That contradicts the strict-mode product promise. In strict mode, either keep a sufficiently healthy pre-click buffer or skip the capture with a visible diagnostic; never label a post-click fallback as strict.
7. **Linux evdev state is reported incorrectly.** `startEvdevWatcher()` does not set `clickWatcher`, so `state().clickCapture` is false even when evdev is active. The UI can say “hotkey only” while clicks are being watched. Device stream errors are swallowed and do not trigger fallback. Represent trigger sources as explicit states (`windows-hook`, `x11`, `wayland-helper`, `hotkey`, `interval`, `unavailable`) rather than a boolean.
8. **Power blocker ownership is wrong.** The IPC `start` action starts a power-save blocker even though new sessions begin paused. Tray/second-instance pauses bypass the main-process closure that stops it. Move power management behind capture state transitions so there is exactly one owner and assert that paused/finished sessions release it.
9. **File URLs are assembled by string concatenation.** `file://${p}` breaks on spaces, `#`, `%`, Windows drive letters, and other characters. Use `pathToFileURL()` and remove renderer control of arbitrary filesystem paths.
10. **Search can silently remain empty.** If the index is missing, corrupt, or version-mismatched, the constructor starts empty and does not reconcile all existing guides. Rebuild incrementally at startup, store a source revision/fingerprint, and expose recovery status instead of silently swallowing failures.
11. **Corrupt user data is silently hidden.** `listGuides()` and `listSteps()` skip unreadable entries. Corrupt settings are silently replaced in memory. Quarantine corrupt files, preserve originals, surface a recovery UI/report, and never make a guide disappear without explanation.
12. **Several public settings/schema fields are dead or incomplete.** `language`, `capture.includeCursor`, `editor.autoTitleTemplate`, `library.sortBy`, automatic backup settings, `themeOverride`, `exportProfiles`, `extraImages`, and parts of `links` are unused or only partially used. Implement them end-to-end or remove/migrate them; do not keep misleading UI/data contracts.
13. **Linked-guide locking is racy and barely observable.** Lock acquisition is read-then-write rather than exclusive creation, locks exist only for the short save operation, and two writers can race. Define whether the lock covers the editing session or just a write transaction. Use atomic exclusive creation plus ownership token, heartbeat/stale recovery if session-scoped, and conflict detection based on archive hash/revision before overwrite.
### Export correctness and scalability
1. `renderAllImages()` retains every decoded/rendered RGBA image. A single 4K image is roughly 32 MiB before copies; a large guide can consume gigabytes and terminate the export worker. Render one step at a time, write/consume it, release buffers, and report progress/cancellation.
2. PNG and ZIP decoding are synchronous and memory-heavy. Dimension-only limits still permit enormous allocations (up to 32,768 squared). Set total pixel/byte budgets, validate exact inflated length rather than only “at least,” and move heavy work off the main process.
3. The PDF writer replaces unsupported Unicode with `?`; raster annotation text uses an ASCII 8x8 font; the editor uses system fonts. Therefore the documented WYSIWYG and international-text claims are false. Vendor a properly licensed Unicode font or adopt a vetted renderer, embed/subset fonts, and add multilingual fixtures.
4. Editor and export rendering differ for blur, typography, antialiasing, tooltip layout, and potentially focused-view geometry. Build shared geometry/style calculations and golden-image comparisons with explicit tolerances.
5. Text-block position behavior needs a complete format matrix. Current grouping supports six positions for text blocks, while code/table blocks always fall into `rest`. Existing tests do not prove every position in PDF, DOCX, PPTX, HTML, Markdown, Confluence, and Wiki.js. Fix the reported callout movement issue by defining one canonical ordered content stream and testing every exporter against it.
6. Image sizing is format-specific and inconsistent. Introduce a canonical image layout policy (`natural`, `fit-content-width`, explicit max width/height, preserve aspect ratio, no-upscale) and map physical units correctly for HTML/CSS pixels, PDF points, DOCX twips, and PPTX EMUs. Make it configurable in export profiles and test portrait, landscape, ultrawide, small, and 4K images.
7. Markdown output does not robustly escape table pipes/newlines or choose a safe code-fence length when code contains backticks. Add escaping/conformance tests. Validate Office packages by opening/rendering with LibreOffice in Linux CI, not just by checking ZIP/XML structure. Validate PDF with Ghostscript/Poppler and images/GIF with external tools in a dedicated integration job.
8. Export writes directly into the selected output directory and can leave partial/stale files. Export into a temporary sibling directory, validate, then atomically publish. Define overwrite behavior and clean obsolete sidecar images.
### Build, packaging, and release
1. The current Linux package script is not production packaging. It copies all `node_modules` (including dev tools and vulnerable build dependencies), docs, prompts, examples, and stale audit files; hardcodes `amd64`; declares only `xinput`; lacks desktop entry/icons/MIME integration; and copies a nonexistent root `LICENSE`. The portable tarball excludes the generated `/usr/bin/stepforge` launcher because it archives only `opt/stepforge`.
2. A clean run can build a package without `node_modules`, producing an unusable artifact while tests still pass. Package tests only inspect file existence, not launch/install behavior.
3. Linux startup currently falls back to `--no-sandbox` whenever `chrome-sandbox` is not setuid-root. Do not normalize an unsandboxed production launch. Use a packaging method/configuration that supports Chromium sandboxing (or user namespaces where supported), fail with actionable diagnostics, and reserve `--no-sandbox` for explicitly marked development/CI environments.
4. The launcher auto-repairs/reinstalls Electron with npm and ignores the lockfile. Remove all runtime installation. Development setup uses `npm ci`; packaged applications contain a fixed Electron runtime.
5. The Windows artifact finder returns the first `.exe` encountered and can select the unpacked app executable instead of the NSIS installer. Select the expected artifact by exact pattern/metadata and fail on zero or multiple matches.
6. There is no real app icon/assets directory even though architecture docs claim one, and the Windows test explicitly asserts assets are not packaged. Add licensed original assets and verify Windows/Linux metadata.
7. Versioning is inconsistent: package version, four-part build version, tags, changelog, and committed build reports disagree. Use SemVer for releases, a separate platform file/build version where needed, and generate all metadata from one source. Do not commit stale machine-specific build reports/manifests as if current.
8. The license is contradictory. `package.json` says `MPL-2.0`, contribution docs require MPL-2.0/DCO, while `docs/LICENSE` and README impose a noncommercial license. There is no root `LICENSE`. The owner must choose one license before the next release; then make the SPDX field, root license text, README, contribution policy, package contents, and generated About view agree. This is a legal release blocker and cannot be guessed by an implementation agent.
9. GitHub CI runs only `npm test`, only on pushes to `main`; docs claim full checks on pull requests. Release builds only Windows. Add `pull_request`, run the same authoritative commands everywhere, and add Linux package jobs. Do not allow the click E2E test to convert arbitrary startup failures into skips.
## Target architecture
Refactor incrementally toward these boundaries; do not perform a blind rewrite of the whole app.
```text
app/
main/ lifecycle, window policy, IPC composition
renderer/ views/components with no filesystem privilege
capture/ platform-neutral session state machine and frame pairing
platform/
windows/ Windows hooks, context, power behavior
linux/ Linux session detection, portal/X11 input, window policy
services/ export, AI/OCR, search coordination
core/
domain/ guide/step/block model and migrations
storage/ transactional repository, recovery, snapshots, locks
render/ canonical document layout and annotation geometry
exporters/ thin format adapters consuming canonical layout
packaging/
windows/
linux/
debian/
fedora/
scripts/
linux/apt/
linux/dnf/
tests/
unit/
integration/
e2e/
fixtures/
```
Use dependency injection for OS adapters. The platform-neutral capture coordinator should consume interfaces such as `ClickSource`, `ScreenFrameSource`, `WindowContextProvider`, `WindowVisibilityPolicy`, and `PowerPolicy`. It should never inspect `process.platform` itself. `app/platform/index.js` is the only factory that selects a platform implementation.
Introduce a schema-v2 migration with a single ordered `blocks[]` collection instead of three arrays plus synthesized order. Keep a tested v1 reader/migrator and never rewrite user data without a pre-migration snapshot. Add `revision` fields for optimistic concurrency.
## Phased implementation plan
### Phase 0 — freeze the contract and make the baseline reproducible
- Resolve the license decision and the offline/local-AI product wording with the owner.
- Choose one supported Node LTS that satisfies the entire locked dependency graph (the current graph requires at least Node 22.12 for packaging), add `engines`, `.nvmrc` or `.node-version`, and a hard version check in setup/CI.
- Make `npm ci` the only dependency installation path. Remove auto-install/repair from `scripts/electron-launcher.js` and keep clear diagnostics.
- Refresh and pin the lockfile on the chosen Node/npm version; remediate the two audited build dependency issues. Add production and full dependency audits as separate CI signals with an explicit policy.
- Split tests into deterministic unit, desktop smoke, platform capture E2E, export integration, and package install/launch suites. A missing display may skip only a capture scenario after the app has demonstrably started; a missing shared library or crash must fail.
- Add `pull_request` CI. Run syntax/lint/type checks, unit tests, and artifact checks on Linux and Windows. Keep macOS core tests only if macOS is an intended support target; otherwise stop implying app support.
- Record baseline performance fixtures: 100-step 1080p guide, 25-step 4K guide, large archive, and rapid-click session. Track peak RSS, export time, save latency, and dropped-click count.
### Phase 1 — close privilege boundaries and data-loss paths
- In the main window, reject all navigation away from the exact local app entry URL. Add `setWindowOpenHandler(() => ({ action: "deny" }))`. Route safe external links through a narrow `openExternal` handler after scheme validation and optional confirmation.
- Validate IPC sender/webContents for every handler. Add per-channel input schemas, length/size limits, enum checks, and ownership/path checks. Remove generic `shell:openPath`/`showItemInFolder` from the renderer; replace them with intent-specific commands for known export, preview, data, and linked-archive paths.
- Set `sandbox: true` explicitly for every renderer. Deny all permissions by default and grant display capture only to the capture worker and only for the apps local URL. Add security regression tests for remote navigation, popup attempts, permission requests, malicious stored HTML, and hostile template archives.
- Remove or default-disable global printable-key capture. Add a privacy disclosure for screenshot/OCR/window-title/AI data. Never persist raw typed text unless the user explicitly opts in.
- Add AI timeouts, cancellation, concurrency limits, loopback policy if required, payload limits, and error states. Ensure stale AI responses cannot overwrite edited revisions.
- Build the serialized revision-aware autosave queue. Keep dirty state on failure, show last successful save time, flush before navigation/quit, and block destructive close only when a flush genuinely fails.
- Make guide/step save operations transactional at the guide level where multiple files must change. Add recovery journals or temp-directory swaps for add/delete/reorder/import/restore. Quarantine and report corrupt data.
- Add archive/template limits and preflight validation. Extract/import into temp storage, validate manifest/schema/all referenced files, then publish atomically.
### Phase 2 — fix known workflows before larger refactors
- Fix region captures nested result and listener cleanup. Add tests covering capture service → IPC → renderer selection → optional AI.
- Implement automatic snapshots or remove the dead settings. Make restore atomic and verify rollback after injected failures.
- Rebuild/reconcile the search index at startup and test deletion/corruption/version upgrade.
- Replace file URL concatenation with `pathToFileURL()` and test Windows, spaces, Unicode, `#`, and `%` paths.
- Fix power blocker transitions, evdev trigger reporting, watcher-loss fallback, pending click drain on application shutdown, and strict-mode post-click behavior.
- Validate and clamp all persisted geometry and settings. Reject NaN/infinite/negative image sizes, cyclic parent relationships, invalid block IDs/orders, unsafe image paths, out-of-range focused views, and oversized strings/arrays.
- Inventory every schema/settings field and either implement, migrate, or delete it. Update UI and docs in the same PR.
### Phase 3 — Linux rewrite with separate files (apt and dnf)
This phase must not add more Linux conditionals to `app/capture.js`, `app/text-intel.js`, `app/main.js`, or the Windows hook. First introduce platform interfaces, preserve the tested Windows adapter, and then write Linux implementations in new files.
Create at minimum:
```text
app/platform/index.js
app/platform/windows/capture-adapter.js
app/platform/windows/click-hook.cs
app/platform/windows/window-context.ps1
app/platform/windows/power-policy.js
app/platform/linux/capture-adapter.js
app/platform/linux/session-detection.js
app/platform/linux/portal-frame-source.js
app/platform/linux/x11-click-source.js
app/platform/linux/wayland-click-source.js
app/platform/linux/window-context-x11.js
app/platform/linux/window-policy.js
app/platform/linux/diagnostics.js
scripts/linux/apt/install-build-deps.sh
scripts/linux/apt/install-runtime-deps.sh
scripts/linux/dnf/install-build-deps.sh
scripts/linux/dnf/install-runtime-deps.sh
packaging/linux/debian/package.sh
packaging/linux/debian/control.in
packaging/linux/fedora/package.sh
packaging/linux/fedora/stepforge.spec
packaging/linux/common/stepforge.desktop
packaging/linux/common/stepforge-mime.xml
packaging/linux/common/launcher.sh
docs/linux/apt.md
docs/linux/dnf.md
tests/integration/linux/x11-capture.test.js
tests/integration/linux/wayland-capture.test.js
tests/integration/linux/package-deb.test.sh
tests/integration/linux/package-rpm.test.sh
```
Requirements:
- Support both X11 and Wayland as different capability profiles. X11 may use a separately implemented `xinput` adapter with event-time coordinates. Wayland must use XDG Desktop Portal/PipeWire for screen selection and capture.
- Do not promise global per-click capture with coordinates on Wayland when the platform does not expose it. The safe baseline is portal screen capture plus user-triggered global hotkey or interval capture. Treat direct `/dev/input` access as an optional, explicitly consented privileged mode, not default setup.
- Remove documentation that casually tells every user to join the broad `input` group. If a privileged helper is retained, perform a threat review, use least-privilege device rules, never read keyboard devices, package it separately, and show the security tradeoff before enabling it.
- Detect portal, PipeWire, compositor/session, sandbox, required shared libraries, xinput availability, and permission state in Linux diagnostics. Return actionable UI messages instead of console-only failures.
- On Wayland, map the portal-selected monitor to actual frame metadata. Do not assume `displays[0]` represents the selected screen. Test single/multiple monitors, mixed DPI, negative origins on X11, portal cancellation, stream revocation, suspend/resume, and monitor hotplug.
- Keep Linux minimize/restore/tray behavior in `window-policy.js`; do not branch inside the capture coordinator.
- Apt and dnf runtime dependency lists must be maintained in their separate files and verified in clean Debian/Ubuntu and Fedora containers/VMs. Include Chromium/Electron shared libraries, portal/PipeWire integration, and X11 tools only where needed. Do not install build tools in end-user packages.
- Produce a real `.deb` and `.rpm` from a pruned packaged Electron application. Never copy the development `node_modules` tree. Include architecture mapping (`x64`, `arm64` if supported), icons, desktop entry, categories, MIME registration, license, uninstall behavior, and sandbox-compatible permissions.
- Add Linux artifacts to release CI with checksums/SBOM. Install each artifact in a clean VM/container where possible, launch a smoke screen under Xvfb for X11, and run a real Wayland compositor test job for portal behavior. A package is not accepted merely because `dpkg-deb` or `rpmbuild` produced a file.
Linux acceptance criteria:
- Fresh apt-based and dnf-based systems can follow separate documented setup paths and launch StepForge without `--no-sandbox` or manual npm commands.
- Fullscreen, region, clipboard/import, edit, save, reopen, and export workflows pass on both distro families.
- X11 click capture preserves event time and marker position across DPI/monitors.
- Wayland asks for screen sharing once per recording, handles cancel/revoke, never loops portal prompts, and accurately reports whether the active trigger is hotkey, interval, or an approved click source.
- `.deb` and `.rpm` contain only runtime files and pass install, upgrade, uninstall, dependency, license, desktop-entry, and launch tests.
### Phase 4 — canonical editor/document model
- Migrate to a single ordered block list with text/code/table discriminated types and explicit anchors (`before-title`, `after-title`, `before-description`, `after-description`, `before-image`, `after-image`). Decide whether code/table can use anchors; enforce the decision consistently.
- Refactor the editor into bounded modules: guide state/autosave, step tree, properties form, block editor, annotation controls, capture controls, export dialog, and command history. Avoid framework migration unless it has a measured benefit and an approved dependency cost.
- Replace deprecated `document.execCommand` with an explicit editor model or a small audited implementation. Preserve selection safely, sanitize paste, and implement real link editing instead of inserting `[Text](Link)` placeholders.
- Unify undo/redo around commands and revisions. Include block edits, step metadata, crop/reset, reorder, delete/restore, and AI changes. Do not keep full base64 image copies in renderer history without a bounded disk-backed strategy.
- Make annotation geometry bounded and reusable. Share style/geometry calculations between canvas and raster export. Add rotation/layering only after parity is tested.
- Fix callout placement with exporter matrix fixtures. Add export image sizing controls and saved per-format profiles.
- Add accessibility: semantic buttons, modal roles/names, focus trap and restoration, keyboard traversal, visible focus, screen-reader labels, reduced-motion support, high contrast, and a non-canvas representation of annotations. Run automated accessibility checks plus manual keyboard testing.
- Improve responsive behavior below the current 880px minimum and at 125200% UI scale. Preserve pane sizes and window bounds per platform.
### Phase 5 — storage, performance, and export hardening
- Add explicit schema migration functions and fixture coverage for every historical schema. Back up before migration and make migrations idempotent.
- Add storage integrity scanning: guide/order references, orphan steps/images, duplicate IDs, missing originals/workings, invalid parents, and recoverable temp files. Provide repair/dry-run output.
- Replace repeated synchronous whole-index writes with an incremental, crash-safe index and background reconciliation. Measure search on large libraries.
- Stream archives and exports where practical. At minimum enforce byte/pixel budgets and render/release one step at a time. Add export progress, cancellation, and worker termination cleanup.
- Introduce a canonical layout layer that computes content order and image constraints once. Keep exporters thin.
- Add Unicode-capable text rendering and licensed embedded fonts. Test CJK, RTL, emoji policy, combining marks, smart punctuation, and long unbroken tokens. If a format cannot support a case, fail or document it rather than substituting silently.
- Add reproducible/golden output tests. Normalize timestamps/IDs where required, render PDF/DOCX/PPTX to images in integration CI, and compare meaningful layout rather than only container structure.
- Add stress/fault tests: disk full, permission denied, interrupted atomic rename, corrupted JSON, ZIP bomb, huge PNG, export worker crash, Ollama timeout, capture worker death, rapid app quit, and concurrent saves.
### Phase 6 — packaging, release, and documentation completion
- Use one packaging system/config source for Windows and Linux where possible, with platform-specific files under `packaging/`. Prune production dependencies and generate an SBOM/license notice.
- Add original icons at required resolutions. Sign Windows artifacts before recommending users bypass SmartScreen; sign/package Linux repositories if repository distribution is introduced.
- Build release artifacts from a clean checkout with `npm ci`, fixed toolchain versions, no dirty files, and no network during the packaging stage. Generate checksums and provenance.
- Test upgrade compatibility using real prior-version user data and installed packages.
- Rewrite README, architecture, security, getting-started, Linux apt/dnf, privacy/AI, file format, and troubleshooting docs to match tested behavior. Remove stale “WIP”/“fixed” claims and stale machine-specific build reports.
- Correct spelling/grammar and links, compress oversized documentation screenshots, and keep generated sample outputs either reproducible and CI-verified or out of version control.
## File-specific work map
- `app/main.js`: split lifecycle/IPC/security policy; navigation guards; permission allowlist; sender/input validation; path intents; capture power ownership.
- `app/capture.js`: reduce to platform-neutral session coordinator, then move every OS branch to adapters; fix region result/listener, strict fallback, shutdown drain, explicit trigger state.
- `app/text-intel.js`: split OCR, platform window context, and Ollama client; remove embedded OS scripts; add privacy controls, timeout/cancel/limits.
- `app/stream-backend.js` and worker: authenticated worker-only IPC, selected-display metadata, cancellation, bounded frames/encodes, lifecycle tests.
- `app/renderer/editor.js`: save state machine, revision conflicts, module split, canonical blocks, reliable undo, modern rich text, accessibility.
- `app/renderer/dialogs.js`: typed settings forms, validation, modal focus/ARIA, safe template options, AI disclosure.
- `core/schema.js`: schema v2, strict validation, bounds, migrations, revisions, unified blocks.
- `core/store.js`: transactions, corruption quarantine, async/heavy-operation strategy, integrity scan, conflict-aware patches.
- `core/archive.js`, `core/zip.js`, `core/snapshots.js`, `core/locks.js`: resource limits, temp validation/atomic swap, exclusive locks/revisions, rollback tests.
- `core/search.js`: startup reconciliation, incremental persistence, visible recovery.
- `core/renderast.js`, `core/raster.js`, `core/pdf.js`: canonical layout, lazy image rendering, WYSIWYG parity, Unicode/font work, resource limits.
- `exporters/*`: typed option schemas, safe escaping, streaming/lazy images, consistent anchors/image sizing, external conformance tests.
- `scripts/electron-launcher.js`: diagnostics only; never install or weaken production sandbox.
- `scripts/package-windows.js`: exact installer selection, assets, signing hooks, clean artifact verification.
- `.github/workflows/*` and `.gitea/workflows/*`: PR triggers, authoritative test commands, Linux distro/package matrix, non-masking E2E behavior, release artifacts/provenance.
- `README.md`, `docs/*`, `package.json`, root `LICENSE`: reconcile support, dependencies, AI/network/privacy, version, license, and build instructions.
## Required test layers
1. **Pure unit tests:** schema/migrations, storage transactions, sanitizer/URLs, archive limits, frame selection, platform parsers, layout calculations, exporter escaping.
2. **IPC contract tests:** instantiate handlers with fake senders and prove invalid origins, paths, sizes, and payloads are rejected. Do not rely on regex extraction alone.
3. **Renderer tests:** save failures/retries, navigation with dirty state, capture-added and AI races, block placement, modals/focus, keyboard and accessibility.
4. **Desktop E2E:** launch packaged/unpackaged app, create/capture/import/edit/save/restart/export. Separate Windows, Linux X11, and Linux Wayland scenarios with explicit capability expectations.
5. **Artifact tests:** install/launch/uninstall `.exe`, `.deb`, and `.rpm`; inspect file lists and dependencies; verify sandbox, icons, desktop integration, version, license, and clean upgrades.
6. **External output tests:** open/render PDF, DOCX, PPTX, HTML, GIF, and images with independent tools; include visual fixtures and multilingual content.
7. **Security/fault tests:** hostile navigation, malicious HTML/template/archive, ZIP bomb budgets, arbitrary IPC paths, permission denial, disk failures, worker crashes, stale AI responses, and captured-secret prevention.
## Definition of done
- No release-blocking security or license contradiction remains.
- No production launch path downloads dependencies or uses `--no-sandbox` by default.
- User edits remain visibly dirty until durably saved; injected failures and concurrent AI/capture updates do not lose data.
- Automatic backups, restore, archive import, and linked saves are transactional and tested.
- Windows, apt-based Linux, and dnf-based Linux use separate platform/setup/package files and pass their documented capability matrices.
- Linux `.deb` and `.rpm` install and launch from clean systems with only runtime dependencies.
- Region capture, callout placement, image sizing, Unicode, large-guide exports, and click-session shutdown have regression tests.
- CI runs on pull requests, cannot hide startup crashes as skips, and tests the same commands documented for contributors.
- README, Security, Architecture, Privacy/AI, support matrix, package metadata, changelog, and license all describe the shipping application accurately.
## Recommended PR sequence
1. Reproducible toolchain/CI and test-runner truthfulness.
2. Navigation/IPC/permission security boundary.
3. Privacy and AI/network contract.
4. Revision-aware autosave and transactional storage.
5. Region capture, power/session state, and shutdown fixes.
6. Archive/snapshot/lock/search recovery hardening.
7. Platform interface extraction with Windows behavior preserved.
8. Linux apt/X11 implementation and `.deb` packaging.
9. Linux dnf/X11 implementation and `.rpm` packaging.
10. Linux Wayland portal implementation and honest fallback behavior.
11. Canonical blocks/callout placement and image sizing.
12. Lazy exports, Unicode rendering, and external conformance tests.
13. Editor modularization/accessibility/UX polish.
14. Signed, reproducible release pipeline and final documentation reconciliation.
Do not combine these into one PR. Each PR must include migration/rollback notes where user data or package layout changes, automated tests proportional to risk, and a short manual verification matrix for the affected operating systems.
+603 -50
View File
@@ -1,9 +1,9 @@
'use strict'; 'use strict';
const path = require('node:path'); const path = require('node:path');
const fs = require('node:fs');
const { spawn, execFileSync } = require('node:child_process'); const { spawn, execFileSync } = require('node:child_process');
const { desktopCapturer, screen, BrowserWindow, nativeImage, Tray, Menu } = require('electron'); const { desktopCapturer, screen, BrowserWindow, nativeImage, Tray, Menu } = require('electron');
const { expandPlaceholders } = require('../core/placeholders');
const raster = require('../core/raster'); const raster = require('../core/raster');
const { encodePng } = require('../core/png'); const { encodePng } = require('../core/png');
const { const {
@@ -14,6 +14,7 @@ const {
DEFAULT_START_SLACK_MS, DEFAULT_START_SLACK_MS,
} = require('./click-frames'); } = require('./click-frames');
const { physicalToDip } = require('./coords'); const { physicalToDip } = require('./coords');
const { DEFAULT_CAPTURE_TITLES } = require('../core/text-intel');
/** /**
* Capture service: full-screen, active-window, and region capture, plus a * Capture service: full-screen, active-window, and region capture, plus a
@@ -107,6 +108,88 @@ function hasBinary(name) {
} }
} }
// On Wayland, xinput only sees XWayland (X11-bridge) events — native Wayland
// app clicks are delivered via the Wayland protocol and never reach xinput.
// Treating it as "available" would leave the session with no click capture AND
// no interval fallback, so zero steps get captured.
function isWayland() {
if (process.platform !== 'linux') return false;
// XDG_SESSION_TYPE is the authoritative session hint when present. Some
// desktops still export WAYLAND_DISPLAY even when the active session is X11,
// so only fall back to it when XDG_SESSION_TYPE is unavailable.
const sessionType = String(process.env.XDG_SESSION_TYPE || '').toLowerCase();
if (sessionType) return sessionType === 'wayland';
return Boolean(process.env.WAYLAND_DISPLAY);
}
// ---- evdev (Linux kernel input) click reader --------------------------------
// Reading /dev/input/event* directly sees mouse-button presses on BOTH X11 and
// Wayland, because it taps the kernel input layer below the display server —
// the one global-click source that survives Wayland's security model. It needs
// read access to the device nodes (the user must be in the `input` group).
//
// Each event is a fixed-size `struct input_event`: a timeval, then u16 type,
// u16 code, s32 value. The timeval is two `long`s, so the record is 24 bytes on
// 64-bit and 16 on 32-bit; type/code/value always sit in the last 8 bytes.
const EV_KEY = 0x01;
const EVDEV_PRESS = 1;
// BTN_LEFT/RIGHT/MIDDLE -> the same button-N naming the xinput path emits
// (1=left, 2=middle, 3=right) so downstream debounce/marker logic is identical.
const EVDEV_BUTTONS = { 272: 'button-1', 273: 'button-3', 274: 'button-2' };
const EVDEV_RECORD_SIZE = (process.arch === 'x64' || process.arch === 'arm64'
|| process.arch === 'ppc64' || process.arch === 's390x' || process.arch === 'loong64'
|| process.arch === 'riscv64') ? 24 : 16;
/**
* Decode a buffer of packed input_event records, returning the button presses
* found and any trailing partial record (a device read can split mid-record).
* Pure and size-parameterised so it is unit-testable without real devices.
*/
function decodeEvdevButtonPresses(buffer, recordSize = EVDEV_RECORD_SIZE) {
const presses = [];
let offset = 0;
while (buffer.length - offset >= recordSize) {
const type = buffer.readUInt16LE(offset + recordSize - 8);
const code = buffer.readUInt16LE(offset + recordSize - 6);
const value = buffer.readInt32LE(offset + recordSize - 4);
offset += recordSize;
if (type === EV_KEY && value === EVDEV_PRESS && EVDEV_BUTTONS[code]) {
presses.push(EVDEV_BUTTONS[code]);
}
}
return { presses, rest: buffer.subarray(offset) };
}
/**
* The /dev/input/event* nodes for pointing devices that are readable by this
* process. /proc/bus/input/devices lists every device with its Handlers line;
* a pointing device exposes a `mouseN` handler, and the matching `eventN` is
* the node to read. Unreadable nodes (no `input` group membership) are skipped.
*/
function readableEvdevMouseNodes() {
const nodes = [];
let table;
try {
table = fs.readFileSync('/proc/bus/input/devices', 'utf8');
} catch {
return nodes;
}
for (const block of table.split('\n\n')) {
const handlers = /H:\s*Handlers=([^\n]*)/.exec(block);
if (!handlers || !/\bmouse\d+\b/.test(handlers[1])) continue;
const event = /\bevent(\d+)\b/.exec(handlers[1]);
if (!event) continue;
const node = `/dev/input/event${event[1]}`;
try {
fs.accessSync(node, fs.constants.R_OK);
nodes.push(node);
} catch {
// Not readable — user is not in the `input` group for this node.
}
}
return nodes;
}
class CaptureService { class CaptureService {
constructor({ constructor({
store, store,
@@ -114,6 +197,7 @@ class CaptureService {
getWindow, getWindow,
notify, notify,
screenApi = screen, screenApi = screen,
textIntel = null,
}) { }) {
this.store = store; this.store = store;
this.settings = settings; this.settings = settings;
@@ -122,6 +206,10 @@ class CaptureService {
// Injectable for tests; the click/coordinate paths must never reach for // Injectable for tests; the click/coordinate paths must never reach for
// the global `screen` directly so coordinate handling stays testable. // the global `screen` directly so coordinate handling stays testable.
this.screen = screenApi; this.screen = screenApi;
this.textIntel = textIntel;
// Cached display-server detection. A method (onWayland) reads this so tests
// can flip platform behavior without touching process.env.
this._wayland = isWayland();
this.session = null; // { guideId, paused, count, intervalSec } this.session = null; // { guideId, paused, count, intervalSec }
this.intervalTimer = null; this.intervalTimer = null;
this.clickWatcher = null; this.clickWatcher = null;
@@ -133,6 +221,11 @@ class CaptureService {
this.clickWatcherErrTail = ''; this.clickWatcherErrTail = '';
this.linuxEvent = null; // event block currently being parsed this.linuxEvent = null; // event block currently being parsed
this.pendingRawClick = null; // raw press waiting for its coordinate twin this.pendingRawClick = null; // raw press waiting for its coordinate twin
this.pendingClickOsPoint = null;
this._keyBuffer = ''; // printable chars typed since last capture
this._lastShortcut = ''; // last Ctrl+X / Alt+X combination
this._keyLastAt = 0; // timestamp of last key event
this._pendingWindowContext = null; // buffered CTX+ELEM from click watcher, consumed on CLICK
this.clickQueue = Promise.resolve(); this.clickQueue = Promise.resolve();
this.frameLoopInFlight = false; this.frameLoopInFlight = false;
this.frameLoopGrabStartedAt = null; this.frameLoopGrabStartedAt = null;
@@ -175,21 +268,62 @@ class CaptureService {
return this.settings.get('capture.strictClickFrames') !== false; return this.settings.get('capture.strictClickFrames') !== false;
} }
fallbackCaptureTrigger() {
const raw = String(this.settings.get('capture.fallbackTrigger') || 'interval').toLowerCase();
return raw === 'hotkey' ? 'hotkey' : 'interval';
}
fallbackIntervalSec() {
const raw = Number(this.settings.get('capture.autoIntervalSec'));
return Number.isFinite(raw) && raw > 0 ? raw : 5;
}
clickCaptureAvailable() { clickCaptureAvailable() {
if (this._clickAvail === undefined) { if (this._clickAvail === undefined) {
this._clickAvail = process.platform === 'win32' || (process.platform === 'linux' && hasBinary('xinput')); // Three click sources, in order of fidelity:
// - Windows: the low-level mouse hook (position + timing);
// - X11: xinput test-xi2 (position + timing) — but it can't see native
// Wayland clicks, only XWayland ones, so it's gated to non-Wayland;
// - Linux evdev (/dev/input): button presses on X11 AND Wayland, but no
// cursor position on Wayland — used for per-click capture there (no
// marker). Requires the user to be in the `input` group.
this._clickAvail = process.platform === 'win32'
|| (process.platform === 'linux' && !this.onWayland() && hasBinary('xinput'))
|| (process.platform === 'linux' && readableEvdevMouseNodes().length > 0);
} }
return this._clickAvail; return this._clickAvail;
} }
/** Whether this is a Wayland session (cached; overridable in tests). */
onWayland() {
return this._wayland;
}
/**
* Whether the in-process frame loop is a usable fallback recorder. It grabs
* via desktopCapturer.getSources(), which on Wayland is broken (throws) and
* pops the portal — so the loop is viable only off Wayland. On Wayland the
* portal-backed stream backend is the sole capture path.
*/
canUseFrameLoop() {
return !this.onWayland();
}
startSession(guideId, { intervalSec = null } = {}) { startSession(guideId, { intervalSec = null } = {}) {
this.finishSession(); this.finishSession();
// Default trigger: clicks when the platform supports it, otherwise an // Default trigger: clicks when the platform supports it, otherwise the
// interval so a session always produces steps even if the global hotkey // user-selected fallback (timer or hotkey-only). That keeps Linux from
// never fires (common under Wayland/WSLg). // silently dropping into an unwanted 5-second timer when click capture
// is unavailable.
let interval = intervalSec; let interval = intervalSec;
if (interval == null) { if (interval == null) {
interval = this.clickCaptureAvailable() ? 0 : (this.settings.get('capture.autoIntervalSec') || 5); if (this.clickCaptureAvailable()) {
interval = 0;
} else if (this.fallbackCaptureTrigger() === 'hotkey') {
interval = 0;
} else {
interval = this.fallbackIntervalSec();
}
} }
// Sessions start paused: nothing hides and no capturing happens until // Sessions start paused: nothing hides and no capturing happens until
// the user explicitly presses "Start recording" in the capture bar, so // the user explicitly presses "Start recording" in the capture bar, so
@@ -293,6 +427,7 @@ class CaptureService {
showWindow() { showWindow() {
const win = this.getWindow(); const win = this.getWindow();
if (win && !win.isDestroyed()) { if (win && !win.isDestroyed()) {
if (win.isMinimized()) win.restore();
win.show(); win.show();
win.focus(); win.focus();
} }
@@ -314,7 +449,14 @@ class CaptureService {
const sec = this.session && this.session.intervalSec; const sec = this.session && this.session.intervalSec;
if (sec > 0) { if (sec > 0) {
this.intervalTimer = setInterval(() => { this.intervalTimer = setInterval(() => {
this.sessionCapture('interval').catch(() => {}); // Don't let a slow capture (e.g. a multi-second software PNG encode on
// a GPU-less host) overlap with the next tick — overlapping requests
// would pile up and could trip the backend's failure counter.
if (this.intervalCapturing) return;
this.intervalCapturing = true;
this.sessionCapture('interval')
.catch(() => {})
.finally(() => { this.intervalCapturing = false; });
}, sec * 1000); }, sec * 1000);
} }
} }
@@ -351,8 +493,13 @@ class CaptureService {
armRecording() { armRecording() {
const win = this.getWindow(); const win = this.getWindow();
const wantHide = Boolean(this.hiddenForSession && win && !win.isDestroyed()); const wantHide = Boolean(this.hiddenForSession && win && !win.isDestroyed());
const recorderWanted = this.settings.get('capture.captureOutsideClicks') !== false // Always start the frame recorder when stream capture is enabled — it
&& this.clickCaptureAvailable(); // buffers frames for click captures AND is used for interval/hotkey
// captures to avoid calling desktopCapturer.getSources() on every capture.
// On Linux/Wayland, each getSources() call goes through the XDG portal and
// shows a permission dialog; the stream backend eliminates that by keeping
// a live video stream open for the duration of the recording session.
const recorderWanted = this.settings.get('capture.streamCapture') !== false;
// Recording is not "live" until the window is hidden and the buffer is // Recording is not "live" until the window is hidden and the buffer is
// primed. While warming up, the window is still visible and over the // primed. While warming up, the window is still visible and over the
// user's work, so clicks in this period are ignored (onOsClick checks // user's work, so clicks in this period are ignored (onOsClick checks
@@ -376,7 +523,17 @@ class CaptureService {
if (!this.session || this.session.paused) { this.warmingUp = false; return; } if (!this.session || this.session.paused) { this.warmingUp = false; return; }
} }
if (wantHide && win && !win.isDestroyed() && win.isVisible()) { if (wantHide && win && !win.isDestroyed() && win.isVisible()) {
// On Linux, always minimize rather than hide. GNOME's system tray
// (StatusNotifier) is unreliable — it can fail or half-export over
// dbus — so a hidden window can be impossible to bring back, leaving
// the user unable to stop the recording. A minimized window is always
// restorable from the taskbar, and minimized windows aren't rendered
// so they still stay out of the fullscreen capture.
if (process.platform === 'linux') {
win.minimize();
} else {
win.hide(); win.hide();
}
// Let a couple of frames of the now-unobscured screen land before // Let a couple of frames of the now-unobscured screen land before
// the user's first click, so that frame shows their work, not the // the user's first click, so that frame shows their work, not the
// app window that was just dismissed. // app window that was just dismissed.
@@ -463,7 +620,7 @@ class CaptureService {
if (frame) { if (frame) {
clog('click@', clickAt, 'frame', frame.source || 'loop', clog('click@', clickAt, 'frame', frame.source || 'loop',
'started', frame.startedAt - clickAt, 'ms, captured', frame.capturedAt - clickAt, 'ms rel. click'); 'started', frame.startedAt - clickAt, 'ms, captured', frame.capturedAt - clickAt, 'ms rel. click');
const result = this.storeFrameAsStep(guideId, frame.mode, frame, clickPos); const result = await this.storeFrameAsStep(guideId, frame.mode, frame, clickPos, clickMeta);
if (result.ok) this.noteStepAdded(result.step, trigger, guideId); if (result.ok) this.noteStepAdded(result.step, trigger, guideId);
return result; return result;
} }
@@ -474,6 +631,46 @@ class CaptureService {
if (!sessionLive) return { ok: false, reason: 'session ended before the fallback shot' }; if (!sessionLive) return { ok: false, reason: 'session ended before the fallback shot' };
} }
// For non-click triggers (interval, hotkey, manual) pull the latest frame
// from the stream backend's ring buffer when available. This avoids a
// desktopCapturer.getSources() call per capture — on Linux/Wayland that
// call goes through the XDG portal and shows a dialog every time.
//
// No clickPos: a timed capture has no click position, and passing a cursor
// point here is actively harmful on Wayland — getCursorScreenPoint() can
// return a stale/out-of-bounds point, which makes the backend reject the
// frame (wrong display / out of bounds) and fall through to a getSources()
// shot, i.e. a portal dialog on every interval tick.
if (trigger !== 'click' && this.streamBackend && this.streamBackend.isActive()) {
const frame = await this.streamBackend.frameForClick({
clickPos: null,
clickAt: Date.now(),
strict: false, // no pre/post-click constraint for timed captures
leadMs: 0,
failable: false, // a slow timed-capture encode must not kill the stream
}).catch(() => null);
if (frame) {
const result = await this.storeFrameAsStep(this.session.guideId, 'fullscreen', frame);
if (result.ok) this.noteStepAdded(result.step, trigger);
clog(trigger, 'capture stored from stream; total', this.session && this.session.count);
return result;
}
clog(trigger, 'capture: no frame from stream this tick — will retry next tick');
} else if (trigger !== 'click' && this.onWayland()) {
clog(trigger, 'capture: stream backend not active (active=',
Boolean(this.streamBackend && this.streamBackend.isActive()), ')');
}
// On Wayland the only screen-grab fallback below is desktopCapturer
// .getSources(), which pops the XDG portal dialog every call. For the
// automatic timed triggers that would mean a dialog on every tick, so skip
// the fallback and wait for the open stream to deliver a frame on a later
// tick. Explicit captures (manual, and click on X11) still fall through —
// one dialog for one deliberate action.
if (this.onWayland() && (trigger === 'interval' || trigger === 'hotkey')) {
return { ok: false, reason: 'waiting for the screen-share stream' };
}
if (this.shooting) return { ok: false, reason: 'capture already in progress' }; if (this.shooting) return { ok: false, reason: 'capture already in progress' };
this.shooting = true; this.shooting = true;
try { try {
@@ -635,7 +832,11 @@ class CaptureService {
}; };
if (this.streamBackend && this.streamBackend.isActive() && grabMode === 'fullscreen') { if (this.streamBackend && this.streamBackend.isActive() && grabMode === 'fullscreen') {
const frame = await this.streamBackend.frameForClick({ clickPos, clickAt: clickTime, strict, leadMs }); // On Wayland the stream is the only capture path (no frame-loop fallback),
// so a slow PNG encode must not let the 2-strikes rule tear it down.
const frame = await this.streamBackend.frameForClick({
clickPos, clickAt: clickTime, strict, leadMs, failable: !this.onWayland(),
});
if (frame) return frame; if (frame) return frame;
// No qualifying frame (or the backend just went unhealthy): fall // No qualifying frame (or the backend just went unhealthy): fall
// through to the loop buffer / fresh-shot fallbacks below. // through to the loop buffer / fresh-shot fallbacks below.
@@ -688,8 +889,11 @@ class CaptureService {
async startClickFrameBackend() { async startClickFrameBackend() {
const mode = this.settings.get('capture.mode') || 'fullscreen'; const mode = this.settings.get('capture.mode') || 'fullscreen';
// The worker streams screens; window-mode grabs need the loop's // The worker streams screens; window-mode grabs need the loop's
// source-filtering logic. // source-filtering logic. But the loop isn't viable on Wayland (getSources
if (this.settings.get('capture.streamCapture') === false || mode === 'window') { // is broken/portal), so there we always take the stream backend regardless
// of the streamCapture/window settings.
if (this.canUseFrameLoop()
&& (this.settings.get('capture.streamCapture') === false || mode === 'window')) {
this.startFrameLoop(); this.startFrameLoop();
return; return;
} }
@@ -709,7 +913,11 @@ class CaptureService {
onUnhealthy: () => this.degradeToFrameLoop(), onUnhealthy: () => this.degradeToFrameLoop(),
}); });
const displays = this.screen.getAllDisplays(); const displays = this.screen.getAllDisplays();
const sources = await desktopCapturer.getSources({ // On Wayland, desktopCapturer.getSources() both fails to yield usable
// source ids AND pops the portal dialog. Skip it entirely and drive the
// worker through getDisplayMedia (the portal picker chooses the screen).
const useDisplayMedia = this.onWayland();
const sources = useDisplayMedia ? [] : await desktopCapturer.getSources({
types: ['screen'], types: ['screen'],
thumbnailSize: { width: 1, height: 1 }, // ids only — skip thumbnail work thumbnailSize: { width: 1, height: 1 }, // ids only — skip thumbnail work
}); });
@@ -717,23 +925,38 @@ class CaptureService {
displays, displays,
sources: sources.map((s) => ({ id: s.id, display_id: s.display_id })), sources: sources.map((s) => ({ id: s.id, display_id: s.display_id })),
sampleMs: this.settings.get('capture.frameSampleMs') || 100, sampleMs: this.settings.get('capture.frameSampleMs') || 100,
useDisplayMedia,
}); });
const stale = gen !== this.captureGen; const stale = gen !== this.captureGen;
if (!ok || stale || !this.session || this.session.paused) { if (!ok || stale || !this.session || this.session.paused) {
backend.stop(); backend.stop();
if (!stale && this.session && !this.session.paused) { if (!stale && this.session && !this.session.paused) {
if (this.canUseFrameLoop()) {
console.error('[stepforge] stream capture backend failed to start — using in-process frame loop'); console.error('[stepforge] stream capture backend failed to start — using in-process frame loop');
this.startFrameLoop(); this.startFrameLoop();
} else {
// On Wayland the frame loop would spam getSources() (portal) with
// nothing usable, so there's no fallback — the recording needs the
// portal stream. Tell the user how to recover.
console.error('[stepforge] screen-share stream did not start — pick a screen in the share dialog, or stop and start recording again');
}
} }
return; return;
} }
this.streamBackend = backend; this.streamBackend = backend;
clog('stream capture backend active'); // Visible in normal output (one line per recording): confirms the screen
// stream came up, so a "nothing records" report can be told apart from a
// stream that never started (which logs the failure paths above).
console.log(`[stepforge] screen-capture stream active (${useDisplayMedia ? 'getDisplayMedia/portal' : 'desktopCapturer'})`);
this.notify('capture:state', this.state()); this.notify('capture:state', this.state());
} catch (err) { } catch (err) {
if (gen === this.captureGen && this.session && !this.session.paused) { if (gen === this.captureGen && this.session && !this.session.paused) {
if (this.canUseFrameLoop()) {
console.error(`[stepforge] stream capture backend error (${err && err.message}) — using in-process frame loop`); console.error(`[stepforge] stream capture backend error (${err && err.message}) — using in-process frame loop`);
this.startFrameLoop(); this.startFrameLoop();
} else {
console.error(`[stepforge] screen-share stream error (${err && err.message}) — stop and start recording again`);
}
} }
} finally { } finally {
if (gen === this.captureGen) this.streamBackendStarting = false; if (gen === this.captureGen) this.streamBackendStarting = false;
@@ -758,8 +981,14 @@ class CaptureService {
*/ */
degradeToFrameLoop() { degradeToFrameLoop() {
this.streamBackend = null; this.streamBackend = null;
if (this.canUseFrameLoop()) {
console.error('[stepforge] stream capture backend unhealthy — falling back to in-process frame loop'); console.error('[stepforge] stream capture backend unhealthy — falling back to in-process frame loop');
if (this.session && !this.session.paused) this.startFrameLoop(); if (this.session && !this.session.paused) this.startFrameLoop();
} else {
// On Wayland the frame loop isn't viable (getSources is broken/portal),
// so there's nothing to fall back to — the stream is the only path.
console.error('[stepforge] screen-share stream stopped — stop and start recording again to re-share');
}
this.notify('capture:state', this.state()); this.notify('capture:state', this.state());
} }
@@ -768,13 +997,14 @@ class CaptureService {
try { try {
this.clickWatcherBuf = ''; this.clickWatcherBuf = '';
this.linuxEvent = null; this.linuxEvent = null;
if (process.platform === 'linux' && hasBinary('xinput')) { if (process.platform === 'linux' && !this.onWayland() && hasBinary('xinput')) {
// Stream raw button events from the X server; one capture per press. // Stream raw button events from the X server; one capture per press.
// xinput block-buffers stdout when piped, so a press event can sit // xinput block-buffers stdout when piped, so a press event can sit
// in its buffer until later motion events flush it — by then the // in its buffer until later motion events flush it — by then the
// cursor read in onOsClick lands where the mouse moved *after* the // cursor read in onOsClick lands where the mouse moved *after* the
// click. stdbuf -oL forces line-buffering so events (and the cursor // click. stdbuf -oL forces line-buffering so events (and the cursor
// read) line up with the actual click instant. // read) line up with the actual click instant.
// (Skipped on Wayland: xinput only sees XWayland events — see isWayland.)
const argv = hasBinary('stdbuf') const argv = hasBinary('stdbuf')
? ['stdbuf', '-oL', 'xinput', 'test-xi2', '--root'] ? ['stdbuf', '-oL', 'xinput', 'test-xi2', '--root']
: ['xinput', 'test-xi2', '--root']; : ['xinput', 'test-xi2', '--root'];
@@ -782,6 +1012,13 @@ class CaptureService {
this.clickWatcher.stdout.on('data', (chunk) => { this.clickWatcher.stdout.on('data', (chunk) => {
this.ingestClickWatcherChunk(chunk.toString(), 'linux'); this.ingestClickWatcherChunk(chunk.toString(), 'linux');
}); });
} else if (process.platform === 'linux' && readableEvdevMouseNodes().length > 0) {
// Wayland (or X11 without xinput): read mouse buttons from the kernel
// input layer. This is the only global click source on Wayland, but it
// carries no cursor position — onOsClick gets a null point, so steps are
// captured per click without a marker. (X11 prefers the xinput branch
// above, which does carry root coordinates for the marker.)
this.startEvdevWatcher();
} else if (process.platform === 'win32') { } else if (process.platform === 'win32') {
// Use a low-level Windows mouse hook instead of polling // Use a low-level Windows mouse hook instead of polling
// GetAsyncKeyState. The low bit from GetAsyncKeyState can be consumed // GetAsyncKeyState. The low bit from GetAsyncKeyState can be consumed
@@ -796,12 +1033,15 @@ using System.Collections.Concurrent;
using System.Runtime.InteropServices; using System.Runtime.InteropServices;
using System.Threading; using System.Threading;
public static class SFMouseHook { public static class SFHook {
private const int WH_MOUSE_LL = 14; private const int WH_MOUSE_LL = 14;
private const int WH_KEYBOARD_LL = 13;
private const int WM_LBUTTONDOWN = 0x0201; private const int WM_LBUTTONDOWN = 0x0201;
private const int WM_RBUTTONDOWN = 0x0204; private const int WM_RBUTTONDOWN = 0x0204;
private const int WM_MBUTTONDOWN = 0x0207; private const int WM_MBUTTONDOWN = 0x0207;
private const int WM_XBUTTONDOWN = 0x020B; private const int WM_XBUTTONDOWN = 0x020B;
private const int WM_KEYDOWN = 0x0100;
private const int WM_SYSKEYDOWN = 0x0104;
private const long UnixEpochMilliseconds = 62135596800000L; private const long UnixEpochMilliseconds = 62135596800000L;
// Opting this process out of Windows Power Throttling (EcoQoS). In a // Opting this process out of Windows Power Throttling (EcoQoS). In a
// power-saving plan the OS CPU-starves background processes; a starved // power-saving plan the OS CPU-starves background processes; a starved
@@ -815,7 +1055,9 @@ public static class SFMouseHook {
private const uint HIGH_PRIORITY_CLASS = 0x00000080; private const uint HIGH_PRIORITY_CLASS = 0x00000080;
private static IntPtr hook = IntPtr.Zero; private static IntPtr hook = IntPtr.Zero;
private static LowLevelMouseProc proc = HookCallback; private static IntPtr keyHook = IntPtr.Zero;
private static LowLevelMouseProc proc = MouseHookCallback;
private static LowLevelKeyboardProc keyProc = KeyboardHookCallback;
private static readonly ConcurrentQueue<string> queue = new ConcurrentQueue<string>(); private static readonly ConcurrentQueue<string> queue = new ConcurrentQueue<string>();
private static readonly AutoResetEvent signal = new AutoResetEvent(false); private static readonly AutoResetEvent signal = new AutoResetEvent(false);
@@ -851,11 +1093,34 @@ public static class SFMouseHook {
public uint StateMask; public uint StateMask;
} }
[StructLayout(LayoutKind.Sequential)]
private struct KBDLLHOOKSTRUCT {
public uint vkCode;
public uint scanCode;
public uint flags;
public uint time;
public UIntPtr dwExtraInfo;
}
private delegate IntPtr LowLevelMouseProc(int nCode, IntPtr wParam, IntPtr lParam); private delegate IntPtr LowLevelMouseProc(int nCode, IntPtr wParam, IntPtr lParam);
private delegate IntPtr LowLevelKeyboardProc(int nCode, IntPtr wParam, IntPtr lParam);
[DllImport("user32.dll", SetLastError = true)] [DllImport("user32.dll", SetLastError = true)]
private static extern IntPtr SetWindowsHookEx(int idHook, LowLevelMouseProc lpfn, IntPtr hMod, uint dwThreadId); private static extern IntPtr SetWindowsHookEx(int idHook, LowLevelMouseProc lpfn, IntPtr hMod, uint dwThreadId);
[DllImport("user32.dll", SetLastError = true)]
private static extern IntPtr SetWindowsHookEx(int idHook, LowLevelKeyboardProc lpfn, IntPtr hMod, uint dwThreadId);
[DllImport("user32.dll")]
private static extern short GetKeyState(int nVirtKey);
[DllImport("user32.dll")]
private static extern bool GetKeyboardState([Out] byte[] lpKeyState);
[DllImport("user32.dll", CharSet = CharSet.Unicode)]
private static extern int ToUnicode(uint wVirtKey, uint wScanCode, byte[] lpKeyState,
System.Text.StringBuilder pwszBuff, int cchBuff, uint wFlags);
[DllImport("user32.dll", SetLastError = true)] [DllImport("user32.dll", SetLastError = true)]
private static extern bool UnhookWindowsHookEx(IntPtr hhk); private static extern bool UnhookWindowsHookEx(IntPtr hhk);
@@ -886,6 +1151,58 @@ public static class SFMouseHook {
[DllImport("kernel32.dll")] [DllImport("kernel32.dll")]
private static extern bool SetPriorityClass(IntPtr hProcess, uint dwPriorityClass); private static extern bool SetPriorityClass(IntPtr hProcess, uint dwPriorityClass);
[DllImport("user32.dll")]
private static extern IntPtr GetForegroundWindow();
[DllImport("user32.dll", CharSet = CharSet.Unicode)]
private static extern int GetWindowText(IntPtr hWnd, System.Text.StringBuilder text, int count);
[DllImport("user32.dll")]
private static extern uint GetWindowThreadProcessId(IntPtr hWnd, out uint processId);
[DllImport("kernel32.dll", SetLastError = true)]
private static extern IntPtr OpenProcess(uint processAccess, bool bInheritHandle, uint processId);
[DllImport("kernel32.dll", SetLastError = true)]
private static extern bool CloseHandle(IntPtr hObject);
[DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)]
private static extern bool QueryFullProcessImageName(IntPtr hProcess, uint dwFlags, System.Text.StringBuilder lpExeName, ref uint lpdwSize);
private static string GetFwTitle() {
try {
IntPtr hwnd = GetForegroundWindow();
if (hwnd == IntPtr.Zero) return "";
var sb = new System.Text.StringBuilder(512);
GetWindowText(hwnd, sb, sb.Capacity);
return sb.ToString();
} catch { return ""; }
}
private static string GetFwApp() {
try {
IntPtr hwnd = GetForegroundWindow();
if (hwnd == IntPtr.Zero) return "";
uint pid = 0;
GetWindowThreadProcessId(hwnd, out pid);
if (pid == 0) return "";
IntPtr hProc = OpenProcess(0x1000u, false, pid);
if (hProc == IntPtr.Zero) return "";
var sb = new System.Text.StringBuilder(260);
uint sz = (uint)sb.Capacity;
QueryFullProcessImageName(hProc, 0u, sb, ref sz);
CloseHandle(hProc);
string path = sb.ToString();
return string.IsNullOrEmpty(path) ? "" : System.IO.Path.GetFileNameWithoutExtension(path);
} catch { return ""; }
}
// Base64-encodes a string for safe line-protocol transmission; "-" for empty.
private static string B64(string s) {
if (string.IsNullOrEmpty(s)) return "-";
try { return Convert.ToBase64String(System.Text.Encoding.UTF8.GetBytes(s)); } catch { return "-"; }
}
// Force this process to run at full CPU speed regardless of the power plan, // Force this process to run at full CPU speed regardless of the power plan,
// so the mouse-hook callback never trips LowLevelHooksTimeout and clicks // so the mouse-hook callback never trips LowLevelHooksTimeout and clicks
// keep being delivered while the laptop is in eco / power-saving mode. // keep being delivered while the laptop is in eco / power-saving mode.
@@ -914,6 +1231,8 @@ public static class SFMouseHook {
if (hook == IntPtr.Zero) { if (hook == IntPtr.Zero) {
throw new System.ComponentModel.Win32Exception(Marshal.GetLastWin32Error()); throw new System.ComponentModel.Win32Exception(Marshal.GetLastWin32Error());
} }
// Keyboard hook is optional — if it fails we still get click titles.
try { keyHook = SetWindowsHookEx(WH_KEYBOARD_LL, keyProc, GetModuleHandle(null), 0); } catch { }
Console.Out.WriteLine("READY"); Console.Out.WriteLine("READY");
Console.Out.Flush(); Console.Out.Flush();
@@ -925,6 +1244,7 @@ public static class SFMouseHook {
} }
UnhookWindowsHookEx(hook); UnhookWindowsHookEx(hook);
if (keyHook != IntPtr.Zero) UnhookWindowsHookEx(keyHook);
} }
private static void WriterLoop() { private static void WriterLoop() {
@@ -938,13 +1258,19 @@ public static class SFMouseHook {
} }
} }
private static IntPtr HookCallback(int nCode, IntPtr wParam, IntPtr lParam) { private static IntPtr MouseHookCallback(int nCode, IntPtr wParam, IntPtr lParam) {
if (nCode >= 0) { if (nCode >= 0) {
int message = wParam.ToInt32(); int message = wParam.ToInt32();
string button = ButtonName(message, lParam); string button = ButtonName(message, lParam);
if (button != null) { if (button != null) {
MSLLHOOKSTRUCT data = (MSLLHOOKSTRUCT)Marshal.PtrToStructure(lParam, typeof(MSLLHOOKSTRUCT)); MSLLHOOKSTRUCT data = (MSLLHOOKSTRUCT)Marshal.PtrToStructure(lParam, typeof(MSLLHOOKSTRUCT));
long unixMs = DateTime.UtcNow.Ticks / TimeSpan.TicksPerMillisecond - UnixEpochMilliseconds; long unixMs = DateTime.UtcNow.Ticks / TimeSpan.TicksPerMillisecond - UnixEpochMilliseconds;
// Capture window context while the user's app is still in foreground.
// GetForegroundWindow is synchronous and fast (no IPC overhead).
try {
string t = GetFwTitle(), a = GetFwApp();
queue.Enqueue("CTX " + B64(t) + " " + B64(a) + " " + unixMs);
} catch { }
queue.Enqueue("CLICK " + data.pt.x + " " + data.pt.y + " " + button + " " + unixMs); queue.Enqueue("CLICK " + data.pt.x + " " + data.pt.y + " " + button + " " + unixMs);
signal.Set(); signal.Set();
} }
@@ -952,6 +1278,75 @@ public static class SFMouseHook {
return CallNextHookEx(hook, nCode, wParam, lParam); return CallNextHookEx(hook, nCode, wParam, lParam);
} }
private static IntPtr KeyboardHookCallback(int nCode, IntPtr wParam, IntPtr lParam) {
if (nCode >= 0 && (wParam.ToInt32() == WM_KEYDOWN || wParam.ToInt32() == WM_SYSKEYDOWN)) {
try {
KBDLLHOOKSTRUCT kb = (KBDLLHOOKSTRUCT)Marshal.PtrToStructure(lParam, typeof(KBDLLHOOKSTRUCT));
int vk = (int)kb.vkCode;
bool ctrl = (GetKeyState(0x11) & 0x8000) != 0;
bool alt = (GetKeyState(0x12) & 0x8000) != 0;
bool shift = (GetKeyState(0x10) & 0x8000) != 0;
// Skip standalone modifier keys.
bool isMod = vk == 0x10 || vk == 0x11 || vk == 0x12 || vk == 0x5B || vk == 0x5C;
if (!isMod) {
long unixMs = DateTime.UtcNow.Ticks / TimeSpan.TicksPerMillisecond - UnixEpochMilliseconds;
if (ctrl || alt) {
// Emit shortcut: "KEY Ctrl+T 123456"
string name = VkName(vk);
if (name != null) {
string mods = (ctrl ? "Ctrl" : "") + (ctrl && (alt || shift) ? "+" : "") +
(alt ? "Alt" : "") + ((alt || ctrl) && shift ? "+" : "") +
(shift ? "Shift" : "");
queue.Enqueue("KEY " + mods + "+" + name + " " + unixMs);
signal.Set();
}
} else if (vk == 0x08) {
queue.Enqueue("KEY Backspace " + unixMs); signal.Set();
} else if (vk == 0x1B) {
queue.Enqueue("KEY Escape " + unixMs); signal.Set();
} else if (vk == 0x0D) {
queue.Enqueue("KEY Enter " + unixMs); signal.Set();
} else {
// Map to Unicode character using current keyboard layout + shift state.
byte[] ks = new byte[256];
GetKeyboardState(ks);
var sb = new System.Text.StringBuilder(4);
int res = ToUnicode(kb.vkCode, kb.scanCode, ks, sb, 4, 0);
if (res > 0) {
char ch = sb[0];
if (ch >= 0x20 && ch < 0x7F) {
queue.Enqueue("CHAR " + (int)ch + " " + unixMs);
signal.Set();
}
}
}
}
} catch { }
}
return CallNextHookEx(keyHook, nCode, wParam, lParam);
}
private static string VkName(int vk) {
if (vk >= 0x41 && vk <= 0x5A) return ((char)vk).ToString(); // A-Z
if (vk >= 0x30 && vk <= 0x39) return ((char)vk).ToString(); // 0-9
if (vk >= 0x70 && vk <= 0x87) return "F" + (vk - 0x6F); // F1-F24
switch (vk) {
case 0x09: return "Tab";
case 0x0D: return "Enter";
case 0x1B: return "Escape";
case 0x20: return "Space";
case 0x21: return "PageUp"; case 0x22: return "PageDown";
case 0x23: return "End"; case 0x24: return "Home";
case 0x25: return "Left"; case 0x26: return "Up";
case 0x27: return "Right"; case 0x28: return "Down";
case 0x2E: return "Delete";
case 0x6B: case 0xBB: return "Plus";
case 0x6D: case 0xBD: return "Minus";
case 0xBE: return "Period"; case 0xBF: return "Slash";
default: return null;
}
}
private static string ButtonName(int message, IntPtr lParam) { private static string ButtonName(int message, IntPtr lParam) {
if (message == WM_LBUTTONDOWN) return "left"; if (message == WM_LBUTTONDOWN) return "left";
if (message == WM_RBUTTONDOWN) return "right"; if (message == WM_RBUTTONDOWN) return "right";
@@ -965,7 +1360,7 @@ public static class SFMouseHook {
} }
} }
'@ '@
[SFMouseHook]::Run() [SFHook]::Run()
`; `;
this.clickWatcher = spawn('powershell.exe', ['-NoProfile', '-NonInteractive', '-ExecutionPolicy', 'Bypass', '-Command', ps], { this.clickWatcher = spawn('powershell.exe', ['-NoProfile', '-NonInteractive', '-ExecutionPolicy', 'Bypass', '-Command', ps], {
stdio: ['ignore', 'pipe', 'pipe'], stdio: ['ignore', 'pipe', 'pipe'],
@@ -1008,7 +1403,9 @@ public static class SFMouseHook {
console.error(`[stepforge] click watcher stopped${detail ? `: ${detail}` : ''}`); console.error(`[stepforge] click watcher stopped${detail ? `: ${detail}` : ''}`);
if (!this.session) return; if (!this.session) return;
if (!this.session.intervalSec) { if (!this.session.intervalSec) {
this.session.intervalSec = this.settings.get('capture.autoIntervalSec') || 5; this.session.intervalSec = this.fallbackCaptureTrigger() === 'hotkey'
? 0
: this.fallbackIntervalSec();
this.applyInterval(); this.applyInterval();
} }
this.notify('capture:state', this.state()); this.notify('capture:state', this.state());
@@ -1019,12 +1416,54 @@ public static class SFMouseHook {
try { this.clickWatcher.kill(); } catch { /* already gone */ } try { this.clickWatcher.kill(); } catch { /* already gone */ }
this.clickWatcher = null; this.clickWatcher = null;
} }
this.stopEvdevWatcher();
this.clickWatcherBuf = ''; this.clickWatcherBuf = '';
this.linuxEvent = null; this.linuxEvent = null;
this.discardPendingRawClick(); this.discardPendingRawClick();
this.lastAcceptedClickByButton.clear(); this.lastAcceptedClickByButton.clear();
} }
/**
* Open every readable mouse device node and turn button-down events into
* onOsClick calls. One physical mouse is normally one node; the leading-edge
* debounce in onOsClick collapses any cross-node duplicates. Frames are still
* served by the stream backend; this only supplies the click *trigger*.
*/
startEvdevWatcher() {
const nodes = readableEvdevMouseNodes();
this.evdevStreams = [];
for (const node of nodes) {
try {
const stream = fs.createReadStream(node);
let buf = Buffer.alloc(0);
stream.on('data', (chunk) => {
buf = buf.length ? Buffer.concat([buf, chunk]) : chunk;
const { presses, rest } = decodeEvdevButtonPresses(buf);
buf = rest;
for (const button of presses) this.onOsClick(Date.now(), null, button);
});
// A device can disappear (unplugged); just drop that stream.
stream.on('error', () => {});
this.evdevStreams.push(stream);
} catch {
// Node became unreadable between enumeration and open — skip it.
}
}
if (!this.evdevStreams.length) {
console.error('[stepforge] no readable mouse input devices — add your user to the "input" group for per-click capture: sudo usermod -aG input "$USER" (then log out and back in)');
} else {
console.log(`[stepforge] per-click capture via evdev on ${this.evdevStreams.length} device(s)${this.onWayland() ? ' (Wayland: no click marker)' : ''}`);
}
}
stopEvdevWatcher() {
if (!this.evdevStreams) return;
for (const stream of this.evdevStreams) {
try { stream.destroy(); } catch { /* already closed */ }
}
this.evdevStreams = null;
}
/** /**
* Buffer stdout chunks and only parse complete lines: a chunk boundary * Buffer stdout chunks and only parse complete lines: a chunk boundary
* can split an event line in half, which used to corrupt press/release * can split an event line in half, which used to corrupt press/release
@@ -1100,11 +1539,33 @@ public static class SFMouseHook {
} }
if (platform === 'win32') { if (platform === 'win32') {
for (const line of lines) { for (const line of lines) {
const m = /^CLICK(?:\s+(-?\d+)\s+(-?\d+)(?:\s+([A-Za-z0-9_-]+))?(?:\s+(\d+))?)?\s*$/.exec(line.trim()); const trimmed = line.trim();
if (m) { const clickM = /^CLICK(?:\s+(-?\d+)\s+(-?\d+)(?:\s+([A-Za-z0-9_-]+))?(?:\s+(\d+))?)?\s*$/.exec(trimmed);
const osPoint = m[1] === undefined ? null : { x: Number(m[1]), y: Number(m[2]) }; if (clickM) {
const eventAt = m[4] === undefined ? Date.now() : Number(m[4]); const osPoint = clickM[1] === undefined ? null : { x: Number(clickM[1]), y: Number(clickM[2]) };
this.onOsClick(Number.isFinite(eventAt) ? eventAt : Date.now(), osPoint, m[3] || 'mouse'); const eventAt = clickM[4] === undefined ? Date.now() : Number(clickM[4]);
this.onOsClick(Number.isFinite(eventAt) ? eventAt : Date.now(), osPoint, clickM[3] || 'mouse');
continue;
}
const keyM = /^KEY\s+(\S+)\s+(\d+)\s*$/.exec(trimmed);
if (keyM) {
this.onKeyboardEvent('KEY', keyM[1], Number(keyM[2]));
continue;
}
const charM = /^CHAR\s+(\d+)\s+(\d+)\s*$/.exec(trimmed);
if (charM) {
this.onKeyboardEvent('CHAR', Number(charM[1]), Number(charM[2]));
continue;
}
// CTX is emitted just before its paired CLICK from MouseHookCallback.
const ctxM = /^CTX\s+(\S+)\s+(\S+)\s+\d+$/.exec(trimmed);
if (ctxM) {
const decode = s => s === '-' ? '' : Buffer.from(s, 'base64').toString('utf8');
this._pendingWindowContext = {
windowTitle: decode(ctxM[1]),
appName: decode(ctxM[2]),
};
continue;
} }
} }
} }
@@ -1209,8 +1670,15 @@ public static class SFMouseHook {
// filtered by the cursor-position check in sessionCapture, not by // filtered by the cursor-position check in sessionCapture, not by
// window focus — WSLg reports focus unreliably.) // window focus — WSLg reports focus unreliably.)
let clickPos = osPoint ? this.osPointToDip(osPoint) : null; let clickPos = osPoint ? this.osPointToDip(osPoint) : null;
if (!clickPos) clickPos = this.screen.getCursorScreenPoint(); // Read the live cursor as a fallback only off Wayland: Wayland refuses to
// report the global pointer position (getCursorScreenPoint returns 0,0), so
// a fallback there would stamp every click marker in the top-left corner.
// Leaving clickPos null means the step is captured with no (wrong) marker.
if (!clickPos && !this.onWayland()) clickPos = this.screen.getCursorScreenPoint();
clog('click@', clickAt, button, 'os', osPoint, '-> dip', clickPos); clog('click@', clickAt, button, 'os', osPoint, '-> dip', clickPos);
this.pendingClickOsPoint = osPoint && Number.isFinite(osPoint.x) && Number.isFinite(osPoint.y)
? { x: osPoint.x, y: osPoint.y }
: null;
this.enqueueClickCapture(clickPos, clickAt, button || 'mouse'); this.enqueueClickCapture(clickPos, clickAt, button || 'mouse');
} }
@@ -1238,20 +1706,32 @@ public static class SFMouseHook {
* scaled away from 100% and on secondary monitors. * scaled away from 100% and on secondary monitors.
*/ */
osPointToDip(osPoint) { osPointToDip(osPoint) {
if (this.screen && typeof this.screen.screenToDipPoint === 'function') { let geometryDip = null;
try {
const dip = this.screen.screenToDipPoint(osPoint);
if (dip && Number.isFinite(dip.x) && Number.isFinite(dip.y)) return dip;
} catch { /* fall through to manual conversion */ }
}
try { try {
const displays = this.screen && typeof this.screen.getAllDisplays === 'function' const displays = this.screen && typeof this.screen.getAllDisplays === 'function'
? this.screen.getAllDisplays() ? this.screen.getAllDisplays()
: []; : [];
const dip = physicalToDip(osPoint, displays); geometryDip = physicalToDip(osPoint, displays);
if (dip) return dip; } catch {
} catch { /* no display geometry available */ } geometryDip = null;
return osPoint; }
if (this.screen && typeof this.screen.screenToDipPoint === 'function') {
try {
const dip = this.screen.screenToDipPoint(osPoint);
if (dip && Number.isFinite(dip.x) && Number.isFinite(dip.y)) {
if (!geometryDip) return dip;
const offByX = Math.abs(dip.x - geometryDip.x);
const offByY = Math.abs(dip.y - geometryDip.y);
// Some Windows/Electron combinations have been observed to return a
// raw physical point here. That keeps the click marker off-screen on
// scaled displays, so trust the geometry path when the two disagree
// by more than a tiny rounding margin.
if (offByX <= 1 && offByY <= 1) return dip;
return geometryDip;
}
} catch { /* fall through to manual conversion */ }
}
return geometryDip || osPoint;
} }
/** /**
@@ -1268,7 +1748,24 @@ public static class SFMouseHook {
* fast the user clicks or how slow the encoder is. * fast the user clicks or how slow the encoder is.
*/ */
enqueueClickCapture(clickPos, clickAt = Date.now(), button = 'mouse') { enqueueClickCapture(clickPos, clickAt = Date.now(), button = 'mouse') {
const clickMeta = { at: Number.isFinite(clickAt) ? clickAt : Date.now(), button: button || 'mouse' }; const osPoint = this.pendingClickOsPoint && Number.isFinite(this.pendingClickOsPoint.x) && Number.isFinite(this.pendingClickOsPoint.y)
? this.pendingClickOsPoint
: null;
this.pendingClickOsPoint = null;
const clickMeta = {
at: Number.isFinite(clickAt) ? clickAt : Date.now(),
button: button || 'mouse',
};
if (osPoint) {
clickMeta.osPoint = { x: osPoint.x, y: osPoint.y };
}
// Snapshot keyboard context accumulated since the last capture.
clickMeta.keyContext = this.snapshotKeyContext();
// Attach the window + element context emitted by the click watcher.
if (this._pendingWindowContext) {
clickMeta.windowContext = this._pendingWindowContext;
this._pendingWindowContext = null;
}
if (this.session && !this.session.paused && !this.userIsInApp()) { if (this.session && !this.session.paused && !this.userIsInApp()) {
// The guide id pins the click to its recording so it can still be // The guide id pins the click to its recording so it can still be
// stored if the session stops while this click waits in the queue. // stored if the session stops while this click waits in the queue.
@@ -1282,6 +1779,46 @@ public static class SFMouseHook {
return this.clickQueue; return this.clickQueue;
} }
// --- Keyboard context tracking -------------------------------------------
onKeyboardEvent(type, data, eventAt) {
const now = Number.isFinite(eventAt) ? eventAt : Date.now();
// Discard stale typing that happened more than 8 seconds ago (user moved on).
if (now - this._keyLastAt > 8000) {
this._keyBuffer = '';
}
this._keyLastAt = now;
if (type === 'CHAR') {
const ch = typeof data === 'number' ? String.fromCharCode(data) : String(data);
this._keyBuffer = (this._keyBuffer + ch).slice(-200);
} else if (type === 'KEY') {
const key = String(data);
if (key === 'Backspace') {
this._keyBuffer = this._keyBuffer.slice(0, -1);
} else if (key === 'Escape') {
this._keyBuffer = '';
} else if (key === 'Enter') {
// Keep typed text — Enter often submits what was typed.
} else {
// It's a modifier+key shortcut (Ctrl+T etc.).
this._lastShortcut = key;
this._keyBuffer = ''; // a shortcut resets the typed-text buffer
}
}
}
snapshotKeyContext() {
const ctx = {
recentTyped: this._keyBuffer.trim(),
recentShortcut: this._lastShortcut,
};
// Reset after snapshot so each step gets its own context.
this._keyBuffer = '';
this._lastShortcut = '';
return ctx;
}
async captureCurrentFrame(mode, capturePoint = null, startedAt = Date.now()) { async captureCurrentFrame(mode, capturePoint = null, startedAt = Date.now()) {
const grabbed = await this.grab(mode, capturePoint); const grabbed = await this.grab(mode, capturePoint);
return { return {
@@ -1300,7 +1837,18 @@ public static class SFMouseHook {
}; };
} }
storeFrameAsStep(guideId, mode, frame, clickPos = null) { async buildStepMeta(mode, frame, clickPos = null, clickMeta = null) {
try {
if (this.textIntel && typeof this.textIntel.buildCaptureContext === 'function') {
return await this.textIntel.buildCaptureContext({ mode, frame, clickPos, clickMeta });
}
} catch {
// fall back
}
return { title: this.autoTitle(mode), captureMetadata: null };
}
async storeFrameAsStep(guideId, mode, frame, clickPos = null, clickMeta = null) {
if (!frame) return { ok: false, reason: 'no capture frame available' }; if (!frame) return { ok: false, reason: 'no capture frame available' };
const annotations = []; const annotations = [];
// The click position (DIP, read at event time) wins over the frame's // The click position (DIP, read at event time) wins over the frame's
@@ -1323,8 +1871,10 @@ public static class SFMouseHook {
} }
} }
const { title, captureMetadata } = await this.buildStepMeta(mode, frame, clickPos, clickMeta);
const step = this.store.addStep(guideId, { const step = this.store.addStep(guideId, {
title: this.autoTitle(mode), title,
captureMetadata,
annotations, annotations,
focusedView: { focusedView: {
enabled: Boolean(this.settings.get('editor.focusedViewDefaultForNewSteps')), enabled: Boolean(this.settings.get('editor.focusedViewDefaultForNewSteps')),
@@ -1335,14 +1885,7 @@ public static class SFMouseHook {
} }
autoTitle(mode) { autoTitle(mode) {
const tplStr = this.settings.get('editor.autoTitleTemplate') || '[[Mode]] capture [[Time]]'; return DEFAULT_CAPTURE_TITLES[mode] || 'Capture';
const now = new Date();
const pad = (n) => String(n).padStart(2, '0');
return expandPlaceholders(tplStr, {
Mode: { fullscreen: 'Screen', window: 'Window', region: 'Region' }[mode] || 'Screen',
Time: `${pad(now.getHours())}:${pad(now.getMinutes())}:${pad(now.getSeconds())}`,
Date: `${now.getFullYear()}-${pad(now.getMonth() + 1)}-${pad(now.getDate())}`,
});
} }
/** Grab the screen/window image as { image, display } or throw. */ /** Grab the screen/window image as { image, display } or throw. */
@@ -1451,8 +1994,12 @@ public static class SFMouseHook {
const cropped = image.crop(rect); const cropped = image.crop(rect);
const size = cropped.getSize(); const size = cropped.getSize();
if (!size.width || !size.height) return { ok: false, reason: 'empty selection' }; if (!size.width || !size.height) return { ok: false, reason: 'empty selection' };
const step = this.store.addStep(guideId, { title: this.autoTitle('region') }, const step = await this.storeFrameAsStep(guideId, 'region', {
cropped.toPNG(), size); image: cropped,
size,
display,
cursor: null,
}, null, null);
return { ok: true, step }; return { ok: true, step };
} }
@@ -1472,8 +2019,12 @@ public static class SFMouseHook {
webPreferences: { webPreferences: {
preload: path.join(__dirname, 'region-preload.js'), preload: path.join(__dirname, 'region-preload.js'),
contextIsolation: true, contextIsolation: true,
nodeIntegration: false,
sandbox: true,
}, },
}); });
// The overlay may only display region.html; deny navigation/popups.
require('./security').installWindowSecurity(overlay, 'region');
let settled = false; let settled = false;
const finish = (rect) => { const finish = (rect) => {
if (settled) return; if (settled) return;
@@ -1504,3 +2055,5 @@ public static class SFMouseHook {
} }
module.exports = CaptureService; module.exports = CaptureService;
// Exposed for unit tests (pure, no device access).
module.exports.decodeEvdevButtonPresses = decodeEvdevButtonPresses;
+300 -49
View File
@@ -5,7 +5,7 @@ const fs = require('node:fs');
const os = require('node:os'); const os = require('node:os');
const { const {
app, BrowserWindow, ipcMain, dialog, shell, nativeTheme, globalShortcut, app, BrowserWindow, ipcMain, dialog, shell, nativeTheme, globalShortcut,
clipboard, nativeImage, screen, powerSaveBlocker, clipboard, nativeImage, screen, powerSaveBlocker, session, desktopCapturer,
} = require('electron'); } = require('electron');
const { GuideStore } = require('../core/store'); const { GuideStore } = require('../core/store');
@@ -15,11 +15,14 @@ const { TemplateManager, FORMATS, FORMAT_LABELS } = require('../core/templates')
const { buildRenderAst } = require('../core/renderast'); const { buildRenderAst } = require('../core/renderast');
const { runExport, EXPORTERS } = require('../exporters'); const { runExport, EXPORTERS } = require('../exporters');
const { runExportInWorker } = require('./export-runner'); const { runExportInWorker } = require('./export-runner');
const { exportGuideArchive, importGuideArchive, saveLinkedGuide, readArchive } = require('../core/archive'); const { exportGuideArchive, importGuideArchive, saveLinkedGuide } = require('../core/archive');
const { createSnapshot, listSnapshots, restoreSnapshot } = require('../core/snapshots'); const { createSnapshot, listSnapshots, restoreSnapshot } = require('../core/snapshots');
const { readLock } = require('../core/locks'); const { readLock } = require('../core/locks');
const CaptureService = require('./capture'); const CaptureService = require('./capture');
const { TextIntelService } = require('./text-intel');
const { keepProcessesResponsive } = require('./win-power'); const { keepProcessesResponsive } = require('./win-power');
const security = require('./security');
const PACKAGE_JSON = require(path.join(__dirname, '..', 'package.json'));
const APP_ID = 'com.stepforge.app'; const APP_ID = 'com.stepforge.app';
@@ -59,6 +62,7 @@ let settings;
let searchIndex; let searchIndex;
let templates; let templates;
let capture; let capture;
let textIntel;
let mainWindow; let mainWindow;
function reindex(guideId) { function reindex(guideId) {
@@ -92,9 +96,19 @@ function createWindow() {
preload: path.join(__dirname, 'preload.js'), preload: path.join(__dirname, 'preload.js'),
contextIsolation: true, contextIsolation: true,
nodeIntegration: false, nodeIntegration: false,
sandbox: true,
spellcheck: Boolean(settings.get('spellcheck')), spellcheck: Boolean(settings.get('spellcheck')),
// During a recording the window is minimized (Linux) or hidden (Windows).
// A throttled renderer stops processing capture:added events, so the step
// list and capture bar appear "stuck" even though steps are saved. Keep
// the renderer live so the UI updates in real time while recording.
backgroundThrottling: false,
}, },
}); });
// The main window may only ever display our index.html: all navigation
// away from it and every popup is denied, so no other document can run
// with this window's preload bridge.
security.installWindowSecurity(mainWindow, 'main');
mainWindow.loadFile(path.join(__dirname, 'renderer', 'index.html')); mainWindow.loadFile(path.join(__dirname, 'renderer', 'index.html'));
mainWindow.once('ready-to-show', () => { mainWindow.once('ready-to-show', () => {
mainWindow.show(); mainWindow.show();
@@ -367,7 +381,37 @@ function sendToRenderer(channel, payload) {
// ---- IPC ------------------------------------------------------------------ // ---- IPC ------------------------------------------------------------------
function setupIpc() { function setupIpc() {
const h = (channel, fn) => ipcMain.handle(channel, async (event, args = {}) => fn(args)); // Every invoke channel is guarded: the event must come from the current
// main window's top frame showing our index.html, the argument bag must be
// a plain object within a per-channel payload budget, and channels with
// risky inputs additionally validate fields before the handler runs.
const trustedSender = security.makeIpcSenderGuard({
getMainWebContents: () => (mainWindow && !mainWindow.isDestroyed() ? mainWindow.webContents : null),
});
const c = security.check;
const IMAGE_BUDGET = 256 * 1024 * 1024; // channels that carry base64 PNGs
const h = (channel, fn, opts = {}) => {
const { maxChars = 2 * 1024 * 1024, validate = null } = opts;
ipcMain.handle(channel, async (event, args = {}) => {
if (!trustedSender(event)) {
throw new Error(`${channel}: rejected — untrusted IPC sender`);
}
const a = args === undefined || args === null ? {} : args;
if (!security.isPlainArgs(a) || !security.payloadWithinBudget(a, maxChars)) {
throw new Error(`${channel}: rejected — invalid or oversized arguments`);
}
if (validate && !validate(a)) {
throw new Error(`${channel}: rejected — arguments failed validation`);
}
return fn(a);
});
};
// Files the main process itself produced (exports, previews); only these
// may be re-opened via the shell on renderer request.
const producedFiles = new security.ProducedFiles();
// Output directories the user actually picked in a dialog this session.
const chosenOutputDirs = new Set();
// library // library
h('library:list', () => ({ h('library:list', () => ({
@@ -385,60 +429,71 @@ function setupIpc() {
}); });
reindex(guide.guideId); reindex(guide.guideId);
return guide; return guide;
}); }, { validate: (a) => c.optionalString(a.title, 500) });
h('library:duplicate', ({ guideId }) => { h('library:duplicate', ({ guideId }) => {
const copy = store.duplicateGuide(guideId); const copy = store.duplicateGuide(guideId);
reindex(copy.guideId); reindex(copy.guideId);
return copy; return copy;
}); }, { validate: (a) => c.id(a.guideId) });
h('library:delete', ({ guideId }) => { h('library:delete', ({ guideId }) => {
store.deleteGuide(guideId); store.deleteGuide(guideId);
searchIndex.removeGuide(guideId); searchIndex.removeGuide(guideId);
return true; return true;
}); }, { validate: (a) => c.id(a.guideId) });
h('library:setFavorite', ({ guideId, favorite }) => store.setFavorite(guideId, favorite)); h('library:setFavorite', ({ guideId, favorite }) => store.setFavorite(guideId, favorite),
{ validate: (a) => c.id(a.guideId) });
h('library:trash:list', () => store.listTrash()); h('library:trash:list', () => store.listTrash());
h('library:trash:restore', ({ name }) => { h('library:trash:restore', ({ name }) => {
const id = store.restoreFromTrash(name); const id = store.restoreFromTrash(name);
reindex(id); reindex(id);
return id; return id;
}); }, { validate: (a) => c.fileName(a.name) });
h('library:trash:purge', ({ names } = {}) => { h('library:trash:purge', ({ names } = {}) => {
if (names && names.length) store.purgeTrashItems(names); if (names && names.length) store.purgeTrashItems(names);
else store.purgeTrash(); else store.purgeTrash();
return true; return true;
}, {
validate: (a) => a.names === undefined || a.names === null
|| (Array.isArray(a.names) && a.names.length <= 1000 && a.names.every((n) => c.fileName(n))),
}); });
h('folders:create', ({ name, parentId }) => store.createFolder(name, parentId || null)); h('folders:create', ({ name, parentId }) => store.createFolder(name, parentId || null),
h('folders:rename', ({ folderId, name }) => store.renameFolder(folderId, name)); { validate: (a) => c.string(a.name, 200) && c.optionalId(a.parentId) });
h('folders:delete', ({ folderId }) => store.deleteFolder(folderId)); h('folders:rename', ({ folderId, name }) => store.renameFolder(folderId, name),
h('folders:moveGuide', ({ guideId, folderId }) => store.moveGuideToFolder(guideId, folderId || null)); { validate: (a) => c.id(a.folderId) && c.string(a.name, 200) });
h('folders:delete', ({ folderId }) => store.deleteFolder(folderId),
{ validate: (a) => c.id(a.folderId) });
h('folders:moveGuide', ({ guideId, folderId }) => store.moveGuideToFolder(guideId, folderId || null),
{ validate: (a) => c.id(a.guideId) && c.optionalId(a.folderId) });
// guide + steps // guide + steps
h('guide:get', ({ guideId }) => ({ h('guide:get', ({ guideId }) => ({
guide: store.getGuide(guideId), guide: store.getGuide(guideId),
steps: orderedSteps(guideId), steps: orderedSteps(guideId),
})); }), { validate: (a) => c.id(a.guideId) });
h('guide:save', ({ guide }) => { h('guide:save', ({ guide }) => {
const saved = store.saveGuide(guide); const saved = store.saveGuide(guide);
reindex(guide.guideId); reindex(guide.guideId);
return saved; return saved;
}); }, { validate: (a) => security.isPlainArgs(a.guide) && a.guide && c.id(a.guide.guideId) });
h('step:add', ({ guideId, fields, imageBase64, size, position }) => { h('step:add', ({ guideId, fields, imageBase64, size, position }) => {
const buf = imageBase64 ? Buffer.from(imageBase64, 'base64') : null; const buf = imageBase64 ? Buffer.from(imageBase64, 'base64') : null;
const step = store.addStep(guideId, fields || {}, buf, size || null, { position }); const step = store.addStep(guideId, fields || {}, buf, size || null, { position });
reindex(guideId); reindex(guideId);
return step; return step;
}, {
maxChars: IMAGE_BUDGET,
validate: (a) => c.id(a.guideId) && c.optionalBase64(a.imageBase64) && c.optionalNumber(a.position, 0, 100000),
}); });
h('step:save', ({ guideId, step }) => { h('step:save', ({ guideId, step }) => {
const saved = store.saveStep(guideId, step); const saved = store.saveStep(guideId, step);
reindex(guideId); reindex(guideId);
return saved; return saved;
}); }, { validate: (a) => c.id(a.guideId) && security.isPlainArgs(a.step) && a.step && c.id(a.step.stepId) });
h('step:delete', ({ guideId, stepId }) => { h('step:delete', ({ guideId, stepId }) => {
store.deleteStep(guideId, stepId); store.deleteStep(guideId, stepId);
reindex(guideId); reindex(guideId);
return true; return true;
}); }, { validate: (a) => c.id(a.guideId) && c.id(a.stepId) });
h('step:restore', ({ guideId, step, originalBase64, workingBase64, position }) => { h('step:restore', ({ guideId, step, originalBase64, workingBase64, position }) => {
const images = { const images = {
original: originalBase64 ? Buffer.from(originalBase64, 'base64') : null, original: originalBase64 ? Buffer.from(originalBase64, 'base64') : null,
@@ -447,20 +502,34 @@ function setupIpc() {
const restored = store.restoreStep(guideId, step, images, position); const restored = store.restoreStep(guideId, step, images, position);
reindex(guideId); reindex(guideId);
return restored; return restored;
}, {
maxChars: IMAGE_BUDGET,
validate: (a) => c.id(a.guideId) && security.isPlainArgs(a.step) && a.step
&& c.optionalBase64(a.originalBase64) && c.optionalBase64(a.workingBase64)
&& c.optionalNumber(a.position, 0, 100000),
});
h('steps:reorder', ({ guideId, order }) => store.reorderSteps(guideId, order), {
validate: (a) => c.id(a.guideId)
&& Array.isArray(a.order) && a.order.length <= 100000 && a.order.every((id) => c.id(id)),
}); });
h('steps:reorder', ({ guideId, order }) => store.reorderSteps(guideId, order));
h('step:imagePath', ({ guideId, stepId, which }) => { h('step:imagePath', ({ guideId, stepId, which }) => {
const p = store.stepImagePath(guideId, stepId, which || 'working'); const p = store.stepImagePath(guideId, stepId, which || 'working');
return p && fs.existsSync(p) ? `file://${p}?v=${fs.statSync(p).mtimeMs}` : null; return p && fs.existsSync(p) ? `file://${p}?v=${fs.statSync(p).mtimeMs}` : null;
}, {
validate: (a) => c.id(a.guideId) && c.id(a.stepId)
&& (a.which === undefined || a.which === null || c.oneOf(a.which, ['original', 'working'])),
}); });
h('step:setWorkingImage', ({ guideId, stepId, pngBase64, size, step }) => h('step:setWorkingImage', ({ guideId, stepId, pngBase64, size, step }) =>
store.setWorkingImage(guideId, stepId, Buffer.from(pngBase64, 'base64'), size, step || null)); store.setWorkingImage(guideId, stepId, Buffer.from(pngBase64, 'base64'), size, step || null), {
maxChars: IMAGE_BUDGET,
validate: (a) => c.id(a.guideId) && c.id(a.stepId) && c.base64(a.pngBase64),
});
h('step:resetWorkingImage', ({ guideId, stepId }) => { h('step:resetWorkingImage', ({ guideId, stepId }) => {
const p = store.stepImagePath(guideId, stepId, 'original'); const p = store.stepImagePath(guideId, stepId, 'original');
const img = nativeImage.createFromPath(p); const img = nativeImage.createFromPath(p);
const { width, height } = img.getSize(); const { width, height } = img.getSize();
return store.resetWorkingImage(guideId, stepId, { width, height }); return store.resetWorkingImage(guideId, stepId, { width, height });
}); }, { validate: (a) => c.id(a.guideId) && c.id(a.stepId) });
h('step:fromClipboard', ({ guideId, position }) => { h('step:fromClipboard', ({ guideId, position }) => {
const img = clipboard.readImage(); const img = clipboard.readImage();
if (img.isEmpty()) return { ok: false, reason: 'clipboard has no image' }; if (img.isEmpty()) return { ok: false, reason: 'clipboard has no image' };
@@ -471,7 +540,7 @@ function setupIpc() {
}, img.toPNG(), { width, height }, { position }); }, img.toPNG(), { width, height }, { position });
reindex(guideId); reindex(guideId);
return { ok: true, step }; return { ok: true, step };
}); }, { validate: (a) => c.id(a.guideId) && c.optionalNumber(a.position, 0, 100000) });
h('step:importImage', async ({ guideId }) => { h('step:importImage', async ({ guideId }) => {
const res = await dialog.showOpenDialog(mainWindow, { const res = await dialog.showOpenDialog(mainWindow, {
title: 'Import images as steps', title: 'Import images as steps',
@@ -489,11 +558,13 @@ function setupIpc() {
} }
reindex(guideId); reindex(guideId);
return { ok: true, steps }; return { ok: true, steps };
}); }, { validate: (a) => c.id(a.guideId) });
// search // search
h('search:query', ({ q, guideId }) => searchIndex.search(q, { guideId: guideId || null })); h('search:query', ({ q, guideId }) => searchIndex.search(q, { guideId: guideId || null }),
h('search:titles', ({ q }) => searchIndex.searchTitles(q)); { validate: (a) => c.optionalString(a.q, 1000) && c.optionalId(a.guideId) });
h('search:titles', ({ q }) => searchIndex.searchTitles(q),
{ validate: (a) => c.optionalString(a.q, 1000) });
// settings + placeholders // settings + placeholders
h('settings:all', () => settings.data); h('settings:all', () => settings.data);
@@ -502,6 +573,31 @@ function setupIpc() {
if (keyPath === 'appearance') applyTheme(); if (keyPath === 'appearance') applyTheme();
if (keyPath.startsWith('capture.hotkey')) registerHotkeys(); if (keyPath.startsWith('capture.hotkey')) registerHotkeys();
return settings.data; return settings.data;
}, { validate: (a) => c.settingsKeyPath(a.keyPath) });
h('ai:test', async ({ enabled = null, ollama = null } = {}) => {
return textIntel.testAiConnection({
enabled,
ollama,
});
}, { validate: (a) => (a.ollama === undefined || a.ollama === null || security.isPlainArgs(a.ollama)) });
h('ai:fillStep', async ({ guideId, stepId, target = 'all', blockId = null } = {}) => {
const result = await textIntel.generateStepPatch({
guideId,
stepId,
target,
blockId,
});
if (result.ok) reindex(guideId);
return result;
}, {
validate: (a) => c.id(a.guideId) && c.id(a.stepId) && c.optionalId(a.blockId)
&& (a.target === undefined || c.oneOf(a.target, ['all', 'title', 'description', 'block'])),
});
h('ai:rewriteText', async ({ text, guideTitle = '', stepTitle = '' } = {}) => {
return textIntel.rewriteText({ text, guideTitle, stepTitle });
}, {
validate: (a) => c.string(a.text, 200000)
&& c.optionalString(a.guideTitle, 1000) && c.optionalString(a.stepTitle, 1000),
}); });
h('placeholders:globals:get', () => settings.getGlobalPlaceholders()); h('placeholders:globals:get', () => settings.getGlobalPlaceholders());
h('placeholders:globals:set', (values) => settings.setGlobalPlaceholders(values)); h('placeholders:globals:set', (values) => settings.setGlobalPlaceholders(values));
@@ -509,14 +605,46 @@ function setupIpc() {
// capture // capture
h('capture:shoot', async ({ guideId, mode, delayMs }) => { h('capture:shoot', async ({ guideId, mode, delayMs }) => {
const result = await capture.shoot({ guideId, mode, delayMs }); const result = await capture.shoot({ guideId, mode, delayMs });
if (result.ok) reindex(guideId); if (result.ok) {
reindex(guideId);
const aiConf = settings.get('ai') || {};
if (aiConf.enabled && aiConf.autoDoc && result.step) {
const aiResult = await textIntel.generateStepPatch({
guideId,
stepId: result.step.stepId,
target: 'all',
}).catch(() => null);
if (aiResult?.ok) {
reindex(guideId);
result.step = aiResult.step;
}
}
}
return result; return result;
}, {
validate: (a) => c.id(a.guideId)
&& (a.mode === undefined || c.oneOf(a.mode, ['fullscreen', 'window', 'region']))
&& c.optionalNumber(a.delayMs, 0, 600000),
}); });
h('capture:region', async ({ guideId }) => { h('capture:region', async ({ guideId }) => {
const result = await capture.regionCapture(guideId); const result = await capture.regionCapture(guideId);
if (result.ok) reindex(guideId); if (result.ok) {
reindex(guideId);
const aiConf = settings.get('ai') || {};
if (aiConf.enabled && aiConf.autoDoc && result.step) {
const aiResult = await textIntel.generateStepPatch({
guideId,
stepId: result.step.stepId,
target: 'all',
}).catch(() => null);
if (aiResult?.ok) {
reindex(guideId);
result.step = aiResult.step;
}
}
}
return result; return result;
}); }, { validate: (a) => c.id(a.guideId) });
let capturePowerBlocker = -1; let capturePowerBlocker = -1;
const startCapturePower = () => { const startCapturePower = () => {
if (!powerSaveBlocker.isStarted(capturePowerBlocker)) { if (!powerSaveBlocker.isStarted(capturePowerBlocker)) {
@@ -560,6 +688,10 @@ function setupIpc() {
const state = capture.state(); const state = capture.state();
sendToRenderer('capture:state', state); sendToRenderer('capture:state', state);
return state; return state;
}, {
validate: (a) => c.oneOf(a.action, ['start', 'pause', 'resume', 'finish', 'interval'])
&& (a.action !== 'start' || c.id(a.guideId))
&& c.optionalNumber(a.intervalSec, 0, 86400),
}); });
h('capture:state', () => capture.state()); h('capture:state', () => capture.state());
@@ -574,7 +706,7 @@ function setupIpc() {
if (res.canceled) return { ok: false }; if (res.canceled) return { ok: false };
exportGuideArchive(store, guideId, res.filePath); exportGuideArchive(store, guideId, res.filePath);
return { ok: true, path: res.filePath }; return { ok: true, path: res.filePath };
}); }, { validate: (a) => c.id(a.guideId) });
h('archive:open', async ({ mode }) => { h('archive:open', async ({ mode }) => {
const res = await dialog.showOpenDialog(mainWindow, { const res = await dialog.showOpenDialog(mainWindow, {
title: 'Open guide archive', title: 'Open guide archive',
@@ -589,30 +721,38 @@ function setupIpc() {
} catch (err) { } catch (err) {
return { ok: false, reason: err.message }; return { ok: false, reason: err.message };
} }
}); }, { validate: (a) => a.mode === undefined || c.oneOf(a.mode, ['copy', 'linked']) });
h('archive:peek', ({ file }) => { // archive:peek was removed: nothing in the renderer used it, and it let a
const { manifest } = readArchive(file); // compromised renderer read arbitrary local archives by path.
return manifest; h('archive:saveLinked', ({ guideId, force }) => saveLinkedGuide(store, guideId, { force: Boolean(force) }),
}); { validate: (a) => c.id(a.guideId) });
h('archive:saveLinked', ({ guideId, force }) => saveLinkedGuide(store, guideId, { force: Boolean(force) }));
// snapshots // snapshots
h('snapshots:list', ({ guideId }) => listSnapshots(store, guideId)); h('snapshots:list', ({ guideId }) => listSnapshots(store, guideId),
{ validate: (a) => c.id(a.guideId) });
h('snapshots:create', ({ guideId, label }) => h('snapshots:create', ({ guideId, label }) =>
createSnapshot(store, guideId, { label: label || 'manual', keepLast: settings.get('backups.keepLast') })); createSnapshot(store, guideId, { label: label || 'manual', keepLast: settings.get('backups.keepLast') }),
{ validate: (a) => c.id(a.guideId) && c.optionalString(a.label, 200) });
h('snapshots:restore', ({ guideId, name }) => { h('snapshots:restore', ({ guideId, name }) => {
const guide = restoreSnapshot(store, guideId, name); const guide = restoreSnapshot(store, guideId, name);
reindex(guideId); reindex(guideId);
return guide; return guide;
}); }, { validate: (a) => c.id(a.guideId) && c.fileName(a.name) });
// templates // templates
h('templates:list', ({ format }) => templates.list(format)); const validFormat = (v) => c.oneOf(v, FORMATS);
h('templates:load', ({ format, name }) => templates.load(format, name)); h('templates:list', ({ format }) => templates.list(format),
h('templates:save', ({ format, name, options }) => templates.save(format, name, options)); { validate: (a) => validFormat(a.format) });
h('templates:delete', ({ format, name }) => templates.remove(format, name)); h('templates:load', ({ format, name }) => templates.load(format, name),
h('templates:rename', ({ format, name, newName }) => templates.rename(format, name, newName)); { validate: (a) => validFormat(a.format) && c.fileName(a.name) });
h('templates:duplicate', ({ format, name }) => templates.duplicate(format, name)); h('templates:save', ({ format, name, options }) => templates.save(format, name, options),
{ validate: (a) => validFormat(a.format) && c.fileName(a.name) && security.isPlainArgs(a.options) });
h('templates:delete', ({ format, name }) => templates.remove(format, name),
{ validate: (a) => validFormat(a.format) && c.fileName(a.name) });
h('templates:rename', ({ format, name, newName }) => templates.rename(format, name, newName),
{ validate: (a) => validFormat(a.format) && c.fileName(a.name) && c.fileName(a.newName) });
h('templates:duplicate', ({ format, name }) => templates.duplicate(format, name),
{ validate: (a) => validFormat(a.format) && c.fileName(a.name) });
h('templates:export', async ({ format, name }) => { h('templates:export', async ({ format, name }) => {
const res = await dialog.showSaveDialog(mainWindow, { const res = await dialog.showSaveDialog(mainWindow, {
defaultPath: `${name}.sfglt`, defaultPath: `${name}.sfglt`,
@@ -621,7 +761,7 @@ function setupIpc() {
if (res.canceled) return { ok: false }; if (res.canceled) return { ok: false };
templates.exportTemplate(format, name, res.filePath); templates.exportTemplate(format, name, res.filePath);
return { ok: true }; return { ok: true };
}); }, { validate: (a) => validFormat(a.format) && c.fileName(a.name) });
h('templates:import', async () => { h('templates:import', async () => {
const res = await dialog.showOpenDialog(mainWindow, { const res = await dialog.showOpenDialog(mainWindow, {
filters: [{ name: 'StepForge template', extensions: ['sfglt'] }], filters: [{ name: 'StepForge template', extensions: ['sfglt'] }],
@@ -654,15 +794,24 @@ function setupIpc() {
}[format]; }[format];
if (!mod) return {}; if (!mod) return {};
return { ...require(mod).DEFAULT_TEMPLATE }; return { ...require(mod).DEFAULT_TEMPLATE };
}); }, { validate: (a) => c.string(a.format, 40) });
h('export:run', async ({ guideId, format, options, outDir }) => { h('export:run', async ({ guideId, format, options, outDir }) => {
let dir = outDir || settings.get(`exports.lastOutputDirs.${format}`); // The renderer may only nominate directories that came from this main
// process: a dialog pick from this session or a remembered last-output
// directory. Anything else is ignored and re-asked via the dialog.
const rememberedDirs = Object.values(settings.get('exports.lastOutputDirs') || {});
let dir = null;
if (outDir && (chosenOutputDirs.has(outDir) || rememberedDirs.includes(outDir))) {
dir = outDir;
}
if (!dir) dir = settings.get(`exports.lastOutputDirs.${format}`);
if (!dir) { if (!dir) {
const res = await dialog.showOpenDialog(mainWindow, { const res = await dialog.showOpenDialog(mainWindow, {
title: 'Choose output folder', properties: ['openDirectory', 'createDirectory'], title: 'Choose output folder', properties: ['openDirectory', 'createDirectory'],
}); });
if (res.canceled) return { ok: false }; if (res.canceled) return { ok: false };
dir = res.filePaths[0]; dir = res.filePaths[0];
chosenOutputDirs.add(dir);
} }
settings.set(`exports.lastOutputDirs.${format}`, dir); settings.set(`exports.lastOutputDirs.${format}`, dir);
const result = await runExportInWorker({ const result = await runExportInWorker({
@@ -673,17 +822,23 @@ function setupIpc() {
outDir: dir, outDir: dir,
globals: settings.getGlobalPlaceholders(), globals: settings.getGlobalPlaceholders(),
}); });
producedFiles.add(result.file);
if (settings.get('exports.openFolderAfterExport')) shell.showItemInFolder(result.file); if (settings.get('exports.openFolderAfterExport')) shell.showItemInFolder(result.file);
return { ok: true, ...result }; return { ok: true, ...result };
}, {
validate: (a) => c.id(a.guideId) && validFormat(a.format)
&& (a.options === undefined || security.isPlainArgs(a.options))
&& c.optionalString(a.outDir, 1000),
}); });
h('export:chooseDir', async ({ format }) => { h('export:chooseDir', async ({ format }) => {
const res = await dialog.showOpenDialog(mainWindow, { const res = await dialog.showOpenDialog(mainWindow, {
title: 'Choose output folder', properties: ['openDirectory', 'createDirectory'], title: 'Choose output folder', properties: ['openDirectory', 'createDirectory'],
}); });
if (res.canceled) return null; if (res.canceled) return null;
chosenOutputDirs.add(res.filePaths[0]);
settings.set(`exports.lastOutputDirs.${format}`, res.filePaths[0]); settings.set(`exports.lastOutputDirs.${format}`, res.filePaths[0]);
return res.filePaths[0]; return res.filePaths[0];
}); }, { validate: (a) => validFormat(a.format) });
h('export:preview', ({ guideId, format, options }) => { h('export:preview', ({ guideId, format, options }) => {
const previewDir = path.join(store.tempDir, `preview-${guideId}-${format}`); const previewDir = path.join(store.tempDir, `preview-${guideId}-${format}`);
fs.rmSync(previewDir, { recursive: true, force: true }); fs.rmSync(previewDir, { recursive: true, force: true });
@@ -692,7 +847,11 @@ function setupIpc() {
maxSteps: settings.get('exports.previewStepCount') || 3, maxSteps: settings.get('exports.previewStepCount') || 3,
}); });
const result = runExport(format, ast, previewDir, options || {}); const result = runExport(format, ast, previewDir, options || {});
producedFiles.add(result.file);
return { ok: true, file: result.file, fileUrl: `file://${result.file}` }; return { ok: true, file: result.file, fileUrl: `file://${result.file}` };
}, {
validate: (a) => c.id(a.guideId) && validFormat(a.format)
&& (a.options === undefined || security.isPlainArgs(a.options)),
}); });
h('preview:cleanup', () => { h('preview:cleanup', () => {
for (const entry of fs.readdirSync(store.tempDir)) { for (const entry of fs.readdirSync(store.tempDir)) {
@@ -703,11 +862,35 @@ function setupIpc() {
return true; return true;
}); });
// shell helpers // shell helpers — intent-specific, no arbitrary paths from the renderer.
h('shell:openPath', ({ target }) => shell.openPath(target)); // Only files this main process produced (exports/previews) may be opened.
h('shell:showItemInFolder', ({ target }) => shell.showItemInFolder(target)); h('shell:openProduced', ({ target }) => {
if (!producedFiles.has(target)) {
return { ok: false, reason: 'not a StepForge-produced file' };
}
shell.openPath(target);
return { ok: true };
}, { validate: (a) => c.string(a.target, 2000) });
// Reveal the linked archive of a guide; the path comes from the store,
// never from the renderer.
h('shell:revealLinkedArchive', ({ guideId }) => {
const guide = store.getGuide(guideId);
const target = guide && guide.linkedSource && guide.linkedSource.path;
if (!target || !fs.existsSync(target)) return { ok: false, reason: 'no linked archive' };
shell.showItemInFolder(target);
return { ok: true };
}, { validate: (a) => c.id(a.guideId) });
// Open a user-clicked link in the system browser. Scheme-validated;
// everything that is not plain http(s)/mailto is refused.
h('shell:openExternal', ({ url }) => {
const safe = security.validateExternalUrl(url);
if (!safe) return { ok: false, reason: 'blocked URL' };
shell.openExternal(safe);
return { ok: true };
}, { validate: (a) => c.string(a.url, 2048) });
h('app:info', () => ({ h('app:info', () => ({
version: app.getVersion(), version: app.getVersion(),
buildVersion: PACKAGE_JSON.buildVersion || app.getVersion(),
dataDir: store.root, dataDir: store.root,
platform: process.platform, platform: process.platform,
})); }));
@@ -739,6 +922,13 @@ if (!gotLock) {
settings = new Settings(store.settingsDir); settings = new Settings(store.settingsDir);
searchIndex = new SearchIndex(store.indexDir); searchIndex = new SearchIndex(store.indexDir);
templates = new TemplateManager(store.templatesDir); templates = new TemplateManager(store.templatesDir);
textIntel = new TextIntelService({
store,
settings,
getWindow: () => mainWindow,
dataDir,
screenApi: screen,
});
// Bringing up the desktop-capture stream spawns/upgrades Chromium's GPU // Bringing up the desktop-capture stream spawns/upgrades Chromium's GPU
// and screen-capture utility processes — which can be born after a session // and screen-capture utility processes — which can be born after a session
// already started, so the start-time EcoQoS opt-out misses them. Re-apply // already started, so the start-time EcoQoS opt-out misses them. Re-apply
@@ -752,6 +942,23 @@ if (!gotLock) {
try { keepProcessesResponsive(app.getAppMetrics().map((m) => m.pid)); } catch { /* best effort */ } try { keepProcessesResponsive(app.getAppMetrics().map((m) => m.pid)); } catch { /* best effort */ }
} }
} }
// Auto-document session captures in the background when autoDoc is enabled.
// Single-shot captures (capture:shoot) are handled synchronously in the IPC handler.
if (channel === 'capture:added' && payload?.step && payload?.guideId) {
const aiConf = settings.get('ai') || {};
if (aiConf.enabled && aiConf.autoDoc) {
textIntel.generateStepPatch({
guideId: payload.guideId,
stepId: payload.step.stepId,
target: 'all',
}).then((result) => {
if (result.ok) {
reindex(payload.guideId);
sendToRenderer('step:updated', { guideId: payload.guideId, step: result.step });
}
}).catch(() => {});
}
}
}; };
capture = new CaptureService({ capture = new CaptureService({
@@ -759,8 +966,49 @@ if (!gotLock) {
settings, settings,
getWindow: () => mainWindow, getWindow: () => mainWindow,
notify: captureNotify, notify: captureNotify,
textIntel,
}); });
// Deny-by-default permission policy. The only grant in the entire app is
// display capture (and the media permission getDisplayMedia consults) for
// the dedicated hidden capture-worker page. Electron 29+ requires that
// explicit grant; everything else — including our own main window — is
// rejected. "Content is local" is not a security control.
session.defaultSession.setPermissionCheckHandler((wc, permission, requestingOrigin, details) => {
const url = (details && details.requestingUrl) || (wc && wc.getURL()) || requestingOrigin;
return security.permissionAllowed(permission, url);
});
session.defaultSession.setPermissionRequestHandler((wc, permission, cb, details) => {
const url = (details && details.requestingUrl) || (wc && wc.getURL());
cb(security.permissionAllowed(permission, url));
});
// On GNOME Wayland the only working screen-capture path is the portal-backed
// getDisplayMedia (desktopCapturer source ids fail with "device not found").
// The worker calls getDisplayMedia; this handler answers it. Calling
// getSources() *inside* the handler is the documented Wayland path: it
// drives the XDG portal picker (shown once when a recording starts), and
// the chosen source then streams for the whole session. (useSystemPicker is
// macOS-only today, harmless elsewhere.)
session.defaultSession.setDisplayMediaRequestHandler((request, callback) => {
// Only the capture worker page may open a desktop stream.
const frameUrl = request && request.frame && request.frame.url;
if (!security.isAppPageUrl(frameUrl, 'captureWorker')) {
console.error('[stepforge] display-media request denied for', frameUrl || '(unknown frame)');
callback({});
return;
}
desktopCapturer.getSources({ types: ['screen'] })
.then((sources) => {
console.log(`[stepforge] display-media request resolved: ${sources.length} screen source(s)`);
callback(sources.length ? { video: sources[0] } : {});
})
.catch((err) => {
console.error(`[stepforge] display-media getSources failed: ${err && err.message}`);
callback({});
});
}, { useSystemPicker: true });
applyTheme(); applyTheme();
setupIpc(); setupIpc();
createWindow(); createWindow();
@@ -778,6 +1026,9 @@ if (!gotLock) {
capture.stopClickWatcher(); capture.stopClickWatcher();
capture.destroySessionTray(); capture.destroySessionTray();
} }
if (textIntel) {
textIntel.shutdown().catch(() => {});
}
// clean preview temp files on close // clean preview temp files on close
try { try {
for (const entry of fs.readdirSync(store.tempDir)) { for (const entry of fs.readdirSync(store.tempDir)) {
+11 -2
View File
@@ -52,6 +52,11 @@ const api = {
globalPlaceholders: invoke('placeholders:globals:get'), globalPlaceholders: invoke('placeholders:globals:get'),
setGlobalPlaceholders: invoke('placeholders:globals:set'), setGlobalPlaceholders: invoke('placeholders:globals:set'),
}, },
ai: {
test: invoke('ai:test'),
fillStep: invoke('ai:fillStep'),
rewriteText: invoke('ai:rewriteText'),
},
capture: { capture: {
shoot: invoke('capture:shoot'), shoot: invoke('capture:shoot'),
region: invoke('capture:region'), region: invoke('capture:region'),
@@ -59,6 +64,7 @@ const api = {
state: invoke('capture:state'), state: invoke('capture:state'),
onAdded: (fn) => ipcRenderer.on('capture:added', (e, payload) => fn(payload)), onAdded: (fn) => ipcRenderer.on('capture:added', (e, payload) => fn(payload)),
onState: (fn) => ipcRenderer.on('capture:state', (e, payload) => fn(payload)), onState: (fn) => ipcRenderer.on('capture:state', (e, payload) => fn(payload)),
onStepUpdated: (fn) => ipcRenderer.on('step:updated', (e, payload) => fn(payload)),
}, },
archive: { archive: {
export: invoke('archive:export'), export: invoke('archive:export'),
@@ -89,8 +95,11 @@ const api = {
cleanupPreviews: invoke('preview:cleanup'), cleanupPreviews: invoke('preview:cleanup'),
}, },
shell: { shell: {
openPath: invoke('shell:openPath'), // Intent-specific shell access only: files the main process produced,
showItemInFolder: invoke('shell:showItemInFolder'), // the guide's linked archive, and scheme-validated external links.
openProduced: invoke('shell:openProduced'),
revealLinkedArchive: invoke('shell:revealLinkedArchive'),
openExternal: invoke('shell:openExternal'),
}, },
app: { app: {
info: invoke('app:info'), info: invoke('app:info'),
+51 -5
View File
@@ -80,6 +80,18 @@ class StepForgeApp {
api.capture.onAdded((payload) => this.onCaptureAdded(payload)); api.capture.onAdded((payload) => this.onCaptureAdded(payload));
api.capture.onState((payload) => this.updateCaptureState(payload)); api.capture.onState((payload) => this.updateCaptureState(payload));
api.capture.onStepUpdated((payload) => this.onStepUpdated(payload));
}
async onStepUpdated(payload) {
if (!payload || !payload.guideId || !payload.step) return;
if (this.state.view === 'editor' && this.editor.guideId === payload.guideId) {
const currentStep = this.editor.currentStep;
if (currentStep && currentStep.stepId === payload.step.stepId) {
await this.editor.reload(payload.step.stepId);
toast('Documentation generated.');
}
}
} }
async onCaptureAdded(payload) { async onCaptureAdded(payload) {
@@ -110,6 +122,7 @@ class StepForgeApp {
api.library.trashList(), api.library.trashList(),
]); ]);
this.state.info = info; this.state.info = info;
document.body.classList.toggle('platform-linux', info.platform === 'linux');
this.state.settings = settings; this.state.settings = settings;
this.state.library = { this.state.library = {
guides: library.guides || [], guides: library.guides || [],
@@ -117,6 +130,7 @@ class StepForgeApp {
guideFolders: library.folders?.guideFolders || {}, guideFolders: library.folders?.guideFolders || {},
}; };
this.state.trash = trash; this.state.trash = trash;
this.editor.setSettings(settings);
} }
async refreshLibrary({ keepFilter = true } = {}) { async refreshLibrary({ keepFilter = true } = {}) {
@@ -250,15 +264,26 @@ class StepForgeApp {
updateCaptureState(state) { updateCaptureState(state) {
this.captureState = state || { active: false }; this.captureState = state || { active: false };
clearNode(this.captureStatus); clearNode(this.captureStatus);
// The capture bar only makes sense alongside the editor it's recording // The capture bar is editor-only — hide it everywhere else (library, welcome).
// into — hide it everywhere else (e.g. the library) even if a session if (this.state.view !== 'editor') {
// is still active in the background.
if (!this.captureState.active || this.state.view !== 'editor') {
this.captureStatus.classList.add('hidden'); this.captureStatus.classList.add('hidden');
return; return;
} }
this.captureStatus.classList.remove('hidden'); this.captureStatus.classList.remove('hidden');
const s = this.captureState; const s = this.captureState;
// No active session: show a button to start a new one for the open guide.
if (!s.active) {
this.captureStatus.append(
el('span', {}, 'Recording - stopped'),
el('button', {
type: 'button',
onClick: () => this.armCaptureSession(this.editor.guideId),
}, 'New recording'),
);
return;
}
const send = (payload) => api.capture.session(payload).then((next) => this.updateCaptureState(next)); const send = (payload) => api.capture.session(payload).then((next) => this.updateCaptureState(next));
// What is currently triggering captures, so the user knows what to do. // What is currently triggering captures, so the user knows what to do.
@@ -320,6 +345,7 @@ class StepForgeApp {
{ label: 'Guide information…', action: () => this.editor.openGuideInfo() }, { label: 'Guide information…', action: () => this.editor.openGuideInfo() },
{ label: 'Guide placeholders…', action: () => this.editor.openGuidePlaceholders() }, { label: 'Guide placeholders…', action: () => this.editor.openGuidePlaceholders() },
{ label: 'Backups & snapshots…', action: () => this.editor.openBackupsDialog() }, { label: 'Backups & snapshots…', action: () => this.editor.openBackupsDialog() },
{ label: 'Generate all text fields with AI (experimental)', action: () => this.editor.generateAllTextFieldsWithAi() },
{ label: guide && guide.linkedSource ? 'Linked guide…' : 'Linked guide (not linked)', action: () => this.editor.openLinkedGuide() }, { label: guide && guide.linkedSource ? 'Linked guide…' : 'Linked guide (not linked)', action: () => this.editor.openLinkedGuide() },
'sep', 'sep',
{ label: 'Keyboard shortcuts…', action: () => this.editor.openShortcutsHelp() }, { label: 'Keyboard shortcuts…', action: () => this.editor.openShortcutsHelp() },
@@ -375,7 +401,7 @@ class StepForgeApp {
el('div', { style: { fontWeight: 650 } }, folderLabel), el('div', { style: { fontWeight: 650 } }, folderLabel),
q ? el('div.muted', {}, `Search: ${q}`) : el('div.muted', {}, `${this.state.library.guides.length} guides`), q ? el('div.muted', {}, `Search: ${q}`) : el('div.muted', {}, `${this.state.library.guides.length} guides`),
), ),
el('div.muted', {}, this.state.info ? `StepForge ${this.state.info.version}` : ''), el('div.muted', {}, this.state.info ? `StepForge ${this.state.info.buildVersion || this.state.info.version}` : ''),
), ),
this.domBulkBar = el('div', {}), this.domBulkBar = el('div', {}),
this.domLibraryResults = el('div', {}), this.domLibraryResults = el('div', {}),
@@ -859,6 +885,7 @@ class StepForgeApp {
const settings = await api.settings.all(); const settings = await api.settings.all();
const placeholders = await api.settings.globalPlaceholders(); const placeholders = await api.settings.globalPlaceholders();
await dialogs.showSettingsDialog({ await dialogs.showSettingsDialog({
api,
settings, settings,
placeholders, placeholders,
onSave: async (next) => { onSave: async (next) => {
@@ -866,6 +893,7 @@ class StepForgeApp {
await api.settings.set({ keyPath: 'spellcheck', value: next.spellcheck }); await api.settings.set({ keyPath: 'spellcheck', value: next.spellcheck });
await api.settings.set({ keyPath: 'capture', value: next.capture }); await api.settings.set({ keyPath: 'capture', value: next.capture });
await api.settings.set({ keyPath: 'editor', value: next.editor }); await api.settings.set({ keyPath: 'editor', value: next.editor });
await api.settings.set({ keyPath: 'ai', value: next.ai });
await api.settings.set({ keyPath: 'exports', value: next.exports }); await api.settings.set({ keyPath: 'exports', value: next.exports });
await api.settings.set({ keyPath: 'backups', value: next.backups }); await api.settings.set({ keyPath: 'backups', value: next.backups });
await api.settings.setGlobalPlaceholders(next.placeholders || {}); await api.settings.setGlobalPlaceholders(next.placeholders || {});
@@ -906,6 +934,24 @@ class StepForgeApp {
window.StepForgeApp = StepForgeApp; window.StepForgeApp = StepForgeApp;
// Links never navigate this window. http(s)/mailto links from guide content
// open externally via the scheme-validated main-process handler; internal
// step:/# links are handled by their own click handlers; everything else is
// inert. The main process additionally denies all navigation, so this is the
// user-experience half of a two-layer guarantee.
document.addEventListener('click', (e) => {
const anchor = e.target && e.target.closest ? e.target.closest('a[href]') : null;
if (!anchor) return;
const href = anchor.getAttribute('href') || '';
if (/^(https?|mailto):/i.test(href)) {
e.preventDefault();
api.shell.openExternal({ url: href });
return;
}
// Ensure a stray href can never navigate the privileged window.
if (!href.startsWith('#')) e.preventDefault();
}, true);
function boot() { function boot() {
const app = new StepForgeApp(); const app = new StepForgeApp();
app.init(); app.init();
+20 -5
View File
@@ -68,8 +68,21 @@
}; };
streams.set(key, state); streams.set(key, state);
try { try {
// The chromeMediaSource constraint set is Electron's documented bridge if (cmd.useDisplayMedia) {
// from a desktopCapturer source id to a live media stream. // GNOME Wayland path: desktopCapturer source ids fail with
// getUserMedia, so go through the portal-backed getDisplayMedia. The
// main process installs a setDisplayMediaRequestHandler that answers
// this request; the OS portal picker chooses the screen. The stream
// then stays open for the whole session — one prompt, not one per shot.
state.media = await navigator.mediaDevices.getDisplayMedia({
audio: false,
video: true,
});
} else {
// Keep the legacy desktop-capture constraint wrapper here: it binds
// the stream to the exact desktop source chosen in the main process.
// Without it Chromium can treat the request like a normal media
// request and pick the default camera device instead.
state.media = await navigator.mediaDevices.getUserMedia({ state.media = await navigator.mediaDevices.getUserMedia({
audio: false, audio: false,
video: { video: {
@@ -80,12 +93,14 @@
maxWidth: physWidth, maxWidth: physWidth,
minHeight: physHeight, minHeight: physHeight,
maxHeight: physHeight, maxHeight: physHeight,
// No maxFrameRate: sampling cadence is controlled by the setInterval // No maxFrameRate: sampling cadence is controlled by the
// timer below, so the actual capture rate is always sampleMs-driven // setInterval timer below, so the actual capture rate is always
// regardless of display refresh rate or power mode. // sampleMs-driven regardless of display refresh rate or power
// mode.
}, },
}, },
}); });
}
const video = document.createElement('video'); const video = document.createElement('video');
video.muted = true; video.muted = true;
video.srcObject = state.media; video.srcObject = state.media;
+97
View File
@@ -285,6 +285,7 @@ function showQuickActions({ query = '', commands = [], searchFn, onOpenItem, onC
} }
function showSettingsDialog({ function showSettingsDialog({
api,
settings, settings,
placeholders = {}, placeholders = {},
onSave, onSave,
@@ -311,8 +312,65 @@ function showSettingsDialog({
const previewCount = makeInput(settings.exports?.previewStepCount ?? 3, 'number', { min: 1, step: 1 }); const previewCount = makeInput(settings.exports?.previewStepCount ?? 3, 'number', { min: 1, step: 1 });
const openFolder = el('input', { type: 'checkbox', checked: Boolean(settings.exports?.openFolderAfterExport) }); const openFolder = el('input', { type: 'checkbox', checked: Boolean(settings.exports?.openFolderAfterExport) });
const captureOutside = el('input', { type: 'checkbox', checked: Boolean(settings.capture?.captureOutsideClicks) }); const captureOutside = el('input', { type: 'checkbox', checked: Boolean(settings.capture?.captureOutsideClicks) });
const fallbackTrigger = makeSelect(settings.capture?.fallbackTrigger || 'interval', [
{ value: 'interval', label: 'Timed interval' },
{ value: 'hotkey', label: 'Hotkey only' },
]);
const autoIntervalSec = makeInput(settings.capture?.autoIntervalSec ?? 5, 'number', { min: 1, step: 1 });
const confirmSimple = el('input', { type: 'checkbox', checked: Boolean(settings.capture?.confirmSimpleCapture) }); const confirmSimple = el('input', { type: 'checkbox', checked: Boolean(settings.capture?.confirmSimpleCapture) });
const keepLast = makeInput(settings.backups?.keepLast ?? 10, 'number', { min: 0, step: 1 }); const keepLast = makeInput(settings.backups?.keepLast ?? 10, 'number', { min: 0, step: 1 });
const aiEnabled = el('input', { type: 'checkbox', checked: Boolean(settings.ai?.enabled) });
const aiAutoDoc = el('input', { type: 'checkbox', checked: Boolean(settings.ai?.autoDoc) });
const ollamaHost = makeInput(settings.ai?.ollama?.host || 'http://127.0.0.1:11434');
const ollamaModel = makeInput(settings.ai?.ollama?.model || 'llama3.2:1b');
const aiStatus = el('div', { className: 'muted ai-status' }, 'AI stays local through Ollama. Vision-capable models can also inspect the screenshot attached to each step.');
const testAiBtn = el('button', { type: 'button' }, 'Test connection');
const persistOllamaModel = debounce(() => {
const model = ollamaModel.value.trim();
// Keep the last model choice even if the dialog is dismissed without a full save.
void api.settings.set({ keyPath: 'ai.ollama.model', value: model }).catch(() => {});
}, 250);
const syncFallbackUi = () => {
autoIntervalSec.disabled = fallbackTrigger.value === 'hotkey';
};
fallbackTrigger.addEventListener('change', syncFallbackUi);
syncFallbackUi();
const updateAiStatus = (message, { error = false } = {}) => {
aiStatus.textContent = message;
aiStatus.classList.toggle('error', Boolean(error));
};
const testAiConnection = async () => {
setButtonLoading(testAiBtn, true, 'Testing…');
updateAiStatus('Checking Ollama at the configured host…');
try {
const result = await api.ai.test({
ollama: {
host: ollamaHost.value.trim(),
model: ollamaModel.value.trim(),
},
});
if (!result.ok) {
updateAiStatus(result.reason || 'Could not connect to Ollama.', { error: true });
return;
}
if (result.installed) {
updateAiStatus(result.vision
? `Connected to ${result.host} with ${result.model}. It can inspect screenshots.`
: `Connected to ${result.host} with ${result.model}. This model is text-only, so StepForge will use OCR and metadata only.`);
} else {
updateAiStatus(`Connected to ${result.host}. Model ${result.model} is not installed yet.`, { error: true });
}
} catch (err) {
updateAiStatus(err.message || 'Could not connect to Ollama.', { error: true });
} finally {
setButtonLoading(testAiBtn, false);
}
};
ollamaModel.addEventListener('input', () => persistOllamaModel());
ollamaModel.addEventListener('blur', () => persistOllamaModel.flush());
const placeholderRows = el('div', { className: 'placeholder-rows' }); const placeholderRows = el('div', { className: 'placeholder-rows' });
const rows = []; const rows = [];
@@ -356,9 +414,14 @@ function showSettingsDialog({
labeledRow('Delay (ms)', delayMs), labeledRow('Delay (ms)', delayMs),
labeledRow('Click marker', clickMarker), labeledRow('Click marker', clickMarker),
labeledRow('Capture outside clicks', captureOutside), labeledRow('Capture outside clicks', captureOutside),
labeledRow('When clicks are unavailable', fallbackTrigger),
labeledRow('Timer interval (seconds)', autoIntervalSec),
labeledRow('Confirm simple capture', confirmSimple), labeledRow('Confirm simple capture', confirmSimple),
labeledRow('Capture hotkey', captureHotkey), labeledRow('Capture hotkey', captureHotkey),
labeledRow('Pause / resume hotkey', pauseHotkey), labeledRow('Pause / resume hotkey', pauseHotkey),
el('div.muted', {},
'Hotkey fallback uses the Capture hotkey. Timer fallback uses the interval above.',
),
), ),
el('fieldset', {}, el('fieldset', {},
el('legend', {}, 'Editor'), el('legend', {}, 'Editor'),
@@ -369,6 +432,20 @@ function showSettingsDialog({
el('legend', {}, 'Backups'), el('legend', {}, 'Backups'),
labeledRow('Keep last snapshots', keepLast), labeledRow('Keep last snapshots', keepLast),
), ),
el('fieldset', {},
el('legend', {}, 'AI'),
labeledRow('Enable AI (experimental)', aiEnabled),
labeledRow('Auto-document captures', aiAutoDoc),
labeledRow('Ollama host', ollamaHost),
labeledRow('Ollama model', ollamaModel),
el('div.row', { style: { justifyContent: 'space-between' } },
aiStatus,
testAiBtn,
),
el('div.muted', {},
'When auto-document is on, each capture is automatically documented by AI. Turn it off to use AI manually only.',
),
),
el('fieldset', {}, el('fieldset', {},
el('legend', {}, 'Global placeholders'), el('legend', {}, 'Global placeholders'),
placeholderRows, placeholderRows,
@@ -394,6 +471,8 @@ function showSettingsDialog({
delayMs: Number(delayMs.value || 0), delayMs: Number(delayMs.value || 0),
mode: captureMode.value, mode: captureMode.value,
clickMarker: clickMarker.checked, clickMarker: clickMarker.checked,
fallbackTrigger: fallbackTrigger.value === 'hotkey' ? 'hotkey' : 'interval',
autoIntervalSec: Math.max(1, Number(autoIntervalSec.value || 5)),
hotkeyCapture: captureHotkey.value.trim(), hotkeyCapture: captureHotkey.value.trim(),
hotkeyPauseResume: pauseHotkey.value.trim(), hotkeyPauseResume: pauseHotkey.value.trim(),
captureOutsideClicks: captureOutside.checked, captureOutsideClicks: captureOutside.checked,
@@ -412,6 +491,16 @@ function showSettingsDialog({
...settings.backups, ...settings.backups,
keepLast: Number(keepLast.value || 0), keepLast: Number(keepLast.value || 0),
}, },
ai: {
...settings.ai,
enabled: aiEnabled.checked,
autoDoc: aiAutoDoc.checked,
ollama: {
...(settings.ai?.ollama || {}),
host: ollamaHost.value.trim(),
model: ollamaModel.value.trim(),
},
},
placeholders: rows.reduce((acc, row) => { placeholders: rows.reduce((acc, row) => {
const inputs = row.querySelectorAll('input'); const inputs = row.querySelectorAll('input');
const key = inputs[0].value.trim(); const key = inputs[0].value.trim();
@@ -430,6 +519,14 @@ function showSettingsDialog({
}); });
form.addEventListener('submit', (e) => e.preventDefault()); form.addEventListener('submit', (e) => e.preventDefault());
testAiBtn.addEventListener('click', testAiConnection);
aiEnabled.addEventListener('change', () => {
updateAiStatus(
aiEnabled.checked
? 'AI generation will be available once Ollama is reachable.'
: 'AI generation is disabled. The settings are still saved for later.',
);
});
}); });
} }
+182 -4
View File
@@ -135,6 +135,7 @@ class GuideEditor {
this.descriptionDirty = false; this.descriptionDirty = false;
this.titleDirty = false; this.titleDirty = false;
this.active = true; this.active = true;
this.settings = {};
this.saveStepDebounced = debounce(() => this.flushStep(), 180); this.saveStepDebounced = debounce(() => this.flushStep(), 180);
this.saveGuideDebounced = debounce(() => this.flushGuide(), 180); this.saveGuideDebounced = debounce(() => this.flushGuide(), 180);
@@ -152,6 +153,149 @@ class GuideEditor {
this.active = Boolean(active); this.active = Boolean(active);
} }
setSettings(settings) {
this.settings = settings || {};
this.updateAiButtonState();
}
isAiEnabled() {
return Boolean(this.settings?.ai?.enabled);
}
updateAiButtonState() {
const enabled = this.isAiEnabled();
const buttons = [
this.dom?.titleAiBtn,
this.dom?.descAiBtn,
...(this.dom?.blocksList ? [...this.dom.blocksList.querySelectorAll('button[data-ai-action]')] : []),
].filter(Boolean);
for (const button of buttons) {
button.disabled = !enabled;
button.title = enabled
? button.dataset.aiTitle || 'Generate with AI'
: 'Enable AI in Settings first.';
}
}
async runAiGeneration(target, { blockId = null, button = null } = {}) {
if (!this.currentStep) {
this.onToast('Select a step first.', { error: true });
return null;
}
if (!this.isAiEnabled()) {
this.onToast('Enable AI in Settings first.', { error: true });
return null;
}
if (this.pendingSave) await this.flushStep();
if (button) setButtonLoading(button, true, 'Generating…');
try {
const result = await api.ai.fillStep({
guideId: this.guideId,
stepId: this.currentStep.stepId,
target,
blockId,
});
if (!result || !result.ok) {
this.onToast(result?.reason || 'AI generation failed.', { error: true });
return null;
}
await this.reload(result.step.stepId);
this.onToast('AI text filled.');
return result.step;
} catch (err) {
this.onToast(err.message || 'AI generation failed.', { error: true });
return null;
} finally {
if (button) setButtonLoading(button, false);
}
}
async generateTitleWithAi(button = null) {
return this.runAiGeneration('title', { button });
}
async generateDescriptionWithAi(button = null) {
return this.runAiGeneration('description', { button });
}
async generateAllTextFieldsWithAi(button = null) {
if (!this.steps.length) {
this.onToast('No steps to generate.', { error: true });
return;
}
if (!this.isAiEnabled()) {
this.onToast('Enable AI in Settings first.', { error: true });
return;
}
if (this.pendingSave) await this.flushStep();
// Only fill fields that are actually empty — never overwrite user-written content.
const PLACEHOLDER_TITLES = new Set([
'', 'screen capture', 'window capture', 'region capture', 'capture', 'untitled step',
]);
const isEmptyDesc = (html) => !(html || '').replace(/<[^>]*>/g, '').trim();
const queue = this.steps
.map((step) => {
const titleEmpty = !step.title || PLACEHOLDER_TITLES.has(step.title.toLowerCase());
const descEmpty = isEmptyDesc(step.descriptionHtml);
if (!titleEmpty && !descEmpty) return null;
const target = titleEmpty && descEmpty ? 'all' : titleEmpty ? 'title' : 'description';
return { stepId: step.stepId, target };
})
.filter(Boolean);
if (!queue.length) {
this.onToast('All steps already have titles and descriptions.');
return;
}
if (button) setButtonLoading(button, true, 'Generating…');
let done = 0;
let failed = 0;
const total = queue.length;
try {
for (const { stepId, target } of queue) {
this.onToast(`AI: filling step ${done + failed + 1} of ${total}`);
try {
const result = await api.ai.fillStep({ guideId: this.guideId, stepId, target });
if (result?.ok) done++;
else failed++;
} catch {
failed++;
}
}
// Reload re-fetches all steps from the store so the editor and list both reflect the new text.
await this.reload(this.selectedStepId);
const msg = failed
? `AI filled ${done} step${done === 1 ? '' : 's'} (${failed} failed).`
: `AI filled ${done} step${done === 1 ? '' : 's'}.`;
this.onToast(msg, failed ? { error: true } : undefined);
} finally {
if (button) setButtonLoading(button, false);
}
}
async generateBlockWithAi(kind, block, button = null) {
if (!block) return null;
return this.runAiGeneration('block', { blockId: block.id, button });
}
updateAiButtonHints() {
const PLACEHOLDER_TITLES = new Set([
'', 'screen capture', 'window capture', 'region capture', 'capture',
]);
if (this.dom.titleAiBtn) {
const val = (this.dom.titleInput?.value || '').trim();
const hasDraft = Boolean(val) && !PLACEHOLDER_TITLES.has(val.toLowerCase());
this.dom.titleAiBtn.title = hasDraft ? 'Rewrite step title with AI' : 'Generate step title with AI';
}
if (this.dom.descAiBtn) {
const hasDesc = Boolean((this.dom.descEditor?.innerText || '').trim());
this.dom.descAiBtn.title = hasDesc ? 'Rewrite description with AI' : 'Generate description with AI';
}
}
get currentStep() { get currentStep() {
return this.stepMap.get(this.selectedStepId) || null; return this.stepMap.get(this.selectedStepId) || null;
} }
@@ -271,7 +415,14 @@ class GuideEditor {
el('aside.pane-props', {}, el('aside.pane-props', {},
el('section', {}, el('section', {},
el('h3', {}, 'Step'), el('h3', {}, 'Step'),
this.dom.titleInput = el('input', { type: 'text', placeholder: 'Step title' }), el('div.row', {},
this.dom.titleInput = el('input', { type: 'text', placeholder: 'Step title', style: { flex: 1 } }),
this.dom.titleAiBtn = el('button.ai', {
type: 'button',
title: 'Generate the step title with AI',
dataset: { aiAction: 'title', aiTitle: 'Generate the step title with AI' },
}, 'AI'),
),
this.dom.statusSelect = makeSelect('todo', [ this.dom.statusSelect = makeSelect('todo', [
{ value: 'todo', label: 'Todo' }, { value: 'todo', label: 'Todo' },
{ value: 'in-progress', label: 'In progress' }, { value: 'in-progress', label: 'In progress' },
@@ -296,7 +447,14 @@ class GuideEditor {
), ),
), ),
el('section', {}, el('section', {},
el('h3', {}, 'Description'), el('div.row', { style: { justifyContent: 'space-between', alignItems: 'center' } },
el('h3', { style: { margin: 0 } }, 'Description'),
this.dom.descAiBtn = el('button.ai', {
type: 'button',
title: 'Generate the description with AI',
dataset: { aiAction: 'description', aiTitle: 'Generate the description with AI' },
}, 'AI'),
),
this.dom.richToolbar = el('div.rich-toolbar', {}, this.dom.richToolbar = el('div.rich-toolbar', {},
this.toolbarBtn('bold', 'Bold'), this.toolbarBtn('bold', 'Bold'),
this.toolbarBtn('italic', 'Italic'), this.toolbarBtn('italic', 'Italic'),
@@ -415,7 +573,9 @@ class GuideEditor {
this.saveStepDebounced(); this.saveStepDebounced();
this.renderStepList(); this.renderStepList();
this.emitMeta(); this.emitMeta();
this.updateAiButtonHints();
}); });
this.dom.titleAiBtn.addEventListener('click', () => this.generateTitleWithAi(this.dom.titleAiBtn));
this.dom.statusSelect.addEventListener('change', () => { this.dom.statusSelect.addEventListener('change', () => {
if (!this.currentStep) return; if (!this.currentStep) return;
@@ -485,12 +645,14 @@ class GuideEditor {
this.saveStepDebounced(); this.saveStepDebounced();
this.emitMeta(); this.emitMeta();
this.updateToolbarState(); this.updateToolbarState();
this.updateAiButtonHints();
}); });
this.dom.descEditor.addEventListener('keyup', () => this.updateToolbarState()); this.dom.descEditor.addEventListener('keyup', () => this.updateToolbarState());
this.dom.descEditor.addEventListener('mouseup', () => this.updateToolbarState()); this.dom.descEditor.addEventListener('mouseup', () => this.updateToolbarState());
document.addEventListener('selectionchange', () => { document.addEventListener('selectionchange', () => {
if (document.activeElement === this.dom.descEditor) this.updateToolbarState(); if (document.activeElement === this.dom.descEditor) this.updateToolbarState();
}); });
this.dom.descAiBtn.addEventListener('click', () => this.generateDescriptionWithAi(this.dom.descAiBtn));
this.dom.descEditor.addEventListener('paste', (e) => { this.dom.descEditor.addEventListener('paste', (e) => {
// Keep pasted text simple; backend sanitization will handle the rest. // Keep pasted text simple; backend sanitization will handle the rest.
@@ -504,6 +666,8 @@ class GuideEditor {
if (!item) return; if (!item) return;
this.canvas.select(item.dataset.annId); this.canvas.select(item.dataset.annId);
}); });
this.updateAiButtonState();
} }
renderAll() { renderAll() {
@@ -513,6 +677,7 @@ class GuideEditor {
this.renderCanvas(); this.renderCanvas();
this.renderAnnotationPanel(); this.renderAnnotationPanel();
this.renderBlocksPanel(); this.renderBlocksPanel();
this.updateAiButtonState();
this.emitMeta(); this.emitMeta();
} }
@@ -670,6 +835,15 @@ class GuideEditor {
el('span.spacer'), el('span.spacer'),
el('button.icon', { type: 'button', title: 'Move block up', disabled: !canMoveUp, onClick: moveUp, dataset: { blockMove: 'up' } }, '↑'), el('button.icon', { type: 'button', title: 'Move block up', disabled: !canMoveUp, onClick: moveUp, dataset: { blockMove: 'up' } }, '↑'),
el('button.icon', { type: 'button', title: 'Move block down', disabled: !canMoveDown, onClick: moveDown, dataset: { blockMove: 'down' } }, '↓'), el('button.icon', { type: 'button', title: 'Move block down', disabled: !canMoveDown, onClick: moveDown, dataset: { blockMove: 'down' } }, '↓'),
(() => {
const aiBtn = el('button.ai', {
type: 'button',
title: 'Generate this block with AI',
dataset: { aiAction: 'block', aiTitle: 'Generate this block with AI' },
onClick: () => this.generateBlockWithAi(kind, block, aiBtn),
}, 'AI');
return aiBtn;
})(),
removeBtn(() => { removeBtn(() => {
if (kind === 'text') step.textBlocks = (step.textBlocks || []).filter((b) => b !== block); if (kind === 'text') step.textBlocks = (step.textBlocks || []).filter((b) => b !== block);
else if (kind === 'code') step.codeBlocks = (step.codeBlocks || []).filter((b) => b !== block); else if (kind === 'code') step.codeBlocks = (step.codeBlocks || []).filter((b) => b !== block);
@@ -761,6 +935,7 @@ class GuideEditor {
if (!blocks.length) { if (!blocks.length) {
this.dom.blocksList.append(el('div.muted', {}, 'Informational text, code, and table blocks can be reordered with drag handles or arrows.')); this.dom.blocksList.append(el('div.muted', {}, 'Informational text, code, and table blocks can be reordered with drag handles or arrows.'));
} }
this.updateAiButtonState();
} }
renderStepList() { renderStepList() {
@@ -915,6 +1090,7 @@ class GuideEditor {
if (document.activeElement !== this.dom.titleInput) this.dom.titleInput.value = step.title || ''; if (document.activeElement !== this.dom.titleInput) this.dom.titleInput.value = step.title || '';
if (document.activeElement !== this.dom.descEditor) this.dom.descEditor.innerHTML = step.descriptionHtml || ''; if (document.activeElement !== this.dom.descEditor) this.dom.descEditor.innerHTML = step.descriptionHtml || '';
this.dom.statusSelect.value = step.status || 'todo'; this.dom.statusSelect.value = step.status || 'todo';
this.updateAiButtonHints();
this.dom.hiddenToggle.querySelector('input').checked = Boolean(step.hidden); this.dom.hiddenToggle.querySelector('input').checked = Boolean(step.hidden);
this.dom.skippedToggle.querySelector('input').checked = Boolean(step.skipped); this.dom.skippedToggle.querySelector('input').checked = Boolean(step.skipped);
this.dom.forceNewPageToggle.querySelector('input').checked = Boolean(step.forceNewPage); this.dom.forceNewPageToggle.querySelector('input').checked = Boolean(step.forceNewPage);
@@ -1651,6 +1827,7 @@ class GuideEditor {
const settings = await api.settings.all(); const settings = await api.settings.all();
const placeholders = await api.settings.globalPlaceholders(); const placeholders = await api.settings.globalPlaceholders();
await dialogs.showSettingsDialog({ await dialogs.showSettingsDialog({
api,
settings, settings,
placeholders, placeholders,
onSave: async (next) => { onSave: async (next) => {
@@ -1658,6 +1835,7 @@ class GuideEditor {
await api.settings.set({ keyPath: 'spellcheck', value: next.spellcheck }); await api.settings.set({ keyPath: 'spellcheck', value: next.spellcheck });
await api.settings.set({ keyPath: 'capture', value: next.capture }); await api.settings.set({ keyPath: 'capture', value: next.capture });
await api.settings.set({ keyPath: 'editor', value: next.editor }); await api.settings.set({ keyPath: 'editor', value: next.editor });
await api.settings.set({ keyPath: 'ai', value: next.ai });
await api.settings.set({ keyPath: 'exports', value: next.exports }); await api.settings.set({ keyPath: 'exports', value: next.exports });
await api.settings.set({ keyPath: 'backups', value: next.backups }); await api.settings.set({ keyPath: 'backups', value: next.backups });
await api.settings.setGlobalPlaceholders(next.placeholders || {}); await api.settings.setGlobalPlaceholders(next.placeholders || {});
@@ -1721,7 +1899,7 @@ class GuideEditor {
onPreview: async ({ format, options }) => { onPreview: async ({ format, options }) => {
const preview = await api.export.preview({ guideId: this.guideId, format, options }); const preview = await api.export.preview({ guideId: this.guideId, format, options });
if (preview && preview.file) { if (preview && preview.file) {
await api.shell.openPath({ target: preview.file }); // open in default viewer await api.shell.openProduced({ target: preview.file }); // open in default viewer
this.onToast('Preview opened (first steps only).'); this.onToast('Preview opened (first steps only).');
} }
return true; return true;
@@ -1757,7 +1935,7 @@ class GuideEditor {
else this.onToast('Could not save linked archive.', { error: true }); else this.onToast('Could not save linked archive.', { error: true });
}, },
onOpenArchive: async () => { onOpenArchive: async () => {
await api.shell.showItemInFolder({ target: this.guide.linkedSource.path }); await api.shell.revealLinkedArchive({ guideId: this.guideId });
}, },
}); });
} }
+36
View File
@@ -49,6 +49,16 @@ body {
overflow: hidden; overflow: hidden;
} }
body.platform-linux button {
padding: 5px 11px;
}
body.platform-linux input,
body.platform-linux select,
body.platform-linux textarea {
padding: 6px 9px;
}
*::selection { background: rgba(0, 104, 255, 0.2); } *::selection { background: rgba(0, 104, 255, 0.2); }
#app { display: flex; flex-direction: column; height: 100vh; } #app { display: flex; flex-direction: column; height: 100vh; }
@@ -90,6 +100,16 @@ button.tool.active {
border-color: var(--accent); border-color: var(--accent);
color: var(--accent-fg); color: var(--accent-fg);
} }
button.ai {
padding: 5px 10px;
border-color: color-mix(in srgb, var(--accent) 38%, var(--border));
background: color-mix(in srgb, var(--accent) 10%, var(--panel-solid));
color: var(--accent-strong);
font-weight: 650;
letter-spacing: 0.02em;
}
button.ai:hover { background: color-mix(in srgb, var(--accent) 16%, var(--panel-2)); }
button.ai:disabled { opacity: 0.6; }
button:disabled { opacity: 0.6; cursor: default; } button:disabled { opacity: 0.6; cursor: default; }
button.loading { button.loading {
display: inline-flex; display: inline-flex;
@@ -173,6 +193,21 @@ kbd {
color: #fff; color: #fff;
} }
body.platform-linux #topbar {
height: 52px;
gap: 10px;
padding: 0 14px;
}
body.platform-linux #capture-status {
padding: 5px 9px;
gap: 7px;
}
body.platform-linux #capture-status button {
padding: 2px 7px;
}
.library, .editor { flex: 1; min-height: 0; display: flex; } .library, .editor { flex: 1; min-height: 0; display: flex; }
.lib-side { .lib-side {
@@ -751,6 +786,7 @@ fieldset legend {
color: var(--muted); color: var(--muted);
line-height: 1.5; line-height: 1.5;
} }
.ai-status.error { color: var(--danger); }
#modal-root:not(:empty) { #modal-root:not(:empty) {
position: fixed; position: fixed;
+302
View File
@@ -0,0 +1,302 @@
'use strict';
/**
* Privilege-boundary policy for every renderer surface.
*
* This module is deliberately loadable under plain Node (no electron
* require at module scope) so the decision logic is unit-testable:
* - which URLs count as our own app pages,
* - whether a navigation/popup may proceed (it may not),
* - which permission a renderer may be granted (display capture for the
* dedicated capture worker only),
* - whether an IPC event comes from the trusted main-window frame,
* - which external URLs may be handed to shell.openExternal,
* - which filesystem paths the renderer may ask the shell to open
* (only files the main process itself produced).
*/
const path = require('node:path');
const { pathToFileURL } = require('node:url');
const RENDERER_DIR = path.join(__dirname, 'renderer');
const APP_PAGES = {
main: pathToFileURL(path.join(RENDERER_DIR, 'index.html')).href,
region: pathToFileURL(path.join(RENDERER_DIR, 'region.html')).href,
captureWorker: pathToFileURL(path.join(RENDERER_DIR, 'capture-worker.html')).href,
};
/** Normalize a URL for identity comparison: drop query/hash, decode path. */
function normalizeAppUrl(rawUrl) {
let url;
try {
url = new URL(String(rawUrl));
} catch {
return null;
}
if (url.protocol !== 'file:') return null;
let pathname;
try {
pathname = decodeURIComponent(url.pathname);
} catch {
pathname = url.pathname;
}
// Windows drive letters may differ in case between loadFile and senderFrame.
if (process.platform === 'win32') pathname = pathname.toLowerCase();
return pathname;
}
function isAppPageUrl(rawUrl, page) {
const expected = APP_PAGES[page];
if (!expected) return false;
const a = normalizeAppUrl(rawUrl);
const b = normalizeAppUrl(expected);
return a !== null && a === b;
}
/**
* Navigation policy: a privileged window may only ever stay on the exact
* page it was created with. Everything else remote URLs, other local
* files, javascript:, data: is denied.
*/
function navigationAllowed(targetUrl, page) {
return isAppPageUrl(targetUrl, page);
}
/**
* Permission policy for Electron sessions. Display capture (and the media
* permission that getDisplayMedia consults) is granted only to the dedicated
* hidden capture-worker page. Everything else is denied for everyone,
* including our own main window.
*/
function permissionAllowed(permission, requestingUrl) {
if (permission === 'media' || permission === 'display-capture') {
return isAppPageUrl(requestingUrl, 'captureWorker');
}
return false;
}
/**
* External-link policy: only well-formed http(s) and mailto URLs may reach
* shell.openExternal. Returns the normalized URL string or null.
*/
function validateExternalUrl(rawUrl) {
if (typeof rawUrl !== 'string' || rawUrl.length > 2048) return null;
let url;
try {
url = new URL(rawUrl);
} catch {
return null;
}
if (url.protocol === 'https:' || url.protocol === 'http:') {
if (!url.hostname) return null;
return url.href;
}
if (url.protocol === 'mailto:') return url.href;
return null;
}
/**
* IPC sender guard: an invoke event is trusted only when it originates from
* the current main window's top frame, and that frame is our index.html.
* Destroyed frames, subframes, other windows, and navigated-away frames are
* all rejected.
*/
function makeIpcSenderGuard({ getMainWebContents }) {
return function trustedSender(event) {
if (!event || typeof event !== 'object') return false;
const expected = getMainWebContents();
if (!expected || event.sender !== expected) return false;
const frame = event.senderFrame;
if (!frame) return false;
try {
if (frame.parent) return false; // top frame only
return isAppPageUrl(frame.url, 'main');
} catch {
// Accessing a disposed WebFrameMain throws — reject.
return false;
}
};
}
/**
* Cheap recursive payload budget: sums string lengths (and key counts) with
* early exit. Protects handlers from absurd payloads without JSON.stringify
* on hundred-megabyte image saves.
*/
function payloadWithinBudget(value, maxChars, depth = 0) {
let budget = maxChars;
const walk = (v, d) => {
if (budget < 0 || d > 16) return false;
if (v == null) return true;
const t = typeof v;
if (t === 'string') {
budget -= v.length;
return budget >= 0;
}
if (t === 'number' || t === 'boolean') {
budget -= 8;
return budget >= 0;
}
if (t === 'function' || t === 'symbol' || t === 'bigint') return false;
if (Array.isArray(v)) {
if (v.length > 100000) return false;
for (const item of v) if (!walk(item, d + 1)) return false;
return true;
}
if (t === 'object') {
const keys = Object.keys(v);
if (keys.length > 4096) return false;
for (const key of keys) {
budget -= key.length;
if (budget < 0) return false;
if (!walk(v[key], d + 1)) return false;
}
return true;
}
return false;
};
return walk(value, depth);
}
/** True for plain-object argument bags (what every IPC channel expects). */
function isPlainArgs(args) {
if (args === undefined || args === null) return true;
if (typeof args !== 'object' || Array.isArray(args)) return false;
const proto = Object.getPrototypeOf(args);
return proto === Object.prototype || proto === null;
}
// ---- field validators (used by main.js per-channel checks) ----------------
const ID_RE = /^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/;
const check = {
/** Guide/step/folder/template identifiers and snapshot/trash entry names:
* single path segment, no separators, no dot-dot, sane length. */
id(v) {
return typeof v === 'string' && ID_RE.test(v) && !v.includes('..');
},
optionalId(v) {
return v === null || v === undefined || check.id(v);
},
string(v, max = 4096) {
return typeof v === 'string' && v.length <= max;
},
optionalString(v, max = 4096) {
return v === null || v === undefined || check.string(v, max);
},
bool(v) {
return typeof v === 'boolean';
},
number(v, min = -1e15, max = 1e15) {
return typeof v === 'number' && Number.isFinite(v) && v >= min && v <= max;
},
optionalNumber(v, min, max) {
return v === null || v === undefined || check.number(v, min, max);
},
oneOf(v, values) {
return values.includes(v);
},
base64(v, maxChars = 192 * 1024 * 1024) {
return typeof v === 'string' && v.length <= maxChars && /^[A-Za-z0-9+/=\r\n]*$/.test(v.slice(0, 4096));
},
optionalBase64(v, maxChars) {
return v === null || v === undefined || check.base64(v, maxChars);
},
/** Single filesystem name (template/snapshot/trash entries): no path
* separators, no traversal, no NUL, printable, bounded length. */
fileName(v, max = 160) {
return (
typeof v === 'string' &&
v.trim().length > 0 &&
v.length <= max &&
!/[/\\\0]/.test(v) &&
!v.includes('..') &&
v !== '.' &&
// eslint-disable-next-line no-control-regex
!/[\x00-\x1f]/.test(v)
);
},
optionalFileName(v, max) {
return v === null || v === undefined || check.fileName(v, max);
},
/** settings keyPath: dotted segments, no prototype-pollution segments. */
settingsKeyPath(v) {
if (typeof v !== 'string' || v.length === 0 || v.length > 200) return false;
const segments = v.split('.');
return segments.every(
(segment) =>
/^[A-Za-z0-9_-]+$/.test(segment) &&
!['__proto__', 'constructor', 'prototype'].includes(segment)
);
},
};
/**
* Registry of files the main process itself produced (export outputs,
* previews) and may therefore re-open on renderer request. Bounded LRU.
*/
class ProducedFiles {
constructor(limit = 256) {
this.limit = limit;
this.paths = new Set();
}
key(p) {
const resolved = path.resolve(String(p));
return process.platform === 'win32' ? resolved.toLowerCase() : resolved;
}
add(p) {
if (!p || typeof p !== 'string') return;
const key = this.key(p);
this.paths.delete(key);
this.paths.add(key);
while (this.paths.size > this.limit) {
this.paths.delete(this.paths.values().next().value);
}
}
has(p) {
if (!p || typeof p !== 'string') return false;
return this.paths.has(this.key(p));
}
}
/**
* Attach the navigation/popup policy to a BrowserWindow. `page` names the
* app page this window is allowed to display (key of APP_PAGES).
*/
function installWindowSecurity(win, page) {
const contents = win.webContents;
contents.setWindowOpenHandler(() => ({ action: 'deny' }));
contents.on('will-navigate', (event, url) => {
if (!navigationAllowed(url, page)) event.preventDefault();
});
contents.on('will-frame-navigate', (details) => {
if (!navigationAllowed(details.url, page) && typeof details.preventDefault === 'function') {
details.preventDefault();
}
});
// Defense in depth: if a disallowed document somehow starts loading,
// never let it attach webviews either.
contents.on('will-attach-webview', (event) => event.preventDefault());
}
module.exports = {
APP_PAGES,
ProducedFiles,
check,
installWindowSecurity,
isAppPageUrl,
isPlainArgs,
makeIpcSenderGuard,
navigationAllowed,
normalizeAppUrl,
payloadWithinBudget,
permissionAllowed,
validateExternalUrl,
};
+28 -6
View File
@@ -83,9 +83,19 @@ class StreamCaptureBackend {
* Spin up the worker and one stream per display that has a matching screen * Spin up the worker and one stream per display that has a matching screen
* source. Resolves true when at least one stream is delivering frames. * source. Resolves true when at least one stream is delivering frames.
*/ */
async start({ displays = [], sources = [], sampleMs = DEFAULT_SAMPLE_MS, retentionMs = null, frameLimit = null } = {}) { async start({
displays = [], sources = [], sampleMs = DEFAULT_SAMPLE_MS,
retentionMs = null, frameLimit = null, useDisplayMedia = false,
} = {}) {
if (this.host) return this.active; if (this.host) return this.active;
const pairs = pairDisplaysToSources(displays, sources); // On GNOME Wayland, desktopCapturer source ids can't be reopened with
// getUserMedia ("Requested device not found") — the portal-backed
// getDisplayMedia path is the only one that works. There's no per-display
// source id in that mode, so capture a single stream against the primary
// display (the portal picker decides which screen is actually shared).
const pairs = useDisplayMedia
? (displays.length ? [{ display: displays[0], sourceId: null }] : [])
: pairDisplaysToSources(displays, sources);
if (!pairs.length) return false; if (!pairs.length) return false;
try { try {
this.host = await this.createHost((msg) => this.handleWorkerEvent(msg)); this.host = await this.createHost((msg) => this.handleWorkerEvent(msg));
@@ -99,6 +109,7 @@ class StreamCaptureBackend {
type: 'start-stream', type: 'start-stream',
displayId: display.id, displayId: display.id,
sourceId, sourceId,
useDisplayMedia,
// The worker needs the physical pixel size to request a full-res // The worker needs the physical pixel size to request a full-res
// stream; bounds stay in DIP for marker math back in the main process. // stream; bounds stay in DIP for marker math back in the main process.
display: { display: {
@@ -154,6 +165,9 @@ class StreamCaptureBackend {
stream.ready = msg.type === 'stream-ready'; stream.ready = msg.type === 'stream-ready';
stream.failed = msg.type === 'stream-error'; stream.failed = msg.type === 'stream-error';
} }
if (msg.type === 'stream-error') {
console.error(`[stepforge] capture worker stream-error display=${msg.displayId}: ${msg.reason}`);
}
for (const check of [...this.startWaiters]) check(); for (const check of [...this.startWaiters]) check();
return; return;
} }
@@ -167,7 +181,7 @@ class StreamCaptureBackend {
clearTimeout(pending.timer); clearTimeout(pending.timer);
pending.timer = setTimeout(() => { pending.timer = setTimeout(() => {
this.settleRequest(msg.requestId, null); this.settleRequest(msg.requestId, null);
this.noteFailure(); if (pending.failable !== false) this.noteFailure();
}, this.encodeTimeoutMs); }, this.encodeTimeoutMs);
return; return;
} }
@@ -210,7 +224,7 @@ class StreamCaptureBackend {
* Resolves null when no frame qualifies (caller falls back) and also on * Resolves null when no frame qualifies (caller falls back) and also on
* timeout, which additionally counts toward unhealthiness. * timeout, which additionally counts toward unhealthiness.
*/ */
frameForClick({ clickPos = null, clickAt = Date.now(), strict = true, leadMs = 0 } = {}) { frameForClick({ clickPos = null, clickAt = Date.now(), strict = true, leadMs = 0, failable = true } = {}) {
if (!this.active || !this.host) return Promise.resolve(null); if (!this.active || !this.host) return Promise.resolve(null);
const displays = [...this.streams.values()].filter((s) => s.ready).map((s) => s.display); const displays = [...this.streams.values()].filter((s) => s.ready).map((s) => s.display);
const display = clickPos ? displayForDipPoint(clickPos, displays) : (displays[0] || null); const display = clickPos ? displayForDipPoint(clickPos, displays) : (displays[0] || null);
@@ -222,10 +236,15 @@ class StreamCaptureBackend {
if (clickPos && !pointInBounds(clickPos, display.bounds)) return Promise.resolve(null); if (clickPos && !pointInBounds(clickPos, display.bounds)) return Promise.resolve(null);
const requestId = this.nextRequestId++; const requestId = this.nextRequestId++;
return new Promise((resolve) => { return new Promise((resolve) => {
const pending = { resolve, display, timer: null }; // failable=false: a timeout resolves null but does NOT count toward
// unhealthiness. Interval/timed captures use this so a single slow PNG
// encode (common on software-rendered hosts with no GPU) can't trip the
// 2-strikes rule and tear down an otherwise-healthy stream — the bug
// that left Wayland recordings "stuck after two captures".
const pending = { resolve, display, timer: null, failable };
pending.timer = setTimeout(() => { pending.timer = setTimeout(() => {
this.settleRequest(requestId, null); this.settleRequest(requestId, null);
this.noteFailure(); if (failable) this.noteFailure();
}, this.ackTimeoutMs); }, this.ackTimeoutMs);
this.requests.set(requestId, pending); this.requests.set(requestId, pending);
this.hostSend({ this.hostSend({
@@ -324,11 +343,14 @@ async function createElectronHost(onEvent) {
preload: path.join(__dirname, 'capture-worker-preload.js'), preload: path.join(__dirname, 'capture-worker-preload.js'),
contextIsolation: true, contextIsolation: true,
nodeIntegration: false, nodeIntegration: false,
sandbox: true,
// The worker must keep sampling while hidden — throttling a hidden // The worker must keep sampling while hidden — throttling a hidden
// window is exactly the wrong default for a frame recorder. // window is exactly the wrong default for a frame recorder.
backgroundThrottling: false, backgroundThrottling: false,
}, },
}); });
// The worker may only display capture-worker.html; deny navigation/popups.
require('./security').installWindowSecurity(win, 'captureWorker');
const listener = (event, msg) => { const listener = (event, msg) => {
if (event.sender === win.webContents) onEvent(msg); if (event.sender === win.webContents) onEvent(msg);
}; };
+654
View File
@@ -0,0 +1,654 @@
'use strict';
const fs = require('node:fs');
const path = require('node:path');
const { execFileSync, execFile } = require('node:child_process');
const {
DEFAULT_CAPTURE_TITLES,
buildCaptureTitle,
normalizeOllamaHost,
normalizeAiPatch,
buildAiPrompt,
applyAiPatchToStep,
displayText,
normalizeWhitespace,
} = require('../core/text-intel');
const DEFAULT_TITLE_VALUES = new Set(Object.values(DEFAULT_CAPTURE_TITLES).concat(['Capture']));
const OCR_CROP = {
width: 420,
height: 220,
};
function hasBinary(name) {
try {
execFileSync('which', [name], { stdio: 'pipe' });
return true;
} catch {
return false;
}
}
function clamp(v, min, max) {
return Math.min(max, Math.max(min, v));
}
function modelLooksVisionCapable(model) {
const clean = normalizeWhitespace(model).toLowerCase();
if (!clean) return false;
return clean.includes('vision')
|| clean.includes('llava')
|| clean.includes('gemma4')
|| /(^|[^a-z0-9])qwen[23](?:\.[0-9]+)?vl([^a-z0-9]|$)/.test(clean);
}
let createWorkerImpl = null;
function loadCreateWorker() {
if (createWorkerImpl) return createWorkerImpl;
// OCR is optional at startup; lazy-load it so the app can still boot when
// the dependency has not been installed yet.
// eslint-disable-next-line global-require
({ createWorker: createWorkerImpl } = require('tesseract.js'));
return createWorkerImpl;
}
class TextIntelService {
constructor({
store,
settings,
getWindow = () => null,
dataDir,
fetchImpl = global.fetch,
screenApi = null,
}) {
this.store = store;
this.settings = settings;
this.getWindow = getWindow;
this.dataDir = dataDir;
this.fetch = fetchImpl;
this.screen = screenApi;
this.worker = null;
this.workerPromise = null;
this.workerQueue = Promise.resolve();
this.ocrDataDir = path.join(dataDir, 'ocr', 'eng');
this.modelCapabilityCache = new Map();
}
async shutdown() {
if (this.worker) {
try {
await this.worker.terminate();
} catch {
// best effort
}
this.worker = null;
this.workerPromise = null;
}
}
ensureLangData() {
const packageDir = path.dirname(require.resolve('@tesseract.js-data/eng/package.json'));
const source = path.join(packageDir, '4.0.0_best_int', 'eng.traineddata.gz');
const targetDir = this.ocrDataDir;
const target = path.join(targetDir, 'eng.traineddata.gz');
if (!fs.existsSync(target)) {
fs.mkdirSync(targetDir, { recursive: true });
fs.copyFileSync(source, target);
}
return targetDir;
}
async getWorker() {
if (this.workerPromise) return this.workerPromise;
this.workerPromise = (async () => {
const workerFactory = loadCreateWorker();
const langPath = this.ensureLangData();
const worker = await workerFactory('eng', 1, {
langPath,
});
await worker.setParameters({
preserve_interword_spaces: '1',
});
this.worker = worker;
return worker;
})();
this.workerPromise.catch(() => {
this.workerPromise = null;
});
return this.workerPromise;
}
async recognizeCrop(image, rect = null) {
const worker = await this.getWorker();
const cropped = rect ? image.crop(rect) : image;
const buffer = cropped.toPNG();
const result = await worker.recognize(buffer);
const text = String(result?.data?.text || '').trim();
return {
text,
confidence: Number.isFinite(result?.data?.confidence) ? result.data.confidence : null,
raw: result,
};
}
cropRectForPoint(frame, clickPos, { width = OCR_CROP.width, height = OCR_CROP.height } = {}) {
if (!frame || !frame.size) return null;
const bounds = frame.display.bounds || { x: 0, y: 0, width: frame.size.width, height: frame.size.height };
const scaleX = frame.size.width / bounds.width;
const scaleY = frame.size.height / bounds.height;
const point = clickPos || {
x: bounds.x + bounds.width / 2,
y: bounds.y + bounds.height / 2,
};
const centerX = (point.x - bounds.x) * scaleX;
const centerY = (point.y - bounds.y) * scaleY;
const rectW = Math.max(1, Math.round(width * scaleX));
const rectH = Math.max(1, Math.round(height * scaleY));
const rect = {
x: Math.round(centerX - rectW / 2),
y: Math.round(centerY - rectH / 2),
width: rectW,
height: rectH,
};
rect.x = clamp(rect.x, 0, Math.max(0, frame.size.width - rect.width));
rect.y = clamp(rect.y, 0, Math.max(0, frame.size.height - rect.height));
rect.width = clamp(rect.width, 1, frame.size.width);
rect.height = clamp(rect.height, 1, frame.size.height);
return rect;
}
async ocrAroundClick(frame, clickPos) {
if (!frame || !frame.image) return { text: '', confidence: null };
// Use a full-width horizontal strip at the click height. This preserves complete
// link text (e.g. "Oracle | Cloud Applications and Cloud Platform") rather than
// cropping through it when the element spans more than the 420 px default width.
const bounds = frame.display?.bounds || { x: 0, y: 0, width: frame.size.width, height: frame.size.height };
const rect = this.cropRectForPoint(frame, clickPos, {
width: bounds.width, // full display width → full image width after DPI scaling
height: 100, // ~2 lines tall, enough context without too much noise
});
try {
return await this.recognizeCrop(frame.image, rect);
} catch {
return { text: '', confidence: null };
}
}
async collectForegroundWindowContext(osPoint = null) {
try {
if (process.platform === 'win32') return this.collectWindowsWindowContext(osPoint);
if (process.platform === 'darwin') return this.collectMacWindowContext();
if (process.platform === 'linux') return this.collectLinuxWindowContext();
} catch {
// best effort only
}
return { appName: '', windowTitle: '' };
}
async collectWindowsWindowContext(osPoint = null) {
const hasPoint = osPoint && Number.isFinite(osPoint.x) && Number.isFinite(osPoint.y);
const clickX = hasPoint ? Number(osPoint.x) : 0;
const clickY = hasPoint ? Number(osPoint.y) : 0;
const script = `
$clickX = ${clickX};
$clickY = ${clickY};
$elementLabel = '';
$elementRole = '';
$elementClass = '';
$elementProcessId = 0;
$elementValue = '';
if (${hasPoint ? '$true' : '$false'}) {
try {
Add-Type -AssemblyName UIAutomationClient,UIAutomationTypes,WindowsBase | Out-Null
$point = New-Object System.Windows.Point($clickX, $clickY);
$element = [System.Windows.Automation.AutomationElement]::FromPoint($point);
if ($element) {
$current = $element.Current;
$elementLabel = $current.Name;
$elementRole = $current.LocalizedControlType;
$elementClass = $current.ClassName;
$elementProcessId = $current.ProcessId;
try {
$valPattern = [System.Windows.Automation.ValuePattern]::Pattern;
if ($element.GetSupportedPatterns() -contains $valPattern) {
$elementValue = $element.GetCurrentPattern($valPattern).Current.Value;
}
} catch { }
}
} catch { }
}
Add-Type @"
using System;
using System.Runtime.InteropServices;
using System.Text;
public static class Win32 {
[DllImport("user32.dll")] public static extern IntPtr GetForegroundWindow();
[DllImport("user32.dll", CharSet = CharSet.Unicode)]
public static extern int GetWindowText(IntPtr hWnd, StringBuilder text, int count);
[DllImport("user32.dll")] public static extern uint GetWindowThreadProcessId(IntPtr hWnd, out uint processId);
}
"@;
$hWnd = [Win32]::GetForegroundWindow();
$sb = New-Object System.Text.StringBuilder 512;
[void][Win32]::GetWindowText($hWnd, $sb, $sb.Capacity);
$pid = 0;
[void][Win32]::GetWindowThreadProcessId($hWnd, [ref]$pid);
$proc = Get-Process -Id $pid -ErrorAction SilentlyContinue | Select-Object -First 1;
$out = [ordered]@{
appName = if ($proc) { $proc.ProcessName } else { '' };
windowTitle = $sb.ToString();
elementLabel = $elementLabel;
elementRole = $elementRole;
elementClass = $elementClass;
elementValue = $elementValue;
elementProcessId = $elementProcessId;
pid = $pid;
};
$out | ConvertTo-Json -Compress;
`;
return new Promise(resolve => {
execFile('powershell.exe', ['-NoProfile', '-NonInteractive', '-ExecutionPolicy', 'Bypass', '-Command', script], {
encoding: 'utf8',
timeout: 4000,
windowsHide: true,
}, (err, stdout) => {
if (err) { resolve({}); return; }
try { resolve(JSON.parse(stdout.trim() || '{}')); }
catch { resolve({}); }
});
});
}
collectMacWindowContext() {
const script = `
set appName to ""
set windowTitle to ""
tell application "System Events"
try
set frontApp to first application process whose frontmost is true
set appName to name of frontApp
try
set windowTitle to name of front window of frontApp
end try
end try
end tell
return appName & linefeed & windowTitle
`;
const result = execFileSync('osascript', ['-e', script], {
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'pipe'],
timeout: 1200,
}).trimEnd();
const [appName = '', windowTitle = ''] = result.split(/\r?\n/);
return { appName, windowTitle };
}
collectLinuxWindowContext() {
if (!hasBinary('xprop')) return { appName: '', windowTitle: '' };
const active = execFileSync('xprop', ['-root', '_NET_ACTIVE_WINDOW'], {
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'pipe'],
timeout: 1200,
});
const activeMatch = active.match(/window id # (0x[0-9a-fA-F]+)/);
if (!activeMatch) return { appName: '', windowTitle: '' };
const winId = activeMatch[1];
const details = execFileSync('xprop', ['-id', winId, '_NET_WM_NAME', 'WM_NAME', 'WM_CLASS'], {
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'pipe'],
timeout: 1200,
});
const titleMatch = details.match(/(?:_NET_WM_NAME\(UTF8_STRING\)|WM_NAME\(STRING\)|WM_NAME\(UTF8_STRING\)) = "([^"]*)"/);
const classMatch = details.match(/WM_CLASS\(STRING\) = "([^"]*)"(?:, "([^"]*)")?/);
return {
appName: classMatch ? (classMatch[2] || classMatch[1] || '') : '',
windowTitle: titleMatch ? titleMatch[1] : '',
};
}
async buildCaptureTitle({ mode, frame, clickPos, clickMeta = null }) {
const ctx = await this.buildCaptureContext({ mode, frame, clickPos, clickMeta });
return ctx.title;
}
async buildCaptureContext({ mode, frame, clickPos, clickMeta = null }) {
const keyContext = clickMeta?.keyContext || {};
const recentTyped = keyContext.recentTyped || '';
const recentShortcut = keyContext.recentShortcut || '';
// Use window context pre-captured by the click watcher when available.
// This avoids a costly PowerShell cold-start (13 s) on every capture.
const fastContext = clickMeta?.windowContext || null;
const [metadata, ocr] = await Promise.all([
fastContext
? Promise.resolve(fastContext)
: this.collectForegroundWindowContext(clickMeta?.osPoint || null),
this.ocrAroundClick(frame, clickPos),
]);
const title = buildCaptureTitle({ mode, metadata, ocrText: ocr.text, recentTyped, recentShortcut });
return {
title,
captureMetadata: {
ocrText: ocr.text || '',
windowTitle: metadata.windowTitle || '',
appName: metadata.appName || '',
elementLabel: metadata.elementLabel || '',
elementRole: metadata.elementRole || '',
elementValue: metadata.elementValue || '',
recentTyped,
recentShortcut,
mode,
},
};
}
aiEnabled() {
return Boolean(this.settings.get('ai.enabled'));
}
aiConfig(override = null) {
const stored = this.settings.get('ai') || {};
const merged = override ? {
...stored,
...override,
ollama: {
...(stored.ollama || {}),
...(override.ollama || {}),
},
} : stored;
return {
...merged,
enabled: override && Object.prototype.hasOwnProperty.call(override, 'enabled')
? Boolean(override.enabled)
: Boolean(stored.enabled),
ollama: {
host: normalizeOllamaHost(merged.ollama?.host || ''),
model: normalizeWhitespace(merged.ollama?.model || ''),
},
};
}
async testAiConnection(override = null) {
const config = this.aiConfig(override);
if (!config.ollama.host) {
return { ok: false, reason: 'Set an Ollama host first.' };
}
const tagsUrl = new URL('/api/tags', `${config.ollama.host.replace(/\/+$/, '')}/`);
const res = await this.fetch(tagsUrl, { method: 'GET' });
if (!res.ok) {
return { ok: false, reason: `Ollama check failed (${res.status})` };
}
const data = await res.json();
const models = Array.isArray(data?.models) ? data.models.map((model) => model.name).filter(Boolean) : [];
const installed = config.ollama.model ? models.includes(config.ollama.model) : false;
const vision = installed ? await this.modelSupportsVision({
host: config.ollama.host,
model: config.ollama.model,
}) : false;
return {
ok: true,
installed,
vision,
models,
host: config.ollama.host,
model: config.ollama.model,
};
}
async modelCapabilities({ host, model }) {
const normalizedHost = normalizeOllamaHost(host);
const normalizedModel = normalizeWhitespace(model);
if (!normalizedHost || !normalizedModel) return [];
const cacheKey = `${normalizedHost}::${normalizedModel}`;
if (this.modelCapabilityCache.has(cacheKey)) {
return this.modelCapabilityCache.get(cacheKey);
}
const url = new URL('/api/show', `${normalizedHost.replace(/\/+$/, '')}/`);
let capabilities = [];
try {
const response = await this.fetch(url, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ model: normalizedModel }),
});
if (response.ok) {
const payload = await response.json();
capabilities = Array.isArray(payload?.capabilities)
? payload.capabilities.map((cap) => normalizeWhitespace(cap).toLowerCase()).filter(Boolean)
: [];
}
} catch {
capabilities = [];
}
if (!capabilities.includes('vision') && modelLooksVisionCapable(normalizedModel)) {
capabilities = [...capabilities, 'vision'];
}
this.modelCapabilityCache.set(cacheKey, capabilities);
return capabilities;
}
async modelSupportsVision({ host, model }) {
const capabilities = await this.modelCapabilities({ host, model });
return capabilities.includes('vision');
}
readStepImageBase64(guideId, stepId) {
const imagePath = this.store.stepImagePath(guideId, stepId, 'working') || this.store.stepImagePath(guideId, stepId, 'original');
if (!imagePath || !fs.existsSync(imagePath)) return '';
return fs.readFileSync(imagePath).toString('base64');
}
async callOllamaText({ host, model, prompt, systemPrompt }) {
const url = new URL('/api/chat', `${host.replace(/\/+$/, '')}/`);
const response = await this.fetch(url, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model,
stream: false,
messages: [
{ role: 'system', content: systemPrompt },
{ role: 'user', content: prompt },
],
options: { temperature: 0.4 },
}),
});
if (!response.ok) throw new Error(`Ollama request failed (${response.status})`);
const payload = await response.json();
const content = payload?.message?.content;
if (typeof content !== 'string' || !content.trim()) throw new Error('Ollama returned an empty response');
return content.trim();
}
async callOllama({ host, model, prompt, systemPrompt, images = [] }) {
const url = new URL('/api/chat', `${host.replace(/\/+$/, '')}/`);
const userMessage = { role: 'user', content: prompt };
if (Array.isArray(images) && images.length) {
userMessage.images = images;
}
const response = await this.fetch(url, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model,
stream: false,
format: 'json',
messages: [
{ role: 'system', content: systemPrompt },
userMessage,
],
options: {
temperature: 0.2,
},
}),
});
if (!response.ok) {
throw new Error(`Ollama request failed (${response.status})`);
}
const payload = await response.json();
const content = payload?.message?.content;
if (typeof content !== 'string' || !content.trim()) {
throw new Error('Ollama returned an empty response');
}
return content;
}
async generateStepPatch({
guideId,
stepId,
target = 'all',
blockId = null,
}) {
try {
const config = this.aiConfig();
if (!config.enabled) {
return { ok: false, reason: 'Enable AI in settings first.' };
}
if (!config.ollama.host || !config.ollama.model) {
return { ok: false, reason: 'Configure Ollama host and model in Settings.' };
}
const guide = this.store.getGuide(guideId);
const step = this.store.getStep(guideId, stepId);
if (!guide || !step) {
return { ok: false, reason: 'Guide or step not found.' };
}
const currentBlock = blockId
? [...(step.textBlocks || []), ...(step.codeBlocks || []), ...(step.tableBlocks || [])].find((b) => b.id === blockId) || null
: null;
if (blockId && target === 'block' && !currentBlock) {
return { ok: false, reason: 'Block not found.' };
}
const screenshotBase64 = step.image ? this.readStepImageBase64(guideId, stepId) : '';
const screenshotAttached = Boolean(screenshotBase64)
? await this.modelSupportsVision({
host: config.ollama.host,
model: config.ollama.model,
})
: false;
let captureContext = null;
// Use stored capture metadata when available (best context, from capture time).
// Fall back to re-running OCR on the stored image only when metadata is absent.
if (step.captureMetadata) {
const rawCandidate = buildCaptureTitle({
mode: step.captureMetadata.mode || 'fullscreen',
metadata: {
windowTitle: step.captureMetadata.windowTitle,
appName: step.captureMetadata.appName,
elementLabel: step.captureMetadata.elementLabel,
elementRole: step.captureMetadata.elementRole,
elementValue: step.captureMetadata.elementValue,
},
ocrText: step.captureMetadata.ocrText,
recentTyped: step.captureMetadata.recentTyped,
recentShortcut: step.captureMetadata.recentShortcut,
});
captureContext = {
...step.captureMetadata,
// Don't suggest a generic fallback title — leave it blank so AI generates from context.
titleCandidate: DEFAULT_TITLE_VALUES.has(rawCandidate) ? '' : rawCandidate,
};
} else if (step.image) {
const imagePath = this.store.stepImagePath(guideId, stepId, 'working') || this.store.stepImagePath(guideId, stepId, 'original');
if (imagePath && fs.existsSync(imagePath)) {
const { nativeImage } = require('electron');
const image = nativeImage.createFromPath(imagePath);
if (!image.isEmpty()) {
const clickPoint = this.clickPointFromStep(step, image);
const [metadata, ocr] = await Promise.all([
this.collectForegroundWindowContext(),
this.ocrAroundClick({ image, size: image.getSize(), display: { bounds: { x: 0, y: 0, width: image.getSize().width, height: image.getSize().height } } }, clickPoint),
]);
const rawCandidate2 = buildCaptureTitle({
mode: step.kind === 'image' ? 'fullscreen' : 'window',
metadata,
ocrText: ocr.text,
});
captureContext = {
...metadata,
ocrText: ocr.text,
titleCandidate: DEFAULT_TITLE_VALUES.has(rawCandidate2) ? '' : rawCandidate2,
mode: step.kind === 'image' ? 'fullscreen' : 'content',
};
}
}
}
const { systemPrompt, prompt } = buildAiPrompt({
target,
guide,
step,
captureContext,
block: currentBlock,
screenshotAttached,
});
const raw = await this.callOllama({
host: config.ollama.host,
model: config.ollama.model,
prompt,
systemPrompt,
images: screenshotAttached ? [screenshotBase64] : [],
});
const patch = normalizeAiPatch(raw);
const updated = applyAiPatchToStep(step, patch, { target, blockId });
const saved = this.store.saveStep(guideId, updated);
return { ok: true, step: saved, patch };
} catch (err) {
return { ok: false, reason: err && err.message ? err.message : 'AI generation failed.' };
}
}
async rewriteText({ text, guideTitle = '', stepTitle = '' }) {
try {
const config = this.aiConfig();
if (!config.enabled) return { ok: false, reason: 'Enable AI in settings first.' };
if (!config.ollama.host || !config.ollama.model) {
return { ok: false, reason: 'Configure Ollama host and model in Settings.' };
}
const trimmed = normalizeWhitespace(text);
if (!trimmed) return { ok: false, reason: 'No text to rewrite.' };
const contextHint = [
guideTitle ? `Guide: ${guideTitle}` : '',
stepTitle ? `Step: ${stepTitle}` : '',
].filter(Boolean).join('\n');
const prompt = [
contextHint,
contextHint ? '' : null,
'Rewrite the following text to sound professional and clear as step-by-step documentation.',
'Keep it concise. Do not add extra information. Return only the rewritten text.',
'',
trimmed,
].filter((l) => l !== null).join('\n');
const result = await this.callOllamaText({
host: config.ollama.host,
model: config.ollama.model,
prompt,
systemPrompt: 'You are a documentation editor. Return only the improved text, nothing else.',
});
return { ok: true, text: result };
} catch (err) {
return { ok: false, reason: err?.message || 'Rewrite failed.' };
}
}
clickPointFromStep(step, image = null) {
const marker = (step.annotations || []).find((ann) => ann.type === 'oval' && Number.isFinite(ann.x) && Number.isFinite(ann.y) && Number.isFinite(ann.w) && Number.isFinite(ann.h));
if (!marker) return null;
const size = image ? image.getSize() : step.image?.size || { width: 0, height: 0 };
if (!size.width || !size.height) return null;
return {
x: Math.round((marker.x + marker.w / 2) * size.width),
y: Math.round((marker.y + marker.h / 2) * size.height),
};
}
}
module.exports = { TextIntelService };
-218
View File
@@ -1,218 +0,0 @@
{
"format": "stepforge-artifacts-manifest",
"version": 1,
"generatedAt": "2026-06-11T21:54:13.294Z",
"packageVersion": "0.1.0",
"files": [
{
"kind": "artifact",
"path": "artifacts/stepforge_0.1.0_amd64.deb",
"size": 103691640,
"sha256": "320faa345f5997905fdc831c045dbe243490a7b4328680d105934f2aec1b4ffb"
},
{
"kind": "artifact",
"path": "artifacts/stepforge_0.1.0_linux-x64.tar.gz",
"size": 139378628,
"sha256": "32971595d4df40b429cb41ba644e57b84351ec1239b4bbe8d5b82fe3b01a4cc9"
},
{
"kind": "sample",
"path": "../examples/sample-data/library/guides/guide-sample-reset-password/guide.json",
"size": 843,
"sha256": "0d7760246d96a85a4d79c15c1ab1a7e481bd79e1d8ff29d177ac6d35ffe6cde6"
},
{
"kind": "sample",
"path": "../examples/sample-data/library/guides/guide-sample-reset-password/steps/step-sample-01-open-users/original.png",
"size": 13643,
"sha256": "09e12f935511bb6fabb5637501aa7743516b96d990adfab62ccfa311a7b60606"
},
{
"kind": "sample",
"path": "../examples/sample-data/library/guides/guide-sample-reset-password/steps/step-sample-01-open-users/step.json",
"size": 1593,
"sha256": "36deb8a8b51952a668e871a03b1c7fba2aaf72b1722fd2d88c7e5d5cbbd8da91"
},
{
"kind": "sample",
"path": "../examples/sample-data/library/guides/guide-sample-reset-password/steps/step-sample-01-open-users/working.png",
"size": 13643,
"sha256": "09e12f935511bb6fabb5637501aa7743516b96d990adfab62ccfa311a7b60606"
},
{
"kind": "sample",
"path": "../examples/sample-data/library/guides/guide-sample-reset-password/steps/step-sample-02-enable-policy/original.png",
"size": 14031,
"sha256": "b5e93a0ee74e2bdbbdf0871e901726dfbdc8b45dd648c959743520f92b02e7a2"
},
{
"kind": "sample",
"path": "../examples/sample-data/library/guides/guide-sample-reset-password/steps/step-sample-02-enable-policy/step.json",
"size": 1873,
"sha256": "4b0b3b74f851d034a2769c69df2e9d0428373ae15009fb3e2100afb5c10f7ebb"
},
{
"kind": "sample",
"path": "../examples/sample-data/library/guides/guide-sample-reset-password/steps/step-sample-02-enable-policy/working.png",
"size": 14031,
"sha256": "b5e93a0ee74e2bdbbdf0871e901726dfbdc8b45dd648c959743520f92b02e7a2"
},
{
"kind": "sample",
"path": "../examples/sample-data/library/guides/guide-sample-reset-password/steps/step-sample-02a-permission-prompt/step.json",
"size": 763,
"sha256": "e8539addbe730e3a1baaa6898bf9779d2ce80564aecb3f196619d8c7ff67cbae"
},
{
"kind": "sample",
"path": "../examples/sample-data/library/guides/guide-sample-reset-password/steps/step-sample-03-review-confirmation/original.png",
"size": 13602,
"sha256": "c96eedffdc5fd2eb9b63942cc00f1c8a91d01a0c5c2316c1d12d750b9b49e3d0"
},
{
"kind": "sample",
"path": "../examples/sample-data/library/guides/guide-sample-reset-password/steps/step-sample-03-review-confirmation/step.json",
"size": 1968,
"sha256": "6f1cbb9d2ed32b89dec3d221822c6e973dbba42c11e93c2b35499179e8481532"
},
{
"kind": "sample",
"path": "../examples/sample-data/library/guides/guide-sample-reset-password/steps/step-sample-03-review-confirmation/working.png",
"size": 13602,
"sha256": "c96eedffdc5fd2eb9b63942cc00f1c8a91d01a0c5c2316c1d12d750b9b49e3d0"
},
{
"kind": "sample",
"path": "../examples/sample-data/library/guides/guide-sample-reset-password/steps/step-sample-04-legacy-note/step.json",
"size": 506,
"sha256": "c6f78405f86f4183612f5865820b2454471cbc47975e9c15e055e7bf742b1eba"
},
{
"kind": "sample",
"path": "../examples/sample-data/library/guides/guide-sample-reset-password/steps/step-sample-05-deprecated-flow/step.json",
"size": 536,
"sha256": "709887c0a5debddc851216920ee3f6f2766e986059b21ac399e2533611ab5aed"
},
{
"kind": "sample",
"path": "../examples/sample-exports/docx/reset-a-password-in-admin-portal.docx",
"size": 110463,
"sha256": "be9e2b550b732fc6cd4a171b72749b49fe67730ee9969f69c23175688b0bea14"
},
{
"kind": "sample",
"path": "../examples/sample-exports/gif/reset-a-password-in-admin-portal.gif",
"size": 32090,
"sha256": "70a789c6ce1aa6154c65d0ee88a286d888fdfd9fd5c3045a14242e13df5ca263"
},
{
"kind": "sample",
"path": "../examples/sample-exports/html-rich/reset-a-password-in-admin-portal-rich.html",
"size": 149884,
"sha256": "70695d55c37d69abb191e65675f726eaa7aea6686a15a2e7d387195441e3ca8c"
},
{
"kind": "sample",
"path": "../examples/sample-exports/html-simple/reset-a-password-in-admin-portal.html",
"size": 146646,
"sha256": "860c774b9b8bfd821b22deadc21f8021dc66bc9cacf43702eecbf55e944c6a9a"
},
{
"kind": "sample",
"path": "../examples/sample-exports/image-bundle/reset-a-password-in-admin-portal-bundle.json",
"size": 779,
"sha256": "0108a63331925ea1fdd03326dd37dce1af3642a02d167e583d0d67b50fce27fa"
},
{
"kind": "sample",
"path": "../examples/sample-exports/image-bundle/steps-reset-a-password-in-admin-portal/001-open-admin-portal-users.png",
"size": 31424,
"sha256": "892a3174e9876ebfab4c879d6d579cf8ad6eba2f299ad95feff33adaf205e042"
},
{
"kind": "sample",
"path": "../examples/sample-exports/image-bundle/steps-reset-a-password-in-admin-portal/002-enable-the-reset-policy.png",
"size": 38991,
"sha256": "f1c79fb2baa1ef41dd85b04be59cc796f3822858654612044a22e27f20d0696a"
},
{
"kind": "sample",
"path": "../examples/sample-exports/image-bundle/steps-reset-a-password-in-admin-portal/004-review-the-confirmation.png",
"size": 37274,
"sha256": "32c5e15f8a04d6af01b7d139cff732872d5ce7e3e519be6ece99cca4167856ed"
},
{
"kind": "sample",
"path": "../examples/sample-exports/json/reset-a-password-in-admin-portal.json",
"size": 6740,
"sha256": "4a27376e75c3bb33fa8e1c4117c3314a04ec4766d67a9b5c729180d116d384d1"
},
{
"kind": "sample",
"path": "../examples/sample-exports/json/steps-reset-a-password-in-admin-portal/001-open-admin-portal-users.png",
"size": 31424,
"sha256": "892a3174e9876ebfab4c879d6d579cf8ad6eba2f299ad95feff33adaf205e042"
},
{
"kind": "sample",
"path": "../examples/sample-exports/json/steps-reset-a-password-in-admin-portal/002-enable-the-reset-policy.png",
"size": 38991,
"sha256": "f1c79fb2baa1ef41dd85b04be59cc796f3822858654612044a22e27f20d0696a"
},
{
"kind": "sample",
"path": "../examples/sample-exports/json/steps-reset-a-password-in-admin-portal/004-review-the-confirmation.png",
"size": 37274,
"sha256": "32c5e15f8a04d6af01b7d139cff732872d5ce7e3e519be6ece99cca4167856ed"
},
{
"kind": "sample",
"path": "../examples/sample-exports/markdown/reset-a-password-in-admin-portal.md",
"size": 1186,
"sha256": "16bbd2eb8850d8a55914abe9ece4d2aee465820fcd6c5ffde8925f37dd2b115b"
},
{
"kind": "sample",
"path": "../examples/sample-exports/markdown/steps-reset-a-password-in-admin-portal/001-open-admin-portal-users.png",
"size": 31424,
"sha256": "892a3174e9876ebfab4c879d6d579cf8ad6eba2f299ad95feff33adaf205e042"
},
{
"kind": "sample",
"path": "../examples/sample-exports/markdown/steps-reset-a-password-in-admin-portal/002-enable-the-reset-policy.png",
"size": 38991,
"sha256": "f1c79fb2baa1ef41dd85b04be59cc796f3822858654612044a22e27f20d0696a"
},
{
"kind": "sample",
"path": "../examples/sample-exports/markdown/steps-reset-a-password-in-admin-portal/004-review-the-confirmation.png",
"size": 37274,
"sha256": "32c5e15f8a04d6af01b7d139cff732872d5ce7e3e519be6ece99cca4167856ed"
},
{
"kind": "sample",
"path": "../examples/sample-exports/pdf/reset-a-password-in-admin-portal.pdf",
"size": 103120,
"sha256": "58a952a2f95653a91d4e662e4bc2ddcb0177de33e51a8f9e454cdf158ba3a795"
},
{
"kind": "sample",
"path": "../examples/sample-exports/pptx/reset-a-password-in-admin-portal.pptx",
"size": 117643,
"sha256": "4ce1e82903b726c549e53235e3fd4c70cf647123ccd823052030e2e1b9449864"
},
{
"kind": "sample",
"path": "../examples/sample-guide.sfgz",
"size": 88427,
"sha256": "313b88f48e53e5ad7fb4e0a8189700ba0f6be642ec2311b1a2fe7ea4f3dd0481"
},
{
"kind": "sample",
"path": "../examples/sample-manifest.json",
"size": 1163,
"sha256": "cb2920e7500758074f1f54867db1ed187d3304c8badbceb06fd611057bd6fe5d"
}
]
}
-43
View File
@@ -1,43 +0,0 @@
# StepForge Build Report
Version: 0.1.0
Generated: 2026-06-11T21:54:13.292Z
Host: linux x64 (node v20.20.2)
## Outputs
- Portable tarball: artifacts/stepforge_0.1.0_linux-x64.tar.gz
- Debian package: artifacts/stepforge_0.1.0_amd64.deb
- Sample guide archive: ../examples/sample-guide.sfgz
- Sample exports (9 formats): see examples/sample-exports/
- Full artifact list with sha256 checksums: artifacts_manifest.json
## Packaging tool availability
| Tool | Status |
|---|---|
| dpkg-deb (Linux .deb) | available |
| rpmbuild (Linux .rpm) | **missing** |
| appimagetool (Linux AppImage) | **missing** |
| makensis (Windows installer .exe) | **missing** |
| wixl / WiX (Windows .msi) | **missing** |
Fallback policy: when a packaging tool is missing the build still produces
the runnable app (portable tarball with launcher) plus whatever package
formats the available tools allow. Windows artifacts are produced by
`npm run package:windows` (electron-builder, portable .exe); .msi/.rpm/
AppImage require the tools listed above and are skipped on this host.
## Offline guarantee
- The shipped app opens no sockets: no telemetry, update checks, license
checks, cloud sync, or remote AI. See docs/SECURITY.md.
- All exporters (PNG/GIF/PDF/DOCX/PPTX/ZIP) are implemented in-repo with
Node built-ins; Electron is the only third-party dependency
(dev-time fetch recorded in build/agent_audit.md).
## Verification
- `bash tests/run_test.sh` runs the workflow suites (node --test), a
startup smoke test of the Electron launcher, the sample-artifact
pipeline, and this release build.
+110
View File
@@ -0,0 +1,110 @@
!include LogicLib.nsh
!include nsDialogs.nsh
!ifndef BUILD_UNINSTALLER
Var StepForgeDesktopShortcutCheckbox
Var StepForgeDesktopShortcutState
; Assisted installer page for the desktop shortcut choice.
!macro customInit
StrCpy $StepForgeDesktopShortcutState "true"
${If} ${isNoDesktopShortcut}
StrCpy $StepForgeDesktopShortcutState "false"
${EndIf}
!macroend
!macro customHeader
Function StepForgeDesktopShortcutPagePre
!insertmacro MUI_PAGE_FUNCTION_CUSTOM PRE
!insertmacro MUI_HEADER_TEXT "Desktop Icon" "Choose whether StepForge creates a desktop icon."
nsDialogs::Create 1018
Pop $0
${If} $0 == error
Abort
${EndIf}
${NSD_CreateLabel} 0u 0u 280u 24u "StepForge can create a desktop icon for quick access from the desktop."
Pop $0
${NSD_CreateCheckbox} 0u 34u 280u 12u "Create a desktop icon"
Pop $StepForgeDesktopShortcutCheckbox
StrCpy $StepForgeDesktopShortcutState "true"
${If} ${isNoDesktopShortcut}
StrCpy $StepForgeDesktopShortcutState "false"
EnableWindow $StepForgeDesktopShortcutCheckbox 0
${Else}
ReadRegStr $0 SHELL_CONTEXT "${INSTALL_REGISTRY_KEY}" CreateDesktopShortcut
${If} $0 == "false"
StrCpy $StepForgeDesktopShortcutState "false"
${Else}
ReadRegStr $1 SHELL_CONTEXT "${INSTALL_REGISTRY_KEY}" ShortcutName
${If} $1 == ""
StrCpy $1 "${SHORTCUT_NAME}"
${EndIf}
${If} ${FileExists} "$DESKTOP\$1.lnk"
StrCpy $StepForgeDesktopShortcutState "true"
${ElseIf} ${FileExists} "$INSTDIR\${APP_EXECUTABLE_FILENAME}"
StrCpy $StepForgeDesktopShortcutState "false"
${EndIf}
${EndIf}
${EndIf}
${If} $StepForgeDesktopShortcutState == "false"
SendMessage $StepForgeDesktopShortcutCheckbox ${BM_SETCHECK} ${BST_UNCHECKED} 0
${Else}
SendMessage $StepForgeDesktopShortcutCheckbox ${BM_SETCHECK} ${BST_CHECKED} 0
${EndIf}
!insertmacro MUI_PAGE_FUNCTION_CUSTOM SHOW
nsDialogs::Show
FunctionEnd
Function StepForgeDesktopShortcutPageLeave
!insertmacro MUI_PAGE_FUNCTION_CUSTOM LEAVE
SendMessage $StepForgeDesktopShortcutCheckbox ${BM_GETCHECK} 0 0 $StepForgeDesktopShortcutState
${If} $StepForgeDesktopShortcutState == ${BST_UNCHECKED}
StrCpy $StepForgeDesktopShortcutState "false"
${Else}
StrCpy $StepForgeDesktopShortcutState "true"
${EndIf}
FunctionEnd
!macroend
!macro customPageAfterChangeDir
!insertmacro MUI_PAGE_INIT
PageEx custom
PageCallbacks StepForgeDesktopShortcutPagePre StepForgeDesktopShortcutPageLeave
Caption " "
PageExEnd
!macroend
!macro customInstall
; Reconcile the desktop shortcut after the default installer logic runs.
${If} ${isNoDesktopShortcut}
StrCpy $StepForgeDesktopShortcutState "false"
${EndIf}
WriteRegStr SHELL_CONTEXT "${INSTALL_REGISTRY_KEY}" CreateDesktopShortcut "$StepForgeDesktopShortcutState"
${If} $StepForgeDesktopShortcutState == "false"
Delete "$newDesktopLink"
Delete "$oldDesktopLink"
${Else}
${IfNot} ${FileExists} "$newDesktopLink"
${If} ${FileExists} "$oldDesktopLink"
Rename "$oldDesktopLink" "$newDesktopLink"
${Else}
CreateShortCut "$newDesktopLink" "$appExe" "" "$appExe" 0 "" "" "${APP_DESCRIPTION}"
${EndIf}
ClearErrors
WinShell::SetLnkAUMI "$newDesktopLink" "${APP_ID}"
${EndIf}
${EndIf}
System::Call 'shell32::SHChangeNotify(i 0x08000000, i 0, i 0, i 0)'
!macroend
!endif
+3
View File
@@ -86,6 +86,9 @@ function createStep(fields = {}) {
codeBlocks: (fields.codeBlocks || []).map((cb) => normalizeCodeBlock(cb, takeOrder(cb))), codeBlocks: (fields.codeBlocks || []).map((cb) => normalizeCodeBlock(cb, takeOrder(cb))),
tableBlocks: (fields.tableBlocks || []).map((tb) => normalizeTableBlock(tb, takeOrder(tb))), tableBlocks: (fields.tableBlocks || []).map((tb) => normalizeTableBlock(tb, takeOrder(tb))),
links: fields.links || [], // { id, label, targetStepId } links: fields.links || [], // { id, label, targetStepId }
captureMetadata: (fields.captureMetadata && typeof fields.captureMetadata === 'object' && !Array.isArray(fields.captureMetadata))
? { ...fields.captureMetadata }
: null,
}; };
} }
+10
View File
@@ -18,6 +18,9 @@ const DEFAULT_SETTINGS = {
hotkeyPauseResume: 'CommandOrControl+Shift+2', hotkeyPauseResume: 'CommandOrControl+Shift+2',
captureOutsideClicks: true, captureOutsideClicks: true,
confirmSimpleCapture: false, confirmSimpleCapture: false,
// Fallback trigger when click capture is unavailable: keep the old timer
// fallback by default, but let users switch to hotkey-only recordings.
fallbackTrigger: 'interval', // interval | hotkey
// Leading-edge click debounce (ms): clicks of the same button closer // Leading-edge click debounce (ms): clicks of the same button closer
// together than this collapse into one step, so accidental fast/double // together than this collapse into one step, so accidental fast/double
// clicks don't each become a step. Clicks spaced further apart always // clicks don't each become a step. Clicks spaced further apart always
@@ -47,6 +50,13 @@ const DEFAULT_SETTINGS = {
focusedViewDefaultForNewSteps: false, focusedViewDefaultForNewSteps: false,
autoTitleTemplate: '[[Mode]] capture [[Time]]', autoTitleTemplate: '[[Mode]] capture [[Time]]',
}, },
ai: {
enabled: false,
ollama: {
host: 'http://127.0.0.1:11434',
model: 'llama3.2:1b',
},
},
exports: { exports: {
previewStepCount: 3, previewStepCount: 3,
openFolderAfterExport: true, openFolderAfterExport: true,
+857
View File
@@ -0,0 +1,857 @@
'use strict';
const { deepClone, htmlToText, escapeHtml } = require('./util');
const { sanitizeHtml } = require('./sanitize');
const {
TEXTBLOCK_LEVELS,
TEXTBLOCK_POSITIONS,
normalizeTextBlock,
normalizeCodeBlock,
normalizeTableBlock,
} = require('./schema');
const DEFAULT_CAPTURE_TITLES = {
fullscreen: 'Screen capture',
window: 'Window capture',
region: 'Region capture',
};
const AI_LEVEL_ALIASES = new Map([
['note', 'info'],
['info', 'info'],
['tip', 'success'],
['success', 'success'],
['warning', 'warn'],
['warn', 'warn'],
['important', 'error'],
['error', 'error'],
]);
const GENERIC_OCR_PHRASES = new Set([
'button',
'click',
'double click',
'menu',
'item',
'field',
'text field',
'search',
'submit',
'cancel',
'ok',
'open',
'select',
'enter',
'type',
]);
// Generic OS/browser chrome titles that tell us nothing about what the user did.
const GENERIC_WINDOW_TITLES = new Set([
'new tab', 'new window', 'new incognito window', 'new incognito tab',
'new document', 'untitled', 'blank page', 'home page', 'homepage',
'start page', 'speed dial', 'loading', 'loading…', 'loading...',
]);
const BROWSER_NAME_PHRASES = new Set([
'google chrome',
'chrome',
'chromium',
'microsoft edge',
'edge',
'brave',
'firefox',
'safari',
'opera',
'vivaldi',
]);
// Known search engine page title suffixes (what appears after the query in the window title).
const SEARCH_ENGINE_PAGE_NAMES = new Set([
'google search',
'google',
'bing',
'duckduckgo',
'yahoo search',
'yahoo',
'startpage',
'ecosia',
'brave search',
]);
// Common keyboard shortcuts → short action descriptions used as step titles.
const SHORTCUT_TITLES = {
'Ctrl+T': 'Open new tab',
'Ctrl+N': 'Open new window',
'Ctrl+W': 'Close tab',
'Ctrl+Shift+T': 'Reopen closed tab',
'Ctrl+Shift+N': 'Open incognito window',
'Ctrl+S': 'Save',
'Ctrl+Shift+S': 'Save as',
'Ctrl+Z': 'Undo',
'Ctrl+Y': 'Redo',
'Ctrl+Shift+Z': 'Redo',
'Ctrl+C': 'Copy selection',
'Ctrl+V': 'Paste',
'Ctrl+X': 'Cut selection',
'Ctrl+A': 'Select all',
'Ctrl+F': 'Open Find',
'Ctrl+H': 'Open Find and Replace',
'Ctrl+R': 'Reload page',
'Ctrl+Shift+R': 'Hard reload page',
'Ctrl+L': 'Focus address bar',
'Ctrl+D': 'Bookmark page',
'Ctrl+Tab': 'Switch to next tab',
'Ctrl+Shift+Tab': 'Switch to previous tab',
'Ctrl+Plus': 'Zoom in',
'Ctrl+Minus': 'Zoom out',
'Ctrl+0': 'Reset zoom',
'Ctrl+P': 'Print',
'Ctrl+O': 'Open file',
'Ctrl+E': 'Focus search bar',
'Ctrl+K': 'Focus search bar',
'Ctrl+G': 'Go to line',
'Ctrl+B': 'Toggle sidebar',
'Ctrl+Shift+P': 'Open command palette',
'Ctrl+Shift+E': 'Show file explorer',
'Ctrl+Shift+G': 'Show source control',
'Ctrl+Shift+D': 'Show debug panel',
'Ctrl+Shift+X': 'Show extensions',
'Alt+F4': 'Close window',
'Alt+Left': 'Go back',
'Alt+Right': 'Go forward',
'Alt+Tab': 'Switch application',
'F2': 'Rename',
'F3': 'Find next',
'F4': 'Open address bar',
'F5': 'Reload page',
'F11': 'Toggle fullscreen',
'F12': 'Open developer tools',
};
// Process name → human-readable display name (used to append "in Chrome" etc. to titles).
const APP_DISPLAY_NAMES = {
chrome: 'Chrome',
msedge: 'Edge',
firefox: 'Firefox',
safari: 'Safari',
opera: 'Opera',
brave: 'Brave',
vivaldi: 'Vivaldi',
code: 'VS Code',
cursor: 'Cursor',
'sublime_text': 'Sublime Text',
atom: 'Atom',
notepad: 'Notepad',
'notepad++': 'Notepad++',
winword: 'Word',
excel: 'Excel',
powerpnt: 'PowerPoint',
outlook: 'Outlook',
teams: 'Teams',
slack: 'Slack',
discord: 'Discord',
zoom: 'Zoom',
figma: 'Figma',
postman: 'Postman',
insomnia: 'Insomnia',
notion: 'Notion',
obsidian: 'Obsidian',
spotify: 'Spotify',
terminal: 'Terminal',
cmd: 'Command Prompt',
powershell: 'PowerShell',
windowsterminal: 'Windows Terminal',
wt: 'Windows Terminal',
iterm2: 'iTerm',
wezterm: 'WezTerm',
alacritty: 'Alacritty',
kitty: 'Kitty',
'gnome-terminal': 'Terminal',
konsole: 'Konsole',
xterm: 'Terminal',
xfce4terminal: 'Terminal',
bash: 'Terminal',
zsh: 'Terminal',
fish: 'Terminal',
finder: 'Finder',
explorer: 'File Explorer',
'files-uwp': 'File Explorer',
steam: 'Steam',
'steamwebhelper': 'Steam',
};
function cleanAppName(rawName) {
if (!rawName) return '';
const key = normalizeWhitespace(rawName).toLowerCase().replace(/\.exe$/i, '');
return APP_DISPLAY_NAMES[key] || sentenceCase(rawName.replace(/\.exe$/i, ''));
}
function qualifyTitleWithApp(title, appName) {
const app = cleanAppName(appName);
if (!app) return title;
if (new RegExp(`\\b${app.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}\\b`, 'i').test(title)) return title;
return `${title} in ${app}`;
}
const ACTION_PREFIXES = [
'click',
'select',
'open',
'choose',
'enter',
'type',
'search',
'switch to',
'go to',
'navigate to',
'toggle',
'turn on',
'turn off',
'enable',
'disable',
'pick',
'focus',
'launch',
'activate',
];
function normalizeWhitespace(text) {
return String(text == null ? '' : text)
.replace(/\s+/g, ' ')
.trim();
}
function titleCaseWord(word) {
if (!word) return word;
if (/^[A-Z0-9]{2,}$/.test(word)) return word;
if (/^\d+$/.test(word)) return word;
return word[0].toUpperCase() + word.slice(1).toLowerCase();
}
function displayText(text) {
const clean = normalizeWhitespace(text)
.replace(/^[\s"'`([{<]+|[\s"'`)}\]>.,;:!?]+$/g, '')
.trim();
if (!clean) return '';
if (clean === clean.toUpperCase()) {
return clean.split(/\s+/).map(titleCaseWord).join(' ');
}
return clean.replace(/\s+/g, ' ');
}
function sentenceCase(text) {
const clean = displayText(text);
if (!clean) return '';
return clean.charAt(0).toUpperCase() + clean.slice(1);
}
function isPathOrUrlLike(text) {
return /^(?:https?:\/\/|file:\/\/|about:blank|chrome:\/\/|edge:\/\/|moz-extension:\/\/|view-source:|localhost(?:[:/]|$)|www\.)/i.test(text) ||
/[A-Za-z]:\\/.test(text) ||
/\/(?:[^/\s]+\/){2,}/.test(text) ||
/\\/.test(text);
}
function isBrowserNoise(text) {
const clean = normalizeWhitespace(text).toLowerCase();
if (!clean) return true;
if (BROWSER_NAME_PHRASES.has(clean)) return true;
if (isPathOrUrlLike(clean)) return true;
let foundBrowserName = false;
for (const name of BROWSER_NAME_PHRASES) {
if (clean.includes(name)) {
foundBrowserName = true;
break;
}
}
return foundBrowserName && /[\s|•·*]{2,}|[-–—]|\/|\\/.test(clean);
}
function isUsefulTitleCandidate(text, { source = 'ocr' } = {}) {
const clean = displayText(text);
if (!clean) return false;
const lower = clean.toLowerCase();
if (GENERIC_OCR_PHRASES.has(lower)) return false;
if (BROWSER_NAME_PHRASES.has(lower)) return false;
if (isPathOrUrlLike(clean)) return false;
if ((source === 'window' || source === 'app') && isBrowserNoise(clean)) return false;
if (source === 'window' && GENERIC_WINDOW_TITLES.has(lower)) return false;
if (/^[\p{P}\p{S}0-9]+$/u.test(clean)) return false;
return true;
}
function splitTitleFragments(text) {
const clean = normalizeWhitespace(text);
if (!clean) return [];
return clean
.split(/\s*(?:\*\*+|[|•·]+|::|\/+|\\+|\s[-–—]\s|\s{2,})\s*/g)
.map((part) => displayText(part))
.filter(Boolean);
}
function candidateWords(text) {
const clean = normalizeWhitespace(text);
if (!clean) return [];
// Exclude standalone punctuation tokens (e.g. "|" in "Oracle | Cloud...") from word count.
return clean.split(/\s+/).filter((w) => /[a-zA-Z0-9]/.test(w));
}
// Remove trailing "- Google Chrome", "| Firefox", etc. from a window title.
// When appName is supplied, also strips the specific app's display name suffix:
// "Document1 - Word" → "Document1" when appName is "winword".
function stripBrowserNameSuffix(text, appName) {
let clean = normalizeWhitespace(text);
// Always strip known browser names first.
for (const name of BROWSER_NAME_PHRASES) {
const escaped = name.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
clean = clean.replace(new RegExp(`\\s*[-–—·|•]\\s*${escaped}\\s*$`, 'i'), '').trim();
}
// Also strip the specific app's display name when provided.
if (appName) {
const display = cleanAppName(appName);
const raw = normalizeWhitespace(appName).replace(/\.exe$/i, '');
for (const name of [display, raw].filter(Boolean)) {
const escaped = name.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
clean = clean.replace(new RegExp(`\\s*[-–—·|•]\\s*${escaped}\\s*$`, 'i'), '').trim();
}
}
return clean;
}
// Detect "[query] - Google Search" or "[query] - Bing" patterns in a (already-stripped) page title.
// Returns the query word(s) if found, otherwise ''.
function extractSearchQuery(pageTitle) {
const frags = splitTitleFragments(pageTitle);
if (frags.length < 2) return '';
const last = frags[frags.length - 1].toLowerCase();
if (SEARCH_ENGINE_PAGE_NAMES.has(last)) {
const query = frags[0];
if (query && isUsefulTitleCandidate(query, { source: 'ocr' })) return query;
}
return '';
}
function scoreCandidate(text, { source = 'ocr' } = {}) {
const clean = displayText(text);
if (!clean) return -Infinity;
const words = candidateWords(clean);
if (!words.length) return -Infinity;
let score = 0;
score += source === 'ocr' ? 140 : source === 'element' ? 95 : source === 'window' ? 35 : source === 'app' ? 25 : 90;
score += Math.min(words.length, 5) * 10;
score -= Math.max(0, words.length - 5) * 11;
score -= Math.max(0, clean.length - 42) * 0.8;
if (GENERIC_OCR_PHRASES.has(clean.toLowerCase())) score -= 50;
if (BROWSER_NAME_PHRASES.has(clean.toLowerCase())) score -= 80;
if (isBrowserNoise(clean)) score -= 60;
if (clean.length <= 24) score += 10;
if (/^(click|select|open|choose|enter|type|search|switch to|go to|navigate to|toggle|turn on|turn off|enable|disable|pick|focus|launch|activate)\b/i.test(clean)) score += 12;
if (/^[\p{P}\p{S}0-9]+$/u.test(clean)) score -= 100;
return score;
}
function pickBestOcrPhrase(ocrText) {
const text = normalizeWhitespace(ocrText);
if (!text) return '';
let best = '';
let bestScore = -Infinity;
for (const rawLine of text.split(/\n+/)) {
const line = normalizeWhitespace(rawLine);
if (!line) continue;
// For short lines (link text, button labels) try the FULL line first before splitting.
// This preserves "Oracle | Cloud Applications and Cloud Platform" instead of splitting on |.
// Full-line bonus (+35) nudges it ahead of its own fragments.
const candidates = line.length <= 80
? [[line, 35], ...splitTitleFragments(line).map((f) => [f, 0])]
: splitTitleFragments(line).map((f) => [f, 0]);
for (const [part, bonus] of candidates) {
if (!isUsefulTitleCandidate(part, { source: 'ocr' })) continue;
const score = scoreCandidate(part, { source: 'ocr' }) + bonus;
if (score > bestScore) {
best = part;
bestScore = score;
}
}
}
return best;
}
function isShortUiLabel(text) {
const words = candidateWords(text);
return words.length > 0 && words.length <= 2 && text.length <= 24;
}
function isDirectiveTitle(text) {
const clean = displayText(text);
if (!clean) return false;
const lower = clean.toLowerCase();
return ACTION_PREFIXES.some((prefix) => lower.startsWith(prefix));
}
function verbForElementRole(role) {
const clean = normalizeWhitespace(role).toLowerCase();
if (!clean) return null;
if (/(tab|menu item|menuitem|option|list item|tree item|radio button|dropdown list|combo box option|hyperlink|link)/.test(clean)) {
return 'Select';
}
if (/(search box|searchbox|search field|search bar|search input)/.test(clean)) {
return 'Search for';
}
if (/(button|check box|checkbox|toggle button|switch|item|command)/.test(clean)) {
return 'Click';
}
if (/(text field|edit|combo box|textbox|text box|input|field)/.test(clean)) {
return 'Click';
}
return null;
}
function formatCaptureTitle(text, { source = 'ocr', metadata = {} } = {}) {
const clean = displayText(text);
if (!clean) return '';
if (isDirectiveTitle(clean)) {
return sentenceCase(clean);
}
const roleVerb = (source === 'ocr' || source === 'element') ? verbForElementRole(metadata.elementRole) : null;
if (roleVerb) {
return `${roleVerb} ${sentenceCase(clean)}`;
}
if (source === 'window' || source === 'app') {
return `Open ${sentenceCase(clean)}`;
}
if (source === 'ocr' || source === 'element') {
return isShortUiLabel(clean) ? `Click ${sentenceCase(clean)}` : sentenceCase(clean);
}
return sentenceCase(clean);
}
function pickBestTitleFragment(text, { source = 'window', metadata = {} } = {}) {
const fragments = splitTitleFragments(text).filter((line) => isUsefulTitleCandidate(line, { source }));
if (!fragments.length) return '';
let best = '';
let bestScore = -Infinity;
for (const part of fragments) {
const score = scoreCandidate(part, { source });
if (score > bestScore) {
best = part;
bestScore = score;
}
}
return best ? formatCaptureTitle(best, { source, metadata }) : '';
}
function buildCaptureTitle({ mode = 'fullscreen', metadata = {}, ocrText = '', recentTyped = '', recentShortcut = '' } = {}) {
const app = cleanAppName(metadata.appName);
// 1. Keyboard shortcut → most reliable signal for "what action did the user take".
if (recentShortcut && SHORTCUT_TITLES[recentShortcut]) {
const base = SHORTCUT_TITLES[recentShortcut];
return app ? qualifyTitleWithApp(base, metadata.appName) : base;
}
// 2. UIAutomation element value — what's actually typed inside the clicked field.
const elementValue = normalizeWhitespace(metadata.elementValue || '');
if (elementValue) {
const roleLower = normalizeWhitespace(metadata.elementRole || '').toLowerCase();
const labelLower = normalizeWhitespace(metadata.elementLabel || '').toLowerCase();
const looksLikeSearch = /(search|find|query|omnibox|address bar)/.test(roleLower + ' ' + labelLower);
const action = looksLikeSearch ? 'Search for' : 'Type';
const base = `${action} "${elementValue}"`;
return app ? qualifyTitleWithApp(base, metadata.appName) : base;
}
// 3. Keyboard-buffer text (typed between captures) + input role context.
const typed = normalizeWhitespace(recentTyped || '');
if (typed) {
const roleLower = normalizeWhitespace(metadata.elementRole || '').toLowerCase();
const labelLower = normalizeWhitespace(metadata.elementLabel || '').toLowerCase();
const isSearchRole = /(search box|searchbox|search field|search bar|search input)/.test(roleLower);
const looksLikeSearch = isSearchRole || /(search|find|query|omnibox|address bar)/.test(roleLower + ' ' + labelLower);
const isAnyInput = /(text field|edit|input|field|combo box|textbox|text box)/.test(roleLower);
if (looksLikeSearch) {
const base = `Search for "${typed}"`;
return app ? qualifyTitleWithApp(base, metadata.appName) : base;
}
if (isAnyInput) {
const base = `Type "${typed}"`;
return app ? qualifyTitleWithApp(base, metadata.appName) : base;
}
}
// 4. OCR text around the click — link text, button labels, menu items.
const ocrPhrase = pickBestOcrPhrase(ocrText);
if (ocrPhrase) {
const title = formatCaptureTitle(ocrPhrase, { source: 'ocr', metadata });
return app ? qualifyTitleWithApp(title, metadata.appName) : title;
}
// 5. UIAutomation element label.
const elementPhrase = pickBestTitleFragment(metadata.elementLabel, { source: 'element', metadata });
if (elementPhrase) {
return app ? qualifyTitleWithApp(elementPhrase, metadata.appName) : elementPhrase;
}
// 6. Window title (browser suffix + app name stripped) → page title or search query.
const strippedWindowTitle = stripBrowserNameSuffix(metadata.windowTitle || '', metadata.appName);
if (strippedWindowTitle) {
const searchQuery = extractSearchQuery(strippedWindowTitle);
if (searchQuery) {
// Only claim this step IS the search action when the user was actually typing
// (recentTyped). Without typing context, the search page title is from the
// PREVIOUS step — the current step is a click ON the search results page.
if (recentTyped) {
const base = `Search for ${sentenceCase(searchQuery)}`;
return app ? qualifyTitleWithApp(base, metadata.appName) : base;
}
// User is clicking something on the search results page — don't claim they searched.
const base = `Select a ${sentenceCase(searchQuery)} result`;
return app ? qualifyTitleWithApp(base, metadata.appName) : base;
}
const windowPhrase = pickBestTitleFragment(strippedWindowTitle, { source: 'window', metadata });
if (windowPhrase) return windowPhrase;
}
// 7. App name alone as last resort.
const appPhrase = pickBestTitleFragment(metadata.appName, { source: 'app', metadata });
if (appPhrase) return appPhrase;
return DEFAULT_CAPTURE_TITLES[mode] || 'Capture';
}
function plainTextToHtml(text) {
const trimmed = normalizeWhitespace(text);
if (!trimmed) return '';
return trimmed
.split(/\n{2,}/)
.map((para) => `<p>${escapeHtml(para).replace(/\n/g, '<br>')}</p>`)
.join('');
}
function normalizeOllamaHost(host) {
const raw = normalizeWhitespace(host);
if (!raw) return '';
if (/^https?:\/\//i.test(raw)) return raw.replace(/\/+$/, '');
return `http://${raw.replace(/\/+$/, '')}`;
}
function normalizeAiLevel(level) {
const key = normalizeWhitespace(level).toLowerCase();
return AI_LEVEL_ALIASES.get(key) || (TEXTBLOCK_LEVELS.includes(key) ? key : 'info');
}
function normalizeAiPosition(position) {
const key = normalizeWhitespace(position).toLowerCase();
return TEXTBLOCK_POSITIONS.includes(key) ? key : 'after-description';
}
function normalizeAiBlock(block) {
if (!block || typeof block !== 'object') return null;
const kind = normalizeWhitespace(block.kind).toLowerCase();
if (kind === 'text') {
const normalized = normalizeTextBlock({
id: block.id,
order: Number.isFinite(block.order) ? block.order : null,
position: normalizeAiPosition(block.position),
level: normalizeAiLevel(block.level),
title: displayText(block.title),
descriptionHtml: plainTextToHtml(block.body ?? block.description ?? block.text ?? ''),
}, Number.isFinite(block.order) ? block.order : null);
return { ...normalized, kind: 'text' };
}
if (kind === 'code') {
return {
...normalizeCodeBlock({
id: block.id,
order: Number.isFinite(block.order) ? block.order : null,
language: displayText(block.language).toLowerCase(),
code: String(block.code ?? ''),
}, Number.isFinite(block.order) ? block.order : null),
kind: 'code',
};
}
if (kind === 'table') {
const rows = Array.isArray(block.rows)
? block.rows.map((row) => (Array.isArray(row) ? row.map((cell) => displayText(cell)) : []))
: [];
return {
...normalizeTableBlock({
id: block.id,
order: Number.isFinite(block.order) ? block.order : null,
rows,
}, Number.isFinite(block.order) ? block.order : null),
kind: 'table',
};
}
return null;
}
function normalizeAiPatch(raw) {
let data = raw;
if (typeof raw === 'string') {
const trimmed = raw.trim().replace(/^```(?:json)?\s*|\s*```$/g, '');
const start = trimmed.indexOf('{');
const end = trimmed.lastIndexOf('}');
const jsonText = start >= 0 && end > start ? trimmed.slice(start, end + 1) : trimmed;
data = JSON.parse(jsonText);
}
if (!data || typeof data !== 'object' || Array.isArray(data)) {
throw new Error('AI response must be a JSON object');
}
const out = {
title: displayText(data.title),
descriptionHtml: plainTextToHtml(data.description ?? data.descriptionText ?? ''),
blocks: Array.isArray(data.blocks)
? data.blocks.map((block) => normalizeAiBlock(block)).filter(Boolean)
: [],
};
return out;
}
function summarizeBlocks(step = {}) {
const parts = [];
for (const block of step.textBlocks || []) {
const body = htmlToText(block.descriptionHtml || '');
parts.push(`- Text (${block.level || 'info'}, ${block.position || 'after-description'}): ${block.title || ''}${body ? `${body}` : ''}`.trim());
}
for (const block of step.codeBlocks || []) {
const code = String(block.code || '').trim();
parts.push(`- Code (${block.language || 'plain'}):\n${code || '(empty)'}`);
}
for (const block of step.tableBlocks || []) {
const rows = Array.isArray(block.rows) ? block.rows.length : 0;
const cols = rows > 0 && Array.isArray(block.rows[0]) ? block.rows[0].length : 0;
parts.push(`- Table (${rows}x${cols})`);
}
return parts.length ? parts.join('\n') : '(none)';
}
const DEFAULT_PLACEHOLDER_TITLES = new Set(
Object.values(DEFAULT_CAPTURE_TITLES).concat(['Capture', 'Untitled step']),
);
function isPlaceholderTitle(title) {
return !title || DEFAULT_PLACEHOLDER_TITLES.has(title);
}
function summarizeStepForAi(step = {}) {
const titleLine = isPlaceholderTitle(step.title)
? 'Step title: (not set — generate a specific action title from the capture context)'
: `Step title: ${step.title}`;
const descText = htmlToText(step.descriptionHtml || '');
return [
titleLine,
`Step description: ${descText || '(empty)'}`,
`Step status: ${step.status || 'todo'}`,
`Blocks:\n${summarizeBlocks(step)}`,
].join('\n');
}
function summarizeGuideForAi(guide = {}) {
return [
`Guide title: ${guide.title || '(untitled)'}`,
`Guide description: ${htmlToText(guide.descriptionHtml || '') || '(empty)'}`,
].join('\n');
}
function hasRichCaptureContext(captureContext) {
if (!captureContext) return false;
const ocr = normalizeWhitespace(captureContext.ocrText || '');
const win = normalizeWhitespace(captureContext.windowTitle || '');
const app = normalizeWhitespace(captureContext.appName || '');
const element = normalizeWhitespace(captureContext.elementLabel || '');
// Any non-trivial context signal is enough — even just an app name.
return ocr.length > 3 || win.length > 2 || app.length > 1 || element.length > 1;
}
function buildAiPrompt({
target = 'all',
guide = null,
step = null,
captureContext = null,
block = null,
screenshotAttached = false,
} = {}) {
const hasDraftTitle = step && !isPlaceholderTitle(step.title);
const hasDraftDesc = step && Boolean(htmlToText(step.descriptionHtml || ''));
const targetText = {
title: hasDraftTitle
? 'improve the user\'s draft step title — keep their intent, make it read like professional documentation'
: 'write a specific action title for this step using the capture context',
description: hasDraftDesc
? 'improve the user\'s draft description — keep their intent, make it read like professional documentation'
: 'write a 12 sentence description of what the user does in this step, using the capture context',
block: 'rewrite only the target block',
all: 'write the step title and description from the capture context',
}[target] || 'rewrite the step';
const richContext = hasRichCaptureContext(captureContext);
const allowedBlockNote = target === 'block' ? [
'Use block.kind = "text" with level in [info, warn, error, success] for note / warning / important / tip blocks.',
'Use block.kind = "code" for code snippets.',
'Use block.kind = "table" for tables, with rows as arrays of strings.',
'Use block.position values from [before-title, after-title, before-image, after-image, before-description, after-description].',
].join(' ') : null;
// When the user already has a draft, surface it prominently so the model
// knows exactly what text to polish rather than generating from scratch.
const descText = htmlToText(step?.descriptionHtml || '');
const draftTitleLine = hasDraftTitle && (target === 'title' || target === 'all')
? `User's draft title (rewrite this): "${step.title}"` : null;
const draftDescLine = hasDraftDesc && (target === 'description' || target === 'all')
? `User's draft description (rewrite this): "${descText}"` : null;
const contextLines = [
...(captureContext ? [
captureContext.windowTitle ? `Active window: ${captureContext.windowTitle}` : null,
captureContext.appName ? `App: ${captureContext.appName}` : null,
captureContext.elementLabel ? `UI element: ${captureContext.elementLabel}${captureContext.elementRole ? ` (${captureContext.elementRole})` : ''}` : null,
captureContext.elementValue ? `Element content (what was typed): ${captureContext.elementValue}` : null,
captureContext.recentTyped ? `Keyboard input before this step: ${captureContext.recentTyped}` : null,
captureContext.recentShortcut ? `Keyboard shortcut used: ${captureContext.recentShortcut}` : null,
captureContext.ocrText ? `OCR text near click:\n${captureContext.ocrText}` : null,
(!hasDraftTitle || target === 'description') && captureContext.titleCandidate
? `Suggested title: ${captureContext.titleCandidate}` : null,
] : []),
screenshotAttached ? 'Screenshot: attached to this request.' : null,
draftTitleLine,
draftDescLine,
].filter(Boolean);
const prompt = [
'You write concise, action-focused step-by-step documentation for a desktop application guide.',
'Return JSON only. No markdown fences, no commentary, no extra keys outside the schema below.',
'Schema:',
target === 'block' ? [
'{',
' "title": string,',
' "description": string,',
' "blocks": [{',
' "kind": "text" | "code" | "table",',
' "position"?: "before-title" | "after-title" | "before-image" | "after-image" | "before-description" | "after-description",',
' "level"?: "info" | "warn" | "error" | "success",',
' "title"?: string,',
' "body"?: string,',
' "language"?: string,',
' "code"?: string,',
' "rows"?: string[][]',
' }]',
'}',
].join('\n') : '{ "title": string, "description": string }',
'',
`Target: ${targetText}.`,
allowedBlockNote,
'',
guide ? summarizeGuideForAi(guide) : 'Guide: (not provided)',
'',
step ? summarizeStepForAi(step) : 'Step: (not provided)',
'',
contextLines.length
? `Capture context:\n${contextLines.join('\n')}`
: 'Capture context: (not available)',
'',
block ? `Target block:\n${JSON.stringify(block, null, 2)}` : null,
'',
'Rules:',
'- Titles must be short imperative actions: "Click Save", "Select New document", "Open Settings".',
'- NEVER output "Screen capture", "Window capture", "Region capture", or "Capture" as a title — always produce something specific.',
hasDraftTitle && (target === 'title' || target === 'all')
? '- The user wrote their own title (shown above). Your only job is to polish its grammar and phrasing. Do NOT replace it with something different. Do NOT change what action or subject it describes.'
: '- No title yet. Use the capture context (OCR text, window, app) to write a specific action title.',
hasDraftDesc && (target === 'description' || target === 'all')
? '- The user wrote their own description (shown above). Polish the wording to sound professional but preserve every fact and intent they stated.'
: '- No description yet. Write 12 sentences describing exactly what the user does.',
target === 'block'
? '- Only include blocks that provide genuinely useful supplemental information (warnings, tips, code).'
: '- Do NOT add any blocks array. Only output "title" and "description".',
richContext
? '- Use the OCR text, window title, app name, and element info to make the documentation specific.'
: '- Context is limited. Use the app name or window title if available; generate a reasonable action title.',
screenshotAttached
? '- A screenshot is attached. Use it together with the OCR and metadata to resolve visual details, but do not mention the screenshot in the output.'
: '- No screenshot is attached. Rely on OCR, the window title, app name, and element info.',
'- Do NOT generate blocks that describe the technical capture process or mention OCR.',
'- Do NOT invent details not supported by the capture context.',
'- If the target is one block, only rewrite that block.',
].filter((l) => l !== null).join('\n');
return {
systemPrompt: 'You are a technical documentation writer. Emit only valid JSON matching the schema. Never add commentary or markdown.',
prompt,
};
}
function applyAiPatchToStep(step, patch, { target = 'all', blockId = null } = {}) {
const next = deepClone(step);
if ((target === 'all' || target === 'title') && patch.title) {
next.title = displayText(patch.title);
}
if ((target === 'all' || target === 'description') && patch.descriptionHtml) {
next.descriptionHtml = sanitizeHtml(patch.descriptionHtml);
}
if (target === 'all' && Array.isArray(patch.blocks) && patch.blocks.length) {
const textBlocks = [];
const codeBlocks = [];
const tableBlocks = [];
let nextOrder = 1;
for (const block of patch.blocks) {
const clone = deepClone(block);
clone.order = nextOrder++;
if (clone.kind === 'text') textBlocks.push(clone);
else if (clone.kind === 'code') codeBlocks.push(clone);
else if (clone.kind === 'table') tableBlocks.push(clone);
}
next.textBlocks = textBlocks;
next.codeBlocks = codeBlocks;
next.tableBlocks = tableBlocks;
} else if (target === 'block' && blockId && Array.isArray(patch.blocks) && patch.blocks.length) {
const replacement = patch.blocks[0];
const textBlock = (next.textBlocks || []).find((block) => block.id === blockId);
const codeBlock = (next.codeBlocks || []).find((block) => block.id === blockId);
const tableBlock = (next.tableBlocks || []).find((block) => block.id === blockId);
if (textBlock && replacement.kind === 'text') {
if (replacement.position) textBlock.position = replacement.position;
if (replacement.level) textBlock.level = replacement.level;
if (replacement.title) textBlock.title = replacement.title;
if (replacement.descriptionHtml) textBlock.descriptionHtml = sanitizeHtml(replacement.descriptionHtml);
} else if (codeBlock && replacement.kind === 'code') {
if (replacement.language) codeBlock.language = replacement.language;
if (replacement.code) codeBlock.code = replacement.code;
} else if (tableBlock && replacement.kind === 'table') {
if (replacement.rows) tableBlock.rows = replacement.rows;
}
}
if (!next.image) {
const hasBody = Boolean(
next.title ||
htmlToText(next.descriptionHtml || '') ||
(next.textBlocks || []).length ||
(next.codeBlocks || []).length ||
(next.tableBlocks || []).length,
);
if (hasBody) next.kind = 'content';
}
return next;
}
module.exports = {
DEFAULT_CAPTURE_TITLES,
buildCaptureTitle,
plainTextToHtml,
normalizeOllamaHost,
normalizeAiPatch,
buildAiPrompt,
applyAiPatchToStep,
summarizeStepForAi,
summarizeGuideForAi,
displayText,
normalizeWhitespace,
scoreCandidate,
pickBestOcrPhrase,
};
+1 -1
View File
@@ -111,7 +111,7 @@ click position. Three pieces make that hold:
1. **OS click events** (`app/capture.js`): a low-level mouse hook on Windows 1. **OS click events** (`app/capture.js`): a low-level mouse hook on Windows
(`CLICK x y button unixMs` lines), an `xinput test-xi2 --root` watcher on (`CLICK x y button unixMs` lines), an `xinput test-xi2 --root` watcher on
X11. The Linux parser carries event-time `root:` coordinates and merges X11. The Linux (WIP) parser carries event-time `root:` coordinates and merges
raw/regular twin blocks structurally — there is no time-based debounce raw/regular twin blocks structurally — there is no time-based debounce
that could drop fast clicks, only suppression of identical duplicate that could drop fast clicks, only suppression of identical duplicate
deliveries. Physical coordinates convert to DIP via deliveries. Physical coordinates convert to DIP via
+13 -6
View File
@@ -11,13 +11,19 @@ For the windows installation, please see [windows_installation](windows_installa
## 1. Install ## 1. Install
Install the pinned Node toolchain first — Node 22.12 or newer (see
`.nvmrc`; with nvm: `nvm install && nvm use`). Installs are refused on
older Nodes.
From the repository root: From the repository root:
```bash ```bash
npm install npm ci
``` ```
That installs Electron and the local packaging tools used by the scripts. That installs the locked dependency tree — Electron and the local packaging
tools used by the scripts. `npm ci` is the only supported installation path;
the app never installs or repairs dependencies at runtime.
## 2. Launch the app ## 2. Launch the app
@@ -25,9 +31,9 @@ That installs Electron and the local packaging tools used by the scripts.
npm start npm start
``` ```
The first launch creates the local StepForge data directory. On Linux it is The first launch creates the local StepForge data directory. On Linux (WIP)
usually under `~/.local/share/stepforge`. On Windows it is usually under it is usually under `~/.local/share/stepforge`. On Windows it is usually
`%APPDATA%/stepforge`. under `%APPDATA%/stepforge`.
## 3. Create your first guide ## 3. Create your first guide
@@ -95,4 +101,5 @@ If you want to find commands quickly, press `Ctrl+/` for Quick Actions.
1. `bash scripts/build-release.sh` assembles the offline release layout. 1. `bash scripts/build-release.sh` assembles the offline release layout.
2. `npm run package:windows` creates the Windows installer `.exe` in 2. `npm run package:windows` creates the Windows installer `.exe` in
`releases/`. `releases/`.
3. `bash scripts/package-linux.sh` creates Linux release artifacts. 3. `bash scripts/package-linux.sh` creates Linux release artifacts (WIP;
local only).
+156
View File
@@ -0,0 +1,156 @@
# Getting Started with StepForge on Linux
> ⚠️ **Work in progress.** Linux support is still under active development.
> Expect rough edges — especially on Wayland (see the limitations below). X11 /
> Xorg is the most complete path today. Please report issues.
StepForge was built on Windows, where the OS lets an app watch every click and
grab the screen freely. Linux is more restrictive, and **how much works depends
on whether you are running X11 (Xorg) or Wayland.** This guide explains the
difference, how to get the best experience, and how to enable per-click capture.
## TL;DR
| | **X11 / "Ubuntu on Xorg"** | **Wayland (default on Ubuntu)** |
|---|---|---|
| Screenshot per click | ✅ Yes | ✅ Yes (needs `input` group) |
| Red circle on the click | ✅ Yes | ❌ No (Wayland hides the cursor position) |
| "Share your screen" prompt | Never | Once per recording session |
| Setup needed | None | Add yourself to the `input` group |
**If you want the full Windows-like experience (click capture *with* the red
marker), use an Xorg session — see [Option A](#option-a-best-experience--use-xorg).**
---
## 1. Check which session you are running
```bash
echo $XDG_SESSION_TYPE
```
- `x11` → you're on Xorg. Everything works, including the red click marker. No setup needed.
- `wayland` → you're on Wayland. Read on.
---
## Option A (best experience) — use Xorg
On Xorg, StepForge captures a screenshot **on every click** and draws the **red
marker** at the exact click position, exactly like Windows. No dependencies, no
permissions, no portal dialogs.
To switch:
1. Log out.
2. On the login (password) screen, click the **⚙ gear icon** in the bottom-right corner.
3. Choose **"Ubuntu on Xorg"**.
4. Log back in.
That's it — open StepForge and record. (To go back to Wayland later, pick
"Ubuntu" at the gear menu again.)
---
## Option B — stay on Wayland
Wayland deliberately blocks apps from monitoring global input and from grabbing
the screen silently. StepForge works around this as far as the platform allows:
### Screen capture
The first time you press **Start recording**, the system shows a **"Share your
screen"** dialog (the XDG desktop portal). Pick your screen and click **Share**.
This happens **once per recording session** — not per screenshot. The shared
stream stays open until you stop recording.
> If you never see steps appear, make sure you actually picked a screen and
> clicked **Share** in that dialog.
### Per-click capture (requires the `input` group)
By default on Wayland, StepForge cannot see your clicks, so it falls back to
**capturing a screenshot every few seconds** (timed capture).
To get a screenshot **on every click** instead, give your user read access to
the mouse devices by joining the `input` group:
```bash
sudo usermod -aG input "$USER"
```
Then **log out and log back in** (group membership only applies to new sessions).
Verify it took effect:
```bash
groups | tr ' ' '\n' | grep input # should print: input
```
Now StepForge reads mouse buttons directly from the kernel (`/dev/input`) and
captures a screenshot on each click.
> **No red marker on Wayland.** Even with per-click capture working, Wayland
> does not tell apps *where* the pointer is, so StepForge cannot draw the circle
> at the click. The screenshot is still captured per click — just without the
> marker. If you need the marker, use [Option A (Xorg)](#option-a-best-experience--use-xorg).
### Adjusting the timed-capture interval
If you don't enable the `input` group, StepForge captures on a timer. Change the
fallback in **Settings → Capture**:
- `When clicks are unavailable` -> `Hotkey only` to use the Capture hotkey
instead of a timer.
- `When clicks are unavailable` -> `Timed interval`, then set
`Timer interval (seconds)` (`capture.autoIntervalSec`, default 5 seconds)
if you want timed captures.
---
## How StepForge picks a capture method (for reference)
On launch StepForge chooses the best available click source:
1. **Windows** — low-level mouse hook (position + timing).
2. **X11**`xinput` (position + timing → full red marker).
3. **Linux evdev** (`/dev/input`) — button presses on X11 *and* Wayland, no
position on Wayland. Used when `xinput` can't see clicks (i.e. Wayland), if
you're in the `input` group.
4. **Timed capture** — the always-works fallback (a screenshot every N seconds)
when no click source is available.
Screen frames come from a single long-lived capture stream per recording, so
clicks/timer ticks never re-open the screen-share dialog.
---
## Troubleshooting
**"It asks to share my screen every time."**
You're likely on an older build. Update to the current version — the screen
stream is now opened once per recording session. If it persists, confirm
`echo $XDG_SESSION_TYPE` and that you clicked **Share** (not Cancel) in the dialog.
**"Recording captures a couple of steps then stops."**
Fixed in the current version (a slow, GPU-less PNG encode used to trip a
failure guard and tear down the stream). Update and retry.
**"The window disappeared and I can't stop the recording."**
On Linux the window **minimizes** while recording (GNOME's system tray is
unreliable). Bring it back from the **taskbar / dock**, then click **Stop
recording**.
**"No steps at all on Wayland, even after picking a screen."**
Run from a terminal with logging and look for the diagnostic lines:
```bash
STEPFORGE_CAPTURE_LOG=1 npm start
```
- `[stepforge] screen-capture stream active …` — the stream is up.
- `[stepforge] per-click capture via evdev on N device(s) …` — clicks are wired up.
- `[stepforge] no readable mouse input devices …` — you need the `input` group (see above).
**Harmless console noise.** Lines like `vaInitialize failed`, `Frame latency is
negative`, and `StatusNotifierItem … already exported` come from Chromium/GNOME,
not StepForge, and don't affect recording.
+84
View File
@@ -0,0 +1,84 @@
# Getting Started With AI
StepForge keeps AI local. It talks to your own Ollama server on your machine and does not send guide content to the cloud.
## 1. Install Ollama
Install Ollama from https://ollama.com and make sure the service is running.
On most systems you can verify it with:
```bash
ollama --version
```
## 2. Pull a lightweight model
The recommended default is:
```bash
ollama pull llama3.2:1b
```
That model is small enough to feel responsive on modest hardware, but still good enough for human-sounding titles and short text blocks.
If you want StepForge to send the screenshot itself to the model, pull a vision-capable model instead:
```bash
ollama pull gemma3
```
That model can inspect pictures as well as text, so it is better when you want the AI to read the UI directly from the screenshot.
If you need something even smaller, try:
```bash
ollama pull qwen3:0.6b
```
or:
```bash
ollama pull gemma3:270m
```
Those are lighter, but they are usually weaker at writing polished step text.
## 3. Open StepForge settings
In StepForge, open `Settings` and find the `AI` section.
Set:
* `Enable AI text filling` to on
* `Ollama host` to your local Ollama server
* `Ollama model` to `llama3.2:1b` for text-only mode, or `llama3.2-vision` if you want screenshot-aware AI
The default host is:
```text
http://127.0.0.1:11434
```
## 4. Test the connection
Use the `Test connection` button in the AI settings section.
If the model is installed, StepForge should confirm the host and model.
## 5. Use AI manually
AI is never automatic. After capture, use the `AI` button next to:
* the step title
* the step description
* each text, code, and table block
You can also use `More -> Generate all text fields with AI` to fill the whole step in one pass.
## Notes
* Capture titles are still generated automatically without AI.
* AI generation only works when `Enable AI text filling` is turned on.
* The app always uses local OCR around the click area first, then local AI only when you ask for it.
* When the selected Ollama model supports vision, StepForge also sends the screenshot to the model so it can cross-check OCR and visual context.
+43 -8
View File
@@ -56,17 +56,52 @@ function stepBlocks(step) {
} }
/** /**
* Split a step's blocks into the groups exporters lay out around the * Split a step's blocks into the layout buckets exporters use around the
* description/image: text blocks pinned to 'before-description', and * step title, description, and image. Text blocks keep their block-list
* everything else (code/table blocks plus 'after-description' and * ordering inside each bucket; code/table blocks stay in `rest`.
* 'after-image' text blocks) in the same relative order they appear in the
* editor's Blocks list.
*/ */
function stepContentGroups(step) { function stepContentGroups(step) {
const all = stepBlocks(step); const all = stepBlocks(step);
const before = all.filter((b) => b.kind === 'text' && b.position === 'before-description'); const groups = {
const rest = all.filter((b) => !(b.kind === 'text' && b.position === 'before-description')); beforeTitle: [],
return { before, rest }; afterTitle: [],
beforeDescription: [],
afterDescription: [],
beforeImage: [],
afterImage: [],
rest: [],
};
for (const block of all) {
if (block.kind !== 'text') {
groups.rest.push(block);
continue;
}
switch (block.position) {
case 'before-title':
groups.beforeTitle.push(block);
break;
case 'after-title':
groups.afterTitle.push(block);
break;
case 'before-image':
groups.beforeImage.push(block);
break;
case 'after-image':
groups.afterImage.push(block);
break;
case 'before-description':
groups.beforeDescription.push(block);
break;
case 'after-description':
default:
groups.afterDescription.push(block);
break;
}
}
return groups;
} }
function codeBlockText(block) { function codeBlockText(block) {
+29 -4
View File
@@ -62,11 +62,25 @@ function exportConfluence(ast, outDir, template = {}) {
} }
const stepXml = ast.steps.map((step) => { const stepXml = ast.steps.map((step) => {
const parts = [`<a id="${anchorFor(step)}"></a>`, `<h2>${escapeXml(step.number)}. ${escapeXml(step.title || 'Untitled step')}</h2>`]; const {
beforeTitle,
afterTitle,
beforeDescription,
afterDescription,
beforeImage,
afterImage,
rest,
} = stepContentGroups(step);
const parts = [`<a id="${anchorFor(step)}"></a>`];
for (const tb of beforeTitle) {
parts.push(blockMacro(tb, ast));
}
parts.push(`<h2>${escapeXml(step.number)}. ${escapeXml(step.title || 'Untitled step')}</h2>`);
if (step.skipped) parts.push('<p><em>(skipped)</em></p>'); if (step.skipped) parts.push('<p><em>(skipped)</em></p>');
for (const tb of afterTitle) {
const { before, rest } = stepContentGroups(step); parts.push(blockMacro(tb, ast));
for (const tb of before) { }
for (const tb of beforeDescription) {
parts.push(blockMacro(tb, ast)); parts.push(blockMacro(tb, ast));
} }
@@ -74,11 +88,22 @@ function exportConfluence(ast, outDir, template = {}) {
parts.push(`<div>${stepLinkRewrite(step.descriptionHtml, ast)}</div>`); parts.push(`<div>${stepLinkRewrite(step.descriptionHtml, ast)}</div>`);
} }
for (const tb of afterDescription) {
parts.push(blockMacro(tb, ast));
}
for (const tb of beforeImage) {
parts.push(blockMacro(tb, ast));
}
const attachment = attachmentNames.get(step.stepId); const attachment = attachmentNames.get(step.stepId);
if (attachment) { if (attachment) {
parts.push(`<p><ac:image><ri:attachment ri:filename="${escapeXml(attachment)}" /></ac:image></p>`); parts.push(`<p><ac:image><ri:attachment ri:filename="${escapeXml(attachment)}" /></ac:image></p>`);
} }
for (const tb of afterImage) {
parts.push(blockMacro(tb, ast));
}
for (const block of rest) { for (const block of rest) {
if (block.kind === 'text') { if (block.kind === 'text') {
parts.push(blockMacro(block, ast)); parts.push(blockMacro(block, ast));
+18 -5
View File
@@ -10,8 +10,8 @@ const { guideMetaLines, guideSummary, tocEntries } = require('./document-layout'
/** /**
* DOCX exporter: WordprocessingML built directly (no dependency), one * DOCX exporter: WordprocessingML built directly (no dependency), one
* heading + description + screenshot per step, text blocks, code blocks * heading per step with positioned text blocks around the description and
* (Courier), and tables. * screenshot, plus code blocks (Courier) and tables.
*/ */
const DEFAULT_TEMPLATE = { const DEFAULT_TEMPLATE = {
@@ -274,16 +274,27 @@ function exportDocx(ast, outDir, template = {}) {
const headSize = headingLevel === 1 ? 30 : headingLevel === 2 ? 26 : 22; const headSize = headingLevel === 1 ? 30 : headingLevel === 2 ? 26 : 22;
const bookmarkId = ++bookmarkCounter; const bookmarkId = ++bookmarkCounter;
const anchor = bookmarkName(step); const anchor = bookmarkName(step);
const {
beforeTitle,
afterTitle,
beforeDescription,
afterDescription,
beforeImage,
afterImage,
rest,
} = stepContentGroups(step);
for (const tb of beforeTitle) emitTextBlock(tb);
body.push(p( body.push(p(
`<w:bookmarkStart w:id="${bookmarkId}" w:name="${anchor}"/>` + `<w:bookmarkStart w:id="${bookmarkId}" w:name="${anchor}"/>` +
run(`${step.number}. ${step.title || 'Untitled step'}`, { bold: true, size: headSize }) + run(`${step.number}. ${step.title || 'Untitled step'}`, { bold: true, size: headSize }) +
`<w:bookmarkEnd w:id="${bookmarkId}"/>`, `<w:bookmarkEnd w:id="${bookmarkId}"/>`,
headingParagraphProps(step.depth, step.forceNewPage) headingParagraphProps(step.depth, step.forceNewPage)
)); ));
for (const tb of afterTitle) emitTextBlock(tb);
const { before, rest } = stepContentGroups(step); for (const tb of beforeDescription) emitTextBlock(tb);
for (const tb of before) emitTextBlock(tb);
if (step.descriptionText) body.push(p(run(step.descriptionText, { size: 20, color: '1F2937' }))); if (step.descriptionText) body.push(p(run(step.descriptionText, { size: 20, color: '1F2937' })));
for (const tb of afterDescription) emitTextBlock(tb);
for (const tb of beforeImage) emitTextBlock(tb);
const img = images.get(step.stepId); const img = images.get(step.stepId);
if (img) { if (img) {
@@ -295,6 +306,8 @@ function exportDocx(ast, outDir, template = {}) {
stepImageCount += 1; stepImageCount += 1;
} }
for (const tb of afterImage) emitTextBlock(tb);
for (const block of rest) { for (const block of rest) {
if (block.kind === 'text') { if (block.kind === 'text') {
emitTextBlock(block); emitTextBlock(block);
+28 -4
View File
@@ -40,6 +40,10 @@ function blockHtml(tb) {
return `<div class="block block-${tb.level}"><strong>${escapeHtml(LEVEL_LABEL[tb.level] || 'Note')}${tb.title ? `: ${escapeHtml(tb.title)}` : ''}</strong>${tb.descriptionHtml ? `<div>${tb.descriptionHtml}</div>` : ''}</div>`; return `<div class="block block-${tb.level}"><strong>${escapeHtml(LEVEL_LABEL[tb.level] || 'Note')}${tb.title ? `: ${escapeHtml(tb.title)}` : ''}</strong>${tb.descriptionHtml ? `<div>${tb.descriptionHtml}</div>` : ''}</div>`;
} }
function renderBlockListHtml(blocks) {
return blocks.map((block) => blockHtml(block)).join('\n');
}
function renderMetaChips(ast) { function renderMetaChips(ast) {
return [ return [
...guideMetaLines(ast).map((line) => `<span class="chip">${escapeHtml(line)}</span>`), ...guideMetaLines(ast).map((line) => `<span class="chip">${escapeHtml(line)}</span>`),
@@ -85,11 +89,22 @@ function renderStepCard(step, ast, images, tpl, { rich = false, selected = false
<span class="step-title">${escapeHtml(step.title || 'Untitled step')}</span> <span class="step-title">${escapeHtml(step.title || 'Untitled step')}</span>
<span class="status-chip status-${step.status || 'todo'}${step.skipped ? ' skipped' : ''}">${escapeHtml(statusText)}</span> <span class="status-chip status-${step.status || 'todo'}${step.skipped ? ' skipped' : ''}">${escapeHtml(statusText)}</span>
</h2>` : `<h2>${title}</h2>`; </h2>` : `<h2>${title}</h2>`;
const {
beforeTitle,
afterTitle,
beforeDescription,
afterDescription,
beforeImage,
afterImage,
rest,
} = stepContentGroups(step);
return ` return `
<section class="step-card${step.skipped ? ' skipped' : ''}${rich && selected ? ' selected' : ''}" id="${anchorFor(step)}"> <section class="step-card${step.skipped ? ' skipped' : ''}${rich && selected ? ' selected' : ''}" id="${anchorFor(step)}">
${renderBlockListHtml(beforeTitle)}
${head} ${head}
${renderBlockListHtml(afterTitle)}
<div class="step-body"> <div class="step-body">
${stepBodyHtml(step, ast, images, tpl)} ${renderStepBodyHtml({ beforeDescription, afterDescription, beforeImage, afterImage, rest }, step, ast, images, tpl)}
</div> </div>
</section>`; </section>`;
} }
@@ -117,15 +132,24 @@ function bodyStyle(tpl) {
return `--accent:${tpl.accentColor};--accent-rgb:${r},${g},${b};`; return `--accent:${tpl.accentColor};--accent-rgb:${r},${g},${b};`;
} }
function stepBodyHtml(step, ast, images, tpl) { function renderStepBodyHtml(groups, step, ast, images, tpl) {
const parts = []; const parts = [];
const { before, rest } = stepContentGroups(step); const {
for (const tb of before) parts.push(blockHtml(tb)); beforeDescription,
afterDescription,
beforeImage,
afterImage,
rest,
} = groups;
for (const tb of beforeDescription) parts.push(blockHtml(tb));
if (step.descriptionHtml) parts.push(`<div class="desc">${stepLinkRewrite(step.descriptionHtml, ast)}</div>`); if (step.descriptionHtml) parts.push(`<div class="desc">${stepLinkRewrite(step.descriptionHtml, ast)}</div>`);
for (const tb of afterDescription) parts.push(blockHtml(tb));
for (const tb of beforeImage) parts.push(blockHtml(tb));
const img = images.get(step.stepId); const img = images.get(step.stepId);
if (img && tpl.includeImages) { if (img && tpl.includeImages) {
parts.push(`<img class="shot" alt="Step ${escapeHtml(step.number)}" src="${dataUri(img)}" width="${img.width}">`); parts.push(`<img class="shot" alt="Step ${escapeHtml(step.number)}" src="${dataUri(img)}" width="${img.width}">`);
} }
for (const tb of afterImage) parts.push(blockHtml(tb));
for (const block of rest) { for (const block of rest) {
if (block.kind === 'text') { if (block.kind === 'text') {
parts.push(blockHtml(block)); parts.push(blockHtml(block));
+15 -4
View File
@@ -96,15 +96,25 @@ function renderMarkdownGuide(ast, outDir, template = {}, {
for (const step of ast.steps) { for (const step of ast.steps) {
const heading = step.depth > 0 ? '###' : '##'; const heading = step.depth > 0 ? '###' : '##';
const {
beforeTitle,
afterTitle,
beforeDescription,
afterDescription,
beforeImage,
afterImage,
rest,
} = stepContentGroups(step);
lines.push(`<a id="${anchorFor(step)}"></a>`, ''); lines.push(`<a id="${anchorFor(step)}"></a>`, '');
for (const tb of beforeTitle) emitBlock(lines, tb, { alertStyle });
lines.push(`${heading} ${step.number}. ${step.title || 'Untitled step'}`, ''); lines.push(`${heading} ${step.number}. ${step.title || 'Untitled step'}`, '');
if (step.skipped) lines.push('*(skipped)*', ''); if (step.skipped) lines.push('*(skipped)*', '');
for (const tb of afterTitle) emitBlock(lines, tb, { alertStyle });
const { before, rest } = stepContentGroups(step); for (const tb of beforeDescription) emitBlock(lines, tb, { alertStyle });
for (const tb of before) emitBlock(lines, tb, { alertStyle });
if (step.descriptionHtml) lines.push(htmlToMarkdown(step.descriptionHtml), ''); if (step.descriptionHtml) lines.push(htmlToMarkdown(step.descriptionHtml), '');
for (const tb of afterDescription) emitBlock(lines, tb, { alertStyle });
for (const tb of beforeImage) emitBlock(lines, tb, { alertStyle });
const img = images.get(step.stepId); const img = images.get(step.stepId);
if (img) { if (img) {
if (tpl.azureWiki && tpl.imageMaxWidth > 0) { if (tpl.azureWiki && tpl.imageMaxWidth > 0) {
@@ -113,6 +123,7 @@ function renderMarkdownGuide(ast, outDir, template = {}, {
lines.push(`![Step ${step.number}](${img.relPath})`, ''); lines.push(`![Step ${step.number}](${img.relPath})`, '');
} }
} }
for (const tb of afterImage) emitBlock(lines, tb, { alertStyle });
for (const block of rest) { for (const block of rest) {
if (block.kind === 'text') { if (block.kind === 'text') {
+31 -8
View File
@@ -203,10 +203,10 @@ function exportPdf(ast, outDir, template = {}) {
/** /**
* Compute the vertical space a step will need: `head` is the title, the * Compute the vertical space a step will need: `head` is the title, the
* accent rule, any before-description blocks, the description, and the * accent rule, positioned text blocks, the description, and the image
* image kept together on one page and `total` adds the remaining * kept together on one page and `total` adds the remaining blocks plus
* blocks plus the trailing gap, used to decide whether the whole step * the trailing gap, used to decide whether the whole step fits on a
* fits on a fresh page. * fresh page.
*/ */
const measureStep = (step) => { const measureStep = (step) => {
const headSize = step.depth > 0 ? 12 : 14; const headSize = step.depth > 0 ? 12 : 14;
@@ -214,8 +214,18 @@ function exportPdf(ast, outDir, template = {}) {
const titleLines = pdf.wrapText(titleText, headSize, usableW, 'F2'); const titleLines = pdf.wrapText(titleText, headSize, usableW, 'F2');
let head = Math.max(40, titleLines.length * headSize * 1.35 + 8); let head = Math.max(40, titleLines.length * headSize * 1.35 + 8);
const { before, rest } = stepContentGroups(step); const {
for (const tb of before) head += measureBlock(tb); beforeTitle,
afterTitle,
beforeDescription,
afterDescription,
beforeImage,
afterImage,
rest,
} = stepContentGroups(step);
for (const tb of [...beforeTitle, ...afterTitle, ...beforeDescription, ...afterDescription, ...beforeImage, ...afterImage]) {
head += measureBlock(tb);
}
if (step.descriptionHtml) head += layoutDescription(pdf, step.descriptionHtml, usableW, 10.5).height; if (step.descriptionHtml) head += layoutDescription(pdf, step.descriptionHtml, usableW, 10.5).height;
head += measureImage(step.stepId); head += measureImage(step.stepId);
@@ -298,14 +308,26 @@ function exportPdf(ast, outDir, template = {}) {
pdf.bookmark(`${step.number}. ${step.title || 'Untitled step'}`); pdf.bookmark(`${step.number}. ${step.title || 'Untitled step'}`);
const tocTarget = tocTargets.get(step.stepId); const tocTarget = tocTargets.get(step.stepId);
if (tocTarget) tocTarget.pageIndex = pdf.pages.length - 1; if (tocTarget) tocTarget.pageIndex = pdf.pages.length - 1;
const {
beforeTitle,
afterTitle,
beforeDescription,
afterDescription,
beforeImage,
afterImage,
rest,
} = stepContentGroups(step);
for (const tb of beforeTitle) emitBlock(tb);
const headSize = step.depth > 0 ? 12 : 14; const headSize = step.depth > 0 ? 12 : 14;
writeLines(`${step.number}. ${step.title || 'Untitled step'}${step.skipped ? ' (skipped)' : ''}`, { size: headSize, font: 'F2' }); writeLines(`${step.number}. ${step.title || 'Untitled step'}${step.skipped ? ' (skipped)' : ''}`, { size: headSize, font: 'F2' });
pdf.rect(M, y, usableW, 0.8, { fill: [225, 228, 232] }); pdf.rect(M, y, usableW, 0.8, { fill: [225, 228, 232] });
y += 8; y += 8;
const { before, rest } = stepContentGroups(step); for (const tb of afterTitle) emitBlock(tb);
for (const tb of before) emitBlock(tb); for (const tb of beforeDescription) emitBlock(tb);
if (step.descriptionHtml) writeDescription(step.descriptionHtml); if (step.descriptionHtml) writeDescription(step.descriptionHtml);
for (const tb of afterDescription) emitBlock(tb);
for (const tb of beforeImage) emitBlock(tb);
const img = images.get(step.stepId); const img = images.get(step.stepId);
if (img) { if (img) {
@@ -317,6 +339,7 @@ function exportPdf(ast, outDir, template = {}) {
pdf.image(img, M, y, w, h); pdf.image(img, M, y, w, h);
y += h + 10; y += h + 10;
} }
for (const tb of afterImage) emitBlock(tb);
for (const block of rest) { for (const block of rest) {
if (block.kind === 'text') { if (block.kind === 'text') {
+56 -22
View File
@@ -9,8 +9,9 @@ const { guideSlug, renderAllImages, LEVEL_LABEL, stepContentGroups } = require('
const { tocEntries, guideMetaLines, guideSummary } = require('./document-layout'); const { tocEntries, guideMetaLines, guideSummary } = require('./document-layout');
/** /**
* PPTX exporter: a title slide plus one 16:9 slide per step (title bar + * PPTX exporter: a title slide plus one 16:9 slide per step, with
* screenshot + description). PresentationML written directly. * positioned text blocks laid out around the step title, description, and
* screenshot. PresentationML written directly.
*/ */
const DEFAULT_TEMPLATE = { const DEFAULT_TEMPLATE = {
@@ -208,24 +209,63 @@ function exportPptx(ast, outDir, template = {}) {
let mediaCounter = 0; let mediaCounter = 0;
for (const step of ast.steps) { for (const step of ast.steps) {
const { before, rest } = stepContentGroups(step); const {
const beforeBlocks = before.filter((block) => block.kind === 'text'); beforeTitle,
const restBlocks = rest.filter((block) => block.kind === 'text'); afterTitle,
beforeDescription,
afterDescription,
beforeImage,
afterImage,
} = stepContentGroups(step);
let content = rectShape(0, 0, SLIDE_W, 18000, '2563EB'); let content = rectShape(0, 0, SLIDE_W, 18000, '2563EB');
content += textBox(457200, TITLE_Y, SLIDE_W - 914400, TITLE_H, const beforeTitleReserve = beforeTitle.reduce((sum, tb) => sum + calloutHeight(tb) + CALL_OUT_GAP, 0);
para(`${step.number}. ${step.title || 'Untitled step'}`, { size: 2600, bold: true })); const titleY = beforeTitle.length ? 220000 + beforeTitleReserve + 120000 : TITLE_Y;
content += rectShape(457200, TITLE_RULE_Y, 2400000, 12000, '2563EB'); const titleRuleY = beforeTitle.length ? titleY + TITLE_H + 120000 : TITLE_RULE_Y;
const contentY = beforeTitle.length ? Math.max(CONTENT_Y, titleRuleY + 180000) : CONTENT_Y;
const beforeTitleY = beforeTitle.length ? 220000 : CONTENT_Y;
let y = CONTENT_Y; let y = beforeTitleY;
for (const tb of beforeBlocks) { for (const tb of beforeTitle) {
const block = calloutXml(tb, 457200, y, SLIDE_W - 914400); const block = calloutXml(tb, 457200, y, SLIDE_W - 914400);
content += block.xml; content += block.xml;
y += block.height + CALL_OUT_GAP; y += block.height + CALL_OUT_GAP;
} }
const descReserve = step.descriptionText ? descriptionHeight(step.descriptionText) + 120000 : 0; content += textBox(457200, titleY, SLIDE_W - 914400, TITLE_H,
const restReserve = restBlocks.reduce((sum, tb) => sum + calloutHeight(tb) + CALL_OUT_GAP, 0); para(`${step.number}. ${step.title || 'Untitled step'}`, { size: 2600, bold: true }));
content += rectShape(457200, titleRuleY, 2400000, 12000, '2563EB');
y = contentY;
for (const tb of afterTitle) {
const block = calloutXml(tb, 457200, y, SLIDE_W - 914400);
content += block.xml;
y += block.height + CALL_OUT_GAP;
}
for (const tb of beforeDescription) {
const block = calloutXml(tb, 457200, y, SLIDE_W - 914400);
content += block.xml;
y += block.height + CALL_OUT_GAP;
}
if (step.descriptionText) {
const descH = descriptionHeight(step.descriptionText);
content += textBox(457200, y, SLIDE_W - 914400, Math.max(360000, descH || 420000),
para(step.descriptionText.slice(0, 300), { size: 1400, color: '374151' }));
y += Math.max(360000, descH || 420000) + 90000;
}
for (const tb of afterDescription) {
const block = calloutXml(tb, 457200, y, SLIDE_W - 914400);
content += block.xml;
y += block.height + CALL_OUT_GAP;
}
for (const tb of beforeImage) {
const block = calloutXml(tb, 457200, y, SLIDE_W - 914400);
content += block.xml;
y += block.height + CALL_OUT_GAP;
}
const postImageReserve = afterImage.reduce((sum, tb) => sum + calloutHeight(tb) + CALL_OUT_GAP, 0);
const rels = []; const rels = [];
const media = []; const media = [];
@@ -236,23 +276,17 @@ function exportPptx(ast, outDir, template = {}) {
media.push({ name, data: encodePng(img) }); media.push({ name, data: encodePng(img) });
const relId = 2; // rId1 = layout, rId2 = image const relId = 2; // rId1 = layout, rId2 = image
rels.push({ id: relId, name }); rels.push({ id: relId, name });
// Fit image into a centered region below the title. // Fit image into the remaining centered region before the trailing blocks.
const maxW = SLIDE_W - 1219200; const maxW = SLIDE_W - 1219200;
const maxH = Math.max(0, SLIDE_H - y - descReserve - restReserve - 100000); const maxH = Math.max(0, SLIDE_H - y - postImageReserve - 100000);
let w = img.width * EMU_PER_PX, h = img.height * EMU_PER_PX; let w = img.width * EMU_PER_PX, h = img.height * EMU_PER_PX;
const scale = Math.min(maxW / w, maxH / h, 1); const scale = Math.min(maxW / w, maxH / h, 1);
w = Math.round(w * scale); h = Math.round(h * scale); w = Math.round(w * scale); h = Math.round(h * scale);
content += picture(relId, Math.round((SLIDE_W - w) / 2), y, w, h); content += picture(relId, Math.round((SLIDE_W - w) / 2), y, w, h);
y += h + 100000; y += h + 100000;
} }
if (step.descriptionText) {
const descH = Math.max(360000, descReserve || 420000);
content += textBox(457200, Math.max(y, SLIDE_H - CONTENT_FOOTER_Y - descH), SLIDE_W - 914400, descH,
para(step.descriptionText.slice(0, 300), { size: 1400, color: '374151' }));
y = Math.max(y, SLIDE_H - CONTENT_FOOTER_Y - descH) + descH + 90000;
}
for (const tb of restBlocks) { for (const tb of afterImage) {
const block = calloutXml(tb, 457200, y, SLIDE_W - 914400); const block = calloutXml(tb, 457200, y, SLIDE_W - 914400);
content += block.xml; content += block.xml;
y += block.height + CALL_OUT_GAP; y += block.height + CALL_OUT_GAP;
+137 -10
View File
@@ -1,16 +1,23 @@
{ {
"name": "stepforge", "name": "stepforge",
"version": "0.1.0", "version": "0.3.2",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "stepforge", "name": "stepforge",
"version": "0.1.0", "version": "0.3.2",
"license": "MPL-2.0", "license": "MPL-2.0",
"dependencies": {
"@tesseract.js-data/eng": "^1.0.0",
"tesseract.js": "^7.0.0"
},
"devDependencies": { "devDependencies": {
"electron": "^41.7.1", "electron": "^41.7.1",
"electron-builder": "^26.15.2" "electron-builder": "^26.15.2"
},
"engines": {
"node": ">=22.12.0"
} }
}, },
"node_modules/@electron/asar": { "node_modules/@electron/asar": {
@@ -624,6 +631,12 @@
"node": ">=10" "node": ">=10"
} }
}, },
"node_modules/@tesseract.js-data/eng": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/@tesseract.js-data/eng/-/eng-1.0.0.tgz",
"integrity": "sha512-mbTumm6KQPUHyzTPQaF3ObXYnx0SqqfV2nabqFVQBwD6Kl7PhGSLSzOlfFTWy0P3BjghaSKA2W9GB19Jk+ZcTg==",
"license": "MIT"
},
"node_modules/@types/cacheable-request": { "node_modules/@types/cacheable-request": {
"version": "6.0.3", "version": "6.0.3",
"resolved": "https://registry.npmjs.org/@types/cacheable-request/-/cacheable-request-6.0.3.tgz", "resolved": "https://registry.npmjs.org/@types/cacheable-request/-/cacheable-request-6.0.3.tgz",
@@ -1057,6 +1070,12 @@
"dev": true, "dev": true,
"license": "MIT" "license": "MIT"
}, },
"node_modules/bmp-js": {
"version": "0.1.0",
"resolved": "https://registry.npmjs.org/bmp-js/-/bmp-js-0.1.0.tgz",
"integrity": "sha512-vHdS19CnY3hwiNdkaqk93DvjVLfbEcI8mys4UjuWrlX1haDmroo8o4xCzh4wD6DGV6HxRCyauwhHRqMTfERtjw==",
"license": "MIT"
},
"node_modules/boolean": { "node_modules/boolean": {
"version": "3.2.0", "version": "3.2.0",
"resolved": "https://registry.npmjs.org/boolean/-/boolean-3.2.0.tgz", "resolved": "https://registry.npmjs.org/boolean/-/boolean-3.2.0.tgz",
@@ -2121,17 +2140,17 @@
} }
}, },
"node_modules/form-data": { "node_modules/form-data": {
"version": "4.0.5", "version": "4.0.6",
"resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.5.tgz", "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.6.tgz",
"integrity": "sha512-8RipRLol37bNs2bhoV67fiTEvdTrbMUYcFTiy3+wuuOnUog2QBHCZWXDRijWQfAkhBj2Uf5UnVaiWwA5vdd82w==", "integrity": "sha512-vKatAh4SlVfgbv+YtmhiRjhEMJsYpsG1Y2rMQtR+SVSbytsSD1YGzDIcrAJmdFec88u/+VoGmxnl+80gL1tRCQ==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"asynckit": "^0.4.0", "asynckit": "^0.4.0",
"combined-stream": "^1.0.8", "combined-stream": "^1.0.8",
"es-set-tostringtag": "^2.1.0", "es-set-tostringtag": "^2.1.0",
"hasown": "^2.0.2", "hasown": "^2.0.4",
"mime-types": "^2.1.12" "mime-types": "^2.1.35"
}, },
"engines": { "engines": {
"node": ">= 6" "node": ">= 6"
@@ -2512,6 +2531,12 @@
"node": ">= 14" "node": ">= 14"
} }
}, },
"node_modules/idb-keyval": {
"version": "6.2.5",
"resolved": "https://registry.npmjs.org/idb-keyval/-/idb-keyval-6.2.5.tgz",
"integrity": "sha512-eKQkTnS0relYsSOYomx8ozIbmdsQCKUdhyuIaQ2DZgKuaxtyQQMkyD/wlnQN32pO3yutN1b1L8uqwcDKaJd7/Q==",
"license": "Apache-2.0"
},
"node_modules/inflight": { "node_modules/inflight": {
"version": "1.0.6", "version": "1.0.6",
"resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz", "resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz",
@@ -2541,6 +2566,12 @@
"node": ">=8" "node": ">=8"
} }
}, },
"node_modules/is-url": {
"version": "1.2.4",
"resolved": "https://registry.npmjs.org/is-url/-/is-url-1.2.4.tgz",
"integrity": "sha512-ITvGim8FhRiYe4IQ5uHSkj7pVaPDrCTkNd3yq3cV7iZAcJdHTUMPMEHcqSOy9xZ9qFenQCvi+2wjH9a1nXqHww==",
"license": "MIT"
},
"node_modules/isarray": { "node_modules/isarray": {
"version": "1.0.0", "version": "1.0.0",
"resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz", "resolved": "https://registry.npmjs.org/isarray/-/isarray-1.0.0.tgz",
@@ -2903,6 +2934,26 @@
"node": ">=10" "node": ">=10"
} }
}, },
"node_modules/node-fetch": {
"version": "2.7.0",
"resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-2.7.0.tgz",
"integrity": "sha512-c4FRfUm/dbcWZ7U+1Wq0AwCyFL+3nt2bEw05wfxSz+DWpWsitgmSgYmy2dQdWyKC1694ELPqMs/YzUSNozLt8A==",
"license": "MIT",
"dependencies": {
"whatwg-url": "^5.0.0"
},
"engines": {
"node": "4.x || >=6.0.0"
},
"peerDependencies": {
"encoding": "^0.1.0"
},
"peerDependenciesMeta": {
"encoding": {
"optional": true
}
}
},
"node_modules/node-gyp": { "node_modules/node-gyp": {
"version": "12.4.0", "version": "12.4.0",
"resolved": "https://registry.npmjs.org/node-gyp/-/node-gyp-12.4.0.tgz", "resolved": "https://registry.npmjs.org/node-gyp/-/node-gyp-12.4.0.tgz",
@@ -3024,6 +3075,15 @@
"wrappy": "1" "wrappy": "1"
} }
}, },
"node_modules/opencollective-postinstall": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/opencollective-postinstall/-/opencollective-postinstall-2.0.3.tgz",
"integrity": "sha512-8AV/sCtuzUeTo8gQK5qDZzARrulB3egtLzFgteqB2tcT4Mw7B8Kt7JcDHmltjz6FOAHsvTevk70gZEbhM4ZS9Q==",
"license": "MIT",
"bin": {
"opencollective-postinstall": "index.js"
}
},
"node_modules/p-cancelable": { "node_modules/p-cancelable": {
"version": "2.1.1", "version": "2.1.1",
"resolved": "https://registry.npmjs.org/p-cancelable/-/p-cancelable-2.1.1.tgz", "resolved": "https://registry.npmjs.org/p-cancelable/-/p-cancelable-2.1.1.tgz",
@@ -3314,6 +3374,12 @@
"util-deprecate": "~1.0.1" "util-deprecate": "~1.0.1"
} }
}, },
"node_modules/regenerator-runtime": {
"version": "0.13.11",
"resolved": "https://registry.npmjs.org/regenerator-runtime/-/regenerator-runtime-0.13.11.tgz",
"integrity": "sha512-kY1AZVr2Ra+t+piVaJ4gxaFaReZVH40AKNo7UCX6W+dEwBo/2oZJzqfuN1qLq1oL45o56cPaTXELwrTh8Fpggg==",
"license": "MIT"
},
"node_modules/require-directory": { "node_modules/require-directory": {
"version": "2.1.1", "version": "2.1.1",
"resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz", "resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz",
@@ -3728,6 +3794,30 @@
"node": ">= 10.0.0" "node": ">= 10.0.0"
} }
}, },
"node_modules/tesseract.js": {
"version": "7.0.0",
"resolved": "https://registry.npmjs.org/tesseract.js/-/tesseract.js-7.0.0.tgz",
"integrity": "sha512-exPBkd+z+wM1BuMkx/Bjv43OeLBxhL5kKWsz/9JY+DXcXdiBjiAch0V49QR3oAJqCaL5qURE0vx9Eo+G5YE7mA==",
"hasInstallScript": true,
"license": "Apache-2.0",
"dependencies": {
"bmp-js": "^0.1.0",
"idb-keyval": "^6.2.0",
"is-url": "^1.2.4",
"node-fetch": "^2.6.9",
"opencollective-postinstall": "^2.0.3",
"regenerator-runtime": "^0.13.3",
"tesseract.js-core": "^7.0.0",
"wasm-feature-detect": "^1.8.0",
"zlibjs": "^0.3.1"
}
},
"node_modules/tesseract.js-core": {
"version": "7.0.0",
"resolved": "https://registry.npmjs.org/tesseract.js-core/-/tesseract.js-core-7.0.0.tgz",
"integrity": "sha512-WnNH518NzmbSq9zgTPeoF8c+xmilS8rFIl1YKbk/ptuuc7p6cLNELNuPAzcmsYw450ca6bLa8j3t0VAtq435Vw==",
"license": "Apache-2.0"
},
"node_modules/tiny-async-pool": { "node_modules/tiny-async-pool": {
"version": "1.3.0", "version": "1.3.0",
"resolved": "https://registry.npmjs.org/tiny-async-pool/-/tiny-async-pool-1.3.0.tgz", "resolved": "https://registry.npmjs.org/tiny-async-pool/-/tiny-async-pool-1.3.0.tgz",
@@ -3785,6 +3875,12 @@
"tmp": "^0.2.0" "tmp": "^0.2.0"
} }
}, },
"node_modules/tr46": {
"version": "0.0.3",
"resolved": "https://registry.npmjs.org/tr46/-/tr46-0.0.3.tgz",
"integrity": "sha512-N3WMsuqV66lT30CrXNbEjx4GEwlow3v6rr4mCcv6prnfwhS01rkgyFdjPNBYd9br7LpXV1+Emh01fHnq2Gdgrw==",
"license": "MIT"
},
"node_modules/truncate-utf8-bytes": { "node_modules/truncate-utf8-bytes": {
"version": "1.0.2", "version": "1.0.2",
"resolved": "https://registry.npmjs.org/truncate-utf8-bytes/-/truncate-utf8-bytes-1.0.2.tgz", "resolved": "https://registry.npmjs.org/truncate-utf8-bytes/-/truncate-utf8-bytes-1.0.2.tgz",
@@ -3817,9 +3913,9 @@
} }
}, },
"node_modules/undici": { "node_modules/undici": {
"version": "6.26.0", "version": "6.27.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-6.26.0.tgz", "resolved": "https://registry.npmjs.org/undici/-/undici-6.27.0.tgz",
"integrity": "sha512-4yqz8a3n5HmGTlsbADNtr/dJlhkh/55Rq798G6ibiULcXbDtaLpTl1pvdqcbFfeoj3iSi52lePFM7h9H21cw/A==", "integrity": "sha512-YmfV3YnEDzXRC5lZ2jWtWWHKGUm1zIt8AhesR1tens+HTNv+YZlN/dp6G727LOvMJ8xjP9Be7Y2Sdr96LDm+pg==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"engines": { "engines": {
@@ -3909,6 +4005,12 @@
"dev": true, "dev": true,
"license": "MIT" "license": "MIT"
}, },
"node_modules/wasm-feature-detect": {
"version": "1.8.0",
"resolved": "https://registry.npmjs.org/wasm-feature-detect/-/wasm-feature-detect-1.8.0.tgz",
"integrity": "sha512-zksaLKM2fVlnB5jQQDqKXXwYHLQUVH9es+5TOOHwGOVJOCeRBCiPjwSg+3tN2AdTCzjgli4jijCH290kXb/zWQ==",
"license": "Apache-2.0"
},
"node_modules/webcrypto-core": { "node_modules/webcrypto-core": {
"version": "1.9.2", "version": "1.9.2",
"resolved": "https://registry.npmjs.org/webcrypto-core/-/webcrypto-core-1.9.2.tgz", "resolved": "https://registry.npmjs.org/webcrypto-core/-/webcrypto-core-1.9.2.tgz",
@@ -3923,6 +4025,22 @@
"tslib": "^2.8.1" "tslib": "^2.8.1"
} }
}, },
"node_modules/webidl-conversions": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-3.0.1.tgz",
"integrity": "sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==",
"license": "BSD-2-Clause"
},
"node_modules/whatwg-url": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/whatwg-url/-/whatwg-url-5.0.0.tgz",
"integrity": "sha512-saE57nupxk6v3HY35+jzBwYa0rKSy0XR8JSxZPwgLr7ys0IBzhGviA1/TUGJLmSVqs8pb9AnvICXEuOHLprYTw==",
"license": "MIT",
"dependencies": {
"tr46": "~0.0.3",
"webidl-conversions": "^3.0.0"
}
},
"node_modules/which": { "node_modules/which": {
"version": "5.0.0", "version": "5.0.0",
"resolved": "https://registry.npmjs.org/which/-/which-5.0.0.tgz", "resolved": "https://registry.npmjs.org/which/-/which-5.0.0.tgz",
@@ -4043,6 +4161,15 @@
"funding": { "funding": {
"url": "https://github.com/sponsors/sindresorhus" "url": "https://github.com/sponsors/sindresorhus"
} }
},
"node_modules/zlibjs": {
"version": "0.3.1",
"resolved": "https://registry.npmjs.org/zlibjs/-/zlibjs-0.3.1.tgz",
"integrity": "sha512-+J9RrgTKOmlxFSDHo0pI1xM6BLVUv+o0ZT9ANtCxGkjIVCCUdx9alUF8Gm+dGLKbkkkidWIHFDZHDMpfITt4+w==",
"license": "MIT",
"engines": {
"node": "*"
}
} }
} }
} }
+10 -2
View File
@@ -1,20 +1,28 @@
{ {
"name": "stepforge", "name": "stepforge",
"version": "0.1.0", "version": "0.3.2",
"buildVersion": "0.3.2.1",
"description": "Fully offline desktop tool for capturing, annotating, and exporting step-by-step guides.", "description": "Fully offline desktop tool for capturing, annotating, and exporting step-by-step guides.",
"main": "app/main.js", "main": "app/main.js",
"author": "StepForge [email protected]", "author": "StepForge [email protected]",
"license": "MPL-2.0", "license": "MPL-2.0",
"private": true, "private": true,
"engines": {
"node": ">=22.12.0"
},
"scripts": { "scripts": {
"start": "node scripts/start-electron.js", "start": "node scripts/start-electron.js",
"test": "node --test tests/unit/", "test": "node scripts/run-unit-tests.js",
"sample": "node scripts/make-sample-guide.js", "sample": "node scripts/make-sample-guide.js",
"package:windows": "node scripts/package-windows.js", "package:windows": "node scripts/package-windows.js",
"build": "bash scripts/build-release.sh", "build": "bash scripts/build-release.sh",
"verify": "bash scripts/verify.sh", "verify": "bash scripts/verify.sh",
"bootstrap": "bash scripts/bootstrap-offline.sh" "bootstrap": "bash scripts/bootstrap-offline.sh"
}, },
"dependencies": {
"@tesseract.js-data/eng": "^1.0.0",
"tesseract.js": "^7.0.0"
},
"devDependencies": { "devDependencies": {
"electron": "^41.7.1", "electron": "^41.7.1",
"electron-builder": "^26.15.2" "electron-builder": "^26.15.2"
+1 -1
View File
@@ -20,5 +20,5 @@ fi
node - <<'NODE' node - <<'NODE'
const pkg = require('./package.json'); const pkg = require('./package.json');
console.log(`StepForge ${pkg.version} bootstrap OK`); console.log(`StepForge ${pkg.buildVersion || pkg.version} bootstrap OK`);
NODE NODE
+4 -1
View File
@@ -70,6 +70,7 @@ for (const rel of walk(examplesRoot, examplesRoot)) {
} }
const pkg = require(path.join(rootDir, 'package.json')); const pkg = require(path.join(rootDir, 'package.json'));
const buildVersion = pkg.buildVersion || pkg.version;
const { execSync } = require('node:child_process'); const { execSync } = require('node:child_process');
function toolAvailable(cmd) { function toolAvailable(cmd) {
@@ -88,7 +89,8 @@ const toolRows = Object.entries(tools)
const report = `# StepForge Build Report const report = `# StepForge Build Report
Version: ${pkg.version} Build version: ${buildVersion}
Package version: ${pkg.version}
Generated: ${new Date().toISOString()} Generated: ${new Date().toISOString()}
Host: ${process.platform} ${process.arch} (node ${process.version}) Host: ${process.platform} ${process.arch} (node ${process.version})
@@ -133,6 +135,7 @@ fs.writeFileSync(manifestFile, JSON.stringify({
version: 1, version: 1,
generatedAt: new Date().toISOString(), generatedAt: new Date().toISOString(),
packageVersion: pkg.version, packageVersion: pkg.version,
buildVersion,
files, files,
}, null, 2) + '\n'); }, null, 2) + '\n');
NODE NODE
+79
View File
@@ -0,0 +1,79 @@
'use strict';
// Hard prerequisite check for the supported Node toolchain.
//
// The locked dependency graph (notably the electron-builder packaging
// toolchain) requires Node >= 22.12. Older Nodes fail late with confusing
// errors (ERR_REQUIRE_ESM deep inside dependencies) instead of a clear
// message, so every entry point calls this first.
const fs = require('node:fs');
const path = require('node:path');
function parseVersion(version) {
const match = /^v?(\d+)\.(\d+)\.(\d+)/.exec(String(version).trim());
if (!match) return null;
return [Number(match[1]), Number(match[2]), Number(match[3])];
}
function compareVersions(a, b) {
for (let i = 0; i < 3; i += 1) {
if (a[i] !== b[i]) return a[i] < b[i] ? -1 : 1;
}
return 0;
}
function requiredNodeVersion(projectRoot = path.join(__dirname, '..')) {
const pkg = JSON.parse(fs.readFileSync(path.join(projectRoot, 'package.json'), 'utf8'));
const range = pkg.engines && pkg.engines.node ? String(pkg.engines.node) : null;
if (!range) return null;
const match = /(\d+\.\d+\.\d+)/.exec(range);
return match ? match[1] : null;
}
function checkNodeVersion({
currentVersion = process.versions.node,
projectRoot = path.join(__dirname, '..'),
} = {}) {
const required = requiredNodeVersion(projectRoot);
if (!required) return { ok: true, required: null, current: currentVersion };
const current = parseVersion(currentVersion);
const minimum = parseVersion(required);
if (!current || !minimum) return { ok: true, required, current: currentVersion };
return {
ok: compareVersions(current, minimum) >= 0,
required,
current: currentVersion,
};
}
function assertSupportedNode(options = {}) {
const result = checkNodeVersion(options);
if (result.ok) return result;
const message = [
`StepForge requires Node ${result.required} or newer; this is Node ${result.current}.`,
'',
'Install the pinned toolchain (see .nvmrc) and reinstall dependencies:',
'',
' nvm install && nvm use # or install Node 22 LTS another way',
' npm ci',
'',
'Older Nodes fail unpredictably inside the packaging dependency graph,',
'so this check stops early instead.',
].join('\n');
const error = new Error(message);
error.code = 'STEPFORGE_UNSUPPORTED_NODE';
throw error;
}
module.exports = {
assertSupportedNode,
checkNodeVersion,
compareVersions,
parseVersion,
requiredNodeVersion,
};
+118 -98
View File
@@ -1,6 +1,14 @@
'use strict'; 'use strict';
const { spawnSync } = require('node:child_process'); // Diagnostics-only Electron launcher helpers.
//
// This module never installs, rebuilds, or repairs dependencies at runtime.
// The only supported dependency installation path is `npm ci` on the pinned
// Node toolchain (see .nvmrc / package.json engines). A desktop launcher that
// mutates node_modules silently drifts away from package-lock.json and can
// download code at runtime; when the runtime is missing we fail with
// actionable diagnostics instead.
const fs = require('node:fs'); const fs = require('node:fs');
const path = require('node:path'); const path = require('node:path');
@@ -10,6 +18,11 @@ const ELECTRON_SKIP_ENV_KEYS = [
'NPM_CONFIG_ELECTRON_SKIP_BINARY_DOWNLOAD', 'NPM_CONFIG_ELECTRON_SKIP_BINARY_DOWNLOAD',
]; ];
const NPM_IGNORE_SCRIPTS_ENV_KEYS = [
'npm_config_ignore_scripts',
'NPM_CONFIG_IGNORE_SCRIPTS',
];
function resolveElectronPackageRoot() { function resolveElectronPackageRoot() {
try { try {
return path.dirname(require.resolve('electron/package.json')); return path.dirname(require.resolve('electron/package.json'));
@@ -48,10 +61,92 @@ function sanitizeElectronEnv(baseEnv = process.env) {
for (const key of ELECTRON_SKIP_ENV_KEYS) { for (const key of ELECTRON_SKIP_ENV_KEYS) {
delete env[key]; delete env[key];
} }
for (const key of NPM_IGNORE_SCRIPTS_ENV_KEYS) {
delete env[key];
}
return env; return env;
} }
// True only when the caller has explicitly marked this as a development or
// CI environment where launching without the Chromium sandbox is acceptable.
function noSandboxExplicitlyAllowed(env = process.env) {
return env.STEPFORGE_ALLOW_NO_SANDBOX === '1' || env.ELECTRON_DISABLE_SANDBOX === '1';
}
function sandboxHelperUsable(electronPath, statSync = fs.statSync) {
if (!electronPath) return false;
const helperPath = path.join(path.dirname(electronPath), 'chrome-sandbox');
try {
const stat = statSync(helperPath);
return stat.uid === 0 && Boolean(stat.mode & 0o4000);
} catch {
return false;
}
}
// Decide how to launch on Linux with respect to the Chromium sandbox.
// { args: [] } sandbox is available, launch normally
// { args: ['--no-sandbox'] } explicitly allowed dev/CI launch
// throws sandbox unavailable and not explicitly
// allowed: refuse to normalize an
// unsandboxed launch, explain how to fix it
function linuxSandboxLaunchArgs({
electronPath,
platform = process.platform,
statSync = fs.statSync,
env = process.env,
userNamespaces = userNamespacesAvailable,
} = {}) {
if (platform !== 'linux') return [];
// Modern kernels with unprivileged user namespaces do not need the setuid
// helper; Chromium falls back to the namespace sandbox on its own. The
// setuid helper check below covers kernels where that is disabled.
if (sandboxHelperUsable(electronPath, statSync)) return [];
if (userNamespaces()) return [];
if (noSandboxExplicitlyAllowed(env)) return ['--no-sandbox'];
const helperPath = electronPath
? path.join(path.dirname(electronPath), 'chrome-sandbox')
: '<node_modules/electron/dist>/chrome-sandbox';
throw new Error(
[
'The Chromium sandbox is not available on this system, and StepForge',
'refuses to silently launch unsandboxed.',
'',
'Fix one of the following:',
` 1. Make the setuid sandbox helper usable:`,
` sudo chown root:root "${helperPath}"`,
` sudo chmod 4755 "${helperPath}"`,
' 2. Enable unprivileged user namespaces (kernel/sysctl dependent):',
' sudo sysctl -w kernel.unprivileged_userns_clone=1',
'',
'For development or CI only, you may explicitly opt in to an',
'unsandboxed launch with STEPFORGE_ALLOW_NO_SANDBOX=1.',
].join('\n')
);
}
function userNamespacesAvailable() {
try {
// Debian/Ubuntu specific knob; absent elsewhere (treated as enabled).
const knob = '/proc/sys/kernel/unprivileged_userns_clone';
if (fs.existsSync(knob)) {
return fs.readFileSync(knob, 'utf8').trim() === '1';
}
// Ubuntu 23.10+ AppArmor restriction on unprivileged user namespaces.
const apparmorKnob = '/proc/sys/kernel/apparmor_restrict_unprivileged_userns';
if (fs.existsSync(apparmorKnob)) {
return fs.readFileSync(apparmorKnob, 'utf8').trim() === '0';
}
return fs.existsSync('/proc/self/ns/user');
} catch {
return false;
}
}
function electronBinaryCandidates({ packageRoot, distDir, platform }) { function electronBinaryCandidates({ packageRoot, distDir, platform }) {
const candidatePaths = []; const candidatePaths = [];
const pathHint = packageRoot ? readElectronPathHint(packageRoot) : null; const pathHint = packageRoot ? readElectronPathHint(packageRoot) : null;
@@ -67,84 +162,21 @@ function electronBinaryCandidates({ packageRoot, distDir, platform }) {
return candidatePaths; return candidatePaths;
} }
function runNpmRebuild({
packageRoot,
npmExecPath = process.env.npm_execpath || null,
npmNodeExecPath = process.env.npm_node_execpath || process.execPath,
}) {
if (!npmExecPath) {
return false;
}
const result = spawnSync(
npmNodeExecPath,
[npmExecPath, 'rebuild', 'electron', '--force', '--foreground-scripts'],
{
cwd: packageRoot,
env: sanitizeElectronEnv(),
stdio: 'inherit',
}
);
if (result.error) {
throw result.error;
}
if (result.signal) {
throw new Error(`Electron repair was interrupted by ${result.signal}`);
}
if (result.status !== 0) {
throw new Error(`Electron rebuild failed with exit code ${result.status ?? 1}`);
}
return true;
}
function repairElectronInstall({
packageRoot,
}) {
const installScript = path.join(packageRoot, 'install.js');
if (!fs.existsSync(installScript)) {
return false;
}
const result = spawnSync(process.execPath, [installScript], {
cwd: packageRoot,
env: sanitizeElectronEnv(),
stdio: 'inherit',
});
if (result.error) {
throw result.error;
}
if (result.signal) {
throw new Error(`Electron repair was interrupted by ${result.signal}`);
}
if (result.status !== 0) {
throw new Error(`Electron repair failed with exit code ${result.status ?? 1}`);
}
return true;
}
function buildMissingElectronError({ packageRoot, distDir, candidatePaths }) { function buildMissingElectronError({ packageRoot, distDir, candidatePaths }) {
const tried = candidatePaths.map((candidate) => ` - ${candidate}`).join('\n'); const tried = (candidatePaths || []).map((candidate) => ` - ${candidate}`).join('\n');
return [ return [
'Electron could not be started because the desktop runtime is missing.', 'Electron could not be started because the desktop runtime is missing.',
'', '',
`Looked under: ${packageRoot}`, `Looked under: ${packageRoot || '(electron package not installed)'}`,
`Expected the binary in: ${distDir}`, `Expected the binary in: ${distDir || '(unknown)'}`,
'', '',
'Try reinstalling dependencies from the repo root:', 'StepForge never installs dependencies at runtime. Reinstall them from',
'the repo root on the pinned Node toolchain (see .nvmrc):',
'', '',
' npm install', ' npm ci',
' npm rebuild electron --force --foreground-scripts',
' make sure ELECTRON_SKIP_BINARY_DOWNLOAD is not set',
'', '',
'If that does not help, delete node_modules/electron and install again.', 'Make sure ELECTRON_SKIP_BINARY_DOWNLOAD is not set while installing.',
'If the problem persists, delete node_modules entirely and run npm ci again.',
'', '',
'Searched:', 'Searched:',
tried, tried,
@@ -153,54 +185,42 @@ function buildMissingElectronError({ packageRoot, distDir, candidatePaths }) {
function resolveElectronBinary({ function resolveElectronBinary({
packageRoot = resolveElectronPackageRoot(), packageRoot = resolveElectronPackageRoot(),
projectRoot = process.cwd(),
platform = process.platform, platform = process.platform,
overrideDistPath = process.env.ELECTRON_OVERRIDE_DIST_PATH || null, overrideDistPath = process.env.ELECTRON_OVERRIDE_DIST_PATH || null,
} = {}) { } = {}) {
if (!packageRoot) {
const conventionalRoot = path.join(projectRoot, 'node_modules', 'electron');
if (fs.existsSync(path.join(conventionalRoot, 'package.json'))) {
packageRoot = conventionalRoot;
}
}
if (!packageRoot && !overrideDistPath) { if (!packageRoot && !overrideDistPath) {
throw new Error( throw new Error(
'Electron could not be started because node_modules/electron is not installed.\n\n' + 'Electron could not be started because node_modules/electron is not installed.\n\n' +
'Run `npm install` from the repo root, then try `npm start` again.' 'StepForge never installs dependencies at runtime. Run `npm ci` from the\n' +
'repo root on the pinned Node toolchain (see .nvmrc), then try again.'
); );
} }
const distDir = overrideDistPath || path.join(packageRoot, 'dist'); const distDir = overrideDistPath || path.join(packageRoot, 'dist');
const candidatePaths = electronBinaryCandidates({ packageRoot, distDir, platform }); const candidatePaths = electronBinaryCandidates({ packageRoot, distDir, platform });
const resolved = candidatePaths.find((candidate) => fs.existsSync(candidate)); const resolved = candidatePaths.find((candidate) => fs.existsSync(candidate));
if (!resolved) { if (resolved) {
if (packageRoot) { return resolved;
if (runNpmRebuild({ packageRoot })) {
const rebuilt = electronBinaryCandidates({ packageRoot, distDir, platform }).find((candidate) =>
fs.existsSync(candidate)
);
if (rebuilt) {
return rebuilt;
}
}
if (repairElectronInstall({ packageRoot })) {
const repaired = electronBinaryCandidates({ packageRoot, distDir, platform }).find((candidate) =>
fs.existsSync(candidate)
);
if (repaired) {
return repaired;
}
}
} }
throw new Error(buildMissingElectronError({ packageRoot, distDir, candidatePaths })); throw new Error(buildMissingElectronError({ packageRoot, distDir, candidatePaths }));
} }
return resolved;
}
module.exports = { module.exports = {
buildMissingElectronError, buildMissingElectronError,
electronBinaryCandidates, electronBinaryCandidates,
readElectronPathHint, readElectronPathHint,
repairElectronInstall,
runNpmRebuild,
sanitizeElectronEnv, sanitizeElectronEnv,
noSandboxExplicitlyAllowed,
linuxSandboxLaunchArgs,
resolveElectronBinary, resolveElectronBinary,
resolveElectronPackageRoot, resolveElectronPackageRoot,
platformBinaryCandidates, platformBinaryCandidates,
+15 -2
View File
@@ -3,7 +3,7 @@
set -euo pipefail set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
VERSION="$(node -p "require('${ROOT_DIR}/package.json').version" 2>/dev/null || echo 0.0.0)" VERSION="$(node -p "const pkg=require('${ROOT_DIR}/package.json'); pkg.buildVersion || pkg.version" 2>/dev/null || echo 0.0.0)"
OUT_DIR="${STEPFORGE_PACKAGE_DIR:-$ROOT_DIR/build/artifacts}" OUT_DIR="${STEPFORGE_PACKAGE_DIR:-$ROOT_DIR/build/artifacts}"
mkdir -p "$OUT_DIR" mkdir -p "$OUT_DIR"
WORK_DIR="$(mktemp -d "${OUT_DIR%/}/.pkg.XXXXXX")" WORK_DIR="$(mktemp -d "${OUT_DIR%/}/.pkg.XXXXXX")"
@@ -46,8 +46,21 @@ fi
cat > "$WORK_DIR/usr/bin/stepforge" <<'EOF' cat > "$WORK_DIR/usr/bin/stepforge" <<'EOF'
#!/usr/bin/env sh #!/usr/bin/env sh
APP_DIR=/opt/stepforge APP_DIR=/opt/stepforge
ELECTRON="$APP_DIR/node_modules/.bin/electron"
SANDBOX_HELPER="$APP_DIR/node_modules/electron/dist/chrome-sandbox"
cd "$APP_DIR" || exit 1 cd "$APP_DIR" || exit 1
exec "$APP_DIR/node_modules/.bin/electron" "$APP_DIR" "$@" if command -v stat >/dev/null 2>&1 && [ -e "$SANDBOX_HELPER" ]; then
helper_uid="$(stat -c '%u' "$SANDBOX_HELPER" 2>/dev/null || echo '')"
helper_mode="$(stat -c '%a' "$SANDBOX_HELPER" 2>/dev/null || echo '')"
if [ "$helper_uid" = "0" ] && [ -n "$helper_mode" ]; then
helper_mode_num=$((8#$helper_mode))
if [ $((helper_mode_num & 04000)) -ne 0 ]; then
exec "$ELECTRON" "$APP_DIR" "$@"
fi
fi
fi
printf '%s\n' '[stepforge] Electron sandbox helper is not configured for this install; starting with --no-sandbox' >&2
exec "$ELECTRON" --no-sandbox "$APP_DIR" "$@"
EOF EOF
chmod 0755 "$WORK_DIR/usr/bin/stepforge" chmod 0755 "$WORK_DIR/usr/bin/stepforge"
+5 -1
View File
@@ -10,6 +10,7 @@ const { build, Platform } = require('electron-builder');
const ROOT_DIR = path.resolve(__dirname, '..'); const ROOT_DIR = path.resolve(__dirname, '..');
const PACKAGE_JSON = require(path.join(ROOT_DIR, 'package.json')); const PACKAGE_JSON = require(path.join(ROOT_DIR, 'package.json'));
const APP_ID = 'com.stepforge.app'; const APP_ID = 'com.stepforge.app';
const BUILD_VERSION = PACKAGE_JSON.buildVersion || PACKAGE_JSON.version;
function findInstallerExe(dir) { function findInstallerExe(dir) {
if (!fs.existsSync(dir)) return null; if (!fs.existsSync(dir)) return null;
@@ -32,6 +33,7 @@ function createWindowsInstallerConfig(outputDir) {
return { return {
appId: APP_ID, appId: APP_ID,
productName: 'StepForge', productName: 'StepForge',
buildVersion: BUILD_VERSION,
directories: { directories: {
output: outputDir, output: outputDir,
}, },
@@ -52,6 +54,8 @@ function createWindowsInstallerConfig(outputDir) {
createDesktopShortcut: true, createDesktopShortcut: true,
createStartMenuShortcut: true, createStartMenuShortcut: true,
shortcutName: 'StepForge', shortcutName: 'StepForge',
artifactName: '${productName} Setup ${buildVersion}.${ext}',
include: 'build/installer.nsh',
}, },
}; };
} }
@@ -86,7 +90,7 @@ async function buildWindowsInstaller() {
const releaseInstaller = path.join(releaseDir, path.basename(builtInstaller)); const releaseInstaller = path.join(releaseDir, path.basename(builtInstaller));
fs.copyFileSync(builtInstaller, releaseInstaller); fs.copyFileSync(builtInstaller, releaseInstaller);
console.log(`StepForge ${PACKAGE_JSON.version} Windows installer written to ${releaseInstaller}`); console.log(`StepForge ${BUILD_VERSION} Windows installer written to ${releaseInstaller}`);
} }
if (require.main === module) { if (require.main === module) {
+51
View File
@@ -0,0 +1,51 @@
'use strict';
const fs = require('node:fs');
const path = require('node:path');
const { spawnSync } = require('node:child_process');
const { assertSupportedNode } = require('./check-node-version');
try {
assertSupportedNode();
} catch (error) {
console.error(error.message);
process.exit(1);
}
function collectTestFiles(rootDir) {
const files = [];
if (!fs.existsSync(rootDir)) return files;
const walk = (dir) => {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const full = path.join(dir, entry.name);
if (entry.isDirectory()) {
walk(full);
continue;
}
if (/\.(test|spec)\.(js|mjs|cjs)$/.test(entry.name)) files.push(full);
}
};
walk(rootDir);
files.sort((a, b) => a.localeCompare(b));
return files;
}
const root = path.join(process.cwd(), 'tests', 'unit');
const tests = collectTestFiles(root);
if (!tests.length) {
console.log('No unit test files found under tests/unit, skipping unit tests.');
process.exit(0);
}
const result = spawnSync(process.execPath, ['--test', ...tests], { stdio: 'inherit' });
if (result.error) {
console.error(result.error.message);
process.exit(result.status ?? 1);
}
process.exit(result.status ?? 0);
+52
View File
@@ -0,0 +1,52 @@
'use strict';
const fs = require('node:fs');
const path = require('node:path');
function readJson(file) {
return JSON.parse(fs.readFileSync(file, 'utf8'));
}
function writeJson(file, value) {
fs.writeFileSync(file, JSON.stringify(value, null, 2) + '\n');
}
function stampVersion(rootDir, version) {
if (!version || typeof version !== 'string') {
throw new Error('version is required');
}
const normalized = version.replace(/^v/i, '');
const parts = normalized.split('.');
const isFourPartBuild = parts.length === 4 && parts.every((part) => /^\d+$/.test(part));
const packageVersion = isFourPartBuild ? parts.slice(0, 3).join('.') : normalized;
const buildVersion = normalized;
const pkgPath = path.join(rootDir, 'package.json');
const pkg = readJson(pkgPath);
pkg.version = packageVersion;
pkg.buildVersion = buildVersion;
writeJson(pkgPath, pkg);
const lockPath = path.join(rootDir, 'package-lock.json');
if (!fs.existsSync(lockPath)) return;
const lock = readJson(lockPath);
lock.version = packageVersion;
if (lock.packages && lock.packages['']) {
lock.packages[''].version = packageVersion;
}
writeJson(lockPath, lock);
}
if (require.main === module) {
try {
const version = process.argv[2] || process.env.VERSION;
stampVersion(process.cwd(), version);
} catch (err) {
console.error(err && err.message ? err.message : err);
process.exitCode = 1;
}
}
module.exports = { stampVersion };
+23 -2
View File
@@ -3,18 +3,39 @@
const { spawn } = require('node:child_process'); const { spawn } = require('node:child_process');
const { resolveElectronBinary, sanitizeElectronEnv } = require('./electron-launcher'); const { assertSupportedNode } = require('./check-node-version');
const {
linuxSandboxLaunchArgs,
resolveElectronBinary,
sanitizeElectronEnv,
} = require('./electron-launcher');
let electronPath; let electronPath;
let sandboxArgs;
try { try {
assertSupportedNode();
electronPath = resolveElectronBinary(); electronPath = resolveElectronBinary();
sandboxArgs = linuxSandboxLaunchArgs({ electronPath });
} catch (error) { } catch (error) {
console.error(error && error.message ? error.message : error); console.error(error && error.message ? error.message : error);
process.exit(1); process.exit(1);
} }
const env = sanitizeElectronEnv(); const env = sanitizeElectronEnv();
if (sandboxArgs.includes('--no-sandbox')) {
console.warn(
'[stepforge] launching WITHOUT the Chromium sandbox (explicitly allowed via ' +
'STEPFORGE_ALLOW_NO_SANDBOX/ELECTRON_DISABLE_SANDBOX — development/CI only)'
);
}
const child = spawn(electronPath, ['.'], { // On Linux, prefer the native Ozone path when available and enable PipeWire-
// based screen capture so desktopCapturer can go through the XDG Desktop
// Portal on Wayland without affecting X11.
const extraArgs = process.platform === 'linux'
? ['--enable-features=WebRTCPipeWireCapturer', '--ozone-platform-hint=auto', ...sandboxArgs]
: [];
const child = spawn(electronPath, [...extraArgs, '.'], {
stdio: 'inherit', stdio: 'inherit',
env, env,
windowsHide: false, windowsHide: false,
+20 -6
View File
@@ -13,14 +13,22 @@
# arm: warmup click ignored, first armed click captured # arm: warmup click ignored, first armed click captured
# debounce: 4 of 4 (40ms burst collapses to 1, three 300ms clicks kept) # debounce: 4 of 4 (40ms burst collapses to 1, three 300ms clicks kept)
# #
# If the environment can't run a desktop capture at all (no display/stream), # Skip policy (kept honest on purpose): the ONLY allowed skip is the upfront
# the scenarios never print, so the check skips rather than failing CI. # absence of a display server, detected BEFORE launching. Once the app is
# launched, failing to reach the scenarios is a real failure — a startup
# crash (missing shared library, launcher bug) must never be reported as
# "no capture environment".
set -euo pipefail set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
cd "$ROOT_DIR" cd "$ROOT_DIR"
if [[ "$(uname -s)" == "Linux" && -z "${DISPLAY:-}" && -z "${WAYLAND_DISPLAY:-}" ]]; then
echo "click capture selftest SKIPPED: no display server (set DISPLAY or run under xvfb-run)"
exit 0
fi
TMP_ROOT="$(mktemp -d)" TMP_ROOT="$(mktemp -d)"
trap 'rm -rf "$TMP_ROOT"' EXIT trap 'rm -rf "$TMP_ROOT"' EXIT
@@ -30,11 +38,17 @@ STEPFORGE_DATA_DIR="$TMP_ROOT/data" STEPFORGE_CLICK_SELFTEST=1 \
timeout 120s npm start >"$LOG_FILE" 2>&1 timeout 120s npm start >"$LOG_FILE" 2>&1
set -e set -e
# The self-test always prints this first line once it begins; without it the # The self-test always prints this line once the app is up (the frame source
# app never reached the scenarios (couldn't launch / no capture environment). # is printed as a diagnostic even when no capture backend is available).
# Its absence means the app never started — that is a failure, not a skip.
if ! grep -q 'CLICK-SELFTEST source:' "$LOG_FILE"; then if ! grep -q 'CLICK-SELFTEST source:' "$LOG_FILE"; then
echo "click capture selftest SKIPPED (no capture environment on this host)" echo "click capture selftest FAILED: the app never reached the self-test scenarios" >&2
exit 0 if grep -Eq 'error while loading shared libraries' "$LOG_FILE"; then
echo "cause: Electron is missing system shared libraries on this host" >&2
fi
echo "----- startup output (last 40 lines) -----" >&2
tail -n 40 "$LOG_FILE" >&2
exit 1
fi fi
fail() { fail() {
+7
View File
@@ -7,6 +7,13 @@ set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
cd "$ROOT_DIR" cd "$ROOT_DIR"
# The only allowed skip is the upfront absence of a display server. Any
# failure after launch (missing shared library, crash) must fail the check.
if [[ "$(uname -s)" == "Linux" && -z "${DISPLAY:-}" && -z "${WAYLAND_DISPLAY:-}" ]]; then
echo "startup smoke SKIPPED: no display server (set DISPLAY or run under xvfb-run)"
exit 0
fi
TMP_ROOT="$(mktemp -d)" TMP_ROOT="$(mktemp -d)"
trap 'rm -rf "$TMP_ROOT"' EXIT trap 'rm -rf "$TMP_ROOT"' EXIT
+146
View File
@@ -0,0 +1,146 @@
'use strict';
const test = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const vm = require('node:vm');
test('capture worker requests the selected desktop source, not a plain camera stream', async () => {
const scriptPath = path.join(__dirname, '../../app/renderer/capture-worker.js');
const script = fs.readFileSync(scriptPath, 'utf8');
const mediaCalls = [];
const messages = [];
let onCommand = null;
let resolveStreamReady;
const streamReady = new Promise((resolve) => {
resolveStreamReady = resolve;
});
const context = {
console: {
error() {},
log() {},
warn() {},
},
captureWorkerBridge: {
onCommand(fn) {
onCommand = fn;
},
send(msg) {
messages.push(msg);
if (msg.type === 'stream-ready') resolveStreamReady();
},
},
StepForgeClickFrames: {
FrameRing: class {
constructor() {
this._frames = [];
}
push(frame) {
this._frames.push(frame);
}
frames() {
return [...this._frames];
}
latest() {
return this._frames[this._frames.length - 1] || null;
}
clear() {
this._frames = [];
}
},
selectFrameForClick() {
return null;
},
},
navigator: {
mediaDevices: {
async getUserMedia(constraints) {
mediaCalls.push(constraints);
return {
getTracks() {
return [{ stop() {} }];
},
};
},
async getDisplayMedia() {
throw new Error('unexpected getDisplayMedia call');
},
},
},
document: {
createElement(tag) {
assert.equal(tag, 'video');
return {
muted: false,
srcObject: null,
readyState: 2,
play: async () => {},
videoWidth: 1920,
videoHeight: 1080,
};
},
},
createImageBitmap: async () => ({ width: 1920, height: 1080, close() {} }),
setInterval: () => 1,
clearInterval: () => {},
OffscreenCanvas: class {
constructor(width, height) {
this.width = width;
this.height = height;
}
getContext() {
return { drawImage() {} };
}
async convertToBlob() {
return {
async arrayBuffer() {
return new ArrayBuffer(0);
},
};
}
},
};
vm.createContext(context);
vm.runInContext(script, context, { filename: scriptPath });
assert.equal(typeof onCommand, 'function', 'worker should register a command handler');
onCommand({
type: 'start-stream',
displayId: 7,
sourceId: 'screen:1:0',
display: {
bounds: { width: 1920, height: 1080 },
scaleFactor: 1,
},
sampleMs: 50,
});
await streamReady;
assert.equal(mediaCalls.length, 1);
const constraints = JSON.parse(JSON.stringify(mediaCalls[0]));
assert.deepEqual(constraints, {
audio: false,
video: {
mandatory: {
chromeMediaSource: 'desktop',
chromeMediaSourceId: 'screen:1:0',
minWidth: 1920,
maxWidth: 1920,
minHeight: 1080,
maxHeight: 1080,
},
},
});
assert.ok(messages.some((msg) => msg.type === 'stream-ready'));
});
+174 -6
View File
@@ -33,6 +33,22 @@ function makeService({ settings: settingsOverrides, screenApi } = {}) {
}); });
} }
test('XDG_SESSION_TYPE=x11 wins over a stray WAYLAND_DISPLAY value', () => {
const prevSessionType = process.env.XDG_SESSION_TYPE;
const prevWaylandDisplay = process.env.WAYLAND_DISPLAY;
try {
process.env.XDG_SESSION_TYPE = 'x11';
process.env.WAYLAND_DISPLAY = 'wayland-0';
const service = makeService();
assert.equal(service.onWayland(), false);
} finally {
if (prevSessionType === undefined) delete process.env.XDG_SESSION_TYPE;
else process.env.XDG_SESSION_TYPE = prevSessionType;
if (prevWaylandDisplay === undefined) delete process.env.WAYLAND_DISPLAY;
else process.env.WAYLAND_DISPLAY = prevWaylandDisplay;
}
});
// The raw/regular twin window plus margin: how long a test must wait for a // The raw/regular twin window plus margin: how long a test must wait for a
// held Linux raw press to fire when no coordinate twin arrives. // held Linux raw press to fire when no coordinate twin arrives.
const TWIN_FLUSH_MS = 60; const TWIN_FLUSH_MS = 60;
@@ -375,7 +391,7 @@ test('queued click captures preserve the original event time and button', async
assert.deepEqual(seen, [{ assert.deepEqual(seen, [{
trigger: 'click', trigger: 'click',
clickPos: { x: 7, y: 8 }, clickPos: { x: 7, y: 8 },
clickMeta: { at: 1770000000456, button: 'left' }, clickMeta: { at: 1770000000456, button: 'left', keyContext: { recentTyped: '', recentShortcut: '' } },
}]); }]);
}); });
@@ -596,6 +612,7 @@ test('armRecording warms while visible, then hides and arms the session', async
isVisible() { return this.visible; }, isVisible() { return this.visible; },
isMinimized() { return false; }, isMinimized() { return false; },
hide() { this.visible = false; }, hide() { this.visible = false; },
minimize() { this.visible = false; },
show() { this.visible = true; }, show() { this.visible = true; },
focus() {}, getTitle() { return 'StepForge'; }, focus() {}, getTitle() { return 'StepForge'; },
getBounds() { return { x: 0, y: 0, width: 800, height: 600 }; }, getBounds() { return { x: 0, y: 0, width: 800, height: 600 }; },
@@ -637,6 +654,7 @@ test('a slow recorder start still arms within the warmup cap', async () => {
isVisible() { return this.visible; }, isVisible() { return this.visible; },
isMinimized() { return false; }, isMinimized() { return false; },
hide() { this.visible = false; }, hide() { this.visible = false; },
minimize() { this.visible = false; },
show() { this.visible = true; }, show() { this.visible = true; },
focus() {}, getTitle() { return 'StepForge'; }, focus() {}, getTitle() { return 'StepForge'; },
getBounds() { return { x: 0, y: 0, width: 800, height: 600 }; }, getBounds() { return { x: 0, y: 0, width: 800, height: 600 }; },
@@ -718,6 +736,27 @@ test('hook coordinates are converted physical → DIP via screenToDipPoint when
assert.deepEqual(seen, [{ x: 640, y: 320 }]); assert.deepEqual(seen, [{ x: 640, y: 320 }]);
}); });
test('bogus Windows screenToDipPoint results fall back to display geometry', () => {
const service = makeService({
screenApi: {
screenToDipPoint: (p) => ({ x: p.x, y: p.y }), // raw physical point: wrong on scaled displays
getAllDisplays: () => [
{ id: 1, scaleFactor: 2, bounds: { x: 0, y: 0, width: 1440, height: 900 } },
],
getCursorScreenPoint: () => { throw new Error('must not fall back to a cursor read'); },
},
});
service.session = { guideId: 'guide-dip-fallback', paused: false, count: 0, intervalSec: 0 };
const seen = [];
service.enqueueClickCapture = (clickPos) => {
seen.push(clickPos);
};
service.onOsClick(1770000000000, { x: 1500, y: 900 }, 'left');
assert.deepEqual(seen, [{ x: 750, y: 450 }]);
});
test('without screenToDipPoint, coordinates convert via display geometry (Linux/X11)', () => { test('without screenToDipPoint, coordinates convert via display geometry (Linux/X11)', () => {
const service = makeService({ const service = makeService({
screenApi: { screenApi: {
@@ -746,6 +785,7 @@ test('clicks without event coordinates fall back to a live cursor read', () => {
getAllDisplays: () => [], getAllDisplays: () => [],
}, },
}); });
service._wayland = false; // X11/Windows: the live-cursor fallback applies
service.session = { guideId: 'guide-cursor', paused: false, count: 0, intervalSec: 0 }; service.session = { guideId: 'guide-cursor', paused: false, count: 0, intervalSec: 0 };
const seen = []; const seen = [];
service.enqueueClickCapture = (clickPos) => { service.enqueueClickCapture = (clickPos) => {
@@ -757,6 +797,64 @@ test('clicks without event coordinates fall back to a live cursor read', () => {
assert.deepEqual(seen, [{ x: 11, y: 22 }]); assert.deepEqual(seen, [{ x: 11, y: 22 }]);
}); });
test('on Wayland an evdev click does not stamp a bogus top-left cursor position', () => {
// Wayland's getCursorScreenPoint returns {0,0}; a fallback read would mark
// every click in the corner. The click must capture with a null position
// (no marker) instead.
const service = makeService({
screenApi: {
getCursorScreenPoint: () => ({ x: 0, y: 0 }),
getAllDisplays: () => [],
},
});
service._wayland = true;
service.session = { guideId: 'guide-wl-click', paused: false, count: 0, intervalSec: 0 };
const seen = [];
service.enqueueClickCapture = (clickPos) => { seen.push(clickPos); };
service.onOsClick(1770000000000, null, 'button-1');
assert.deepEqual(seen, [null], 'no position is passed, so no marker is drawn');
});
// ---- evdev decoding ---------------------------------------------------------------
const { decodeEvdevButtonPresses } = CaptureService;
// Pack a 64-bit struct input_event (24 bytes): 16-byte timeval, u16 type,
// u16 code, s32 value.
function evdevRecord(type, code, value) {
const b = Buffer.alloc(24);
b.writeUInt16LE(type, 16);
b.writeUInt16LE(code, 18);
b.writeInt32LE(value, 20);
return b;
}
test('evdev decoder extracts left/right/middle button presses, ignores the rest', () => {
const EV_KEY = 0x01;
const EV_REL = 0x02;
const buf = Buffer.concat([
evdevRecord(EV_KEY, 272, 1), // BTN_LEFT down -> button-1
evdevRecord(EV_KEY, 272, 0), // BTN_LEFT up -> ignored (release)
evdevRecord(EV_REL, 0, 5), // motion -> ignored (not a key)
evdevRecord(EV_KEY, 273, 1), // BTN_RIGHT down -> button-3
evdevRecord(EV_KEY, 274, 1), // BTN_MIDDLE down-> button-2
evdevRecord(EV_KEY, 0x110 + 8, 1), // a non-mouse key -> ignored
]);
const { presses, rest } = decodeEvdevButtonPresses(buf, 24);
assert.deepEqual(presses, ['button-1', 'button-3', 'button-2']);
assert.equal(rest.length, 0);
});
test('evdev decoder keeps a trailing partial record for the next chunk', () => {
const full = evdevRecord(0x01, 272, 1);
const buf = Buffer.concat([full, full.subarray(0, 10)]); // 1.4 records
const { presses, rest } = decodeEvdevButtonPresses(buf, 24);
assert.deepEqual(presses, ['button-1']);
assert.equal(rest.length, 10, 'the half record is returned to be completed later');
});
// ---- watcher loss ----------------------------------------------------------------- // ---- watcher loss -----------------------------------------------------------------
test('losing the click watcher mid-session falls back to interval capture', () => { test('losing the click watcher mid-session falls back to interval capture', () => {
@@ -779,6 +877,49 @@ test('losing the click watcher mid-session falls back to interval capture', () =
} }
}); });
test('losing the click watcher mid-session can fall back to hotkey-only', () => {
const service = makeService();
service.settings.get = (key) => {
if (key === 'capture.fallbackTrigger') return 'hotkey';
if (key === 'capture.autoIntervalSec') return 3;
return null;
};
service.session = { guideId: 'guide-loss-hotkey', paused: false, count: 0, intervalSec: 0 };
const states = [];
service.notify = (channel, payload) => {
states.push({ channel, payload });
};
try {
service.handleClickWatcherLoss('exited with code 1');
assert.equal(service.session.intervalSec, 0,
'hotkey fallback must not silently start the 5-second timer');
assert.equal(service.intervalTimer, null);
assert.ok(states.some((s) => s.channel === 'capture:state'));
} finally {
service.finishSession();
}
});
test('starting a session with hotkey fallback keeps the timer off when clicks are unavailable', () => {
const service = makeService();
service.clickCaptureAvailable = () => false;
service.settings.get = (key) => {
if (key === 'capture.fallbackTrigger') return 'hotkey';
if (key === 'capture.autoIntervalSec') return 7;
if (key === 'capture.captureOutsideClicks') return false;
return null;
};
service.startSession('guide-hotkey-fallback');
assert.equal(service.session.intervalSec, 0);
assert.equal(service.state().clickCaptureAvailable, false);
assert.equal(service.state().clickCapture, false);
service.finishSession();
});
// ---- strict frame selection ----------------------------------------------------- // ---- strict frame selection -----------------------------------------------------
test('a click is served instantly from the freshly buffered frame', async () => { test('a click is served instantly from the freshly buffered frame', async () => {
@@ -1004,6 +1145,7 @@ test('clicks during an in-flight pre-click grab wait for the frame instead of be
test('click frames come from the stream backend when it is active', async () => { test('click frames come from the stream backend when it is active', async () => {
const service = makeService(); const service = makeService();
service._wayland = false; // X11/Windows: click frame-requests are failable
const clickAt = Date.now(); const clickAt = Date.now();
service.session = { guideId: 'guide-stream', paused: false, count: 0, intervalSec: 0 }; service.session = { guideId: 'guide-stream', paused: false, count: 0, intervalSec: 0 };
const requests = []; const requests = [];
@@ -1036,8 +1178,8 @@ test('click frames come from the stream backend when it is active', async () =>
assert.equal(result.ok, true); assert.equal(result.ok, true);
assert.deepEqual(added, ['stream-frame']); assert.deepEqual(added, ['stream-frame']);
assert.deepEqual(requests, [{ clickPos: { x: 10, y: 10 }, clickAt, strict: true, leadMs: 0 }], assert.deepEqual(requests, [{ clickPos: { x: 10, y: 10 }, clickAt, strict: true, leadMs: 0, failable: true }],
'the worker receives the hook-time click timestamp, strictness, and lead'); 'the worker receives the hook-time click timestamp, strictness, lead, and failability');
}); });
test('a stream backend with no qualifying frame falls through to the fresh-shot path', async () => { test('a stream backend with no qualifying frame falls through to the fresh-shot path', async () => {
@@ -1080,6 +1222,9 @@ test('an unhealthy stream backend degrades to the in-process frame loop', () =>
const service = makeService(); const service = makeService();
service.session = { guideId: 'guide-degrade', paused: false, count: 0, intervalSec: 0 }; service.session = { guideId: 'guide-degrade', paused: false, count: 0, intervalSec: 0 };
service.streamBackend = { isActive: () => true, stop: () => {} }; service.streamBackend = { isActive: () => true, stop: () => {} };
// Off Wayland the in-process loop is viable (getSources works), so an
// unhealthy worker must degrade to the loop rather than silently stopping.
service._wayland = false;
let loopStarted = false; let loopStarted = false;
service.startFrameLoop = () => { loopStarted = true; }; service.startFrameLoop = () => { loopStarted = true; };
const states = []; const states = [];
@@ -1092,6 +1237,24 @@ test('an unhealthy stream backend degrades to the in-process frame loop', () =>
assert.ok(states.includes('capture:state')); assert.ok(states.includes('capture:state'));
}); });
test('an unhealthy stream backend does NOT start the frame loop on Wayland', () => {
// On Wayland the 200ms getSources() frame loop is broken (throws) and pops
// the portal, so it's not a usable fallback — the portal stream is the only
// capture path and the user must re-share if it stops.
const service = makeService();
service.session = { guideId: 'guide-degrade-wl', paused: false, count: 0, intervalSec: 0 };
service.streamBackend = { isActive: () => true, stop: () => {} };
service._wayland = true;
let loopStarted = false;
service.startFrameLoop = () => { loopStarted = true; };
service.notify = () => {};
service.degradeToFrameLoop();
assert.equal(service.streamBackend, null);
assert.equal(loopStarted, false, 'no frame loop on Wayland — getSources is unusable there');
});
test('session state reports which frame recorder is serving clicks', () => { test('session state reports which frame recorder is serving clicks', () => {
const service = makeService(); const service = makeService();
service.session = { guideId: 'guide-state', paused: false, count: 0, intervalSec: 0 }; service.session = { guideId: 'guide-state', paused: false, count: 0, intervalSec: 0 };
@@ -1156,7 +1319,12 @@ test('a new session starts paused and does not hide the window until "Start reco
isDestroyed() { return this.destroyed; }, isDestroyed() { return this.destroyed; },
isVisible() { return this.visible; }, isVisible() { return this.visible; },
isMinimized() { return this.minimized; }, isMinimized() { return this.minimized; },
// The window is "tucked away" for recording either by hide() (with a tray)
// or minimize() (Linux without a tray). Both make it not visible; assert on
// visibility so the test is independent of which mechanism the platform picks.
hide() { this.visible = false; this.hidden += 1; }, hide() { this.visible = false; this.hidden += 1; },
minimize() { this.visible = false; this.minimized = true; },
restore() { this.minimized = false; this.visible = true; },
show() { this.visible = true; this.shown += 1; }, show() { this.visible = true; this.shown += 1; },
showInactive() { this.visible = true; this.shown += 1; }, showInactive() { this.visible = true; this.shown += 1; },
focus() {}, focus() {},
@@ -1171,15 +1339,15 @@ test('a new session starts paused and does not hide the window until "Start reco
assert.equal(service.session.paused, true, 'sessions start paused'); assert.equal(service.session.paused, true, 'sessions start paused');
assert.equal(service.state().paused, true); assert.equal(service.state().paused, true);
assert.equal(win.hidden, 0, 'window must stay visible until recording starts'); assert.equal(win.visible, true, 'window must stay visible until recording starts');
// User clicks "Start recording" (the resume action). // User clicks "Start recording" (the resume action).
service.togglePause(false); service.togglePause(false);
assert.equal(service.session.paused, false); assert.equal(service.session.paused, false);
assert.equal(win.hidden, 0, 'hide is deferred until the resume path runs'); assert.equal(win.visible, true, 'tuck-away is deferred until the resume path runs');
await new Promise((r) => setTimeout(r, 25)); await new Promise((r) => setTimeout(r, 25));
assert.equal(win.hidden, 1, 'window hides once recording actually starts'); assert.equal(win.visible, false, 'window is tucked away once recording actually starts');
} finally { } finally {
service.finishSession(); service.finishSession();
} }
+105 -79
View File
@@ -7,9 +7,11 @@ const path = require('node:path');
const { const {
buildMissingElectronError, buildMissingElectronError,
repairElectronInstall, linuxSandboxLaunchArgs,
noSandboxExplicitlyAllowed,
resolveElectronBinary, resolveElectronBinary,
} = require('../../scripts/electron-launcher'); } = require('../../scripts/electron-launcher');
const { checkNodeVersion, assertSupportedNode } = require('../../scripts/check-node-version');
const { makeTmpDir, rmrf } = require('./helpers'); const { makeTmpDir, rmrf } = require('./helpers');
test('resolves the Electron binary from path.txt when present', (t) => { test('resolves the Electron binary from path.txt when present', (t) => {
@@ -39,95 +41,119 @@ test('falls back to the platform binary when path.txt is absent', (t) => {
); );
}); });
test('repairs a broken Electron install before resolving the binary', (t) => { test('never runs npm when the runtime is missing: fails with npm ci diagnostics', (t) => {
const root = makeTmpDir('electron-repair');
t.after(() => rmrf(root));
fs.mkdirSync(path.join(root, 'dist'), { recursive: true });
fs.writeFileSync(
path.join(root, 'install.js'),
[
"const fs = require('node:fs');",
"const path = require('node:path');",
"if (process.env.ELECTRON_SKIP_BINARY_DOWNLOAD) process.exit(2);",
"fs.mkdirSync(path.join(__dirname, 'dist'), { recursive: true });",
"fs.writeFileSync(path.join(__dirname, 'dist', 'electron.exe'), 'binary');",
"fs.writeFileSync(path.join(__dirname, 'path.txt'), 'electron.exe');",
].join('\n')
);
const originalSkip = process.env.ELECTRON_SKIP_BINARY_DOWNLOAD;
process.env.ELECTRON_SKIP_BINARY_DOWNLOAD = '1';
t.after(() => {
if (originalSkip === undefined) delete process.env.ELECTRON_SKIP_BINARY_DOWNLOAD;
else process.env.ELECTRON_SKIP_BINARY_DOWNLOAD = originalSkip;
});
assert.equal(
repairElectronInstall({ packageRoot: root }),
true
);
assert.equal(
resolveElectronBinary({ packageRoot: root, platform: 'win32' }),
path.join(root, 'dist', 'electron.exe')
);
});
test('rebuilds Electron through npm when the binary is missing', (t) => {
const root = makeTmpDir('electron-rebuild');
t.after(() => rmrf(root));
fs.mkdirSync(path.join(root, 'dist'), { recursive: true });
const fakeNpmCli = path.join(root, 'fake-npm-cli.js');
fs.writeFileSync(
fakeNpmCli,
[
"const fs = require('node:fs');",
"const path = require('node:path');",
"if (process.env.ELECTRON_SKIP_BINARY_DOWNLOAD) process.exit(2);",
"fs.mkdirSync(path.join(__dirname, 'dist'), { recursive: true });",
"fs.writeFileSync(path.join(__dirname, 'dist', 'electron.exe'), 'binary');",
"fs.writeFileSync(path.join(__dirname, 'path.txt'), 'electron.exe');",
].join('\n')
);
const originalNpmExecPath = process.env.npm_execpath;
const originalNpmNodeExecPath = process.env.npm_node_execpath;
const originalSkip = process.env.ELECTRON_SKIP_BINARY_DOWNLOAD;
process.env.npm_execpath = fakeNpmCli;
process.env.npm_node_execpath = process.execPath;
process.env.ELECTRON_SKIP_BINARY_DOWNLOAD = '1';
t.after(() => {
if (originalNpmExecPath === undefined) delete process.env.npm_execpath;
else process.env.npm_execpath = originalNpmExecPath;
if (originalNpmNodeExecPath === undefined) delete process.env.npm_node_execpath;
else process.env.npm_node_execpath = originalNpmNodeExecPath;
if (originalSkip === undefined) delete process.env.ELECTRON_SKIP_BINARY_DOWNLOAD;
else process.env.ELECTRON_SKIP_BINARY_DOWNLOAD = originalSkip;
});
assert.equal(
resolveElectronBinary({ packageRoot: root, platform: 'win32' }),
path.join(root, 'dist', 'electron.exe')
);
});
test('reports a helpful error when the runtime is missing', (t) => {
const root = makeTmpDir('electron-missing'); const root = makeTmpDir('electron-missing');
t.after(() => rmrf(root)); t.after(() => rmrf(root));
fs.mkdirSync(path.join(root, 'dist'), { recursive: true }); fs.mkdirSync(path.join(root, 'dist'), { recursive: true });
assert.throws( // Point npm_execpath at a script that would create the binary if executed.
() => resolveElectronBinary({ packageRoot: root, platform: 'win32' }), // The launcher must NOT execute it: runtime self-repair is forbidden.
/npm install/ const trapNpmCli = path.join(root, 'trap-npm-cli.js');
fs.writeFileSync(
trapNpmCli,
[
"const fs = require('node:fs');",
"const path = require('node:path');",
"fs.writeFileSync(path.join(__dirname, 'npm-was-invoked'), '1');",
].join('\n')
); );
const originalNpmExecPath = process.env.npm_execpath;
process.env.npm_execpath = trapNpmCli;
t.after(() => {
if (originalNpmExecPath === undefined) delete process.env.npm_execpath;
else process.env.npm_execpath = originalNpmExecPath;
});
assert.throws(
() => resolveElectronBinary({ packageRoot: root, projectRoot: root, platform: 'win32' }),
/npm ci/
);
assert.equal(fs.existsSync(path.join(root, 'npm-was-invoked')), false);
});
test('missing electron package fails with npm ci diagnostics, not an install', (t) => {
const root = makeTmpDir('electron-no-package');
t.after(() => rmrf(root));
assert.throws(
() => resolveElectronBinary({ packageRoot: null, projectRoot: root, platform: 'win32' }),
/never installs dependencies at runtime[\s\S]*npm ci/
);
});
test('missing runtime error message explains recovery without runtime installs', (t) => {
const root = makeTmpDir('electron-missing-msg');
t.after(() => rmrf(root));
const message = buildMissingElectronError({ const message = buildMissingElectronError({
packageRoot: root, packageRoot: root,
distDir: path.join(root, 'dist'), distDir: path.join(root, 'dist'),
candidatePaths: [path.join(root, 'dist', 'electron.exe')], candidatePaths: [path.join(root, 'dist', 'electron.exe')],
}); });
assert.match(message, /Electron could not be started/); assert.match(message, /Electron could not be started/);
assert.match(message, /Expected the binary in:/); assert.match(message, /npm ci/);
assert.doesNotMatch(message, /npm install/);
});
test('refuses an unsandboxed Linux launch unless explicitly allowed', () => {
assert.throws(
() =>
linuxSandboxLaunchArgs({
electronPath: '/tmp/stepforge/node_modules/electron/dist/electron',
platform: 'linux',
statSync: () => ({ uid: 1000, mode: 0o100755 }),
env: {},
userNamespaces: () => false,
}),
/refuses to silently launch unsandboxed[\s\S]*STEPFORGE_ALLOW_NO_SANDBOX/
);
});
test('allows --no-sandbox only with an explicit dev/CI opt-in', () => {
for (const env of [
{ STEPFORGE_ALLOW_NO_SANDBOX: '1' },
{ ELECTRON_DISABLE_SANDBOX: '1' },
]) {
const args = linuxSandboxLaunchArgs({
electronPath: '/tmp/stepforge/node_modules/electron/dist/electron',
platform: 'linux',
statSync: () => ({ uid: 1000, mode: 0o100755 }),
env,
userNamespaces: () => false,
});
assert.deepEqual(args, ['--no-sandbox']);
}
assert.equal(noSandboxExplicitlyAllowed({ STEPFORGE_ALLOW_NO_SANDBOX: '1' }), true);
assert.equal(noSandboxExplicitlyAllowed({}), false);
});
test('keeps the sandbox enabled when the Linux helper is root-owned and setuid', () => {
const args = linuxSandboxLaunchArgs({
electronPath: '/tmp/stepforge/node_modules/electron/dist/electron',
platform: 'linux',
statSync: () => ({ uid: 0, mode: 0o104755 }),
env: {},
});
assert.deepEqual(args, []);
});
test('non-Linux platforms never receive sandbox launch flags', () => {
assert.deepEqual(linuxSandboxLaunchArgs({ platform: 'win32', env: {} }), []);
assert.deepEqual(linuxSandboxLaunchArgs({ platform: 'darwin', env: {} }), []);
});
test('node toolchain check compares against package.json engines', () => {
const ok = checkNodeVersion({ currentVersion: '99.0.0' });
assert.equal(ok.ok, true);
const tooOld = checkNodeVersion({ currentVersion: '18.19.1' });
assert.equal(tooOld.ok, false);
assert.match(tooOld.required, /^\d+\.\d+\.\d+$/);
assert.throws(
() => assertSupportedNode({ currentVersion: '18.19.1' }),
/requires Node .* or newer/
);
}); });
+65
View File
@@ -12,6 +12,7 @@ const { exportWikiJs } = require('../../exporters/wikijs');
const { exportHtmlSimple, exportHtmlRich } = require('../../exporters/html'); const { exportHtmlSimple, exportHtmlRich } = require('../../exporters/html');
const { exportConfluence } = require('../../exporters/confluence'); const { exportConfluence } = require('../../exporters/confluence');
const { htmlToMarkdown } = require('../../exporters/htmlmd'); const { htmlToMarkdown } = require('../../exporters/htmlmd');
const { stepContentGroups } = require('../../exporters/common');
const { decodePng } = require('../../core/png'); const { decodePng } = require('../../core/png');
const { buildFixtureGuide } = require('./fixture-guide'); const { buildFixtureGuide } = require('./fixture-guide');
const { makeTmpDir, rmrf } = require('./helpers'); const { makeTmpDir, rmrf } = require('./helpers');
@@ -146,6 +147,70 @@ test('Markdown export: TOC anchors resolve, images exist, blocks rendered', (t)
assert.ok(md.includes('<p>Admins only.</p>')); assert.ok(md.includes('<p>Admins only.</p>'));
}); });
test('text block positions render around the title, description, and image', (t) => {
const root = makeTmpDir('exppositions');
t.after(() => rmrf(root));
const { store, guide, s1 } = buildFixtureGuide(path.join(root, 'data'));
const step = store.getStep(guide.guideId, s1.stepId);
step.textBlocks = [
{ id: 'tb-before-title', order: 1, position: 'before-title', level: 'info', title: 'Before title', descriptionHtml: '<p>Before title.</p>' },
{ id: 'tb-after-title', order: 2, position: 'after-title', level: 'info', title: 'After title', descriptionHtml: '<p>After title.</p>' },
{ id: 'tb-before-description', order: 3, position: 'before-description', level: 'info', title: 'Before description', descriptionHtml: '<p>Before description.</p>' },
{ id: 'tb-after-description', order: 4, position: 'after-description', level: 'info', title: 'After description', descriptionHtml: '<p>After description.</p>' },
{ id: 'tb-before-image', order: 5, position: 'before-image', level: 'info', title: 'Before image', descriptionHtml: '<p>Before image.</p>' },
{ id: 'tb-after-image', order: 6, position: 'after-image', level: 'info', title: 'After image', descriptionHtml: '<p>After image.</p>' },
];
store.saveStep(guide.guideId, step);
const ast = buildRenderAst(store, guide.guideId);
const groups = stepContentGroups(ast.steps[0]);
assert.equal(groups.beforeTitle.length, 1);
assert.equal(groups.afterTitle.length, 1);
assert.equal(groups.beforeDescription.length, 1);
assert.equal(groups.afterDescription.length, 1);
assert.equal(groups.beforeImage.length, 1);
assert.equal(groups.afterImage.length, 1);
const assertIncreasingOrder = (text, markers) => {
let previous = -1;
for (const marker of markers) {
const index = text.indexOf(marker);
assert.ok(index >= 0, `missing marker: ${marker}`);
assert.ok(index > previous, `marker out of order: ${marker}`);
previous = index;
}
};
const mdOut = path.join(root, 'md');
const md = fs.readFileSync(exportMarkdown(ast, mdOut).file, 'utf8');
assertIncreasingOrder(md, [
'Before title',
'## 1. Open AcmeSync settings',
'After title',
'Before description',
'docs.example.com',
'After description',
'Before image',
'![Step 1](',
'After image',
]);
const htmlOut = path.join(root, 'html');
const html = fs.readFileSync(exportHtmlSimple(ast, htmlOut).file, 'utf8');
assertIncreasingOrder(html, [
'Before title',
'<h2>1. Open AcmeSync settings</h2>',
'After title',
'Before description',
'docs.example.com',
'After description',
'Before image',
'alt="Step 1"',
'After image',
]);
});
test('Wiki.js export: TOC is included, wiki callouts render, images exist', (t) => { test('Wiki.js export: TOC is included, wiki callouts render, images exist', (t) => {
const root = makeTmpDir('expwikijs'); const root = makeTmpDir('expwikijs');
t.after(() => rmrf(root)); t.after(() => rmrf(root));
+4 -1
View File
@@ -21,6 +21,9 @@ test('Windows packaging uses an assisted NSIS installer', (t) => {
assert.equal(config.nsis.createDesktopShortcut, true); assert.equal(config.nsis.createDesktopShortcut, true);
assert.equal(config.nsis.createStartMenuShortcut, true); assert.equal(config.nsis.createStartMenuShortcut, true);
assert.equal(config.nsis.shortcutName, 'StepForge'); assert.equal(config.nsis.shortcutName, 'StepForge');
assert.equal(config.buildVersion, '0.3.2.1');
assert.equal(config.nsis.artifactName, '${productName} Setup ${buildVersion}.${ext}');
assert.equal(config.nsis.include, 'build/installer.nsh');
assert.equal(config.asar, true); assert.equal(config.asar, true);
assert.ok(config.files.includes('app/**/*')); assert.ok(config.files.includes('app/**/*'));
assert.ok(config.files.includes('core/**/*')); assert.ok(config.files.includes('core/**/*'));
@@ -34,7 +37,7 @@ test('Windows packaging uses an assisted NSIS installer', (t) => {
const tmp = makeTmpDir('windows-installer'); const tmp = makeTmpDir('windows-installer');
t.after(() => rmrf(tmp)); t.after(() => rmrf(tmp));
fs.mkdirSync(path.join(tmp, 'nested', 'deeper'), { recursive: true }); fs.mkdirSync(path.join(tmp, 'nested', 'deeper'), { recursive: true });
const installer = path.join(tmp, 'nested', 'deeper', 'StepForge Setup 0.2.0.exe'); const installer = path.join(tmp, 'nested', 'deeper', 'StepForge Setup 0.3.2.1.exe');
fs.writeFileSync(installer, Buffer.from('fake installer')); fs.writeFileSync(installer, Buffer.from('fake installer'));
assert.equal(findInstallerExe(tmp), installer); assert.equal(findInstallerExe(tmp), installer);
}); });
+3
View File
@@ -56,13 +56,16 @@ test('settings persist, deep-merge with defaults, and store global placeholders'
assert.equal(s1.get('appearance'), DEFAULT_SETTINGS.appearance); assert.equal(s1.get('appearance'), DEFAULT_SETTINGS.appearance);
s1.set('appearance', 'dark'); s1.set('appearance', 'dark');
s1.set('capture.delayMs', 1500); s1.set('capture.delayMs', 1500);
s1.set('ai.ollama.model', 'qwen3:0.6b');
s1.setGlobalPlaceholders({ Company: 'Acme', Author: 'Tyler' }); s1.setGlobalPlaceholders({ Company: 'Acme', Author: 'Tyler' });
// A fresh instance reads back the changed values merged over defaults. // A fresh instance reads back the changed values merged over defaults.
const s2 = new Settings(dir); const s2 = new Settings(dir);
assert.equal(s2.get('appearance'), 'dark'); assert.equal(s2.get('appearance'), 'dark');
assert.equal(s2.get('capture.delayMs'), 1500); assert.equal(s2.get('capture.delayMs'), 1500);
assert.equal(s2.get('ai.ollama.model'), 'qwen3:0.6b');
assert.equal(s2.get('capture.clickMarker'), DEFAULT_SETTINGS.capture.clickMarker); assert.equal(s2.get('capture.clickMarker'), DEFAULT_SETTINGS.capture.clickMarker);
assert.equal(s2.get('capture.fallbackTrigger'), DEFAULT_SETTINGS.capture.fallbackTrigger);
assert.deepEqual(s2.getGlobalPlaceholders(), { Company: 'Acme', Author: 'Tyler' }); assert.deepEqual(s2.getGlobalPlaceholders(), { Company: 'Acme', Author: 'Tyler' });
}); });
+218
View File
@@ -0,0 +1,218 @@
'use strict';
const test = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const security = require('../../app/security');
const MAIN_URL = security.APP_PAGES.main;
const WORKER_URL = security.APP_PAGES.captureWorker;
const REGION_URL = security.APP_PAGES.region;
const ROOT = path.resolve(__dirname, '..', '..');
const read = (rel) => fs.readFileSync(path.join(ROOT, rel), 'utf8');
// ---- navigation policy -----------------------------------------------------
test('navigation: a window may only stay on its own app page', () => {
assert.equal(security.navigationAllowed(MAIN_URL, 'main'), true);
assert.equal(security.navigationAllowed(`${MAIN_URL}?q=1#frag`, 'main'), true);
assert.equal(security.navigationAllowed(REGION_URL, 'region'), true);
// Hostile / cross-page navigations are all denied.
for (const target of [
'https://evil.example/phish.html',
'http://127.0.0.1:8080/',
'javascript:alert(1)',
'data:text/html,<script>1</script>',
'about:blank',
REGION_URL, // a *different* app page is still a denial for 'main'
'file:///etc/passwd',
'not a url',
'',
]) {
assert.equal(security.navigationAllowed(target, 'main'), false, `should deny ${target}`);
}
assert.equal(security.navigationAllowed(MAIN_URL, 'nonexistent-page'), false);
});
// ---- permission policy -----------------------------------------------------
test('permissions: display capture only for the capture worker, nothing else for anyone', () => {
assert.equal(security.permissionAllowed('display-capture', WORKER_URL), true);
assert.equal(security.permissionAllowed('media', WORKER_URL), true);
// The main window gets nothing, including display capture.
assert.equal(security.permissionAllowed('display-capture', MAIN_URL), false);
assert.equal(security.permissionAllowed('media', MAIN_URL), false);
// Everything else is denied even for the worker.
for (const permission of ['geolocation', 'notifications', 'clipboard-read', 'openExternal', 'fullscreen', 'pointerLock', 'hid', 'usb', 'serial']) {
assert.equal(security.permissionAllowed(permission, WORKER_URL), false, permission);
}
// Remote origins never get anything.
assert.equal(security.permissionAllowed('display-capture', 'https://evil.example/'), false);
assert.equal(security.permissionAllowed('media', undefined), false);
});
// ---- external URL policy ---------------------------------------------------
test('external links: only well-formed http(s)/mailto pass', () => {
assert.equal(security.validateExternalUrl('https://example.com/docs'), 'https://example.com/docs');
assert.equal(security.validateExternalUrl('http://example.com'), 'http://example.com/');
assert.match(security.validateExternalUrl('mailto:[email protected]'), /^mailto:/);
for (const url of [
'javascript:alert(1)',
'file:///etc/passwd',
'data:text/html,x',
'ftp://example.com/x',
'smb://server/share',
'chrome://settings',
'vbscript:x',
'https://',
'not a url',
123,
null,
`https://example.com/${'a'.repeat(3000)}`,
]) {
assert.equal(security.validateExternalUrl(url), null, `should reject ${String(url).slice(0, 60)}`);
}
});
// ---- IPC sender guard --------------------------------------------------------
function makeGuard(mainWc) {
return security.makeIpcSenderGuard({ getMainWebContents: () => mainWc });
}
test('ipc guard: accepts only the main window top frame on index.html', () => {
const wc = { id: 1 };
const guard = makeGuard(wc);
assert.equal(guard({ sender: wc, senderFrame: { parent: null, url: MAIN_URL } }), true);
});
test('ipc guard: rejects other windows, subframes, navigated and disposed frames', () => {
const wc = { id: 1 };
const otherWc = { id: 2 };
const guard = makeGuard(wc);
// Different webContents (popup, worker, region overlay).
assert.equal(guard({ sender: otherWc, senderFrame: { parent: null, url: MAIN_URL } }), false);
// Subframe of our own window.
assert.equal(guard({ sender: wc, senderFrame: { parent: {}, url: MAIN_URL } }), false);
// Frame that navigated somewhere else but kept the preload bridge.
assert.equal(guard({ sender: wc, senderFrame: { parent: null, url: 'https://evil.example/' } }), false);
assert.equal(guard({ sender: wc, senderFrame: { parent: null, url: WORKER_URL } }), false);
// Disposed frame accessor throws.
const disposed = {};
Object.defineProperty(disposed, 'parent', { get() { throw new Error('disposed'); } });
assert.equal(guard({ sender: wc, senderFrame: disposed }), false);
// Missing frame or window entirely.
assert.equal(guard({ sender: wc, senderFrame: null }), false);
assert.equal(makeGuard(null)({ sender: wc, senderFrame: { parent: null, url: MAIN_URL } }), false);
assert.equal(guard(null), false);
});
// ---- argument hygiene --------------------------------------------------------
test('args must be plain object bags', () => {
assert.equal(security.isPlainArgs({ a: 1 }), true);
assert.equal(security.isPlainArgs(Object.create(null)), true);
assert.equal(security.isPlainArgs(undefined), true);
assert.equal(security.isPlainArgs(null), true);
assert.equal(security.isPlainArgs([1, 2]), false);
assert.equal(security.isPlainArgs('x'), false);
class Weird {}
assert.equal(security.isPlainArgs(new Weird()), false);
});
test('payload budget rejects oversized and non-data values', () => {
assert.equal(security.payloadWithinBudget({ a: 'small' }, 1000), true);
assert.equal(security.payloadWithinBudget({ a: 'x'.repeat(2000) }, 1000), false);
assert.equal(security.payloadWithinBudget({ fn: () => 1 }, 1000), false);
// Deep nesting is cut off.
let deep = 'leaf';
for (let i = 0; i < 40; i += 1) deep = { deep };
assert.equal(security.payloadWithinBudget(deep, 100000), false);
// Numbers/booleans/null are fine.
assert.equal(security.payloadWithinBudget({ n: 5, b: true, z: null }, 1000), true);
});
test('field validators refuse traversal, separators, and pollution', () => {
const c = security.check;
assert.equal(c.id('guide-123_A.b'), true);
assert.equal(c.id('../../etc/passwd'), false);
assert.equal(c.id('a/b'), false);
assert.equal(c.id('a\\b'), false);
assert.equal(c.id(''), false);
assert.equal(c.id(42), false);
assert.equal(c.fileName('My snapshot 2026-07-03'), true);
assert.equal(c.fileName('..'), false);
assert.equal(c.fileName('a/../b'), false);
assert.equal(c.fileName('a/b'), false);
assert.equal(c.fileName('a\\b'), false);
assert.equal(c.fileName('a\0b'), false);
assert.equal(c.fileName(' '), false);
assert.equal(c.settingsKeyPath('capture.hotkeyCapture'), true);
assert.equal(c.settingsKeyPath('__proto__.polluted'), false);
assert.equal(c.settingsKeyPath('a.constructor.b'), false);
assert.equal(c.settingsKeyPath('a..b'), false);
assert.equal(c.settingsKeyPath(''), false);
assert.equal(c.base64('aGVsbG8=', 100), true);
assert.equal(c.base64('<script>', 100), false);
});
test('produced-files registry only re-opens what main created', () => {
const produced = new security.ProducedFiles(3);
produced.add('/tmp/exports/guide.pdf');
assert.equal(produced.has('/tmp/exports/guide.pdf'), true);
assert.equal(produced.has('/tmp/exports/../exports/guide.pdf'), true, 'path normalization');
assert.equal(produced.has('/etc/passwd'), false);
assert.equal(produced.has(null), false);
produced.add('/tmp/a');
produced.add('/tmp/b');
produced.add('/tmp/c'); // evicts guide.pdf (LRU bound)
assert.equal(produced.has('/tmp/exports/guide.pdf'), false);
assert.equal(produced.has('/tmp/c'), true);
});
// ---- source-level regression guards -----------------------------------------
test('main process never grants blanket permissions again', () => {
const src = read('app/main.js');
assert.doesNotMatch(src, /setPermissionCheckHandler\(\(\)\s*=>\s*true\)/);
assert.doesNotMatch(src, /cb\(true\)\)/);
assert.match(src, /security\.permissionAllowed/);
assert.match(src, /installWindowSecurity\(mainWindow, 'main'\)/);
});
test('no generic shell path channels remain on the bridge', () => {
const preload = read('app/preload.js');
assert.doesNotMatch(preload, /shell:openPath/);
assert.doesNotMatch(preload, /shell:showItemInFolder/);
assert.match(preload, /shell:openProduced/);
assert.match(preload, /shell:openExternal/);
});
test('every renderer window is created sandboxed', () => {
for (const file of ['app/main.js', 'app/capture.js', 'app/stream-backend.js']) {
const src = read(file);
const created = src.match(/new BrowserWindow\(/g) || [];
const sandboxed = src.match(/sandbox: true/g) || [];
assert.ok(created.length > 0, `${file} should create a window`);
assert.equal(
sandboxed.length,
created.length,
`${file}: every BrowserWindow must set sandbox: true (${sandboxed.length}/${created.length})`
);
}
});
+46
View File
@@ -0,0 +1,46 @@
'use strict';
const fs = require('node:fs');
const path = require('node:path');
const test = require('node:test');
const assert = require('node:assert/strict');
const { makeTmpDir, rmrf } = require('./helpers');
const { stampVersion } = require('../../scripts/stamp-version');
test('stampVersion splits build labels into package and build versions', () => {
const root = makeTmpDir('stamp-version');
try {
fs.writeFileSync(path.join(root, 'package.json'), JSON.stringify({
name: 'stepforge',
version: '0.1.0',
private: true,
buildVersion: '0.1.0',
}, null, 2));
fs.writeFileSync(path.join(root, 'package-lock.json'), JSON.stringify({
name: 'stepforge',
version: '0.1.0',
lockfileVersion: 3,
requires: true,
packages: {
'': {
name: 'stepforge',
version: '0.1.0',
},
},
}, null, 2));
stampVersion(root, '0.3.2.1');
const pkg = JSON.parse(fs.readFileSync(path.join(root, 'package.json'), 'utf8'));
const lock = JSON.parse(fs.readFileSync(path.join(root, 'package-lock.json'), 'utf8'));
assert.equal(pkg.version, '0.3.2');
assert.equal(pkg.buildVersion, '0.3.2.1');
assert.equal(lock.version, '0.3.2');
assert.equal(lock.packages[''].version, '0.3.2');
} finally {
rmrf(root);
}
});
+464
View File
@@ -0,0 +1,464 @@
'use strict';
const fs = require('node:fs');
const path = require('node:path');
const test = require('node:test');
const assert = require('node:assert/strict');
const { makeTmpDir, rmrf } = require('./helpers');
const { createStep } = require('../../core/schema');
const {
buildCaptureTitle,
buildAiPrompt,
normalizeAiPatch,
applyAiPatchToStep,
} = require('../../core/text-intel');
const { TextIntelService } = require('../../app/text-intel');
function makeSettings(values = {}) {
const data = {
ai: {
enabled: true,
ollama: {
host: 'http://127.0.0.1:11434',
model: 'llama3.2:1b',
},
},
...values,
};
return {
get(key) {
return key.split('.').reduce((acc, part) => (acc == null ? undefined : acc[part]), data);
},
};
}
test('capture titles prefer semantic metadata before OCR fallback', () => {
const title = buildCaptureTitle({
mode: 'fullscreen',
metadata: { windowTitle: 'Reset user password in admin portal' },
ocrText: 'Save',
});
assert.equal(title, 'Click Save');
});
test('capture titles prefer element metadata before window chrome and OCR', () => {
const title = buildCaptureTitle({
mode: 'window',
metadata: {
elementLabel: 'Open advanced settings',
windowTitle: 'Preferences',
},
ocrText: 'Cancel',
});
assert.equal(title, 'Open advanced settings');
});
test('capture titles ignore browser chrome noise in favor of OCR', () => {
const title = buildCaptureTitle({
mode: 'window',
metadata: {
windowTitle: 'Google Chrome ** PR reviews ** /chrome/tyler/autodoc',
appName: 'Google Chrome',
},
ocrText: 'New tab',
});
// OCR wins over the noisy window title; app name is appended for context.
assert.equal(title, 'Click New tab in Google Chrome');
});
test('tab-like roles use select when OCR identifies a tab label', () => {
const title = buildCaptureTitle({
mode: 'window',
metadata: {
elementLabel: 'New tab',
elementRole: 'tab item',
windowTitle: 'Google Chrome - PR reviews',
},
ocrText: 'New tab',
});
assert.equal(title, 'Select New tab');
});
test('capture titles fall back to OCR when metadata is absent', () => {
const title = buildCaptureTitle({
mode: 'window',
metadata: {},
ocrText: 'Save changes',
});
assert.equal(title, 'Click Save changes');
});
test('browser window title strips browser name and falls back to page title', () => {
// OCR fails; browser window title should give something useful, not "Screen capture".
const title = buildCaptureTitle({
mode: 'fullscreen',
metadata: {
windowTitle: 'Oracle | Cloud Applications and Cloud Platform - Google Chrome',
appName: 'chrome',
},
ocrText: '',
});
// Stripped title "Oracle | Cloud Applications and Cloud Platform" → best fragment
assert.ok(title !== 'Screen capture', `Expected smart title, got: ${title}`);
assert.ok(title.toLowerCase().includes('oracle') || title.toLowerCase().includes('cloud'), `Expected oracle/cloud in title, got: ${title}`);
});
test('search query is extracted when user was typing (search step)', () => {
const title = buildCaptureTitle({
mode: 'fullscreen',
metadata: { windowTitle: 'oracle - Google Search - Google Chrome', appName: 'chrome' },
ocrText: '',
recentTyped: 'oracle', // user was typing → this IS the search step
});
assert.equal(title, 'Search for Oracle in Chrome');
});
test('search results window title produces select-result title when no typing (click on results page)', () => {
const title = buildCaptureTitle({
mode: 'fullscreen',
metadata: { windowTitle: 'oracle - Google Search - Google Chrome', appName: 'chrome' },
ocrText: '',
recentTyped: '', // no recent typing → user is clicking a result, not searching
});
assert.equal(title, 'Select a Oracle result in Chrome');
});
test('full link text with pipe separator is preserved in OCR phrases', () => {
const title = buildCaptureTitle({
mode: 'fullscreen',
metadata: { elementRole: 'hyperlink' },
ocrText: 'Oracle | Cloud Applications and Cloud Platform',
});
assert.equal(title, 'Select Oracle | Cloud Applications and Cloud Platform');
});
test('link element role uses Select verb', () => {
const title = buildCaptureTitle({
mode: 'fullscreen',
metadata: { elementLabel: 'Sign in', elementRole: 'hyperlink' },
ocrText: '',
});
assert.equal(title, 'Select Sign in');
});
test('search box element role uses Search for verb', () => {
const title = buildCaptureTitle({
mode: 'fullscreen',
metadata: { elementLabel: 'oracle', elementRole: 'search box' },
ocrText: '',
});
assert.equal(title, 'Search for Oracle');
});
test('keyboard shortcut produces action title qualified with app', () => {
const title = buildCaptureTitle({
mode: 'fullscreen',
metadata: { appName: 'chrome' },
ocrText: '',
recentShortcut: 'Ctrl+T',
});
assert.equal(title, 'Open new tab in Chrome');
});
test('keyboard shortcut title without app name', () => {
const title = buildCaptureTitle({
mode: 'fullscreen',
metadata: {},
ocrText: '',
recentShortcut: 'Ctrl+S',
});
assert.equal(title, 'Save');
});
test('typed text with search input role produces Search for title', () => {
const title = buildCaptureTitle({
mode: 'fullscreen',
metadata: { elementRole: 'search box', appName: 'chrome' },
ocrText: '',
recentTyped: 'oracle',
});
assert.equal(title, 'Search for "oracle" in Chrome');
});
test('UIAutomation element value takes priority over keyboard buffer', () => {
const title = buildCaptureTitle({
mode: 'fullscreen',
metadata: { elementRole: 'edit', elementValue: 'oracle', appName: 'chrome' },
ocrText: '',
recentTyped: 'ignored',
});
// elementValue (from UIAutomation) wins over the keyboard buffer
assert.ok(title.includes('oracle'), `expected oracle in title, got: ${title}`);
});
test('app-qualified OCR title includes app name', () => {
const title = buildCaptureTitle({
mode: 'fullscreen',
metadata: { appName: 'code' },
ocrText: 'Save',
});
assert.equal(title, 'Click Save in VS Code');
});
test('ai prompts include the deterministic OCR-backed title candidate', () => {
const { prompt } = buildAiPrompt({
captureContext: {
windowTitle: 'Google Chrome ** PR reviews ** /chrome/tyler/autodoc',
appName: 'Google Chrome',
ocrText: 'New tab',
titleCandidate: 'Click New tab',
mode: 'content',
},
});
assert.match(prompt, /Suggested title: Click New tab/);
});
test('ocr crop rectangles clamp to the image bounds', (t) => {
const root = makeTmpDir('text-intel-crop');
t.after(() => rmrf(root));
const service = new TextIntelService({
store: { settingsDir: root },
settings: makeSettings(),
getWindow: () => null,
dataDir: root,
fetchImpl: global.fetch,
});
const frame = {
size: { width: 1000, height: 500 },
display: { bounds: { x: 0, y: 0, width: 1000, height: 500 } },
};
const topLeft = service.cropRectForPoint(frame, { x: 5, y: 5 });
assert.deepEqual(topLeft, { x: 0, y: 0, width: 420, height: 220 });
const bottomRight = service.cropRectForPoint(frame, { x: 995, y: 495 });
assert.deepEqual(bottomRight, { x: 580, y: 280, width: 420, height: 220 });
});
test('ocr failures fall back to empty text instead of crashing', async (t) => {
const root = makeTmpDir('text-intel-ocr-fallback');
t.after(() => rmrf(root));
const service = new TextIntelService({
store: { settingsDir: root },
settings: makeSettings(),
getWindow: () => null,
dataDir: root,
fetchImpl: global.fetch,
});
service.getWorker = async () => {
throw new Error('tesseract missing');
};
const result = await service.ocrAroundClick({
image: {},
size: { width: 100, height: 100 },
display: { bounds: { x: 0, y: 0, width: 100, height: 100 } },
}, { x: 50, y: 50 });
assert.deepEqual(result, { text: '', confidence: null });
});
test('ai response normalization and application keeps fields structured', () => {
const patch = normalizeAiPatch(JSON.stringify({
title: 'Open settings',
description: 'Pick the AI tab.',
blocks: [
{
kind: 'text',
position: 'after-description',
level: 'tip',
title: 'Tip',
body: 'Use the local Ollama model.',
},
{
kind: 'code',
language: 'bash',
code: 'ollama pull llama3.2:1b',
},
{
kind: 'table',
rows: [['Name', 'Value'], ['Host', '127.0.0.1']],
},
],
}));
const step = createStep({
title: 'Old title',
descriptionHtml: '<p>Old text</p>',
textBlocks: [{ id: 'tb1', order: 1, position: 'after-description', level: 'info', title: 'Old tip', descriptionHtml: '<p>Old body</p>' }],
codeBlocks: [{ id: 'cb1', order: 2, language: 'text', code: 'old' }],
tableBlocks: [{ id: 'tbl1', order: 3, rows: [['x']] }],
});
const updated = applyAiPatchToStep(step, patch, { target: 'all' });
assert.equal(updated.title, 'Open settings');
assert.equal(updated.descriptionHtml, '<p>Pick the AI tab.</p>');
assert.equal(updated.textBlocks.length, 1);
assert.equal(updated.textBlocks[0].level, 'success');
assert.equal(updated.textBlocks[0].descriptionHtml, '<p>Use the local Ollama model.</p>');
assert.equal(updated.codeBlocks[0].code, 'ollama pull llama3.2:1b');
assert.deepEqual(updated.tableBlocks[0].rows, [['Name', 'Value'], ['Host', '127.0.0.1']]);
});
test('ollama connection test reports installed models', async (t) => {
const root = makeTmpDir('text-intel-ai');
t.after(() => rmrf(root));
const service = new TextIntelService({
store: { settingsDir: root },
settings: makeSettings(),
getWindow: () => null,
dataDir: root,
fetchImpl: async (url) => {
const pathname = new URL(url).pathname;
if (pathname === '/api/tags') {
return {
ok: true,
json: async () => ({
models: [
{ name: 'llama3.2:1b' },
{ name: 'qwen3:0.6b' },
],
}),
};
}
if (pathname === '/api/show') {
return {
ok: true,
json: async () => ({
capabilities: ['completion', 'vision'],
}),
};
}
throw new Error(`unexpected fetch: ${pathname}`);
},
});
const result = await service.testAiConnection();
assert.equal(result.ok, true);
assert.equal(result.installed, true);
assert.equal(result.model, 'llama3.2:1b');
assert.equal(result.vision, true);
});
test('vision-capable models receive the screenshot in the chat request', async (t) => {
const root = makeTmpDir('text-intel-ai-vision');
t.after(() => rmrf(root));
const imagePath = path.join(root, 'step.png');
fs.writeFileSync(imagePath, Buffer.from('fake screenshot bytes'));
const step = createStep({
title: 'Old title',
descriptionHtml: '<p>Old text</p>',
image: {
originalPath: 'original.png',
workingPath: 'working.png',
size: { width: 10, height: 10 },
},
captureMetadata: {
windowTitle: 'Settings',
appName: 'chrome',
ocrText: 'Open settings',
titleCandidate: 'Open settings',
mode: 'fullscreen',
},
});
const fetchCalls = [];
const service = new TextIntelService({
store: {
settingsDir: root,
getGuide: () => ({ guideId: 'g1', title: 'Guide', descriptionHtml: '', stepsOrder: ['s1'] }),
getStep: () => step,
stepImagePath: () => imagePath,
saveStep: (_, next) => next,
},
settings: makeSettings(),
getWindow: () => null,
dataDir: root,
fetchImpl: async (url, init = {}) => {
const pathname = new URL(url).pathname;
fetchCalls.push({ pathname, init });
if (pathname === '/api/show') {
return {
ok: true,
json: async () => ({
capabilities: ['completion', 'vision'],
}),
};
}
if (pathname === '/api/chat') {
return {
ok: true,
json: async () => ({
message: {
content: JSON.stringify({
title: 'Open settings',
description: 'Use the AI tab.',
}),
},
}),
};
}
throw new Error(`unexpected fetch: ${pathname}`);
},
});
const result = await service.generateStepPatch({
guideId: 'g1',
stepId: 's1',
target: 'all',
});
assert.equal(result.ok, true);
const chatCall = fetchCalls.find((call) => call.pathname === '/api/chat');
assert.ok(chatCall, 'expected an Ollama chat request');
const body = JSON.parse(chatCall.init.body);
assert.deepEqual(body.messages[1].images, [fs.readFileSync(imagePath).toString('base64')]);
assert.match(body.messages[1].content, /Screenshot: attached/i);
});
test('invalid ollama output fails safely without saving the step', async (t) => {
const root = makeTmpDir('text-intel-ai-invalid');
t.after(() => rmrf(root));
let saveCalls = 0;
const step = createStep({
title: 'Old title',
descriptionHtml: '<p>Old text</p>',
});
const service = new TextIntelService({
store: {
settingsDir: root,
getGuide: () => ({ guideId: 'g1', title: 'Guide', descriptionHtml: '', stepsOrder: ['s1'] }),
getStep: () => step,
stepImagePath: () => null,
saveStep: () => {
saveCalls += 1;
throw new Error('save should not be called');
},
},
settings: makeSettings(),
getWindow: () => null,
dataDir: root,
fetchImpl: async () => ({
ok: true,
json: async () => ({
message: {
content: 'not json at all',
},
}),
}),
});
const result = await service.generateStepPatch({
guideId: 'g1',
stepId: 's1',
target: 'all',
});
assert.equal(result.ok, false);
assert.match(result.reason, /JSON/i);
assert.equal(saveCalls, 0);
assert.equal(step.title, 'Old title');
});