ai smart auto-documentation and text rewrite

- Store captureMetadata (OCR text, window/app/element info) with each
  step at capture time so AI always has the original rich context.
- Add buildCaptureContext() to TextIntelService; capture.js uses it
  instead of buildCaptureTitle() so both title and metadata come from
  one pass.
- generateStepPatch() prefers stored captureMetadata over re-running
  OCR, giving the AI the best possible context when the user clicks
  an AI button later.
- Add autoDoc setting: when enabled every capture (shoot, region, and
  session hotkey/click) is automatically documented by AI. Manual
  captures await AI before returning; session captures fire-and-forget
  and push a step:updated event so the renderer reloads seamlessly.
- Add ai:rewriteText IPC and rewriteText() method for plain-text
  polishing via a separate callOllamaText() that skips JSON mode.
- Add "AI Rewrite" section in the editor right panel: textarea + AI
  button that rewrites whatever the user types in place.
- Improve buildAiPrompt() rules: action-focused title instructions,
  explicit anti-junk rules (no "Capture the screen / OCR" blocks),
  and a context-quality gate that suppresses blocks when context is
  thin.
- Add autoDoc checkbox to AI settings dialog.
- Renderer handles step:updated to reload the selected step after
  background auto-doc finishes.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
This commit is contained in:
2026-06-24 09:17:31 -05:00
co-authored by Claude Sonnet 4.6
parent 375c14b028
commit d244626583
10 changed files with 255 additions and 32 deletions
+8 -6
View File
@@ -1316,15 +1316,15 @@ public static class SFMouseHook {
};
}
async buildStepTitle(mode, frame, clickPos = null, clickMeta = null) {
async buildStepMeta(mode, frame, clickPos = null, clickMeta = null) {
try {
if (this.textIntel && typeof this.textIntel.buildCaptureTitle === 'function') {
return await this.textIntel.buildCaptureTitle({ mode, frame, clickPos, clickMeta });
if (this.textIntel && typeof this.textIntel.buildCaptureContext === 'function') {
return await this.textIntel.buildCaptureContext({ mode, frame, clickPos, clickMeta });
}
} catch {
// fall back to the local semantic title below
// fall back
}
return this.autoTitle(mode);
return { title: this.autoTitle(mode), captureMetadata: null };
}
async storeFrameAsStep(guideId, mode, frame, clickPos = null, clickMeta = null) {
@@ -1350,8 +1350,10 @@ public static class SFMouseHook {
}
}
const { title, captureMetadata } = await this.buildStepMeta(mode, frame, clickPos, clickMeta);
const step = this.store.addStep(guideId, {
title: await this.buildStepTitle(mode, frame, clickPos, clickMeta),
title,
captureMetadata,
annotations,
focusedView: {
enabled: Boolean(this.settings.get('editor.focusedViewDefaultForNewSteps')),