diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..51ba3b1 --- /dev/null +++ b/Makefile @@ -0,0 +1,56 @@ +.PHONY: help dev web server migrate ext ext:watch ext:test test typecheck build prod down logs + +help: + @echo "make dev — postgres + API in docker, web panel on :5173" + @echo "make web — web panel dev server (Vite, :5173)" + @echo "make migrate — apply alembic migrations (docker)" + @echo "make ext — build the extension (chrome + firefox)" + @echo "make ext:watch — rebuild the extension on change" + @echo "make ext:test — e2e smoke test of the built extension (needs dev stack up)" + @echo "make test — server tests" + @echo "make typecheck — vue-tsc for web + extension" + @echo "make build — production build of the web panel" + @echo "make prod — production stack (caddy :8081 + API + postgres)" + @echo "make down — stop the dev stack" + @echo "make logs — follow server logs" + +dev: + docker compose up -d --build + $(MAKE) web + +web: + npm run dev -w @ltt/web + +migrate: + docker compose exec server alembic upgrade head + +ext: + npm run build -w @ltt/extension + +ext:watch: + npm run watch:chrome -w @ltt/extension + +ext:test: + node packages/extension/e2e.mjs + +test: + docker compose exec server pytest tests/ -q + +typecheck: + npm run typecheck -w @ltt/web + npm run typecheck -w @ltt/extension + +build: + npm run build -w @ltt/web + npm run build -w @ltt/extension + +prod: + npm run build -w @ltt/web + docker compose -f docker-compose.prod.yml up -d --build + +down: + docker compose down + -docker compose -f docker-compose.prod.yml down + +logs: + docker compose logs -f server \ No newline at end of file diff --git a/README.md b/README.md new file mode 100644 index 0000000..394317f --- /dev/null +++ b/README.md @@ -0,0 +1,88 @@ +# Live Testing Tool + +Сервис для передачи багов от тестировщиков разработчикам: заметки на элементах +страницы со скриншотами и аннотациями, запись алгоритма воспроизведения, +веб-панель с неугадываемыми ссылками для Jira. + +## Состав + +| Часть | Стек | Где | +|---|---|---| +| Сервер (API) | Python 3.13, FastAPI, SQLAlchemy async, Postgres 16, Alembic | `server/` (Docker) | +| Веб-панель | Vue 3, vue-router, vue-i18n (en/ru), [gnexus-ui-kit](https://git.gnexus.space/git/root/gnexus-ui-kit.git) | `packages/web` | +| Общий код UI | Аннотации на скриншотах, контекст элемента | `packages/ui` | +| Расширение | Chrome + Firefox (MV3), Vite, closed shadow-DOM overlay | `packages/extension` | +| Общие типы/клиент API | TypeScript | `packages/shared` | + +## Быстрый старт + +```sh +npm install # workspaces: shared, ui, web, extension +make dev # postgres + API в docker, панель на http://localhost:5173 +``` + +API: `http://localhost:8001` (healthcheck: `/api/healthz`). +Миграции применяются при старте контейнера; вручную — `make migrate`. + +## Расширение + +```sh +make ext # сборка в packages/extension/dist/{chrome,firefox} +``` + +Установка (dev): + +- **Chrome/Chromium**: `chrome://extensions` → Developer mode → *Load unpacked* → `packages/extension/dist/chrome` +- **Firefox**: `about:debugging#/runtime/this-firefox` → *Load Temporary Add-on* → `packages/extension/dist/firefox/manifest.json` + +Затем: иконка расширения → Settings (или Alt+Shift+B → контекстное меню настроек +не используется, страница настроек открывается из chrome://extensions → Details → +Extension options): указать URL сервера, войти (email/пароль), выбрать проект по умолчанию. + +Управление: + +- **Alt+Shift+B** — пикер элемента: клик по элементу → скриншот → аннотации + (перо/стрелка/прямоугольник/текст) → комментарий → Submit. +- **Alt+Shift+R** — старт/стоп записи: клики, ввод (с дебаунсом 250 мс), переходы + по URL; скриншоты ключевых шагов прикладываются автоматически. Кнопка **Note** + в рекордер-баре открывает тот же пикер для заметки на элемент. + +Пароли не сохраняются: маскирование ввода дублируется на сервере. + +### Почему /assets + +`kit.css` из gnexus-ui-kit ссылается на шрифты абсолютными путями `/assets/...`. +Веб-панель копирует ассеты кита в свой `dist/assets` (dev-миддлварь + `vite-plugin-static-copy`). +Расширение раздаёт их как `web_accessible_resources` и переписывает пути на +`chrome-extension://.../assets/...` при загрузке стилей в shadow root +(constructable stylesheets — CSS хост-страницы и её CSP не затрагиваются). + +## E2E-проверка расширения + +`make ext:test` (нужен поднятый `make dev`) — headless Chromium грузит собранное +расширение, проходит сценарий «пикер → заметка» и «рекордер: клики + ввод», +проверяет репорты через API. + +## Прод + +```sh +make prod # web dist + docker-compose.prod.yml +# панель и API на одном origin: http://localhost:8081 (caddy: статика + /api → server) +``` + +## Ссылки и доступ + +- PK — UUIDv7; публичные ссылки — случайные 128-битные токены (`/p/`, `/r/`). +- Токен = авторизация: страницы проекта и репорта открываются без логина. +- Ротация токена: кнопка в панели (`POST .../share/rotate`). + +## Структура + +``` +server/app/routers/ auth, projects, reports, uploads +packages/web/src/ pages (Login, Register, Projects, Project, Report, Settings), i18n +packages/ui/src/ AnnotationEditor, ScreenshotViewer, ElementContext, EnvironmentInfo +packages/extension/src/ background (SW: сеть, captureVisibleTab, буфер рекордера), + content (shadow-DOM оверлей: PickerLayer, NoteComposer, RecorderBar), + options (сервер, логин, проект по умолчанию) +``` \ No newline at end of file diff --git a/deploy/Caddyfile b/deploy/Caddyfile new file mode 100644 index 0000000..1f73dce --- /dev/null +++ b/deploy/Caddyfile @@ -0,0 +1,10 @@ +:80 { + handle /api/* { + reverse_proxy server:8000 + } + handle { + root * /srv/web + try_files {path} /index.html + file_server + } +} \ No newline at end of file diff --git a/docker-compose.prod.yml b/docker-compose.prod.yml new file mode 100644 index 0000000..ad48d81 --- /dev/null +++ b/docker-compose.prod.yml @@ -0,0 +1,42 @@ +# Production stack: caddy (static panel + /api proxy) + server + postgres. +# Build the web panel first: npm run build -w @ltt/web +services: + postgres: + image: postgres:16-alpine + environment: + POSTGRES_USER: ltt + POSTGRES_PASSWORD: ltt + POSTGRES_DB: ltt + volumes: + - postgres_data:/var/lib/postgresql/data + healthcheck: + test: ["CMD-SHELL", "pg_isready -U ltt"] + interval: 2s + timeout: 3s + retries: 15 + + server: + build: ./server + environment: + DATABASE_URL: postgresql+asyncpg://ltt:ltt@postgres:5432/ltt + FILES_DIR: /srv/data/files + depends_on: + postgres: + condition: service_healthy + volumes: + - ./data:/srv/data + + caddy: + image: caddy:2-alpine + ports: + - "8081:80" # panel + API on http://host:8081 + volumes: + - ./deploy/Caddyfile:/etc/caddy/Caddyfile:ro + - ./packages/web/dist:/srv/web:ro + - caddy_data:/data + depends_on: + - server + +volumes: + postgres_data: + caddy_data: \ No newline at end of file diff --git a/package-lock.json b/package-lock.json index 25b1e9e..7f81c91 100644 --- a/package-lock.json +++ b/package-lock.json @@ -573,6 +573,10 @@ "integrity": "sha512-T7jf+5zgsZHwNJ4lvQ7/aezbyk0nNX+zJVWpmHA7VYsEx7a7qr5Rg5IbtJFqkgze5Y2sruq1RUY8Q837Od7iFw==", "license": "MIT" }, + "node_modules/@ltt/extension": { + "resolved": "packages/extension", + "link": true + }, "node_modules/@ltt/shared": { "resolved": "packages/shared", "link": true @@ -1095,6 +1099,13 @@ "undici-types": "~6.21.0" } }, + "node_modules/@types/webextension-polyfill": { + "version": "0.12.6", + "resolved": "https://registry.npmjs.org/@types/webextension-polyfill/-/webextension-polyfill-0.12.6.tgz", + "integrity": "sha512-XTNQSGiQaipt1iocbpHSjYHd+Ujcya+K5H4H2uH7JwycReTFixoYqGWIKih2Ec072NuEW2lAyXRDYmlEx00+cw==", + "dev": true, + "license": "MIT" + }, "node_modules/@vitejs/plugin-vue": { "version": "5.2.4", "resolved": "https://registry.npmjs.org/@vitejs/plugin-vue/-/plugin-vue-5.2.4.tgz", @@ -2719,6 +2730,12 @@ "typescript": ">=5.0.0" } }, + "node_modules/webextension-polyfill": { + "version": "0.12.0", + "resolved": "https://registry.npmjs.org/webextension-polyfill/-/webextension-polyfill-0.12.0.tgz", + "integrity": "sha512-97TBmpoWJEE+3nFBQ4VocyCdLKfw54rFaJ6EVQYLBCXqCIpLSZkwGgASpv4oPt9gdKCJ80RJlcmNzNn008Ag6Q==", + "license": "MPL-2.0" + }, "node_modules/wrap-ansi": { "version": "7.0.0", "resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz", @@ -2783,6 +2800,24 @@ "node": ">=12" } }, + "packages/extension": { + "name": "@ltt/extension", + "version": "0.1.0", + "dependencies": { + "@ltt/shared": "*", + "@ltt/ui": "*", + "gnexus-ui-kit": "git+https://git.gnexus.space/git/root/gnexus-ui-kit.git#5227ba022e5da5ef7df5a4b4ed463c25abd1f85b", + "vue": "^3.5.0", + "webextension-polyfill": "^0.12.0" + }, + "devDependencies": { + "@types/webextension-polyfill": "^0.12.0", + "@vitejs/plugin-vue": "^5.2.0", + "typescript": "^5.7.0", + "vite": "^6.0.0", + "vue-tsc": "^2.1.10" + } + }, "packages/shared": { "name": "@ltt/shared", "version": "0.1.0", diff --git a/packages/extension/e2e.mjs b/packages/extension/e2e.mjs new file mode 100644 index 0000000..cc90368 --- /dev/null +++ b/packages/extension/e2e.mjs @@ -0,0 +1,174 @@ +/** + * E2E smoke test for the extension (chrome target) against the dev server: + * register a user via API, sign in through the options page, pick an element + * on a local test page, and submit a note report. + * Usage: node e2e.mjs (server on :8001 must be running) + */ +import { chromium } from "playwright-core"; +import { createServer } from "node:http"; +import { readFile } from "node:fs/promises"; +import { join, dirname } from "node:path"; +import { fileURLToPath } from "node:url"; +import { homedir } from "node:os"; + +const root = dirname(fileURLToPath(import.meta.url)); +const SERVER = "http://localhost:8001"; +const PAGE_PORT = 8899; +const stamp = Date.now(); +const EMAIL = `ext-e2e-${stamp}@example.com`; +const PASSWORD = "e2e-password-1"; + +const pageHtml = `E2E target page + +

E2E page

+ +
+ +
+`; + +const httpServer = createServer((req, res) => { + res.writeHead(200, { "Content-Type": "text/html" }); + res.end(pageHtml); +}); +await new Promise((resolve) => httpServer.listen(PAGE_PORT, resolve)); + +async function api(path, options = {}) { + const response = await fetch(SERVER + path, options); + if (!response.ok) throw new Error(`API ${path} -> ${response.status}: ${await response.text()}`); + return response.json(); +} + +// --- seed: user + project via API --- +const user = await api("/api/auth/register", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ nickname: `ext-e2e-${stamp}`, email: EMAIL, password: PASSWORD }), +}); +const login = await api("/api/auth/login", { + method: "POST", + headers: { "Content-Type": "application/json", "X-Client": "extension" }, + body: JSON.stringify({ email: EMAIL, password: PASSWORD }), +}); +const project = await api("/api/projects", { + method: "POST", + headers: { "Content-Type": "application/json", Authorization: `Bearer ${login.token}` }, + body: JSON.stringify({ name: "Extension E2E" }), +}); +console.log("seeded user+project:", project.id); + +// --- launch with the extension --- +const executablePath = join(homedir(), ".cache/ms-playwright/chromium-1234/chrome-linux64/chrome"); +const context = await chromium.launchPersistentContext("", { + executablePath, + headless: true, + args: [ + `--disable-extensions-except=${join(root, "dist/chrome")}`, + `--load-extension=${join(root, "dist/chrome")}`, + ], +}); + +let [worker] = context.serviceWorkers(); +if (!worker) worker = await context.waitForEvent("serviceworker", { timeout: 10000 }); +const extensionId = new URL(worker.url()).host; +console.log("extension id:", extensionId); + +try { + // --- options page: sign in, choose default project --- + const options = await context.newPage(); + await options.goto(`chrome-extension://${extensionId}/src/options/options.html`); + await options.fill('input[type="email"]', EMAIL); + await options.fill('input[type="password"]', PASSWORD); + await options.click("text=Sign in"); + await options.waitForSelector("text=Signed in as", { timeout: 10000 }); + await options.selectOption("select", project.id); + console.log("options page: signed in, project selected"); + + // --- target page: pick an element and submit a note --- + const page = await context.newPage(); + page.on("console", (msg) => { + if (msg.type() === "error") console.log("[page error]", msg.text()); + }); + await page.goto(`http://localhost:${PAGE_PORT}/`); + // headless Chromium doesn't deliver extension shortcut keys; trigger the + // same code path the toolbar button runs through the debug hook + await worker.evaluate(() => self.__lttTriggerAction()); + await page.waitForTimeout(1000); + // the picker overlay intentionally intercepts pointer events, so use raw + // mouse events at the target's coordinates instead of locator.hover/click + const box = await page.locator("#target").boundingBox(); + if (!box) throw new Error("target not found"); + await page.mouse.move(box.x + box.width / 2, box.y + box.height / 2); + await page.waitForTimeout(300); + await page.mouse.down(); + await page.mouse.up(); + + // the composer autofocuses its title input; drive it with real keys since + // Playwright locators can't pierce the closed shadow root + await page.waitForTimeout(1500); // picker click → composer mount + focus + await page.keyboard.type("Button looks broken after hover"); + await page.keyboard.press("Tab"); + await page.keyboard.type("Repro: hover, then click."); + await page.keyboard.press("Control+Enter"); + await page.waitForTimeout(4000); // capture + upload + submit + await page.screenshot({ path: "/tmp/ltt-e2e-after-submit.png" }); + + // --- verify the report landed on the server --- + const reports = await api(`/api/projects/${project.id}/reports`, { + headers: { Authorization: `Bearer ${login.token}` }, + }); + const items = reports.items ?? reports; + if (!items.length) throw new Error("No reports found after submit"); + const report = await api(`/api/reports/${items[0].share_token}`); + console.log("report:", JSON.stringify({ title: report.title, selector: report.element?.selector })); + const noteOk = + report.title.includes("Button looks broken") && + report.element?.selector?.includes("buggy-button") && + report.attachments?.length > 0; + + // --- recorder: start, click the button, type, stop → recording report --- + await worker.evaluate(() => self.__lttToggleRecorder()); + await page.waitForTimeout(800); + const buttonBox = await page.locator("#target").boundingBox(); + await page.mouse.click(buttonBox.x + buttonBox.width / 2, buttonBox.y + buttonBox.height / 2); + const inputBox = await page.locator("#name-input").boundingBox(); + await page.mouse.click(inputBox.x + 10, inputBox.y + inputBox.height / 2); + await page.keyboard.type("Hello recorder"); + await page.waitForTimeout(600); // let the input debounce flush + await worker.evaluate(() => self.__lttToggleRecorder()); + await page.waitForTimeout(4000); // upload step screenshots + submit + + const itemsAfter = (await api(`/api/projects/${project.id}/reports`, { + headers: { Authorization: `Bearer ${login.token}` }, + })); + const all = itemsAfter.items ?? itemsAfter; + const recordingSummary = await api(`/api/reports/${all[0].share_token}`); + const steps = recordingSummary.steps ?? []; + console.log( + "recording:", + JSON.stringify({ + type: recordingSummary.type, + stepCount: steps.length, + types: steps.map((s) => s.type), + inputValue: steps.find((s) => s.type === "input")?.data?.value, + }) + ); + const recorderOk = + recordingSummary.type === "recording" && + steps.some((s) => s.type === "click") && + steps.some((s) => s.type === "input" && s.data?.value === "Hello recorder") && + steps.some((s) => s.screenshot_attachment_id != null); + + const ok = noteOk && recorderOk; + console.log("background log:", JSON.stringify(await worker.evaluate(() => self.__lttDebug()), null, 1)); + console.log(noteOk ? "note flow OK" : "note flow MISMATCH"); + console.log(recorderOk ? "recorder flow OK" : "recorder flow MISMATCH"); + console.log(ok ? "E2E OK" : "E2E FAILED"); + process.exitCode = ok ? 0 : 1; +} catch (error) { + console.error("E2E FAILED:", error); + process.exitCode = 1; +} finally { + await context.close(); + httpServer.close(); +} \ No newline at end of file diff --git a/packages/extension/manifest.template.json b/packages/extension/manifest.template.json new file mode 100644 index 0000000..38eba83 --- /dev/null +++ b/packages/extension/manifest.template.json @@ -0,0 +1,31 @@ +{ + "manifest_version": 3, + "name": "Live Testing Tool", + "description": "Capture bug reports: element notes with screenshots and recorded reproduction steps.", + "version": "0.1.0", + "permissions": ["activeTab", "scripting", "storage", "tabs"], + "host_permissions": [""], + "action": { + "default_title": "Capture a bug report" + }, + "web_accessible_resources": [ + { + "resources": ["assets/*"], + "matches": [""] + } + ], + "options_ui": { + "page": "src/options/options.html", + "open_in_tab": true + }, + "commands": { + "start-picker": { + "suggested_key": { "default": "Alt+Shift+B" }, + "description": "Pick an element to annotate" + }, + "toggle-recorder": { + "suggested_key": { "default": "Alt+Shift+R" }, + "description": "Start/stop recording reproduction steps" + } + } +} \ No newline at end of file diff --git a/packages/extension/package.json b/packages/extension/package.json new file mode 100644 index 0000000..a870a82 --- /dev/null +++ b/packages/extension/package.json @@ -0,0 +1,29 @@ +{ + "name": "@ltt/extension", + "version": "0.1.0", + "private": true, + "type": "module", + "scripts": { + "build": "npm run build:chrome && npm run build:firefox", + "build:chrome": "node scripts/build.mjs chrome", + "build:firefox": "node scripts/build.mjs firefox", + "watch:chrome": "node scripts/build.mjs chrome --watch", + "watch:firefox": "node scripts/build.mjs firefox --watch", + "typecheck": "vue-tsc --noEmit", + "manifest": "node scripts/build-manifest.mjs chrome && node scripts/build-manifest.mjs firefox" + }, + "dependencies": { + "@ltt/shared": "*", + "@ltt/ui": "*", + "gnexus-ui-kit": "git+https://git.gnexus.space/git/root/gnexus-ui-kit.git#5227ba022e5da5ef7df5a4b4ed463c25abd1f85b", + "vue": "^3.5.0", + "webextension-polyfill": "^0.12.0" + }, + "devDependencies": { + "@types/webextension-polyfill": "^0.12.0", + "@vitejs/plugin-vue": "^5.2.0", + "typescript": "^5.7.0", + "vite": "^6.0.0", + "vue-tsc": "^2.1.10" + } +} diff --git a/packages/extension/scripts/build-manifest.mjs b/packages/extension/scripts/build-manifest.mjs new file mode 100644 index 0000000..23d3b06 --- /dev/null +++ b/packages/extension/scripts/build-manifest.mjs @@ -0,0 +1,51 @@ +/** + * Emits manifest.json into the build output for the given target. + * Usage: node scripts/build-manifest.mjs chrome|firefox + * (or import { writeManifest } from another script) + */ +import { readFileSync, writeFileSync, existsSync } from "node:fs"; +import { join, dirname } from "node:path"; +import { fileURLToPath } from "node:url"; + +const root = dirname(fileURLToPath(import.meta.url)); + +export function writeManifest(target) { + const dist = join(root, "..", "dist", target); + + const template = JSON.parse(readFileSync(join(root, "..", "manifest.template.json"), "utf8")); + + const manifest = structuredClone(template); + manifest.version = "0.1.0"; + + if (target === "chrome") { + // bundles are IIFE (self-contained), so the worker stays classic + manifest.background = { + service_worker: "background.js", + }; + } else if (target === "firefox") { + manifest.background = { + scripts: ["background.js"], + }; + manifest.browser_specific_settings = { + gecko: { + id: "live-testing-tool@gnexus.space", + strict_min_version: "128.0", + }, + }; + } + + // sanity check: referenced files must exist in dist + for (const file of ["background.js", "content.js"]) { + if (!existsSync(join(dist, file))) { + console.error(`build-manifest: missing ${file} in ${dist} — run vite build first`); + process.exit(1); + } + } + + writeFileSync(join(dist, "manifest.json"), JSON.stringify(manifest, null, 2)); + console.log(`manifest.json written for ${target} -> ${dist}`); +} + +if (import.meta.url === `file://${process.argv[1]}`) { + writeManifest(process.argv[2] ?? "chrome"); +} \ No newline at end of file diff --git a/packages/extension/scripts/build.mjs b/packages/extension/scripts/build.mjs new file mode 100644 index 0000000..a15275b --- /dev/null +++ b/packages/extension/scripts/build.mjs @@ -0,0 +1,58 @@ +import { spawn } from "node:child_process"; +import { createRequire } from "node:module"; +import { dirname } from "node:path"; +import { writeManifest } from "./build-manifest.mjs"; + +const require = createRequire(import.meta.url); +// vite's ./bin subpath isn't exported; resolve via the package directory +const viteBin = new URL("file://" + dirname(require.resolve("vite/package.json")) + "/bin/vite.js").pathname; + +/** + * Vite can't emit IIFE for a multi-entry (code-splitting) build, so each + * extension entry (background, content, options) is built as its own + * single-chunk bundle into the same dist/ directory. + * Usage: node scripts/build.mjs [--watch] + */ +const target = process.argv[2]; +const watch = process.argv.includes("--watch"); +if (target !== "chrome" && target !== "firefox") { + console.error("Usage: node scripts/build.mjs [--watch]"); + process.exit(1); +} + +// order matters: every pass writes assets/style.css (cssCodeSplit:false), so +// the content pass — whose stylesheet is the union of all styles — runs last +const entries = ["background", "options", "content"]; + +function run(entry) { + return new Promise((resolve, reject) => { + const child = spawn( + process.execPath, + [viteBin, "build", "--mode", target, ...(watch ? ["--watch"] : [])], + { + cwd: new URL("..", import.meta.url).pathname, + env: { ...process.env, LTT_ENTRY: entry }, + stdio: "inherit", + } + ); + child.on("exit", (code) => (code === 0 ? resolve() : reject(new Error(`${target}/${entry} exited with ${code}`)))); + }); +} + +try { + for (const entry of entries) { + if (watch) { + // keep all three watchers alive + run(entry).catch((error) => { + console.error(error.message); + process.exit(1); + }); + } else { + await run(entry); + } + } + if (!watch) writeManifest(target); +} catch (error) { + console.error(error.message); + process.exit(1); +} \ No newline at end of file diff --git a/packages/extension/src/background/api.ts b/packages/extension/src/background/api.ts new file mode 100644 index 0000000..229e0b8 --- /dev/null +++ b/packages/extension/src/background/api.ts @@ -0,0 +1,69 @@ +import type { HttpClient, ReportCreateIn, ReportDetail, AnnotationShape } from "@ltt/shared"; +import { getHttpClient, getSettings, type Settings } from "./settings"; +import type { SubmitNotePayload } from "../lib/messages"; + +async function dataUrlToFile(dataUrl: string, filename: string): Promise { + const response = await fetch(dataUrl); + const blob = await response.blob(); + return new File([blob], filename, { type: blob.type || "image/png" }); +} + +async function uploadFile(client: HttpClient, dataUrl: string, filename: string): Promise { + const file = await dataUrlToFile(dataUrl, filename); + const result = await client.upload<{ file_id: string }>("/api/uploads", file); + return result.file_id; +} + +/** Prepares and submits an element-note report from the NoteComposer payload. */ +export async function submitNoteReport(payload: SubmitNotePayload): Promise<{ report_token: string }> { + const settings = await getSettings(); + + if (!settings.token) throw new Error("Not signed in — open the extension settings"); + if (!settings.defaultProjectId) throw new Error("No default project selected — open the extension settings"); + + const client = await getHttpClient(); + const attachmentIds: string[] = []; + const shapesByFile: Record = {}; + + const screenshotId = await uploadFile(client, payload.screenshotDataUrl, "screenshot.png"); + attachmentIds.push(screenshotId); + if (payload.annotationShapes?.length) { + shapesByFile[screenshotId] = payload.annotationShapes; + } + + for (const [index, file] of payload.files.entries()) { + attachmentIds.push(await uploadFile(client, file.dataUrl, file.filename || `attachment-${index + 1}`)); + } + + const body: ReportCreateIn = { + project_id: settings.defaultProjectId, + type: "element_note", + title: payload.title || "Element note", + description: [payload.comment, ...payload.links.map((l) => l)].filter(Boolean).join("\n\n") || null, + page_url: payload.pageUrl || null, + page_title: payload.pageTitle || null, + environment: payload.environment, + element: payload.element, + attachment_ids: attachmentIds, + annotation_shapes: Object.keys(shapesByFile).length ? shapesByFile : null, + }; + + const report = await client.post("/api/reports", body); + return { report_token: report.share_token }; +} + +/** Prepares and submits a recording report from buffered recorder steps. */ +export async function submitRecordingReport(input: { + payload: Omit; + settings?: Settings; +}): Promise<{ report_token: string }> { + const settings = input.settings ?? (await getSettings()); + if (!settings.token) throw new Error("Not signed in — open the extension settings"); + if (!settings.defaultProjectId) throw new Error("No default project selected — open the extension settings"); + const client = await getHttpClient(); + const report = await client.post("/api/reports", { + ...input.payload, + project_id: settings.defaultProjectId, + }); + return { report_token: report.share_token }; +} \ No newline at end of file diff --git a/packages/extension/src/background/index.ts b/packages/extension/src/background/index.ts new file mode 100644 index 0000000..74f0772 --- /dev/null +++ b/packages/extension/src/background/index.ts @@ -0,0 +1,385 @@ +import browser from "webextension-polyfill"; +import type { Runtime } from "webextension-polyfill"; +import { submitNoteReport, submitRecordingReport } from "./api"; +import { getSettings, saveSettings, login, logout, getHttpClient } from "./settings"; +import type { + BackgroundResponse, + RecordEvent, + RecorderState, +} from "../lib/messages"; + +const OVERLAY_ACTIVE = new Set(); + +// debug ring buffer (inspected from the e2e harness / DevTools) +const DEBUG_LOG: string[] = []; +function debugLog(line: string) { + DEBUG_LOG.push(`${new Date().toISOString()} ${line}`); + if (DEBUG_LOG.length > 50) DEBUG_LOG.shift(); +} + +// ---------- recorder state (lives in the worker; survives page navigations) ---------- + +interface RecorderBuffer { + recording: boolean; + startedAt: number | null; + events: RecordEvent[]; + pageUrl: string | null; + pageTitle: string | null; + environment: Record | null; + /** screenshots captured for click events, keyed by event index */ + screenshots: Map; + lastUrl: string | null; +} + +const recorders = new Map(); + +function recorder(tabId: number): RecorderBuffer { + let buffer = recorders.get(tabId); + if (!buffer) { + buffer = { + recording: false, + startedAt: null, + events: [], + pageUrl: null, + pageTitle: null, + environment: null, + screenshots: new Map(), + lastUrl: null, + }; + recorders.set(tabId, buffer); + } + return buffer; +} + +async function captureVisibleTab(tabId: number): Promise { + // the first argument of captureVisibleTab is a *window* id + const tab = await browser.tabs.get(tabId); + return (await browser.tabs.captureVisibleTab(tab.windowId ?? undefined, { format: "png" })) as string; +} + +async function captureForEvent(tabId: number, eventIndex: number): Promise { + try { + const dataUrl = await captureVisibleTab(tabId); + recorder(tabId).screenshots.set(eventIndex, dataUrl); + debugLog(`event screenshot ${eventIndex} captured`); + } catch (error) { + // tab may be inactive or chrome-internal — skip the screenshot + debugLog(`event screenshot ${eventIndex} failed: ${error instanceof Error ? error.message : String(error)}`); + } +} + +function ensureRecordingFreshness(buffer: RecorderBuffer) { + // drop stale buffers older than 2 hours + if (buffer.startedAt && Date.now() - buffer.startedAt > 2 * 60 * 60 * 1000) { + buffer.recording = false; + buffer.events = []; + buffer.startedAt = null; + } +} + +async function recorderStart(tabId: number): Promise { + const buffer = recorder(tabId); + const tab = await browser.tabs.get(tabId); + buffer.recording = true; + buffer.startedAt = Date.now(); + buffer.events = []; + buffer.screenshots = new Map(); + buffer.pageUrl = tab.url ?? null; + buffer.pageTitle = tab.title ?? null; + buffer.lastUrl = tab.url ?? null; + buffer.environment = null; // content script supplies it with the first event + return recorderStateOf(buffer); +} + +function recorderStateOf(buffer: RecorderBuffer): RecorderState { + return { + recording: buffer.recording, + stepCount: buffer.events.length, + startedAt: buffer.startedAt, + }; +} + +async function recorderStop(tabId: number): Promise { + const buffer = recorder(tabId); + ensureRecordingFreshness(buffer); + if (!buffer.recording) return recorderStateOf(buffer); + + const settings = await getSettings(); + const attachmentIdByStep = await uploadStepScreenshots(buffer); + const steps = buffer.events.map((event, index) => ({ + type: event.type, + offset_ms: buffer.startedAt ? Math.max(event.at - buffer.startedAt, 0) : 0, + data: event.data, + attachment_id: attachmentIdByStep.get(index) ?? null, + })); + + if (buffer.events.length > 0) { + await submitRecordingReport({ + payload: { + type: "recording", + title: `Recording ${new Date().toLocaleString()} — ${buffer.pageTitle ?? "page"}`, + description: null, + page_url: buffer.pageUrl, + page_title: buffer.pageTitle, + environment: buffer.environment ?? {}, + steps, + attachment_ids: [...attachmentIdByStep.values()], + }, + settings, + }).catch(async (error) => { + // surface submit failure to the content script + buffer.recording = false; + throw error; + }); + } + + buffer.recording = false; + buffer.startedAt = null; + buffer.events = []; + buffer.screenshots = new Map(); + return recorderStateOf(buffer); +} + +/** Uploads captured step screenshots; returns attachment id by event index. */ +async function uploadStepScreenshots(buffer: RecorderBuffer): Promise> { + const client = await getHttpClient(); + // buffer.screenshots is keyed by the event index, which matches the step index + const attachmentIdByStep = new Map(); + + for (const [eventIndex, dataUrl] of buffer.screenshots) { + try { + const response = await fetch(dataUrl); + const blob = await response.blob(); + const file = new File([blob], `step-${eventIndex + 1}.png`, { type: "image/png" }); + const uploaded = await client.upload<{ file_id: string }>("/api/uploads", file); + attachmentIdByStep.set(eventIndex, uploaded.file_id); + debugLog(`step screenshot ${eventIndex} uploaded: ${uploaded.file_id}`); + } catch (error) { + debugLog(`step screenshot ${eventIndex} failed: ${error instanceof Error ? error.message : String(error)}`); + } + } + + return attachmentIdByStep; +} + +// ---------- content-script injection ---------- + +async function ensureContentScript(tabId: number): Promise { + try { + await browser.tabs.sendMessage(tabId, { type: "ping" }); + return; + } catch { + // not injected yet + } + await browser.scripting.executeScript({ + target: { tabId }, + files: ["content.js"], + }); +} + +async function startPicker(tabId: number): Promise { + await ensureContentScript(tabId); + OVERLAY_ACTIVE.add(tabId); + await browser.tabs.sendMessage(tabId, { type: "start_picker" }).catch(async (error) => { + // the page may need a reload for the fresh script; retry once + await browser.scripting.executeScript({ target: { tabId }, files: ["content.js"] }); + await browser.tabs.sendMessage(tabId, { type: "start_picker" }).catch(() => { + throw error; + }); + }); +} + +async function startRecorder(tabId: number): Promise { + await ensureContentScript(tabId); + const state = await recorderStart(tabId); + await browser.tabs.sendMessage(tabId, { type: "recorder_state", recording: true }).catch(() => {}); + return state; +} + +// ---------- wiring ---------- + +browser.action.onClicked.addListener(async (tab) => { + if (tab.id == null) return; + try { + await startPicker(tab.id); + } catch (error) { + console.error("[ltt] failed to start picker:", error); + } +}); + +browser.commands?.onCommand.addListener(async (command) => { + const [tab] = await browser.tabs.query({ active: true, currentWindow: true }); + if (tab?.id == null) return; + if (command === "start-picker") { + await startPicker(tab.id).catch(console.error); + } else if (command === "toggle-recorder") { + const buffer = recorder(tab.id); + if (buffer.recording) { + await recorderStop(tab.id).catch(console.error); + await browser.tabs.sendMessage(tab.id, { type: "recorder_state", recording: false }).catch(() => {}); + } else { + await startRecorder(tab.id).catch(console.error); + } + } +}); + +browser.tabs.onRemoved.addListener((tabId) => { + OVERLAY_ACTIVE.delete(tabId); + recorders.delete(tabId); +}); + +// debug/test hook: lets the e2e harness (and DevTools) trigger the action +// and recorder-toggle handlers the way the toolbar button and shortcuts do +Object.assign(self as unknown as Record, { + __lttDebug: () => DEBUG_LOG, + __lttTriggerAction: async () => { + const [tab] = await browser.tabs.query({ active: true, currentWindow: true }); + if (tab?.id == null) return; + await startPicker(tab.id).catch(console.error); + }, + __lttToggleRecorder: async () => { + const [tab] = await browser.tabs.query({ active: true, currentWindow: true }); + if (tab?.id == null) return; + const buffer = recorder(tab.id); + if (buffer.recording) { + await recorderStop(tab.id).catch(console.error); + await browser.tabs.sendMessage(tab.id, { type: "recorder_state", recording: false }).catch(() => {}); + } else { + await startRecorder(tab.id).catch(console.error); + } + }, +}); + +// keep the recorder's page metadata fresh on SPA navigations +browser.tabs.onUpdated.addListener((tabId, changeInfo, tab) => { + const buffer = recorders.get(tabId); + if (!buffer?.recording) return; + if (changeInfo.url) buffer.pageUrl = changeInfo.url; + if (changeInfo.title) buffer.pageTitle = changeInfo.title; + if (changeInfo.url && buffer.lastUrl !== changeInfo.url && tab.url) { + buffer.events.push({ + type: "url_change", + at: Date.now(), + data: { from_url: buffer.lastUrl, to_url: tab.url, trigger: "tab_updated" }, + }); + buffer.lastUrl = tab.url; + } +}); + +browser.runtime.onMessage.addListener(async (message: unknown, sender: Runtime.MessageSender) => { + const msg = message as { type: string; [key: string]: unknown }; + const tabId = sender.tab?.id; + + switch (msg.type) { + case "ping": + return { ok: true, data: "pong" }; + + case "capture": { + debugLog("capture requested"); + try { + const dataUrl = await captureVisibleTab(tabId!); + debugLog(`capture ok (${dataUrl.length} bytes)`); + return { ok: true, data: { dataUrl } }; + } catch (error) { + debugLog(`capture failed: ${String(error)}`); + return { ok: false, error: String(error) }; + } + } + + case "submit_note": { + debugLog("submit_note received"); + try { + const result = await submitNoteReport(msg.payload as Parameters[0]); + debugLog(`submit_note ok: ${JSON.stringify(result)}`); + return { ok: true, data: result }; + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + debugLog(`submit_note failed: ${message}`); + return { ok: false, error: message }; + } + } + + case "recorder_event": { + const buffer = recorder(tabId!); + if (!buffer.recording) return { ok: true, data: { recording: false } }; + ensureRecordingFreshness(buffer); + const { event } = msg as unknown as { event: RecordEvent }; + buffer.events.push(event); + // auto-screenshot on clicks (bounded to avoid runaway captures) + if (event.type === "click" && buffer.events.length <= 30) { + await captureForEvent(tabId!, buffer.events.length - 1); + } + return { ok: true, data: { stepCount: buffer.events.length } }; + } + + case "recorder_environment": { + const buffer = recorder(tabId!); + buffer.environment = msg.environment as Record; + return { ok: true }; + } + + case "recorder_screenshot": { + // a manual screenshot becomes its own step attached to the recording + const buffer = recorder(tabId!); + if (!buffer.recording) return { ok: true, data: { recording: false } }; + ensureRecordingFreshness(buffer); + buffer.events.push({ + type: "screenshot", + at: Date.now(), + data: { trigger: "manual" }, + }); + await captureForEvent(tabId!, buffer.events.length - 1); + return { ok: true, data: { stepCount: buffer.events.length } }; + } + + case "list_projects": { + try { + const client = await getHttpClient(); + const projects = await client.get<{ id: string; name: string }[]>("/api/projects"); + return { ok: true, data: projects }; + } catch (error) { + return { ok: false, error: error instanceof Error ? error.message : String(error) }; + } + } + + case "recorder_stop": { + try { + const state = await recorderStop(tabId!); + return { ok: true, data: state }; + } catch (error) { + return { ok: false, error: error instanceof Error ? error.message : String(error) }; + } + } + + case "recorder_state_request": { + const buffer = tabId != null ? recorders.get(tabId) : undefined; + return { ok: true, data: buffer ? recorderStateOf(buffer) : { recording: false, stepCount: 0, startedAt: null } }; + } + + case "settings_get": { + return { ok: true, data: await getSettings() }; + } + + case "settings_save": { + await saveSettings(msg.patch as Record); + return { ok: true }; + } + + case "login": { + try { + const user = await login(msg.email as string, msg.password as string); + return { ok: true, data: user }; + } catch (error) { + return { ok: false, error: error instanceof Error ? error.message : String(error) }; + } + } + + case "logout": { + await logout(); + return { ok: true }; + } + + default: + return { ok: false, error: `Unknown message type: ${msg.type}` }; + } +}); \ No newline at end of file diff --git a/packages/extension/src/background/settings.ts b/packages/extension/src/background/settings.ts new file mode 100644 index 0000000..3390ed1 --- /dev/null +++ b/packages/extension/src/background/settings.ts @@ -0,0 +1,73 @@ +import { createHttpClient, type HttpClient } from "@ltt/shared"; +import browser from "webextension-polyfill"; + +/** Server settings persisted in storage.local. */ +export interface Settings { + serverUrl: string; + token: string | null; + tokenExpiresAt: string | null; + defaultProjectId: string | null; + user: { id: string; nickname: string; email: string } | null; +} + +const DEFAULTS: Settings = { + serverUrl: "http://localhost:8001", + token: null, + tokenExpiresAt: null, + defaultProjectId: null, + user: null, +}; + +export async function getSettings(): Promise { + const stored = await browser.storage.local.get(Object.keys(DEFAULTS)); + return { ...DEFAULTS, ...stored } as Settings; +} + +export async function saveSettings(patch: Partial): Promise { + await browser.storage.local.set(patch); +} + +export async function clearSession(): Promise { + await saveSettings({ token: null, tokenExpiresAt: null, user: null, defaultProjectId: null }); +} + +export async function login(email: string, password: string): Promise<{ nickname: string; email: string }> { + const settings = await getSettings(); + const response = await fetch(`${settings.serverUrl}/api/auth/login`, { + method: "POST", + headers: { "Content-Type": "application/json", "X-Client": "extension" }, + body: JSON.stringify({ email, password }), + }); + if (!response.ok) { + const detail = await response.json().catch(() => null); + throw new Error(detail?.detail ?? `Login failed (${response.status})`); + } + const { token, expires_at } = (await response.json()) as { token: string; expires_at: string }; + + // fetch the profile with the new token + const meResponse = await fetch(`${settings.serverUrl}/api/auth/me`, { + headers: { Authorization: `Bearer ${token}` }, + }); + if (!meResponse.ok) throw new Error("Login succeeded but profile fetch failed"); + const user = (await meResponse.json()) as { id: string; nickname: string; email: string }; + + await saveSettings({ token, tokenExpiresAt: expires_at, user }); + return user; +} + +export async function logout(): Promise { + const settings = await getSettings(); + if (settings.token) { + await fetch(`${settings.serverUrl}/api/auth/logout`, { + method: "POST", + headers: { Authorization: `Bearer ${settings.token}` }, + }).catch(() => {}); + } + await clearSession(); +} + +/** Client with the current stored token; refreshes settings each call. */ +export async function getHttpClient(): Promise { + const settings = await getSettings(); + return createHttpClient({ baseUrl: settings.serverUrl, token: settings.token }); +} \ No newline at end of file diff --git a/packages/extension/src/content/index.ts b/packages/extension/src/content/index.ts new file mode 100644 index 0000000..a05f18f --- /dev/null +++ b/packages/extension/src/content/index.ts @@ -0,0 +1,130 @@ +/** + * Content script bootstrap — deliberately minimal: + * no UI, no network; mounts the shadow-DOM overlay, relays DOM events + * and (while recording) captures clicks/inputs for the background buffer. + */ +import browser from "webextension-polyfill"; +import type { ElementContext } from "@ltt/shared"; +import { mountOverlay } from "./overlay/mount"; +import { buildElementContext, collectEnvironment } from "../lib/selector"; + +let overlay: ReturnType | null = null; + +function getOverlay() { + if (!overlay) overlay = mountOverlay(); + return overlay; +} + +// ---------- recorder capture (event listeners, no UI) ---------- + +let captureListeners: (() => void) | null = null; + +/** Longest input debounced per element; flushes a single input event per pause. */ +const pendingInputs = new Map(); + +function sendRecorderEvent(type: "click" | "input", element: ElementContext, data: Record) { + void browser.runtime + .sendMessage({ + type: "recorder_event", + event: { type, data: { element, ...data }, at: Date.now() }, + }) + .catch(() => {}); +} + +function flushInput(element: Element) { + const pending = pendingInputs.get(element); + if (!pending) return; + pendingInputs.delete(element); + window.clearTimeout(pending.timer); + const input = element as HTMLInputElement | HTMLTextAreaElement | HTMLSelectElement; + const isPassword = (input as HTMLInputElement).type === "password"; + const value = isPassword ? null : input.value.slice(0, 500); + sendRecorderEvent("input", pending.element, { + value_length: input.value.length, + value, + input_type: (input as HTMLInputElement).type ?? null, + }); +} + +function isOurOverlay(target: EventTarget | null): boolean { + // events originating in the closed shadow root are retargeted to the host + return target instanceof Element && target.id === "ltt-overlay-host"; +} + +function attachCaptureListeners() { + if (captureListeners) return; + + const onClick = (event: MouseEvent) => { + if (isOurOverlay(event.target)) return; + const target = event.target; + if (!(target instanceof Element)) return; + sendRecorderEvent("click", buildElementContext(target), { + button: event.button, + }); + }; + + const onInput = (event: Event) => { + if (isOurOverlay(event.target)) return; + const target = event.target; + if ( + !(target instanceof HTMLInputElement) && + !(target instanceof HTMLTextAreaElement) && + !(target instanceof HTMLSelectElement) + ) { + return true; + } + const existing = pendingInputs.get(target); + if (existing) window.clearTimeout(existing.timer); + const timer = window.setTimeout(() => flushInput(target), 250); + pendingInputs.set(target, { timer, element: buildElementContext(target) }); + }; + + document.addEventListener("click", onClick, true); + document.addEventListener("input", onInput, true); + captureListeners = () => { + document.removeEventListener("click", onClick, true); + document.removeEventListener("input", onInput, true); + for (const [element, pending] of pendingInputs) window.clearTimeout(pending.timer); + pendingInputs.clear(); + }; +} + +function detachCaptureListeners() { + if (!captureListeners) return; + captureListeners(); + captureListeners = null; +} + +// ---------- message wiring ---------- + +browser.runtime.onMessage.addListener((message, _sender, sendResponse) => { + const msg = message as { type: string; [key: string]: unknown }; + switch (msg.type) { + case "ping": + sendResponse({ ok: true, data: "pong" }); + return true; + + case "start_picker": + getOverlay().startPicker(); + sendResponse({ ok: true }); + return true; + + case "recorder_state": { + const recording = Boolean((message as { recording?: unknown }).recording); + getOverlay().setRecorderState(recording); + if (recording) { + attachCaptureListeners(); + void browser.runtime + .sendMessage({ type: "recorder_environment", environment: collectEnvironment() }) + .catch(() => {}); + } else { + detachCaptureListeners(); + } + sendResponse({ ok: true }); + return true; + } + + default: + return true; + } +}); \ No newline at end of file diff --git a/packages/extension/src/content/overlay/App.vue b/packages/extension/src/content/overlay/App.vue new file mode 100644 index 0000000..defa534 --- /dev/null +++ b/packages/extension/src/content/overlay/App.vue @@ -0,0 +1,64 @@ + + + \ No newline at end of file diff --git a/packages/extension/src/content/overlay/NoteComposer.vue b/packages/extension/src/content/overlay/NoteComposer.vue new file mode 100644 index 0000000..ef87f32 --- /dev/null +++ b/packages/extension/src/content/overlay/NoteComposer.vue @@ -0,0 +1,283 @@ + + +