feat: publish Agent Studio integrations and setup guide
This commit is contained in:
Wenxin Duan
@@ -0,0 +1,11 @@
|
||||
# Drama Minis 网页接入
|
||||
|
||||
用户已授权直接实施，并允许 swads 数据传递到已配置的 Qwen 网关。正式经营查询仍须由页面用户确认周期、对比周期和账户范围。
|
||||
|
||||
1. 扩展 Skill 快照：仅复制显式允许的引用、脚本、CSS 和结构示例；保持原始 Skill 不变，拒绝路径逃逸和符号链接。
|
||||
2. 接入官方 MCP HTTP 客户端：仅映射 swads_whoami、metrics_catalog、metrics_semantic_query；凭证来自服务器 SWADS_MCP_TOKEN，不经过浏览器或模型。真实查询逐次等待页面批准。
|
||||
3. 封装固定报告生成器：校验 JSON，在会话私有目录运行已知 Python 脚本；生成 HTML 和 A4 PDF。PDF 失败时明确返回 HTML 与原因，不伪造成功。
|
||||
4. 网页增加 Skill 调用入口、真实查询确认和报告下载卡片；固定 Session 产物端点禁止路径遍历。
|
||||
5. 验收：先失败测试后实现，运行全套检查；真实网页验证 Skill 读取、引用、swads 连接和参数确认。未获日期/范围确认前不自动查询真实指标。报告渲染使用明确标注的测试数据验证，不作为真实经营报告交付。
|
||||
|
||||
不开放通用 shell，不导入其他 Codex MCP 权限，不修改原始 Skill，不做公网部署。
|
||||
@@ -0,0 +1,52 @@
|
||||
# Editor CMS Connection Implementation Plan
|
||||
|
||||
> **For agentic workers:** Use superpowers:executing-plans inline. The user requests direct execution; do not pause for another execution-mode choice.
|
||||
|
||||
**Goal:** Let the web app load the real editor Skill and authenticate its own CMS session, then read genuine drama inventory and subtitles without exposing credentials.
|
||||
**Architecture:** First implement the CMS/auth boundary as a separately testable milestone. It does not claim render readiness; the subsequent media milestone requires an authenticated source and explicit model/footage authorization. Original Skill and CMS code remain unchanged.
|
||||
**Tech Stack:** Existing Fastify, React, Pi, Zod, pinned official Supabase JS client.
|
||||
**Spec:** `docs/superpowers/specs/2026-09-10-short-drama-web-design.md`.
|
||||
|
||||
**2026-09-14 execution checkpoint:** Auth/API/UI/read-only tools are implemented. Full suite: 146 passed, lint/typecheck/build passed. Real login, 30-item catalog, Skill + CMS contract reads and status tool passed through the Vite/API/real-Qwen chain. Browser login/catalog also passed. Detailed evidence and remaining limitations: `docs/editor-cms-web.md`. Checklists below retain unverified cases (including explicit expiry, busy API mutation and browser content approval); do not interpret this milestone as full video acceptance.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- localhost single-user app; no CMS writes, RLS changes, service_role, arbitrary shell or uploads.
|
||||
- Authentication is a separate UI form; no password/token in chat, model tools, SSE or disk. Session reset clears only this app's CMS connection.
|
||||
- Fixed Supabase project and CMS origin; reject redirects. Only verified app_metadata.role=admin is accepted, matching actual backend checks.
|
||||
- Skill source is read-only; copy only explicitly enumerated references and script dependencies.
|
||||
- Do not equate readable Skill/catalog with a finished video pipeline.
|
||||
|
||||
### Task 1: Authenticated CMS boundary
|
||||
|
||||
Files: `packages/harness/src/cms/cms-connection.ts`, `packages/harness/tests/cms-connection.test.ts`.
|
||||
Interface: `CmsConnection({publishableKey,fetch?})`: `login(email,password)`, `status()`, `clear()`, `listDramas(query,offset)`, `inventory(dramaId)`, `subtitle(dramaId,episodeId,locale)`.
|
||||
|
||||
- [ ] RED: create real service tests with injected external HTTP responses. Assert no data query before login; fail non-admin even with user_metadata.admin; status omits passwords/tokens; subtitle rejects an episode not in the requested drama; inventory completeness preserves missing sources. Example: `await expect(cms.listDramas('',0)).rejects.toMatchObject({code:'CMS_LOGIN_REQUIRED'})`.
|
||||
- [ ] Run `../../node_modules/.bin/vitest run tests/cms-connection.test.ts` in harness and observe missing implementation failure.
|
||||
- [ ] GREEN: use official `createClient(url,key,{auth:{persistSession:false,autoRefreshToken:false,detectSessionInUrl:false}})`. Call `signInWithPassword` then `getUser` and verify role. Fetch only known tables/selects, validate all returns with Zod; fixed CMS subtitle route and bounded responses. Never return raw SDK errors.
|
||||
- [ ] Verify tests and TypeScript, including expiry and disconnect; review credential lifecycle and request destinations.
|
||||
|
||||
### Task 2: Session API and login UI
|
||||
|
||||
Files: `apps/server/src/routes/cms.ts`, `apps/server/src/app.ts`, `apps/server/src/index.ts`, `apps/web/src/components/cms/CmsPanel.tsx`, `apps/web/src/App.tsx`, corresponding route/component tests.
|
||||
Interface: GET/POST/DELETE `/api/sessions/:sessionId/cms/connection`; GET `/api/sessions/:sessionId/cms/dramas?q=&offset=`. All authenticated CMS objects are mapped by Session; dispose clears them.
|
||||
|
||||
- [ ] RED: API rejects unknown Session, cross-origin mutation, concurrent login, busy mutation; private fields never appear in status. UI password empties on submission/failure/close; authenticated catalog selection fills an exact drama ID in Chat.
|
||||
- [ ] Run test files and observe failures before implementation.
|
||||
- [ ] GREEN: fixed-origin backend API, JSON-only login body, same-origin/fetch metadata guard, bounded login attempts, no-store. Status communicates `connected` and `expiresAt`; no user email or credentials to model. UI accessible labels, pending/error states and an explicit notice of the destination receiving credentials.
|
||||
- [ ] Verify route and UI tests; unchanged report flow passes.
|
||||
|
||||
### Task 3: Real Skill and read-only tools
|
||||
|
||||
Files: `packages/harness/src/cms/editor-tools.ts`, `apps/server/src/index.ts`, tool policy, Skill and web tests.
|
||||
Interfaces: `cms_connection_status`, `cms_list_dramas`, `cms_drama_inventory`, `cms_read_subtitle`. Model never calls login or changes the connection. Inventory and subtitle access require page approval because they can disclose licensed/nonpublic content to the configured model.
|
||||
|
||||
- [ ] RED: Skill snapshot reads full reference hierarchy; tools reject data access before login; policy asks for content reads; discovery alone does not claim rendering ready.
|
||||
- [ ] GREEN: enumerate trusted Skill resources, register tools only with that Skill, strict tool argument schemas, inject runtime instruction to report missing media/vision/research capability honestly and never use demo data.
|
||||
- [ ] Run all tests, lint, typecheck, build, then restart only the local Agent Studio service with secrets via hidden stdin.
|
||||
- [ ] Browser verify Skill entry, original instructions, login form and capability/status tools; request the user to authenticate through the form for the real CMS flow. Stop at a genuine missing login instead of inventing an admin session.
|
||||
|
||||
## Remaining full-workflow milestones (not claimed by this plan)
|
||||
|
||||
After real CMS login is available: approved bounded HLS download; vision evidence and analysis v2 gate; configured reference research and scoring; immutable plan approval; fixed Python/FFmpeg worker with cancellation/quota; machine QA; human sound/viewing review; versioned downloads/export. Each requires its own test-first plan against the full spec. The task remains incomplete until these are verified on user-selected licensed footage.
|
||||
@@ -0,0 +1,54 @@
|
||||
# Editor Media Pipeline Implementation Plan
|
||||
|
||||
> Execute inline using executing-plans and test-driven-development; the user already requested execution, no further execution-mode confirmation.
|
||||
|
||||
**Goal:** Prepare actual licensed source media and return auditable frames/subtitles through Agent Studio, then enforce original analysis, reference, recipe, render and review gates.
|
||||
**Architecture:** Fixed CMS/admin signing; bounded same-CDN HTTPS HLS retrieval to local files; FFmpeg/FFprobe without network or inherited secrets. Per-session service and explicit approval tools reuse the existing Harness. Rendering cannot run until analysis and reference gates pass.
|
||||
**Tech Stack:** Existing TypeScript/Fastify/Pi/React; pinned ffmpeg-static 5.3.0, ffprobe-static 3.1.0; bundled Python 3.12 and original Skill validators.
|
||||
**Spec:** `docs/superpowers/specs/2026-09-10-short-drama-web-design.md`.
|
||||
|
||||
## Constraints
|
||||
|
||||
No CMS writes, uploads, source overwrites, arbitrary shell or credential/URL exposure. First acceptance title is `06aa396d-175a-48ed-ad76-02786ef4d0a9` (Beneath the Ice). Read-only preflight is already PASS. Metadata completeness is not media analysis. Content review must follow the newly added adult-content gate, never fabricate reviewer PASS.
|
||||
|
||||
### 1. Source preparation
|
||||
|
||||
Files: `cms/cms-connection.ts`, `media/hls.ts`, `media/media-worker.ts`, `media/editor-service.ts`, matching harness tests.
|
||||
|
||||
- [ ] Write failing tests for HLS target-origin/path escapes, encrypted/byte-range playlists, master nesting, segment count/duration/byte limits; valid playlists become local filenames without signed URLs.
|
||||
- [x] Implement `parseHls(text:string): {kind:'master',variants:string[]} | {kind:'media',segments:{uri:string,duration:number}[],duration:number}`. Only finite VOD MPEG-TS sources; reject unsupported formats explicitly, never silently skip encryption or audio renditions.
|
||||
- [x] Add `CmsConnection.playback(dramaId,episodeId)` returning signed URL only to server-side service after exact inventory binding and admin checks.
|
||||
- [x] `EditorService.prepare({dramaId,episodeNumbers},signal)` allows at most 5 episodes, 30 minutes/episode, 1 GiB/task download. Fetch fixed public CDN, no redirects, preserve CDN-issued per-path signing query only on same episode-directory assets. Localize manifests before invoking FFmpeg with `file,crypto,data` protocol allowlist; no network credentials in child env.
|
||||
- [ ] Probe real local MP4 for audio and video, hash sources; create representative JPEGs, save captions and source/time bindings in private runtime. Cancellation aborts fetch and kills process group.
|
||||
|
||||
### 2. Web tools and artifacts
|
||||
|
||||
Files: `media/editor-tools.ts`, server assembly and routes, tool policy and approval UI.
|
||||
|
||||
- [x] Expose `prepare_drama_sources` with exact IDs/quotas (approval), `read_drama_frames` returning small selected JPEG images plus time/source binding (approval), `editor_job_status` (read-only).
|
||||
- [x] Keep normal image-tool results out of SSE base64 payloads. Expose only registered frame IDs scoped to owning Session, with no-store/nosniff and inline JPEG display. Model must read Skill before analysis; no shell access.
|
||||
- [ ] Verify actual first/middle/last source download, image tool result acceptance by existing Qwen, and genuine transcript evidence. A safe extraction result can be claimed only after probing output.
|
||||
|
||||
### 3. Analysis and downstream gates
|
||||
|
||||
Files: validator bridge and video job routes/review components, separate tests following the verified interfaces above.
|
||||
|
||||
- [x] Bind every analysis submission to freshly prepared drama_id/fetched_at and server-issued analysis_id. Server compares actual subtitle/frame IDs before invoking original validator; reject invented evidence, paths or PASS flags.
|
||||
- [ ] After diagnosis PASS, obtain verified public short-drama references through a separately configured source or user-supplied direct evidence; never claim current Codex search is inherited by the webpage. Missing reference access/evidence is a genuine gate requiring input, not an automatic Tier C downgrade.
|
||||
- [x] Bind immutable recipe and content-review records to the same analysis and approved source IDs; user approves exact plan hash. Run original render_exact and machine QA with bounded child process and minimum environment. Verified with synthetic engineering fixtures; real-drama acceptance remains pending.
|
||||
- [ ] Serve REVIEW preview/ranged download only for registered artifacts; final export requires human full-playback-with-sound review bound to exact video hash, plus existing content/story gates.
|
||||
|
||||
### Verification and persistence
|
||||
|
||||
- [ ] Run baseline/regression tests, original validators, typecheck/lint/build. Keep progress in this plan, avoid claiming later unchecked gates complete.
|
||||
- [x] Restart only the local service after build; fresh web Session must enable verified Qwen image capability by default. Credentials remain in memory, require re-login after restart.
|
||||
- [ ] Run real browser task to a genuine review/approval gate. Do not stop at a preflight or after writing a plan.
|
||||
|
||||
## 2026-09-14 execution evidence
|
||||
|
||||
- Real local acceptance passed: episodes 1/3/5 downloaded, remuxed with audio/video, probed, SHA-256 recorded, 9 JPEGs and English VTT saved.
|
||||
- Real browser run loaded the original Skill and CMS contract, requested page approval, then successfully ran `prepare_drama_sources` for all three episodes. Job `job_b1280a06-450e-4982-935a-b2bd8dfb726f`; nine actual 288×512 images loaded in the result card.
|
||||
- Licensed subtitles/frames have NOT yet been sent to Qwen. Separate transmission authorization remains pending. Do not equate the model's earlier synthetic-image test with a real-drama analysis pass.
|
||||
- Original analysis/recipe/hook validators, reference scorer, renderer and machine QA are now connected. Source and plan review endpoints are human-only; tools cannot submit approval. REVIEW video routes support byte ranges, with frontend playback/download. Actual model transmission and real-title end-to-end acceptance remain pending.
|
||||
- Fresh synthetic integration produced actual 10-second H.264/AAC/1080×1920 with original source audio and mapped English subtitles. It does not establish licensed-drama analysis, real reference evidence, or semantic content PASS.
|
||||
- Local persisted files are not yet restart-resumable. Global disk-cache quota, automatic public reference search and final-stage export remain open; final PASS is never synthesized.
|
||||
@@ -0,0 +1,71 @@
|
||||
# Image-to-video Web Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Integrate the real CMS-free image-to-video Skill with durable H3 tasks and a usable web interface.
|
||||
**Architecture:** Reuse attachment validation and approval broker; add a separate durable service and restricted MCP adapter. Keep image generation jobs independent of in-memory CMS/editor jobs and chat history.
|
||||
**Tech Stack:** Existing TypeScript/Pi/Fastify/React/Vitest; original Skill Python media validator; existing ffmpeg-static/ffprobe-static.
|
||||
**Spec:** `docs/superpowers/specs/2026-09-15-image-to-video-web-design.md` (approved by user).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- 不依赖 CMS 登录、剧名、字幕或旧剪辑器的参考/选段审核。
|
||||
- 首版只做单图新生成，不提供原片续接、批量生成、自动选图或生成封面功能。
|
||||
- 不自动调用 Seedance 或其他付费提供商；不宣称 H3 免费或已支持 10 秒。
|
||||
- 本地参考图外传和生成提交分别审批；不得自动审核素材、分配账户或发布广告。
|
||||
- 重启后已知 job_id 只查原任务；未知提交不得盲重投。不得伪造时长或审核 PASS。
|
||||
- 不发送密钥至模型/前端/日志。媒体端点必须防路径穿越、SSRF、重定向和错误内容。
|
||||
- 保留所有已有脏改动；仅提交本任务新增文件和本任务增量，不提交无关旧改动。
|
||||
|
||||
### Task 1: Complete vertical slice (durable service, tools, routes and UI)
|
||||
|
||||
**Files:**
|
||||
- Create focused modules under `packages/harness/src/image-video/`: contracts, store/service, MCP adapter, media validation, tools. Tests under `packages/harness/tests/image-video*.test.ts`.
|
||||
- Modify `packages/harness/src/tools/swads-mcp.ts` only to expose reusable validated discovery/call plumbing if needed; preserve existing analytics allowlist. Export new modules through `packages/harness/src/index.ts`.
|
||||
- Modify `apps/server/src/index.ts`, `apps/server/src/app.ts`; create `apps/server/src/routes/image-video.ts` and tests `apps/server/tests/image-video-api.test.ts`.
|
||||
- Create `apps/web/src/components/tools/ImageVideoPanel.tsx` and tests `apps/web/tests/image-video.test.tsx`; integrate `App.tsx`, `ApprovalDialog.tsx`, `ToolFallback.tsx` and `components.css` as needed. Preserve current CMS UI but never require it for this flow.
|
||||
- Modify `packages/harness/src/tools/tool-policy.ts`, `pi-event-adapter.ts` for the new tools and safe public details.
|
||||
- Create `docs/image-to-video-web.md` with actual usage and explicit test/live limitations.
|
||||
|
||||
**Interfaces / implementation guidance:**
|
||||
|
||||
Use task IDs `i2v_<uuid>` and a versioned private JSON record with server-owned paths, ownerSessionId, stable request hash, reference hash, upload receipts, jobId, state, timestamps and revision. A local draft operation (`i2v_create_task`) resolves an attachment from the requesting Session and copies approved-local image bytes into the durable task directory; it must not upload. Subsequent tools `i2v_upload_reference` (ask), `i2v_submit` (ask), `i2v_status` (allow) consume only registered taskId, not arbitrary URL, tenant, file path or provider arguments. Preserve prompt/duration/image bindings from the draft through approval. Do not give a model a general mutation tool.
|
||||
|
||||
```ts
|
||||
type ImageVideoDraft = { attachmentId: string; title: string; prompt: string; durationSeconds: number };
|
||||
type ImageVideoPublicTask = {
|
||||
id: string; title: string; prompt: string; durationSeconds: number;
|
||||
state: string; createdAt: string; updatedAt: string;
|
||||
referenceUrl: string; videoUrl?: string; jobId?: string;
|
||||
error?: string; validation?: unknown;
|
||||
};
|
||||
```
|
||||
|
||||
Use private service methods `create(sessionId, draft)`, `upload(sessionId, taskId)`, `submit(sessionId, taskId)`, `refresh(taskId)` and `list()/view(taskId)` with public redacted return DTOs. Tools must enforce owner Session. A same-origin, read-only local task history can display registered tasks across process restarts without granting a new Session permission to mutate them. No automatic cross-session reassignment. Keep already-submitted jobs collectible after restart, even when their old chat Session no longer exists. A timer or UI read-only refresh should resume job polling without involving the LLM. Query at most once per 30–60 seconds; serialize each task and coalesce concurrent requests.
|
||||
|
||||
Use `/api/image-video/tasks` for redacted local history and registered media read endpoints. Creation is a Session-scoped route and/or tool. Prefer a small dedicated image input + draft form that uses the existing attachment endpoint, then invokes this Skill through the composer with the exact draft/attachment identifiers; this allows references even without vision-enabled Qwen. Keep all transfer approvals explicit. If the model needs image inspection, explain current model image transmission separately; server thumbnail display is not model transmission.
|
||||
|
||||
Discover and validate required MCP tool schemas via `listTools` before submission; rely on actual structured response shapes, not guessed backend endpoints. Root controller will supply read-only live contract observations. Identity/platform ambiguity blocks upload. Persist every potentially mutating intent before calling remote tools. A timeout around submit must persist unknown_submission, never return to ready. A known returned job ID must be persisted even if user cancelled waiting; preserve in-flight result handling. Before a source upload retry, distinguish uninitiated, known upload receipt and unknown initialization/finalization; never duplicate an uncertain import blindly.
|
||||
|
||||
MP4 download uses only registered job artifact HTTPS URL, DNS/public-address checks and bounded bytes/time, no redirects. Validate full decode, video dimensions/frame rate/duration and audio using the original Skill `scripts/media.py verify`; exact-duration mismatch is failure, not ready. Store validation report and hash; preview remains not human reviewed. Bound process time and prohibit media tools from network access. Reuse existing safe media runtime helpers where suitable, without copying CMS download logic or fixed CMS hosts.
|
||||
|
||||
- [x] **Step 1: RED — durable state tests.** Test draft ownership, no external operations on create, separate approvals, duplicate concurrent submit, persisted job after reload, unknown submit blocks retry, unknown import blocks retry, stale task revisions, safe public DTO, independent CMS. Example behavioral assertion:
|
||||
```ts
|
||||
await expect(service.submit("ses_other", task.id)).rejects.toMatchObject({code:"I2V_TASK_NOT_FOUND"});
|
||||
expect(remoteSubmissions).toHaveLength(0);
|
||||
```
|
||||
Run the focused harness tests and record real failing assertions before implementation. External provider/network calls may be faked, not service state or persistence.
|
||||
- [x] **Step 2: GREEN — implement minimal service and remote boundary.** Atomic records, strict persisted schema, locked task operations, known-ID polling, safe URL/media validation. Validate recorded state by constructing a new service against the same temporary directory and proving no duplicate remote submit.
|
||||
- [x] **Step 3: RED/GREEN — API and tool access.** Add tests of denied approval/no side effects, cross-session task binding, durable history after registry restart, same-origin JSON mutation requirements, registered-only media, HTTP 206/416 and no-store. Implement routes and tool policy, original Skill resource snapshot and production wiring. Deny unknown tool names.
|
||||
- [x] **Step 4: RED/GREEN — real UI behavior.** Test image/draft input, exact approval parameters and thumbnail, unchecked decisions, loading/error/queue states, restored tasks, periodic status fetch with cleanup, registered-only video links and actual native player/download. Use jsdom only as component evidence, not end-to-end real generation proof.
|
||||
- [x] **Step 5: Verify and self-review.** Run focused tests per iteration then full `CI=true pnpm --config.verify-deps-before-run=false test`, lint, typecheck and build. Review actual diff for credential reflection, arbitrary paths, duplicate requests and existing dirty edits. Write report with RED/GREEN evidence and all remaining live unknowns. Commit only isolated task changes; if modifying already-dirty files, use a task-specific delta commit or leave those files uncommitted and document why rather than absorbing unrelated edits.
|
||||
|
||||
### Task 2: Independent review and live acceptance
|
||||
|
||||
**Files:** Review package/report in plan workspace; update `docs/image-to-video-web.md` with exact evidence. Root controller owns service start, browser and live acceptance, not the implementer.
|
||||
|
||||
- [x] **Step 1:** Independent review the new feature diff for spec compliance and code quality, especially restart/approval/unknown-submission semantics. Fix important findings with covering tests before live mutation.
|
||||
- [x] **Step 2:** Run latest test/lint/typecheck/build; start local backend using existing hidden credential input without printing or saving keys. Read-only current MCP identity/schema checks must use the configured application token as well as connected-tool discovery.
|
||||
- [x] **Step 3:** Browser test catalog discovery, new draft, reference preview, approval refusal and task history restoration with no CMS connection. Synthetic boundary tests must be visibly labelled as such.
|
||||
- [ ] **Step 4:** If there is a user-authorized reference image, submit one actual H3 task through page approvals. Otherwise stop at image selection; do not transmit earlier CMS images or manufacture approval of a new image. User's approval of this implementation is not approval of an unspecified remote submission.
|
||||
- [ ] **Step 5:** If a real job exists, poll it rather than resubmit, collect its exact MP4, run technical checks, watch/listen and verify actual webpage playback/download. If blocked on duration/provider/image approval, report that specifically; do not claim real generation completion.
|
||||
Reference in New Issue
Block a user