Compare commits
13
Commits
89ce7973c3
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9c19375ece | ||
|
|
8e7aece9fd | ||
|
|
c7fc7f6ee1 | ||
|
|
05e6060490 | ||
|
|
2ad089d68a | ||
|
|
cc80265bb2 | ||
|
|
40e03e393c | ||
|
|
9349317eb8 | ||
|
|
cc1613fa3b | ||
|
|
9f247d22b2 | ||
|
|
ba45382747 | ||
|
|
8fa006344d | ||
|
|
da7558ab49 |
@@ -2,10 +2,26 @@ OPENAI_API_KEY=
|
|||||||
ANTHROPIC_API_KEY=
|
ANTHROPIC_API_KEY=
|
||||||
AGENT_PROVIDER=
|
AGENT_PROVIDER=
|
||||||
AGENT_MODEL=
|
AGENT_MODEL=
|
||||||
|
AGENT_MODEL_SUPPORTS_IMAGES=true
|
||||||
|
AGENT_SKILLS_DIRS=
|
||||||
LLM_BASE_URL=
|
LLM_BASE_URL=
|
||||||
LLM_API=openai-responses
|
LLM_API=openai-responses
|
||||||
|
HOST=127.0.0.1
|
||||||
|
PORT=3001
|
||||||
|
SESSIONS_ROOT=.agent-studio-sessions
|
||||||
|
AGENT_DIR=.agent-studio-runtime
|
||||||
ALLOW_PRIVATE_LLM_BASE_URLS=false
|
ALLOW_PRIVATE_LLM_BASE_URLS=false
|
||||||
SESSION_RUN_TIMEOUT_MS=120000
|
SESSION_RUN_TIMEOUT_MS=120000
|
||||||
APPROVAL_TIMEOUT_MS=60000
|
APPROVAL_TIMEOUT_MS=60000
|
||||||
MAX_IMAGE_BYTES=5242880
|
MAX_IMAGE_BYTES=5242880
|
||||||
MAX_IMAGES_PER_MESSAGE=4
|
MAX_IMAGES_PER_MESSAGE=4
|
||||||
|
|
||||||
|
# Optional real Skill integrations. Never commit filled credentials.
|
||||||
|
SWADS_MCP_TOKEN=
|
||||||
|
# Absolute directory paths containing SKILL.md; leave overrides commented unless set.
|
||||||
|
# IMAGE_VIDEO_SKILL_DIR=/absolute/path/to/swads-image-to-video
|
||||||
|
# DRAMA_MINIS_SKILL_DIR=/absolute/path/to/drama-minis-diagnosis
|
||||||
|
# DRAMA_EDITOR_SKILL_DIR=/absolute/path/to/swads-short-drama-editor
|
||||||
|
# PYTHON_EXECUTABLE=/absolute/path/to/python3
|
||||||
|
# Optional CMS public key, not required for image-to-video.
|
||||||
|
CMS_SUPABASE_PUBLISHABLE_KEY=
|
||||||
|
|||||||
@@ -9,3 +9,10 @@ workspace/sessions/*
|
|||||||
!workspace/sessions/.gitkeep
|
!workspace/sessions/.gitkeep
|
||||||
*.log
|
*.log
|
||||||
.DS_Store
|
.DS_Store
|
||||||
|
.agent-studio-sessions/
|
||||||
|
.agent-studio-runtime/
|
||||||
|
|
||||||
|
# One-shot operator repairs contain machine/task-specific recovery assumptions.
|
||||||
|
/scripts/recover-*.mjs
|
||||||
|
/scripts/diagnose-authorized-agent.mjs
|
||||||
|
.superpowers/
|
||||||
|
|||||||
@@ -1,15 +1,212 @@
|
|||||||
# Agent Studio Mini
|
# Agent Studio Mini
|
||||||
|
|
||||||
Training-grade Agent Harness built with Pi Coding Agent SDK, Fastify, SSE, React, and assistant-ui.
|
基于 React、assistant-ui、Fastify 和 Pi Coding Agent SDK 的本地 Agent 工作台。通过网页调用模型和真实 Skill,观察工具执行过程,并在图片外传、视频生成等操作前逐项审批。
|
||||||
|
|
||||||
The project is under active construction. Verified setup, security guidance, and demo instructions are added with the completed implementation.
|
> 当前定位是单用户、本地运行的开发与验收版本,不是可直接暴露到公网的多租户产品。图生视频仍依赖 SW Ads / H3;历史成功样片不代表所有任务都已实现无人干预稳定出片。
|
||||||
|
|
||||||
## Verified dependency baseline
|
## 界面预览
|
||||||
|
|
||||||
- Node.js 22.22.3 (Pi 0.85.1 requires Node.js 22.19 or newer)
|
以下为本地运行页面的真实截图(2026-09-16,窄屏布局),不是设计稿。为避免公开私人素材、任务提示词和凭证,仅展示工作台首页与 Skill 目录;截图不代表图生视频链路已通过稳定性验收。
|
||||||
- pnpm 11.7.0
|
|
||||||
- TypeScript 5.9.3 (the newest release supported by typescript-eslint 8.69)
|
|
||||||
- Pi Coding Agent SDK 0.85.1, using `typebox` 1.3.7
|
|
||||||
- assistant-ui React 0.15.18
|
|
||||||
|
|
||||||
The original brief allowed Node.js 20+ and referred generically to the current TypeBox package. The installed Pi release has a higher Node.js floor and directly uses the `typebox` package rather than `@sinclair/typebox`; this project follows the installed public types.
|
| 工作台首页 | 真实 Skill 目录 |
|
||||||
|
| :---: | :---: |
|
||||||
|
|  |  |
|
||||||
|
| 描述任务,观察执行过程与审批状态 | 按需选择 Skill,完整指令由 Agent 在执行时读取 |
|
||||||
|
|
||||||
|
### Skill 生成效果:猫咪参考图 → 3 秒视频
|
||||||
|
|
||||||
|
用户在网页提供猫咪参考图,填写提示词 **“随意发挥”**、目标时长 **3 秒**,调用 **swads-image-to-video**,由 SW Ads / H3 生成下方视频。参考图和这次成片经用户授权用于 README 展示。
|
||||||
|
|
||||||
|
| 输入参考图 | Skill 生成结果 |
|
||||||
|
| :---: | :---: |
|
||||||
|
|  | [](docs/examples/skill-cat-3s.mp4) |
|
||||||
|
| 本次任务实际使用的参考图 | 原速、静音循环预览;点击查看 MP4 |
|
||||||
|
|
||||||
|
**[查看 / 下载带音轨的猫咪 MP4](docs/examples/skill-cat-3s.mp4)** · 3.000 秒 · 768 × 1344 · 24 fps · 约 1.75 MiB
|
||||||
|
|
||||||
|
上方 GIF 从成片转为 320 像素宽、12 fps 的静音循环预览,保持原速;循环只是 README 展示方式,MP4 本身只有 3 秒。若仓库页面不内嵌播放视频,可打开文件后选择原始文件或下载。
|
||||||
|
|
||||||
|
> 样片生成于 2026-09-16,任务状态为 ready,已通过时长、音轨和解码技术检查,画面与声音仍待正式视听验收。生成画面的构图和场景与参考图有所变化,不代表严格保留原图细节。单次成功不代表之前的模型输出中断与错误提示问题已修复;此次仅发布已有参考图与成片,不重新提交生成。
|
||||||
|
|
||||||
|
## 功能与状态
|
||||||
|
|
||||||
|
| 功能 | 当前状态 |
|
||||||
|
| --- | --- |
|
||||||
|
| 模型聊天 | 流式回复、HTTP + SSE 事件、取消运行;支持 OpenAI、Anthropic 和兼容网关配置 |
|
||||||
|
| 图片输入 | JPEG / PNG / WebP 上传、规范化和图片能力校验;视觉能力需按实际模型验证 |
|
||||||
|
| Skill 调用 | 展示目录元信息,执行时读取真实 SKILL.md 和允许的参考资源 |
|
||||||
|
| 工具审批 | 服务端管理审批、超时与拒绝;浏览器不能伪造执行成功 |
|
||||||
|
| 图生视频 | 本机草稿 → 上传审批 → SW Ads 入库 → H3 生成审批 → 查询、校验、播放、下载 MP4 |
|
||||||
|
| 旧任务接续 | 重启后,经明确确认将草稿或待生成任务接续到新会话;不重复上传、不自动生成 |
|
||||||
|
| Drama Minis 报告 | 真实指标查询与 HTML / JSON 报告;PDF 需要 Chrome / Chromium |
|
||||||
|
| CMS 剪辑 | 已有连接、素材准备和受控预览工具;完整剪辑验收仍有限制 |
|
||||||
|
|
||||||
|
## 快速开始
|
||||||
|
|
||||||
|
### 环境要求
|
||||||
|
|
||||||
|
- Node.js **22.19+**;建议使用受支持的 LTS 版本。
|
||||||
|
- pnpm **11.7.0**,版本见 package.json。
|
||||||
|
- 可调用的 LLM API Key;仅启动界面和运行默认测试不需要真实 Key。
|
||||||
|
- 图生视频额外需要 SW Ads MCP Token、完整的 swads-image-to-video Skill 和 Python 3。
|
||||||
|
- FFmpeg / FFprobe 使用项目依赖中的二进制;安装时需要允许相关依赖构建和下载。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone https://git.mincode.cn/wenxinduan50/agent.git
|
||||||
|
cd agent
|
||||||
|
pnpm install --frozen-lockfile
|
||||||
|
pnpm build
|
||||||
|
cp .env.example .env
|
||||||
|
```
|
||||||
|
|
||||||
|
编辑 .env,填写自己的配置。**不要提交 .env、API Key、Token 或 CMS 密码。**
|
||||||
|
|
||||||
|
在项目根目录启动后端:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
node --env-file=.env apps/server/dist/index.js
|
||||||
|
```
|
||||||
|
|
||||||
|
另开一个终端,在同一目录启动网页:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm --filter @agent-studio/web dev
|
||||||
|
```
|
||||||
|
|
||||||
|
- 网页:<http://127.0.0.1:5173/>
|
||||||
|
- 后端:<http://127.0.0.1:3001/>
|
||||||
|
- Vite 将 /api 和 /health 代理到后端。
|
||||||
|
|
||||||
|
首次构建不能省略,工作区包通过 dist 导出。后端源码变化后重新构建并重启;网页开发服务支持热更新。
|
||||||
|
|
||||||
|
如果已经通过 shell / IDE 注入环境变量,也可使用 `pnpm dev` 同时启动前后端。**pnpm dev 不会自动加载 .env**;上面的 `node --env-file` 才负责读取文件。
|
||||||
|
|
||||||
|
## 配置模型
|
||||||
|
|
||||||
|
可以使用服务端环境变量,也可以在网页右上角“模型设置”中配置当前会话。优先级:当前会话配置 → 环境变量 → 明确提示未配置;不会自动切换模型。
|
||||||
|
|
||||||
|
| 变量 | 用途 |
|
||||||
|
| --- | --- |
|
||||||
|
| AGENT_PROVIDER | openai、anthropic 或 compatible |
|
||||||
|
| AGENT_MODEL | 提供商支持的准确模型 ID |
|
||||||
|
| LLM_API | openai-responses、openai-completions 或 anthropic-messages |
|
||||||
|
| LLM_BASE_URL | 自定义兼容接口地址;默认限制为安全的 HTTPS 目标 |
|
||||||
|
| OPENAI_API_KEY / ANTHROPIC_API_KEY | 对应模型凭证 |
|
||||||
|
| AGENT_MODEL_SUPPORTS_IMAGES | 显式声明图片能力,不是检测结果 |
|
||||||
|
| SESSION_RUN_TIMEOUT_MS | 单次 Agent 运行上限,默认 120000 毫秒 |
|
||||||
|
| APPROVAL_TIMEOUT_MS | 审批有效时间,默认 60000 毫秒 |
|
||||||
|
|
||||||
|
OpenAI 兼容服务通常使用 `AGENT_PROVIDER=compatible`、`LLM_API=openai-completions`,具体以所用服务契约为准。聊天模型与 H3 视频模型是独立链路,聊天 API Key 不能代替视频凭证。
|
||||||
|
|
||||||
|
网页填写的凭证只保存在服务端内存,接口返回脱敏提示;重启或清除会话后失效。网页“连接测试”可能仅返回“首次运行验证”,不能据此声称真实调用成功。
|
||||||
|
|
||||||
|
## 安装真实 Skills
|
||||||
|
|
||||||
|
仓库自带 ads-analysis、report-writer 演示 Skill。三个业务 Skill 的原始文件**不随本仓库分发**,需另行取得完整且获授权的目录,包括 references、scripts、assets 等资源。
|
||||||
|
|
||||||
|
| Skill | 默认位置 | 覆盖变量 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| swads-image-to-video | ~/.codex/skills/swads-image-to-video | IMAGE_VIDEO_SKILL_DIR |
|
||||||
|
| drama-minis-diagnosis | ~/.codex/skills/drama-minis-diagnosis | DRAMA_MINIS_SKILL_DIR |
|
||||||
|
| swads-short-drama-editor | ~/.codex/skills/swads-short-drama-editor | DRAMA_EDITOR_SKILL_DIR |
|
||||||
|
|
||||||
|
覆盖变量填写 **Skill 目录的绝对路径**,不是 SKILL.md 文件路径。其他 Skill 可使用 AGENT_SKILLS_DIRS 配置,多个目录用操作系统路径分隔符连接(macOS / Linux 为冒号)。
|
||||||
|
|
||||||
|
图生视频和剪辑还需将 PYTHON_EXECUTABLE 设置为 Python 3 可执行文件的绝对路径,可用 `command -v python3` 查询。代码中的默认路径面向本机 Codex 运行时,其他电脑不一定存在。
|
||||||
|
|
||||||
|
仅复制 Skill 文件,不等于自动接通它引用的所有工具。工具仍须经过 Harness 白名单、参数校验和审批策略适配;缺少资源或后端能力时应明确报告不可用。
|
||||||
|
|
||||||
|
## 在网页生成视频
|
||||||
|
|
||||||
|
1. 配置 SWADS_MCP_TOKEN、图生视频 Skill 和 Python,并启动服务。当前 MCP 地址由代码固定为 https://ads.mincode.cn/mcp。
|
||||||
|
2. 在 Skills 区域的“图片 → 有声视频”表单选择参考图,填写标题、动作与声音、目标秒数。
|
||||||
|
3. 点击“准备本地任务”。这一步只保存在本机并填写聊天框,**不会自动发送**。
|
||||||
|
4. 点击“发送”。Agent 读取 Skill,并申请参考图上传审批。
|
||||||
|
5. 核对图片、提示词和时长,勾选确认后批准上传。该操作会产生 SW Ads 中待审核、未分配账户的参考素材,不是完全无入库副作用。
|
||||||
|
6. 参考图就绪后,另行确认 H3 生成审批。费用以服务和账户为准,不假定免费。
|
||||||
|
7. 网页按原任务轮询。成片通过技术检查后,任务卡显示播放器和“下载 MP4”。
|
||||||
|
|
||||||
|
当前使用 H3 ref2va、9:16、768p。成片检查时长、尺寸、帧率、完整解码和音轨;符合约束的少量超长尾帧会按原速裁齐,不通过降速、循环或定格伪造时长。**技术检查通过不等于画面、声音和提示词效果已经人工验收。**
|
||||||
|
|
||||||
|
### 重启后继续任务
|
||||||
|
|
||||||
|
在原任务卡点击“继续此任务”,核对信息并确认接续到当前会话。已入库任务复用原图和回执,聊天框只准备生成审批请求,不重新上传、不自动生成。
|
||||||
|
|
||||||
|
- 只允许本地草稿和“参考图已入库 · 待生成审批”的任务接续。
|
||||||
|
- 原会话仍有效时,请回原页面继续,不能由其他会话接管。
|
||||||
|
- 排队或生成中的任务按原 job ID 查询,不重新提交。
|
||||||
|
- unknown_upload / unknown_submission 必须核实回执,不能盲目重试。
|
||||||
|
- 接续恢复任务操作权,不恢复旧聊天历史或 API Key。
|
||||||
|
|
||||||
|
当前表单要求提供图片;未传图时自动 AI 生图尚未作为完整链路交付。
|
||||||
|
|
||||||
|
## 其他工作流
|
||||||
|
|
||||||
|
- **课堂 Demo**:分析内置 demo-account 的模拟广告数据,经审批保存报告。fixtures 不是真实账户。
|
||||||
|
- **Drama Minis**:需要业务 Skill 和 SW Ads Token;真实指标查询、报告生成均受审批控制。PDF 依赖 Chrome / Chromium,失败时保留 HTML / JSON。
|
||||||
|
- **CMS 剪辑**:需要独立 CMS 配置与授权素材;图生视频不需要 CMS。详见 [CMS 接入说明](docs/editor-cms-web.md)。
|
||||||
|
|
||||||
|
## 项目结构
|
||||||
|
|
||||||
|
```text
|
||||||
|
apps/web/ React + assistant-ui,聊天、任务卡、审批、事件界面
|
||||||
|
apps/server/ Fastify API、SSE、本地任务与媒体路由
|
||||||
|
packages/harness/ Pi 适配、Skills、模型、工具策略、审批和任务状态机
|
||||||
|
packages/shared/ 前后端共享协议与 Zod 校验
|
||||||
|
.agents/skills/ ads-analysis 演示 Skill
|
||||||
|
.pi/skills/ report-writer 演示 Skill
|
||||||
|
fixtures/ 测试与演示数据
|
||||||
|
docs/ 架构、安全、接入和验收说明
|
||||||
|
scripts/ 可选诊断和真实服务验证工具
|
||||||
|
```
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
Web[网页工作台] -->|HTTP / SSE| API[Fastify + Harness]
|
||||||
|
API --> Agent[Pi Agent + 聊天模型]
|
||||||
|
Agent --> Skills[Skill 与受控工具]
|
||||||
|
Skills --> Approval[服务端审批]
|
||||||
|
Approval --> SW[SW Ads MCP / H3]
|
||||||
|
SW --> Tasks[本机任务记录与 MP4 校验]
|
||||||
|
Tasks --> Web
|
||||||
|
```
|
||||||
|
|
||||||
|
## 数据与安全
|
||||||
|
|
||||||
|
- .agent-studio-runtime/image-video/ 保存持久任务、冻结参考图、私有回执和已校验 MP4。备份时视为私有业务数据;回执可能含签名 URL。
|
||||||
|
- .agent-studio-sessions/ 是临时会话工作区;模型凭证、审批和事件主要驻留内存。不要依赖刷新页面恢复聊天。
|
||||||
|
- 图片外传、视频生成分开审批;服务端保留幂等键、回执和原 job ID,结果未知时停止重投。
|
||||||
|
- 媒体访问有 HTTPS、DNS / 地址校验、大小边界和文件哈希检查。本地任务接口限制 loopback 和同源访问。
|
||||||
|
- .env 文件、运行目录、依赖、构建产物、日志和一次性本机恢复脚本不进入 Git。
|
||||||
|
- **不要把本地端口直接暴露到公网。** 目前没有生产级身份认证、租户隔离或分布式锁。
|
||||||
|
- Pi 和扩展继承 Node 进程权限;工具白名单和路径校验不是操作系统沙箱。
|
||||||
|
|
||||||
|
## 开发与验证
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm lint
|
||||||
|
pnpm typecheck
|
||||||
|
pnpm test
|
||||||
|
pnpm build
|
||||||
|
```
|
||||||
|
|
||||||
|
2026-09-16 的验收基线为 **236 项测试**:shared 13、harness 152、server 25、web 46。默认测试使用隔离的临时数据与模拟远端,不提交真实视频任务;部分媒体检查会调用本机 FFmpeg。
|
||||||
|
|
||||||
|
scripts/verify-*.mjs 是可选的真实环境验证工具,部分会访问模型、上传素材或生成视频,可能产生费用。先阅读脚本并核对目标和副作用,不要作为默认安装步骤。start-private-server.mjs 是当前本机联调启动器,含特定网关和模型配置;通用部署请用环境变量和上面的启动方式。
|
||||||
|
|
||||||
|
## 已知限制与验收口径
|
||||||
|
|
||||||
|
- SW Ads 上传曾返回 HTTP 500;原猫图经受控恢复后入库成功,但历史 500 的服务端根因尚未确认。
|
||||||
|
- 既往 2 秒、10 秒样片测试有过人工恢复,不应描述成全程无人干预的稳定性验证。
|
||||||
|
- 旧猫图任务已验证网页接续成功;本页另展示了 2026-09-16 成功生成的 3 秒猫咪样片。每次生成仍需单独审批,接续通过本身不等于已经生成视频。
|
||||||
|
- 模型、网络、共享 GPU 排队、存储和第三方接口都会影响完成时间。重启后能查询任务,不代表旧 Agent 运行能自动重放。
|
||||||
|
- 当前不是完整 CMS 自动剪辑产品;画面质量、连续性和声音仍需检查。
|
||||||
|
- 尚无多用户鉴权、跨设备同步、分布式任务调度和完整生产监控。
|
||||||
|
|
||||||
|
## 文档
|
||||||
|
|
||||||
|
- [架构](docs/architecture.md) · [事件映射](docs/event-mapping.md) · [安全边界](docs/security-boundaries.md)
|
||||||
|
- [课堂演示](docs/classroom-demo.md) · [生产化路线](docs/production-roadmap.md)
|
||||||
|
- [图生视频说明](docs/image-to-video-web.md) · [验收清单](docs/image-to-video-acceptance.md)
|
||||||
|
- [任务接续与猫图恢复记录](docs/image-to-video-cat-recovery.md)
|
||||||
|
- [Drama Minis 报告](docs/drama-minis-web.md) · [CMS 接入](docs/editor-cms-web.md)
|
||||||
|
|||||||
@@ -1,4 +1,6 @@
|
|||||||
import type {
|
import type {
|
||||||
|
ImageVideoService,
|
||||||
|
CmsConnection,
|
||||||
ApprovalBroker,
|
ApprovalBroker,
|
||||||
AttachmentStore,
|
AttachmentStore,
|
||||||
EventStore,
|
EventStore,
|
||||||
@@ -18,6 +20,10 @@ import { registerEventRoute } from "./routes/events.js";
|
|||||||
import { registerModelRoutes } from "./routes/models.js";
|
import { registerModelRoutes } from "./routes/models.js";
|
||||||
import { registerSessionRoutes } from "./routes/sessions.js";
|
import { registerSessionRoutes } from "./routes/sessions.js";
|
||||||
import { registerSkillsRoute } from "./routes/skills.js";
|
import { registerSkillsRoute } from "./routes/skills.js";
|
||||||
|
import { registerReportRoutes } from "./routes/reports.js";
|
||||||
|
import { registerCmsRoutes } from "./routes/cms.js";
|
||||||
|
import { registerEditorRoutes } from "./routes/editor.js";
|
||||||
|
import { registerImageVideoRoutes } from "./routes/image-video.js";
|
||||||
import { SseRegistry, type SseRegistryPort } from "./sse/sse-registry.js";
|
import { SseRegistry, type SseRegistryPort } from "./sse/sse-registry.js";
|
||||||
|
|
||||||
export type ApiSession = SessionRecord & {
|
export type ApiSession = SessionRecord & {
|
||||||
@@ -27,6 +33,8 @@ export type ApiSession = SessionRecord & {
|
|||||||
};
|
};
|
||||||
|
|
||||||
export type AppDependencies = {
|
export type AppDependencies = {
|
||||||
|
imageVideo?: ImageVideoService;
|
||||||
|
imageVideoPorts?: number[];
|
||||||
sessionRegistry: {
|
sessionRegistry: {
|
||||||
create(): Promise<ApiSession>;
|
create(): Promise<ApiSession>;
|
||||||
get(sessionId: string): ApiSession | undefined;
|
get(sessionId: string): ApiSession | undefined;
|
||||||
@@ -38,6 +46,10 @@ export type AppDependencies = {
|
|||||||
attachmentStore: Pick<AttachmentStore, "add" | "remove">;
|
attachmentStore: Pick<AttachmentStore, "add" | "remove">;
|
||||||
approvalBroker: Pick<ApprovalBroker, "approve" | "deny">;
|
approvalBroker: Pick<ApprovalBroker, "approve" | "deny">;
|
||||||
eventStore?: EventStore;
|
eventStore?: EventStore;
|
||||||
|
readReport?: (sessionId: string, reportId: string, format: string) => Promise<{ bytes: Buffer; contentType: string }>;
|
||||||
|
readEditorFrame?: (sessionId: string, jobId: string, frameId: string) => Promise<Buffer>;
|
||||||
|
editorForSession?: (sessionId:string)=>{workflowView:(jobId:string)=>unknown;review:(jobId:string,stage:"selection"|"plan",value:unknown)=>Promise<unknown>;video:(jobId:string,kind:"sources"|"plans",id:string)=>Promise<{path:string;size:number}>};
|
||||||
|
cmsForSession?: (sessionId: string) => CmsConnection;
|
||||||
sseRegistry?: SseRegistryPort;
|
sseRegistry?: SseRegistryPort;
|
||||||
testModelConnection?: (
|
testModelConnection?: (
|
||||||
resolved: ResolvedModelConnection,
|
resolved: ResolvedModelConnection,
|
||||||
@@ -57,6 +69,10 @@ export function buildApp(dependencies: AppDependencies) {
|
|||||||
installErrorHandler(app);
|
installErrorHandler(app);
|
||||||
registerHealthRoute(app);
|
registerHealthRoute(app);
|
||||||
registerSkillsRoute(app, effectiveDependencies);
|
registerSkillsRoute(app, effectiveDependencies);
|
||||||
|
registerReportRoutes(app, effectiveDependencies);
|
||||||
|
registerCmsRoutes(app, effectiveDependencies);
|
||||||
|
registerEditorRoutes(app, effectiveDependencies);
|
||||||
|
registerImageVideoRoutes(app, effectiveDependencies);
|
||||||
registerSessionRoutes(app, effectiveDependencies);
|
registerSessionRoutes(app, effectiveDependencies);
|
||||||
registerModelRoutes(app, effectiveDependencies);
|
registerModelRoutes(app, effectiveDependencies);
|
||||||
registerAttachmentRoutes(app, effectiveDependencies);
|
registerAttachmentRoutes(app, effectiveDependencies);
|
||||||
|
|||||||
@@ -40,10 +40,15 @@ function mapError(error: FastifyError | Error): {
|
|||||||
if ("statusCode" in error && error.statusCode === 413) {
|
if ("statusCode" in error && error.statusCode === 413) {
|
||||||
return { statusCode: 413, code: "PAYLOAD_TOO_LARGE", message: "Request payload is too large", details: {} };
|
return { statusCode: 413, code: "PAYLOAD_TOO_LARGE", message: "Request payload is too large", details: {} };
|
||||||
}
|
}
|
||||||
|
if ("statusCode" in error && error.statusCode === 400) {
|
||||||
|
return { statusCode: 400, code: "INVALID_REQUEST_BODY", message: "Request body could not be processed", details: {} };
|
||||||
|
}
|
||||||
return { statusCode: 500, code: "INTERNAL_ERROR", message: "Internal server error", details: {} };
|
return { statusCode: 500, code: "INTERNAL_ERROR", message: "Internal server error", details: {} };
|
||||||
}
|
}
|
||||||
|
|
||||||
function statusForHarnessCode(code: string): number {
|
function statusForHarnessCode(code: string): number {
|
||||||
|
if (code === "CMS_ORIGIN_DENIED" || code === "CMS_ADMIN_REQUIRED") return 403;
|
||||||
|
if (code === "CMS_RATE_LIMITED") return 429;
|
||||||
if (code.includes("NOT_FOUND") || code === "SESSION_NOT_FOUND") return 404;
|
if (code.includes("NOT_FOUND") || code === "SESSION_NOT_FOUND") return 404;
|
||||||
if (code === "SESSION_BUSY" || code.includes("ALREADY_RESOLVED") || code.includes("DUPLICATE")) return 409;
|
if (code === "SESSION_BUSY" || code.includes("ALREADY_RESOLVED") || code.includes("DUPLICATE")) return 409;
|
||||||
if (code === "IMAGE_TOO_LARGE") return 413;
|
if (code === "IMAGE_TOO_LARGE") return 413;
|
||||||
|
|||||||
@@ -1,9 +1,26 @@
|
|||||||
import { mkdir } from "node:fs/promises";
|
import { mkdir } from "node:fs/promises";
|
||||||
import { join, resolve } from "node:path";
|
import { delimiter, dirname, join, resolve } from "node:path";
|
||||||
import { pathToFileURL } from "node:url";
|
import { homedir } from "node:os";
|
||||||
|
import { fileURLToPath, pathToFileURL } from "node:url";
|
||||||
import {
|
import {
|
||||||
|
ImageVideoService,
|
||||||
|
ImageVideoMcpRemote,
|
||||||
|
createImageVideoCollector,
|
||||||
|
createImageVideoTools,
|
||||||
|
IMAGE_VIDEO_RESOURCES,
|
||||||
FileAttachmentStore,
|
FileAttachmentStore,
|
||||||
|
CmsConnection,
|
||||||
|
EditorService,
|
||||||
|
SkillRunner,
|
||||||
|
createMediaTools,
|
||||||
|
createEditorTools,
|
||||||
|
EDITOR_RESOURCES,
|
||||||
HarnessSession,
|
HarnessSession,
|
||||||
|
HarnessError,
|
||||||
|
MinisReportService,
|
||||||
|
SwadsMcpClient,
|
||||||
|
createSwadsTools,
|
||||||
|
createMinisReportTool,
|
||||||
InMemoryApprovalBroker,
|
InMemoryApprovalBroker,
|
||||||
InMemoryEventStore,
|
InMemoryEventStore,
|
||||||
InMemoryModelConnectionStore,
|
InMemoryModelConnectionStore,
|
||||||
@@ -22,19 +39,28 @@ import { loadServerConfig } from "./config.js";
|
|||||||
|
|
||||||
export async function buildDefaultApp(environment: NodeJS.ProcessEnv = process.env) {
|
export async function buildDefaultApp(environment: NodeJS.ProcessEnv = process.env) {
|
||||||
const config = loadServerConfig(environment);
|
const config = loadServerConfig(environment);
|
||||||
const projectRoot = process.cwd();
|
const projectRoot = resolve(dirname(fileURLToPath(import.meta.url)), "../../..");
|
||||||
const sessionsRoot = resolve(projectRoot, config.sessionsRoot);
|
const sessionsRoot = resolve(projectRoot, config.sessionsRoot);
|
||||||
const agentDir = resolve(projectRoot, config.agentDir);
|
const agentDir = resolve(projectRoot, config.agentDir);
|
||||||
await mkdir(sessionsRoot, { recursive: true, mode: 0o700 });
|
await mkdir(sessionsRoot, { recursive: true, mode: 0o700 });
|
||||||
await mkdir(agentDir, { recursive: true, mode: 0o700 });
|
await mkdir(agentDir, { recursive: true, mode: 0o700 });
|
||||||
|
|
||||||
const eventStore = new InMemoryEventStore();
|
const eventStore = new InMemoryEventStore();
|
||||||
|
const dramaSkillRoot = resolve(environment.DRAMA_MINIS_SKILL_DIR ?? join(homedir(), ".codex", "skills", "drama-minis-diagnosis"));
|
||||||
|
const editorSkillRoot = resolve(environment.DRAMA_EDITOR_SKILL_DIR ?? join(homedir(), ".codex", "skills", "swads-short-drama-editor"));
|
||||||
|
const imageVideoSkillRoot = resolve(environment.IMAGE_VIDEO_SKILL_DIR ?? join(homedir(), ".codex", "skills", "swads-image-to-video"));
|
||||||
|
const imageVideoRemote = new ImageVideoMcpRemote({ token: environment.SWADS_MCP_TOKEN ?? "", receiptRoot: join(agentDir, "image-video") });
|
||||||
|
const cmsConnections = new Map<string, CmsConnection>();
|
||||||
|
const editors = new Map<string, EditorService>();
|
||||||
|
const swads = new SwadsMcpClient({ token: environment.SWADS_MCP_TOKEN ?? "" });
|
||||||
|
const reports = new WeakMap<HarnessSession, MinisReportService>();
|
||||||
const skillCatalog = new SkillCatalog({
|
const skillCatalog = new SkillCatalog({
|
||||||
cwd: projectRoot,
|
cwd: projectRoot,
|
||||||
agentDir,
|
agentDir,
|
||||||
...(environment.AGENT_SKILLS_DIRS === undefined
|
extraDirsValue: [dramaSkillRoot, editorSkillRoot, imageVideoSkillRoot, environment.AGENT_SKILLS_DIRS].filter(Boolean).join(delimiter),
|
||||||
? {}
|
resources: { [join(dramaSkillRoot, "SKILL.md")]: [
|
||||||
: { extraDirsValue: environment.AGENT_SKILLS_DIRS }),
|
"references/metrics.md", "references/reporting-playbook.md", "assets/report-theme.css", "scripts/generate_report.py", "examples/sample-report.json", "examples/weekly-report-example.json",
|
||||||
|
], [join(editorSkillRoot, "SKILL.md")]: EDITOR_RESOURCES, [join(imageVideoSkillRoot, "SKILL.md")]: IMAGE_VIDEO_RESOURCES },
|
||||||
});
|
});
|
||||||
await skillCatalog.reload();
|
await skillCatalog.reload();
|
||||||
const environmentConnection = await readEnvironmentConnection(environment, config.allowPrivateLlmBaseUrls);
|
const environmentConnection = await readEnvironmentConnection(environment, config.allowPrivateLlmBaseUrls);
|
||||||
@@ -55,11 +81,17 @@ export async function buildDefaultApp(environment: NodeJS.ProcessEnv = process.e
|
|||||||
maxBytes: config.maxImageBytes,
|
maxBytes: config.maxImageBytes,
|
||||||
maxAttachmentsPerMessage: config.maxImagesPerMessage,
|
maxAttachmentsPerMessage: config.maxImagesPerMessage,
|
||||||
});
|
});
|
||||||
|
const imageVideo = new ImageVideoService({ root: join(agentDir, "image-video"), attachments: attachmentStore, remote: imageVideoRemote, collect: createImageVideoCollector({ skillRoot: imageVideoSkillRoot, python: environment.PYTHON_EXECUTABLE ?? join(homedir(), ".cache/codex-runtimes/codex-primary-runtime/dependencies/python/bin/python3") }) });
|
||||||
const registry = new InMemorySessionRegistry<HarnessSession>({
|
const registry = new InMemorySessionRegistry<HarnessSession>({
|
||||||
sessionsRoot,
|
sessionsRoot,
|
||||||
createSession: ({ sessionId, workspaceRoot }) => {
|
createSession: ({ sessionId, workspaceRoot }) => {
|
||||||
|
const cms = new CmsConnection({ publishableKey: environment.CMS_SUPABASE_PUBLISHABLE_KEY ?? "" });
|
||||||
|
cmsConnections.set(sessionId, cms);
|
||||||
|
const editor = new EditorService({ cms, sessionId, runtimeRoot: join(projectRoot, ".agent-studio-runtime", "videos"),runner:new SkillRunner({skillRoot:editorSkillRoot,pythonExecutable:environment.PYTHON_EXECUTABLE??join(homedir(),".cache/codex-runtimes/codex-primary-runtime/dependencies/python/bin/python3")}) });
|
||||||
|
editors.set(sessionId, editor);
|
||||||
|
let sessionSkillIndex = skillCatalog.index();
|
||||||
eventStore.append(sessionId, { type: "session.started", payload: { cwd: "." } });
|
eventStore.append(sessionId, { type: "session.started", payload: { cwd: "." } });
|
||||||
return new HarnessSession({
|
const session = new HarnessSession({
|
||||||
sessionId,
|
sessionId,
|
||||||
workspaceRoot,
|
workspaceRoot,
|
||||||
eventStore,
|
eventStore,
|
||||||
@@ -71,31 +103,86 @@ export async function buildDefaultApp(environment: NodeJS.ProcessEnv = process.e
|
|||||||
sessionId,
|
sessionId,
|
||||||
runId,
|
runId,
|
||||||
eventStore,
|
eventStore,
|
||||||
skillIndex: skillCatalog.index(),
|
skillIndex: sessionSkillIndex,
|
||||||
}),
|
}),
|
||||||
createAgentPort: async (input) => {
|
createAgentPort: async (input) => {
|
||||||
const workspace = await SafeWorkspace.create(workspaceRoot);
|
const workspace = await SafeWorkspace.create(workspaceRoot);
|
||||||
|
const snapshot = await skillCatalog.createSessionSnapshot(workspace.root);
|
||||||
|
sessionSkillIndex = snapshot.index;
|
||||||
|
const drama = snapshot.skills.find((skill) => skill.name === "drama-minis-diagnosis");
|
||||||
|
let reportService = reports.get(session);
|
||||||
|
if (drama !== undefined && reportService === undefined) {
|
||||||
|
reportService = new MinisReportService({ workspace, sessionId, skillRoot: drama.baseDir,
|
||||||
|
...(environment.PYTHON_EXECUTABLE === undefined ? {} : { pythonExecutable: environment.PYTHON_EXECUTABLE }),
|
||||||
|
...(environment.CHROME_EXECUTABLE === undefined ? {} : { chromeExecutable: environment.CHROME_EXECUTABLE }),
|
||||||
|
});
|
||||||
|
reports.set(session, reportService);
|
||||||
|
}
|
||||||
|
const service = reportService;
|
||||||
return createPiAgentSessionPortFactory({
|
return createPiAgentSessionPortFactory({
|
||||||
agentDir,
|
agentDir,
|
||||||
|
skills: snapshot.skills,
|
||||||
|
appendSystemPrompt: ["用户选择 Skill 时必须先用 read 读取对应 SKILL.md。Drama Minis 任务必须按该 Skill 执行,并按相对路径读取 references。不要把 Skill 全文重复给用户。swads_whoami 和 metrics_catalog 是真实只读 MCP 工具;metrics_semantic_query 必须经用户确认周期、对比周期和范围,并在页面批准后才执行。不得用 mock_ads_metrics 分析真实 Minis。生成日报/周报必须用 generate_minis_report,使用本次查询返回的 sourceQueryId 和新的 JSON,不能使用示例数字。此受控工具代替 shell 运行 Python,提供 HTML、A4 PDF、JSON 下载。只在工具成功后声称文件已生成。缺少配置或数据时明确报错,不编造数据。"],
|
||||||
approvalBroker,
|
approvalBroker,
|
||||||
customTools: [
|
customTools: [
|
||||||
|
...(snapshot.skills.some(skill => skill.name === "swads-image-to-video") ? createImageVideoTools(imageVideo, sessionId) : []),
|
||||||
|
...(snapshot.skills.some((skill) => skill.name === "swads-short-drama-editor") ? [
|
||||||
|
...createEditorTools(cms,()=>({rendering:true,vision:modelStore.resolveConnection(sessionId).connection.supportsImages,humanReview:true})),
|
||||||
|
...createMediaTools(editor, () => modelStore.resolveConnection(sessionId).connection.supportsImages),
|
||||||
|
] : []),
|
||||||
createMockAdsMetricsTool(join(projectRoot, "fixtures", "ads-account.json")),
|
createMockAdsMetricsTool(join(projectRoot, "fixtures", "ads-account.json")),
|
||||||
createSaveReportDraftTool(workspace),
|
createSaveReportDraftTool(workspace),
|
||||||
|
...(service === undefined ? [] : [
|
||||||
|
...createSwadsTools(swads, (args, result) => service.recordQuery(args, result)),
|
||||||
|
createMinisReportTool(service),
|
||||||
|
]),
|
||||||
],
|
],
|
||||||
})(input);
|
})(input);
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
return session;
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
|
|
||||||
return buildApp({
|
const app = buildApp({
|
||||||
|
imageVideo,
|
||||||
|
imageVideoPorts: [config.port, 5173, 4173],
|
||||||
|
editorForSession:(sessionId)=>{const editor=editors.get(sessionId);if(!editor)throw new HarnessError("SESSION_NOT_FOUND","Session not found");return editor;},
|
||||||
|
readEditorFrame: (sessionId, jobId, frameId) => {
|
||||||
|
const editor = editors.get(sessionId);
|
||||||
|
if (!editor) throw new HarnessError("FRAME_NOT_FOUND", "Frame not found in this Session");
|
||||||
|
return editor.readFrame(jobId, frameId);
|
||||||
|
},
|
||||||
|
cmsForSession: (sessionId) => {
|
||||||
|
const cms = cmsConnections.get(sessionId);
|
||||||
|
if (cms === undefined) throw new HarnessError("SESSION_NOT_FOUND", "Session not found");
|
||||||
|
return cms;
|
||||||
|
},
|
||||||
sessionRegistry: registry,
|
sessionRegistry: registry,
|
||||||
skillCatalog,
|
skillCatalog,
|
||||||
modelStore,
|
modelStore,
|
||||||
attachmentStore,
|
attachmentStore,
|
||||||
approvalBroker,
|
approvalBroker,
|
||||||
eventStore,
|
eventStore,
|
||||||
|
readReport: async (sessionId, reportId, format) => {
|
||||||
|
const session = registry.get(sessionId);
|
||||||
|
const service = session === undefined ? undefined : reports.get(session);
|
||||||
|
if (service === undefined) throw new HarnessError("REPORT_NOT_FOUND", "Report not found in this Session");
|
||||||
|
return service.read(reportId, format);
|
||||||
|
},
|
||||||
});
|
});
|
||||||
|
app.addHook("onClose", () => swads.close());
|
||||||
|
app.addHook("onClose", () => imageVideoRemote.close());
|
||||||
|
app.addHook("onClose", () => { editors.clear(); });
|
||||||
|
app.addHook("onClose", () => { for (const cms of cmsConnections.values()) cms.clear(); cmsConnections.clear(); });
|
||||||
|
app.addHook("onResponse", (request, _reply, done) => {
|
||||||
|
if (request.method === "DELETE" && /^\/api\/sessions\/ses_[a-zA-Z0-9_-]+$/.test(request.url)) {
|
||||||
|
const id = request.url.split("/").at(-1) ?? "";
|
||||||
|
if (registry.get(id) === undefined) { cmsConnections.get(id)?.clear(); cmsConnections.delete(id); editors.delete(id); }
|
||||||
|
}
|
||||||
|
done();
|
||||||
|
});
|
||||||
|
return app;
|
||||||
}
|
}
|
||||||
|
|
||||||
async function readEnvironmentConnection(
|
async function readEnvironmentConnection(
|
||||||
@@ -122,6 +209,7 @@ async function readEnvironmentConnection(
|
|||||||
modelId,
|
modelId,
|
||||||
api,
|
api,
|
||||||
supportsImages: environment.AGENT_MODEL_SUPPORTS_IMAGES !== "false",
|
supportsImages: environment.AGENT_MODEL_SUPPORTS_IMAGES !== "false",
|
||||||
|
...(environment.AGENT_MODEL_MAX_TOKENS === undefined ? {} : { maxTokens: Number(environment.AGENT_MODEL_MAX_TOKENS) }),
|
||||||
...(baseUrl === undefined ? {} : { baseUrl }),
|
...(baseUrl === undefined ? {} : { baseUrl }),
|
||||||
...(apiKey === undefined || apiKey === "" ? {} : { apiKey }),
|
...(apiKey === undefined || apiKey === "" ? {} : { apiKey }),
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -0,0 +1,58 @@
|
|||||||
|
import { HarnessError, type CmsConnection } from "@agent-studio/harness";
|
||||||
|
import type { FastifyInstance, FastifyRequest } from "fastify";
|
||||||
|
import { z } from "zod";
|
||||||
|
import type { AppDependencies } from "../app.js";
|
||||||
|
import { requireSession } from "./sessions.js";
|
||||||
|
|
||||||
|
const Params = z.object({ sessionId: z.string().regex(/^ses_[a-zA-Z0-9_-]+$/) });
|
||||||
|
const Login = z.object({ email: z.string().email().max(254), password: z.string().min(1).max(1024) }).strict();
|
||||||
|
|
||||||
|
export function registerCmsRoutes(app: FastifyInstance, dependencies: AppDependencies): void {
|
||||||
|
const attempts = new Map<string, { count: number; until: number }>();
|
||||||
|
const connection = (request: FastifyRequest, mutate = false): CmsConnection => {
|
||||||
|
const { sessionId } = Params.parse(request.params);
|
||||||
|
const session = requireSession(dependencies, sessionId);
|
||||||
|
if (mutate && session.isBusy === true) throw new HarnessError("SESSION_BUSY", "运行期间不能修改 CMS 连接");
|
||||||
|
if (dependencies.cmsForSession === undefined) throw new HarnessError("CMS_NOT_CONFIGURED", "CMS 连接不可用");
|
||||||
|
return dependencies.cmsForSession(sessionId);
|
||||||
|
};
|
||||||
|
const guard = (request: FastifyRequest): void => {
|
||||||
|
const origin = request.headers.origin;
|
||||||
|
if (request.headers["sec-fetch-site"] === "cross-site" || (origin !== undefined && origin !== `http://${request.headers.host ?? ""}`)) {
|
||||||
|
throw new HarnessError("CMS_ORIGIN_DENIED", "CMS 请求必须来自当前本机页面");
|
||||||
|
}
|
||||||
|
};
|
||||||
|
app.get("/api/sessions/:sessionId/cms/connection", (request, reply) => {
|
||||||
|
guard(request);
|
||||||
|
void reply.header("Cache-Control", "no-store");
|
||||||
|
return { connection: connection(request).status() };
|
||||||
|
});
|
||||||
|
app.post("/api/sessions/:sessionId/cms/connection", async (request, reply) => {
|
||||||
|
guard(request);
|
||||||
|
void reply.header("Cache-Control", "no-store");
|
||||||
|
if (!request.headers["content-type"]?.startsWith("application/json")) throw new HarnessError("CMS_INVALID_CONTENT_TYPE", "CMS login requires JSON");
|
||||||
|
const cms = connection(request, true);
|
||||||
|
const input = Login.parse(request.body);
|
||||||
|
const key = request.ip;
|
||||||
|
const now = Date.now();
|
||||||
|
for (const [id, value] of attempts) if (value.until < now) attempts.delete(id);
|
||||||
|
const record = attempts.get(key) ?? { count: 0, until: now + 60_000 };
|
||||||
|
if (record.count >= 5) throw new HarnessError("CMS_RATE_LIMITED", "登录尝试过多,请一分钟后再试");
|
||||||
|
record.count++; attempts.set(key, record);
|
||||||
|
try { return { connection: await cms.login(input.email, input.password) }; }
|
||||||
|
finally { input.password = ""; }
|
||||||
|
});
|
||||||
|
app.delete("/api/sessions/:sessionId/cms/connection", (request, reply) => {
|
||||||
|
guard(request);
|
||||||
|
void reply.header("Cache-Control", "no-store");
|
||||||
|
const cms = connection(request, true);
|
||||||
|
cms.clear();
|
||||||
|
return { connection: cms.status() };
|
||||||
|
});
|
||||||
|
app.get("/api/sessions/:sessionId/cms/dramas", async (request, reply) => {
|
||||||
|
guard(request);
|
||||||
|
void reply.header("Cache-Control", "no-store");
|
||||||
|
const query = z.object({ q: z.string().max(120).default(""), offset: z.coerce.number().int().min(0).max(10_000).default(0) }).parse(request.query);
|
||||||
|
return connection(request).listDramas(query.q, query.offset);
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
import { HarnessError } from "@agent-studio/harness";
|
||||||
|
import type { FastifyInstance,FastifyRequest } from "fastify";
|
||||||
|
import {createReadStream} from "node:fs";
|
||||||
|
import { z } from "zod";
|
||||||
|
import type { AppDependencies } from "../app.js";
|
||||||
|
import { requireSession } from "./sessions.js";
|
||||||
|
|
||||||
|
const Params = z.object({ sessionId: z.string().regex(/^ses_[\w-]+$/), jobId: z.string().regex(/^job_[\w-]+$/), frameId: z.string().regex(/^frame_[\w-]+$/) }).strict();
|
||||||
|
export function registerEditorRoutes(app: FastifyInstance, dependencies: AppDependencies): void {
|
||||||
|
const context=(request:FastifyRequest)=>{
|
||||||
|
const params=z.object({sessionId:z.string().regex(/^ses_[\w-]+$/),jobId:z.string().regex(/^job_[\w-]+$/)}).parse(request.params);
|
||||||
|
requireSession(dependencies,params.sessionId);
|
||||||
|
if(request.headers["sec-fetch-site"]==="cross-site"||(request.headers.origin!==undefined&&request.headers.origin!==`http://${request.headers.host??""}`))throw new HarnessError("CMS_ORIGIN_DENIED","媒体和审核仅限当前本机页面");
|
||||||
|
if(!dependencies.editorForSession)throw new HarnessError("EDITOR_JOB_NOT_FOUND","Editor unavailable");
|
||||||
|
return {...params,editor:dependencies.editorForSession(params.sessionId)};
|
||||||
|
};
|
||||||
|
app.get("/api/sessions/:sessionId/editor/:jobId/workflow",(request,reply)=>{const ctx=context(request);void reply.header("Cache-Control","no-store");return ctx.editor.workflowView(ctx.jobId);});
|
||||||
|
app.post("/api/sessions/:sessionId/editor/:jobId/review/:stage",async(request,reply)=>{
|
||||||
|
const ctx=context(request);const {stage}=z.object({stage:z.enum(["selection","plan"])}).parse(request.params);
|
||||||
|
if(!request.headers["content-type"]?.startsWith("application/json"))throw new HarnessError("CMS_INVALID_CONTENT_TYPE","Review requires JSON");
|
||||||
|
void reply.header("Cache-Control","no-store");return ctx.editor.review(ctx.jobId,stage,request.body);
|
||||||
|
});
|
||||||
|
for(const kind of ["sources","plans"] as const){
|
||||||
|
app.get(`/api/sessions/:sessionId/editor/:jobId/${kind}/:mediaId${kind==="plans"?"/preview":""}`,async(request,reply)=>{
|
||||||
|
const ctx=context(request);const {mediaId}=z.object({mediaId:z.string().regex(/^[\w-]{1,100}$/)}).parse(request.params);
|
||||||
|
const media=await ctx.editor.video(ctx.jobId,kind,mediaId);
|
||||||
|
void reply.header("Cache-Control","no-store").header("X-Content-Type-Options","nosniff").header("Accept-Ranges","bytes").type("video/mp4");
|
||||||
|
const query=z.object({download:z.enum(["0","1"]).optional()}).parse(request.query);
|
||||||
|
if(query.download==="1")void reply.header("Content-Disposition",`attachment; filename="${kind==="plans"?"REVIEW-preview":"licensed-source"}.mp4"`);
|
||||||
|
const range=request.headers.range;
|
||||||
|
if(range!==undefined){
|
||||||
|
const match=/^bytes=(\d*)-(\d*)$/.exec(range);let start=0;let end=media.size-1;
|
||||||
|
if(match?.[1]){start=Number(match[1]);if(match[2])end=Math.min(Number(match[2]),end);}
|
||||||
|
else if(match?.[2]){start=Math.max(0,media.size-Number(match[2]));}
|
||||||
|
if(!match||(!match[1]&&!match[2])||!Number.isSafeInteger(start)||!Number.isSafeInteger(end)||start>end||start>=media.size||end<0){return reply.code(416).header("Content-Range",`bytes */${String(media.size)}`).send();}
|
||||||
|
return reply.code(206).header("Content-Range",`bytes ${String(start)}-${String(end)}/${String(media.size)}`).header("Content-Length",end-start+1).send(createReadStream(media.path,{start,end}));
|
||||||
|
}
|
||||||
|
return reply.header("Content-Length",media.size).send(createReadStream(media.path));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
app.get("/api/sessions/:sessionId/editor/:jobId/frames/:frameId", async (request, reply) => {
|
||||||
|
const { sessionId, jobId, frameId } = Params.parse(request.params);
|
||||||
|
requireSession(dependencies, sessionId);
|
||||||
|
if (!dependencies.readEditorFrame) throw new HarnessError("FRAME_NOT_FOUND", "Frame not found");
|
||||||
|
const bytes = await dependencies.readEditorFrame(sessionId, jobId, frameId);
|
||||||
|
return reply.header("Cache-Control", "no-store").header("X-Content-Type-Options", "nosniff").type("image/jpeg").send(bytes);
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
import { createReadStream } from "node:fs";
|
||||||
|
import { HarnessError, ImageVideoDraftSchema, taskIdSchema } from "@agent-studio/harness";
|
||||||
|
import type { FastifyInstance, FastifyRequest } from "fastify";
|
||||||
|
import { z } from "zod";
|
||||||
|
import type { AppDependencies } from "../app.js";
|
||||||
|
import { requireSession } from "./sessions.js";
|
||||||
|
|
||||||
|
export function registerImageVideoRoutes(app: FastifyInstance, dependencies: AppDependencies) {
|
||||||
|
const context = (request: FastifyRequest) => {
|
||||||
|
const host = request.headers.host ?? "";
|
||||||
|
const ports = dependencies.imageVideoPorts ?? [80, 3001, 5173, 4173];
|
||||||
|
let url: URL;
|
||||||
|
try { url = new URL(`http://${host}`); } catch { throw new HarnessError("I2V_ORIGIN_DENIED", "Local origin required"); }
|
||||||
|
if (!["localhost", "127.0.0.1", "[::1]"].includes(url.hostname) || !ports.includes(Number(url.port || 80)) || request.headers["sec-fetch-site"] === "cross-site" || (request.headers.origin !== undefined && request.headers.origin !== url.origin)) throw new HarnessError("I2V_ORIGIN_DENIED", "Image video tasks are available only from the local page");
|
||||||
|
if (!dependencies.imageVideo) throw new HarnessError("I2V_TASK_NOT_FOUND", "Image video service unavailable");
|
||||||
|
return dependencies.imageVideo;
|
||||||
|
};
|
||||||
|
app.get("/api/image-video/tasks", async (request, reply) => {
|
||||||
|
const service = context(request); void reply.header("Cache-Control", "no-store");
|
||||||
|
const history = await service.list();
|
||||||
|
return { tasks: await Promise.all(history.map(task => service.refresh(task.id))) };
|
||||||
|
});
|
||||||
|
app.get("/api/image-video/tasks/:taskId", async (request, reply) => {
|
||||||
|
const service = context(request); const { taskId } = z.object({ taskId: taskIdSchema }).parse(request.params);
|
||||||
|
void reply.header("Cache-Control", "no-store"); return { task: await service.refresh(taskId) };
|
||||||
|
});
|
||||||
|
app.post("/api/sessions/:sessionId/image-video/tasks", async (request, reply) => {
|
||||||
|
const service = context(request); const { sessionId } = z.object({ sessionId: z.string().regex(/^ses_[\w-]+$/) }).parse(request.params);
|
||||||
|
requireSession(dependencies, sessionId);
|
||||||
|
if (!request.headers["content-type"]?.startsWith("application/json")) throw new HarnessError("I2V_JSON_REQUIRED", "Draft creation requires JSON");
|
||||||
|
void reply.header("Cache-Control", "no-store"); return { task: await service.create(sessionId, ImageVideoDraftSchema.parse(request.body)) };
|
||||||
|
});
|
||||||
|
app.post("/api/sessions/:sessionId/image-video/tasks/:taskId/continue", async (request, reply) => {
|
||||||
|
const service = context(request);
|
||||||
|
const { sessionId, taskId } = z.object({ sessionId: z.string().regex(/^ses_[\w-]+$/), taskId: taskIdSchema }).parse(request.params);
|
||||||
|
const session = requireSession(dependencies, sessionId);
|
||||||
|
if (session.isBusy) throw new HarnessError("SESSION_BUSY", "请等待当前会话结束再接续任务。");
|
||||||
|
if (!request.headers["content-type"]?.startsWith("application/json")) throw new HarnessError("I2V_JSON_REQUIRED", "Continuation requires JSON");
|
||||||
|
const { expectedRevision } = z.object({ confirmed: z.literal(true), expectedRevision: z.number().int().nonnegative() }).strict().parse(request.body);
|
||||||
|
void reply.header("Cache-Control", "no-store");
|
||||||
|
return { task: await service.continueTask(sessionId, taskId, expectedRevision, owner => dependencies.sessionRegistry.get(owner) !== undefined, () => {
|
||||||
|
const current = requireSession(dependencies, sessionId);
|
||||||
|
if (current !== session || current.isBusy) throw new HarnessError("SESSION_BUSY", "会话状态已改变,请重新确认接续。");
|
||||||
|
}) };
|
||||||
|
});
|
||||||
|
for (const kind of ["reference", "video"] as const) app.get(`/api/image-video/tasks/:taskId/${kind}`, async (request, reply) => {
|
||||||
|
const service = context(request); const { taskId } = z.object({ taskId: taskIdSchema }).parse(request.params);
|
||||||
|
const query = z.object({ download: z.literal("1").optional() }).strict().parse(request.query);
|
||||||
|
const media = await service.media(taskId, kind);
|
||||||
|
void reply.header("Cache-Control", "no-store").header("X-Content-Type-Options", "nosniff").header("Cross-Origin-Resource-Policy", "same-origin").header("Accept-Ranges", "bytes").type(media.contentType);
|
||||||
|
if (query.download === "1") void reply.header("Content-Disposition", `attachment; filename="${taskId}${kind === "video" ? ".mp4" : ".image"}"`);
|
||||||
|
const range = request.headers.range;
|
||||||
|
if (range !== undefined) {
|
||||||
|
const match = /^bytes=(\d*)-(\d*)$/.exec(range); let start = 0; let end = media.size - 1;
|
||||||
|
if (match?.[1]) { start = Number(match[1]); if (match[2]) end = Math.min(Number(match[2]), end); }
|
||||||
|
else if (match?.[2]) start = Math.max(0, media.size - Number(match[2]));
|
||||||
|
if (!match || (!match[1] && !match[2]) || !Number.isSafeInteger(start) || !Number.isSafeInteger(end) || start > end || start >= media.size || end < 0) return reply.code(416).header("Content-Range", `bytes */${String(media.size)}`).send();
|
||||||
|
return reply.code(206).header("Content-Range", `bytes ${String(start)}-${String(end)}/${String(media.size)}`).header("Content-Length", end - start + 1).send(createReadStream(media.path, { start, end }));
|
||||||
|
}
|
||||||
|
return reply.header("Content-Length", media.size).send(createReadStream(media.path));
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -21,9 +21,32 @@ export function registerModelRoutes(app: FastifyInstance, dependencies: AppDepen
|
|||||||
if (session.isBusy === true) {
|
if (session.isBusy === true) {
|
||||||
throw new HarnessError("SESSION_BUSY", "Model configuration cannot change during a run");
|
throw new HarnessError("SESSION_BUSY", "Model configuration cannot change during a run");
|
||||||
}
|
}
|
||||||
const input = ModelConnectionInputSchema.parse(request.body);
|
try {
|
||||||
await dependencies.modelStore.configure(sessionId, input);
|
const input = ModelConnectionInputSchema.parse(request.body);
|
||||||
return { modelConfig: dependencies.modelStore.getRedacted(sessionId), requestId: request.id };
|
await dependencies.modelStore.configure(sessionId, input);
|
||||||
|
const modelConfig = dependencies.modelStore.getRedacted(sessionId);
|
||||||
|
if (modelConfig !== undefined) {
|
||||||
|
dependencies.eventStore?.append(sessionId, {
|
||||||
|
type: "model.configured",
|
||||||
|
payload: {
|
||||||
|
providerId: modelConfig.providerId,
|
||||||
|
modelId: modelConfig.modelId,
|
||||||
|
supportsImages: modelConfig.supportsImages,
|
||||||
|
source: modelConfig.source,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return { modelConfig, requestId: request.id };
|
||||||
|
} catch (error) {
|
||||||
|
dependencies.eventStore?.append(sessionId, {
|
||||||
|
type: "model.validation_failed",
|
||||||
|
payload: {
|
||||||
|
code: error instanceof HarnessError ? error.code : "MODEL_VALIDATION_FAILED",
|
||||||
|
message: "Model configuration validation failed",
|
||||||
|
},
|
||||||
|
});
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
app.delete("/api/sessions/:sessionId/model-config/credentials", (request) => {
|
app.delete("/api/sessions/:sessionId/model-config/credentials", (request) => {
|
||||||
@@ -37,9 +60,17 @@ export function registerModelRoutes(app: FastifyInstance, dependencies: AppDepen
|
|||||||
const { sessionId } = Params.parse(request.params);
|
const { sessionId } = Params.parse(request.params);
|
||||||
requireSession(dependencies, sessionId);
|
requireSession(dependencies, sessionId);
|
||||||
const resolved = dependencies.modelStore.resolveConnection(sessionId);
|
const resolved = dependencies.modelStore.resolveConnection(sessionId);
|
||||||
const result = dependencies.testModelConnection === undefined
|
try {
|
||||||
? { status: "deferred" as const, message: "Saved; validation will occur on first run." }
|
const result = dependencies.testModelConnection === undefined
|
||||||
: await dependencies.testModelConnection(resolved, AbortSignal.timeout(5_000));
|
? { status: "deferred" as const, message: "Saved; validation will occur on first run." }
|
||||||
return { ...result, requestId: request.id };
|
: await dependencies.testModelConnection(resolved, AbortSignal.timeout(5_000));
|
||||||
|
return { ...result, requestId: request.id };
|
||||||
|
} catch {
|
||||||
|
dependencies.eventStore?.append(sessionId, {
|
||||||
|
type: "model.validation_failed",
|
||||||
|
payload: { code: "MODEL_CONNECTION_FAILED", message: "Model connection validation failed" },
|
||||||
|
});
|
||||||
|
throw new HarnessError("MODEL_CONNECTION_FAILED", "Model connection validation failed");
|
||||||
|
}
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,20 @@
|
|||||||
|
import { HarnessError } from "@agent-studio/harness";
|
||||||
|
import type { FastifyInstance } from "fastify";
|
||||||
|
import { z } from "zod";
|
||||||
|
import type { AppDependencies } from "../app.js";
|
||||||
|
import { requireSession } from "./sessions.js";
|
||||||
|
|
||||||
|
const Params = z.object({ sessionId: z.string().regex(/^ses_[\w-]+$/), reportId: z.string().regex(/^rpt_[\w-]+$/), format: z.enum(["html", "pdf", "json"]) }).strict();
|
||||||
|
|
||||||
|
export function registerReportRoutes(app: FastifyInstance, dependencies: AppDependencies): void {
|
||||||
|
app.get("/api/sessions/:sessionId/reports/:reportId/:format", async (request, reply) => {
|
||||||
|
const { sessionId, reportId, format } = Params.parse(request.params);
|
||||||
|
requireSession(dependencies, sessionId);
|
||||||
|
if (dependencies.readReport === undefined) throw new HarnessError("REPORT_NOT_FOUND", "Report not found");
|
||||||
|
const file = await dependencies.readReport(sessionId, reportId, format);
|
||||||
|
return reply.header("Cache-Control", "no-store").header("X-Content-Type-Options", "nosniff")
|
||||||
|
.header("Content-Security-Policy", "sandbox allow-scripts; default-src 'none'; style-src 'unsafe-inline'; script-src 'unsafe-inline'; img-src data:")
|
||||||
|
.header("Content-Disposition", `attachment; filename="${reportId}.${format}"`)
|
||||||
|
.type(file.contentType).send(file.bytes);
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -53,4 +53,21 @@ describe("HTTP API", () => {
|
|||||||
expect(busy.statusCode).toBe(409);
|
expect(busy.statusCode).toBe(409);
|
||||||
expect(busy.body).not.toContain("stack");
|
expect(busy.body).not.toContain("stack");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("keeps Fastify body parse errors as safe client errors", async () => {
|
||||||
|
const { app } = createTestApp();
|
||||||
|
apps.push(app);
|
||||||
|
const response = await app.inject({
|
||||||
|
method: "POST",
|
||||||
|
url: "/api/sessions",
|
||||||
|
headers: { "content-type": "application/json" },
|
||||||
|
payload: "",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(response.statusCode).toBe(400);
|
||||||
|
expect(response.json()).toMatchObject({
|
||||||
|
error: { code: "INVALID_REQUEST_BODY", message: "Request body could not be processed" },
|
||||||
|
});
|
||||||
|
expect(response.body).not.toContain("stack");
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -0,0 +1,21 @@
|
|||||||
|
import { CmsConnection } from "@agent-studio/harness";
|
||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import { createTestApp } from "./test-app.js";
|
||||||
|
|
||||||
|
describe("CMS API safety", () => {
|
||||||
|
it("exposes only redacted connection status and rejects cross-origin login before contacting CMS", async () => {
|
||||||
|
const cms = new CmsConnection({ publishableKey: "sb_publishable_test", fetch: () => Promise.reject(new Error("Must not reach CMS")) });
|
||||||
|
const { app } = createTestApp({ cmsForSession: () => cms });
|
||||||
|
try {
|
||||||
|
const status = await app.inject({ method: "GET", url: "/api/sessions/ses_1/cms/connection" });
|
||||||
|
expect(status.statusCode).toBe(200);
|
||||||
|
expect(status.json()).toMatchObject({ connection: { connected: false, configured: true } });
|
||||||
|
expect(status.headers["cache-control"]).toBe("no-store");
|
||||||
|
const denied = await app.inject({ method: "POST", url: "/api/sessions/ses_1/cms/connection", headers: { origin: "https://attacker.example", "sec-fetch-site": "cross-site" }, payload: { email: "fixture@example.com", password: "never-expose" } });
|
||||||
|
expect(denied.statusCode).toBe(403);
|
||||||
|
expect(denied.body).not.toContain("never-expose");
|
||||||
|
expect((await app.inject({ method: "GET", url: "/api/sessions/ses_missing/cms/connection" })).statusCode).toBe(404);
|
||||||
|
expect((await app.inject({ method: "GET", url: "/api/sessions/ses_1/cms/dramas" })).json()).toMatchObject({ error: { code: "CMS_LOGIN_REQUIRED" } });
|
||||||
|
} finally { await app.close(); }
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import { HarnessError } from "@agent-studio/harness";
|
||||||
|
import { createTestApp } from "./test-app.js";
|
||||||
|
import {mkdtemp,writeFile,rm} from "node:fs/promises";
|
||||||
|
import {tmpdir} from "node:os";
|
||||||
|
import {join} from "node:path";
|
||||||
|
|
||||||
|
describe("private editor frames", () => {
|
||||||
|
it("only serves a registered Session frame and forbids caching", async () => {
|
||||||
|
const { app } = createTestApp({ readEditorFrame: (sessionId, jobId, frameId) => {
|
||||||
|
if (sessionId !== "ses_1" || jobId !== "job_known" || frameId !== "frame_known") throw new HarnessError("FRAME_NOT_FOUND", "Missing frame");
|
||||||
|
return Promise.resolve(Buffer.from([255, 216, 255, 217]));
|
||||||
|
} });
|
||||||
|
try {
|
||||||
|
const response = await app.inject({ method: "GET", url: "/api/sessions/ses_1/editor/job_known/frames/frame_known" });
|
||||||
|
expect(response.statusCode).toBe(200);
|
||||||
|
expect(response.headers["content-type"]).toBe("image/jpeg");
|
||||||
|
expect(response.headers["cache-control"]).toBe("no-store");
|
||||||
|
expect(response.rawPayload).toEqual(Buffer.from([255, 216, 255, 217]));
|
||||||
|
expect((await app.inject({ method: "GET", url: "/api/sessions/ses_missing/editor/job_known/frames/frame_known" })).statusCode).toBe(404);
|
||||||
|
expect((await app.inject({ method: "GET", url: "/api/sessions/ses_1/editor/job_known/frames/secret.env" })).statusCode).toBe(400);
|
||||||
|
} finally { await app.close(); }
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("streams only registered media ranges and keeps review mutations same-origin",async()=>{
|
||||||
|
const root=await mkdtemp(join(tmpdir(),"editor-api-"));const path=join(root,"preview.mp4");await writeFile(path,Buffer.from("0123456789"));
|
||||||
|
const {app}=createTestApp({editorForSession:()=>({workflowView:()=>({jobId:"job_test"}),review:()=>Promise.resolve({saved:true}),video:()=>Promise.resolve({path,size:10})})});
|
||||||
|
try{
|
||||||
|
const range=await app.inject({method:"GET",url:"/api/sessions/ses_1/editor/job_test/plans/plan_test/preview",headers:{range:"bytes=2-5"}});
|
||||||
|
expect(range.statusCode).toBe(206);expect(range.body).toBe("2345");expect(range.headers["content-range"]).toBe("bytes 2-5/10");expect(range.headers["content-type"]).toBe("video/mp4");
|
||||||
|
expect((await app.inject({method:"GET",url:"/api/sessions/ses_1/editor/job_test/plans/plan_test/preview",headers:{range:"bytes=20-30"}})).statusCode).toBe(416);
|
||||||
|
expect((await app.inject({method:"GET",url:"/api/sessions/ses_missing/editor/job_test/plans/plan_test/preview"})).statusCode).toBe(404);
|
||||||
|
expect((await app.inject({method:"POST",url:"/api/sessions/ses_1/editor/job_test/review/selection",headers:{origin:"https://evil.example","content-type":"application/json"},payload:{}})).statusCode).toBeGreaterThanOrEqual(400);
|
||||||
|
expect((await app.inject({method:"POST",url:"/api/sessions/ses_1/editor/job_test/review/selection",payload:{}})).json()).toEqual({saved:true});
|
||||||
|
}finally{await app.close();await rm(root,{recursive:true,force:true});}
|
||||||
|
});
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import { buildApp, type AppDependencies } from "../src/app.js";
|
||||||
|
import { ImageVideoService, FileAttachmentStore } from "@agent-studio/harness";
|
||||||
|
import type { ImageVideoPublicTask } from "@agent-studio/harness";
|
||||||
|
import { mkdtemp, mkdir } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { Readable } from "node:stream";
|
||||||
|
|
||||||
|
async function fixture() {
|
||||||
|
const root = await mkdtemp(join(tmpdir(), "i2v-api-")); await mkdir(join(root, "ses_1"));
|
||||||
|
const attachments = new FileAttachmentStore({ sessionsRoot: root });
|
||||||
|
const image = await attachments.add("ses_1", { name: "pixel.png", mediaType: "image/png", stream: Readable.from(Buffer.from("iVBORw0KGgoAAAANSUhEUgAAAAIAAAACCAIAAAD91JpzAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAAE0lEQVQImWP4z8DwnwGM/zMwAAAf7gP9qS/A4gAAAABJRU5ErkJggg==", "base64")) });
|
||||||
|
const service = new ImageVideoService({ root: join(root, "tasks"), attachments, remote: {} as never, collect: () => Promise.reject(new Error("No external operations permitted")) });
|
||||||
|
const dependencies = { imageVideo: service, sessionRegistry: { get: (id: string) => id === "ses_1" ? { sessionId: id } : undefined, disposeAll: async () => {} }, skillCatalog: { list: () => [] }, modelStore: {}, attachmentStore: attachments, approvalBroker: {} } as unknown as AppDependencies;
|
||||||
|
return { app: buildApp(dependencies), service, dependencies, draft: { attachmentId: image.attachmentId, title: "Reference", prompt: "Natural motion and ambient sound", durationSeconds: 10 } };
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("image video local API", () => {
|
||||||
|
it("does not persist a target session that disappeared during continuation", async () => {
|
||||||
|
const { app, service, dependencies, draft } = await fixture();
|
||||||
|
const task = await service.create("ses_1", draft);
|
||||||
|
let lookups = 0;
|
||||||
|
dependencies.sessionRegistry.get = id => id === "ses_new" && lookups++ === 0 ? { sessionId: id } as never : undefined;
|
||||||
|
try {
|
||||||
|
const result = await app.inject({ method: "POST", url: `/api/sessions/ses_new/image-video/tasks/${task.id}/continue`, payload: { confirmed: true, expectedRevision: 0 } });
|
||||||
|
expect(result.statusCode).toBe(404);
|
||||||
|
expect((await service.owned("ses_1", task.id)).revision).toBe(0);
|
||||||
|
} finally { await app.close(); }
|
||||||
|
});
|
||||||
|
it("requires an explicit same-origin confirmation to continue an orphaned task in a live session", async () => {
|
||||||
|
const { app, service, dependencies, draft } = await fixture();
|
||||||
|
const task = await service.create("ses_1", draft);
|
||||||
|
const liveSession = { sessionId: "ses_new" } as never;
|
||||||
|
dependencies.sessionRegistry.get = id => id === "ses_new" ? liveSession : undefined;
|
||||||
|
const url = `/api/sessions/ses_new/image-video/tasks/${task.id}/continue`;
|
||||||
|
const payload = { confirmed: true, expectedRevision: 0 };
|
||||||
|
try {
|
||||||
|
expect((await app.inject({ method: "POST", url, payload: { expectedRevision: 0 } })).statusCode).toBe(400);
|
||||||
|
expect((await app.inject({ method: "POST", url, payload: { ...payload, confirmed: false } })).statusCode).toBe(400);
|
||||||
|
expect((await app.inject({ method: "POST", url, payload, headers: { origin: "https://evil.example" } })).statusCode).toBeGreaterThanOrEqual(400);
|
||||||
|
expect((await app.inject({ method: "POST", url, payload, headers: { "sec-fetch-site": "cross-site" } })).statusCode).toBeGreaterThanOrEqual(400);
|
||||||
|
expect((await app.inject({ method: "POST", url: url.replace("ses_new", "ses_unknown"), payload })).statusCode).toBe(404);
|
||||||
|
expect((await app.inject({ method: "POST", url, payload: { ...payload, prompt: "changed" } })).statusCode).toBe(400);
|
||||||
|
await expect(service.owned("ses_new", task.id)).rejects.toMatchObject({ code: "I2V_TASK_NOT_FOUND" });
|
||||||
|
const result = await app.inject({ method: "POST", url, payload });
|
||||||
|
expect(result.statusCode).toBe(200);
|
||||||
|
expect(result.json<{ task: ImageVideoPublicTask }>().task).toMatchObject({ id: task.id, state: "draft", revision: 1 });
|
||||||
|
expect((await service.owned("ses_new", task.id)).prompt).toBe(draft.prompt);
|
||||||
|
expect((await service.list())).toHaveLength(1);
|
||||||
|
} finally { await app.close(); }
|
||||||
|
});
|
||||||
|
it("creates locally, persists history independently from Sessions and serves registered reference ranges", async () => {
|
||||||
|
const { app, service, draft } = await fixture();
|
||||||
|
try {
|
||||||
|
const response = await app.inject({ method: "POST", url: "/api/sessions/ses_1/image-video/tasks", payload: draft });
|
||||||
|
expect(response.statusCode).toBe(200); const task = response.json<{ task: ImageVideoPublicTask }>().task;
|
||||||
|
expect(task.state).toBe("draft");
|
||||||
|
expect((await app.inject({ method: "GET", url: "/api/image-video/tasks" })).json<{ tasks: ImageVideoPublicTask[] }>().tasks).toHaveLength(1);
|
||||||
|
const ranged = await app.inject({ method: "GET", url: task.referenceUrl, headers: { range: "bytes=0-3" } });
|
||||||
|
expect(ranged.statusCode).toBe(206); expect(ranged.rawPayload.length).toBe(4); expect(ranged.headers["cache-control"]).toBe("no-store");
|
||||||
|
expect((await app.inject({ method: "GET", url: task.referenceUrl, headers: { range: "bytes=999999-" } })).statusCode).toBe(416);
|
||||||
|
expect((await app.inject({ method: "GET", url: `/api/image-video/tasks/${task.id}/video` })).statusCode).toBe(404);
|
||||||
|
expect((await service.list())[0]?.id).toBe(task.id);
|
||||||
|
} finally { await app.close(); }
|
||||||
|
});
|
||||||
|
it("rejects foreign origins, rebinding hosts, unknown sessions, arbitrary media paths and mutation types", async () => {
|
||||||
|
const { app, draft } = await fixture();
|
||||||
|
try {
|
||||||
|
for (const headers of [{ host: "evil.example", origin: "http://evil.example" }, { origin: "https://evil.example" }, { "sec-fetch-site": "cross-site" }]) expect((await app.inject({ method: "GET", url: "/api/image-video/tasks", headers })).statusCode).toBeGreaterThanOrEqual(400);
|
||||||
|
expect((await app.inject({ method: "POST", url: "/api/sessions/ses_other/image-video/tasks", payload: draft })).statusCode).toBe(404);
|
||||||
|
expect((await app.inject({ method: "POST", url: "/api/sessions/ses_1/image-video/tasks", payload: "text", headers: { "content-type": "text/plain" } })).statusCode).toBe(400);
|
||||||
|
expect((await app.inject({ method: "GET", url: "/api/image-video/tasks/secret.env/reference" })).statusCode).toBe(400);
|
||||||
|
expect((await app.inject({ method: "POST", url: "/api/sessions/ses_1/image-video/tasks", payload: { ...draft, path: "/etc/passwd" } })).statusCode).toBe(400);
|
||||||
|
} finally { await app.close(); }
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -6,7 +6,7 @@ afterEach(async () => Promise.all(apps.splice(0).map((app) => app.close())));
|
|||||||
|
|
||||||
describe("model API", () => {
|
describe("model API", () => {
|
||||||
it("configures and returns only redacted model metadata", async () => {
|
it("configures and returns only redacted model metadata", async () => {
|
||||||
const { app, modelStore } = createTestApp();
|
const { app, modelStore, eventStore } = createTestApp();
|
||||||
apps.push(app);
|
apps.push(app);
|
||||||
const configured = await app.inject({ method: "PUT", url: "/api/sessions/ses_1/model-config", payload: {
|
const configured = await app.inject({ method: "PUT", url: "/api/sessions/ses_1/model-config", payload: {
|
||||||
providerId: "training", api: "openai-responses", apiKey: "sk-super-secret",
|
providerId: "training", api: "openai-responses", apiKey: "sk-super-secret",
|
||||||
@@ -21,6 +21,10 @@ describe("model API", () => {
|
|||||||
const body = response.json<{ modelConfig: { keyHint: string }; requestId: string }>();
|
const body = response.json<{ modelConfig: { keyHint: string }; requestId: string }>();
|
||||||
expect(body.modelConfig.keyHint).toBe("••••cret");
|
expect(body.modelConfig.keyHint).toBe("••••cret");
|
||||||
expect(body.requestId).toMatch(/^req_/);
|
expect(body.requestId).toMatch(/^req_/);
|
||||||
|
expect(eventStore.listAfter("ses_1").events.at(-1)).toMatchObject({
|
||||||
|
type: "model.configured",
|
||||||
|
payload: { providerId: "training", modelId: "training-model", supportsImages: true, source: "session" },
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
it("clears credentials idempotently and returns an honest deferred connection test", async () => {
|
it("clears credentials idempotently and returns an honest deferred connection test", async () => {
|
||||||
@@ -36,4 +40,21 @@ describe("model API", () => {
|
|||||||
expect(body.requestId).toMatch(/^req_/);
|
expect(body.requestId).toMatch(/^req_/);
|
||||||
expect(tested.body).not.toContain("server-only-secret");
|
expect(tested.body).not.toContain("server-only-secret");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("emits a safe validation event when model configuration fails", async () => {
|
||||||
|
const { app, modelStore, eventStore } = createTestApp();
|
||||||
|
apps.push(app);
|
||||||
|
modelStore.configure.mockRejectedValueOnce(new Error("provider leaked sk-super-secret"));
|
||||||
|
|
||||||
|
const response = await app.inject({ method: "PUT", url: "/api/sessions/ses_1/model-config", payload: {
|
||||||
|
providerId: "training", api: "openai-responses", apiKey: "sk-super-secret",
|
||||||
|
modelId: "training-model", supportsImages: true,
|
||||||
|
} });
|
||||||
|
|
||||||
|
expect(response.statusCode).toBe(500);
|
||||||
|
const serialized = JSON.stringify(eventStore.listAfter("ses_1").events);
|
||||||
|
expect(serialized).toContain("model.validation_failed");
|
||||||
|
expect(serialized).not.toContain("sk-super-secret");
|
||||||
|
expect(serialized).not.toContain("provider leaked");
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -0,0 +1,20 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import { createTestApp } from "./test-app.js";
|
||||||
|
|
||||||
|
describe("report download API", () => {
|
||||||
|
it("requires a known Session and serves only a fixed report format with sandbox and no-store headers", async () => {
|
||||||
|
const { app } = createTestApp({
|
||||||
|
readReport: (_session, _report, format) => Promise.resolve({ bytes: Buffer.from("<!doctype html><p>report</p>"), contentType: format === "html" ? "text/html" : "application/json" }),
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
const response = await app.inject({ method: "GET", url: "/api/sessions/ses_1/reports/rpt_test/html" });
|
||||||
|
expect(response.statusCode).toBe(200);
|
||||||
|
expect(response.body).toContain("<p>report</p>");
|
||||||
|
expect(response.headers["cache-control"]).toBe("no-store");
|
||||||
|
expect(response.headers["content-security-policy"]).toContain("sandbox");
|
||||||
|
expect(response.headers["content-disposition"]).toContain("attachment");
|
||||||
|
expect((await app.inject({ method: "GET", url: "/api/sessions/ses_missing/reports/rpt_test/html" })).statusCode).toBe(404);
|
||||||
|
expect((await app.inject({ method: "GET", url: "/api/sessions/ses_1/reports/rpt_test/env" })).statusCode).toBe(400);
|
||||||
|
} finally { await app.close(); }
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -1,9 +1,9 @@
|
|||||||
import { HarnessError } from "@agent-studio/harness";
|
import { HarnessError, InMemoryEventStore } from "@agent-studio/harness";
|
||||||
import type { FastifyInstance } from "fastify";
|
import type { FastifyInstance } from "fastify";
|
||||||
import { vi } from "vitest";
|
import { vi } from "vitest";
|
||||||
import { buildApp } from "../src/app.js";
|
import { buildApp, type AppDependencies } from "../src/app.js";
|
||||||
|
|
||||||
export function createTestApp(): {
|
export function createTestApp(options: { editorForSession?:AppDependencies["editorForSession"]; readReport?: AppDependencies["readReport"]; readEditorFrame?: AppDependencies["readEditorFrame"]; cmsForSession?: AppDependencies["cmsForSession"] } = {}): {
|
||||||
app: FastifyInstance;
|
app: FastifyInstance;
|
||||||
session: { startRun: ReturnType<typeof vi.fn>; cancelRun: ReturnType<typeof vi.fn> };
|
session: { startRun: ReturnType<typeof vi.fn>; cancelRun: ReturnType<typeof vi.fn> };
|
||||||
modelStore: {
|
modelStore: {
|
||||||
@@ -14,6 +14,7 @@ export function createTestApp(): {
|
|||||||
};
|
};
|
||||||
attachmentStore: { add: ReturnType<typeof vi.fn>; remove: ReturnType<typeof vi.fn> };
|
attachmentStore: { add: ReturnType<typeof vi.fn>; remove: ReturnType<typeof vi.fn> };
|
||||||
approvalBroker: { approve: ReturnType<typeof vi.fn>; deny: ReturnType<typeof vi.fn> };
|
approvalBroker: { approve: ReturnType<typeof vi.fn>; deny: ReturnType<typeof vi.fn> };
|
||||||
|
eventStore: InMemoryEventStore;
|
||||||
} {
|
} {
|
||||||
const session = {
|
const session = {
|
||||||
sessionId: "ses_1",
|
sessionId: "ses_1",
|
||||||
@@ -57,15 +58,21 @@ export function createTestApp(): {
|
|||||||
approve: vi.fn((approvalId: string) => ({ approvalId, status: "approved" as const })),
|
approve: vi.fn((approvalId: string) => ({ approvalId, status: "approved" as const })),
|
||||||
deny: vi.fn((approvalId: string) => ({ approvalId, status: "denied" as const })),
|
deny: vi.fn((approvalId: string) => ({ approvalId, status: "denied" as const })),
|
||||||
};
|
};
|
||||||
|
const eventStore = new InMemoryEventStore();
|
||||||
const app = buildApp({
|
const app = buildApp({
|
||||||
|
...(options.editorForSession===undefined?{}:{editorForSession:options.editorForSession}),
|
||||||
|
...(options.readEditorFrame === undefined ? {} : { readEditorFrame: options.readEditorFrame }),
|
||||||
|
...(options.readReport === undefined ? {} : { readReport: options.readReport }),
|
||||||
|
...(options.cmsForSession === undefined ? {} : { cmsForSession: options.cmsForSession }),
|
||||||
sessionRegistry: registry,
|
sessionRegistry: registry,
|
||||||
skillCatalog: { list: () => [{ name: "ads-analysis", description: "Analyze", source: "project", filePath: ".agents/skills/ads-analysis/SKILL.md", diagnostics: [] }] },
|
skillCatalog: { list: () => [{ name: "ads-analysis", description: "Analyze", source: "project", filePath: ".agents/skills/ads-analysis/SKILL.md", diagnostics: [] }] },
|
||||||
modelStore,
|
modelStore,
|
||||||
attachmentStore,
|
attachmentStore,
|
||||||
approvalBroker,
|
approvalBroker,
|
||||||
|
eventStore,
|
||||||
testModelConnection: () => Promise.resolve({ status: "deferred", message: "Saved; validation will occur on first run." }),
|
testModelConnection: () => Promise.resolve({ status: "deferred", message: "Saved; validation will occur on first run." }),
|
||||||
});
|
});
|
||||||
return { app, session, modelStore, attachmentStore, approvalBroker };
|
return { app, session, modelStore, attachmentStore, approvalBroker, eventStore };
|
||||||
}
|
}
|
||||||
|
|
||||||
export function makeSessionBusy(session: { startRun: ReturnType<typeof vi.fn> }): void {
|
export function makeSessionBusy(session: { startRun: ReturnType<typeof vi.fn> }): void {
|
||||||
|
|||||||
+28
-3
@@ -4,16 +4,19 @@ import type { ImageAttachment, ModelConnectionInput } from "@agent-studio/shared
|
|||||||
import { HarnessAttachmentAdapter } from "./assistant/attachment-adapter.js";
|
import { HarnessAttachmentAdapter } from "./assistant/attachment-adapter.js";
|
||||||
import { useHarnessRuntime, type HarnessRuntimeStore } from "./assistant/use-harness-runtime.js";
|
import { useHarnessRuntime, type HarnessRuntimeStore } from "./assistant/use-harness-runtime.js";
|
||||||
import { Thread } from "./components/chat/Thread.js";
|
import { Thread } from "./components/chat/Thread.js";
|
||||||
import { AppShell } from "./components/layout/AppShell.js";
|
import { AppShell, type WorkspaceTab } from "./components/layout/AppShell.js";
|
||||||
import { SkillsPanel } from "./components/skills/SkillsPanel.js";
|
import { SkillsPanel } from "./components/skills/SkillsPanel.js";
|
||||||
|
import { CmsPanel } from "./components/cms/CmsPanel.js";
|
||||||
import { EventTimeline } from "./components/timeline/EventTimeline.js";
|
import { EventTimeline } from "./components/timeline/EventTimeline.js";
|
||||||
import { ApprovalDialog } from "./components/approvals/ApprovalDialog.js";
|
import { ApprovalDialog } from "./components/approvals/ApprovalDialog.js";
|
||||||
import { ModelSettings, type ModelView } from "./components/settings/ModelSettings.js";
|
import { ModelSettings, type ModelView } from "./components/settings/ModelSettings.js";
|
||||||
import { CompletionAnnouncer } from "./components/common/CompletionAnnouncer.js";
|
import { CompletionAnnouncer } from "./components/common/CompletionAnnouncer.js";
|
||||||
import { HarnessClientStore } from "./store/harness-client-store.js";
|
import { HarnessClientStore } from "./store/harness-client-store.js";
|
||||||
|
import { ImageVideoPanel } from "./components/tools/ImageVideoPanel.js";
|
||||||
|
|
||||||
type AppStore = HarnessRuntimeStore & {
|
type AppStore = HarnessRuntimeStore & {
|
||||||
start(): Promise<void>;
|
start(): Promise<void>;
|
||||||
|
dispose?: () => Promise<void>;
|
||||||
getModelConfig(): Promise<{ modelConfig: ModelView }>;
|
getModelConfig(): Promise<{ modelConfig: ModelView }>;
|
||||||
putModelConfig?: (input: ModelConnectionInput) => Promise<{ modelConfig: ModelView }>;
|
putModelConfig?: (input: ModelConnectionInput) => Promise<{ modelConfig: ModelView }>;
|
||||||
testModelConfig?: () => Promise<{ status: "ok" | "deferred"; message: string }>;
|
testModelConfig?: () => Promise<{ status: "ok" | "deferred"; message: string }>;
|
||||||
@@ -34,6 +37,7 @@ export function App({ store: suppliedStore, autoStart = true, modelConfigured: s
|
|||||||
const [ownedStore] = useState(() => new HarnessClientStore());
|
const [ownedStore] = useState(() => new HarnessClientStore());
|
||||||
const store = suppliedStore ?? ownedStore;
|
const store = suppliedStore ?? ownedStore;
|
||||||
const [settingsOpen, setSettingsOpen] = useState(false);
|
const [settingsOpen, setSettingsOpen] = useState(false);
|
||||||
|
const [activeTab, setActiveTab] = useState<WorkspaceTab>("chat");
|
||||||
const snapshot = useSyncExternalStore(
|
const snapshot = useSyncExternalStore(
|
||||||
(subscriber) => store.subscribe(subscriber),
|
(subscriber) => store.subscribe(subscriber),
|
||||||
() => store.getSnapshot(),
|
() => store.getSnapshot(),
|
||||||
@@ -81,15 +85,35 @@ export function App({ store: suppliedStore, autoStart = true, modelConfigured: s
|
|||||||
return (
|
return (
|
||||||
<AssistantRuntimeProvider runtime={runtime}>
|
<AssistantRuntimeProvider runtime={runtime}>
|
||||||
<AppShell
|
<AppShell
|
||||||
|
activeTab={activeTab}
|
||||||
|
onTabChange={setActiveTab}
|
||||||
topBar={{
|
topBar={{
|
||||||
...(snapshot.sessionId === undefined ? {} : { sessionId: snapshot.sessionId }),
|
...(snapshot.sessionId === undefined ? {} : { sessionId: snapshot.sessionId }),
|
||||||
connection: snapshot.connection,
|
connection: snapshot.connection,
|
||||||
modelLabel: model.modelId ?? "No model",
|
modelLabel: model.modelId ?? "No model",
|
||||||
supportsImages: model.supportsImages ?? false,
|
supportsImages: model.supportsImages ?? false,
|
||||||
onOpenSettings: () => setSettingsOpen(true),
|
onOpenSettings: () => setSettingsOpen(true),
|
||||||
onReset: () => window.location.reload(),
|
onReset: () => {
|
||||||
|
if (store.dispose === undefined) {
|
||||||
|
window.location.reload();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
void store.dispose()
|
||||||
|
.then(() => store.start())
|
||||||
|
.then(() => store.getModelConfig())
|
||||||
|
.then((response) => setModel(response.modelConfig))
|
||||||
|
.catch(() => undefined);
|
||||||
|
},
|
||||||
}}
|
}}
|
||||||
skills={<SkillsPanel skills={snapshot.skills} />}
|
skills={<><ImageVideoPanel disabled={busy || approvalPending} {...(snapshot.sessionId && snapshot.skills.some(skill => skill.name === "swads-image-to-video") ? { sessionId: snapshot.sessionId } : {})} onDraft={task => { runtime.thread.composer.setText(`请使用 swads-image-to-video,先读取完整 Skill 和 references。本地冻结任务已创建 taskId=${task.id},标题=${task.title},精确提示词=${task.prompt},目标=${String(task.durationSeconds)} 秒。不要重复创建任务,不使用 CMS。${task.state === "awaiting_submission" ? "参考图已上传入库,本任务已接续到当前会话;不得重新上传。先用 i2v_status 核对状态,再申请 i2v_submit 独立生成审批。" : "先申请 i2v_upload_reference 独立图片外传审批;参考图入库就绪后再申请 i2v_submit 独立生成审批。"}不得改变提示词或时长,结果未知禁止重投。`); setActiveTab("chat"); }} /><SkillsPanel skills={snapshot.skills} disabled={busy || !model.configured} onUse={(name) => { runtime.thread.composer.setText(
|
||||||
|
name === "drama-minis-diagnosis"
|
||||||
|
? "请使用 drama-minis-diagnosis Skill 生成 Drama Minis 日报。先读取 Skill,向我确认分析周期、对比周期和账户范围,确认前不要查询经营数据。"
|
||||||
|
: name === "swads-short-drama-editor"
|
||||||
|
? "请使用 swads-short-drama-editor。先读取完整 Skill 和 CMS source contract,再检查 cms_connection_status,说明实际可用能力,并询问我要剪哪部剧。不要在聊天索要密码,不要在能力未就绪时声称视频已生成。"
|
||||||
|
: `请使用 ${name} Skill,先读取指令并告诉我需要补充哪些任务信息。`,
|
||||||
|
); setActiveTab("chat"); }} />{snapshot.sessionId === undefined ? null : <CmsPanel key={snapshot.sessionId} sessionId={snapshot.sessionId} busy={busy} onSelect={(drama) => {
|
||||||
|
runtime.thread.composer.setText(`请使用 swads-short-drama-editor 分析这部授权剧:${drama.title},精确 drama_id=${drama.id}。先读取 Skill 和参考文件,获取完整目录前申请审批。不要在缺少画面/参考证据时生成诊断 PASS 或渲染视频。`); setActiveTab("chat");
|
||||||
|
}} />}</>}
|
||||||
chat={<Thread
|
chat={<Thread
|
||||||
runtime={runtime}
|
runtime={runtime}
|
||||||
busy={busy}
|
busy={busy}
|
||||||
@@ -103,6 +127,7 @@ export function App({ store: suppliedStore, autoStart = true, modelConfigured: s
|
|||||||
/>
|
/>
|
||||||
{pendingApproval !== undefined ? (
|
{pendingApproval !== undefined ? (
|
||||||
<ApprovalDialog
|
<ApprovalDialog
|
||||||
|
key={pendingApproval.approvalId}
|
||||||
request={pendingApproval}
|
request={pendingApproval}
|
||||||
onApprove={(approvalId) => store.approve?.(approvalId) ?? Promise.reject(new Error("Approval API unavailable"))}
|
onApprove={(approvalId) => store.approve?.(approvalId) ?? Promise.reject(new Error("Approval API unavailable"))}
|
||||||
onDeny={(approvalId) => store.deny?.(approvalId) ?? Promise.reject(new Error("Approval API unavailable"))}
|
onDeny={(approvalId) => store.deny?.(approvalId) ?? Promise.reject(new Error("Approval API unavailable"))}
|
||||||
|
|||||||
@@ -87,7 +87,7 @@ export class HarnessApi {
|
|||||||
const { body, ...init } = options;
|
const { body, ...init } = options;
|
||||||
const isForm = body instanceof FormData;
|
const isForm = body instanceof FormData;
|
||||||
const headers = new Headers(init.headers);
|
const headers = new Headers(init.headers);
|
||||||
if (!isForm) headers.set("Content-Type", "application/json");
|
if (body !== undefined && !isForm) headers.set("Content-Type", "application/json");
|
||||||
const response = await fetch(`${this.baseUrl}${path}`, {
|
const response = await fetch(`${this.baseUrl}${path}`, {
|
||||||
...init,
|
...init,
|
||||||
headers,
|
headers,
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
import { ShieldAlert } from "lucide-react";
|
import { ShieldAlert } from "lucide-react";
|
||||||
import { useEffect, useRef, useState } from "react";
|
import { useEffect, useRef, useState } from "react";
|
||||||
|
import { ImageVideoApprovalDetails } from "../tools/ImageVideoPanel.js";
|
||||||
import type { ClientApproval } from "../../store/reducer.js";
|
import type { ClientApproval } from "../../store/reducer.js";
|
||||||
|
|
||||||
export function ApprovalDialog({ request, onApprove, onDeny }: {
|
export function ApprovalDialog({ request, onApprove, onDeny }: {
|
||||||
@@ -10,6 +11,17 @@ export function ApprovalDialog({ request, onApprove, onDeny }: {
|
|||||||
const [submitting, setSubmitting] = useState(false);
|
const [submitting, setSubmitting] = useState(false);
|
||||||
const approveRef = useRef<HTMLButtonElement>(null);
|
const approveRef = useRef<HTMLButtonElement>(null);
|
||||||
const pending = request.status === "pending";
|
const pending = request.status === "pending";
|
||||||
|
const query = request.toolName === "metrics_semantic_query";
|
||||||
|
const minis = request.toolName === "generate_minis_report";
|
||||||
|
const cms = ["cms_list_dramas", "cms_drama_inventory", "cms_read_subtitle"].includes(request.toolName);
|
||||||
|
const prepare = request.toolName === "prepare_drama_sources";
|
||||||
|
const frames = request.toolName === "read_drama_frames";
|
||||||
|
const render = request.toolName === "render_editor_preview";
|
||||||
|
const i2vUpload = request.toolName === "i2v_upload_reference";
|
||||||
|
const i2vSubmit = request.toolName === "i2v_submit";
|
||||||
|
const i2v = i2vUpload || i2vSubmit;
|
||||||
|
const [taskReady, setTaskReady] = useState(false);
|
||||||
|
const [confirmed, setConfirmed] = useState(false);
|
||||||
const [remaining, setRemaining] = useState(() => secondsRemaining(request.expiresAt));
|
const [remaining, setRemaining] = useState(() => secondsRemaining(request.expiresAt));
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
@@ -21,7 +33,7 @@ export function ApprovalDialog({ request, onApprove, onDeny }: {
|
|||||||
}, [request.expiresAt]);
|
}, [request.expiresAt]);
|
||||||
|
|
||||||
const decide = async (decision: "approve" | "deny") => {
|
const decide = async (decision: "approve" | "deny") => {
|
||||||
if (submitting || !pending) return;
|
if (submitting || !pending || (decision === "approve" && i2v && (!taskReady || !confirmed))) return;
|
||||||
setSubmitting(true);
|
setSubmitting(true);
|
||||||
try {
|
try {
|
||||||
await (decision === "approve" ? onApprove(request.approvalId) : onDeny(request.approvalId));
|
await (decision === "approve" ? onApprove(request.approvalId) : onDeny(request.approvalId));
|
||||||
@@ -41,17 +53,19 @@ export function ApprovalDialog({ request, onApprove, onDeny }: {
|
|||||||
>
|
>
|
||||||
<span className="dialog-icon dialog-icon--warning"><ShieldAlert size={22} aria-hidden="true" /></span>
|
<span className="dialog-icon dialog-icon--warning"><ShieldAlert size={22} aria-hidden="true" /></span>
|
||||||
<span className="eyebrow">SERVER APPROVAL</span>
|
<span className="eyebrow">SERVER APPROVAL</span>
|
||||||
<h2 id="approval-title">允许写入报告草稿吗?</h2>
|
<h2 id="approval-title">{i2vUpload ? "允许将这张参考图上传到 SW Ads 吗?" : i2vSubmit ? "允许提交本次 H3 生成吗?" : render ? "允许生成 10 秒 REVIEW 预览吗?" : prepare ? "允许下载所选分集并在本机抽帧吗?" : frames ? "允许将字幕和代表帧发送给当前模型吗?" : cms ? "允许读取 CMS 内容并交给模型吗?" : query ? "允许查询真实经营数据吗?" : minis ? "允许生成 HTML / PDF 报告吗?" : "允许写入报告草稿吗?"}</h2>
|
||||||
<p>Agent 请求执行 <code>{request.toolName}</code>。批准后只会写入当前训练会话的固定报告文件。</p>
|
<p>Agent 请求执行 <code>{request.toolName}</code>。{i2vUpload ? "将本次冻结参考图发送到 SW Ads 存储,并创建 review_pending、未分配账户的参考素材记录。" : i2vSubmit ? "将下方冻结图片和提示词提交到共享 H3 队列。费用未知,目标时长支持尚待实际验证;不自动换模型或重投。" : render ? "仅在本机运行原 Skill 渲染和技术 QA,使用已审核版本的授权片源及原声,生成可播放下载的 REVIEW MP4。不会发布、上传或替代最终带声音审片。" : prepare ? "仅将所选分集下载到本机并抽帧,每次最多 5 集 / 1 GiB。不会修改 CMS;此步仅返回元数据,不向模型发送字幕或图片。" : frames ? "将此任务的英文字幕和每集 3 张代表帧发送给当前配置的模型网关。代表帧分析不能替代完整带声音审片。请核对任务范围。" : cms ? "本次剧集简介、目录或字幕可能包含未公开内容,将发送给当前配置的模型网关。请核对剧集和读取范围;不会修改 CMS。" : query ? "请核对日期、账户范围和指标。查询结果将发送给当前配置的模型网关进行分析,不会更改广告投放。" : "批准后只会写入当前训练会话的报告文件。"}</p>
|
||||||
|
{i2v ? <><ImageVideoApprovalDetails taskId={(request.arguments as { taskId?: unknown } | undefined)?.taskId} onReady={setTaskReady} /><label><input type="checkbox" checked={confirmed} disabled={!taskReady || submitting} onChange={event => setConfirmed(event.currentTarget.checked)} />我已核对本次图片、提示词、时长和上述副作用</label></> : null}
|
||||||
|
{i2v || query || minis || cms || prepare || frames || render ? <details open={!minis}><summary>{minis ? "报告数据与来源" : "本次执行参数"}</summary><pre className="approval-arguments">{JSON.stringify(request.arguments, null, 2)}</pre></details> : null}
|
||||||
<dl className="approval-details">
|
<dl className="approval-details">
|
||||||
<div><dt>风险</dt><dd>{riskLabel(request.riskLevel)}</dd></div>
|
<div><dt>风险</dt><dd>{riskLabel(request.riskLevel)}</dd></div>
|
||||||
<div><dt>目标</dt><dd>report.md</dd></div>
|
<div><dt>目标</dt><dd>{i2vUpload ? "本机参考图 → SW Ads 存储(待审核、未归属)" : i2vSubmit ? "本次参考图与提示词 → H3 ref2va 生成" : render ? "已审核片源 → 本机 REVIEW MP4" : prepare ? "CMS CDN → 本机媒体目录" : frames ? "本机授权素材 → 当前模型网关" : cms ? "SWAVE CMS → 当前模型网关" : query ? "swads MCP → 当前模型网关" : minis ? "report.html / report.pdf / report.json" : "report.md"}</dd></div>
|
||||||
<div><dt>审批 ID</dt><dd><code>{request.approvalId}</code></dd></div>
|
<div><dt>审批 ID</dt><dd><code>{request.approvalId}</code></dd></div>
|
||||||
<div><dt>剩余时间</dt><dd>{remaining > 0 ? `${String(remaining)} 秒` : "等待服务器确认"}</dd></div>
|
<div><dt>剩余时间</dt><dd>{remaining > 0 ? `${String(remaining)} 秒` : "等待服务器确认"}</dd></div>
|
||||||
</dl>
|
</dl>
|
||||||
<div className="dialog-actions">
|
<div className="dialog-actions">
|
||||||
<button type="button" className="secondary-button" disabled={submitting || !pending} onClick={() => void decide("deny")}>拒绝</button>
|
<button type="button" className="secondary-button" disabled={submitting || !pending} onClick={() => void decide("deny")}>拒绝</button>
|
||||||
<button ref={approveRef} type="button" className="send-button" disabled={submitting || !pending} onClick={() => void decide("approve")}>批准并保存</button>
|
<button ref={approveRef} type="button" className="send-button" disabled={submitting || !pending || (i2v && (!taskReady || !confirmed))} onClick={() => void decide("approve")}>{i2vUpload ? "批准参考图上传" : i2vSubmit ? "批准 H3 生成" : render ? "批准本次渲染" : prepare ? "批准本次下载" : frames ? "批准本次分析" : query || cms ? "批准本次查询" : minis ? "批准生成报告" : "批准并保存"}</button>
|
||||||
</div>
|
</div>
|
||||||
<p className="dialog-note">提交后等待服务器事件确认,重复点击不会重复执行。</p>
|
<p className="dialog-note">提交后等待服务器事件确认,重复点击不会重复执行。</p>
|
||||||
</section>
|
</section>
|
||||||
|
|||||||
@@ -0,0 +1,66 @@
|
|||||||
|
import { useState } from "react";
|
||||||
|
|
||||||
|
type Connection = { configured: boolean; connected: boolean; expiresAt?: string };
|
||||||
|
type Drama = { id: string; title: string; language: string; total_episodes: number };
|
||||||
|
type ApiResult = { connection?: Connection; dramas?: Drama[] };
|
||||||
|
|
||||||
|
export function CmsPanel({ sessionId, busy, onSelect }: { sessionId: string; busy: boolean; onSelect: (drama: Drama) => void }) {
|
||||||
|
const [open, setOpen] = useState(false);
|
||||||
|
const [connection, setConnection] = useState<Connection>();
|
||||||
|
const [pending, setPending] = useState(false);
|
||||||
|
const [email, setEmail] = useState("");
|
||||||
|
const [password, setPassword] = useState("");
|
||||||
|
const [error, setError] = useState("");
|
||||||
|
const [query, setQuery] = useState("");
|
||||||
|
const [offset, setOffset] = useState(0);
|
||||||
|
const [dramas, setDramas] = useState<Drama[]>([]);
|
||||||
|
const base = `/api/sessions/${encodeURIComponent(sessionId)}/cms`;
|
||||||
|
const request = async (path: string, method = "GET", body?: unknown): Promise<ApiResult> => {
|
||||||
|
const response = await fetch(`${base}${path}`, { method, cache: "no-store", headers: { "Content-Type": "application/json" }, ...(body === undefined ? {} : { body: JSON.stringify(body) }) });
|
||||||
|
if (!response.ok) {
|
||||||
|
if (path === "/connection" && method === "POST") throw new Error("CMS 登录失败:请检查账户、密码和管理员权限;若启用了验证码需先完成相应认证。");
|
||||||
|
throw new Error("CMS 请求失败,请检查连接是否过期并重新登录。");
|
||||||
|
}
|
||||||
|
return await response.json() as ApiResult;
|
||||||
|
};
|
||||||
|
const act = async (operation: () => Promise<void>) => {
|
||||||
|
if (pending) return;
|
||||||
|
setPending(true); setError("");
|
||||||
|
try { await operation(); } catch (failure) { setError(failure instanceof Error ? failure.message : "CMS 请求失败"); }
|
||||||
|
finally { setPending(false); }
|
||||||
|
};
|
||||||
|
const loadDramas = async (page: number) => {
|
||||||
|
const result = await request(`/dramas?q=${encodeURIComponent(query)}&offset=${String(page)}`);
|
||||||
|
setDramas(result.dramas ?? []); setOffset(page);
|
||||||
|
};
|
||||||
|
return <section className="cms-panel" aria-label="CMS 素材连接">
|
||||||
|
<button className="secondary-button" type="button" aria-expanded={open} onClick={() => {
|
||||||
|
setPassword(""); setOpen(!open);
|
||||||
|
if (!open) void act(async () => { setConnection((await request("/connection")).connection); });
|
||||||
|
}}>CMS 连接与选剧</button>
|
||||||
|
{open ? <div className="cms-panel__body">
|
||||||
|
<p>使用 SWAVE 的 Supabase 管理员账户。密码通过本机后端发送至 shorts-sg 登录服务,不进入聊天或浏览器存储。</p>
|
||||||
|
<p>已提供选段审核和 10 秒 REVIEW 预览工具;需先授权素材分析、通过原始校验并核实参考。技术预览不等于最终审片通过。</p>
|
||||||
|
{connection?.connected ? <>
|
||||||
|
<p role="status">CMS 已连接(本次会话)</p>
|
||||||
|
<button type="button" className="secondary-button" disabled={pending || busy} onClick={() => void act(async () => { setConnection((await request("/connection", "DELETE")).connection); setDramas([]); })}>断开 CMS</button>
|
||||||
|
<form onSubmit={(event) => { event.preventDefault(); void act(() => loadDramas(0)); }}>
|
||||||
|
<label>搜索剧名<input aria-label="搜索剧名" value={query} maxLength={120} onChange={(event) => setQuery(event.target.value)} /></label>
|
||||||
|
<button className="secondary-button" disabled={pending} type="submit">读取剧集目录</button>
|
||||||
|
</form>
|
||||||
|
<ul className="cms-drama-list">{dramas.map((drama) => <li key={drama.id}><strong>{drama.title}</strong><span>{drama.language} · {drama.total_episodes} 集</span><button type="button" className="secondary-button" disabled={busy || pending} onClick={() => onSelect(drama)}>选择此剧</button></li>)}</ul>
|
||||||
|
{dramas.length || offset ? <div><button type="button" disabled={pending || offset === 0} onClick={() => void act(() => loadDramas(Math.max(0, offset - 30)))}>上一页</button><button type="button" disabled={pending || dramas.length < 30} onClick={() => void act(() => loadDramas(offset + 30))}>下一页</button></div> : null}
|
||||||
|
</> : <form autoComplete="off" onSubmit={(event) => {
|
||||||
|
event.preventDefault(); if (busy || pending) return;
|
||||||
|
const enteredPassword = password; setPassword("");
|
||||||
|
void act(async () => { setConnection((await request("/connection", "POST", { email, password: enteredPassword })).connection); });
|
||||||
|
}}>
|
||||||
|
<label>CMS 邮箱<input type="email" aria-label="CMS 邮箱" value={email} required autoComplete="off" maxLength={254} onChange={(event) => setEmail(event.target.value)} /></label>
|
||||||
|
<label>CMS 密码<input type="password" aria-label="CMS 密码" value={password} required autoComplete="new-password" maxLength={1024} onChange={(event) => setPassword(event.target.value)} /></label>
|
||||||
|
<button className="send-button" type="submit" disabled={pending || busy || connection?.configured === false}>{pending ? "连接中…" : "登录 CMS"}</button>
|
||||||
|
{connection?.configured === false ? <p>后端尚未配置 CMS publishable key。</p> : null}
|
||||||
|
</form>}
|
||||||
|
{error ? <p role="alert">{error}</p> : null}
|
||||||
|
</div> : null}
|
||||||
|
</section>;
|
||||||
|
}
|
||||||
@@ -2,10 +2,9 @@ import { Activity, MessageSquare, PanelsTopLeft } from "lucide-react";
|
|||||||
import { useEffect, useState, type KeyboardEvent, type ReactNode } from "react";
|
import { useEffect, useState, type KeyboardEvent, type ReactNode } from "react";
|
||||||
import { TopBar, type TopBarProps } from "../topbar/TopBar.js";
|
import { TopBar, type TopBarProps } from "../topbar/TopBar.js";
|
||||||
|
|
||||||
type WorkspaceTab = "chat" | "skills" | "events";
|
export type WorkspaceTab = "chat" | "skills" | "events";
|
||||||
|
|
||||||
export function AppShell({ topBar, skills, chat, timeline }: { topBar: TopBarProps; skills: ReactNode; chat: ReactNode; timeline: ReactNode }) {
|
export function AppShell({ topBar, skills, chat, timeline, activeTab, onTabChange: setActiveTab }: { topBar: TopBarProps; skills: ReactNode; chat: ReactNode; timeline: ReactNode; activeTab: WorkspaceTab; onTabChange: (tab: WorkspaceTab) => void }) {
|
||||||
const [activeTab, setActiveTab] = useState<WorkspaceTab>("chat");
|
|
||||||
const [mobile, setMobile] = useState(false);
|
const [mobile, setMobile] = useState(false);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
|
|||||||
@@ -83,17 +83,17 @@ export function ModelSettings({ open, busy, current, onClose, onSave, onTest, on
|
|||||||
<button className="icon-button" type="button" aria-label="关闭模型设置" onClick={onClose}><X size={19} /></button>
|
<button className="icon-button" type="button" aria-label="关闭模型设置" onClick={onClose}><X size={19} /></button>
|
||||||
</div>
|
</div>
|
||||||
<p className="settings-intro">密钥仅发送到本机服务并保存在当前会话内存中,不会写入浏览器存储。</p>
|
<p className="settings-intro">密钥仅发送到本机服务并保存在当前会话内存中,不会写入浏览器存储。</p>
|
||||||
<form className="settings-form" onSubmit={(event) => void submit(event)}>
|
<form className="settings-form" autoComplete="off" onSubmit={(event) => void submit(event)}>
|
||||||
<label>服务商<select ref={firstInput} value={providerId} onChange={(event) => {
|
<label>服务商<select ref={firstInput} value={providerId} onChange={(event) => {
|
||||||
const value = event.target.value;
|
const value = event.target.value;
|
||||||
setProviderId(value);
|
setProviderId(value);
|
||||||
setApi(value === "anthropic" ? "anthropic-messages" : "openai-responses");
|
setApi(value === "anthropic" ? "anthropic-messages" : "openai-responses");
|
||||||
}}><option value="openai">OpenAI</option><option value="anthropic">Anthropic</option><option value="compatible">OpenAI Compatible</option></select></label>
|
}}><option value="openai">OpenAI</option><option value="anthropic">Anthropic</option><option value="compatible">OpenAI Compatible</option></select></label>
|
||||||
<label>API 类型<select value={api} onChange={(event) => setApi(event.target.value as ModelApi)}><option value="openai-responses">OpenAI Responses</option><option value="openai-completions">OpenAI Completions</option><option value="anthropic-messages">Anthropic Messages</option></select></label>
|
<label>API 类型<select value={api} onChange={(event) => setApi(event.target.value as ModelApi)}><option value="openai-responses">OpenAI Responses</option><option value="openai-completions">OpenAI Completions</option><option value="anthropic-messages">Anthropic Messages</option></select></label>
|
||||||
<label className="settings-form__wide">Base URL(可选)<input type="url" value={baseUrl} placeholder="https://api.example.com/v1" onChange={(event) => setBaseUrl(event.target.value)} /></label>
|
<label className="settings-form__wide">Base URL(可选)<input name="agent-base-url" autoComplete="off" type="url" value={baseUrl} placeholder="https://api.example.com/v1" onChange={(event) => setBaseUrl(event.target.value)} /></label>
|
||||||
<label>模型 ID<input aria-label="模型 ID" required value={modelId} onChange={(event) => setModelId(event.target.value)} /></label>
|
<label>模型 ID<input name="agent-model-id" autoComplete="off" aria-label="模型 ID" required value={modelId} onChange={(event) => setModelId(event.target.value)} /></label>
|
||||||
<label>图片能力<select value={supportsImages ? "yes" : "no"} onChange={(event) => setSupportsImages(event.target.value === "yes")}><option value="yes">支持图片</option><option value="no">仅文本</option></select></label>
|
<label>图片能力<select value={supportsImages ? "yes" : "no"} onChange={(event) => setSupportsImages(event.target.value === "yes")}><option value="yes">支持图片</option><option value="no">仅文本</option></select></label>
|
||||||
<label className="settings-form__wide">API Key<div className="password-field"><KeyRound size={17} /><input aria-label="API Key" type={reveal ? "text" : "password"} value={apiKey} autoComplete="off" onChange={(event) => setApiKey(event.target.value)} /><button type="button" aria-label={reveal ? "隐藏 API Key" : "显示 API Key"} onClick={() => setReveal((value) => !value)}>{reveal ? <EyeOff size={17} /> : <Eye size={17} />}</button></div></label>
|
<label className="settings-form__wide">API Key<div className="password-field"><KeyRound size={17} /><input name="agent-api-key" aria-label="API Key" type={reveal ? "text" : "password"} value={apiKey} autoComplete="new-password" onChange={(event) => setApiKey(event.target.value)} /><button type="button" aria-label={reveal ? "隐藏 API Key" : "显示 API Key"} onClick={() => setReveal((value) => !value)}>{reveal ? <EyeOff size={17} /> : <Eye size={17} />}</button></div></label>
|
||||||
{saved?.keyHint === undefined ? null : <p className="key-hint">服务器中的密钥:<strong>{saved.keyHint}</strong></p>}
|
{saved?.keyHint === undefined ? null : <p className="key-hint">服务器中的密钥:<strong>{saved.keyHint}</strong></p>}
|
||||||
{message === "" ? null : <p className="settings-message" role="status">{message}</p>}
|
{message === "" ? null : <p className="settings-message" role="status">{message}</p>}
|
||||||
{busy ? <p className="settings-message settings-message--warning">运行期间不能更改模型配置</p> : null}
|
{busy ? <p className="settings-message settings-message--warning">运行期间不能更改模型配置</p> : null}
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
import { AlertCircle, BookOpen, Check, LoaderCircle } from "lucide-react";
|
import { AlertCircle, BookOpen, Check, LoaderCircle } from "lucide-react";
|
||||||
import type { ClientSkill } from "../../store/reducer.js";
|
import type { ClientSkill } from "../../store/reducer.js";
|
||||||
|
|
||||||
export function SkillsPanel({ skills }: { skills: ClientSkill[] }) {
|
export function SkillsPanel({ skills, onUse, disabled = false }: { skills: ClientSkill[]; onUse?: (name: string) => void; disabled?: boolean }) {
|
||||||
return (
|
return (
|
||||||
<section className="side-panel skills-panel" aria-label="Skills Catalog">
|
<section className="side-panel skills-panel" aria-label="Skills Catalog">
|
||||||
<div className="panel-heading">
|
<div className="panel-heading">
|
||||||
@@ -24,6 +24,7 @@ export function SkillsPanel({ skills }: { skills: ClientSkill[] }) {
|
|||||||
{skill.activity === "loaded" ? <Check size={15} aria-label="已加载" /> : null}
|
{skill.activity === "loaded" ? <Check size={15} aria-label="已加载" /> : null}
|
||||||
</div>
|
</div>
|
||||||
<p>{skill.description}</p>
|
<p>{skill.description}</p>
|
||||||
|
{onUse === undefined ? null : <button type="button" className="secondary-button" aria-label={`使用 ${skill.name}`} disabled={disabled} onClick={() => onUse(skill.name)}>使用此 Skill</button>}
|
||||||
<div className="skill-meta"><span>{skill.source}</span><code>{skill.filePath}</code></div>
|
<div className="skill-meta"><span>{skill.source}</span><code>{skill.filePath}</code></div>
|
||||||
{skill.diagnostics.map((diagnostic, index) => (
|
{skill.diagnostics.map((diagnostic, index) => (
|
||||||
<div className="diagnostic" key={`${diagnostic.message}:${String(index)}`}>
|
<div className="diagnostic" key={`${diagnostic.message}:${String(index)}`}>
|
||||||
|
|||||||
@@ -0,0 +1,72 @@
|
|||||||
|
import {useState,type FormEvent} from "react";
|
||||||
|
|
||||||
|
type Workflow={jobId:string;selection:null|{hash:string;reviewed:boolean;segments:{id:string;episodeNo:number;start:number;end:number;reason:string;videoUrl:string;subtitles:string}[]};plan:null|{hash:string;reviewed:boolean;reference:Record<string,unknown>;artifacts:Record<string,unknown>;subtitles:string;segmentIds:string[]};preview:null|{url:string;duration:number;sha256:string;machinePass:boolean;checks:Record<string,boolean>}};
|
||||||
|
const urlPattern=/^\/api\/sessions\/ses_[\w-]+\/editor\/job_[\w-]+\/workflow$/;
|
||||||
|
export function EditorWorkflowPanel({url}:{url:string}){
|
||||||
|
const [data,setData]=useState<Workflow|null>(null);const [busy,setBusy]=useState(false);const [message,setMessage]=useState("");
|
||||||
|
if(!urlPattern.test(url))return null;
|
||||||
|
const base=url.slice(0,-"/workflow".length);
|
||||||
|
const allowedVideo=(value:string,kind:"sources"|"plans")=>typeof value==="string"&&value.startsWith(`${base}/${kind}/`)&&new RegExp(`^${kind}/[\\w-]+${kind==="plans"?"/preview":""}$`).test(value.slice(base.length+1));
|
||||||
|
const request=async(target:string,body?:unknown)=>{
|
||||||
|
setBusy(true);setMessage("");
|
||||||
|
try{
|
||||||
|
const response=await fetch(target,body===undefined?{cache:"no-store"}:{method:"POST",headers:{"Content-Type":"application/json"},body:JSON.stringify(body)});
|
||||||
|
if(!response.ok){const failure=await response.json() as {error?:{message?:unknown}};throw new Error(typeof failure.error?.message==="string"?failure.error.message:`请求未通过(${String(response.status)}),请刷新状态并核对审核记录。`);}
|
||||||
|
const next=await response.json() as Workflow;
|
||||||
|
if(typeof next.jobId!=="string"||!base.endsWith(`/editor/${next.jobId}`))throw new Error("媒体任务不匹配");
|
||||||
|
setData(next);if(body!==undefined)setMessage("审核记录已保存。请在聊天中输入“继续”,让 Agent 执行下一步。");
|
||||||
|
}catch(error){setMessage(error instanceof Error?error.message:"请求失败");}finally{setBusy(false);}
|
||||||
|
};
|
||||||
|
const selected=data?.selection;const plan=data?.plan;
|
||||||
|
const stage=selected&&!selected.reviewed?"selection":plan&&!plan.reviewed?"plan":undefined;
|
||||||
|
const submit=(event:FormEvent<HTMLFormElement>)=>{
|
||||||
|
event.preventDefault();if(!stage||!selected)return;
|
||||||
|
const fields=new FormData(event.currentTarget);const text=(name:string)=>{const value=fields.get(name);return typeof value==="string"?value:"";};
|
||||||
|
const checked=(name:string)=>fields.get(name)==="on";
|
||||||
|
const common={planHash:stage==="selection"?selected.hash:plan?.hash,reviewer:text("reviewer"),policyCheckedAt:new Date(text("policyCheckedAt")).toISOString(),pictureAudioReviewed:checked("pictureAudioReviewed"),noUnresolvedFindings:checked("noUnresolvedFindings")};
|
||||||
|
const payload=stage==="selection"?{...common,policyPageUpdated:text("policyPageUpdated"),segmentEvidence:selected.segments.map(s=>({segmentId:s.id,pictureEvidence:text(`${s.id}-picture`),audioEvidence:text(`${s.id}-audio`),subtitleEvidence:text(`${s.id}-subtitle`)}))}:{...common,referenceReviewed:checked("referenceReviewed"),storyReviewed:checked("storyReviewed"),packagingReviewed:checked("packagingReviewed"),segmentEvidence:(plan?.segmentIds??[]).map(id=>({segmentId:id,observation:text(`${id}-observation`)}))};
|
||||||
|
void request(`${base}/review/${stage}`,payload);
|
||||||
|
};
|
||||||
|
return <section className="editor-workflow" aria-label="剪辑审核与预览">
|
||||||
|
<button className="secondary-button" type="button" disabled={busy} onClick={()=>void request(url)}>{data?"刷新剪辑状态":"打开剪辑审核与预览"}</button>
|
||||||
|
{message?<p role="status">{message}</p>:null}
|
||||||
|
{data?<>
|
||||||
|
<p>分析已通过不等于成片。选段审核、方案审核和最终审片分别记录;不会上传或发布广告。</p>
|
||||||
|
{selected?<div><strong>候选源片段 · {selected.reviewed?"已有人工审核记录":"待带声音审核"}</strong>
|
||||||
|
{selected.segments.map(span=><div key={span.id} className="editor-review-segment">
|
||||||
|
<p>选段 {span.id} · 第 {span.episodeNo} 集 · {span.start}–{span.end} 秒</p><p>{span.reason}</p>
|
||||||
|
{allowedVideo(span.videoUrl,"sources")?<video aria-label={`选段 ${span.id} 原声视频`} src={`${span.videoUrl}#t=${String(span.start)},${String(span.end)}`} controls playsInline preload="metadata" onLoadedMetadata={e=>{e.currentTarget.currentTime=span.start;}} onTimeUpdate={e=>{if(e.currentTarget.currentTime>=span.end)e.currentTarget.pause();}}/>:<p>源视频地址无效,不能确认已审片。</p>}
|
||||||
|
<details><summary>该选段原始英文字幕</summary><pre>{span.subtitles||"所选区间没有字幕 cue,仍需实际听原声。"}</pre></details>
|
||||||
|
</div>)}
|
||||||
|
</div>:<p>等待 Agent 提交候选选段。</p>}
|
||||||
|
{plan?<div><strong>剪辑方案 · {plan.reviewed?"已有人工审核记录":"待核对参考、故事与字幕"}</strong>
|
||||||
|
<details><summary>本次参考证据与完整方案</summary><pre>{JSON.stringify({reference:plan.reference,artifacts:plan.artifacts,finalSubtitles:plan.subtitles},null,2)}</pre></details>
|
||||||
|
</div>:null}
|
||||||
|
{stage&&selected?<form key={stage==="selection"?selected.hash:plan?.hash} onSubmit={submit}>
|
||||||
|
<p>请亲自观看完整选段并开启声音。下列记录由审核人填写,模型不能代勾。若内容不合适,不要确认,请在聊天中要求替换完整场景。</p>
|
||||||
|
<a href="https://ads.tiktok.com/resources/help/article/tiktok-ads-policy-adult-content" target="_blank" rel="noreferrer">核对 TikTok Adult Content 官方政策</a>
|
||||||
|
<label>审核人<input name="reviewer" required maxLength={100}/></label>
|
||||||
|
<label>本次实际政策核对时间<input name="policyCheckedAt" type="datetime-local" required/></label>
|
||||||
|
{stage==="selection"?<label>政策页面显示的更新日期<input name="policyPageUpdated" required minLength={4} maxLength={100}/></label>:null}
|
||||||
|
<label><input name="pictureAudioReviewed" type="checkbox" required/>我已连续观看全部选段并听取原声,不是只看代表帧</label>
|
||||||
|
{stage==="plan"?<>
|
||||||
|
<label><input name="referenceReviewed" type="checkbox" required/>已完整观看并听过全部候选参考,核实短剧营销来源、A/B 证据及成人内容适用性;没有仅凭搜索摘要通过的候选</label>
|
||||||
|
<label><input name="storyReviewed" type="checkbox" required/>已核对所有因果、对白、动作反应和未揭晓信息,10 秒结尾完整</label>
|
||||||
|
<label><input name="packagingReviewed" type="checkbox" required/>已核对最终英文字幕;本次没有新增封面、叠字、CTA 或广告文案</label>
|
||||||
|
</>:null}
|
||||||
|
<label><input name="noUnresolvedFindings" type="checkbox" required/>本次所选内容没有未解决的成人内容疑点;不代表平台批准</label>
|
||||||
|
{(stage==="selection"?selected.segments.map(s=>s.id):plan?.segmentIds??[]).map(id=><fieldset key={id}><legend>选段 {id} 审核证据(每项至少 20 字符)</legend>
|
||||||
|
{stage==="selection"?<><label>连续画面、服装与互动观察<textarea name={`${id}-picture`} required minLength={20} maxLength={2000}/></label><label>原声对白与声音观察<textarea name={`${id}-audio`} required minLength={20} maxLength={2000}/></label><label>字幕及其结合语境<textarea name={`${id}-subtitle`} required minLength={20} maxLength={2000}/></label></>:<label>参考依据、完整因果、字幕、切点及结尾观察<textarea name={`${id}-observation`} required minLength={20} maxLength={3000}/></label>}
|
||||||
|
</fieldset>)}
|
||||||
|
<button type="submit" className="secondary-button" disabled={busy}>保存本版本的人工审核记录</button>
|
||||||
|
</form>:null}
|
||||||
|
{data.preview&&allowedVideo(data.preview.url,"plans")?<div>
|
||||||
|
<h3>REVIEW 预览 · {data.preview.duration} 秒</h3>
|
||||||
|
<video aria-label="REVIEW 视频预览" src={data.preview.url} controls playsInline preload="metadata"/>
|
||||||
|
<p>技术 QA:{data.preview.machinePass?"脚本通过":"存在未通过项目"}。尚未完成最终带声音审片,不是最终审核 PASS。</p>
|
||||||
|
<details><summary>文件校验信息</summary><pre>{JSON.stringify({sha256:data.preview.sha256,checks:data.preview.checks},null,2)}</pre></details>
|
||||||
|
<a href={`${data.preview.url}?download=1`} download>下载 REVIEW MP4</a>
|
||||||
|
</div>:null}
|
||||||
|
</>:null}
|
||||||
|
</section>;
|
||||||
|
}
|
||||||
@@ -0,0 +1,116 @@
|
|||||||
|
import { useEffect, useRef, useState } from "react";
|
||||||
|
|
||||||
|
export type ImageVideoTask = { id: string; revision?: number; title: string; prompt: string; durationSeconds: number; state: string; referenceUrl: string; videoUrl?: string; jobId?: string; error?: string; validation?: { videoSeconds?: number; width?: number; height?: number; fps?: number } };
|
||||||
|
export function isImageVideoTask(value: unknown): value is ImageVideoTask {
|
||||||
|
if (!value || typeof value !== "object") return false;
|
||||||
|
const task = value as Partial<ImageVideoTask>;
|
||||||
|
return typeof task.id === "string" && /^i2v_[0-9a-f-]{36}$/.test(task.id) && typeof task.title === "string" && typeof task.prompt === "string" && typeof task.state === "string" && typeof task.durationSeconds === "number" && task.referenceUrl === `/api/image-video/tasks/${task.id}/reference`;
|
||||||
|
}
|
||||||
|
const stateNames: Record<string, string> = { draft: "本地草稿 · 待图片上传审批", initializing_upload: "准备上传", uploading: "上传中", finalizing: "入库确认中", importing: "参考图入库中", awaiting_submission: "参考图已入库 · 待生成审批", submitting: "提交中", queued: "排队中", running: "生成中", validating: "成片技术校验中", ready: "生成预览 · 待视听检查", failed: "失败", unknown_submission: "提交状态未知 · 禁止重投", unknown_upload: "上传状态未知 · 禁止重复导入" };
|
||||||
|
|
||||||
|
export function ImageVideoTaskCard({ task }: { task: ImageVideoTask }) {
|
||||||
|
if (!isImageVideoTask(task)) return null;
|
||||||
|
const video = task.state === "ready" && task.videoUrl === `/api/image-video/tasks/${task.id}/video` ? task.videoUrl : undefined;
|
||||||
|
return <article className="i2v-task">
|
||||||
|
<img className="i2v-reference" src={task.referenceUrl} alt={`${task.title} 参考图`} loading="lazy" />
|
||||||
|
<div><h3>{task.title}</h3><p role="status">{stateNames[task.state] ?? "未知状态 · 请核实"}</p><p>{task.durationSeconds} 秒 · H3 ref2va · 9:16 · 768p</p><p>{task.prompt}</p>
|
||||||
|
{task.jobId ? <p>生成任务:<code>{task.jobId}</code></p> : null}
|
||||||
|
{task.error ? <p role="alert">{task.error}</p> : null}
|
||||||
|
{video ? <><video aria-label="生成预览播放器" src={video} controls playsInline preload="metadata" /><a className="secondary-button" href={`${video}?download=1`} download>下载 MP4</a><p>技术验证通过,待视听检查;音轨存在不能证明声音内容正确。</p></> : null}
|
||||||
|
</div>
|
||||||
|
</article>;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function jsonRequest(url: string, init?: RequestInit): Promise<Record<string, unknown>> {
|
||||||
|
const response = await fetch(url, { cache: "no-store", ...init });
|
||||||
|
if (!response.ok) throw new Error("图生视频请求失败,请检查连接或任务状态。");
|
||||||
|
return await response.json() as Record<string, unknown>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function ImageVideoPanel({ sessionId, onDraft, disabled = false }: { sessionId?: string; onDraft?: (task: ImageVideoTask) => void; disabled?: boolean }) {
|
||||||
|
const [tasks, setTasks] = useState<ImageVideoTask[]>([]); const [loading, setLoading] = useState(true); const [error, setError] = useState<string>();
|
||||||
|
const [file, setFile] = useState<File>(); const [title, setTitle] = useState(""); const [prompt, setPrompt] = useState(""); const [duration, setDuration] = useState(10); const [saving, setSaving] = useState(false);
|
||||||
|
const [reference, setReference] = useState<string>(); const mounted = useRef(true);
|
||||||
|
const [continuing, setContinuing] = useState<ImageVideoTask>(); const [confirmed, setConfirmed] = useState(false); const [resuming, setResuming] = useState(false);
|
||||||
|
const continuationEpoch = useRef(0);
|
||||||
|
useEffect(() => { continuationEpoch.current++; setContinuing(undefined); setConfirmed(false); setResuming(false); return () => { continuationEpoch.current++; }; }, [sessionId]);
|
||||||
|
useEffect(() => {
|
||||||
|
mounted.current = true; const controller = new AbortController(); let active = false;
|
||||||
|
const refresh = async () => { if (active) return; active = true; try { const data = await jsonRequest("/api/image-video/tasks", { signal: controller.signal }); if (!controller.signal.aborted) { setTasks(Array.isArray(data.tasks) ? data.tasks.filter(isImageVideoTask) : []); setError(undefined); } } catch { if (!controller.signal.aborted) setError("无法读取任务历史,请检查本机服务。"); } finally { active = false; if (!controller.signal.aborted) setLoading(false); } };
|
||||||
|
void refresh(); const timer = window.setInterval(() => void refresh(), 30_000);
|
||||||
|
return () => { mounted.current = false; controller.abort(); window.clearInterval(timer); };
|
||||||
|
}, []);
|
||||||
|
useEffect(() => { if (!file) { setReference(undefined); return; } const url = URL.createObjectURL(file); setReference(url); return () => URL.revokeObjectURL(url); }, [file]);
|
||||||
|
const create = async () => {
|
||||||
|
if (!file || !sessionId || saving || !title.trim() || !prompt.trim()) return;
|
||||||
|
setSaving(true); setError(undefined);
|
||||||
|
try {
|
||||||
|
const body = new FormData(); body.append("file", file);
|
||||||
|
const upload = await jsonRequest(`/api/sessions/${encodeURIComponent(sessionId)}/attachments`, { method: "POST", body });
|
||||||
|
const attachment = upload.attachment as { attachmentId?: unknown } | undefined;
|
||||||
|
if (typeof attachment?.attachmentId !== "string") throw new Error("附件接收失败。");
|
||||||
|
const result = await jsonRequest(`/api/sessions/${encodeURIComponent(sessionId)}/image-video/tasks`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ attachmentId: attachment.attachmentId, title, prompt, durationSeconds: duration }) });
|
||||||
|
if (!isImageVideoTask(result.task)) throw new Error("任务响应无效。");
|
||||||
|
const task = result.task;
|
||||||
|
if (mounted.current) { setTasks(current => [task, ...current]); onDraft?.(task); }
|
||||||
|
} catch (error) { if (mounted.current) setError(error instanceof Error ? error.message : "准备任务失败。"); } finally { if (mounted.current) setSaving(false); }
|
||||||
|
};
|
||||||
|
const continueTask = async () => {
|
||||||
|
if (!sessionId || !continuing || !confirmed || disabled || resuming) return;
|
||||||
|
const epoch = continuationEpoch.current;
|
||||||
|
const current = () => mounted.current && continuationEpoch.current === epoch;
|
||||||
|
setResuming(true); setError(undefined);
|
||||||
|
try {
|
||||||
|
const response = await fetch(`/api/sessions/${encodeURIComponent(sessionId)}/image-video/tasks/${continuing.id}/continue`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ confirmed: true, expectedRevision: continuing.revision }) });
|
||||||
|
const result = await response.json() as Record<string, unknown>;
|
||||||
|
if (!response.ok) {
|
||||||
|
const code = (result.error as { code?: string } | undefined)?.code;
|
||||||
|
throw new Error(code === "I2V_OWNER_ACTIVE" ? "原会话仍有效,请回原页面继续。" : code === "I2V_STALE_REVISION" ? "任务状态已更新,请取消后重新核对。" : "接续失败,请检查连接和任务状态;没有重新上传或生成。");
|
||||||
|
}
|
||||||
|
if (!isImageVideoTask(result.task) || result.task.id !== continuing.id) throw new Error("接续响应无效,请核实原任务。");
|
||||||
|
const resumed = result.task;
|
||||||
|
if (current()) { setTasks(tasks => tasks.map(task => task.id === resumed.id ? resumed : task)); setContinuing(undefined); setConfirmed(false); onDraft?.(resumed); }
|
||||||
|
} catch (error) { if (current()) setError(error instanceof Error ? error.message : "接续失败。"); }
|
||||||
|
finally { if (current()) setResuming(false); }
|
||||||
|
};
|
||||||
|
return <section className="i2v-panel" aria-label="图生视频工作区">
|
||||||
|
<h2>图片 → 有声视频</h2><p>单张参考图,H3 ref2va。无需 CMS。默认竖屏、自然速度、10 秒;成片自动校验时长,少量超长尾帧会原速裁齐。声音与画面效果需检查。</p>
|
||||||
|
{sessionId && onDraft ? <form onSubmit={event => { event.preventDefault(); void create(); }}>
|
||||||
|
<label>参考图片<input aria-label="参考图片" type="file" accept="image/png,image/jpeg,image/webp" disabled={saving} onChange={event => setFile(event.currentTarget.files?.[0])} /></label>
|
||||||
|
{reference ? <img className="i2v-reference" src={reference} alt="本机参考图片预览" /> : null}
|
||||||
|
<label>标题<input aria-label="标题" value={title} maxLength={120} disabled={saving} onChange={event => setTitle(event.currentTarget.value)} /></label>
|
||||||
|
<label>动作与声音<textarea aria-label="动作与声音" placeholder="例如:人物自然转向窗外,镜头缓慢推近,有雨声。" value={prompt} maxLength={4000} disabled={saving} onChange={event => setPrompt(event.currentTarget.value)} /></label>
|
||||||
|
<label>目标时长(秒)<input aria-label="目标时长(秒)" type="number" min="0.001" max="60" step="0.001" value={duration} disabled={saving} onChange={event => setDuration(Number(event.currentTarget.value))} /></label>
|
||||||
|
<p>准备仅保存到本机。随后在聊天中分别确认图片外传与 H3 生成。此缩略图不会传给聊天模型;H3 将在批准后接收图片。费用未知。</p>
|
||||||
|
<button className="secondary-button" disabled={saving || !file || !title.trim() || !prompt.trim() || duration <= 0 || duration > 60} type="submit">{saving ? "保存中…" : "准备本地任务"}</button>
|
||||||
|
</form> : null}
|
||||||
|
{loading ? <p role="status">正在读取持久任务…</p> : null}
|
||||||
|
{error ? <p role="alert">{error}</p> : null}
|
||||||
|
{!loading && tasks.length === 0 ? <p>尚无图生视频任务。</p> : null}
|
||||||
|
<div className="i2v-history">{tasks.map(task => <div key={task.id}>
|
||||||
|
<ImageVideoTaskCard task={task} />
|
||||||
|
{sessionId && onDraft && ["draft", "awaiting_submission"].includes(task.state) && Number.isSafeInteger(task.revision) ? <button type="button" className="secondary-button" disabled={disabled || resuming} onClick={() => { setContinuing(task); setConfirmed(false); setError(undefined); }}>继续此任务</button> : null}
|
||||||
|
{continuing?.id === task.id ? <section className="i2v-continuation" aria-label={`接续 ${task.title}`}>
|
||||||
|
<p>仅接续原任务到当前会话,保留图片、提示词、时长和上传结果。不重新上传,不自动生成;后续生成仍需单独审批。</p>
|
||||||
|
<label><input type="checkbox" checked={confirmed} disabled={resuming} onChange={event => setConfirmed(event.currentTarget.checked)} />确认将此任务接续到当前会话</label>
|
||||||
|
<button type="button" className="secondary-button" disabled={!confirmed || resuming || disabled} onClick={() => { void continueTask(); }}>{resuming ? "接续中…" : "确认接续(不生成)"}</button>
|
||||||
|
<button type="button" className="secondary-button" disabled={resuming} onClick={() => { setContinuing(undefined); setConfirmed(false); }}>取消</button>
|
||||||
|
</section> : null}
|
||||||
|
</div>)}</div>
|
||||||
|
</section>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function ImageVideoApprovalDetails({ taskId, onReady }: { taskId: unknown; onReady: (ready: boolean) => void }) {
|
||||||
|
const [task, setTask] = useState<ImageVideoTask>(); const [error, setError] = useState(false);
|
||||||
|
useEffect(() => {
|
||||||
|
const controller = new AbortController(); onReady(false); setTask(undefined); setError(false);
|
||||||
|
if (typeof taskId !== "string" || !/^i2v_[0-9a-f-]{36}$/.test(taskId)) { setError(true); return; }
|
||||||
|
void jsonRequest(`/api/image-video/tasks/${taskId}`, { signal: controller.signal }).then(result => {
|
||||||
|
if (!isImageVideoTask(result.task) || result.task.id !== taskId) throw new Error("Task mismatch");
|
||||||
|
if (!controller.signal.aborted) { setTask(result.task); onReady(true); }
|
||||||
|
}).catch(() => { if (!controller.signal.aborted) setError(true); });
|
||||||
|
return () => controller.abort();
|
||||||
|
}, [taskId, onReady]);
|
||||||
|
if (error) return <p role="alert">无法读取本次冻结参数,审批已禁用。请拒绝并核实任务。</p>;
|
||||||
|
return task ? <ImageVideoTaskCard task={task} /> : <p role="status">读取本次参考图和冻结参数…</p>;
|
||||||
|
}
|
||||||
@@ -1,7 +1,13 @@
|
|||||||
import type { ToolCallMessagePartProps } from "@assistant-ui/react";
|
import type { ToolCallMessagePartProps } from "@assistant-ui/react";
|
||||||
import { Wrench } from "lucide-react";
|
import { Wrench } from "lucide-react";
|
||||||
|
import {EditorWorkflowPanel} from "./EditorWorkflowPanel.js";
|
||||||
|
import { ImageVideoTaskCard, isImageVideoTask } from "./ImageVideoPanel.js";
|
||||||
|
|
||||||
export function ToolFallback({ toolName, status }: ToolCallMessagePartProps) {
|
export function ToolFallback({ toolName, status, result }: ToolCallMessagePartProps) {
|
||||||
|
const report = toolName === "generate_minis_report" && typeof result === "object" && result !== null ? result as { files?: unknown; warning?: unknown } : undefined;
|
||||||
|
const files = Array.isArray(report?.files) ? report.files.filter(isReportFile) : [];
|
||||||
|
const media = ["prepare_drama_sources", "editor_job_status", "read_drama_frames"].includes(toolName) ? mediaResult(result) : undefined;
|
||||||
|
const workflow=typeof result==="object"&&result!==null?result as {workflowUrl?:unknown}:undefined;
|
||||||
return (
|
return (
|
||||||
<section className="tool-card" aria-label={`${toolName} 工具`}>
|
<section className="tool-card" aria-label={`${toolName} 工具`}>
|
||||||
<div className="tool-card__head">
|
<div className="tool-card__head">
|
||||||
@@ -9,10 +15,48 @@ export function ToolFallback({ toolName, status }: ToolCallMessagePartProps) {
|
|||||||
<div><span className="eyebrow">TOOL</span><strong>{safeName(toolName)}</strong></div>
|
<div><span className="eyebrow">TOOL</span><strong>{safeName(toolName)}</strong></div>
|
||||||
<span className="tool-status">{status?.type === "complete" ? "完成" : status?.type === "incomplete" ? "失败" : "运行中"}</span>
|
<span className="tool-status">{status?.type === "complete" ? "完成" : status?.type === "incomplete" ? "失败" : "运行中"}</span>
|
||||||
</div>
|
</div>
|
||||||
|
{files.length === 0 ? null : <div className="report-downloads">{files.map((file) => <a className="secondary-button" key={file.url} href={file.url} download>下载 {file.format.toUpperCase()}</a>)}</div>}
|
||||||
|
{toolName.startsWith("i2v_") && isImageVideoTask(result) ? <ImageVideoTaskCard task={result} /> : null}
|
||||||
|
{typeof report?.warning === "string" ? <p role="status">{report.warning}</p> : null}
|
||||||
|
{typeof workflow?.workflowUrl==="string"?<EditorWorkflowPanel url={workflow.workflowUrl}/>:null}
|
||||||
|
{media ? <div>
|
||||||
|
<p><strong>{media.title}</strong> · {media.state}</p>
|
||||||
|
<code>{media.jobId}</code>
|
||||||
|
<p>仅代表帧与素材准备;尚未完成剧情验证、内容审核或最终成片。</p>
|
||||||
|
<div className="editor-frames">{media.frames.map(frame => <figure key={frame.url}>
|
||||||
|
<img src={frame.url} alt={`第 ${String(frame.episodeNo)} 集 ${String(frame.time)} 秒代表帧`} loading="lazy" />
|
||||||
|
<figcaption>第 {frame.episodeNo} 集 · {frame.time} s</figcaption>
|
||||||
|
</figure>)}</div>
|
||||||
|
</div> : null}
|
||||||
</section>
|
</section>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function mediaResult(value: unknown) {
|
||||||
|
if (typeof value !== "object" || value === null) return undefined;
|
||||||
|
const job = value as { jobId?: unknown; title?: unknown; state?: unknown; sources?: unknown };
|
||||||
|
if (typeof job.jobId !== "string" || typeof job.title !== "string" || typeof job.state !== "string" || !Array.isArray(job.sources)) return undefined;
|
||||||
|
const frames: { episodeNo: number; time: number; url: string }[] = [];
|
||||||
|
for (const value of job.sources.slice(0, 5)) {
|
||||||
|
if (typeof value !== "object" || value === null) continue;
|
||||||
|
const source = value as { episodeNo?: unknown; frames?: unknown };
|
||||||
|
if (typeof source.episodeNo !== "number" || !Array.isArray(source.frames)) continue;
|
||||||
|
for (const value of source.frames.slice(0, 3)) {
|
||||||
|
if (typeof value !== "object" || value === null) continue;
|
||||||
|
const frame = value as { time?: unknown; url?: unknown };
|
||||||
|
if (typeof frame.time !== "number" || typeof frame.url !== "string" || !/^\/api\/sessions\/ses_[\w-]+\/editor\/job_[\w-]+\/frames\/frame_[\w-]+$/.test(frame.url)) continue;
|
||||||
|
frames.push({ episodeNo: source.episodeNo, time: frame.time, url: frame.url });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return { jobId: job.jobId, title: job.title, state: job.state, frames };
|
||||||
|
}
|
||||||
|
|
||||||
|
function isReportFile(value: unknown): value is { format: string; url: string } {
|
||||||
|
if (typeof value !== "object" || value === null) return false;
|
||||||
|
const file = value as { format?: unknown; url?: unknown };
|
||||||
|
return typeof file.format === "string" && ["html", "pdf", "json"].includes(file.format) && typeof file.url === "string" && /^\/api\/sessions\/ses_[\w-]+\/reports\/rpt_[\w-]+\/(html|pdf|json)$/.test(file.url);
|
||||||
|
}
|
||||||
|
|
||||||
function safeName(name: string): string {
|
function safeName(name: string): string {
|
||||||
return name.slice(0, 80).replace(/[^a-zA-Z0-9._-]/g, "?");
|
return name.slice(0, 80).replace(/[^a-zA-Z0-9._-]/g, "?");
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -38,6 +38,7 @@ export class HarnessClientStore {
|
|||||||
public async dispose(): Promise<void> {
|
public async dispose(): Promise<void> {
|
||||||
this.#eventSource?.close();
|
this.#eventSource?.close();
|
||||||
this.#eventSource = undefined;
|
this.#eventSource = undefined;
|
||||||
|
this.#starting = undefined;
|
||||||
const sessionId = this.#state.sessionId;
|
const sessionId = this.#state.sessionId;
|
||||||
if (sessionId !== undefined) await this.api.deleteSession(sessionId);
|
if (sessionId !== undefined) await this.api.deleteSession(sessionId);
|
||||||
this.#setState(createInitialHarnessState());
|
this.#setState(createInitialHarnessState());
|
||||||
|
|||||||
@@ -154,6 +154,12 @@ export function reduceHarnessEvent(state: HarnessClientState, event: HarnessEven
|
|||||||
return updateMessageForRun(next, event.runId, { status: "failed", error: event.payload.message });
|
return updateMessageForRun(next, event.runId, { status: "failed", error: event.payload.message });
|
||||||
}
|
}
|
||||||
case "tool.requested": {
|
case "tool.requested": {
|
||||||
|
if (event.runId !== undefined && !Object.values(next.messages).some((message) => message.role === "assistant" && message.runId === event.runId)) {
|
||||||
|
next = upsertMessage(next, {
|
||||||
|
id: `msg_${event.runId}`, role: "assistant", runId: event.runId,
|
||||||
|
text: "", status: "running", attachmentIds: [],
|
||||||
|
});
|
||||||
|
}
|
||||||
const tool: ClientTool = {
|
const tool: ClientTool = {
|
||||||
toolCallId: event.payload.toolCallId,
|
toolCallId: event.payload.toolCallId,
|
||||||
name: event.payload.toolName,
|
name: event.payload.toolName,
|
||||||
|
|||||||
@@ -135,3 +135,43 @@
|
|||||||
@keyframes spin { to { transform: rotate(360deg); } }
|
@keyframes spin { to { transform: rotate(360deg); } }
|
||||||
@media (prefers-reduced-motion: reduce) { *, *::before, *::after { scroll-behavior: auto !important; animation-duration: .01ms !important; animation-iteration-count: 1 !important; transition-duration: .01ms !important; } }
|
@media (prefers-reduced-motion: reduce) { *, *::before, *::after { scroll-behavior: auto !important; animation-duration: .01ms !important; animation-iteration-count: 1 !important; transition-duration: .01ms !important; } }
|
||||||
@media (max-width: 720px) { .topbar { grid-template-columns: 1fr auto; padding: 0 12px; } .topbar__actions { grid-column: 2; } .thread-viewport { padding: 20px 16px; } .composer-area { padding: 0 10px 10px; } .tool-summary { grid-template-columns: repeat(2, 1fr); } .tool-summary > div + div { border-left: 0; } .settings-form { grid-template-columns: 1fr; } .settings-form > * { grid-column: 1; } .settings-actions { flex-wrap: wrap; } }
|
@media (max-width: 720px) { .topbar { grid-template-columns: 1fr auto; padding: 0 12px; } .topbar__actions { grid-column: 2; } .thread-viewport { padding: 20px 16px; } .composer-area { padding: 0 10px 10px; } .tool-summary { grid-template-columns: repeat(2, 1fr); } .tool-summary > div + div { border-left: 0; } .settings-form { grid-template-columns: 1fr; } .settings-form > * { grid-column: 1; } .settings-actions { flex-wrap: wrap; } }
|
||||||
|
.approval-arguments { max-height: 240px; overflow: auto; white-space: pre-wrap; overflow-wrap: anywhere; font-size: 12px; text-align: left; }
|
||||||
|
.report-downloads { display: flex; gap: 8px; flex-wrap: wrap; padding-top: 12px; }
|
||||||
|
.cms-panel { padding: 16px; border-top: 1px solid var(--border); }
|
||||||
|
.cms-panel__body { display: grid; gap: 12px; margin-top: 12px; font-size: 13px; }
|
||||||
|
.cms-panel__body form, .cms-panel__body label { display: grid; gap: 8px; }
|
||||||
|
.cms-panel__body input { min-width: 0; width: 100%; padding: 8px; border: 1px solid #cbd5e1; border-radius: 6px; }
|
||||||
|
.cms-drama-list { list-style: none; padding: 0; display: grid; gap: 12px; }
|
||||||
|
.cms-drama-list li { display: grid; gap: 6px; padding: 10px; border: 1px solid #cbd5e1; border-radius: 6px; }
|
||||||
|
.editor-frames { display: grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 8px; margin-top: 12px; }
|
||||||
|
.editor-frames figure { margin: 0; min-width: 0; }
|
||||||
|
.editor-frames img { display: block; width: 100%; max-height: 190px; object-fit: contain; background: #111; border-radius: 6px; }
|
||||||
|
.editor-frames figcaption { font-size: 11px; text-align: center; margin-top: 4px; }
|
||||||
|
.editor-workflow { margin-top: 12px; display: grid; gap: 12px; }
|
||||||
|
.editor-workflow video { display: block; width: min(100%, 270px); max-height: 480px; background: #111; border-radius: 10px; }
|
||||||
|
.editor-workflow form, .editor-workflow fieldset { display: grid; gap: 12px; min-width: 0; }
|
||||||
|
.editor-workflow label { display: grid; gap: 6px; }
|
||||||
|
.editor-workflow label:has(input[type="checkbox"]) { display: flex; align-items: flex-start; gap: 8px; }
|
||||||
|
.editor-workflow input:not([type="checkbox"]), .editor-workflow textarea { width: 100%; box-sizing: border-box; padding: 8px; border: 1px solid #ccd2dc; border-radius: 6px; background: white; color: #192233; }
|
||||||
|
.editor-workflow textarea { min-height: 72px; }
|
||||||
|
.editor-workflow pre { white-space: pre-wrap; overflow-wrap: anywhere; max-height: 360px; overflow: auto; }
|
||||||
|
.editor-review-segment { margin-block: 16px; }
|
||||||
|
.i2v-panel { box-sizing: border-box; min-width: 0; width: 100%; max-width: 100vw; padding: 18px; border-bottom: 1px solid var(--border); }
|
||||||
|
.i2v-panel h2 { font-size: 18px; margin: 0 0 10px; }
|
||||||
|
.i2v-panel p { font-size: 12px; line-height: 1.65; overflow-wrap: anywhere; }
|
||||||
|
.i2v-panel form { display: grid; grid-template-columns: minmax(0, 1fr); min-width: 0; gap: 10px; margin: 16px 0; }
|
||||||
|
.i2v-panel label { display: grid; grid-template-columns: minmax(0, 1fr); min-width: 0; gap: 5px; font-size: 12px; }
|
||||||
|
.i2v-panel input, .i2v-panel textarea { width: 100%; min-width: 0; padding: 8px; border: 1px solid #d6dbe1; border-radius: 7px; background: #fff; color: #182431; }
|
||||||
|
.i2v-panel textarea { min-height: 90px; resize: vertical; }
|
||||||
|
.i2v-reference { width: 88px; height: 116px; object-fit: contain; border-radius: 6px; background: #f0f2f4; }
|
||||||
|
.i2v-history { display: grid; gap: 14px; }
|
||||||
|
.i2v-continuation { margin-top: 10px; padding: 10px; border: 1px solid var(--border); border-radius: 7px; }
|
||||||
|
.i2v-continuation label { display: flex; align-items: flex-start; gap: 8px; margin-bottom: 10px; }
|
||||||
|
.i2v-continuation input[type="checkbox"] { width: auto; margin-top: 2px; }
|
||||||
|
.i2v-continuation button { margin: 0 8px 6px 0; }
|
||||||
|
.i2v-task { display: flex; gap: 12px; padding: 12px 0; min-width: 0; }
|
||||||
|
.i2v-task > div { min-width: 0; flex: 1; }
|
||||||
|
.i2v-task h3 { margin: 0 0 6px; font-size: 14px; }
|
||||||
|
.i2v-task p, .i2v-task code { font-size: 12px; overflow-wrap: anywhere; }
|
||||||
|
.i2v-task video { display: block; width: 100%; max-height: 440px; margin: 12px 0; background: #15191e; border-radius: 8px; }
|
||||||
|
.approval-dialog:has(.i2v-task) { max-height: 90vh; overflow-y: auto; }
|
||||||
|
|||||||
@@ -1,12 +1,13 @@
|
|||||||
.app-shell { height: 100dvh; min-height: 620px; display: grid; grid-template-rows: 64px minmax(0, 1fr); overflow: hidden; }
|
.app-shell { height: 100dvh; min-height: 620px; display: grid; grid-template-rows: 64px minmax(0, 1fr); overflow: hidden; }
|
||||||
.workspace-grid { display: grid; grid-template-columns: minmax(220px, 272px) minmax(420px, 1fr) minmax(260px, 330px); min-height: 0; }
|
.workspace-grid { display: grid; grid-template-columns: minmax(220px, 272px) minmax(420px, 1fr) minmax(260px, 330px); min-height: 0; }
|
||||||
.workspace-grid > div { min-width: 0; min-height: 0; }
|
.workspace-grid > div { min-width: 0; min-height: 0; }
|
||||||
|
#skills-panel { overflow-y: auto; overscroll-behavior-y: contain; scrollbar-gutter: stable; }
|
||||||
.workspace-grid > div + div { border-left: 1px solid var(--border); }
|
.workspace-grid > div + div { border-left: 1px solid var(--border); }
|
||||||
.side-panel, .chat-panel { height: 100%; min-height: 0; }
|
.side-panel, .chat-panel { height: 100%; min-height: 0; }
|
||||||
.side-panel { display: flex; flex-direction: column; overflow: hidden; background: var(--surface); }
|
.side-panel { display: flex; flex-direction: column; overflow: hidden; background: var(--surface); }
|
||||||
.chat-panel { background: var(--bg); }
|
.chat-panel { background: var(--bg); }
|
||||||
.mobile-tabs { display: none; }
|
.mobile-tabs { display: none; }
|
||||||
@media (max-width: 1080px) {
|
@media (min-width: 721px) and (max-width: 1080px) {
|
||||||
.workspace-grid { grid-template-columns: minmax(210px, 240px) minmax(400px, 1fr); }
|
.workspace-grid { grid-template-columns: minmax(210px, 240px) minmax(400px, 1fr); }
|
||||||
.workspace-grid > div:last-child { display: none; }
|
.workspace-grid > div:last-child { display: none; }
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,34 @@
|
|||||||
|
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { HarnessApi } from "../src/api/harness-api.js";
|
||||||
|
|
||||||
|
afterEach(() => vi.unstubAllGlobals());
|
||||||
|
|
||||||
|
describe("HarnessApi request headers", () => {
|
||||||
|
it("does not claim an empty POST contains JSON", async () => {
|
||||||
|
const fetchMock = vi.fn().mockResolvedValue(new Response(JSON.stringify({
|
||||||
|
sessionId: "ses_1",
|
||||||
|
requestId: "req_1",
|
||||||
|
}), { status: 201, headers: { "Content-Type": "application/json" } }));
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
|
||||||
|
await new HarnessApi().createSession();
|
||||||
|
|
||||||
|
const request = fetchMock.mock.calls[0]?.[1] as RequestInit;
|
||||||
|
expect(new Headers(request.headers).has("Content-Type")).toBe(false);
|
||||||
|
expect(request.body).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("sets JSON content type when a structured body is present", async () => {
|
||||||
|
const fetchMock = vi.fn().mockResolvedValue(new Response(JSON.stringify({
|
||||||
|
runId: "run_1",
|
||||||
|
requestId: "req_1",
|
||||||
|
}), { status: 202, headers: { "Content-Type": "application/json" } }));
|
||||||
|
vi.stubGlobal("fetch", fetchMock);
|
||||||
|
|
||||||
|
await new HarnessApi().sendMessage("ses_1", { text: "hello", attachmentIds: [] });
|
||||||
|
|
||||||
|
const request = fetchMock.mock.calls[0]?.[1] as RequestInit;
|
||||||
|
expect(new Headers(request.headers).get("Content-Type")).toBe("application/json");
|
||||||
|
expect(request.body).toBe(JSON.stringify({ text: "hello", attachmentIds: [] }));
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
import { render, screen } from "@testing-library/react";
|
import { render, screen } from "@testing-library/react";
|
||||||
import "@testing-library/jest-dom/vitest";
|
import "@testing-library/jest-dom/vitest";
|
||||||
import userEvent from "@testing-library/user-event";
|
import userEvent from "@testing-library/user-event";
|
||||||
import { describe, expect, it, vi } from "vitest";
|
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||||
import { App } from "../src/App.js";
|
import { App } from "../src/App.js";
|
||||||
import { createInitialHarnessState, type HarnessClientState } from "../src/store/reducer.js";
|
import { createInitialHarnessState, type HarnessClientState } from "../src/store/reducer.js";
|
||||||
|
|
||||||
@@ -24,6 +24,7 @@ function makeStore() {
|
|||||||
subscribe: () => () => undefined,
|
subscribe: () => () => undefined,
|
||||||
sendMessage,
|
sendMessage,
|
||||||
cancel: vi.fn().mockResolvedValue({ cancelled: false }),
|
cancel: vi.fn().mockResolvedValue({ cancelled: false }),
|
||||||
|
dispose: vi.fn().mockResolvedValue(undefined),
|
||||||
start: vi.fn().mockResolvedValue(undefined),
|
start: vi.fn().mockResolvedValue(undefined),
|
||||||
getModelConfig: vi.fn().mockResolvedValue({
|
getModelConfig: vi.fn().mockResolvedValue({
|
||||||
modelConfig: { configured: true, source: "environment", providerId: "openai", api: "openai-responses", modelId: "gpt-test", supportsImages: true },
|
modelConfig: { configured: true, source: "environment", providerId: "openai", api: "openai-responses", modelId: "gpt-test", supportsImages: true },
|
||||||
@@ -31,7 +32,47 @@ function makeStore() {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
afterEach(() => vi.unstubAllGlobals());
|
||||||
|
|
||||||
describe("App", () => {
|
describe("App", () => {
|
||||||
|
it("fills a generation-approval prompt for a continued uploaded task without sending or asking for another upload", async () => {
|
||||||
|
const store = makeStore();
|
||||||
|
store.getSnapshot().skills.push({ name: "swads-image-to-video", description: "Image video", source: "local", filePath: "external/swads-image-to-video/SKILL.md", diagnostics: [] });
|
||||||
|
const id = "i2v_11111111-1111-1111-1111-111111111111";
|
||||||
|
const task = { id, revision: 4, title: "Cat", prompt: "Cat blinks", state: "awaiting_submission", durationSeconds: 3, referenceUrl: `/api/image-video/tasks/${id}/reference` };
|
||||||
|
vi.stubGlobal("fetch", (_url: string, init?: RequestInit) => Promise.resolve(new Response(JSON.stringify(init?.method === "POST" ? { task } : { tasks: [task] }))));
|
||||||
|
render(<App store={store} autoStart={false} modelConfigured />);
|
||||||
|
const user = userEvent.setup();
|
||||||
|
await user.click(await screen.findByRole("button", { name: "继续此任务" }));
|
||||||
|
await user.click(screen.getByRole("checkbox", { name: /确认将此任务接续到当前会话/ }));
|
||||||
|
await user.click(screen.getByRole("button", { name: "确认接续(不生成)" }));
|
||||||
|
const composer = screen.getByRole<HTMLTextAreaElement>("textbox", { name: "消息" });
|
||||||
|
expect(composer.value).toContain(id);
|
||||||
|
expect(composer.value).toContain("i2v_submit");
|
||||||
|
expect(composer.value).not.toContain("i2v_upload_reference");
|
||||||
|
expect(composer.value).toContain("3 秒");
|
||||||
|
expect(store.sendMessage).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
it("lets the user select a discovered Skill without submitting data immediately", async () => {
|
||||||
|
const store = makeStore();
|
||||||
|
render(<App store={store} autoStart={false} modelConfigured />);
|
||||||
|
await userEvent.setup().click(screen.getByRole("button", { name: "使用 ads-analysis" }));
|
||||||
|
expect(screen.getByRole<HTMLTextAreaElement>("textbox").value).toContain("ads-analysis");
|
||||||
|
expect(store.sendMessage).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("returns to Chat after choosing a Skill in the mobile layout", async () => {
|
||||||
|
vi.stubGlobal("matchMedia", vi.fn(() => ({ matches: true, addEventListener: vi.fn(), removeEventListener: vi.fn() })));
|
||||||
|
const store = makeStore();
|
||||||
|
render(<App store={store} autoStart={false} modelConfigured />);
|
||||||
|
const user = userEvent.setup();
|
||||||
|
await user.click(screen.getByRole("tab", { name: "Skills" }));
|
||||||
|
await user.click(screen.getByRole("button", { name: "使用 ads-analysis" }));
|
||||||
|
expect(screen.getByRole("tab", { name: "Chat" })).toHaveAttribute("aria-selected", "true");
|
||||||
|
expect(screen.getByRole<HTMLTextAreaElement>("textbox").value).toContain("ads-analysis");
|
||||||
|
expect(store.sendMessage).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
it("renders the three primary work areas", () => {
|
it("renders the three primary work areas", () => {
|
||||||
render(<App store={makeStore()} autoStart={false} modelConfigured />);
|
render(<App store={makeStore()} autoStart={false} modelConfigured />);
|
||||||
expect(screen.getByRole("region", { name: "Skills Catalog" })).toBeVisible();
|
expect(screen.getByRole("region", { name: "Skills Catalog" })).toBeVisible();
|
||||||
@@ -55,4 +96,37 @@ describe("App", () => {
|
|||||||
expect(screen.getByRole("button", { name: "发送消息" })).toBeDisabled();
|
expect(screen.getByRole("button", { name: "发送消息" })).toBeDisabled();
|
||||||
expect(screen.getByText("请先配置模型连接")).toBeVisible();
|
expect(screen.getByText("请先配置模型连接")).toBeVisible();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("switches to the event timeline in the mobile tab layout", async () => {
|
||||||
|
vi.stubGlobal("matchMedia", vi.fn(() => ({
|
||||||
|
matches: true,
|
||||||
|
media: "(max-width: 720px)",
|
||||||
|
onchange: null,
|
||||||
|
addEventListener: vi.fn(),
|
||||||
|
removeEventListener: vi.fn(),
|
||||||
|
addListener: vi.fn(),
|
||||||
|
removeListener: vi.fn(),
|
||||||
|
dispatchEvent: vi.fn(),
|
||||||
|
})));
|
||||||
|
const user = userEvent.setup();
|
||||||
|
render(<App store={makeStore()} autoStart={false} modelConfigured />);
|
||||||
|
|
||||||
|
await user.click(screen.getByRole("tab", { name: "Events" }));
|
||||||
|
|
||||||
|
expect(screen.getByRole("tab", { name: "Events" })).toHaveAttribute("aria-selected", "true");
|
||||||
|
expect(screen.getByRole("region", { name: "Event Timeline" })).toBeVisible();
|
||||||
|
expect(screen.queryByRole("main", { name: "Streaming Chat" })).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("deletes the old session before starting a reset session", async () => {
|
||||||
|
const user = userEvent.setup();
|
||||||
|
const store = makeStore();
|
||||||
|
render(<App store={store} autoStart={false} modelConfigured />);
|
||||||
|
|
||||||
|
await user.click(screen.getByRole("button", { name: "重置训练会话" }));
|
||||||
|
|
||||||
|
expect(store.dispose).toHaveBeenCalledTimes(1);
|
||||||
|
expect(store.start).toHaveBeenCalledTimes(1);
|
||||||
|
expect(store.dispose.mock.invocationCallOrder[0]).toBeLessThan(store.start.mock.invocationCallOrder[0] ?? 0);
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -0,0 +1,34 @@
|
|||||||
|
import { describe, expect, it, vi } from "vitest";
|
||||||
|
import type { HarnessApi } from "../src/api/harness-api.js";
|
||||||
|
import { HarnessClientStore } from "../src/store/harness-client-store.js";
|
||||||
|
|
||||||
|
describe("HarnessClientStore lifecycle", () => {
|
||||||
|
it("deletes the current session and can start a fresh session after dispose", async () => {
|
||||||
|
const createSession = vi.fn()
|
||||||
|
.mockResolvedValueOnce({ sessionId: "ses_1", requestId: "req_1" })
|
||||||
|
.mockResolvedValueOnce({ sessionId: "ses_2", requestId: "req_2" });
|
||||||
|
const deleteSession = vi.fn().mockResolvedValue({ deleted: true, requestId: "req_delete" });
|
||||||
|
const api = {
|
||||||
|
createSession,
|
||||||
|
deleteSession,
|
||||||
|
getSkills: vi.fn().mockResolvedValue({ skills: [], requestId: "req_skills" }),
|
||||||
|
eventsUrl: (sessionId: string) => `/api/sessions/${sessionId}/events`,
|
||||||
|
} as unknown as HarnessApi;
|
||||||
|
const eventSources: Array<{ close: ReturnType<typeof vi.fn> }> = [];
|
||||||
|
const store = new HarnessClientStore(api, () => {
|
||||||
|
const source = { addEventListener: vi.fn(), close: vi.fn() };
|
||||||
|
eventSources.push(source);
|
||||||
|
return source;
|
||||||
|
});
|
||||||
|
|
||||||
|
await store.start();
|
||||||
|
expect(store.getSnapshot().sessionId).toBe("ses_1");
|
||||||
|
await store.dispose();
|
||||||
|
await store.start();
|
||||||
|
|
||||||
|
expect(deleteSession).toHaveBeenCalledWith("ses_1");
|
||||||
|
expect(createSession).toHaveBeenCalledTimes(2);
|
||||||
|
expect(eventSources[0]?.close).toHaveBeenCalledTimes(1);
|
||||||
|
expect(store.getSnapshot().sessionId).toBe("ses_2");
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
import "@testing-library/jest-dom/vitest";
|
||||||
|
import { render, screen } from "@testing-library/react";
|
||||||
|
import userEvent from "@testing-library/user-event";
|
||||||
|
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { CmsPanel } from "../src/components/cms/CmsPanel.js";
|
||||||
|
|
||||||
|
afterEach(() => vi.unstubAllGlobals());
|
||||||
|
describe("CMS panel", () => {
|
||||||
|
it("clears passwords after failed login and never renders reflected upstream secrets", async () => {
|
||||||
|
vi.stubGlobal("fetch", (_url: string, init?: RequestInit) => Promise.resolve(new Response(JSON.stringify(init?.method === "POST" ? { error: { code: "CMS_LOGIN_FAILED", message: "echo password-secret" } } : { connection: { configured: true, connected: false } }), { status: init?.method === "POST" ? 400 : 200 })));
|
||||||
|
render(<CmsPanel sessionId="ses_1" busy={false} onSelect={() => undefined} />);
|
||||||
|
const user = userEvent.setup();
|
||||||
|
await user.click(screen.getByRole("button", { name: "CMS 连接与选剧" }));
|
||||||
|
expect(screen.getByText(/选段审核和 10 秒 REVIEW 预览/)).toBeVisible();
|
||||||
|
await user.type(screen.getByLabelText("CMS 邮箱"), "fixture@example.com");
|
||||||
|
await user.type(screen.getByLabelText("CMS 密码"), "password-secret");
|
||||||
|
await user.click(screen.getByRole("button", { name: "登录 CMS" }));
|
||||||
|
expect(screen.getByLabelText("CMS 密码")).toHaveValue("");
|
||||||
|
expect(await screen.findByRole("alert")).toHaveTextContent("登录失败");
|
||||||
|
expect(screen.queryByText(/password-secret/)).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
import {render,screen} from "@testing-library/react";
|
||||||
|
import userEvent from "@testing-library/user-event";
|
||||||
|
import {it,expect,vi,afterEach} from "vitest";
|
||||||
|
import {EditorWorkflowPanel} from "../src/components/tools/EditorWorkflowPanel.js";
|
||||||
|
const url="/api/sessions/ses_1/editor/job_1/workflow";
|
||||||
|
afterEach(()=>vi.unstubAllGlobals());
|
||||||
|
it("shows the actual registered preview with REVIEW labeling and no auto approval",async()=>{
|
||||||
|
vi.stubGlobal("fetch",()=>Promise.resolve({ok:true,json:()=>Promise.resolve({jobId:"job_1",selection:null,plan:null,preview:{url:"/api/sessions/ses_1/editor/job_1/plans/plan_1/preview",duration:10,reviewStatus:"REVIEW",sha256:"abc",machinePass:true}})}));
|
||||||
|
render(<EditorWorkflowPanel url={url}/>);
|
||||||
|
await userEvent.click(screen.getByRole("button",{name:"打开剪辑审核与预览"}));
|
||||||
|
expect(await screen.findByText(/10 秒/)).toBeVisible();
|
||||||
|
expect(screen.getByLabelText("REVIEW 视频预览")).toHaveAttribute("src","/api/sessions/ses_1/editor/job_1/plans/plan_1/preview");
|
||||||
|
expect(screen.getByRole("link",{name:"下载 REVIEW MP4"})).toHaveAttribute("href","/api/sessions/ses_1/editor/job_1/plans/plan_1/preview?download=1");
|
||||||
|
expect(screen.queryByText("最终审核 PASS")).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
it("keeps human review boxes unchecked and rejects off-job media URLs",async()=>{
|
||||||
|
vi.stubGlobal("fetch",()=>Promise.resolve({ok:true,json:()=>Promise.resolve({jobId:"job_1",selection:{hash:"abc",reviewed:false,segments:[{id:"one",episodeNo:1,start:0,end:10,videoUrl:"https://external.example/secret",reason:"Test",subtitles:"Test"}]},plan:null,preview:null})}));
|
||||||
|
render(<EditorWorkflowPanel url={url}/>);await userEvent.click(screen.getByRole("button",{name:"打开剪辑审核与预览"}));
|
||||||
|
for(const checkbox of await screen.findAllByRole("checkbox"))expect(checkbox).not.toBeChecked();
|
||||||
|
expect(screen.queryByLabelText("选段 one 原声视频")).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
import { act, render, screen, waitFor } from "@testing-library/react";
|
||||||
|
import userEvent from "@testing-library/user-event";
|
||||||
|
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { ApprovalDialog } from "../src/components/approvals/ApprovalDialog.js";
|
||||||
|
import { ImageVideoPanel, ImageVideoTaskCard } from "../src/components/tools/ImageVideoPanel.js";
|
||||||
|
|
||||||
|
const id = "i2v_11111111-1111-1111-1111-111111111111";
|
||||||
|
const task = { id, title: "Window", prompt: "Turn naturally toward the window; ambient rain", durationSeconds: 10, state: "queued", referenceUrl: `/api/image-video/tasks/${id}/reference`, createdAt: "2026-09-15T00:00:00Z", updatedAt: "2026-09-15T00:00:00Z" };
|
||||||
|
afterEach(() => { vi.unstubAllGlobals(); vi.useRealTimers(); });
|
||||||
|
|
||||||
|
describe("image video UI", () => {
|
||||||
|
it("continues the exact task only after confirmation and never sends or generates automatically", async () => {
|
||||||
|
const onDraft = vi.fn(); const requests: { url: string; body: unknown }[] = [];
|
||||||
|
const pending = { ...task, state: "awaiting_submission", revision: 9 };
|
||||||
|
vi.stubGlobal("fetch", (url: string, init?: RequestInit) => {
|
||||||
|
if (init?.method === "POST") requests.push({ url, body: init.body });
|
||||||
|
return Promise.resolve(new Response(JSON.stringify(init?.method === "POST" ? { task: { ...pending, revision: 10 } } : { tasks: [pending] })));
|
||||||
|
});
|
||||||
|
render(<ImageVideoPanel sessionId="ses_new" onDraft={onDraft} />);
|
||||||
|
const user = userEvent.setup();
|
||||||
|
await user.click(await screen.findByRole("button", { name: "继续此任务" }));
|
||||||
|
expect(screen.getByRole("button", { name: "确认接续(不生成)" })).toBeDisabled();
|
||||||
|
expect(requests).toEqual([]);
|
||||||
|
await user.click(screen.getByRole("checkbox", { name: /确认将此任务接续到当前会话/ }));
|
||||||
|
await user.click(screen.getByRole("button", { name: "确认接续(不生成)" }));
|
||||||
|
await waitFor(() => expect(onDraft).toHaveBeenCalledWith(expect.objectContaining({ id, state: "awaiting_submission", revision: 10 })));
|
||||||
|
expect(requests).toEqual([{ url: `/api/sessions/ses_new/image-video/tasks/${id}/continue`, body: JSON.stringify({ confirmed: true, expectedRevision: 9 }) }]);
|
||||||
|
});
|
||||||
|
it("does not offer continuation for unknown submission/upload outcomes", async () => {
|
||||||
|
vi.stubGlobal("fetch", () => Promise.resolve(new Response(JSON.stringify({ tasks: [{ ...task, revision: 2, state: "unknown_upload" }] }))));
|
||||||
|
render(<ImageVideoPanel sessionId="ses_new" onDraft={vi.fn()} />);
|
||||||
|
expect(await screen.findByText(/上传状态未知/)).toBeVisible();
|
||||||
|
expect(screen.queryByRole("button", { name: "继续此任务" })).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
it("ignores a late continuation response after switching sessions", async () => {
|
||||||
|
const onDraft = vi.fn(); const pending = { ...task, revision: 9, state: "awaiting_submission" };
|
||||||
|
let finish!: (response: Response) => void;
|
||||||
|
vi.stubGlobal("fetch", (_url: string, init?: RequestInit) => init?.method === "POST" ? new Promise<Response>(resolve => { finish = resolve; }) : Promise.resolve(new Response(JSON.stringify({ tasks: [pending] }))));
|
||||||
|
const view = render(<ImageVideoPanel sessionId="ses_old" onDraft={onDraft} />); const user = userEvent.setup();
|
||||||
|
await user.click(await screen.findByRole("button", { name: "继续此任务" }));
|
||||||
|
await user.click(screen.getByRole("checkbox", { name: /确认将此任务接续到当前会话/ }));
|
||||||
|
await user.click(screen.getByRole("button", { name: "确认接续(不生成)" }));
|
||||||
|
view.rerender(<ImageVideoPanel sessionId="ses_new" onDraft={onDraft} />);
|
||||||
|
await act(() => { finish(new Response(JSON.stringify({ task: pending }))); return Promise.resolve(); });
|
||||||
|
expect(onDraft).not.toHaveBeenCalled();
|
||||||
|
expect(screen.getByRole("button", { name: "继续此任务" })).toBeEnabled();
|
||||||
|
});
|
||||||
|
it("restores queued tasks, polls at 30 seconds, and cleans up after unmount", async () => {
|
||||||
|
vi.useFakeTimers();
|
||||||
|
const fetcher = vi.fn(() => Promise.resolve(new Response(JSON.stringify({ tasks: [task] }))));
|
||||||
|
vi.stubGlobal("fetch", fetcher); const view = render(<ImageVideoPanel />);
|
||||||
|
await act(() => vi.advanceTimersByTimeAsync(0));
|
||||||
|
expect(screen.getByText("Window")).toBeVisible(); expect(screen.getByText(/排队中/)).toBeVisible();
|
||||||
|
await act(() => vi.advanceTimersByTimeAsync(29_999)); expect(fetcher).toHaveBeenCalledTimes(1);
|
||||||
|
await act(() => vi.advanceTimersByTimeAsync(1)); expect(fetcher).toHaveBeenCalledTimes(2);
|
||||||
|
view.unmount(); await act(() => vi.advanceTimersByTimeAsync(60_000)); expect(fetcher).toHaveBeenCalledTimes(2);
|
||||||
|
});
|
||||||
|
it("creates a draft from a local image without needing model image capability", async () => {
|
||||||
|
const onDraft = vi.fn(); const requests: { url: string; body: unknown }[] = [];
|
||||||
|
vi.stubGlobal("fetch", (url: string, init?: RequestInit) => { requests.push({ url, body: init?.body }); return Promise.resolve(new Response(JSON.stringify(url.includes("attachments") ? { attachment: { attachmentId: "att_local" } } : init?.method === "POST" ? { task } : { tasks: [] }))); });
|
||||||
|
render(<ImageVideoPanel sessionId="ses_1" onDraft={onDraft} />); const user = userEvent.setup();
|
||||||
|
await user.upload(screen.getByLabelText("参考图片"), new File(["test"], "test.png", { type: "image/png" }));
|
||||||
|
await user.type(screen.getByLabelText("标题"), "Window"); await user.type(screen.getByLabelText("动作与声音"), task.prompt);
|
||||||
|
await user.click(screen.getByRole("button", { name: "准备本地任务" }));
|
||||||
|
await waitFor(() => expect(onDraft).toHaveBeenCalledWith(expect.objectContaining({ id })));
|
||||||
|
expect(requests.some(request => request.url.includes("/sessions/ses_1/attachments"))).toBe(true);
|
||||||
|
expect(requests.filter(request => request.url.endsWith("/image-video/tasks") && typeof request.body === "string")[0]?.body).toContain('"attachmentId":"att_local"');
|
||||||
|
expect(requests.some(request => /submit|upload_reference/.test(request.url))).toBe(false);
|
||||||
|
});
|
||||||
|
it("renders a native player/download only for this registered task video", () => {
|
||||||
|
const view = render(<ImageVideoTaskCard task={{ ...task, state: "ready", videoUrl: `/api/image-video/tasks/${id}/video` }} />);
|
||||||
|
expect(screen.getByLabelText("生成预览播放器")).toHaveAttribute("controls");
|
||||||
|
expect(screen.getByRole("link", { name: "下载 MP4" })).toHaveAttribute("href", `/api/image-video/tasks/${id}/video?download=1`);
|
||||||
|
expect(screen.getByRole("status")).toHaveTextContent("待视听检查");
|
||||||
|
view.rerender(<ImageVideoTaskCard task={{ ...task, state: "ready", videoUrl: "https://evil.example/video" }} />);
|
||||||
|
expect(screen.queryByRole("link", { name: "下载 MP4" })).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
it("requires an unchecked confirmation after showing the frozen reference/prompt/duration", async () => {
|
||||||
|
vi.stubGlobal("fetch", () => Promise.resolve(new Response(JSON.stringify({ task }))));
|
||||||
|
const approve = vi.fn();
|
||||||
|
render(<ApprovalDialog request={{ approvalId: "ap_i2v", toolCallId: "tc", toolName: "i2v_submit", arguments: { taskId: id }, riskLevel: "medium", expiresAt: "2099-01-01T00:00:00Z", status: "pending" }} onApprove={approve} onDeny={vi.fn()} />);
|
||||||
|
expect(await screen.findByText(task.prompt)).toBeVisible();
|
||||||
|
expect(screen.getByRole("img")).toHaveAttribute("src", task.referenceUrl);
|
||||||
|
expect(screen.getByText(/10 秒/)).toBeVisible(); expect(screen.getByText(/费用未知/)).toBeVisible();
|
||||||
|
const button = screen.getByRole("button", { name: "批准 H3 生成" }); expect(button).toBeDisabled();
|
||||||
|
await userEvent.click(screen.getByRole("checkbox")); expect(button).toBeEnabled();
|
||||||
|
await userEvent.click(button); expect(approve).toHaveBeenCalledWith("ap_i2v");
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -16,6 +16,17 @@ function event<T extends HarnessEvent>(value: Omit<T, "eventId" | "sessionId" |
|
|||||||
}
|
}
|
||||||
|
|
||||||
describe("reduceHarnessEvent", () => {
|
describe("reduceHarnessEvent", () => {
|
||||||
|
it("shows tool-only assistant turns before the first text token without duplicating the final message", () => {
|
||||||
|
const pending = [
|
||||||
|
event<HarnessEvent>({ type: "run.started", sequence: 1, runId: "run_1", payload: { modelId: "qwen" } }),
|
||||||
|
event<HarnessEvent>({ type: "tool.requested", sequence: 2, runId: "run_1", payload: { toolCallId: "tool_1", toolName: "mock_ads_metrics", arguments: { accountId: "demo-account", days: 7 } } }),
|
||||||
|
].reduce(reduceHarnessEvent, createInitialHarnessState());
|
||||||
|
expect(pending.messages.msg_run_1).toMatchObject({ role: "assistant", runId: "run_1", status: "running", text: "" });
|
||||||
|
const final = reduceHarnessEvent(pending, event<HarnessEvent>({ type: "assistant.delta", sequence: 3, runId: "run_1", payload: { messageId: "msg_run_1", delta: "已获取指标" } }));
|
||||||
|
expect(final.messageOrder).toEqual(["msg_run_1"]);
|
||||||
|
expect(final.messages.msg_run_1?.text).toBe("已获取指标");
|
||||||
|
});
|
||||||
|
|
||||||
it("builds messages, run state, tools, approvals, and timeline from ordered events", () => {
|
it("builds messages, run state, tools, approvals, and timeline from ordered events", () => {
|
||||||
const events: HarnessEvent[] = [
|
const events: HarnessEvent[] = [
|
||||||
event({ type: "user.message", sequence: 1, payload: { messageId: "msg_user", text: "分析素材", attachmentIds: [] } }),
|
event({ type: "user.message", sequence: 1, payload: { messageId: "msg_user", text: "分析素材", attachmentIds: [] } }),
|
||||||
|
|||||||
@@ -10,6 +10,8 @@ describe("ModelSettings", () => {
|
|||||||
const save = vi.fn().mockResolvedValue({ configured: true, source: "session", keyHint: "••••cret" });
|
const save = vi.fn().mockResolvedValue({ configured: true, source: "session", keyHint: "••••cret" });
|
||||||
render(<ModelSettings open onClose={vi.fn()} busy={false} onSave={save} onTest={vi.fn()} onClear={vi.fn()} />);
|
render(<ModelSettings open onClose={vi.fn()} busy={false} onSave={save} onTest={vi.fn()} onClear={vi.fn()} />);
|
||||||
|
|
||||||
|
expect(screen.getByLabelText("模型 ID")).toHaveAttribute("autocomplete", "off");
|
||||||
|
expect(screen.getByLabelText("API Key")).toHaveAttribute("autocomplete", "new-password");
|
||||||
await user.type(screen.getByLabelText("API Key"), "sk-browser-secret");
|
await user.type(screen.getByLabelText("API Key"), "sk-browser-secret");
|
||||||
await user.type(screen.getByLabelText("模型 ID"), "gpt-test");
|
await user.type(screen.getByLabelText("模型 ID"), "gpt-test");
|
||||||
await user.click(screen.getByRole("button", { name: "保存配置" }));
|
await user.click(screen.getByRole("button", { name: "保存配置" }));
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ import userEvent from "@testing-library/user-event";
|
|||||||
import { describe, expect, it, vi } from "vitest";
|
import { describe, expect, it, vi } from "vitest";
|
||||||
import { ApprovalDialog } from "../src/components/approvals/ApprovalDialog.js";
|
import { ApprovalDialog } from "../src/components/approvals/ApprovalDialog.js";
|
||||||
import { AdsMetricsTool } from "../src/components/tools/AdsMetricsTool.js";
|
import { AdsMetricsTool } from "../src/components/tools/AdsMetricsTool.js";
|
||||||
|
import { ToolFallback } from "../src/components/tools/ToolFallback.js";
|
||||||
import type { ClientApproval } from "../src/store/reducer.js";
|
import type { ClientApproval } from "../src/store/reducer.js";
|
||||||
|
|
||||||
const pendingApproval: ClientApproval = {
|
const pendingApproval: ClientApproval = {
|
||||||
@@ -16,6 +17,49 @@ const pendingApproval: ClientApproval = {
|
|||||||
};
|
};
|
||||||
|
|
||||||
describe("tool and approval UI", () => {
|
describe("tool and approval UI", () => {
|
||||||
|
it("shows registered representative frames but never external frame URLs or a fake final pass", () => {
|
||||||
|
render(<ToolFallback type="tool-call" toolName="prepare_drama_sources" toolCallId="tool_media" args={{}} argsText="" result={{ jobId: "job_known", title: "Test drama", state: "sources_ready", sources: [{ episodeNo: 1, duration: 72.3, frames: [
|
||||||
|
{ frameId: "frame_known", time: 10.5, url: "/api/sessions/ses_1/editor/job_known/frames/frame_known" },
|
||||||
|
{ frameId: "frame_bad", time: 22, url: "https://external.example/image.jpg" },
|
||||||
|
] }] }} status={{ type: "complete" }} addResult={vi.fn()} resume={vi.fn()} respondToApproval={vi.fn()} />);
|
||||||
|
expect(screen.getAllByRole("img")).toHaveLength(1);
|
||||||
|
expect(screen.getByRole("img")).toHaveAttribute("src", "/api/sessions/ses_1/editor/job_known/frames/frame_known");
|
||||||
|
expect(screen.getByText(/尚未完成剧情验证/)).toBeVisible();
|
||||||
|
expect(screen.getByText("job_known")).toBeVisible();
|
||||||
|
});
|
||||||
|
it.each([
|
||||||
|
["prepare_drama_sources", "允许下载所选分集并在本机抽帧吗?", "批准本次下载"],
|
||||||
|
["read_drama_frames", "允许将字幕和代表帧发送给当前模型吗?", "批准本次分析"],
|
||||||
|
["render_editor_preview", "允许生成 10 秒 REVIEW 预览吗?", "批准本次渲染"],
|
||||||
|
])("explains the media boundary for %s", (toolName, title, button) => {
|
||||||
|
render(<ApprovalDialog request={{ ...pendingApproval, toolName, arguments: { jobId: "job_authorized" } }} onApprove={vi.fn()} onDeny={vi.fn()} />);
|
||||||
|
expect(screen.getByRole("dialog", { name: title })).toBeVisible();
|
||||||
|
expect(screen.getByRole("button", { name: button })).toBeVisible();
|
||||||
|
expect(screen.getByText(/job_authorized/)).toBeVisible();
|
||||||
|
expect(screen.queryByText("report.md")).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
it("identifies licensed CMS content reads and their model destination", () => {
|
||||||
|
render(<ApprovalDialog request={{ ...pendingApproval, toolName: "cms_drama_inventory", arguments: { dramaId: "fixture-drama" } }} onApprove={vi.fn()} onDeny={vi.fn()} />);
|
||||||
|
expect(screen.getByRole("dialog", { name: "允许读取 CMS 内容并交给模型吗?" })).toBeVisible();
|
||||||
|
expect(screen.getByText(/fixture-drama/)).toBeVisible();
|
||||||
|
expect(screen.queryByText("report.md")).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
it("offers only Session report download links and exposes PDF generation failures", () => {
|
||||||
|
render(<ToolFallback type="tool-call" toolName="generate_minis_report" toolCallId="tool_report" args={{}} argsText="" result={{ pdfStatus: "failed", warning: "PDF 未生成", files: [{ format: "html", url: "/api/sessions/ses_1/reports/rpt_1/html" }, { format: "pdf", url: "https://untrusted.example/private" }] }} status={{ type: "complete" }} addResult={vi.fn()} resume={vi.fn()} respondToApproval={vi.fn()} />);
|
||||||
|
expect(screen.getByRole("link", { name: "下载 HTML" })).toHaveAttribute("href", "/api/sessions/ses_1/reports/rpt_1/html");
|
||||||
|
expect(screen.queryByRole("link", { name: "下载 PDF" })).not.toBeInTheDocument();
|
||||||
|
expect(screen.getByText("PDF 未生成")).toBeVisible();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("shows actual analytics scope and destination rather than pretending the action only saves a file", () => {
|
||||||
|
render(<ApprovalDialog request={{ ...pendingApproval, toolName: "metrics_semantic_query", arguments: { date_from: "2026-09-08", date_to: "2026-09-08", external_account_id: "test-account", metrics: ["spend"] } }} onApprove={vi.fn()} onDeny={vi.fn()} />);
|
||||||
|
expect(screen.getByRole("dialog", { name: "允许查询真实经营数据吗?" })).toBeVisible();
|
||||||
|
expect(screen.getByText(/2026-09-08/)).toBeVisible();
|
||||||
|
expect(screen.getByText(/test-account/)).toBeVisible();
|
||||||
|
expect(screen.getByRole("button", { name: "批准本次查询" })).toBeVisible();
|
||||||
|
expect(screen.queryByText("report.md")).not.toBeInTheDocument();
|
||||||
|
});
|
||||||
|
|
||||||
it("disables both decisions after submitting approval", async () => {
|
it("disables both decisions after submitting approval", async () => {
|
||||||
const user = userEvent.setup();
|
const user = userEvent.setup();
|
||||||
let resolveApproval: (() => void) | undefined;
|
let resolveApproval: (() => void) | undefined;
|
||||||
|
|||||||
@@ -9,8 +9,9 @@ export default defineConfig({
|
|||||||
server: {
|
server: {
|
||||||
port: 5173,
|
port: 5173,
|
||||||
proxy: {
|
proxy: {
|
||||||
"/api": "http://127.0.0.1:3000",
|
// Preserve the page Host so the backend can verify Origin for CMS login.
|
||||||
"/health": "http://127.0.0.1:3000"
|
"/api": { target: "http://127.0.0.1:3001", changeOrigin: false },
|
||||||
|
"/health": { target: "http://127.0.0.1:3001", changeOrigin: false }
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -0,0 +1,78 @@
|
|||||||
|
# 架构与数据流
|
||||||
|
|
||||||
|
## 设计目标
|
||||||
|
|
||||||
|
Agent Studio Mini 把课堂上最容易被隐藏的 Agent 行为变成可观察事件:模型流式输出、Skill 渐进加载、工具生命周期、写操作审批和 Session 清理。浏览器是投影,服务端 Harness 才是状态权威。
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
subgraph Browser
|
||||||
|
Composer
|
||||||
|
Thread[assistant-ui Thread]
|
||||||
|
ToolUI[Tool UI / Approval Dialog]
|
||||||
|
Timeline
|
||||||
|
Store[HarnessClientStore]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph Fastify
|
||||||
|
Routes[HTTP Routes]
|
||||||
|
SSE[SSE Connection]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph Harness
|
||||||
|
Registry[SessionRegistry]
|
||||||
|
Session[HarnessSession]
|
||||||
|
Events[EventStore]
|
||||||
|
Broker[ApprovalBroker]
|
||||||
|
Attachments[AttachmentStore]
|
||||||
|
Models[ModelConnectionStore]
|
||||||
|
Skills[SkillCatalog / Inference]
|
||||||
|
Workspace[SafeWorkspace]
|
||||||
|
end
|
||||||
|
|
||||||
|
subgraph Pi
|
||||||
|
Agent[AgentSession]
|
||||||
|
Loader[DefaultResourceLoader]
|
||||||
|
Runtime[ModelRuntime]
|
||||||
|
Tools[Pi Tools + Extension]
|
||||||
|
end
|
||||||
|
|
||||||
|
Composer -->|multipart / JSON| Routes
|
||||||
|
Routes --> Registry --> Session --> Agent
|
||||||
|
Store -->|EventSource| SSE --> Events
|
||||||
|
Events --> Store --> Thread
|
||||||
|
Store --> Timeline
|
||||||
|
Store --> ToolUI
|
||||||
|
Session --> Attachments
|
||||||
|
Session --> Models --> Runtime
|
||||||
|
Session --> Skills
|
||||||
|
Agent --> Loader
|
||||||
|
Agent --> Tools --> Workspace
|
||||||
|
Tools --> Broker --> ToolUI
|
||||||
|
```
|
||||||
|
|
||||||
|
## 一次消息的生命周期
|
||||||
|
|
||||||
|
1. Web 创建 Session,读取 Skill Catalog,并建立 SSE。
|
||||||
|
2. 图片由 `AttachmentAdapter` 先上传,Composer 只保留 `attachmentId`。
|
||||||
|
3. Web 提交文本和 attachment IDs;Server 校验后立即返回 `202 + runId`。
|
||||||
|
4. `HarnessSession` 显式解析当前模型,转换图片,创建或复用对应模型指纹的 Pi Session。
|
||||||
|
5. Pi 事件由 `PiEventAdapter` 串行映射。异步 Skill 路径识别完成之前不会越过后续 read-completed 或 `run.completed`。
|
||||||
|
6. `InMemoryEventStore` 分配单调 sequence;SSE 支持 cursor replay、heartbeat 和断线清理。
|
||||||
|
7. Web reducer 去重并拒绝 sequence gap,映射为 assistant-ui 外部 Runtime、工具卡、审批框和时间线。
|
||||||
|
8. run 完成、取消或失败后释放订阅;删除 Session 时取消审批、终止 run、删除附件和工作区。
|
||||||
|
|
||||||
|
## 核心所有权
|
||||||
|
|
||||||
|
| 状态 | 权威组件 | Web 的角色 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Session / active run | `SessionRegistry` / `HarnessSession` | 发起命令、展示结果 |
|
||||||
|
| Event sequence | `EventStore` | 去重、检测缺口、重连 |
|
||||||
|
| Approval | `ApprovalBroker` | 提交一次决策、等待事件确认 |
|
||||||
|
| API Key | `ModelConnectionStore` + Pi 内存凭证 | 仅在 password input 中临时持有 |
|
||||||
|
| Attachment bytes | `AttachmentStore` | 上传阶段持有 File/object URL |
|
||||||
|
| Tool write target | `SafeWorkspace` | 不参与路径选择 |
|
||||||
|
|
||||||
|
## 进程与存储模型
|
||||||
|
|
||||||
|
课堂版只有一个 Node 进程。EventStore、ApprovalBroker 和模型配置均在内存;Session 工作区位于 `.agent-studio-sessions/<sessionId>`。这种结构便于展示边界和清理行为,但不提供横向扩展、崩溃恢复或多租户隔离。
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
# 课堂演示脚本(10–15 分钟)
|
||||||
|
|
||||||
|
## 演示前准备(1 分钟)
|
||||||
|
|
||||||
|
运行 `pnpm dev`,打开 `http://127.0.0.1:5173`。准备一个可用模型配置;若演示图片,确认 `supportsImages` 已开启并准备一张 JPEG、PNG 或 WebP。
|
||||||
|
|
||||||
|
先指出三栏:左侧是被发现的 Skills,中间是 assistant-ui 对话与工具卡,右侧是服务器事件时间线。强调页面不是 Agent 本体,而是 Harness 状态的投影。
|
||||||
|
|
||||||
|
## 1. 未配置模型(1 分钟)
|
||||||
|
|
||||||
|
展示 Composer 的明确禁用提示。解释系统不会偷偷选择任意可用模型,也不会把图片悄悄丢弃。打开 Settings,填写 Provider、API、Model、可选 Base URL 和 API Key。
|
||||||
|
|
||||||
|
指出 API Key 字段刷新后不会恢复,GET 响应也只能看到末四位提示。
|
||||||
|
|
||||||
|
## 2. Skill 渐进披露(2 分钟)
|
||||||
|
|
||||||
|
点击“填入演示任务”,不要立刻发送:
|
||||||
|
|
||||||
|
> 请分析 demo-account 最近 7 天广告数据,并生成一份简短报告草稿。
|
||||||
|
|
||||||
|
先看左侧只有 `ads-analysis` 的名称、描述和 diagnostics,没有把正文放进浏览器。发送后观察 `read`、`skill.requested` 和 `skill.loaded`。说明两个 Skill 事件由 Harness 根据规范路径和 read 成功状态推导,不是模型自报。
|
||||||
|
|
||||||
|
## 3. 只读工具与流式回复(2–3 分钟)
|
||||||
|
|
||||||
|
观察中央文字逐段增加,右侧 sequence 单调增长。`mock_ads_metrics` 工具卡展示 account、days、spend、revenue 和 ROAS。
|
||||||
|
|
||||||
|
明确数据来自仓库 fixture,不连接真实广告平台。Skill 约束模型先取工具结果再做数字结论,这体现“指令披露 + 结构化工具”的组合。
|
||||||
|
|
||||||
|
## 4. 服务器审批(2–3 分钟)
|
||||||
|
|
||||||
|
当 `save_report_draft` 请求出现时,Dialog 显示风险、固定目标、approval ID 和剩余时间。先讲清浏览器按钮不是权限边界,Broker 的状态才是权威。
|
||||||
|
|
||||||
|
第一轮点击 Deny:工具显示未执行,Session 目录不产生 `report.md`。
|
||||||
|
|
||||||
|
第二轮重发任务并点击 Approve:只有这次会在当前 Session 临时目录写入固定 `report.md`。重复点击被 UI 锁定,冲突决策被 Server 拒绝。也可等待超时展示 expired 不写文件。
|
||||||
|
|
||||||
|
## 5. 图片多模态(2 分钟)
|
||||||
|
|
||||||
|
附加一张广告截图,输入:
|
||||||
|
|
||||||
|
> 请结合这张素材解释可能的创意问题,并用 demo-account 最近 7 天数据验证建议。不要从图片猜测账户指标。
|
||||||
|
|
||||||
|
展示上传、缩略图、预览与移除。说明浏览器/SSE 只保留 attachmentId,Server 规范化文件后才转换为 Pi `ImageContent[]`。若将模型设置为 text-only,系统会明确阻止发送图片。
|
||||||
|
|
||||||
|
## 6. 重连与清理(1–2 分钟)
|
||||||
|
|
||||||
|
说明 SSE 使用 sequence 和 Last-Event-ID 重放,重复事件会去重,缺口会触发全量重建。点击重置或删除 Session,展示 run、审批、附件、报告与内存凭证一并清理。
|
||||||
|
|
||||||
|
最后强调:当前的 SafeWorkspace、工具策略和审批是教学用应用边界;Pi 没有内置 Sandbox,生产环境仍要用容器或 microVM 隔离。
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# 网页调用 Drama Minis
|
||||||
|
|
||||||
|
## 使用
|
||||||
|
|
||||||
|
打开本机 http://127.0.0.1:5173/,在 Skills 中找到 `drama-minis-diagnosis`,点击「使用此 Skill」。页面返回 Chat 并填入任务草稿,点击发送即可读取原始 Skill。
|
||||||
|
|
||||||
|
模型会确认分析日期、对比日期和账户范围。确认后,每次真实指标查询会弹出审批,检查日期与账户参数后点击「批准本次查询」。数据会由当前配置的模型处理。报告生成有独立审批,成功后工具卡提供 JSON、HTML 和 PDF 下载;PDF 失败不会显示虚假的 PDF 链接。
|
||||||
|
|
||||||
|
`ads-analysis` 和「填入演示任务」仍使用模拟数据,不是这条真实业务链路。
|
||||||
|
|
||||||
|
## 服务配置
|
||||||
|
|
||||||
|
- `SWADS_MCP_TOKEN`:后端环境中的凭证,不写前端、源码或 Git。
|
||||||
|
- `DRAMA_MINIS_SKILL_DIR`:默认用户目录 `.codex/skills/drama-minis-diagnosis`。
|
||||||
|
- `PYTHON_EXECUTABLE`:默认 python3。
|
||||||
|
- `CHROME_EXECUTABLE`:macOS 默认 Google Chrome;用于本地 A4 PDF。
|
||||||
|
- 模型配置沿用网页 Settings 或服务器环境。
|
||||||
|
|
||||||
|
本机另有 `scripts/start-private-server.mjs`,在构建后通过关闭回显的交互终端读取一行包含 swadsToken 与 llmKey 的 JSON,启动指定的 Qwen 网关后端。不要把凭证放在命令行参数或文件中;退出后恢复终端回显。凭证只在进程内,服务退出后需重新注入,脚本不是持久化密钥管理器。
|
||||||
|
|
||||||
|
## 验证与限制
|
||||||
|
|
||||||
|
2026-09-09 已通过真实网页验证:Qwen 成功读取 SKILL.md、metrics.md、reporting-playbook.md,成功调用 swads_whoami 与 metrics_catalog,随后询问周期和账户范围。未调用 metrics_semantic_query,未制作真实经营报告。
|
||||||
|
|
||||||
|
验证结果:137 条自动化测试通过(包含新增窄屏 Skill→Chat 切换测试);typecheck、lint、build 通过。原始 Python 渲染器的合成测试生成了 1 页中文 A4 PDF(594.96 × 841.92 pt),完成图像检查;不是经营数据证据。构建仍有约 599 kB 主包的非阻塞体积提示。
|
||||||
|
|
||||||
|
默认测试覆盖 MCP 白名单、脱敏与错误、资源快照、审批、报告来源 ID、下载边界和前端下载卡。`node scripts/verify-minis-render.mjs` 是可选的离线渲染验证,使用人工合成数据调用原始 Skill Python 渲染器与本地 Chrome,不查询经营数据,不是经营报告。
|
||||||
|
|
||||||
|
当前原始渲染器仅支持 USD 图表;非 USD 报告明确拒绝。报告与查询记录属于内存 Session,重启或重置后下载登记失效,请及时下载。工具审批与来源留档不能替代人工核对模型分析。
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
# 首次剪辑接入验收:Beneath the Ice
|
||||||
|
|
||||||
|
用户于 2026-09-14 授权自行选择 CMS 剧目。固定选择:
|
||||||
|
|
||||||
|
- 剧名:Beneath the Ice
|
||||||
|
- drama_id:`06aa396d-175a-48ed-ad76-02786ef4d0a9`
|
||||||
|
- CMS 语言:en;已发布;总集数 5。
|
||||||
|
- 实际数据库:5 个分集均有非空片源记录,5 个分集均有英文字幕记录。
|
||||||
|
|
||||||
|
选择理由:集数较少、源和英文字幕记录完整,便于首次全季证据覆盖及受限下载验收。这不是剧情、视觉质量或广告合规的评判。
|
||||||
|
|
||||||
|
## 真实片源预检
|
||||||
|
|
||||||
|
运行 `scripts/verify-editor-source.mjs`,使用用户已授权的管理员凭据,仅隐藏 stdin 输入,不记录账号密码或签名 URL。
|
||||||
|
|
||||||
|
2026-09-14T02:10:23.661Z:第 1、3、5 集 CMS/admin 签名成功;签名 HLS manifest 全部 HTTP 200 且包含 `#EXTM3U`;英文字幕端点全部返回有效 WEBVTT 和时间轴。
|
||||||
|
|
||||||
|
只读预检 PASS,不等于渲染或全片审核 PASS。本次没有下载视频分片,没有向模型发送 CMS 内容,没有修改数据库或上传广告。用户不需要再选择剧目。
|
||||||
|
|
||||||
|
## 后续固定范围
|
||||||
|
|
||||||
|
先在批准的素材范围内接通下载/抽帧及 Qwen 图片分析,再建立 fresh analysis_id + drama_id + fetched_at 绑定的 analysis v2;不能将本预检冒充视觉证据或诊断。
|
||||||
|
|
||||||
|
2026-09-14 原始 Skill 已增加 `references/tiktok-adult-content-policy.md`,已读并补入网页 Skill 的受限资源清单。TikTok 官方成人内容页显示更新于 August 2026,本批已重新打开核对:<https://ads.tiktok.com/resources/help/article/tiktok-ads-policy-adult-content>。此时尚未选定任何广告视频片段,未生成内容审核 PASS;连续画面、音频与最终包装仍须按原 Skill 实审。
|
||||||
|
|
||||||
|
禁止用仅抽帧/字幕/机器 QA 代替完整带声观看。下载、参考研究、方案审批、渲染及审核能力仍待完成,不声明 Agent Studio 已能自动成片。
|
||||||
|
|
||||||
|
## 真实本地媒体验收(2026-09-14 11:50 CST)
|
||||||
|
|
||||||
|
`scripts/verify-editor-media.mjs` 已执行成功,任务 `job_893a9bfb-1a78-4ecf-816c-c9c2cb4b7310`,数据位于 `.agent-studio-runtime/videos/job-kuZwme`。
|
||||||
|
|
||||||
|
- 第 1 集:72.363 秒;第 3 集:84.694 秒;第 5 集:71.531 秒。
|
||||||
|
- 三集真实 TS 分片下载、本地 MP4 解复用、音视频流检测、SHA-256 和英文 WEBVTT 保存成功。
|
||||||
|
- 三集分别生成 3 张代表帧,合计 9 张,附实际时间和分集绑定。
|
||||||
|
- 修复 CDN 对每条子路径重签 auth_key 的兼容问题,保留同源和同集目录约束;不保存签名地址。
|
||||||
|
- 此次未向 Qwen 发送 CMS 字幕/图片;analysisValidated、renderingReady、humanReviewPassed 全为 false。
|
||||||
|
|
||||||
|
这是本地准备验收,不代表模型诊断或最终成片完成。
|
||||||
|
|
||||||
|
## 真实网页调用(2026-09-14 11:55 CST)
|
||||||
|
|
||||||
|
网页真实 Qwen 已读取原 Skill 和 CMS 契约,并在页面批准后完成 `prepare_drama_sources`。任务 `job_b1280a06-450e-4982-935a-b2bd8dfb726f`,analysisId `analysis_d1b59a1a-e3a9-4706-9e19-e393cf894b48`,fetchedAt `2026-09-14T03:54:29.848Z`。三集的 9 张 288×512 JPEG 在结果卡全部实际加载。此任务仅传递准备结果元数据,仍未向 Qwen 发送字幕或图片。
|
||||||
|
|
||||||
|
## 授权后的真实模型验收(2026-09-14 13:20 CST)
|
||||||
|
|
||||||
|
用户已批准向现有 Qwen 网关发送本剧 E1/E3/E5 英文字幕和九张代表帧,不包含整集上传或广告发布。本次网页 Session 为 `ses_15453ae6-eb84-4926-8bf1-50950fa3f70b`,实际 job 为 `job_9e5a84cb-2f1f-4fac-8221-1f0e15681f5c`,analysisId 为 `analysis_98be9661-7738-477b-8d7d-e4f8f705d084`,fetchedAt 为 `2026-09-14T04:54:32.866Z`。真实素材保留于 `.agent-studio-runtime/videos/job-rdUnOV`。
|
||||||
|
|
||||||
|
- 下载和 `read_drama_frames` 成功;网页分步观察已输出三集实际画面说明,引用九个真实 frameId/episodeId/time。
|
||||||
|
- 综合分析及随后的分析校验请求仍出现空终止回复;没有调用成功的 `submit_drama_analysis`,没有分析 PASS、候选审核或真实 10 秒 MP4。
|
||||||
|
- 独立 Agent 诊断能够处理同一九帧和字幕并返回正文(8192 输出预算;实际输出 2493 token)。这不能证明网页综合分析成功,尚未确定空回复根因。
|
||||||
|
- 后端事件适配源码已增加空正文/仅思考和输出额度耗尽的失败处理,避免宣告成功;三项新测试先失败再通过。全量 183 项测试及 lint/typecheck/build 通过,构建仍有 613.62 kB 主包提示。为保留内存中的当前素材任务,运行中后端尚未重启加载此修复。
|
||||||
|
- 更进一步的重复证据 + Skill 指令/校验脚本外传诊断被自动安全审批拒绝,需要用户明确批准后才能继续;未绕过拒绝。
|
||||||
|
|
||||||
|
模型上下文曾刷新,媒体任务和 CMS 登录保留;不把对素材传输的批准当作人已观看、已审核或已授权发布。
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
# Short Drama Editor:当前网页接入状态
|
||||||
|
|
||||||
|
2026-09-14。实现位于 `feature/agent-studio-build` 本地工作树,尚未声明完整剪辑能力。
|
||||||
|
|
||||||
|
## 已接通
|
||||||
|
|
||||||
|
- 发现本机原始 `swads-short-drama-editor`,按显式清单复制其指令、引用和脚本到会话只读快照;原 Skill 不变。
|
||||||
|
- 页面左侧 `CMS 连接与选剧`:Supabase shorts-sg 管理员登录、目录搜索、分页、选剧填入聊天。
|
||||||
|
- 密码只经本机 API 发往固定 Supabase 登录服务;不写文件、浏览器存储、聊天或模型工具。会话凭证留在后端内存。断开/重置清除本应用连接,不注销其他 CMS 客户端。
|
||||||
|
- 模型工具:`cms_connection_status`、`cms_list_dramas`、`cms_drama_inventory`、`cms_read_subtitle`。后三者需要页面审批,明确将所选内容发送给当前模型网关。
|
||||||
|
- `complete` 仅代表数据库目录及源记录齐全,`playbackVerified` 仍为 false,不能代替真实取片测试。
|
||||||
|
- `prepare_drama_sources`:单次最多 5 集 / 1 GiB 下载;固定 CMS/CDN、分路径签名、有限 VOD TS 清单、本地 FFmpeg 无网络解复用和 3 张/集代表帧。`editor_job_status` 返回本会话任务元数据;`read_drama_frames` 必须经另一次页面审批才将字幕和图片交给当前模型。
|
||||||
|
- 素材结果卡展示绑定分集和时间的本地代表帧。图片端点只接受已登记 Session/job/frame,禁止缓存;SSE 仅保留图片数量和元数据,不写入原图 base64 或原始字幕。
|
||||||
|
- `submit_drama_analysis` 将实际 CMS 分类、简介、分集、字幕原文和帧 ID/时间与本次不可变 analysis_id 对齐,调用原始分析校验器;失败不放行。
|
||||||
|
- `propose_drama_selection` 提交候选区间,网页提供带原声源视频和逐段审核记录。模型没有审核批准工具。
|
||||||
|
- `submit_editor_plan` 绑定已审核源区间和全部分析工件,调用原始 recipe/hook 校验器;候选参考先由人核实完整视频、来源、内容与证据,再运行原始参考评分。默认不启用 C 级参考。
|
||||||
|
- `render_editor_preview` 在版本审核及页面批准后调用原始渲染和 QA,输出 10 秒 REVIEW MP4。渲染前重新校验源文件 SHA-256;字幕由真实 VTT 映射,截断句子的边界阻断。播放器/下载支持登记文件的 HTTP Range,不接受任意文件路径。
|
||||||
|
|
||||||
|
## 使用
|
||||||
|
|
||||||
|
1. 在已启动的本地网页展开 `CMS 连接与选剧` 并登录,密码不要发进聊天框。
|
||||||
|
2. 搜索剧目,选定一部授权英文剧,再点击该 Skill 的使用入口或直接描述任务。
|
||||||
|
3. 审批前核对剧目、分集、语言与当前模型网关。下载到本机和发送模型分析是两次独立审批。
|
||||||
|
4. 分析通过后,点工具卡中的“打开剪辑审核与预览”。完整观看所选区间并听原声,填写真实逐段观察和政策核对记录。不能把勾选当作形式步骤,疑点未解决就要求更换完整场景。
|
||||||
|
5. 提供真实短剧营销参考证据,核对方案、完整参考与字幕。审核通过后在聊天输入“继续”,批准本次渲染。预览可播放/下载,但仍为 REVIEW,不允许当作已完成最终内容审核的广告或直接上传。
|
||||||
|
|
||||||
|
后端需通过启动环境设置 `CMS_SUPABASE_PUBLISHABLE_KEY`(公开客户端 key,不是 service_role);可通过 `scripts/start-private-server.mjs` 的隐藏输入字段 `cmsPublishableKey` 传入。邮箱和密码不能作为此脚本的启动配置。重启后须重新登录;新会话不能继承旧会话认证。
|
||||||
|
|
||||||
|
## 验证证据
|
||||||
|
|
||||||
|
- 最新素材流水线回归:162 项测试通过,lint、typecheck、build 通过;9 张代表帧已在真实网页实际加载。构建仍提示主 JS 包超过 500 kB,不影响当前验收。
|
||||||
|
- 146 项测试通过,lint、typecheck、build 通过;构建仍有主 JS 包超过 500 kB 的体积提示。
|
||||||
|
- 真实管理员登录及网页 API 首页面目录读取成功(30 项);没有修改 CMS。
|
||||||
|
- `scripts/verify-editor-api.mjs` 实测 PASS:走 Vite 网页代理,真实 Qwen 读取 Skill 和 CMS 契约,调用已连接状态工具并返回回复。测试不向模型发送 CMS 业务内容,结束后清除测试连接和会话。
|
||||||
|
- 浏览器实测显示 4 个 Skill,登录成功并显示真实目录;从聊天框发起检查后,两个 `read` 及 `cms_connection_status` 均显示完成,回复明确列出尚未接入的能力。旧错误页无法直接恢复,新测试页可正常访问并已保留。
|
||||||
|
- 修复 Vite 字符串代理默认 `changeOrigin: true` 导致 CMS 同源校验失败的问题,显式保留 Host;未放宽安全校验。
|
||||||
|
|
||||||
|
## 尚未接通,不能宣称完成
|
||||||
|
|
||||||
|
授权 HLS 下载、本地 FFmpeg 解复用及抽帧已完成真实验收。本轮已实现原始校验/渲染/QA、不可变审核记录与 REVIEW 下载,但尚未以真实剧集跑通整个网页链路。2026-09-14 用户已批准本次 Beneath the Ice 第 1/3/5 集英文字幕和九帧传输 Qwen;该批准不包含整集上传、发布或人工内容审核。真实参考证据仍需提供,没有配置网页自动公共搜索。最终完整带声审片及 final/ 导出没有自动放行功能。
|
||||||
|
|
||||||
|
本轮工程证据:180 项回归测试通过。独立测试使用明确标注的合成视频和测试证据串联原始脚本,生成 10 秒 1080×1920 H.264/AAC 字幕 MP4;这些测试数据不代表真实剧情、爆款参考或内容审核 PASS。测试覆盖未审核阻断、未知源、版本哈希、源文件变化、C 级参考阻断、字幕映射、私有视频 Range 及前端无默认勾选。
|
||||||
|
|
||||||
|
2026-09-14 12:48(上海)新版服务已重启,验证页新会话 `ses_f7cc79...3f75` 已用原授权账号重新登录 CMS。实际从网页发起只读检查,Qwen 成功读取原 SKILL.md 并调用 `cms_connection_status`,确认 `submit_drama_analysis` / `propose_drama_selection` / `submit_editor_plan` / `render_editor_preview` 已注册,公共参考搜索未配置。本轮网页检查未向模型发送剧情、字幕或图片,也未下载/渲染真实剧集。旧媒体文件保留,但旧会话任务不能自动续跑。
|
||||||
|
|
||||||
|
Qwen3.8-27B-FP8 已通过同一网关 Responses API 的双图识别测试,不需要另接视觉模型。随后已将当前网页会话设置为支持图片,并通过真实网页上传/附件 API/模型流式回复链路测试:附件中的蓝色三角形、绿色圆形、红色正方形及其左右顺序全部识别正确。测试只发送合成几何图,未发送 CMS 素材。
|
||||||
|
|
||||||
|
2026-09-14 已通过更新后的启动脚本重启本地后端,新会话默认启用已验证的 Qwen 图片能力;已在验证网页重新登录 CMS。重启会清除本应用会话认证,但不会修改 CMS 账户。
|
||||||
|
|
||||||
|
聊天附件识图已可用。`cms_connection_status` 已返回实际注册的媒体能力;能力已注册不等于当前任务已过审核。具体以 `editor_job_status` 和网页审核状态为准。完整验收仍需真实参考证据及逐段内容审核。不能用几何图测试或代表帧准备成功替代真实视频分析、成片审核或原 Skill 的 PASS 门槛。
|
||||||
|
|
||||||
|
## 本地缓存限制
|
||||||
|
|
||||||
|
2026-09-14 13:20 更新:本次授权后的九帧/字幕读取及网页画面观察成功;综合分析仍空回复,真实 MP4 尚未生成。排障与当前任务标识见 `editor-acceptance-title.md`。新增空回复/输出额度失败处理已通过 183 项测试及构建,但当前服务为保留内存任务尚未重启加载。进一步将 Skill 指令、校验脚本连同既定证据发送网关的重复诊断被自动安全审批阻止,等待明确范围授权。
|
||||||
|
|
||||||
|
每会话最多 3 次素材准备,全进程同时只运行一个重型任务;下载上限不含 MP4 解复用副本。素材保存在私有 `.agent-studio-runtime/videos/job-*`,失败任务和已完成媒体不会自动清除。当前仅保存元数据和文件,不支持重启后自动认领/续跑;跨会话全局磁盘配额与缓存清理界面尚未实现。
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
# 事件映射
|
||||||
|
|
||||||
|
## 三层映射
|
||||||
|
|
||||||
|
Pi 发出的事件首先经过 `PiEventAdapter` 变成经过裁剪和脱敏的 `HarnessEvent`。Web 的 reducer 再把有序事件投影为 assistant-ui 消息、工具和运行状态。Pi 原始对象不会直接进入浏览器。
|
||||||
|
|
||||||
|
| Pi / Harness 来源 | HarnessEvent | Web / assistant-ui 状态 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Session 创建 | `session.started` | 建立初始 timeline |
|
||||||
|
| HTTP message input | `user.message` | user message,附件仅为 IDs |
|
||||||
|
| Harness 开始 run | `run.started` | `isRunning=true`,显示停止按钮 |
|
||||||
|
| `message_update.text_delta` | `assistant.delta` | 按稳定 messageId 追加文本 |
|
||||||
|
| 最终 assistant `message_end`(非 toolUse / error) | `assistant.completed` | assistant message complete |
|
||||||
|
| `tool_execution_start` | `tool.requested` | 创建工具调用 part |
|
||||||
|
| `tool_execution_end` 成功 | `tool.completed` | 自定义工具使用 Pi result.details 展示指标和保存结果 |
|
||||||
|
| `tool_execution_end` 失败 | `tool.failed` | 工具卡显示安全错误 |
|
||||||
|
| Broker 请求 | `approval.required` | 打开服务器审批 Dialog |
|
||||||
|
| Broker 决策 / 超时 / 取消 | `approval.resolved` | 关闭 pending 状态 |
|
||||||
|
| AttachmentStore 成功 | `attachment.uploaded` | 保存安全附件元数据 |
|
||||||
|
| AttachmentStore 拒绝 | `attachment.rejected` | timeline / 错误状态 |
|
||||||
|
| ModelStore 配置 | `model.configured` | 更新安全模型摘要 |
|
||||||
|
| Harness 正常结束 | `run.completed` | run 和消息标为 complete |
|
||||||
|
| Harness 取消 | `run.cancelled` | 消息 incomplete/cancelled |
|
||||||
|
| Harness 失败 | `run.failed` | 消息 incomplete/error |
|
||||||
|
| Session 删除 | `session.ended` | 连接回到 idle |
|
||||||
|
|
||||||
|
## Skill 推导事件
|
||||||
|
|
||||||
|
`skill.requested` 和 `skill.loaded` 不是 Pi SDK 的原生事件。Harness 按下面条件推导:
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
A[Pi read start] --> B[realpath]
|
||||||
|
B --> C{是否匹配 SkillIndex}
|
||||||
|
C -->|否| D[只保留普通 tool 事件]
|
||||||
|
C -->|是| E[skill.requested]
|
||||||
|
E --> F{同一 toolCallId read 成功}
|
||||||
|
F -->|是| G[skill.loaded]
|
||||||
|
F -->|否| H[不发 loaded]
|
||||||
|
```
|
||||||
|
|
||||||
|
真实路径匹配避免仅凭用户提供的字符串伪造 Skill 活动。事件由 Session 队列串行处理,因此异步 `realpath` 不会导致 read end 越过 read start。
|
||||||
|
|
||||||
|
运行时索引匹配 Session 内的只读 Skill 正文快照,目录面板仍展示来源路径。模型尚未输出文本时,首个工具事件也会建立 assistant 消息,避免工具卡片一直隐藏到最终回复。
|
||||||
|
|
||||||
|
## 顺序、重放与去重
|
||||||
|
|
||||||
|
- 每个 Session 的 sequence 从 1 单调递增。
|
||||||
|
- SSE 初次连接可用 `afterSequence`,重连可用 `Last-Event-ID`。
|
||||||
|
- Server 先重放 cursor 之后的事件,再订阅新事件,并发送 heartbeat。
|
||||||
|
- Web 忽略 sequence 小于等于当前值的重复事件。
|
||||||
|
- sequence 出现缺口时,Web 不应用该事件,而是重置投影并从 0 重连。
|
||||||
|
- timeline 最多保留 500 条客户端状态,界面显示最近 100 条筛选结果。
|
||||||
|
|
||||||
|
## 数据最小化
|
||||||
|
|
||||||
|
事件负载经过 `redactAndLimit`。API Key、authorization、token 等敏感字段被脱敏;工具参数和结果有字节上限。图片事件只含 ID、MIME、字节数和尺寸,不含 base64。错误映射只保留明确 code 和有限长度的安全消息。
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 40 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 4.1 MiB |
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 1.1 MiB |
Binary file not shown.
@@ -0,0 +1,43 @@
|
|||||||
|
# 同图 2 秒网页复测
|
||||||
|
|
||||||
|
日期:2026-09-15,Asia/Shanghai。用户要求再生成一条2秒视频,沿用其提供的 IMG_5598.jpeg。只制作预览,不发布广告、不分配账户、不使用 CMS。
|
||||||
|
|
||||||
|
## 首次复测与修复
|
||||||
|
|
||||||
|
网页任务 `i2v_db33dcee-a002-4139-8d20-382d21b73b2e` 在独立图片审批后停于 initializing_upload。保存的真实回执为 `reused:true`、空 upload_url,并包含匹配图片哈希、格式、大小的 reused_file 和已有 asset ID。程序错误地要求新 PUT URL,因而报 unknown_upload。没有执行 PUT、完成上传或 H3 提交。原记录和回执保持不动。
|
||||||
|
|
||||||
|
修复:initialize 显式区分新票据和已有素材;复用必须核对 SHA256、MIME、字节数、image 类型、非空素材 ID。服务保存已核对素材后进入 awaiting_submission,不执行 PUT/finalize,也不自动生成。独立生成审批不变。两项回归先失败后通过;全量228项测试、lint/typecheck/build通过,独立只读复审无阻断问题。
|
||||||
|
|
||||||
|
重置会话被安全检查阻止,未执行;另开测试页,原页和任务保留。没有绕过删除保护、重置未知记录或运行恢复脚本。
|
||||||
|
|
||||||
|
## 修复后的正常网页流程
|
||||||
|
|
||||||
|
- 新任务:`i2v_e0785c65-afb1-45d6-9b57-3b6053921d77`。
|
||||||
|
- 标题:海边洞窟画作 · 2秒网页验收。
|
||||||
|
- 图片哈希沿用原图规范化结果,未更换素材;固定2秒、9:16、768p、H3 ref2va。
|
||||||
|
- 网页选图、填表、准备任务并发送自动填入的消息。
|
||||||
|
- Qwen 读取真实 Skill 和全部三份参考说明。
|
||||||
|
- 网页图片审批:`ap_a4bd5c08-4a63-492c-86ad-0d503fd26d86`,程序正确接收已有素材,不再传图片字节。
|
||||||
|
- 网页独立 H3 审批:`ap_f1d32143-9bc1-4709-9be3-d75fb5f0db9d`。
|
||||||
|
- 唯一真实生成任务:`cmjob-f7b9c8c644c6488e`,已保存原始提交回执和 job ID。
|
||||||
|
- 此阶段状态:running。后续只查询这个 job,不重投。
|
||||||
|
|
||||||
|
至生成提交阶段,新的验收任务没有人工改记录或后台恢复。下面记录实际收片结果,不能将此前阶段的无干预结论扩展到整个流程。
|
||||||
|
|
||||||
|
## 成片、原生帧数与最终结果
|
||||||
|
|
||||||
|
同一 job 成功,网页轮询自动收取 MP4。但原片为56帧、24fps,即2.333333秒,音轨2.304秒,超过旧0.25秒自动裁尾上限,故严格校验使任务失败。没有重新生成。
|
||||||
|
|
||||||
|
只读核对实际 H3 worker `h3_frame_count`:24fps 帧数向上对齐到 `17k+5`。2秒的48帧对齐后为56帧;此前10秒的240帧对齐后为243帧。修复只在24fps且实测时长匹配这个精确网格时放宽裁尾范围;其他输出保留原0.25秒上限,短片、音轨缺失、非整帧目标等继续拒绝。真实FFmpeg回归覆盖56帧裁至48帧,并证明原片保留、帧率不变和非网格超长输出被拒绝。
|
||||||
|
|
||||||
|
已失败的这一条使用 `recover-two-second-duration.mjs` 做**本地恢复**:停止本机服务器,核对任务、revision13、job、原片SHA256,备份原记录,调用生产裁切函数及原Skill验证,再注册成片。没有再次上传或生成。最终结果:
|
||||||
|
|
||||||
|
- 实际音视频均 **2.000秒**,768×1344,24fps,48帧。
|
||||||
|
- 原片 candidate.mp4 保留,SHA256 `61c84bd7a19c9025656810e6afb5f3954d9f3a6f9aa0a5078d7753a710862810`。
|
||||||
|
- 最终 video.mp4 为854,692字节,SHA256 `e3fd8717f80eeb350e1e281fcb505fb7bf145605225137761c91f3a06ab399b1`。
|
||||||
|
- 原Skill完整解码、时长、音轨及音量技术检查通过,声音内容未试听,不标记audio reviewed。
|
||||||
|
- 5个时点画面抽查未见上一条新增的真实手持画作镜头,保留整幅画及竖屏留白,变化较轻微;不作正式素材审核通过声明。
|
||||||
|
- 网页播放器实际加载duration2、readyState4、errornull,可正常启动播放;下载HTTP200、video/mp4、正确attachment文件名及字节数。
|
||||||
|
- 最终228项全量测试、lint/typecheck/build通过,git diff --check通过;复用与原生帧数修复均经独立只读复审,无阻断问题。
|
||||||
|
|
||||||
|
**验收结论:2秒视频已交付,网页播放/下载可用;本轮仍经过收片后的本地恢复,不是无干预全链路验收通过。** 修复已进入正常程序,但没有为了再次证明无干预而追加生成任务。
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# 图生视频接入验收记录
|
||||||
|
|
||||||
|
日期:2026-09-15(Asia/Shanghai)。本页区分工程验证与真实生成结果。
|
||||||
|
|
||||||
|
追加2秒复测:已交付2.000秒音视频,并补修同图去重返回及H3 `17k+5`原生帧数处理。网页审批、真实提交、自动下载成功;收片校验后仍需一次本地恢复,因此尚不能声称全程无干预。最终228项测试及lint/typecheck/build通过。详见 image-to-video-2s-web-test.md。
|
||||||
|
|
||||||
|
最新现场结果:用户图片已生成真实10秒视频,任务 cmjob-894ca2faddd34ff7。Agent Studio 已自动收取原片,经原速裁掉0.125秒超长尾帧后,最终音视频均10.000秒、768×1344、24fps。网页播放器已实际播放到片尾,下载接口返回200。恢复有操作者干预,不是无干预的全网页成功;画面有模型添加手持画作的偏差,作为接入测试预览交付,不是正式素材质量通过。详情见 image-to-video-live-artwork-test.md。
|
||||||
|
|
||||||
|
## 已验证
|
||||||
|
|
||||||
|
- 当前页面 Skills →「图片 → 有声视频」可接收本地单张图片,固定图片、提示词和目标时长,创建本地草稿并填入聊天输入框。
|
||||||
|
- 当前配置的 Qwen 在真实运行中读取原始 swads-image-to-video Skill 和 references/swads.md;流程不调用 CMS。
|
||||||
|
- 用明确标注 LOCAL UI TEST 的合成图验证了网页审批:外传对话框展示准确缩略图、提示词、10 秒目标、H3 ref2va、9:16、768p 和入库副作用;确认未默认勾选。
|
||||||
|
- 拒绝上传后模型停止,草稿仍在本机。刷新页面和重启后端后,同一草稿及缩略图仍可查看。
|
||||||
|
- 347px 实际侧栏宽度下的新表单布局已验证无横向裁切。
|
||||||
|
- 当前 MCP 的只读工具发现与身份查询通过,权限含生成/上传。本次已实际上传参考图、完成入库、尝试一次生成提交;未分配账户、审核或发布。
|
||||||
|
- 原 Skill 视频校验器在本地合成 10 秒测试视频上通过完整解码、音轨、尺寸、帧率、时长和哈希检查;视听人工审核仍为 pending。这不是 H3 成片。
|
||||||
|
- 修复了提交回执写入后、主记录更新前中断的恢复窗口。自动化测试证明匹配回执恢复原任务查询、不重投;冲突或无效回执保持隔离。
|
||||||
|
- 临时取片失败可在重启后继续查询原 job、收取新的 artifact;确定性媒体验证失败仍阻断。恢复与取片问题均已独立复审通过。
|
||||||
|
- 初始实现验证为218项测试;本次新增上传参数、错误识别与回执留存回归。已有主包超过 500 kB 提示在此前验收文档中也存在。
|
||||||
|
- 本次修复最终验证:223项全量测试通过;lint/typecheck/build退出0;类型修正后24项相关回归再次通过,git diff --check通过。独立只读复审未发现阻断问题。已重启加载最新修复版本,未知提交继续保留,未重投。
|
||||||
|
|
||||||
|
## 尚未验证
|
||||||
|
|
||||||
|
用户参考图及两项审批、上传入库、真实生成、实际10秒成片、完整解码、MP4下载和网页播放均已核实。尚未核实实际费用和声音内容;画面抽查已发现手持画作/运镜偏离提示词,不作效果合格声明。
|
||||||
|
|
||||||
|
旧超长标识请求经后台核查确认未创建任务;修复后的真实任务已完成并交付网页预览,没有重复生成。原片保留,最终10秒由保留原速的短尾裁切得到。LOCAL UI TEST 草稿禁止外传;旧初始化失败任务保持隔离。最新全量226项测试、lint/typecheck通过,独立复审无阻断问题。
|
||||||
|
|
||||||
|
## 本地使用
|
||||||
|
|
||||||
|
打开 http://127.0.0.1:5173/,进入 Skills 的「图片 → 有声视频」,选择参考图并填写要求。点击「准备本地任务」后,在聊天中发送已填好的消息,逐项核对图片外传和生成审批。无需 CMS 登录。遇到提交状态未知时不要新建任务盲目重试。
|
||||||
|
|
||||||
|
原有工作树和未提交修改均保留;本次新增实现与修复记录为 40e03e3、cc80265、2ad089d,既有脏文件中的接线增量仍保留在工作树中,不能仅靠这些提交重建整个运行版本。
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
# Cat upload recovery — 2026-09-16
|
||||||
|
|
||||||
|
## Scope and evidence
|
||||||
|
|
||||||
|
- Original task: `i2v_6d07ef38-e230-4da3-b337-435c65cb912d`, title `test`, exact duration 3 seconds.
|
||||||
|
- Frozen reference SHA-256: `5a4220f1fc37a110f3801bbc35119921ec7871b4c94d2233078af20951f4c880`.
|
||||||
|
- Original initialization failed with upstream HTTP 500 at 2026-09-15T09:23:36.444Z. No upload ticket, asset or generation job was stored locally. The historical server-side root cause remains unconfirmed.
|
||||||
|
- On 2026-09-16 immediately before recovery, a READ ONLY query of `direct_upload_tickets`, scoped to this exact content hash and original tenant, returned zero matching rows. This is current reconciliation, not proof that no historical ticket ever existed.
|
||||||
|
- The original record (revision 2), original error receipt, owner, frozen prompt, duration and reference bytes were preserved. An exclusive one-shot recovery marker and backup were written privately. No token or signed storage URL is recorded in this document.
|
||||||
|
|
||||||
|
## Executed recovery
|
||||||
|
|
||||||
|
`scripts/recover-confirmed-cat-upload.mjs` performed one user-authorized upload attempt with the local server stopped, using the production image-video service and adapter. It fails closed on changed task/revision/hash/identity, stale reconciliation, an existing recovery marker, or an online local server. Its remote wrapper prohibits generation submission.
|
||||||
|
|
||||||
|
- Initialize: success, 2026-09-16T04:11:36.699Z.
|
||||||
|
- Image PUT: completed; finalization receipt returned `finalizing` at 2026-09-16T04:11:37.317Z.
|
||||||
|
- Normal server read/refresh: `awaiting_submission` at 2026-09-16T04:12:41.023Z.
|
||||||
|
- Browser: existing page reconnected and Skills panel showed the cat task as `参考图已入库 · 待生成审批`, with original title, prompt and 3-second target.
|
||||||
|
- H3 submission: **not performed**. No generated cat video is claimed.
|
||||||
|
- Focused regression run: 3 files / 29 tests passed (`image-video.test.ts`, `image-video-mcp.test.ts`, `image-video-duration.test.ts`).
|
||||||
|
|
||||||
|
## Continuation repair — completed 2026-09-16
|
||||||
|
|
||||||
|
Session registry and browser chat are in-memory. After restart the task remains visible, but its original session no longer exists. The new explicit continuation flow preserves the existing tool owner checks; it does not silently reassign tasks or restore old chat history.
|
||||||
|
|
||||||
|
After user design approval, implemented `继续此任务` for local drafts and references awaiting generation. Clicking it shows an unchecked local-session continuation confirmation. The strict JSON, local-origin route validates the live idle target, original owner inactivity, frozen revision, state and reference hash. Ownership history is saved privately and all original image/request/upload/asset bindings remain intact. Unknown results and already-submitted jobs cannot be reset through continuation. The model has no continuation tool.
|
||||||
|
|
||||||
|
After confirmation, the UI fills (but does not send) the original task's chat request. An already-imported reference requests only status verification followed by separate H3 generation approval, not another upload. Late responses are discarded on session changes; the target is revalidated within the task lock. If the original session is still active, return to its existing page instead of transferring it.
|
||||||
|
|
||||||
|
Live browser verification on the user's cat task:
|
||||||
|
|
||||||
|
- Confirmation button initially disabled until the checkbox was checked.
|
||||||
|
- Local continuation succeeded, revision 9 → 10, one ownership-history entry.
|
||||||
|
- State remained `awaiting_submission`, no generation job appeared.
|
||||||
|
- Upload ID, asset ID, reference hash and request hash were unchanged.
|
||||||
|
- Remote receipt count remained 3; no additional upload/finalize/submit.
|
||||||
|
- Browser returned to Chat with exact original 3-second request filled in and Send enabled; no message was automatically sent.
|
||||||
|
- Regression verification: 236 tests passed (13 shared, 152 harness, 46 web, 25 server); typecheck, lint and build passed. Existing web chunk-size warning remains.
|
||||||
|
- Bounded independent review found two session-lifecycle races; both received failing regression tests and fixes, and follow-up review reported no blockers.
|
||||||
|
|
||||||
|
This verifies continuation, not unattended image-to-video acceptance or a proven root-cause fix for the historical upstream 500. Cat video generation still requires the user's separate approval.
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
# User artwork live H3 test
|
||||||
|
|
||||||
|
2026-09-15 Asia/Shanghai. User supplied IMG_5598.jpeg and explicitly asked to test video generation after discussion of SW Ads upload + H3 generation. Only this supplied image is authorized; old synthetic drafts are not.
|
||||||
|
|
||||||
|
Target: one10-second H3 ref2va video through Agent Studio UI, existing9:16/768p defaults. Preserve original painted composition with neutral margins if necessary; subtle flags/sails/water/guest movement; sea and breeze ambience, no dialogue/music. Do not change duration/provider or submit another job without a confirmed safe reason. No account assignment, review approval or advertisement publication.
|
||||||
|
|
||||||
|
Task: i2v_2ef50a3d-c9ce-4264-b484-539e0122bb67. Title: 海边洞窟画作 · 10秒实测.
|
||||||
|
|
||||||
|
Original photo kept unchanged. Local webpage attachment normalized and copied into the registered durable task. This record currently proves local draft/Skill request only, not upload or H3 success.
|
||||||
|
|
||||||
|
Read-only video_h3_workers: four online idle workers before submission. Availability does not prove duration support or cost.
|
||||||
|
|
||||||
|
## Actual result: FAILED before reference transfer / H3 submission
|
||||||
|
|
||||||
|
- Web form created the registered task and Qwen read the actual Skill and all three references.
|
||||||
|
- Confirmed the exact supplied artwork/10-second prompt in the separate upload approval and approved once. No H3 generation approval was requested or granted.
|
||||||
|
- Result became unknown_upload. On-disk record revision2 has no upload receipt, assetId or jobId. State transitioned from initializing_upload to unknown_upload before an upload receipt could be parsed/saved; the sequential put() stage was therefore not reached. Whether a remote initialization ticket was reserved is unknown.
|
||||||
|
- Read-only current MCP listTools + exact submitted-argument validation passed. Normalized local reference is JPEG,2571635bytes. This diagnostic did not invoke upload or generation.
|
||||||
|
- Existing code discards the original remote/response-parser error, so the specific upstream or parser cause cannot be recovered from the task record. No blind initialization/import retry, new task or alternate-provider call was made.
|
||||||
|
- This is a real integration failure, not a video-generation success. The previous automated/boundary tests did not validate this actual initialization response. Repair needs sanitized private error/receipt preservation, actual response-contract diagnosis and another controlled same-image acceptance run.
|
||||||
|
|
||||||
|
## Repair investigation
|
||||||
|
|
||||||
|
User authorized repair. One controlled metadata-only diagnostic initialization used the same normalized reference hash and a distinct `-diagnostic-1.jpeg` filename. Actual tool response: `{"error_code":"invalid_arguments","detail":"push begin needs: ratio, prompt"}` with MCP `isError:false`; no upload ticket was returned and no picture bytes or H3 request were sent.
|
||||||
|
|
||||||
|
Root cause: live input JSON Schema marks these fields optional, but direct-upload begin requires ratio and prompt at runtime. The adapter omitted both and treated the provider error body as a successful payload, then lost the error during receipt parsing. The fix sends the frozen prompt and existing 9:16 ratio on begin/finish, recognizes provider error_code responses, and persists a sanitized failure stage/code without exposing remote details. Three new regression cases failed before the fix and passed afterward. Original unknown task remains quarantined; controlled new same-image acceptance run follows, not a reset of the original record.
|
||||||
|
|
||||||
|
## Controlled repair acceptance
|
||||||
|
|
||||||
|
New browser-created task: `i2v_4d14f754-c120-4a1d-a1cc-cc39af4bb4e7`, title 海边洞窟画作 · 修复验收10秒. Same user reference, same normalized hash, prompt and 10-second target. Actual Skill and references read; independent upload approval accepted in browser.
|
||||||
|
|
||||||
|
Initialization and image-byte PUT succeeded. Saved upload ID `du-f9eae1fb99e045f6`. Finish exposed a second runtime requirement: `{"error_code":"invalid_arguments","detail":"push finish needs: file_name, content_type, size_bytes"}`. Read-only lookup of this exact upload returned initiated, matching filename/hash/2571635 bytes and no asset ID. No second image upload was performed.
|
||||||
|
|
||||||
|
Fix now shares identical file metadata between begin and finish, passing the actual validated bytes through service.finalize. Regression test reproduced this finish rejection before the fix and passes afterward. A controlled operator recovery completed this same upload ticket with corrected metadata and returned finalizing. This was an explicit repair intervention, **not an uninterrupted all-web success**. The browser's own i2v_status then recovered the existing task to awaiting_submission. No record reset, ownership reassignment or alternate-provider request occurred.
|
||||||
|
|
||||||
|
## Earlier blocker: H3 submission outcome unknown (reconciled below)
|
||||||
|
|
||||||
|
The configured Qwen Responses connection returned MODEL_EMPTY_RESPONSE on three continuations. Switching the same gateway/model to Chat Completions (and therefore refreshing the model conversation context) restored actual Skill reads, status tool calls and the independent H3 approval. This does not establish which of protocol/context caused the original empty response. No provider or model change was made.
|
||||||
|
|
||||||
|
One H3 generation approval was accepted in the browser, but i2v_submit returned unknown_submission without a job ID. The deployed submit catch still discarded the specific response/parse failure, so no claim that H3 accepted, rejected, or generated a video is supported. Original 10-second parameters retained; no new submission or automated retry followed. Workers being idle does not prove no job was created. Available read-only tools do not expose lookup by idempotency key; harness_run_inspect requires a known run ID or flow kind and did not resolve this task.
|
||||||
|
|
||||||
|
Reconciliation identifiers (not credentials):
|
||||||
|
- Task: i2v_4d14f754-c120-4a1d-a1cc-cc39af4bb4e7
|
||||||
|
- Reference asset: casset-01ddea3b9bb14c2f
|
||||||
|
- Submit outcome timestamp: 2026-09-15T05:50:46.577Z
|
||||||
|
- Idempotency key: i2v_4d14f754-c120-4a1d-a1cc-cc39af4bb4e7/ba340e1bc788b28d9d54bcb7861027df1197135de3b08556e24d8a32c39a9d06
|
||||||
|
|
||||||
|
Follow-up repair saves original mutation responses in private 0600 per-task files before interpreting them, binding operation/revision/request hash; public messages contain only controlled stage/error codes. Regression reproduced missing receipt persistence and sanitized submit diagnostic failure before fixes. At this checkpoint no video had been delivered and the task remained quarantined pending backend reconciliation.
|
||||||
|
|
||||||
|
## Confirmed rejection and controlled recovery — 2026-09-15
|
||||||
|
|
||||||
|
The operator located the actual SW Ads gateway source and performed a narrowly scoped READ ONLY query against the corresponding service database. Matching the original idempotency key or reference asset within the correct organization returned **0 media jobs**. Only after that verification, a same-original-key diagnostic returned `invalid_arguments: idempotency_key must be at most 87 characters`. The gateway adds a caller namespace within a 128-character limit and rejects overlength keys before job creation. No duplicate generation was created by this diagnostic.
|
||||||
|
|
||||||
|
The adapter now sends a 68-character key containing a full SHA-256 binding of task ID and frozen request hash. Regression first reproduced the rejection, then proved accepted submission, stable duplicate identity and distinct identities for different tasks/requests. Existing unknown-submission guards were not relaxed.
|
||||||
|
|
||||||
|
`scripts/recover-confirmed-artwork-submit.mjs` performed one exact-task recovery with the local server stopped: required revision14/unknown state/no job/exact asset and hashes; saved the original record and reconciliation evidence privately; preserved owner and draft; reused the existing image asset without another upload; submitted through the production service. It is a one-shot operator intervention, not evidence of uninterrupted all-web success.
|
||||||
|
|
||||||
|
Confirmed real job: **cmjob-894ca2faddd34ff7**. Receipt and main task record are saved. The restarted Agent Studio shows this same job as generating and polls it without resubmission.
|
||||||
|
|
||||||
|
The real backend artifact contract uses `storage_url`, not `url`. Another red-green regression covers registered MP4 collection alongside JSON metadata and blocks conflicting URLs or foreign-job artifacts.
|
||||||
|
|
||||||
|
## Actual delivered preview and browser verification
|
||||||
|
|
||||||
|
The same job succeeded. The normal server collected its MP4 automatically, but strict validation correctly rejected the provider's 10.125-second video / 10.112-second audio (243 frames at 24fps, 768×1344). No second generation was submitted.
|
||||||
|
|
||||||
|
The collector now handles a bounded provider frame overrun of at most0.25seconds: it requires sufficient audio/video, compatible constant frame timing, trims only the tail at original speed, preserves the provider MP4, and applies the original Skill's full technical verification to the result. It never stretches, loops, pads missing time or accepts a too-short output. A real-FFmpeg regression exercises original-speed frame count, exact duration, original-byte preservation, and short/excessive-output rejection.
|
||||||
|
|
||||||
|
For this already-failed task, the stopped-server local-only `recover-artwork-duration.mjs` checks exact revision28/job/source hash, backs up the record, processes the registered local candidate, and registers the verified result. This is a second explicit operator recovery, not an uninterrupted all-web claim.
|
||||||
|
|
||||||
|
- Original `candidate.mp4`: SHA256 e7f0597eede161024de72aed6106e513f6972a7b7006ff40dcdbdea90cee583d, retained unchanged.
|
||||||
|
- Final `video.mp4`: SHA256 26a7b49e5f49b9b5507340669dc902ebd1583ebfcad06fd724e8831c8afd06c2, 9,772,338bytes.
|
||||||
|
- Actual video **10.000s**, actual audio **10.000s**, **768×1344,24fps**, complete decode and original Skill audio checks passed.
|
||||||
|
- Agent Studio task is `ready` and displays player/download. Browser metadata: duration10,readyState4,errornull. Keyboard-started playback reached currentTime10,endedtrue,errornull.
|
||||||
|
- Local download endpoint returned HTTP200,video/mp4,attachment disposition and the exact final byte count.
|
||||||
|
- Contact sheet sampled five moments. **Creative deviation:** H3 introduced real hands handling the painting and camera movement; it did not follow the intended fixed-camera subtle animation inside the painting. This is an integration preview, not a creative-quality pass. Audio content has not been listened to; no claim of full audio review.
|
||||||
|
- Final full verification before copy-only UI text update: **226 tests**, lint and typecheck passed; build passed. Independent read-only review found no blocking issue in key/artifact handling or duration processing. The one-shot credential helper now checks terminal echo is disabled; the actual credential entry already used disabled echo.
|
||||||
|
|
||||||
|
Result: actual supplied image → real H3 job → automatic local collection → exact-duration local processing → web playback/download demonstrated, with the operator repair interventions and visual limitation explicitly recorded. No CMS, account assignment, review approval, publication or duplicate generation.
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
# SW Ads image-to-video in Agent Studio
|
||||||
|
|
||||||
|
The Skills pane contains “图片 → 有声视频”. Add one JPEG, PNG or WebP reference, a title, the exact motion/sound prompt and a duration (10 seconds initially). “准备本地任务” receives the image through the existing Session attachment validator, copies normalized image bytes to private durable storage, and fills the chat composer with the registered task ID. Send that prepared message to use the original `swads-image-to-video` Skill. No CMS login or vision-enabled chat model is required. The local thumbnail itself is not transmitted to the chat model.
|
||||||
|
|
||||||
|
Two separate page approvals are required. Reference upload sends the frozen image to SW Ads storage and creates a `review_pending`, unassigned reference asset. Generation sends that asset and the frozen prompt to H3 `ref2va`, 9:16, 768p with the exact requested duration. Each dialog loads the registered image, prompt and duration and starts with an unchecked confirmation. Refusal causes no operation of that type. Costs and arbitrary duration support are not assumed. There is no provider fallback, automatic repair, account assignment, moderation approval or publication.
|
||||||
|
|
||||||
|
Only four controlled tools are exposed: `i2v_create_task`, `i2v_upload_reference`, `i2v_submit` and `i2v_status`. The latter three accept only a task ID owned by that Session. The analytics MCP allowlist remains separate. The server discovers current input schemas and the current MCP account’s `tenant_id` and `platform`; ambiguity blocks upload. Original Skill snapshots include SKILL.md, three references and scripts/media.py.
|
||||||
|
|
||||||
|
The durable history is stored under `AGENT_DIR/image-video` (default `.agent-studio-runtime/image-video`). It persists independently of chat Sessions. Returning to the local page lists registered tasks and polls at most every 30 seconds per task, including after restart. Old tasks cannot be reassigned to a new Session for mutation. Only one server process may own a runtime directory. Each operation serializes its task, validates a strict versioned record and reference hash, and atomically persists revisioned intents before network mutations. Unknown upload/submission outcomes are quarantined; never delete their records and retry blindly. A returned generation receipt is retained separately before updating its task state, so stale results cannot silently overwrite a newer revision.
|
||||||
|
|
||||||
|
The local history and media routes are `/api/image-video/tasks`, `/api/image-video/tasks/:taskId`, and the registered `reference` / `video` endpoints. Draft creation is `/api/sessions/:sessionId/image-video/tasks`. All require a loopback Host with an allowed local port, reject cross-site requests and mismatched Origin, and media/history are `no-store`. No endpoint accepts a remote media URL or arbitrary filesystem path. Media supports HTTP Range and download.
|
||||||
|
|
||||||
|
MP4 collection uses the registered job’s single `video/mp4` artifact only, public HTTPS DNS validation with the actual connection pinned to the validated IP, no redirects, bounded bytes and elapsed time. The original Skill verifier runs with every FFmpeg/FFprobe subprocess restricted to file/pipe protocols and the MP4 demuxer. Full decode, sound presence, portrait geometry, positive frame rate and exact video duration are checked; audio container packet rounding has a 50 ms tolerance. A local SHA-256 binds the validated file to the registered player/download endpoint. A successful technical preview remains “待视听检查”; no human visual or audio approval is inferred.
|
||||||
|
|
||||||
|
Server configuration: `SWADS_MCP_TOKEN` stays server-only; optional `IMAGE_VIDEO_SKILL_DIR` overrides the original Skill directory; `PYTHON_EXECUTABLE` may select the installed local Python. Existing bundled FFmpeg/FFprobe are reused. Never place credentials in task prompts or browser input.
|
||||||
|
|
||||||
|
Validation boundary: automated tests use synthetic local references and fake provider boundary responses. Read-only live discovery confirms current input schemas and identity structure; current tools do not advertise output schemas. The adapter conservatively accepts documented top-level upload fields (`upload_id`, `object_key`, `upload_url`), import `creative_factory_asset_id`, submit `job_id`, and status `status` plus `artifacts[{content_type,url}]`; unsupported or ambiguous envelopes stop safely. Actual upload/finalization/status envelopes must still be checked with an explicitly approved live reference. A real target-duration H3 generation, actual browser playback, download hash match and human sound/visual review are required before claiming end-to-end real generation success. Synthetic test video and jsdom component evidence do not satisfy that final gate.
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 43 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 26 KiB |
@@ -0,0 +1,40 @@
|
|||||||
|
# 真实模型联调验收
|
||||||
|
|
||||||
|
日期:2026-09-09。本地开发版本,未部署到公网。
|
||||||
|
|
||||||
|
## 使用的连接
|
||||||
|
|
||||||
|
- 模型:`Qwen3.8-27B-FP8`。
|
||||||
|
- 协议:OpenAI Responses。
|
||||||
|
- 网关:`https://gateway.pencil-stub.com/v1`。
|
||||||
|
- API Key 仅在当前 Session / Pi 凭证内存中使用,没有写入源代码、配置文件或 Git。
|
||||||
|
- 仅验收文本能力;未确认该模型的图片能力,当前连接配置为 text-only。Anthropic 协议未做完整链路验收。
|
||||||
|
|
||||||
|
## 实测结果
|
||||||
|
|
||||||
|
- 简单文字请求返回“连接成功”,SSE 产生文本增量和正常结束事件。
|
||||||
|
- `ads-analysis`、`report-writer` 均产生 `skill.requested` 和 `skill.loaded`。
|
||||||
|
- `mock_ads_metrics` 返回演示数据;页面显示 Spend 10000、Revenue 21800、ROAS 2.18。
|
||||||
|
- 页面出现待审批对话框。点击“批准并保存”后观察到 `approval.resolved: approved`,继而 `save_report_draft` 成功。
|
||||||
|
- 最终报告实际写入 Session 的 `report.md`,879 字节;包含五个章节并注明模拟数据。
|
||||||
|
- 模型返回“保存成功:report.md”,运行正常结束。
|
||||||
|
- 从页面输入并发送测试消息后,真实模型返回“页面发送正常。”;页面未记录 console error。
|
||||||
|
|
||||||
|
验收 Session:`ses_ffbe9bca-275c-4004-9fa2-271c2db9cc1a`。该标识仅用于本次本地验收,不是持久会话;重启后不能据此恢复内存数据。
|
||||||
|
|
||||||
|
## 修复与回归
|
||||||
|
|
||||||
|
- 将 Skill 正文复制到会话内只读快照,并同步运行时索引,保持原有路径守卫。
|
||||||
|
- 将 Pi 自定义工具的 `result.details` 映射给页面工具卡。
|
||||||
|
- 不再将工具调用轮次结束误报为整个回复完成;模型错误不再被当成正常完成。
|
||||||
|
- 模型先调用工具、尚未输出文本时,页面仍即时显示工具卡。
|
||||||
|
- 本地 Session 和 runtime 目录加入 Git 忽略。
|
||||||
|
|
||||||
|
最终检查全部通过:127 条测试(shared 13、harness 71、server 17、web 26)、生产构建、类型检查、Lint、`git diff --check`。构建仍有主 JS 包超过 500 kB 的体积提示,不影响本次功能验收。
|
||||||
|
|
||||||
|
## 仍需注意
|
||||||
|
|
||||||
|
- 广告数据是内置 fixture,不是真实投放平台数据。
|
||||||
|
- 本轮首次较长报告在结尾出现不完整句子;最终验收改用短报告并确认完整输出。长报告仍需控制长度或提高模型输出上限,并人工审阅数字与结论。
|
||||||
|
- Skill 快照目前只包含 `SKILL.md`,额外引用、脚本和资源文件没有自动开放。
|
||||||
|
- 保留当前页面可继续使用已配置会话;刷新页面、重置会话或重启服务后可能需要重新配置。
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
# 从课堂 MVP 到生产 Harness
|
||||||
|
|
||||||
|
## 目标架构
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart LR
|
||||||
|
Web --> Gateway[Gateway / Auth / CSRF / Rate Limit]
|
||||||
|
Gateway --> API[Stateless API]
|
||||||
|
API --> DB[(Postgres)]
|
||||||
|
API --> Redis[(Redis Streams / Locks)]
|
||||||
|
API --> KMS[KMS / Secret Manager]
|
||||||
|
API --> Blob[Object Storage]
|
||||||
|
Redis --> Workers[Agent Workers]
|
||||||
|
Workers --> Sandbox[Per-run Container / microVM]
|
||||||
|
Sandbox --> Egress[Egress Proxy / Allowlist]
|
||||||
|
Workers --> Audit[Immutable Audit Log]
|
||||||
|
Redis --> SSE[Distributed SSE Gateway]
|
||||||
|
SSE --> Web
|
||||||
|
```
|
||||||
|
|
||||||
|
## 阶段 1:身份与持久化
|
||||||
|
|
||||||
|
- 引入用户、组织、Session 所有权与 RBAC。
|
||||||
|
- Postgres 保存 Session、Run、Message、Event、ToolCall 和 Approval 的不可变记录。
|
||||||
|
- 以 `(session_id, sequence)` 唯一约束保证事件顺序;命令使用幂等键。
|
||||||
|
- Redis 保存短期锁、队列和 fan-out,避免同 Session 多 Worker 并发。
|
||||||
|
- 为迁移和清理任务定义明确的数据保留周期。
|
||||||
|
|
||||||
|
## 阶段 2:凭证与 Web 安全
|
||||||
|
|
||||||
|
- API Key 进入 KMS / Secret Manager,应用只持有短期解密结果。
|
||||||
|
- OAuth 或短期 provider token 优先于长期静态密钥。
|
||||||
|
- Secure、HttpOnly、SameSite cookie;CSRF token;严格 CSP 与 frame-ancestors。
|
||||||
|
- 对 Session、模型测试、消息、上传和审批分别限速并设置租户配额。
|
||||||
|
- 所有错误响应使用稳定 code,不回显 provider body 或敏感 headers。
|
||||||
|
|
||||||
|
## 阶段 3:文件与执行隔离
|
||||||
|
|
||||||
|
- 原图和规范化图进入私有对象存储,使用租户前缀、服务端加密和生命周期规则。
|
||||||
|
- 图片扫描、解码超时、内存限制和异步规范化。
|
||||||
|
- 每个 run 在低权限容器或 microVM 内执行;只挂载该 Session 工作区。
|
||||||
|
- 使用只读根文件系统、CPU/内存/文件数/时长配额、seccomp 和无特权用户。
|
||||||
|
- 出站流量经过代理,只允许已批准的模型域名;禁止 metadata 和横向内网访问。
|
||||||
|
|
||||||
|
## 阶段 4:可靠 Agent 运行
|
||||||
|
|
||||||
|
- Durable queue 驱动 run 状态机,Worker 崩溃后可重试或终止。
|
||||||
|
- Approval request 持久化并使用 compare-and-set 解决重复、冲突和超时竞态。
|
||||||
|
- 工具 side effect 使用幂等键或事务 outbox;审批不等于可以重复执行。
|
||||||
|
- 版本化 Prompt、Skill、模型配置和工具 schema,以便重放与审计。
|
||||||
|
- 对 token、费用、延迟、tool error、approval time 和 cancellation 建立指标。
|
||||||
|
|
||||||
|
## 阶段 5:分布式事件与审计
|
||||||
|
|
||||||
|
- Redis Streams、Kafka 或数据库 outbox 发布事件。
|
||||||
|
- SSE Gateway 按租户鉴权、从持久 cursor 重放,并支持跨实例 fan-out。
|
||||||
|
- 审计日志记录操作者、决策、策略版本、输入摘要和 side effect,不记录秘密或原始敏感内容。
|
||||||
|
- 定期验证删除任务确实清除了数据库、对象存储、缓存、凭证与 Sandbox volume。
|
||||||
|
|
||||||
|
## 上线门槛
|
||||||
|
|
||||||
|
生产发布前需要威胁建模、依赖与镜像扫描、SSRF/路径/上传模糊测试、故障注入、备份恢复演练、租户逃逸测试和第三方安全评审。课堂测试套件应保留,并新增真实 provider 的隔离式 smoke test;后者不能进入默认 CI,也不能使用个人密钥。
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
# 安全边界
|
||||||
|
|
||||||
|
## 信任模型
|
||||||
|
|
||||||
|
浏览器输入、模型输出、工具参数、上传文件名、MIME、Base URL、Skill 额外路径和 SSE cursor 都是不可信输入。Zod / TypeBox 负责结构校验,Harness 负责语义与权限校验,操作系统仍是最终权限边界。
|
||||||
|
|
||||||
|
## 已实现的课堂版保护
|
||||||
|
|
||||||
|
### 凭证
|
||||||
|
|
||||||
|
- Session 配置覆盖服务器环境变量;没有配置时明确报错。
|
||||||
|
- API Key 只保存在 Server 内存和 Pi `InMemoryCredentialStore`。
|
||||||
|
- 响应只返回 `keyHint`,不会返回原值或长度。
|
||||||
|
- 不写浏览器 Storage、日志、事件、URL、Pi 用户目录或仓库文件。
|
||||||
|
- 运行中禁止修改模型;清除凭证幂等。
|
||||||
|
|
||||||
|
### Base URL / SSRF
|
||||||
|
|
||||||
|
- 使用 WHATWG URL,默认仅 HTTPS。
|
||||||
|
- 拒绝 userinfo、fragment、非 HTTP(S)、异常端口和含凭证 query。
|
||||||
|
- DNS 结果若命中 loopback、private、link-local、unspecified、multicast 或 metadata 范围则拒绝。
|
||||||
|
- 开发模式只对显式 localhost HTTP 开例外;企业内网需要服务器管理员设置环境变量。
|
||||||
|
- 当前请求策略要求同源重定向,不能依赖首次 URL 校验后任意跳转。
|
||||||
|
|
||||||
|
### 路径与写入
|
||||||
|
|
||||||
|
- 每个 Session 使用独立随机目录。
|
||||||
|
- 已发现的 Skill 正文复制到 Session 内只读快照;Pi 仅使用该快照清单,不向模型工具开放项目或用户目录。Drama Minis 额外复制明确白名单内的两份参考文档、主题、渲染脚本及两份结构示例;逐段拒绝符号链接,单文件上限 1 MiB。其他 Skill 默认仍只复制正文。
|
||||||
|
- realpath + relative containment 防止 `..` 和绝对路径逃逸。
|
||||||
|
- 逐段拒绝符号链接,并保护 `.env`、`.git`、`.ssh` 和常见私钥名。
|
||||||
|
- write tool 只能调用 `SafeWorkspace.writeReportDraft`,目标固定为 `report.md`,临时文件以独占创建并原子 rename。
|
||||||
|
- 删除 Session 会取消 run 和审批,并清除附件和整个 Session 目录。
|
||||||
|
|
||||||
|
### 图片
|
||||||
|
|
||||||
|
- 只收 JPEG、PNG、WebP;同时检查扩展名、MIME 和 magic bytes。
|
||||||
|
- 流式字节上限、宽高上限与总像素上限阻止常见资源耗尽输入。
|
||||||
|
- Sharp 自动旋转并重新编码,去掉不需要的元数据。
|
||||||
|
- 随机服务端文件名,权限 0600;不提供任意路径下载或静态目录。
|
||||||
|
- base64 只在 Server 构造 Pi prompt 的最后阶段短暂存在。
|
||||||
|
|
||||||
|
### 工具与审批
|
||||||
|
|
||||||
|
- allowlist 只开放课堂所需 read-only 工具。
|
||||||
|
- 未知工具默认拒绝。
|
||||||
|
- `save_report_draft`、真实指标查询 `metrics_semantic_query`、报告生成 `generate_minis_report` 在 Pi tool preflight 阶段等待服务器 Broker。查询审批展示实际参数与第三方模型数据处理提示。
|
||||||
|
- 审批绑定 approvalId、runId 和 toolCallId;支持 approve、deny、expired、cancelled。
|
||||||
|
- Browser 点击不是最终事实,只有 `approval.resolved` 事件能改变投影状态。
|
||||||
|
|
||||||
|
### Drama Minis 真实数据与报告
|
||||||
|
|
||||||
|
- 服务端使用官方 MCP SDK,固定 swads HTTPS 地址,禁止重定向,只开放身份、目录和语义查询;未开放广告修改接口。
|
||||||
|
- swads token 仅由服务端环境注入,缺失时明确失败。源工具结果按 token 精确脱敏,不使用模拟数据替代失败。
|
||||||
|
- 报告必须引用同一 Session 已成功的真实查询 ID,JSON 保存查询参数和原始返回;这提供可追溯性,但不自动证明模型计算的每个字段正确。
|
||||||
|
- 仅运行管理员指定 Skill 中的固定 Python 渲染脚本,模型不能指定任意命令。Python/Chrome 子进程不继承 API 凭证。Chrome 禁止背景网络并使用临时 profile;这不是通用网络隔离。
|
||||||
|
- 当前原始渲染器的图表单位固定为 USD,因此其他币种明确拒绝生成,不静默换算或合并。
|
||||||
|
- 下载限定 Session、注册 report ID 与 json/html/pdf 三种格式;no-store、nosniff、attachment、沙盒 CSP。PDF 失败时仅返回 HTML/JSON 及明确提示。
|
||||||
|
- 仍是单用户 localhost 原型,无用户级身份认证。不得直接暴露公网或作为多租户生产服务。
|
||||||
|
|
||||||
|
## 不提供的隔离
|
||||||
|
|
||||||
|
Pi 没有内置 Sandbox,Project Trust 也不是 Sandbox。Pi Agent、Extension、Sharp 和所有 Node 依赖继承启动进程的用户权限与网络权限。应用层路径校验和审批不能防御依赖漏洞、Node 进程逃逸、同用户数据读取或不受控的出站模型请求。
|
||||||
|
|
||||||
|
因此不要用该课堂版处理生产密钥、不可信租户代码或敏感业务文件。真正部署至少需要:低权限系统用户、只读基础镜像、每 Session 容器或 microVM、挂载白名单、seccomp、出站网络 allowlist、资源配额和销毁验证。
|
||||||
|
|
||||||
|
## 部署前仍需补齐
|
||||||
|
|
||||||
|
- 登录、Session 所有权、租户授权和管理员权限。
|
||||||
|
- SameSite Secure cookie、CSRF、防点击劫持和严格 CSP。
|
||||||
|
- API、上传、run 和审批的速率限制与配额。
|
||||||
|
- KMS / Secret Manager,不在应用堆中长期保存明文凭证。
|
||||||
|
- 持久化且防篡改的审计日志。
|
||||||
|
- 恶意图片扫描、对象存储隔离和生命周期策略。
|
||||||
|
- 分布式锁、幂等键、Broker 持久化和崩溃恢复。
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
# Drama Minis 网页接入
|
||||||
|
|
||||||
|
用户已授权直接实施,并允许 swads 数据传递到已配置的 Qwen 网关。正式经营查询仍须由页面用户确认周期、对比周期和账户范围。
|
||||||
|
|
||||||
|
1. 扩展 Skill 快照:仅复制显式允许的引用、脚本、CSS 和结构示例;保持原始 Skill 不变,拒绝路径逃逸和符号链接。
|
||||||
|
2. 接入官方 MCP HTTP 客户端:仅映射 swads_whoami、metrics_catalog、metrics_semantic_query;凭证来自服务器 SWADS_MCP_TOKEN,不经过浏览器或模型。真实查询逐次等待页面批准。
|
||||||
|
3. 封装固定报告生成器:校验 JSON,在会话私有目录运行已知 Python 脚本;生成 HTML 和 A4 PDF。PDF 失败时明确返回 HTML 与原因,不伪造成功。
|
||||||
|
4. 网页增加 Skill 调用入口、真实查询确认和报告下载卡片;固定 Session 产物端点禁止路径遍历。
|
||||||
|
5. 验收:先失败测试后实现,运行全套检查;真实网页验证 Skill 读取、引用、swads 连接和参数确认。未获日期/范围确认前不自动查询真实指标。报告渲染使用明确标注的测试数据验证,不作为真实经营报告交付。
|
||||||
|
|
||||||
|
不开放通用 shell,不导入其他 Codex MCP 权限,不修改原始 Skill,不做公网部署。
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
# Editor CMS Connection Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** Use superpowers:executing-plans inline. The user requests direct execution; do not pause for another execution-mode choice.
|
||||||
|
|
||||||
|
**Goal:** Let the web app load the real editor Skill and authenticate its own CMS session, then read genuine drama inventory and subtitles without exposing credentials.
|
||||||
|
**Architecture:** First implement the CMS/auth boundary as a separately testable milestone. It does not claim render readiness; the subsequent media milestone requires an authenticated source and explicit model/footage authorization. Original Skill and CMS code remain unchanged.
|
||||||
|
**Tech Stack:** Existing Fastify, React, Pi, Zod, pinned official Supabase JS client.
|
||||||
|
**Spec:** `docs/superpowers/specs/2026-09-10-short-drama-web-design.md`.
|
||||||
|
|
||||||
|
**2026-09-14 execution checkpoint:** Auth/API/UI/read-only tools are implemented. Full suite: 146 passed, lint/typecheck/build passed. Real login, 30-item catalog, Skill + CMS contract reads and status tool passed through the Vite/API/real-Qwen chain. Browser login/catalog also passed. Detailed evidence and remaining limitations: `docs/editor-cms-web.md`. Checklists below retain unverified cases (including explicit expiry, busy API mutation and browser content approval); do not interpret this milestone as full video acceptance.
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- localhost single-user app; no CMS writes, RLS changes, service_role, arbitrary shell or uploads.
|
||||||
|
- Authentication is a separate UI form; no password/token in chat, model tools, SSE or disk. Session reset clears only this app's CMS connection.
|
||||||
|
- Fixed Supabase project and CMS origin; reject redirects. Only verified app_metadata.role=admin is accepted, matching actual backend checks.
|
||||||
|
- Skill source is read-only; copy only explicitly enumerated references and script dependencies.
|
||||||
|
- Do not equate readable Skill/catalog with a finished video pipeline.
|
||||||
|
|
||||||
|
### Task 1: Authenticated CMS boundary
|
||||||
|
|
||||||
|
Files: `packages/harness/src/cms/cms-connection.ts`, `packages/harness/tests/cms-connection.test.ts`.
|
||||||
|
Interface: `CmsConnection({publishableKey,fetch?})`: `login(email,password)`, `status()`, `clear()`, `listDramas(query,offset)`, `inventory(dramaId)`, `subtitle(dramaId,episodeId,locale)`.
|
||||||
|
|
||||||
|
- [ ] RED: create real service tests with injected external HTTP responses. Assert no data query before login; fail non-admin even with user_metadata.admin; status omits passwords/tokens; subtitle rejects an episode not in the requested drama; inventory completeness preserves missing sources. Example: `await expect(cms.listDramas('',0)).rejects.toMatchObject({code:'CMS_LOGIN_REQUIRED'})`.
|
||||||
|
- [ ] Run `../../node_modules/.bin/vitest run tests/cms-connection.test.ts` in harness and observe missing implementation failure.
|
||||||
|
- [ ] GREEN: use official `createClient(url,key,{auth:{persistSession:false,autoRefreshToken:false,detectSessionInUrl:false}})`. Call `signInWithPassword` then `getUser` and verify role. Fetch only known tables/selects, validate all returns with Zod; fixed CMS subtitle route and bounded responses. Never return raw SDK errors.
|
||||||
|
- [ ] Verify tests and TypeScript, including expiry and disconnect; review credential lifecycle and request destinations.
|
||||||
|
|
||||||
|
### Task 2: Session API and login UI
|
||||||
|
|
||||||
|
Files: `apps/server/src/routes/cms.ts`, `apps/server/src/app.ts`, `apps/server/src/index.ts`, `apps/web/src/components/cms/CmsPanel.tsx`, `apps/web/src/App.tsx`, corresponding route/component tests.
|
||||||
|
Interface: GET/POST/DELETE `/api/sessions/:sessionId/cms/connection`; GET `/api/sessions/:sessionId/cms/dramas?q=&offset=`. All authenticated CMS objects are mapped by Session; dispose clears them.
|
||||||
|
|
||||||
|
- [ ] RED: API rejects unknown Session, cross-origin mutation, concurrent login, busy mutation; private fields never appear in status. UI password empties on submission/failure/close; authenticated catalog selection fills an exact drama ID in Chat.
|
||||||
|
- [ ] Run test files and observe failures before implementation.
|
||||||
|
- [ ] GREEN: fixed-origin backend API, JSON-only login body, same-origin/fetch metadata guard, bounded login attempts, no-store. Status communicates `connected` and `expiresAt`; no user email or credentials to model. UI accessible labels, pending/error states and an explicit notice of the destination receiving credentials.
|
||||||
|
- [ ] Verify route and UI tests; unchanged report flow passes.
|
||||||
|
|
||||||
|
### Task 3: Real Skill and read-only tools
|
||||||
|
|
||||||
|
Files: `packages/harness/src/cms/editor-tools.ts`, `apps/server/src/index.ts`, tool policy, Skill and web tests.
|
||||||
|
Interfaces: `cms_connection_status`, `cms_list_dramas`, `cms_drama_inventory`, `cms_read_subtitle`. Model never calls login or changes the connection. Inventory and subtitle access require page approval because they can disclose licensed/nonpublic content to the configured model.
|
||||||
|
|
||||||
|
- [ ] RED: Skill snapshot reads full reference hierarchy; tools reject data access before login; policy asks for content reads; discovery alone does not claim rendering ready.
|
||||||
|
- [ ] GREEN: enumerate trusted Skill resources, register tools only with that Skill, strict tool argument schemas, inject runtime instruction to report missing media/vision/research capability honestly and never use demo data.
|
||||||
|
- [ ] Run all tests, lint, typecheck, build, then restart only the local Agent Studio service with secrets via hidden stdin.
|
||||||
|
- [ ] Browser verify Skill entry, original instructions, login form and capability/status tools; request the user to authenticate through the form for the real CMS flow. Stop at a genuine missing login instead of inventing an admin session.
|
||||||
|
|
||||||
|
## Remaining full-workflow milestones (not claimed by this plan)
|
||||||
|
|
||||||
|
After real CMS login is available: approved bounded HLS download; vision evidence and analysis v2 gate; configured reference research and scoring; immutable plan approval; fixed Python/FFmpeg worker with cancellation/quota; machine QA; human sound/viewing review; versioned downloads/export. Each requires its own test-first plan against the full spec. The task remains incomplete until these are verified on user-selected licensed footage.
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
# Editor Media Pipeline Implementation Plan
|
||||||
|
|
||||||
|
> Execute inline using executing-plans and test-driven-development; the user already requested execution, no further execution-mode confirmation.
|
||||||
|
|
||||||
|
**Goal:** Prepare actual licensed source media and return auditable frames/subtitles through Agent Studio, then enforce original analysis, reference, recipe, render and review gates.
|
||||||
|
**Architecture:** Fixed CMS/admin signing; bounded same-CDN HTTPS HLS retrieval to local files; FFmpeg/FFprobe without network or inherited secrets. Per-session service and explicit approval tools reuse the existing Harness. Rendering cannot run until analysis and reference gates pass.
|
||||||
|
**Tech Stack:** Existing TypeScript/Fastify/Pi/React; pinned ffmpeg-static 5.3.0, ffprobe-static 3.1.0; bundled Python 3.12 and original Skill validators.
|
||||||
|
**Spec:** `docs/superpowers/specs/2026-09-10-short-drama-web-design.md`.
|
||||||
|
|
||||||
|
## Constraints
|
||||||
|
|
||||||
|
No CMS writes, uploads, source overwrites, arbitrary shell or credential/URL exposure. First acceptance title is `06aa396d-175a-48ed-ad76-02786ef4d0a9` (Beneath the Ice). Read-only preflight is already PASS. Metadata completeness is not media analysis. Content review must follow the newly added adult-content gate, never fabricate reviewer PASS.
|
||||||
|
|
||||||
|
### 1. Source preparation
|
||||||
|
|
||||||
|
Files: `cms/cms-connection.ts`, `media/hls.ts`, `media/media-worker.ts`, `media/editor-service.ts`, matching harness tests.
|
||||||
|
|
||||||
|
- [ ] Write failing tests for HLS target-origin/path escapes, encrypted/byte-range playlists, master nesting, segment count/duration/byte limits; valid playlists become local filenames without signed URLs.
|
||||||
|
- [x] Implement `parseHls(text:string): {kind:'master',variants:string[]} | {kind:'media',segments:{uri:string,duration:number}[],duration:number}`. Only finite VOD MPEG-TS sources; reject unsupported formats explicitly, never silently skip encryption or audio renditions.
|
||||||
|
- [x] Add `CmsConnection.playback(dramaId,episodeId)` returning signed URL only to server-side service after exact inventory binding and admin checks.
|
||||||
|
- [x] `EditorService.prepare({dramaId,episodeNumbers},signal)` allows at most 5 episodes, 30 minutes/episode, 1 GiB/task download. Fetch fixed public CDN, no redirects, preserve CDN-issued per-path signing query only on same episode-directory assets. Localize manifests before invoking FFmpeg with `file,crypto,data` protocol allowlist; no network credentials in child env.
|
||||||
|
- [ ] Probe real local MP4 for audio and video, hash sources; create representative JPEGs, save captions and source/time bindings in private runtime. Cancellation aborts fetch and kills process group.
|
||||||
|
|
||||||
|
### 2. Web tools and artifacts
|
||||||
|
|
||||||
|
Files: `media/editor-tools.ts`, server assembly and routes, tool policy and approval UI.
|
||||||
|
|
||||||
|
- [x] Expose `prepare_drama_sources` with exact IDs/quotas (approval), `read_drama_frames` returning small selected JPEG images plus time/source binding (approval), `editor_job_status` (read-only).
|
||||||
|
- [x] Keep normal image-tool results out of SSE base64 payloads. Expose only registered frame IDs scoped to owning Session, with no-store/nosniff and inline JPEG display. Model must read Skill before analysis; no shell access.
|
||||||
|
- [ ] Verify actual first/middle/last source download, image tool result acceptance by existing Qwen, and genuine transcript evidence. A safe extraction result can be claimed only after probing output.
|
||||||
|
|
||||||
|
### 3. Analysis and downstream gates
|
||||||
|
|
||||||
|
Files: validator bridge and video job routes/review components, separate tests following the verified interfaces above.
|
||||||
|
|
||||||
|
- [x] Bind every analysis submission to freshly prepared drama_id/fetched_at and server-issued analysis_id. Server compares actual subtitle/frame IDs before invoking original validator; reject invented evidence, paths or PASS flags.
|
||||||
|
- [ ] After diagnosis PASS, obtain verified public short-drama references through a separately configured source or user-supplied direct evidence; never claim current Codex search is inherited by the webpage. Missing reference access/evidence is a genuine gate requiring input, not an automatic Tier C downgrade.
|
||||||
|
- [x] Bind immutable recipe and content-review records to the same analysis and approved source IDs; user approves exact plan hash. Run original render_exact and machine QA with bounded child process and minimum environment. Verified with synthetic engineering fixtures; real-drama acceptance remains pending.
|
||||||
|
- [ ] Serve REVIEW preview/ranged download only for registered artifacts; final export requires human full-playback-with-sound review bound to exact video hash, plus existing content/story gates.
|
||||||
|
|
||||||
|
### Verification and persistence
|
||||||
|
|
||||||
|
- [ ] Run baseline/regression tests, original validators, typecheck/lint/build. Keep progress in this plan, avoid claiming later unchecked gates complete.
|
||||||
|
- [x] Restart only the local service after build; fresh web Session must enable verified Qwen image capability by default. Credentials remain in memory, require re-login after restart.
|
||||||
|
- [ ] Run real browser task to a genuine review/approval gate. Do not stop at a preflight or after writing a plan.
|
||||||
|
|
||||||
|
## 2026-09-14 execution evidence
|
||||||
|
|
||||||
|
- Real local acceptance passed: episodes 1/3/5 downloaded, remuxed with audio/video, probed, SHA-256 recorded, 9 JPEGs and English VTT saved.
|
||||||
|
- Real browser run loaded the original Skill and CMS contract, requested page approval, then successfully ran `prepare_drama_sources` for all three episodes. Job `job_b1280a06-450e-4982-935a-b2bd8dfb726f`; nine actual 288×512 images loaded in the result card.
|
||||||
|
- Licensed subtitles/frames have NOT yet been sent to Qwen. Separate transmission authorization remains pending. Do not equate the model's earlier synthetic-image test with a real-drama analysis pass.
|
||||||
|
- Original analysis/recipe/hook validators, reference scorer, renderer and machine QA are now connected. Source and plan review endpoints are human-only; tools cannot submit approval. REVIEW video routes support byte ranges, with frontend playback/download. Actual model transmission and real-title end-to-end acceptance remain pending.
|
||||||
|
- Fresh synthetic integration produced actual 10-second H.264/AAC/1080×1920 with original source audio and mapped English subtitles. It does not establish licensed-drama analysis, real reference evidence, or semantic content PASS.
|
||||||
|
- Local persisted files are not yet restart-resumable. Global disk-cache quota, automatic public reference search and final-stage export remain open; final PASS is never synthesized.
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
# Image-to-video Web Implementation Plan
|
||||||
|
|
||||||
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||||
|
|
||||||
|
**Goal:** Integrate the real CMS-free image-to-video Skill with durable H3 tasks and a usable web interface.
|
||||||
|
**Architecture:** Reuse attachment validation and approval broker; add a separate durable service and restricted MCP adapter. Keep image generation jobs independent of in-memory CMS/editor jobs and chat history.
|
||||||
|
**Tech Stack:** Existing TypeScript/Pi/Fastify/React/Vitest; original Skill Python media validator; existing ffmpeg-static/ffprobe-static.
|
||||||
|
**Spec:** `docs/superpowers/specs/2026-09-15-image-to-video-web-design.md` (approved by user).
|
||||||
|
|
||||||
|
## Global Constraints
|
||||||
|
|
||||||
|
- 不依赖 CMS 登录、剧名、字幕或旧剪辑器的参考/选段审核。
|
||||||
|
- 首版只做单图新生成,不提供原片续接、批量生成、自动选图或生成封面功能。
|
||||||
|
- 不自动调用 Seedance 或其他付费提供商;不宣称 H3 免费或已支持 10 秒。
|
||||||
|
- 本地参考图外传和生成提交分别审批;不得自动审核素材、分配账户或发布广告。
|
||||||
|
- 重启后已知 job_id 只查原任务;未知提交不得盲重投。不得伪造时长或审核 PASS。
|
||||||
|
- 不发送密钥至模型/前端/日志。媒体端点必须防路径穿越、SSRF、重定向和错误内容。
|
||||||
|
- 保留所有已有脏改动;仅提交本任务新增文件和本任务增量,不提交无关旧改动。
|
||||||
|
|
||||||
|
### Task 1: Complete vertical slice (durable service, tools, routes and UI)
|
||||||
|
|
||||||
|
**Files:**
|
||||||
|
- Create focused modules under `packages/harness/src/image-video/`: contracts, store/service, MCP adapter, media validation, tools. Tests under `packages/harness/tests/image-video*.test.ts`.
|
||||||
|
- Modify `packages/harness/src/tools/swads-mcp.ts` only to expose reusable validated discovery/call plumbing if needed; preserve existing analytics allowlist. Export new modules through `packages/harness/src/index.ts`.
|
||||||
|
- Modify `apps/server/src/index.ts`, `apps/server/src/app.ts`; create `apps/server/src/routes/image-video.ts` and tests `apps/server/tests/image-video-api.test.ts`.
|
||||||
|
- Create `apps/web/src/components/tools/ImageVideoPanel.tsx` and tests `apps/web/tests/image-video.test.tsx`; integrate `App.tsx`, `ApprovalDialog.tsx`, `ToolFallback.tsx` and `components.css` as needed. Preserve current CMS UI but never require it for this flow.
|
||||||
|
- Modify `packages/harness/src/tools/tool-policy.ts`, `pi-event-adapter.ts` for the new tools and safe public details.
|
||||||
|
- Create `docs/image-to-video-web.md` with actual usage and explicit test/live limitations.
|
||||||
|
|
||||||
|
**Interfaces / implementation guidance:**
|
||||||
|
|
||||||
|
Use task IDs `i2v_<uuid>` and a versioned private JSON record with server-owned paths, ownerSessionId, stable request hash, reference hash, upload receipts, jobId, state, timestamps and revision. A local draft operation (`i2v_create_task`) resolves an attachment from the requesting Session and copies approved-local image bytes into the durable task directory; it must not upload. Subsequent tools `i2v_upload_reference` (ask), `i2v_submit` (ask), `i2v_status` (allow) consume only registered taskId, not arbitrary URL, tenant, file path or provider arguments. Preserve prompt/duration/image bindings from the draft through approval. Do not give a model a general mutation tool.
|
||||||
|
|
||||||
|
```ts
|
||||||
|
type ImageVideoDraft = { attachmentId: string; title: string; prompt: string; durationSeconds: number };
|
||||||
|
type ImageVideoPublicTask = {
|
||||||
|
id: string; title: string; prompt: string; durationSeconds: number;
|
||||||
|
state: string; createdAt: string; updatedAt: string;
|
||||||
|
referenceUrl: string; videoUrl?: string; jobId?: string;
|
||||||
|
error?: string; validation?: unknown;
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
Use private service methods `create(sessionId, draft)`, `upload(sessionId, taskId)`, `submit(sessionId, taskId)`, `refresh(taskId)` and `list()/view(taskId)` with public redacted return DTOs. Tools must enforce owner Session. A same-origin, read-only local task history can display registered tasks across process restarts without granting a new Session permission to mutate them. No automatic cross-session reassignment. Keep already-submitted jobs collectible after restart, even when their old chat Session no longer exists. A timer or UI read-only refresh should resume job polling without involving the LLM. Query at most once per 30–60 seconds; serialize each task and coalesce concurrent requests.
|
||||||
|
|
||||||
|
Use `/api/image-video/tasks` for redacted local history and registered media read endpoints. Creation is a Session-scoped route and/or tool. Prefer a small dedicated image input + draft form that uses the existing attachment endpoint, then invokes this Skill through the composer with the exact draft/attachment identifiers; this allows references even without vision-enabled Qwen. Keep all transfer approvals explicit. If the model needs image inspection, explain current model image transmission separately; server thumbnail display is not model transmission.
|
||||||
|
|
||||||
|
Discover and validate required MCP tool schemas via `listTools` before submission; rely on actual structured response shapes, not guessed backend endpoints. Root controller will supply read-only live contract observations. Identity/platform ambiguity blocks upload. Persist every potentially mutating intent before calling remote tools. A timeout around submit must persist unknown_submission, never return to ready. A known returned job ID must be persisted even if user cancelled waiting; preserve in-flight result handling. Before a source upload retry, distinguish uninitiated, known upload receipt and unknown initialization/finalization; never duplicate an uncertain import blindly.
|
||||||
|
|
||||||
|
MP4 download uses only registered job artifact HTTPS URL, DNS/public-address checks and bounded bytes/time, no redirects. Validate full decode, video dimensions/frame rate/duration and audio using the original Skill `scripts/media.py verify`; exact-duration mismatch is failure, not ready. Store validation report and hash; preview remains not human reviewed. Bound process time and prohibit media tools from network access. Reuse existing safe media runtime helpers where suitable, without copying CMS download logic or fixed CMS hosts.
|
||||||
|
|
||||||
|
- [x] **Step 1: RED — durable state tests.** Test draft ownership, no external operations on create, separate approvals, duplicate concurrent submit, persisted job after reload, unknown submit blocks retry, unknown import blocks retry, stale task revisions, safe public DTO, independent CMS. Example behavioral assertion:
|
||||||
|
```ts
|
||||||
|
await expect(service.submit("ses_other", task.id)).rejects.toMatchObject({code:"I2V_TASK_NOT_FOUND"});
|
||||||
|
expect(remoteSubmissions).toHaveLength(0);
|
||||||
|
```
|
||||||
|
Run the focused harness tests and record real failing assertions before implementation. External provider/network calls may be faked, not service state or persistence.
|
||||||
|
- [x] **Step 2: GREEN — implement minimal service and remote boundary.** Atomic records, strict persisted schema, locked task operations, known-ID polling, safe URL/media validation. Validate recorded state by constructing a new service against the same temporary directory and proving no duplicate remote submit.
|
||||||
|
- [x] **Step 3: RED/GREEN — API and tool access.** Add tests of denied approval/no side effects, cross-session task binding, durable history after registry restart, same-origin JSON mutation requirements, registered-only media, HTTP 206/416 and no-store. Implement routes and tool policy, original Skill resource snapshot and production wiring. Deny unknown tool names.
|
||||||
|
- [x] **Step 4: RED/GREEN — real UI behavior.** Test image/draft input, exact approval parameters and thumbnail, unchecked decisions, loading/error/queue states, restored tasks, periodic status fetch with cleanup, registered-only video links and actual native player/download. Use jsdom only as component evidence, not end-to-end real generation proof.
|
||||||
|
- [x] **Step 5: Verify and self-review.** Run focused tests per iteration then full `CI=true pnpm --config.verify-deps-before-run=false test`, lint, typecheck and build. Review actual diff for credential reflection, arbitrary paths, duplicate requests and existing dirty edits. Write report with RED/GREEN evidence and all remaining live unknowns. Commit only isolated task changes; if modifying already-dirty files, use a task-specific delta commit or leave those files uncommitted and document why rather than absorbing unrelated edits.
|
||||||
|
|
||||||
|
### Task 2: Independent review and live acceptance
|
||||||
|
|
||||||
|
**Files:** Review package/report in plan workspace; update `docs/image-to-video-web.md` with exact evidence. Root controller owns service start, browser and live acceptance, not the implementer.
|
||||||
|
|
||||||
|
- [x] **Step 1:** Independent review the new feature diff for spec compliance and code quality, especially restart/approval/unknown-submission semantics. Fix important findings with covering tests before live mutation.
|
||||||
|
- [x] **Step 2:** Run latest test/lint/typecheck/build; start local backend using existing hidden credential input without printing or saving keys. Read-only current MCP identity/schema checks must use the configured application token as well as connected-tool discovery.
|
||||||
|
- [x] **Step 3:** Browser test catalog discovery, new draft, reference preview, approval refusal and task history restoration with no CMS connection. Synthetic boundary tests must be visibly labelled as such.
|
||||||
|
- [ ] **Step 4:** If there is a user-authorized reference image, submit one actual H3 task through page approvals. Otherwise stop at image selection; do not transmit earlier CMS images or manufacture approval of a new image. User's approval of this implementation is not approval of an unspecified remote submission.
|
||||||
|
- [ ] **Step 5:** If a real job exists, poll it rather than resubmit, collect its exact MP4, run technical checks, watch/listen and verify actual webpage playback/download. If blocked on duration/provider/image approval, report that specifically; do not claim real generation completion.
|
||||||
@@ -0,0 +1,96 @@
|
|||||||
|
# Short Drama Editor 网页接入设计
|
||||||
|
|
||||||
|
日期:2026-09-10。用户已同意“网页选剧 → 分析与剪辑方案 → 确认后渲染 → 预览审核 → 下载”的范围。本文件是实现前设计,不代表能力已上线。
|
||||||
|
|
||||||
|
## 1. 目标与边界
|
||||||
|
|
||||||
|
在既有 Agent Studio Mini 中调用原始 `swads-short-drama-editor`,提供可观察、可取消、可审核的授权英文短剧剪辑任务。原始 Skill、CMS 素材、已交付版本保持不变。保留现有 Drama Minis 报表流程。
|
||||||
|
|
||||||
|
不是仅注册 Skill 名称,也不把模拟素材、模型描述或机器检查通过当成真实成片验收。首版是本机单用户工具,不新增公网部署、广告投放、SWADS 上传、配音或合成镜头。
|
||||||
|
|
||||||
|
## 2. 已核对的基础条件
|
||||||
|
|
||||||
|
- 既有后端已有 Skill 只读快照、工具白名单、审批、Session/SSE 和受限报告下载。
|
||||||
|
- 现有 swads 连接只映射身份、指标目录、指标查询;没有网页 CMS 取片能力。swads token 不等于 CMS 管理员凭证。
|
||||||
|
- 最初 Qwen 连接在项目中声明为 text-only;2026-09-14 已实测同一网关的 Responses 双图识别成功,当前网页会话启用图片后也通过上传及识别测试。无需额外视觉模型,但 CMS 抽帧/证据绑定流程仍未实现;不能仅打开 supportsImages 开关就视为该流程已完成。
|
||||||
|
- bundled Python 已实测为 3.12.14,可用于 Skill 所需 audioop;当前 PATH 与已检查的 bundled bin 中未发现 FFmpeg/FFprobe。安装或指定媒体运行时需先确认可用来源。
|
||||||
|
- 2026-09-14 用户提供 `https://swave.pencil-stub.com/`。已从本机 `/Users/wenxinduan/codex-outputs/Short` 核对到同一项目:`.ops/README.md` 中 CMS 为 `/swave/`,后端为 `/swave-api/`。Supabase 为 `shorts-sg`(`lmcysqgfhoxppwuorsmu`),已只读核对 drama、episode、episode_source、subtitle_track 表结构。不得搜索或抽取浏览器存储中的凭证。
|
||||||
|
|
||||||
|
## 3. 方案选择
|
||||||
|
|
||||||
|
推荐在现有 Fastify 后端增加受控视频任务服务和媒体 worker,复用现有 Chat、Skills、事件和审批。只注册固定业务工具,不开放通用 shell。
|
||||||
|
|
||||||
|
仅导入 Skill 目录不足以实现取片、研究、视觉分析与渲染。把任意 shell 交给模型会扩大文件和网络权限,且难以绑定审核结果。这两种方式均不作为实现目标。
|
||||||
|
|
||||||
|
## 4. 用户流程
|
||||||
|
|
||||||
|
1. Skills 新增该 Skill 的“使用”入口,返回 Chat;状态区显示 CMS、媒体运行时、画面理解、参考研究分别是否可用。
|
||||||
|
2. 用户指定剧名或精确 drama_id、目标市场、用途和数量。同名剧必须先确认,不自动选择;不默认下载所有剧集。
|
||||||
|
3. 从已配置 CMS 读取完整剧集目录、简介、标签和字幕信息,验证首/中/尾集 HLS 访问。审批显示选定剧集、下载范围和预计磁盘量;凭证及签名 URL 不进入模型、事件或产物。
|
||||||
|
4. 通过字幕和代表帧建立跨开头/中段/后段的证据,生成新的 analysis_id,固定 drama_id/fetched_at 并执行原始 v2 验证器。视觉证据缺失时阻塞,不用字幕冒充观片。
|
||||||
|
5. 诊断 PASS 后才创建策略和研究参考。参考须具备短剧营销来源、完整可分析视频、捕获时间、证据链接及 A/B 等级;搜索摘要不能代替来源。三轮无合格参考即阻塞;Tier C 单独请求明确授权。
|
||||||
|
6. 页面展示剪辑策略、选段、预期长度、参考证据和不揭示的信息。用户批准的对象是不可变计划版本及摘要哈希,而不是无限制的渲染权限。
|
||||||
|
7. 固定 worker 校验全部门槛后运行原始 render_exact.py,再运行 QA。聊天请求可结束,渲染任务独立继续;页面显示进度并可取消。
|
||||||
|
8. 机器检查通过后产生 REVIEW 视频。页面展示视频、全片联系表、转场条、字幕风险帧与审核清单。用户需完整带声试听并确认画面、字幕、故事因果和结尾,登记审核人和时间。
|
||||||
|
9. 只有同一版本的机器检查和人工检查都 PASS,才可导出正式版本;待审视频允许以明确 REVIEW 标识下载。归档旧版本,不覆盖源片和已审成片。
|
||||||
|
|
||||||
|
## 5. 后端职责与接口
|
||||||
|
|
||||||
|
### Skill 与工具
|
||||||
|
|
||||||
|
扩展显式资源清单,复制本 Skill 的引用、schema 和确定性脚本到只读快照,记录哈希。测试 fixture、缓存和历史报告不作为真实输入。所有相对引用必须完整可读。
|
||||||
|
|
||||||
|
业务工具负责能力检查、CMS 目录、授权素材准备、画面证据、诊断校验、参考证据、计划提交、渲染提交与状态读取。下载、将素材发往视觉服务、Tier C、渲染分别审批。模型不得调用人工审核通过接口,也不得伪造 PASS 文件。
|
||||||
|
|
||||||
|
### CMS 连接器
|
||||||
|
|
||||||
|
独立接口提供 list/search drama、exact drama inventory、subtitles、签名播放。具体实现以用户指定 CMS 的真实代码或已验证接口为依据;只读目录并按管理员播放路径取源,绝不使用付费观看接口判断缺片。
|
||||||
|
|
||||||
|
CMS 凭证留在服务端内存或用户指定的密钥管理设施。固定目标主机和重定向策略;限制媒体协议、清单递归、字节数、时长与超时。HLS 列表及分片下载在受控连接层完成,渲染器仅接触本地批准素材。
|
||||||
|
|
||||||
|
### 模型与参考研究
|
||||||
|
|
||||||
|
沿用现有模型设置进行文本推理;视觉步骤必须使用已实际验证的图像输入连接并取得素材传输授权。不可静默更换供应商、模型或复用其他服务凭证。
|
||||||
|
|
||||||
|
Web 应用不能继承当前 Codex 会话的浏览器或搜索工具。研究服务必须单独配置并验证;未配置则显示所缺能力。允许用户提供可访问的参考及直接证据,但不自动将其标记为合格或绕过原始评分门槛。
|
||||||
|
|
||||||
|
### 媒体 worker
|
||||||
|
|
||||||
|
只执行固定 Python 3.12 脚本与已解析的 FFmpeg/FFprobe,不拼接 shell。子进程使用最小环境,不继承 CMS、模型或 swads 密钥。脚本需能在 Python 隔离启动时导入其受信任同目录 common 等模块。
|
||||||
|
|
||||||
|
所有模型指定的素材引用先解析成已批准 source ID,再由服务端生成本地路径;拒绝任意路径、符号链接、远程 URL、跨剧引用与写入原片。脚本输出也需要路径和大小校验。
|
||||||
|
|
||||||
|
首版全局最多一个重型媒体任务。每个任务有磁盘、媒体时长、运行时间和子进程数量上限;清单完整性不等于授权无限量下载。取消必须终止整个任务子进程树,而不仅是父 Python;清理仅针对该任务的精确临时目录。
|
||||||
|
|
||||||
|
### 任务与下载
|
||||||
|
|
||||||
|
VideoJob 使用独立 UUID、Session 所属关系、固定 drama_id 和不可变版本号。状态包括 waiting_input、preparing、analyzing、awaiting_plan_approval、rendering、machine_qa、review_required、approved、blocked、failed、cancelled。每次转移由服务端验证。
|
||||||
|
|
||||||
|
元数据原子持久化到项目内专用 runtime 目录,不复用重置 Session 时删除的临时报告目录。进程重启后将中断的运行标记为 interrupted/failed,绝不假装自动恢复。新 Session 只经显式恢复动作接管既有任务。
|
||||||
|
|
||||||
|
新增本机任务 API:列出/创建、读取状态、取消、读取审核资料、提交人工审核、导出。接口校验 task ID、版本及所属关系;下载只接受注册 artifact ID,不接受文件路径。视频预览支持受限 Range,JSON/审核文件下载为 attachment,禁止缓存。
|
||||||
|
|
||||||
|
本机导出遵守 Skill 的日期 → 剧名 → 素材结构;默认目的地是用户素材目录。确认路径及写权限后才复制,并校验字节和 SHA-256。没有目的地权限时保留项目内 REVIEW/已审文件并提供下载,不擅自写入其他目录。
|
||||||
|
|
||||||
|
## 6. 必须保留的校验
|
||||||
|
|
||||||
|
- 诊断 v2 PASS;drama_id、analysis_id、fetched_at 与全流程一致。
|
||||||
|
- 策略、参考评分、hook-map、recipe、matches、continuity、字幕、QA 等必需产物齐全;禁止模型只写 pass: true 就绕过校验器。
|
||||||
|
- 原始音频、1.00x、精确 24fps 边界;1080×1920、H.264/AAC、48kHz 双声道、最长 300 秒。
|
||||||
|
- 只用目标剧素材,保留动作/反应和 withheld_information,比较历次交付的故事线与选段,不把换字幕当新故事。
|
||||||
|
- 机器 PASS 与人工带声审核分别记录。媒体、计划或字幕哈希改变后旧审批与 QA 失效。
|
||||||
|
- 本地 PASS 不是平台批准。SWADS 上传本次不实现;未来单独授权并验证真实上传发起器及任务状态后再接入。
|
||||||
|
|
||||||
|
## 7. 验收与完成定义
|
||||||
|
|
||||||
|
先写失败测试再实现,覆盖目录/资源、未配置状态、路径逃逸、跨剧绑定、伪造 PASS、拒绝/超时审批、版本变更、任务取消及进程退出、重启中断、Range/下载边界和人工审核门槛。
|
||||||
|
|
||||||
|
运行原始 Skill 验证器测试与明确标记的合成媒体流水线测试;它们只证明技术链路,不算真实剪辑验收。保留原有日报自动化回归,运行 typecheck、lint、test、build。
|
||||||
|
|
||||||
|
网页端最终验收需要一部用户指定的授权英文剧、可用 CMS 和视觉/参考研究连接:选剧、完整目录检查、真实分析、方案审批、渲染、REVIEW 预览和下载。只有用户或获授权的真实审阅完成后才验证正式导出。缺任何一环,交付时逐项标明未验证,不宣称端到端可剪辑。
|
||||||
|
|
||||||
|
## 8. 实现前需要的输入
|
||||||
|
|
||||||
|
CMS 入口和本机源码已定位,不再要求用户重复提供。`backend/app/routers/cms_playback.py` 确认专用签名接口为 `GET /api/v1/cms/playback/{episode_id}`,字幕为同路径下 `/subtitles/{locale}.vtt`;生产完整前缀为 `https://swave.pencil-stub.com/swave-api`。它们依赖有效 Supabase 用户会话及后端管理员判定,不能用 swads token 或匿名 key 替代。
|
||||||
|
|
||||||
|
还需通过受支持的登录/授权流程配置 Agent Studio 自己的 CMS 会话,不能抽取另一浏览器的登录存储;然后确认视觉与参考研究连接并执行实际取片验收。源码接口已确认不等于线上授权取片已验证。不得因缺少这些条件而退化成假素材演示并标记完成。
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
# Agent Studio:SW Ads 图生视频接入
|
||||||
|
|
||||||
|
## 目标和已确认范围
|
||||||
|
|
||||||
|
用户已同意在 Agent Studio 中接入本机 `swads-image-to-video`,并明确不需要 CMS。暂停旧短剧剪辑 Skill 的排障;不删除原有代码、会话或素材。新流程不依赖 CMS 登录、剧名、字幕或旧剪辑器的参考/选段审核。
|
||||||
|
|
||||||
|
网页输入图片和动作要求,真实调用 H3,显示任务进度,最终播放、下载生成的有声 MP4。加载 SKILL.md 不算接入成功;必须验证真实提交、按原 job_id 收取结果和网页播放链路。
|
||||||
|
|
||||||
|
## 方案选择
|
||||||
|
|
||||||
|
1. **采用:受控 H3 工具接入。** 复用现有 Skill 目录、图片附件、审批、工具卡和本地视频 Range 响应模式,新增独立图生视频任务存储。不继承旧 CMS 编辑流水线。
|
||||||
|
2. 不采用直接开放全部 SW Ads MCP 工具:会扩大到素材审核、账户分配和投放,不符合本次范围。
|
||||||
|
3. 不采用多供应商自动切换:收费、时长与输入契约不同。首版不自动调用 Seedance 或其他付费提供商。
|
||||||
|
|
||||||
|
## 首版用户流程
|
||||||
|
|
||||||
|
1. 选择此 Skill,添加一张参考图片,填写动作、声音和目标时长。图片在本机接收后经现有附件校验,展示确认缩略图;同会话绑定,不能引用任意磁盘路径。
|
||||||
|
2. Agent 读取真实 Skill 及相关引用,生成简短提示词。页面展示精确参考图、提示词、H3 模式、尺寸和时长供本次确认。原先 CMS 素材的传输批准不复用到本次图片。
|
||||||
|
3. 本地参考图外传单独审批:明确上传到 SW Ads 存储,并形成 `review_pending`、未分配账户的参考素材记录。拒绝则不上传、不提交;默认不审核素材、不分配账户。
|
||||||
|
4. 服务端从当前 MCP 身份和明确返回的环境信息解析上传所需 tenant/platform;信息缺失或有歧义时阻断,不猜账户。按实际 upload_url 上传,按同一 upload_id/object_key 完成,按 import_status 获取 asset_id。
|
||||||
|
5. H3 生成提交单独审批。使用 `ref2va`;提交前保存请求和稳定幂等键,返回后立即保存 job_id。费用未知就如实提示,不宣称免费;不通过切换提供商绕过能力限制。
|
||||||
|
6. 工具卡和独立任务列表显示准备、上传中、入库中、待批准、提交中、排队、生成中、校验中、可预览、失败、提交状态未知。长任务不占用一次 LLM 对话,查询不重复提交。
|
||||||
|
7. 按返回任务的 MP4 artifact 收取成片,本地技术验证后提供受控播放器和下载。未完成视听检查时标为“生成预览/待检查”,不称最终广告审核通过。
|
||||||
|
|
||||||
|
## 真实能力约束
|
||||||
|
|
||||||
|
已现场发现 `video_h3_submit`、`video_job_status`、`creative_upload_from_url`、`creative_import_status`、`swads_whoami` 和 `video_h3_workers`。实现时通过同一 MCP 连接重新发现并核对 schema,仅允许上述范围内所需操作。
|
||||||
|
|
||||||
|
H3 当前是多参考图模式,1–9 张参考图,不保证首帧锁定,也不是原生视频延长。首版只做单图新生成,不提供原片续接、批量生成、自动选图或生成封面功能;这些不是本轮验收条件。原 Skill 所附本地验证脚本可以复用,不修改原 Skill。
|
||||||
|
|
||||||
|
默认竖屏 9:16、自然速度、有环境声。沿用此前 10 秒作为首个端到端验收的目标,而非声称 H3 已支持 10 秒:当前工具 schema 没有列出时长枚举,须验证实际支持。若 H3 不支持目标时长,明确报告限制,不自行改时长,不用减速、定格、循环或重复帧凑秒数,不自动多次生成或换付费模型。
|
||||||
|
|
||||||
|
## 实现边界
|
||||||
|
|
||||||
|
- `packages/harness`:受控 MCP 图生视频适配器、任务状态机、原子磁盘记录、参考图上传/收据绑定、H3 提交/查询和本地验证。原指标工具白名单保持隔离。
|
||||||
|
- `apps/server`:同源受限的图生视频任务列表、查询与注册 MP4 端点;只接受登记任务 ID,不接受任意 URL 或磁盘路径。既有附件在本次批准上传前复制到任务私有目录,保存哈希;删除聊天附件不破坏已批准任务。
|
||||||
|
- `apps/web`:Skill 使用入口、参数与审批提示、持久任务列表和生成卡片;展示真实状态、错误原因、播放器、下载和检查状态。任务列表不依赖恢复旧聊天历史。
|
||||||
|
- 原始 Skill 资源:显式清单复制 `SKILL.md`、三份 references 和 `scripts/media.py` 到受限快照。不给模型 shell 或任意网络访问。
|
||||||
|
- 密钥仅保留服务端运行环境,不进入模型上下文、网页、日志、任务 JSON 或仓库。私有上传回执保留于受限目录,不在 SSE 暴露签名上传地址或原图字节。
|
||||||
|
|
||||||
|
## 恢复与安全
|
||||||
|
|
||||||
|
独立任务目录保存 batch/id/revision、图片与输出哈希、原始请求、幂等键、upload_id、asset_id、job_id、时区时间、状态、检查报告。临时文件原子替换;同任务单写入者;重复点击不能产生重复生成。
|
||||||
|
|
||||||
|
刷新和后端重启后恢复任务列表,已取得 job_id 的只查旧任务。已写提交意图但未收到 job_id 的进入 `unknown_submission`:只有服务端确认支持按键查重/幂等才安全重试,否则要求人工核实,不盲重投。旧 revision 的迟到结果不得覆盖新 revision。
|
||||||
|
|
||||||
|
任务查询 30–60 秒节奏并退避,页面关闭不导致重复提交;后端恢复后可继续查询。生成拒绝、失败、超时和排队必须分开。首版不自动返修;用户可明确发起新修订,累计自动返修不能超过原 Skill 的两次上限。
|
||||||
|
|
||||||
|
参考上传只发送批准图片的实际字节至工具返回并验证的 HTTPS 地址。媒体下载仅限本任务 artifact 的已验证安全地址,限制大小和超时,禁止私网、localhost、凭据 URL 和未验证重定向;不开放通用代理。成片检查在本地受限进程执行,禁止 FFmpeg 自行联网。
|
||||||
|
|
||||||
|
删除旧任务、批量清理、最终成片再入库、账户分配和广告发布都不在本次授权内。
|
||||||
|
|
||||||
|
## 验证与完成标准
|
||||||
|
|
||||||
|
按 TDD 先写失败测试,再实现。覆盖:CMS 完全未调用;跨会话/任意路径拒绝;上传和生成审批拒绝无副作用;重复提交;入库未就绪;进程中断后未知提交;原 job 恢复;迟到 revision;错误或 HTML 假冒 MP4;私网/重定向拒绝;HTTP Range;页面真实状态和未默认批准。
|
||||||
|
|
||||||
|
先用明确标注的测试图片和伪造边界响应验证工程行为,不将其宣称为真实 H3 成片。之后使用用户提供或明确确认可发送的样图,页面批准后真实生成一条目标时长视频:核对返回 job_id、非零 MP4、实际时长、分辨率、帧率、完整解码和音轨,并实际观看、试听。生成模型不保证声音或人物一致,失败应如实展示,不能只凭音轨存在就称音频检查通过。
|
||||||
|
|
||||||
|
运行 lint、typecheck、全量测试、build 和网页端到端验收。只有真实目标时长视频在网页可播放、下载并验证一致时,才声明该目标已完成。若生成后端不支持时长或不可用,明确区分工程已接入与真实出片尚未完成。
|
||||||
|
|
||||||
|
## 评审状态
|
||||||
|
|
||||||
|
本文件是新增生成子系统的待确认设计,不代表代码或生成验证已经完成。用户确认后在当前已有工作树内按实现计划推进;不再询问此前已确认的“不用 CMS / 使用 H3 / 网页播放下载”范围。
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
<svg xmlns="http://www.w3.org/2000/svg" width="600" height="360">
|
||||||
|
<rect width="600" height="360" fill="white"/>
|
||||||
|
<polygon points="110,100 45,245 175,245" fill="blue"/>
|
||||||
|
<circle cx="300" cy="180" r="65" fill="green"/>
|
||||||
|
<rect x="430" y="115" width="130" height="130" fill="red"/>
|
||||||
|
</svg>
|
||||||
|
After Width: | Height: | Size: 290 B |
+1
-1
@@ -5,7 +5,7 @@
|
|||||||
"type": "module",
|
"type": "module",
|
||||||
"packageManager": "pnpm@11.7.0",
|
"packageManager": "pnpm@11.7.0",
|
||||||
"engines": {
|
"engines": {
|
||||||
"node": ">=20"
|
"node": ">=22.19.0"
|
||||||
},
|
},
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"dev": "pnpm --parallel --filter @agent-studio/server --filter @agent-studio/web dev",
|
"dev": "pnpm --parallel --filter @agent-studio/server --filter @agent-studio/web dev",
|
||||||
|
|||||||
@@ -20,7 +20,12 @@
|
|||||||
"@agent-studio/shared": "workspace:*",
|
"@agent-studio/shared": "workspace:*",
|
||||||
"@earendil-works/pi-ai": "0.85.1",
|
"@earendil-works/pi-ai": "0.85.1",
|
||||||
"@earendil-works/pi-coding-agent": "^0.85.1",
|
"@earendil-works/pi-coding-agent": "^0.85.1",
|
||||||
|
"@modelcontextprotocol/sdk": "^1.30.0",
|
||||||
|
"@supabase/supabase-js": "2.116.0",
|
||||||
|
"ffmpeg-static": "5.3.0",
|
||||||
|
"ffprobe-static": "3.1.0",
|
||||||
"sharp": "^0.35.4",
|
"sharp": "^0.35.4",
|
||||||
"typebox": "1.3.7"
|
"typebox": "1.3.7",
|
||||||
|
"zod": "4.5.4"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import {
|
|||||||
createAgentSession,
|
createAgentSession,
|
||||||
type AgentSession,
|
type AgentSession,
|
||||||
type ExtensionFactory,
|
type ExtensionFactory,
|
||||||
|
type Skill,
|
||||||
type ToolDefinition,
|
type ToolDefinition,
|
||||||
} from "@earendil-works/pi-coding-agent";
|
} from "@earendil-works/pi-coding-agent";
|
||||||
import type { ApprovalBroker } from "../approvals/approval-broker.js";
|
import type { ApprovalBroker } from "../approvals/approval-broker.js";
|
||||||
@@ -40,6 +41,8 @@ export type CreatePiAgentSessionPortOptions = {
|
|||||||
agentDir: string;
|
agentDir: string;
|
||||||
customTools?: ToolDefinition[];
|
customTools?: ToolDefinition[];
|
||||||
approvalBroker?: ApprovalBroker;
|
approvalBroker?: ApprovalBroker;
|
||||||
|
skills?: Skill[];
|
||||||
|
appendSystemPrompt?: string[];
|
||||||
};
|
};
|
||||||
|
|
||||||
export function createPiAgentSessionPortFactory(
|
export function createPiAgentSessionPortFactory(
|
||||||
@@ -70,6 +73,9 @@ export function createPiAgentSessionPortFactory(
|
|||||||
settingsManager,
|
settingsManager,
|
||||||
extensionFactories,
|
extensionFactories,
|
||||||
noExtensions: true,
|
noExtensions: true,
|
||||||
|
noSkills: true,
|
||||||
|
skillsOverride: () => ({ skills: options.skills ?? [], diagnostics: [] }),
|
||||||
|
...(options.appendSystemPrompt === undefined ? {} : { appendSystemPrompt: options.appendSystemPrompt }),
|
||||||
noPromptTemplates: true,
|
noPromptTemplates: true,
|
||||||
noThemes: true,
|
noThemes: true,
|
||||||
noContextFiles: true,
|
noContextFiles: true,
|
||||||
@@ -80,7 +86,7 @@ export function createPiAgentSessionPortFactory(
|
|||||||
agentDir: options.agentDir,
|
agentDir: options.agentDir,
|
||||||
modelRuntime,
|
modelRuntime,
|
||||||
model,
|
model,
|
||||||
tools: ["read", "grep", "find", "ls", "mock_ads_metrics", "save_report_draft"],
|
tools: ["read", "grep", "find", "ls", ...(options.customTools ?? []).map((tool) => tool.name)],
|
||||||
customTools: options.customTools ?? [],
|
customTools: options.customTools ?? [],
|
||||||
resourceLoader,
|
resourceLoader,
|
||||||
settingsManager,
|
settingsManager,
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import type { EventStore } from "../events/event-store.js";
|
import type { EventStore } from "../events/event-store.js";
|
||||||
|
import { HarnessError } from "../errors/harness-error.js";
|
||||||
import { redactAndLimit } from "../security/redact.js";
|
import { redactAndLimit } from "../security/redact.js";
|
||||||
|
|
||||||
type SkillInferencePort = {
|
type SkillInferencePort = {
|
||||||
@@ -46,6 +47,18 @@ export class PiEventAdapter {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
if (value.type === "message_end" && isRecord(value.message) && value.message.role === "assistant") {
|
if (value.type === "message_end" && isRecord(value.message) && value.message.role === "assistant") {
|
||||||
|
if (value.message.stopReason === "toolUse") return;
|
||||||
|
if (value.message.stopReason === "error") {
|
||||||
|
throw new HarnessError("MODEL_REQUEST_FAILED", "Model request failed; check the configured model and gateway");
|
||||||
|
}
|
||||||
|
if (value.message.stopReason === "length") {
|
||||||
|
throw new HarnessError("MODEL_OUTPUT_LIMIT", "模型达到输出额度,回复未完成;请缩小本轮任务或检查模型输出配置");
|
||||||
|
}
|
||||||
|
if (Array.isArray(value.message.content) && !value.message.content.some(
|
||||||
|
(block: unknown) => isRecord(block) && block.type === "text" && typeof block.text === "string" && block.text.trim() !== "",
|
||||||
|
)) {
|
||||||
|
throw new HarnessError("MODEL_EMPTY_RESPONSE", "模型未返回可用正文或工具调用;本轮未成功完成,请检查网关响应与模型配置");
|
||||||
|
}
|
||||||
if (!this.#assistantCompleted) {
|
if (!this.#assistantCompleted) {
|
||||||
this.#assistantCompleted = true;
|
this.#assistantCompleted = true;
|
||||||
this.#eventStore.append(this.#sessionId, {
|
this.#eventStore.append(this.#sessionId, {
|
||||||
@@ -108,7 +121,12 @@ export class PiEventAdapter {
|
|||||||
payload: {
|
payload: {
|
||||||
toolCallId: event.toolCallId,
|
toolCallId: event.toolCallId,
|
||||||
toolName: event.toolName,
|
toolName: event.toolName,
|
||||||
result: redactAndLimit(event.result, this.#maxPayloadBytes),
|
result: redactAndLimit(
|
||||||
|
["i2v_create_task", "i2v_upload_reference", "i2v_submit", "i2v_status", "mock_ads_metrics", "save_report_draft", "generate_minis_report", "metrics_catalog", "metrics_semantic_query", "swads_whoami", "prepare_drama_sources", "editor_job_status", "read_drama_frames", "submit_drama_analysis", "propose_drama_selection", "submit_editor_plan", "render_editor_preview"].includes(event.toolName) &&
|
||||||
|
isRecord(event.result) && event.result.details !== undefined
|
||||||
|
? event.result.details : event.result,
|
||||||
|
this.#maxPayloadBytes,
|
||||||
|
),
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,205 @@
|
|||||||
|
import { createClient, type SupabaseClient } from "@supabase/supabase-js";
|
||||||
|
import { z } from "zod";
|
||||||
|
import { HarnessError } from "../errors/harness-error.js";
|
||||||
|
|
||||||
|
export const CMS_SUPABASE_URL = "https://lmcysqgfhoxppwuorsmu.supabase.co";
|
||||||
|
export const CMS_BACKEND_URL = "https://swave.pencil-stub.com/swave-api";
|
||||||
|
const Id = z.string().uuid();
|
||||||
|
const Drama = z.object({ id: Id, title: z.string(), synopsis: z.string().nullable(), language: z.string(), total_episodes: z.number().int().nonnegative(), status: z.string() });
|
||||||
|
const Episode = z.object({ id: Id, drama_id: Id, episode_no: z.number().int().positive(), title: z.string().nullable(), duration_seconds: z.number().nullable(), status: z.string() });
|
||||||
|
const Source = z.object({ episode_id: Id, provider: z.string(), playback_path: z.string().nullable() });
|
||||||
|
const Subtitle = z.object({ episode_id: Id, locale: z.string(), path: z.string().nullable() });
|
||||||
|
const dramaSelect = "id,title,synopsis,language,total_episodes,status";
|
||||||
|
type Options = { publishableKey: string; fetch?: typeof fetch };
|
||||||
|
|
||||||
|
export class CmsConnection {
|
||||||
|
#client: SupabaseClient | undefined;
|
||||||
|
#expiresAt = 0;
|
||||||
|
#generation = 0;
|
||||||
|
#loggingIn = false;
|
||||||
|
readonly #key: string;
|
||||||
|
readonly #fetch: typeof fetch;
|
||||||
|
|
||||||
|
public constructor(options: Options) {
|
||||||
|
this.#key = options.publishableKey;
|
||||||
|
const externalFetch = options.fetch ?? fetch;
|
||||||
|
this.#fetch = async (input, init) => {
|
||||||
|
const url = new URL(typeof input === "string" ? input : input instanceof URL ? input.href : input.url);
|
||||||
|
if (![new URL(CMS_SUPABASE_URL).origin, new URL(CMS_BACKEND_URL).origin].includes(url.origin)) throw new Error("CMS destination denied");
|
||||||
|
const timeout = AbortSignal.timeout(20_000);
|
||||||
|
const signal = init?.signal == null ? timeout : AbortSignal.any([init.signal, timeout]);
|
||||||
|
const response = await externalFetch(input, { ...init, redirect: "error", signal });
|
||||||
|
const reader = response.body?.getReader();
|
||||||
|
if (reader === undefined) return response;
|
||||||
|
const chunks: Uint8Array[] = [];
|
||||||
|
let size = 0;
|
||||||
|
try {
|
||||||
|
for (;;) {
|
||||||
|
const chunk = await reader.read();
|
||||||
|
if (chunk.done) break;
|
||||||
|
size += chunk.value.byteLength;
|
||||||
|
if (size > 4 * 1024 * 1024) throw new Error("CMS response limit exceeded");
|
||||||
|
chunks.push(chunk.value);
|
||||||
|
}
|
||||||
|
} finally { await reader.cancel().catch(() => undefined); }
|
||||||
|
return new Response(Buffer.concat(chunks), { status: response.status, statusText: response.statusText, headers: response.headers });
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
public status() {
|
||||||
|
const connected = this.#client !== undefined && this.#expiresAt > Date.now();
|
||||||
|
return { configured: this.#key.length > 0, connected, ...(connected ? { expiresAt: new Date(this.#expiresAt).toISOString() } : {}), project: "shorts-sg", cmsUrl: "https://swave.pencil-stub.com/swave/" };
|
||||||
|
}
|
||||||
|
|
||||||
|
public async login(email: string, password: string) {
|
||||||
|
if (!this.#key) throw new HarnessError("CMS_NOT_CONFIGURED", "CMS publishable key is missing");
|
||||||
|
if (this.#loggingIn) throw new HarnessError("SESSION_BUSY", "CMS login is already running");
|
||||||
|
z.object({ email: z.string().email().max(254), password: z.string().min(1).max(1024) }).parse({ email, password });
|
||||||
|
this.clear();
|
||||||
|
const generation = this.#generation;
|
||||||
|
this.#loggingIn = true;
|
||||||
|
const client = createClient(CMS_SUPABASE_URL, this.#key, { auth: { persistSession: false, autoRefreshToken: false, detectSessionInUrl: false }, global: { fetch: this.#fetch } });
|
||||||
|
try {
|
||||||
|
const result = await client.auth.signInWithPassword({ email, password });
|
||||||
|
if (result.error !== null || result.data.session === null) throw new HarnessError("CMS_LOGIN_FAILED", "CMS 登录失败,请检查账户、密码或验证码要求");
|
||||||
|
const user = await client.auth.getUser(result.data.session.access_token);
|
||||||
|
if (user.error !== null || user.data.user?.app_metadata.role !== "admin") throw new HarnessError("CMS_ADMIN_REQUIRED", "需要具有 CMS 管理员权限的 Supabase 账户");
|
||||||
|
if (generation !== this.#generation) throw new HarnessError("CMS_LOGIN_CANCELLED", "CMS login was cancelled");
|
||||||
|
this.#expiresAt = (result.data.session.expires_at ?? 0) * 1000;
|
||||||
|
if (this.#expiresAt <= Date.now()) throw new HarnessError("CMS_LOGIN_FAILED", "CMS 返回的登录会话已过期");
|
||||||
|
this.#client = client;
|
||||||
|
return this.status();
|
||||||
|
} catch (error) {
|
||||||
|
if (generation === this.#generation) this.clear();
|
||||||
|
if (error instanceof HarnessError) throw error;
|
||||||
|
throw new HarnessError("CMS_LOGIN_FAILED", "CMS 登录请求失败;未保存密码");
|
||||||
|
} finally { this.#loggingIn = false; }
|
||||||
|
}
|
||||||
|
|
||||||
|
public clear(): void {
|
||||||
|
this.#generation++;
|
||||||
|
this.#client = undefined;
|
||||||
|
this.#expiresAt = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
async #authenticated(): Promise<SupabaseClient> {
|
||||||
|
const client = this.#client;
|
||||||
|
if (client === undefined || this.#expiresAt <= Date.now()) {
|
||||||
|
this.clear();
|
||||||
|
throw new HarnessError("CMS_LOGIN_REQUIRED", "请先在网页 CMS 连接面板登录管理员账户");
|
||||||
|
}
|
||||||
|
const generation = this.#generation;
|
||||||
|
const { data, error } = await client.auth.getUser();
|
||||||
|
if (generation !== this.#generation) throw new HarnessError("CMS_LOGIN_REQUIRED", "CMS connection changed");
|
||||||
|
if (error !== null || data.user?.app_metadata.role !== "admin") {
|
||||||
|
this.clear();
|
||||||
|
throw new HarnessError("CMS_LOGIN_REQUIRED", "CMS 会话失效或管理员权限已撤销,请重新登录");
|
||||||
|
}
|
||||||
|
return client;
|
||||||
|
}
|
||||||
|
|
||||||
|
public async listDramas(query: string, offset: number) {
|
||||||
|
z.string().max(120).parse(query);
|
||||||
|
z.number().int().min(0).max(10_000).parse(offset);
|
||||||
|
return this.#safe(async () => {
|
||||||
|
const client = await this.#authenticated();
|
||||||
|
let request = client.from("drama").select(dramaSelect).order("title").order("id").range(offset, offset + 29);
|
||||||
|
if (query.trim()) request = request.ilike("title", `%${query.trim().replaceAll(/[\\%_]/g, "\\$&")}%`);
|
||||||
|
const { data, error } = await request;
|
||||||
|
if (error !== null) throw new Error();
|
||||||
|
return { dramas: z.array(Drama).parse(data), offset, pageSize: 30 };
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
public async inventory(dramaId: string) {
|
||||||
|
Id.parse(dramaId);
|
||||||
|
return this.#safe(async () => {
|
||||||
|
const client = await this.#authenticated();
|
||||||
|
const [dramaResult, episodeResult, categoryResult] = await Promise.all([
|
||||||
|
client.from("drama").select(dramaSelect).eq("id", dramaId).limit(1),
|
||||||
|
client.from("episode").select("id,drama_id,episode_no,title,duration_seconds,status").eq("drama_id", dramaId).order("episode_no").range(0, 999),
|
||||||
|
client.from("drama_category").select("drama_id,category(name)").eq("drama_id",dramaId).order("category_id").range(0,99),
|
||||||
|
]);
|
||||||
|
if (dramaResult.error !== null || episodeResult.error !== null || categoryResult.error !== null) throw new Error();
|
||||||
|
const categories=z.array(z.object({drama_id:Id,category:z.object({name:z.string()})})).parse(categoryResult.data);
|
||||||
|
if(categories.length>=100||categories.some(category=>category.drama_id!==dramaId))throw new HarnessError("CMS_CATEGORY_INVALID","CMS 分类返回不完整或不属于目标剧");
|
||||||
|
const tags=[...new Set(categories.map(category=>category.category.name))];
|
||||||
|
const drama = z.array(Drama).parse(dramaResult.data)[0];
|
||||||
|
if (drama === undefined || drama.id !== dramaId) throw new HarnessError("CMS_DRAMA_NOT_FOUND", "CMS 剧集不存在或无权访问");
|
||||||
|
const episodes = z.array(Episode).parse(episodeResult.data);
|
||||||
|
if (episodes.length >= 1000 || drama.total_episodes > 1000) throw new HarnessError("CMS_INVENTORY_TOO_LARGE", "剧集超过单次安全处理范围,未声称完整读取");
|
||||||
|
if (episodes.some((episode) => episode.drama_id !== dramaId)) throw new Error();
|
||||||
|
const sources: z.infer<typeof Source>[] = [];
|
||||||
|
const subtitles: z.infer<typeof Subtitle>[] = [];
|
||||||
|
for (let index = 0; index < episodes.length; index += 25) {
|
||||||
|
const ids = episodes.slice(index, index + 25).map((episode) => episode.id);
|
||||||
|
const [sourceResult, subtitleResult] = await Promise.all([
|
||||||
|
client.from("episode_source").select("episode_id,provider,playback_path").in("episode_id", ids).range(0, 999),
|
||||||
|
client.from("subtitle_track").select("episode_id,locale,path").in("episode_id", ids).range(0, 999),
|
||||||
|
]);
|
||||||
|
if (sourceResult.error !== null || subtitleResult.error !== null) throw new Error();
|
||||||
|
const sourceRows = z.array(Source).parse(sourceResult.data);
|
||||||
|
const subtitleRows = z.array(Subtitle).parse(subtitleResult.data);
|
||||||
|
if (sourceRows.length >= 1000 || subtitleRows.length >= 1000) throw new HarnessError("CMS_INVENTORY_TOO_LARGE", "源或字幕分页超限,目录不完整");
|
||||||
|
sources.push(...sourceRows); subtitles.push(...subtitleRows);
|
||||||
|
}
|
||||||
|
const missingEpisodeNumbers = Array.from({ length: drama.total_episodes }, (_, index) => index + 1).filter((number) => !episodes.some((episode) => episode.episode_no === number));
|
||||||
|
const rows = episodes.map((episode) => ({ id: episode.id, episodeNo: episode.episode_no, title: episode.title, durationSeconds: episode.duration_seconds, status: episode.status,
|
||||||
|
hasSource: sources.some((source) => source.episode_id === episode.id && Boolean(source.playback_path?.trim())),
|
||||||
|
subtitleLocales: [...new Set(subtitles.filter((subtitle) => subtitle.episode_id === episode.id && subtitle.path?.trim()).map((subtitle) => subtitle.locale))],
|
||||||
|
}));
|
||||||
|
return { drama:{...drama,tags}, fetchedAt: new Date().toISOString(), episodes: rows, missingEpisodeNumbers,
|
||||||
|
complete: drama.total_episodes > 0 && rows.length === drama.total_episodes && missingEpisodeNumbers.length === 0 && rows.every((episode) => episode.hasSource),
|
||||||
|
playbackVerified: false,
|
||||||
|
};
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
public async subtitle(dramaId: string, episodeId: string, locale: string) {
|
||||||
|
Id.parse(episodeId); z.string().regex(/^[a-zA-Z]{2,3}(?:-[a-zA-Z0-9]{2,8})*$/).max(24).parse(locale);
|
||||||
|
const inventory = await this.inventory(dramaId);
|
||||||
|
const episode = inventory.episodes.find((item) => item.id === episodeId);
|
||||||
|
if (episode === undefined) throw new HarnessError("CMS_EPISODE_NOT_FOUND", "该分集不属于所选剧集");
|
||||||
|
if (!episode.subtitleLocales.includes(locale)) throw new HarnessError("CMS_SUBTITLE_NOT_FOUND", "该分集未配置所选语言字幕");
|
||||||
|
return this.#safe(async () => {
|
||||||
|
const client = await this.#authenticated();
|
||||||
|
const { data } = await client.auth.getSession();
|
||||||
|
if (data.session === null) throw new HarnessError("CMS_LOGIN_REQUIRED", "CMS 会话已失效");
|
||||||
|
const response = await this.#fetch(`${CMS_BACKEND_URL}/api/v1/cms/playback/${episodeId}/subtitles/${locale}.vtt`, { headers: { Authorization: `Bearer ${data.session.access_token}` } });
|
||||||
|
if (!response.ok) throw new HarnessError("CMS_SUBTITLE_FAILED", `CMS 字幕接口返回 HTTP ${response.status.toString()}`);
|
||||||
|
const text = await response.text();
|
||||||
|
if (!text.trimStart().startsWith("WEBVTT") || text.length > 200_000) throw new HarnessError("CMS_SUBTITLE_INVALID", "CMS 字幕格式无效或过大");
|
||||||
|
return { dramaId, episodeId, locale, text, fetchedAt: new Date().toISOString() };
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Server-only capability. Never expose the signed URL in a model tool or event. */
|
||||||
|
public async playback(dramaId: string, episodeId: string) {
|
||||||
|
Id.parse(episodeId);
|
||||||
|
const inventory = await this.inventory(dramaId);
|
||||||
|
if (!inventory.episodes.some((episode) => episode.id === episodeId && episode.hasSource)) throw new HarnessError("CMS_EPISODE_NOT_FOUND", "所选分集没有属于该剧的可用源记录");
|
||||||
|
return this.#safe(async () => {
|
||||||
|
const client = await this.#authenticated();
|
||||||
|
const {data} = await client.auth.getSession();
|
||||||
|
if (!data.session) throw new HarnessError("CMS_LOGIN_REQUIRED", "CMS 会话已失效");
|
||||||
|
const response = await this.#fetch(`${CMS_BACKEND_URL}/api/v1/cms/playback/${episodeId}`, {headers:{Authorization:`Bearer ${data.session.access_token}`}});
|
||||||
|
if (!response.ok) throw new HarnessError("CMS_SIGNING_FAILED", "CMS 管理员签名接口失败;不代表片源缺失");
|
||||||
|
const signed = z.object({url:z.string().url(),expires_at:z.string().datetime({offset:true})}).parse(await response.json());
|
||||||
|
const url = new URL(signed.url);
|
||||||
|
if (url.origin !== "https://video.pencil-stub.com" || url.username || url.password || url.hash || !url.pathname.endsWith(".m3u8") || Date.parse(signed.expires_at)<=Date.now()) throw new HarnessError("CMS_SIGNING_INVALID", "CMS 返回的签名源不符合固定 CDN 安全范围");
|
||||||
|
return signed;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async #safe<T>(operation: () => Promise<T>): Promise<T> {
|
||||||
|
const generation = this.#generation;
|
||||||
|
try {
|
||||||
|
const result = await operation();
|
||||||
|
if (generation !== this.#generation) throw new HarnessError("CMS_LOGIN_REQUIRED", "CMS connection changed");
|
||||||
|
return result;
|
||||||
|
} catch (error) {
|
||||||
|
if (error instanceof HarnessError) throw error;
|
||||||
|
throw new HarnessError("CMS_REQUEST_FAILED", "CMS 请求失败或返回格式不符;未使用模拟数据替代");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
import { defineTool } from "@earendil-works/pi-coding-agent";
|
||||||
|
import { Type } from "typebox";
|
||||||
|
import type { CmsConnection } from "./cms-connection.js";
|
||||||
|
|
||||||
|
export const EDITOR_RESOURCES = [
|
||||||
|
"references/cms-source-contract.md", "references/target-drama-analysis.md", "references/creative-strategy-contract.md",
|
||||||
|
"references/viral-reference-research.md", "references/story-editing-rules.md", "references/recipe-schema.md",
|
||||||
|
"references/recipe.schema.json", "references/reference-candidates.schema.json", "references/continuity-contract.md",
|
||||||
|
"references/hook-design-contract.md", "references/qa-contract.md", "references/editorial-gates.md",
|
||||||
|
"references/local-delivery.md", "references/swads-delivery-contract.md", "requirements.txt",
|
||||||
|
"references/tiktok-adult-content-policy.md",
|
||||||
|
"scripts/common.py", "scripts/inspect_media.py", "scripts/validate_target_drama_analysis.py", "scripts/score_reference_candidates.py",
|
||||||
|
"scripts/validate_hook_map.py", "scripts/validate_recipe.py", "scripts/render_exact.py", "scripts/verify_audio_sync.py", "scripts/qa_video.py",
|
||||||
|
] as const;
|
||||||
|
|
||||||
|
function output(details: unknown) { return { content: [{ type: "text" as const, text: JSON.stringify(details) }], details }; }
|
||||||
|
const id = Type.String({ pattern: "^[a-fA-F0-9-]{36}$" });
|
||||||
|
export function createEditorTools(cms: CmsConnection,mediaCapabilities:()=>{rendering:boolean;vision:boolean;humanReview:boolean}=()=>({rendering:false,vision:false,humanReview:false})) {
|
||||||
|
return [
|
||||||
|
defineTool({ name: "cms_connection_status", label: "CMS connection", description: "Check this Session's CMS connection and configured media tools, not per-job readiness. Never request passwords in chat. If disconnected, direct the user to CMS 连接与选剧. Rendering remains gated by real analysis, reference and human reviews. Public reference search is not configured; actual reference evidence must be supplied. Do not fabricate readiness or files.", parameters: Type.Object({}, { additionalProperties: false }),
|
||||||
|
execute() { return Promise.resolve(output({ ...cms.status(), capabilities: { inventory: true, subtitles: true, referenceResearch: false, ...mediaCapabilities() }, note:"Capabilities mean registered tools, not analysis/content PASS. Use editor_job_status for an actual job." })); } }),
|
||||||
|
defineTool({ name: "cms_list_dramas", label: "CMS drama catalog", description: "Search the real CMS by title. Requires CMS administrator login and page approval before sending CMS metadata to the model. Resolve ambiguous titles with the user.",
|
||||||
|
parameters: Type.Object({ query: Type.String({ maxLength: 120 }), offset: Type.Integer({ minimum: 0, maximum: 10_000 }) }, { additionalProperties: false }),
|
||||||
|
async execute(_id, args) { return output(await cms.listDramas(args.query, args.offset)); } }),
|
||||||
|
defineTool({ name: "cms_drama_inventory", label: "CMS complete inventory", description: "Read real synopsis, episode inventory and configured subtitle languages for an exact drama ID. Requires page approval. complete checks database source coverage only; playbackVerified=false does NOT prove footage can be downloaded. Preserve fetchedAt for analysis v2.",
|
||||||
|
parameters: Type.Object({ dramaId: id }, { additionalProperties: false }),
|
||||||
|
async execute(_id, args) { return output(await cms.inventory(args.dramaId)); } }),
|
||||||
|
defineTool({ name: "cms_read_subtitle", label: "CMS subtitles", description: "Read real WebVTT from the authenticated CMS admin subtitle endpoint. Requires page approval and exact drama/episode binding. Subtitle evidence is not frame evidence and cannot substitute for viewing footage.",
|
||||||
|
parameters: Type.Object({ dramaId: id, episodeId: id, locale: Type.String({ maxLength: 24 }) }, { additionalProperties: false }),
|
||||||
|
async execute(_id, args) { return output(await cms.subtitle(args.dramaId, args.episodeId, args.locale)); } }),
|
||||||
|
];
|
||||||
|
}
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
import { z } from "zod";
|
||||||
|
|
||||||
|
export const ImageVideoDraftSchema = z.object({ attachmentId: z.string().regex(/^att_[\w-]+$/), title: z.string().trim().min(1).max(120), prompt: z.string().trim().min(1).max(4000), durationSeconds: z.number().positive().max(60) }).strict();
|
||||||
|
export type ImageVideoDraft = z.infer<typeof ImageVideoDraftSchema>;
|
||||||
|
export const taskIdSchema = z.string().regex(/^i2v_[0-9a-f-]{36}$/);
|
||||||
|
export const validationSchema = z.object({ videoSeconds: z.number().positive(), audioSeconds: z.number().positive(), width: z.number().int().positive(), height: z.number().int().positive(), fps: z.number().positive(), decoded: z.literal(true), sha256: z.string().regex(/^[a-f0-9]{64}$/), visualReview: z.literal("pending"), audioReview: z.literal("pending") }).strict();
|
||||||
|
export type ImageVideoValidation = z.infer<typeof validationSchema>;
|
||||||
|
export const recordSchema = z.object({
|
||||||
|
version: z.literal(1), id: taskIdSchema, ownerSessionId: z.string().regex(/^ses_[\w-]+$/), revision: z.number().int().nonnegative(),
|
||||||
|
draft: ImageVideoDraftSchema, requestHash: z.string().regex(/^[a-f0-9]{64}$/), referenceHash: z.string().regex(/^[a-f0-9]{64}$/), referenceType: z.enum(["image/jpeg", "image/png", "image/webp"]),
|
||||||
|
state: z.enum(["draft", "initializing_upload", "uploading", "finalizing", "importing", "awaiting_submission", "submitting", "queued", "running", "validating", "ready", "failed", "unknown_submission", "unknown_upload"]),
|
||||||
|
createdAt: z.string().datetime(), updatedAt: z.string().datetime(), lastPollAt: z.number().optional(),
|
||||||
|
identity: z.object({ tenantId: z.string().min(1), platform: z.string().min(1) }).strict().optional(),
|
||||||
|
continuations: z.array(z.object({ fromSessionId: z.string().regex(/^ses_[\w-]+$/), toSessionId: z.string().regex(/^ses_[\w-]+$/), revision: z.number().int().nonnegative(), at: z.string().datetime() }).strict()).max(100).optional(),
|
||||||
|
upload: z.object({ uploadId: z.string().min(1), objectKey: z.string().min(1), uploadUrl: z.string().url() }).strict().optional(),
|
||||||
|
assetId: z.string().min(1).optional(), jobId: z.string().min(1).max(200).optional(), artifactUrl: z.string().url().optional(),
|
||||||
|
error: z.string().max(500).optional(), validation: validationSchema.optional(),
|
||||||
|
}).strict();
|
||||||
|
export type ImageVideoRecord = z.infer<typeof recordSchema>;
|
||||||
|
export type ImageVideoPublicTask = { id: string; revision: number; title: string; prompt: string; durationSeconds: number; state: string; createdAt: string; updatedAt: string; referenceUrl: string; videoUrl?: string; jobId?: string; error?: string; validation?: ImageVideoValidation };
|
||||||
|
export interface ImageVideoRemote {
|
||||||
|
preflight(): Promise<{ tenantId: string; platform: string }>;
|
||||||
|
initialize(record: ImageVideoRecord, bytes: Buffer): Promise<{ uploadId: string; objectKey: string; uploadUrl: string } | { reused: true; assetId: string }>;
|
||||||
|
put(record: ImageVideoRecord, bytes: Buffer): Promise<void>;
|
||||||
|
finalize(record: ImageVideoRecord, bytes: Buffer): Promise<void>;
|
||||||
|
importStatus(record: ImageVideoRecord): Promise<{ state: "pending" | "ready" | "failed"; assetId?: string }>;
|
||||||
|
submit(record: ImageVideoRecord): Promise<{ jobId: string }>;
|
||||||
|
status(record: ImageVideoRecord): Promise<{ state: "queued" | "running" | "succeeded" | "failed"; artifactUrl?: string }>;
|
||||||
|
}
|
||||||
|
export const IMAGE_VIDEO_RESOURCES = ["references/selection.md", "references/swads.md", "references/local-processing.md", "scripts/media.py"];
|
||||||
@@ -0,0 +1,157 @@
|
|||||||
|
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
|
||||||
|
import { createHash, randomUUID } from "node:crypto";
|
||||||
|
import { lstat, open, rename } from "node:fs/promises";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";
|
||||||
|
import type { Transport } from "@modelcontextprotocol/sdk/shared/transport.js";
|
||||||
|
import { z } from "zod";
|
||||||
|
import { Check } from "typebox/value";
|
||||||
|
import type { TSchema } from "typebox";
|
||||||
|
import { HarnessError } from "../errors/harness-error.js";
|
||||||
|
import { SWADS_MCP_URL } from "../tools/swads-mcp.js";
|
||||||
|
import type { ImageVideoRecord, ImageVideoRemote } from "./contracts.js";
|
||||||
|
import { taskIdSchema } from "./contracts.js";
|
||||||
|
import { imageVideoTransfer, validateImageVideoUrl } from "./media.js";
|
||||||
|
|
||||||
|
const names = ["swads_whoami", "creative_upload_from_url", "creative_import_status", "video_h3_submit", "video_job_status"] as const;
|
||||||
|
const nonempty = z.string().min(1).max(2048);
|
||||||
|
export function assertImageVideoArguments(schema: Record<string, unknown>, args: Record<string, unknown>): void {
|
||||||
|
const properties = z.record(z.string(), z.unknown()).parse(schema.properties ?? {});
|
||||||
|
if (Object.keys(args).some(key => !(key in properties)) || !Check(schema as TSchema, args)) throw new HarnessError("I2V_SCHEMA_CHANGED", "SW Ads request schema changed; operation blocked");
|
||||||
|
}
|
||||||
|
export function imageVideoResponse(value: unknown): Record<string, unknown> {
|
||||||
|
try {
|
||||||
|
const envelope = z.object({ isError: z.boolean().optional(), structuredContent: z.unknown().optional(), content: z.array(z.object({ type: z.string(), text: z.string().optional() })).optional() }).parse(value);
|
||||||
|
if (envelope.isError) throw new Error("Remote error");
|
||||||
|
const texts = envelope.content?.filter(part => part.type === "text");
|
||||||
|
const result = envelope.structuredContent ?? (texts?.length === 1 && texts[0]?.text ? JSON.parse(texts[0].text) as unknown : undefined);
|
||||||
|
const body = z.record(z.string(), z.unknown()).parse(result);
|
||||||
|
if (body.error_code !== undefined) {
|
||||||
|
// Provider errors are sometimes returned as successful MCP text envelopes.
|
||||||
|
// Never forward arbitrary error details (which can contain credentials).
|
||||||
|
const code = body.error_code === "invalid_arguments" ? "invalid_arguments" : "remote_error";
|
||||||
|
throw new HarnessError("I2V_REMOTE_REJECTED", `SW Ads rejected the request (${code})`);
|
||||||
|
}
|
||||||
|
return body;
|
||||||
|
} catch (error) {
|
||||||
|
if (error instanceof HarnessError) throw error;
|
||||||
|
throw new HarnessError("I2V_RESPONSE_UNSUPPORTED", "SW Ads response contract is unclear; retain the task and inspect its receipt");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Dedicated capability set. The analytics client's allowlist is never broadened. */
|
||||||
|
export class ImageVideoMcpRemote implements ImageVideoRemote {
|
||||||
|
#connection: Promise<Client> | undefined;
|
||||||
|
#schemas = new Map<string, Record<string, unknown>>();
|
||||||
|
constructor(private readonly options: { token: string; receiptRoot?: string }) {}
|
||||||
|
private async connect() {
|
||||||
|
if (!this.options.token.trim()) throw new HarnessError("SWADS_NOT_CONFIGURED", "SWADS_MCP_TOKEN is not configured on the server");
|
||||||
|
this.#connection ??= (async () => {
|
||||||
|
const client = new Client({ name: "agent-studio-image-video", version: "1.0.0" });
|
||||||
|
try { await client.connect(new StreamableHTTPClientTransport(new URL(SWADS_MCP_URL), { requestInit: { redirect: "error", headers: { Authorization: `Bearer ${this.options.token}` } } }) as Transport, { timeout: 20_000 }); return client; }
|
||||||
|
catch { await client.close().catch(() => undefined); this.#connection = undefined; throw new HarnessError("I2V_CONNECTION_FAILED", "SW Ads connection failed"); }
|
||||||
|
})();
|
||||||
|
return this.#connection;
|
||||||
|
}
|
||||||
|
async close() { const client = await this.#connection?.catch(() => undefined); this.#connection = undefined; await client?.close(); }
|
||||||
|
async preflight() {
|
||||||
|
const client = await this.connect(); this.#schemas.clear();
|
||||||
|
let cursor: string | undefined;
|
||||||
|
do {
|
||||||
|
const page = await client.listTools(cursor ? { cursor } : {}, { timeout: 20_000 });
|
||||||
|
for (const tool of page.tools) if (names.some(name => name === tool.name)) this.#schemas.set(tool.name, tool.inputSchema);
|
||||||
|
cursor = page.nextCursor;
|
||||||
|
} while (cursor);
|
||||||
|
for (const name of names) if (!this.#schemas.has(name)) throw new HarnessError("I2V_CAPABILITY_MISSING", `Required capability unavailable: ${name}`);
|
||||||
|
const identity = z.object({ access: z.object({ current_account: z.object({ tenant_id: nonempty, platform: nonempty }) }) }).safeParse(await this.call("swads_whoami", {}));
|
||||||
|
if (!identity.success) throw new HarnessError("I2V_IDENTITY_AMBIGUOUS", "Current MCP tenant/platform is not unambiguous; upload blocked");
|
||||||
|
return { tenantId: identity.data.access.current_account.tenant_id, platform: identity.data.access.current_account.platform };
|
||||||
|
}
|
||||||
|
private async call(name: typeof names[number], args: Record<string, unknown>, receipt?: { record: ImageVideoRecord; operation: "initialize" | "finalize" | "submit" }): Promise<Record<string, unknown>> {
|
||||||
|
if (!names.includes(name)) throw new HarnessError("I2V_TOOL_DENIED", "Tool unavailable");
|
||||||
|
const schema = this.#schemas.get(name);
|
||||||
|
// Status collection after restart discovers the same live capability set.
|
||||||
|
if (!schema) { await this.preflight(); return this.call(name, args, receipt); }
|
||||||
|
assertImageVideoArguments(schema, args);
|
||||||
|
try {
|
||||||
|
const result = await (await this.connect()).callTool({ name, arguments: args }, undefined, { timeout: 60_000 });
|
||||||
|
if (receipt && this.options.receiptRoot) {
|
||||||
|
// Retain the exact upstream response BEFORE interpreting it. Only this
|
||||||
|
// process owner's private task directory receives signed URLs/details.
|
||||||
|
const r = receipt.record;
|
||||||
|
const directory = join(this.options.receiptRoot, taskIdSchema.parse(r.id));
|
||||||
|
const info = await lstat(directory);
|
||||||
|
if (!info.isDirectory() || info.isSymbolicLink()) throw new Error("Unsafe receipt directory");
|
||||||
|
const path = join(directory, `remote-${receipt.operation}-${randomUUID()}.json`);
|
||||||
|
const temp = `${path}.tmp`;
|
||||||
|
const file = await open(temp, "wx", 0o600);
|
||||||
|
try { await file.writeFile(JSON.stringify({ id: r.id, revision: r.revision, requestHash: r.requestHash, operation: receipt.operation, receivedAt: new Date().toISOString(), response: result })); await file.sync(); }
|
||||||
|
finally { await file.close(); }
|
||||||
|
await rename(temp, path);
|
||||||
|
}
|
||||||
|
return imageVideoResponse(result);
|
||||||
|
} catch (error) {
|
||||||
|
if (error instanceof HarnessError) throw error;
|
||||||
|
throw new HarnessError("I2V_REMOTE_FAILED", "SW Ads request failed or its response is unsupported; inspect saved task state");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
private uploadArgs(r: ImageVideoRecord) {
|
||||||
|
if (!r.identity) throw new HarnessError("I2V_IDENTITY_AMBIGUOUS", "Missing approved upload identity");
|
||||||
|
return { tenant_id: r.identity.tenantId, platforms: [r.identity.platform], platform: r.identity.platform, media_kind: "image", review_status: "review_pending", asset_role: "reference", ratio: "9:16", prompt: r.draft.prompt };
|
||||||
|
}
|
||||||
|
async initialize(r: ImageVideoRecord, bytes: Buffer) {
|
||||||
|
const body = await this.call("creative_upload_from_url", this.uploadMetadata(r, bytes), { record: r, operation: "initialize" });
|
||||||
|
if (body.reused === true) {
|
||||||
|
// Deduplicated uploads have no PUT URL. Accept only an existing asset
|
||||||
|
// cryptographically bound to the approved local image, never a URL guess.
|
||||||
|
const file = z.object({ creative_factory_asset_id: nonempty, mime_type: z.literal(r.referenceType), size_bytes: z.literal(bytes.length), attachment_kind: z.literal("image"), asset_metadata: z.object({ content_sha256: z.literal(r.referenceHash) }) }).safeParse(body.reused_file);
|
||||||
|
if (!file.success) throw new HarnessError("I2V_RESPONSE_UNSUPPORTED", "Reused reference does not match the frozen image");
|
||||||
|
return { reused: true as const, assetId: file.data.creative_factory_asset_id };
|
||||||
|
}
|
||||||
|
const receipt = z.object({ upload_id: nonempty, object_key: nonempty, upload_url: z.string().url() }).parse(body);
|
||||||
|
return { uploadId: receipt.upload_id, objectKey: receipt.object_key, uploadUrl: receipt.upload_url };
|
||||||
|
}
|
||||||
|
async put(r: ImageVideoRecord, bytes: Buffer) { if (!r.upload) throw new Error("Missing receipt"); await imageVideoTransfer(r.upload.uploadUrl, { bytes, contentType: r.referenceType }); }
|
||||||
|
private uploadMetadata(r: ImageVideoRecord, bytes: Buffer) {
|
||||||
|
return { ...this.uploadArgs(r), file_name: `${r.id}.${r.referenceType.split("/")[1]}`, content_type: r.referenceType, size_bytes: bytes.length, content_sha256: r.referenceHash };
|
||||||
|
}
|
||||||
|
async finalize(r: ImageVideoRecord, bytes: Buffer) {
|
||||||
|
if (!r.upload) throw new Error("Missing receipt");
|
||||||
|
const result = await this.call("creative_upload_from_url", { ...this.uploadMetadata(r, bytes), upload_id: r.upload.uploadId, object_key: r.upload.objectKey }, { record: r, operation: "finalize" });
|
||||||
|
if (result.status !== "finalizing") throw new HarnessError("I2V_RESPONSE_UNSUPPORTED", "Finalization receipt is unclear; poll the saved upload ID");
|
||||||
|
}
|
||||||
|
async importStatus(r: ImageVideoRecord) {
|
||||||
|
if (!r.upload) throw new Error("Missing receipt");
|
||||||
|
const value = await this.call("creative_import_status", { upload_id: r.upload.uploadId });
|
||||||
|
if (typeof value.creative_factory_asset_id === "string" && value.creative_factory_asset_id) return { state: "ready" as const, assetId: value.creative_factory_asset_id };
|
||||||
|
if (value.status === "failed") return { state: "failed" as const };
|
||||||
|
if (["pending", "finalizing", "processing"].includes(String(value.status))) return { state: "pending" as const };
|
||||||
|
throw new HarnessError("I2V_RESPONSE_UNSUPPORTED", "Import response has no unambiguous asset receipt");
|
||||||
|
}
|
||||||
|
async submit(r: ImageVideoRecord) {
|
||||||
|
if (!r.assetId) throw new Error("Missing reference asset");
|
||||||
|
// The gateway adds a caller namespace within a 128-character limit.
|
||||||
|
// Keep the full collision-resistant task/request binding in just 68 chars.
|
||||||
|
// Unknown submissions remain blocked by the service; never re-key a retry.
|
||||||
|
const key = `i2v:${createHash("sha256").update(`${r.id}/${r.requestHash}`).digest("hex")}`;
|
||||||
|
const result = z.object({ job_id: nonempty }).parse(await this.call("video_h3_submit", { prompt: r.draft.prompt, title: r.draft.title, reference_asset_ids: [r.assetId], aspect_ratio: "9:16", duration_seconds: r.draft.durationSeconds, resolution: "768p", h3_mode: "ref2va", idempotency_key: key, skill_id: "swads-image-to-video" }, { record: r, operation: "submit" }));
|
||||||
|
return { jobId: result.job_id };
|
||||||
|
}
|
||||||
|
async status(r: ImageVideoRecord) {
|
||||||
|
if (!r.jobId) throw new Error("Missing job");
|
||||||
|
const value = await this.call("video_job_status", { job_id: r.jobId });
|
||||||
|
if (value.job_id !== undefined && value.job_id !== r.jobId) throw new HarnessError("I2V_RESPONSE_UNSUPPORTED", "Returned job does not match registered job");
|
||||||
|
if (["pending", "queued"].includes(String(value.status))) return { state: "queued" as const };
|
||||||
|
if (["running", "processing"].includes(String(value.status))) return { state: "running" as const };
|
||||||
|
if (["failed", "rejected", "cancelled"].includes(String(value.status))) return { state: "failed" as const };
|
||||||
|
if (!["succeeded", "completed", "success"].includes(String(value.status))) throw new HarnessError("I2V_RESPONSE_UNSUPPORTED", "Unknown generation state");
|
||||||
|
const artifacts = z.array(z.object({ content_type: z.string(), job_id: nonempty.optional(), storage_url: z.string().url().nullish(), url: z.string().url().nullish() })).parse(value.artifacts).filter(artifact => artifact.content_type === "video/mp4");
|
||||||
|
if (artifacts.length !== 1 || !artifacts[0]) throw new HarnessError("I2V_RESPONSE_UNSUPPORTED", "Expected one registered MP4 artifact");
|
||||||
|
const artifact = artifacts[0];
|
||||||
|
if ((artifact.job_id !== undefined && artifact.job_id !== r.jobId) || (artifact.storage_url && artifact.url && artifact.storage_url !== artifact.url)) throw new HarnessError("I2V_RESPONSE_UNSUPPORTED", "Artifact receipt conflicts with registered job or URL");
|
||||||
|
const artifactUrl = artifact.storage_url ?? artifact.url;
|
||||||
|
if (!artifactUrl) throw new HarnessError("I2V_RESPONSE_UNSUPPORTED", "Registered MP4 has no storage URL");
|
||||||
|
await validateImageVideoUrl(artifactUrl);
|
||||||
|
return { state: "succeeded" as const, artifactUrl };
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,125 @@
|
|||||||
|
import { request } from "node:https";
|
||||||
|
import { lookup } from "node:dns/promises";
|
||||||
|
import { isIP } from "node:net";
|
||||||
|
import { randomUUID } from "node:crypto";
|
||||||
|
import { readFile, writeFile, rename } from "node:fs/promises";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { z } from "zod";
|
||||||
|
import { HarnessError } from "../errors/harness-error.js";
|
||||||
|
import { validateBaseUrl, type AddressResolver } from "../models/base-url-policy.js";
|
||||||
|
import { mediaRuntime, runMedia } from "../media/media-worker.js";
|
||||||
|
import { validationSchema } from "./contracts.js";
|
||||||
|
import { sha256 } from "./service.js";
|
||||||
|
|
||||||
|
export async function validateImageVideoUrl(value: string, resolver?: AddressResolver) {
|
||||||
|
let url: URL;
|
||||||
|
try { url = new URL(value); } catch { throw new HarnessError("I2V_URL_DENIED", "Invalid media URL"); }
|
||||||
|
// The storage provider may require a signed query. Only remove it from host validation.
|
||||||
|
const hostOnly = new URL(url); hostOnly.search = "";
|
||||||
|
let resolutionFailed = false;
|
||||||
|
const resolveAddresses: AddressResolver = resolver ?? (async hostname => (await lookup(hostname, { all: true })).map(result => ({ address: result.address, family: result.family as 4 | 6 })));
|
||||||
|
let checked;
|
||||||
|
try {
|
||||||
|
checked = await validateBaseUrl(hostOnly.href, {}, async hostname => {
|
||||||
|
try { return await resolveAddresses(hostname); } catch { resolutionFailed = true; throw new Error("DNS lookup failed"); }
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
if (resolutionFailed) throw new HarnessError("I2V_TRANSFER_TRANSIENT", "Media DNS lookup temporarily unavailable; retain the original job");
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
const address = checked.addresses[0];
|
||||||
|
if (!address) throw new HarnessError("I2V_URL_DENIED", "No public media address");
|
||||||
|
return { url, address };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** HTTPS with DNS pinned to the validated address; redirects and oversized bodies fail closed. */
|
||||||
|
export async function imageVideoTransfer(value: string, options: { bytes?: Buffer; contentType?: string; maxBytes?: number } = {}): Promise<Buffer> {
|
||||||
|
const { url, address } = await validateImageVideoUrl(value);
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const transient = () => new HarnessError("I2V_TRANSFER_TRANSIENT", "Media transfer temporarily unavailable; retain the original job");
|
||||||
|
const rejected = () => new HarnessError("I2V_TRANSFER_REJECTED", "Media response violates the transfer policy or bounds");
|
||||||
|
let failure: HarnessError | undefined;
|
||||||
|
const stop = (error: HarnessError) => { failure = error; req.destroy(error); };
|
||||||
|
const req = request(url, {
|
||||||
|
family: isIP(address),
|
||||||
|
method: options.bytes ? "PUT" : "GET",
|
||||||
|
headers: options.bytes ? { "Content-Type": options.contentType ?? "application/octet-stream", "Content-Length": options.bytes.length } : {},
|
||||||
|
lookup: (_hostname, _options, callback) => callback(null, address, isIP(address)),
|
||||||
|
}, response => {
|
||||||
|
if (!response.statusCode || response.statusCode < 200 || response.statusCode >= 300) {
|
||||||
|
const retryable = response.statusCode !== undefined && ([401, 403, 404, 408, 410, 429].includes(response.statusCode) || response.statusCode >= 500);
|
||||||
|
stop(retryable ? transient() : rejected()); response.destroy(); return;
|
||||||
|
}
|
||||||
|
const limit = options.maxBytes ?? (options.bytes ? 1024 * 1024 : 256 * 1024 * 1024);
|
||||||
|
if (Number(response.headers["content-length"] ?? 0) > limit) { stop(rejected()); response.destroy(); return; }
|
||||||
|
if (!options.bytes && !/^video\/mp4(?:;|$)/i.test(response.headers["content-type"] ?? "")) { stop(rejected()); response.destroy(); return; }
|
||||||
|
let total = 0; const chunks: Buffer[] = [];
|
||||||
|
response.on("data", (chunk: Buffer) => { total += chunk.length; if (total > limit) stop(rejected()); else chunks.push(chunk); });
|
||||||
|
response.once("error", () => { clearTimeout(timer); reject(failure ?? transient()); });
|
||||||
|
response.once("end", () => { clearTimeout(timer); resolve(Buffer.concat(chunks)); });
|
||||||
|
});
|
||||||
|
const timer = setTimeout(() => stop(transient()), 120_000);
|
||||||
|
req.once("error", () => { clearTimeout(timer); reject(failure ?? transient()); });
|
||||||
|
req.end(options.bytes);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export function parseImageVideoValidation(value: unknown, duration: number) {
|
||||||
|
const raw = z.object({ video_seconds: z.number().positive(), audio_seconds: z.number().positive(), width: z.number().positive(), height: z.number().positive(), fps: z.string(), decoded: z.literal(true), sha256: z.string() }).parse(value);
|
||||||
|
const parts = raw.fps.split("/"); const fps = Number(parts[0]) / Number(parts[1] ?? 1);
|
||||||
|
if (Math.abs(raw.video_seconds - duration) > 0.002 || Math.abs(raw.audio_seconds - duration) > 0.05 || Math.abs(raw.width / raw.height - 9 / 16) > 0.01 || fps > 120) throw new HarnessError("I2V_VALIDATION_FAILED", "H3 output does not match approved duration or portrait dimensions");
|
||||||
|
return validationSchema.parse({ videoSeconds: raw.video_seconds, audioSeconds: raw.audio_seconds, width: raw.width, height: raw.height, fps, decoded: true, sha256: raw.sha256, visualReview: "pending", audioReview: "pending" });
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function verifyImageVideo(path: string, directory: string, duration: number, options: { skillRoot: string; python: string }) {
|
||||||
|
const { ffmpeg, ffprobe } = await mediaRuntime();
|
||||||
|
// Patch the module object used by the unmodified original Skill. Every subprocess is
|
||||||
|
// restricted, including ffprobe, full decode and the audio-peak ffmpeg invocation.
|
||||||
|
const wrapper = `import runpy,sys,subprocess\noriginal=subprocess.run\nallowed=set(sys.argv[2:4])\ndef local_run(command,*args,**kwargs):\n if not isinstance(command,list) or command[0] not in allowed: raise RuntimeError('Unexpected media command')\n return original([command[0],'-protocol_whitelist','file,pipe','-format_whitelist','mov']+command[1:],*args,**kwargs)\nsubprocess.run=local_run\nscript=sys.argv[1]\nsys.argv=[script,'--ffmpeg',sys.argv[2],'--ffprobe',sys.argv[3],'verify','--input',sys.argv[4],'--minimum',sys.argv[5]]\nrunpy.run_path(script,run_name='__main__')`;
|
||||||
|
const result: unknown = JSON.parse(await runMedia(options.python, ["-c", wrapper, join(options.skillRoot, "scripts/media.py"), ffmpeg, ffprobe, path, String(duration)], directory, new AbortController().signal, 180_000));
|
||||||
|
return parseImageVideoValidation(result, duration);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** H3 may return a few extra frames. Trim only a small overrun, never stretch,
|
||||||
|
* loop, pad missing time or overwrite the provider's original MP4. */
|
||||||
|
export async function conformImageVideoDuration(source: string, directory: string, duration: number): Promise<string> {
|
||||||
|
const { ffmpeg, ffprobe } = await mediaRuntime();
|
||||||
|
const signal = new AbortController().signal;
|
||||||
|
const probe = z.object({ streams: z.array(z.object({ codec_type: z.string(), duration: z.string().optional(), start_time: z.string().optional(), avg_frame_rate: z.string().optional(), r_frame_rate: z.string().optional() })) }).parse(JSON.parse(await runMedia(ffprobe, ["-v", "error", "-protocol_whitelist", "file,pipe", "-format_whitelist", "mov", "-show_streams", "-of", "json", source], directory, signal)));
|
||||||
|
const video = probe.streams.filter(s => s.codec_type === "video");
|
||||||
|
const audio = probe.streams.filter(s => s.codec_type === "audio");
|
||||||
|
const vd = Number(video[0]?.duration); const ad = Number(audio[0]?.duration);
|
||||||
|
const [n, d = "1"] = (video[0]?.avg_frame_rate ?? "0").split("/"); const fps = Number(n) / Number(d);
|
||||||
|
// H3's verified worker rounds 24fps output up to 17k+5 frames:
|
||||||
|
// 2s -> 56 frames (2.333s), 10s -> 243 frames (10.125s).
|
||||||
|
// Extend the small-overrun allowance ONLY for that exact native grid result.
|
||||||
|
const nativeSeconds = (5 + 17 * Math.ceil((Math.max(5, Math.round(duration * 24)) - 5) / 17)) / 24;
|
||||||
|
const maximum = fps === 24 && Math.abs(vd - nativeSeconds) <= 0.002 ? Math.max(duration + 0.25, nativeSeconds + 0.002) : duration + 0.25;
|
||||||
|
if (video.length !== 1 || audio.length !== 1 || !Number.isFinite(duration) || duration <= 0 || !Number.isFinite(vd) || !Number.isFinite(ad) || !Number.isFinite(fps) || fps <= 0 || fps > 120 || video[0]?.avg_frame_rate !== video[0]?.r_frame_rate || Math.abs(Number(video[0]?.start_time)) > 0.002 || Math.abs(Number(audio[0]?.start_time)) > 0.05 || vd < duration - 0.002 || ad < duration - 0.002 || vd > maximum || ad > maximum || Math.abs(duration * fps - Math.round(duration * fps)) > 0.002) throw new HarnessError("I2V_VALIDATION_FAILED", "Output duration cannot be safely conformed without changing speed or adding content");
|
||||||
|
if (Math.abs(vd - duration) <= 0.002 && Math.abs(ad - duration) <= 0.05) return source;
|
||||||
|
const output = join(directory, `conformed-${randomUUID()}.mp4`);
|
||||||
|
await runMedia(ffmpeg, ["-nostdin", "-v", "error", "-protocol_whitelist", "file,pipe", "-format_whitelist", "mov", "-i", source, "-map", "0:v:0", "-map", "0:a:0", "-vf", `trim=duration=${duration},setpts=PTS-STARTPTS`, "-af", `atrim=duration=${duration},asetpts=PTS-STARTPTS`, "-c:v", "libx264", "-preset", "fast", "-crf", "18", "-c:a", "aac", "-b:a", "192k", "-movflags", "+faststart", "-n", output], directory, signal);
|
||||||
|
return output;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createImageVideoCollector(options: { skillRoot: string; python: string }) {
|
||||||
|
return async (url: string, directory: string, duration: number) => {
|
||||||
|
const bytes = await imageVideoTransfer(url);
|
||||||
|
if (bytes.length < 16 || bytes.subarray(4, 8).toString("ascii") !== "ftyp") throw new HarnessError("I2V_INVALID_MP4", "Artifact is not an MP4 file");
|
||||||
|
const source = join(directory, "candidate.mp4"); await localMediaStorage(writeFile(source, bytes, { mode: 0o600 }));
|
||||||
|
const path = await conformImageVideoDuration(source, directory, duration);
|
||||||
|
const validation = await verifyImageVideo(path, directory, duration, options);
|
||||||
|
if (validation.sha256 !== sha256(await localMediaStorage(readFile(path)))) throw new HarnessError("I2V_INVALID_MP4", "Validated media changed");
|
||||||
|
await localMediaStorage(writeFile(join(directory, "artifact-provenance.json"), JSON.stringify({ sourceSha256: sha256(bytes), outputSha256: validation.sha256, durationSeconds: duration, operation: source === path ? "unchanged" : "trim-small-tail-overrun-original-speed", checkedAt: new Date().toISOString() }), { mode: 0o600 }));
|
||||||
|
await localMediaStorage(rename(path, join(directory, "video.mp4")));
|
||||||
|
return validation;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function localMediaStorage<T>(operation: Promise<T>): Promise<T> {
|
||||||
|
try { return await operation; } catch (error) {
|
||||||
|
const code = error instanceof Error && "code" in error ? error.code : undefined;
|
||||||
|
if (typeof code === "string" && ["EIO", "EBUSY", "EMFILE", "ENFILE", "ENOSPC", "EAGAIN"].includes(code)) throw new HarnessError("I2V_TRANSFER_TRANSIENT", "Local media storage temporarily unavailable; retain the original job");
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,196 @@
|
|||||||
|
import { createHash, randomUUID } from "node:crypto";
|
||||||
|
import { lstat, mkdir, open, readFile, readdir, rename, stat } from "node:fs/promises";
|
||||||
|
import { join, resolve } from "node:path";
|
||||||
|
import type { AttachmentStore } from "../attachments/attachment-store.js";
|
||||||
|
import { HarnessError } from "../errors/harness-error.js";
|
||||||
|
import { ImageVideoDraftSchema, recordSchema, taskIdSchema, validationSchema, type ImageVideoDraft, type ImageVideoPublicTask, type ImageVideoRecord, type ImageVideoRemote, type ImageVideoValidation } from "./contracts.js";
|
||||||
|
|
||||||
|
const locks = new Map<string, Promise<unknown>>();
|
||||||
|
const submissionReceiptSchema = recordSchema.pick({ version: true, id: true, revision: true, requestHash: true, jobId: true }).required().strict();
|
||||||
|
export const sha256 = (bytes: Buffer | string) => createHash("sha256").update(bytes).digest("hex");
|
||||||
|
const fail = (code: string, message: string): never => { throw new HarnessError(code, message); };
|
||||||
|
function safeRemoteCode(error: unknown): string {
|
||||||
|
const codes = ["I2V_REMOTE_REJECTED", "I2V_RESPONSE_UNSUPPORTED", "I2V_REMOTE_FAILED", "I2V_SCHEMA_CHANGED", "I2V_TRANSFER_TRANSIENT"];
|
||||||
|
return error instanceof HarnessError && codes.includes(error.code) ? error.code : "I2V_OPERATION_FAILED";
|
||||||
|
}
|
||||||
|
export type ImageVideoOptions = { root: string; attachments: Pick<AttachmentStore, "get" | "openNormalized">; remote: ImageVideoRemote; collect: (url: string, directory: string, duration: number) => Promise<ImageVideoValidation> };
|
||||||
|
|
||||||
|
/** One process owns this private store. Per-task locks are shared by service instances. */
|
||||||
|
export class ImageVideoService {
|
||||||
|
readonly root: string;
|
||||||
|
constructor(private readonly options: ImageVideoOptions) { this.root = resolve(options.root); }
|
||||||
|
|
||||||
|
async create(sessionId: string, input: ImageVideoDraft): Promise<ImageVideoPublicTask> {
|
||||||
|
const draft = ImageVideoDraftSchema.parse(input);
|
||||||
|
const attachment = this.options.attachments.get(sessionId, draft.attachmentId);
|
||||||
|
if (!attachment || attachment.sessionId !== sessionId) return fail("I2V_TASK_NOT_FOUND", "Reference image not found in this Session");
|
||||||
|
const bytes = await this.options.attachments.openNormalized(sessionId, draft.attachmentId);
|
||||||
|
const now = new Date().toISOString(); const id = `i2v_${randomUUID()}`;
|
||||||
|
const record = recordSchema.parse({ version: 1, id, ownerSessionId: sessionId, revision: 0, draft, referenceHash: sha256(bytes), requestHash: sha256(JSON.stringify(draft)), referenceType: attachment.mediaType, state: "draft", createdAt: now, updatedAt: now });
|
||||||
|
await mkdir(join(this.root, id), { recursive: true, mode: 0o700 });
|
||||||
|
await this.atomic(join(this.root, id, "reference"), bytes);
|
||||||
|
await this.atomic(join(this.root, id, "record.json"), JSON.stringify(record));
|
||||||
|
return this.public(record);
|
||||||
|
}
|
||||||
|
|
||||||
|
async list(): Promise<ImageVideoPublicTask[]> {
|
||||||
|
const entries = await readdir(this.root).catch(() => []);
|
||||||
|
const tasks = await Promise.all(entries.filter(id => taskIdSchema.safeParse(id).success).map(id => this.view(id).catch(() => undefined)));
|
||||||
|
return tasks.filter((task): task is ImageVideoPublicTask => task !== undefined).sort((a, b) => b.createdAt.localeCompare(a.createdAt));
|
||||||
|
}
|
||||||
|
async view(id: string): Promise<ImageVideoPublicTask> { return this.lock(id, async () => this.public(await this.read(id))); }
|
||||||
|
async owned(sessionId: string, id: string): Promise<ImageVideoPublicTask> { return this.lock(id, async () => { const r = await this.read(id); this.owner(r, sessionId); return this.public(r); }); }
|
||||||
|
|
||||||
|
/** Local-page confirmation only. Not exposed to the model as a tool. */
|
||||||
|
async continueTask(sessionId: string, id: string, expectedRevision: number, ownerIsActive: (sessionId: string) => boolean, requireTargetSession: () => void): Promise<ImageVideoPublicTask> {
|
||||||
|
return this.lock(id, async () => {
|
||||||
|
const r = await this.read(id);
|
||||||
|
requireTargetSession();
|
||||||
|
if (r.revision !== expectedRevision) return fail("I2V_STALE_REVISION", "任务已更新,请重新核对后接续。");
|
||||||
|
if (!["draft", "awaiting_submission"].includes(r.state) || r.jobId || (r.state === "awaiting_submission" && !r.assetId)) return fail("I2V_CONTINUE_BLOCKED", "当前状态不允许接续;不得重置未知结果或重新生成已有任务。");
|
||||||
|
if (r.ownerSessionId === sessionId) return this.public(r);
|
||||||
|
if (ownerIsActive(r.ownerSessionId)) return fail("I2V_OWNER_ACTIVE", "原会话仍有效,请回原页面继续,不能由其他会话接管。");
|
||||||
|
if ((r.continuations?.length ?? 0) >= 100) return fail("I2V_CONTINUE_BLOCKED", "接续记录已达上限,请人工核实。");
|
||||||
|
await this.reference(r);
|
||||||
|
requireTargetSession();
|
||||||
|
r.continuations = [...(r.continuations ?? []), { fromSessionId: r.ownerSessionId, toSessionId: sessionId, revision: r.revision, at: new Date().toISOString() }];
|
||||||
|
r.ownerSessionId = sessionId;
|
||||||
|
await this.save(r);
|
||||||
|
return this.public(r);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async upload(sessionId: string, id: string): Promise<ImageVideoPublicTask> {
|
||||||
|
return this.lock(id, async () => {
|
||||||
|
const r = await this.read(id); this.owner(r, sessionId);
|
||||||
|
if (r.state === "unknown_upload") return fail("I2V_UNKNOWN_UPLOAD", "Upload outcome unknown; inspect the saved receipt before any new import");
|
||||||
|
if (r.state !== "draft") return this.public(r);
|
||||||
|
// Read-only discovery and identity validation happen before any remote mutation intent.
|
||||||
|
r.identity = await this.options.remote.preflight();
|
||||||
|
const bytes = await this.reference(r);
|
||||||
|
r.state = "initializing_upload"; await this.save(r);
|
||||||
|
try {
|
||||||
|
const initialized = await this.options.remote.initialize(r, bytes);
|
||||||
|
if ("reused" in initialized) {
|
||||||
|
r.assetId = initialized.assetId; r.state = "awaiting_submission"; delete r.error; await this.save(r);
|
||||||
|
return this.public(r);
|
||||||
|
}
|
||||||
|
r.upload = initialized; r.state = "uploading"; await this.save(r);
|
||||||
|
await this.options.remote.put(r, bytes); r.state = "finalizing"; await this.save(r);
|
||||||
|
await this.options.remote.finalize(r, bytes); r.state = "importing"; await this.save(r);
|
||||||
|
} catch (error) {
|
||||||
|
const stage = r.state;
|
||||||
|
const code = safeRemoteCode(error);
|
||||||
|
r.state = "unknown_upload"; r.error = `参考图上传或入库未完成(${stage}/${code});禁止重复导入,请核实本任务收据。`; await this.save(r);
|
||||||
|
}
|
||||||
|
return this.public(r);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async submit(sessionId: string, id: string): Promise<ImageVideoPublicTask> {
|
||||||
|
return this.lock(id, async () => {
|
||||||
|
const r = await this.read(id); this.owner(r, sessionId);
|
||||||
|
if (r.state === "unknown_submission") return fail("I2V_UNKNOWN_SUBMISSION", "Submission outcome unknown; no safe retry or new billing request is allowed");
|
||||||
|
if (r.jobId) return this.public(r);
|
||||||
|
if (r.state !== "awaiting_submission" || !r.assetId) return fail("I2V_NOT_READY", "Reference import must be ready before generation approval");
|
||||||
|
await this.options.remote.preflight(); await this.reference(r);
|
||||||
|
r.state = "submitting"; await this.save(r);
|
||||||
|
try {
|
||||||
|
// Deliberately independent of chat cancellation: retain any returned receipt.
|
||||||
|
const receipt = await this.options.remote.submit(r);
|
||||||
|
if (!receipt.jobId) throw new Error("Missing job ID");
|
||||||
|
await this.atomic(join(this.root, id, "submission-receipt.json"), JSON.stringify({ version: 1, id, revision: r.revision, requestHash: r.requestHash, jobId: receipt.jobId }));
|
||||||
|
r.jobId = receipt.jobId; r.state = "queued"; delete r.lastPollAt; await this.save(r);
|
||||||
|
} catch (error) { r.state = "unknown_submission"; r.error = `生成提交结果未知(submitting/${safeRemoteCode(error)});禁止重投。请核实私有回执,避免重复计费。`; await this.save(r); }
|
||||||
|
return this.public(r);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async refresh(id: string): Promise<ImageVideoPublicTask> {
|
||||||
|
return this.lock(id, async () => {
|
||||||
|
const r = await this.read(id);
|
||||||
|
if (r.state === "unknown_submission" || r.state === "ready" || r.state === "failed" || Date.now() - (r.lastPollAt ?? 0) < 30_000) return this.public(r);
|
||||||
|
if (!r.jobId && !(r.upload && ["importing", "unknown_upload"].includes(r.state))) return this.public(r);
|
||||||
|
r.lastPollAt = Date.now(); await this.save(r);
|
||||||
|
let collecting = false;
|
||||||
|
try {
|
||||||
|
if (!r.jobId) {
|
||||||
|
const result = await this.options.remote.importStatus(r);
|
||||||
|
if (result.state === "ready" && result.assetId) { r.assetId = result.assetId; r.state = "awaiting_submission"; delete r.error; }
|
||||||
|
else if (result.state === "failed") { r.state = "failed"; r.error = "参考图入库失败。"; }
|
||||||
|
} else {
|
||||||
|
const result = await this.options.remote.status(r);
|
||||||
|
if (result.state === "succeeded") {
|
||||||
|
if (!result.artifactUrl) throw new Error("No unambiguous MP4 artifact");
|
||||||
|
r.artifactUrl = result.artifactUrl; r.state = "validating"; await this.save(r);
|
||||||
|
collecting = true;
|
||||||
|
r.validation = validationSchema.parse(await this.options.collect(result.artifactUrl, join(this.root, id), r.draft.durationSeconds));
|
||||||
|
if (Math.abs(r.validation.videoSeconds - r.draft.durationSeconds) > 0.002 || Math.abs(r.validation.audioSeconds - r.draft.durationSeconds) > 0.05) throw new Error("Duration mismatch");
|
||||||
|
r.state = "ready"; delete r.error;
|
||||||
|
} else { r.state = result.state; if (result.state === "failed") r.error = "H3 生成失败;未自动重投。"; }
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
const retryCollection = collecting && error instanceof HarnessError && error.code === "I2V_TRANSFER_TRANSIENT";
|
||||||
|
if (collecting && !retryCollection) r.state = "failed";
|
||||||
|
r.error = retryCollection ? "成片收取暂时失败;30 秒后查询原任务并重试收取,不会重新生成。" : r.state === "failed" ? "成片技术验证失败:请核实 MP4、精确时长、尺寸、帧率和音轨;未自动重投。" : "状态响应暂不可用或契约不明确;保留原任务,下次只读查询。";
|
||||||
|
}
|
||||||
|
await this.save(r); return this.public(r);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async media(id: string, kind: "reference" | "video") {
|
||||||
|
return this.lock(id, async () => {
|
||||||
|
const r = await this.read(id);
|
||||||
|
if (kind === "video" && (r.state !== "ready" || !r.validation)) return fail("I2V_MEDIA_NOT_FOUND", "Validated video not ready");
|
||||||
|
const path = join(this.root, id, kind === "reference" ? "reference" : "video.mp4");
|
||||||
|
await this.regular(path); const bytes = await readFile(path);
|
||||||
|
if (sha256(bytes) !== (kind === "reference" ? r.referenceHash : r.validation?.sha256)) return fail("I2V_MEDIA_INVALID", "Registered media hash mismatch");
|
||||||
|
return { path, size: (await stat(path)).size, contentType: kind === "reference" ? r.referenceType : "video/mp4" };
|
||||||
|
});
|
||||||
|
}
|
||||||
|
private owner(r: ImageVideoRecord, sessionId: string) { if (r.ownerSessionId !== sessionId) fail("I2V_TASK_NOT_FOUND", "Task not found in this Session"); }
|
||||||
|
private public(r: ImageVideoRecord): ImageVideoPublicTask {
|
||||||
|
return { id: r.id, revision: r.revision, title: r.draft.title, prompt: r.draft.prompt, durationSeconds: r.draft.durationSeconds, state: r.state, createdAt: r.createdAt, updatedAt: r.updatedAt, referenceUrl: `/api/image-video/tasks/${r.id}/reference`, ...(r.jobId ? { jobId: r.jobId } : {}), ...(r.error ? { error: r.error } : {}), ...(r.validation ? { validation: r.validation } : {}), ...(r.state === "ready" ? { videoUrl: `/api/image-video/tasks/${r.id}/video` } : {}) };
|
||||||
|
}
|
||||||
|
private async reference(r: ImageVideoRecord) { await this.regular(join(this.root, r.id, "reference")); const bytes = await readFile(join(this.root, r.id, "reference")); if (sha256(bytes) !== r.referenceHash) fail("I2V_MEDIA_INVALID", "Reference changed after draft"); return bytes; }
|
||||||
|
private async regular(path: string) { const info = await lstat(path); if (!info.isFile() || info.isSymbolicLink()) fail("I2V_RECORD_INVALID", "Unsafe registered file"); }
|
||||||
|
private async matchingSubmissionReceipt(r: ImageVideoRecord): Promise<string | undefined> {
|
||||||
|
const path = join(this.root, r.id, "submission-receipt.json");
|
||||||
|
try {
|
||||||
|
await this.regular(path);
|
||||||
|
const receipt = submissionReceiptSchema.parse(JSON.parse(await readFile(path, "utf8")));
|
||||||
|
if (receipt.id !== r.id || receipt.requestHash !== r.requestHash || receipt.revision !== r.revision || !receipt.jobId.trim() || (r.jobId !== undefined && r.jobId !== receipt.jobId)) return undefined;
|
||||||
|
return receipt.jobId;
|
||||||
|
} catch { return undefined; } // Preserve absent, malformed and conflicting evidence for manual reconciliation.
|
||||||
|
}
|
||||||
|
private async read(id: string): Promise<ImageVideoRecord> {
|
||||||
|
if (!taskIdSchema.safeParse(id).success) return fail("I2V_TASK_NOT_FOUND", "Task not found");
|
||||||
|
const path = join(this.root, id, "record.json");
|
||||||
|
const dir = await lstat(join(this.root, id)).catch(() => undefined);
|
||||||
|
if (!dir || !dir.isDirectory() || dir.isSymbolicLink()) return fail("I2V_TASK_NOT_FOUND", "Task not found");
|
||||||
|
try {
|
||||||
|
await this.regular(path); const r = recordSchema.parse(JSON.parse(await readFile(path, "utf8")));
|
||||||
|
if (r.id !== id || r.requestHash !== sha256(JSON.stringify(r.draft))) throw new Error("Binding mismatch");
|
||||||
|
if (r.state === "submitting") {
|
||||||
|
const jobId = await this.matchingSubmissionReceipt(r);
|
||||||
|
if (jobId !== undefined) { r.jobId = jobId; r.state = "queued"; delete r.lastPollAt; delete r.error; await this.save(r); }
|
||||||
|
else { r.state = "unknown_submission"; await this.save(r); }
|
||||||
|
}
|
||||||
|
if (["initializing_upload", "uploading", "finalizing"].includes(r.state)) { r.state = "unknown_upload"; await this.save(r); }
|
||||||
|
return r;
|
||||||
|
} catch { return fail("I2V_RECORD_INVALID", "Task record is invalid; manual recovery required"); }
|
||||||
|
}
|
||||||
|
private async save(r: ImageVideoRecord) {
|
||||||
|
const path = join(this.root, r.id, "record.json");
|
||||||
|
const previous = recordSchema.parse(JSON.parse(await readFile(path, "utf8")));
|
||||||
|
if (previous.revision !== r.revision) return fail("I2V_STALE_REVISION", "Stale task result rejected");
|
||||||
|
r.revision++; r.updatedAt = new Date().toISOString();
|
||||||
|
await this.atomic(path, JSON.stringify(recordSchema.parse(r)));
|
||||||
|
}
|
||||||
|
private async atomic(path: string, content: Buffer | string) { const temp = `${path}.${randomUUID()}.tmp`; const file = await open(temp, "wx", 0o600); try { await file.writeFile(content); await file.sync(); } finally { await file.close(); } await rename(temp, path); }
|
||||||
|
private async lock<T>(id: string, operation: () => Promise<T>): Promise<T> {
|
||||||
|
const key = `${this.root}/${id}`; const previous = locks.get(key) ?? Promise.resolve();
|
||||||
|
const work = previous.catch(() => undefined).then(operation); locks.set(key, work);
|
||||||
|
try { return await work; } finally { if (locks.get(key) === work) locks.delete(key); }
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
import { defineTool } from "@earendil-works/pi-coding-agent";
|
||||||
|
import { Type } from "typebox";
|
||||||
|
import type { ImageVideoService } from "./service.js";
|
||||||
|
const task = Type.Object({ taskId: Type.String({ pattern: "^i2v_[0-9a-f-]{36}$" }) }, { additionalProperties: false });
|
||||||
|
export function createImageVideoTools(service: ImageVideoService, sessionId: string) {
|
||||||
|
const output = (details: unknown) => ({ content: [{ type: "text" as const, text: JSON.stringify(details) }], details });
|
||||||
|
return [
|
||||||
|
defineTool({ name: "i2v_create_task", label: "Prepare local image video draft", description: "Read swads-image-to-video SKILL.md and references first. Copy this Session's existing attachment into a durable local draft. No external transmission. Use the user's exact title, prompt and duration; default portrait 9:16, H3 ref2va,768p. A form-created taskId already exists: do not create it again. Image display does not send it to the chat model. Never require CMS.", parameters: Type.Object({ attachmentId: Type.String(), title: Type.String(), prompt: Type.String(), durationSeconds: Type.Number() }, { additionalProperties: false }), async execute(_id, args) { return output(await service.create(sessionId, args)); } }),
|
||||||
|
defineTool({ name: "i2v_upload_reference", label: "Approve reference transfer", description: "Separate explicit page approval: transfer this registered draft's frozen image to SW Ads storage and create review_pending, unassigned reference asset. No account assignment or publishing. Arguments accept only taskId. Refusal performs no transfer. Wait for import status before requesting generation approval.", parameters: task, async execute(_id, args) { return output(await service.upload(sessionId, args.taskId)); } }),
|
||||||
|
defineTool({ name: "i2v_submit", label: "Approve H3 generation", description: "Separate explicit page approval: submit this registered task's exact frozen prompt, reference asset and target duration once to H3 ref2va,9:16,768p. Cost unknown; target duration support unverified. Never modify duration, retry unknown submission, or switch provider. Returns durable jobId immediately; UI polls the original job without an LLM. Technical preview is not human-reviewed.", parameters: task, async execute(_id, args) { return output(await service.submit(sessionId, args.taskId)); } }),
|
||||||
|
defineTool({ name: "i2v_status", label: "Read registered image video task", description: "Read only this Session's registered task. Poll at most every 30 seconds; preserve queued or unknown state. Does not submit or approve. The webpage restores task history after restart.", parameters: task, async execute(_id, args) { await service.owned(sessionId, args.taskId); return output(await service.refresh(args.taskId)); } }),
|
||||||
|
];
|
||||||
|
}
|
||||||
@@ -23,4 +23,16 @@ export * from "./testing/fake-agent-session.js";
|
|||||||
export * from "./tools/mock-ads-metrics.js";
|
export * from "./tools/mock-ads-metrics.js";
|
||||||
export * from "./tools/save-report-draft.js";
|
export * from "./tools/save-report-draft.js";
|
||||||
export * from "./tools/tool-policy.js";
|
export * from "./tools/tool-policy.js";
|
||||||
|
export * from "./tools/swads-mcp.js";
|
||||||
|
export * from "./tools/minis-report.js";
|
||||||
export * from "./workspace/safe-workspace.js";
|
export * from "./workspace/safe-workspace.js";
|
||||||
|
export * from "./cms/cms-connection.js";
|
||||||
|
export * from "./cms/editor-tools.js";
|
||||||
|
export * from "./media/editor-service.js";
|
||||||
|
export * from "./media/editor-tools.js";
|
||||||
|
export * from "./media/skill-runner.js";
|
||||||
|
export * from "./image-video/contracts.js";
|
||||||
|
export * from "./image-video/service.js";
|
||||||
|
export * from "./image-video/media.js";
|
||||||
|
export * from "./image-video/mcp.js";
|
||||||
|
export * from "./image-video/tools.js";
|
||||||
|
|||||||
@@ -0,0 +1,30 @@
|
|||||||
|
import {z} from "zod";
|
||||||
|
import {HarnessError} from "../errors/harness-error.js";
|
||||||
|
|
||||||
|
export type AnalysisEvidence={analysisId:string;dramaId:string;fetchedAt:string;title:string;synopsis:string|null;tags:string[];episodeCount:number;
|
||||||
|
sources:{episodeId:string;episodeNo:number;duration:number;subtitle:string;frames:{id:string;time:number}[]}[]};
|
||||||
|
const Shape=z.object({analysis_id:z.string(),drama:z.object({drama_id:z.string(),title:z.string(),synopsis:z.string(),tags:z.array(z.string()),episode_count:z.number()}),cms_source:z.object({fetched_at:z.string()}),evidence:z.object({
|
||||||
|
frame_evidence:z.array(z.object({episode:z.number(),episode_id:z.string(),frame_id:z.string(),time:z.number().finite(),claim:z.string().min(1)})).min(2),
|
||||||
|
subtitle_evidence:z.array(z.object({episode:z.number(),episode_id:z.string(),story_position:z.enum(["opening","middle","late"]),quote:z.string().trim().min(3)})).min(3),
|
||||||
|
})});
|
||||||
|
export function assertAnalysisBinding(value:unknown,expected:AnalysisEvidence):void {
|
||||||
|
const fail=():never=>{throw new HarnessError("ANALYSIS_EVIDENCE_MISMATCH","分析必须绑定当前任务的真实 CMS、字幕和帧证据,不能自行修改身份或补造证据");};
|
||||||
|
const parsed=Shape.safeParse(value);if(!parsed.success)fail();
|
||||||
|
const actual=parsed.data!;
|
||||||
|
if(actual.analysis_id!==expected.analysisId||actual.drama.drama_id!==expected.dramaId||actual.cms_source.fetched_at!==expected.fetchedAt||actual.drama.title!==expected.title||actual.drama.synopsis!==expected.synopsis||actual.drama.episode_count!==expected.episodeCount||JSON.stringify([...actual.drama.tags].sort())!==JSON.stringify([...expected.tags].sort()))fail();
|
||||||
|
const seen=new Set<string>();
|
||||||
|
for(const frame of actual.evidence.frame_evidence){
|
||||||
|
const source=expected.sources.find(s=>s.episodeId===frame.episode_id&&s.episodeNo===frame.episode);
|
||||||
|
const original=source?.frames.find(f=>f.id===frame.frame_id);
|
||||||
|
if(!original||Math.abs(original.time-frame.time)>0.001||seen.has(frame.frame_id))fail();
|
||||||
|
seen.add(frame.frame_id);
|
||||||
|
}
|
||||||
|
const positions={opening:1,middle:Math.floor(expected.episodeCount/2)+1,late:expected.episodeCount};
|
||||||
|
const covered=new Set<string>();
|
||||||
|
for(const subtitle of actual.evidence.subtitle_evidence){
|
||||||
|
const source=expected.sources.find(s=>s.episodeId===subtitle.episode_id&&s.episodeNo===subtitle.episode);
|
||||||
|
if(!source||positions[subtitle.story_position]!==source.episodeNo||!source.subtitle.includes(subtitle.quote))fail();
|
||||||
|
covered.add(subtitle.story_position);
|
||||||
|
}
|
||||||
|
if(covered.size!==3)fail();
|
||||||
|
}
|
||||||
@@ -0,0 +1,125 @@
|
|||||||
|
import { randomUUID, createHash } from "node:crypto";
|
||||||
|
import {createReadStream} from "node:fs";
|
||||||
|
import { mkdir, mkdtemp, readFile, writeFile, rename, stat } from "node:fs/promises";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { z } from "zod";
|
||||||
|
import type { CmsConnection } from "../cms/cms-connection.js";
|
||||||
|
import { HarnessError } from "../errors/harness-error.js";
|
||||||
|
import { downloadHls, mediaRuntime, prepareLocalMedia } from "./media-worker.js";
|
||||||
|
import { SafeWorkspace } from "../workspace/safe-workspace.js";
|
||||||
|
import {assertAnalysisBinding} from "./analysis-binding.js";
|
||||||
|
import type {SkillRunner} from "./skill-runner.js";
|
||||||
|
import {EditorWorkflow} from "./editor-workflow.js";
|
||||||
|
|
||||||
|
type Frame={id:string;time:number;path:string};
|
||||||
|
type Source={episodeId:string;episodeNo:number;source:string;sha256:string;duration:number;frames:Frame[];subtitle:string};
|
||||||
|
type Job={id:string;analysisId:string;dramaId:string;title:string;drama:Awaited<ReturnType<CmsConnection["inventory"]>>["drama"];fetchedAt:string;root:string;state:"preparing"|"sources_ready"|"failed"|"cancelled";error?:string;sources:Source[];evidenceShared:boolean;analysisValidated:boolean;validationAttempts:number};
|
||||||
|
const Input=z.object({dramaId:z.string().uuid(),episodeNumbers:z.array(z.number().int().positive()).min(1).max(5)}).strict();
|
||||||
|
let mediaBusy=false;
|
||||||
|
|
||||||
|
export class EditorService {
|
||||||
|
readonly #jobs=new Map<string,Job>();
|
||||||
|
readonly #workflows=new Map<string,EditorWorkflow>();
|
||||||
|
public constructor(private readonly options:{cms:CmsConnection;runtimeRoot:string;sessionId:string;runner?:SkillRunner}){}
|
||||||
|
public status(jobId:string) {
|
||||||
|
const job=this.#get(jobId);
|
||||||
|
return {jobId:job.id,analysisId:job.analysisId,dramaId:job.dramaId,title:job.title,fetchedAt:job.fetchedAt,state:job.state,error:job.error,
|
||||||
|
sources:job.sources.map(source=>({episodeId:source.episodeId,episodeNo:source.episodeNo,duration:source.duration,sha256:source.sha256,
|
||||||
|
frames:source.frames.map(frame=>({frameId:frame.id,time:frame.time,url:`/api/sessions/${this.options.sessionId}/editor/${job.id}/frames/${frame.id}`}))})),
|
||||||
|
analysisValidated:job.analysisValidated,workflowUrl:job.analysisValidated?`/api/sessions/${this.options.sessionId}/editor/${job.id}/workflow`:undefined,renderingReady:this.#workflows.get(job.id)?.view().plan?.reviewed===true,humanReviewPassed:false};
|
||||||
|
}
|
||||||
|
public async prepare(input:z.infer<typeof Input>,signal:AbortSignal) {
|
||||||
|
Input.parse(input);
|
||||||
|
if(new Set(input.episodeNumbers).size!==input.episodeNumbers.length) throw new HarnessError("MEDIA_DUPLICATE_EPISODE","不能重复选择同一分集");
|
||||||
|
if(!this.options.cms.status().connected) throw new HarnessError("CMS_LOGIN_REQUIRED","请先登录 CMS");
|
||||||
|
if(mediaBusy) throw new HarnessError("MEDIA_BUSY","已有重型媒体任务在运行");
|
||||||
|
if(this.#jobs.size>=3) throw new HarnessError("MEDIA_JOB_LIMIT","本会话最多准备三次素材;请先检查已有任务");
|
||||||
|
mediaBusy=true;let job:Job|undefined;
|
||||||
|
try {
|
||||||
|
signal.throwIfAborted();await mediaRuntime();
|
||||||
|
const inventory=await this.options.cms.inventory(input.dramaId);
|
||||||
|
if(!inventory.complete || inventory.drama.language!=="en") throw new HarnessError("MEDIA_INVENTORY_INCOMPLETE","需要完整的英文剧目录和片源记录");
|
||||||
|
const episodes=input.episodeNumbers.map(number=>inventory.episodes.find(episode=>episode.episodeNo===number));
|
||||||
|
if(episodes.some(episode=>!episode || !episode.subtitleLocales.includes("en"))) throw new HarnessError("MEDIA_EPISODE_INVALID","所选分集不存在或缺少英文字幕");
|
||||||
|
await mkdir(this.options.runtimeRoot,{recursive:true,mode:0o700});
|
||||||
|
const root=await mkdtemp(join(this.options.runtimeRoot,"job-"));
|
||||||
|
job={id:`job_${randomUUID()}`,analysisId:`analysis_${randomUUID()}`,dramaId:input.dramaId,title:inventory.drama.title,drama:inventory.drama,fetchedAt:inventory.fetchedAt,root,state:"preparing",sources:[],evidenceShared:false,analysisValidated:false,validationAttempts:0};
|
||||||
|
this.#jobs.set(job.id,job);await this.#persist(job);
|
||||||
|
const budget={remaining:1024*1024*1024};
|
||||||
|
for(const episode of episodes) {
|
||||||
|
signal.throwIfAborted();if(!episode)throw new Error();
|
||||||
|
const episodeRoot=join(root,`episode-${episode.episodeNo.toString()}`);await mkdir(episodeRoot,{mode:0o700});
|
||||||
|
const signed=await this.options.cms.playback(input.dramaId,episode.id);
|
||||||
|
await downloadHls(new URL(signed.url),episodeRoot,signal,budget);
|
||||||
|
const media=await prepareLocalMedia(episodeRoot,signal);
|
||||||
|
const subtitle=await this.options.cms.subtitle(input.dramaId,episode.id,"en");
|
||||||
|
await writeFile(join(episodeRoot,"subtitles.vtt"),subtitle.text,{mode:0o600,flag:"wx"});
|
||||||
|
// MP4 is bounded by the task download quota; no signed source URL is retained.
|
||||||
|
const hash=createHash("sha256");for await(const chunk of createReadStream(media.source)){signal.throwIfAborted();hash.update(chunk as Buffer);}const sha256=hash.digest("hex");
|
||||||
|
job.sources.push({episodeId:episode.id,episodeNo:episode.episodeNo,source:media.source,sha256,duration:media.duration,subtitle:subtitle.text,
|
||||||
|
frames:media.frames.map(frame=>({id:`frame_${randomUUID()}`,time:frame.time,path:frame.path}))});
|
||||||
|
await this.#persist(job);
|
||||||
|
}
|
||||||
|
signal.throwIfAborted();job.state="sources_ready";await this.#persist(job);return this.status(job.id);
|
||||||
|
} catch(error) {
|
||||||
|
if(job){job.state=signal.aborted?"cancelled":"failed";job.error=error instanceof HarnessError?error.code:"MEDIA_PREPARATION_FAILED";await this.#persist(job);}
|
||||||
|
if(error instanceof HarnessError)throw error;
|
||||||
|
throw new HarnessError(signal.aborted?"MEDIA_CANCELLED":"MEDIA_PREPARATION_FAILED","真实素材准备失败;未使用模拟素材替代");
|
||||||
|
} finally {mediaBusy=false;}
|
||||||
|
}
|
||||||
|
public async evidence(jobId:string) {
|
||||||
|
const job=this.#get(jobId);
|
||||||
|
if(job.state!=="sources_ready")throw new HarnessError("MEDIA_NOT_READY","素材尚未准备完成");
|
||||||
|
const metadata={...this.status(jobId),drama:job.drama,subtitles:job.sources.map(source=>({episodeId:source.episodeId,episodeNo:source.episodeNo,text:source.subtitle})),
|
||||||
|
instruction:"Images follow in sources/frame order. For submit_drama_analysis, use original v2 schema plus evidence.frame_evidence[].episode_id, frame_id, time and evidence.subtitle_evidence[].episode_id, quote (exact substring of actual VTT). Preserve CMS synopsis, tags, episode_count and server analysisId/fetchedAt exactly. These samples are NOT full-clip viewing or content clearance. Never claim analysis PASS before submit_drama_analysis returns pass:true."};
|
||||||
|
if(JSON.stringify(metadata).length>100_000) throw new HarnessError("MEDIA_EVIDENCE_TOO_LARGE","字幕超过单次模型证据范围,需分批处理");
|
||||||
|
const images=await Promise.all(job.sources.flatMap(source=>source.frames).map(async frame=>({type:"image" as const,mimeType:"image/jpeg",data:(await this.readFrame(jobId,frame.id)).toString("base64")})));
|
||||||
|
job.evidenceShared=true;
|
||||||
|
return {content:[{type:"text" as const,text:JSON.stringify(metadata)},...images],details:{...this.status(jobId),imageCount:images.length,subtitleCount:job.sources.length}};
|
||||||
|
}
|
||||||
|
public async submitAnalysis(jobId:string,value:unknown,signal:AbortSignal){
|
||||||
|
const job=this.#get(jobId);
|
||||||
|
if(job.state!=="sources_ready"||!job.evidenceShared)throw new HarnessError("ANALYSIS_EVIDENCE_REQUIRED","请先完成授权后的真实字幕与帧读取");
|
||||||
|
if(job.analysisValidated)throw new HarnessError("ANALYSIS_IMMUTABLE","已通过的分析不能覆盖,请使用新的素材任务");
|
||||||
|
if(!this.options.runner)throw new HarnessError("EDITOR_VALIDATOR_UNAVAILABLE","校验运行时未配置");
|
||||||
|
if(job.validationAttempts>=10)throw new HarnessError("ANALYSIS_ATTEMPT_LIMIT","本任务分析提交次数达到限制");
|
||||||
|
const json=JSON.stringify(value);if(json.length>128*1024)throw new HarnessError("ANALYSIS_TOO_LARGE","分析文件超过限制");
|
||||||
|
assertAnalysisBinding(value,{analysisId:job.analysisId,dramaId:job.dramaId,fetchedAt:job.fetchedAt,title:job.title,synopsis:job.drama.synopsis,tags:job.drama.tags,episodeCount:job.drama.total_episodes,sources:job.sources});
|
||||||
|
job.validationAttempts++;
|
||||||
|
const attempt=await mkdtemp(join(job.root,"analysis-"));const input=join(attempt,"target-drama-analysis.json");
|
||||||
|
await writeFile(input,json,{mode:0o600,flag:"wx"});
|
||||||
|
const result=await this.options.runner.validate("analysis",input,attempt,signal);
|
||||||
|
if(result.pass){await writeFile(join(job.root,"target-drama-analysis.json"),json,{mode:0o600,flag:"wx"});job.analysisValidated=true;this.#workflows.set(job.id,new EditorWorkflow({root:job.root,jobId:job.id,sessionId:this.options.sessionId,analysisId:job.analysisId,dramaId:job.dramaId,sources:job.sources,runner:this.options.runner}));}
|
||||||
|
await writeFile(join(attempt,"target-drama-analysis-validation.json"),JSON.stringify(result),{mode:0o600,flag:"wx"});
|
||||||
|
await this.#persist(job);return {...result,jobId,analysisId:job.analysisId,dramaId:job.dramaId,workflowUrl:this.status(jobId).workflowUrl,renderingReady:false,contentReview:"required"};
|
||||||
|
}
|
||||||
|
public workflowView(jobId:string){return this.#workflow(jobId).view();}
|
||||||
|
public async proposeSelection(jobId:string,value:unknown){await this.#workflow(jobId).proposeSelection(value);return this.workflowSummary(jobId);}
|
||||||
|
public async submitPlan(jobId:string,value:unknown,signal:AbortSignal){const result=await this.#workflow(jobId).submitPlan(value,signal);return {...result,...this.workflowSummary(jobId),plan:undefined,selection:undefined};}
|
||||||
|
public async renderPreview(jobId:string,signal:AbortSignal){if(mediaBusy)throw new HarnessError("MEDIA_BUSY","已有重型媒体任务在运行");mediaBusy=true;try{await this.#workflow(jobId).render(signal);return this.workflowSummary(jobId);}finally{mediaBusy=false;}}
|
||||||
|
public async review(jobId:string,stage:"selection"|"plan",value:unknown){return this.#workflow(jobId).review(stage,value);}
|
||||||
|
public workflowSummary(jobId:string){const view=this.workflowView(jobId);return {jobId,workflowUrl:view.workflowUrl,selectionReviewed:view.selection?.reviewed??false,planReviewed:view.plan?.reviewed??false,preview:view.preview,humanReviewPassed:false};}
|
||||||
|
public async video(jobId:string,kind:"sources"|"plans",id:string){
|
||||||
|
const job=this.#get(jobId);const path=kind==="sources"?job.sources.find(source=>source.episodeId===id)?.source:this.#workflow(jobId).previewPath(id);
|
||||||
|
if(!path)throw new HarnessError("EDITOR_SOURCE_NOT_FOUND","该会话没有此媒体文件");
|
||||||
|
const workspace=await SafeWorkspace.create(job.root);const resolved=await workspace.resolveReadable(path);const info=await stat(resolved);
|
||||||
|
if(!info.isFile())throw new HarnessError("EDITOR_SOURCE_NOT_FOUND","媒体文件不存在");
|
||||||
|
return {path:resolved,size:info.size};
|
||||||
|
}
|
||||||
|
public async readFrame(jobId:string,frameId:string) {
|
||||||
|
const job=this.#get(jobId);const frame=job.sources.flatMap(source=>source.frames).find(item=>item.id===frameId);
|
||||||
|
if(!frame)throw new HarnessError("FRAME_NOT_FOUND","该会话中不存在此图片");
|
||||||
|
const workspace=await SafeWorkspace.create(job.root);
|
||||||
|
const bytes=await readFile(await workspace.resolveReadable(frame.path));
|
||||||
|
if(bytes.length>512*1024)throw new HarnessError("FRAME_TOO_LARGE","图片超过安全范围");
|
||||||
|
return bytes;
|
||||||
|
}
|
||||||
|
#get(id:string):Job {const job=this.#jobs.get(id);if(!job)throw new HarnessError("EDITOR_JOB_NOT_FOUND","该会话没有此媒体任务");return job;}
|
||||||
|
#workflow(id:string){this.#get(id);const workflow=this.#workflows.get(id);if(!workflow)throw new HarnessError("ANALYSIS_REQUIRED","先完成真实剧情分析并通过原始校验器");return workflow;}
|
||||||
|
async #persist(job:Job) {
|
||||||
|
// Atomic metadata; source/frames persist for recovery, but no automatic Session takeover.
|
||||||
|
const temporary=join(job.root,"job.json.tmp");
|
||||||
|
await writeFile(temporary,JSON.stringify({...this.status(job.id),ownerSessionId:this.options.sessionId},null,2),{mode:0o600});
|
||||||
|
await rename(temporary,join(job.root,"job.json"));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
import {HarnessError} from "../errors/harness-error.js";
|
||||||
|
const time=(text:string)=>text.split(":").reduce((total,item)=>total*60+Number(item),0);
|
||||||
|
const stamp=(seconds:number)=>{const ms=Math.round(seconds*1000);return `${String(Math.floor(ms/3600000)).padStart(2,"0")}:${String(Math.floor(ms/60000)%60).padStart(2,"0")}:${String(Math.floor(ms/1000)%60).padStart(2,"0")},${String(ms%1000).padStart(3,"0")}`;};
|
||||||
|
export function cutSubtitles(spans:{subtitle:string;sourceStart:number;sourceEnd:number;targetStart:number;segmentId:string}[]){
|
||||||
|
const cues:string[]=[];const warnings=new Set<string>();
|
||||||
|
for(const span of spans){
|
||||||
|
if(!span.subtitle.trimStart().startsWith("WEBVTT"))throw new HarnessError("SUBTITLE_INVALID","需要真实英文 WebVTT");
|
||||||
|
for(const block of span.subtitle.replaceAll("\r","").split(/\n\s*\n/)){
|
||||||
|
const lines=block.split("\n");const index=lines.findIndex(line=>line.includes(" --> "));if(index<0)continue;
|
||||||
|
const match=/((?:\d{2}:)?\d{2}:\d{2}\.\d{3})\s+-->\s+((?:\d{2}:)?\d{2}:\d{2}\.\d{3})/.exec(lines[index]??"");
|
||||||
|
if(!match?.[1]||!match[2])throw new HarnessError("SUBTITLE_INVALID","不支持此字幕时间格式");
|
||||||
|
const start=time(match[1]);const end=time(match[2]);
|
||||||
|
if(end<=span.sourceStart||start>=span.sourceEnd)continue;
|
||||||
|
if(start<span.sourceStart-0.001||end>span.sourceEnd+0.001)warnings.add(span.segmentId);
|
||||||
|
const text=lines.slice(index+1).join("\n").replaceAll(/<[^>]*>/g,"").replaceAll("&","&").replaceAll("<","<").replaceAll(">",">").replaceAll(" "," ").trim();
|
||||||
|
if(!text)continue;
|
||||||
|
cues.push(`${String(cues.length+1)}\n${stamp(Math.max(start,span.sourceStart)-span.sourceStart+span.targetStart)} --> ${stamp(Math.min(end,span.sourceEnd)-span.sourceStart+span.targetStart)}\n${text}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return {srt:cues.join("\n\n")+(cues.length?"\n":""),boundaryWarnings:[...warnings]};
|
||||||
|
}
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
import {defineTool} from "@earendil-works/pi-coding-agent";
|
||||||
|
import {Type} from "typebox";
|
||||||
|
import type {EditorService} from "./editor-service.js";
|
||||||
|
import {HarnessError} from "../errors/harness-error.js";
|
||||||
|
const job=Type.Object({jobId:Type.String({pattern:"^job_[a-zA-Z0-9-]+$"})},{additionalProperties:false});
|
||||||
|
export function createMediaTools(service:EditorService,supportsImages:()=>boolean) {
|
||||||
|
return [
|
||||||
|
defineTool({name:"propose_drama_selection",label:"Review source selection",description:"After real analysis PASS and before strategy/reference research, propose exact candidate spans for human adult-content review in the webpage. Input selection={segments:[{id,episodeId,start,end,reason}]}; times in seconds on 24fps boundaries, at most 10 spans and 60 seconds each. These are candidates, not approved clips. The user must watch with original sound and record findings. No tool can approve the review. Initial output is a 10-second original-audio preview, no new CTA/cover/overlays.",parameters:Type.Object({jobId:Type.String(),selection:Type.Record(Type.String(),Type.Unknown())},{additionalProperties:false}),async execute(_id,args){const details=await service.proposeSelection(args.jobId,args.selection);return {content:[{type:"text" as const,text:JSON.stringify(details)}],details};}}),
|
||||||
|
defineTool({name:"submit_editor_plan",label:"Validate bound editing plan",description:"Only after human selection review. Read the original Skill references first. bundle requires recipe, hookMap, strategy, referenceCandidates, referenceAnalysis, referenceRecipe, continuity JSON objects. All except referenceCandidates/recipe require target_binding={drama_id,analysis_id} and exact withheld_information. recipe v2 segments add episode_id and must exactly match reviewed segment IDs/source ranges; target duration exactly 10s,24fps. Supply no file paths, subtitle_path or invented PASS flags. Reference candidates need actual full short-drama marketing video plus A/B evidence; webpage does not inherit Codex web search, request actual URLs/evidence when missing. C is not enabled. Original reference/recipe/hook scripts run, then human confirms the exact plan, reference and story. No rendering from this tool.",parameters:Type.Object({jobId:Type.String(),bundle:Type.Record(Type.String(),Type.Unknown())},{additionalProperties:false}),async execute(_id,args,signal){const details=await service.submitPlan(args.jobId,args.bundle,signal??new AbortController().signal);return {content:[{type:"text" as const,text:JSON.stringify(details)}],details};}}),
|
||||||
|
defineTool({name:"render_editor_preview",label:"Render actual REVIEW MP4",description:"Render a 10-second local REVIEW preview using the original Skill renderer and machine QA. Requires analysis, reference/recipe/hook checks, human source and exact plan reviews, plus this page approval. Original audio at 1.00x. Returns registered video URL, duration, hash and real technical QA, never final content PASS. No publishing or upload.",parameters:job,async execute(_id,args,signal){const details=await service.renderPreview(args.jobId,signal??new AbortController().signal);return {content:[{type:"text" as const,text:JSON.stringify(details)}],details};}}),
|
||||||
|
defineTool({name:"submit_drama_analysis",label:"Validate actual drama analysis",description:"Submit original Skill v2 analysis JSON after approved read_drama_frames. Required: exact server analysisId as analysis_id, CMS drama_id/title/synopsis/tags/episode_count and fetchedAt as cms_source.fetched_at. Frame evidence adds exact episode_id/frame_id/time; subtitle evidence adds episode_id/quote from actual subtitles. Runs original validator; returns real issue codes. Passing analysis is not adult-content or rendering approval. No caller file paths are used.",parameters:Type.Object({jobId:Type.String(),analysis:Type.Record(Type.String(),Type.Unknown())},{additionalProperties:false}),
|
||||||
|
async execute(_id,args,signal){const details=await service.submitAnalysis(args.jobId,args.analysis,signal??new AbortController().signal);return {content:[{type:"text" as const,text:JSON.stringify(details)}],details};}}),
|
||||||
|
defineTool({name:"prepare_drama_sources",label:"Prepare licensed CMS sources",description:"After reading the editor Skill, download only approved exact episodeNumbers of an English CMS drama, maximum 5 episodes / 1 GiB per job. Fixed CMS admin/CDN only. Remux original audio/video locally and extract 3 representative frames per episode; no images or subtitles sent to the model by this tool. Requires page approval and CMS login. Returns a jobId for read_drama_frames; sources_ready is NOT analysis or content clearance. Cannot render yet.",
|
||||||
|
parameters:Type.Object({dramaId:Type.String({pattern:"^[a-fA-F0-9-]{36}$"}),episodeNumbers:Type.Array(Type.Integer({minimum:1}),{minItems:1,maxItems:5,uniqueItems:true})},{additionalProperties:false}),
|
||||||
|
async execute(_id,args,signal){const details=await service.prepare(args,signal??new AbortController().signal);return {content:[{type:"text" as const,text:JSON.stringify(details)}],details};}}),
|
||||||
|
defineTool({name:"editor_job_status",label:"Editor source job",description:"Read this Session's actual media preparation status by jobId. No source URLs or credentials. Does not grant any rendering or review approval.",parameters:job,
|
||||||
|
execute(_id,args){const details=service.status(args.jobId);return Promise.resolve({content:[{type:"text" as const,text:JSON.stringify(details)}],details});}}),
|
||||||
|
defineTool({name:"read_drama_frames",label:"Approved frame and subtitle evidence",description:"Requires separate page approval: sends the prepared job's English subtitles, CMS synopsis/categories and 3 representative JPEG frames per episode to the currently configured image-capable model. Cite exact frameId/episodeId/time in observations. Sampled frames cannot prove full-clip continuity, audio review or adult-content PASS. Submit analysis using submit_drama_analysis; no reference research or rendering until it passes.",parameters:job,
|
||||||
|
async execute(_id,args){if(!supportsImages())throw new HarnessError("MODEL_DOES_NOT_SUPPORT_IMAGES","请为已验证模型启用图片能力");return service.evidence(args.jobId);}}),
|
||||||
|
];
|
||||||
|
}
|
||||||
@@ -0,0 +1,135 @@
|
|||||||
|
import {createHash,randomUUID} from "node:crypto";
|
||||||
|
import {createReadStream} from "node:fs";
|
||||||
|
import {readFile,writeFile,mkdtemp} from "node:fs/promises";
|
||||||
|
import {join} from "node:path";
|
||||||
|
import {z} from "zod";
|
||||||
|
import {HarnessError} from "../errors/harness-error.js";
|
||||||
|
import {bindRecipe} from "./recipe-binding.js";
|
||||||
|
import {assertSelectionReview} from "./review-ledger.js";
|
||||||
|
import type {SkillRunner} from "./skill-runner.js";
|
||||||
|
import {cutSubtitles} from "./editor-subtitles.js";
|
||||||
|
|
||||||
|
type Source={episodeId:string;episodeNo:number;source:string;duration:number;sha256:string;subtitle:string};
|
||||||
|
const Span=z.object({id:z.string().regex(/^[\w-]{1,80}$/),episodeId:z.string(),start:z.number().nonnegative(),end:z.number().positive(),reason:z.string().min(10).max(2000)}).strict();
|
||||||
|
const Selection=z.object({segments:z.array(Span).min(1).max(10)}).strict();
|
||||||
|
const SourceReview=z.object({planHash:z.string(),reviewer:z.string().trim().min(1).max(100),policyCheckedAt:z.string().datetime({offset:true}),policyPageUpdated:z.string().min(4).max(100),pictureAudioReviewed:z.literal(true),noUnresolvedFindings:z.literal(true),segmentEvidence:z.array(z.object({segmentId:z.string(),pictureEvidence:z.string().trim().min(20).max(2000),audioEvidence:z.string().trim().min(20).max(2000),subtitleEvidence:z.string().trim().min(20).max(2000)}).strict()).min(1).max(10)}).strict();
|
||||||
|
const Json=z.record(z.string(),z.unknown());
|
||||||
|
const Bundle=z.object({recipe:Json,hookMap:Json,strategy:Json,referenceCandidates:Json,referenceAnalysis:Json,referenceRecipe:Json,continuity:Json}).strict();
|
||||||
|
type Selected={id:string;hash:string;root:string;input:z.infer<typeof Selection>;review?:z.infer<typeof SourceReview>};
|
||||||
|
type Plan={id:string;hash:string;root:string;recipe:ReturnType<typeof bindRecipe>;bundle:z.infer<typeof Bundle>;reference:Record<string,unknown>;subtitles:string;review?:ReturnType<typeof assertSelectionReview>;preview?:Awaited<ReturnType<SkillRunner["render"]>>};
|
||||||
|
const policyUrl="https://ads.tiktok.com/resources/help/article/tiktok-ads-policy-adult-content";
|
||||||
|
const hash=(value:unknown)=>createHash("sha256").update(JSON.stringify(value)).digest("hex");
|
||||||
|
|
||||||
|
/** Created only after EditorService's real analysis validator passes. Human reviews are API-only. */
|
||||||
|
export class EditorWorkflow {
|
||||||
|
#selection:Selected|undefined;
|
||||||
|
#plan:Plan|undefined;
|
||||||
|
#attempts=0;
|
||||||
|
#busy=false;
|
||||||
|
constructor(private readonly context:{root:string;sessionId:string;jobId:string;analysisId:string;dramaId:string;sources:Source[];runner:SkillRunner}){}
|
||||||
|
view(){
|
||||||
|
const base=`/api/sessions/${this.context.sessionId}/editor/${this.context.jobId}`;
|
||||||
|
const selection=this.#selection;const plan=this.#plan;
|
||||||
|
return {jobId:this.context.jobId,workflowUrl:`${base}/workflow`,policyUrl,
|
||||||
|
selection:selection?{id:selection.id,hash:selection.hash,reviewed:!!selection.review,segments:selection.input.segments.map(span=>({...span,episodeNo:this.source(span.episodeId).episodeNo,videoUrl:`${base}/sources/${span.episodeId}`,subtitles:cutSubtitles([{subtitle:this.source(span.episodeId).subtitle,sourceStart:span.start,sourceEnd:span.end,targetStart:0,segmentId:span.id}]).srt}))}:null,
|
||||||
|
plan:plan?{id:plan.id,hash:plan.hash,reviewed:!!plan.review,reference:plan.reference,artifacts:plan.bundle,subtitles:plan.subtitles,segmentIds:plan.recipe.segments.map(s=>s.id)}:null,
|
||||||
|
preview:plan?.preview?{url:`${base}/plans/${plan.id}/preview`,duration:plan.preview.duration,sha256:plan.preview.sha256,machinePass:plan.preview.machinePass,checks:plan.preview.checks,events:plan.preview.events,reviewStatus:"REVIEW",pending:["完整播放最终 MP4 并听原声","逐项核对字幕、边界与内容;预览不是投放审批"]}:null,
|
||||||
|
humanReviewPassed:false};
|
||||||
|
}
|
||||||
|
source(id:string){const source=this.context.sources.find(item=>item.episodeId===id);if(!source)throw new HarnessError("EDITOR_SOURCE_NOT_FOUND","当前任务中没有此源文件");return source;}
|
||||||
|
previewPath(id:string){if(this.#plan?.id!==id||!this.#plan.preview)throw new HarnessError("EDITOR_PREVIEW_NOT_FOUND","当前版本没有预览");return this.#plan.preview.output;}
|
||||||
|
async proposeSelection(value:unknown){
|
||||||
|
if(this.#busy)throw new HarnessError("EDITOR_BUSY","任务运行中");
|
||||||
|
const input=Selection.parse(value);
|
||||||
|
if(++this.#attempts>10)throw new HarnessError("EDITOR_ATTEMPT_LIMIT","本任务版本数已达限制");
|
||||||
|
const ids=new Set<string>();
|
||||||
|
for(const span of input.segments){
|
||||||
|
const source=this.context.sources.find(s=>s.episodeId===span.episodeId);
|
||||||
|
if(!source||ids.has(span.id)||span.end<=span.start||span.end>source.duration||span.end-span.start>60||[span.start,span.end].some(t=>Math.abs(t-Math.round(t*24)/24)>0.001))throw new HarnessError("SELECTION_INVALID","选段必须绑定本任务源文件及有效的 24fps 边界");
|
||||||
|
ids.add(span.id);
|
||||||
|
}
|
||||||
|
const root=await mkdtemp(join(this.context.root,"selection-"));
|
||||||
|
const record={target_binding:this.binding(),input,sources:input.segments.map(span=>({episodeId:span.episodeId,sha256:this.source(span.episodeId).sha256,subtitleHash:hash(this.source(span.episodeId).subtitle)})),packaging:{overlays:"not_present",cover:"not_present",cta:"not_present",adCopy:"not_present",subtitles:"original English captions only"}};
|
||||||
|
const selection={id:`selection_${randomUUID()}`,hash:hash(record),input,root};
|
||||||
|
await this.write(root,"selection.json",record);this.#selection=selection;this.#plan=undefined;return this.view();
|
||||||
|
}
|
||||||
|
async review(stage:"selection"|"plan",value:unknown){
|
||||||
|
if(this.#busy)throw new HarnessError("EDITOR_BUSY","任务运行中");
|
||||||
|
const selected=this.#selection;
|
||||||
|
if(!selected)throw new HarnessError("SELECTION_REVIEW_REQUIRED","请先提交候选选段");
|
||||||
|
if(stage==="selection"){
|
||||||
|
if(selected.review)throw new HarnessError("REVIEW_IMMUTABLE","审核已保存;修改选段需要新版本");
|
||||||
|
const review=SourceReview.parse(value);const checked=Date.parse(review.policyCheckedAt);
|
||||||
|
if(review.planHash!==selected.hash||checked>Date.now()+60_000||Date.now()-checked>86400000||review.segmentEvidence.length!==selected.input.segments.length||new Set(review.segmentEvidence.map(e=>e.segmentId)).size!==selected.input.segments.length||review.segmentEvidence.some(e=>!selected.input.segments.some(s=>s.id===e.segmentId)))throw new HarnessError("SELECTION_REVIEW_INVALID","审核必须对应当前版本每一选段和本批次政策核对时间");
|
||||||
|
await this.write(selected.root,"adult-content-review.json",{policy_source:{url:policyUrl,page_updated:review.policyPageUpdated,checked_at:review.policyCheckedAt,verification_status:"verified"},target_binding:this.binding(),stage:"selection",status:"PASS",reviewer:review.reviewer,reviewed_at:new Date().toISOString(),segments:selected.input.segments.map(span=>({...span,...review.segmentEvidence.find(e=>e.segmentId===span.id),decision:"KEEP"})),surfaces:{picture:"reviewed",original_audio:"reviewed",subtitles:"reviewed",overlays:"not_present",cover:"not_present",CTA:"not_present",ad_copy:"not_present"},findings:[],unresolved_findings:[],excluded_candidates:[],selectionHash:selected.hash});
|
||||||
|
selected.review=review;
|
||||||
|
}else{
|
||||||
|
const plan=this.#plan;if(!plan)throw new HarnessError("PLAN_REVIEW_REQUIRED","剪辑方案尚未通过脚本校验");
|
||||||
|
if(plan.review)throw new HarnessError("REVIEW_IMMUTABLE","审核已保存,修改需要新版本");
|
||||||
|
const review=assertSelectionReview(value,{hash:plan.hash,segmentIds:plan.recipe.segments.map(s=>s.id)});
|
||||||
|
// Candidate content/full-video evidence is reviewed BEFORE ranking, not cleared by scores.
|
||||||
|
this.#busy=true;
|
||||||
|
try{
|
||||||
|
const scored=await this.context.runner.scoreReferences(join(plan.root,"reference-candidates.json"),plan.root,AbortSignal.timeout(60_000));
|
||||||
|
if(scored.status!=="selected"||!scored.selected_reference)throw new HarnessError("VERIFIED_REFERENCE_REQUIRED","没有 A/B 级短剧营销参考;需要真实可播放来源及证据,不能自动降为 C 级");
|
||||||
|
if(plan.recipe.reference_id!==scored.selected_reference.id||plan.recipe.reference_evidence_tier!==scored.selected_reference.evidence_tier||plan.bundle.referenceAnalysis.reference_id!==scored.selected_reference.id||plan.bundle.referenceRecipe.reference_id!==scored.selected_reference.id)throw new HarnessError("REFERENCE_BINDING_INVALID","方案必须绑定本次评分选中的参考及其证据等级");
|
||||||
|
await this.write(plan.root,"selected-reference.json",{...scored,target_binding:this.binding()});
|
||||||
|
await this.write(plan.root,"plan-review.json",{...review,target_binding:this.binding(),reviewedAt:new Date().toISOString()});plan.reference=scored.selected_reference;plan.review=review;
|
||||||
|
}finally{this.#busy=false;}
|
||||||
|
}
|
||||||
|
return this.view();
|
||||||
|
}
|
||||||
|
async submitPlan(value:unknown,signal:AbortSignal){
|
||||||
|
const selection=this.#selection;
|
||||||
|
if(!selection?.review)throw new HarnessError("SELECTION_REVIEW_REQUIRED","策略与参考研究前,先在网页带声音审核候选源片段");
|
||||||
|
if(this.#busy)throw new HarnessError("EDITOR_BUSY","任务运行中");
|
||||||
|
if(++this.#attempts>10)throw new HarnessError("EDITOR_ATTEMPT_LIMIT","本任务版本数已达限制");
|
||||||
|
if(JSON.stringify(value).length>256*1024)throw new HarnessError("PLAN_TOO_LARGE","方案超过限制");
|
||||||
|
const bundle=Bundle.parse(value);
|
||||||
|
const analysis=Json.parse(JSON.parse(await readFile(join(this.context.root,"target-drama-analysis.json"),"utf8")));
|
||||||
|
const withheld=z.array(z.string()).min(1).parse(analysis.withheld_information);
|
||||||
|
const recipe=bindRecipe(bundle.recipe,{analysisId:this.context.analysisId,dramaId:this.context.dramaId,analysisPath:join(this.context.root,"target-drama-analysis.json"),withheldInformation:withheld,sources:this.context.sources});
|
||||||
|
for(const segment of recipe.segments)if(!selection.input.segments.some(s=>s.id===segment.id&&s.episodeId===segment.episode_id&&s.start===segment.source_start&&s.end===segment.source_end))throw new HarnessError("UNREVIEWED_SELECTION","时间范围改变后必须重新审核,不能沿用原 PASS");
|
||||||
|
const captions=cutSubtitles(recipe.segments.map(s=>({subtitle:this.source(s.episode_id).subtitle,sourceStart:s.source_start,sourceEnd:s.source_end,targetStart:s.target_start,segmentId:s.id})));
|
||||||
|
if(captions.boundaryWarnings.length)throw new HarnessError("SUBTITLE_BOUNDARY_REVIEW","切点截断了完整字幕句,需要重新选择完整对白区间");
|
||||||
|
for(const artifact of [bundle.hookMap,bundle.strategy,bundle.referenceAnalysis,bundle.referenceRecipe,bundle.continuity]){
|
||||||
|
const binding=Json.parse(artifact.target_binding);
|
||||||
|
if(binding.analysis_id!==this.context.analysisId||binding.drama_id!==this.context.dramaId||hash(artifact.withheld_information)!==hash(withheld))throw new HarnessError("PLAN_BINDING_INVALID","所有方案文件必须绑定当前分析及相同 withheld_information");
|
||||||
|
}
|
||||||
|
const strategy=z.object({narrative_structure:z.object({mode:z.enum(["linear","reverse"]),reason:z.string().min(10)}),opening:z.object({mode:z.literal("original_footage"),voiceover_authorized:z.literal(false)}),first_30_seconds:Json,character_amplification:Json,event_density:Json}).parse(bundle.strategy);void strategy;
|
||||||
|
for(const artifact of [bundle.referenceAnalysis,bundle.referenceRecipe,bundle.continuity])if(Object.keys(artifact).length<4)throw new HarnessError("PLAN_EVIDENCE_REQUIRED","参考与连续性报告需要实质分析,不接受仅有绑定和 PASS 标记");
|
||||||
|
const candidates={...bundle.referenceCandidates,target_analysis_path:join(this.context.root,"target-drama-analysis.json"),...this.binding(),allow_unverified_reference:false};
|
||||||
|
const root=await mkdtemp(join(this.context.root,"plan-"));this.#busy=true;this.#plan=undefined;
|
||||||
|
try{
|
||||||
|
await this.write(root,"reference-candidates.json",candidates);
|
||||||
|
await writeFile(join(root,"final-subtitles.srt"),captions.srt,{mode:0o600,flag:"wx"});
|
||||||
|
const renderedRecipe={...recipe,...(captions.srt?{subtitle_path:join(root,"final-subtitles.srt")}:{} )};
|
||||||
|
const artifacts={"recipe.json":renderedRecipe,"hook-map.json":bundle.hookMap,"creative-strategy.json":bundle.strategy,"reference-analysis.json":bundle.referenceAnalysis,"reference-recipe.json":bundle.referenceRecipe,"continuity-report.json":bundle.continuity};
|
||||||
|
for(const [name,data]of Object.entries(artifacts))await this.write(root,name,data);
|
||||||
|
const recipeReport=await this.context.runner.validate("recipe",join(root,"recipe.json"),root,signal);
|
||||||
|
const hookReport=await this.context.runner.validate("hook",join(root,"hook-map.json"),root,signal,join(root,"recipe.json"));
|
||||||
|
await this.write(root,"recipe-report.json",recipeReport);await this.write(root,"hook-report.json",hookReport);
|
||||||
|
if(!recipeReport.pass||!hookReport.pass)return {pass:false,recipeReport,hookReport,workflowUrl:this.view().workflowUrl};
|
||||||
|
const planHash=hash({selectionHash:selection.hash,bundle,subtitles:captions.srt});
|
||||||
|
this.#plan={id:`plan_${randomUUID()}`,hash:planHash,root,recipe,bundle,reference:{status:"pending_human_content_review_before_scoring",candidates:bundle.referenceCandidates.candidates},subtitles:captions.srt};
|
||||||
|
return {pass:true,validationStage:"recipe_and_hook_only",referenceScoring:"after_human_candidate_review",...this.view()};
|
||||||
|
}finally{this.#busy=false;}
|
||||||
|
}
|
||||||
|
async render(signal:AbortSignal){
|
||||||
|
const plan=this.#plan;if(!plan?.review||!this.#selection?.review)throw new HarnessError("PLAN_REVIEW_REQUIRED","生成前需要在网页完成当前版本的源片段、参考和故事审核");
|
||||||
|
if(plan.preview)return this.view();
|
||||||
|
if(this.#busy)throw new HarnessError("EDITOR_BUSY","任务运行中");
|
||||||
|
this.#busy=true;
|
||||||
|
try{
|
||||||
|
for(const id of new Set(plan.recipe.segments.map(s=>s.episode_id))){
|
||||||
|
const source=this.source(id);const actual=createHash("sha256");
|
||||||
|
for await(const chunk of createReadStream(source.source)){signal.throwIfAborted();actual.update(chunk as Buffer);}
|
||||||
|
if(actual.digest("hex")!==source.sha256)throw new HarnessError("REVIEWED_SOURCE_CHANGED","源文件已改变,旧审核不可复用");
|
||||||
|
}
|
||||||
|
plan.preview=await this.context.runner.render(join(plan.root,"recipe.json"),plan.root,signal);return this.view();
|
||||||
|
}
|
||||||
|
finally{this.#busy=false;}
|
||||||
|
}
|
||||||
|
private binding(){return {drama_id:this.context.dramaId,analysis_id:this.context.analysisId};}
|
||||||
|
private async write(root:string,name:string,data:unknown){await writeFile(join(root,name),JSON.stringify(data,null,2),{mode:0o600,flag:"wx"});}
|
||||||
|
}
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
import { HarnessError } from "../errors/harness-error.js";
|
||||||
|
|
||||||
|
type Playlist = {kind:"master";variants:string[]} | {kind:"media";segments:{uri:string;duration:number}[];duration:number};
|
||||||
|
function invalid(): never { throw new HarnessError("HLS_UNSUPPORTED", "HLS 清单格式不支持或超过安全范围;未下载替代素材"); }
|
||||||
|
|
||||||
|
export function parseHls(text: string): Playlist {
|
||||||
|
if (Buffer.byteLength(text)>1_048_576 || !text.trimStart().startsWith("#EXTM3U")) invalid();
|
||||||
|
const lines=text.trim().split(/\r?\n/).map((line)=>line.trim()).filter(Boolean);
|
||||||
|
if (lines.some((line)=>/^#EXT-X-(KEY|MAP|BYTERANGE|MEDIA:|I-FRAMES-ONLY|GAP|DISCONTINUITY)/.test(line))) invalid();
|
||||||
|
const variants:string[]=[];
|
||||||
|
const segments:{uri:string;duration:number}[]=[];
|
||||||
|
let pending:number|"variant"|undefined;
|
||||||
|
for (const line of lines.slice(1)) {
|
||||||
|
if (line.startsWith("#EXT-X-STREAM-INF:")) { if(pending!==undefined) invalid(); pending="variant"; }
|
||||||
|
else if (line.startsWith("#EXTINF:")) {
|
||||||
|
if(pending!==undefined) invalid();
|
||||||
|
const duration=Number(line.slice(8).split(",")[0]);
|
||||||
|
if(!Number.isFinite(duration)||duration<=0||duration>60) invalid();
|
||||||
|
pending=duration;
|
||||||
|
} else if (!line.startsWith("#")) {
|
||||||
|
if(pending==="variant") variants.push(line);
|
||||||
|
else if(typeof pending==="number") segments.push({uri:line,duration:pending});
|
||||||
|
else invalid();
|
||||||
|
pending=undefined;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if(pending!==undefined || (variants.length>0 && segments.length>0)) invalid();
|
||||||
|
if(variants.length>0) { if(variants.length>10) invalid(); return {kind:"master",variants}; }
|
||||||
|
const duration=segments.reduce((sum,item)=>sum+item.duration,0);
|
||||||
|
if(!segments.length||segments.length>1000||duration>1800||!lines.includes("#EXT-X-ENDLIST")) invalid();
|
||||||
|
return {kind:"media",segments,duration};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function resolveHlsAsset(base:URL, uri:string, episodePrefix:string):URL {
|
||||||
|
if(uri.length>4096 || /[\s\\]/.test(uri) || [...uri].some(character=>character.charCodeAt(0)<32)) invalid();
|
||||||
|
const result=new URL(uri,base);
|
||||||
|
if(result.origin!==base.origin || result.protocol!=="https:" || result.username || result.password || result.hash) invalid();
|
||||||
|
let decoded:string;
|
||||||
|
try { decoded=decodeURIComponent(result.pathname); } catch { return invalid(); }
|
||||||
|
if(!result.pathname.startsWith(episodePrefix)||!decoded.startsWith(episodePrefix)||decoded.includes("..")||decoded.includes("\\")||!(/\.(ts|m3u8)$/i.test(result.pathname))) invalid();
|
||||||
|
// CDN inheritance signs each child path separately; never replace that signature with the master's.
|
||||||
|
if(result.search && result.search!==base.search) {
|
||||||
|
const entries=[...result.searchParams.entries()];
|
||||||
|
if(entries.length!==1 || entries[0]?.[0]!=="auth_key" || !/^\d+-[a-zA-Z0-9]+-[a-zA-Z0-9]+-[a-fA-F0-9]+$/.test(entries[0][1])) invalid();
|
||||||
|
}
|
||||||
|
if(!result.search) result.search=base.search;
|
||||||
|
return result;
|
||||||
|
}
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
import { spawn } from "node:child_process";
|
||||||
|
import { createRequire } from "node:module";
|
||||||
|
import { access, writeFile } from "node:fs/promises";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { HarnessError } from "../errors/harness-error.js";
|
||||||
|
import { parseHls, resolveHlsAsset } from "./hls.js";
|
||||||
|
|
||||||
|
const require=createRequire(import.meta.url);
|
||||||
|
export async function mediaRuntime() {
|
||||||
|
const ffmpeg=require("ffmpeg-static") as string|null;
|
||||||
|
const ffprobe=(require("ffprobe-static") as {path:string}).path;
|
||||||
|
if(!ffmpeg) throw new HarnessError("MEDIA_RUNTIME_MISSING","当前平台缺少 FFmpeg");
|
||||||
|
try { await Promise.all([access(ffmpeg),access(ffprobe)]); } catch { throw new HarnessError("MEDIA_RUNTIME_MISSING","FFmpeg/FFprobe 安装尚未完成"); }
|
||||||
|
return {ffmpeg,ffprobe};
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function downloadHls(url:URL,root:string,signal:AbortSignal,budget:{remaining:number},fetcher:typeof fetch=fetch) {
|
||||||
|
if(url.origin!=="https://video.pencil-stub.com"||url.username||url.password||url.hash) throw new HarnessError("MEDIA_ORIGIN_DENIED","只允许 CMS 固定 CDN 源");
|
||||||
|
const prefix=url.pathname.slice(0,url.pathname.lastIndexOf("/")+1);
|
||||||
|
const request=async(target:URL,limit:number)=>{
|
||||||
|
signal.throwIfAborted();
|
||||||
|
const response=await fetcher(target,{redirect:"error",signal:AbortSignal.any([signal,AbortSignal.timeout(30_000)])});
|
||||||
|
if(response.status!==200||!response.body) throw new HarnessError("MEDIA_FETCH_FAILED",`片源请求返回 HTTP ${response.status.toString()}`);
|
||||||
|
const reader=response.body.getReader();const chunks:Uint8Array[]=[];let size=0;
|
||||||
|
try {
|
||||||
|
for(;;){ const chunk=await reader.read();if(chunk.done)break;size+=chunk.value.byteLength;budget.remaining-=chunk.value.byteLength;
|
||||||
|
if(size>limit||budget.remaining<0) throw new HarnessError("MEDIA_BYTE_LIMIT","下载超过单次安全字节上限");
|
||||||
|
chunks.push(chunk.value);
|
||||||
|
}
|
||||||
|
} finally { await reader.cancel().catch(()=>undefined); }
|
||||||
|
return Buffer.concat(chunks);
|
||||||
|
};
|
||||||
|
let current=url;
|
||||||
|
let playlist=parseHls((await request(current,1_048_576)).toString("utf8"));
|
||||||
|
for(let depth=0;playlist.kind==="master" && depth<3;depth++) {
|
||||||
|
current=resolveHlsAsset(current,playlist.variants[0]!,prefix);
|
||||||
|
playlist=parseHls((await request(current,1_048_576)).toString("utf8"));
|
||||||
|
}
|
||||||
|
if(playlist.kind!=="media") throw new HarnessError("MEDIA_PLAYLIST_DEPTH","HLS 主清单嵌套超限");
|
||||||
|
const local=["#EXTM3U","#EXT-X-VERSION:3",`#EXT-X-TARGETDURATION:${Math.ceil(Math.max(...playlist.segments.map(s=>s.duration))).toString()}`,"#EXT-X-MEDIA-SEQUENCE:0"];
|
||||||
|
for(const [index,segment] of playlist.segments.entries()) {
|
||||||
|
const data=await request(resolveHlsAsset(current,segment.uri,prefix),32*1024*1024);
|
||||||
|
const name=`segment-${index.toString()}.ts`;
|
||||||
|
await writeFile(join(root,name),data,{flag:"wx",mode:0o600});
|
||||||
|
local.push(`#EXTINF:${segment.duration.toString()},`,name);
|
||||||
|
}
|
||||||
|
local.push("#EXT-X-ENDLIST","");
|
||||||
|
const path=join(root,"source.m3u8");await writeFile(path,local.join("\n"),{flag:"wx",mode:0o600});
|
||||||
|
return {path,duration:playlist.duration,segmentCount:playlist.segments.length};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function runMedia(executable:string,args:string[],cwd:string,signal:AbortSignal,timeoutMs=180_000):Promise<string> {
|
||||||
|
signal.throwIfAborted();
|
||||||
|
return new Promise((resolve,reject)=>{
|
||||||
|
const child=spawn(executable,args,{cwd,env:{PATH:"/usr/bin:/bin",LANG:"en_US.UTF-8"},detached:process.platform!=="win32",stdio:["ignore","pipe","pipe"]});
|
||||||
|
let stdout="";let bytes=0;let terminated=false;
|
||||||
|
const kill=()=>{terminated=true;try{if(process.platform!=="win32"&&child.pid)process.kill(-child.pid,"SIGKILL");else child.kill("SIGKILL");}catch{/* Already exited. */}};
|
||||||
|
const timer=setTimeout(kill,timeoutMs);signal.addEventListener("abort",kill,{once:true});
|
||||||
|
const cleanup=()=>{clearTimeout(timer);signal.removeEventListener("abort",kill);};
|
||||||
|
child.stdout.on("data",(chunk:Buffer)=>{bytes+=chunk.length;if(bytes>2*1024*1024)kill();else stdout+=chunk.toString();});
|
||||||
|
child.stderr.on("data",(chunk:Buffer)=>{bytes+=chunk.length;if(bytes>2*1024*1024)kill();});
|
||||||
|
child.once("error",()=>{cleanup();reject(new HarnessError("MEDIA_EXEC_FAILED","媒体进程启动失败"));});
|
||||||
|
child.once("close",code=>{cleanup();if(code===0&&!terminated&&!signal.aborted)resolve(stdout);else reject(new HarnessError(signal.aborted?"MEDIA_CANCELLED":"MEDIA_EXEC_FAILED","媒体进程失败、取消或超过限制"));});
|
||||||
|
if(signal.aborted)kill();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function prepareLocalMedia(root:string,signal:AbortSignal) {
|
||||||
|
const {ffmpeg,ffprobe}=await mediaRuntime();
|
||||||
|
const source=join(root,"episode.mp4");
|
||||||
|
await runMedia(ffmpeg,["-nostdin","-v","error","-protocol_whitelist","file,crypto,data","-i",join(root,"source.m3u8"),"-map","0:v:0","-map","0:a:0","-c","copy","-movflags","+faststart","-n",source],root,signal);
|
||||||
|
const raw=JSON.parse(await runMedia(ffprobe,["-v","error","-protocol_whitelist","file","-show_streams","-show_format","-of","json",source],root,signal)) as {streams?:{codec_type?:string;width?:number;height?:number}[];format?:{duration?:string}};
|
||||||
|
const duration=Number(raw.format?.duration);const video=raw.streams?.find(s=>s.codec_type==="video");
|
||||||
|
if(!video||!raw.streams?.some(s=>s.codec_type==="audio")||!Number.isFinite(duration)||duration<=0||duration>1800||!video.width||!video.height) throw new HarnessError("MEDIA_STREAM_INVALID","素材缺少有效音视频流或超出时长范围");
|
||||||
|
const frames:{path:string;time:number}[]=[];
|
||||||
|
for(const [index,fraction] of [0.15,0.5,0.85].entries()) {
|
||||||
|
const time=Math.round(duration*fraction*1000)/1000;const path=join(root,`frame-${index.toString()}.jpg`);
|
||||||
|
await runMedia(ffmpeg,["-nostdin","-v","error","-protocol_whitelist","file","-ss",time.toString(),"-i",source,"-frames:v","1","-vf","scale=512:512:force_original_aspect_ratio=decrease","-q:v","3","-n",path],root,signal);
|
||||||
|
frames.push({path,time});
|
||||||
|
}
|
||||||
|
return {source,duration,width:video.width,height:video.height,frames};
|
||||||
|
}
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
import {z} from "zod";
|
||||||
|
import {HarnessError} from "../errors/harness-error.js";
|
||||||
|
const number=z.number().finite().nonnegative();
|
||||||
|
const Segment=z.object({id:z.string().min(1),hook_id:z.string().min(1),episode_id:z.string(),episode:z.number().int().positive(),source_start:number,source_end:number,target_start:number,target_end:number,hold:number.default(0),semantic_role:z.string().min(1),continuity_group:z.string().min(1),source_gap_before:z.number().finite(),confidence:z.enum(["high","medium","low"]),reveal_risk:z.enum(["low","medium","high"]),cut_reason:z.string().min(1),intentional_discontinuity:z.boolean().optional(),discontinuity_reason:z.string().optional()});
|
||||||
|
const Recipe=z.object({schema_version:z.literal("2.0"),analysis_id:z.string(),drama_id:z.string(),title:z.string().optional(),creative_goal:z.string().optional(),reference_id:z.string().optional(),reference_evidence_tier:z.enum(["A","B","C"]).optional(),duration_strategy:z.literal("content_led"),withheld_information:z.array(z.string()).min(1),ending:z.object({cutoff_reason:z.string().min(1),before_withheld_reveal:z.literal(true),hold:number.default(0)}),segments:z.array(Segment).min(1).max(10),transitions:z.array(z.unknown()).optional(),subtitle_path:z.never().optional()});
|
||||||
|
export function bindRecipe(value:unknown,binding:{analysisId:string;dramaId:string;analysisPath:string;withheldInformation:string[];sources:{episodeId:string;episodeNo:number;source:string;duration:number}[]}){
|
||||||
|
const recipe=Recipe.parse(value);const fail=():never=>{throw new HarnessError("RECIPE_BINDING_INVALID","剪辑必须使用当前分析和授权源、精确 24fps 时间线,总长必须为 10 秒");};
|
||||||
|
if(recipe.analysis_id!==binding.analysisId||recipe.drama_id!==binding.dramaId||JSON.stringify(recipe.withheld_information)!==JSON.stringify(binding.withheldInformation))fail();
|
||||||
|
let end=0;const ids=new Set<string>();
|
||||||
|
const segments=recipe.segments.map(segment=>{
|
||||||
|
const source=binding.sources.find(s=>s.episodeId===segment.episode_id&&s.episodeNo===segment.episode);
|
||||||
|
if(!source||ids.has(segment.id))return fail();ids.add(segment.id);
|
||||||
|
for(const value of [segment.source_start,segment.source_end,segment.target_start,segment.target_end,segment.hold])if(Math.abs(value-Math.round(value*24)/24)>0.001)fail();
|
||||||
|
if(segment.source_end>source.duration||segment.source_end<=segment.source_start||Math.abs(segment.target_start-end)>0.001||Math.abs((segment.source_end-segment.source_start+segment.hold)-(segment.target_end-segment.target_start))>0.001)fail();
|
||||||
|
end=segment.target_end;return {...segment,source:source.source};
|
||||||
|
});
|
||||||
|
if(Math.abs(end-10)>0.001)fail();
|
||||||
|
return {...recipe,segments,target_analysis_path:binding.analysisPath,target_duration_preference:10,maximum_duration_seconds:300};
|
||||||
|
}
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
import {z} from "zod";
|
||||||
|
import {HarnessError} from "../errors/harness-error.js";
|
||||||
|
const Review=z.object({planHash:z.string(),reviewer:z.string().trim().min(1).max(100),policyCheckedAt:z.string().datetime({offset:true}),pictureAudioReviewed:z.literal(true),referenceReviewed:z.literal(true),storyReviewed:z.literal(true),packagingReviewed:z.literal(true),noUnresolvedFindings:z.literal(true),segmentEvidence:z.array(z.object({segmentId:z.string(),observation:z.string().trim().min(20).max(3000)})).min(1).max(10)}).strict();
|
||||||
|
export function assertSelectionReview(value:unknown,plan:{hash:string;segmentIds:string[]}){
|
||||||
|
const review=Review.parse(value);
|
||||||
|
const checked=Date.parse(review.policyCheckedAt);
|
||||||
|
if(review.planHash!==plan.hash||checked>Date.now()+60_000||Date.now()-checked>24*60*60*1000||review.segmentEvidence.length!==plan.segmentIds.length||new Set(review.segmentEvidence.map(item=>item.segmentId)).size!==plan.segmentIds.length||review.segmentEvidence.some(item=>!plan.segmentIds.includes(item.segmentId)))throw new HarnessError("SELECTION_REVIEW_INVALID","审核必须对应当前版本、每一个选段及本批次政策核对时间");
|
||||||
|
return review;
|
||||||
|
}
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
import {mkdtemp,readFile,stat} from "node:fs/promises";
|
||||||
|
import {join} from "node:path";
|
||||||
|
import {z} from "zod";
|
||||||
|
import {HarnessError} from "../errors/harness-error.js";
|
||||||
|
import {runMedia,mediaRuntime} from "./media-worker.js";
|
||||||
|
|
||||||
|
const Report=z.object({pass:z.boolean(),issues:z.array(z.object({code:z.string(),severity:z.string(),message:z.string()}))});
|
||||||
|
const validators={analysis:["validate_target_drama_analysis.py","--analysis"],recipe:["validate_recipe.py","--recipe"],hook:["validate_hook_map.py","--hook-map"]} as const;
|
||||||
|
export class SkillRunner{
|
||||||
|
constructor(private readonly options:{skillRoot:string;pythonExecutable:string}){}
|
||||||
|
async validate(kind:keyof typeof validators,input:string,root:string,signal:AbortSignal,recipe?:string){
|
||||||
|
const work=await mkdtemp(join(root,"validation-"));
|
||||||
|
const reportPath=join(work,"report.json");
|
||||||
|
const [script,flag]=validators[kind];
|
||||||
|
const args=["-B",join(this.options.skillRoot,"scripts",script),flag,input,"--report",reportPath];
|
||||||
|
if(kind==="hook"&&recipe)args.push("--recipe",recipe);
|
||||||
|
try{await runMedia(this.options.pythonExecutable,args,work,signal,30_000);}
|
||||||
|
catch(error){if(signal.aborted)throw error;if(!(error instanceof HarnessError)||error.code!=="MEDIA_EXEC_FAILED")throw error;}
|
||||||
|
signal.throwIfAborted();
|
||||||
|
try{
|
||||||
|
if((await stat(reportPath)).size>128*1024)throw new Error();
|
||||||
|
const result=Report.parse(JSON.parse(await readFile(reportPath,"utf8")));
|
||||||
|
// Output only issue codes: original script messages may contain private absolute paths.
|
||||||
|
return {pass:result.pass&&result.issues.every(issue=>issue.severity!=="error"),issues:result.issues.map(issue=>({code:issue.code,severity:issue.severity}))};
|
||||||
|
}catch{throw new HarnessError("EDITOR_VALIDATOR_FAILED","原始 Skill 校验器未生成有效报告,未放行");}
|
||||||
|
}
|
||||||
|
/** Internal only: the workflow must bind paths and enforce all pre-render gates first. */
|
||||||
|
async scoreReferences(input:string,root:string,signal:AbortSignal){
|
||||||
|
const work=await mkdtemp(join(root,"references-"));const output=join(work,"selected-reference.json");
|
||||||
|
try{await runMedia(this.options.pythonExecutable,["-B",join(this.options.skillRoot,"scripts/score_reference_candidates.py"),"--input",input,"--output",output],work,signal,30_000);}
|
||||||
|
catch(error){if(signal.aborted||!(error instanceof HarnessError)||error.code!=="MEDIA_EXEC_FAILED")throw error;}
|
||||||
|
signal.throwIfAborted();if((await stat(output)).size>256*1024)throw new HarnessError("REFERENCE_REPORT_INVALID","参考报告超限");
|
||||||
|
return z.object({status:z.string(),selected_reference:z.record(z.string(),z.unknown()).nullable(),warnings:z.array(z.string())}).parse(JSON.parse(await readFile(output,"utf8")));
|
||||||
|
}
|
||||||
|
/** Internal only: the workflow must bind paths and enforce all pre-render gates first. */
|
||||||
|
async render(recipe:string,root:string,signal:AbortSignal){
|
||||||
|
const work=await mkdtemp(join(root,"render-"));const output=join(work,"preview.mp4");
|
||||||
|
const runtime=await mediaRuntime();
|
||||||
|
const binaries=["--ffmpeg",runtime.ffmpeg,"--ffprobe",runtime.ffprobe];
|
||||||
|
await runMedia(this.options.pythonExecutable,["-B",join(this.options.skillRoot,"scripts/render_exact.py"),"--recipe",recipe,"--work-dir",work,"--output",output,...binaries],work,signal,300_000);
|
||||||
|
try{await runMedia(this.options.pythonExecutable,["-B",join(this.options.skillRoot,"scripts/qa_video.py"),"--video",output,"--recipe",recipe,"--matches",join(work,"matches.json"),"--work-dir",work,...binaries],work,signal,300_000);}
|
||||||
|
catch(error){if(signal.aborted||!(error instanceof HarnessError)||error.code!=="MEDIA_EXEC_FAILED")throw error;}
|
||||||
|
signal.throwIfAborted();
|
||||||
|
const report=join(work,"qa-report.json");if((await stat(report)).size>256*1024)throw new HarnessError("QA_REPORT_INVALID","QA 报告超过限制");
|
||||||
|
const qa=z.object({pass:z.boolean(),sha256:z.string().regex(/^[a-f0-9]{64}$/),probe:z.object({duration:z.number().positive().max(300)}),checks:z.record(z.string(),z.boolean()),audio_sync:z.object({results:z.array(z.unknown())}),black_events:z.array(z.string()),silence_events:z.array(z.string()),freeze_events:z.array(z.string())}).parse(JSON.parse(await readFile(report,"utf8")));
|
||||||
|
return {output,duration:qa.probe.duration,sha256:qa.sha256,checks:qa.checks,machinePass:qa.pass,reviewStatus:"REVIEW" as const,events:{black:qa.black_events.length,silence:qa.silence_events.length,freeze:qa.freeze_events.length},audioSync:qa.audio_sync.results};
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -83,14 +83,14 @@ export class HarnessSession {
|
|||||||
...(skillInference === undefined ? {} : { skillInference }),
|
...(skillInference === undefined ? {} : { skillInference }),
|
||||||
});
|
});
|
||||||
let eventQueue = Promise.resolve();
|
let eventQueue = Promise.resolve();
|
||||||
let eventError: unknown;
|
let eventError: Error | undefined;
|
||||||
const unsubscribe = port.subscribe((event) => {
|
const unsubscribe = port.subscribe((event) => {
|
||||||
eventQueue = eventQueue.then(async () => {
|
eventQueue = eventQueue.then(async () => {
|
||||||
if (eventError !== undefined) return;
|
if (eventError !== undefined) return;
|
||||||
try {
|
try {
|
||||||
await adapter.observe(event);
|
await adapter.observe(event);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
eventError = error;
|
eventError = error instanceof Error ? error : new Error("Agent event mapping failed");
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import { access, realpath, stat } from "node:fs/promises";
|
import { access, mkdir, mkdtemp, readFile, realpath, stat, writeFile } from "node:fs/promises";
|
||||||
import { realpathSync } from "node:fs";
|
import { realpathSync } from "node:fs";
|
||||||
import { delimiter, isAbsolute, relative, resolve, sep } from "node:path";
|
import { delimiter, dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
|
||||||
|
import { SafeWorkspace } from "../workspace/safe-workspace.js";
|
||||||
import {
|
import {
|
||||||
DefaultResourceLoader,
|
DefaultResourceLoader,
|
||||||
SettingsManager,
|
SettingsManager,
|
||||||
@@ -49,13 +50,16 @@ export type SkillCatalogOptions = {
|
|||||||
cwd: string;
|
cwd: string;
|
||||||
agentDir: string;
|
agentDir: string;
|
||||||
extraDirsValue?: string;
|
extraDirsValue?: string;
|
||||||
|
resources?: Record<string, readonly string[]>;
|
||||||
};
|
};
|
||||||
|
|
||||||
export class SkillCatalog {
|
export class SkillCatalog {
|
||||||
readonly #cwd: string;
|
readonly #cwd: string;
|
||||||
readonly #agentDir: string;
|
readonly #agentDir: string;
|
||||||
readonly #extraDirsValue: string | undefined;
|
readonly #extraDirsValue: string | undefined;
|
||||||
|
readonly #resources: Record<string, readonly string[]>;
|
||||||
#entries: SkillCatalogEntry[] = [];
|
#entries: SkillCatalogEntry[] = [];
|
||||||
|
#skills: Skill[] = [];
|
||||||
#indexedSkills: IndexedSkill[] = [];
|
#indexedSkills: IndexedSkill[] = [];
|
||||||
#diagnostics: SkillDiagnostic[] = [];
|
#diagnostics: SkillDiagnostic[] = [];
|
||||||
|
|
||||||
@@ -63,6 +67,7 @@ export class SkillCatalog {
|
|||||||
this.#cwd = canonicalPathSync(options.cwd);
|
this.#cwd = canonicalPathSync(options.cwd);
|
||||||
this.#agentDir = canonicalPathSync(options.agentDir);
|
this.#agentDir = canonicalPathSync(options.agentDir);
|
||||||
this.#extraDirsValue = options.extraDirsValue;
|
this.#extraDirsValue = options.extraDirsValue;
|
||||||
|
this.#resources = Object.fromEntries(Object.entries(options.resources ?? {}).map(([path, files]) => [canonicalPathSync(path), files]));
|
||||||
}
|
}
|
||||||
|
|
||||||
public async reload(): Promise<void> {
|
public async reload(): Promise<void> {
|
||||||
@@ -81,6 +86,7 @@ export class SkillCatalog {
|
|||||||
|
|
||||||
await loader.reload();
|
await loader.reload();
|
||||||
const loaded = loader.getSkills();
|
const loaded = loader.getSkills();
|
||||||
|
this.#skills = loaded.skills;
|
||||||
const loaderDiagnostics = loaded.diagnostics.map((diagnostic) =>
|
const loaderDiagnostics = loaded.diagnostics.map((diagnostic) =>
|
||||||
mapDiagnostic(diagnostic, this.#cwd),
|
mapDiagnostic(diagnostic, this.#cwd),
|
||||||
);
|
);
|
||||||
@@ -106,6 +112,34 @@ export class SkillCatalog {
|
|||||||
return new SkillIndex(this.#indexedSkills);
|
return new SkillIndex(this.#indexedSkills);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Copy only catalogued instruction bodies, never arbitrary project files or credentials. */
|
||||||
|
public async createSessionSnapshot(workspaceRoot: string): Promise<{ skills: Skill[]; index: SkillIndex }> {
|
||||||
|
const root = await realpath(workspaceRoot);
|
||||||
|
const snapshotRoot = await mkdtemp(join(root, ".skills-"));
|
||||||
|
const skills = await Promise.all(this.#skills.map(async (skill, index) => {
|
||||||
|
const baseDir = join(snapshotRoot, String(index));
|
||||||
|
await mkdir(baseDir, { mode: 0o700 });
|
||||||
|
const filePath = join(baseDir, "SKILL.md");
|
||||||
|
await writeFile(filePath, await readFile(skill.filePath), { flag: "wx", mode: 0o400 });
|
||||||
|
const source = await SafeWorkspace.create(skill.baseDir);
|
||||||
|
for (const resource of this.#resources[canonicalPathSync(skill.filePath)] ?? []) {
|
||||||
|
if (isAbsolute(resource) || resource.split(/[\\/]/).includes("..")) throw new Error("Unsafe Skill resource path");
|
||||||
|
const from = await source.resolveReadable(resource);
|
||||||
|
if ((await stat(from)).size > 1_048_576) throw new Error("Skill resource exceeds size limit");
|
||||||
|
const target = join(baseDir, resource);
|
||||||
|
await mkdir(dirname(target), { recursive: true, mode: 0o700 });
|
||||||
|
await writeFile(target, await readFile(from), { flag: "wx", mode: 0o400 });
|
||||||
|
}
|
||||||
|
return { ...skill, baseDir, filePath };
|
||||||
|
}));
|
||||||
|
return {
|
||||||
|
skills,
|
||||||
|
index: new SkillIndex(skills.map((skill) => ({
|
||||||
|
name: skill.name, source: displaySource(skill), canonicalFilePath: skill.filePath,
|
||||||
|
}))),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
public readonly diagnostics = (): SkillDiagnostic[] => {
|
public readonly diagnostics = (): SkillDiagnostic[] => {
|
||||||
return this.#diagnostics.map((diagnostic) => ({ ...diagnostic }));
|
return this.#diagnostics.map((diagnostic) => ({ ...diagnostic }));
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -0,0 +1,108 @@
|
|||||||
|
import { execFile } from "node:child_process";
|
||||||
|
import { randomUUID } from "node:crypto";
|
||||||
|
import { chmod, mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { pathToFileURL } from "node:url";
|
||||||
|
import { promisify } from "node:util";
|
||||||
|
import { defineTool } from "@earendil-works/pi-coding-agent";
|
||||||
|
import { Type } from "typebox";
|
||||||
|
import { HarnessError } from "../errors/harness-error.js";
|
||||||
|
import { SafeWorkspace } from "../workspace/safe-workspace.js";
|
||||||
|
|
||||||
|
const exec = promisify(execFile);
|
||||||
|
const formats = { json: "application/json", html: "text/html; charset=utf-8", pdf: "application/pdf" } as const;
|
||||||
|
type Format = keyof typeof formats;
|
||||||
|
type QueryRecord = { queryId: string; requestedAt: string; arguments: Record<string, unknown>; result: unknown };
|
||||||
|
type ReportInput = { data: Record<string, unknown>; source_query_ids: string[] };
|
||||||
|
export type MinisReportOptions = { workspace: SafeWorkspace; sessionId: string; skillRoot: string; pythonExecutable?: string; chromeExecutable?: string };
|
||||||
|
|
||||||
|
export class MinisReportService {
|
||||||
|
readonly #queries = new Map<string, QueryRecord>();
|
||||||
|
readonly #reports = new Map<string, { root: string; formats: Set<Format> }>();
|
||||||
|
public constructor(private readonly options: MinisReportOptions) {}
|
||||||
|
|
||||||
|
public recordQuery(args: Record<string, unknown>, result: unknown): string {
|
||||||
|
const queryId = `query_${randomUUID()}`;
|
||||||
|
this.#queries.set(queryId, { queryId, requestedAt: new Date().toISOString(), arguments: structuredClone(args), result: structuredClone(result) });
|
||||||
|
return queryId;
|
||||||
|
}
|
||||||
|
|
||||||
|
public async generate(input: ReportInput, signal?: AbortSignal) {
|
||||||
|
if (!input.source_query_ids.length || input.source_query_ids.some((id) => !this.#queries.has(id))) {
|
||||||
|
throw new HarnessError("REPORT_SOURCE_REQUIRED", "Report requires successful swads query IDs from this Session; never use example data");
|
||||||
|
}
|
||||||
|
const { data } = input;
|
||||||
|
for (const key of ["title", "report_type", "period", "comparison_period", "currency", "timezone", "data_status", "conclusion"]) {
|
||||||
|
if (typeof data[key] !== "string" || !data[key].trim()) throw new HarnessError("INVALID_REPORT", `Report requires ${key}`);
|
||||||
|
}
|
||||||
|
if (!["daily", "weekly"].includes(String(data.report_type)) || !Array.isArray(data.cards) || !Array.isArray(data.issues)) throw new HarnessError("INVALID_REPORT", "Invalid report type, cards or issues");
|
||||||
|
if (data.currency !== "USD") throw new HarnessError("REPORT_CURRENCY_UNSUPPORTED", "This renderer labels charts in USD. Keep other currencies separate and do not generate misleading USD charts.");
|
||||||
|
const payload = JSON.stringify({ ...data, source_queries: input.source_query_ids.map((id) => this.#queries.get(id)) }, null, 2);
|
||||||
|
if (Buffer.byteLength(payload) > 4_194_304) throw new HarnessError("REPORT_TOO_LARGE", "Report source data exceeds 4 MiB");
|
||||||
|
const reportId = `rpt_${randomUUID()}`;
|
||||||
|
// A fresh random directory avoids overwrites and untrusted symlink targets.
|
||||||
|
const root = await mkdtemp(join(this.options.workspace.root, ".report-"));
|
||||||
|
const jsonPath = join(root, "report.json");
|
||||||
|
const htmlPath = join(root, "report.html");
|
||||||
|
await writeFile(jsonPath, payload, { mode: 0o600, flag: "wx" });
|
||||||
|
const source = await SafeWorkspace.create(this.options.skillRoot);
|
||||||
|
const script = await source.resolveReadable("scripts/generate_report.py");
|
||||||
|
const env = { PATH: process.env.PATH ?? "/usr/bin:/bin", LANG: "en_US.UTF-8", PYTHONDONTWRITEBYTECODE: "1", PYTHONNOUSERSITE: "1" };
|
||||||
|
try {
|
||||||
|
await exec(this.options.pythonExecutable ?? "python3", ["-I", script, jsonPath, htmlPath], {
|
||||||
|
cwd: root, env, timeout: 30_000, maxBuffer: 65_536, ...(signal === undefined ? {} : { signal }),
|
||||||
|
});
|
||||||
|
await chmod(htmlPath, 0o600);
|
||||||
|
} catch {
|
||||||
|
throw new HarnessError("REPORT_RENDER_FAILED", "HTML generation failed. Check report JSON against the Skill schema and the configured Python runtime.");
|
||||||
|
}
|
||||||
|
const available = new Set<Format>(["json", "html"]);
|
||||||
|
let pdfWarning: string | undefined;
|
||||||
|
const pdfPath = join(root, "report.pdf");
|
||||||
|
const profile = await mkdtemp(join(root, ".chrome-"));
|
||||||
|
try {
|
||||||
|
await exec(this.options.chromeExecutable ?? (process.platform === "darwin" ? "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" : "chromium"), [
|
||||||
|
"--headless", "--disable-gpu", "--disable-extensions", "--disable-background-networking", "--no-first-run",
|
||||||
|
"--host-resolver-rules=MAP * ~NOTFOUND", "--no-pdf-header-footer", `--user-data-dir=${profile}`, `--print-to-pdf=${pdfPath}`, pathToFileURL(htmlPath).href,
|
||||||
|
], { cwd: root, env, timeout: 60_000, maxBuffer: 65_536, ...(signal === undefined ? {} : { signal }) });
|
||||||
|
const pdf = await readFile(pdfPath);
|
||||||
|
if (!pdf.subarray(0, 5).equals(Buffer.from("%PDF-")) || pdf.byteLength < 1_000 || !pdf.includes(Buffer.from("%%EOF"))) throw new Error("Invalid PDF output");
|
||||||
|
await chmod(pdfPath, 0o600);
|
||||||
|
available.add("pdf");
|
||||||
|
} catch {
|
||||||
|
pdfWarning = "PDF 生成失败:请检查 Chrome/Chromium 安装与运行权限;HTML 和 JSON 已保留。";
|
||||||
|
} finally {
|
||||||
|
await rm(profile, { recursive: true, force: true }).catch(() => undefined);
|
||||||
|
}
|
||||||
|
if (signal?.aborted) throw new HarnessError("REPORT_CANCELLED", "Report generation was cancelled");
|
||||||
|
this.#reports.set(reportId, { root, formats: available });
|
||||||
|
return {
|
||||||
|
reportId, title: data.title, pdfStatus: available.has("pdf") ? "ready" : "failed",
|
||||||
|
...(pdfWarning === undefined ? {} : { warning: pdfWarning }),
|
||||||
|
files: await Promise.all([...available].map(async (format) => ({
|
||||||
|
format, url: `/api/sessions/${this.options.sessionId}/reports/${reportId}/${format}`,
|
||||||
|
byteSize: (await stat(join(root, `report.${format}`))).size,
|
||||||
|
}))),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
public async read(reportId: string, format: string): Promise<{ bytes: Buffer; contentType: string }> {
|
||||||
|
const report = this.#reports.get(reportId);
|
||||||
|
if (report === undefined || !Object.hasOwn(formats, format) || !report.formats.has(format as Format)) throw new HarnessError("REPORT_NOT_FOUND", "Report file not found");
|
||||||
|
const file = await this.options.workspace.resolveReadable(join(report.root, `report.${format}`));
|
||||||
|
return { bytes: await readFile(file), contentType: formats[format as Format] };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createMinisReportTool(service: MinisReportService) {
|
||||||
|
return defineTool({
|
||||||
|
name: "generate_minis_report", label: "Drama Minis HTML / PDF",
|
||||||
|
description: "Generate HTML + A4 PDF + editable JSON using the installed Drama Minis Skill renderer. Read the Skill, references and example JSON structure first, NEVER reuse example values. Pass source_query_ids returned by successful REAL metrics_semantic_query calls in this Session, and freshly normalized report data (title, report_type daily/weekly, period, comparison_period, currency USD, timezone, data_status, conclusion, cards, issues, funnel, countries, country_revenue, country_funnels; weekly trend). Preserve missing values and source/account coverage in footnote. Requires page approval. Only declare PDF success when pdfStatus is ready. This replaces shell execution for the Skill report script.",
|
||||||
|
parameters: Type.Object({ source_query_ids: Type.Array(Type.String(), { minItems: 1 }), data: Type.Record(Type.String(), Type.Unknown()) }, { additionalProperties: false }),
|
||||||
|
executionMode: "sequential",
|
||||||
|
async execute(_id, params, signal) {
|
||||||
|
const details = await service.generate(params, signal);
|
||||||
|
return { content: [{ type: "text" as const, text: JSON.stringify(details) }], details };
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
|
||||||
|
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";
|
||||||
|
import type { Transport } from "@modelcontextprotocol/sdk/shared/transport.js";
|
||||||
|
import { defineTool } from "@earendil-works/pi-coding-agent";
|
||||||
|
import { Type } from "typebox";
|
||||||
|
import { HarnessError } from "../errors/harness-error.js";
|
||||||
|
|
||||||
|
const allowed = ["swads_whoami", "metrics_catalog", "metrics_semantic_query"] as const;
|
||||||
|
export const SWADS_MCP_URL = "https://ads.mincode.cn/mcp";
|
||||||
|
|
||||||
|
export class SwadsMcpClient {
|
||||||
|
#connection: Promise<Client> | undefined;
|
||||||
|
public constructor(private readonly options: { token: string; fetch?: typeof fetch }) {}
|
||||||
|
|
||||||
|
async #connect(): Promise<Client> {
|
||||||
|
if (!this.options.token.trim()) throw new HarnessError("SWADS_NOT_CONFIGURED", "SWADS_MCP_TOKEN is not configured on the server");
|
||||||
|
this.#connection ??= (async () => {
|
||||||
|
const client = new Client({ name: "agent-studio-mini", version: "0.1.0" });
|
||||||
|
try {
|
||||||
|
await client.connect(new StreamableHTTPClientTransport(new URL(SWADS_MCP_URL), {
|
||||||
|
requestInit: { redirect: "error", headers: { Authorization: `Bearer ${this.options.token}` } },
|
||||||
|
...(this.options.fetch === undefined ? {} : { fetch: this.options.fetch }),
|
||||||
|
}) as Transport, { timeout: 20_000 });
|
||||||
|
return client;
|
||||||
|
} catch {
|
||||||
|
await client.close().catch(() => undefined);
|
||||||
|
this.#connection = undefined;
|
||||||
|
throw new HarnessError("SWADS_CONNECTION_FAILED", "swads MCP connection failed; check the server credential and network");
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
return this.#connection;
|
||||||
|
}
|
||||||
|
|
||||||
|
public async call(name: string, args: Record<string, unknown>, signal?: AbortSignal) {
|
||||||
|
if (!allowed.some((item) => item === name)) throw new HarnessError("SWADS_TOOL_DENIED", "MCP tool is not allowed");
|
||||||
|
const client = await this.#connect();
|
||||||
|
try {
|
||||||
|
const result = await client.callTool({ name, arguments: args }, undefined, {
|
||||||
|
timeout: 60_000, ...(signal === undefined ? {} : { signal }),
|
||||||
|
});
|
||||||
|
// Remove the credential even if an upstream error reflects request headers in text.
|
||||||
|
return JSON.parse(JSON.stringify(result).replaceAll(this.options.token, "[REDACTED]")) as typeof result;
|
||||||
|
} catch {
|
||||||
|
throw new HarnessError("SWADS_REQUEST_FAILED", "swads MCP request failed or was cancelled; no metric values were fabricated");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public async close(): Promise<void> {
|
||||||
|
const connection = this.#connection;
|
||||||
|
this.#connection = undefined;
|
||||||
|
const client = await connection?.catch(() => undefined);
|
||||||
|
await client?.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createSwadsTools(client: SwadsMcpClient, onQueryResult: (args: Record<string, unknown>, result: unknown) => string) {
|
||||||
|
const descriptions = {
|
||||||
|
swads_whoami: "Read the authenticated swads identity and all visible advertising accounts. Do not assume the current account is the whole platform.",
|
||||||
|
metrics_catalog: "Read actual swads metric definitions, dimensions and request_schema before querying. No advertising metrics are queried by this call.",
|
||||||
|
metrics_semantic_query: "Query REAL swads analytics using metrics_catalog request_schema. Confirm analysis period, comparison period and account scope with the user first. Each call requires page approval. Pass date_from/date_to explicitly; external_account_id selects an account (not a campaign). Gateway can fill current tenant/org/platform/account. Never substitute mock data. Query accounts separately, preserve currencies/timezones and distinguish null from zero.",
|
||||||
|
};
|
||||||
|
return allowed.map((name) => defineTool({
|
||||||
|
name, label: name, description: descriptions[name],
|
||||||
|
parameters: Type.Object({}, { additionalProperties: true }),
|
||||||
|
executionMode: "sequential",
|
||||||
|
async execute(_id, params, signal) {
|
||||||
|
const args = params as Record<string, unknown>;
|
||||||
|
const result = await client.call(name, args, signal);
|
||||||
|
if (result.isError) throw new HarnessError("SWADS_TOOL_FAILED", JSON.stringify(result.content).slice(0, 2_000));
|
||||||
|
const sourceQueryId = name === "metrics_semantic_query" ? onQueryResult(args, result) : undefined;
|
||||||
|
const details = { source: "swads MCP", toolName: name, ...(sourceQueryId === undefined ? {} : { sourceQueryId }), result: result.structuredContent ?? result.content };
|
||||||
|
return {
|
||||||
|
content: [{ type: "text" as const, text: JSON.stringify(details) }],
|
||||||
|
details,
|
||||||
|
};
|
||||||
|
},
|
||||||
|
}));
|
||||||
|
}
|
||||||
@@ -1,9 +1,11 @@
|
|||||||
export type ToolPolicyDecision = "allow" | "ask" | "deny";
|
export type ToolPolicyDecision = "allow" | "ask" | "deny";
|
||||||
|
|
||||||
const allowedTools = new Set(["read", "grep", "find", "ls", "mock_ads_metrics"]);
|
const allowedTools = new Set(["read", "grep", "find", "ls", "mock_ads_metrics", "swads_whoami", "metrics_catalog", "cms_connection_status", "editor_job_status", "submit_drama_analysis", "propose_drama_selection", "submit_editor_plan"]);
|
||||||
|
|
||||||
export function decideToolPolicy(toolName: string): ToolPolicyDecision {
|
export function decideToolPolicy(toolName: string): ToolPolicyDecision {
|
||||||
|
if (["i2v_create_task", "i2v_status"].includes(toolName)) return "allow";
|
||||||
|
if (["i2v_upload_reference", "i2v_submit"].includes(toolName)) return "ask";
|
||||||
if (allowedTools.has(toolName)) return "allow";
|
if (allowedTools.has(toolName)) return "allow";
|
||||||
if (toolName === "save_report_draft") return "ask";
|
if (["save_report_draft", "metrics_semantic_query", "generate_minis_report", "cms_list_dramas", "cms_drama_inventory", "cms_read_subtitle", "prepare_drama_sources", "read_drama_frames", "render_editor_preview"].includes(toolName)) return "ask";
|
||||||
return "deny";
|
return "deny";
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,27 @@
|
|||||||
|
import {describe,it,expect} from "vitest";
|
||||||
|
import {assertAnalysisBinding} from "../src/media/analysis-binding.js";
|
||||||
|
|
||||||
|
const expected={analysisId:"analysis_real",dramaId:"drama_real",fetchedAt:"2026-09-14T00:00:00Z",title:"Actual title",synopsis:"Actual CMS synopsis",tags:["Mystery"],episodeCount:5,
|
||||||
|
sources:[{episodeId:"e1",episodeNo:1,duration:30,subtitle:"WEBVTT\n\n00:00:01.000 --> 00:00:03.000\nWhere is she?",frames:[{id:"f1",time:2}]},{episodeId:"e3",episodeNo:3,duration:30,subtitle:"WEBVTT\n\n00:00:01.000 --> 00:00:03.000\nFind her.",frames:[{id:"f3",time:2}]},{episodeId:"e5",episodeNo:5,duration:30,subtitle:"WEBVTT\n\n00:00:01.000 --> 00:00:03.000\nWho did this?",frames:[{id:"f5",time:2}]}]};
|
||||||
|
function analysis(){return {analysis_id:"analysis_real",drama:{drama_id:"drama_real",title:"Actual title",synopsis:"Actual CMS synopsis",tags:["Mystery"],episode_count:5},cms_source:{fetched_at:expected.fetchedAt},evidence:{
|
||||||
|
frame_evidence:[{episode:1,episode_id:"e1",frame_id:"f1",time:2,claim:"A person stands"},{episode:3,episode_id:"e3",frame_id:"f3",time:2,claim:"A person speaks"}],
|
||||||
|
subtitle_evidence:[{episode:1,episode_id:"e1",story_position:"opening",quote:"Where is she?"},{episode:3,episode_id:"e3",story_position:"middle",quote:"Find her."},{episode:5,episode_id:"e5",story_position:"late",quote:"Who did this?"}]}};}
|
||||||
|
describe("analysis evidence binding",()=>{
|
||||||
|
it("accepts only the actual job identity, CMS record and registered frame/subtitle evidence",()=>{
|
||||||
|
expect(()=>assertAnalysisBinding(analysis(),expected)).not.toThrow();
|
||||||
|
const changed=analysis();changed.analysis_id="analysis_other";
|
||||||
|
expect(()=>assertAnalysisBinding(changed,expected)).toThrow();
|
||||||
|
changed.analysis_id="analysis_real";changed.drama.synopsis="Model invented synopsis";
|
||||||
|
expect(()=>assertAnalysisBinding(changed,expected)).toThrow();
|
||||||
|
});
|
||||||
|
it("rejects invented frames, mismatched timestamps, duplicate frames and invented subtitle quotes",()=>{
|
||||||
|
const unknown=analysis();unknown.evidence.frame_evidence[0]!.frame_id="unknown";
|
||||||
|
expect(()=>assertAnalysisBinding(unknown,expected)).toThrow();
|
||||||
|
const timestamp=analysis();timestamp.evidence.frame_evidence[0]!.time=29;
|
||||||
|
expect(()=>assertAnalysisBinding(timestamp,expected)).toThrow();
|
||||||
|
const duplicate=analysis();duplicate.evidence.frame_evidence[1]=duplicate.evidence.frame_evidence[0]!;
|
||||||
|
expect(()=>assertAnalysisBinding(duplicate,expected)).toThrow();
|
||||||
|
const quote=analysis();quote.evidence.subtitle_evidence[0]!.quote="Invented dialogue";
|
||||||
|
expect(()=>assertAnalysisBinding(quote,expected)).toThrow();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,102 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import { CmsConnection } from "../src/cms/cms-connection.js";
|
||||||
|
|
||||||
|
const dramaId = "11111111-1111-4111-8111-111111111111";
|
||||||
|
const episodeId = "22222222-2222-4222-8222-222222222222";
|
||||||
|
const userId = "33333333-3333-4333-8333-333333333333";
|
||||||
|
const token = "test-access-secret";
|
||||||
|
function server(role = "admin") {
|
||||||
|
const requests: { url: URL; init: RequestInit | undefined }[] = [];
|
||||||
|
const fetcher: typeof fetch = (input, init) => {
|
||||||
|
const url = new URL(typeof input === "string" ? input : input instanceof URL ? input.href : input.url);
|
||||||
|
requests.push({ url, init });
|
||||||
|
const user = { id: userId, aud: "authenticated", role: "authenticated", email: "fixture@example.com", app_metadata: { role }, user_metadata: { role: "admin" }, created_at: "2026-01-01T00:00:00Z" };
|
||||||
|
let data: unknown;
|
||||||
|
if (url.pathname === "/auth/v1/token") data = { access_token: token, refresh_token: "refresh-secret", token_type: "bearer", expires_in: 3600, user };
|
||||||
|
else if (url.pathname === "/auth/v1/user") data = user;
|
||||||
|
else if (url.pathname === "/rest/v1/drama") data = [{ id: dramaId, title: "Test drama", synopsis: "Fixture only", language: "en", total_episodes: 2, status: "published" }];
|
||||||
|
else if (url.pathname === "/rest/v1/episode") data = [{ id: episodeId, drama_id: dramaId, episode_no: 1, title: "One", duration_seconds: 90, status: "published" }];
|
||||||
|
else if (url.pathname === "/rest/v1/drama_category") data = [{drama_id:dramaId,category:{name:"Mystery"}}];
|
||||||
|
else if (url.pathname === "/rest/v1/episode_source") data = [{ episode_id: episodeId, provider: "fixture", playback_path: "private/fixture.m3u8" }];
|
||||||
|
else if (url.pathname === "/rest/v1/subtitle_track") data = [{ episode_id: episodeId, locale: "en", path: "private/fixture.srt" }];
|
||||||
|
else if (url.pathname.endsWith("/subtitles/en.vtt")) return Promise.resolve(new Response("WEBVTT\n\n00:00:00.000 --> 00:00:01.000\nFixture", { headers: { "content-type": "text/vtt" } }));
|
||||||
|
else return Promise.resolve(new Response("unexpected destination", { status: 500 }));
|
||||||
|
return Promise.resolve(new Response(JSON.stringify(data), { headers: { "content-type": "application/json" } }));
|
||||||
|
};
|
||||||
|
return { requests, fetcher };
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("CMS connection", () => {
|
||||||
|
it("binds signing to the selected drama and keeps it on the fixed CMS admin route", async () => {
|
||||||
|
const fixture = server();
|
||||||
|
const cms = new CmsConnection({publishableKey:"sb_publishable_test",fetch:async(input,init)=>{
|
||||||
|
const url=new URL(typeof input==="string"?input:input instanceof URL?input.href:input.url);
|
||||||
|
if(url.pathname.endsWith(`/cms/playback/${episodeId}`)) {
|
||||||
|
expect(url.origin).toBe("https://swave.pencil-stub.com");
|
||||||
|
expect(new Headers(init?.headers).get("authorization")).toBe(`Bearer ${token}`);
|
||||||
|
return new Response(JSON.stringify({url:"https://video.pencil-stub.com/fixture/master.m3u8?auth_key=private",expires_at:new Date(Date.now()+600000).toISOString()}));
|
||||||
|
}
|
||||||
|
return fixture.fetcher(input,init);
|
||||||
|
}});
|
||||||
|
await cms.login("fixture@example.com","secret");
|
||||||
|
await expect(cms.playback(dramaId,userId)).rejects.toMatchObject({code:"CMS_EPISODE_NOT_FOUND"});
|
||||||
|
expect((await cms.playback(dramaId,episodeId)).url).toContain("https://video.pencil-stub.com/");
|
||||||
|
expect(JSON.stringify(cms.status())).not.toContain("auth_key");
|
||||||
|
});
|
||||||
|
it("blocks content before login and never sends credentials to a model-facing status", async () => {
|
||||||
|
const { fetcher } = server();
|
||||||
|
const cms = new CmsConnection({ publishableKey: "sb_publishable_test", fetch: fetcher });
|
||||||
|
await expect(cms.listDramas("", 0)).rejects.toMatchObject({ code: "CMS_LOGIN_REQUIRED" });
|
||||||
|
await cms.login("fixture@example.com", "password-secret");
|
||||||
|
expect(cms.status()).toMatchObject({ configured: true, connected: true });
|
||||||
|
expect(JSON.stringify(cms.status())).not.toMatch(/password-secret|test-access-secret|refresh-secret|fixture@example/);
|
||||||
|
expect((await cms.listDramas("", 0)).dramas[0]?.title).toBe("Test drama");
|
||||||
|
cms.clear();
|
||||||
|
expect(cms.status().connected).toBe(false);
|
||||||
|
await expect(cms.listDramas("", 0)).rejects.toMatchObject({ code: "CMS_LOGIN_REQUIRED" });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects user-editable administrator claims and discards the session", async () => {
|
||||||
|
const { fetcher } = server("viewer");
|
||||||
|
const cms = new CmsConnection({ publishableKey: "sb_publishable_test", fetch: fetcher });
|
||||||
|
await expect(cms.login("fixture@example.com", "secret")).rejects.toMatchObject({ code: "CMS_ADMIN_REQUIRED" });
|
||||||
|
expect(cms.status().connected).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reports incomplete inventory without leaking playback or subtitle paths", async () => {
|
||||||
|
const { fetcher } = server();
|
||||||
|
const cms = new CmsConnection({ publishableKey: "sb_publishable_test", fetch: fetcher });
|
||||||
|
await cms.login("fixture@example.com", "secret");
|
||||||
|
const inventory = await cms.inventory(dramaId);
|
||||||
|
expect(inventory.complete).toBe(false);
|
||||||
|
expect(inventory.drama.tags).toEqual(["Mystery"]);
|
||||||
|
expect(inventory.missingEpisodeNumbers).toEqual([2]);
|
||||||
|
expect(inventory.episodes[0]).toMatchObject({ episodeNo: 1, hasSource: true, subtitleLocales: ["en"] });
|
||||||
|
expect(JSON.stringify(inventory)).not.toContain("private/");
|
||||||
|
expect(inventory.fetchedAt).toMatch(/^\d{4}-/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("binds subtitle access to an actual episode in the chosen drama", async () => {
|
||||||
|
const { fetcher, requests } = server();
|
||||||
|
const cms = new CmsConnection({ publishableKey: "sb_publishable_test", fetch: fetcher });
|
||||||
|
await cms.login("fixture@example.com", "secret");
|
||||||
|
await expect(cms.subtitle(dramaId, userId, "en")).rejects.toMatchObject({ code: "CMS_EPISODE_NOT_FOUND" });
|
||||||
|
expect((await cms.subtitle(dramaId, episodeId, "en")).text).toContain("WEBVTT");
|
||||||
|
const subtitleRequest = requests.find((request) => request.url.pathname.endsWith(".vtt"));
|
||||||
|
expect(subtitleRequest?.url.origin).toBe("https://swave.pencil-stub.com");
|
||||||
|
expect(subtitleRequest?.init?.redirect).toBe("error");
|
||||||
|
expect(new Headers(subtitleRequest?.init?.headers).get("authorization")).toBe(`Bearer ${token}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects a late login result after the connection was cleared", async () => {
|
||||||
|
const { fetcher } = server();
|
||||||
|
let release: (() => void) | undefined;
|
||||||
|
const gate = new Promise<void>((resolve) => { release = resolve; });
|
||||||
|
const cms = new CmsConnection({ publishableKey: "sb_publishable_test", fetch: async (input, init) => { await gate; return fetcher(input, init); } });
|
||||||
|
const pending = cms.login("fixture@example.com", "secret");
|
||||||
|
cms.clear();
|
||||||
|
release?.();
|
||||||
|
await expect(pending).rejects.toMatchObject({ code: "CMS_LOGIN_CANCELLED" });
|
||||||
|
expect(cms.status().connected).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
import {describe,it,expect} from "vitest";
|
||||||
|
import {EditorService} from "../src/media/editor-service.js";
|
||||||
|
import {CmsConnection} from "../src/cms/cms-connection.js";
|
||||||
|
|
||||||
|
describe("editor job boundary",()=>{
|
||||||
|
it("requires CMS login before preparing media and rejects unknown jobs",async()=>{
|
||||||
|
const service=new EditorService({cms:new CmsConnection({publishableKey:""}),runtimeRoot:"/unused/editor-tests",sessionId:"ses_test"});
|
||||||
|
await expect(service.prepare({dramaId:"06aa396d-175a-48ed-ad76-02786ef4d0a9",episodeNumbers:[1,3,5]},new AbortController().signal)).rejects.toMatchObject({code:"CMS_LOGIN_REQUIRED"});
|
||||||
|
expect(()=>service.status("job_unknown")).toThrow();
|
||||||
|
await expect(service.submitAnalysis("job_unknown",{},new AbortController().signal)).rejects.toMatchObject({code:"EDITOR_JOB_NOT_FOUND"});
|
||||||
|
expect(()=>service.workflowView("job_unknown")).toThrow();
|
||||||
|
await expect(service.review("job_unknown","selection",{})).rejects.toMatchObject({code:"EDITOR_JOB_NOT_FOUND"});
|
||||||
|
});
|
||||||
|
it("rejects duplicate episode requests and oversized preparation scope before fetching",async()=>{
|
||||||
|
const service=new EditorService({cms:new CmsConnection({publishableKey:""}),runtimeRoot:"/unused/editor-tests",sessionId:"ses_test"});
|
||||||
|
for(const numbers of [[1,1],[1,2,3,4,5,6],[]])await expect(service.prepare({dramaId:"06aa396d-175a-48ed-ad76-02786ef4d0a9",episodeNumbers:numbers},new AbortController().signal)).rejects.toThrow();
|
||||||
|
});
|
||||||
|
});
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user