18 lines
4.6 KiB
Markdown
18 lines
4.6 KiB
Markdown
# SW Ads image-to-video in Agent Studio
|
||
|
||
The Skills pane contains “图片 → 有声视频”. Add one JPEG, PNG or WebP reference, a title, the exact motion/sound prompt and a duration (10 seconds initially). “准备本地任务” receives the image through the existing Session attachment validator, copies normalized image bytes to private durable storage, and fills the chat composer with the registered task ID. Send that prepared message to use the original `swads-image-to-video` Skill. No CMS login or vision-enabled chat model is required. The local thumbnail itself is not transmitted to the chat model.
|
||
|
||
Two separate page approvals are required. Reference upload sends the frozen image to SW Ads storage and creates a `review_pending`, unassigned reference asset. Generation sends that asset and the frozen prompt to H3 `ref2va`, 9:16, 768p with the exact requested duration. Each dialog loads the registered image, prompt and duration and starts with an unchecked confirmation. Refusal causes no operation of that type. Costs and arbitrary duration support are not assumed. There is no provider fallback, automatic repair, account assignment, moderation approval or publication.
|
||
|
||
Only four controlled tools are exposed: `i2v_create_task`, `i2v_upload_reference`, `i2v_submit` and `i2v_status`. The latter three accept only a task ID owned by that Session. The analytics MCP allowlist remains separate. The server discovers current input schemas and the current MCP account’s `tenant_id` and `platform`; ambiguity blocks upload. Original Skill snapshots include SKILL.md, three references and scripts/media.py.
|
||
|
||
The durable history is stored under `AGENT_DIR/image-video` (default `.agent-studio-runtime/image-video`). It persists independently of chat Sessions. Returning to the local page lists registered tasks and polls at most every 30 seconds per task, including after restart. Old tasks cannot be reassigned to a new Session for mutation. Only one server process may own a runtime directory. Each operation serializes its task, validates a strict versioned record and reference hash, and atomically persists revisioned intents before network mutations. Unknown upload/submission outcomes are quarantined; never delete their records and retry blindly. A returned generation receipt is retained separately before updating its task state, so stale results cannot silently overwrite a newer revision.
|
||
|
||
The local history and media routes are `/api/image-video/tasks`, `/api/image-video/tasks/:taskId`, and the registered `reference` / `video` endpoints. Draft creation is `/api/sessions/:sessionId/image-video/tasks`. All require a loopback Host with an allowed local port, reject cross-site requests and mismatched Origin, and media/history are `no-store`. No endpoint accepts a remote media URL or arbitrary filesystem path. Media supports HTTP Range and download.
|
||
|
||
MP4 collection uses the registered job’s single `video/mp4` artifact only, public HTTPS DNS validation with the actual connection pinned to the validated IP, no redirects, bounded bytes and elapsed time. The original Skill verifier runs with every FFmpeg/FFprobe subprocess restricted to file/pipe protocols and the MP4 demuxer. Full decode, sound presence, portrait geometry, positive frame rate and exact video duration are checked; audio container packet rounding has a 50 ms tolerance. A local SHA-256 binds the validated file to the registered player/download endpoint. A successful technical preview remains “待视听检查”; no human visual or audio approval is inferred.
|
||
|
||
Server configuration: `SWADS_MCP_TOKEN` stays server-only; optional `IMAGE_VIDEO_SKILL_DIR` overrides the original Skill directory; `PYTHON_EXECUTABLE` may select the installed local Python. Existing bundled FFmpeg/FFprobe are reused. Never place credentials in task prompts or browser input.
|
||
|
||
Validation boundary: automated tests use synthetic local references and fake provider boundary responses. Read-only live discovery confirms current input schemas and identity structure; current tools do not advertise output schemas. The adapter conservatively accepts documented top-level upload fields (`upload_id`, `object_key`, `upload_url`), import `creative_factory_asset_id`, submit `job_id`, and status `status` plus `artifacts[{content_type,url}]`; unsupported or ambiguous envelopes stop safely. Actual upload/finalization/status envelopes must still be checked with an explicitly approved live reference. A real target-duration H3 generation, actual browser playback, download hash match and human sound/visual review are required before claiming end-to-end real generation success. Synthetic test video and jsdom component evidence do not satisfy that final gate.
|