4.6 KiB
SW Ads image-to-video in Agent Studio
The Skills pane contains “图片 → 有声视频”. Add one JPEG, PNG or WebP reference, a title, the exact motion/sound prompt and a duration (10 seconds initially). “准备本地任务” receives the image through the existing Session attachment validator, copies normalized image bytes to private durable storage, and fills the chat composer with the registered task ID. Send that prepared message to use the original swads-image-to-video Skill. No CMS login or vision-enabled chat model is required. The local thumbnail itself is not transmitted to the chat model.
Two separate page approvals are required. Reference upload sends the frozen image to SW Ads storage and creates a review_pending, unassigned reference asset. Generation sends that asset and the frozen prompt to H3 ref2va, 9:16, 768p with the exact requested duration. Each dialog loads the registered image, prompt and duration and starts with an unchecked confirmation. Refusal causes no operation of that type. Costs and arbitrary duration support are not assumed. There is no provider fallback, automatic repair, account assignment, moderation approval or publication.
Only four controlled tools are exposed: i2v_create_task, i2v_upload_reference, i2v_submit and i2v_status. The latter three accept only a task ID owned by that Session. The analytics MCP allowlist remains separate. The server discovers current input schemas and the current MCP account’s tenant_id and platform; ambiguity blocks upload. Original Skill snapshots include SKILL.md, three references and scripts/media.py.
The durable history is stored under AGENT_DIR/image-video (default .agent-studio-runtime/image-video). It persists independently of chat Sessions. Returning to the local page lists registered tasks and polls at most every 30 seconds per task, including after restart. Old tasks cannot be reassigned to a new Session for mutation. Only one server process may own a runtime directory. Each operation serializes its task, validates a strict versioned record and reference hash, and atomically persists revisioned intents before network mutations. Unknown upload/submission outcomes are quarantined; never delete their records and retry blindly. A returned generation receipt is retained separately before updating its task state, so stale results cannot silently overwrite a newer revision.
The local history and media routes are /api/image-video/tasks, /api/image-video/tasks/:taskId, and the registered reference / video endpoints. Draft creation is /api/sessions/:sessionId/image-video/tasks. All require a loopback Host with an allowed local port, reject cross-site requests and mismatched Origin, and media/history are no-store. No endpoint accepts a remote media URL or arbitrary filesystem path. Media supports HTTP Range and download.
MP4 collection uses the registered job’s single video/mp4 artifact only, public HTTPS DNS validation with the actual connection pinned to the validated IP, no redirects, bounded bytes and elapsed time. The original Skill verifier runs with every FFmpeg/FFprobe subprocess restricted to file/pipe protocols and the MP4 demuxer. Full decode, sound presence, portrait geometry, positive frame rate and exact video duration are checked; audio container packet rounding has a 50 ms tolerance. A local SHA-256 binds the validated file to the registered player/download endpoint. A successful technical preview remains “待视听检查”; no human visual or audio approval is inferred.
Server configuration: SWADS_MCP_TOKEN stays server-only; optional IMAGE_VIDEO_SKILL_DIR overrides the original Skill directory; PYTHON_EXECUTABLE may select the installed local Python. Existing bundled FFmpeg/FFprobe are reused. Never place credentials in task prompts or browser input.
Validation boundary: automated tests use synthetic local references and fake provider boundary responses. Read-only live discovery confirms current input schemas and identity structure; current tools do not advertise output schemas. The adapter conservatively accepts documented top-level upload fields (upload_id, object_key, upload_url), import creative_factory_asset_id, submit job_id, and status status plus artifacts[{content_type,url}]; unsupported or ambiguous envelopes stop safely. Actual upload/finalization/status envelopes must still be checked with an explicitly approved live reference. A real target-duration H3 generation, actual browser playback, download hash match and human sound/visual review are required before claiming end-to-end real generation success. Synthetic test video and jsdom component evidence do not satisfy that final gate.