Add Beryl Agent and harness source
This commit is contained in:
@@ -0,0 +1,366 @@
|
||||
---
|
||||
name: h3-video-production
|
||||
description: >-
|
||||
Render approved shot plans on the private MiniMax H3 ComfyUI fleet with `uv run piecesai h3`, then assemble, verify and derive the delivery. Load it at two points: early, to preflight a brief's duration, aspect ratio, spoken language and frame rate before a shot plan is built on them, and again once an official creative Skill has produced an approved prompt and the user authorises generation. Not for replacing the creative workflow, skipping its confirmation gates, or diagnosing a broken worker.
|
||||
---
|
||||
|
||||
# H3 Video Production
|
||||
|
||||
Repository-local execution companion to the untouched official MiniMax H3 Skills.
|
||||
The official Skill owns the brief, shot plan and prompt. This Skill owns
|
||||
submission, delivery post-processing, and the evidence trail.
|
||||
|
||||
Everything runs through `uv run piecesai h3`. Never call a ComfyUI endpoint
|
||||
directly and never hard-code a provider URL — the fleet addresses live in
|
||||
`MINIMAX_H3_BASE_URLS` and the CLI owns worker selection.
|
||||
|
||||
## Fleet
|
||||
|
||||
Four ComfyUI workers: one RTX 5090 (remote) and three RTX 4090s (local).
|
||||
`uv run piecesai h3 workers` reports each one's status, device and queue depth.
|
||||
|
||||
- Mode: **Ref2VA only.** Pass `--h3-mode ref2va` explicitly. `auto` is *not*
|
||||
equivalent: `infer_h3_mode` resolves it by reference count (0 → `t2va`,
|
||||
1 → `i2va`, 2 → `fl2va`, 3+ → `ref2va`), and the `u06` workflow profile this
|
||||
Skill always renders on only supports `ref2va`. With `--h3-mode auto` and
|
||||
one or two `--reference` flags, the render fails with `LightX2V profile u06
|
||||
does not support i2va` (or `fl2va`). `auto` only lands on `ref2va` by
|
||||
itself once you supply three or more references.
|
||||
- References: **at least one required.** `--reference` takes a local path and
|
||||
uploads it; a public URL is not needed. Because `--h3-mode ref2va` is
|
||||
passed explicitly (see above), one or two references work fine — `auto`'s
|
||||
three-reference floor does not apply when the mode is explicit.
|
||||
- Cost knob: `--megapixels` (default `1.03`, range `0.1`–`1.03`). The cap
|
||||
was lowered from `1.5`; values above `1.03` are rejected at validation.
|
||||
`--resolution` only selects which aspect the canvas derives from —
|
||||
`--resolution 256p` at default megapixels still renders full cost.
|
||||
- Super-resolution: only the 5090 carries the node. Off by default.
|
||||
- Audio: H3 renders native audio. Keep it; standardise only at delivery.
|
||||
|
||||
### Render shots in parallel
|
||||
|
||||
One `render` call occupies one worker for the length of that shot, so a
|
||||
sequential loop over N shots leaves the rest of the fleet idle.
|
||||
|
||||
Use `batch` rather than hand-rolling concurrency:
|
||||
|
||||
```bash
|
||||
uv run piecesai h3 batch --jobs jobs.json --project-id <proj_id> [--dry-run]
|
||||
```
|
||||
|
||||
`jobs.json` is a list of clip jobs:
|
||||
|
||||
```json
|
||||
[{"clip": "P03_S01_C1", "group": "P03_S01",
|
||||
"prompt_file": "/abs/prompts/video/P03_S01_C1.txt",
|
||||
"refs": ["/abs/plate.png", "/abs/anchor.png"],
|
||||
"duration": 5.0, "megapixels": 1.03}]
|
||||
```
|
||||
|
||||
Batch pins one worker per concurrent task, benches a worker that fails twice in
|
||||
a row and requeues its work, skips clips whose request fingerprint is unchanged,
|
||||
and joins each `group` once its clips are complete **and agree on frame size**.
|
||||
`--dry-run` prints the plan without touching a worker.
|
||||
|
||||
The fingerprint reads the **contents** of the prompt file and the references,
|
||||
not their paths, so a rewritten prompt or an edited anchor is different work.
|
||||
It hashed filenames until 2026-08-20, when six clips were rewritten to fix a
|
||||
product-lock violation and every one came back `skip: unchanged`. After editing
|
||||
prompts, `--dry-run` and confirm the clips you changed say `would render`.
|
||||
|
||||
`megapixels` decides the canvas — `1.03` renders 768x1344 — so every clip
|
||||
in one `group` must use the same value. Mixing them is refused at assembly
|
||||
rather than silently rescaled.
|
||||
|
||||
Loudness normalisation and cover extraction stay outside batch: they depend on
|
||||
platform safe areas and delivery targets, which belong to the campaign rather
|
||||
than the runtime. `scripts/finalize_audio.py` is the loudness pass.
|
||||
|
||||
**Captions ship no tool at all.** `post` has `verify`, `assemble`, `frames` and
|
||||
`derive` and nothing else; there is no subtitle burn-in anywhere in this
|
||||
repository. When a brief needs burned-in subtitles, that is hand-written ffmpeg
|
||||
work — say so at the brief instead of treating it as a step that already exists.
|
||||
|
||||
## Comparing settings: change the seed
|
||||
|
||||
A render whose request fingerprint matches a completed run resumes from that
|
||||
run's receipt and returns the old file in seconds. Correct for resuming a batch,
|
||||
wrong for an A/B: two configurations at the same seed can come back
|
||||
byte-identical while the timings suggest the settings were free.
|
||||
|
||||
Give each comparison round its own `--seed`, then confirm the outputs actually
|
||||
differ before reading anything into them:
|
||||
|
||||
```bash
|
||||
ffmpeg -v error -i a.png -f rawvideo -pix_fmt rgb24 - | cmp -s - <(...)
|
||||
```
|
||||
|
||||
A mean channel difference near zero means the comparison never happened. This
|
||||
has already cost two rounds — `references/render-settings-evidence.md`.
|
||||
|
||||
## Steps and the turbo LoRA
|
||||
|
||||
The default is the **v4 step600 turbo LoRA at strength 1.0, 8 sampling steps**.
|
||||
Do not turn it off and do not raise steps without a reason from the shot in
|
||||
front of you: `--no-turbo-lora --sampling-steps 14` costs about 30% more
|
||||
wall-clock per clip and was measured as no better.
|
||||
|
||||
Strength is tuned for 1.0. Raise toward 1.2 against ghosting and smear, lower
|
||||
toward 0.8 against over-sharp grain. Stay inside 0.0-2.0.
|
||||
|
||||
```bash
|
||||
# only when a specific shot argues for it
|
||||
uv run piecesai h3 render ... --lora-strength 1.2 # ghosting on fast motion
|
||||
uv run piecesai h3 render ... --no-turbo-lora --sampling-steps 14 # slower, not better
|
||||
```
|
||||
|
||||
**Do not judge motion with a sharpness metric.** Motion coherence needs eyes:
|
||||
render the variants, show them, let the owner pick.
|
||||
|
||||
The four-round selection, the measured timings and why the sharpness proxy was
|
||||
rejected are in `references/render-settings-evidence.md`. Its numbers answer to
|
||||
the comment block above `U06_V4_SAMPLING_STEPS` in
|
||||
`src/piecesai/generators/minimax_h3.py`, which is the authority when they
|
||||
disagree.
|
||||
|
||||
## Preflight the brief — before the shot plan, not before the render
|
||||
|
||||
The official creative Skills settle duration, aspect ratio, language and beat
|
||||
timing in their first step, then build a shot plan on top. Several of those
|
||||
answers are constrained by the renderer, and each is cheap to change at the
|
||||
brief and expensive to change afterwards. Check them the moment they are given:
|
||||
|
||||
```bash
|
||||
uv run piecesai h3 preflight \
|
||||
--aspect-ratio 9:16 --duration 30 --language English \
|
||||
[--clip-duration 5] [--planning-fps 30] [--separate-audio narration] [--json]
|
||||
```
|
||||
|
||||
Exit `2` means a FAIL that no render will survive. WARN findings are soft gates:
|
||||
put the trade-off to the user, do not silently resolve it.
|
||||
|
||||
**The command owns the values; this section owns the reasons.** Do not restate
|
||||
its lists here — a stale copy of a supported-language list or a megapixel
|
||||
ceiling is exactly how the LoRA section above went three commits out of date.
|
||||
Run it and read what it prints.
|
||||
|
||||
The traps it exists to catch:
|
||||
|
||||
- **Aspect.** H3 renders two aspects. The official Skills offer five in their
|
||||
intake, and the other three are *delivery* aspects only — reachable through
|
||||
`post derive`, which refuses a crop that would discard the composition. Plan
|
||||
the shots for a render aspect; treat anything else as a derived cut, never as
|
||||
a promise.
|
||||
- **Duration.** One clip is 5-15 s, so a 30 s promo is several clips. Preflight
|
||||
splits the total evenly instead of taking clips off the front and leaving a
|
||||
short tail, because that tail is rejected at render *after* the other clips
|
||||
have been paid for.
|
||||
- **Frame rate.** H3 renders at **24 fps** (`frame_count = max(5, round(duration
|
||||
* 24))`). Several official Skills plan beats at 30 fps and count transition
|
||||
overlaps in frames. Those boundaries land on the wrong grid — re-express beats
|
||||
in seconds, or replan on a 24 fps grid.
|
||||
- **Spoken language.** H3 performs the words in `<d>…</d>` itself and is stable
|
||||
in a fixed set of languages; outside it the vendor says only "supported to
|
||||
varying degrees", which is an untested render rather than a refusal. The
|
||||
limit covers spoken and sung lines only — the prompt body stays English
|
||||
either way, and on-screen text is a glyph problem, not a speech one.
|
||||
`references/spoken-language-support.md` holds the list, the `<d>` tag values,
|
||||
the failure modes to listen for and the probe command.
|
||||
- **Assets with no local equivalent.** No TTS, no standalone music generation,
|
||||
no caption burn-in. A separately editable narration track cannot be delivered;
|
||||
spoken lines and score are performed by H3 inside the clip. Say so at the
|
||||
brief rather than promising a track that has no tool behind it.
|
||||
|
||||
None of this replaces the calling Skill's own confirmation gate. It supplies the
|
||||
constraints that gate should be confirming against.
|
||||
|
||||
## Write the prompt first — mandatory gate
|
||||
|
||||
**Do not call `render` with a prompt you wrote freehand.** Load
|
||||
`h3-prompt-writing` and write the prompt in H3's own Ref2VA structure first.
|
||||
That skill is not optional styling; its structure is what keeps a render
|
||||
faithful to its references.
|
||||
|
||||
Ref2VA prompts have six sections, in this order:
|
||||
|
||||
`subject_definitions` · `summary` · `retention_analysis` ·
|
||||
`detailed_description` · `overall_soundscape` · `non_diegetic_music`
|
||||
|
||||
Read `h3-prompt-writing/references/ref-en.txt` for the label rules and a
|
||||
complete example. Two parts of that structure do real work and are the
|
||||
reason this gate exists:
|
||||
|
||||
- **`subject_definitions`** forces you to name what each reference actually
|
||||
contributes — an identity, an environment, a composition anchor. A product
|
||||
photo cited as `<Subject 1>` behaves differently from the same file cited
|
||||
as `<Picture 1>` first frame.
|
||||
- **`retention_analysis`** forces one line per reference stating whether it is
|
||||
`fully_preserved`, `partially_preserved`, `attribute_transfer` or
|
||||
`weak_reference`, and in which shots. Writing that line is what surfaces a
|
||||
contradiction *before* you spend a render on it.
|
||||
|
||||
`<Picture N>` entries carry **no** `sources`, and no gate should compare their
|
||||
source set. A picture's source is itself, so `sources: <Picture 1>` is a
|
||||
tautology — and the one thing a picture definition must do is say who is in that
|
||||
frame, which forces it to mention that the same person also appears in
|
||||
`<Picture 2>`. The set becomes `[1,2]` and the entry is rejected for a
|
||||
contradiction that the format invented. `sources` belongs to `<Subject N>`;
|
||||
`h3-prompt-writing/references/ref-en.txt` never gives pictures one.
|
||||
|
||||
Getting this wrong is expensive and not obviously the instruction's fault: two
|
||||
different model tiers burned five attempts on it in one session. **The same
|
||||
error from two model tiers means the instruction is ambiguous, not that the
|
||||
model is too weak** — read the failing constraint and ask whether it carries any
|
||||
information at all. Full account in `references/failure-modes.md`. The only
|
||||
cross-image identity that genuinely needs strict checking is `<Subject j>`
|
||||
narrowing: a character losing identity across shots.
|
||||
|
||||
Freehand prose skips both checks, and this repository has the re-renders to
|
||||
show for it in the same file.
|
||||
|
||||
Keep the written prompt next to the render — it is the record of what was
|
||||
asked for, and the starting point when a shot has to be done again.
|
||||
|
||||
## Never write "no X" — it draws X
|
||||
|
||||
When a render puts an unwanted object in frame, the instinct is to name the
|
||||
object and forbid it. That reliably makes it worse: the text conditioning does
|
||||
not parse negation, so `no jug` and `jug` enter cross-attention as the same
|
||||
token. Negating an *attribute* (`no lip`) is safe; negating a whole object noun
|
||||
is what backfires.
|
||||
|
||||
To exclude an object, do not mention it. Occupy the space it would fill:
|
||||
|
||||
- **Separate the confusable classes into their own Subjects and distinguish
|
||||
them by form, not by count** — "the only vessel in this video that has a
|
||||
pouring lip" beats "no second pitcher".
|
||||
- **State counts positively:** `the whole table holds five glass objects in
|
||||
total`, `exactly one stream of liquid is visible at any moment`.
|
||||
- **Make the causal chain checkable** — keep source and target in the same frame
|
||||
so the render can be verified rather than argued about.
|
||||
- **Give a countable set a container sized to exactly that count.** Four cups
|
||||
that must stay four go on a board defined as four cups long and one cup deep,
|
||||
filled end to end. A physical boundary outperforms every counting phrase
|
||||
tried here, because it leaves nowhere for an extra one to stand.
|
||||
|
||||
It holds only while the whole container stays in frame. Any camera move that
|
||||
crops an end re-opens the space beyond it and the count drifts again — the
|
||||
shots that kept their board ends inside the frame rendered the right number
|
||||
on the first try, and the one that cropped them needed three attempts and a
|
||||
fixed wide frame before it agreed. When a shot has to assert a count, hold
|
||||
one frame that contains the whole container with bare ground visible beyond
|
||||
both ends, and let something else in the film carry the camera movement.
|
||||
|
||||
This applies to the boilerplate too. A `retention_analysis` line reading
|
||||
`no spoon, ladle, chopstick or peeler merges with it` carries the same risk and
|
||||
should be rewritten the same way.
|
||||
|
||||
The render that poured a pitcher into another pitcher is in
|
||||
`references/failure-modes.md`.
|
||||
|
||||
## Render
|
||||
|
||||
```bash
|
||||
uv run piecesai h3 init --name "<project name>"
|
||||
|
||||
uv run piecesai h3 render \
|
||||
--project-id <proj_id> --title "<shot title>" --skill <calling-skill-name> \
|
||||
--prompt-file <path> \
|
||||
--reference <character.png> --reference <scene.png> \
|
||||
--aspect-ratio 16:9 --duration 5.0 --resolution 768p \
|
||||
--h3-mode ref2va --megapixels 1.03
|
||||
```
|
||||
|
||||
Each render writes `generations/<run_id>/` under the project, holding
|
||||
`manifest.json`, `references/`, `prompts/`, `parts/`, `receipts/`, `qa/` and
|
||||
`finals/`. The finished master is `finals/final_master_<slug>.mp4`. That
|
||||
directory is the record — keep it.
|
||||
|
||||
**Validate the riskiest shot first at a low `--megapixels`** before committing
|
||||
the whole film at full cost. What "riskiest" means comes from the calling
|
||||
Skill's own shot table.
|
||||
|
||||
**A cheap probe validates motion, structure and composition — not how many
|
||||
objects appear.** Object count is a function of canvas size: at a low
|
||||
megapixel tier the frame is small and the model fills it with what you asked
|
||||
for, and at `1.03` the same prompt has more room and puts more things in it.
|
||||
A shot that must contain exactly N of something is only proven at the
|
||||
megapixel tier it will ship at. Probe the motion cheaply, then probe the
|
||||
count at full cost on that one shot before committing the batch.
|
||||
|
||||
## Delivery
|
||||
|
||||
```bash
|
||||
uv run piecesai h3 post assemble --shot s01.mp4 --shot s02.mp4 --out film.mp4
|
||||
uv run piecesai h3 post verify --input film.mp4 --expect-aspect 16:9 --expect-duration 20
|
||||
uv run piecesai h3 post frames --input film.mp4 --at 2.6,11.4,17.0 --out-dir review/
|
||||
uv run piecesai h3 post derive --input film.mp4 --aspect 9:16 --out vertical.mp4
|
||||
```
|
||||
|
||||
- `assemble` normalises each shot before concatenating, so mismatched time bases
|
||||
cannot drop audio or fail the join. Shot order is the order of `--shot`.
|
||||
- `verify` exits `2` on any FAIL. Read the failing line before re-rendering.
|
||||
- `derive` **refuses** a crop that would discard the composition and tells you to
|
||||
re-render at the target aspect instead. A refusal is a correct answer, not a
|
||||
tool failure.
|
||||
|
||||
For the standard delivery audio pass (AAC 256k / 48 kHz, 2× gain, peak limit):
|
||||
|
||||
```bash
|
||||
uv run python skills/h3-video-production/scripts/finalize_audio.py \
|
||||
film.mp4 film_audio_boost2x.mp4 --receipt film_audio_boost2x.receipt.json
|
||||
```
|
||||
|
||||
Keep both files. Do not enable denoising unless inspection proves persistent noise.
|
||||
|
||||
## When one clip's audio collapses
|
||||
|
||||
H3 drops a clip's audio at some rate: the sound cuts out partway and the tail
|
||||
turns into low-frequency rumble **louder than anything else in the clip**. It is
|
||||
a dice roll, not a prompt fault. Re-render the same prompt and it is usually
|
||||
clean.
|
||||
|
||||
The tell is the tail out-energising the whole clip, plus a harmonic band in the
|
||||
spectrogram that stops partway and is replaced by strong energy near DC. A clean
|
||||
take keeps its harmonic lines to the end. Check it without listening:
|
||||
|
||||
```bash
|
||||
ffmpeg -v error -i clip.mp4 -af volumedetect -f null - # per-segment energy
|
||||
ffmpeg -v error -i clip.mp4 -lavfi showspectrumpic=s=800x400 spec.png
|
||||
```
|
||||
|
||||
**Re-roll before theorising.** One collapsed sample is never grounds for
|
||||
changing the prompt scaffold that every other clip depends on. The trigger for
|
||||
that is the same prompt collapsing **twice in a row** — that is the difference
|
||||
between a bad die and a bad prompt. Measurements in
|
||||
`references/failure-modes.md`.
|
||||
|
||||
## Failures
|
||||
|
||||
Read the error before acting. Out of memory, a missing checkpoint or a rejected
|
||||
workflow is deterministic — change the request, do not retry it. A transient
|
||||
network error is already retried by the client.
|
||||
|
||||
When a render succeeds but the content is wrong, the fault is usually the prompt,
|
||||
not the fleet:
|
||||
|
||||
1. Quote the shot's reference anchors verbatim into the prompt and re-render.
|
||||
2. Still wrong — split the shot into two shorter ones, update the calling Skill's
|
||||
shot table, and re-render.
|
||||
3. Still wrong — stop and take it to the user with what you observed. There is no
|
||||
second model to fall back to here.
|
||||
|
||||
If renders fail across every shot rather than one, the problem is the fleet, not
|
||||
the prompt. Load `h3-fleet-ops` — do not diagnose workers from this Skill.
|
||||
|
||||
## swads MCP migration (not yet live)
|
||||
|
||||
An approved migration moves execution to the SW Ads queue via `video_h3_submit` /
|
||||
`video_h3_status` / `video_h3_workers`
|
||||
(`docs/superpowers/specs/2026-08-15-h3-execution-moves-to-swads-design.md`).
|
||||
P1 changed the routing; **P2 is not live and those tools do not exist yet.**
|
||||
|
||||
Check by whether `video_h3_submit` is available in the session. While it is not,
|
||||
`uv run piecesai h3` is the execution path — the migration design keeps it
|
||||
deliberately for exactly this window. When P2 lands, this section and the CLI
|
||||
instructions above retire together.
|
||||
Reference in New Issue
Block a user