19926533763/short-video-viral-replicator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add Beryl Agent and harness source

Browse Source
This commit is contained in:
19926533763
2026-09-08 20:10:03 +08:00
parent 06d72d001c
commit 47cfbcd66a
70 changed files with 12558 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
agent-studio-mini/.agents/skills/h3-video-production/SKILL.md
+366
View File
@@ -0,0 +1,366 @@
---
name: h3-video-production
description: >-
Render approved shot plans on the private MiniMax H3 ComfyUI fleet with `uv run piecesai h3`, then assemble, verify and derive the delivery. Load it at two points: early, to preflight a brief's duration, aspect ratio, spoken language and frame rate before a shot plan is built on them, and again once an official creative Skill has produced an approved prompt and the user authorises generation. Not for replacing the creative workflow, skipping its confirmation gates, or diagnosing a broken worker.
---
# H3 Video Production
Repository-local execution companion to the untouched official MiniMax H3 Skills.
The official Skill owns the brief, shot plan and prompt. This Skill owns
submission, delivery post-processing, and the evidence trail.
Everything runs through `uv run piecesai h3`. Never call a ComfyUI endpoint
directly and never hard-code a provider URL — the fleet addresses live in
`MINIMAX_H3_BASE_URLS` and the CLI owns worker selection.
## Fleet
Four ComfyUI workers: one RTX 5090 (remote) and three RTX 4090s (local).
`uv run piecesai h3 workers` reports each one's status, device and queue depth.
- Mode: **Ref2VA only.** Pass `--h3-mode ref2va` explicitly. `auto` is *not*
equivalent: `infer_h3_mode` resolves it by reference count (0 → `t2va`,
1 → `i2va`, 2 → `fl2va`, 3+ → `ref2va`), and the `u06` workflow profile this
Skill always renders on only supports `ref2va`. With `--h3-mode auto` and
one or two `--reference` flags, the render fails with `LightX2V profile u06
does not support i2va` (or `fl2va`). `auto` only lands on `ref2va` by
itself once you supply three or more references.
- References: **at least one required.** `--reference` takes a local path and
uploads it; a public URL is not needed. Because `--h3-mode ref2va` is
passed explicitly (see above), one or two references work fine — `auto`'s
three-reference floor does not apply when the mode is explicit.
- Cost knob: `--megapixels` (default `1.03`, range `0.1`–`1.03`). The cap
was lowered from `1.5`; values above `1.03` are rejected at validation.
`--resolution` only selects which aspect the canvas derives from —
`--resolution 256p` at default megapixels still renders full cost.
- Super-resolution: only the 5090 carries the node. Off by default.
- Audio: H3 renders native audio. Keep it; standardise only at delivery.
### Render shots in parallel
One `render` call occupies one worker for the length of that shot, so a
sequential loop over N shots leaves the rest of the fleet idle.
Use `batch` rather than hand-rolling concurrency:
```bash
uv run piecesai h3 batch --jobs jobs.json --project-id <proj_id> [--dry-run]
```
`jobs.json` is a list of clip jobs:
```json
[{"clip": "P03_S01_C1", "group": "P03_S01",
"prompt_file": "/abs/prompts/video/P03_S01_C1.txt",
"refs": ["/abs/plate.png", "/abs/anchor.png"],
"duration": 5.0, "megapixels": 1.03}]
```
Batch pins one worker per concurrent task, benches a worker that fails twice in
a row and requeues its work, skips clips whose request fingerprint is unchanged,
and joins each `group` once its clips are complete **and agree on frame size**.
`--dry-run` prints the plan without touching a worker.
The fingerprint reads the **contents** of the prompt file and the references,
not their paths, so a rewritten prompt or an edited anchor is different work.
It hashed filenames until 2026-08-20, when six clips were rewritten to fix a
product-lock violation and every one came back `skip: unchanged`. After editing
prompts, `--dry-run` and confirm the clips you changed say `would render`.
`megapixels` decides the canvas — `1.03` renders 768x1344 — so every clip
in one `group` must use the same value. Mixing them is refused at assembly
rather than silently rescaled.
Loudness normalisation and cover extraction stay outside batch: they depend on
platform safe areas and delivery targets, which belong to the campaign rather
than the runtime. `scripts/finalize_audio.py` is the loudness pass.
**Captions ship no tool at all.** `post` has `verify`, `assemble`, `frames` and
`derive` and nothing else; there is no subtitle burn-in anywhere in this
repository. When a brief needs burned-in subtitles, that is hand-written ffmpeg
work — say so at the brief instead of treating it as a step that already exists.
## Comparing settings: change the seed
A render whose request fingerprint matches a completed run resumes from that
run's receipt and returns the old file in seconds. Correct for resuming a batch,
wrong for an A/B: two configurations at the same seed can come back
byte-identical while the timings suggest the settings were free.
Give each comparison round its own `--seed`, then confirm the outputs actually
differ before reading anything into them:
```bash
ffmpeg -v error -i a.png -f rawvideo -pix_fmt rgb24 - | cmp -s - <(...)
```
A mean channel difference near zero means the comparison never happened. This
has already cost two rounds — `references/render-settings-evidence.md`.
## Steps and the turbo LoRA
The default is the **v4 step600 turbo LoRA at strength 1.0, 8 sampling steps**.
Do not turn it off and do not raise steps without a reason from the shot in
front of you: `--no-turbo-lora --sampling-steps 14` costs about 30% more
wall-clock per clip and was measured as no better.
Strength is tuned for 1.0. Raise toward 1.2 against ghosting and smear, lower
toward 0.8 against over-sharp grain. Stay inside 0.0-2.0.
```bash
# only when a specific shot argues for it
uv run piecesai h3 render ... --lora-strength 1.2 # ghosting on fast motion
uv run piecesai h3 render ... --no-turbo-lora --sampling-steps 14 # slower, not better
```
**Do not judge motion with a sharpness metric.** Motion coherence needs eyes:
render the variants, show them, let the owner pick.
The four-round selection, the measured timings and why the sharpness proxy was
rejected are in `references/render-settings-evidence.md`. Its numbers answer to
the comment block above `U06_V4_SAMPLING_STEPS` in
`src/piecesai/generators/minimax_h3.py`, which is the authority when they
disagree.
## Preflight the brief — before the shot plan, not before the render
The official creative Skills settle duration, aspect ratio, language and beat
timing in their first step, then build a shot plan on top. Several of those
answers are constrained by the renderer, and each is cheap to change at the
brief and expensive to change afterwards. Check them the moment they are given:
```bash
uv run piecesai h3 preflight \
--aspect-ratio 9:16 --duration 30 --language English \
[--clip-duration 5] [--planning-fps 30] [--separate-audio narration] [--json]
```
Exit `2` means a FAIL that no render will survive. WARN findings are soft gates:
put the trade-off to the user, do not silently resolve it.
**The command owns the values; this section owns the reasons.** Do not restate
its lists here — a stale copy of a supported-language list or a megapixel
ceiling is exactly how the LoRA section above went three commits out of date.
Run it and read what it prints.
The traps it exists to catch:
- **Aspect.** H3 renders two aspects. The official Skills offer five in their
intake, and the other three are *delivery* aspects only — reachable through
`post derive`, which refuses a crop that would discard the composition. Plan
the shots for a render aspect; treat anything else as a derived cut, never as
a promise.
- **Duration.** One clip is 5-15 s, so a 30 s promo is several clips. Preflight
splits the total evenly instead of taking clips off the front and leaving a
short tail, because that tail is rejected at render *after* the other clips
have been paid for.
- **Frame rate.** H3 renders at **24 fps** (`frame_count = max(5, round(duration
* 24))`). Several official Skills plan beats at 30 fps and count transition
overlaps in frames. Those boundaries land on the wrong grid — re-express beats
in seconds, or replan on a 24 fps grid.
- **Spoken language.** H3 performs the words in `<d>…</d>` itself and is stable
in a fixed set of languages; outside it the vendor says only "supported to
varying degrees", which is an untested render rather than a refusal. The
limit covers spoken and sung lines only — the prompt body stays English
either way, and on-screen text is a glyph problem, not a speech one.
`references/spoken-language-support.md` holds the list, the `<d>` tag values,
the failure modes to listen for and the probe command.
- **Assets with no local equivalent.** No TTS, no standalone music generation,
no caption burn-in. A separately editable narration track cannot be delivered;
spoken lines and score are performed by H3 inside the clip. Say so at the
brief rather than promising a track that has no tool behind it.
None of this replaces the calling Skill's own confirmation gate. It supplies the
constraints that gate should be confirming against.
## Write the prompt first — mandatory gate
**Do not call `render` with a prompt you wrote freehand.** Load
`h3-prompt-writing` and write the prompt in H3's own Ref2VA structure first.
That skill is not optional styling; its structure is what keeps a render
faithful to its references.
Ref2VA prompts have six sections, in this order:
`subject_definitions` · `summary` · `retention_analysis` ·
`detailed_description` · `overall_soundscape` · `non_diegetic_music`
Read `h3-prompt-writing/references/ref-en.txt` for the label rules and a
complete example. Two parts of that structure do real work and are the
reason this gate exists:
- **`subject_definitions`** forces you to name what each reference actually
contributes — an identity, an environment, a composition anchor. A product
photo cited as `<Subject 1>` behaves differently from the same file cited
as `<Picture 1>` first frame.
- **`retention_analysis`** forces one line per reference stating whether it is
`fully_preserved`, `partially_preserved`, `attribute_transfer` or
`weak_reference`, and in which shots. Writing that line is what surfaces a
contradiction *before* you spend a render on it.
`<Picture N>` entries carry **no** `sources`, and no gate should compare their
source set. A picture's source is itself, so `sources: <Picture 1>` is a
tautology — and the one thing a picture definition must do is say who is in that
frame, which forces it to mention that the same person also appears in
`<Picture 2>`. The set becomes `[1,2]` and the entry is rejected for a
contradiction that the format invented. `sources` belongs to `<Subject N>`;
`h3-prompt-writing/references/ref-en.txt` never gives pictures one.
Getting this wrong is expensive and not obviously the instruction's fault: two
different model tiers burned five attempts on it in one session. **The same
error from two model tiers means the instruction is ambiguous, not that the
model is too weak** — read the failing constraint and ask whether it carries any
information at all. Full account in `references/failure-modes.md`. The only
cross-image identity that genuinely needs strict checking is `<Subject j>`
narrowing: a character losing identity across shots.
Freehand prose skips both checks, and this repository has the re-renders to
show for it in the same file.
Keep the written prompt next to the render — it is the record of what was
asked for, and the starting point when a shot has to be done again.
## Never write "no X" — it draws X
When a render puts an unwanted object in frame, the instinct is to name the
object and forbid it. That reliably makes it worse: the text conditioning does
not parse negation, so `no jug` and `jug` enter cross-attention as the same
token. Negating an *attribute* (`no lip`) is safe; negating a whole object noun
is what backfires.
To exclude an object, do not mention it. Occupy the space it would fill:
- **Separate the confusable classes into their own Subjects and distinguish
them by form, not by count** — "the only vessel in this video that has a
pouring lip" beats "no second pitcher".
- **State counts positively:** `the whole table holds five glass objects in
total`, `exactly one stream of liquid is visible at any moment`.
- **Make the causal chain checkable** — keep source and target in the same frame
so the render can be verified rather than argued about.
- **Give a countable set a container sized to exactly that count.** Four cups
that must stay four go on a board defined as four cups long and one cup deep,
filled end to end. A physical boundary outperforms every counting phrase
tried here, because it leaves nowhere for an extra one to stand.
It holds only while the whole container stays in frame. Any camera move that
crops an end re-opens the space beyond it and the count drifts again — the
shots that kept their board ends inside the frame rendered the right number
on the first try, and the one that cropped them needed three attempts and a
fixed wide frame before it agreed. When a shot has to assert a count, hold
one frame that contains the whole container with bare ground visible beyond
both ends, and let something else in the film carry the camera movement.
This applies to the boilerplate too. A `retention_analysis` line reading
`no spoon, ladle, chopstick or peeler merges with it` carries the same risk and
should be rewritten the same way.
The render that poured a pitcher into another pitcher is in
`references/failure-modes.md`.
## Render
```bash
uv run piecesai h3 init --name "<project name>"
uv run piecesai h3 render \
--project-id <proj_id> --title "<shot title>" --skill <calling-skill-name> \
--prompt-file <path> \
--reference <character.png> --reference <scene.png> \
--aspect-ratio 16:9 --duration 5.0 --resolution 768p \
--h3-mode ref2va --megapixels 1.03
```
Each render writes `generations/<run_id>/` under the project, holding
`manifest.json`, `references/`, `prompts/`, `parts/`, `receipts/`, `qa/` and
`finals/`. The finished master is `finals/final_master_<slug>.mp4`. That
directory is the record — keep it.
**Validate the riskiest shot first at a low `--megapixels`** before committing
the whole film at full cost. What "riskiest" means comes from the calling
Skill's own shot table.
**A cheap probe validates motion, structure and composition — not how many
objects appear.** Object count is a function of canvas size: at a low
megapixel tier the frame is small and the model fills it with what you asked
for, and at `1.03` the same prompt has more room and puts more things in it.
A shot that must contain exactly N of something is only proven at the
megapixel tier it will ship at. Probe the motion cheaply, then probe the
count at full cost on that one shot before committing the batch.
## Delivery
```bash
uv run piecesai h3 post assemble --shot s01.mp4 --shot s02.mp4 --out film.mp4
uv run piecesai h3 post verify --input film.mp4 --expect-aspect 16:9 --expect-duration 20
uv run piecesai h3 post frames --input film.mp4 --at 2.6,11.4,17.0 --out-dir review/
uv run piecesai h3 post derive --input film.mp4 --aspect 9:16 --out vertical.mp4
```
- `assemble` normalises each shot before concatenating, so mismatched time bases
cannot drop audio or fail the join. Shot order is the order of `--shot`.
- `verify` exits `2` on any FAIL. Read the failing line before re-rendering.
- `derive` **refuses** a crop that would discard the composition and tells you to
re-render at the target aspect instead. A refusal is a correct answer, not a
tool failure.
For the standard delivery audio pass (AAC 256k / 48 kHz, 2× gain, peak limit):
```bash
uv run python skills/h3-video-production/scripts/finalize_audio.py \
film.mp4 film_audio_boost2x.mp4 --receipt film_audio_boost2x.receipt.json
```
Keep both files. Do not enable denoising unless inspection proves persistent noise.
## When one clip's audio collapses
H3 drops a clip's audio at some rate: the sound cuts out partway and the tail
turns into low-frequency rumble **louder than anything else in the clip**. It is
a dice roll, not a prompt fault. Re-render the same prompt and it is usually
clean.
The tell is the tail out-energising the whole clip, plus a harmonic band in the
spectrogram that stops partway and is replaced by strong energy near DC. A clean
take keeps its harmonic lines to the end. Check it without listening:
```bash
ffmpeg -v error -i clip.mp4 -af volumedetect -f null - # per-segment energy
ffmpeg -v error -i clip.mp4 -lavfi showspectrumpic=s=800x400 spec.png
```
**Re-roll before theorising.** One collapsed sample is never grounds for
changing the prompt scaffold that every other clip depends on. The trigger for
that is the same prompt collapsing **twice in a row** — that is the difference
between a bad die and a bad prompt. Measurements in
`references/failure-modes.md`.
## Failures
Read the error before acting. Out of memory, a missing checkpoint or a rejected
workflow is deterministic — change the request, do not retry it. A transient
network error is already retried by the client.
When a render succeeds but the content is wrong, the fault is usually the prompt,
not the fleet:
1. Quote the shot's reference anchors verbatim into the prompt and re-render.
2. Still wrong — split the shot into two shorter ones, update the calling Skill's
shot table, and re-render.
3. Still wrong — stop and take it to the user with what you observed. There is no
second model to fall back to here.
If renders fail across every shot rather than one, the problem is the fleet, not
the prompt. Load `h3-fleet-ops` — do not diagnose workers from this Skill.
## swads MCP migration (not yet live)
An approved migration moves execution to the SW Ads queue via `video_h3_submit` /
`video_h3_status` / `video_h3_workers`
(`docs/superpowers/specs/2026-08-15-h3-execution-moves-to-swads-design.md`).
P1 changed the routing; **P2 is not live and those tools do not exist yet.**
Check by whether `video_h3_submit` is available in the session. While it is not,
`uv run piecesai h3` is the execution path — the migration design keeps it
deliberately for exactly this window. When P2 lands, this section and the CLI
instructions above retire together.
agent-studio-mini/.agents/skills/h3-video-production/agents/openai.yaml
+4
View File
@@ -0,0 +1,4 @@
interface:
display_name: "PiecesAI H3 Video Production"
short_description: "Render H3 shorts on private multi-GPU workers"
default_prompt: "Use $h3-video-production to render this approved short-video plan on the PiecesAI H3 worker pool."
agent-studio-mini/.agents/skills/h3-video-production/references/failure-modes.md
+122
View File
@@ -0,0 +1,122 @@
# Failure modes measured on this repository
The rules live in `SKILL.md`. This file is what was actually observed, with the
numbers. Read it when a rule looks arbitrary, or when you are tempted to fix a
bad render by doing more of what caused it.
## Negation summons the object — 2026-08-17
A pitcher-and-cups shot kept growing extra pouring vessels, so the retention
line was hardened with an explicit ban:
```
no second pitcher, no fifth cup, no jug, no bottle, no carafe
```
The next render was **worse**: the pitcher now poured *into another pitcher*,
the receiving cup having grown a spout. Deleting the whole list and rewriting
the same constraint positively passed on the first try.
The text conditioning does not parse negation — `no jug` and `jug` enter
cross-attention as the same token. Negating an *attribute* (`no lip`) is safe;
what backfires is negating a whole object noun.
What worked instead: the pitcher became "the only vessel in this video that has
a pouring lip"; the cups became `no lip and no handle` and `only ever receives
liquid`; the count was stated positively as `the whole table holds five glass
objects in total`.
## `<Picture N>` entries and `sources` — 2026-08-10
Both `qwen3.8-max` and `claude-opus-5` hit the same rejection three times each
and burned five attempts between them.
The cause was the instruction, not the models. A picture's source is itself, so
`sources: <Picture 1>` is a tautology — and the one thing a picture definition
must do is say who is in that frame, which forces it to mention that the same
person also appears in `<Picture 2>`. The set becomes `[1,2]` and the entry is
rejected for a contradiction the format invented.
**The same error from two different model tiers means the instruction is
ambiguous, not that the model is too weak.** Reaching for a bigger model there
is the wrong move. Read the failing constraint and ask whether it carries any
information at all.
## What freehand prompts produced
Observed on this repository's own production runs, each costing a re-render the
Ref2VA structure would have caught for free:
- strands of vegetable appearing with no vegetable in frame
- two copies of a single-item product in one shot
- two tools merging onto one subject instead of staying on opposite sides of
the board
## Audio collapse is a dice roll — 2026-08-09
One beat, an alarm bell held for the full 5 s, same prompt both times:
| | collapsed | re-rolled |
|---|---|---|
| last 1.4 s | −8.8 dB — loudest point in the clip | −16.5 dB |
| span across the clip | 25.8 dB | 6.5 dB |
| clipped samples | 844 | 85 |
Two explanations looked equally sound at the time — the sound was described too
vaguely, and H3 cannot hold a sustained high-energy ambience. Both would have
sent someone rewriting the prompt scaffold that every other clip depends on.
One re-render falsified both: same wording, same sustained requirement, clean
take.
One collapsed sample is never grounds for changing the scaffold. The trigger for
that is the same prompt collapsing twice in a row — that is the difference
between a bad die and a bad prompt.
## Object count is set by canvas size — 2026-08-20
A pitcher-and-four-cups delivery, the same product family as the negation case
above. The brief's hard lock was "exactly one pitcher and four cups, countable".
The riskiest shot was probed three times at `--megapixels 0.3`. All three came
back with exactly four cups. The same prompts at `1.03` grew a fifth in three of
the six clips. Nothing about the prompt changed — only the canvas. At the low
tier the frame is small and the model fills it with what was asked for; at full
tier the extra horizontal room gets filled too.
So a cheap probe is evidence about motion, structure and composition, and no
evidence at all about how many things appear.
What finally held was a container, not a phrase. The four cups were placed on a
serving board defined as *four cups long and one cup deep, covered end to end*.
Where the whole board stayed inside the frame the count was correct on the first
render. Where a camera move cropped an end, the count drifted into the cropped
space:
| shot | camera | attempts to correct |
|---|---|---|
| pitcher lift | fixed wide, whole board in frame | 1 |
| pour | fixed wide, whole board in frame | 1 |
| final settle | opens wide, only widens | 1 |
| ice drops | close-up, then truck, then "static wide" wording | 3, fixed only by reusing a working shot's establishing geometry verbatim |
The last row is the useful one. Three separate rewordings of "keep all four in
frame" did not move it. Copying the opening sentence from a shot that already
rendered four correctly did. When an instruction has failed twice, stop
rewording it and transplant the wording that works.
## The batch fingerprint hashed filenames — 2026-08-20
Same delivery. All six prompts were rewritten to fix the product-lock violation
above, and `batch` returned `skip: unchanged` for all six: `job_fingerprint`
hashed the prompt file's *path*, while its own docstring claimed it covered the
same inputs as the single-render fingerprint, which hashes the prompt *text*.
The stale clips were caught by frame review, not by the tool. Fixed in
`src/piecesai/h3/batch.py` — the fingerprint now digests prompt contents and
reference bytes — with regression tests for a rewritten prompt, an edited
anchor, and a moved-but-unchanged prompt file.
The general shape is the one already on the seed comparison: **a fingerprint
match is a claim about work, and a claim you did not check is a result you did
not get.** The seed version costs a wasted A/B. This version silently returns
content you already know is wrong.
agent-studio-mini/.agents/skills/h3-video-production/references/render-settings-evidence.md
+64
View File
@@ -0,0 +1,64 @@
# How the render settings were chosen
The rules live in `SKILL.md`. This file is the evidence behind them — read it
when you are about to argue with a default, not before every render.
The authority for the numbers is the comment block above
`U06_V4_SAMPLING_STEPS` in `src/piecesai/generators/minimax_h3.py`. When the
code default moves, that comment moves with it and this file is stale until it
is rewritten from there. It has been stale once already: three commits of
default changes landed after the first version was written, and it went on
telling readers that the shipped default had been "rejected on motion".
## The v4 step600 LoRA at 8 steps
Chosen 2026-08-16 by side-by-side viewing across four rounds.
Everything before that round used the **fl2v** LoRA — trained for first/last
frame work, while this deployment renders ref2va. Once that mismatch was fixed
the comparison stopped being a speed-versus-quality trade, and two candidates
were left: the ref2v-matched 4-step weight, and v4 step600 at 8 steps.
| round | owner picked |
|---|---|
| peeler, seed 661120 | ref2v + v4 |
| peeler, seed 314159 | no-LoRA + v4 |
| two-person exchange | ref2v + v4 |
| Malay piece to camera | ref2v + v4 |
v4 is the only one present in every round. The ref2v-matched weight scored
higher more often but dropped out when the seed changed, and a twelve-clip
delivery cannot depend on drawing a good seed. The steadier of two equals wins.
8 steps is v4's own contract: its documentation reports motion smear at 4 steps
under large fast motion, largely gone by 6-8, and no gain past 8. Measured on a
5090 at 1.03 MP:
| setting | seconds per clip |
|---|---|
| v4 step600 @ 8 steps (current default) | 119-122 |
| old fl2v default | 128 |
| undistilled @ 14 steps | 156 |
Strength is tuned for 1.0 by the LoRA's own documentation. The 0.75 that shipped
before sat below the documented range and under-corrected the velocity
prediction, which is exactly what smears on large motion.
## Why a sharpness metric was rejected
A sharpness proxy was measured here once and ranked the runs the wrong way. It
scored horizontal high-frequency energy, and over-sharp grain is precisely the
failure mode these LoRAs name in their own documentation — so the metric
rewarded the artefact.
Motion coherence needs eyes. Render the variants, show them, let the owner pick.
## Why an A/B needs a fresh seed
A render whose request fingerprint matches a completed run resumes from that
run's receipt and returns the old file in seconds. Correct for resuming a batch;
wrong for a comparison.
This cost two rounds on 2026-08-16. One set of four variants came back
pixel-identical. Another "finished" in 3-5 seconds. Both times the giveaway was
the clock, not the picture — the frames looked plausible either way.
agent-studio-mini/.agents/skills/h3-video-production/references/spoken-language-support.md
+101
View File
@@ -0,0 +1,101 @@
# Spoken language support
H3 renders its own audio. The words in `<d>…</d>` are *spoken by the model*,
not dubbed on afterwards, so the language of a line is a model capability
question — not a localisation question that post-production can fix.
## The eleven stable languages
MiniMax's own model card states:
> Stable support for 11 languages: Arabic, Chinese, English, French, German,
> Italian, Japanese, Korean, Portuguese, Russian, and Spanish.
> Additional languages are also supported to varying degrees.
Source: <https://huggingface.co/MiniMaxAI/MiniMax-H3>, checked 2026-08-20.
Use exactly these names as the tag inside `<d>`:
```
<d>[English] First batch of the morning.</d>
<d>[Chinese] 今天的第一炉。</d>
```
## What the limit covers, and what it does not
- **Covered:** every spoken or sung line the model performs — dialogue,
voiceover narration, lyrics. Anything inside `<d>…</d>`.
- **Not covered:** the prompt body itself. All six rewrite sections stay in
English regardless of the campaign language — that is `h3-prompt-writing`'s
rule and this gate does not change it.
- **Not covered:** on-screen text. Signs, lower thirds and product copy are a
*glyph rendering* problem, not a speech one. A language can be outside the
eleven and still render as visible text, and a language inside the eleven can
still render its glyphs badly. Judge on-screen copy from a still.
- **Not covered:** `overall_soundscape` and `non_diegetic_music`. Wordless
audio has no language.
"Supported to varying degrees" is the vendor's phrasing for the rest. It is
not a promise and it is not a refusal. Treat an out-of-list language as an
untested render, priced like one.
The failure modes to watch for on an out-of-list line are accent drift toward
the nearest stable language, substituted phonemes, lip movement that no longer
matches the words, and — occasionally — the line coming back in English. None
of these are measured on this repo's fleet; they are what the probe below is
for.
## Subtitles are outside this gate, and outside the toolchain
Delivery subtitles carry no model risk in any language — they are not spoken.
But no subtitle burn-in tool ships here: `post` has `verify`, `assemble`,
`frames` and `derive` and nothing else. Burned-in captions are hand-written
ffmpeg work. Price them as work, never as an existing step.
## The gate
Fires the moment a spoken language is chosen — at the brief, not at render
time. The official creative Skills ask for narration language early
(`brand-promo-video-generator` asks in Step 1, alongside duration and aspect
ratio); that answer is the trigger.
`uv run piecesai h3 preflight --language <name> ...` answers this without
reading anything: the language check is one of its findings, and it warns
rather than fails, which is exactly the gate described here.
If the language is one of the eleven, say nothing and carry on.
If it is not, this is a **soft** gate. Do not refuse, do not silently swap the
language, and do not quietly drop the voiceover. Tell the user plainly what
the constraint is and let them pick:
1. **Keep the language, probe first.** Render the single densest dialogue
shot at low `--megapixels` and listen to it before committing the batch.
Cheapest way to turn the question into an answer.
2. **Speak a stable language, subtitle the target one.** Voice the line in
the nearest stable language and carry the campaign language as subtitles.
Usually the right answer for a promo, where the read is short and the copy
carries the message. Quote the subtitles as work — see above.
3. **Drop the spoken layer.** Music, soundscape and on-screen copy only.
Costs nothing in render risk and often suits a 15-second promo better than
a rushed voiceover.
Record which one the user chose next to the prompt, the same way the prompt
itself is kept. When a later shot comes back with wrong-sounding speech, that
line is the difference between a known trade-off and a mystery.
## The probe
```bash
uv run piecesai h3 render \
--project-id <proj_id> --title "lang probe" --skill <calling-skill-name> \
--prompt-file <path> --reference <anchor.png> \
--h3-mode ref2va --duration 5.0 --megapixels 0.3
```
Pick the shot with the most words per second, not the first shot. Listen for
the words themselves, and check the mouth against them — a line can be
intelligible and still be lip-synced to a different language's phonemes.
A failed probe is not a fleet problem. Do not load `h3-fleet-ops` for it.
Go back to the user with what you heard and pick option 2 or 3.
agent-studio-mini/.agents/skills/h3-video-production/scripts/finalize_audio.py
+234
View File
@@ -0,0 +1,234 @@
#!/usr/bin/env python3
"""Apply the PiecesAI delivery-audio standard without re-encoding video."""
from __future__ import annotations
import argparse
import json
import math
import os
import re
import shutil
import subprocess
import sys
import uuid
from pathlib import Path
VOLUME_RE = re.compile(r"(?P<name>mean_volume|max_volume): (?P<value>-?inf|-?\d+(?:\.\d+)?) dB")
def _binary(name: str) -> str:
path = shutil.which(name)
if path is None:
raise RuntimeError(f"required binary not found: {name}")
return path
def _run(command: list[str], *, capture: bool = False) -> subprocess.CompletedProcess[str]:
return subprocess.run(
command,
check=True,
text=True,
stdout=subprocess.PIPE if capture else None,
stderr=subprocess.PIPE if capture else None,
)
def _probe(ffprobe: str, media_path: Path) -> dict:
result = _run(
[
ffprobe,
"-v",
"error",
"-show_entries",
"stream=index,codec_type,codec_name,width,height,r_frame_rate,sample_rate,channels",
"-show_entries",
"format=duration,size",
"-of",
"json",
str(media_path),
],
capture=True,
)
return json.loads(result.stdout)
def _volume_stats(ffmpeg: str, media_path: Path) -> dict[str, float]:
result = _run(
[
ffmpeg,
"-hide_banner",
"-i",
str(media_path),
"-af",
"volumedetect",
"-f",
"null",
"-",
],
capture=True,
)
stats: dict[str, float] = {}
for match in VOLUME_RE.finditer(result.stderr):
stats[match.group("name")] = float(match.group("value"))
if set(stats) != {"mean_volume", "max_volume"}:
raise RuntimeError(f"could not read volume statistics from {media_path}")
return stats
def _audio_filter(*, gain: float, peak_limit: float, denoise: str) -> str:
filters: list[str] = []
if denoise == "afftdn":
filters.append("afftdn=nr=10:nf=-45:tn=1")
elif denoise == "anlmdn":
filters.append("anlmdn=s=1e-5:p=0.002:r=0.006:m=15")
filters.extend(
(
f"volume={gain:.6f}",
f"alimiter=limit={peak_limit:.6f}:attack=5:release=50:level=false",
)
)
return ",".join(filters)
def finalize_audio(
input_path: Path,
output_path: Path,
*,
gain: float = 2.0,
peak_limit: float = 0.89,
denoise: str = "none",
audio_bitrate: str = "256k",
sample_rate: int = 48_000,
force: bool = False,
) -> dict:
input_path = input_path.resolve()
output_path = output_path.resolve()
if input_path == output_path:
raise ValueError("input and output paths must differ")
if not input_path.is_file():
raise FileNotFoundError(input_path)
if output_path.exists() and not force:
raise FileExistsError(f"output exists; pass --force to replace it: {output_path}")
if gain <= 0:
raise ValueError("gain must be greater than zero")
if not 0 < peak_limit <= 1:
raise ValueError("peak limit must be within (0, 1]")
ffmpeg = _binary("ffmpeg")
ffprobe = _binary("ffprobe")
source_probe = _probe(ffprobe, input_path)
stream_types = {stream.get("codec_type") for stream in source_probe.get("streams", [])}
if not {"video", "audio"}.issubset(stream_types):
raise ValueError("input must contain both video and audio streams")
source_volume = _volume_stats(ffmpeg, input_path)
output_path.parent.mkdir(parents=True, exist_ok=True)
temporary = output_path.with_name(
f".{output_path.stem}.{uuid.uuid4().hex}.tmp{output_path.suffix or '.mp4'}"
)
try:
_run(
[
ffmpeg,
"-hide_banner",
"-loglevel",
"error",
"-y",
"-i",
str(input_path),
"-map",
"0:v:0",
"-map",
"0:a:0",
"-map_metadata",
"0",
"-c:v",
"copy",
"-af",
_audio_filter(gain=gain, peak_limit=peak_limit, denoise=denoise),
"-c:a",
"aac",
"-b:a",
audio_bitrate,
"-ar",
str(sample_rate),
"-movflags",
"+faststart",
str(temporary),
]
)
output_probe = _probe(ffprobe, temporary)
output_volume = _volume_stats(ffmpeg, temporary)
# AAC can overshoot the linear limiter slightly. The standard keeps at
# least 0.5 dB of encoded-sample headroom.
if output_volume["max_volume"] > -0.5:
raise RuntimeError(
f"unsafe output peak: {output_volume['max_volume']:.1f} dB; "
"lower --peak-limit"
)
os.replace(temporary, output_path)
finally:
temporary.unlink(missing_ok=True)
return {
"input": str(input_path),
"output": str(output_path),
"video_mode": "stream_copy",
"gain": gain,
"gain_db": 20 * math.log10(gain),
"peak_limit": peak_limit,
"denoise": denoise,
"audio_codec": "aac",
"audio_bitrate": audio_bitrate,
"sample_rate": sample_rate,
"source_volume": source_volume,
"output_volume": output_volume,
"source_probe": source_probe,
"output_probe": output_probe,
}
def _parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
description="Boost delivery audio while stream-copying the video track."
)
parser.add_argument("input", type=Path)
parser.add_argument("output", type=Path)
parser.add_argument("--gain", type=float, default=2.0)
parser.add_argument("--peak-limit", type=float, default=0.89)
parser.add_argument("--denoise", choices=("none", "afftdn", "anlmdn"), default="none")
parser.add_argument("--audio-bitrate", default="256k")
parser.add_argument("--sample-rate", type=int, default=48_000)
parser.add_argument("--receipt", type=Path)
parser.add_argument("--force", action="store_true")
return parser
def main() -> int:
args = _parser().parse_args()
try:
receipt = finalize_audio(
args.input,
args.output,
gain=args.gain,
peak_limit=args.peak_limit,
denoise=args.denoise,
audio_bitrate=args.audio_bitrate,
sample_rate=args.sample_rate,
force=args.force,
)
except (FileNotFoundError, FileExistsError, RuntimeError, ValueError) as exc:
print(f"error: {exc}", file=sys.stderr)
return 2
encoded = json.dumps(receipt, ensure_ascii=False, indent=2)
if args.receipt:
args.receipt.parent.mkdir(parents=True, exist_ok=True)
args.receipt.write_text(encoded + "\n", encoding="utf-8")
print(encoded)
return 0
if __name__ == "__main__":
raise SystemExit(main())