HTTP API for agents

This website documents selected supported workflows and fields, not the complete internal API.

Use original or stylised characters, or your self-depiction with documented consent. Any depiction of another identifiable person requires their documented consent. Follow the Acceptable Use Policy for all reference images, training data, audio and outputs.

External agents (Claude Code, Codex, OSS, or anything that speaks HTTP) drive Phosphene through this API. The in-panel chat was removed 2026-05-15 — there’s no agent embedded inside the panel anymore. Everything that the chat used to do can be done by an external agent through these endpoints.

Ready-made tools on top of this API: the phosphene command and an MCP server for Claude Code, Codex or any MCP client, in agents/. See Agent tools.

Server

• Base URL: http://127.0.0.1:8198 on a normal install. 8199 is the DEV profile’s port, and every example on this page uses it because that is the panel these were written against. The rule is one line in the source (mlx_ltx_panel.py:275): DEFAULT_PORT = 8199 if PROFILE == "dev" else 8198, overridable with LTX_PORT. If a request here returns nothing, you are almost certainly talking to a port that has no panel on it — try the other one.
• Process: mlx_ltx_panel.py (in the repo root)
• Wire format: form-encoded POST bodies; JSON exceptions are noted per-endpoint. Responses are HTTP/1.0 — BaseHTTPRequestHandler’s default, never overridden — so the connection closes after every request and there is no keep-alive to pipeline against.
• Auth: none — bound to loopback only.

Conventions

• Every JSON response is Content-Type: application/json.
• Errors return {"error": "<message>"} plus a non-2xx HTTP status.
• 451 licence_required (4.19.8). A request that would download or run a model whose licence asks for an explicit “I accept” - Hailuo H3 (MiniMax H3 Community License) and the non-commercial models Ideogram 4, YuE2, FLUX.1 Krea [dev] and Bria FIBO - answers HTTP 451 with {"ok": false, "code": "licence_required", "licence": "<key>", "error": "...", "url": "<licence>"} until that licence is accepted once on this Mac. Applies to /queue/add, /queue/retry, /queue/batch, /image/generate, /h3/turbo/install, /h3/tristep/install, /music/install, /music/cover/install and /music/lora/fetch; a job that arrives any other way fails with the same sentence. Read the terms with GET /licences and accept with POST /licences/accept (below), or once in the panel, which shows the screen by itself.
• The /agent/* namespace is reserved-and-removed: any path under /agent/ except /agent/image/config returns 410 Gone with {"error": "agentic flows removed; see docs/API.md"}. New endpoints should not be added under /agent/.

---

Generation — submit and manage jobs

POST /queue/add — enqueue a video render

Add a job to the panel’s queue. Returns immediately; the helper renders it asynchronously.

Form fields (all required unless marked optional):

Hailuo H3 (engine=h3), 4.19 fields. H3 renders t2v, i2v, keyframe, extend, a2v and v2a; each mode past i2v needs a flag on the INSTALLED runner (/status.h3.keyframes / .extend / .audio_drive / .video_to_audio), and a job for one the runner lacks renders on LTX with a sentence (Add sound has no LTX lane and is refused with the update sentence instead).

LoRA payload: the loras field is a JSON-encoded array of {path, strength} objects. Stack as many as needed: On Hailuo H3, stacking needs the engine’s repeatable --lora (/status.h3.loras.max_stack > 1); an older pack takes one adapter and the h3_lora_slot field decides whether it goes to Turbo or to the LoRA.

[
  {"path": "<panel-root>/mlx_models/loras/explorertrn_v2.safetensors", "strength": 1.0}
]

Return: {"ok": true, "id": "j-<...>"} — the job ID for polling.

Example:

curl -s -X POST http://127.0.0.1:8199/queue/add \
  --data-urlencode "mode=t2v" \
  --data-urlencode "prompt=Cinematic close-up of bizarrotrn man in a wood-paneled study, golden hour, photorealistic." \
  --data-urlencode "width=1024" --data-urlencode "height=576" \
  --data-urlencode "frames=169" --data-urlencode "steps=8" \
  --data-urlencode "quality=high" --data-urlencode "temporal_mode=native" \
  --data-urlencode "stage1_steps=10" --data-urlencode "stage2_steps=3" \
  --data-urlencode "teacache_thresh=2.0" --data-urlencode "cfg_scale=3.0" \
  --data-urlencode "bongmath_max_iter=100" --data-urlencode "accel=off" \
  --data-urlencode "enhance=false" --data-urlencode "upscale=fit_720p" \
  --data-urlencode "upscale_method=lanczos" --data-urlencode "seed=-1" \

POST /run — alias for /queue/add (identical behavior).

POST /queue/retry

Re-queue a job from history with the same params. params.open_when_done is forced to false (retries are background).

Returns {"ok": true, "id": "<new-id>", "source_id": "<old-id>"}.

POST /queue/remove

Remove a queued job. (Running job can’t be removed — use /stop.)

Returns {"ok": true}.

POST /queue/clear

Empty the queue. Returns {"cleared": <count>, "ids": [...], "undo_token": "<hex>"|null}. The token undoes THIS clear once, for 60 s (POST /queue/restore with {"token": "<hex>"} puts back exactly the removed jobs, original ids included, ahead of anything queued since). A later clear supersedes it; an expired or spent token answers 410.

POST /queue/pause, POST /queue/resume

Pause/resume the queue dispatcher (does not affect a job already running).

POST /stop

Request the running job to stop. Returns {"ok": true}. Optional ?id=<job id> (what the panel sends): stop only if that job is still the one running — otherwise 409 {"stale": true} and nothing is stopped. /stop/after_part takes the same id.

POST /queue/batch

Submit multiple prompts as one batch.

---

Status — poll job state

One Shot — its own door: POST /oneshot (2026-09-08)

One Shot is a workflow tab and a first-class API. A script posts the same document the tab posts — one JSON object — and gets the queued job back:

curl -s -X POST http://127.0.0.1:8198/oneshot -H 'Content-Type: application/json' -d '{
  "prompt": "A red vintage tram rolls through a rain-soaked Lisbon street at night; a woman with a yellow umbrella waits at the stop. The bell rings twice; rain on the roof.",
  "seconds": 60,
  "engine": "ltx",
  "quality": "balanced",
  "camera": "a slow, steady push in along the street",
  "beats": ["the tram comes round the corner", "", "the woman steps to the kerb"],
  "handoff": "last"
}'
# → {"ok": true, "id": "j-…", "plan": {"seconds": 60, "parts": 6, "beats": 12, "engine": "ltx", "handoff": "last", "minutes": 28.2, "eta": "~28 min"}}

GET /oneshot/options — what this Mac can do: seconds, engines.{ltx,h3} (available, part_seconds, qualities), character_qualities, characters (id, name, trigger), planner. GET /oneshot/estimate?engine=&quality=&seconds= — parts, part_seconds, beats, minutes, eta. POST /oneshot/plan {prompt, seconds, engine} — the Storyboard planner writes one beat per five seconds from the prompt (blocks 20–40 s): {ok, beats:[…]}. GET /oneshot/status — current (id, label, part, parts, lipsync per part, drift, handoff), queued, recent, and the take’s own log lines.

Under the hood every One Shot is the take contract below — POST /oneshot maps the document onto make_job’s form, so the tab, the API and /queue/add render exactly the same job.

One take — take_seconds + beats on POST /queue/add

A clip longer than a single pass, on either engine. take_seconds is one of 30 / 45 / 60 / 90 / 120; beats is a JSON list (or newline text) with one prompt per five seconds — a blank beat holds the previous moment, extras are dropped. No beats at all (the field absent, empty, or only blank lines) means the prompt is every beat: the shot is the prompt sustained for the whole take. Before 4.12.1 that case rendered every window as the hold sentence — a take about nothing. The take overrules the engine’s own length field. On BOTH engines it renders as PARTS chained by last-frame handoff — run_take_job_inner runs one ordinary render per part, each starting from the last frame of the one before (i2v, anchored), hides the parts from the gallery and joins them into <name>_take<seconds>s.mp4:

Two optional fields, both default on:

params.take on the job: {seconds, beats, parts, frames, part_frames, beats_per_part, engine, beat_prompts, light_lock, retake} — frames is the whole take (seconds·24+1 on LTX), part_frames one full part. The output’s sidecar carries take.{seconds, beats, parts, engine, beats_per_part, part_frames, light_lock, retake} with parts the hidden part files, and image only when the take opened on the user’s own anchor. Load Params restores a take as a take.

GET /take/estimate

?engine=h3|ltx&quality=<engine quality key>&seconds=<take_seconds> → { ok, seconds, beats, parts, beats_per_part, part_frames, engine, frames, minutes, eta, needs_q8 }. parts is the part count on both engines. Minutes are parts × the single-clip price of one part from the same cost model as every other estimate (H3: the measured 15 s cell; LTX: the quality’s 10s cell of LTX_TIERS, the one-beat tail at its 5s cell); null where this Mac has no number. needs_q8 is always false now — an LTX take renders on whatever the chosen quality renders on.

GET /status

Every outputs[] row carries q: the output’s searchable words, lower-cased, built from its sidecar in the listing pass that already reads it — prompt, label, mode, quality, engine, character, seed, WxH, Nf, LoRA file stems, the model directory name, the temporal mode and windows. The gallery’s search box is a substring test over name + q; every typed word must match, and typing pulls the older outputs in so the search is over everything.

Every video row also carries silent: true when the clip has no sound (no audio track in its mp4 header, or a track that never rises above −80 dB), false with sound, null while not known yet. The header read is inline; the level check runs once per file on a background thread and is cached per (mtime, size) in state/output_silence.json, so a poll never decodes anything. The gallery card and the player offer Add sound on a silent clip when /status.h3.video_to_audio is true.

Returns the full panel state.

{
  "running": false,
  "paused": false,
  "current": null,
  "queue": [],
  "history": [ { "id": "j-...", "status": "done", "params": {...}, "elapsed_sec": 426.1, "output_path": "...", "raw_path": "...", "error": null }, ... ],
  "log": [ "[HH:MM:SS] helper line...", ... ]
}

Field semantics:

• deferred_downloads (4.20) — { active, state, key, message, pending: [{key, name, size_gb, left_gb}], left_gb, finished }. The packs a fresh install queued for after Start (state/install_deferred.json, written by install.js; required_files.json -> install.after_start). state is idle, starting, downloading, waiting (another download holds the slot), waiting_disk (the panel keeps install.background_floor_gb free; message says how much to free), retry_wait, failed or done. Empty and idle on any install install.js never queued — an updated install downloads nothing. download carries background: true while the fetcher holds the slot.
• current — the job actively running (null when idle).
• gpu (4.19.10) — { busy, holder, since, outside_queue }: the panel’s one GPU gate. holder is "queue" for the job in current, or what holds the GPU outside the queue: "image" (an Image Studio render), "character_sheet", "enhance" (Prompt Enhance), "lyrics", "update" (the post-update script). A client that must not interrupt work (stopping or updating the panel) waits for current to be null, the queue to be paused or empty, AND gpu.busy to be false.
• app_models (Mac app only) — the app’s own model download: { state, packs, active, locked, done_bytes, total_bytes }. locked (4.19.10) is true while the app holds its download lock (<Phosphene home>/.locks/models.lock); the panel then offers no Download for those packs and refuses one with 409 app_downloading. A second scripts/fetch_pack_release.py on a pack folder that is already being written stops untouched with exit code 75.
• queue — jobs waiting, FIFO order.
• history — completed/failed jobs. Each entry has status, elapsed_sec, output_path, raw_path (native pre-upscale), error.
• log — rolling buffer of helper stdout lines (last ~50 jobs).

Polling loop: GET /status every 15–30s. A job is terminal when its status is one of done, failed, cancelled, stopped, error. Until then it’s queued or running.

stopped means the user stopped the render on purpose (see POST /stop?mode=early). It is not a failure: nothing was saved because the clip was never finished, but nothing went wrong. stopped_reason carries the runner’s own sentence.

current.progress

Present while a job runs.

{
  "pct": 41, "phase": "denoise", "phase_label": "Denoising · step 5 / 10",
  "elapsed_sec": 62, "remaining_sec": 84, "eta_sec": 146,
  "preview": {
    "url": "/image?path=…/live/preview_latest.png&t=1786…",  // cache-busted server-side
                          // /image, NOT /file: the preview lives under STATE_DIR and
                          // /file serves OUTPUT only. Writing /file here is what shipped
                          // a correct preview to a broken-image glyph in every client.
    "estimate": 4, "total": 12,
    "meaningful": true,   // THE gate — the server decides, the client renders
    "abortable": true,    // meaningful && the runner published an abort sentinel
    "saves_sec": 182      // remaining_sec at the moment it became abortable
  },
  "preview_url": "/image?path=…"  // top-level alias of preview.url, same /image
                                  // form, present only once meaningful
}

preview is absent unless the live preview is on (Settings → Memory / speed) and the tiny decoder is installed. meaningful is computed server-side and must not be re-derived: the rule differs per pipeline — estimate 6 of 8 on the distilled schedule, whose first estimates are still essentially noise; estimate 2 on the two-stage HQ path, which evaluates the denoiser twice per stage-2 step. A client counting estimates would have to know which schedule is running.

POST /stop?mode=early

Asks the running render to stop at the next forward boundary by touching an ABORT sentinel. The runner exits 75 — a distinct code meaning “the user stopped this”, not a crash — and the job resolves status: "stopped" with no output file. Returns 404 when nothing is rendering and 409 when the render has no live preview to stop through (use the hard POST /stop, which kills the helper).

Encoding caveat: the log buffer can contain literal control characters from prompts with embedded newlines. Strip them before JSON-parsing:

import re, json
raw = re.sub(r'[\x00-\x08\x0b\x0c\x0e-\x1f]', ' ', raw)
data = json.loads(raw, strict=False)

---

LoRAs

GET /loras

Returns {"user": [...], "curated": [...], "loras_dir": "...", ...}. Each entry has id, name, path, filename, size_bytes, recommended_strength, kind (train_character for LoRAs trained via the panel; null otherwise), adapter_verdict (see below), and CivitAI metadata if known.

adapter_verdict — whether this file can do anything

A LoRA can train to completion, write a valid safetensors, load without a warning, and change nothing: the deltas it learned are numerically too small to move the model. Every structural gate passes, because every structural gate asks whether the KEYS land and none asks whether there is anything in them (#61, #62).

adapter_verdict (_adapter_verdict) is read out of the sidecar’s adapter_strength block and is one of:

Anything else is whatever the trainer called it, passed through verbatim.

Callers should surface weak before a render is spent finding out, and should not decorate unknown.

GET /train/list carries the same verdict plus adapter_advice — one sentence naming what to do about it, empty when the verdict is ok or unknown. It is hardware-aware: on a sub-64 GB Mac it does not say “train again on High”, because that machine cannot reach the graded recipe (see the profile section below). The panel’s Train tab renders it as a banner above the trained-character chips and in the finished job’s history row; a verdict a user cannot act on is a measurement, not an answer (#62).

GET /hf/loras

Hugging Face as a browsable source beside CivitAI. ?lane=h3|ltx&q=<a name, author:someone, or owner/repo>&refresh=1. Phosphene names no org: the query is a search (empty = the lane’s default search term), an author listing, or one repo; results are kept by lane from the repo name and tags, and each repo’s largest .safetensors plus its example clip become one item in the same shape the CivitAI grid renders (name, creator, likes, size_kb, preview_url / preview_type, download_url, filename, base_model, hf_url, source: "huggingface"). Cached ten minutes per lane and query.

POST /hf/loras/download

Form repo, filename, meta (the item JSON). Downloads through huggingface_hub (the saved HF token when present) into the lane’s LoRA directory, runs the H3 layout probe, and writes the same sidecar a CivitAI install writes plus source, hf_repo, hf_url. Returns { ok, skipped, name, path, sidecar_path, lane, layout, converted }.

GET /civitai/download/state · POST /civitai/download/cancel (4.18.0)

POST /civitai/download takes an optional form field token (8-64 characters, A-Z a-z 0-9 _ -) chosen by the caller. While that download runs, GET /civitai/download/state?token=… returns { ok, state: "running"|"cancelled", bytes, total, name } (404 once it has finished), and POST /civitai/download/cancel with token stops it at the next 256 KB chunk: the partial file is removed, nothing is registered, and the download’s own response is { ok: false, cancelled: true, error }. Without a token a download runs exactly as before.

GET /loras/updates

Asks CivitAI whether any installed LoRA with a civitai_id in its sidecar has a newer version (modelVersions[0].id ≠ the sidecar’s civitai_version_id). One request per such LoRA; nothing is downloaded.

{ "ok": true, "checked": 3,
  "items": [ { "path": "...", "name": "...", "current_version_id": 1, "latest_version_id": 2,
               "latest_version_name": "v2", "published_at": "...", "base_model": "LTXV 2.3",
               "download_url": "...",            // what /civitai/download takes
               "meta": { ... } } ],             // the sidecar the download will write
  "errors": [ { "path": "...", "error": "..." } ] }

Installing the update is the ordinary POST /civitai/download with that download_url + meta; the new file lands beside the old one, which stays until deleted.

POST /loras/guide

Form: path (an installed LoRA from GET /loras). The planner model writes a one-paragraph guide — what the LoRA does, how to prompt it, a strength to start from, what it fights with — from the sidecar’s own name / description / trigger words, and saves it in the sidecar under guide; GET /loras rows then carry guide. Refused with 409 while a render runs (the planner is a 12B model). Returns { ok, guide }.

POST /loras/refresh

Re-scan mlx_models/loras/. Returns the same payload as /loras.

POST /loras/delete

Deletes the file. Returns {"ok": true}.

POST /h3/loras/import

multipart/form-data. Installs one Hailuo H3 adapter you already have into H3’s own LoRA library (mlx_models/hailuo-h3/loras/, or wherever the installed pack keeps it). Deliberately NOT /upload: reference media can be any image or audio, whereas an H3 adapter has to pass the runner’s layout contract before it may appear in the picker.

The body is streamed to a hidden staging directory — not buffered — so a multi-gigabyte adapter costs about 1 MiB of resident memory. Content-Length is required and capped at H3_LORA_UPLOAD_MAX_BYTES (4 GiB).

What happens to the file, in order: staged under .import.<pid>.<tid>/<name> → _h3_lora_prepare (a ComfyUI repack has its diffusion_model. namespace stripped in place, a header-only rewrite) → the import-only payload and target-module checks → os.replace into the library → sidecar JSON. A refusal at any step leaves nothing behind.

Returns:

{
  "ok": true,
  "path": "/abs/path/in/the/h3/library.safetensors",
  "filename": "my_lora.safetensors",
  "name": "my lora",
  "layout": "bare" | "comfyui",
  "converted": false,
  "pairs": 208,
  "recommended_strength": 1.0,
  "scale_source": "folded" | "per_module" | "uniform" | "advisory" | "none",
  "scale_evidence": "…"
}

recommended_strength is written into the sidecar and picked up by the picker’s per-LoRA strength control. The H3 loader applies B @ A and no alpha, so an adapter whose PEFT alpha/rank is not already folded into lora_B needs that number here — this is reported, never used as grounds for refusal.

Errors: 411 when Content-Length is absent or zero, 413 when the body exceeds the cap, 400 for a bad file (wrong extension, empty, duplicate name, unusable layout, malformed payload, wrong model), 500 otherwise. Every message names the file the user chose.

---

Training

POST /train/upload

multipart/form-data upload. Adds one image to a training dataset — this route is one-file-per-request, not a batch. Send the images in a loop, echoing the returned train_job_id back on every call after the first so the whole set lands in one directory.

The request body is capped at TRAIN_MAX_BYTES_PER_IMAGE (32 MB). Because that cap is on the whole request and the route takes one file, a multi-file body is refused even when every individual image is small.

Returns the dataset state (train_job_id, image count, etc.).

POST /train/start

Kick off a character LoRA training. The panel shells out to lora_lab.train_character and streams progress events back into STATE['log'].

Advanced overrides (all optional, fall back to preset defaults):

Returns {"ok": true, "id": "j-<...>"}. Track progress via /status (the training job appears in current then history).

What the presets actually are

Steps are derived, not fixed: steps = epochs × image_count, floored at 1 and capped by the preset’s max_steps (_preset_steps_for, mlx_ltx_panel.py:1629). The cap is part of the server contract — a hand-written API call must not be able to enqueue a multi-thousand-step run on a memory-constrained Mac.

On a Mac with 64 GB or more (TRAIN_PRESETS, mlx_ltx_panel.py:1558):

Measured with lora_compat.py: rank-32 adapters that carry an identity sit at 5.4e-04 to 1.6e-03 delta_rms; two independent users’ rank-8 quick runs measured 1.54e-04 and 1.98e-04, both at or under the WEAK_DELTA_RMS floor of 2.0e-04 (#62).

So high at 50 images is 5000 steps, at 20 images 2000, at 300 images 20000 (capped). Only the rank-32 recipe has ever been put in front of a human eye and called good.

⚠️ Under 64 GB, every preset is quietly a different preset

_select_train_profile (mlx_ltx_panel.py:1650) runs once at import and, when total RAM is 0 < ram < 64 GB, it mutates TRAIN_PRESETS and TRAIN_STYLE_PRESETS in place. The names do not change. The numbers all do:

make_job mirrors the same clamps server-side, so rank / steps / resolution passed as advanced overrides are capped too (max_rank 8, max_resolution 448, max_steps 500) — you cannot ask for the graded recipe on this hardware and get it.

This is a deliberate trade and the reason for it is real: full 512 px LTX LoRA training materialises backward activations for the dev transformer and a 48 GB Mac falls into swap thrash and never finishes. But be clear about what it means: on a sub-64 GB Mac there is no preset that trains the recipe anybody has graded. rank 8 at ≤500 steps is exactly the regime adapter_verdict was built to catch, so expect weak more often here, and read it as “this hardware, this recipe” rather than “this dataset”.

GET /status exposes the active profile as train_profile (key: compact_training | full_training, plus label, ram_gb, max_rank, max_resolution, max_steps, note) and the effective tables themselves as train_presets / train_style_presets (mlx_ltx_panel.py:21842-21844). Read those before trusting a preset name — they are the post-mutation values, so they are the only place the two tables above are told apart at runtime.

The adapter_strength trainer event

Not an analytics event — nothing about it leaves the machine (see docs/ANALYTICS.md for the complete list of what does). It is a progress event on the lora_lab subprocess stream, emitted once after the trainer measures the file it just wrote (mlx_ltx_panel.py:18005). The panel logs the median delta RMS, the max, and how many modules are carrying, then finishes the job in a WARNING state rather than a bare done when the verdict is not ok. The verdict is written into the LoRA’s sidecar as adapter_strength: {"verdict": …} and read back out as adapter_verdict on /loras and /characters.

POST /train/install

After training succeeds, the LoRA is automatically copied into mlx_models/loras/. This endpoint is for explicit re-install / metadata refresh.

POST /train/delete, POST /train/remove-image

Delete training datasets / individual training images.

---

Characters

Discover and drive trained character LoRAs as bundles.

GET /characters

Returns {"characters": [...]}. Each entry describes one character discovered in mlx_models/loras/:

Discovery rule: <trigger>_v2.safetensors is required; everything else is optional.

GET /characters/<id>/preview

Serves the sample training image for the character (PNG/JPEG).

GET /characters/<id>/sheet

Serves the generated character sheet PNG. ?w=<px> returns a cached thumbnail (JPEG) through the same resize lane as /image?w=. 404 until a sheet has been generated.

POST /characters/<id>/sheet/generate

Renders a multi-view turnaround sheet from the character’s reference image (bundle avatar first, training-sample fallback) and composites it into mlx_models/characters/<id>/sheet.png + a sheet.json sidecar (phosphene/character_sheet@1). Synchronous, like /image/generate — the caller blocks for the whole render; 429 when the GPU is already held by a render/training/image job (never queued).

Content-Type: application/json. Every field optional (an empty body renders the default 3-view sheet):

{
  "engine_override": "qwen_edit_inline",
  "views": ["front", "profile_left", "three_quarter"],
  "wardrobe": "a red flight jacket",
  "seed": -1
}

Only ref-honoring engines are accepted (hidream_*_inline, qwen_edit_*_inline, mock_inline) — a text-only engine would render a stranger. seed >= 0 gives view i seed + i so a whole sheet reproduces from one number. Every view prompt pins wardrobe to the reference (“wearing exactly the same clothes as in the reference image”); wardrobe re-states the outfit in words on top of that. Views after the first also chain the first rendered view as a second reference: measured on a dim reference image, solo side-angle renders re-imagined hair color across seeds and phrasings, and a clean frontal from the same run re-anchors the attributes the raw reference under-specifies. Per-view framing can still drift (a subject off-center in one view) — regenerate or vary the seed. On success, bundle.json’s preview is pointed at sheet.png only if it was null — a curated preview is never clobbered.

POST /characters/<id>/generate

Assembles the locked production recipe + queues a T2V render. The endpoint accepts prompt_body, framing, duration, quality, and an optional integer seed (-1 means random, matching /queue/add). For exact replay/debug calls, callers may pass full_prompt plus HQ override fields such as stage1_steps, stage2_steps, teacache_thresh, cfg_scale, bongmath_max_iter, video_skip_step, and audio_skip_step.

---

Image generation (separate from video)

POST /image/generate

Generates still images via the bundled image engine (mflux, BFL, mock backend). Pluggable per call.

Content-Type: application/json. Payload:

{
  "prompt": "...",
  "n": 4,
  "engine_override": "mflux",
  "aspect": "1:1",
  "refs": ["/path/to/ref.png"],
  "loras": [{"path": "...", "strength": 1.0}]
}

See image_engine.py for backend-specific options.

GET /agent/image/config, POST /agent/image/config

Kept under the legacy /agent/ path for backward compatibility with the Image Studio frontend. Read/write the image engine config (default backend, mflux family, BFL key, etc.). Despite the path, this is the image-engine config, NOT the removed chat agent.

---

Storyboard — plan a film, then shoot it

A board is a film: a concept, a cast, locations, and a numbered shot list. Every render it starts goes through the same make_job → queue contract as /queue/add; none of these routes has a private execution path. Boards live under state/storyboards/<board_id>/, with storyboard.json written atomically — renders run for hours and a torn board would lose the whole run.

Read

Brief switches (on plan)

Each field is patched only when sent; an absent field keeps the board’s value.

POST /storyboard/restill — form id, n, optional pass: forgets shot n’s still and the clip rendered from it, turns anchor_stills on if it was off, and renders that shot alone (the still first, then the clip from it). Same refusals as render (planning, already rendering).

Write — POST /storyboard/<action>

The Director. plan accepts soundtrack (an audio file on this Mac) and bars_per_shot (1/2/4, default 2). With a track the plan is a MUSIC VIDEO: beat_map() — the same fit the Editor’s Prepare runs — turns the downbeats into slots of bars_per_shot bars, the shot count is the slot count (the shots chips stand down; capped at STORYBOARD_MAX_SHOTS with a note), the planner’s brief carries the bpm, the slot length, an arc instruction and two laws (begin every description with the movement; no dialogue, the track replaces every clip’s sound), and each planned shot’s duration_s is its slot plus a one-second handle, with slot: {start, end} on the shot. The song map decides how often. song_map() (storyboard_edit) puts SECTIONS on the same bar grid — {start, end, label, energy, brightness} with labels intro / verse / chorus / bridge / outro from position, relative loudness and brightness — and director_pacing() turns bars_per_shot into a per-section stride: the chorus cuts twice as often, the intro and outro half as often. Each slot carries its section; the brief tells the planner which shots fall in which section and how hot it is; a track the map cannot read is cut at the base stride and the log says so. The board records soundtrack: {path, bpm, bars_per_shot, total_sec, count, slots, sections}, the Editor’s first auto-cut uses that track when Prepare has not cached another — so the film opens already cut on the beat — and GET /storyboard/edit returns sections so the ruler can paint the arc. A bad path is a 400 before the planner runs; the planner stage grid is reported while the beat is read.

Auto. plan accepts auto=1: when the plan lands the draft render of every shot starts on its own (_sb_auto_after_plan, once the planner has given the memory back), and when the render thread finishes with every shot rendered the film is cut (_sbe_auto_edit — on the beat when there is a track) and assembled by the same assembler Render uses (_sb_auto_film); the board records auto_film. A shot that did not render leaves the film waiting and the log says how many. Stop still stops the render; the cut and the film are the ordinary ones and can be re-cut afterwards.

Form-encoded unless noted. id is the board id throughout.

Board schema — the geography fields

A shot is composed from its location. Since 2026-08-18 a location can also be faced from more than one direction, which is what makes a reverse angle a reverse angle instead of a second location.

"locations": [
  {"id": "carwash", "name": "The car wash", "description": "…",
   "views": [                                  // ABSENT, never [], when there are none
     {"id": "establishing", "name": "…",
      "light": "camera left",                  // flips with the camera
      "description": "…"},                     // self-contained: readable without the floor plan
     {"id": "reverse", "name": "…",
      "light": "camera right",
      "description": "… no car in frame …"}    // says what it does NOT hold, in those words
   ]}
],
"shots": [
  {"n": 1, "location_id": "carwash",
   "view": "reverse",        // must be a view id OF that location
   "eyeline": "right",       // "left" | "right" | "lens"
   …}
]

What the server does with them. shot_scene_text injects the VIEW’s description when a shot names one and falls back to the location’s otherwise — so a board with no views, or a shot that names no view, renders exactly as it always did. eyeline_clause composes "his eyes fixed past the right edge of frame" at render time; lens deliberately produces nothing (telling these models to look down the barrel buys a stiffer performance than silence), and it exists so a shot can say it holds the lens, which is what lets the 180-degree check tell “faces the camera” apart from “nobody wrote one”.

The 180-degree rule is enforced, not suggested. After planning, _enforce_eyelines walks the shots in screen order and flips the second of any two adjacent shots that cut between two different characters in the same location and both claim the same side, and says so in warnings. It repairs mechanically because eyeline is a discrete field with exactly one complement — flipping it cannot damage anybody’s prose. Prose-level laws get a model round trip instead, precisely because they lack that property.

---

Editor — the timeline

The Editor is a top-level workflow, not a stage of the storyboard: engine-agnostic, opens without a board being mid-render, and its document is edit.json under the board dir. One video track, one music lane. Per-clip sound is linked by default and can be pulled apart (J/L cuts) — that is not a second audio track.

Read

Write — POST /storyboard/edit/<sub>

save semantics. revision is the server’s counter, taken from disk, never from what the client remembered — a stale tab must not be able to wind it backwards. Pass expect_revision to find out you were overtaken instead of silently overwriting the other tab: mismatch returns 409 with conflict: true and the current revision. Validation errors come back as all of them at once (errors: [...], 400) with the file on disk untouched, so a client can highlight every bad clip in one pass instead of playing whack-a-mole.

edit.json schema

version is 2 (EDIT_VERSION). Version 1 is upgraded on READ only (migrate_edit) — bumping the version without a read-path upgrade would not refuse old builds, it would refuse every timeline anybody already had. A version from the future is left alone so the validator can say the honest thing.

{
  "version": 2,
  "board_id": "sb_…",
  "revision": 17,
  "origin": "manual",          // "manual" | "auto" | "backup" — who asked for this save
  "updated_at": 1786…,
  "duration": 43.5,
  "clips": [ /* below */ ],
  "audio": { /* below */ },
  "beats": null,
  "settings": {}
}

clips[]

clip.audio — the J-cut and the L-cut

"audio": {"start": 3.5, "end": 7.0, "film_start": 12.25}

Video clips only. Same three numbers as the picture, for the SOUND — so the sound can start before its picture (J) or run past it (L), which is how a cut stops being a butt join.

Absent means linked, and that is the whole migration: every clip ever written has no audio key and every one of them plays its own sound under its own picture. EDIT_VERSION did not move for it and no document was rewritten.

The PRESENCE of the field is the switch, not the values in it. Do not derive “linked” from audio equalling the picture window: unlinking writes the window the clip already has, so a clip the user had just unlinked would read as linked and refuse to be dragged. The toggle adds the object or deletes it; nothing else decides. normalise_edit rounds the field but never removes it for the same reason, and strips it entirely from a still or a slug.

Audio windows may not overlap each other any more than the pictures may. A split edit is a butt join that lands somewhere else, not a mix — the mix is what audio_tracks is for.

audio_tracks[] — A3, A4, … under the music

"audio_tracks": [
  {"id": "t…", "name": "Laughs", "muted": true, "gain": 0.8,
   "strips": [{"id": "k…", "path": "/…/laugh.wav", "start": 0.0, "end": 2.0,
               "film_start": 12.5, "duration": 4.1, "gain": 0.5,
               "muted": true, "locked": true, "title": "laugh (copy)",
               "afx": {"fade_in": 0.5, "fade_out": 0.25, "points": [[1.0, 0.6]]}}]}]

Any number of tracks, each holding any number of strips. start/end are the in/out points inside the file (a strip plays at 1x, so it is end - start long on the film); film_start is a timeline second, laid out with adelay exactly as the bed is. afx is the same envelope object a clip’s sound and the bed carry, read by audio_gain_points; the strip’s gain and the track’s gain (linear 0–1) multiply it — track_strip_gain_points is the one curve the preview, the render and the export read. Absent is none and neutral is absent: an edit with no tracks has no key, so it loads, saves and renders byte-identically; an empty track is kept.

Strips on the same track may not overlap (audio_track_strips_overlap, an error, TOUCH_TOLERANCE for a butt join); strips on different tracks overlap and mix. Shape errors: audio_tracks_shape, audio_track_shape, audio_track_name, audio_track_muted, audio_track_gain(_range), audio_track_strips, track_strip_shape, track_strip_path, track_strip_start, track_strip_end, track_strip_film_start, track_strip_window, track_strip_past_the_end, track_strip_gain(_range), track_strip_muted, track_strip_locked, track_strip_title, track_strip_afx*.

Render: every audible strip (not muted, track not muted, curve not silent) is one -vn -i after the overlays, atrim → volume (its curve, on the strip clock) → aresample → adelay (in samples) → apad → atrim to the film, and all of them are amixed (normalize=0, duration=first) over the clips’ sound and the bed under the one asoftclip limiter; the film facts carry sound_strips and mix_peak. A strip whose file is gone refuses the render with its path and track. Export: each track is its own FCP7 audio track (clipitems with in/out, level keyframes, <enabled>FALSE</enabled> for a mute) and each strip an After Effects layer named <track> · <title>.

transitions[] — a typed object that owns a BOUNDARY

"transitions": [{"id": "t1", "after_clip": "<clip id>",
                 "kind": "dissolve" | "fade_black", "duration": 0.5}]

A cross-dissolve is two clips in the same second, and the picture lane’s one-clip-at-a-time rule (clips_overlap) is load-bearing — so a transition is not an overlap. It names the OUTGOING clip and sits on the cut between it and its successor in film order; one per boundary. The clips’ own film_start/film_end do not move and the film stays exactly as long as the timeline says. The render pulls half the duration of extra tail from beyond the outgoing clip’s out-point and half of extra head from before the incoming clip’s in-point (source the trims left behind), splits the picture concat at the boundary and joins the two runs with xfade (fade / fadeblack) centred on the cut. The sound needs nothing new: it takes the lane path (_sb_split_audio_plan) exactly as a J-cut does.

duration is clamped on every read (transition_duration) to min(asked, half the shorter neighbour, 2.0 s) and snapped to an even number of frames; the document keeps the number typed. A side with no spare material is refused — transition_no_handles names the side and the shortfall (“clip 2 has only 0.10s before its in-point and the dissolve needs 0.25s there”). Every transition code (transitions_shape, transition_shape, transition_unknown_clip, transition_last_clip, transition_duplicate_boundary, transition_kind, transition_duration, transition_no_handles) is an error; none is in WARNING_CODES. render refuses with the same sentence. A still and a slug have all the handles anybody could ask for.

overlays[] — cards, mattes, and titles

kind is still | video | text. An explicit kind wins; with none, the path’s suffix decides (.png/.webp/.tif are stills), so every overlay written before titles existed reads as it did. A title is an overlay whose pixels the render draws:

{"id": "o1", "kind": "text", "text": "FIN", "film_start": 40.0, "film_end": 44.0,
 "style": {"font_size": 96, "color": "#ffffff", "align": "center", "x": 0.5, "y": 0.8,
           "box": true, "box_color": "#000000", "box_opacity": 0.5},
 "fx": {"fade_in": 0.5, "fade_out": 0.5}}

font_size is px at a 1080-high frame and scales with the film; x/y are the anchor as fractions of the frame; align says which edge of the text sits on x. overlay_text() is the one accessor and the style is written only where it differs from the defaults. The font is a FILE — LTX_TITLE_FONT, else the first of TITLE_FONT_CANDIDATES present on the machine — verified before ffmpeg is built; with none, render answers 400 with a sentence rather than a film with a hole in it. The raster is a frame-sized RGBA PNG beside the film (.titles/) fed through the same overlay chain as an uploaded card, so a title inherits the lane’s alpha, fades and z-order. Titles are drawn on the stage in the DOM at the same anchor and size; they do not travel in the NLE export (no path).

audio — the soundtrack, as an object on the timeline

"audio": {"path": "…", "duration": 180.0, "peaks": "peaks.json",
          "offset": -4.0, "trim_start": 12.0, "trim_end": 96.0}

music_window(audio, duration=…) is the single place these become the numbers ffmpeg and every exporter need: start = max(trim_start, offset) (a head trim and a positive offset are the same gesture from two directions; whichever cuts more wins), end or None for “play to the end”, and delay/film_start = start - offset. Trimming the left edge does not slide the rest of the track earlier — that is a ripple, and music does not ripple — so the seconds a head trim removes come back as silence in front. A window the trims close entirely returns “no music” rather than a zero-length atrim ffmpeg refuses.

Drafts, backups, and history — three different things

Drafts are named variations of the same film, in drafts/ with an index.json naming the active one. A board that has only edit.json has exactly one draft and always did; it just had no name for it. load_draft_index names it on the way out rather than in a rewrite pass, so the upgrade cannot half-happen.

POST /storyboard/edit/draft is one route with five verbs — they are five edits to the same active pointer, and splitting them would be five places for that pointer to be wrong:

new, duplicate, delete and activate return 409 with busy: true while that film is rendering — the render is reading this document clip by clip, and swapping the timeline under it is the same hazard import-shots and restore refuse.

The backup is ours, the saving is the user’s. POST /storyboard/edit/backup writes history/backup-<draft>.json and nothing else — no edit.json, no revision, no conflict check. The user’s saved draft stays exactly what he last saved. It is surfaced by GET …/drafts as backup and offered, never applied: recover adopts it, discard-backup throws it away.

History is two lanes plus a keep lane, told apart by filename prefix rather than by opening the files — a prune that has to read fifty documents to decide what to delete fails halfway on the first corrupt one:

Every save archives its predecessor before the new document lands (~5 KB each). History failing must never block a save — losing a breadcrumb is nothing next to refusing to persist the arrangement on screen. origin is also stamped inside the JSON, for anything reading one file on its own.

---

Output management

POST /output/delete

POST /output/hide, POST /output/show, POST /output/show_all

Hide/unhide outputs from the gallery without deleting the file.

POST /output/open_folder

Opens the output’s containing folder in Finder (Mac).

---

Model licences (4.19.8)

GET /licences

Every licence the panel asks about, with this Mac’s acceptance state: {"licences": [{"key": "h3", "model": ..., "licence": ..., "licensor": ..., "version": ..., "url": ..., "non_commercial": false, "title": ..., "summary": ..., "terms": [...], "badge": ..., "accept_label": ..., "accepted": false, "accepted_at": null, ...}, ...], "ui": {...}}. Keys: h3, ideogram4, yue2, flux1_dev, fibo. H3’s entry also carries territory_notice, excluded_territories, report_line, report_url and credit_line.

POST /licences/accept

Form fields key=<licence key> and accept=1 (the box the user ticked). Records the acceptance for the licence’s current version in state/model_licences.json (plus a marker file the H3 installer reads) and returns {"ok": true, "key": ..., "record": {...}, "licence": {...}}. A changed licence version asks again. Accepting on someone’s behalf is accepting for them: an agent should show the user the terms from GET /licences first.

Helper / system

POST /helper/restart

Restart the warm helper subprocess (used after settings changes that need a fresh process).

POST /stop_comfy

Shut down a ComfyUI process if Phosphene started one.

GET /version/check, POST /version/pull

Self-update against the GitHub repo. /version/pull is a git pull only: it does not run scripts/post_update.sh, so anything an Update installs (new packages, weights) is not installed by it. Since 4.17.4 the panel installs a missing Lip-sync voice separator itself at the next start (see /a2v/separator).

POST /engine/repair

Re-runs only the engine-environment step (scripts/pinokio/ltx_engine_env.sh, the same step Install runs): re-installs the render engine’s Python packages into its venv, in the background. No downloads, no venv rebuild, no Stop. Answers 202 (started), 200 {nothing_to_do} (the engine is fine), or 409 (a repair is running, or installing: true - Pinokio’s own Install is still writing the venv). Progress and outcome ride GET /status -> engine_env (ok, fault, installing, repair.{state, active, log, error}).

POST /h3/repair

4.19.5. Re-runs only scripts/pinokio/h3_deps.sh on the Hailuo H3 checkout: re-installs H3’s Python packages into its own venv at the set this release was tested with (scripts/pinokio/h3_constraints.txt, passed with -c; uv moves anything already outside it back). No weights, no Stop - the panel never imports the H3 venv. Answers 202 (started) or 409 (H3 is not installed, an H3 render is running, or a repair is already running). Progress and outcome ride GET /status -> h3_env (ok, fault, repair.{state, active, log, error}). An H3 render on a venv off the set is refused at its start with H3 cannot start: the Hailuo H3 venv ... and this repair on the card.

Push alerts — the completion alert for a closed tab

The in-tab chime and browser alert (Settings → Completion alerts) need the page open. Web Push does not: the browser keeps a subscription for this origin, the panel signs each message with a VAPID key pair it generates once (state/vapid.json), and the browser’s push service wakes /sw.js to show the notification. No relay: the panel posts straight to the browser vendor. Offered only when pywebpush imports (GET /settings → push_available).

Every job that ends done or failed pushes Phosphene — render done / — a render failed with the job’s label (or the first 60 characters of its prompt), off the GPU lock, only while notify_done is on. A subscription the vendor reports gone (404 / 410) is dropped.

POST /settings

POST /prompt/enhance

Form-encoded. Pre-rewrites a prompt with Gemma. Used by the panel’s Enhance buttons (Video and Lip-sync).

Returns { ok, original, enhanced, mode, elapsed_sec, source_non_latin, translated, lora_notes_used: [names], lora_triggers_added: [words] }; { error } with 409 while a render holds the GPU.

409 needs_pack (4.20). Enhance runs on Gemma 3, which a fresh install downloads in the background after the first boot (see deferred_downloads under GET /status). Until it is on disk the route answers 409 { ok: false, error, needs_pack: "gemma", pack_name, pack_size, pack_state } with pack_state one of downloading, queued, waiting_disk, missing, and never asks the helper to load it. /storyboard/plan, /storyboard/replan-shots, /oneshot/plan and /train/auto-caption answer the same shape for the same reason. POST /models/download repo_key=<needs_pack> fetches it now: while the background fetcher holds the download slot that answers 202 { queued: true } and moves the pack to the front of the queue ({ already: true } when it is the one downloading).

POST /upload

multipart/form-data. Uploads a reference image for I2V or keyframe modes. Returns {"path": "<absolute path under panel_uploads/>"}.

---

Sidecar JSON format

Every completed clip writes a sidecar at <output>.mp4.json alongside the MP4. Schema (simplified):

{
  "output": "/abs/path/to/clip_720p.mp4",
  "raw_output": "/abs/path/to/clip.mp4",
  "params": { /* the full params from /queue/add, plus seed_used, model, etc. */ },
  "command": "helper",
  "started": "YYYY-MM-DD HH:MM:SS",
  "elapsed_sec": 426.13,
  "video_duration_sec": 7.0,
  "fps": 24,
  "model": "mlx_models/ltx-2.5-mlx-q4",
  "queue_id": "j-<...>",
  "helper_elapsed_sec": 425.5,
  "output_codec": { "preset": "standard", "pix_fmt": "yuv420p", "crf": "18" },
  "memory_policy": { "requested": "auto", "effective": "auto", "frames": 169, "tier": "standard", "pressure_pct": 30, "swap_gb": 3.9 }
}

Read sidecars to compare runs without re-querying the panel.

---

Production recipe (validated 2026-05-15)

For character-LoRA video work, the validated locked recipe:

mode:               t2v
width × height:     1024 × 576   (also validated: 736 × 416 for ~2× faster wall)
frames:             169 (7s) / 241 (10s) / 361 (15s)
quality:            high          ← CRITICAL — forces Q8 dev transformer
temporal_mode:      native
stage1_steps:       10
stage2_steps:       3
teacache_thresh:    1.8–2.0       (plateau speedup; below 1.6 = no speedup)
cfg_scale:          3.0
bongmath_max_iter:  100
upscale:            off | fit_720p (lanczos)
accel:              off
enhance:            false         ← critical when trigger words are in the prompt
seed:               42 for A/B, -1 for production variety
LoRAs:              <trigger>_v2.safetensors @ 1.0

Wall on M4 Max 64 GB: ~7:06 per 7s clip at 1024×576, ~3:24 at 736×416.

---

Worked example — one agent driver

Submit a clip with two LoRAs, poll until done, print the output path. Pure stdlib.

import json, time, re, urllib.parse, urllib.request

PANEL = "http://127.0.0.1:8199"

def submit(prompt, loras, **overrides):
    form = {
        "mode": "t2v",
        "prompt": prompt,
        "width": "1024", "height": "576", "frames": "169", "steps": "8",
        "seed": "-1",
        "quality": "high", "temporal_mode": "native",
        "stage1_steps": "10", "stage2_steps": "3",
        "teacache_thresh": "1.8", "cfg_scale": "3.0",
        "bongmath_max_iter": "100", "accel": "off",
        "enhance": "false",
        "upscale": "fit_720p", "upscale_method": "lanczos",
        "loras": json.dumps(loras),
    }
    form.update({k: str(v) for k, v in overrides.items()})
    data = urllib.parse.urlencode(form).encode()
    req = urllib.request.Request(f"{PANEL}/queue/add", data=data, method="POST")
    with urllib.request.urlopen(req, timeout=15) as resp:
        return json.loads(resp.read())["id"]

def wait_terminal(job_id, timeout=1800):
    deadline = time.time() + timeout
    while time.time() < deadline:
        with urllib.request.urlopen(f"{PANEL}/status", timeout=15) as resp:
            raw = re.sub(r"[\x00-\x08\x0b\x0c\x0e-\x1f]", " ", resp.read().decode())
        d = json.loads(raw, strict=False)
        for src in ([d.get("current")] if d.get("current") else [], d.get("queue") or [], d.get("history") or []):
            for j in src:
                if not j or j.get("id") != job_id:
                    continue
                st = j.get("status")
                if st in ("done", "failed", "cancelled", "error"):
                    return j
        time.sleep(20)
    raise TimeoutError(f"job {job_id} did not finish")

if __name__ == "__main__":
    # Adjust to wherever your Phosphene install lives:
    LORAS_DIR = "<panel-root>/mlx_models/loras"
    job_id = submit(
        prompt="An original illustrated explorer in a wood-paneled study, storybook illustration.",
        loras=[
            {"path": f"{LORAS_DIR}/explorertrn_v2.safetensors", "strength": 1.0},
        ],
    )
    print(f"queued: {job_id}")
    result = wait_terminal(job_id)
    print(f"status={result['status']} wall={result['elapsed_sec']}s output={result.get('output_path')}")

That’s the full pattern: submit → poll → read output path.

---

What was removed (and what replaced it)

The in-panel agentic flows feature (chat-driven shot planner) was removed 2026-05-15. Pre-removal snapshot is durable on GitHub as tag pre-agent-removal-2026-05-15.

Everything that the chat used to do — plan multiple shots, generate candidate anchor images, render I2V conditioned on those anchors, manage sessions — is achievable by an external agent through the API documented above. Specifically:

• Shot planning: agent-side text reasoning, no panel involvement.
• Anchor generation: POST /image/generate.
• Anchor-to-video: POST /queue/add with mode=i2v and image=<anchor_path>.
• Session state: agent maintains it externally.

External agents can use the documented workflows through this API.