venc — Standalone Video Encoder & Streamer

Standalone H.265/H.264 video encoder and RTP streamer for SigmaStar Infinity6E (Star6E) and Infinity6C (Maruko) camera SoCs. Designed for low-latency FPV and IP camera applications with full runtime control via HTTP API.

Features

H.265 (HEVC) and H.264 encoding with CBR/VBR/AVBR/FIXQP rate control
RTP packetization with adaptive payload sizing
Compact UDP streaming mode (raw NAL units)
Built-in web dashboard at / for configuration, API docs, and IQ tuning
HTTP API for live parameter tuning without pipeline restart
ISP IQ parameter system: 60+ params with multi-field structs, export/import (both backends)
Custom 3A: built-in AE and AWB with configurable gain limits and convergence
ISP control: exposure, AWB mode, color temperature
ROI-based QP gradient for FPV center-priority encoding
Sensor FPS unlock for IMX415/IMX335 (up to 120fps)
Optional audio capture (Opus / G.711a / G.711µ / raw PCM) on both backends, RTP or compact UDP output, mute via live API
SD card recording: MPEG-TS mux (HEVC + audio in TS, PCM/A-law/µ-law/Opus alongside video), power-loss safe; raw .hevc available on Star6E
Gemini mode: dual VENC for concurrent stream + high-quality record (both backends; Maruko via Phase 7 port, Star6E reference)
Adaptive recording bitrate: auto-reduces if SD card can't keep up
Dual-backend: Star6E and Maruko from shared codebase (dlopen for all MI libs)
Maruko-specific opt-in 3A throttle (isp.aeMode="throttle") — saves ~24% sys CPU at 120 fps with no visible AE quality loss
BMI270 IMU driver with frame-synced FIFO (both backends) — module compiled in but disabled by default; ready for telemetry/sidecar consumers

Build

From repository root:

# Star6E (Infinity6E)
make build SOC_BUILD=star6e

# Maruko (Infinity6C)
make build SOC_BUILD=maruko

The toolchain is auto-downloaded on first build. Each backend builds to its own output directory:

out/star6e/venc    # Star6E binary
out/maruko/venc    # Maruko binary

Both backends can coexist — no clean needed between them.

Stage a deployable bundle with shared libraries:

make stage SOC_BUILD=star6e
# Output: out/star6e/venc + out/star6e/lib/*.so

Run host tests:

make test-ci

Deployment

Copy the binary to the target device:

scp out/star6e/venc root@<device-ip>:/usr/bin/venc

For the current Star6E bench workflow, prefer the helper:

scripts/star6e_direct_deploy.sh cycle

It deploys /usr/bin/venc, uses the production /etc/venc.json, waits for HTTP readiness, and captures /tmp/venc.log.

The binary resolves shared libraries from /usr/lib. For staged bundles, set LD_LIBRARY_PATH to the lib directory.

Configuration

venc loads configuration from /etc/venc.json on startup. A default template is provided at config/venc.default.json.

{
  "system": { "webPort": 80, "overclockLevel": 0, "verbose": false },
  "sensor": {
    "index": -1, "mode": -1,
    "unlockEnabled": true, "unlockCmd": 35,
    "unlockReg": 12298, "unlockValue": 128, "unlockDir": 0
  },
  "isp": {
    "sensorBin": "",
    "legacyAe": true, "aeFps": 15,
    "aeMode": "native",
    "gainMax": 0,
    "awbMode": "auto", "awbCt": 5500,
    "keepAspect": true
  },
  "image": { "mirror": false, "flip": false, "rotate": 0 },
  "video0": {
    "codec": "h265", "rcMode": "cbr", "fps": 30,
    "bitrate": 8192, "gopSize": 1.0,
    "qpDelta": -4
  },
  "outgoing": {
    "enabled": false, "server": "", "streamMode": "rtp",
    "maxPayloadSize": 1400,
    "connectedUdp": true, "audioPort": 5601, "sidecarPort": 5602
  },
  "fpv": {
    "roiEnabled": true, "roiQp": 0, "roiSteps": 2,
    "roiCenter": 0.25, "noiseLevel": 0
  },
  "audio": {
    "enabled": false, "sampleRate": 16000, "channels": 1,
    "codec": "g711a", "volume": 80, "mute": false
  },
  "imu": {
    "enabled": false, "i2cDevice": "/dev/i2c-1", "i2cAddr": "0x68",
    "sampleRateHz": 200, "gyroRangeDps": 1000,
    "calFile": "/etc/imu.cal", "calSamples": 400
  },
  "record": {
    "enabled": false, "mode": "mirror", "dir": "/mnt/mmcblk0p1",
    "format": "ts", "maxSeconds": 300, "maxMB": 500,
    "bitrate": 0, "fps": 0, "gopSize": 0, "server": ""
  }
}

Set outgoing.enabled to true and outgoing.server to udp://<receiver_ip>:5600, unix://<abstract_name>, or shm://<ring_name> to start streaming.

HTTP API

All endpoints use HTTP GET (BusyBox wget compatible). The default port is 80 (configurable via system.webPort). Responses are JSON with an {"ok": true/false, ...} envelope.

Endpoints

GET /api/v1/version

Returns version info.

curl http://<device-ip>:<port>/api/v1/version

{"ok":true,"data":{"app_version":"...","backend":"star6e","contract_version":"0.2.0","config_schema_version":"0.2.0"}}

GET /api/v1/config

Returns the full active configuration as JSON.

curl http://<device-ip>:<port>/api/v1/config

GET /api/v1/capabilities

Returns every field with its mutability (live or restart_required) and support status. Support is backend-specific; for example, Star6E reports video0.scene_threshold / video0.scene_holdoff as supported, while Maruko reports them as unsupported. Use this to discover which fields can be changed at runtime.

curl http://<device-ip>:<port>/api/v1/capabilities

GET /api/v1/get?field_name

Read a single configuration field.

curl "http://<device-ip>:<port>/api/v1/get?video0.bitrate"

{"ok":true,"data":{"field":"video0.bitrate","value":8192}}

GET /api/v1/set?field_name=value

Write a field. Live fields take effect immediately. Restart-required fields trigger an automatic pipeline reinit.

# Live change — immediate
curl "http://<device-ip>:<port>/api/v1/set?video0.bitrate=4096"

# Live multi-set — all fields must be live
curl "http://<device-ip>:<port>/api/v1/set?video0.bitrate=4096&system.verbose=true"

# Restart-required — triggers pipeline reinit
curl "http://<device-ip>:<port>/api/v1/set?video0.size=1280x720"

{"ok":true,"data":{"field":"video0.bitrate","value":4096}}
{"ok":true,"data":{"applied":[{"field":"video0.bitrate","value":4096},{"field":"system.verbose","value":true}]}}
{"ok":true,"data":{"field":"video0.size","value":"1280x720","reinit_pending":true}}

Multi-set is supported only for live fields. If any restart-required field is present, the full request is rejected and restart/reinit changes must be sent one at a time.

Returns HTTP 409 on validation failure (e.g., invalid AWB mode).

GET /api/v1/restart

Trigger a full pipeline reinit. Reloads /etc/venc.json and restarts the camera pipeline without exiting the process.

curl http://<device-ip>:<port>/api/v1/restart

GET /api/v1/awb

Query current AWB (auto white balance) state from the ISP.

curl http://<device-ip>:<port>/api/v1/awb

GET /request/idr

Request an IDR keyframe from the encoder.

curl http://<device-ip>:<port>/request/idr

GET /api/v1/record/start

Start SD card recording. Uses the configured record.dir, or override with a ?dir= query parameter.

curl "http://<device-ip>:<port>/api/v1/record/start"
curl "http://<device-ip>:<port>/api/v1/record/start?dir=/mnt/mmcblk0p1"

GET /api/v1/record/stop

Stop SD card recording.

curl "http://<device-ip>:<port>/api/v1/record/stop"

GET /api/v1/record/status

Query recording status.

curl "http://<device-ip>:<port>/api/v1/record/status"

{"ok":true,"data":{"active":true,"format":"ts","path":"/mnt/mmcblk0p1/rec_01h23m45s_abcd.ts","frames":1500,"bytes":12345678,"segments":1,"stop_reason":"none"}}

GET /api/v1/dual/status

Query the secondary VENC channel status (dual/dual-stream modes only).

curl "http://<device-ip>:<port>/api/v1/dual/status"

{"ok":true,"data":{"active":true,"channel":1,"bitrate":20000,"fps":120,"gop":240}}

Returns 404 when dual VENC is not active.

GET /api/v1/dual/set?param=value

Live-change secondary VENC channel parameters.

# Change ch1 bitrate
curl "http://<device-ip>:<port>/api/v1/dual/set?bitrate=10000"

# Change ch1 GOP (in seconds)
curl "http://<device-ip>:<port>/api/v1/dual/set?gop=1.0"

GET /api/v1/dual/idr

Request an IDR keyframe on the secondary VENC channel.

curl "http://<device-ip>:<port>/api/v1/dual/idr"

GET /api/v1/audio/status

Live snapshot of the audio capture/encode pipeline (lib loaded, capture running, codec, rate, channels, Opus initialization). Both backends. See HTTP_API_CONTRACT.md for full field reference.

curl http://<device-ip>:<port>/api/v1/audio/status

GET /api/v1/transport/status

Live observability for the active video transport (UDP / Unix / SHM): fill percentage, backpressure flag, lifetime drop counters. Used by the WebUI status bar and external link controllers.

curl http://<device-ip>:<port>/api/v1/transport/status

GET /api/v1/idr/stats

Per-channel IDR-rate-limit counters: how many requests were honored vs. coalesced.

curl http://<device-ip>:<port>/api/v1/idr/stats

GET /api/v1/modes

Sensor pad and resolution mode introspection — populates the WebUI sensor-mode dropdown. Reports the currently-active selection plus every mode the SDK enumerates.

curl http://<device-ip>:<port>/api/v1/modes

Field Reference

Fields marked live can be changed at runtime without interrupting the video stream. Fields marked restart trigger a pipeline reinit.

System

Field	Type	Mutability	Description
`system.web_port`	uint16	restart	HTTP API port
`system.overclock_level`	int	restart	CPU overclock level
`system.verbose`	bool	live	Enable verbose logging

Sensor

Field	Type	Mutability	Description
`sensor.index`	int	restart	Sensor pad index (-1 = auto)
`sensor.mode`	int	restart	Sensor mode (-1 = auto)
`sensor.unlock_enabled`	bool	restart	Enable high-FPS sensor unlock
`sensor.unlock_cmd`	uint	restart	I2C register write command
`sensor.unlock_reg`	uint16	restart	Unlock register address
`sensor.unlock_value`	uint16	restart	Unlock register value
`sensor.unlock_dir`	int	restart	I2C direction flag

ISP

Field	Type	Mutability	Description
`isp.sensor_bin`	string	live	ISP tuning binary path (empty = auto-detect /etc/sensors/<sensor>.bin)
`isp.legacy_ae`	bool	restart	Use ISP internal AE instead of custom 3A (Star6E)
`isp.ae_fps`	uint	restart	Custom 3A processing rate in Hz (default 15)
`isp.ae_mode`	string	restart	Maruko-only: `"native"` (default, SDK runs AE/AWB at sensor rate) or `"throttle"` (no-op AE adaptor + 15 Hz manual AE; saves ~24% sys CPU at 120 fps). Alias: `isp.aeMode`.
`isp.gain_max`	uint	live	AE max ISP gain ceiling (0 = use ISP bin default)
`isp.awb_mode`	string	live	`"auto"` or `"ct_manual"`
`isp.awb_ct`	uint	live	Color temperature in K (for ct_manual)
`isp.keep_aspect`	bool	restart	When `true` (default), VIF/SCL crop preserves sensor AR; `false` lets downstream stretch. Star6E + Maruko (Phase 1, v0.9.9).

Image

Field	Type	Mutability	Description
`image.mirror`	bool	restart	Horizontal mirror
`image.flip`	bool	restart	Vertical flip
`image.rotate`	int	restart	Rotation (0, 90, 180, 270)

Video

Field	Type	Mutability	Description
`video0.codec`	string	restart	`"h265"` (Maruko also supports `"h264"`; Star6E RTP remains h265-only)
`video0.rc_mode`	string	restart	`"cbr"`, `"vbr"`, `"avbr"`, `"fixqp"`
`video0.fps`	uint	live	Output frame rate
`video0.size`	string	restart	Encode resolution: `"auto"` (default, uses sensor native), `"1920x1080"`, `"720p"`, `"1080p"`
`video0.bitrate`	uint	live	Target bitrate in kbps
`video0.gop_size`	double	live	GOP interval in seconds (0 = all-intra)
`video0.qp_delta`	int	live	Relative I/P QP delta (-12..12)
`video0.frame_lost`	bool	restart	Enable frame-lost safety net
`video0.zoom_pct`	double	restart	Digital zoom crop fraction (`0.0` = off, `0.25..1.0` = crop fraction)
`video0.zoom_x`	double	live	Zoom crop center X (`0.0` left to `1.0` right)
`video0.zoom_y`	double	live	Zoom crop center Y (`0.0` top to `1.0` bottom)

Digital Zoom (Star6E + Maruko)

Approach-C digital zoom shrinks both the crop window and encoded output resolution. The SCL path reads the crop at 1:1 and emits it unchanged, so there is no upscale pass and no extra bandwidth pressure. Receivers see the smaller resolution in SPS/PPS.

Field	Type	Mutability	Description
`video0.zoom_pct`	double	restart	`0.0` = off/full frame; `0.25..1.0` = crop fraction (smaller = deeper zoom)
`video0.zoom_x`	double	live	Crop center X, `0.0` = left, `1.0` = right
`video0.zoom_y`	double	live	Crop center Y, `0.0` = top, `1.0` = bottom

CamelCase aliases: video0.zoomPct, video0.zoomX, video0.zoomY.

Examples:

# Restart-required: enable a 2x crop.
curl "http://<device>/api/v1/set?video0.zoomPct=0.5"

# Live pan inside the current crop size.
curl "http://<device>/api/v1/set?video0.zoomX=0.25&video0.zoomY=0.75"

# Disable zoom on the next reinit.
curl "http://<device>/api/v1/set?video0.zoomPct=0.0"

When debug.showOsd=true and zoom is active, the overlay adds rows after existing OSD stats:

zoom  2.00x 960x540
crop  960x540+480+270

zoom shows magnification and encoded resolution. crop shows the source crop size and placement within the sensor/precrop surface.

Adaptive Encoder Control (Star6E + Maruko)

Field	Type	Mutability	Description
`video0.scene_threshold`	uint16	restart	Scene spike threshold ratio x100 (0=off, 150=1.5x EMA spike detection)
`video0.scene_holdoff`	uint8	restart	Consecutive spike frames required (default 2)

CamelCase aliases: video0.sceneThreshold, video0.sceneHoldoff.

When scene_threshold is non-zero, the inline scene detector tracks frame size EMA, computes complexity, and requests an IDR after a spike above the threshold settles. Use /api/v1/capabilities to check backend support before writing these fields.

Typical usage:

Leave video0.scene_threshold=0 for fixed-GOP behavior controlled by video0.gop_size.
Set video0.scene_threshold=150 for FPV/live links where scene-change-triggered IDRs improve stream recovery.
Pair scene detection with outgoing.sidecar_port>0 when an external controller needs per-frame frame_type, complexity, scene_change, idr_inserted, and frames_since_idr telemetry on the sidecar.

Current Star6E IMX335 bench starting point:

"video0": {
  "sceneThreshold": 150,
  "sceneHoldoff": 2
}

Tuning notes:

sceneThreshold is a frame-size-spike ratio scaled by 100, so 150 means roughly "trigger near a 1.5x spike over the rolling baseline". Raise to reduce false positives, lower to increase sensitivity.
Keep sceneChangeHoldoff=2 unless threshold changes alone cannot suppress false positives. Raising holdoff reduces responsiveness faster than raising threshold does.

Codec note:

Star6E with outgoing.stream_mode="rtp" requires video0.codec="h265".
Maruko accepts both h264 and h265.

Intra Refresh (Star6E + Maruko)

GDR-style rolling stripe: a configurable number of MB/LCU rows in each P-frame are intra-coded so a decoder that joins mid-stream — or recovers from a packet loss burst — can resync without waiting for the next IDR. Layered over normal GOP-based IDRs (Majestic-style belt-and-suspenders).

Single mode knob picks intent (self-heal target window); GOP, lines, and QP all derive from the mode. Per-field overrides remain available for power users — non-zero overrides win.

Field	Type	Mutability	Description
`video0.intra_refresh_mode`	string	restart	`off` \| `fast` \| `balanced` \| `robust` (default `off`)
`video0.intra_refresh_lines`	uint16	restart	LCU/MB rows refreshed per P-frame (`0` = mode auto)
`video0.intra_refresh_qp`	uint8	restart	QP for the intra-refreshed rows (`0` = codec default: 48 H.265 / 45 H.264)

CamelCase aliases: video0.intraRefreshMode, video0.intraRefreshLines, video0.intraRefreshQp.

Mode targets (self-heal window from packet loss to fully-refreshed picture):

Mode	target	Use case
`off`	—	feature disabled
`fast`	150 ms	FPV racing, low-latency, clean link
`balanced`	500 ms	general FPV (recommended starting point)
`robust`	1000 ms	lossy long-range, high packet loss

Stripe QP defaults (codec-aware, lower = better quality + more bitrate cost):

Mode	H.265 QP	H.264 QP
`fast`	36	33
`balanced`	32	29
`robust`	28	25

Robust runs the lowest QP because lossy links want the cleanest possible recovery anchor; fast runs the highest because clean links can absorb minor stripe banding without artifacts. Override with intraRefreshQp (1–51).

When a mode is active the encoder computes:

total_rows     = ceil(height / lcu_h)            // lcu_h: 32 H.265, 16 H.264
refresh_frames = round(fps * target_ms / 1000)
auto_lines     = ceil(total_rows / refresh_frames)
auto_gop       = ceil(total_rows / effective_lines)   // one IDR per GDR pass

Auto-GOP overrides gop_size so each IDR aligns with one full GDR pass — no half-cycles, no cycle without a hard recovery anchor. Setting an explicit gopSize > 0 suppresses auto-GOP and keeps the user value (logged at boot).

Precomputed values @ 60 fps H.265

For other framerates: gop_sec scales as 60 / fps. Lines stays the same unless refresh_frames rounds differently — at 30 fps fast doubles its window, at 120 fps it halves.

Resolution	total_rows	mode	lines	gop frames	gop sec	qp
1280×720	23	fast	3	8	0.133	36
1280×720	23	balanced	1	23	0.383	32
1280×720	23	robust	1	23	0.383 ⚠	28
1456×816	26	fast	3	9	0.150	36
1456×816	26	balanced	1	26	0.433	32
1456×816	26	robust	1	26	0.433 ⚠	28
1920×1080	34	fast	4	9	0.150	36
1920×1080	34	balanced	2	17	0.283	32
1920×1080	34	robust	1	34	0.567	28
2560×1440	45	fast	5	9	0.150	36
2560×1440	45	balanced	2	23	0.383	32
2560×1440	45	robust	1	45	0.750	28
3840×2160	68	fast	8	9	0.150	36
3840×2160	68	balanced	3	23	0.383	32
3840×2160	68	robust	2	34	0.567	28

⚠ At 720p and below, robust and balanced collapse to identical numbers because total_rows is small enough that even balanced refreshes in 1 line per P-frame. The mode label still ships through (recorded in status endpoint) but the encoder behavior is identical.

H.264 doubles total_rows (lcu_h = 16 → 720p has 45 rows, 1080p has 68 rows) so lines and gop scale up roughly 2×, but the gop seconds match the H.265 column closely.

Quick start — one HTTP call:

curl -X POST 'http://<device>/api/v1/intra/mode?mode=balanced'

This sets the mode, clears any per-field overrides, persists, and reinits the encoder. Equivalent to editing the config JSON and triggering reload.

Notes:

Budget +20–30 % bitrate when enabling refresh; intra-coded rows compress worse than inter-coded ones.
Refresh is applied to ch0 only. The dual-VENC recorder (ch1) is intentionally skipped — TS containers expect IDRs at GOP boundaries.
Explicit intraRefreshLines greater than the picture's LCU-row count are clamped (with a [venc] WARNING) to avoid SDK underflow.
Both backends use the identical MI_VENC_IntraRefresh_t layout (bEnable, u32RefreshLineNum, u32ReqIQp); the Maruko symbol takes (MI_VENC_DEV, MI_VENC_CHN, *cfg) while Star6E takes (MI_VENC_CHN, *cfg).
Maruko: MI_VENC_SetIntraRefresh is treated as an optional symbol — the loader logs a warning if dlsym misses on older firmware drops, and the pipeline falls back to plain GOP-based IDRs (mi_supported=false).

Status endpoint:

curl http://<device>/api/v1/intra/status
# { "ok":true, "data":{
#     "mode": "balanced",
#     "active": true,
#     "mi_supported": true,
#     "apply_ok": true,
#     "target_ms": 500,
#     "total_rows": 34,
#     "lines": { "requested": 0,    "effective": 2,    "clamped": false },
#     "qp":    { "requested": 0,    "effective": 32 },
#     "gop":   { "explicit_sec": 0.0, "effective_sec": 0.283, "auto": true }
# }}

Boot log (from stderr):

[venc] intraRefresh: mode=balanced lines/P=2 qp=32 gop=0.28s (auto)

When debug.showOsd=true and a mode is active, two extra OSD rows render the live values. If zoom is also active, zoom rows are appended below them instead of replacing the intra rows:

intra balanced L2 q32
gop   0.28s auto
zoom  2.00x 960x540
crop  960x540+480+270

intra shows mode, effective stripe lines per P-frame, and effective QP. gop shows the IDR period in seconds and whether it came from auto or an explicit gopSize override. zoom shows magnification and encoded resolution; crop shows source placement.

Outgoing (Streaming)

Field	Type	Mutability	Description
`outgoing.enabled`	bool	live	Enable/disable streaming output
`outgoing.server`	string	live	Destination URI (`udp://ip:port`, `unix://name`, or `shm://name`)
`outgoing.stream_mode`	string	restart	`"rtp"` or `"compact"`
`outgoing.max_payload_size`	uint16	restart	Max UDP payload bytes
`outgoing.connected_udp`	bool	restart	Connect UDP socket (applies only to `udp://`)
`outgoing.audio_port`	uint16	restart	`0` = shared video destination; nonzero = dedicated audio port. With `unix://`, dedicated audio is sent to `127.0.0.1:<audioPort>`
`outgoing.sidecar_port`	uint16	restart	RTP timing sidecar port (0 = disabled)

unix:// uses Linux abstract Unix datagram sockets and is available in both rtp and compact mode. On Star6E, audioPort=0 piggybacks on the same active video destination for both udp:// and unix://. shm:// remains RTP-only; it cannot share audio, but a nonzero audioPort still uses a dedicated local UDP audio destination.

FPV

Field	Type	Mutability	Description
`fpv.roi_enabled`	bool	live	Enable horizontal ROI bands
`fpv.roi_qp`	int	live	Signed ROI delta QP (-30..30, negative = sharper center)
`fpv.roi_steps`	uint16	live	Number of horizontal bands (1-4)
`fpv.roi_center`	double	live	Center band width ratio (0.1-0.9)
`fpv.noise_level`	int	restart	3DNR noise reduction level

Audio

Field	Type	Mutability	Description
`audio.mute`	bool	live	Mute/unmute audio output

Audio configuration (enabled, sample rate, channels, codec, volume) is set in /etc/venc.json only and requires a process restart to change.

Supported codecs: "pcm" (raw 16-bit, big-endian L16 per RFC 3551), "g711a" (A-law), "g711u" (µ-law), "opus" (requires libopus.so at runtime; falls back to PCM with a warning if the library or encoder is unavailable).

RTP payload types: When streaming in RTP mode, venc uses standard static payload types when the sample rate matches the RFC 3551 standard:

Codec	Sample rate	RTP PT	Notes
`g711u`	8000	0 (PCMU)	RFC 3551 standard
`g711a`	8000	8 (PCMA)	RFC 3551 standard
`g711u`	non-8kHz	112	Dynamic, Waybeam convention
`g711a`	non-8kHz	113	Dynamic, Waybeam convention
`pcm`	44100	11 (L16 mono)	RFC 3551 standard
`pcm`	other	110	Dynamic PCM
`opus`	any	98	Dynamic, majestic-compatible (RFC 7587)

Sample rate range: 8000–48000 Hz (clamped by config parser). For Opus the recommended sample rate is 48000 Hz (native Opus clock, no resampling); the RTP clock is fixed at 48 kHz per RFC 7587 regardless of capture rate. For voice-only FPV audio, 16 kHz G.711a remains a low-latency choice.

Frame timing: Each RTP packet carries one 20 ms frame. The RTP timestamp advances by sample_rate / 50 samples for PCM/G.711, and by 960 (the 48 kHz nominal Opus tick) for Opus.

Receiving Opus:

gst-launch-1.0 udpsrc port=5601 \
    caps="application/x-rtp,media=audio,payload=98,clock-rate=48000,encoding-name=OPUS" \
  ! rtpopusdepay ! opusdec ! audioconvert ! autoaudiosink

Recording

Field	Type	Mutability	Description
`record.enabled`	bool	restart	Start recording on launch
`record.mode`	string	restart	`"off"`, `"mirror"`, `"dual"`, `"dual-stream"`
`record.dir`	string	restart	Output directory (must be mounted)
`record.format`	string	restart	`"ts"` (MPEG-TS + audio) or `"hevc"` (raw); on Maruko only `"ts"` is implemented
`record.max_seconds`	uint	restart	Rotate file after N seconds (0 = off)
`record.max_mb`	uint	restart	Rotate file after N MB (0 = off)
`record.bitrate`	uint	restart	Dual mode: ch1 bitrate in kbps (0 = same as video0)
`record.fps`	uint	restart	Dual mode: ch1 fps (0 = sensor max)
`record.gop_size`	double	restart	Dual mode: ch1 GOP in seconds (0 = same as video0)
`record.server`	string	restart	Dual-stream: second RTP destination URI

Backend support:

Star6E — full feature set: mirror/dual/dual-stream modes, both ts and hevc formats, HTTP-driven start/stop via /api/v1/record/start|stop, adaptive bitrate while SD-bound.
Maruko (Phase 6, v0.9.14) — mirror and dual modes wired, ts format only, config-driven only: set record.enabled=true
- record.mode=... in /etc/venc.json and reload. HTTP /api/v1/record/start|stop returns 501 not_implemented on Maruko (Phase 6.5 backlog: wire the runtime poll loop and record_status_callback). Audio is interleaved into the TS file whenever Phase 5 audio capture is active (audio.enabled=true).

Recording can also be controlled at runtime via the HTTP API. In dual/dual-stream modes, the secondary channel parameters can be adjusted live via /api/v1/dual/set.

HTTP /api/v1/record/start|stop behavior vs configured record.mode:

/api/v1/record/start always writes ch0 (the live encoded stream) to disk in the configured record.format — it cannot change the pipeline topology, only open or close a recording file. Whether that even runs depends on whether a dedicated recording thread already owns the recorder:

`record.enabled`	`record.mode`	Auto-start at boot	Dashboard Start / `/record/start`
false	any	no	starts ch0 mirror, `record.format` respected
true	`off`	no	starts ch0 mirror, `record.format` respected
true	`mirror`	yes (ch0 → disk)	restarts ch0 mirror
true	`dual`	yes (ch1 → disk, dedicated)	silently ignored — ch1 thread owns the recorder
true	`dual-stream`	no (ch1 → RTP via `record.server`)	silently ignored — ch1 is streamed, not recorded

The "silently ignored" rows exist because ps->dual is non-NULL only when record.enabled=true && mode ∈ {dual, dual-stream}; the runtime loop explicitly skips the HTTP start/stop poll in that case to avoid racing the dedicated recording thread. The dashboard Recordings tab detects this configuration and disables the Start/Stop buttons with a reason note.

File rotation (record.max_seconds, record.max_mb) applies equally to config-started and HTTP-started recordings — both use the same ts_recorder / recorder object.

IMU (both backends, POC consumer)

Field	Type	Mutability	Description
`imu.enabled`	bool	restart	Enable BMI270 IMU driver
`imu.i2c_device`	string	restart	I2C device path
`imu.i2c_addr`	uint8	restart	I2C address (decimal or hex string, e.g. `104` or `"0x68"`)
`imu.sample_rate_hz`	int	restart	ODR in Hz (25-1600). Alias: `imu.sampleRateHz`.
`imu.gyro_range_dps`	int	restart	Gyro range in ±dps. Alias: `imu.gyroRangeDps`.
`imu.cal_file`	string	restart	Calibration file path
`imu.cal_samples`	int	restart	Auto-bias samples at startup

Phase 3 (PR #84, v0.9.13) ported the IMU driver to Maruko with one caveat: on Maruko, init must run before MI_VENC_StartRecvPic because the 2 s auto-bias loop blocking the main thread post-VENC leaves the encoder fd in a state where poll() never returns POLLIN. This ordering constraint is captured in maruko_pipeline.c; do not re-order without re-running the bench check on 192.168.2.12.

Usage Examples

Start streaming to a receiver:

curl "http://<device-ip>:<port>/api/v1/set?outgoing.server=udp://<receiver-ip>:5600"
curl "http://<device-ip>:<port>/api/v1/set?outgoing.enabled=true"

Switch to 720p at 90fps with lower bitrate:

curl "http://<device-ip>:<port>/api/v1/set?video0.size=1280x720"
curl "http://<device-ip>:<port>/api/v1/set?video0.fps=90"
curl "http://<device-ip>:<port>/api/v1/set?video0.bitrate=4096"

Manual white balance at 6500K:

curl "http://<device-ip>:<port>/api/v1/set?isp.awb_mode=ct_manual"
curl "http://<device-ip>:<port>/api/v1/set?isp.awb_ct=6500"

Enable center-priority ROI encoding:

curl "http://<device-ip>:<port>/api/v1/set?fpv.roi_enabled=true"
curl "http://<device-ip>:<port>/api/v1/set?fpv.roi_qp=-18"
curl "http://<device-ip>:<port>/api/v1/set?fpv.roi_steps=2"

Request an IDR keyframe (useful after stream start):

curl http://<device-ip>:<port>/request/idr

Start/stop SD card recording:

# Start recording (MPEG-TS with audio)
curl "http://<device-ip>:<port>/api/v1/record/start"

# Check recording status
curl "http://<device-ip>:<port>/api/v1/record/status"

# Stop recording
curl "http://<device-ip>:<port>/api/v1/record/stop"

SD Card Recording

venc records HEVC video with PCM audio to SD card in MPEG-TS format. Recording runs concurrently with RTP streaming at minimal CPU overhead (1-4% additional load measured across 30-120fps at 4-22 Mbps).

Key properties:

Power-loss safe — MPEG-TS requires no finalization; partial files are playable up to the last written packet.
Gemini mode — dual VENC channels for independent stream and record quality. Stream at 30fps 4 Mbps over WiFi while recording at 120fps 20 Mbps to SD card. Four modes: off, mirror, dual, dual-stream.
Recording thread — dedicated pthread drains the secondary encoder channel at full speed, with adaptive bitrate reduction (10%/s) if the SD card can't keep up.
File rotation — splits at IDR keyframe boundaries by time (default 5 minutes) or size (default 500 MB). Each segment is independently playable.
Disk safety — periodic free-space checks with automatic stop when below 50 MB. Handles ENOSPC gracefully.
Audio interleaving — raw 16-bit PCM from the hardware audio input is muxed alongside HEVC video in the TS container.
Live API control — /api/v1/dual/set for runtime bitrate/GOP changes on the secondary channel.

Enable in config or use the HTTP API for runtime control. The SD card must be pre-mounted at the configured directory (OpenIPC auto-mounts to /mnt/mmcblk0p1).

Verify recordings with:

ffprobe recording.ts          # check streams and format
ffmpeg -i recording.ts -f null -   # full decode test
ffplay recording.ts           # play directly

See documentation/SD_CARD_RECORDING.md for the full guide including performance benchmarks, limitations, and architecture details.

RTP Timing Sidecar

An optional out-of-band UDP channel that sends per-frame timing metadata alongside the RTP video stream. Set outgoing.sidecarPort=0 to disable it.

Purpose

When enabled, the sidecar provides frame-level diagnostics for the entire sender-side pipeline:

capture_us → [encode] → frame_ready_us → [packetise+send] → last_pkt_send_us
                                                              ↕ (network)
                                                        recv_last_us (probe)

This enables measurement of:

Encode duration — time from sensor capture to encoder output
Send spread — time to packetise and hand all RTP packets to the kernel
One-way latency — frame-ready on venc to first-packet-received on ground (requires clock synchronisation)
Frame intervals — jitter and regularity of both sender and receiver clocks
RTP packet counts and gaps — per-frame packet accounting
Encoded frame size / type / QP — when Star6E scene detection is active
Scene detection state — complexity, scene-change flag, IDR decision, frames-since-IDR

Enabling

Set the sidecar port in the configuration:

curl "http://<device-ip>:<port>/api/v1/set?outgoing.sidecar_port=6666"

Or in /etc/venc.json:

"outgoing": { "sidecarPort": 6666 }

A pipeline restart is required after changing this setting. The sidecar socket is silent until a probe subscribes — zero network overhead when no probe is connected.

When the sidecar is disabled (port 0), no socket is created and there is no runtime overhead.

Wire Protocol

The sidecar uses a simple binary UDP protocol:

Message	Direction	Size	Purpose
SUBSCRIBE	probe -> venc	8 B	Start/refresh metadata subscription
FRAME	venc -> probe	52 B base, 64 B with trailer	Per-frame timing + RTP sequence info, plus optional encoder telemetry
SYNC_REQ	probe -> venc	16 B	NTP-style clock offset request
SYNC_RESP	venc -> probe	32 B	Clock offset response (t1, t2, t3)

All messages share a common 6-byte header: 4-byte magic (0x52545053 = "RTPS"), 1-byte version, 1-byte message type. Fields are network byte order.

Subscription expires after 5 seconds without any probe message. Both SUBSCRIBE and SYNC_REQ refresh the expiry timer.

When Star6E adaptive encoder control is enabled, FRAME appends a 12-byte trailer carrying frame_size_bytes, frame_type, qp, complexity, scene_change, idr_inserted, and frames_since_idr. Maruko and timing-only Star6E runs keep sending the original 52-byte frame.

Link-control / FEC usage:

RTP video keeps using outgoing.server as usual.
Set outgoing.sidecarPort to expose sidecar metadata on a separate UDP port.
Base timing fields are available whenever the sidecar is enabled.
The extra encoder trailer requires Star6E with video0.scene_threshold>0.
The sender tracks one active sidecar subscriber at a time; the most recent probe or consumer to subscribe receives the frame metadata.

Reference Probe

A host-native reference probe is included at tools/rtp_timing_probe.c. It listens for RTP on one port and communicates with the venc sidecar on another, correlating frames by (SSRC, RTP timestamp).

Build (no cross-compiler needed):

make rtp_timing_probe

Usage:

./rtp_timing_probe --venc-ip <device-ip> [--rtp-port 5600] [--sidecar-port 6666] [--stats]

Without --stats, the probe outputs tab-separated frame records to stdout (one line per frame) suitable for piping to analysis tools. The TSV includes columns for all timing fields, sequence numbers, gaps, intervals, estimated latency, and optional encoder-feedback fields when the sidecar trailer is present. For timing-only frames, the encoder-feedback columns print -.

With --stats, a summary is printed to stderr on exit:

=== Timing Probe Summary ===

Duration:             20.0 s
Frames:               936 (46.8 fps)
RTP packets:          8484 (9.1 avg/frame)
RTP gaps:             0

--- Send spread (frame_ready -> last_pkt_send) ---
  Mean:    294 us
  P50:     265 us
  P95:     331 us
  P99:     1710 us

--- Encode duration (capture -> frame_ready) ---
  Mean:    4254 us

--- Clock sync ---
  Samples:  8
  Best RTT: 347 us

The probe uses burst-then-coast clock synchronisation: 8 fast samples at 200 ms intervals, then one sample every 10 seconds. Only the sample with the lowest RTT is used for offset estimation.

Sidecar Overhead

At 90 fps with an active subscriber:

venc -> probe: ~90 frame packets/s (52 B each) + sync responses
probe -> venc: ~0.5 subscribe/s + ~0.1 sync/s
Bandwidth: ~40 kbps total (both directions)
CPU: single poll() per frame + one sendto() per frame

When no probe is subscribed, the sidecar socket exists but no packets are sent.

Sensor Unlock

IMX415 and IMX335 sensors support high-FPS modes (90/120fps) via a register unlock sequence applied before pipeline initialization. This is enabled by default (sensor.unlock_enabled=true) with preset values for IMX415.

For different sensors, adjust sensor.unlock_cmd, sensor.unlock_reg, and sensor.unlock_value in the config file or via the API before a restart.

See documentation/SENSOR_UNLOCK_IMX415_IMX335.md for register details.

Sensor Driver Sources

Full sensor driver source code is available in the sensors-src/ submodule (from OpenIPC/sensors). This includes drivers for IMX335, IMX415, GC4653, and other SigmaStar Infinity6E sensors.

# Fetch the sensor sources (not cloned by default)
git submodule update --init sensors-src

# Driver sources for Infinity6E
ls sensors-src/sigmastar/infinity6e/sensor/

Pre-built kernel modules (.ko) for IMX335 and IMX415 remain in sensors/.

Maruko IMX335 Sensor Modes

Custom Maruko driver in drivers/sensor_imx335_maruko.c (built via make -C drivers sensor):

Mode	Resolution	Max FPS	Verified	Init table
0	1920x1080	60	59fps	Star6E 120fps windowed
1	1920x1080	90	89fps	Star6E 120fps windowed

Deploy: scp sensor_imx335_maruko.ko root@device:/lib/modules/5.10.61/sigmastar/sensor_imx335_mipi.ko

The driver uses no-op pCus_poweroff (sensor stays powered from boot) and a VTS 120% cap to prevent AE from dropping FPS in low light. A delayed MI_SNR_SetFps kick after ~1s fixes cold-boot FPS lock.

Deployment Testing

For the current Star6E bench, use the direct helper against the production /etc/venc.json workflow:

./scripts/star6e_direct_deploy.sh cycle
./scripts/star6e_direct_deploy.sh status

This deploys /usr/bin/venc, waits for the HTTP API, and captures /tmp/venc.log.

Use remote_test.sh for bounded CLI runs such as sensor-mode discovery, max-FPS sweeps, and dedicated test binaries:

./scripts/remote_test.sh --help

Run the API test suite against a live device after venc is already running:

./scripts/api_test_suite.sh 192.168.1.13 80

Verify the single-instance pidfile + flock gate by trying to launch a second venc while one is already running on the device:

./scripts/test_pidfile_lock.sh root@192.168.1.13

The second launch must exit with rc=1 and the "venc already running" banner; the first instance must remain alive.

Scene-change IDR control is configured through video0.scene_threshold in /etc/venc.json. Leave video0.scene_threshold=0 for baseline behavior.

Web Dashboard

venc includes a built-in web dashboard served at the root URL (/). Open http://<device-ip>/ in any browser to access it.

Settings Tab

All configuration fields across 12 sections (System, Sensor, ISP, Image, Video, Outgoing, Audio, FPV, IMU, Recording, Adaptive Encoder Control, Debug) with:

Collapsible sections — start collapsed for a clean overview
Live/Restart badges — green for immediate changes, orange for restart-required
Tooltips — hover any field label for a description
Change tracking — modified fields highlighted; Apply only sends changes
Apply Changes — applies all modified fields via the API
Save & Restart — applies changes then triggers pipeline reinit
Restore Defaults — reloads on-disk config and resets the form

API Reference Tab

Documentation for all HTTP endpoints with descriptions and example responses, grouped by category: Configuration, Encoder Control, ISP & Image Quality, Recording, and Dual-Stream.

Image Quality Tab

Direct access to 62 SigmaStar ISP parameters organized by category.

Parameter Categories — expandable sections with clickable parameter chips. Multi-field parameters (colortrans, obc, demosaic, false_color, crosstalk, r2y, wdr_curve_adv) show sub-field chips for individual field access.

Expanded Editor — click a multi-field parameter to open an inline form with all sub-fields pre-filled from live ISP values. Array fields (e.g., colortrans 3x3 matrix) render as editable grids. Changed fields highlight and Apply All writes only the modified fields.

Export / Import — save all IQ parameters as a timestamped JSON file, or import a previously saved file to restore tuning. Partial imports are supported — only the parameters present in the JSON are applied, leaving others untouched.

# Export current IQ state
curl http://<device>/api/v1/iq > my_tuning.json

# Import (full or partial)
curl -X POST -H "Content-Type: application/json" \
  -d @my_tuning.json http://<device>/api/v1/iq/import

# Partial import example — only set specific params
echo '{"lightness":{"value":75},"demosaic":{"fields":{"dir_thrd":30}}}' | \
  curl -X POST -H "Content-Type: application/json" -d @- http://<device>/api/v1/iq/import

IQ Dot-Notation API

Multi-field parameters support dot-notation for individual field access:

# Set a single field
curl "http://<device>/api/v1/iq/set?colortrans.y_ofst=200"

# Set an array field (comma-separated)
curl "http://<device>/api/v1/iq/set?colortrans.matrix=23,45,9,1005,987,56,56,977,1015"

# Query shows all fields
curl http://<device>/api/v1/iq
# Returns: "colortrans":{"enabled":true,"value":200,"fields":{"y_ofst":200,"u_ofst":0,"v_ofst":0,"matrix":[23,45,...]}}

Legacy single-value set (?colortrans=200) still works for backward compatibility.

Status Bar

The top telemetry bar shows version, backend type, live FPS (auto-refreshes every 2s), recording status indicator, and an Export Config button to download the full configuration as JSON.

IMU (BMI270 gyro module)

The BMI270 driver is compiled into the binary on both backends but disabled by default (imu.enabled = false). When enabled, it samples gyro+accel via the hardware FIFO at 200 Hz, drains per video frame, and hands samples to a caller-supplied push callback.

The previous EIS consumer (gyroglide crop-based stabilization) was removed in 0.8.0 — see HISTORY.md for the rationale and documentation/EIS_INTEGRATION_PLAN.md for what a future replacement (LDC-warp Phase C) would look like. The push callback in both star6e_pipeline.c and maruko_pipeline.c is currently a stub that discards samples; a future telemetry export, sidecar gcsv logging, or an HTTP /api/v1/imu peek would slot in there.

Maruko ordering caveat. On Maruko, IMU init must run before MI_VENC_StartRecvPic (i.e. before bind_maruko_pipeline()) because the auto-bias loop blocks the main thread for ~2 s. Empirically, blocking the main thread for 2 s after StartRecvPic leaves the VENC fd in a state where poll() never returns POLLIN and the stream loop never progresses. Star6E does not exhibit this — IMU init can stay post-VENC there.

To enable the IMU for development:

{
  "imu": {
    "enabled": true,
    "i2cDevice": "/dev/i2c-1",
    "i2cAddr": "0x68",
    "sampleRateHz": 200,
    "gyroRangeDps": 1000,
    "calFile": "/etc/imu.cal",
    "calSamples": 400
  }
}

Restart venc. The 2-second auto-calibration runs at startup — hold the board still during it. With no consumer wired up, samples are discarded after the per-frame drain (~negligible CPU).

Name		Name	Last commit message	Last commit date
Latest commit History 85 Commits
.agents/skills		.agents/skills
.claude		.claude
.codex		.codex
.github		.github
config		config
documentation		documentation
drivers		drivers
include		include
iq-profiles		iq-profiles
lib		lib
libs		libs
scripts		scripts
sdk/ssc338q		sdk/ssc338q
sensors-src @ c225b92		sensors-src @ c225b92
sensors		sensors
src		src
tests		tests
tools		tools
vendor-libs/maruko		vendor-libs/maruko
web		web
.gitignore		.gitignore
.gitmodules		.gitmodules
AGENTS.md		AGENTS.md
CLAUDE.md		CLAUDE.md
HISTORY.md		HISTORY.md
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
VERSION		VERSION

Folders and files

Latest commit

History

Repository files navigation