System Architecture

rust-srec is an automated stream recorder built around a clear separation of concerns:

• A control plane (REST API + configuration + orchestration)
• A data plane (live status detection + downloads + danmu + post-processing)
• A persistence layer (SQLite + filesystem outputs)

It is implemented as a set of long-running Tokio services managed by the runtime ServiceContainer.

High-level topology

Arrows between runtime services show logical event routes. ServiceContainer implements that wiring with broadcast subscriptions, bounded queues, and handler tasks rather than direct service coupling.

Three ownership boundaries are important in this topology:

• ServiceContainer is the composition root and event-wiring layer, not the owner of domain state.
• StreamMonitor detects and filters platform status; SessionLifecycle exclusively owns the in-memory session state machine and durable start/end decisions.
• A live event starts the download path first. Danmu collection starts only after the download manager's start_with_slot call returns a real download ID.

Runtime root: ServiceContainer

The ServiceContainer (in rust-srec/src/services/container.rs) wires everything together:

• Initializes repositories and services (DB, config cache, managers)
• Starts background tasks (scheduler actors, pipeline workers, outbox flushers)
• Subscribes to event streams and forwards events between services
• Owns the CancellationToken used for graceful shutdown

This gives the project one clear place to reason about lifecycle, dependencies, and shutdown order.

Core components (what each one actually does)

ConfigService (configuration + hot reload)

ConfigService is the configuration control plane. It loads and merges a 4-level hierarchy:

1. Global defaults
2. Platform configuration
3. Template configuration
4. Streamer-specific overrides

It also caches merged results and broadcasts ConfigUpdateEvent so runtime services can respond to changes without a restart.

See also: Configuration

StreamerManager (runtime state source of truth)

StreamerManager maintains the in-memory streamer metadata used by orchestration and downloads, with write-through persistence to SQLite.

Important correctness detail: on startup it performs restart recovery by resetting any streamers left in Live back to NotLive, so the normal NotLive → Live edge can trigger downloads again.

Scheduler (actor model orchestration)

The scheduler is a supervisor that manages self-scheduling actors:

• StreamerActor: owns the timing and state loop for one streamer
• PlatformActor: coordinates batch detection for batch-capable platforms
• Supervisor: handles actor lifecycle, restart tracking, and shutdown reporting

Actors call into StreamMonitor for real status checks; the scheduler also reacts to configuration events to spawn/stop actors dynamically.

StreamMonitor (detect + filter + outbox)

StreamMonitor is the data-plane detector. It:

• Resolves platform information and checks live status
• Applies filters (time/keyword/category, etc.)
• Delegates session changes to SessionLifecycle
• Emits MonitorEvent via a DB-backed outbox for consistency

Outbox pattern: Monitor events are written in the same DB transaction as state/session updates, then a background task flushes the outbox to a Tokio broadcast channel. This reduces the chance of “state changed but event lost” during crashes or restarts.

SessionLifecycle (single owner of session state)

SessionLifecycle owns the recording state machine, including hysteresis and terminal-cause classification. Fresh starts and durable ends commit their required database changes before broadcasting Started or Ended. Hysteresis Ending and Resumed are in-memory transitions: their audit rows are best-effort, and the session end_time remains unset until the lifecycle reaches Ended. Download terminal events feed back into this service. Ended drives session-complete pipelines, danmu cleanup, and download bookkeeping; a resumed Started restarts the same session.

DownloadManager (downloads + engine abstraction)

The download manager owns:

• Concurrency limits (including extra slots for high priority downloads)
• Failure classification and circuit breakers keyed by engine type, configuration, and optional streamer scope
• Failure/rejection events and retry-after hints; scheduler actors decide when to check again and re-enter the download-start path
• Engine abstraction:
  • External processes: ffmpeg, streamlink
  • In-process Rust engine: mesio

It emits DownloadManagerEvent for lifecycle, segment boundaries, and (optionally) progress.

For persisted session segments, the backend keeps three separate timestamps:

• created_at: when the segment started recording
• completed_at: when the segment finished recording
• persisted_at: when the segment metadata row was stored in SQLite

DanmuService (chat capture)

Danmu collection is session-scoped but writes files per segment:

• A websocket connection stays alive for the session
• Segment boundaries (from download events) open/close danmu files (e.g. XML)
• Danmu events are forwarded to the pipeline for paired/session coordination

PipelineManager (job queue + DAG + worker pools)

The pipeline manager is the post-processing engine:

• Maintains a DB-backed job queue (with recovery on restart)
• Executes a DAG pipeline model (fan-in / fan-out)
• Uses separate worker pools for CPU-bound and IO-bound processors
• Coordinates multi-stage triggers:
  • Segment pipelines (single output file)
  • Paired-segment pipelines (video + danmu for the same segment index)
  • Session-complete pipelines (once all segments are complete)

See also: DAG Pipeline

NotificationService (event fan-out)

Notifications subscribe to monitor/download/session/pipeline events and deliver them to configured channels (Discord / Email / Gotify / Telegram / Webhook), with retry, circuit breakers, and dead-letter persistence. Optional browser Web Push delivery is handled by WebPushService.

See also: Notifications

Key flows

Recording lifecycle (end-to-end)

API request flow (control plane)

Most protected routes use JWT middleware when JWT is configured. The full health and readiness handlers validate bearer tokens themselves and return 401 when JWT authentication is not configured; liveness remains public. WebSocket, media, and stream-proxy routes use their documented query-parameter authentication paths.

Event-driven communication

Most cross-service coordination happens via Tokio broadcast channels.

Stream	Publisher	Typical consumers	Notes
ConfigUpdateEvent	ConfigService, StreamerManager	Scheduler, ServiceContainer	Drives actor changes, runtime reconfiguration, and cleanup
MonitorEvent	StreamMonitor	ServiceContainer, NotificationService	Emitted through the DB outbox (best-effort delivery under restarts)
DownloadManagerEvent	DownloadManager	Scheduler, NotificationService, ServiceContainer handlers	Handlers feed segments to PipelineManager and terminal outcomes to SessionLifecycle
SessionTransition	SessionLifecycle	ServiceContainer handlers, NotificationService	Ended drives cleanup and session pipelines; resumed Started restarts the same session
DanmuEvent	DanmuService	ServiceContainer handlers	Handlers feed segment pairing to PipelineManager and terminal signals to download/session handling
PipelineEvent	PipelineManager	NotificationService	Job lifecycle events for observability

About throttling

PipelineManager contains an optional throttling subsystem (ThrottleController) that can emit events and apply download concurrency adjustments if a DownloadLimitAdjuster is wired in.

Output-root write gate

The download manager runs an output-root write gate (in downloader::output_root_gate) that operates at the filesystem boundary, complementing the engine-level circuit breakers that operate at the network/process boundary. It exists so that a single filesystem failure (disk full, stale bind mount, lost permissions) does not cascade into dozens of per-streamer retries that would flood the logs and DB outbox.

Healthy ──(record_failure: pre-start ENOENT / runtime ENOSPC / startup probe)──► Degraded
                                                                                    │
                              (mark_healthy: next real ensure_output_dir succeeds)  │
Healthy ◄───────────────────────────────────────────────────────────────────────────┘

Key properties:

• Lock-free fast path. check() on a Healthy root is an atomic load plus a DashMap::get. No mutex on the hot path, no cost when there are no tracked failures.
• Single-flight cooldown via CAS. When a root is Degraded, only one caller per cooldown window (30s default) is allowed through to attempt the real create_dir_all. Other concurrent callers fast-reject with the cached error. Mirrors the half-open pattern in CircuitBreaker.
• No background probe task. The real ensure_output_dir call is the probe — the gate piggybacks on actual download attempts. A single one-shot probe runs at container startup to surface broken mounts from second zero.
• Recovery hook. On Degraded → Healthy transition the gate clears consecutive_error_count, disabled_until, and last_error for every streamer whose backoff was caused by the gate (filtered by the "output-root blocked:" prefix). The whole affected fleet cascades out of backoff on the same tick.
• One notification per transition. The Healthy → Degraded CAS is also what decides which caller emits the critical output_path_inaccessible notification, so users see exactly one alert per incident regardless of how many concurrent streamers are affected.

Exposed in /api/health as a single aggregated output-root component listing each Degraded root with its classified io::ErrorKind, rejected count, and staleness. See the notifications doc for the event shape and the Docker troubleshooting guide for the stale-mount failure mode.

Observability, health, and shutdown

• Logging uses tracing with a reloadable filter and log retention cleanup
• Health endpoints:
  • GET /api/health/live (no auth; suitable for container liveness)
  • GET /api/health and GET /api/health/ready require a valid bearer token and return 401 when JWT authentication is not configured
• Shutdown:
  • The standalone executable keeps SQLite, sockets, and recording files inside an OS-contained worker process. A dedicated parent thread observes termination signals and arms the absolute shutdown deadline even while startup or async runtime work is blocked. The watchdog remains armed through durable marker updates, terminal diagnostics, and parent process exit.
  • The ServiceContainer performs phased graceful shutdown inside that worker, keeping required event consumers alive until final segment facts are persisted.
  • SIGINT triggers graceful shutdown on all supported platforms; SIGTERM is additionally handled on Unix. The parent and the worker both register handlers, so a signal delivered straight to the worker — as KillMode=control-group or pkill does — runs the same graceful finalization as one relayed over the control pipe. A worker-local critical failure still fail-stops that worker; the parent then contains its descendants and retains recovery state.
  • A forced or crashed worker leaves a dirty-generation marker beside SQLite so the next launch reports that recovery may be required. Earlier recovery debt survives later clean generations; the marker is a detection mechanism, not artifact replay.
  • That marker keeps the oldest and newest unresolved generations plus a count of the ones in between, so a restart loop cannot grow it. Startup and exit messages report how many generations still owe recovery.