- 800+
- YouTube channels automated
- 80,000+
- Videos per day
- 99%+
- Uptime
- 6–10
- Person team replaced
The constraint
Architecture
Decisions
Containerised per-workload execution on Kubernetes rather than a fixed worker fleet.
Video and audio processing is bursty and uneven — some jobs are 30 seconds, some are 20 minutes. Fixed workers either sit idle or become the bottleneck. Scaling deployments per workload meant the platform could go from a handful of containers to thousands of Kubernetes deployments without redesigning the execution model.
Account isolation enforced at the scheduler, not in application code.
With hundreds of accounts sharing one pipeline, one noisy tenant can starve every other one. Putting per-account concurrency limits in shared state — rather than trusting each service to behave — meant fairness held even when a single client queued thousands of jobs at once.
Publishing decoupled from generation.
Generation and upload fail for completely different reasons: model refusals and encoding errors on one side, API quotas and auth expiry on the other. Keeping them as separate stages with independent retry meant a publishing outage never wasted generated video, and a generation failure never blocked the publishing queue.
Failure modes
- worker crash mid-job
- Job stays claimed in the processing set and is recovered on the next worker start rather than silently disappearing
- upstream model refusal
- Detected as a distinct failure class from a transport error, then retried through a prompt-rewrite path instead of burning the retry budget
- per-account quota exhaustion
- Account is throttled at the scheduler; other accounts keep running at full rate
- frequent platform-wide outages
- Root cause was a single-point architecture; replaced with a distributed design, which is what took the platform to 99%+
Outcome
Stack