Articles

Your AI Video's Subtitles Drift More the Longer It Runs: Stop Fixing the Aligner — the Bug Is 'Generate the Whole Audio First'

Building explainer videos with TTS + auto subtitles: the first 30 seconds were perfectly synced, then the subtitles drifted further and further behind, and the tail of the .srt degraded to 00:00:00 timestamps. The first instinct — patch the alignment (normalize numbers, loosen similarity thresholds) — treats symptoms. This postmortem shows the real cause: any 'whole-clip TTS + whisper transcription + post-hoc matching' pipeline drifts by construction, because it hands timing — which the generation step could produce directly — to a lossy reconstruction chain. The fix: sentence-level TTS with sample-accurate concatenation, where each subtitle's timestamp is the running sum of real wav sample counts. Sentence boundaries reset the error, so drift is physically impossible. Includes splitting rules, caching design, and the limits of the approach.