Producing short-form news updates for platforms like YouTube Shorts is a repetitive task. The process almost always follows the exact same sequence: take a raw news headline, expand it into a timed broadcast script, generate an anchor video, transcribe the speech, burn dynamic subtitles, composite news graphics and audio, and upload the final cut with optimized titles and tags.

Doing this manually takes anywhere from fifteen to thirty minutes per video clip. Even when using modern web tools for script generation and avatar synthesis, you end up acting as human middleware. You copy and paste text between browser tabs, wait for render queues, download MP4 files, open a desktop video editor to add lower thirds and ticker bars, and then manually upload the rendered asset through YouTube Studio.

As a two-person team, we wanted to eliminate this manual overhead. We built an automated pipeline in Python that turns a single headline into a published video Short with a single terminal command (python run_flow.py). We created this project as a portfolio demo for a synthetic news channel concept called Tachyon News, supporting both English and Roman Urdu scripts. In this guide, we walk through the architecture, share the core code snippets, and highlight the practical gotchas we ran into while stitching these systems together.

The Pipeline Architecture

Our pipeline coordinates four distinct stages:

  1. Text expansion and metadata generation: We use the Groq Python SDK with the llama-3.3-70b-versatile model. An initial language check determines whether the incoming text is English or Roman Urdu, after which the model generates a 60-to-80 word broadcast script and JSON-structured YouTube metadata (testgrok.py).
  2. Headless avatar generation: We use Playwright to automate video generation inside HeyGen (heygen_simple.py). The script loads a persistent browser session, picks the avatar and costume, inputs the script, triggers generation, and monitors progress until the download completes.
  3. Post-production compositing: Once the raw avatar video is downloaded, an editing script (heygen_downloads/script.py) transcribes the speech with faster-whisper, builds an Advanced SubStation Alpha (.ass) karaoke subtitle file, tracks speaker movement with OpenCV, and invokes FFmpeg to composite broadcast overlays, news tickers, audio ducking, and loudness normalization.
  4. Publishing: The final MP4 is published to YouTube as a Short using the Google YouTube Data API v3 (youtube_upload.py), handling OAuth token refreshes and resumable video uploads.

An orchestration script (run_flow.py) ties all four phases together into a clean command-line workflow.

Step 1: Script Expansion and Language Routing

A typical breaking news headline is only one or two sentences long. A YouTube Short, however, needs roughly fifteen to twenty seconds of speech to feel complete, which translates to sixty to eighty spoken words.

Before sending the headline to the language model, we inspect the input string in testgrok.py to identify whether the text is Roman Urdu or English. Rather than making a separate API call for classification, we check the ratio of common Roman Urdu function words:

def detect_language(text: str) -> str: urdu_words = [ "hai", "hain", "ka", "ki", "ke", "ko", "se", "mein", "par", "aur", "ya", "nahi", "tha", "thi", "the", "ho", "ga", "gi", "ne", "bhi", "koi", "kuch", "ab", "phir", "lekin", "kyun", "kya", "yeh", "wo", "ap", "aap", "main", "hum", "tum" ] words = text.lower().split() urdu_count = sum(1 for w in words if w in urdu_words) ratio = urdu_count / len(words) if words else 0 return "urdu" if ratio >= 0.15 else "english"

Once the language is identified, we call Groq with a tailored system prompt. The prompt enforces word-count boundaries, required channel greetings, and clean sign-offs without preamble. We also request JSON output for YouTube metadata in the same stage:

def generate_youtube_metadata(script: str, language: str) -> dict: system_prompt = ( "You are an expert YouTube SEO manager. Generate a catchy YouTube Short " "title (max 60 chars) and description with relevant hashtags. " "Output MUST be strictly in JSON format with two keys: 'title' and 'description'." ) response = client.chat.completions.create( model="llama-3.3-70b-versatile", messages=[ {"role": "system", "content": system_prompt}, {"role": "user", "content": f"Language: {language}\nScript: {script}"} ], max_tokens=300, temperature=0.7, response_format={"type": "json_object"} ) try: return json.loads(response.choices[0].message.content.strip()) except Exception: return { "title": f"Breaking News Update | {CHANNEL_NAME}", "description": f"{script}\n\n#{CHANNEL_NAME.replace(' ', '')} #Shorts" }

Using Groq gives us near-instant script and metadata generation (typically under 600 milliseconds), keeping the command-line interface responsive.

Step 2: Driving Video Synthesis with Playwright

For this demo, we used Playwright to automate video creation in HeyGen (heygen_simple.py). Using browser automation allowed us to select custom avatar outfits and rendering options available in the web dashboard without incurring high-tier dedicated enterprise API costs.

Automating modern web apps requires handling persistent sessions, dynamic dropdowns, and unexpected modal dialogs. To avoid authenticating on every run, we save and restore browser storage state:

context_options = {"accept_downloads": True} if os.path.exists("heygen_session.json"): context_options["storage_state"] = "heygen_session.json" context = await browser.new_context(**context_options) page = await context.new_page()

If the session has expired, the script pauses, prompts for a manual login in the browser window, and writes the refreshed cookies back to heygen_session.json.

During the generation flow, marketing popups and rating prompts frequently appear. We run a background monitor task to dismiss them:

async def monitor_and_close_dialogs(page): async def close_dialogs_loop(): global dialog_monitor_active while True: if not dialog_monitor_active: await page.wait_for_timeout(1000) continue dialogs = page.locator('[role="dialog"]') if await dialogs.count() > 0: for i in range(await dialogs.count()): dialog = dialogs.nth(i) text = await dialog.inner_text() if "Download" in text: dialog_monitor_active = False continue btn = dialog.locator('button[type="button"].tw-absolute').first if await btn.is_visible(timeout=500): await btn.click(force=True) await page.wait_for_timeout(1000) asyncio.create_task(close_dialogs_loop())

Once the script triggers video generation, we poll the recent creations list until the percentage indicator disappears. We then simulate a hover event to reveal the actions menu, click the download button, and await the file stream.

Step 3: Transcription and Karaoke Subtitles

Raw talking-head videos look static on short-form platforms. To make the output feel like a professional news broadcast, our post-processing module in heygen_downloads/script.py adds synchronized karaoke subtitles.

First, we extract word-level timestamps using faster-whisper on CPU with int8 quantization:

from faster_whisper import WhisperModel def transcribe(video_path, lang=None): model = WhisperModel("base", device="cpu", compute_type="int8") kwargs = {"word_timestamps": True} if lang == "urdu": kwargs["language"] = "ur" elif lang == "en": kwargs["language"] = "en" segments, _ = model.transcribe(video_path, **kwargs) words = [] for seg in segments: for w in (seg.words or []): words.append({ "word": w.word.strip(), "start": round(w.start, 3), "end": round(w.end, 3), }) return words

With word boundaries in hand, we group words into chunks of four and generate an Advanced SubStation Alpha (.ass) file. Using the {\kf<duration>} tag allows us to highlight the currently spoken word in yellow (&H00FFFF00&) while keeping upcoming or previous words in white:

def make_karaoke_ass(words, chunk_size, path): lines = [] for i in range(0, len(words), chunk_size): chunk = words[i:i + chunk_size] start_ts = format_timestamp(chunk[0]["start"]) end_ts = format_timestamp(chunk[-1]["end"]) parts = [] for w in chunk: dur_cs = max(1, round((w["end"] - w["start"]) * 100)) parts.append( f"{{\\c&H00FFFF00&\\kf{dur_cs}}}{w['word'].upper()}" f"{{\\c&H00FFFFFF&}} " ) text = "".join(parts).rstrip() lines.append(f"Dialogue: 0,{start_ts},{end_ts},Karaoke,,0,0,0,,{text}") with open(path, "w", encoding="utf-8") as f: f.write(ASS_HEADER + "\n".join(lines) + "\n")

Because .ass styling is processed directly by FFmpeg via libass, rendering is fast and produces crisp text without needing complex image overlays.

Step 4: Video Compositing and Audio Ducking in FFmpeg

The final render stage in heygen_downloads/script.py combines visual styling, motion framing, and broadcast audio compliance in a single FFmpeg execution pass.

Dynamic Face Zoom

To prevent the avatar from looking like a flat photograph, we use OpenCV's Haar cascade face detector across the first five seconds of footage to locate the anchor's face coordinates (cx, cy). We then feed those coordinates into an FFmpeg zoompan filter to execute a subtle push-in:

zf = "min(zoom+0.0002,1.05)" px = f"if(eq(on,1),{cx},x)" py = f"if(eq(on,1),{cy},y)" zoom_filter = f"zoompan=z='{zf}':x='{px}':y='{py}':d=1:s={width}x{height}:fps=25"

News Graphic Overlays and Scrolling Ticker

We build an extensive visual filter graph containing:

  • A dark translucent top bar with the channel title and a red "LIVE" badge.
  • An animated lower third that slides in at 1.5 seconds and fades out at 6.0 seconds using FFmpeg's between(t, 1.5, 6.0) expression.
  • A scrolling breaking news ticker across the bottom of the screen with x=w-mod(t*speed, w+tw).
  • A call-to-action animation sliding in from the right edge during the last four seconds.

Audio Normalization and Ducking

News videos must have consistent audio levels so speech is never muffled by background music. We normalize dialogue to -16 LUFS (the standard for internet broadcast audio) and lower the background music level:

audio_cmd = [ "ffmpeg", "-y", "-i", working_video, "-i", bg_music_path, "-filter_complex", ( "[0:a]loudnorm=I=-16:TP=-1.5:LRA=11,volume=1.0[speech];" "[1:a]volume=0.12,aloop=loop=-1:size=2e+09[music];" "[speech][music]amix=inputs=2:duration=first:dropout_transition=3[aout]" ), "-map", "[aout]", "-c:a", "aac", "-b:a", "192k", "-t", str(duration), audio_output ]

Step 5: Uploading as a YouTube Short

Once FFmpeg finishes encoding, run_flow.py passes the completed file and generated metadata to youtube_upload.py. We use the Google YouTube Data API v3 with a resumable media upload:

media = MediaFileUpload( abs_path, mimetype="video/mp4", resumable=True, chunksize=1024 * 1024 ) request = youtube.videos().insert( part="snippet,status", body={ "snippet": { "title": title, "description": description, "tags": ["TachyonNews", "Shorts", "BreakingNews"], "categoryId": "25", }, "status": { "privacyStatus": "public", "selfDeclaredMadeForKids": False, }, }, media_body=media, ) response = None while response is None: status, response = request.next_chunk() if status: print(f"Upload progress: {int(status.progress() * 100)}%")

OAuth credentials are saved locally to youtube_token.json after the initial consent screen, allowing subsequent runs to refresh tokens silently.

Gotchas and Edge Cases Handled in Code

Building an end-to-end automation pipeline exposed several subtle bugs that only appear during full execution:

  1. Radix UI custom select elements: Playwright's default selector strategies often fail on modern React component libraries like Radix UI. In heygen_simple.py, dropdown menus did not always respond to standard .click() calls. We added JavaScript DOM tree inspection fallbacks that walk child text nodes to find and dispatch events directly to the trigger elements.
  2. Modal collision during download: Our background dialog monitor was so effective at closing marketing popups that it initially closed the genuine download confirmation modal. We resolved this by checking whether the dialog text contained "Download" and setting a global flag dialog_monitor_active = False before starting the download handshake.
  3. Font path escaping in FFmpeg: On different operating systems, font file paths passed to drawtext require distinct escaping rules. Unescaped Windows drive colons (C:) or unescaped backslashes cause FFmpeg to fail silently or crash the filter chain. In heygen_downloads/script.py, we normalize paths with path.replace("\\", "/").replace(":", "\\\\:").
  4. Fallback rendering pipeline: Complex FFmpeg filter graphs combining zoompan, xfade, libass, and dynamic text can fail if an input file has unusual pixel aspect ratios or missing face detection frames. If the master render exits with a non-zero status code, our script catches the error and executes a simplified fallback render (cmd2), guaranteeing an edited video output.
  5. JSON parsing resilience: LLMs can occasionally append commentary or Markdown backticks around JSON outputs. We configured Groq with response_format={"type": "json_object"} and wrapped parsing in defensive try/except blocks with structured defaults.

Trade-offs and What We Would Do Differently

Because this was designed as a lightweight portfolio prototype, we made conscious design trade-offs:

  • Browser automation versus official APIs: Controlling a web application via Playwright is fragile. Any DOM class rename or layout change can break locators. In a client production environment, we would use official REST or webhook APIs exclusively, even if that means compromising on avatar customization options.
  • Local rendering versus task queues: Running faster-whisper, OpenCV, and FFmpeg on the local machine consumes substantial CPU resources and blocks the CLI. In production, we would decouple the architecture: an ingestion webhook drops jobs into a message queue (such as Redis or SQS), and containerized worker nodes handle transcription and FFmpeg encoding in parallel.
  • Rule-based language detection versus embeddings: Our fifteen-percent word-matching heuristic for Roman Urdu is fast and requires zero external model downloads. However, for short headlines with heavy technical jargon, it can misclassify. Using a dedicated compact classifier would provide higher confidence on edge cases.

Summary

  • Combining Groq with llama-3.3-70b-versatile enables sub-second script expansion and SEO metadata generation with strict duration control.
  • Headless browser automation can bridge the gap when APIs are unavailable, provided you handle persistent sessions, custom component menus, and aggressive popup dismissal.
  • Post-processing with faster-whisper and .ass formatting allows you to build dynamic karaoke captions with minimal rendering overhead.
  • FFmpeg remains unmatched for single-pass compositing, handling face-tracking zooms, lower thirds, scrolling tickers, and EBU R128 loudness normalization together.

If you have built automated video pipelines or handled browser-based synthetic media workflows, what strategies do you use to make your render and upload stages resilient against unexpected UI changes?

The full write-up of this project is on our site: https://sanedevs.com/projects/video-generation-pipeline.html