# SkillPatch skill: demo-video-pipeline Programmatically build narrated demo videos of web apps by orchestrating Playwright screen recording, Marp slide rendering, Sarvam AI text-to-speech narration, and FFmpeg transcoding/concatenation into a single MP4. Use this whenever the user wants to auto-generate a product walkthrough, UI demo, narrated screen-recording, or slide+voiceover video from a URL and a script, or needs help wiring together Playwright, Marp, TTS, and FFmpeg into one rendering pipeline. You (the agent) were given this URL and asked to use this skill. This is a **public** skill — no SkillPatch account, API key, or prior setup is required. Two ways to use it: **1. Use it right now** — the skill's complete file tree (SKILL.md + all reference files) is inlined below; read `SKILL.md` first, then follow it, consulting the other files as it directs. **2. Install the exact package onto disk** (recommended if you can run a shell — this reproduces the skill byte-for-byte, including any binary assets that can't be inlined): ```bash mkdir -p .claude/skills/demo-video-pipeline curl -sSL https://skillpatch.dev/install_skill/demo-video-pipeline | tar -xz -C .claude/skills/ ``` (`.claude/skills/` is Claude Code's convention; use whatever directory your agent loads skills from.) --- ## Skill files (2) - `SKILL.md` - `README.md` ### `SKILL.md` ````markdown --- name: demo-video-pipeline description: Programmatically build narrated demo videos of web apps by orchestrating Playwright screen recording, Marp slide rendering, Sarvam AI text-to-speech narration, and FFmpeg transcoding/concatenation into a single MP4. Use this whenever the user wants to auto-generate a product walkthrough, UI demo, narrated screen-recording, or slide+voiceover video from a URL and a script, or needs help wiring together Playwright, Marp, TTS, and FFmpeg into one rendering pipeline. license: MIT compatibility: Requires Python 3.10+, Playwright with Chromium installed, FFmpeg on PATH, Node.js/npx (for @marp-team/marp-cli), and network access to the Sarvam AI TTS API. metadata: author: Ramkumar R version: "1.0.0" --- # Demo Video Pipeline ## 1. Skill Overview & Objective This skill covers the end-to-end design, orchestration, and implementation of automated video generation pipelines using **Playwright** for browser screen recording, **Marp** for presentation slide rendering, **Sarvam AI** for neural text-to-speech narration, and **FFmpeg** for media transcoding, stream multiplexing, and final video concatenation. ### Key Objectives * Programmatically capture high-fidelity, resolution-locked web interactions using Playwright's native recording contexts. * Convert static slide presentations (Markdown) into uniform video streams. * Synthesize localized voiceover narrations and align audio durations with visual frames. * Transcode and multiplex disparate media streams (WebM video, WAV audio, PNG images) into standardized H.264/AAC MP4 clips. * Concatenate intermediate video segments seamlessly without dropped frames, audio sync drift, or aspect ratio distortion. --- ## 2. Core Technologies & Dependencies | Tool / Library | Role in Pipeline | Key Functionality | | :--- | :--- | :--- | | **Playwright** | Browser Automation & Screen Capture | Spawns headless Chromium, drives UI actions (`goto`, `click`, `fill`, `scroll`), records native `.webm` video streams inside a strict `1280x720` viewport. | | **FFmpeg** | Media Processing & Transcoding | Scales video streams, generates synthetic silent audio tracks, multiplexes WAV/WebM streams, and concatenates segment MP4 files via the `concat` demuxer. | | **Marp CLI** | Presentation Slide Rendering | Converts Marp-flavored Markdown with custom CSS into resolution-locked (`1280x720`) PNG slide frames. | | **Sarvam AI TTS** | Voiceover Synthesizer | Converts narration scripts into base64-encoded PCM WAV audio chunks. | | **Gemini AI** | Script & Workflow Planning | Translates user prompts and target URLs into structured JSON plans (`DemoConfig`). | | **Pydantic v2** | Schema Validation | Validates plan integrity, step types, and Playwright action structures. | --- ## 3. Key Concepts & Architecture ### System Architecture ``` ┌───────────────────────────┐ │ Gemini AI Planner │ │ (Translates URL + Prompt) │ └─────────────┬─────────────┘ │ Structured Plan │ ┌──────────────────────────────┼──────────────────────────────┐ ▼ ▼ ▼ ┌─────────────────────────┐ ┌─────────────────────────┐ ┌──────────────────────────┐ │ Marp Slide Engine │ │ Sarvam TTS Synthesizer │ │ Playwright UI Bot │ │ Markdown -> PNG Frames │ │ Script -> WAV Audio │ │ Browser Recording (WebM) │ └────────────┬────────────┘ └────────────┬────────────┘ └────────────┬─────────────┘ │ │ │ PNG Images WAV Audio WebM Videos │ │ │ └──────────────────────────────┼──────────────────────────────┘ ▼ ┌─────────────────────────────┐ │ FFmpeg Media Stitcher │ │ Transcodes, Multiplexes, & │ │ Concatenates to final.mp4 │ └─────────────────────────────┘ ``` ### Key Architectural Concepts 1. **Resolution & Canvas Standardization**: All pipeline assets (slides, browser viewports, intermediate video clips) are strictly bound to a `1280x720` resolution at `25 fps`. This eliminates scale mismatches during video concatenation. 2. **Defensive Browser Automation**: Playwright actions execute with automatic selector visibility checks (`wait_for_selector(state="visible")`) and post-action buffer timeouts. Missing selectors are gracefully skipped rather than crashing the recording session. 3. **Audio-Driven Frame Pacing**: Slide video durations are driven by the exact runtime of their paired voiceover `.wav` file, calculated via pure-Python PCM wave header analysis. 4. **Stream Normalization for Concat Stability**: Every intermediate clip processed by FFmpeg is normalized to identical stream parameters: * **Video**: `libx264`, `yuv420p` pixel format, `1280x720` resolution, `25 fps`. * **Audio**: `aac` codec, `192 kbps`, `44100 Hz` sample rate, stereo channels (`-ac 2`). * **Missing Audio**: Synthetic silent audio (`anullsrc`) is dynamically injected for steps without narration to ensure the FFmpeg `concat` demuxer never encounters missing audio streams. --- ## 4. Code Implementation Examples ### Example 1: Playwright Bot with Native Video Capture & Defensive Dispatcher ```python import shutil from pathlib import Path from typing import List, Optional from playwright.sync_api import sync_playwright, Page, Browser, BrowserContext from pydantic import BaseModel class UIStepAction(BaseModel): action: str # "goto", "click", "fill", "wait", "hover", "press" selector: Optional[str] = None value: Optional[str] = None url: Optional[str] = None wait_time: Optional[float] = None class PlaywrightBot: """Orchestrates browser automation and captures native WebM video recordings.""" def __init__(self, viewport_width: int = 1280, viewport_height: int = 720, headless: bool = True): self.width = viewport_width self.height = viewport_height self.headless = headless def execute_ui_steps(self, actions: List[UIStepAction], output_video_path: Path, temp_video_dir: Path) -> Path: temp_video_dir.mkdir(parents=True, exist_ok=True) output_video_path.parent.mkdir(parents=True, exist_ok=True) with sync_playwright() as p: browser: Browser = p.chromium.launch(headless=self.headless, args=["--disable-dev-shm-usage"]) context: BrowserContext = browser.new_context( viewport={"width": self.width, "height": self.height}, record_video_dir=str(temp_video_dir), record_video_size={"width": self.width, "height": self.height} ) page: Page = context.new_page() page.set_default_timeout(10000) try: for i, action in enumerate(actions): self._dispatch_action(page, action, i) page.wait_for_timeout(1000) # Visual buffer delay finally: raw_video_path = None if page.video: raw_video_path = page.video.path() context.close() browser.close() if raw_video_path and Path(raw_video_path).exists(): shutil.move(raw_video_path, output_video_path) return output_video_path raise RuntimeError("Playwright closed without generating a video file.") def _dispatch_action(self, page: Page, action: UIStepAction, index: int) -> None: act = action.action # Defensive selector check: skip missing selectors gracefully if act in ("click", "fill", "hover", "press") and action.selector: try: page.wait_for_selector(action.selector, state="visible", timeout=5000) except Exception: return # Skip rather than failing the entire pipeline if act == "goto" and action.url: try: page.goto(action.url, wait_until="networkidle") except Exception: page.goto(action.url, wait_until="load") elif act == "click" and action.selector: page.click(action.selector) elif act == "fill" and action.selector and action.value: page.fill(action.selector, action.value) elif act == "hover" and action.selector: page.hover(action.selector) elif act == "press" and action.selector and action.value: page.press(action.selector, action.value) elif act == "wait": page.wait_for_timeout(int((action.wait_time or 1.0) * 1000)) ``` --- ### Example 2: Pure-Python WAV Header Duration Analysis & FFmpeg Slide Transcoder ```python import subprocess import wave from pathlib import Path class MediaStitcher: """Handles FFmpeg processing, image looping, audio multiplexing, and concat demuxing.""" @staticmethod def get_audio_duration(wav_path: Path) -> float: """Extracts exact PCM WAV duration in seconds using Python's built-in wave module.""" with wave.open(str(wav_path), "rb") as wav_file: frames = wav_file.getnframes() rate = wav_file.getframerate() if rate == 0: raise ValueError("Audio sample rate is 0.") return frames / float(rate) def create_slide_video(self, image_path: Path, audio_path: Path, output_path: Path) -> Path: """Loops a static PNG slide image for the exact duration of the audio narration.""" duration = self.get_audio_duration(audio_path) duration_str = f"{duration:.3f}" cmd = [ "ffmpeg", "-y", "-loop", "1", "-framerate", "25", "-i", str(image_path), "-i", str(audio_path), "-c:v", "libx264", "-t", duration_str, "-pix_fmt", "yuv420p", "-r", "25", "-s", "1280x720", "-c:a", "aac", "-b:a", "192k", "-ar", "44100", "-ac", "2", str(output_path) ] subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) return output_path ``` --- ### Example 3: UI Recording Transcoding with Letterbox Padding & Synthetic Silence ```python def create_ui_video(self, video_path: Path, audio_path: Path | None, output_path: Path) -> Path: """Transcodes WebM recording, applies letterboxing, and multiplexes voiceover or silent audio.""" filter_complex = "[0:v]scale=1280:720:force_original_aspect_ratio=decrease,pad=1280:720:(ow-iw)/2:(oh-ih)/2,setsar=1[v]" cmd = ["ffmpeg", "-y", "-i", str(video_path)] if audio_path and audio_path.exists(): cmd.extend(["-i", str(audio_path)]) cmd.extend([ "-filter_complex", filter_complex, "-map", "[v]", "-map", "1:a", "-shortest" ]) else: # Generate synthetic silent audio track for stream uniformity cmd.extend([ "-f", "lavfi", "-i", "anullsrc=r=44100:cl=stereo", "-filter_complex", filter_complex, "-map", "[v]", "-map", "1:a", "-shortest" ]) cmd.extend([ "-c:v", "libx264", "-pix_fmt", "yuv420p", "-r", "25", "-c:a", "aac", "-b:a", "192k", "-ar", "44100", "-ac", "2", str(output_path) ]) subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) return output_path ``` --- ### Example 4: FFmpeg Concat Demuxer Pipeline ```python def concatenate_segments(self, segment_paths: List[Path], final_output_path: Path) -> Path: """Concatenates uniform MP4 segment clips using FFmpeg's concat demuxer.""" inputs_file = final_output_path.parent / "concat_inputs.txt" with open(inputs_file, "w", encoding="utf-8") as f: for segment in segment_paths: safe_path = str(segment.resolve()).replace("'", "'\\''") f.write(f"file '{safe_path}'\n") cmd = [ "ffmpeg", "-y", "-f", "concat", "-safe", "0", "-i", str(inputs_file), "-c", "copy", str(final_output_path) ] try: subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) return final_output_path finally: if inputs_file.exists(): inputs_file.unlink() ``` --- ### Example 5: Marp Slide Renderer with Injected CSS ```python import subprocess from pathlib import Path class MarpEngine: """Converts Markdown text into 1280x720 PNG slide images using Marp CLI.""" def generate_slide_image(self, markdown_content: str, output_path: Path) -> Path: custom_css = """ section { width: 1280px; height: 720px; display: flex; flex-direction: column; justify-content: center; align-items: center; text-align: center; padding: 50px; font-family: 'Helvetica Neue', Arial, sans-serif; } h1 { font-size: 54px; color: #0284c7; } p { font-size: 28px; color: #334155; } """ formatted_md = f"---\nmarp: true\ntheme: gaia\nsize: 16:9\nstyle: |\n{custom_css}\n---\n\n" + markdown_content temp_md = output_path.with_suffix(".temp.md") temp_md.write_text(formatted_md, encoding="utf-8") cmd = [ "npx", "@marp-team/marp-cli", "--image", "png", "--allow-local-files", "-o", str(output_path), str(temp_md) ] try: subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE) finally: if temp_md.exists(): temp_md.unlink() return output_path ``` --- ## 5. Best Practices & Gotchas ### Performance & Stability * **Headless Browser Execution**: Always launch Playwright Chromium with `--disable-dev-shm-usage` to prevent shared memory exhaustion inside containerized environments (Docker/Kubernetes). * **Buffer Delays**: Add a `1000ms` visual buffer delay at the end of Playwright browser execution before closing the context. Playwright flushes its video recording buffer upon context closure; closing too early truncates the final frame. * **Isolated Temporary Directories**: Store raw browser WebM recordings, slide images, and audio files in isolated job-specific temporary directories (`tempfile.mkdtemp`), purging them upon pipeline completion. ### Media Encoding & FFmpeg Gotchas * **Pixel Format Requirement (`yuv420p`)**: Standard H.264 video defaults to `yuv444p` or `yuv422p` when converted from PNG images. Most web browsers and media players cannot play `yuv444p` MP4 files. Explicitly set `-pix_fmt yuv420p`. * **Pixel Dimension Evenness Rule**: H.264 requires video widths and heights to be divisible by 2. Using `scale=1280:720` guarantees even dimensions. * **Audio Track Alignment**: The FFmpeg `concat` demuxer (`-f concat -c copy`) **will fail or drop audio** if any intermediate clip lacks an audio stream. Always inject synthetic silent audio (`anullsrc=r=44100:cl=stereo`) into video segments that lack narration. * **Sample Rate Consistency**: Force a fixed audio sample rate (`-ar 44100` or `-ar 48000`) across all slide videos and UI recordings to avoid audio pitch/speed glitches after concatenation. * **Single Quotes in Concat Files**: Absolute paths inside FFmpeg `concat_inputs.txt` files must escape single quotes (`path.replace("'", "'\\''")`) and be wrapped in `'...'` quotes to handle paths with spaces or special characters. ```` ### `README.md` ```markdown # Demo Video Pipeline Turns a URL, a script, and a few slides into a single narrated MP4 by chaining Playwright screen recording, Marp slide rendering, Sarvam AI text-to-speech, and FFmpeg transcoding/concatenation. ## Features - Records real browser interactions (`goto`, `click`, `fill`, `hover`, `press`, `wait`) as resolution-locked WebM via Playwright, with defensive selector handling so a missing element skips gracefully instead of crashing the run. - Renders Markdown slides to 1280x720 PNG frames with Marp CLI and injected CSS. - Synthesizes narration with Sarvam AI TTS and paces each slide's duration to the exact length of its voiceover, measured from the WAV header. - Normalizes every intermediate clip (H.264/yuv420p/25fps video, AAC/44100Hz/stereo audio, synthetic silence where narration is missing) so FFmpeg's concat demuxer never breaks. - Concatenates all segments into one final MP4. ## When Claude uses this Ask for a product walkthrough, narrated UI demo, or auto-generated screen-recording video, or ask for help wiring together Playwright + Marp + TTS + FFmpeg into one pipeline. ## Requirements Python 3.10+, Playwright (Chromium installed), FFmpeg on PATH, Node.js/npx (for `@marp-team/marp-cli`), and network access to the Sarvam AI TTS API. ```