multimedia-production-mastery

name multimedia-production-mastery version 1.2.1 description Automates music- and curve-driven video edits: librosa beat-sync, MoviePy 2.x time_transform ramps, FFmpeg zoompan easing, camera shake, and word-highlight .ass. Use when beats, easing math, or time-remap drive the cut. Not for plain trim/concat (ffmpeg-video-editing), format conversion (ffmpeg-format-conversion), or audio loudness (ffmpeg-audio-processing). category media-automation risk safe source openrouter-deepsearch date_added 2026-06-20T00:00:00.000Z last_updated 2026-07-01T00:00:00.000Z trigger User asks to cut or transition video on music beats or onsets, build a speed ramp (fast -> slow-mo -> fast), animate smooth zoom/pan with easing curves, add bass-impact camera shake, or generate karaoke-style word-highlight subtitles. action Run the matching bundled script (beat_sync.py, speed_ramp.py, subtitles_styled.py) or compose the documented MoviePy 2.x / FFmpeg recipe; apply the easing and time-remap formulas; enforce even output dimensions and close every clip handle. exception Outputting odd-dimension video; leaving MoviePy clip handles open on Windows; applying atempo outside 0.5-2.0 without chaining; calling time_transform without restoring the clip duration. not-triggered Format conversion, simple trims or concatenation, static image editing or graphic design, generative AI video creation, CapCut or other NLE project-file manipulation. Multimedia Production Mastery Automation recipes, mathematical formulations, and Python/FFmpeg templates for edits that are driven by a signal or a curve : audio beats, easing functions, and time-remap profiles. The unit of work here is a computed timeline, not a manual cut. When to Use Use this skill when one or more of the following are true: Beat synchronization — cutting or transitioning clips to align with the tempo, beats, or onsets of an audio track (librosa analysis feeding MoviePy/FFmpeg cuts). Speed ramping — smooth, non-linear acceleration/deceleration profiles (fast → slow-mo → fast) via time remapping. Zoom & pan curves — Ken Burns motion using sinusoidal or exponential ease-in/ease-out instead of robotic linear interpolation. Camera shake — procedural handheld or bass-impact shake generated from noise or onset strength. Styled dynamic subtitles — .ass files with word-by-word highlight colors, custom fonts, borders, and margins. Pipeline orchestration — wiring librosa + MoviePy 2.x + FFmpeg around pre-rendered clips (including AI-generated ones). When NOT to use — routing table Request Route to Cut, trim, join, or re-encode segments with no timing math ffmpeg-video-editing Constant-rate scale, crop, watermark, blur, or uniform speed change ffmpeg-video-filters Assemble finished clips, audio, and titles into a final deliverable video-assembly-pipeline Extract, normalise, or mix audio tracks ffmpeg-audio-processing Programmatic timeline edits inside CapCut Desktop drafts capcut-video-automation Container/format conversion with no timeline changes ffmpeg-format-conversion Prerequisites Python environment All bundled scripts require the following packages in the active Python environment: Package Version constraint Purpose librosa ≥ 0.11 (tempo returns NumPy array) Beat/onset detection moviepy 2.x only — 1.x API is incompatible Video composition, time remapping numpy any recent Numerical integration, interpolation scipy any recent cumulative_trapezoid for speed profiles Verify before running any script: python -c "import librosa, moviepy, numpy, scipy; print(librosa.__version__, moviepy.__version__)" Expected output should show moviepy version starting with 2. . FFmpeg FFmpeg must be on PATH . Verify: ffmpeg -version Bundled scripts location All scripts live in scripts/ relative to this skill directory: Script Purpose Key flags beat_sync.py Detect beats/onsets in a track and cut a video (or stitch a clip folder) on them --audio (req), --video | --dir , --output (req), --onset , --export-beats beats.json speed_ramp.py Three-phase non-linear speed ramp --video (req), --output (req), --start-speed 2.0 , --mid-speed 0.2 , --end-speed 2.0 , --ramp-start 0.2 , --ramp-end 0.8 subtitles_styled.py Timed-words JSON → styled .ass with per-word highlights --input (req), --output (req), --font , --size , --color #RRGGBB , --highlight-color , --outline-color , --outline , --shadow , --alignment , --margin-v Bundled references Load references/effects-landscape.md when the user asks about available AI-enhanced or classical video effects, their parameters, tooling, or underlying math — it is a catalog you can search for effect names and implementation details. Procedure Step 1 — Identify the driving signal Determine what drives the edit: Signal Tool Script Musical beats or tempo librosa.beat.beat_track beat_sync.py Onsets (percussive/transient) librosa.onset.onset_detect beat_sync.py --onset Custom speed curve scipy.integrate.cumulative_trapezoid speed_ramp.py Easing curve for zoom/pan FFmpeg zoompan with cosine easing Recipe 1 below Word timings for subtitles External JSON (whisper or manual) subtitles_styled.py Step 2 — Run or compose the matching recipe Recipe A: Beat-synced cut from a folder of clips python scripts/beat_sync.py --audio music.mp3 --dir .\raw_clips --output synced.mp4 Export detected beats for inspection or reuse: python scripts/beat_sync.py --audio music.mp3 --dir .\raw_clips --output synced.mp4 --export-beats beats.json For sparse or syncopated tracks where beat tracking misses, use onset detection: python scripts/beat_sync.py --audio ambient.mp3 --dir .\raw_clips --output synced.mp4 --onset Recipe B: Three-phase speed ramp (fast → slow-mo → fast) python scripts/speed_ramp.py --video in.mp4 --output ramped.mp4 --mid-speed 0.15 Full parameter control: python scripts/speed_ramp.py ` --video in.mp4 ` --output ramped.mp4 ` --start-speed 2.0 ` --mid-speed 0.2 ` --end-speed 2.0 ` --ramp-start 0.2 ` --ramp-end 0.8 Recipe C: Word-highlight styled subtitles Input JSON format (word-level timings, highlight flags the accent color): [ { "start" : 0.5 , "end" : 1.2 , "text" : "Welcome" , "highlight" : false } , { "start" : 1.8 , "end" : 3.0 , "text" : "Future!" , "highlight" : true } ] Generate the .ass file: python scripts/subtitles_styled.py ` --input words.json ` --output subs.ass ` --highlight-color "#00FFFF" Step 3 — Core formulas (reference when composing custom code) Non-linear motion curves HARD RULE: Never linearly interpolate zoom or pan — it reads as robotic. Use sinusoidal ease-in-out: z(t) = z_start + (z_end - z_start) * (1 - cos(pi * t / d)) / 2 where t is current time, d the ease duration. To keep pixel (x_f, y_f) centered on a W x H frame while zooming by z(t) : x(t) = x_f - W / (2 * z(t)) y(t) = y_f - H / (2 * z(t)) Time remapping (speed ramps) Define S(t) = source time at output time t ; the derivative S'(t) is the instantaneous speed multiplier. Build S(t) by integrating a speed profile. MoviePy 2.x: clip.time_transform(time_func) — the 1.x fl_time is gone. HARD RULE: time_transform() leaves the clip's duration undefined. Call .with_duration(target) immediately after, or output is truncated. FFmpeg: dynamic speed requires splitting into segments and applying setpts (video) + atempo (audio) per segment. HARD RULE: atempo only accepts [0.5, 2.0] ; chain filters for anything outside (e.g. atempo=2.0,atempo=2.0 for 4x, atempo=2.0,atempo=1.5 for 3x). ASS color formats Two different hex conventions inside one file: Context Format Example Style definitions ( [V4+ Styles] ) &HAABBGGRR — 8 digits, no trailing & &H0000FFFF = opaque yellow Inline override tags \c&HBBGGRR& — 6 digits, trailing & , no alpha {\c&H00FFFF&} = yellow from here on HARD RULE: Alpha is inverted relative to RGBA: 00 = fully opaque, FF = fully transparent. Invisible text usually means FF (transparent) where 00 (opaque) was intended. Step 4 — FFmpeg recipes Recipe 1: Jitter-free FFmpeg zoompan (Ken Burns) zoompan evaluates positions in integer pixels, causing visible jitter at low zoom speeds. Pre-upscale to 4K first so the truncation error becomes sub-pixel: ffmpeg -i in.jpg -vf "scale=3840:-2,zoompan=z='1+0.5*(1-cos(PI*on/149))/2':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':d=150:fps=30:s=1920x1080" -c:v libx264 -pix_fmt yuv420p out.mp4 Recipe 2: The even-dimensions rule HARD RULE: Most encoders (yuv420p especially) fail on odd width/height. After any crop or scale, force even dimensions: ffmpeg -i input.mp4 -vf "scale='trunc(iw/2)*2:trunc(ih/2)*2'" output.mp4 Recipe 3: MoviePy 2.x non-linear speed ramp (custom code) Integrate a speed profile to get the time-remap function, then apply it: import numpy as np from scipy.integrate import cumulative_trapezoid from moviepy import VideoFileClip src = VideoFileClip( "in.mp4" ) fps = src.fps out_dur = 6.0 t_out = np.linspace( 0 , out_dur, int (out_dur * fps) + 1 ) # Speed profile: fast -> slow -> fast speed = 1.0 + 0.5 * (np.cos( 2 * np.pi * t_out / out_dur) * 0.5 + 0.5 ) # Integrate speed to get source timestamps for each output frame s_src = np.clip(cumulative_trapezoid(speed, t_out, initial= 0.0 ), 0 , src.duration - 1e-3 ) time_func = lambda t: float (np.interp(t, t_out, s_src)) ramped = src.time_transform(time_func).with_duration(out_dur) ramped.write_videofile( "out.mp4" , fps=fps, audio= False ) src.close() ramped.close() Recipe 4: Proxy workflow for heavy footage Validate cuts and beat alignment on a cheap proxy, render once on the original: Generate proxy: ffmpeg -i input.mp4 -vf scale=-2:720 -c:v libx264 -crf 20 -c:a copy proxy.mp4 Iterate timing/beat scripts against proxy.mp4 . Swap the source path back to the 4K original for the final render only. Step 5 — Onset fallback for difficult tracks Standard beat tracking misses beats in sparse or heavily syncopated material. Fall back to onsets in custom code: onset_frames = librosa.onset.onset_detect(y=y, sr=sr, backtrack= True ) times = librosa.frames_to_time(onset_frames, sr=sr) ( beat_sync.py --onset does this automatically.) Pitfalls MoviePy duration reset — time_transform() leaves the clip's duration undefined. Call .with_duration(target) immediately after, or output is truncated. ASS inverted alpha — invisible text usually means FF (transparent) where 00 (opaque) was intended. Alpha is inverted relative to RGBA. atempo range — multipliers outside [0.5, 2.0] silently fail unless chained ( atempo=2.0,atempo=1.5 for 3x, atempo=2.0,atempo=2.0 for 4x). librosa tempo type — beat_track() returns a NumPy array for tempo in librosa 0.11+; cast with float() before formatting. Beat tracking on ambient/vocal tracks — standard beat tracking misses beats in sparse or heavily syncopated material. Use --onset flag or librosa.onset.onset_detect with backtrack=True . Windows file locks — unclosed VideoFileClip / AudioFileClip handles keep files locked and crash subsequent runs. HARD RULE: Always .close() every clip before the script exits, including in error paths. Odd output dimensions — yuv420p and most encoders fail on odd width/height. Always force even dimensions after any crop or scale operation. MoviePy 1.x vs 2.x API — only use 2.x methods: time_transform / with_duration / subclipped / with_audio . The 1.x fl_time / set_duration / subclip methods do not exist in 2.x and will raise AttributeError . zoompan integer jitter — zoompan evaluates positions in integer pixels. Without pre-upscaling to 4K, low zoom speeds produce visible jitter. Always pre-scale to a high resolution before zoompan . Verification Run through this checklist before considering the task complete: Environment check: librosa , moviepy (2.x), numpy , and scipy import successfully: python -c "import librosa, moviepy, numpy, scipy; print(librosa.__version__, moviepy.__version__)" Output must show moviepy version starting with 2. . MoviePy 2.x API only: Only these methods are used: time_transform / with_duration / subclipped / with_audio . No 1.x fl_time / set_duration / subclip appears anywhere in the code. Even dimensions: Every scale/crop expression forces even output dimensions. Verify with: