carocut-media-visual

name carocut-media-visual description 视觉素材获取与生成。根据 resources.yaml 定义，通过库存图片/视频搜索（Pexels/Pixabay）或 AI 生图（Caro LLM）获取视觉素材。包含搜索策略、视觉隐喻搜索（counterpoint 声画关系）、构图感知搜索（framing 字段）、prompt 工程指南、素材质量评估标准。 Visual Assets Retrieves stock images and generates custom images for video production. This step fulfills visual resource requirements defined in manifests/resources.yaml . Decision Flow digraph visual_decision { "Resource from resources.yaml" [shape=doublecircle]; "source = existing?" [shape=diamond]; "Already in raws/" [shape=box]; "source = retrieve?" [shape=diamond]; "Search stock images" [shape=box]; "source = generate?" [shape=diamond]; "Generate with AI" [shape=box]; "Needs transparency?" [shape=diamond]; "Remove background" [shape=box]; "Save to raws/" [shape=box]; "Update resources.yaml" [shape=doublecircle]; "Resource from resources.yaml" -> "source = existing?"; "source = existing?" -> "Already in raws/" [label="yes"]; "source = existing?" -> "source = retrieve?" [label="no"]; "source = retrieve?" -> "Search stock images" [label="yes"]; "source = retrieve?" -> "source = generate?" [label="no"]; "source = generate?" -> "Generate with AI" [label="yes"]; "Search stock images" -> "Needs transparency?"; "Generate with AI" -> "Needs transparency?"; "Needs transparency?" -> "Remove background" [label="yes"]; "Needs transparency?" -> "Save to raws/" [label="no"]; "Remove background" -> "Save to raws/"; "Already in raws/" -> "Update resources.yaml"; "Save to raws/" -> "Update resources.yaml"; } Resource Source Types 资源类型完整定义见 carocut-shared-schema skill。Media 只处理 visual 类型（image/video/sprite），不处理 data 和 components 类型。 Source When to Use existing User-provided materials already in raws/ retrieve Objective backgrounds, stock photos, stock videos (Pexels/Pixabay) generate Custom characters, foreground elements with transparency Sprite Sheets (Recommended for Character Animation) When the storyboard includes animated characters, mascots, or any element with multiple poses/actions, prefer generating sprite sheets over individual images. Sprite sheets provide: Consistent character appearance across all animation frames Smooth frame-by-frame animation in Remotion Efficient asset management (one file per character instead of many) Video Resources (视频素材) When the storyboard requires motion footage (product demos, b-roll, ambient shots), use type: video in resources.yaml: - id: demo_recording type: video description: "产品演示录屏" source: retrieve search_keywords: [ "product demo" , "screen recording" ] duration_range: "5-15s" Video Search Strategy Use Pexels for STOCK video retrieval when using tool videos_search Search keywords follow the same English-keyword rules as image search Filter by duration range when specified in the resource definition Prefer HD (1080p) or higher resolution Video Output Storage path: raws/videos/ Preferred format: MP4 Directory structure: raws/ videos/ retrieved/ product_demo_001.mp4 aerial_city_002.mp4 Video in resources.yaml - id: vid_001 type: video description: "Aerial city timelapse" source: retrieve search_keywords: [ "aerial city" , "timelapse" , "urban" ] duration_range: "5-10s" path: videos/retrieved/aerial_city_001.mp4 Image Retrieval (Stock Photos) Search royalty-free images from Pexels and Pixabay. Search Keywords Tips Need Keywords Video backgrounds abstract , texture , gradient , bokeh , blur Business office , meeting , professional , workspace Nature mountain , forest , ocean , sunset , aerial Technology technology , computer , coding , digital Minimal minimal , white background , flat lay , clean Image Search Strategy Use English keywords for all searches Keep search result count low (3-5) to avoid context bloat Use specific keywords with style descriptors for better relevance When initial results are poor, expand with synonyms (e.g., "server room" -> "data center", "network infrastructure") Filter by orientation when the layout requires it (landscape for backgrounds, portrait for mobile) Filter by color when brand consistency matters Visual Metaphor Search (视觉隐喻搜索) When the storyboard specifies audio_visual_relation: counterpoint , the visual should not literally illustrate the narration. Instead, switch to metaphorical/associative search: Narrative Concept Metaphor Search Keywords 信息过载 overflowing water, paper avalanche, cluttered desk 快速增长 sprouting plant timelapse, rising tide, rocket launch 技术落后 vintage technology, old computer, floppy disk 复杂度 tangled wires, maze, puzzle pieces 突破/创新 breaking through wall, sunrise, butterfly emerging 竞争 race track, chess game, climbing mountain 连接/协作 bridge, handshake, puzzle fitting together 衰退/消失 wilting flower, melting ice, fading light Rules: Only activate metaphor search when audio_visual_relation: counterpoint For sync or lead-visual , use literal/direct search as usual For lead-audio , visuals can be more abstract/atmospheric but not necessarily metaphorical When using metaphor search, pick 2-3 keywords from the table and combine with style descriptors If the narrative concept is not in the table, derive a visual metaphor by analogy Framing-Aware Search (构图感知搜索) Adjust search keywords based on the storyboard's framing field to find images matching the intended shot composition: Framing Append to Search Keywords ECU (Extreme Close-Up) "close up", "detail", "macro", "texture" CU (Close-Up) "close up", "detail", "macro", "texture" MCU (Medium Close-Up) (no modification, default search) MS (Medium Shot) (no modification, default search) LS (Long Shot) "wide shot", "landscape", "panoramic", "aerial" ELS (Extreme Long Shot) "wide shot", "landscape", "panoramic", "aerial" Rules: Append the framing keywords to the existing search query (do not replace) For MCU and MS , no framing keywords are needed Combine with metaphor search when applicable (e.g., counterpoint + ECU → "close up texture tangled wires") License Information Platform License Attribution Commercial Use Pexels Pexels License Appreciated but not required Yes Pixabay CC0 / Pixabay License Not required Yes Image Generation (AI) Generate custom images via OpenAI-compatible API. Prompt Rules Write all prompts in English For subjects needing transparency: Include "on a solid white background" or "on a plain white background" For Chinese text in image: Use format sign that reads "Chinese text here" Be specific: Describe composition, style, colors, lighting, perspective Prompt Examples Background image: Abstract gradient background, soft blue to purple transition, subtle geometric patterns, clean modern aesthetic, suitable for tech presentation Character with transparency: A friendly 3D cartoon scientist character, female, wearing white lab coat and glasses, standing pose facing camera, on a plain white background, Pixar style rendering Sprite sheet: Grid: 8 columns x 4 rows. Row 1: Idle - character breathing subtly Row 2: Walking - 8-frame walk cycle from left Row 3: Running - 8-frame run cycle Row 4: Jumping - jump up and land sequence Character: cute robot with round blue body, small antenna, expressive digital eyes. Style: 3D cartoon, soft lighting, clean design. Background Removal Remove background from generated images to create transparent subjects. Works best when the original image has a clean, solid-color background. Use Cases Characters/subjects generated on white background that need transparency Stock photos where only the subject is needed Foreground elements that will be composited over video backgrounds Sprite Sheet Validation (MANDATORY) After generating a sprite sheet, validation is performed automatically by the images_generate tool when using mode: sprite . You can also run validation independently with the images_validate_sprite tool. Automatic Validation (Built-in to Generate) When using images_generate with mode: sprite : Grid dimensions auto-detected from the prompt (patterns like "8 columns x 4 rows"), or explicitly via cols / rows parameters Post-generation validation runs automatically: Checks image dimensions are evenly divisible by cols/rows Checks magenta chroma key coverage >= 15% Auto-retry on failure (up to max_retries times, default 3): Each retry enhances the prompt with progressively stronger dimension constraints Adds explicit pixel resolution requirements (e.g., "output MUST be 4096x2048") Reports final validation result in JSON Recommended usage: images_generate( prompt="Grid: 8 columns x 4 rows. ...", output="/path/to/sprite.png", mode="sprite", cols=8, # explicit is more reliable than auto-inference rows=4, ) Independent Validation Tool Use images_validate_sprite for post-hoc validation or re-checking: images_validate_sprite( image="/path/to/sprite.png", cols=8, rows=4, fix_chroma=true, # optional: replace magenta with transparency output="/path/to/sprite_transparent.png", extract_frames="/path/to/frames/", # optional: extract individual frames ) Returns JSON with check results: { "success" : true , "checks" : [ { "check" : "dimensions" , "passed" : true , "frame_size" : "512x512" } , { "check" : "aspect_ratio" , "passed" : true , "frame_aspect_ratio" : 1.0 } , { "check" : "chroma_key" , "passed" : true , "magenta_coverage" : 42.3 } ] } Sprite Sheet Failure Handling If sprite sheet generation fails after all retries: Log failure details (prompt, dimensions, error) in execution summary Do NOT silently skip — report the failure to orchestrator as partial_failure Suggest alternatives to the user: Retry with simplified prompt (fewer cells, simpler subjects) Fall back to static images instead of sprite animation Provide alternative asset manually Continue processing remaining (non-failed) resources Manual Fallback Validation If the tools are unavailable, use Python/Pillow to verify: from PIL import Image img = Image. open ( "raws/images/generated/robot_sprite.png" ) cols, rows = 8 , 4 w, h = img.size assert w % cols == 0 , f"Width {w} not divisible by {cols} columns" assert h % rows == 0 , f"Height {h} not divisible by {rows} rows" frame_w, frame_h = w // cols, h // rows print ( f"Sprite: {w} x {h} , Frame: {frame_w} x {frame_h} , Grid: {cols} x {rows} " ) Validation Checklist Check Expected Auto-handled Manual Action if Fails Width divisible by columns width % cols == 0 Yes (auto-retry with enhanced prompt) Regenerate with explicit resolution Height divisible by rows height % rows == 0 Yes (auto-retry with enhanced prompt) Regenerate with explicit resolution Frame aspect ratio ~1:1 ratio ≈ 1.0 ± 0.3 Warning only Adjust cols/rows ratio Magenta coverage > 15% Background is clean Yes (auto-retry) Regenerate with stronger magenta instruction No magenta on character Character pixels distinct Not auto-checked Add "avoid magenta/pink colors on character" to prompt Frame-to-frame continuity Smooth motion Warning only Regenerate with "smooth transition" emphasis Sprite Metadata in resources.yaml After validation, record sprite metadata for downstream consumption: - id: vis_sprite_001 type: sprite description: "Robot character animation" source: generate path: images/generated/robot_sprite.png sprite: cols: 8 rows: 4 frameWidth: 512 frameHeight: 512 chromaKey: "#ff00ff" animations: idle: { row: 0 , frameCount: 8 }