{
    "name": "comfyui",
    "version": "1.0.0",
    "description": "Generate images, video, and audio via diffusion workflows.",
    "system_prompt": "name comfyui description Generate images, video, and audio via diffusion workflows. version 5.1.0 author [\"kshitijk4poor\",\"alt-glitch\",\"purzbeats\"] license MIT platforms [\"macos\",\"linux\",\"windows\"] compatibility Requires ComfyUI (local, Comfy Desktop, or Comfy Cloud) and comfy-cli (auto-installed via pipx/uvx by the setup script). prerequisites {\"commands\":[\"python\"]} setup {\"help\":\"Run scripts/hardware_check.py FIRST to decide local vs Comfy Cloud; then scripts/comfyui_setup.sh auto-installs locally (or use Cloud API key for platform.comfy.org).\"} metadata {\"hermes\":{\"tags\":[\"comfyui\",\"image-generation\",\"stable-diffusion\",\"flux\",\"sd3\",\"wan-video\",\"hunyuan-video\",\"creative\",\"generative-ai\",\"video-generation\"],\"related_skills\":[\"stable-diffusion\"],\"category\":\"creative\"}} ComfyUI Generate images, video, audio, and 3D content through ComfyUI using the official comfy-cli for setup/lifecycle and direct REST/WebSocket API for workflow execution. What's in this skill Reference docs ( references/ ): official-cli.md — every comfy ... command, with flags rest-api.md — REST + WebSocket endpoints (local + cloud), payload schemas workflow-format.md — API-format JSON, common node types, param mapping template-integrity.md — converting comfyui-workflow-templates from editor format to API format: Reroute bypass, dotted dynamic-input keys ( values.a , resize_type.width ), Cloud quirks (302 redirect, 1 concurrent free-tier job, 1080p VRAM ceiling), Discord-compatible ffmpeg stitch. Authored by @purzbeats . Load this whenever you're starting from an official template. Scripts ( scripts/ ): Script Purpose _common.py Shared HTTP, cloud routing, node catalogs (don't run directly) hardware_check.py Probe GPU/VRAM/disk → recommend local vs Comfy Cloud comfyui_setup.sh Hardware check + comfy-cli + ComfyUI install + launch + verify extract_schema.py Read a workflow → list controllable params + model deps check_deps.py Check workflow against running server → list missing nodes/models auto_fix_deps.py Run check_deps then comfy node install / comfy model download run_workflow.py Inject params, submit, monitor, download outputs (HTTP or WS) run_batch.py Submit a workflow N times with sweeps, parallel up to your tier ws_monitor.py Real-time WebSocket viewer for executing jobs (live progress) health_check.py Verification checklist runner — comfy-cli + server + models + smoke test fetch_logs.py Pull traceback / status messages for a given prompt_id Example workflows ( workflows/ ): SD 1.5, SDXL, Flux Dev, SDXL img2img, SDXL inpaint, ESRGAN upscale, AnimateDiff video, Wan T2V. See workflows/README.md . When to Use User asks to generate images with Stable Diffusion, SDXL, Flux, SD3, etc. User wants to run a specific ComfyUI workflow file User wants to chain generative steps (txt2img → upscale → face restore) User needs ControlNet, inpainting, img2img, or other advanced pipelines User asks to manage ComfyUI queue, check models, or install custom nodes User wants video/audio/3D generation via AnimateDiff, Hunyuan, Wan, AudioCraft, etc. Architecture: Two Layers ┌─────────────────────────────────────────────────────┐ │ Layer 1: comfy-cli (official lifecycle tool) │ │ Setup, server lifecycle, custom nodes, models │ │ → comfy install / launch / stop / node / model │ └─────────────────────────┬───────────────────────────┘ │ ┌─────────────────────────▼───────────────────────────┐ │ Layer 2: REST/WebSocket API + skill scripts │ │ Workflow execution, param injection, monitoring │ │ POST /api/prompt, GET /api/view, WS /ws │ │ → run_workflow.py, run_batch.py, ws_monitor.py │ └─────────────────────────────────────────────────────┘ Why two layers? The official CLI is excellent for installation and server management but has minimal workflow execution support. The REST/WS API fills that gap — the scripts handle param injection, execution monitoring, and output download that the CLI doesn't do. Quick Start Detect environment # What's available? command -v comfy >/dev/null 2>&1 && echo \"comfy-cli: installed\" curl -s http://127.0.0.1:8188/system_stats 2>/dev/null && echo \"server: running\" # Can this machine run ComfyUI locally? (GPU/VRAM/disk check) python scripts/hardware_check.py If nothing is installed, see Setup & Onboarding below — but always run the hardware check first. One-line health check python scripts/health_check.py # → JSON: comfy_cli on PATH? server reachable? at least one checkpoint? smoke-test passes? Core Workflow Step 1: Get a workflow JSON in API format Workflows must be in API format (each node has class_type ). They come from: ComfyUI web UI → Workflow → Export (API) (newer UI) or the legacy \"Save (API Format)\" button (older UI) This skill's workflows/ directory (ready-to-run examples) Community downloads (civitai, Reddit, Discord) — usually editor format, must be loaded into ComfyUI then re-exported Editor format (top-level nodes and links arrays) is not directly executable . The scripts detect this and tell you to re-export. Step 2: See what's controllable python scripts/extract_schema.py workflow_api.json --summary-only # → {\"parameter_count\": 12, \"has_negative_prompt\": true, \"has_seed\": true, ...} python scripts/extract_schema.py workflow_api.json # → full schema with parameters, model deps, embedding refs Step 3: Run with parameters # Local (defaults to http://127.0.0.1:8188) python scripts/run_workflow.py \\ --workflow workflow_api.json \\ --args '{\"prompt\": \"a beautiful sunset over mountains\", \"seed\": -1, \"steps\": 30}' \\ --output-dir ./outputs # Cloud (export API key once; uses correct /api routing automatically) export COMFY_CLOUD_API_KEY= \"comfyui-...\" python scripts/run_workflow.py \\ --workflow workflow_api.json \\ --args '{\"prompt\": \"...\"}' \\ --host https://cloud.comfy.org \\ --output-dir ./outputs # Real-time progress via WebSocket (requires `pip install websocket-client`) python scripts/run_workflow.py \\ --workflow flux_dev.json \\ --args '{\"prompt\": \"...\"}' \\ --ws # img2img / inpaint: pass --input-image to upload + reference automatically python scripts/run_workflow.py \\ --workflow sdxl_img2img.json \\ --input-image image=./photo.png \\ --args '{\"prompt\": \"make it watercolor\", \"denoise\": 0.6}' # Batch / sweep: 8 random seeds, parallel up to cloud tier limit python scripts/run_batch.py \\ --workflow sdxl.json \\ --args '{\"prompt\": \"abstract\"}' \\ --count 8 --randomize-seed --parallel 3 \\ --output-dir ./outputs/batch -1 for seed (or omitting it with --randomize-seed ) generates a fresh random seed per run. Step 4: Present results The scripts emit JSON to stdout describing every output file: { \"status\" : \"success\" , \"prompt_id\" : \"abc-123\" , \"outputs\" : [ { \"file\" : \"./outputs/sdxl_00001_.png\" , \"node_id\" : \"9\" , \"type\" : \"image\" , \"filename\" : \"sdxl_00001_.png\" } ] } Decision Tree User says Tool Command Lifecycle (use comfy-cli) \"install ComfyUI\" comfy-cli bash scripts/comfyui_setup.sh \"start ComfyUI\" comfy-cli comfy launch --background \"stop ComfyUI\" comfy-cli comfy stop \"install X node\" comfy-cli comfy node install <name> \"download X model\" comfy-cli comfy model download --url <url> --relative-path models/checkpoints \"list installed models\" comfy-cli comfy model list \"list installed nodes\" comfy-cli comfy node show installed Execution (use scripts) \"is everything ready?\" script health_check.py (optionally with --workflow X --smoke-test ) \"what can I change in this workflow?\" script extract_schema.py W.json \"check if W's deps are met\" script check_deps.py W.json \"fix missing deps\" script auto_fix_deps.py W.json \"generate an image\" script run_workflow.py --workflow W --args '{...}' \"use this image\" (img2img) script run_workflow.py --input-image image=./x.png ... \"8 variations with random seeds\" script run_batch.py --count 8 --randomize-seed ... \"show me live progress\" script ws_monitor.py --prompt-id <id> \"fetch the error from job X\" script fetch_logs.py <prompt_id> Direct REST \"what's in the queue?\" REST curl http://HOST:8188/queue (local) or --host https://cloud.comfy.org \"cancel that\" REST curl -X POST http://HOST:8188/interrupt \"free GPU memory\" REST curl -X POST http://HOST:8188/free Setup & Onboarding When a user asks to set up ComfyUI, the FIRST thing to do is ask whether they want Comfy Cloud (hosted, zero install, API key) or Local (install ComfyUI on their machine) . Don't start running install commands or hardware checks until they've answered. Official docs: https://docs.comfy.org/installation CLI docs: https://docs.comfy.org/comfy-cli/getting-started Cloud docs: https://docs.comfy.org/get_started/cloud Cloud API: https://docs.comfy.org/development/cloud/overview Step 0: Ask Local vs Cloud (ALWAYS FIRST) Suggested script: \"Do you want to run ComfyUI locally on your machine, or use Comfy Cloud? Comfy Cloud — hosted on RTX 6000 Pro GPUs, all common models pre-installed, zero setup. Requires an API key (paid subscription required to actually run workflows; free tier is read-only). Best if you don't have a capable GPU. Local — free, but your machine MUST meet the hardware requirements: NVIDIA GPU with ≥6 GB VRAM (≥8 GB for SDXL, ≥12 GB for Flux/video), OR AMD GPU with ROCm support (Linux), OR Apple Silicon Mac (M1+) with ≥16 GB unified memory (≥32 GB recommended). Intel Macs and machines with no GPU will NOT work — use Cloud instead. Which would you like?\" Routing: Cloud → skip to Path A . Local → run hardware check first, then pick a path from Paths B–E based on the verdict. Unsure → run the hardware check and let the verdict decide. Step 1: Verify Hardware (ONLY if user chose local) python scripts/hardware_check.py --json # Optional: also probe `torch` for actual CUDA/MPS: python scripts/hardware_check.py --json --check-pytorch Verdict Meaning Action ok ≥8 GB VRAM (discrete) OR ≥32 GB unified (Apple Silicon) Local install — use comfy_cli_flag from report marginal SD1.5 works; SDXL tight; Flux/video unlikely Local OK for light workflows, else Path A (Cloud) cloud No usable GPU, <6 GB VRAM, <16 GB Apple unified, Intel Mac, Rosetta Python Switch to Cloud unless user explicitly forces local The script also surfaces wsl: true (WSL2 with NVIDIA passthrough) and rosetta: true (x86_64 Python on Apple Silicon — must reinstall as ARM64). If verdict is cloud but the user wants local, do not proceed silently. Show the notes array verbatim and ask whether they want to (a) switch to Cloud or (b) force a local install (will OOM or be unusably slow on modern models). Choosing an Installation Path Use the hardware check first. The table below is the fallback for when the user has already told you their hardware: Situation Recommended Path verdict: cloud from hardware check Path A: Comfy Cloud No GPU / want to try without commitment Path A: Comfy Cloud Windows + NVIDIA + non-technical Path B: ComfyUI Desktop Windows + NVIDIA + technical Path C: Portable or Path D: comfy-cli Linux + any GPU Path D: comfy-cli (easiest) macOS + Apple Silicon Path B: Desktop or Path D: comfy-cli Headless / server / CI / agents Path D: comfy-cli For the fully automated path (hardware check → install → launch → verify): bash scripts/comfyui_setup.sh # Or with overrides: bash scripts/comfyui_setup.sh --m-series --port=8190 --workspace=/data/comfy It runs hardware_check.py internally, refuses to install locally when the verdict is cloud (unless --force-cloud-override ), picks the right comfy-cli flag, and prefers pipx / uvx over global pip to avoid polluting system Python.",
    "model_config": {
        "provider": "deepseek",
        "model": "deepseek-chat",
        "temperature": 0.7,
        "max_tokens": 4096,
        "top_p": 0.9
    },
    "trigger_words": [],
    "source": "DeepseekModel",
    "source_url": "https://deepseekmodel.com/skill?id=nousresearch-hermes-agent-optional-skills-creative-comfyui-skill-md"
}