dstack

name dstack description dstack is an open-source control plane for GPU provisioning and orchestration across GPU clouds, Kubernetes, and on-prem clusters. dstack Overview dstack provisions and orchestrates workloads across GPU clouds, Kubernetes, and on-prem via fleets. When to use this skill: Running or managing dev environments, tasks, or services on dstack Creating, editing, or applying *.dstack.yml configurations Managing fleets, volumes, gateways, and checking available offers How it works dstack operates through three core components: dstack server - Can run locally, remotely, or via dstack Sky (managed) dstack CLI - Applies configurations and manages or inspects fleets, runs, logs, events, volumes, gateways, and offers; it uses project configurations stored in ~/.dstack/config.yml , which can be managed with dstack project dstack configuration files - YAML files ending with .dstack.yml dstack apply shows a plan and submits configuration changes. For run configurations, it attaches when the run reaches running by default: it configures SSH access, forwards declared ports, and streams logs. With -d , it submits and exits. Quick agent flow (detached runs) Show plan: echo "n" | dstack apply -f <config> If plan is OK and user confirms, apply detached: dstack apply -f <config> -y -d Check the run: dstack run get <run-name> --json If dev-environment or task with ports and running: attach to surface IDE link/ports/SSH alias (agent runs attach in background); ask to open link If attach fails in sandbox: request escalation; if not approved, ask the user to run dstack attach locally and share the output CRITICAL: Never propose dstack CLI commands or YAML syntaxes that don't exist. Only use CLI commands and YAML syntax documented here or verified via --help If uncertain about a command or its syntax, check the links or use --help NEVER do the following: Invent CLI flags not documented here or shown in --help Guess YAML property names - verify in configuration reference links Run dstack apply for runs without -d in automated contexts (blocks indefinitely) Retry failed commands without addressing the underlying error Summarize or reformat tabular CLI output - show it as-is Use echo "y" | when -y flag is available Assume a command succeeded without checking output for errors Agent execution guidelines Output accuracy NEVER reformat, summarize, or paraphrase CLI output. Display tables, status output, and error messages exactly as returned. When showing command results, use code blocks to preserve formatting. If output is truncated due to length, indicate this clearly (e.g., "Output truncated. Full output shows X entries."). Verification before execution When uncertain about any CLI flag or YAML property, run dstack <command> --help first. Never guess or invent flags. Example verification commands: dstack -- help # List all commands dstack apply -h <configuration type > # Flags for apply per configuration type (dev-environment, task, service, fleet, etc) dstack fleet -- help # Fleet subcommands dstack ps -- help # Flags for ps If a command or flag isn't documented, it doesn't exist. Command timing and confirmation handling Commands that stream indefinitely in the foreground: dstack attach dstack apply without -d for runs dstack ps -w Agents should avoid blocking: use -d , timeouts, or background attach. When attach is needed, run it in the background by default ( nohup ... ), but describe it to the user simply as "attach" unless they ask for a live foreground session. When waiting programmatically for a specific run, use dstack run get <run-name> --json and read its top-level status . Run statuses are pending , submitted , provisioning , running , terminating , terminated , failed , and done ; the last three are terminal. Stop waiting when the run reaches the state needed for the next action or a terminal status. Never parse or grep human-readable dstack ps output; its status column may display a job message such as no offers . All other commands: Use 10-60s timeout. Most complete within this range. While waiting, monitor the output - it may contain errors, warnings, or prompts requiring attention. Confirmation handling: dstack apply , dstack stop , dstack fleet delete require confirmation Use -y flag to auto-confirm when user has already approved For dstack stop , always use -y after the user confirms to avoid interactive prompts Use echo "n" | to preview dstack apply plan without executing (avoid echo "y" | , prefer -y ) Best practices: Prefer modifying configuration files over passing parameters to dstack apply (unless it's an exception) When user confirms deletion/stop operations, use -y flag to skip confirmation prompts Detached run follow-up (after -d ) After submitting a run with -d (dev-environment, task, service), first determine whether submission failed. If the apply output shows errors (validation, no offers, etc.), stop and surface the error. If the run was submitted, check it with dstack run get <run-name> --json , then guide the user through relevant next steps: If you need to prompt for next actions, be explicit about the dstack step and command (avoid vague questions). When speaking to the user, refer to the action as "attach" (not "background attach"). Monitor status: Report the current status and offer to keep watching. If watching, poll dstack run get <run-name> --json every 10-20 seconds until it reaches the state needed for the next action or a terminal status. Attach when running: For agents, run attach in the background by default so the session does not block. Use it to capture IDE links/SSH alias or enable port forwarding; when describing the action to the user, just say "attach". Dev environments or tasks with ports: Once running , attach to surface the IDE link/port forwarding/SSH alias, then ask whether to open the IDE link. Never open links without explicit approval. Services: Prefer using service endpoints. Attach only if the user explicitly needs port forwarding or full log replay. Tasks without ports: Default to dstack logs for progress; attach only if full log replay is required. Attaching behavior (blocking vs non-blocking) dstack attach runs until interrupted and blocks the terminal. Agents must avoid indefinite blocking. If a brief attach is needed, use a timeout to capture initial output (IDE link, SSH alias) and then detach. Note: dstack attach writes SSH alias info under ~/.dstack/ssh/config (and may update ~/.ssh/config ) to enable ssh <run name> , IDE connections, port forwarding, and real-time logs ( dstack attach --logs ). If the sandbox cannot write there, the alias will not be created. Permissions guardrail: If dstack attach fails due to sandbox permissions, request permission escalation to run it outside the sandbox. If escalation isn’t approved or attach still fails, ask the user to run dstack attach locally and share the IDE link/SSH alias output. Background attach (non-blocking default for agents): nohup dstack attach <run name> --logs > /tmp/<run name>.attach.log 2>&1 & echo $! > /tmp/<run name>.attach.pid Then read the output: tail -n 50 /tmp/<run name>.attach.log Offer live follow only if asked: tail -f /tmp/<run name>.attach.log Stop the background attach (preferred): kill " $(cat /tmp/<run name>.attach.pid) " If the PID file is missing, fall back to a specific match (avoid killing all attaches): pkill -f "dstack attach <run name>" Why this helps: it keeps the attach session alive (including port forwarding) while the agent remains usable. IDE links and SSH instructions appear in the log file -- surface them and ask whether to open the link ( open "<link>" on macOS, xdg-open "<link>" on Linux) only after explicit approval. If background attach fails in the sandbox (permissions writing ~/.dstack or ~/.ssh , timeouts), request escalation to run attach outside the sandbox. If not approved, ask the user to run attach locally and share the IDE link/SSH alias. Interpreting user requests "Run something": When the user asks to run a workload (dev environment, task, service), use dstack apply with the appropriate configuration. Note: dstack run only supports dstack run get --json for retrieving run details -- it cannot start workloads. "Connect to" or "open" a dev environment: If a dev environment is already running, use dstack attach <run name> --logs (agent runs it in the background by default) to surface the IDE URL ( cursor:// , vscode:// , etc.) and SSH alias. If sandboxed attach fails, request escalation or ask the user to run attach locally and share the link. Distributed tasks and multi-replica services Unless you use Distributed tasks (see ### 2. Tasks ) or Multi-replica services (see ### 3. Services ), both tasks and services run on a single node. That's why dstack logs <run name> , dstack attach <run name> , and ssh <run name> default to the first replica/job. In a distributed task, each node runs its own job, numbered from 0 in order across node groups. Target a node via dstack logs <run name> --job 1 or dstack attach <run name> --job 1 . In a multi-replica service, replicas are numbered from 0 in order across replica groups. Target a replica via dstack logs <run name> --replica 1 or dstack attach <run name> --replica 1 . Attaching with a non-zero --job or --replica creates the SSH alias ssh <run name>-<job num>-<replica num> . Configuration types dstack supports run configurations (dev environments, tasks, and services) and infrastructure configurations (fleets, volumes, and gateways). Configuration files can be named <name>.dstack.yml or simply .dstack.yml . Common parameters: All run configurations (dev environments, tasks, services) support many parameters including: Git integration: Clone repos automatically ( repo ) or mount existing repos ( repos ) File upload: Upload local files ( files ; see concept docs for examples) Docker support: Use custom Docker images ( image ); use docker: true if you want to use Docker from inside the container (VM-based backends only) Environment: Set environment variables ( env ), often via .envrc . Secrets are supported but less common. Storage: Persistent network volumes ( volumes ), specify disk size Resources: Define GPU, CPU, memory, and disk requirements Best practices: Prefer giving configurations a name property for easier management When configurations need credentials (API keys, tokens), list only env var names in the env section (e.g., - HF_TOKEN ), not values. Recommend storing actual values in a .envrc file alongside the configuration, applied via source .envrc && dstack apply . python and image are mutually exclusive in run configurations. If image is set, do not set python . files and repos intent policy Use files and repos only when the user intends to use local/repo files inside the run. If user asks to use project code/data/config in the run, then add files or repos as appropriate. If it is totally unclear whether files or repos must be mounted, ask one explicit clarification question or default to not mounting. files guidance: Relative paths are valid and preferred for local project files. A relative files path is placed under the run's working_dir (default or set by user). repos + image/working directory guidance: With non-default Docker images, prefer explicit absolute mount targets for repos (e.g., .:/dstack/run ). When setting an explicit repo mount path, also set working_dir to the same path. Reason: custom images may have a different/non-empty default working directory, and mounting a repo into a non-empty path can fail. With dstack default images, the default working_dir is already /dstack/run . 1. Dev environments Use for: Interactive development with IDE integration (VS Code, Cursor, etc.). type: dev-environment name: cursor python: "3.12" ide: vscode resources: gpu: 80GB Concept documentation | Configuration reference 2. Tasks Use for: Batch jobs, training runs, fine-tuning, web applications, any executable workload. Key features: Distributed training (multi-node) and port forwarding for web apps. type: task name: train python: "3.12" env: - HUGGING_FACE_HUB_TOKEN commands: - uv pip install -r requirements.txt - uv run python train.py ports: - 8501 # Optional: expose ports for web apps resources: gpu: A100:40GB:2 Port forwarding: When you specify ports , dstack apply forwards them to localhost while attached. Use dstack attach <run name> to reconnect and restore port forwarding. The run name becomes an SSH alias (e.g., ssh <run name> ) for direct access. Distributed tasks: Set nodes to run a task across multiple nodes, or use groups to define node groups, each with its own nodes count, resources , commands , and ports ( groups and top-level nodes are mutually exclusive). Requires a fleet that supports inter-node communication (see placement: cluster in fleets). Concept documentation | Configuration reference 3. Services Use for: Deploying models or web applications as production endpoints. Key features: OpenAI-compatible model serving, auto-scaling (RPS/queue), custom gateways with HTTPS. type: service name: llama31 python: "3.12" env: - HF_TOKEN commands: - uv pip install vllm - uv run vllm serve meta-llama/Meta-Llama-3.1-8B-Instruct port: 8000 model: meta-llama/Meta-Llama-3.1-8B-Instruct resources: gpu: 80GB disk: 200GB Service endpoints: Without gateway: <server URL>/proxy/services/<project name>/<run name>/ With gateway: https://<run name>.<gateway domain>/ Authentication: Unless auth is false , include Authorization: Bearer <user token> on service requests. Model endpoint: If model is set, service.model.base_url from dstack run get <run name> --json provides the model endpoint. For OpenAI-compatible models (the default, unless format is set otherwise), this will be service.url + /v1 . Example (with gateway): curl -sS -X POST "https://<run name>.<gateway domain>/v1/chat/completions" \ -H "Authorization: Bearer <user token>" \ -H "Content-Type: application/json" \ -d '{"model":"<model name>","messages":[{"role":"user","content":"Hello"}],"max_tokens":64}' Multi-replica services: Set replicas to run multiple replicas, or use groups to define replica groups, each with its own replicas count, resources , and commands ( groups and top-level replicas are mutually exclusive). If replicas require an interconnect (e.g., PD disaggregation), the service must run on a fleet with placement: cluster . Concept documentation | Configuration reference 4. Fleets Use for: Pre-provisioning infrastructure for workloads, managing on-prem GPU servers, creating auto-scaling instance pools. type: fleet name: my-fleet nodes: 0 ..2 resources: gpu: 24GB.. disk: 200GB spot_policy: auto # other values: spot, on-demand idle_duration: 5m