microsoft-foundry

name microsoft-foundry description Build, deploy, evaluate, optimize, fine-tune, and manage Microsoft Foundry agents, models, and resources end to end. USE FOR: azd ai agent, azd provision/deploy, hosted agent scaffold/develop/run/deploy/troubleshoot, prompt agent create, create agent, update agent, add tool to agent, invoke agent, agent.yaml, evaluate agent, batch eval, continuous eval, continuous monitoring, agent CI/CD, optimize prompt, improve prompt, prompt optimizer, optimize agent instructions, Agent Optimizer scaffold, dataset curation from traces, deploy model, model fine-tuning (SFT/DPO/RFT), Foundry project, RBAC, role assignment, permissions, quota, capacity, region, deployment failure, AI Services, create Foundry resource, knowledge index, customize deployment, onboard, availability, training-data, grader, distillation, large file upload. DO NOT USE FOR: Azure Functions, App Service, general Azure deploy (use azure-deploy), general Azure prep (use azure-prepare). license MIT metadata {"author":"Microsoft","version":"1.2.15"} Microsoft Foundry Skill This skill helps developers work with Microsoft Foundry resources, covering model discovery and deployment, complete dev lifecycle of AI agent, evaluation workflows, and troubleshooting. Pre-Execution Requirements Follow each applicable subsection below before starting its corresponding action or workflow. Dependency Check and Setup MANDATORY: As the first step after this skill loads, run the dependency check and setup script below from this skill's root and wait for it to finish before continuing. The script checks first and installs only missing dependencies; it does not reinstall dependencies that are already available. You MUST complete this check before reading or entering any sub-skill, workflow, or workflow-specific reference. ./scripts/check-and-setup-dependencies.sh # macOS / Linux ./scripts/check-and-setup-dependencies.ps1 # Windows (pwsh) Strictly follow the script output for subsequent actions. Workflow Guidance MANDATORY: Before executing ANY workflow-specific steps, you MUST read the corresponding sub-skill document. Do not call workflow-specific MCP tools for a workflow without reading its skill document. This applies even if you already know the MCP tool parameters — the skill document contains required workflow steps, pre-checks, and validation logic that must be followed. This rule applies on every new user message that triggers a different workflow, even if the skill is already loaded. Foundry MCP MANDATORY: Before using Foundry MCP operations, call the Azure MCP foundry tool and inspect the available Foundry MCP tools and related parameters. Treat this as the discovery/help step for MCP-based workflows. azd MANDATORY: Before executing ANY azd command, you MUST read azd-guidance and strictly follow the shared rules defined in it, especially the AZURE_DEV_USER_AGENT setting rules. Sub-Skills This skill includes specialized sub-skills for specific workflows. When a sub-skill matches the task, strictly follow its workflow: Sub-Skill When to Use Reference deploy Deploy hosted agents to Foundry, smoke-test a deployment, create or update prompt agents, and manage agent versions and multi-environment deploys. deploy cicd Set up a CI/CD deployment pipeline for a Foundry agent. cicd invoke Send messages to an agent, single or multi-turn conversations invoke routine Schedule or event-trigger Foundry agents with routines; use azd for CRUD, enable/disable, manual dispatch, and viewing past runs, or define routines in azure.yaml . routine invocations-ws Build, deploy, and connect to hosted agents that speak the invocations_ws duplex WebSocket protocol — voice agents, real-time streams, and signaling for out-of-band media transports. invocations-ws observe Evaluate agent quality, run batch evals, analyze failures, optimize prompts, improve agent instructions, compare versions, set up CI/CD monitoring, and enable continuous production evaluation observe trace Query traces, analyze latency/failures, correlate eval results to specific responses via App Insights customEvents trace troubleshoot View hosted agent logs, query telemetry, diagnose failures troubleshoot validate Use only when the user explicitly asks to use this validation sub-skill or to validate Microsoft Foundry hosted-agent code against best practices. Never invoke it proactively or add it to another workflow. validate create (quick start) Create a new hosted Foundry agent from scratch end-to-end — scaffold, provision or use an existing Foundry project, deploy, and smoke-test. Do not use for any work on existing code. For anything not covered by the quickstart, use create . create/quick-start-hosted.md create Use when the standard end-to-end happy path (quick start) doesn't fit. Create a new Foundry agent, update code of an existing agent, continue development of an existing agent, wire connections at scaffold time, use advanced setup or A2A (Agent2Agent), or recover from a failed quickstart run. create agent-optimizer Make existing Python hosted-agent code optimization-ready, configure eval.yaml, run Agent Optimizer jobs, apply candidates locally, and deploy through azd after review. agent-optimizer eval-datasets Harvest production traces into evaluation datasets, manage dataset versions and splits, track evaluation metrics over time, detect regressions, and maintain full lineage from trace to deployment. Use for: create dataset from traces, dataset versioning, evaluation trending, regression detection, dataset comparison, eval lineage. eval-datasets project/create Creating a new Microsoft Foundry project for hosting agents and models. Use when onboarding to Foundry or setting up new infrastructure. project/create/create-foundry-project.md resource/create Creating Azure AI Services multi-service resource (Foundry resource) using Azure CLI. Use when manually provisioning AI Services resources with granular control. resource/create/create-foundry-resource.md private-network Answer questions about Foundry network isolation and deploy Foundry with VNet isolation (BYO VNet, Managed VNet, hybrid). Covers architecture concepts, template selection, deployment, and post-deployment validation. resource/private-network/private-network.md models/deploy-model Unified model deployment with intelligent routing. Handles quick preset deployments, fully customized deployments (version/SKU/capacity/RAI), and capacity discovery across regions. Routes to sub-skills: preset (quick deploy), customize (full control), capacity (find availability). models/deploy-model/SKILL.md quota Managing quotas and capacity for Microsoft Foundry resources. Use when checking quota usage, troubleshooting deployment failures due to insufficient quota, requesting quota increases, or planning capacity. quota/quota.md rbac Managing RBAC permissions, role assignments, managed identities, and service principals for Microsoft Foundry resources. Use for access control, auditing permissions, and CI/CD setup. rbac/rbac.md finetuning Fine-tune models on Microsoft Foundry — SFT distillation, DPO preference optimization, RFT with graders and tool calling. Dataset preparation, grader calibration, training, checkpoint selection, deployment, evaluation. Use for: fine-tune, SFT, DPO, RFT, training data, grader, distillation, fine-tuned model, large file upload. finetuning/SKILL.md azd-guidance Provide shared azd knowledge and guidance for managing Foundry agents. Read this first for any workflows related to azd. azd-guidance 💡 Tip: For a complete onboarding flow: project/create (public) or private-network (VNet isolation) → models/deploy-model → agent workflows ( create → deploy → invoke ). 💡 Fine-Tuning: Use finetuning for all model customization — SFT distillation, DPO preference optimization, and RFT with graders. Includes quickstart, grader calibration, and training curve analysis. 💡 Model Deployment: Use models/deploy-model for all deployment scenarios — it intelligently routes between quick preset deployment, customized deployment with full control, and capacity discovery across regions. 💡 Prompt Optimization: For requests like "optimize my prompt" or "improve my agent instructions," load observe and use the prompt_optimize MCP tool through that eval-driven workflow. Infrastructure Lifecycle Match user intent to the correct infrastructure workflow. User Intent Workflow "Create Foundry" / "Set up Foundry" (ambiguous) Use AskUserQuestion : (a) just an AI Services resource, (b) a project with public access, or (c) a project with network isolation? Route: (a) → resource/create , (b) → project/create , (c) → private-network Set up Foundry with VNet isolation private-network Create a Foundry project (public) project/create Create a bare Foundry resource resource/create Agent Development Lifecycle Match user intent to the correct agent workflow. Read each sub-skill in order before executing. User Intent Workflow (read in order) Create a new hosted agent end-to-end (scaffold + deploy + test) dependency check and setup → azd-guidance → quick-start-hosted (self-contained end-to-end) Anything beyond the standard quickstart (existing code, migration, re-hosting, deployment customization, scaffold-time connections, A2A (Agent2Agent), recovery) dependency check and setup → azd-guidance → create → deploy → invoke Optimize existing Python hosted agent dependency check and setup → azd-guidance → agent-optimizer → scaffold/review → eval.yaml → optimize → apply candidate → deploy → invoke Deploy an agent (code already exists) dependency check and setup → azd-guidance → deploy (includes eval-suite setup) → invoke → observe (evaluate/optimize) Update/redeploy an agent after code changes dependency check and setup → azd-guidance → deploy (includes eval-suite setup) → invoke → observe (evaluate/optimize) Set up a CI/CD deployment pipeline for a hosted agent dependency check and setup → azd-guidance → cicd Invoke/test/chat with an agent dependency check and setup → azd-guidance → invoke Schedule/event-trigger an agent, or CRUD/enable/disable/dispatch a routine dependency check and setup → azd-guidance → routine Optimize / improve agent prompt or instructions observe (Step 4: Optimize) Evaluate and optimize agent (full loop) observe Enable continuous evaluation monitoring observe (Step 6: CI/CD & Monitoring) Troubleshoot an agent issue dependency check and setup → azd-guidance → invoke → troubleshoot Fix a broken agent (troubleshoot + redeploy) dependency check and setup → azd-guidance → invoke → troubleshoot → apply fixes → deploy → invoke Agent: .foundry Workspace Standard Every agent source folder can keep Foundry-specific cache and overlay state under .foundry/ : <agent-root>/ .foundry/ agent-metadata.yaml agent-metadata.prod.yaml suites/ datasets/ evaluators/ results/ In azd projects, derive deployment context (project endpoint, agent name/version, ACR, App Insights) from azure.yaml plus azd env get-values ; do not duplicate those values in metadata when azd already provides them. agent-metadata.yaml is the preferred local/dev overlay for non-azd values, remote Foundry suite references, local cache paths, result summaries, and explicit overrides. Optional sidecar files such as agent-metadata.prod.yaml can hold a single prod or CI-targeted overlay without mixing multiple environments in one file. suites/ , datasets/ , and evaluators/ are local cache folders. Reuse them when they are current, and ask before refreshing or overwriting them. See Agent Metadata Contract for the canonical schema and workflow rules. Agent: Setup References Standard Agent Setup — advanced setup for production workloads that need data-residency control (bring-your-own Cosmos DB / Storage / AI Search via a Foundry capability host). The default azd ai agent flow uses Basic Agent Setup and does not provision capabilityHosts/agents — do not flag its absence as a bug. For default post-provision state, see the "Expected env-var fingerprint" section in foundry-agent/create/create-hosted.md . Agent: Common Project Context Resolution Agent skills should run this step only when they need configuration values they don't already have . If a value (for example, agent root, environment, project endpoint, or agent name) is already known from the user's message or a previous skill in the same session, skip resolution for that value. Step 1: Discover Agent Roots and azd Context First check whether the workspace has azure.yaml with services using host: azure.ai.agent . One azd agent service -> use that service's project folder as the agent root. Multiple azd agent services -> require the user to choose the target service/folder. No azd agent service -> search the workspace for .foundry/ folders that contain agent-metadata.yaml or agent-metadata.<env>.yaml . One match -> use that agent root. Multiple matches -> require the user to choose the target agent folder. No matches -> for create/deploy workflows, seed a new .foundry/ folder during setup; for all other workflows, stop and ask the user which agent source folder to initialize. After selecting an agent root, keep all local .foundry cache inspection, source inspection, evaluator suggestions, dataset suggestions, and prompt-optimization context inside that folder only. Do not scan sibling agent folders unless the user explicitly switches roots. Step 2: Resolve Environment and Deployment Context If azure.yaml is present, resolve the azd environment first: Environment explicitly named by the user AZURE_ENV_NAME from azd env get-values azd default environment from .azure/config.json Environment already selected earlier in the session Run azd env get-values for the selected environment when project/deployment values are not already known. Prefer azd values for deployment context: azd Variable Resolves To AZURE_AI_PROJECT_ENDPOINT or AZURE_AIPROJECT_ENDPOINT Project endpoint AGENT_<SERVICE>_NAME Agent name for the selected azd service AGENT_<SERVICE>_VERSION Agent version for the selected azd service AZURE_CONTAINER_REGISTRY_NAME or AZURE_CONTAINER_REGISTRY_ENDPOINT ACR registry name / image URL prefix APPLICATIONINSIGHTS_CONNECTION_STRING App Insights connection string for trace workflows AZURE_SUBSCRIPTION_ID , AZURE_RESOURCE_GROUP , AZURE_AI_ACCOUNT_NAME , AZURE_AI_PROJECT_NAME Azure resource lookup and Playground links When azd supplies these values, use them as the source of truth and do not copy them into .foundry/agent-metadata*.yaml on metadata writes. Step 3: Select Metadata Overlay and Resolve Environment Inside the selected agent root, choose the metadata file in this order: Metadata filename or path explicitly provided by the user or workflow If an explicit environment is already known and .foundry/agent-metadata.<env>.yaml exists, use that file .foundry/agent-metadata.yaml If multiple metadata files remain and no rule above selects one, prompt the user to choose Read the selected metadata file and resolve any remaining environment choice in this order: Environment explicitly named by the user If the selected metadata file defines exactly one environment, use it Environment already selected earlier in the session defaultEnvironment from metadata If the selected metadata file still contains multiple environments and none of the rules above selects one, prompt the user to choose. Keep the selected agent root, metadata file, environment, and whether context came from azd or metadata visible in every workflow summary.