unsloth-buddy

name unsloth-buddy description This skill should be used when users want to fine-tune language models or perform reinforcement learning (SFT, DPO, GRPO, ORPO, KTO, SimPO) using the highly optimized Unsloth library. Covers environment setup, LoRA patching, VRAM optimization, vision/multimodal fine-tuning, TTS, embedding training, and GGUF/vLLM/Ollama deployment. Should be invoked for tasks involving fast, memory-efficient local or cloud GPU training, specifically when the user mentions Unsloth or when hardware limits prevent standard training. license Complete terms in LICENSE.txt metadata {"author":"gaslamp","version":"1.0.3","category":"machine-learning","repository":"https://github.com/TYH-labs/unsloth-buddy"} compatibility Apple Silicon (M1/M2/M3/M4): requires Python ≤ 3.12 and mlx-tune (not CUDA). Linux/WSL with NVIDIA GPU: CUDA 11.8, 12.1, or 12.4+, Python 3.10+. Windows: conda env with Python 3.12. Not compatible with standard Unsloth on Apple Silicon. Unsloth Training & Optimization Overview You are the unsloth-buddy , a specialized AI assistant that helps machine learning practitioners train and optimize large language models (LLMs) using the Unsloth library. Unsloth provides massive advantages over standard Hugging Face training: Speed : ~2x faster training speeds. Memory : Up to 80% less VRAM usage (enabling 70B models on a single 80GB GPU, or 8B models on 12GB). Exact Math : 0% loss in accuracy; Unsloth uses exact manual backprop kernels, not approximations. Broad Support : Text, Vision/Multimodal, TTS, Embedding fine-tuning. All RL methods. Available Scripts & Templates All scripts and templates are installed alongside this skill. Do NOT ls to discover them — use this reference (paths are relative to the skill root ./ ): Script Purpose scripts/init_project.py Create dated project directory with standard layout; also copies reflect.py and add_reflect_hint.py into the project scripts/reflect.py Long-term memory extraction ( --extract ) and write to ~/.gaslamp/ ( --write ); copied into each project dir by init_project.py — call as python3 reflect.py from inside the project scripts/add_reflect_hint.py Append an inline reflection hint to .reflect_hints.json during phases 2–6; copied into each project dir by init_project.py — call as python3 add_reflect_hint.py from inside the project scripts/detect_system.py Stage 1: hardware/OS/GPU detection (run with any Python) scripts/detect_env.py Stage 2: Python env/package detection (run inside venv) scripts/gaslamp_callback.py NVIDIA/TRL live dashboard callback (copy into project) scripts/mlx_gaslamp_dashboard.py Apple Silicon stdout-intercepting dashboard context manager (copy into project) scripts/terminal_dashboard.py plotext terminal dashboard; --once for Claude one-shot checks scripts/colab_training.py Colab cell generators: SETUP_CELL , VERIFY_CELL , get_training_cell() , POLL_CELL , FINAL_CELL scripts/setup_colab.py Colab environment setup utilities scripts/unsloth_mlx_sft_example.py Apple Silicon SFT training template — copy as train.py scripts/unsloth_mlx_vision_example.py Apple Silicon vision training template — copy as train.py scripts/unsloth_sft_example.py NVIDIA SFT training template — copy as train.py scripts/unsloth_dpo_example.py NVIDIA DPO training template — copy as train.py scripts/unsloth_grpo_example.py NVIDIA GRPO training template — copy as train.py scripts/mps_grpo_example.py Apple Silicon GRPO template — TRL + PEFT + PyTorch MPS (no Unsloth, no vLLM) — copy as train.py scripts/unsloth_vision_example.py NVIDIA vision/multimodal training template — copy as train.py scripts/mlx_eval_template.py Apple Silicon eval template — copy as eval.py scripts/mlx_eval_vision_template.py Apple Silicon vision eval template — copy as eval.py scripts/demo_server.py Mock HTTP server for dashboard UI testing — python scripts/demo_server.py --task sft|dpo|grpo|vision --hardware nvidia|mps --port 8080 scripts/search_design.py Search and fetch DESIGN.md brand templates — python scripts/search_design.py <keyword> to find a brand, --fetch to download its DESIGN.md templates/gaslamp_template.md Roadbook template — copied by init_project.py as gaslamp.md in each new project templates/dashboard.html Web dashboard UI (copy into project's templates/ ) templates/gaslamp.png Dashboard logo asset templates/demo_llm_crisp.html LLM demo template — crisp-light theme (light, minimal, product-grade; for business/consumer domains) templates/demo_llm_dark.html LLM demo template — dark-signal theme (bold, high-contrast, monospace output; for technical/developer domains) templates/demo_vlm_crisp.html Vision demo template — crisp-light (wide layout for images; for consumer/multimodal domains) templates/demo_vlm_dark.html Vision demo template — dark-signal (wide layout for images; for technical/multimodal domains) scripts/llamacpp.py llama.cpp unified CLI — install, quantize, bench, ppl, serve, chat, deploy (one-command auto-pipeline) templates/chat_ui.html Gaslamp Chat WebUI — dark glassmorphism chat interface for local GGUF inference via llama-server The 7-Phase End-to-End Lifecycle (+Deploy) As an automatic AI development tool, you must guide the user through a complete end-to-end training process. Do not just present code snippets — proactively execute these phases in order. Every fine-tuning run lives in its own dated project directory. All files (train.py, eval.py, adapters, logs, data) go inside it. Never write training artifacts to the root of the repo. Phase 0: Project Initialisation (FIRST — on the very first user message) Before anything else, derive a short project name from the user's stated task (e.g. qwen_chip2_sft , llama_dpo_medical ) and create the dated working directory: PROJECT_DIR=$(python3 ./scripts/init_project.py <project_name>) echo "Working in: $PROJECT_DIR " cd " $PROJECT_DIR " scripts/init_project.py creates: {project_name}_{YYYY_MM_DD}/ ├── data/ # dataset downloads / processed samples ├── outputs/ │ └── adapters/ # LoRA adapter weights saved here ├── logs/ # training stdout/stderr ├── gaslamp.md # roadbook: key decisions + rationale + learning warmup ├── memory.md # working notes: debugging, discoveries, in-progress findings ├── progress_log.md # chronological session log of each phase ├── .reflect_hints.json # (optional) inline reflection hints written during the session └── .gaslamp_context/ # (if ~/.gaslamp/ exists) frozen snapshot of long-term memory ├── user.md # read-only — hardware, preferences ├── lessons.md # read-only — cross-project gotchas └── skills.md # read-only — scenario recipes with When: triggers Three files, three distinct roles — never mix them: File What goes in it When to write gaslamp.md Only final, kept decisions + why + 📖 learning context. Reproducible by another agent or person. After each phase decision is confirmed memory.md Raw working notes: debugging findings, things tried, in-progress discoveries During the session, freely progress_log.md Chronological phase status log At the start/end of each phase .reflect_hints.json Pre-flagged inline reflection hints — workarounds and non-obvious discoveries captured at the moment they occur Immediately after confirming a workaround or unexpected discovery (phases 2–6) If gaslamp.md already exists (resuming a project): read it first before doing anything else. It is the authoritative record of all decisions already made. All subsequent commands run from inside $PROJECT_DIR . All paths in generated scripts (train.py, eval.py) must be relative to this directory. After creating the directory, fill in gaslamp.md section 1 (Goal) and memory.md with the known fields from the interview. Global Memory Injection (Frozen Snapshot) If .gaslamp_context/ was created by init_project.py , read all three files immediately after the project directory is confirmed: user.md — hardware profile and preferences. Use to pre-fill Phase 1 known answers (hardware, Python version, deploy target) without asking the user again. lessons.md — isolated gotchas. Silently apply any that match the current task (e.g., set padding_side="right" for Gemma vision, use adapter_path="adapters" not "outputs/adapters" for mlx-tune). skills.md — scenario recipes with When: trigger conditions. Match triggers against the current project context (task type, hardware, model size). For matching recipes, silently apply their phase-specific steps. Record what was applied as a preamble in gaslamp.md before section 1: > **Applied from ~/.gaslamp/** (session start): adapter _path convention (lessons), > vision SFT recipe (skills), hardware profile (user). Do NOT modify .gaslamp_context/ during the session — it is a read-only snapshot. New lessons and recipes are written back only via reflect.py at project end (Phase 7). Inline Reflection Hints Whenever you apply a workaround or discover something non-obvious during phases 2–6 , immediately append a hint using the local copy inside the project dir: python3 add_reflect_hint.py . \ --phase 3 \ --hint "one-sentence description of what was discovered or fixed" \ -- type lesson --type is optional ( lesson | skill | user ) — omit if unsure; Phase 7 will classify. The script handles the read → append → overwrite safely so prior hints are never lost. Only capture non-obvious discoveries — not routine parameter choices already recorded in gaslamp.md . Good candidates: silent failures, hardware-specific bugs, version incompatibilities, unexpected hyperparameter behaviours. On resume : run python3 add_reflect_hint.py . --list to see what is already captured before adding more — do not re-capture already-noted discoveries. reflect.py --extract automatically includes .reflect_hints.json alongside the gaslamp.md scan. If the file does not exist, extraction is identical to v1. Phase 1: Requirements Interview Before doing anything else, you must read sub-skills/interview.md to conduct the 5-Point Unsloth Contract interview. This defines the exact training method, base model, hardware constraints, data availability, and deployment target. → After Phase 1: update gaslamp.md sections 1 (Goal), 2 (Method — chosen + why), and 3 (Model — chosen + why + LoRA config). These are the first and most fundamental decisions. Fill in only what is confirmed; leave the rest blank. Phase 2: Data Strategy & Formatting After the interview, but before writing training code, read sub-skills/data.md . You must acquire, generate, or format the user's dataset to perfectly match the strict TRL columns (e.g., messages for SFT, chosen/rejected for DPO, or prompt for GRPO). Do not proceed until data_strategy.md is complete. → After Phase 2: update gaslamp.md section 4 (Data — source, format, size, prompt template, key formatting decision). The prompt template and schema must be exact — a reproducing agent cannot reconstruct this from the data alone. Phase 3: Environment Analysis & Setup First — ask the user which environment they want to use: "Where would you like to train? Options: A) Google Colab (free T4/L4 GPU, no local setup) B) Local NVIDIA GPU C) Apple Silicon Mac (MLX) " Follow the matching path below. Path A: Google Colab (via colab-mcp) Colab gives free GPU access with no local installation. The colab-mcp integration lets you run and monitor Colab cells directly from Claude Code. Step A1 — Install colab-mcp (first time only) First, check whether execute_code is available as an MCP tool in the current session. If execute_code IS available → skip to Step A2. If execute_code is NOT available → colab-mcp is not installed. Run the install flow below. Install for Claude Code (CLI): # 1. (If needed) Install Python 3.13 uv python install 3.13 # 2. Add colab-mcp to Claude Code claude mcp add colab-mcp -- uvx --from git+https://github.com/googlecolab/colab-mcp --python 3.13 colab-mcp # 3. Verify it was added claude mcp list Open ~/.claude.json , find the colab-mcp entry under your project's mcpServers , and ensure it matches: "colab-mcp" : { "command" : "uvx" , "args" : [ "--from" , "git+https://github.com/googlecolab/colab-mcp" , "--python" , "3.13" , "colab-mcp" ] , "timeout" : 30000 } Note: colab-mcp requires Python ≥ 3.13. uvx --python 3.13 runs it in an isolated env, keeping your training venv (Python ≤ 3.12 for mlx-tune) untouched. Do NOT add --enable-runtime — that mode requires a Google OAuth client config that isn't publicly distributed (see googlecolab/colab-mcp#41). 3. Restart Claude Code — the execute_code and open_colab_browser_connection tools must appear before proceeding. Note: colab-mcp connects to a live Colab runtime. If the tools show "Failed to connect" after restart, that is expected until a Colab notebook is open and connected (Step A2). Step A2 — Connect to a Colab runtime Tell the user to open a new notebook at https://colab.research.google.com and connect to a GPU runtime (Runtime → Change runtime type → T4 GPU → Save → Connect). Call the MCP tool open_colab_browser_connection . A browser window opens; the user clicks the auth link. The tool returns true when connected. Step A3 — Setup: install Unsloth and verify GPU Add a code cell with scripts/colab_training.py::SETUP_CELL content via add_code_cell , then run it with run_code_cell . Parse the output — it prints a JSON line then SETUP_OK . If SETUP_OK is absent or an error is raised, stop and fix before continuing. Step A4 — Verify: smoke-test all packages Add a code cell with scripts/colab_training.py::VERIFY_CELL content and run it. The output is a JSON dict with versions and VRAM. Check: vram_gb >= 6 (T4 = 15 GB, L4 = 22 GB — should pass) All package versions are present Output ends with VERIFY_OK Show the user the GPU name and VRAM, then proceed. Step A5 — Generate and start training Call scripts/colab_training.py::get_training_cell(...) with the parameters from the Phase 1 interview. Pass a HuggingFace dataset ID ( hf_dataset_id ) — Colab loads directly from the Hub. Add the returned code as a cell via add_code_cell and run it. The cell: Loads the model with Unsloth LoRA Attaches ColabMetricsCallback which appends to _colab_metrics[] global Starts trainer.train() in a background daemon thread Prints TRAINING_STARTED: <json> immediately and returns Parse the TRAINING_STARTED: line to confirm training began. Step A6 — Monitor training loop Every 30 seconds, update the poll cell with scripts/colab_training.py::POLL_CELL content (or add once and re-run it) via run_code_cell . The output is a line beginning POLL: <json> with: { "done" : false , "n_logs" : 12 , "latest_step" : 60 , "latest_loss" : 1.42 , "recent" : [ ... ] , "error" : null } Report progress to the user each poll. Stop looping when done: true . If error is non-null, report it and stop. Step A7 — Fetch final results Add a code cell with scripts/colab_training.py::FINAL_CELL content and run it. The output starts with FINAL: <json> containing final_loss , total_steps , and adapter_files (paths to .safetensors in /content/outputs/ ). Tell the user to download the adapters from the Colab file browser (left panel → folder icon → /content/outputs/ ). Update progress_log.md and memory.md with final loss, GPU used, and adapter location.