最新博客文章

11 / 200

AI 开发实战教程，从零到一构建你的 AI 应用。

Tutorial Date

DeepSeek Data Engineering Tutorial Model Fine-tuning Advanced From data collection to quality evaluation, fully master the large model data engineering pipeline. Covers core aspects such as SFT dataset construction, RLHF preference data preparation, data cleaning and deduplication, data augmentation, and large-scale data processing. 2025-06 ·23 min read DeepSeek Model Architecture Deep Dive Model Fine-tuning Advanced In-depth analysis of DeepSeek-V3's core technologies such as MoE architecture, MLA attention mechanism, and Multi-Token Prediction. Understand how DeepSeek achieves GPT-4o-level performance at extremely low cost, including architecture diagrams and source code analysis. 2025-06 ·28 min read Model Distillation in Practice: Training Small Models with Large Models Model Fine-tuning Advanced Knowledge distillation allows small models to approach large model capabilities while significantly reducing inference costs. This article provides an in-depth explanation of distillation principles, data generation strategies, training techniques, and effect evaluation, teaching you to distill cost-effective small models with DeepSeek. 2026-08 ·24 min read DeepSeek Fine-tuning Hands-on Tutorial Model Fine-tuning Advanced Full practical guide to fine-tuning DeepSeek models: LoRA/QLoRA efficient fine-tuning, full-parameter fine-tuning, data preparation, training configuration, and evaluation validation. Covers fine-tuning solutions for DeepSeek-V3 and DeepSeek-R1, including complete code and best practices. 2025-06 ·26 min read Model Compression: Where Quantization, Pruning, and Distillation Fit Model Fine-tuning Advanced Quantization, pruning, and distillation are often compared in isolation, so it's hard to tell whether a compression win is real or just moved cost. This article puts them on one ledger: bit width, grouping and outliers for weights, how GPTQ and AWQ treat error propagation differently, where KV Cache quantization and FP8 fit, and structured versus unstructured pruning. 2026-10 ·44 min read Model Evaluation and A/B Testing Model Fine-tuning Advanced After fine-tuning, how can you objectively evaluate model performance? How should A/B tests be designed to obtain statistically significant conclusions? This article establishes a scientific model evaluation system from three dimensions: offline evaluation, online A/B testing, and statistical significance. 2026-07 ·20 min read RLHF Reinforcement Fine-Tuning in Practice: From Preference Data to Reward Model Model Fine-tuning Advanced RLHF aligns model outputs with human preferences. This article walks through the complete RLHF pipeline, from preference data collection and reward model training to PPO reinforcement fine-tuning, and provides engineering insights on data quality and effect evaluation. 2026-08 ·26 min read Distributed Training of Large Models: Data, Tensor, Pipeline, and Expert Parallelism Model Fine-tuning Advanced A single GPU cannot hold the model, so parallelism has to be chosen deliberately. Data, tensor, pipeline and expert parallelism are compared by communication pattern and usable scale: why tensor parallelism needs two all-reduces per layer, how communication grows with parallelism degree, how NVLink and cross-node bandwidth cap the choices, and what ZeRO 1/2/3 saves. 2026-10 ·26 min read Detailed Explanation of DPO Preference Alignment Training Model Fine-tuning Advanced DPO (Direct Preference Optimization) is a simplified alternative to RLHF that does not require training a reward model. This article explains the principles, implementation details, and training techniques of DPO in depth. 2026-07 ·25 min read Dataset Construction and Quality Evaluation Model Fine-tuning Advanced Data quality determines the upper limit of fine-tuning effectiveness. This article systematically explains the construction methods of fine-tuning datasets, data cleaning strategies, quality evaluation metrics, and data augmentation techniques, helping you build high-quality fine-tuning datasets. 2025-06 ·17 min read Synthetic Data Construction: From Self-Distillation to Adversarial Samples Model Fine-tuning Advanced Model output can't be gold data, and this article starts there. It walks the synthetic data pipeline: the self-distillation and rejection-sampling loop, why distribution collapse is a probability rather than correctness problem, how to quantify the quality-diversity trade-off, curricula and bucketed sampling matched to model ability, and adversarial samples for error injection. 2026-10 ·45 min read