Dataset is the Soul of Fine-tuning

When fine-tuning large models, data quality is far more important than data quantity. A few hundred high-quality samples can outperform tens of thousands of low-quality samples. Building a dataset requires a systematic methodology and strict quality control.

SFT Data Format

Supervised fine-tuning (SFT) data typically uses a conversation format:

[
  {
    "messages": [
      {"role": "system", "content": "You are a professional Python programming assistant"},
      {"role": "user", "content": "How to read a CSV file in Python?"},
      {"role": "assistant", "content": "You can use the read_csv function from the pandas library..."}
    ]
  },
  {
    "messages": [
      {"role": "system", "content": "You are a professional Python programming assistant"},
      {"role": "user", "content": "What is a list comprehension?"},
      {"role": "assistant", "content": "A list comprehension is a concise syntax for creating lists in Python..."}
    ]
  }
]

Data Sources and Collection

  • Human Annotation: Highest quality, but also highest cost. Suitable for core scenarios.
  • Model Generation + Human Review: First use strong models like GPT-4 to generate candidate data, then manually filter and modify.
  • Open-source Datasets: Such as ShareGPT, Alpaca, OpenOrca, etc., which need to be filtered before use.
  • User Feedback: Collect high-quality conversations from user interactions in production environments.

Data Cleaning Strategies

  1. Length Filtering: Conversations that are too short (<20 characters) or too long (>2000 characters) are usually of low quality.
  2. Duplicate Detection: Use MinHash or semantic similarity to detect and remove duplicate samples.
  3. Language Detection: Ensure data language consistency, avoiding mixed Chinese and English.
  4. Format Validation: Ensure JSON format is correct and fields are complete.
  5. Content Review: Filter samples containing sensitive, harmful, or low-quality content.

Data Quality Evaluation

def evaluate_dataset_quality(dataset):
    metrics = {
        "total_samples": len(dataset),
        "avg_instruction_length": 0,
        "avg_response_length": 0,
        "diversity_score": 0,
        "format_valid_rate": 0
    }

    instruction_lengths = []
    response_lengths = []

    for sample in dataset:
        messages = sample["messages"]
        user_msg = next(m["content"] for m in messages if m["role"]=="user")
        assistant_msg = next(m["content"] for m in messages if m["role"]=="assistant")

        instruction_lengths.append(len(user_msg))
        response_lengths.append(len(assistant_msg))

    metrics["avg_instruction_length"] = sum(instruction_lengths) / len(instruction_lengths)
    metrics["avg_response_length"] = sum(response_lengths) / len(response_lengths)
    metrics["diversity_score"] = calculate_diversity(instruction_lengths)

    return metrics

Data Augmentation Techniques

  • Self-Instruct: Let the model generate new instruction-response pairs itself.
  • Back-translation Augmentation: Chinese → English → Chinese, generating samples with the same meaning but different expressions.
  • Template Expansion: Generate variants with the same intent but different wording based on templates.
  • Difficulty Grading: Build samples with different difficulty levels to help the model learn progressively.

Best Practices

1. Quality over quantity: Prioritize data quality. 2. Coverage diversity: Ensure data covers various scenarios of the target domain. 3. Continuous iteration: Continuously supplement and optimize the dataset based on model performance. 4. Version management: Version control the dataset to trace every change.