pytorch-training

name pytorch-training description Best practices for building robust PyTorch training loops. Use when generating or reviewing ML training code. metadata {"category":"tooling","trigger-keywords":"training,pytorch,torch,deep learning,neural network,model","applicable-stages":"10,12","priority":"3","version":"1.0","author":"researchclaw","references":"PyTorch Performance Tuning Guide, pytorch.org","code-template":"import torch\nimport torch.nn as nn\nfrom torch.utils.data import DataLoader\n\n# Reproducibility\ntorch.manual_seed(seed)\ntorch.cuda.manual_seed_all(seed)\ntorch.backends.cudnn.deterministic = True\n\n# Training loop\nmodel.train()\nfor epoch in range(num_epochs):\n for batch in train_loader:\n optimizer.zero_grad(set_to_none=True)\n loss = criterion(model(batch['input']), batch['target'])\n loss.backward()\n torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)\n optimizer.step()\n scheduler.step()\n"} PyTorch Training Best Practice Use torch.manual_seed() for reproducibility (set for torch, numpy, random) Use DataLoader with num_workers>0 and pin_memory=True for GPU Enable cudnn.benchmark=True for fixed input sizes Use learning rate schedulers (CosineAnnealingLR or OneCycleLR) Implement early stopping based on validation metric Log metrics every epoch, save best model checkpoint Use torch.no_grad() for evaluation Clear gradients with optimizer.zero_grad(set_to_none=True) for efficiency