@zappazack found a real regression: pin_memory page-locks RAM that
NVIDIA's Windows driver wants for shared-memory VRAM-overflow fallback
when VRAM saturates. Page-locked pages can't be relocated, so the driver
falls back to PCIe thrashing - step times went from 1.5s to 20+s when
VRAM saturated on RTX 5090 + Qwen LoRA + 1024x1024.
The original "strict-improvement" claim was wrong. Convert to opt-in:
- Add `pin_memory: bool` field on DatasetConfig (default False)
- Only pin when both CUDA is available AND user opts in via config
Users with stable VRAM headroom can opt in for the transfer speedup.
Users at the VRAM ceiling stay safe with the default.
Refs #758.
torch DataLoader's pin_memory was never set, defaulting to False. Setting
pin_memory=True allocates batch tensors in page-locked memory, which lets
PyTorch issue async cuda host->device copies via DMA - typically 10-30%
faster transfer for image/video training tensors and can reduce per-step
overhead noticeably on Windows where num_workers is forced to 0 (see
6d31c6db / #770) and the main process pays the full transfer cost.
pin_memory is orthogonal to num_workers and is safe with num_workers=0,
so this benefits Windows/macOS users (who currently can't use multi-worker
loading due to pickling constraints) without changing any other behavior.
Gated on torch.cuda.is_available() so it's a no-op on CPU/MPS/ROCm-without
-CUDA-API setups.
Refs #758.
Booyaka101