Files
ai-toolkit/toolkit
Booyaka101 67e223b502 perf: enable pin_memory on CUDA DataLoaders for faster CPU->GPU transfer
torch DataLoader's pin_memory was never set, defaulting to False. Setting
pin_memory=True allocates batch tensors in page-locked memory, which lets
PyTorch issue async cuda host->device copies via DMA - typically 10-30%
faster transfer for image/video training tensors and can reduce per-step
overhead noticeably on Windows where num_workers is forced to 0 (see
6d31c6db / #770) and the main process pays the full transfer cost.

pin_memory is orthogonal to num_workers and is safe with num_workers=0,
so this benefits Windows/macOS users (who currently can't use multi-worker
loading due to pickling constraints) without changing any other behavior.
Gated on torch.cuda.is_available() so it's a no-op on CPU/MPS/ROCm-without
-CUDA-API setups.

Refs #758.
2026-05-01 23:07:43 +08:00
..
2024-07-20 08:21:57 -06:00
2023-07-05 16:44:58 -06:00
2024-07-10 15:20:04 -06:00
2026-04-19 09:51:46 -06:00
2025-04-17 09:00:10 -06:00
2023-08-13 10:25:56 -06:00
2025-12-18 10:08:59 -07:00
2026-01-13 04:55:30 -07:00
2025-10-26 05:47:25 -06:00
2025-06-12 08:00:51 -06:00
2024-02-13 16:00:04 -07:00
2023-07-28 08:16:29 -06:00