@zappazack found a real regression: pin_memory page-locks RAM that
NVIDIA's Windows driver wants for shared-memory VRAM-overflow fallback
when VRAM saturates. Page-locked pages can't be relocated, so the driver
falls back to PCIe thrashing - step times went from 1.5s to 20+s when
VRAM saturated on RTX 5090 + Qwen LoRA + 1024x1024.
The original "strict-improvement" claim was wrong. Convert to opt-in:
- Add `pin_memory: bool` field on DatasetConfig (default False)
- Only pin when both CUDA is available AND user opts in via config
Users with stable VRAM headroom can opt in for the transfer speedup.
Users at the VRAM ceiling stay safe with the default.
Refs #758.
torch DataLoader's pin_memory was never set, defaulting to False. Setting
pin_memory=True allocates batch tensors in page-locked memory, which lets
PyTorch issue async cuda host->device copies via DMA - typically 10-30%
faster transfer for image/video training tensors and can reduce per-step
overhead noticeably on Windows where num_workers is forced to 0 (see
6d31c6db / #770) and the main process pays the full transfer cost.
pin_memory is orthogonal to num_workers and is safe with num_workers=0,
so this benefits Windows/macOS users (who currently can't use multi-worker
loading due to pickling constraints) without changing any other behavior.
Gated on torch.cuda.is_available() so it's a no-op on CPU/MPS/ROCm-without
-CUDA-API setups.
Refs #758.
* Base ace step 1.5 xl added. Generating, still wip on training and ui
* Base training code done
* Fix some issues with caching text embeddings. Update sample cards to show audio
* Fix issue with quantizing ace step
* Add album artwork to samples with waveform.
* Cleanup logs
* Add album art endpoint to speed up album art loading
* Made an make video with artwork script
* Make ui handle basic audio models. Make multi line adjustments to the editor and better syntax hilighting.
* Add prompt tagging system for special tagged models.
* prompt tagging processing for ui working.
* Moved default samples to a special file so we can add more when needed and they can be adjusted for a specific model
* Add a captioner job with music captioner that is prepped for use with the ui
* Add basit ui setup for captioning modal and handeling captioning jobs
* Starting captioning job from ui working. Still better management for it.
* Better filtering of job options in the job view for captioning jobs
* Added qwen3 vl as a captioner for images
* Have an indicator when a dataset is being captioned.
* Adjust the way caption jobs look in the queue
* Fix a few issues. Adjust defaults.
* Version bump
* Added ace step to the readme.
* Made an install script and auto updates env for mac
* GPU sensors and initial training working for MAC. Still WIP.
* Switch dataloader to single threaded until I can work around some mac pickeling issues.
* Get quantization working on mac
* Fix mac exclusive imports so they don't break other builds.
* Add mac instructions to the UI
Jaret Burkett
Booyaka101
Rodrigo Reis