Booyaka101
0c8793ff47
fix: make pin_memory opt-in via dataset config (default off)
...
@zappazack found a real regression: pin_memory page-locks RAM that
NVIDIA's Windows driver wants for shared-memory VRAM-overflow fallback
when VRAM saturates. Page-locked pages can't be relocated, so the driver
falls back to PCIe thrashing - step times went from 1.5s to 20+s when
VRAM saturated on RTX 5090 + Qwen LoRA + 1024x1024.
The original "strict-improvement" claim was wrong. Convert to opt-in:
- Add `pin_memory: bool` field on DatasetConfig (default False)
- Only pin when both CUDA is available AND user opts in via config
Users with stable VRAM headroom can opt in for the transfer speedup.
Users at the VRAM ceiling stay safe with the default.
Refs #758 .
2026-05-02 11:02:41 +08:00
Booyaka101
67e223b502
perf: enable pin_memory on CUDA DataLoaders for faster CPU->GPU transfer
...
torch DataLoader's pin_memory was never set, defaulting to False. Setting
pin_memory=True allocates batch tensors in page-locked memory, which lets
PyTorch issue async cuda host->device copies via DMA - typically 10-30%
faster transfer for image/video training tensors and can reduce per-step
overhead noticeably on Windows where num_workers is forced to 0 (see
6d31c6db#770 ) and the main process pays the full transfer cost.
pin_memory is orthogonal to num_workers and is safe with num_workers=0,
so this benefits Windows/macOS users (who currently can't use multi-worker
loading due to pickling constraints) without changing any other behavior.
Gated on torch.cuda.is_available() so it's a no-op on CPU/MPS/ROCm-without
-CUDA-API setups.
Refs #758 .
2026-05-01 23:07:43 +08:00
Jaret Burkett
0d91fcee9e
Allow edit of captioning job
2026-04-30 16:00:16 -06:00
Jaret Burkett
eadc9a58af
Version bump
2026-04-30 06:00:58 -06:00
Jaret Burkett
e9ab387dfd
Fixed issue with qwen image edit models when using multiple control images when not caching text embeddings.
2026-04-30 11:59:12 +00:00
Jaret Burkett
deb409085a
Made it possible to use the flux 2 small decoder VAE when setting the vae_path manually for flux2 models
2026-04-30 05:42:31 -06:00
Jaret Burkett
7ccec8ec2c
Add checkpointing and a proper decode for flux 2 VAEs so they can be used with DFE
2026-04-30 04:27:13 -06:00
Jaret Burkett
b4f0efb025
Version Bump
2026-04-28 13:42:00 -06:00
Jaret Burkett
af6458d1b5
Enable caching of ACE step latents.
2026-04-28 13:39:20 -06:00
Jaret Burkett
77b8765939
Fix issue with hover styling on select components.
2026-04-28 10:55:20 -06:00
Jaret Burkett
43989cc19e
Add advanced config section to the captioner
2026-04-28 10:49:46 -06:00
Jaret Burkett
f972b750e6
Performance improvements to captioner. Add ability to set a default caption for ace captioner to avoid needing to caption the audio so we only transcribe
2026-04-28 10:15:11 -06:00
Jaret Burkett
acc6a36214
Scale DFE 9 to a velocity equiv weight to match flow matching gradient strength. Probably need to rework all DFEs to do this as the math checks out.
2026-04-28 09:10:02 -06:00
Jaret Burkett
1fc4ad3979
Add sapiens2 as a diffusion feature extractor
2026-04-27 15:59:03 -06:00
Jaret Burkett
67d67f8c1d
Ignore hidden files when captioning
2026-04-25 17:38:31 -06:00
Jaret Burkett
998a02f30e
Yet another pseudo_huber fix. Still drinking coffee.
2026-04-19 10:05:40 -06:00
Jaret Burkett
fc85410c9a
Fix issue with precision on pseudo_huber loss
2026-04-19 10:02:27 -06:00
Jaret Burkett
20a99258b8
Fix issue is is else on pseudo_huber loss
2026-04-19 09:59:18 -06:00
Jaret Burkett
f4445cd78c
Added psuedo_huber loss
2026-04-19 09:51:46 -06:00
Jaret Burkett
488878f354
Use hidden layers in the loss for DFE 7 and 8
2026-04-18 13:07:38 -06:00
Jaret Burkett
beb40ae29b
Add DFE8 with partial step
2026-04-17 17:40:16 -06:00
Jaret Burkett
7c4f18ce51
Fix ernie unpatchify
2026-04-17 12:03:39 -06:00
Jaret Burkett
8cb9649382
Add decode latent to ernie pipe
2026-04-17 12:01:09 -06:00
Jaret Burkett
67048df9f9
Merge branch 'main' of github.com:ostris/ai-toolkit
2026-04-17 06:05:40 -06:00
Jaret Burkett
be54094704
Version bump
2026-04-17 06:05:26 -06:00
Jaret Burkett
a513a1583e
Fixed issue where Qwen VL MOE captioner produced nonsense
2026-04-16 21:24:43 +00:00
Jaret Burkett
22ea3dd620
Fixed issue on some systems where Logger didnt have atty
2026-04-16 21:09:52 +00:00
Jaret Burkett
ab1ee4df34
Hotfix some issues with Wan models caused by diffusers and transformers updates
2026-04-16 20:53:50 +00:00
Jaret Burkett
0c18b39346
Version bump
2026-04-16 13:09:45 -06:00
Jaret Burkett
afb62b1fa5
Add support for Nucleus-Image
2026-04-16 13:09:10 -06:00
Jaret Burkett
2faba22b46
Fix issue when saving advanced prompt embeds. No such file or directory error
2026-04-16 12:22:56 -06:00
Jaret Burkett
0792352dab
version bump
2026-04-16 09:25:14 -06:00
Jaret Burkett
8f67f5022e
Version lock peft version
2026-04-15 14:17:26 -06:00
Jaret Burkett
acc3e60140
Add ernie to readme
2026-04-15 06:43:36 -06:00
Jaret Burkett
dd7074a21f
Fix issue with layer offloading on ernie
2026-04-14 19:19:02 -06:00
Jaret Burkett
e74bc9ac7b
Fix issue with concatinating advanced prompt embeds.
2026-04-14 16:04:34 -06:00
Jaret Burkett
7eb1226a6d
Fix issue with loading advanced prompt configs metadata
2026-04-14 15:42:09 -06:00
Jaret Burkett
97d8c05d75
Fix issue with captioner sometimes outputting not utf-8 characters
2026-04-14 14:37:21 -06:00
Jaret Burkett (Ostris)
3e0c904054
Add support for Baidu's ERNIE-Image ( #793 )
...
* Add support for ERNIE Image
* change float64 to float32
* Version bump
* Update ERNIE defaults
2026-04-14 09:45:12 -06:00
Asaf Agami
e868fca562
fix custom_flowmatch_sampler ( #783 )
2026-04-13 09:42:05 -06:00
Jaret Burkett
233e292256
Added some experimental low step things for zeta
2026-04-13 09:37:34 -06:00
Jaret Burkett
1058ef3513
Made AdvancedPromptEmbeds that is compatable with previous PromptEmbeds functionality, but is more streamlined and can accomidate more model embedding paradigms.
2026-04-11 10:45:47 -06:00
Jaret Burkett
0d11be41fa
Adjust default sample steps.
2026-04-10 17:46:34 -06:00
Jaret Burkett
62e18427b4
Adjust default sample steps.
2026-04-10 17:46:09 -06:00
Jaret Burkett
9b4e2d1b0b
More flac support
2026-04-10 12:27:09 -06:00
Jaret Burkett
0b9c365acb
Add flac and ogg support
2026-04-10 12:10:54 -06:00
Jaret Burkett
bfb373c8fa
Prep for future breaking changes in newer versions of transformers library
2026-04-10 12:04:32 -06:00
Jaret Burkett
145144eee3
Fix issue with auto updating captions when captioning a dataset
2026-04-10 11:51:18 -06:00
Jaret Burkett
765a9d5b2e
Add a download button to music samples in in the gear menu
2026-04-10 10:07:45 -06:00
Jaret Burkett
d08ea8318f
Change default timestep type for ace step to linear
2026-04-09 17:39:00 -06:00
Booyaka101
Jaret Burkett
Asaf Agami