evan/ai-toolkit
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,462 Commits 2 Branches 0 Tags
5fc8fd76dea55164c6f6b912e5540477ef3f45a8
Commit Graph
763 Commits
Author SHA1 Message Date
hitoshic1982 5fc8fd76de Fix cache_text_encoder device preset missing in BaseModel 
BaseModel.set_device_state_preset() has no branch for the
'cache_text_encoder' preset, so when TextEmbeddingCachingMixin calls
set_device_state_preset('cache_text_encoder') the active_modules list
stays empty and the text encoder is placed on CPU. With
cache_text_embeddings enabled on a BaseModel arch (e.g. chroma), every
caption is then encoded on CPU — with a quantized T5-XXL this took
~6.5 minutes per caption (~21 hours for a 205-image dataset) while the
GPU sat idle. StableDiffusion.set_device_state_preset() already has
this branch; this adds the same two lines to BaseModel.
 2026-08-29 01:37:39 +08:00
Jaret Burkett da79ebce99 Add D-OPSD as a distillation handeling option for MiniMax H3 ref2va 2026-08-26 11:06:33 -06:00
Jaret Burkett 8a912564ce Fix video DOP methods 2026-08-26 10:05:14 -06:00
Jaret Burkett 27a03a91f2 Rework progress bar and ui speed string so it gets the same value at a more accurate and snappier wall clock time 2026-08-21 13:31:02 -06:00
Jaret Burkett b96476a841 Allow overriding the batch size on a per dataset level 2026-08-21 12:56:04 -06:00
Jaret Burkett afd1d92722 Fix issue with double sampling qwen vl video frames for hidream h3 for reference videos 2026-08-17 15:39:07 -06:00
Jaret Burkett 127d6f626d Rework img/video reference in Minimax H3 to more closely match the comfy ui implementation. 2026-08-15 07:13:54 -06:00
Jaret Burkett 97bf49edad Add support for video references in MiniMax H3 ref2va 2026-08-15 06:18:09 -06:00
Jaret Burkett 5261d3fcca Add checkpointing to the Wan2.1 encoder 2026-08-14 07:31:16 -06:00
Jaret Burkett 6b7fb60a22 Make contrastive guidance loss do constant instead of sigma loss schedule by default 2026-08-13 09:52:02 -06:00
Jaret Burkett 7f9a142dfd Generate thumbnails for dataset items so things load faster and the ui is more stable. Especially good for videos. 2026-08-12 20:33:42 -06:00
Jaret Burkett a1ddeeef13 Fixed issue with offloading text encoder on ltx 2.5 2026-08-12 10:46:58 -06:00
Jaret Burkett 21dc65972d Only add a blank control if the model has to have it, for unconditionals. Previously it always encoded a blank control on unconditional if the model could take one. Affects DOP and blank prompt preservations. 2026-08-11 08:25:23 -06:00
Jaret Burkett f421542df4 When dropping out, caching, and doing DOP, make sure we select a cahced trigger word when dropped out so DOP matches the drop out embeddings. 2026-08-11 06:59:20 -06:00
Jaret Burkett ab5fef8970 Fix caption dropout so it now works identically when caching text embeddings. 2026-08-10 18:53:33 -06:00
Jaret Burkett 257da9b586 Rework DOP so it works with caching text embeddings 2026-08-09 22:13:49 -06:00
Jaret Burkett 72623ed3d6 When doing auto frame count. Ensure the time is not squeezed or expanded to fit tempooral spacing. trime the few extra frames. Also fixed frame counts of buckets. 2026-08-09 12:31:02 -06:00
Jaret Burkett 682b27c6ee Reworked freeing memory manager for removing text encoder completly when not needed. 2026-08-09 07:08:24 -06:00
Jaret Burkett 8c1a4082fd Allow images to work with auto frame count, and include images in video datasets if they exist. 2026-08-08 20:26:28 -06:00
Fitzy d184c6c622 Fix offload buffer stream lifetime 2026-08-08 19:59:23 -04:00
Jaret Burkett f4e9130547 Fix race condition that can corrupt grads under certain conditions. 2026-08-07 14:52:47 -06:00
Jaret Burkett 1e1418b22c Apply sigma to contrastive guidance to balance loss better. Prevent noise grads on images/non audio datasets. Prep for training adapters on MiniMax H3 2026-08-05 13:27:12 -06:00
Jaret Burkett 3afa270ab5 Fix audio losses for DOP and other preservation losses 2026-08-04 22:08:48 -06:00
Jaret Burkett 0f9094db95 Fix audio loss when doing do_guidance_loss 2026-08-04 21:43:31 -06:00
Jaret Burkett d870e9b68a Fix backwards compatability for older versions of triton 2026-08-04 21:07:57 -06:00
Jaret Burkett 00a93e3830 Add dataset flag to cache the raw tensors 2026-08-04 15:32:48 -06:00
Jaret Burkett 8a0bcf1ffe Fix issue loading H3 test encoder on older GPUS 2026-08-04 15:30:29 -06:00
Jaret Burkett a9a04547e9 Dont move encoders on and off device when caching until the first instance of needing to process a cache item. 2026-08-03 15:54:30 -06:00
Jaret Burkett 41676bb258 Queue up videos with multiple threads when caching latents so the VAE is not waiting on videos to process 2026-08-03 15:41:36 -06:00
Jaret Burkett d3a3f70a2a Speed up quantization processing on H3 2026-08-03 15:21:39 -06:00
Jaret Burkett bf739ff966 Fix issue with layer offloading with MinMax H3 2026-08-03 11:32:36 -06:00
Jaret Burkett 8502a845b1 Add support for MiniMax H3 T2V and I2V training 2026-08-03 10:17:39 -06:00
Jaret Burkett 73cab2acf5 Reworked merge in out of loras with convrot weights for better roundtrip accuracy. 2026-08-02 08:29:17 -06:00
Jaret Burkett a6f6b6b896 Fixed experiment opt name match 2026-08-01 17:43:36 -06:00
Jaret Burkett 6b95282097 Recover from issue when a video model first fram may not have been cached properly 2026-08-01 12:00:33 -06:00
Jaret Burkett 5baa495585 DFE optimizations 2026-08-01 10:06:16 -06:00
Jaret Burkett c68e58083f Add code for automagic experiment 2026-08-01 10:04:46 -06:00
Jaret Burkett 497014bf5d Set i2v to default to false when omitted from the config 2026-07-31 19:11:58 -06:00
Jaret Burkett 7b2386c096 Handle video codecs that fail in opencv 2026-07-29 21:01:57 -06:00
Jaret Burkett 9021caa723 Merge branch 'dev' of github.com:ostris/ai-toolkit into dev 2026-07-29 17:50:50 -06:00
Jaret Burkett 3f8afcac7e Allow dataloader to encode first frame with the text embeddings if the model needs it. 2026-07-29 17:50:45 -06:00
Jaret Burkett 3d472de2f1 Reworked spawning on dataloader so windows and mac can use multiple data loader workers now 2026-07-29 17:47:04 -06:00
Jaret Burkett c3afd95cc4 Improvements for mps convrot quants 2026-07-28 12:11:19 -06:00
Jaret Burkett 461e798708 Rework windows start and stop methods so that command windows dont appear. Stop with signal since we cannot signint. 2026-07-27 17:43:01 -06:00
Jaret Burkett 7e7053fc9a Remove k-sampler requirement and remove it form the cold. not used anymore anyway 2026-07-27 14:58:31 -06:00
Jaret Burkett b677cdb026 Move uintx quantization to ostris quant with bit identical matching. Now we are not bound to an older version of torch ao. 2026-07-27 12:59:42 -06:00
Jaret Burkett fb204b7677 Fixed for DFEs with pixelspace and video models 2026-07-27 11:12:02 -06:00
Jaret Burkett 0c3a5e6970 Default to lokr full rank when not passed. 2026-07-25 09:06:49 -06:00
Jaret Burkett efb58c8641 Switch to thumbnails and thumbnail creation on the sample grid page until clicked. 2026-07-25 08:40:33 -06:00
Jaret Burkett 8f2d001eae Improvements to video frame loading. Added ability to cache as uint8 pixelspace for video 2026-07-24 12:36:47 -06:00