evan/ai-toolkit
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,640 Commits 2 Branches 0 Tags
evan_dev
Commit Graph
237 Commits
Author SHA1 Message Date
Jaret Burkett 1beba8a96d Pass sample width/height/num_frames into cached sample prompt configs so reference sizing matches generation 2026-10-23 12:20:39 -06:00
Jaret Burkett 14f8f95090 Bump node js version for docker and installer 2026-10-21 21:17:12 -06:00
Jaret Burkett 15303bf876 Moved x0 logic to the model class instead of recalculating it multiple times in the trainer 2026-10-21 11:38:09 -06:00
Jaret Burkett 831ccfa7aa Added experimental band loss 2026-10-15 08:26:12 -06:00
Jaret Burkett 07abdbedc7 Add match_target_res for Qwen Image 2.1 so reference images are scaled to the target's pixel area with their own aspect ratio, consistently across text-embedding caching, training, and sampling, and key the embedding cache on it. 2026-09-22 13:57:49 -06:00
Jaret Burkett db8dbd68c2 Add support for training VLLMs, omni models, and LLMs 2026-09-15 17:47:02 -06:00
Jaret Burkett f56b5a1d40 Change hidream to be proper x0 prediction and use a custom weighing scheme for x0 models. 2026-09-10 11:49:54 -06:00
Jaret Burkett 3ab6eaa3c4 Show fft loss in loss log 2026-09-08 05:16:07 -06:00
Jaret Burkett 3e1cb5f67f Rework DFEs to work better with x0 models. Show their loss in the graph 2026-09-07 13:26:01 -06:00
Jaret Burkett bbb1fda457 Fix gradient accumulation scaling so grads scale to the same size as if it was one big batch. 2026-09-06 10:48:10 -06:00
Jaret Burkett 764b5064fb Migrate to a new DTO for latents to carry more information that a normal tensor such as audio. 2026-08-30 10:30:21 -06:00
Jaret Burkett 683fe8afc0 Stability improvements to model offloading. Added D-OPSD bleed loss as well. 2026-08-29 11:23:47 -06:00
Jaret Burkett da79ebce99 Add D-OPSD as a distillation handeling option for MiniMax H3 ref2va 2026-08-26 11:06:33 -06:00
Jaret Burkett 127d6f626d Rework img/video reference in Minimax H3 to more closely match the comfy ui implementation. 2026-08-15 07:13:54 -06:00
Jaret Burkett 97bf49edad Add support for video references in MiniMax H3 ref2va 2026-08-15 06:18:09 -06:00
Jaret Burkett 4b00b61257 When doing an audio loss, show the img and audio loss in the loss log 2026-08-12 18:58:54 -06:00
Jaret Burkett 21dc65972d Only add a blank control if the model has to have it, for unconditionals. Previously it always encoded a blank control on unconditional if the model could take one. Affects DOP and blank prompt preservations. 2026-08-11 08:25:23 -06:00
Jaret Burkett 8d4beedd04 Move guidance loss target selection when given a range before preservation so it is avaliable during preservation. 2026-08-11 06:57:33 -06:00
Jaret Burkett 257da9b586 Rework DOP so it works with caching text embeddings 2026-08-09 22:13:49 -06:00
Jaret Burkett f4e9130547 Fix race condition that can corrupt grads under certain conditions. 2026-08-07 14:52:47 -06:00
Jaret Burkett 817f3dcbcb Fix finite check on inverted masked prior 2026-08-07 09:49:12 -06:00
Jaret Burkett b811636ae4 Check for finite vs isnan on loss before backpropigating. to catch infinity overflows 2026-08-06 09:18:56 -06:00
Jaret Burkett 1e1418b22c Apply sigma to contrastive guidance to balance loss better. Prevent noise grads on images/non audio datasets. Prep for training adapters on MiniMax H3 2026-08-05 13:27:12 -06:00
Jaret Burkett 3afa270ab5 Fix audio losses for DOP and other preservation losses 2026-08-04 22:08:48 -06:00
Jaret Burkett 0f9094db95 Fix audio loss when doing do_guidance_loss 2026-08-04 21:43:31 -06:00
Jaret Burkett fb204b7677 Fixed for DFEs with pixelspace and video models 2026-07-27 11:12:02 -06:00
Jaret Burkett 479c72ada2 Add replacing triggers on prompts when caching text encoder 2026-07-19 09:03:05 -06:00
Jaret Burkett 4e50535478 Allow models to provide an additional loss 2026-06-28 09:49:36 -06:00
Jaret Burkett d14f6e567a Allow individual models to scale the loss after it is calculated. 2026-06-25 10:13:58 -06:00
Jaret Burkett 820d534d6e Add features for models that may need a non masked loss such as inpainiting. 2026-06-22 10:59:17 -06:00
Jaret Burkett cac3815b2c Make DOP run in a single backward pass, should be faster and more stable. Show dop loss on ui 2026-06-08 13:55:47 -06:00
Jaret Burkett 12304e170f Added some experimental loss targets 2026-05-24 14:13:23 -06:00
Jaret Burkett ec58dcde92 Add better gradient checkpointing to dfes 2026-05-18 09:25:23 -06:00
Jaret Burkett (Ostris) fc4d6ebf39 Add support for fine-tuning Hidream O1 (#831) 
* Initial support for hidream. Lora keys likely need work

* Fix saving for hidream-o1

* Remove dependence on flash attention for hidream o1

* Fix gradient checkpointing for hidream o1

* A lot of fixes for hidream. Handle loading and saving as comfy model.

* Omit layers not used in comfy. Fix issue with lora loading keys in comfy

* Version bumpo
 2026-05-12 11:15:16 -06:00
Jaret Burkett e9ab387dfd Fixed issue with qwen image edit models when using multiple control images when not caching text embeddings. 2026-04-30 11:59:12 +00:00
Jaret Burkett 1fc4ad3979 Add sapiens2 as a diffusion feature extractor 2026-04-27 15:59:03 -06:00
Jaret Burkett 998a02f30e Yet another pseudo_huber fix. Still drinking coffee. 2026-04-19 10:05:40 -06:00
Jaret Burkett fc85410c9a Fix issue with precision on pseudo_huber loss 2026-04-19 10:02:27 -06:00
Jaret Burkett 20a99258b8 Fix issue is is else on pseudo_huber loss 2026-04-19 09:59:18 -06:00
Jaret Burkett f4445cd78c Added psuedo_huber loss 2026-04-19 09:51:46 -06:00
Jaret Burkett beb40ae29b Add DFE8 with partial step 2026-04-17 17:40:16 -06:00
Jaret Burkett 233e292256 Added some experimental low step things for zeta 2026-04-13 09:37:34 -06:00
Jaret Burkett 171535833a Add Mac OS support for Apple Silicon (#770) 
* Made an install script and auto updates env for mac

* GPU sensors and initial training working for MAC. Still WIP.

* Switch dataloader to single threaded until I can work around some mac pickeling issues.

* Get quantization working on mac

* Fix mac exclusive imports so they don't break other builds.

* Add mac instructions to the UI
 2026-03-30 09:37:47 -06:00
Jaret Burkett 6a1fc54779 Add t0 loss target 2026-03-28 13:35:21 -06:00
Jaret Burkett 7e1fdc3844 Remove the 0.1 floor for amplification 2026-03-22 09:01:58 -06:00
Jaret Burkett 0f075fc45e Adjust signal amplification target. Allow signal amplification strength in config. 2026-03-22 08:30:13 -06:00
Jaret Burkett dcd98dc0d5 Add signal amplification 2026-03-21 07:44:18 -06:00
Jaret Burkett b04c64e0f8 Add a dino version of DFE 2026-03-04 08:20:37 -07:00
Jaret Burkett 1c74ca5d22 Add audio_loss_multiplier to scale audio loss to larger values if desired. 2026-02-19 11:57:44 -07:00
Jaret Burkett 5b5aadadb8 Add LTX-2 Support (#644) 
* WIP, adding support for LTX2

* Training on images working

* Fix loading comfy models

* Handle converting and deconverting lora so it matches original format

* Reworked ui to habdle ltx and propert dataset default overwriting.

* Update the way lokr saves to it is more compatable with comfy

* Audio loading and synchronization/resampling is working

* Add audio to training. Does it work? Maybe, still testing.

* Fixed fps default issue for sound

* Have ui set fps for accurate audio mapping on ltx

* Added audio procession options to the ui for ltx

* Clean up requirements
 2026-01-13 04:55:30 -07:00