Jaret Burkett
1beba8a96d
Pass sample width/height/num_frames into cached sample prompt configs so reference sizing matches generation
2026-10-23 12:20:39 -06:00
Jaret Burkett
14f8f95090
Bump node js version for docker and installer
2026-10-21 21:17:12 -06:00
Jaret Burkett
15303bf876
Moved x0 logic to the model class instead of recalculating it multiple times in the trainer
2026-10-21 11:38:09 -06:00
Jaret Burkett
831ccfa7aa
Added experimental band loss
2026-10-15 08:26:12 -06:00
Jaret Burkett
07abdbedc7
Add match_target_res for Qwen Image 2.1 so reference images are scaled to the target's pixel area with their own aspect ratio, consistently across text-embedding caching, training, and sampling, and key the embedding cache on it.
2026-09-22 13:57:49 -06:00
Jaret Burkett
db8dbd68c2
Add support for training VLLMs, omni models, and LLMs
2026-09-15 17:47:02 -06:00
Jaret Burkett
f56b5a1d40
Change hidream to be proper x0 prediction and use a custom weighing scheme for x0 models.
2026-09-10 11:49:54 -06:00
Jaret Burkett
3ab6eaa3c4
Show fft loss in loss log
2026-09-08 05:16:07 -06:00
Jaret Burkett
3e1cb5f67f
Rework DFEs to work better with x0 models. Show their loss in the graph
2026-09-07 13:26:01 -06:00
Jaret Burkett
bbb1fda457
Fix gradient accumulation scaling so grads scale to the same size as if it was one big batch.
2026-09-06 10:48:10 -06:00
Jaret Burkett
764b5064fb
Migrate to a new DTO for latents to carry more information that a normal tensor such as audio.
2026-08-30 10:30:21 -06:00
Jaret Burkett
683fe8afc0
Stability improvements to model offloading. Added D-OPSD bleed loss as well.
2026-08-29 11:23:47 -06:00
Jaret Burkett
da79ebce99
Add D-OPSD as a distillation handeling option for MiniMax H3 ref2va
2026-08-26 11:06:33 -06:00
Jaret Burkett
27a03a91f2
Rework progress bar and ui speed string so it gets the same value at a more accurate and snappier wall clock time
2026-08-21 13:31:02 -06:00
Jaret Burkett
127d6f626d
Rework img/video reference in Minimax H3 to more closely match the comfy ui implementation.
2026-08-15 07:13:54 -06:00
Jaret Burkett
97bf49edad
Add support for video references in MiniMax H3 ref2va
2026-08-15 06:18:09 -06:00
Jaret Burkett
4b00b61257
When doing an audio loss, show the img and audio loss in the loss log
2026-08-12 18:58:54 -06:00
Jaret Burkett
21dc65972d
Only add a blank control if the model has to have it, for unconditionals. Previously it always encoded a blank control on unconditional if the model could take one. Affects DOP and blank prompt preservations.
2026-08-11 08:25:23 -06:00
Jaret Burkett
8d4beedd04
Move guidance loss target selection when given a range before preservation so it is avaliable during preservation.
2026-08-11 06:57:33 -06:00
Jaret Burkett
257da9b586
Rework DOP so it works with caching text embeddings
2026-08-09 22:13:49 -06:00
Jaret Burkett
f4e9130547
Fix race condition that can corrupt grads under certain conditions.
2026-08-07 14:52:47 -06:00
Jaret Burkett
817f3dcbcb
Fix finite check on inverted masked prior
2026-08-07 09:49:12 -06:00
Jaret Burkett
b811636ae4
Check for finite vs isnan on loss before backpropigating. to catch infinity overflows
2026-08-06 09:18:56 -06:00
Jaret Burkett
1e1418b22c
Apply sigma to contrastive guidance to balance loss better. Prevent noise grads on images/non audio datasets. Prep for training adapters on MiniMax H3
2026-08-05 13:27:12 -06:00
Jaret Burkett
3afa270ab5
Fix audio losses for DOP and other preservation losses
2026-08-04 22:08:48 -06:00
Jaret Burkett
0f9094db95
Fix audio loss when doing do_guidance_loss
2026-08-04 21:43:31 -06:00
Jaret Burkett
461e798708
Rework windows start and stop methods so that command windows dont appear. Stop with signal since we cannot signint.
2026-07-27 17:43:01 -06:00
Jaret Burkett
fb204b7677
Fixed for DFEs with pixelspace and video models
2026-07-27 11:12:02 -06:00
Jaret Burkett
e8573dad34
Fix trailing progress bar print when stopping a job in the ui
2026-07-22 10:11:08 -06:00
Jaret Burkett
479c72ada2
Add replacing triggers on prompts when caching text encoder
2026-07-19 09:03:05 -06:00
Jaret Burkett
988d891102
Added a Sample Next Step in the job gear dropdown to force a sample on the next step.
2026-07-17 07:59:30 -06:00
fatalis and Jaret Burkett
8bbd051667
Add sample_start_step setting to configure when sampling starts ( #949 )
...
Co-authored-by: Jaret Burkett <jaretburkett@gmail.com >
2026-07-15 11:15:50 -06:00
Jaret Burkett
4e50535478
Allow models to provide an additional loss
2026-06-28 09:49:36 -06:00
Jaret Burkett
1e12b6b73f
Remove duplicate save when triggering save next step
2026-06-28 07:18:16 -06:00
Jaret Burkett
d14f6e567a
Allow individual models to scale the loss after it is calculated.
2026-06-25 10:13:58 -06:00
Jaret Burkett
820d534d6e
Add features for models that may need a non masked loss such as inpainiting.
2026-06-22 10:59:17 -06:00
Jaret Burkett
cac3815b2c
Make DOP run in a single backward pass, should be faster and more stable. Show dop loss on ui
2026-06-08 13:55:47 -06:00
Jaret Burkett
68e9b38220
Add ability to trigger a save from the ui which whill make the trainer save on the next step
2026-05-26 08:38:49 -06:00
Jaret Burkett
12304e170f
Added some experimental loss targets
2026-05-24 14:13:23 -06:00
Jaret Burkett
ec58dcde92
Add better gradient checkpointing to dfes
2026-05-18 09:25:23 -06:00
Jaret Burkett (Ostris)
fc4d6ebf39
Add support for fine-tuning Hidream O1 ( #831 )
...
* Initial support for hidream. Lora keys likely need work
* Fix saving for hidream-o1
* Remove dependence on flash attention for hidream o1
* Fix gradient checkpointing for hidream o1
* A lot of fixes for hidream. Handle loading and saving as comfy model.
* Omit layers not used in comfy. Fix issue with lora loading keys in comfy
* Version bumpo
2026-05-12 11:15:16 -06:00
Jaret Burkett
e9ab387dfd
Fixed issue with qwen image edit models when using multiple control images when not caching text embeddings.
2026-04-30 11:59:12 +00:00
Jaret Burkett
1fc4ad3979
Add sapiens2 as a diffusion feature extractor
2026-04-27 15:59:03 -06:00
Jaret Burkett
998a02f30e
Yet another pseudo_huber fix. Still drinking coffee.
2026-04-19 10:05:40 -06:00
Jaret Burkett
fc85410c9a
Fix issue with precision on pseudo_huber loss
2026-04-19 10:02:27 -06:00
Jaret Burkett
20a99258b8
Fix issue is is else on pseudo_huber loss
2026-04-19 09:59:18 -06:00
Jaret Burkett
f4445cd78c
Added psuedo_huber loss
2026-04-19 09:51:46 -06:00
Jaret Burkett
beb40ae29b
Add DFE8 with partial step
2026-04-17 17:40:16 -06:00
Jaret Burkett
233e292256
Added some experimental low step things for zeta
2026-04-13 09:37:34 -06:00
Jaret Burkett
bf56217c37
Fixed issue where job would fail if DB is locked.
2026-03-31 09:10:33 -06:00
Jaret Burkett
fatalisandJaret Burkett