 Jaret Burkett
|
8ef07a9c36
|
Added training for an experimental decoratgor embedding. Allow for turning off guidance embedding on flux (for unreleased model). Various bug fixes and modifications
|
2024-12-15 08:59:27 -07:00 |
|
|
Jaret Burkett
|
92ce93140e
|
Adjustments to defaults for automagic
|
2024-11-29 10:28:06 -07:00 |
|
|
Jaret Burkett
|
f213996aa5
|
Fixed saving and displaying for automagic
|
2024-11-29 08:00:22 -07:00 |
|
|
Jaret Burkett
|
cbe31eaf0a
|
Initial work on a auto adjusting optimizer
|
2024-11-29 04:48:58 -07:00 |
|
|
Jaret Burkett
|
67c2e44edb
|
Added support for training flux redux adapters
|
2024-11-21 20:01:52 -07:00 |
|
|
Jaret Burkett
|
96d418bb95
|
Added support for full finetuning flux with randomized param activation. Examples coming soon
|
2024-11-21 13:05:32 -07:00 |
|
|
Jaret Burkett
|
894374b2e9
|
Various bug fixes and optimizations for quantized training. Added untested custom adam8bit optimizer. Did some work on LoRM (dont use)
|
2024-11-20 09:16:55 -07:00 |
|
|
Jaret Burkett
|
6509ba4484
|
Fix seed generation to make it deterministic so it is consistant from gpu to gpu
|
2024-11-15 12:11:13 -07:00 |
|
|
Jaret Burkett
|
025ee3dd3d
|
Added ability for adafactor to fully fine tune quantized model.
|
2024-10-30 16:38:07 -06:00 |
|
|
Jaret Burkett
|
58f9d01c2b
|
Added adafactor implementation that handles stochastic rounding of update and accumulation
|
2024-10-30 05:25:57 -06:00 |
|
|
Jaret Burkett
|
e72b59a8e9
|
Added experimental 8bit version of prodigy with stochastic rounding and stochastic gradient accumulation. Still testing.
|
2024-10-29 14:28:28 -06:00 |
|
|
Jaret Burkett
|
4aa19b5c1d
|
Only quantize flux T5 is also quantizing model. Load TE from original name and path if fine tuning.
|
2024-10-29 14:25:31 -06:00 |
|
|
Jaret Burkett
|
4747716867
|
Fixed issue with adapters not providing gradients with new grad activator
|
2024-10-29 14:22:10 -06:00 |
|
|
Jaret Burkett
|
22cd40d7b9
|
Improvements for full tuning flux. Added debugging launch config for vscode
|
2024-10-29 04:54:08 -06:00 |
|
|
Jaret Burkett
|
3400882a80
|
Added preliminary support for SD3.5-large lora training
|
2024-10-22 12:21:36 -06:00 |
|
|
Jaret Burkett
|
9f94c7b61e
|
Added experimental param multiplier to the ema module
|
2024-10-22 09:25:52 -06:00 |
|
|
Jaret Burkett
|
bedb8197a2
|
Fixed issue with sizes for some images being loaded sideways resulting in squished images.
|
2024-10-20 11:51:29 -06:00 |
|
|
Jaret Burkett
|
e3ebd73610
|
Add a projection layer on vision direct when doing image embeds
|
2024-10-20 10:48:23 -06:00 |
|
|
Jaret Burkett
|
dd931757cd
|
Merge branch 'main' of github.com:ostris/ai-toolkit
|
2024-10-20 07:04:29 -06:00 |
|
|
Jaret Burkett
|
0640cdf569
|
Handle errors in loading size database
|
2024-10-20 07:04:19 -06:00 |
|
|
Jaret Burkett
|
0b048d0dde
|
Locked version of quanto as it breaks in later versions
|
2024-10-16 22:41:04 +00:00 |
|
|
Jaret Burkett
|
473d455f44
|
Process empty clip image if there is not one for reg images when training a custom adapter
|
2024-10-15 08:28:04 -06:00 |
|
|
Jaret Burkett
|
ce759ebd8c
|
Normalize the image embeddings on vd adapter forward
|
2024-10-12 15:09:48 +00:00 |
|
|
Jaret Burkett
|
628a7923a3
|
Remove norm on image embeds on custom adapter
|
2024-10-12 00:43:18 +00:00 |
|
|
Jaret Burkett
|
3922981996
|
Added some additional experimental things to the vision direct encoder
|
2024-10-10 19:42:26 +00:00 |
|
|
Jaret Burkett
|
ab22674980
|
Allow for a default caption file in the folder. Minor bug fixes.
|
2024-10-10 07:31:33 -06:00 |
|
|
Jaret Burkett
|
9452929300
|
Apply a mask to the embeds for SD if using T5 encoder
|
2024-10-04 10:55:20 -06:00 |
|
|
Jaret Burkett
|
a800c9d19e
|
Add a method to have an inference only lora
|
2024-10-04 10:06:53 -06:00 |
|
|
Jaret Burkett
|
28e6f00790
|
Fixed bug in returning clip image embed to actually return it
|
2024-10-03 10:49:09 -06:00 |
|
|
Jaret Burkett
|
67e0aca750
|
Added ability to load clip pairs randomly from folder. Other small bug fixes
|
2024-10-03 10:03:49 -06:00 |
|
|
Jaret Burkett
|
f05224970f
|
Added Vision Languate Adapter usage for pixtral vd adapter
|
2024-09-29 19:39:56 -06:00 |
|
|
Jaret Burkett
|
b4f64de4c2
|
Quick patch to scope xformer imports until a better solution
|
2024-09-28 15:36:42 -06:00 |
|
|
Jaret Burkett
|
2e5f6668dc
|
Add xformers ad a dependency
|
2024-09-28 15:30:14 -06:00 |
|
|
Jaret Burkett
|
e4c82803e1
|
Handle random resizing for pixtral input on direct vision adapter
|
2024-09-28 14:53:38 -06:00 |
|
|
Jaret Burkett
|
69aa92bce5
|
Added support for AdEMAMix8bit
|
2024-09-28 14:33:51 -06:00 |
|
|
Jaret Burkett
|
a508caad1d
|
Change pixtral to crop based on number of pixels instead of largest dimension
|
2024-09-28 13:05:26 -06:00 |
|
|
Jaret Burkett
|
58537fc92b
|
Added initial direct vision pixtral support
|
2024-09-28 10:47:51 -06:00 |
|
|
Jaret Burkett
|
86b5938cf3
|
Fixed the webp bug finally.
|
2024-09-25 13:56:00 -06:00 |
|
|
Jaret Burkett
|
6b4034122f
|
REmove layers from direct vision resampler
|
2024-09-24 15:08:29 -06:00 |
|
|
Jaret Burkett
|
10817696fb
|
Fixed issue where direct vision was not passing additional params from resampler when it is added
|
2024-09-24 10:34:11 -06:00 |
|
|
Jaret Burkett
|
037ce11740
|
Always return vision encoder in state dict
|
2024-09-24 07:43:17 -06:00 |
|
|
Jaret Burkett
|
04424fe2d6
|
Added config setting to set the timestep type
|
2024-09-24 06:53:59 -06:00 |
|
|
Jaret Burkett
|
40a8ff5731
|
Load local hugging face packages for assistant adapter
|
2024-09-23 10:37:12 -06:00 |
|
|
Jaret Burkett
|
2776221497
|
Added option to cache empty prompt or trigger and unload text encoders while training
|
2024-09-21 20:54:09 -06:00 |
|
|
Jaret Burkett
|
f85ad452c6
|
Added initial support for pixtral vision as a vision encoder.
|
2024-09-21 15:21:14 -06:00 |
|
|
Jaret Burkett
|
dd889086f4
|
Updates to the docker file for jupyterlab
|
2024-09-21 12:07:07 -06:00 |
|
|
Jaret Burkett
|
d97c55cd96
|
Updated requirements to lock version of albucore, which had breaking changes.
|
2024-09-21 11:19:13 -06:00 |
|
|
Jaret Burkett
|
951e223481
|
Added support to disable single transformers in vision direct adapter
|
2024-09-11 08:54:51 -06:00 |
|
|
Jaret Burkett
|
fc34a69bec
|
Ignore guidance embed when full tuning flux. adjust block scaler to decat to 1.0. Add MLP resampler for reducing vision adapter tokens
|
2024-09-09 16:24:46 -06:00 |
|
|
Jaret Burkett
|
279ee65177
|
Remove block scaler
|
2024-09-06 08:28:17 -06:00 |
|
|
Jaret Burkett
|
3a1f464132
|
Added support for training vision direct weight adapters
|
2024-09-05 10:11:44 -06:00 |
|
|
Jaret Burkett
|
5c8fcc8a4e
|
Fix bug with zeroing out gradients when accumulating
|
2024-09-03 08:29:15 -06:00 |
|
|
Jaret Burkett
|
121a760c19
|
Added proper grad accumulation
|
2024-09-03 07:24:18 -06:00 |
|
|
Jaret Burkett
|
e5fadddd45
|
Added ability to do prompt attn masking for flux
|
2024-09-02 17:29:36 -06:00 |
|
|
Jaret Burkett
|
d44d4eb61a
|
Added a new experimental linear weighing technique
|
2024-09-02 09:22:13 -06:00 |
|
|
Jaret Burkett
|
7d9ab22405
|
Rework ip adapter and vision direct adapters to apply to the single transformer blocks even though they are not cross attn.
|
2024-09-01 10:40:42 -06:00 |
|
|
Jaret Burkett
|
7ed8c51f20
|
Readme cleanup
|
2024-09-01 07:06:09 -06:00 |
|
|
Jaret Burkett
|
6df33156f0
|
Add information about specific weight targeting in the README
|
2024-09-01 06:59:47 -06:00 |
|
|
Jaret Burkett
|
40f5c59da0
|
Fixes for training ilora on flux
|
2024-08-31 16:55:26 -06:00 |
|
|
Jaret Burkett
|
3e71a99df0
|
Check for contains only against clean name for lora, not the adjusted one
|
2024-08-31 07:44:13 -06:00 |
|
|
Jaret Burkett
|
60232def91
|
Made peleminary arch for flux ip adapter training
|
2024-08-28 08:55:39 -06:00 |
|
|
Jaret Burkett
|
3843e0d148
|
Added support for vision direct adapter for flux
|
2024-08-26 16:27:28 -06:00 |
|
|
Jaret Burkett
|
b322d05fa3
|
Added tutorial link to readme
|
2024-08-22 16:25:32 -06:00 |
|
|
Jaret Burkett
|
8577849eeb
|
Fixed wrong discord link. Woops.
|
2024-08-22 14:49:03 -06:00 |
|
|
Jaret Burkett
|
338c77d677
|
Fixed breaking change with diffusers. Allow flowmatch on normal stable diffusion models.
|
2024-08-22 14:36:22 -06:00 |
|
|
Jaret Burkett
|
e07a98a50c
|
Bugfixes for full finetuning at bf16
|
2024-08-22 05:15:33 -06:00 |
|
|
Jaret Burkett
|
6a754b2710
|
Merge branch 'main' of github.com:ostris/ai-toolkit
|
2024-08-22 04:36:50 -06:00 |
|
|
Jaret Burkett
|
a939cf3730
|
WIP - adding support for flux DoRA and ip adapter training
|
2024-08-22 04:36:39 -06:00 |
|
|
Jaret Burkett
|
169dbd22ba
|
Finaized bug reports
|
2024-08-18 16:21:48 -06:00 |
|
|
Jaret Burkett
|
6e7d721382
|
More issues testing
|
2024-08-18 16:20:08 -06:00 |
|
|
Jaret Burkett
|
dc6f36cd82
|
Testing github bug reporting stuff
|
2024-08-18 16:09:52 -06:00 |
|
|
Jaret Burkett
|
c45887192a
|
Unload interum weights when doing multi lora fuse
|
2024-08-18 09:35:10 -06:00 |
|
|
Jaret Burkett
|
13a965a26c
|
Fixed bad key naming on lora fuse I just pushed
|
2024-08-18 09:33:31 -06:00 |
|
|
Jaret Burkett
|
77ee7090e8
|
Update FAQ.md
|
2024-08-18 09:26:22 -06:00 |
|
|
Jaret Burkett
|
078396ceac
|
Added a basic FAQ
|
2024-08-18 09:21:51 -06:00 |
|
|
Jaret Burkett
|
f944eeaa4d
|
Fuse flux schnell assistant adapter in pieces when doing lowvram to drastically speed ip up from minutes to seconds.
|
2024-08-18 09:09:11 -06:00 |
|
|
Jaret Burkett
|
81899310f8
|
Added support for training on flux schnell. Added example config and instructions for training on flux schnell
|
2024-08-17 06:58:39 -06:00 |
|
|
Jaret Burkett
|
f9179540d2
|
Flush after sampling
|
2024-08-16 17:29:42 -06:00 |
|
|
Jaret Burkett
|
452e0e286d
|
For lora assisted training, merge in before quantizing then sample with schnell at -1 weight. Almost doubles training speed with lora adapter.
|
2024-08-16 17:28:44 -06:00 |
|
|
Jaret Burkett
|
165510ace2
|
Dumb typo
|
2024-08-15 12:59:32 -06:00 |
|
|
Jaret Burkett
|
0355662e8e
|
Added support for polarity guidance for flow matching models
|
2024-08-15 12:22:00 -06:00 |
|
|
Jaret Burkett
|
b99d36dfdb
|
fixed issue with batch sizes larget than 1
|
2024-08-15 12:21:38 -06:00 |
|
|
Jaret Burkett
|
9001e5c933
|
Change flux latent spact if so it will not use old cache
|
2024-08-14 11:27:40 -06:00 |
|
|
Jaret Burkett
|
7fed4ea761
|
fixed huge flux training bug. Added ability to use an assistatn lora
|
2024-08-14 10:14:13 -06:00 |
|
|
Jaret Burkett
|
e07bf11727
|
Merge pull request #61 from fofr/patch-1
Fix image name in captions section of README
|
2024-08-14 08:01:51 -06:00 |
|
|
Jaret Burkett
|
00bd3d54a3
|
Actually use the save dtype from the config file.
|
2024-08-13 17:08:27 -06:00 |
|
|
Jaret Burkett
|
f7cf2f866f
|
Make 100% sure lora alpha matches for flux
|
2024-08-13 14:24:03 -06:00 |
|
|
Jaret Burkett
|
465bc1e2f8
|
Update readme again
|
2024-08-13 13:37:22 -06:00 |
|
|
Jaret Burkett
|
0beca0d4a7
|
Updated readme
|
2024-08-13 13:35:20 -06:00 |
|
|
Jaret Burkett
|
418f5f7e8c
|
Added new experimental time step weighing that should solve a lot of issues with distribution. Updated example. Removed a warning
|
2024-08-13 12:02:11 -06:00 |
|
|
Jaret Burkett
|
9ee1ef2a0a
|
Added experimental modified sigma sqrt weight mapping for linear timestep scheduling for flowmatching
|
2024-08-12 17:03:09 -06:00 |
|
|
Jaret Burkett
|
599fafe01f
|
Allow user to have the full flux checkpoint local
|
2024-08-12 09:57:16 -06:00 |
|
|
Jaret Burkett
|
af108bb964
|
Bug fix with dataloader. Added a flag to completly disable sampling
|
2024-08-12 09:19:40 -06:00 |
|
|
Jaret Burkett
|
89d61a3b8e
|
Readme updates
|
2024-08-11 13:23:57 -06:00 |
|
|
Jaret Burkett
|
a6aa4b2c7d
|
Added ability to set timesteps to linear for flowmatching schedule
|
2024-08-11 13:06:08 -06:00 |
|
|
Jaret Burkett
|
f8f0657b68
|
Added a colab notebook for training flux loras
|
2024-08-11 12:27:40 -06:00 |
|
|
Jaret Burkett
|
7f0ecdb377
|
Merge branch 'main' of github.com:ostris/ai-toolkit
|
2024-08-11 11:10:45 -06:00 |
|
|
Jaret Burkett
|
fbed8568fb
|
Actually use the correct timestep sampling instead of calculating it and moving on lol. Tested a few with it and it seems to work better.
|
2024-08-11 11:10:37 -06:00 |
|
|
Jaret Burkett
|
6d31c6db73
|
Added a fix for windows dataloader
|
2024-08-11 10:48:24 -06:00 |
|
|
Jaret Burkett
|
6490a326e5
|
Fixed issue for vaes without a shift
|
2024-08-11 10:30:55 -06:00 |
|