 Jaret Burkett
|
0beca0d4a7
|
Updated readme
|
2024-08-13 13:35:20 -06:00 |
|
|
Jaret Burkett
|
418f5f7e8c
|
Added new experimental time step weighing that should solve a lot of issues with distribution. Updated example. Removed a warning
|
2024-08-13 12:02:11 -06:00 |
|
|
Jaret Burkett
|
9ee1ef2a0a
|
Added experimental modified sigma sqrt weight mapping for linear timestep scheduling for flowmatching
|
2024-08-12 17:03:09 -06:00 |
|
|
Jaret Burkett
|
599fafe01f
|
Allow user to have the full flux checkpoint local
|
2024-08-12 09:57:16 -06:00 |
|
|
Jaret Burkett
|
af108bb964
|
Bug fix with dataloader. Added a flag to completly disable sampling
|
2024-08-12 09:19:40 -06:00 |
|
|
Jaret Burkett
|
89d61a3b8e
|
Readme updates
|
2024-08-11 13:23:57 -06:00 |
|
|
Jaret Burkett
|
a6aa4b2c7d
|
Added ability to set timesteps to linear for flowmatching schedule
|
2024-08-11 13:06:08 -06:00 |
|
|
Jaret Burkett
|
f8f0657b68
|
Added a colab notebook for training flux loras
|
2024-08-11 12:27:40 -06:00 |
|
|
Jaret Burkett
|
7f0ecdb377
|
Merge branch 'main' of github.com:ostris/ai-toolkit
|
2024-08-11 11:10:45 -06:00 |
|
|
Jaret Burkett
|
fbed8568fb
|
Actually use the correct timestep sampling instead of calculating it and moving on lol. Tested a few with it and it seems to work better.
|
2024-08-11 11:10:37 -06:00 |
|
|
Jaret Burkett
|
6d31c6db73
|
Added a fix for windows dataloader
|
2024-08-11 10:48:24 -06:00 |
|
|
Jaret Burkett
|
6490a326e5
|
Fixed issue for vaes without a shift
|
2024-08-11 10:30:55 -06:00 |
|
|
Jaret Burkett
|
8d48ad4e85
|
fixed bug I added to demo config
|
2024-08-11 10:28:39 -06:00 |
|
|
Jaret Burkett
|
ec1ea7aa0e
|
Added support for training on primary gpu with low_vram flag. Updated example script to remove creepy horse sample at that seed
|
2024-08-11 09:54:30 -06:00 |
|
|
Jaret Burkett
|
fa02e774b0
|
Added info about datset
|
2024-08-10 15:08:05 -06:00 |
|
|
Jaret Burkett
|
2308ef2868
|
Added flux training instructions
|
2024-08-10 14:10:02 -06:00 |
|
|
Jaret Burkett
|
b3e03295ad
|
Reworked flux pred. Again
|
2024-08-08 13:06:34 -06:00 |
|
|
Jaret Burkett
|
e69a520616
|
Reworked timestep distribution on flowmatch sampler when training.
|
2024-08-08 06:01:45 -06:00 |
|
|
Jaret Burkett
|
acafe9984f
|
Adjustments to loading of flux. Added a feedback to ema
|
2024-08-07 13:17:26 -06:00 |
|
|
Jaret Burkett
|
653fe60f16
|
Updates to flow matching algo
|
2024-08-07 15:04:17 +00:00 |
|
|
Jaret Burkett
|
c2424087d6
|
8 bit training working on flux
|
2024-08-06 11:53:27 -06:00 |
|
|
Jaret Burkett
|
272c8608c2
|
Make a CFG version of flux pipeline
|
2024-08-05 16:35:53 -06:00 |
|
|
Jaret Burkett
|
99f24cfb0c
|
Added a conversion script to convert my loras to peft format for flux
|
2024-08-05 14:54:10 -06:00 |
|
|
Jaret Burkett
|
187663ab55
|
Use peft format for flux loras so they are compatible with diffusers. allow loading an assistant lora
|
2024-08-05 14:34:37 -06:00 |
|
|
Jaret Burkett
|
edb7e827ee
|
Adjusted flow matching so target noise multiplier works properly with it.
|
2024-08-05 11:40:05 -06:00 |
|
|
Jaret Burkett
|
0ea27011d5
|
Bug fix
|
2024-08-04 11:07:19 -06:00 |
|
|
Jaret Burkett
|
f321de7bdb
|
Setup to retrain guidance embedding for flux. Use defualt timestep distribution for flux
|
2024-08-04 10:37:23 -06:00 |
|
|
Jaret Burkett
|
88acc28d7f
|
Prep for runpod docker
|
2024-08-03 12:41:06 -06:00 |
|
|
Jaret Burkett
|
de2da96a81
|
Updat4ed requirements
|
2024-08-03 09:50:18 -06:00 |
|
|
Jaret Burkett
|
9beea1c268
|
Flux training should work now... maybe
|
2024-08-03 09:17:34 -06:00 |
|
|
Jaret Burkett
|
369aa143bc
|
Only train a few blocks on flux (for now)
|
2024-08-03 07:02:27 -06:00 |
|
|
Jaret Burkett
|
87ba867fdc
|
Added flux training. Still a WIP. Wont train right without rectified flow working right
|
2024-08-02 15:00:30 -06:00 |
|
|
Jaret Burkett
|
03613c523f
|
Bugfixes and cleanup
|
2024-08-01 11:45:12 -06:00 |
|
|
Jaret Burkett
|
47744373f2
|
Change img multiplier math
|
2024-07-30 11:33:41 -06:00 |
|
|
Jaret Burkett
|
443c996e7f
|
Do a noisy unconsitional for vision direct
|
2024-07-29 15:42:26 -06:00 |
|
|
Jaret Burkett
|
8f0f467c20
|
Switch back to old ilora
|
2024-07-29 07:22:05 -06:00 |
|
|
Jaret Burkett
|
e81e19fd0f
|
Added target_norm_std which is a game changer
|
2024-07-28 16:08:33 -06:00 |
|
|
Jaret Burkett
|
0bc4d555c7
|
A lot of pixart sigma training tweaks
|
2024-07-28 11:23:18 -06:00 |
|
|
Jaret Burkett
|
80aa2dbb80
|
New image generation img2img. various tweaks and fixes
|
2024-07-24 04:13:41 -06:00 |
|
|
Jaret Burkett
|
8d799031cf
|
Remove reg as prior pred
|
2024-07-21 02:34:12 -06:00 |
|
|
Jaret Burkett
|
6e92922c14
|
Add a mergable linear to the mid of ilora
|
2024-07-20 21:17:53 -06:00 |
|
|
Jaret Burkett
|
c51235c486
|
Fixed misnamed var
|
2024-07-20 23:00:20 +00:00 |
|
|
Jaret Burkett
|
c2c4e8cf34
|
Added ability to target parts of lora for ilora
|
2024-07-20 22:45:52 +00:00 |
|
|
Jaret Burkett
|
4c249cf607
|
Added ilora2
|
2024-07-20 16:40:57 -06:00 |
|
|
Jaret Burkett
|
c2d5f712a3
|
Reworked ilora arch
|
2024-07-20 15:35:59 -06:00 |
|
|
Jaret Burkett
|
22d2f6e28f
|
Fixed issue with grad scaling
|
2024-07-20 08:21:57 -06:00 |
|
|
Jaret Burkett
|
a2301cf28c
|
Amall bug fixes
|
2024-07-18 10:39:55 -06:00 |
|
|
Jaret Burkett
|
11e426fdf1
|
Various features and fixes. Too much brain fog to do a proper description
|
2024-07-18 07:34:14 -06:00 |
|
|
Jaret Burkett
|
58dffd43a8
|
Added caching to image sizes so we dont do it every time.
|
2024-07-15 19:07:41 -06:00 |
|
|
Jaret Burkett
|
e4558dff4b
|
Partial implementation for training auraflow.
|
2024-07-12 12:11:38 -06:00 |
|
|
Jaret Burkett
|
c062b7716c
|
Varous bug fixes
|
2024-07-10 15:20:04 -06:00 |
|
|
Jaret Burkett
|
c008405480
|
Added after model load hook
|
2024-07-09 15:34:48 -06:00 |
|
|
Jaret Burkett
|
93e5df1d59
|
Merge branch 'main' of github.com:ostris/ai-toolkit
|
2024-07-07 07:56:56 -06:00 |
|
|
Jaret Burkett
|
045e4a6e15
|
Save entire pixart model again
|
2024-07-07 07:56:48 -06:00 |
|
|
Jaret Burkett
|
76f225a467
|
Fixed issue with TE adapter caption projection
|
2024-07-06 19:09:58 +00:00 |
|
|
Jaret Burkett
|
cab8a1c7b8
|
WIP to add the caption_proj weight to pixart sigma TE adapter
|
2024-07-06 13:00:21 -06:00 |
|
|
Jaret Burkett
|
acb06d6ff3
|
Bug fixes
|
2024-07-03 10:56:34 -06:00 |
|
|
Jaret Burkett
|
bb57623a35
|
Merge branch 'main' of github.com:ostris/ai-toolkit
|
2024-06-29 15:53:40 -06:00 |
|
|
Jaret Burkett
|
3072d20f17
|
Add ability to include conv_in and conv_out to full train when doing a lora
|
2024-06-29 14:54:50 -06:00 |
|
|
Jaret Burkett
|
f6b21f47bb
|
Increased the number of heads for ip adapters.
|
2024-06-28 16:09:52 +00:00 |
|
|
Jaret Burkett
|
603ceca3ca
|
added ema
|
2024-06-28 10:03:26 -06:00 |
|
|
Jaret Burkett
|
657fd09f25
|
Added more control over Sigma sizes
|
2024-06-26 08:57:53 -06:00 |
|
|
Jaret Burkett
|
8407c4deea
|
Merge branch 'main' of github.com:ostris/ai-toolkit
|
2024-06-23 14:47:43 -06:00 |
|
|
Jaret Burkett
|
64f2b085b7
|
Minor fixes
|
2024-06-23 14:47:40 -06:00 |
|
|
Jaret Burkett
|
7165f2d25a
|
Work to omprove pixart training
|
2024-06-23 20:46:48 +00:00 |
|
|
Jaret Burkett
|
5d47244c57
|
Added support for pixart sigma loras
|
2024-06-16 11:56:30 -06:00 |
|
|
Jaret Burkett
|
ada722c9e4
|
Fixed issue with heads not being added
|
2024-06-15 08:34:33 -06:00 |
|
|
Jaret Burkett
|
696f73c30d
|
Removed variant
|
2024-06-14 17:09:47 -06:00 |
|
|
Jaret Burkett
|
e3410413b9
|
Rework head on ilora
|
2024-06-14 16:21:26 -06:00 |
|
|
Jaret Burkett
|
37cebd9458
|
WIP Ilora
|
2024-06-14 09:31:01 -06:00 |
|
|
Jaret Burkett
|
bd10d2d668
|
Some work on sd3 training. Not working
|
2024-06-13 12:19:16 -06:00 |
|
|
Jaret Burkett
|
cb5d28cba9
|
Added working ilora trainer
|
2024-06-12 09:33:45 -06:00 |
|
|
Jaret Burkett
|
3f3636b788
|
Bug fixes and little improvements here and there.
|
2024-06-08 06:24:20 -06:00 |
|
|
Jaret Burkett
|
833c833f28
|
WIP on SAFE encoder. Work on fp16 training improvements. Various other tweaks and improvements
|
2024-05-27 10:50:24 -06:00 |
|
|
Jaret Burkett
|
68b7e159bc
|
Bug Fixes
|
2024-05-17 08:41:20 -06:00 |
|
|
Jaret Burkett
|
5a45c709cd
|
Work on ipadapters and custom adapters
|
2024-05-13 06:37:54 -06:00 |
|
|
Jaret Burkett
|
10e1ecf1e8
|
Added single value adapter training
|
2024-04-28 06:04:47 -06:00 |
|
|
Jaret Burkett
|
b96913d73c
|
Improvements to dataloader
|
2024-04-27 09:28:28 -06:00 |
|
|
Jaret Burkett
|
5da3613e0b
|
Bug fixes and minor features
|
2024-04-25 06:14:31 -06:00 |
|
|
Jaret Burkett
|
5a70b7f38d
|
Added pixart sigma support, but it wont work until i address breaking changes with lora code in diffusers so it can be upgraded.
|
2024-04-20 10:46:56 -06:00 |
|
|
Jaret Burkett
|
377b81ee3e
|
Adjustments to guidance
|
2024-04-19 15:00:35 -06:00 |
|
|
Jaret Burkett
|
2d0a1be59d
|
Bug fixes
|
2024-04-16 03:48:13 -06:00 |
|
|
Jaret Burkett
|
7284aab7c0
|
Added specialized scaler training to ip adapters
|
2024-04-05 08:17:09 -06:00 |
|
|
Jaret Burkett
|
427847ac4c
|
Small tweaks and fixes for specialized ip adapter training
|
2024-03-26 11:35:26 -06:00 |
|
|
Jaret Burkett
|
9c1cc9641e
|
Added keep tokens to keep so many tokens in a prompt when dropping
|
2024-03-18 13:18:25 -06:00 |
|
|
Jaret Burkett
|
89f4bcad2e
|
Lock diffusers to 0.26.3 until I can figure out why future versions break LoRA code
|
2024-03-18 10:17:55 -06:00 |
|
|
Jaret Burkett
|
016687bda1
|
Adapter work. Bug fixes. Auto adjust LR when resuming optimizer.
|
2024-03-17 10:21:47 -06:00 |
|
|
Jaret Burkett
|
72de68d8aa
|
WIP on clip vision encoder
|
2024-03-13 07:24:08 -06:00 |
|
|
Jaret Burkett
|
d87b49882c
|
Work on embedding adapters
|
2024-03-11 15:18:42 -06:00 |
|
|
Jaret Burkett
|
f415bac7b5
|
Merge branch 'main' of github.com:ostris/ai-toolkit
|
2024-03-06 09:32:38 -07:00 |
|
|
Jaret Burkett
|
f1cb87fe9e
|
fixed bug the kept learning rates the same
|
2024-03-06 09:23:32 -07:00 |
|
|
Jaret Burkett
|
8f9cd823d1
|
Create LICENSE
|
2024-03-06 07:54:55 -07:00 |
|
|
Jaret Burkett
|
b01e8d889a
|
Added stochastic rounding to adafactor. ILora adjustments
|
2024-03-05 07:07:09 -07:00 |
|
|
Jaret Burkett
|
1325613583
|
rework ilora
|
2024-02-29 07:55:52 -07:00 |
|
|
Jaret Burkett
|
337945de9a
|
Added this not that guidance. Added ability to replace prompts.
|
2024-02-28 20:10:14 -07:00 |
|
|
Jaret Burkett
|
561914d8e6
|
Removed old code for fixing multistep sampler that is no longer needed
|
2024-02-25 11:53:35 -07:00 |
|
|
Jaret Burkett
|
b0a0f28191
|
Bug fixes
|
2024-02-25 08:28:29 -07:00 |
|
|
Jaret Burkett
|
f965a1299f
|
Fixed Dora implementation. Still highly experimental
|
2024-02-24 10:26:01 -07:00 |
|
|
Jaret Burkett
|
1bd94f0f01
|
Added early DoRA support, but will change shortly. Dont use right now.
|
2024-02-23 05:55:41 -07:00 |
|
|
Jaret Burkett
|
9ffa8c3711
|
Fixed issue when there is no adapter
|
2024-02-22 02:59:59 -07:00 |
|