evan/ai-toolkit
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,092 Commits 2 Branches 0 Tags
4260a3c5b6c437ab863aa5f143703d77fc706c88
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Jaret Burkett 4260a3c5b6 Add optimizer test suite and make minor speed adjustments to Automagicv2 2026-05-05 10:02:30 -06:00
Jaret Burkett aeca7fe404 Add Automagic v2 optimizer. It uses significantly less vram and is much more efficient. 2026-05-05 09:09:07 -06:00
Jaret Burkett 0d91fcee9e Allow edit of captioning job 2026-04-30 16:00:16 -06:00
Jaret Burkett eadc9a58af Version bump 2026-04-30 06:00:58 -06:00
Jaret Burkett e9ab387dfd Fixed issue with qwen image edit models when using multiple control images when not caching text embeddings. 2026-04-30 11:59:12 +00:00
Jaret Burkett deb409085a Made it possible to use the flux 2 small decoder VAE when setting the vae_path manually for flux2 models 2026-04-30 05:42:31 -06:00
Jaret Burkett 7ccec8ec2c Add checkpointing and a proper decode for flux 2 VAEs so they can be used with DFE 2026-04-30 04:27:13 -06:00
Jaret Burkett b4f0efb025 Version Bump 2026-04-28 13:42:00 -06:00
Jaret Burkett af6458d1b5 Enable caching of ACE step latents. 2026-04-28 13:39:20 -06:00
Jaret Burkett 77b8765939 Fix issue with hover styling on select components. 2026-04-28 10:55:20 -06:00
Jaret Burkett 43989cc19e Add advanced config section to the captioner 2026-04-28 10:49:46 -06:00
Jaret Burkett f972b750e6 Performance improvements to captioner. Add ability to set a default caption for ace captioner to avoid needing to caption the audio so we only transcribe 2026-04-28 10:15:11 -06:00
Jaret Burkett acc6a36214 Scale DFE 9 to a velocity equiv weight to match flow matching gradient strength. Probably need to rework all DFEs to do this as the math checks out. 2026-04-28 09:10:02 -06:00
Jaret Burkett 1fc4ad3979 Add sapiens2 as a diffusion feature extractor 2026-04-27 15:59:03 -06:00
Jaret Burkett 67d67f8c1d Ignore hidden files when captioning 2026-04-25 17:38:31 -06:00
Jaret Burkett 998a02f30e Yet another pseudo_huber fix. Still drinking coffee. 2026-04-19 10:05:40 -06:00
Jaret Burkett fc85410c9a Fix issue with precision on pseudo_huber loss 2026-04-19 10:02:27 -06:00
Jaret Burkett 20a99258b8 Fix issue is is else on pseudo_huber loss 2026-04-19 09:59:18 -06:00
Jaret Burkett f4445cd78c Added psuedo_huber loss 2026-04-19 09:51:46 -06:00
Jaret Burkett 488878f354 Use hidden layers in the loss for DFE 7 and 8 2026-04-18 13:07:38 -06:00
Jaret Burkett beb40ae29b Add DFE8 with partial step 2026-04-17 17:40:16 -06:00
Jaret Burkett 7c4f18ce51 Fix ernie unpatchify 2026-04-17 12:03:39 -06:00
Jaret Burkett 8cb9649382 Add decode latent to ernie pipe 2026-04-17 12:01:09 -06:00
Jaret Burkett 67048df9f9 Merge branch 'main' of github.com:ostris/ai-toolkit 2026-04-17 06:05:40 -06:00
Jaret Burkett be54094704 Version bump 2026-04-17 06:05:26 -06:00
Jaret Burkett a513a1583e Fixed issue where Qwen VL MOE captioner produced nonsense 2026-04-16 21:24:43 +00:00
Jaret Burkett 22ea3dd620 Fixed issue on some systems where Logger didnt have atty 2026-04-16 21:09:52 +00:00
Jaret Burkett ab1ee4df34 Hotfix some issues with Wan models caused by diffusers and transformers updates 2026-04-16 20:53:50 +00:00
Jaret Burkett 0c18b39346 Version bump 2026-04-16 13:09:45 -06:00
Jaret Burkett afb62b1fa5 Add support for Nucleus-Image 2026-04-16 13:09:10 -06:00
Jaret Burkett 2faba22b46 Fix issue when saving advanced prompt embeds. No such file or directory error 2026-04-16 12:22:56 -06:00
Jaret Burkett 0792352dab version bump 2026-04-16 09:25:14 -06:00
Jaret Burkett 8f67f5022e Version lock peft version 2026-04-15 14:17:26 -06:00
Jaret Burkett acc3e60140 Add ernie to readme 2026-04-15 06:43:36 -06:00
Jaret Burkett dd7074a21f Fix issue with layer offloading on ernie 2026-04-14 19:19:02 -06:00
Jaret Burkett e74bc9ac7b Fix issue with concatinating advanced prompt embeds. 2026-04-14 16:04:34 -06:00
Jaret Burkett 7eb1226a6d Fix issue with loading advanced prompt configs metadata 2026-04-14 15:42:09 -06:00
Jaret Burkett 97d8c05d75 Fix issue with captioner sometimes outputting not utf-8 characters 2026-04-14 14:37:21 -06:00
Jaret Burkett (Ostris) 3e0c904054 Add support for Baidu's ERNIE-Image (#793) 
* Add support for ERNIE Image

* change float64 to float32

* Version bump

* Update ERNIE defaults
 2026-04-14 09:45:12 -06:00
Jaret Burkett 233e292256 Added some experimental low step things for zeta 2026-04-13 09:37:34 -06:00
Jaret Burkett 1058ef3513 Made AdvancedPromptEmbeds that is compatable with previous PromptEmbeds functionality, but is more streamlined and can accomidate more model embedding paradigms. 2026-04-11 10:45:47 -06:00
Jaret Burkett 0d11be41fa Adjust default sample steps. 2026-04-10 17:46:34 -06:00
Jaret Burkett 62e18427b4 Adjust default sample steps. 2026-04-10 17:46:09 -06:00
Jaret Burkett 9b4e2d1b0b More flac support 2026-04-10 12:27:09 -06:00
Jaret Burkett 0b9c365acb Add flac and ogg support 2026-04-10 12:10:54 -06:00
Jaret Burkett bfb373c8fa Prep for future breaking changes in newer versions of transformers library 2026-04-10 12:04:32 -06:00
Jaret Burkett 145144eee3 Fix issue with auto updating captions when captioning a dataset 2026-04-10 11:51:18 -06:00
Jaret Burkett 765a9d5b2e Add a download button to music samples in in the gear menu 2026-04-10 10:07:45 -06:00
Jaret Burkett d08ea8318f Change default timestep type for ace step to linear 2026-04-09 17:39:00 -06:00
Jaret Burkett 78cf049c29 Add support for ACE-Step 1.5 and ACE-Step 1.5 XL. Also added dataset captioning through the UI. (#785) 
* Base ace step 1.5 xl added. Generating, still wip on training and ui

* Base training code done

* Fix some issues with caching text embeddings. Update sample cards to show audio

* Fix issue with quantizing ace step

* Add album artwork to samples with waveform.

* Cleanup logs

* Add album art endpoint to speed up album art loading

* Made an make video with artwork script

* Make ui handle basic audio models. Make multi line adjustments to the editor and better syntax hilighting.

* Add prompt tagging system for special tagged models.

* prompt tagging processing for ui working.

* Moved default samples to a special file so we can add more when needed and they can be adjusted for a specific model

* Add a captioner job with music captioner that is prepped for use with the ui

* Add basit ui setup for captioning modal and handeling captioning jobs

* Starting captioning job from ui working. Still better management for it.

* Better filtering of job options in the job view for captioning jobs

* Added qwen3 vl as a captioner for images

* Have an indicator when a dataset is being captioned.

* Adjust the way caption jobs look in the queue

* Fix a few issues. Adjust defaults.

* Version bump

* Added ace step to the readme.
 2026-04-09 15:02:03 -06:00
Jaret Burkett 9ca58e9aa2 Fixed offload and quantize order of ltx 2.3 text encoder. 2026-04-07 15:11:50 -06:00
Jaret Burkett 0dcbabf6af Fix merge nertwork ref 2026-04-01 10:38:31 -06:00
Jaret Burkett da2a79590f Add a merge network on save strength 2026-04-01 09:21:08 -06:00
Jaret Burkett 853ffaf207 Add light mode support. 2026-03-31 16:54:55 -06:00
Jaret Burkett ad474e3d06 Update and reformat the readme 2026-03-31 12:31:05 -06:00
Jaret Burkett 4a3251640a More work on compiling models 2026-03-31 12:11:56 -06:00
Jaret Burkett 358d684f6f Move compiiling the model after accelerate manipulation 2026-03-31 09:52:27 -06:00
Jaret Burkett 0045260af7 Fix issue where compile true did not actually compile the model 2026-03-31 09:27:54 -06:00
Jaret Burkett e22039e4aa Add more optimizers to the ui 2026-03-31 09:20:20 -06:00
Jaret Burkett bf56217c37 Fixed issue where job would fail if DB is locked. 2026-03-31 09:10:33 -06:00
Jaret Burkett dcb7f465ec Version Bump 2026-03-30 15:57:40 -06:00
Jaret Burkett 626d9674ea Add info about automated coding agent pull requests. So sick of them. 2026-03-30 11:02:12 -06:00
Jaret Burkett b43ea6c2d3 Abort caption requests when they are not in view to tax the server less. 2026-03-30 10:46:25 -06:00
Jaret Burkett a484e55d66 Rework dataset model and file dragging. Use single model for dragging, uploading, and selecting images. 2026-03-30 10:33:10 -06:00
Jaret Burkett ac82ebd852 Dont try to list hidden files in datasets 2026-03-30 10:03:10 -06:00
Jaret Burkett 171535833a Add Mac OS support for Apple Silicon (#770) 
* Made an install script and auto updates env for mac

* GPU sensors and initial training working for MAC. Still WIP.

* Switch dataloader to single threaded until I can work around some mac pickeling issues.

* Get quantization working on mac

* Fix mac exclusive imports so they don't break other builds.

* Add mac instructions to the UI
 2026-03-30 09:37:47 -06:00
Jaret Burkett bc47fd6755 Make a requirements base file to make it easier to maintain requirements across platforms. 2026-03-29 14:04:46 -06:00
Jaret Burkett fbda10d088 Add a duplicate dataset function to the ui 2026-03-29 13:51:18 -06:00
Jaret Burkett 86dcf39eee Allow user to set a training seed via env vars for repeat result testing 2026-03-29 13:34:46 -06:00
Jaret Burkett 45e99664b9 Add icons to the top bar on the job page 2026-03-29 12:38:47 -06:00
Jaret Burkett 540659709d Improved the load time of dataset and sample images and videos by switching to streaming 2026-03-29 10:38:34 -06:00
Jaret Burkett e030f4f2e0 Show the control images in the image viewer when clicked so they can be easily previewed for reference. 2026-03-29 10:00:54 -06:00
Jaret Burkett affa411edc Fixed an issue where Flux.2 model VAE can be left offloaded to CPU when encoding control images while caching latents 2026-03-29 09:49:10 -06:00
Jaret Burkett 6a1fc54779 Add t0 loss target 2026-03-28 13:35:21 -06:00
Jaret Burkett 8302b21f8f Version Bump 2026-03-28 13:23:52 -06:00
Jaret Burkett 5ce87fa48b Version bump 2026-03-27 20:26:31 -06:00
Jaret Burkett 740657e25e Improve dataset uploader. Upload the files one at a time instead of one huge chunk. Show progress for each file. 2026-03-27 09:26:22 -06:00
Jaret Burkett f85bf065bf Use pooler embeddings for DFE v6 with dino v3 2026-03-27 07:02:07 -06:00
Jaret Burkett a802014ec5 Update the torch versions in the README 2026-03-26 12:15:32 -06:00
Jaret Burkett 2782df02c3 Allow HF_HUB_ENABLE_HF_TRANSFER to be set via env variable 2026-03-26 10:45:49 -06:00
Jaret Burkett 2c8d2acdcb On jobs table, sort idle jobs by last updated so recent active ones are at the top 2026-03-26 10:33:17 -06:00
Jaret Burkett 9a77389653 On a new training job, or when editing one, load everything before allowing editing 2026-03-26 10:23:42 -06:00
Jaret Burkett a7bb4ddb2c Work on loss graph. Add smoothed overlay. Allow user to hilite a secton of the graph to zoom into. 2026-03-26 10:05:09 -06:00
Jaret Burkett 401f7df425 Merge branch 'main' of github.com:ostris/ai-toolkit 2026-03-26 09:11:50 -06:00
Jaret Burkett 4df3b0463f Save job pid to the database and sing sigint to kill it when stopping so it stops immediatly. 2026-03-26 09:10:37 -06:00
Jaret Burkett 89d2090962 Fixed race condition that would occasionally set the dataset path to the first one when editing a job 2026-03-25 11:22:42 -06:00
Jaret Burkett 3f7a3d8d87 Shorten stal action to 3 months 2026-03-25 10:42:18 -06:00
Jaret Burkett 45647c15d3 Added github actions to close stale issues automatically. Hopefully it doesnt break things 2026-03-25 10:25:56 -06:00
Jaret Burkett 899ee528f9 Update git ignore 2026-03-25 10:16:36 -06:00
Jaret Burkett 5d5a8ef9da Fixed issue with deleting datasets and jobs with newer version of node.js. Bumped minimum version of node js to 20 2026-03-25 10:04:36 -06:00
Jaret Burkett dfde30f231 Fix issue with ltx2 custom te repo path 2026-03-25 09:50:18 -06:00
Jaret Burkett b8000dbcbc Bump version 2026-03-25 08:18:42 -06:00
Jaret Burkett 7f3309b291 Add support for audo frame count so datasets can have varrying length videos. Varous ltx 2.3 VAE optimizations such as removing tiling articacts, and doing frame split encoding to reduce vram on encoding/decoding. 2026-03-24 12:20:09 -06:00
Jaret Burkett 295094b4b5 Fixed new breaking change in diffusers with with qwen image 2026-03-23 14:10:55 -06:00
Jaret Burkett 5642b656b9 Fix audio issues with ltx2 models. Silent codec fails now raised. Auto convert surround sound audio to stereo. Invalidate old caches just to be safe so they recache now. 2026-03-23 20:08:33 +00:00
Jaret Burkett 561e6f201c Fixed an issue with ltx 2.3 i2v training 2026-03-23 12:41:18 -06:00
Jaret Burkett 330059d8a1 version bump 2026-03-23 11:01:16 -06:00
Jaret Burkett e91827f9be Change gemma repo to lightricks one that is not gated 2026-03-23 11:00:32 -06:00
Jaret Burkett 253cb31362 Fix issue with video and images with no audio on ltx models 2026-03-22 22:09:23 -06:00
Jaret Burkett 4a3d317e2b Fix issue with using the default text encoder with ltx 2.3 2026-03-22 18:53:59 -06:00