Comments (1)
Underscore has no special meaning if it is in the .txt instead. There's a script in the Every Dream tools repo that will push the filename captions into .txt for you, then you can modify from there. You may wish to modify the script to not use _ to delimit if you want prior to running it.
There's also a very simple GUI editor to edit the .txt files after that.
My understanding is most of the booru taggers are creating .txt files as CSV of tags, so they would work fine with underscores if it is putting them in the .txt or .caption file and not the filename. _ is only reserved in filename captions.
There's no doc for this but its very easy to use:
https://github.com/victorchall/EveryDream/blob/main/scripts/createtxtfromfilename.py
Simple gui mstevenson made:
https://github.com/victorchall/EveryDream/blob/main/doc/CAPTION_GUI.md
For tag shuffling, there's a more complex yaml option with more control but there's no easy way to edit or create them in bulk. Some sort of more "interesting" way to perform data augmentation has been floating around in my head and in the discord community, but it's not entirely obvious what the best approach is. Whatever it is, some meaningful editor is needed.
I'm not sure how to shuffle tags other than to have some sort of delimiter. If you have ideas I'm open. Long term I would like to use a DB (parquest/pandas, etc) to store tags or something I suppose?
Using ".." may create its own problems, there's no good way to use just any string for this without an actual structure to the data that exists outside the actual caption data. For instance, the yaml has structure and doesn't require any reserved characters. Reserved characters in general always cause problems, so I don't feel just changing the reserved character is actually a solution, and just causes problems for other people.
CSV is the current method because its pretty standard for booru tagging to make CSV format outputs. It was specifically added just for booru tag shuffling, and built around that.
from everydream2trainer.
Related Issues (20)
- [BUG] HOT 3
- [BUG] Runpod Customize Deployment option ignores password environment variables HOT 2
- [BUG]
- Failed to load checkpoint [ERROR] HOT 1
- [ERROR] HOT 1
- Request - safetensor model support for training on colab HOT 1
- [ERROR] RuntimeError while converting panopstor/EveryDream/sd_v1-5_vae.ckpt to Diffusers format HOT 2
- SDXL support HOT 8
- [ERROR] Something went wrong, attempting to save model HOT 4
- [feature] define dataset metadata outside of dataset folders HOT 1
- [BUG]The current model class (GitModel) is not compatible with `.generate()`, as it doesn't have a language model head. Please use one of the following classes instead: {'GitForCausalLM'} HOT 5
- [BUG] Using --disable_unet_training massively increases VRAM usage and crashes on 12gb VRAM HOT 1
- [BUG] Checkpoint does not load in InvokeAI, `state_dic` issues HOT 1
- Reinstall HOT 1
- [BUG] HOT 1
- Multiple GPUs Parallel Trainning. HOT 3
- [ERROR] HOT 1
- [ERROR] Cannot install the requirements on Linux HOT 5
- [BUG]about multi-gpu training HOT 3
- [ERROR] Colab CUDA setup Error HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from everydream2trainer.