LORA fine-tuning with .txt caption files?

[sashasubbbb]

Hello! I'm having diffuculties adapting my dataset to these scripts. Is there any way to start LORA fine-tuning with .png files and corresponing .txt tag caption files in same directory? How do i ensure i'm starting LORA fine-tuning and not dreambooth style training? Are regularized images and .json files necessary for fine-tuning?
Thank you.

Edit: i managed to successfully start training by manually creating buckets and .jsons from .txt captions following this guide: https://github.com/kohya-ss/sd-scripts/blob/main/fine_tune_README_ja.md
I wonder can this process be automated?

[kohya-ss]

Hi, I'm glad to hear the training is started.

Currently LoRA fine-tuning with caption files requires the metadata in .json file. Because it assumes that both tags and captions are present. Tags and captions are combined in the .json file.

However, if there is only captions or tags, .json might not be necessarily. I have a plan to refactor DreamBooth/fine tuning/LoRA training scripts to use the common Dataset class, I will consider whether I can add the training without .json file.

[catboxanon]

@kohya-ss Apologies for bumping this discussion -- just to clarify, I assume this refactor was completed? As far as I can tell it has been (prior loss weight only seems to be taken into account only when regularization images are used), but there's a few remnants which possibly suggest otherwise, like this:

sd-scripts/train_network.py

Line 85 in 6417f5d

use_dreambooth_method = args.in_json is None

I bring this up since the majority of users (English-speaking, at least) who do LoRA training simply use image and .txt caption pairs without regularization images, and oftentimes with hundreds or thousands of these image/caption pairs, usually with BLIP or WD tagger output as the caption rather than rare tokens, all of which goes against the original idea of Dreambooth.

[TingTingin]

The refractor was completed