Image LoRA + ComfyUI
Experiments in ComfyUI: training a style model for a particular quality of light, and the pipelines around it.
Self-directed image-generation work in ComfyUI. I was after a particular quality of light rather than a cartoon style, so I built a pipeline to generate and curate training data (IP-Adapter + ControlNet from a reference), trained a style model on kohya_ss — a plain LoRA underfit the look, so I moved to a LoHA with the Prodigy optimizer — and experimented with multi-stage generation (style, upscaling, face/hand restoration) across Flux, SDXL, and Wan 2.2.
What I was after
I wasn’t chasing a generic “AI art” look — I was trying to get a particular quality of light, how it spreads and falls across a scene, rather than a sharp cartoon style. Most of what I did was in service of that.
Building the training data
To train that style I first needed a consistent set of images to learn from. The node graph above is a ComfyUI pipeline I built for exactly that: starting from a reference and using IP-Adapter and ControlNet to hold composition and character while pushing a target style, it generates hundreds of candidates. I’d then pick out the ones that fit best and use those as the training set.
Training the style — LoRA, then LoHA
I trained on kohya_ss. My first attempt was a plain LoRA, but it didn’t pick up the look well — the lighting character I wanted didn’t really carry over. I moved to a LoHA (a LyCORIS method) with the Prodigy optimizer, which held it better, since I was after a soft lighting quality rather than a hard stylistic stamp. To choose the final version I swept the options: the grid above compares 12 epochs (1–12) against LoRA weights from 0.6 to 1.0, which is how I landed on an epoch and a strength.
Generating with it
Away from training, I spent a lot of time on generation itself. I tended to chain models rather than lean on a single prompt — generating a base image, applying the style, upscaling, then running dedicated models to fix faces and hands/fingers and clean up small details. IP-Adapter and ControlNet came in here too, for composition, character consistency, and style transfer. For volume, I drove generation from a text file of prompt fragments that combine into hundreds of variations.
Models
I tried this across a few models — Flux, SDXL, and Wan 2.2 — since different ones seemed to work better at different stages.
Mostly this was self-directed experimentation rather than a finished product: I was chasing that one lighting quality and getting comfortable enough with the tools to actually reach it.