It Started with Turbo

I wasn't hunting for a convention fork. I was trying to improve my few-step turbo LoRA — a DMD-style distillation student — and wanted to know what the official turbo distillation had actually changed: partly curiosity, partly to steal a warm start. So I diffed the official turbo checkpoint against the base model and ranked the delta by relative movement.

The result was unambiguous. The single largest weight movers in the entire delta were the AdaLN up-projections (.2, relative Δ 0.008–0.03). Their paired down-projections (.1) barely moved (0.0002–0.006). Attention and MLP — everything my trainer targets — moved less than a family of layers my trainer doesn't touch at all.

Better yet, the movement is cheap to capture. The up-projection's input dim is only 256, so a rank-96 extraction keeps ~99% of the adaln delta's energy; adding adaln lifted the extraction's mean captured energy from 0.803 to 0.860 over the adaln-less version. Concentrated, low-rank, high-leverage — exactly the kind of change a LoRA is for.

So why had I never trained these layers? And what were they, exactly? Answering the second question took an afternoon. Answering the first turned into the git-blame that is the rest of this post.

Two Mysteries About a Module Called "AdaLN-LoRA"

Anima is a DiT that inherits Cosmos-Predict2's per-block modulation. Every transformer block has three small MLPs that turn the timestep embedding into shift / scale / gate coefficients for its self-attention, cross-attention, and MLP branches. They're conditioned on the diffusion timestep only — no text, no spatial signal — so what they encode is global, per-σ behavior: tone, contrast, how hard each branch fires at each noise level. Few-step distillation is precisely a remap of "how the model behaves at each σ," and adaln is the model's per-σ behavior dial. First mystery solved: the official turbo pulls hardest on adaln because adaln is the only place a σ-remap can live that cheaply.

The second mystery was the module's name. Every Anima checkpoint stores modulation in AdaLN-LoRA bottleneck form — Linear 2048→256 down, Linear 256→6144 up, per branch, per block — and for a while I read "LoRA" as "adapter," something optional that the turbo release had bolted on. It isn't. The name is NVIDIA's, and it's a pretraining architecture choice: full-rank modulation would be 2048×6144 × 3 branches × 28 blocks ≈ 1B params, and the 256 bottleneck cuts that to ~176M. These weights exist in every checkpoint, base and turbo alike.

There's even a red herring guarding the answer. The Anima author's own trainer, diffusion-pipe, vendors NVIDIA's reference model code, whose class signature reads use_adaln_lora=False — which looks for all the world like the author switching the module off. Follow the loader instead: it hardcodes use_adaln_lora=True, adaln_lora_dim=256 for every checkpoint it loads, and ComfyUI's model detection does the same. So "why does the official turbo model have an adaln module?" has a boring answer — every Anima does; it's the architecture — and an interesting one: turbo is where those weights move most, because distillation is a σ-remap.

So this is not a norm you can wave away. It's a lever — and, it turns out, one family of LoRAs pulls it while the other leaves it bolted down.

Two Kinds of LoRA for the Same Model

Once you know to look, there are two families of Anima LoRAs in the wild, and you can tell them apart by a set of keys. LoRAs trained with the author's own trainer carry weights for the adaln modulation layers — both projections, plus the LLM adapter, strictly more surface than my target set at equal rank. LoRAs from the sd-scripts lineage don't. Load an adaln-carrying LoRA in ComfyUI and it applies those keys. Load the same file through my training repo's inference path and they get silently dropped, because my target set never included them. Same checkpoint, two different pictures, depending on which tool opens it.

That's not a bug in either tool. It's a fork in convention that happened somewhere upstream:

So the model's own author ships a trainer that adapts adaln, and the most popular inference tool applies it. Even kohya's repo ships a converter (convert_anima_lora_to_comfy.py) that already handles the adaln key names — an sd-scripts user who opts in gets a ComfyUI-shippable adaln LoRA today. The "don't train adaln" default lives entirely in the sd-scripts lineage — and I copied it from there. The question is who put it in sd-scripts, and whether it was ever a decision.

The Question: Who Decided adaln Shouldn't Be Trained?

Short answer, which I'll spend the rest of the post earning: nobody did. No one sat down and concluded that the modulation MLPs are a bad LoRA target. The exclusion is a side effect of a refactor whose stated goal was cleanup, and it has been inherited, comment-hardened, and shipped ever since without anyone re-deciding it. Here's the trail.

Archaeology

git blame on the exclude line in sd-scripts/networks/lora_anima.py:

34e7138b (Kohya S. 2026-02-13) exclude_patterns.append(r".*(_modulation|_norm|_embedder|final_layer).*")

Commit 34e7138, by the maintainer, on 2026-02-13. But the file itself is older than that line. The first commit to touch lora_anima.py is e21a773 — the community pull request (#2260, "Support Anima model," by duongve13112002) that introduced Anima support five days earlier, on 2026-02-08. So the exclude was not part of the original Anima support. Something changed it in between. What did the original do?

In e21a773, adaln was a first-class trainable target with its own rank knob:

# type_dims = [self_attn_dim, cross_attn_dim, mlp_dim, mod_dim, llm_adapter_dim]
identifier_order = [
    (4, ("llm_adapter",)),
    (3, ("adaln_modulation",)),   # <- adaln had its own dim slot: mod_dim
    (0, ("self_attn",)),
    (1, ("cross_attn",)),
    (2, ("mlp",)),
]

adaln sat in the same list as self-attn, cross-attn, and mlp, with a dedicated rank slot (mod_dim). If you passed no per-type dims at all, every Linear — adaln included — got the default rank. In other words, the original community implementation trained adaln by default and gave you a knob to size or disable it. Whatever else you think of that design, it treated adaln as a real, adaptable target.

Then came 34e7138 — pull request #2261, "Add/modify some implementation for anima." It's a grab-bag: a dozen bullet points spanning typo-config fixes, a Qwen-Image VAE swap, fp8 loading, a WIP minimal-inference script. Buried in the middle is one line:

feat: simplify target module selection by regular expression patterns

That bullet is the whole story. The refactor tore out the per-type dim system — self_attn_dim, cross_attn_dim, mlp_dim, mod_dim, llm_adapter_dim, gone — and replaced it with a single rank plus a regex include/exclude scheme. And the new default exclude was:

# add default exclude patterns
exclude_patterns.append(r".*(_modulation|_norm|_embedder|final_layer).*")

Look at the company adaln's _modulation is now keeping: _norm, _embedder, final_layer. That's the conventional "structural plumbing you don't adapt" bucket — the layers every LoRA trainer skips reflexively. adaln got swept into the housekeeping exclude alongside them. Not because someone benchmarked it and found it useless. Because when you're collapsing five explicit target types into one tidy regex, the modulation MLPs look like norm-adjacent structure, and lumping them in makes the diff smaller and cleaner.

There is no ablation in that PR. No discussion of adaln as a target. No note reading "excluding modulation because X." The demotion from first-class target to excluded-by-default is a single line inside a commit about typo configs and VAE compatibility, and it is invisible as a decision precisely because the surrounding diff reads as cleanup. A reviewer skims "simplify target selection by regex," sees a tidier file, and approves. The semantics flipped underneath the syntax.

The Inheritance

Here's where I come in. My repo's exclude regex:

# Default exclude regex appended to user-supplied excludes.
# Skips embedders / norms / modulation projectors that are never adapted.
_DEFAULT_EXCLUDE = (
    r".*(_modulation|_norm|_embedder|final_layer|adaln_fused_down|adaln_up_|"
    r"pooled_text_proj).*"
)

That is kohya's regex, verbatim at its core, extended with a few of my runtime rename names (adaln_up_ is what the module is called after my loader renames it). I inherited the exclude without ever questioning it — and then I did something worse than copy it. I wrote a comment: "modulation projectors that are never adapted."

Never adapted. Stated as a property of the architecture, a law of nature. Except it's false: the model's own author adapts them in diffusion-pipe, ComfyUI applies them, and the original community PR that this whole lineage descends from trained them by default. The assumption didn't just get inherited — it hardened one layer downstream, turning "kohya's regex happened to exclude this" into "this is a thing that is never adapted." That comment is the drift fossilized.

Meanwhile the other half of the ecosystem never drifted at all. diffusion-pipe → ComfyUI is a straight line: the author trains adaln, the popular runtime applies it, third-party LoRAs carry it. It's only the sd-scripts branch — and everyone downstream of it, including me — that quietly stopped. The split isn't two independent conventions. It's one convention (train everything) and one accident (a regex that looked like cleanup), now presented to users as if both were considered choices.

Un-Bolting the Lever

Knowing the history changes what you're allowed to assume, so the repo changed with it — measured moves, not a counter-swing:

• Extraction ships adaln now. The delta extractor grew an --include_adaln flag plus a ComfyUI-layout rename at save time, and the result was verified through ComfyUI's own load_lora: all 364 patch targets consumed — 280 attention/MLP, 84 adaln. The official turbo's biggest movers are no longer left on the extraction floor.
• Training is one regex away. Include patterns beat the default exclude, so a train_adaln knob now wires adaln into the turbo student's target set — optionally at its own smaller rank, which is nearly free (the up-projection's input dim is 256, so rank 32 already keeps 98% of the delta's energy).
• The comment is gone. "Never adapted" no longer appears in my codebase.

What hasn't changed is the default: per my repo's own rules, default changes are bench-gated, and rendered A/Bs will decide whether trained adaln actually helps a few-step student or a style LoRA. I audited every checkpoint I've ever trained — zero adaln keys, nothing mis-shipped; those models just learned to compensate through attention and MLP. The exclude may well survive the bench. The difference is that it would then be a decision, held in place by evidence, instead of a never held in place by nobody.

The Lesson

The concrete cost of the drift was small but real. adaln is where turbo distillation moves hardest, so excluding it by inheritance meant every few-step student I trained was leaving its single highest-leverage target on the floor and forcing attention/MLP to compensate around a shift they can't fully reach. And the cross-tool render mismatch — an ecosystem LoRA looking different in my repo than in ComfyUI — is a genuine support-bug surface that traces straight back to this line.

But the part worth generalizing is the shape of how it happened. The drift didn't enter through a bad decision. It entered through a good one — "simplify target selection with a regex" is a perfectly reasonable refactor — that carried a semantic change as cargo. The most durable defaults in a codebase are rarely the ones someone argued for in a design doc. They're the ones nobody noticed changing, because they rode in on a commit labeled cleanup, and every layer downstream treated the inherited behavior as intentional. By the time it reached me, "excluded by a regex in 2026" had become "never adapted," asserted in a comment, with no memory of the alternative.

So the practical takeaway is narrow and specific: audit your own exclude lists for inherited never comments. Any time a default is justified by "this is just how it's done," there's a decent chance the real answer is "someone made the diff smaller three repos upstream, and no one has re-decided since." It took an official checkpoint shouting — the largest movers in the whole turbo delta, sitting inside my exclude regex — for me to even ask the question.

Trail: official turbo delta (adaln up-projections are the largest movers) → sd-scripts e21a773 (Anima support, adaln trainable) → 34e7138 ("simplify target selection," adaln excluded) → my repo's inherited _DEFAULT_EXCLUDE — now with --include_adaln extraction and a bench-gated train_adaln opt-in.