Training a DiT LoRA on a downscaled latent at high noise should be free: the noise already ate the detail. Measured at the gradient level it is only sometimes free, and the map of when comes from two huge perturbations that nearly cancel each other. Here is the whole story without the appendix.
Sequel to the chaos-floor post. This time everything ran with --deterministic — and a validation gate still failed at 0.33 against committed reference measurements. Nothing changed: not the weights, not torch, not the driver. Just two reboots and an empty /tmp. Determinism pins a kernel path; it can't pin which path you're on.
A YAAR sequel: agents building apps inside the OS keep hitting bugs in the OS itself — so I gave them a GitHub app agent that verifies reports against real source and files the issue.
We seeded everything — init, data order, noise, even per-step σ draws — and still got checkpoint cosine 0.41 between two runs of the identical command. The culprit is the one RNG no seed reaches, and one flag makes training bit-exact.
While improving my turbo LoRA I diffed the official turbo checkpoint against base: the biggest movers in the whole delta were AdaLN layers my trainer never touches. I git-blamed that exclusion back through the ecosystem to a single line in a "simplification" refactor that nobody framed as a decision.
Introducing YAAR, a generative OS where the AI makes the system calls — and a proposal for 'model DX' as the design discipline behind it.
An empirical-ML research workflow where decisions come from measurement, not plausibility — gates kill expensive mistakes early, dead hypotheses still get a tombstone, and even a clean PASS has to answer one last question: "but is this a contribution?" And where Claude rides the loop with you.
Few-step distillation usually means holding a teacher and a student model in memory at once. We distilled a 28-step diffusion teacher into a 4-step LoRA on one 16 GB card by running three roles off a single frozen backbone.
How I got an MoE-style LoRA to actually specialize on a T2I model — through a cold-start deadlock, a failed jitter attempt, orthogonalized experts via SVD slicing, expert warmup, and σ-conditional routing borrowed from T-LoRA.
Sharing a persona prompt, memory layout, and the 'excuse tool' — patterns I kept refining from Opus 4.1 through 4.7.
DiT training has three sources of shape dynamism that cause torch.compile to recompile every step. We eliminated all three and got stable compiled training on a consumer GPU.
Flash Attention 4 doesn't support consumer Blackwell GPUs yet. We fixed three critical bugs and got it running on the RTX 5060 Ti.
A C++ critique video as a lens into vibe coding and the myth of total code comprehension.
Two precision-oriented features for the LoRA training pipeline: lora_fp32_accumulation and attn_softmax_scale.
Personal opinion on the paper 'Epiplexity'
How to build native Windows desktop applications that integrate with the Claude Code CLI using a pure Rust backend.
Personal opinion on AI slops
An exploration of the challenges in long context language model research.
A personal pick of recently published papers that show strong potential.
A personal take on the role of persona in the agentic LLM paradigm.
How to build native Windows desktop applications that integrate with the Claude Code CLI using a pure Rust backend.