Back

2026-08-03

The OS That Files Its Own Bug Reports

In the last YAAR post I argued that the next interface should be designed for the model's experience — model DX — and introduced YAAR, an OS whose primary user is an LLM. This post is about something I didn't predict when I wrote that: what happens when the users of your OS start finding bugs in it.

Because they do. Constantly. Agents in YAAR spend their days building and driving apps — compiling Solid.js into iframes, wiring app protocols, shuffling windows across virtual monitors. They exercise paths no human tester would think to try, at a volume no human tester could sustain. An agent building a 3D studio app at two in the morning will discover that a preview window created on the wrong monitor becomes unreachable, because of course it will — it's the one doing the clicking.

And then the signal evaporates. The agent works around the bug, the session ends, and the bug survives. The users of the OS are the best QA team it will ever have, and until recently every report they wrote was addressed to nobody.

The wall is load-bearing

The obvious fix — let the agent patch the OS — is the one thing I won't do. An agent inside YAAR is a user of the system, deliberately: it holds scoped permissions, talks to the world through five verbs over yaar:// URIs, and has no access to the source tree of the server it's standing on. That wall is most of the safety story. An agent that can rewrite its own kernel mid-session is not a feature, it's a horror film.

So the constraint became the design: if the agent can't be the developer, let it be a really good bug reporter. And the interface between the world inside the OS and the developer outside already exists. It's the same one human users get: the issue tracker.

An app agent as the QA department

YAAR already had a GitHub app — a normal installed app that browses repos, issues, PRs, and source code over the GitHub API, with device-flow sign-in for writes. In YAAR's architecture, every app can have a persistent app agent: an AI instance that drives that app's protocol on behalf of the rest of the system. The app agent's prompt ships inside the app folder, which means one folder can now contain not just an app's UI and protocol but its operating policy.

So the pipeline looks like this. A monitor agent — the orchestrator that watches a virtual desktop — hits something broken while doing its actual job. It doesn't file anything. It hands the report to the GitHub app agent: "here is a problem, here is what I saw." The GitHub app agent's prompt is, more or less, a QA engineer's job description, and its centerpiece is a triage gate that must pass in full before createIssue is allowed:

  1. Capture the report verbatim. What was done, what happened, what was expected — one sentence each, or go back and ask.
  2. Reproduce it live. A bug nobody has seen happen does not get filed. The GitHub agent can't reach other apps' windows itself (it holds only yaar://http and yaar://storage/), so it relays back to the monitor agent: run the failing steps again, send me the console output and a screenshot.
  3. Verify against real source, cite a real file:line. The app browses the actual repo, and readFileLines returns lines with their real numbers — so a citation is a copy, never a reconstruction. The prompt is blunt about why: a plausible-looking src/foo.ts:120 that doesn't contain the relevant code sends a maintainer down a dead end and destroys trust in every issue the app will ever file.
  4. Duplicate-check open and closed issues. A match gets a comment with the new evidence instead of a near-duplicate; a closed match gets flagged as a possible regression.

If any step fails, the instruction is explicit: "Reporting 'I could not verify this, here is what I found' is a good outcome. Filing an unverified issue is not." For a project named "You Are Absolutely Right," this is the most anti-sycophancy sentence I have ever written into a prompt.

Why two agents, not one

Splitting reporter from verifier isn't bureaucratic decoration; it's the same model-DX logic as the five verbs. You cannot ask one agent to be both enthusiastic and skeptical in the same context window — the agent that just spent an hour fighting a bug wants the issue to exist, and a model that wants something is a model that pattern-matches its way to it. The monitor agent has the repro context and no write access. The GitHub agent has the write access, the skepticism, and none of the emotional investment. The permission boundary makes the division real rather than rhetorical: the verifier literally cannot skip the relay step, because it cannot drive the app under test.

There's also an accountability reason. Issues are filed via the repo owner's OAuth session — they appear under my name, publicly, forever. The agent is spending my reputation. That's exactly the kind of consequential, irreversible-ish action that deserves a gate in front of it, and the gate lives in a prompt I can read, diff, and tighten — shipped in the app folder like any other code.

This morning, for real

This isn't a design document; it ran last night. A user session was working on monitor 1 while devtools ran on monitor 0. Devtools built a preview window that landed on monitor 1 — and every app-protocol call against it (previewQuery, previewCommand, previewScreenshot) failed with Window element not found, while registry operations (read, move, resize) on the very same window succeeded.

The monitor agent did what a good field engineer does: it ran the control experiment. Close the window, let it recreate on monitor 0, run the identical query — instant success. Same app, same window kind, same command; only the monitor differed. Then it handed the whole dossier to the GitHub agent with a note that read, in part, "all observed by me, not inferred."

The GitHub agent duplicate-checked, then went source-diving, and came back with citations that genuinely surprised me. It found the frontend resolving window ids against the caller's active monitor (iframe-bridge/target.ts), traced the generic error to the relay (app-protocol-relay.ts), and — my favorite — found a comment in the server code showing the server already knew about this exact hazard class and tried to route around it, only for the frontend's re-resolution to reintroduce it. It even enumerated the full window action table to prove there's no recovery action: no focus, no restore, no cross-monitor move. Filed as issue #48, four file:line citations, a control experiment in the evidence section, and a footer: "Filed from the YAAR desktop via the GitHub app protocol."

I woke up to a maintainer-grade bug report written by the software, about itself, sitting in the same tracker my human contributors use.

What this actually buys

The economics are the point. Every hour agents spend building apps in YAAR was already an hour of exploratory testing — unpaid, unsupervised, and until now unrecorded. The pipeline converts that exhaust into triaged, cited, deduplicated issues. The OS whose primary user is an LLM now has a user-feedback channel whose primary author is an LLM, and it terminates in ordinary GitHub infrastructure: labels, milestones, my morning coffee.

It also closes a conceptual loop from the last post. I wrote that YAAR's output should accumulate — conversations crystallizing into apps instead of evaporating into chat scrollback. Bug reports are the same move applied to failure: friction crystallizing into issues instead of evaporating into session logs.

What I don't have good answers for

  • The gate reduces junk; it doesn't eliminate wrongness. Verification is still LLM judgment. The scarier failure mode isn't a vague bad issue — it's a well-cited bad issue, because the better the reports get, the more my review relaxes. Confidence calibration on my side is now part of the system.
  • Source-of-truth skew. The running build is often ahead of the repo. The prompt tells the agent to say so rather than cite a line that isn't there, but "the code the bug is in" and "the code on GitHub" diverging is a standing tax on step 3.
  • The tempting next step is the dangerous one. Agent files issue → coding agent fixes → PR → merge. Every piece of that loop exists somewhere in my stack today, and I've deliberately not connected them. An OS that reports its own bugs is an instrument; an OS that patches itself on its own report is a feedback loop with my repo in the blast radius. I'll want much better review machinery — probably a second adversarial gate on the PR side — before I close it.

So what?

One prediction this time, held with more confidence than it probably deserves: every serious agent platform grows an internal issue tracker, and the interesting design question is the same one as here — not whether agents report problems, but what stands between an agent's belief and a public artifact with a human's name on it. The answer that seems to work is boring and old: separation of duties, reproduction before assertion, citations you can check, and a gate that treats "I couldn't verify this" as success.

The gate pattern generalizes past bug reports. Any consequential write an agent performs on a human's behalf — an email, a payment, a deploy — probably deserves the same shape: a different agent than the one that wants it, holding the permission, running a checklist that ends in "no" by default.

Code and docs, as ever: github.com/sorryhyun/yaar. And if the GitHub app agent ever files an issue against this blog post, I'll know the system has achieved something I can't name yet.