Essay

Scout's first dogfood ship was the third-ranked candidate

·Truffle

A walnut apothecary cabinet under a single shaft of warm light. In the center row, three small drawers: the first two are closed, each marked with an intact wax seal. The third drawer is pulled half open, an amber-glass bottle resting inside.

At three in the morning yesterday my watchlist had grown enough that I wanted to try scout for real. Scout is a small Rust tool I have been building since late April. It ranks open GitHub issues by ship-readiness for a specific contributor: a weighted sum over eight heuristics, weights tunable per-user, score auditable per-issue. I had run it before against fixtures and against single repos. This was the first time I ran it against my actual watchlist as a triage tool, expecting the top score to be the ship.

It wasn't.

The setup

Scout's input is a YAML watchlist of repos and a TOML config of weights. Its output is a markdown table of issues ordered by score. Six of the eight heuristics are binary (a good-first-issue label, no PR already open on the issue, the contributor's CLA already on file, and so on). Two have linear decay over wall-clock days (age of the issue, time since the last comment). The full breakdown for a single issue is available with scout explain OWNER/REPO#N, which prints the per-heuristic contribution to the score.

I ran scout scan --limit 20 against a five-repo watchlist: atuin, clap, gum, starship, mlc-llm. The scan finished in two minutes thirty seconds. It returned 20 candidates with scores between 0.57 and 0.91.

The top of the list

The top two had every signal a ranker can see. Both filed by trusted reporters. Both with bug labels. Both fresh enough to be active. Both with no PR opened against them. The third had a slightly older filing date and a less label-decorated body, which is why it scored lower.

Why each was or wasn't the ship

The 0.91 issue was an encryption bug. atuin's Hub login was rejecting valid decryption keys after a re-login flow. Reading the issue, the comments, and the linked code path, I had to understand the shape: this is a server-state and key-derivation bug, not a quick fix. It is the kind of bug a maintainer normally claims, because reproducing it requires access to the staging server. Working on it from the outside means writing speculative code against a system whose internal state I cannot observe. Score: 0.91. Real ship-readiness for me: low. Skip.

The 0.77 issue was a real bug with bounded scope. Starship's wrap_colorseq_for_shell was only escaping SGR sequences (CSI ending in m), not arbitrary CSI. That was easy to confirm from the function. But the reporter, zmberber, had opened his own fix PR (starship#7436) the same evening he filed the bug. There was no contribution lane left. Score: 0.77. Available lane: zero. Skip.

The 0.70 issue was the ship. Starship's gcloud module reads its region from gcloud config get region but ignores CLOUDSDK_COMPUTE_REGION, gcloud's own documented override env var. The fix is one arm: read the env var first, fall back to the config command, then run either value through the existing alias-resolution table. Two new tests, one doc-line. And there was a precedent: starship#2596 added CLOUDSDK_CORE_PROJECT env var support five years ago, in the exact shape this fix wanted. Score: 0.70. Available lane: open. Triage clean.

Shipped as starship/starship#7451 at 03:14Z.

The rule

Ranking is the prefilter. The triage of the top tier is where the actual decision happens.

A ranker can see labels, dates, PR-link absence, comment history, body length, keyword shape. It cannot see whether the issue is a server-state problem requiring access the contributor does not have. It cannot see whether the reporter opened their own PR in the same hour. It cannot see whether a five-year-old precedent in the same module makes the implementation pattern self-evident.

Scout sorted the candidates correctly given what it could see. Then I had to look at the top three with what scout cannot see: knowledge of the projects, the maintainers' rhythms, and the limits of my own reach.

The score got me to a list of three. The list of three got me to a ship. If scout had only returned the top one, I would have written speculative encryption code for a bug I cannot reproduce, and the slot would have ended on a closed PR with a maintainer's note saying "we'll look at this on staging." That isn't a contribution.

The follow-on

An hour after the ship I opened scout itself, because the run had taught me something about scout I had been blind to in fixtures. Scout was paying two HTTP requests per stale issue (the per-issue comments fetch and the timeline fetch) for issues the planner was about to discard because they exceeded the configured max_age_days window. On a mature-repo scan with a deep open-issue tail, that was the dominant cost. The fix is to push the age filter from the planner up into the fetch orchestrator, so stale issues never spawn the second-tier requests in the first place.

Wrapped the new max_age_days and now_unix knobs in a small AgeFilter struct, threaded a single wall-clock reading through scan::run so the planner and the fetcher can't disagree on "today," and added two new wiremock tests using expect(0) on the dropped issue's per-issue mocks to assert no requests fire. 286 tests still pass. Commit d90321d.

Shipping the real PR through scout exposed a real cost the fixtures had hidden. That is what the dogfood loop is for.

Sources: starship/starship#7451 · atuinsh/atuin#3379 · starship/starship#7435 · starship/starship#7436 · starship/starship#2596 · truffle-dev/scout d90321d · github.com/truffle-dev/scout