I like Client Testing
and Dislike Galaxy’s
Unit Tests

John Chilton
The problem

The client is throwing 429s

John
“Test is throwing 429s — here is why the client is doing that.”
Hoping admins take pity and increase the rate.
Marius
“Please don’t make client DoS us.”
John
“But I have so much code to review already 😭”
Two fixes for #23977

About 3× as much test code as production code

Datatypes fan-out
2.9×
Stop unnecessary datatype loads and share concurrent requests.
PR #23995 ↗
Retries / hidden counts
3.2×
Space out retries and skip counts for badges that can’t appear.
Merged PR #24141 ↗
Historical diff snapshots · additions + deletions. Both fixes are now merged.
#23910 · a separate CSS fix

One selector. A 50-line test.

/* Before */
:deep(.dataset-details-popover) { max-width: 420px; }

/* After */
.dataset-details-popover { max-width: 420px; }
GPopover moves the content under body. The old descendant selector no longer matches. The plain scoped selector matches the content’s retained scope id.
Vitest doesn’t load SFC styles. The test reads the Vue file, compiles and injects its CSS, mounts the component, and checks the computed 420px cap.
#23910 · Fix storage run dataset popover overflowing the viewport ↗
The problem for the human reviewer

The cognitive load is too high

Better abstractions and cleaner separation
between setup and test code.
Better test abstractions can help

Make the scenario easier to see

Shared fixtures and mount helpers remove repeated setup. Named scenarios and exact assertions make the behavior easier to review.
Chat test harness
Two chat suites share their mounting, API spies and message helpers.
Typed dataset factory
A typed dataset factory replaces private builders; scenario overrides stay visible.
#24038 · Rewrite Client Tests toward Readability ↗
Setup and test code get mixed together

Use responses the server actually sends

Hand-written payloads can type-check and still describe a response Galaxy never sends. Repeated synthetic data also hides the test’s story.
Record backend responses. Load typed fixtures. Keep only the scenario’s overrides in the test.
Branch: vitest_real_api_calls ↗
Recorded API fixtures → history-summary factory → disk-usage tests. Separate from the story lane.
#24030 · Real API Responses / Models in Client Unit Tests ↗
Separating the responses from busy, dense test code is a start.
Storybook stories take it
a step further.
Setup belongs to the story

The test reads like the scenario

const wizard = await mountWizard(stories.WithUserAndDefaultZenodo);
await next(wizard);
expect(destinationCard(wizard, "zenodo-repository").text())
    .toContain("My Zenodo");
The named story owns the API responses and component setup. A reviewer can open that state in Storybook.
Story pass: named setup. Play pass: verify the visible result in Chromium.
Storybook showing the HistoryExportWizard Suggests Remote File Name story, the rendered export summary, named stories in the sidebar, and controls and interaction panels below

Try at home

“Yo agent - create a worktree locally with jmchilton’s fork of galaxyproject/galaxy on the fork’s vitest_story_play branch. Read https://github.com/jmchilton/galaxy-brain/blob/main/vault/projects/just_jesting_around/STORYBOOK_PITCH.md for context and then start the storybook server and open in my local browser.”

October 20 snapshot · combined prototype

The checks move — the browser gate matters

Catalog
100 files
583 collected browser stories, including interaction plays.
Strict browser run
432 / 151
Passing / failing stories. Accessibility failures remain.
Review units
96 tests
Per-test play commits, with accessibility annotations folded in.
A passing unit job does not run these plays or axe checks.
Browser CI is still required for adoption.
Render-only stories are not interaction checks. Counts are not coverage percentages.
Runtime tradeoff

About eight seconds longer overall

Roughly 12% longer
Moving checks out of units offsets much of the browser cost.
Earlier local benchmark: smaller catalog, accessibility excluded.
The current combined branch has not been benchmarked.

Extra Slides

Play functions

Check the state in real Chromium

Browser plays
Click, hover, and scroll. Query by role and label. Assert what the user can see.
Remaining unit tests
Keep checks about spies, request payloads, and controlled timing in Vitest.
Each moved check is tested against a deliberate break. Every original assertion should still have a home.
The shiny features

A browsable catalog is useful beyond tests

Now · stories and plays
Browse real component states and watch interactions step by step.

Now · accessibility alongside plays
Axe checks, violation highlighting and vision simulations in the same branch as plays.

Future · visual regression
Compare the same states during UI migrations. Screenshot baselines and a review workflow still need building.

150 known axe-failing stories ≠ 150 distinct bugs.
Viewport presets alone do not run a screenshot matrix.
“But these aren’t unit tests”
HOT TAKE: Unit tests are
a deeply human thing.
Part of their value is how they push us to think about the design. I don’t see agents responding to that pressure the same way.
Decomposition

Humans feel shame writing a bad test

A ton of mocking. A test more complicated than the feature or fix. We know when we’re writing a bad test — and the bad test is telling us the implementation sucks.
That pushes us to decompose and design better.
Good unit tests require good decomposition. That feedback is part of their value.
What I see with agents
They can write around the implementation
instead of rethinking it.
What I keep seeing: a complicated implementation gets a complicated test. The agent makes it pass, and I’m left reviewing both.
I want the test to help us see when the design needs work.
The review loop

Look at what these tests let us review

These Storybook tests communicate between the implementation, the agents, and the humans. That is the part I care about.
They’re easier to review — and the review covers so much more when you can see the story in action.
Discussion
Our review time matters too.
I want to see the story in action,
understand the behavior, and trust the test.