The Foundry Pattern
The Pattern
Knowledge Base
Karpathy — llm-wiki.md ↗
Mold
Cast
The skill is never hand-maintained. Under-instructed? Fix the Mold and/or its references and cast again. Prefer thin Mold procedures and rich references to maximize reuse.
Provenance
Not “the model probably knew this” —
this claim is real, and here is where it came from.
The picture to hold
The Case
Four arguments: engineering practice, scale, evidence, and resistance to rot.
An Argument from Engineering Practice
No serious engineering culture treats the compiled binary as the place you make changes.
The Foundry and Pinecone Nexus independently converge on the same diagnosis: agents need structured, governed, provenance-bearing knowledge—not raw retrieval reconstructed on every call. The Foundry bets on open-source artifacts; Nexus bets on an adapted runtime.
The Foundry work predates the Nexus announcement.
Pinecone Nexus and KnowQL
An Argument from Scale
All of human knowledge.
Inside the circle, AI generated skills without backing are mostly fine
And the interior use has a job
Spelling out the canonical pipeline is the difference between a small model doing the task and fumbling it.
Progress happens here.
Panel from The illustrated guide to a Ph.D. by Matt Might · CC BY-NC 2.5
A model is not another red shape.
The model is the circle.
At this scale, the world looks different.
Panel from The illustrated guide to a Ph.D. by Matt Might · CC BY-NC 2.5
Interior tools at the boundary
A Foundry is
an attempt to
push that edge and make it
executable.
Panel from The illustrated guide to a Ph.D. by Matt Might · CC BY-NC 2.5
An Appeal with Evidence
Blind regeneration
Delete a science skill. Hand a frontier model only the topic and the format. How much of the original does it write back from memory?
It wrote the skill back
It reproduced the citations too
You cannot tell, from the output,
which of those are correct.
A real citation and a confabulated one look identical on the page.
A smarter model produces more convincing unverifiable citations, not more verifiable ones.
And it added more validity caveats than the original
The two things it could not supply
Restating canonical knowledge is a commodity.
Traceable, check-backed knowledge is not.
The Case for Resisting Rot
The generated skill is
the perishable layer.
A skill frozen in place inherits the limits of the model that authored it.
Keep the source, and recast.
Targets change the package, not the source
Agent platforms will change faster than the domain knowledge they consume.
Binding source knowledge to today’s packaging format makes it inherit the platform’s lifetime.
An Implementation
You can use the pattern without foundry-lib
One concrete implementation: Astro + foundry-lib
Bootstrap the whole Astro stack
with an agent.
STANDING UP A FOUNDRY — WORKING CHECKLIST
vocabulary → reader + contracts → registries + validation + CI
→ corpus → Molds → Cast → external checkVersions, paths, packages, tests, handoffs, and stopping conditions—
enough to build the first reviewable vertical slice.
The Workflow Foundry
The same analysis, two directions
Example 2 extracted differential ATAC-seq from a completed run. A Foundry pipeline constructed the same analysis from the same initial prompt — no history, no execution.
%%{init: {'theme':'base','themeVariables':{'fontFamily':'Atkinson Hyperlegible','primaryColor':'#25537b','primaryTextColor':'#ffffff','primaryBorderColor':'#2c3143','lineColor':'#58585a','fontSize':'15px'}}}%%
graph LR
input_0>"ATAC counts"]
input_1>"sample metadata"]
step_0["DESeq2 differential test"]
step_1["Clean table (NA filter)"]
step_2["Volcano plot"]
step_3["Filter significant peaks"]
step_4["Sort by log2FC"]
step_5["Top gained peaks"]
step_6["Top lost peaks"]
input_0 --> step_0
input_1 --> step_0
step_0 --> step_1
step_1 --> step_2
step_1 --> step_3
step_3 --> step_4
step_4 --> step_5
step_4 --> step_6
classDef input fill:#edf4fa,stroke:#25537b,color:#2c3143;
classDef core fill:#25537b,stroke:#2c3143,color:#ffffff;
class input_0,input_1 input;
class step_0 core;- count matrix + sample sheet → DESeq2 → NA-clean → volcano ∥ filter → sort → top gained / lost
- every step a real, version-pinned Galaxy tool
tool_stateschema-validated against the Tool Shed bygxwf
Diagram emitted by gxwf mermaid from the pipeline’s output workflow.
We’re producing exciting results the model alone probably couldn’t replicate.
But every run also produces Planemo, gxwf, and Galaxy issues and PRs.
Sample sheet uploads, job cache tweaks, workflow draft schema changes, and more.
The problem is not closed — it is active research at the edge.
Panel from The illustrated guide to a Ph.D. by Matt Might · CC BY-NC 2.5
Interview → Galaxy
advance-galaxy-draft-step until no drafty step remains.
Patterns are the reusable moves
Structured Drafting of Workflows
Schemas are inherited as contracts
and enforced as rails
Upstream owns the contract. The Foundry pins it, records its lineage, and carries it into the cast.
draft-validate, validate, and validate-tests gate authored artifacts before expensive execution.The schema travels with the skill. The deterministic runtime owns the verdict.
Turn upstream interfaces into
deterministic agent rails
Instrument the real interface, formalize it, and put it inside the author–validate–fix loop.
cli_metadata; import machine-readable gxwf command specs.draft-next-step → author one step
→ draft-validate --concrete → gxwf validate
→ validate-tests → planemo testThe model translates and repairs. Independent tools select, validate, execute, and classify.
The Statistical Genomics Foundry
A solid rule:
derive abstractions from a second application
For a good developer, these approaches can converge: lessons from the first implementation naturally guide the second.
Fresh context breaks that equivalence
None of the lessons are learned if none of the context survives.
The second Foundry diverged at every level
technology stack
concepts
still produced a very different Foundry.
Extracting the shared surface was tedious. It resisted automation in exactly the place where abstraction was supposed to help.
I seeded the Foundry wrong.
It started with guiding principles that were vaguely fine—perhaps even useful—but did not reflect getting shit done.
I was trying to do four things at once
Each new document propagated values that were documented—and that I did not believe in.
Hundreds of skills are not
hundreds of specifications
You look at bioSkills and think there must be a principled way to reconstruct them.
- They have never been run.
- Inputs are not concretely defined.
- Expected outputs are not documented.
- There is no execution or test path.
- Licensing and installation remain unresolved.
The catalog contains ideas for programs—not enough operational evidence to reconstruct working programs.
A better skill mill is still a skill mill
The Foundry pattern could build a better one. It could even hide most of the implementation details.
That does not reflect who I am or my values.
Writing a program—which a skill is—without a concrete implementation, without documenting its inputs and outputs, and without explaining how to test it is not who I am.
It being 2026, and an agent being remarkably good at figuring out how to use it on the fly, is not going to change me.
So I stepped back.
more focused, more actionable—and still hugely ambitious.
The Topological Data Analysis
for Bioinformatics Foundry
The implementation issues
Planning Ahead
I had already done a huge amount of research—breaking the domain into kinds and collecting concrete examples and goals for each one.
Share durably.
Hardening leaves receipts
Sylverity/petls-pytorch#1
Merged and released: removed an unnecessary runtime dependency.
yubingapril/TopoQA#1
Open: reported the released (x,y,y) coordinate defect.
bdjones13/PETLS#2
Open: requested the missing software license.
TODO: file the prepared TopoMetry fixes and the documented scVelo defects upstream; add their issue and PR numbers here.
Open implementations—and studies that tested the claims
Persistent Laplacians rebuilt for PyTorch and CUDA.
Paper-derived persistent-homology interface features with corrected coordinates.
Paper-derived graph-attention architecture, retrained into a redistributable model.
Reproduced. The headline losses hold; the correlations were pooled and part of the margin depends on a defect.
Partially reproduced. The direction holds; the seeded 2-D layouts do not.
Inconclusive. 67.27% versus 90.90%; the locked Linux/CUDA arm remains.
Provenance boundary: open-topodockq-featurizer is also clean-room, but recovered through black-box observation rather than from the paper alone.
A paper becomes an executable, guarded action
Validate candidate complexes for one target, run the paper-derived open TopoQA featurizer and scorer in one locked environment, preserve failures, then return a stable ranking and selected structures.
A prediction—not measured DockQ or biological validation.
TODO: quantify persistent H1 in the TopoMetry cell-cycle extension and complete HiPoNet’s Linux/CUDA arm.
Panel from The illustrated guide to a Ph.D. by Matt Might · CC BY-NC 2.5
Instead of building a mill,
I feel like I’m pushing on the boundary!
A pattern for pushing into a domain with agents
I think this is a really nice set of ideas for pushing on a domain with agents. It feels like a competent implementation of a pattern that lets a human and an agent build up rich abstractions for interacting with that domain.
It is not a turnkey solution or one-size-fits-all infrastructure.
Knowledge about a domain—and descriptions of actions over it—should reflect the structure of the domain itself.
Discussion Questions
“How will we know when a Foundry skill is needed vs. what’s available in the LLM/harness and/or existing skill repos is not sufficient?”
Checks That Actually Fire
An oracle, not a test suite
## Property: blocks, each marked check: deterministic or check: llm-judged. How to judge any output.## Case: per fixture, carrying its own expect: assertions. What you feed in and what comes back.%%{init: {'theme':'base','themeVariables':{'fontFamily':'Atkinson Hyperlegible','primaryColor':'#25537b','primaryTextColor':'#ffffff','primaryBorderColor':'#2c3143','lineColor':'#58585a','fontSize':'15px'}}}%%
flowchart LR
mold["Mold<br/>index.md + references"]
scen["scenarios.md<br/>bind one case"]
srun["run the cast skill"]
chk{{"eval.md<br/>score every property"}}
jrnl["refinements/<br/>decision: eval-add"]
mold --> srun
scen --> srun
srun --> chk
chk --> jrnl
jrnl -. "reviewed, then re-cast" .-> mold
classDef nfile fill:#edf4fa,stroke:#25537b,color:#2c3143;
classDef nrun fill:#25537b,stroke:#2c3143,color:#ffffff;
classDef nchk fill:#f6f8fa,stroke:#8c6d1f,color:#2c3143;
class mold,scen,jrnl nfile;
class srun nrun;
class chk nchk;A check: deterministic property earns its verdict by running the oracle —
gxwf validate, a structural diff, planemo test. Marked “not run” because the tool
looked expensive, it is a failed trial, not a weaker pass.
One journey, an oracle at every step
%%{init: {'theme':'base','themeVariables':{'fontFamily':'Atkinson Hyperlegible','primaryColor':'#25537b','primaryTextColor':'#ffffff','primaryBorderColor':'#2c3143','lineColor':'#58585a','fontSize':'15px','tertiaryColor':'#ffffff','tertiaryBorderColor':'#d6dbe2','tertiaryTextColor':'#25537b'}}}%%
flowchart LR
subgraph J["one journey · one scenarios.md case"]
direction LR
p1["phase<br/>output"] --> v1{{"Mold<br/>eval.md"}}
v1 -. "fix it here" .-> p1
v1 --> p2["phase<br/>output"] --> v2{{"Mold<br/>eval.md"}}
v2 -. "fix it here" .-> p2
end
v2 --> pe["pipeline eval.md<br/>cross-step, end to end"]
pe -. "refinements to Molds and pipeline" .-> p1
classDef nphase fill:#25537b,stroke:#2c3143,color:#ffffff;
classDef nchk fill:#f6f8fa,stroke:#8c6d1f,color:#2c3143;
classDef nend fill:#edf4fa,stroke:#25537b,color:#2c3143;
class p1,p2 nphase;
class v1,v2 nchk;
class pe nend;[loop] phaseadvance-galaxy-draft-step runs until gxwf draft-next-step reports no drafty step remains.[branch] phaseeval.md is the one that applies.run-workflow-test without a green planemo test means the journey did not complete.A miss caught at the phase that produced it beats the same miss surfacing three phases downstream.
A false miss passes every static check
miss falls through discover-or-author — so the run authors a wrapper for a tool the Tool Shed already ships.integron finder scores 34.60 and resolves to iuc/integron_finder, changeset 5429646e486d. Only the raw id token missed.decision: eval-add — a normalization step in the procedure, a new property in eval.md, and a re-cast.A miss that an obvious name-variant would have turned into a hit is a failure.
ISEScan’s bare token happened to score, which masked the whole class of bug.
You cannot read a bibliography and find a wrong DOI
The repo owns the corpus. The package owns the mechanics.
audit-citations.config.json
Five source globs, trusted citation hosts, request budget.
audit/citation-audit.md
The rendered run. Carries no timestamp and no revision, so it diffs cleanly.
audit/provider-evidence.json
62 normalized provider answers. The offline replay reads only this.
pnpm validate replays the audit offline and fails when a citation has no committed evidence, resolves to a different work, or leaves a flagged finding unreviewed.
citation-audit.yml re-resolves every identifier against live providers and opens a pull request only when the rendered report changes.Three DOIs that named other papers
HERMES: Persistent spectral graph software → 10.3934/fods.2021004 resolves to Markov chain simulation for multilevel Monte Carlo.
Persistent topological Laplacians — a survey → 10.3390/math13020278 resolves to Prioritization of Preventive Measures … Road Infrastructure Projects.
Persistent sheaf Laplacians → 10.3934/fods.2024020 is a different work.
Next Steps
