← Home
Reusable agent system. Library-Agnostic by Design.
Restructuring a team of AI design agents so any designer can use it on any design system
Overview
Our UX team's AI design agents are genuinely powerful, a multi-agent system that takes a designer from a prompt, a PRD, or a Figma file all the way to a working, coded, validated prototype.
🚧
The catch: it was hardcoded to a single design library. Powerful for the one team on that library; unusable for everyone else.
I inherited that system and restructured it to be library-independent so the whole team can adopt it, with the design library as a clean, swappable layer underneath. The goal wasn't a new tool, it was one system every designer can use, whatever their product is built on, so designers own the full outcome of the experience instead of handing it off and hoping the intent survives.
The decision that shaped it
The interesting part wasn't the refactor , it was what I chose not to do.
I started with the obvious restructure: rename the agents, reshuffle the folders. Before executing, I pressure-tested it against the system's actual goals and found that ~70% of the plan was cosmetic, renames and folder moves that would break the orchestrator's routing and every saved workflow for zero functional gain.
The changes that actually mattered were invisible in the file tree:
💡
I optimized for leverage, not motion
I finished the library-agnostic refactor across every agent, collapsed the duplicated rules and accessibility into single sources of truth, fixed portability, and deliberately skipped the cosmetic renames. Drift went from "patched" to structurally impossible. Distinguishing real leverage from change-for-its-own-sake is the part I'm most proud of.
On paper it looked like a team of eight specialists. Underneath, it was a single-library tool wearing a team's clothes. The design library was written into every agent, the rules copied again and again, with no line between how the team works and which design system it's on. There was no home for brand intent or accessibility on their own, and the whole thing only ran on one person's machine.
AFTER - THE LIBRARY BECOMES A LAYBER YOU LIFT OUT
The system separates procedure from brand from library. Three conceptual layers so a designer can swap the design library underneath without touching how the agents work or what the brand stands for.
Everything lives in one agents/ folder by design. The tool only looks there to find the agents, so every file they read, brand rules, library packs, templates, has to sit alongside them. The one exception is raw, unedited library source code, which I keep in a separate folder that never ships with the system. It's just the raw material for building a new library pack.
HOW IT WORKS
The workflow mirrors how designers actually work, but starts wherever the project needs it:
Checkpoints sit at every phase so clarity and design thinking stay at the center, not speed. The design and build agents resolve the active library at runtime, so the output is on-brand whatever the product is built on.
IMPACT
The refactor didn't just clean up the system, it changed what adding a library costs.
5 → 1
copies of the design rule
1 → any
design systems served
drop a folder
to add a library ∙ 0 agent edits
📣
Actively expanding. If you'd like to hear the design decisions behind the system, or where it's going next, I'd love to walk you through it.
