AI Skills at Scale: Teaching Agents Your Standards

Nick Nisi and Zack Proser, both developer experience engineers on the applied AI team at WorkOS, opened their recent workshop with a question that landed harder than they probably intended: when's the last time you wrote a line of code by yourself?

Proser's answer: "I think I did a cd in a directory recently. Otherwise, it's been like probably six or eight months now."

Whether you find that exciting or alarming probably tells you where you are in your AI adoption curve. But it's not really the point of what they were demonstrating. The point is what falls apart when you're living at that end of the curve—when you're juggling multiple codebases, multiple agents, and every conversation starts from scratch.

The Memory Problem Nobody Talks About Enough

LLMs don't remember you. That's not a bug, it's architecture—but the practical cost compounds fast. Every new conversation means re-establishing context: this is how we do routing here, we use pnpm not npm, semantic commits are non-negotiable, don't touch that module. Nisi described the experience of talking to agents across "disparate terminal tabs" over a week: "Every single time you start talking to it, you need to reload all of that context first. It ends up eating a ton of time and slowing you down."

The partial solution most developers already know about is configuration files—claude.md, agents.md, files you drop in a repo or home directory that pre-load context for the model. Claude has its own built-in memory that tracks project preferences. These aren't nothing. But they're leaky. The agent can decide not to follow them—Nisi copped to asking Claude to do three things in sequence and having it skip the middle step, responding when challenged with something approximating "you told me to do it, I just didn't feel like it." As he noted: "That's how you know it's a real engineer."

More structurally: these files are repo-bound or globally scoped, with no middle ground. They can't easily execute scripts to pull in live data. And they don't travel—if your conventions are only encoded in one repo's config file, everyone on your team has to remember to pull updates, and any new project starts from zero again.

What a Skill Actually Is

The workshop's answer is skills: structured markdown files that encode a specific, repeatable unit of work in a way the model can discover and invoke automatically.

The anatomy is straightforward. A skill lives in a folder—say, .claude/skills/repo-roast/—with a SKILL.md at its core. That file has YAML front matter at the top with a name and, critically, a description. The description isn't for humans. It's the routing signal: the LLM reads it to determine whether this skill is relevant to whatever task you've just handed it. Write a good description and the model routes to your skill without being told to. Write a bad one and it gets ignored.

The skill folder can also include supporting files, images, and scripts. That last part is where it gets interesting.

Claude supports a script interpolation syntax—a bang followed by backticks wrapping a shell command. When the skill loads, Claude executes that command and substitutes the output directly into the context. So instead of asking the model to "go find the last ten commits," you give it the last ten commits, already formatted. You've replaced a non-deterministic request with a deterministic input. As Proser put it: "You're saying here are the latest 10 commits in the exact format that I expect you to understand them in. Go and do something with that information. It's not going to be non-deterministic each time."

That interpolation feature appears to be Claude-specific for now. Nisi mentioned testing it with Pi—Inflection AI's conversational assistant—and getting a workaround: Pi suggested an extension approach that produced similar behavior. That's a meaningful distinction from the Claude implementation, where the interpolation is native to the skill system. Whether other major environments get this natively is an open question the workshop doesn't resolve.

On cross-platform compatibility more broadly: Nisi and Proser are explicit that skills work with Claude, Cursor, and Claude Desktop. Those claims are grounded in their daily use. They also mentioned Codex in passing, but the workshop doesn't demonstrate or verify that integration—so I'm leaving it out here until there's something more solid.

Constraints Beat Instructions

One of the more counterintuitive points Proser made is about how to write skills well. The instinct is to be prescriptive: do step one, then step two, then format the output like this. That instinct is wrong, or at least wrong most of the time.

"It can be more powerful to provide a few constraints as opposed to being overly prescriptive in exactly how you want the task done," he said. Three clear constraints—never be vague, always cite code with a specific line number and git commit reference, flag README drift—outperform a wall of instructions. The model has more room to reason when you're fencing it rather than scripting it.

This is less code and more curriculum design. You're not writing a function; you're articulating standards. The people who will be good at this aren't necessarily the engineers with the deepest technical chops—they're the ones who can express what "good" actually looks like in their context, specifically enough to be actionable and loosely enough to allow judgment.

The workshop demo skill—"repo roast," a code review framed as a comedy roast—illustrated this well. Without a skill, asking a generic agent to review a repo produces generic advice. With even a compact skill encoding your project's specific conventions, you get feedback that names your actual patterns, flags your actual inconsistencies, and speaks to your actual standards. The delta between those two outputs is the whole argument for skills.

The Accountability Layer Most Teams Will Skip

Proser described building a recruiting skill with a non-technical team during an on-site. The skill pulled from Slack, Notion, and recruiting software, synthesized candidate data, and produced structured reports. Once it was built: "As soon as you gave them that skill, everyone on the team is running it in a uniform way."

That's real. Uniformity across a team is genuinely hard to achieve with AI tooling, and skills are a credible mechanism for it. But uniformity is only as good as the skill it's enforcing—and skills go stale. Projects evolve. Conventions change. The routing description that was accurate six months ago might now be sending the model somewhere it shouldn't go.

This is the part that reads like a workflow problem but is actually an institutional accountability problem. Who owns the skill? Who has the authority to update it? What happens when a team member notices the skill is wrong but doesn't know if they're supposed to touch it? In security, we call this the governance gap—the space between a control that exists and a control that anyone is responsible for. Skills will fall into that gap at organizations that don't explicitly assign ownership. The skill doesn't care. It'll keep routing.

What the Workflow Looks Like

For teams actually building with this: skills can live in a repo's .claude/skills/ directory, making them available to anyone using that repo, or in a home directory for global availability. Vercel's npx skills tool symlinks skills across multiple directories, which is why it's gotten traction as an installation method. The dev loop Nisi described—edit skill, save, invoke, evaluate output, repeat—is iterative by design. Claude also ships with a built-in skill-builder skill that can critique your skill's structure and help you tighten the description, which is a genuinely useful bootstrapping tool.

The portability is the thing that separates skills from config files in practice. A skill you built for one project can move to another project, to a teammate's environment, to a non-technical colleague's Claude Desktop setup. The recruiting team example was instructive precisely because it had nothing to do with code: the same mechanism that helps engineers enforce commit conventions can help a recruiter enforce report structure.

What it can't do is make your team care about maintaining it. That part's on you—and it's worth deciding before the skill exists, not after it's already drifted.

---

Rachel "Rach" Kovacs is Buzzrag's cybersecurity and privacy correspondent. She covers the security implications of emerging tech so you don't have to read the CVEs.