How Middleware Governs LangChain’s Managed AI Agents

LangChain’s PII middleware can remove an email address before a managed agent sends the prompt to a language model or records it in LangSmith. That creates a useful privacy boundary, but it can also leave the agent unable to perform the task the user requested.

Nathan Drezner, a product manager at LangChain, puts the trade-off unusually plainly in the company’s demonstration of middleware for managed deep agents: “We’re going to break that with the middleware.”

The agent in his example has a tool that looks up a customer’s subscription plan by email address. Without redaction, the model receives the address and passes it to the lookup tool. With email blocking enabled, the address becomes a redacted placeholder before reaching the model. The lookup subsequently fails because its required identifier has disappeared.

That small failure captures the central problem in agent security. A control can protect data perfectly at one boundary while making the surrounding system useless. Good middleware design depends on choosing what the agent may see, what its tools still need, and what the organization may retain.

A Policy Layer Between Decisions and Actions

An AI agent combines model calls with tools, state and operational limits. Depending on its configuration, tools might retrieve customer records, query internal systems or take actions. Middleware inserts programmable checks at selected points in that lifecycle.

“Middleware is a way to extend your agent life cycle with customized behavior,” Drezner says. LangChain identifies policy enforcement, retries, rate limits, prompt transformation, error handling and logging as possible uses.

The familiar web-software term fits. Middleware sits along a processing path and inspects or changes what passes through. In an agent, however, that path has several security-sensitive junctions:

• before a prompt reaches the model;
• around a model call;
• before or after a tool runs;
• when token or rate limits are reached;
• when an error triggers a retry or fallback.

Placement determines the result. Redacting an address before the model call keeps it out of that model interaction. Wrapping a tool call can record which tool ran or stop the invocation under a policy. A rate limiter controls volume rather than content. Calling all three “guardrails” can obscure their separate jobs.

Middleware also runs code, so it joins the system’s trusted computing base. A custom component may inspect prompts, tool arguments and results. Its access makes it useful for enforcement and valuable to anyone who compromises or misconfigures it. Teams need to review middleware with the same care they apply to the agent and its tools, including its dependencies, permissions, failure behavior and logs.

Where the Data Boundary Actually Sits

LangChain’s example applies email redaction as input passes toward the language model. Drezner says the modified prompt “isn’t even stored inside of LangSmith,” the company’s observability platform, because the PII middleware changes the input earlier in the process.

That supports a narrow and valuable claim: the demonstrated email does not reach the model-facing step or the LangSmith trace shown in the setup. It does not establish that the address never entered the application. The user supplied it, and some component had to receive the original text before matching and replacing it.

A useful data-flow review therefore asks more than whether the LLM saw a value:

1. Where did the original request first arrive?
2. Did an API gateway, application log or error tracker record it?
3. Does redaction cover tool inputs and outputs as well as model inputs?
4. Can retries or exception messages restore the original value to a trace?
5. Who can access unredacted memory or state?
6. How long does each system retain the data?

The location of the first durable copy often matters more than the redaction label in an agent graph. If an upstream request logger saves the complete prompt, filtering only before the model reduces model exposure while leaving the earlier copy intact.

Regex Can Recognize a Pattern, Not Its Purpose

According to DeepWiki’s description of LangChain’s middleware implementation, the PII component detects and redacts sensitive information using regex-based detectors. Regular expressions work well for text with recognizable structure, including many email addresses.

Pattern matching has an unavoidable context limit. A detector can recognize that text resembles an email address. It cannot infer, from shape alone, whether the address is a customer identifier required for an authorized lookup, a fake test value, a public support address or personal data that policy forbids sending to a model.

That distinction belongs in system policy. Developers can block the value, mask part of it, replace it with a stable token, or route the request through a component allowed to handle the original. Each choice changes the threat model.

Stable tokenization, for example, could let an agent refer to the same customer without receiving the raw address, provided a trusted service maps the token to the account. Full redaction offers stronger separation but destroys the identifier. Partial masking may preserve usefulness while exposing enough information to remain sensitive. The supplied sources do not document comparative testing of those designs.

The demonstration also supplies no false-positive rate, false-negative rate or latency figure for the email detector. Those omissions do not invalidate the control. They define the testing still required before a team can treat a demo configuration as a production policy.

Observability Creates Its Own Privacy Problem

LangChain’s custom example uses a wrap_tool_call decorator to intercept every tool invocation and write basic information to standard output. Drezner suggests sending such events to tracing or performance systems instead.

Centralized tool-call records can answer important operational questions: Which tools run most often? Which fail? How long do they take? Did the agent attempt an action that policy should have blocked? An audit trail can also help reconstruct an incident after the agent makes an unexpected decision.

Logging can duplicate the data that redaction was supposed to contain. A logger that captures tool arguments may collect customer identifiers, search terms or retrieved records. A logger that records only a tool name and outcome reveals less, but it provides a thinner forensic record.

Teams have to choose the fields deliberately. Useful controls include an allowlist of recorded metadata, redaction before export, restricted access, defined retention and a tested response for logging failures. “Log everything now, sort it out later” is an expensive privacy policy wearing an observability badge.

Middleware order deserves attention too. If an audit wrapper records an argument before a redaction wrapper changes it, the trace may preserve the original. Reverse the order and the audit system sees only the sanitized version. A visual graph helps developers inspect that sequence, but tests should verify the data that each destination receives.

Prebuilt Controls Reduce Code, Not Accountability

Drezner recommends beginning with packaged components: “It’s always worth using the pre-built middleware before you go ahead and build your own.” That approach offers a sensible engineering advantage. A maintained component can reduce duplicated code and provide a common configuration model for routine controls.

Custom middleware remains necessary when an organization’s rules depend on business context. A bank, retailer and healthcare provider may all treat the same identifier differently. Even within one company, a support agent may need access that a marketing agent should never receive.

Prebuilt components still require review against the workflow. Relevant questions include:

• Does the detector cover the formats and languages users submit?
• Does the component fail open or fail closed when it throws an error?
• Can a user bypass matching through formatting or encoding?
• What happens when several middleware components modify the same input?
• Are tool results filtered before returning to the model?
• Can authorized support staff recover the original value when needed?

Custom code adds another set of concerns: maintenance ownership, test coverage and policy drift. A five-line wrapper can become security infrastructure the moment every sensitive tool call depends on it.

The useful production test is the one LangChain’s example accidentally makes memorable. Run a legitimate workflow with privacy controls enabled, inspect every copy of the data, and confirm that the user can still finish the job. If the address vanishes safely but the customer lookup collapses, the middleware has enforced one rule and exposed the next design decision.