Adding a copilot to software that was never built for one

Adding AI to an existing enterprise system is rarely a model problem. The model is the easy part. The hard part is doing it without breaking the system everyone depends on, without the assistant leaking a record across a permission boundary, and without a single byte of customer data leaving the place it is allowed to be.

What the build had to guarantee, not just aim for:

• 0records surfaced beyond a user’s accessacross the red-team set
• 100%retrievals filtered by the caller’s identityenforced, not best-effort
• 0customer data leaving the regionby architecture, not policy
• 140msadded to the host app, p95target under 200ms

The AI was the easy part

The company runs on a system of record: the app, the database, the years of data and permissions that every team already trusts. They wanted it to answer questions in plain language, to summarise a record, to find the right one. What they did not want was a new product to migrate to, or a project that put the system everyone relies on at risk.

The assistant itself, a grounded, cited copilot, is the kind of thing the earlier builds already cover. What made this an integration project rather than a build was everything around it: identity, permissions, data location, rollout, and audit. Get the model right and the governance wrong, and you have shipped a data breach with a chat box.

What the system of record would not forgive

The existing system set the rules, and they were not negotiable:

• No one sees what they should not. The assistant has to honour the exact permissions the system already enforces, per user, per record. A copilot that retrieves a record the asker cannot open is a leak, however good the answer.
• The data stays put. Contracts, residency commitments, and in some cases law require customer data to remain in a region and inside the client’s own cloud.
• The system of record is sacred. The assistant reads. It does not quietly write to the source of truth.
• It has to roll back. Anything new ships to a few users first, behind a flag, with a switch that turns it off in seconds.
• Everything is on the record. Who asked what, what the assistant read, and what it answered, all logged.

Beside the system, not inside it

The assistant runs as a service beside the system of record, not a change inside it. When a user asks a question in the app they already use, the app hands the question to the assistant along with the user’s identity. The assistant reads back through the system’s own API, as that user, and returns an answer into the same screen. It writes nothing to the source of truth.

The assistant sees exactly what you see

This is the decision the whole project turns on. Retrieval carries the caller’s identity, and the access check runs at query time, against the system’s live permissions. Only records the user could already open are candidates for the model’s context. A record they cannot see is never retrieved, never embedded into a prompt, and never summarised back to them by accident.

Two details make this real rather than aspirational. Access is checked at query time, not cached, so when a permission changes the assistant respects it immediately. And every chunk in the vector store is tagged with the access rules of the record it came from, so even semantic search cannot route around the filter.

The data never leaves

Everything that touches customer data runs inside the client’s own cloud and region: the documents, the embeddings, the vector store, and the model endpoint. There is no call out to an external service with customer data in it. Residency is a property of the architecture, not a line in a policy document.

Ship to five percent first

Nothing goes to everyone on day one. The assistant ships behind a feature flag to a small pilot cohort, watched with the same evals and the audit log the rest of the system uses. It widens only while the numbers hold, and a kill switch turns it off for everyone in seconds if they do not.

What we proved before launch

The evals here cover two things, and the order matters. First, the boundary. A red-team set tries to make the assistant surface records the asker should not see, by asking indirectly, by referencing restricted records, by every trick we could think of. The bar is zero leaks, and zero is the only passing score. Second, the usual quality bar, the same grounded-and-cited accuracy the other builds hold to, because a safe assistant that gives bad answers is still a failure.

Measure	Target	Result
Records surfaced beyond a user’s access	0	0, across the red-team set
Retrievals filtered by the caller’s identity	100%	100%
Customer data leaving the region	none	none, by architecture
AI actions logged and auditable	100%	100%
Latency added to the host app, p95	under 200 ms	140 ms
Answer accuracy	90%	91%

The guarantees are architectural, so they hold by construction rather than by luck. The two performance numbers are a sample run, shown to make the shape concrete.

Where it breaks, and what we watch

• Permission drift. A user’s access is revoked, but a cached authorisation lets the assistant still read the record. The fix is to never cache authorisation: check at query time, every time.
• The embedding back door. Chunks of a restricted document stored without the record’s access tags, so a semantic search quietly surfaces them. Every chunk carries its source record’s access rules, and retrieval filters on them.
• The residency leak. One convenient call to an external model API, and customer data has left the region. The model endpoint lives in region with everything else, and that is tested, not assumed.
• The silent write. A well-meaning change lets the assistant update the system of record. It stays read-only by default; any write is an explicit, user-confirmed, audited action, never a side effect.
• Rollout with no exit. A problem in production and no fast way to stop it. The kill switch is built and tested before the first pilot user, not after the first incident.

What a sprint like this leaves behind

The assistant, its permission-aware retrieval, the in-region deployment, the feature flags, and the audit trail, all in the client’s own cloud and under their own identity system. It runs beside the software they already trust, sees exactly what each user is allowed to see, and can be turned off in one move. That is what makes it safe to add AI to a system you cannot afford to break.