PII-Safe RAG and Agents: Reversible Tokenization in Production

The Problem: RAG Leaks

Let me be direct: the moment you build a RAG pipeline over real business documents, you have almost certainly built a data-leakage machine. Customer names, IBANs, health IDs, and case numbers flow from your documents into three untrusted places: the model provider, the vector store, and your logs. Most teams do not notice until a security review or a regulator asks where the personal data went.

The naive fix is to strip it. Replace every name with [REDACTED] and move on. That breaks two things at once. The model loses the context it needs to answer well, and retrieval falls apart because the masked query no longer matches the masked documents. You need the personal data gone from untrusted systems, but the meaning preserved. That is a harder problem, and it is solvable.

The goal is not to delete personal data from your AI pipeline. It is to make sure the raw values never cross into a system you do not trust, while the model still has enough to do its job.

We build this discipline into client systems, and we built a runtime for it: OGuardAI, a proprietary data-protection layer that sits between your application and the model. This guide is the architecture of doing PII-safe AI properly. For the business framing, see our AI data leakage prevention guide, and this is core AI services work.

Who Cares About What

The Trust Boundary

Everything starts with one line drawn through your architecture. On one side, the trusted zone: your application and the protection runtime. On the other, the untrusted zone: the model provider, the vector store, third-party tools, and your logs.

   TRUSTED                          │   UNTRUSTED
                                    │
  App ──▶ Protection runtime ──────▶│──▶ LLM provider
          (detect, tokenize)        │──▶ Vector store
          (restore on the way back) │──▶ Logs, tools
                                    │
  raw PII lives here, transiently   │   only tokens cross this line

Raw personal data exists only transiently inside the runtime, during a request. What crosses the boundary is tokens and safe metadata, never raw values. The design principle that makes this trustworthy: configuration can only add or tighten protection, never loosen the built-in floor. You cannot accidentally configure the leak back in. Our AI governance guide covers why this "fail toward safe" posture matters for agentic systems.

Detection: Finding the Sensitive Bits

You cannot protect what you cannot find. Detection runs in two layers.

The first is a deterministic regex floor: dozens of format-based patterns that are language-agnostic. Emails, phone numbers, IBANs, VAT IDs, credit cards validated by checksum, IPs, national IDs, tax IDs. These are fast, on the order of a millisecond, and they never miss a well-formed IBAN because they do not depend on a model's mood.

The second is named-entity recognition for the things regex cannot catch: person names, organizations, locations, addresses. This runs through a model-based detector and is broadly multilingual, at a higher latency cost.

The important part is the failure policy. You choose a mode: builtin only, both with graceful fallback to the regex floor if NER is down, or advanced where NER is required and the request fails closed if it cannot run. A system that silently degrades to "no PII detection" because a sidecar timed out is worse than one that refuses the request. Fail closed.

Semantic Tokens, Not Redaction

Here is the key idea that makes the model still useful. Instead of deleting a detected entity, you replace it with a structured token that carries its type and a unique identity, but not its value.

Input:  "Email the invoice to Max Mustermann at max.mustermann@beispiel.de"
Masked: "Email the invoice to {{person:p_001:ad4f97}} at {{email:e_002:5873cc}}"

The token tells the model "there is a person here" and "there is an email here," and it keeps them distinct and referenceable, so the model can write a coherent reply about them. The raw name and address never leave your trust boundary. On the way back, the runtime restores the real values into the final output.

This beats [REDACTED] in every way that matters. The model keeps the structure and relationships. Identical values collapse to the same token, so the model understands that two mentions are the same person. Entities near each other can be linked, so a name and its email restore coherently. Redaction throws all of that away.

The Six Restore Modes

Not everything should come back in full. Restoration is governed per entity and per output channel, with six modes.

So a support reply to the customer can restore their name in full, while the same interaction written to an internal log restores only a masked form. The channel decides. This is how you serve a useful answer to the person while keeping your logs clean.

The RAG Problem: Retrieval Must Still Work

This is the part that trips up every team that tries to roll their own. If you mask a document with random tokens at ingestion, and then mask the user's query with different random tokens at search time, retrieval breaks. The query token for "Mustermann" does not match the document token for "Mustermann," because they were minted independently.

The fix is deterministic, corpus-scoped tokens. Within a corpus, a value is tokenized through a keyed hash so that the same value always produces the same token, in every document and in the query. Now a masked query matches masked chunks, because "Mustermann" tokenizes identically on both sides.

Ingestion:  doc chunk  "...contract with Mustermann..."  ──▶  "...contract with {{person:h_9f3a...}}..."
Query:      "what did Mustermann sign"                    ──▶  "what did {{person:h_9f3a...}} sign"
                                                                       │
                                        same value ──▶ same token ──▶ retrieval matches

The tokens are derived per corpus from a secret, so the same value in a different corpus is an unrelated token, and a hash collision fails closed rather than merging two different people. This is what lets you run retrieval over masked documents with masked queries and still get correct results. We cover the retrieval side generally in our enterprise RAG systems guide and vector search architecture guide. Combined with the agent-memory patterns in our agent memory guide, this is how you build agents that remember and retrieve without hoarding personal data.

Sessions: Where the Mapping Lives

The mapping from tokens back to real values has to live somewhere secure and be usable across replicas. The default is a sealed session: the mapping is encrypted with AES-256-GCM into a blob that travels with the request, so the server holds no state and any replica can process any request. For shared setups, an encrypted store keeps only ciphertext plus non-personal routing metadata.

Every session load re-validates the tenant it belongs to and fails closed on a mismatch, so one tenant can never unseal another's mapping. Sessions expire on a TTL, and a tampered or expired blob is rejected by its authentication tag. This is the boring cryptographic plumbing that makes the whole thing trustworthy, and getting it wrong is how "we tokenize PII" becomes "we have a shared secret leak." Our designing systems for failure guide covers this defensive mindset.

Policy: Declaring What Happens

You do not hard-code protection decisions. You declare them in policy, so a security or compliance owner can read and change them without touching application code.

name: support-desk
version: 1.0.0
defaults:
  restore_mode: masked
rules:
  - entity_type: credit_card
    action: redact          # never reversible, never reaches the model
  - entity_type: person
    action: tokenize        # reversible, restored to the customer
    restore_mode: full
  - entity_type: email
    action: tokenize
    restore_mode: masked    # masked in logs, full to the user by channel

Actions are explicit: tokenize for reversible protection, redact for irreversible removal, abstract for a coarse label. An unknown policy name fails the request rather than falling back to something permissive. The floor cannot be weakened by policy, only tightened. This is authorization-as-configuration, and it keeps the security decision out of scattered code and in one reviewable place. Our AI governance guide explains why that separation matters.

The Drop-In Proxy

The fastest way to protect an existing app is to change nothing in it. A transparent proxy speaks the same API as your model provider, so you point your SDK at the proxy URL and it masks on the way out and restores on the way back. No application rewrite.

Your app ──▶ proxy (mask) ──▶ OpenAI / Anthropic ──▶ proxy (restore) ──▶ your app

The proxy holds the session, so your app never handles the token mapping at all. For teams that want tighter control, thin SDKs in several languages give you the same primitives with framework integrations for the common agent and RAG stacks. Either way, the integration is a seam, not a rewrite, which is what makes this practical to adopt. Our custom software and consulting teams wire this into existing systems.

GDPR: Erasure by Revocation

A right-to-erasure request has a clean answer in this model. Because the mapping from token to value is the only link to the real data in your pipeline, you revoke it. The stored mapping keeps only keyed hashes of values, never the raw values, and once a value is revoked it restores to a deletion marker regardless of what any session blob says. A replayed or rolled-back session cannot un-revoke it.

That gives you a targeted, provable erasure across the AI pipeline, with an audit receipt and no raw personal data in the audit trail itself. We cover the wider compliance picture in our GDPR and AI guide, and how we handle it in delivery on our trust and GDPR pages.

Getting Started

1. Map your data flows. Find every point where documents or messages reach the model, the vector store, or logs.
2. Start with the regex floor. Structured PII, IBANs, cards, emails, is the fastest, highest-certainty win.
3. Add NER where names and addresses matter. Pick your failure mode deliberately.
4. Use corpus-scoped tokens for RAG. This is what keeps retrieval working over masked data.
5. Declare policy, not code. Put the protection decisions somewhere a compliance owner can read.

This is staged, low-risk work, and it fits the phased delivery in our methodology. Start at contact or get a scoped quote.

Common Ways This Goes Wrong

1. Redacting instead of tokenizing. You lose model context and break retrieval. Use reversible semantic tokens.
2. Independent tokens for query and documents. Retrieval breaks. Use deterministic corpus-scoped tokens.
3. Silent degradation on detector outage. "No detection" is a leak. Fail closed.
4. Protection logic scattered in code. Nobody can audit it. Declare it in policy.
5. No erasure path. If you cannot revoke a mapping, you cannot honor a GDPR request. Design deletion in.

Who Builds This

Oronts is a founder-led software company in Munich. Refaat Al Ktifan, our founder and solution architect, leads a senior team across backend, security, and AI. OGuardAI is our own data-protection runtime, built because we needed this discipline in our own AI work before we could recommend it to clients. We map your data flows, add the protection layer as a seam, and hand you a pipeline that is defensible under GDPR and the EU AI Act. See our services and solutions pages.

Takeaways

• A RAG pipeline over real documents leaks PII to the model, the vector store, and your logs by default.
• Tokenize, do not redact, so the model keeps context and identical values stay linked.
• Use deterministic corpus-scoped tokens so retrieval still works over masked data.
• Keep the token mapping in encrypted, tenant-bound sessions, and fail closed on any error.
• Declare protection in policy, and honor erasure by revoking the mapping.

"We tokenize PII" is easy to say and hard to do right. The details, corpus-scoped tokens, fail-closed detection, tenant-bound sessions, are what separate a real protection layer from a false sense of security.

If your AI pipeline touches personal data, tell us how it flows today. Start at contact or get a scoped quote.