Before sending text to an LLM API, build a deliberate privacy layer that identifies personal data, removes or masks what the task does not require, and records enough information to test the result. This checklist explains how to choose between redaction and pseudonymization, detect common PII fields, protect reversible mappings, test edge cases, and review the pipeline as data sources, providers, languages, and application requirements change.
Overview
PII redaction before an LLM request is a data-processing step, not a prompt-writing trick. The goal is to reduce unnecessary exposure while preserving the context the model needs to complete a task. For example, a support-ticket classifier may need an order reference and product name but not a customer’s email address, phone number, or full street address.
A practical pipeline usually has five stages:
- Define the task. List the information the model must receive to produce a useful result.
- Identify sensitive fields. Combine deterministic rules, pattern matching, dictionaries, and an entity-recognition method appropriate to the text and language.
- Transform the text. Remove, mask, generalize, or pseudonymize fields that are not necessary.
- Validate the transformed content. Scan again, inspect logs, and test whether the intended meaning remains intact.
- Send only the minimum required payload. Keep raw text and mapping data outside the request path unless there is a defined operational need.
No detector is perfect. Names can resemble ordinary words, identifiers can have multiple formats, and personal data may appear in images, attachments, metadata, or free-form notes. Treat automated detection as a control that requires testing and monitoring, not as proof that a document is safe.
What to track
1. Maintain a PII field reference
Start with a field inventory based on your sources and use cases. The following categories are a useful baseline for an LLM privacy review:
| Category | Examples | Possible transformation |
|---|---|---|
| Direct identifiers | Names, email addresses, phone numbers | Remove or replace with typed tokens such as [PERSON] and [EMAIL] |
| Location data | Street addresses, precise coordinates, workplace locations | Generalize to city, region, or country when precision is unnecessary |
| Account and transaction data | Account numbers, order IDs, customer IDs, payment references | Mask, tokenize, or retain only a non-sensitive suffix if the task requires distinction |
| Government or organizational identifiers | Tax, employee, license, passport, or national identifiers | Usually remove or use a stable opaque token |
| Health, financial, or employment details | Diagnoses, balances, salary information, performance notes | Remove, summarize, or restrict to the exact attribute needed |
| Quasi-identifiers | Age, uncommon job title, dates, or combinations of attributes | Generalize, aggregate, or assess the combination rather than each field alone |
Use labels that make transformed text easy to inspect. A consistent token such as [EMAIL_1] is more useful for debugging than an ambiguous string of asterisks. However, labels should not contain the original value, predictable fragments, or secrets.
2. Choose redaction, masking, or pseudonymization
Redaction permanently removes the value from the model input. Use it when the model does not need the information and the downstream result can stand without it. Example: “Contact [EMAIL] to confirm the shipment.”
Masking hides part of a value while retaining a limited signal. It can help distinguish records without exposing the full identifier, but partial values may still be sensitive. Example: “Card ending in [LAST4].” Do not assume that a few retained characters are harmless in every context.
Pseudonymization replaces a value with a stable substitute, such as Customer_042. This is useful when the model must follow a person or case across several messages. It is not the same as irreversible anonymization when a separate mapping can reconnect the token to the original person.
Use a decision rule: if identity continuity is unnecessary, redact; if a small amount of context is enough, generalize or mask; if continuity is essential, pseudonymize with strict controls. Ask whether the model output can be linked back to the source through the request, response, logs, prompts, traces, or stored conversation history.
3. Protect reversible mappings
If pseudonymization is required, keep the mapping service or table separate from the LLM request. A simple internal record might contain:
{
"token": "PERSON_042",
"entity_type": "person",
"source_reference": "internal-record-id",
"created_at": "internal timestamp",
"purpose": "support-ticket summarization",
"expires_at": "review or deletion date"
}
Do not place the original name, email, or identifier in the token itself. Restrict access to the mapping, encrypt it according to your organization’s security practices, define retention and deletion behavior, and avoid sending the mapping to the model. If the application does not need to restore the original values, prefer one-way transformation or ordinary redaction.
4. Test detection with realistic cases
Build a small, versioned test set before production use. Include ordinary and difficult examples, such as:
- “Email [email protected] about order 84721.”
- “Call +1 555 010 0199 after 3 p.m.”
- “The office is at 14 Example Road, Bristol.”
- “Patient ID AB-123-XY appears in the attached note.”
- “Please ask Jordan to review the file.”
- Names, addresses, and identifiers embedded in quoted replies, signatures, tables, and unusual punctuation.
Test false negatives, where sensitive data remains, and false positives, where useful non-sensitive text is replaced. Also test multiple languages, character sets, date formats, phone conventions, OCR output, and text copied from email or PDFs if those inputs are in scope. Record the detector version, ruleset, configuration, and test result so a later change can be compared with the previous baseline.
Cadence and checkpoints
Review the redaction pipeline on a monthly or quarterly cadence, depending on the sensitivity and volume of the application. A regular review does not need to change the design every time; it should confirm that the assumptions still hold.
- Monthly operational check: sample transformed requests, inspect detection misses, review error rates, and confirm that raw text is not appearing in application logs, traces, analytics, or error messages.
- Quarterly control review: reassess the PII inventory, retention settings, mapping access, supported languages, detector configuration, and test corpus. Confirm that the task still needs every field being sent.
- Release checkpoint: run the full test set whenever you change a detector, transformation rule, prompt, model, SDK, preprocessing library, or output workflow.
- Provider checkpoint: before onboarding or changing an API provider, review the provider’s current documentation and your organization’s requirements for data handling, retention, access, regional processing, and account configuration. Record what was checked rather than relying on an old assumption.
Automate as much as possible. A scheduled job can scan representative payloads for known patterns, compare detection results with a fixed test set, and alert when a previously protected category appears in outbound traffic. An automated check should support, not replace, access controls and human review.
How to interpret changes
A change in the pipeline is meaningful only when you understand its effect on both privacy and task quality. If a new rule increases the number of redactions, determine whether it found previously missed PII or merely replaced legitimate content. If summaries become less accurate, check whether the model needed relationship or timing information that was removed.
Review results across at least four dimensions:
- Coverage: Which intended PII categories were detected, and which formats or languages were missed?
- Precision: How often was ordinary content transformed unnecessarily?
- Utility: Can the model still classify, extract, summarize, or draft the required output?
- Exposure: Could the original value be inferred from surrounding context, retained fragments, stable tokens, logs, or the model response?
When results worsen, isolate one variable at a time. Compare the old and new transformation output, then inspect the request actually sent to the API. A prompt can instruct a model not to repeat sensitive data, but that instruction is an additional safeguard, not a substitute for preprocessing. For broader application controls, see the LLM Prompt Security Checklist and the LLM Evaluation Metrics and Testing framework.
When to revisit
Revisit this checklist immediately when you add a data source, support a new language, process attachments or OCR text, introduce a new model or API provider, change regional routing, or expand the application to health, financial, employment, or other sensitive workflows. Also review it after a suspected data leak, a detector failure, a material prompt change, or an unexpected model output that reproduces source details.
For routine maintenance, schedule a monthly operational check and a quarterly design review. Keep a short change record containing the date, reason for review, detector and transformation versions, test results, provider assumptions, unresolved risks, and the owner responsible for the next check.
Before your next production request, use this final checklist:
- Write down what the model actually needs.
- List direct identifiers, sensitive attributes, and combinations that could identify someone.
- Select redaction, masking, generalization, or pseudonymization for each category.
- Keep reversible mappings separate and access-controlled.
- Run a second scan on the transformed text and inspect representative payloads.
- Test names, emails, locations, identifiers, multilingual text, formatting variations, and attachments in scope.
- Check logs, traces, prompts, responses, and stored conversation data for unintended copies.
- Record the decision and set the next monthly or quarterly review date.
PII protection is strongest when it is treated as a repeatable engineering control: minimize the input, transform it before the API boundary, test the transformation, and revisit the assumptions whenever the system changes.