What TokenVeil detects
Each detected value is replaced with a typed, reversible token (for example <PERSON_1> or <IBAN_CODE_1>). This page lists what is covered out of the box, and where the two editions differ.
Two layers
Section titled “Two layers”- Deterministic detection (both editions): pattern and checksum rules for values that have a fixed shape. High precision, because a checksum or a shape is a proof rather than a guess.
- Named-entity recognition (commercial edition): a statistical model for values that have no fixed shape, such as a person name in the middle of a sentence.
Deterministic, in both editions
Section titled “Deterministic, in both editions”| Category | Detected |
|---|---|
| Contact | email addresses, phone numbers (including bare-digit numbers under a phone key) |
| Financial | IBANs, card numbers (including 15-digit Amex), transaction amounts |
| Network and infra | public and internal IP addresses, MAC addresses, hostnames, system user names |
| Secrets | API secrets, private keys, JWT-shaped tokens, high-entropy tokens |
| National identifiers | French NIR, US SSN, license plates |
| Structured and logs | key-driven detection for structured data (a value is tokenized based on its JSON or column key), log identities, customer references |
| Your own terms | a custom-terms list for internal codenames and identifiers, per user or instance-wide |
Two useful details: a person name that directly follows a title (Mr, Mrs) is caught by the deterministic layer, and transaction amounts keep their order of magnitude (thousands, millions) so a model can still reason about scale without seeing the real figure.
Free text, in the commercial edition
Section titled “Free text, in the commercial edition”The commercial edition adds a trained model for entities that have no fixed shape and no adjacent key:
- person names in free-text prose,
- organizations,
- locations.
This is what catches a surname mid-sentence with no marker, which the deterministic layer alone does not do.
Extending and honesty
Section titled “Extending and honesty”You can add your own recognizers (national IDs, company registrations, policy numbers, internal formats) and languages. See extending detection. No detector is perfect: recall and precision depend on your content, which is why we benchmark on your real data during a pilot rather than quote a single number.