All articles
6 min read

Tarnkappe: a small, open model that finds personal data on web pages

Every BetterPrivacy scan reads hundreds of web pages. To flag names, emails, phone numbers and addresses on all of them, we built a detector that runs in a few milliseconds on an ordinary processor, and we are releasing it as open source.


1.5 ms

per short text on one CPU core, no GPU

100%

of persons found on 23 real scanned pages it was never tuned on

5 MB

model, plus 25 MB of word vectors; English and German

Tarnkappe is the cloak of invisibility from German legend. The model finds the personal data in a text so it can be flagged or hidden. This article explains why we built it, how it works, how well it does, and where it still gets things wrong.

Why we built our own

Good open models for finding personal data already exist. The best known are the GLiNER family, which are accurate, Apache-licensed and label any category you ask for. We tested two of them throughout this work. They have two problems for a service like ours.

  • They are big and slow on ordinary servers. Every open model we found has 150 million parameters or more. On the same 2-core machine, GLiNER and GLiNER2-PII took about 200 to 280 ms per text. A scan covers hundreds of pages, and we run scans for every user.
  • They give us nothing of our own. Anyone can download them. A fast, small, European model that is open and honest about its limits is something no existing model offers.
Tarnkappe
≈ 2 ms
GLiNER PII v1
≈ 280 ms
GLiNER2-PII
≈ 250 ms

Time per text, same 2-core machine, CPU only. Measured on short texts and on scanned page excerpts of about 400 characters.

What else is out there

ModelSizeLicenceNotes
GLiNER multi PII v1≈ 278M parametersApache 2.0Six languages including German; labels chosen per request; trained on synthetic data.
GLiNER2-PII≈ 300MApache 2.0Newer, 42 types; strongest of those we tested; over-flags persons in informal text.
NVIDIA GLiNER-PII570MNVIDIA Open Model LicenceEnglish, 55+ types.
OpenAI Privacy Filter1.5B (≈ 50M active)Apache 2.0Mainly English; an independent test found no organisation or location label.
Tarnkappe2.5MApache 2.0 code, CC BY-SA 4.0 weightsEnglish and German; built for scanned web pages.

Sizes and licences from the linked model cards and articles.

How it works

Cheap checks run on everything; the neural network only handles what patterns cannot.

  1. Patterns and checksums catch structured identifiers: email addresses, IBANs (mod-97 check), card numbers (Luhn check), IP addresses, and phone numbers through Google's libphonenumber. Postcodes count only next to a street or city, or after a word like "PLZ".

  2. A small neural tagger with 2.5 million parameters reads the text once and labels each word as person, organisation, location, street, postcode, email or phone. It combines pretrained word vectors, a character-level network and a two-layer recurrent network, and it sees line breaks, so menu entries and list rows stay apart.

  3. Rules split each person into first and last name, mark "Name GmbH", "Name Ltd" and similar as companies, and never call a street name or a user name a company.

  4. Scan-level context. A scan is about one person and many pages. The person's own name is known in advance and is found everywhere: in a headline, in capitals, inside a domain name. Company names are shared across pages, but only when they carry a legal form or turn up on two different pages, so one wrong guess is not spread.

What it was trained on

We collected no personal data to train it. Everything in training is public, licensed for commercial use, or generated.

DataSizeLicence
Universal NER, English web text12,543 sentencesCC BY-SA 4.0
GermEval 2014, German Wikipedia and news24,000 sentencesCC BY 4.0
WNUT 2017, social media3,395 sentencesCC BY 4.0
Broad Twitter Corpus6,192 tweetsCC BY 4.0
Wikipedia paragraphs, German and English, labelled by GLiNER2-PII and shaped like web pages11,000 paragraphsCC BY-SA 4.0 text; Apache 2.0 teacher
Generated sentences with invented names, addresses and numbers25,000ours
Presidio Research templates, half of them8,000MIT

The Wikipedia step is a form of knowledge distillation: the large GLiNER2 model labels text, and the small model learns from its labels. We only taught labels the teacher was confident about, because GLiNER models tend to mark ordinary words such as "I" or "owner" as persons.

How well it works

On real scanned pages

The test that matters is real pages from BetterPrivacy scans. Four people agreed to be scanned for a test the model had never been tuned on. We saved the model's answers first; afterwards a person marked every person and company by hand.

23 page excerpts, 4 scansPersons foundPersons correctCompanies foundCompanies correct
Tarnkappe100%93%92%34%
GLiNER2-PII82%88%100%28%

"Found" is the share of marked names a system detected. "Correct" is the share of its detections that were right. 39 person and 13 company mentions, so each miss moves a figure by several points.

Persons pass the bar we set for release (95% found, 85% correct). Companies do not: these pages mention few real companies, and the model also flagged medical terms, product names, places and page text. Two fixes made after this test raised correct company detections to 46%. On a further 65 excerpts from five other scans, which we used while tuning, it found 96 to 100% of persons and 78 to 84% of companies, with 78 to 92% of its detections correct.

On public test sets

Test set, never trained onPersonal data found
Persons in English web text (UNER EWT)95.8%
Persons in German Wikipedia and news (GermEval)95.8%
Persons in tweets (Broad Twitter Corpus)79.8%
Persons in social-media posts (WNUT 2017)79.7%
All types, Microsoft Presidio sentences from unseen templates96.1%

On the Presidio set, 93.6% of what it flagged was real personal data. Our phone and postcode rules were written after studying misses on that set, so it overstates those two types.

Where it still fails

  • Company flags. Most company hits on pages with few companies are wrong. Bare site names such as a blog's domain are also missed.
  • Tweets and chat. Retraining for web pages cost accuracy on informal text: persons found in tweets fell from 88% to 80%.
  • Unusual addresses. Postcodes in multi-line addresses from countries our rules do not cover are often missed. German, Austrian, Swiss, UK and US formats are handled.
  • Languages. Only English and German so far.
  • No guarantee of anonymity. It is a tool to find and minimise personal data, not proof that a text is anonymous.

How we handled personal data in testing

Testing on real pages means handling real personal data, so we kept it narrow. Only people who agreed were scanned. From each page we kept only short excerpts that mention the scanned person. The excerpts and their hand labels stay private and were never used as training data. Only totals are published, here and in the repository.

Open source and what comes next

The code is released under Apache 2.0 and the model weights under CC BY-SA 4.0, because part of the training data carries that licence. Both allow commercial use. The repository includes the training script, the benchmark and the full results, including the failures: github.com/ncserny/betterprivacy-tarnkappe.

  • In BetterPrivacy: Tarnkappe will soon be built into the scan, flagging personal data on every page, with company flags shown as hints.
  • Next version: fewer false company flags, informal text back to its earlier level, and a larger set of real pages for testing.
  • API: an EU-hosted detection API is planned. Text is processed in memory and never logged or stored.

Sources

  1. GLiNER multi PII v1 (model card) — Hugging Face
  2. GLiNER2-PII (model card) — Hugging Face
  3. NVIDIA GLiNER-PII (model card) — Hugging Face
  4. OpenAI Privacy Filter: overview — Grepture
  5. OpenAI Privacy Filter: independent test — PrivacyBridge
  6. On synthetic training data and real documents: benchmark — Tonic
  7. UNER English EWT (training and test data) — Universal NER
  8. GermEval 2014 (training and test data) — GermEval
  9. WNUT 2017 (training and test data) — GitHub
  10. Broad Twitter Corpus (training and test data) — GATE, University of Sheffield
  11. Wikipedia dataset (training data) — Wikimedia on Hugging Face
  12. Presidio Research (training and test data) — Microsoft

All Tarnkappe figures were measured by us and can be reproduced with the benchmark in the repository.