A problem most people don't notice until it's too late.
More and more business processes run through AI: contracts, customer correspondence, HR documents, medical records, citizen requests. As soon as personal data (names, addresses, phone numbers, diagnoses, bank details) enters these flows, the company automatically falls within the scope of the GDPR and the responsibility that comes with it.
The typical way to "solve" this: ask the language model itself, in the prompt, not to mention personal data or to anonymize it carefully. It's an intuitive solution, which is exactly why it's so common. The problem is that it doesn't stand up to scrutiny.
A language model is a probabilistic system. It can follow an instruction in 95 cases out of 100 and fail in the remaining five, unpredictably: on an unexpected phrasing, a rare name or an unusual document structure. For most tasks that error rate is acceptable. For personal data it isn't: a regulator, an auditor or an affected customer isn't interested in statistics. They're interested in the specific case of the leak.
Our principle: data protection must not depend on whether the model "understands" the instruction
When we build AI products that handle sensitive information, we follow a simple rule: a safety-critical function cannot rest on a neural network "generally behaving correctly". It has to be moved into a separate, independently verifiable mechanism that either guarantees the data is cleaned or blocks it from passing further down the chain. No in-between states, and no hoping for luck.
In practice this means an architectural separation of roles:
- The language model does what it is good at: understanding meaning, generating answers, working with content.
- A separate protection layer is responsible for exactly one thing, the control of personal data, and works by rules that don't change from run to run and don't depend on how a request is phrased.
This separation is not a technical detail but a deliberate architectural decision. It means that when someone asks "how do you guarantee the protection of personal data?", the product has a substantive answer rather than a reference to the model's good conduct.
The prototype of such a PII filter was developed by our engineer Sergey Kostyaev.
How this is verified in practice
The statement "our system is protected" is worth nothing without systematic testing. Before such a protection mechanism is considered ready for real customer data, it goes through:
- A large set of test scenarios, deliberately built to include not only the obvious cases but also edge cases, provocations and intentionally confusing phrasings. Exactly the situations where a "polite instruction" to the model fails most often.
- Testing in both directions: the system must not only reliably remove what it must remove, but also leave text untouched where there is no personal data. Excessive, aggressive cleaning also counts as a defect, because it makes the product useless.
- Independent control sets that do not overlap with the main tests, to rule out the system simply having "memorized" specific examples instead of following the rule.
- Honest documentation of known limitations. If a mechanism has a documented edge of capability, it is reported openly rather than masked by convenient statistics. We consider transparency about a system's boundaries part of trust in the product, not its weakness.
Where appropriate, a compact model running locally inside the client's infrastructure is used as well, with no data sent to external cloud services. Such a configuration is only enabled once it has passed the same strict testing and shown a stable result under a pinned, reproducible configuration, not "the latest version available today".
Why this matters to you as a client
If AI touches personal or otherwise sensitive data in your product, an internal process or a client-facing service, sooner or later one of three questions will come up:
- From a regulator or auditor: how can you demonstrate compliance with data protection requirements?
- From a client or partner: what happens to their data inside your system?
- From your own team: what happens if the model makes a mistake one day?
The answer "we asked the AI to be careful" fails every one of these checks. The answer "we have a separate, tested, independently verifiable control mechanism" passes them.
If this is relevant to you
We design and implement protection layers like this as part of AI products for our portfolio companies and partners, from analysing which data and risks are present in your process to a finished solution built into the product.
If your business works with personal or sensitive data through AI systems and you want to understand what their protection actually rests on, write to us at info@bergstone.de. We'll discuss your specific case.