Who Actually Controls Your Data Once AI Touches It?

Ownership of data is easy to assert and surprisingly hard to demonstrate.

A useful test: pick a document your organisation holds and ask where copies of it exist right now. Then ask which of those copies you could delete this afternoon if someone asked you to.

Before AI, that was a tedious question with a findable answer. Afterwards, it is a genuinely difficult one — and for one of the answers, the honest response is never.

The five places it comes to rest

When a document enters an AI system, it does not simply sit in storage and get read occasionally. It gets transformed, copied, and transmitted, and each step leaves something behind.

The original. Still in object storage, in a region you chose, under controls you understand. This is the copy everyone thinks about.

The vector index. The document is split into passages and each is converted into a numerical representation for searching. This feels like a harmless technical artefact. It is not. Published research reconstructs substantial portions of source text from these vectors alone, especially for short passages — and splitting a document produces exactly short passages. The index is a copy of the document in a different shape, and it deserves identical treatment.

The context window. Each time someone asks a question, the system builds a brand new document at runtime: parts of your file, parts of others, instructions, conversation history. It exists for seconds, is rarely recorded in full, and usually travels outside your infrastructure to reach the model. Nobody classified it, because it did not exist until the moment it was needed.

The provider's logs. Many model providers retain what you send them for a period, for abuse monitoring. That is a real copy, in a real jurisdiction, governed by a contract most people have not read. Enterprise agreements usually let you shorten or disable it — but the default is frequently not what your privacy documentation claims.

The model weights. If your data was used to train or fine-tune a model, this is the copy you cannot get back.

The one you cannot undo

That last one deserves its own section, because it is the only truly irreversible item on the list.

Deletion, as a concept, assumes data lives in records you can locate and remove. A row in a table. A file in a bucket. An entry in an index. Find it, remove it, done.

Model weights are not records. Training distributes information across billions of parameters in a form nobody can point to. There is no row for a particular person, no file for a particular document. The influence is real and diffuse simultaneously.

So when someone exercises a right to have their data erased, and that data was part of a fine-tuning set, the only honest answer is that removing it requires retraining the model without it. Expensive, slow, and not something anyone wants to do on request.

Which means the decision to train on personal data is, whether or not anyone frames it this way, a decision to accept a permanent obligation. Most teams do not make that choice deliberately. They make it because fine-tuning was the technique someone already knew, and nobody asked the deletion question until the system was live.

The approach that keeps control

There is a straightforward alternative, and it is worth understanding because it changes the ownership picture entirely.

Instead of training a model on your data, you leave the data where it is and have the system look things up when a question arrives. The model never absorbs the information permanently — it reads what it needs, answers, and forgets.

The practical consequence is that deletion works. Remove the document, remove its passages from the index, remove the vectors, and the capability genuinely disappears. Ask the system tomorrow and it knows nothing about that document.

That property — being able to actually delete things — is worth more than it sounds. Regulators increasingly test it rather than accepting assurances, and enterprise buyers ask about it during procurement. A system that can demonstrate clean deletion is easier to sell into than one that cannot.

The other control most people are missing

While we are on the subject of who can see what, there is a widespread design flaw worth knowing about.

In most internal AI systems, the component that searches documents runs with permission to read everything. When someone asks a question, it searches the whole collection, sends the best matches to the model, and only afterwards checks whether that person was allowed to see those documents — hiding the references that fail the check.

The information is already in the answer by then. Hiding the reference removes the citation, not the content.

The result is that people can receive accurate summaries of documents they have no right to open, with nothing unusual appearing in any log, because every step was technically permitted.

The fix is to check permissions before the model reads anything rather than after it writes. It is a change in ordering, not in complexity — but it has to be designed in rather than added later.

What is actually within your control

Not everything here is bleak. Four things you genuinely control:

• Where the data rests. Map every location — storage, index, cache, model endpoint, provider logs — against your obligations. The common error is assuming the model provider's stated region covers the whole chain. It covers the model.
• What the provider keeps. Set retention explicitly in the contract rather than accepting the default.
• What gets trained on. This is the permanent decision. Keep anything that might need deleting out of the training path.
• Who can reach what. Permission checks belong inside the search, evaluated against the person asking.

Four decisions, all made at design time, all cheap then and expensive afterwards.

Frequently Asked Questions

Can data really be recovered from an AI search index?

To a meaningful extent, yes. Inversion research reconstructs substantial portions of short text passages from their numerical representations. Treat the index as a copy of the original.

Why can't data be deleted from a trained model?

Because training spreads information across billions of parameters rather than storing records. There is nothing specific to locate or remove, so deletion means retraining without the data.

Is there a way to use AI and still honour deletion requests?

Yes — use systems that look data up at query time rather than training on it. Removing the source document and its index entries genuinely removes the capability.

Do AI providers keep copies of what we send?

Often, for a limited abuse-monitoring period. Enterprise agreements usually allow this to be reduced or switched off, but you have to ask. Defaults vary.

What is the most common security flaw in these systems?

Checking permissions after the model has generated an answer rather than before it reads. It produces disclosures that leave no trace in security logs.

Does this mean organisations should avoid AI?

No. It means a few decisions — where data rests, what gets trained on, and when permissions are checked — should be made deliberately and early, because they are the ones that are hard to reverse.

---

The full technical treatment is at Enterprise Data Protection in the Age of AI Agents.

We help teams design these systems properly — AI development services and contact us.