Before you build sovereign RAG: a readiness audit

The instinct, when you set out to build a RAG system on sensitive data, is to start with the visible decisions: which model, how many GPUs, which cloud region. Those are the parts you buy. They improve on someone else's roadmap, and they are almost never why the project fails.

Projects fail on the two parts you actually own. The first is the knowledge base: a corpus that looks like documentation to a human can be semantically flat to an embedding model, and no amount of model quality recovers a passage that was never retrievable in the first place. The second is the data path: the route a single query and its retrieved context take through your system, and whether the data is even allowed to travel it. Both are decided upstream of the model, and both are invisible in a demo.

So before the build, run a readiness audit. Not a compliance questionnaire — an engineering pre-flight that answers two questions with evidence: can this corpus become good retrievable chunks, and is the data lawfully allowed down the path I'm about to send it. The good news is that both are cheap to check. The bad news is that the answer is usually 'not yet', and the most expensive surprises hide where nobody looks — in the test set, not the model.

The audit has two axes, and conflating them is the first mistake. Retrievability-readiness asks whether your corpus can become good chunks — and whether you can measure that honestly. Sovereignty-readiness asks whose jurisdiction each leg of the data path sits in. They are orthogonal: a corpus can be beautifully retrievable and completely exposed, or locked down inside your jurisdiction and impossible to retrieve from.

Both gate the project. Strong retrieval on an exposed path is a compliance incident waiting to happen; perfect sovereignty over a corpus nobody can query is an expensive archive. You have to pass both, and you have to assess them separately, because the work to fix one does nothing for the other.

Start with an inventory, and separate two things people merge: the logical source — a body of knowledge, a crawl root — from the physical store it happens to live in. One source can span several stores; one store can hold several sources. Get that mapping explicit, and derive a type for each source rather than hand-maintaining it: a reference corpus, a procedural one, and a Q&A export need different handling downstream, and the type is what routes them.

Then the part that decides everything: structure. A chunk is the unit of retrieval, so its boundaries cap retrieval quality, and structure is what makes boundaries meaningful. The catch is that structure is often not where a naive parser looks — it lives in UI widgets, in bold-as-heading conventions, in layout rather than in real heading tags. I have seen a corpus go from single-digit to roughly two-thirds heading density just by recovering that implicit structure before the splitter ran — same content, the signal was simply being thrown away. Chunking then has to be content-type aware — a procedure, a reference table, and a definition page want different boundaries.

One concrete trap belongs in the audit: check whether your platform re-chunks your chunks. A managed knowledge base with a fixed-size chunking default will happily cut straight through the careful boundaries you just built, and the setting is easy to miss because its default looks reasonable. And where a source is structurally hopeless — a tiny stub, a wall of jargon-only table cells, an index page of near-identical entries — mark it and decide deliberately, rather than ingesting it blind and letting it drag your numbers down.

Here is the finding that reorders the whole audit: most retrieval 'failures' are not retrieval failures. In one diagnosis of a system that looked like it was missing badly, sixteen of eighteen misses turned out to be wrong labels in the test set — the right page was being retrieved, and the ground truth simply insisted a different page was correct. Before you trust a single recall number, audit the thing you are measuring against.

The cheapest first move costs an afternoon and no inference at all: compute your practical ceiling — the share of your expected-answer pages that even exist in the corpus you indexed. If a tenth of your labelled answers point at pages that aren't there, your recall is capped at ninety percent before retrieval does anything, and chasing the last points is chasing noise. Next, separate your query sets by purpose — a frozen benchmark for the verdict, an append-only regression set, a generated diagnostic set — and never let them mix; the full machinery behind that split is told in "Retrieval quality is gated, not asserted". One pile serving three purposes manufactures fake miss rates and hides real ones.

And when a metric moves, break it down by subset before you react. An aggregate number is an average over strata that may be shifting in opposite directions; optimise against the wrong aggregate and you will cheerfully roll back changes that helped the subset you actually care about. None of this is exotic — retrieval is one of the few parts of an LLM system with honest, classical metrics like Recall@k and MRR — but the metrics are only as honest as the ground truth underneath them. Audit that first.

Sovereignty is a property of the data path, not of a data center. Choosing an EU region is the reassuring line in the architecture and the one that protects the least, because what binds is the jurisdiction of the operator, not the location of the server. So walk the path one real query takes — query, embedding, retrieval, generation — and mark every leg that a foreign-jurisdiction operator controls. Each mark is an exposure, whatever the region dropdown says.

What decides how strict you have to be is the data's protection requirement — what German public-sector teams call the Schutzbedarf. And the crucial subtlety is that the classification judges the path and the vendor, not the model's pedigree. Open weights you download and run on your own EU infrastructure send nothing home; a managed API from a foreign provider sees the protected context on every call. So 'is this model American' is the wrong question. 'Who operates it, and what can they be compelled to hand over' is the right one.

Two practical notes that catch people. Regions are asymmetric in what they offer — a reranker or a model you need may exist in only one region — so your compliance constraint and raw availability jointly fix where you can run, and the two can pull against each other. And data paths leak by accident: a deploy can silently reach into a foreign region because a CDN or a managed service quietly requires it. Make it a rule that any infrastructure action prints a plain-language list of the regions and accounts it will touch, before it runs. The exposure you don't see is the one that hurts.

Put the two axes together and the audit becomes a scorecard you can run in hours, not weeks. You don't need every line green to start — you need to know which lines are red and to have decided, on purpose, that you can live with them. Here is the shape of it:

The audit is not free, but it is cheap relative to what it saves. For public content at low stakes, a quick gut-check is enough and a full audit is over-engineering. For personal, public-sector, or health data — where the protection requirement is real and the regulator already expects you to control the flow — it is not optional, and an afternoon spent here is the cheapest insurance the project will buy.

The systems that skip it don't fail loudly at launch. They fail quietly, upstream of the model, in exactly the place the next person to debug them will look last.