A visa assessment is a document somebody acts on. They read it, and then they book flights, pay an application fee, or decide not to travel at all. If a language model invents an eligibility line in that document, the cost is not an awkward chat reply. It is a wasted fee and a cancelled trip, and it is my name on the system that produced it.
That is the difference between a demo and production, and it changes the engineering completely. Below is the pattern I keep coming back to: four gates between the model and the customer. None of them are exotic. They are the same controls you would put around any third-party service you do not trust, which is exactly what a model is.
Gate 1 - The boundary: what the model is allowed to see
Before a single token leaves your system, somebody has to decide what may go. Not as a legal footnote after launch, but as a design decision written down alongside the data model.
- Classify the input fields. Which are public, which are personal, which are sensitive.
- Send the minimum that produces a correct answer. A passport number rarely improves a recommendation; a nationality usually does.
- Read the provider's retention and training terms and treat them as an architecture constraint. Where the data lands, how long it stays and whether it trains a future model are all facts your design has to survive.
- Write the decision per field. When someone asks in year two why a field was excluded, the answer should be in a document, not in somebody's memory.
This is the gate that gets skipped most often, because at prototype stage it costs nothing to send the whole record. It costs a great deal later.
Gate 2 - The contract: a schema, not prose
The model returns structured output, validated against a schema, every time. If validation fails, the request fails. It does not degrade, it does not best-effort parse, and it never hands a half-understood blob to the next function.
This is a real trade-off and worth stating plainly: some requests now error instead of returning something. That is the point. A visible failure the user can retry beats an invented eligibility result they would act on. Choosing to fail loudly is a decision you make once, deliberately, and then defend when someone asks why the success rate is not 100%.
If you cannot describe the shape of a correct answer, you are not ready to put the model in front of a customer. The schema is the specification.
Retry with backoff around the model call, validate, and only then let the result into your domain. In practice this turns a non-deterministic dependency into something that behaves like an ordinary API with a strict response contract.
Gate 3 - The review gate: who signs off, and when
Not everything needs a human. The stakes decide, and the stakes vary inside a single product. A tier that works:
- Informational output - suggestions, summaries, ranked options. Ships straight through. The user can see it is advisory.
- Decision-bearing output - anything the customer will act on financially or legally. Human review before release, or a narrow, heavily constrained generation with a confidence signal.
- Irreversible output - anything that moves money, sends on the customer's behalf, or cannot be retracted. Human in the loop, always.
Where a human reviews, build the queue as a first-class part of the system. A review step that lives in somebody's inbox is not a control; it is a bottleneck that will be bypassed the first week the volume doubles.
Gate 4 - The ledger: audit and cost
Log enough to reconstruct any single answer months later: prompt version, model and model version, a hash of the input, the raw output, the validation result, and who reviewed it if anyone did.
You will be asked "why did it say that in March", and the honest answers are either "here is the record" or "we do not know". Only one of those is survivable in a regulated conversation. Model versions change under you; without the version in your log you cannot even tell whether the behaviour changed or your prompt did.
Cost belongs in the same ledger. Cost per request, monitored, with an alert - not a surprise at the end of the month. AI spend is the one infrastructure line that scales with user enthusiasm rather than user count, and it is the one most teams have no dashboard for.
What this looks like in practice
On a travel and immigration platform I designed and built, three paid products generate customer-facing documents from model output. Every response is schema-validated before it can reach a document. Validation failure raises an error the customer can retry rather than producing a partial assessment. Prompt and model versions are recorded against every generated report, and cost per assessment is a monitored metric rather than a monthly discovery.
The result is unglamorous and that is the intention: the AI part of the system is the part that behaves most predictably, because it is the part wrapped in the most constraints.
The pattern underneath
None of this is really about AI. It is ordinary systems engineering applied to a non-deterministic dependency: validate the response, define the failure path, decide who signs off, keep an audit trail, watch the cost. You already do this for payment providers and third-party APIs.
The teams that struggle with AI in production are the ones treating the model as something new and magical. The teams that ship treat it as an integration - an unusually capable, unusually unreliable one - and reach for the controls they already know.
That reframe is most of the job.