Your AI pilot is useful enough that a department wants to depend on it. Now the questions change. Where will the documents be stored? What happens when the model service stops? Who applies updates, restores an index or checks whether a new model still answers correctly? These are infrastructure and administration decisions, and they need owners before launch.
For a Luxembourg business, AI infrastructure deployment means building the environment around the workload. AI system administration means keeping that environment and its integrations reliable after handover. This checklist follows an illustrative internal knowledge assistant from architecture to ongoing operation; the same questions apply to many customer-support and agent workflows.
Start with the workload and its operating requirements
Write down the task, users, supported languages, data classification and allowed actions. Decide whether a delayed answer is inconvenient or business-critical. Identify the people who can approve changes and the people who must receive an incident notification. A model benchmark cannot make these decisions for you.
Define the whole dependency chain: identity provider, application, model endpoint, document ingestion, embedding service, vector database, agent harness, MCP servers and downstream APIs. Mark which components you operate and which depend on suppliers. Include DNS, certificates, storage and backup services in the diagram.
Our AI infrastructure deployment service starts with these requirements so the resulting environment fits its intended use. Specify acceptable response time, recovery objectives and budget boundaries before comparing hosting options; otherwise an attractive demonstration can become an unsuitable production architecture.
For the illustrative knowledge assistant, assign a document owner before selecting storage. That person decides which sources are authoritative, who may read them and when they should be removed. An administrator can implement those rules, but should not have to infer them from folder names. Record the difference between responsibility for the platform and responsibility for the information the platform makes available.
Choose hosting using the complete data flow
On-premises, European cloud and hybrid designs each introduce different operational responsibilities. Locate the model, document sources, embeddings, logs, backups and support access. An application hosted in Europe may still send prompts to a model elsewhere or export telemetry to another provider.
| Option | Decisions to make | Responsibilities to confirm |
|---|
| On-premises | Hardware, power, capacity and physical access | Patching, replacement equipment, backups and recovery |
| European cloud | Region, service boundaries and supplier access | Cloud permissions, consumption limits and shared operations |
| Hybrid | Which data and requests may cross each boundary | Connectivity, failure behaviour and split incident ownership |
Document retention and deletion rules for each data store, including derived indexes. Check model licences, processor arrangements and access by support staff against the use case. A hosting location is one input to those decisions; it does not establish privacy or regulatory conformity by itself.
Keep a practical exit plan. Record which configurations, documents and application components can move to another environment, and what rebuilding an index would require. Portability is easier to assess before a supplier-specific feature becomes a hidden dependency of every workflow.
Size model serving with representative requests
Capacity depends on the model, runtime, precision, context length, output length and concurrent usage. Test the complete request path with representative documents rather than selecting a GPU from a model name alone. A document assistant processing long contracts has different demands from a short-answer support bot.
Measure the delay before an answer starts, the time to finish, the request queue and behaviour during a burst. Include indexing work and background tasks in the capacity plan. Define what happens when the service is full: a bounded queue, a useful error, a fallback or escalation to a human.
If Kubernetes is appropriate, its GPU scheduling documentation explains the use of device plugins to expose accelerator resources. The orchestration choice should follow the workload and your team's ability to operate it; a smaller deployment may need a simpler platform.
Validate the exact runtime and model configuration you intend to release. Quantisation or a longer context setting can affect quality as well as resource use. Keep capacity measurements alongside evaluation results so a performance change does not silently change the standard users receive.
Separate identities, tools and network access
Give users and services the access they need for a named task. Keep administrative interfaces separate from ordinary inference endpoints. Manage secrets outside prompts and source documents, and assign a responsible owner to credential rotation. A shared administrative token makes both investigation and access review harder.
For the illustrative knowledge assistant, a document lookup and an email-sending tool should have separate permissions. Retrieval must respect the requesting user's access. An agent should not acquire write permission simply because another component of the application has it. Enforce sensitive-action approval in the execution path.
Inventory MCP clients, servers, transports and upstream services. Review the official MCP security guidance for the authorisation flow in use. Treat installation, configuration changes and new tool access as managed changes, with a record of who approved them.
Restrict outbound destinations where the workload permits it. A model service, document-ingestion worker and administration interface may need different network access. Test those boundaries and verify them again after an infrastructure migration or connector update, when inherited permissions can change.
Monitor service health and answer quality
A running process is not the same as a working AI service. Check whether requests reach the model, authorised documents are retrieved and the response is returned. Separate availability checks from representative quality evaluations; the first can pass while the second reveals broken retrieval or a changed model response.
The vLLM production metrics documentation includes measurements such as queue time, end-to-end latency and time to first token. Use metrics supported by your serving stack to explain where delays occur rather than relying only on average response time.
Choose alerts with a concrete response. Name who receives them, what they inspect and when they escalate. Track request failures, resource saturation, unexpected tool activity and abnormal consumption. Keep logs useful for investigation while controlling their access and retention; prompts and retrieved content can contain sensitive material.
Prometheus instrumentation guidance recommends designing metrics around the service and avoiding uncontrolled label cardinality. In practice, do not use arbitrary user prompts or document identifiers as metric labels. Review telemetry as a data flow with its own exposure and operating cost.
Prove recovery before calling the deployment complete
List what must be preserved and what can be rebuilt: source documents, permissions, application configuration, prompts, evaluation datasets, indexes and deployment manifests. A vector index backup alone may not preserve the source ownership or access rules needed to use it safely.
Restore into an isolated environment and run representative checks. Confirm that documents remain restricted to the appropriate users and that tool permissions have not broadened. Measure the actual recovery path against your agreed objectives. Keep the procedure usable by someone other than the person who built the pilot.
Plan for a failed dependency as well as a failed host. Decide whether the application can run with retrieval unavailable, whether model requests should stop during a connector incident and how users are informed. If switching models is an emergency option, validate that fallback beforehand instead of improvising it during an outage.
Make updates and rollback an explicit operating process
Maintain an inventory of model versions, runtimes, drivers, container images, harness dependencies and MCP services. Pin the components needed to reproduce a release. Review supplier changes, licences and security notices through a process that fits the system's support arrangements.
Stage changes before production. Re-run quality, access-boundary and tool-action tests after a model, prompt, retrieval or connector update. The AI pentesting and testing guide explains what that evaluation should cover. A package update can succeed while the resulting application behaviour becomes unacceptable.
Define rollback for both software and state. Restoring an old container may not reverse a changed index schema or an action already taken in a downstream system. Preserve compatible configuration and test the recovery steps for the particular change. Record the release decision and any remaining restrictions.
Rehearse the launch with a limited, authorised user group and realistic tasks. Check the support route, a deliberately failed dependency and the return to the previous release. Record what the team learned and what must be corrected before wider access. This rehearsal should demonstrate that people can operate the service as well as that the model can answer questions.
Agree the administration scope and handover evidence
Ongoing AI system administration should name the systems covered, support hours, escalation route, update responsibilities and reporting cadence. Separate infrastructure maintenance from decisions about acceptable model behaviour. Both need owners, even when one team supplies the service.
Ask for architecture and data-flow diagrams, access inventories, deployment records, dashboards, restore procedures, evaluation results and runbooks. Identify who owns the model selection, document sources and tool approvals. Include expected consumption and a method for reviewing costs when usage changes.
Our guide to AI technical debt explains why missing ownership becomes expensive. This checklist turns that concern into handover decisions. Where regulatory requirements apply, retain the relevant records without describing infrastructure administration as a substitute for the wider conformity process.
To prepare a deployment or review an existing AI environment, talk to ObsidianCorps with your use case, data constraints and current architecture. We can scope the infrastructure work and the administration responsibilities needed to keep the system usable after launch.
