14.1 — Deployment in Customer Environments

1. What is it?

This chapter covers the practical reality of deploying an AI system into an environment you don't fully control — the customer's own cloud account, their on-premises data center, or a fully air-gapped network with no internet connectivity at all — synthesizing Part 7's production engineering and Part 9's security discipline specifically for the constraints this deployment context imposes.

2. Why does it exist?

Every architecture and code example throughout this handbook implicitly assumed you control the deployment environment. Real FDE work very often doesn't have this luxury: an enterprise customer's IT department has its own network policies, its own approved technology stack, its own change-management process, and sometimes explicit requirements (regulatory, security, or simply organizational preference) that the system run entirely within their infrastructure, not yours. This chapter exists because "the system worked perfectly in our environment" is a genuinely different, much weaker claim than "the system works in the customer's actual environment," and the gap between these two is a common, serious source of late-stage engagement friction if not planned for from the start (Part 13.1's lifecycle point about deployment surprises).

3. What problem does it solve?

It solves "how do I design, package, and deploy an AI system so it reliably runs correctly in an environment I don't fully control, whose specific constraints I may not fully know until deployment itself" — requiring both technical flexibility (Part 7.1's Docker portability) and a specific discovery/planning discipline (extending Part 12's process to deployment-environment specifics).

4. How does it work internally?

The deployment-environment spectrum

Fully cloud-hosted by yousimplest, full control — standard Part 7 deployment applies directly

Customer's cloud accounttheir cloud, your deploy — same Docker/K8s patterns, but within THEIR VPC/account, subject to their approval/network policies (Part 7.3)

VPC-hosted / private-endpoint modelcloud-connected, public LLM API calls blocked — otherwise-normal deployment, but the model is reached over a private network path (VPC peering, PrivateLink, Private Endpoint) to a provider-hosted, tenant-isolated model

On-prem, connectedtheir data center, internet-connected — requires the customer's own infrastructure for anything the AI system needs (Part 7.3)

Fully air-gappedno internet connectivity at all — requires a completely self-contained deployment package; even the LLM must be self-hosted (Part 2.2)

Each step right removes something you'd otherwise control or take for granted.

Each step right on this spectrum removes something you'd otherwise control or take for granted — cloud provider choice, network configuration, even (in the air-gapped case) the ability to call any hosted LLM API at all, since that API call is itself outbound internet traffic an air-gapped network structurally cannot make.

The VPC-hosted-model case — the far more common "we can't call your API" scenario

The spectrum's middle case is easy to conflate with air-gapping, but it is a genuinely different, much more common requirement: the customer is otherwise fully cloud-connected — they have internet access, they run normal SaaS tooling, their network is not air-gapped in any sense — but their security team's policy is specifically "no application traffic leaves our cloud account/VPC to a public, multi-tenant API endpoint," full stop, regardless of how that endpoint is secured. This is not a data-sensitivity objection to the model provider itself; it's an objection to the network path the request would otherwise take.

The fix is a private-network path to a still-hosted model, not self-hosting the model:

• AWS Bedrock or Azure OpenAI Service (or equivalent) puts the same underlying models behind an endpoint reachable via AWS PrivateLink or Azure Private Endpoint — traffic from the customer's VPC to the model never traverses the public internet, and the provider's standard enterprise contractual terms (no training on submitted data, data-residency commitments, Part 10.5) satisfy the data-handling half of the customer's concern.
• This preserves nearly everything about a normal hosted-API architecture (Part 3.11) — no self-hosting infrastructure, no GPU provisioning, standard per-token pricing — while satisfying a network-policy requirement that would otherwise (if misdiagnosed as an air-gap requirement) trigger a far more expensive, unnecessary redesign around a self-hosted open-weights model.
• The practical cost is usually availability of specific models (a VPC-hosted offering may lag the provider's newest public-API model release by weeks or months) and sometimes a modest latency/throughput difference versus the public endpoint — real trade-offs, but far smaller than the air-gapped case's full self-hosting burden.

Diagnosing which case you're actually in — during discovery (Part 12.1), ask directly: "when you say you can't call an external API, do you mean no traffic can leave your cloud account to any external endpoint under any circumstances (a true network-isolation/air-gap requirement), or specifically no traffic to a public, shared endpoint (a VPC-hosted/private-endpoint requirement)?" These two answers lead to architecturally and financially very different engagements, and treating the more common, cheaper VPC-hosted case as if it were the rarer, much harder air-gapped case is a costly, avoidable overcorrection.

Air-gapped deployment — the hardest, most distinctive case

An air-gapped environment has no internet connectivity, full stop — meaning every dependency the system needs must be physically transferred in (via an approved, audited transfer process, often involving offline media or a tightly-controlled one-way data diode) and every external service call (Part 3.11's hosted LLM APIs) is structurally impossible. This means:

• The LLM itself must be self-hosted (an open-weights model running on the customer's own hardware, Part 2.2's fine-tuning/self-hosting discussion becomes directly load-bearing here, not just a theoretical alternative) — a materially different, harder engineering and operational problem than calling a hosted API (Part 3.11).
• Every dependency (Python packages, model weights, Docker base images) must be pre-resolved and transferred in, not pip install-ed or docker pull-ed live — Part 7.1's Docker image portability is exactly what makes this tractable: build and fully test the image in a connected environment, then transfer the complete, already-built image in.
• Monitoring/observability tooling (Part 6.1/8.4) that depends on a hosted service (LangSmith's cloud offering, for instance) needs a self-hosted or fully offline alternative — verify what deployment options a given tool actually supports before committing to it for an air-gapped engagement.

The post-deployment operating model — who runs this after go-live

Getting the system deployed is not the end of the deployment conversation — a genuinely complete deployment plan also answers, explicitly and in writing, who does what once the system is live in the customer's environment. Leaving any of the following as an unstated assumption is a common, avoidable source of post-launch conflict:

• Change-advisory-board (CAB) process: many enterprise customers require any production change — a new model version, a config change, a bug-fix deploy — to pass through their own CAB or equivalent change-review process before it ships, even for a vendor-built system. This can add real, non-negotiable lead time to what would otherwise be a same-day fix (Part 13.1's timeline planning needs to account for this explicitly), and the FDE team needs to know, before go-live, exactly what qualifies as a "change" under the customer's process and how far in advance a request must be submitted.
• Pre-go-live security sign-off / penetration-test gates: many customers require a formal security review — sometimes including a third-party penetration test — of the system before it's allowed into production, separate from and in addition to the FDE team's own security work (Part 9). This gate should be scheduled into the engagement timeline as its own milestone with its own lead time, not discovered as a surprise blocker in the final week before a planned launch date.
• Who owns on-call and incident response after go-live: this is a specific, negotiated answer, not an assumption either party should make silently. Does the vendor/FDE team hold the pager for production incidents, does the customer's own ops team take first-line response with vendor escalation for anything AI-specific, or is it a shared model with an explicit handoff boundary (e.g., customer owns infrastructure-level alerts, vendor owns model/application-level alerts)? Whichever model is chosen, it needs to be written down, agreed by both sides, and reflected in the actual on-call rotation and escalation paths (Part 8's incident-response practices) before go-live — not sorted out reactively during the first production incident.
• Credential and secret handoff to the customer's own ops team: at some point, the customer's own team typically needs to take operational ownership of the credentials and secrets the system depends on (API keys, database credentials, service accounts) — this handoff should follow the customer's own secret-management practices (Part 9.7) and be a planned, documented step (who rotates what, who holds a break-glass copy, when the FDE team's own access is revoked) rather than credentials quietly remaining in the FDE team's hands indefinitely after the engagement's active phase ends, which is both an operational and a security liability for both sides.

Each of these four questions should be raised and answered during deployment planning, not discovered as a gap during the first post-launch incident or the first requested change — exactly the same "surface it early, not late" discipline this chapter has already argued for regarding the deployment-environment constraint itself.

Network policy negotiation — the recurring, practical friction point

Even in a merely on-prem-but-internet-connected environment (not fully air-gapped), a customer's IT department typically has firewall rules restricting outbound traffic to an explicit allowlist — meaning your system's specific outbound needs (an LLM provider's API endpoint, a specific SaaS integration, Part 10.3) must be identified explicitly and requested through the customer's own change-management process, which can take real, sometimes substantial time and should be planned for in the engagement timeline (Part 13.1), not discovered as a last-minute blocker right before a planned go-live.

5. Simple mental model

Deploying into a fully cloud-hosted environment you control is like cooking in your own kitchen, with every tool and ingredient exactly where you left it. Deploying into a customer's connected on-prem environment is like cooking in someone else's kitchen — you can bring your own ingredients, but you need their permission to use certain appliances, and you need to ask where things are instead of assuming. Deploying into a fully air-gapped environment is like being asked to cook a full meal on a remote island with no store nearby at all — every single ingredient and tool must be packed and brought in advance, because there's no possibility of running out and quickly buying more once you're there.

6. Real-world example

An engagement with a defense-adjacent logistics customer required fully air-gapped deployment — no exceptions, given the sensitivity of the data involved. This meant the entire architecture built around a hosted LLM API (the team's default assumption from prior engagements) had to be redesigned around a self-hosted open-weights model running on customer-provided GPU hardware, with every Python dependency, the model weights themselves, and the full Docker image pre-built and tested in a connected staging environment, then transferred via the customer's approved offline media process — a materially different engineering effort than a typical hosted-API deployment, discovered as a hard requirement only during discovery (Part 12.1's constraint-surfacing questions) specifically because the FDE team asked directly about deployment environment early, rather than assuming a standard cloud deployment and discovering the air-gap requirement only much later.

7. Architecture diagram

CONNECTED ENVIRONMENT · build and test here

Full CI/CD pipelinePart 7.2

Docker image buildPart 7.1, fully self-contained

Model weights, dependenciespre-resolved

AIR-GAPPED ENVIRONMENTno internet, no live pulls — pre-built image + pre-transferred weights run entirely self-contained: self-hosted LLM, vector store (Part 1.5), observability (or none)

8. Production considerations

• Discover the actual deployment-environment constraint explicitly during initial discovery (Part 12.1) — "what environment will this run in" should be one of the very first architecture-shaping questions, not discovered midway through a build already assuming a different environment (section 6's exact lesson).
• Explicitly distinguish "no public-endpoint API calls" from "no external connectivity at all" during that same discovery conversation — conflating the VPC-hosted-model case with true air-gapping (section 4's dedicated subsection) leads to either a wildly over-engineered self-hosting effort where a private-endpoint hosted model would have sufficed, or the reverse, discovering the true air-gap requirement only after a private-endpoint architecture has already been built and won't work.
• Design for portability from the start when there's any chance of an on-prem/air-gapped requirement — Docker's portability (Part 7.1) is what makes a "build once, deploy anywhere" strategy tractable; an architecture tightly coupled to a specific cloud provider's proprietary services is much harder to port later.
• Plan for network-policy negotiation time explicitly in the engagement timeline — this is a real, often underestimated source of schedule delay (section 4's recurring friction point).
• Verify every dependency's offline/self-hosted deployment option before committing to it for an air-gapped engagement — a tool with no offline mode is simply not usable in that environment, regardless of how good it is otherwise.
• Negotiate and document the post-deployment operating model explicitly before go-live (section 4's dedicated subsection) — CAB process, security sign-off timing, on-call ownership, and credential handoff are each a specific answer that should be agreed in writing, not assumed by either side.

9. Common mistakes

• Assuming a standard cloud-hosted deployment model by default and discovering an on-prem or air-gapped requirement only late in the engagement, requiring a costly architectural redesign (section 6's exact, avoidable scenario).
• Treating a "we can't call external APIs" statement as an air-gap requirement without clarifying whether a VPC-hosted/private-endpoint model would actually satisfy the customer's real concern — a much cheaper, far more common fit (section 4).
• Building tightly coupled to a specific hosted LLM API with no self-hosting fallback consideration, then discovering an air-gapped requirement makes that entire approach unusable.
• Underestimating the time required for a customer's network-policy change-management process, causing a preventable go-live delay.
• Not verifying that every dependency (including observability/monitoring tooling) has a viable offline or self-hosted deployment path before committing to it for a constrained environment.
• Reaching go-live without an explicit, agreed answer for who owns on-call/incident response, leaving this to be improvised during the first production incident.

10. Security considerations

Deployment-environment constraints are frequently driven by security/compliance requirements (Part 10.5) — an air-gap requirement, for instance, is usually a direct consequence of the data sensitivity involved, meaning the deployment-environment discussion and the compliance discussion (Part 10.5) should happen together, not as separate, disconnected conversations.

11. Performance considerations

A self-hosted model (the air-gapped case) has genuinely different performance characteristics than a hosted API call (Part 2.2/3.11) — dependent on the customer's provided hardware, requiring careful capacity planning and realistic performance-expectation setting (Part 12.3's communication discipline) given the customer's actual available infrastructure, which may not match a well-resourced cloud provider's typical performance profile.

12. Cost considerations

Self-hosted deployment (air-gapped or otherwise) shifts cost from per-token API pricing (Part 7.10) to infrastructure/hardware cost (GPU provisioning, ongoing maintenance) — a genuinely different cost model requiring its own explicit analysis (Part 15) rather than simply applying Part 7.10's hosted-API cost framework unchanged.

13. When to use it

Any engagement with genuine on-prem, air-gapped, or heavily network-restricted requirements — verified explicitly during discovery (Part 12.1), never assumed either way.

14. When NOT to over-apply it

Designing every system for maximal portability "just in case" a future air-gapped requirement emerges is disproportionate effort for engagements where this is genuinely never going to be a requirement — proportional investment (Part 3.12) matches design flexibility to actual, discovered likelihood, not a defensive maximum applied universally.

15. Alternatives and trade-offs

Deployment model	Good for	Weak point
Fully cloud-hosted (your infrastructure)	Simplest, fastest to deploy/iterate	Unsuitable for customers requiring their own infrastructure control
Customer's cloud account	Customer data-residency/control, still cloud-native tooling	Subject to customer's own cloud policies/approval processes
VPC-hosted / private-endpoint model	Satisfies "no public-API-endpoint traffic" policies while keeping a hosted model's low operational burden and normal per-token pricing	Model selection sometimes lags the provider's newest public-API release; still a negotiated network-approval process
On-prem, connected	Meets stricter infrastructure-control requirements	More deployment/ops complexity; network-policy negotiation overhead
Fully air-gapped	Meets the strictest security/regulatory requirements	Materially harder engineering (self-hosted LLM), no live external services at all

16. Practical Python/code example

A deployment-environment-aware configuration loader, making the environment's specific constraints an explicit, code-level concern rather than an implicit assumption:

from enum import Enum
import os


class DeploymentEnvironment(str, Enum):
    CLOUD_HOSTED = "cloud_hosted"
    CUSTOMER_CLOUD = "customer_cloud"
    VPC_HOSTED_MODEL = "vpc_hosted_model"
    ON_PREM_CONNECTED = "on_prem_connected"
    AIR_GAPPED = "air_gapped"


def build_model_client(deployment_env: DeploymentEnvironment):
    """
    Builds the appropriate model client for the deployment environment,
    making the air-gapped case's self-hosting requirement, and the
    VPC-hosted case's private-endpoint requirement, explicit enforced
    code paths rather than assumptions that could silently fail.

    Args:
        deployment_env (DeploymentEnvironment): The actual deployment context.

    Returns:
        A model client appropriate for this environment.

    Raises:
        RuntimeError: If a public-endpoint API client is requested in an
            air-gapped or VPC-hosted environment, where it would structurally
            fail (air-gapped) or violate network policy (VPC-hosted) at
            runtime anyway — fail loudly at configuration time instead.
    """
    if deployment_env == DeploymentEnvironment.AIR_GAPPED:
        return build_self_hosted_model_client()
    if deployment_env == DeploymentEnvironment.VPC_HOSTED_MODEL:
        return build_private_endpoint_model_client(
            private_endpoint_url=os.environ["MODEL_PRIVATE_ENDPOINT_URL"]
        )
    return build_hosted_api_client(api_key=os.environ["LLM_API_KEY"])

17. Production-quality example

A pre-transfer dependency manifest generator, directly implementing section 8's "verify every dependency's offline path" recommendation as a concrete, auditable artifact for the customer's transfer-approval process:

import subprocess
import logging

logger = logging.getLogger("air_gapped_packaging")


def generate_transfer_manifest(docker_image_tag: str, model_weights_path: str) -> dict:
    """
    Generates a complete manifest of everything needed for an air-gapped
    deployment transfer, for the customer's audited transfer-approval process.

    Args:
        docker_image_tag (str): The fully-built, self-contained Docker image.
        model_weights_path (str): Path to the pre-downloaded model weights.

    Returns:
        dict: A complete manifest of artifacts requiring transfer, with
            checksums for the customer's own integrity verification.
    """
    image_digest = subprocess.check_output(
        ["docker", "inspect", "--format={{.Id}}", docker_image_tag]
    ).decode().strip()

    import hashlib
    with open(model_weights_path, "rb") as f:
        weights_checksum = hashlib.sha256(f.read()).hexdigest()

    manifest = {
        "docker_image": {"tag": docker_image_tag, "digest": image_digest},
        "model_weights": {"path": model_weights_path, "sha256": weights_checksum},
        "generated_at": datetime.now(timezone.utc).isoformat(),
    }
    logger.info("transfer manifest generated: %s", manifest)
    return manifest

This kind of explicit, checksummed manifest is exactly what a customer's security team needs to audit and approve an air-gapped transfer — a concrete, professional artifact rather than an informal handoff.

18. Short exercise

A customer initially describes their deployment as "on our own servers" without further detail. Using this chapter's spectrum (section 4), write three specific follow-up questions you'd ask during discovery to determine exactly where on the spectrum this actually falls, and explain why the answer materially changes your architecture.

19. Interview questions

• Explain the deployment-environment spectrum from fully cloud-hosted to fully air-gapped, and what changes architecturally at each step.
• Why does an air-gapped requirement make a hosted LLM API architecturally impossible, and what does this require instead?
• Why should deployment-environment constraints be discovered explicitly and early, rather than assumed?
• A customer says "we can't call any external LLM API." What clarifying question distinguishes a VPC-hosted/private-endpoint fix from a true air-gap requirement, and why does getting this wrong get expensive?
• What should be explicitly negotiated and documented before go-live regarding who owns on-call and incident response, and why is this not something either side should assume?

20. FDE/customer scenario

CUSTOMER (discovered mid-engagement): "By the way, this needs to run in our classified network with no internet access at all."

This is precisely the costly, late-discovery scenario section 6 illustrated — the FDE-correct response, going forward, is to have asked this question explicitly during initial discovery (Part 12.1), and upon discovering it now, honestly communicating (Part 12.3) the real architectural implications (a self-hosted model replacing the assumed hosted API, Part 2.2) and the corresponding timeline/scope impact, rather than either quietly absorbing a large unplanned redesign or downplaying the real engineering effort this requirement introduces.

A second, more common variant — CUSTOMER: "Our cloud security policy says we cannot make outbound calls to public API endpoints from our VPC." The FDE-correct response does not jump to "we'll need to self-host the model" — it asks section 4's clarifying question first, and in the far more common case where the answer is "specifically no public endpoints, private connectivity is fine," proposes a VPC-hosted/private-endpoint model (AWS Bedrock, Azure OpenAI via PrivateLink/Private Endpoint) instead: a materially smaller architectural change, a normal per-token cost model, and a much shorter timeline than the self-hosting redesign the first scenario required — precisely because the actual constraint here was the network path, not the absence of internet connectivity altogether.

---

Key takeaways

• Deployment environment exists on a spectrum from fully cloud-hosted to fully air-gapped, and this should be discovered explicitly and early — not assumed or discovered late in an engagement.
• "We can't call external APIs" usually means a VPC-hosted/private-endpoint model (Bedrock, Azure OpenAI via PrivateLink/Private Endpoint) is needed, not a true air-gap — conflating the two leads to a wildly disproportionate self-hosting effort in the far more common case.
• Air-gapped deployment requires a fundamentally different architecture (self-hosted models, pre-resolved dependencies, offline-capable tooling), not just a packaging change to an otherwise-unchanged system.
• Network-policy negotiation with a customer's IT department is a real, often underestimated source of engagement timeline risk that should be planned for explicitly.
• The post-deployment operating model — CAB process, security sign-off timing, on-call ownership, credential handoff — is a set of specific, negotiated answers that must be agreed and documented before go-live, not assumed by either side.

Things you should be able to explain

• The deployment-environment spectrum and what architecturally changes at each step, especially the VPC-hosted and air-gapped cases.
• Why a hosted LLM API architecture is structurally incompatible with an air-gapped requirement, and why it's usually still viable (via a private endpoint) for a "no public API" requirement.
• Why on-call/incident-response ownership after go-live must be an explicit, negotiated answer rather than an assumption.

Things you should be able to build

• A deployment-environment-aware configuration system (including the VPC-hosted-model code path) and a checksummed transfer manifest for air-gapped deployment approval.

Common mistakes

• Assuming a standard cloud deployment and discovering an on-prem/air-gapped requirement late.
• Treating a "no public API calls" policy as an air-gap requirement instead of clarifying whether a VPC-hosted/private-endpoint model would satisfy it.
• Tightly coupling architecture to a hosted API with no self-hosting fallback path considered.
• Underestimating network-policy negotiation time in the engagement timeline.
• Reaching go-live without an explicit, documented answer for CAB process, security sign-off timing, on-call ownership, and credential handoff.

Recommended next chapter

Part 14 complete. Continue to handbook/15-business-impact/01-roi-and-business-metrics.md.