7.1 — Docker

1. What is it?

Docker packages an application together with its exact runtime environment (OS libraries, language runtime, dependencies) into a single, portable image, which runs as an isolated container — a lightweight, process-level isolation (using Linux namespaces and cgroups, Part 1.8) rather than a full virtual machine. For AI systems, Docker is how you package a FastAPI backend, its Python dependencies, and any local model/tooling into something that runs identically on your laptop, in CI, and in a customer's production environment.

2. Why does it exist?

"It works on my machine" is a genuine, historically expensive problem: different OS versions, different installed library versions, different environment configurations produce subtly different behavior for supposedly the same code. Docker exists to eliminate this entire class of problem by making the deployed artifact include its entire runtime environment, not just the application code — the same image that passed tests in CI is, byte-for-byte, the same image that runs in production, with no "well, it was configured slightly differently there" gap remaining.

3. What problem does it solve?

For an AI FDE specifically, it solves "how do I hand a customer's IT team something they can run in their own environment (on-prem, their cloud account, an air-gapped network) with confidence it will behave exactly as tested" — a recurring, concrete need given Part 12/14's emphasis on deploying into environments you don't fully control and can't assume matches your development setup.

4. How does it work internally?

Images, layers, and the build cache

A Docker image is built from a Dockerfile — a sequence of instructions, each producing a layer stacked on top of the previous one:

FROM python:3.12-slim          # base layer
WORKDIR /app
RUN pip install --no-cache-dir uv      # layer: the uv tool itself
COPY uv.lock pyproject.toml ./         # layer: just the dependency manifest + lockfile
RUN uv sync --frozen --no-dev          # layer: installed packages, exactly as locked
COPY . .                       # layer: application code
CMD ["uvicorn", "main:app", "--host", "0.0.0.0"]

Each layer is cached and content-addressed (Part 1.7's Git object model is a close conceptual cousin) — if a layer's inputs haven't changed, Docker reuses the cached layer instead of rebuilding it. This is precisely why instruction order matters for build speed: putting COPY uv.lock pyproject.toml ./ and uv sync before COPY . . means changing application code (which happens far more often than changing dependencies) doesn't invalidate the expensive dependency-installation layer's cache — a very common, easy, high-leverage optimization that's frequently missed by copying everything in one step first. --frozen is what makes this safe in a Docker build specifically: it fails the build rather than silently re-resolving a different dependency graph than the one tested in CI (Part 1.1 covers uv and its lockfile in depth); --no-dev keeps test-only tooling out of the image entirely.

Containers — isolated processes, not virtual machines

A container is a running instance of an image, using Linux namespaces (Part 1.8) to give it an isolated view of the filesystem, process tree, and network, and cgroups to enforce resource limits (CPU, memory) — critically, a container shares the host machine's kernel, unlike a virtual machine which runs its own full guest OS. This is why containers start in milliseconds to a few seconds (no OS boot required) and have much lower resource overhead than VMs, while still providing meaningful isolation for most application-packaging purposes.

Multi-stage builds — keeping production images lean

FROM python:3.12 AS builder
WORKDIR /app
RUN pip install --no-cache-dir uv
COPY uv.lock pyproject.toml ./
RUN uv sync --frozen --no-dev --no-install-project

FROM python:3.12-slim
COPY --from=builder /app/.venv /app/.venv
COPY . /app
WORKDIR /app
ENV PATH="/app/.venv/bin:$PATH"
CMD ["uvicorn", "main:app", "--host", "0.0.0.0"]

A multi-stage build uses one stage (with full build tooling, potentially larger) to compile/install dependencies, then copies only the necessary output into a final, minimal stage — the final production image doesn't carry build-time-only tooling and intermediate artifacts, directly reducing image size (faster pulls/deploys) and attack surface (Part 9's security considerations, fewer installed tools an attacker could leverage if they gained access to a running container).

Baking model weights into the image vs. mounting/downloading them at runtime

For a service that packages a locally-hosted model (an embedding model, a reranker, Part 3.6, or a self-hosted LLM) rather than only calling a remote provider API, a genuinely consequential packaging decision arises that a plain application-code Dockerfile doesn't face: where do the multi-gigabyte model weight files actually live?

• Baking weights into the image (a COPY or RUN-download instruction in the Dockerfile itself, so the weights become one of the image's layers) produces a large image — multi-GB to tens-of-GB, depending on the model — with correspondingly slower registry pushes/pulls and slower cold-starts when a new pod schedules onto a node without the image already cached (Part 7.4's pod-startup-time performance consideration). The decisive advantage: the resulting image is genuinely self-contained — it carries everything it needs to run, with no network access or external volume required at container start.
• Mounting or downloading weights at runtime (a startup script that pulls weights from object storage or a model registry into a mounted volume before the application starts serving) keeps the image itself small and fast to build/push/pull, and lets weight updates happen independently of an image rebuild — but it makes the container's ability to actually start correctly depend on network reachability (or an already-populated volume) at startup time, a dependency the baked-in approach doesn't have at all.

This is precisely a repeat of section 6's air-gapped deployment scenario, made concrete for model weights specifically: an environment with no direct internet access for the deployment process itself is exactly the case where "no network or external volume access required at container start" stops being a nice-to-have and becomes the deciding constraint — baking the weights into the image, resolved once at build time in an environment that does have internet/registry access, is the only one of the two options that a genuinely air-gapped target can run at all. Conversely, for a normal (non-air-gapped) production environment serving several different model variants or that updates weights frequently, mounting/downloading at runtime avoids rebuilding and redistributing a multi-GB image on every weight update, trading a runtime network/volume dependency for meaningfully faster, cheaper iteration.

5. Simple mental model

A Docker image is a fully-stocked, sealed shipping container for your application — everything it needs to run (the specific OS libraries, the exact Python version, every dependency) is packed inside, sealed at build time, so it doesn't matter whether the ship (the host machine) is docked in one port or another (your laptop, CI, a customer's cloud) — the container's contents and behavior are identical everywhere it's unloaded and opened (run).

6. Real-world example

An AI FDE builds a document-extraction service (Part 3.5's RAG pipeline plus Part 1.3's FastAPI layer) and needs to deploy it into a customer's air-gapped, on-prem environment with no direct internet access for the deployment process itself. Docker's self-contained image is exactly what makes this practical: the image (including all Python dependencies, pre-downloaded, baked into a layer during the build) is exported, transferred via the customer's approved internal transfer process, and loaded/run directly on their infrastructure — no dependency-resolution or package-download step needs to happen in their restricted network at deployment time, since everything was already resolved and packaged at build time in an environment that did have internet access.

7. Architecture diagram

Docker Image · built once

Base OS layer

Dependencies layercached if unchanged

Application code layerchanges most often

Laptopdev

CI runnertest

Customer's on-prem serverproduction

8. Production considerations

• Use multi-stage builds and minimal base images (-slim/-alpine variants, though verify compatibility with your specific dependencies — some Python packages with C extensions have known friction with alpine's different C library) to keep production images lean and reduce attack surface (Part 9).
• Run as a non-root user inside the container (Part 1.8's exact recommendation, now expressed in the Dockerfile) — a compromised container process running as root has meaningfully more blast radius.
• Never bake secrets (API keys, credentials) into an image layer — even if a later layer appears to remove them, they remain recoverable from the image's layer history (Part 1.7's parallel warning about Git history applies identically to Docker layers); inject secrets at runtime via environment variables or a secrets manager (Part 9.5) instead.
• Pin exact base image versions and dependency versions (Part 1.7) — FROM python:3.12-slim without a more specific tag can silently pull a different patch version over time; pin to a specific, tested version for reproducible builds.
• Set explicit resource limits at the container-runtime level (Part 1.8's cgroups discussion) rather than relying on defaults, to prevent one runaway container from starving others on the same host.

9. Common mistakes

• Copying the entire application directory before installing dependencies, invalidating the dependency-layer cache on every single code change and dramatically slowing down iterative builds.
• Baking secrets into an image, only to discover they remain recoverable from earlier layers even after a later layer appears to remove them.
• Running the application process as root inside the container by default (many base images default to root unless explicitly configured otherwise).
• Not pinning base image/dependency versions, leading to "it worked yesterday" build inconsistency as upstream images update.

10. Security considerations

• Container isolation (namespaces/cgroups) is real but weaker than a full VM's hardware-level isolation — for genuinely untrusted, adversarial code execution (Part 9.6's sandboxing discussion, relevant for executing LLM-generated code), additional isolation layers (gVisor, Firecracker microVMs, or a fully separate VM) may be warranted beyond plain Docker containers.
• Scan images for known vulnerabilities in base images and dependencies as part of your CI pipeline (Part 7.2) — an outdated base image can carry known, exploitable CVEs regardless of how careful your own application code is.
• Minimize what's installed in the final production image (multi-stage builds, section 4) — every additional installed tool is additional attack surface if a container is ever compromised.

11. Performance considerations

• Layer caching (section 4) is a real, significant build-speed lever — structuring your Dockerfile to maximize cache hits (dependencies before code) can be the difference between a 30-second and a 5-minute CI build.
• Container startup time (milliseconds to seconds, given no OS boot) is a real advantage for horizontal scaling (Part 7.5) and autoscaling responsiveness compared to VM-based deployment.

12. Cost considerations

• Smaller images (multi-stage builds) reduce registry storage cost and, more significantly, reduce pull time on every deployment/scale-out event — a real, compounding cost/time factor at deployment frequency and scale.

13. When to use it

The default packaging/deployment unit for essentially any modern backend service, including AI applications — near-universal in production software engineering today, and a close-to-mandatory skill for FDE work given the variety of customer deployment environments (Part 14) you'll need to target.

14. When NOT to use it

Extremely simple, purely local scripts/notebooks with no deployment target don't need containerization. Some specialized, extremely latency- or resource-constrained edge deployments may need a lighter-weight packaging approach than a full container runtime — a genuinely rare case for typical enterprise AI backend work, but worth knowing exists.

15. Alternatives and trade-offs

Approach	Good for	Weak point
Docker containers	Portable, reproducible, lightweight, industry-standard	Weaker isolation than a full VM for genuinely untrusted code
Virtual machines	Strongest isolation	Heavier, slower to start, more resource overhead
Bare-metal/direct deployment (no containerization)	Simplicity for a single, controlled environment	Loses reproducibility/portability across environments

16. Practical Python/code example

FROM python:3.12-slim AS builder
WORKDIR /app
RUN pip install --no-cache-dir uv
COPY uv.lock pyproject.toml ./
RUN uv sync --frozen --no-dev --no-install-project

FROM python:3.12-slim
RUN useradd --create-home appuser
WORKDIR /app
COPY --from=builder /app/.venv /app/.venv
COPY --chown=appuser:appuser . .
ENV PATH="/app/.venv/bin:$PATH"
USER appuser

HEALTHCHECK --interval=30s --timeout=3s CMD curl -f http://localhost:8000/health || exit 1
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

This combines section 4's multi-stage pattern with Part 1.8's non-root and health-check recommendations into one concrete, production-oriented Dockerfile. Using uv sync --frozen --no-dev (Part 1.1) rather than pip install -r requirements.txt means the image is built from the exact, hash-verified dependency graph committed in uv.lock — a uv.lock drifted out of sync with pyproject.toml fails the build immediately instead of silently resolving something different than what CI tested.

17. Production-quality example

A docker-compose.yml for local development that mirrors production topology (app + Postgres + Redis, Part 1.5/1.6), giving developers an environment consistent with what will actually run in production:

services:
  app:
    build: .
    ports:
      - "8000:8000"
    environment:
      - DATABASE_URL=postgresql://user:pass@db:5432/appdb
      - REDIS_URL=redis://cache:6379
    depends_on:
      - db
      - cache

  db:
    image: postgres:16
    environment:
      - POSTGRES_PASSWORD=pass
      - POSTGRES_DB=appdb
    volumes:
      - pgdata:/var/lib/postgresql/data

  cache:
    image: redis:7

volumes:
  pgdata:

Note secrets here are placeholders for local development only — production secrets must come from a proper secrets manager (Part 9.5), never hardcoded in a compose file committed to version control.

18. Short exercise

Take the Dockerfile from section 16 and identify: (a) which specific instruction ordering choice preserves the dependency-layer cache across code-only changes, and (b) what would happen to build speed if COPY --chown=appuser:appuser . . were moved to before the uv sync --frozen step.

19. Interview questions

• Explain Docker's layer caching mechanism and why instruction order in a Dockerfile affects build speed.
• Why is a container not a lightweight virtual machine, architecturally, and what security implication follows from that distinction?
• Why should secrets never be baked into a Docker image, even if a later Dockerfile instruction appears to remove them?

20. FDE/customer scenario

Customer's IT team: "We can't give you direct access to our production environment — how would you even deploy this?"

Docker's self-contained image is the concrete answer: build and test the image in an environment you control, then hand the customer a specific, versioned, pre-built image (via their approved registry/transfer process) to run in their environment — they never need to reproduce your exact build environment or dependency resolution themselves, only run the already-built, already-tested artifact, which is precisely the deployment model Part 14 will build on for customer-environment deployment more broadly.

---

Key takeaways

• Docker images package an application with its entire runtime environment, eliminating "works on my machine" inconsistency across dev, CI, and customer production environments.
• Layer caching rewards putting rarely-changing instructions (dependency installation) before frequently-changing ones (application code copy).
• Secrets must never be baked into an image layer — they remain recoverable from layer history even if a later layer appears to remove them.

Things you should be able to explain

• Why containers share the host kernel and how that differs from VM isolation.
• Why Dockerfile instruction order affects build cache efficiency.

Things you should be able to build

• A multi-stage, non-root, health-checked production Dockerfile and a docker-compose setup mirroring production topology for local development.

Common mistakes

• Poor instruction ordering invalidating the dependency cache unnecessarily.
• Baking secrets into image layers.
• Running containers as root by default.

Recommended next chapter

02-cicd.md