1.2 — REST APIs

1. What is it?

A REST (Representational State Transfer) API is a set of conventions for exposing resources over HTTP: nouns as URLs (/customers/123), verbs as HTTP methods (GET, POST, PATCH, DELETE), and status codes as outcomes. In an AI system, REST is almost always the boundary layer: the interface between your AI backend and the frontend, between your service and an LLM provider, and between your service and every enterprise system it integrates with (CRM, ticketing, document stores).

2. Why does it exist?

Before REST conventions became dominant, every API had its own bespoke shape for "how do I do X" — different verbs, different error formats, different pagination styles. REST won because HTTP already gave you a rich, well-understood vocabulary (methods, status codes, headers, caching semantics) and REST just asks you to use it consistently instead of inventing your own on top. That consistency is what lets a new engineer (or a customer's engineering team) understand your API by convention rather than by reading every endpoint's bespoke documentation.

3. What problem does it solve?

It solves the "how do two systems that don't share a codebase or a runtime talk to each other reliably" problem. For an AI FDE this is constant: your agent's tool needs to call the customer's ticketing system's API; the customer's frontend needs to call your AI backend; your AI backend needs to call an LLM provider. All three are REST (or REST-like) boundaries, and getting the contract, error handling, and versioning right at each one determines whether integration is a day of work or a week of debugging.

4. How does it work internally?

The request/response contract

Client

Serverauth → validate → execute

Full headers and body shown in prose above; this is the request/response shape.

Idempotency and why it matters more in AI systems

GET, PUT, and DELETE are meant to be idempotent (calling them N times has the same effect as calling them once); POST is not. In AI systems, this matters acutely: a client that times out waiting for a slow LLM-backed endpoint and retries a POST /generate-report can trigger the (expensive, slow) generation twice. Production AI APIs commonly add an Idempotency-Key header pattern on top of REST conventions specifically to guard against this — the server deduplicates requests with the same key within a time window.

Status codes as a contract, not decoration

Code	Meaning in an AI API context
200 / 201	Success; 201 for resource creation (e.g., a new conversation)
400	Malformed request — e.g., a tool-call payload that fails schema validation
401 / 403	Missing/invalid auth vs. valid auth but insufficient permission (critical distinction for RBAC — see Part 9)
404	Resource doesn't exist — don't conflate with 403 in security-sensitive systems (leaking existence via 404 vs 403 is itself an info-leak, see security section)
422	Semantically invalid — e.g., a well-formed JSON body that fails business validation
429	Rate limited — extremely common when calling LLM providers; your API should propagate or handle this explicitly
500	Unhandled server error
502/503/504	Upstream failure (LLM provider down/slow) — distinguish this from your own bugs in monitoring

5. Simple mental model

A REST API is a restaurant menu with a fixed ordering ritual: you don't walk into the kitchen, you point at an item on the menu (a resource, /orders/42) and use one of a small set of standard requests (order it, modify it, cancel it). The waiter (HTTP) enforces the ritual so the kitchen doesn't need custom protocol for every table.

6. Real-world example

An insurance company's claims-processing AI needs to (1) receive a claim document via their internal API, (2) call your AI service to extract structured fields, (3) write results back to their claims system. Steps 1 and 3 are REST calls into their API (which you don't control and must adapt to); step 2 is your REST API (which you do control and should design well). The FDE's job includes reading their API docs, handling their quirks (maybe their auth token expires every 15 minutes, maybe their pagination is non-standard), while making your own API a clean, well-documented contract for them to integrate against.

7. Architecture diagram

Customer Frontend/Systems

Your AI BackendFastAPI

LLM Provider APIOpenAI/Anthropic

Customer's CRM/Ticketing System

Every arrow above is a REST boundary with its own auth, error handling, retries, and versioning concerns.

8. Production considerations

• Version your API (/v1/...) from day one — enterprise customers integrate against your contract and will not tolerate breaking changes without notice.
• Design for partial failure: if your endpoint orchestrates an LLM call + a DB write + a webhook, decide explicitly what happens if the LLM call succeeds but the DB write fails (see Part 7 for idempotency/outbox patterns).
• Pagination for any list endpoint from the start — "we'll add pagination later" always becomes a breaking change later.
• Consistent error body shape across all endpoints (e.g., {"error": {"code": "...", "message": "..."}}) — customers build error handling against this shape, so inconsistency multiplies their integration cost.

9. Common mistakes

• Using GET for actions that have side effects (e.g., GET /trigger-report) — breaks caching assumptions and can be accidentally triggered by prefetching/crawlers.
• Returning 200 OK with an error payload inside the body — breaks every HTTP-level tool (monitoring, retries, gateways) that inspects status codes.
• Not handling the LLM provider's 429 (rate limit) explicitly — it propagates as an opaque 500 to your own customers instead of a handled, retried, or backed-off condition.
• Designing the API around your internal implementation (e.g., exposing your vector DB's exact query shape) instead of around the resource the customer actually needs.

10. Security considerations

• Never leak existence via error codes carelessly: returning 404 for "resource doesn't exist" vs 403 for "exists but you can't see it" needs a deliberate policy — in multi-tenant systems, often you want to return 404 for both, so a tenant can't enumerate other tenants' resource IDs by watching for 403 vs 404.
• Validate and sanitize every field from the request body before it reaches an LLM prompt or a tool call — an API boundary is exactly where injection attacks (Part 9) get their entry point.
• Rate-limit your own API, not just rely on the LLM provider's rate limit — an unauthenticated or poorly-throttled endpoint that triggers LLM calls is a direct cost-attack surface.

11. Performance considerations

• Keep request/response payloads lean — don't return full LLM reasoning traces or entire retrieved documents in a response the frontend doesn't need; it costs bandwidth and can leak internal context.
• Use HTTP-level compression and connection keep-alive for high-throughput internal service-to-service REST calls.
• For any endpoint that must call a slow LLM synchronously, consider whether it should instead be async (return 202 Accepted + a polling/webhook pattern) — Part 7 covers this trade-off in depth.

12. Cost considerations

• Every unnecessary round trip in a multi-step REST orchestration (e.g., calling an enterprise CRM API per-item instead of in bulk) adds latency and, if each hop also triggers an LLM call, multiplies token cost.
• A poorly rate-limited public API is a direct financial risk when it fronts an LLM — a scraping bot or bug can generate a large, unbounded bill.

13. When to use it

REST is the default choice for synchronous, resource-oriented service boundaries — which is the overwhelming majority of AI system integration points.

14. When NOT to use it

• Streaming token-by-token LLM output: plain request/response REST doesn't fit — use Server-Sent Events or WebSockets layered on HTTP (still "RESTful" in spirit for the rest of the API, but the streaming endpoint itself isn't a classic request/response).
• High-frequency, low-latency internal service calls at large scale sometimes favor gRPC over REST/JSON for lower serialization overhead — rare in early-stage AI systems, worth knowing about for scale-sensitive platforms (Part 11).
• Long-running async workflows (a report that takes 10 minutes to generate) are better modeled as job-creation + polling/webhook than a single long-held REST request.

15. Alternatives and trade-offs

Approach	Good for	Weak point
REST/JSON	Universal compatibility, human-readable, easy to debug	Verbose, chatty for many small resources
GraphQL	Client-driven flexible queries, fewer round trips	Added complexity, caching is harder
gRPC	High-performance internal service-to-service calls	Not browser-native, harder to debug ad hoc
WebSockets/SSE	Streaming, real-time (LLM token streaming, live agent status)	Stateful connections complicate horizontal scaling

16. Practical Python/code example

from fastapi import FastAPI, HTTPException, status
from pydantic import BaseModel

app = FastAPI(title="Claims Extraction API", version="1")


class ExtractionRequest(BaseModel):
    """A request to extract structured fields from a claim document."""
    document_text: str
    claim_id: str


class ExtractionResponse(BaseModel):
    """The structured extraction result for a claim document."""
    claim_id: str
    vendor_name: str
    total_amount_usd: float


@app.post(
    "/v1/claims/extract",
    response_model=ExtractionResponse,
    status_code=status.HTTP_200_OK,
)
async def extract_claim(request: ExtractionRequest) -> ExtractionResponse:
    """
    Extracts structured fields from a claim document's raw text.

    Args:
        request (ExtractionRequest): The claim document text and its id.

    Returns:
        ExtractionResponse: The structured extraction result.
    """
    if not request.document_text.strip():
        raise HTTPException(status_code=400, detail="document_text must not be empty")

    result = await run_extraction(request.document_text)
    return ExtractionResponse(
        claim_id=request.claim_id,
        vendor_name=result.vendor_name,
        total_amount_usd=result.total_amount_usd,
    )

17. Production-quality example

The production version adds an idempotency key, a consistent error envelope, and explicit handling for upstream (LLM) failures:

from fastapi import FastAPI, Header, HTTPException, Request
from fastapi.responses import JSONResponse
from pydantic import BaseModel

app = FastAPI(title="Claims Extraction API", version="1")

_idempotency_cache: dict[str, dict] = {}  # replace with Redis in real deployments


class ErrorBody(BaseModel):
    """Standard error envelope returned by every endpoint on failure."""
    code: str
    message: str


@app.exception_handler(HTTPException)
async def http_exception_handler(request: Request, exc: HTTPException) -> JSONResponse:
    """Converts any HTTPException into the API's standard error envelope."""
    return JSONResponse(
        status_code=exc.status_code,
        content={"error": ErrorBody(code=str(exc.status_code), message=exc.detail).model_dump()},
    )


@app.post("/v1/claims/extract")
async def extract_claim(
    request: "ExtractionRequest",
    idempotency_key: str = Header(..., alias="Idempotency-Key"),
) -> dict:
    """
    Extracts structured fields from a claim document, deduplicating retried requests
    that share the same Idempotency-Key.

    Args:
        request (ExtractionRequest): The claim document text and its id.
        idempotency_key (str): Client-supplied key identifying this logical request,
            used to avoid re-running an expensive LLM extraction on client retry.

    Returns:
        dict: The structured extraction result.
    """
    if idempotency_key in _idempotency_cache:
        return _idempotency_cache[idempotency_key]

    try:
        result = await run_extraction(request.document_text)
    except UpstreamLLMError as exc:
        raise HTTPException(status_code=502, detail=f"extraction provider unavailable: {exc}")

    response = {
        "claim_id": request.claim_id,
        "vendor_name": result.vendor_name,
        "total_amount_usd": result.total_amount_usd,
    }
    _idempotency_cache[idempotency_key] = response
    return response

18. Short exercise

Design (on paper, no code required) the REST contract for an endpoint that lets a customer's support team submit a ticket for AI-drafted response generation, where generation can take up to 30 seconds. Specify: the endpoint(s), status codes at each stage, how the client learns the draft is ready, and what happens if the client never polls for the result.

19. Interview questions

• Why is idempotency a bigger concern for POST endpoints that trigger LLM calls than for typical CRUD endpoints?
• When would you choose 404 over 403 for a resource the requester isn't authorized to see, and why does this matter for multi-tenant AI systems?
• How would you design an API for an LLM endpoint that streams output, while keeping the rest of your API RESTful?

20. FDE/customer scenario

Customer: "Your integration keeps double-creating tickets in our system when our network hiccups and our client retries."

This is the idempotency problem in the wild. The fix isn't "tell the customer to stop retrying" (they won't, and shouldn't have to) — it's adding idempotency-key support to your endpoint, documenting it clearly, and confirming their client actually sends a stable key per logical request (not a new UUID per retry, which defeats the purpose).

---

Key takeaways

• REST conventions (verbs, status codes, idempotency) exist so unrelated systems can integrate predictably.
• Every REST boundary in an AI system needs explicit handling for the LLM-specific failure modes (rate limits, slow responses, non-deterministic output).

Things you should be able to explain

• Why idempotency matters more when POST triggers an expensive LLM call.
• The difference between 403 and 404 in a multi-tenant security context.

Things you should be able to build

• A versioned FastAPI endpoint with a consistent error envelope, idempotency-key handling, and explicit upstream-failure status codes.

Common mistakes

• Using GET for side-effecting actions.
• Returning 200 with an error body.
• No idempotency handling on expensive POST endpoints.

Recommended next chapter

03-fastapi.md