16.3 — Project: Multi-Tenant RAG SaaS

This project builds the meeting-notes summarization SaaS designed in Part 11.4 into a real, runnable system, combined with Part 3.5's RAG techniques for a "search past meetings" feature. Read Part 11.4 first.

1. Customer Scenario

A startup (Part 11.4's scenario) offers AI meeting-notes summarization and action-item extraction to business customers ranging from 5-person teams to large enterprises with contractual isolation requirements — this project adds a "search across past meetings" RAG feature on top of Part 11.4's core summarization product.

2. Requirements

Recap of Part 11.4: tiered tenant isolation (shared database for small/mid customers, dedicated database for large enterprise customers), per-tenant customization, zero cross-tenant leakage under any circumstance. This project adds: RAG-based search across a tenant's own historical meeting transcripts, respecting the same tiered isolation.

3. Architecture

Part 11.4's tiered multi-tenant architecture, extended with a per-tenant-scoped vector store for meeting-transcript search (Part 3.4/3.5), using Part 10.5's tenant-tier-aware routing pattern to determine which database connection (and, by extension, which vector store partition/instance) a given tenant's data lives in.

4. Technology Selection

• pgvector (Part 1.5/3.4) for the shared tier, using tenant_id-scoped rows within one Postgres instance.
• Separate pgvector-enabled Postgres instances for the dedicated tier (Part 10.5's database-per-tenant pattern).
• FastAPI with tenant-tier-aware dependency injection (Part 10.5's routing pattern) determining which database connection to use per request.

5. Folder Structure

meeting-saas/
├── src/
│   ├── tenancy/
│   │   ├── models.py               # tenant config, isolation tier
│   │   └── connection_router.py    # Part 10.5's tenant-tier-aware routing
│   ├── ingestion/
│   │   ├── transcript_processor.py # chunking (Part 3.5), tenant-tagged
│   │   └── summarizer.py           # per-tenant customized summary format
│   ├── search/
│   │   ├── retriever.py            # tenant-scoped, pre-filtered retrieval (Part 3.4)
│   │   └── verified_retriever.py   # defense-in-depth assertion (Part 4.5/9.6)
│   ├── billing/
│   │   ├── usage_tracker.py        # per-tenant cost attribution (Part 7.9/7.10)
│   │   └── cache.py                # tenant-scoped cache-key utility (Part 9.6), ahead of section 15's full semantic cache
│   └── api/
│       └── routes.py
├── tests/
│   ├── unit/
│   ├── integration/
│   └── security/
│       └── test_tenant_isolation.py  # THE most important test file in this project
├── infra/
│   ├── Dockerfile
│   ├── k8s/
│   │   └── dedicated-tenant-provisioning/  # templated per-tenant DB provisioning
│   └── terraform/
│       ├── modules/rds/                    # dedicated-tier DB module (Part 18.6)
│       └── environments/production/dedicated_tenants.tf  # per-tenant for_each (section 11)
└── requirements.txt

6. Implementation (key excerpt)

The tenant-tier-aware connection router, directly from Part 10.5's production example, plus the defense-in-depth retrieval verification from Part 4.5:

async def search_meeting_transcripts(query: str, tenant_config, connection_registry, embed_fn) -> list[dict]:
    """Searches a tenant's meeting transcripts, routing to the correct
    isolation tier's connection and applying defense-in-depth verification."""
    connection = get_database_connection_for_tenant(tenant_config, connection_registry)
    query_embedding = (await embed_fn([query]))[0]

    retriever = TenantVerifiedRetriever(
        base_retriever=connection.vector_store.as_retriever(
            search_kwargs={"k": 5, "filter": {"tenant_id": tenant_config.tenant_id}}
        )
    )
    return await retriever.get_relevant_documents(query, tenant_id=tenant_config.tenant_id)

7. Testing

test_tenant_isolation.py is the single highest-priority test suite in this entire project (Part 1.9's tenant-isolation test discussion) — verifying, for every isolation tier, that tenant A's search never returns tenant B's content, including specifically testing the shared-tier pre-filtering behavior (Part 3.4's pre-filter/post-filter distinction) and the dedicated-tier's complete database separation.

Here is that file in full, not just described — fixtures seed two real tenants (one shared-tier, sharing a Postgres instance via tenant_id-scoped rows; one dedicated-tier, its own database instance) with overlapping meeting-transcript content, then assert isolation holds at both the pre-filter layer and the TenantVerifiedRetriever defense-in-depth layer (Part 4.5's exact class, reused here unmodified), plus a cache-key regression test for the shared-tier semantic cache added in section 15/04-langchain/07-production-patterns.md's cache-key pattern (Part 9.6's exact real-world leak this guards against):

"""tests/security/test_tenant_isolation.py

The single highest-priority test file in this project. Every test here
must pass before any deploy — a regression here is a data breach, not a bug.
"""

import pytest
import pytest_asyncio

from src.tenancy.connection_router import get_database_connection_for_tenant
from src.search.verified_retriever import TenantVerifiedRetriever
from src.billing.cache import build_tenant_scoped_cache_key, get_cached_answer, set_cached_answer


TENANT_A = "tenant_acme"       # shared tier
TENANT_B = "tenant_globex"     # shared tier, same Postgres instance as A
TENANT_C = "tenant_initech"    # dedicated tier, its own Postgres instance


@pytest_asyncio.fixture
async def seeded_shared_tier(shared_pg_pool, embed_fn):
    """Seeds two shared-tier tenants into the same pgvector-backed Postgres
    instance with distinct, non-overlapping meeting transcripts.

    Args:
        shared_pg_pool: A connection pool fixture for the shared-tier database.
        embed_fn: An async embedding function fixture (real or a deterministic fake).

    Returns:
        dict: Tenant configs and the shared connection registry.
    """
    async with shared_pg_pool.acquire() as conn:
        await conn.execute(
            "INSERT INTO meeting_chunks (tenant_id, content, embedding) VALUES ($1, $2, $3)",
            TENANT_A, "Acme Q3 roadmap: ship the billing-v2 migration by October.",
            await embed_fn(["Acme Q3 roadmap: ship the billing-v2 migration by October."]),
        )
        await conn.execute(
            "INSERT INTO meeting_chunks (tenant_id, content, embedding) VALUES ($1, $2, $3)",
            TENANT_B, "Globex Q3 roadmap: the billing-v2 migration slipped to November.",
            await embed_fn(["Globex Q3 roadmap: the billing-v2 migration slipped to November."]),
        )
    return {"pool": shared_pg_pool, "tenant_a": TENANT_A, "tenant_b": TENANT_B}


@pytest_asyncio.fixture
async def seeded_dedicated_tier(dedicated_pg_pool_factory, embed_fn):
    """Seeds a dedicated-tier tenant into its own, separately-provisioned
    Postgres instance (Part 10.5's database-per-tenant pattern).

    Args:
        dedicated_pg_pool_factory: A factory fixture returning a connection pool
            scoped to one tenant's dedicated database instance.
        embed_fn: An async embedding function fixture.

    Returns:
        dict: The dedicated tenant's config and its isolated connection pool.
    """
    pool = await dedicated_pg_pool_factory(TENANT_C)
    async with pool.acquire() as conn:
        await conn.execute(
            "INSERT INTO meeting_chunks (tenant_id, content, embedding) VALUES ($1, $2, $3)",
            TENANT_C, "Initech Q3 roadmap: the billing-v2 migration is out of scope this year.",
            await embed_fn(["Initech Q3 roadmap: the billing-v2 migration is out of scope this year."]),
        )
    return {"pool": pool, "tenant_c": TENANT_C}


@pytest.mark.asyncio
async def test_shared_tier_prefilter_excludes_other_tenant(seeded_shared_tier, embed_fn):
    """A shared-tier query for tenant A must never return tenant B's rows,
    even though both tenants' embeddings live in the same table/index —
    this is Part 3.4's pre-filter behavior, tested directly against the query
    both tenants would plausibly ask (they used near-identical wording)."""
    connection = await get_database_connection_for_tenant(
        tenant_config={"tenant_id": TENANT_A, "tier": "shared"},
        connection_registry={"shared": seeded_shared_tier["pool"]},
    )
    query_embedding = (await embed_fn(["What's the billing-v2 migration timeline?"]))[0]
    rows = await connection.fetch(
        "SELECT tenant_id, content FROM meeting_chunks "
        "WHERE tenant_id = $1 ORDER BY embedding <-> $2 LIMIT 5",
        TENANT_A, query_embedding,
    )
    assert len(rows) > 0
    assert all(row["tenant_id"] == TENANT_A for row in rows)
    assert not any("Globex" in row["content"] for row in rows)


@pytest.mark.asyncio
async def test_verified_retriever_raises_on_cross_tenant_leak(seeded_shared_tier, embed_fn):
    """Defense-in-depth check (Part 4.5's TenantVerifiedRetriever): even if a
    misconfigured or buggy base retriever returns a document from the wrong
    tenant, the verification layer must catch it and raise loudly rather than
    silently serving the leaked result."""

    class _BrokenRetriever:
        """A deliberately misconfigured retriever standing in for a
        pre-filter/post-filter bug, to prove the verification layer catches it."""

        async def ainvoke(self, query):
            class _Doc:
                metadata = {"tenant_id": TENANT_B}
                page_content = "Globex Q3 roadmap: leaked content"
            return [_Doc()]

    retriever = TenantVerifiedRetriever(base_retriever=_BrokenRetriever())
    with pytest.raises(RuntimeError, match="tenant isolation"):
        await retriever.get_relevant_documents("billing migration timeline", tenant_id=TENANT_A)


@pytest.mark.asyncio
async def test_dedicated_tier_is_a_fully_separate_database(seeded_shared_tier, seeded_dedicated_tier):
    """A dedicated-tier tenant's data must be physically unreachable from a
    shared-tier connection — not merely filtered out, but on a different
    database instance entirely (Part 10.5's isolation guarantee)."""
    shared_conn = seeded_shared_tier["pool"]
    async with shared_conn.acquire() as conn:
        rows = await conn.fetch(
            "SELECT tenant_id FROM meeting_chunks WHERE tenant_id = $1", TENANT_C
        )
    assert rows == []  # tenant C's rows cannot exist in the shared instance at all


@pytest.mark.asyncio
async def test_cache_key_is_tenant_scoped_not_just_query_scoped(redis_client):
    """Regression test for Part 9.6's real-world Redis cache-leak scenario:
    a cache key built from query text alone lets tenant A's cached answer
    be served to tenant B when they ask a similarly-worded question. This
    project's semantic cache (section 15) must scope every key by tenant_id."""
    query = "What's the billing-v2 migration timeline?"

    key_a = build_tenant_scoped_cache_key(query, tenant_id=TENANT_A)
    key_b = build_tenant_scoped_cache_key(query, tenant_id=TENANT_B)
    assert key_a != key_b  # same query text, different tenants -> different keys

    await set_cached_answer(redis_client, key_a, "Acme's migration ships in October.")

    # Tenant B asking the same question must miss the cache entirely, never
    # receiving tenant A's cached answer back.
    cached_for_b = await get_cached_answer(
        redis_client, build_tenant_scoped_cache_key(query, tenant_id=TENANT_B)
    )
    assert cached_for_b is None

Each of these four tests exercises a different isolation layer named in section 9 (database, vector-store pre-filter, retriever-level defense-in-depth, cache) — matching Part 9.6's central point that isolation must be audited layer-by-layer, since getting three out of four right still leaves a real breach.

8. Evaluation

Search relevance and faithfulness (Part 8.1/8.2) evaluated per tenant tier, since retrieval quality characteristics can genuinely differ between a shared, larger index and a dedicated, smaller one.

9. Security

Full-layer tenant isolation audit (Part 9.6) across database, vector store, cache (if added), and billing data — this project's entire raison d'être is getting this right, given the product category's central risk (Part 11.4's core design challenge).

10. Observability

Per-tenant cost, usage, and search-quality metrics (Part 7.9/8.4) feeding both customer-facing dashboards and internal margin analysis per isolation tier.

11. Deployment

Shared-tier infrastructure scales normally (Part 7.4/7.5); dedicated-tier tenants get their own provisioned database instance via a templated provisioning process (Part 10.5), triggered when a new enterprise contract requiring this tier is signed.

That templated provisioning process is itself Terraform, not a manual runbook — a for_each-based module (Part 18.6's exact for_each pattern, applied here per-tenant instead of per-availability-zone) instantiates one dedicated, pgvector-enabled RDS instance and schema per enterprise-tier tenant listed in a tracked variable, so onboarding a new dedicated-tier tenant is a one-line addition to dedicated_tenants, plan-reviewed like any other infrastructure change rather than a hand-run script:

# environments/production/dedicated_tenants.tf
variable "dedicated_tenants" {
  type = map(object({
    instance_class = string
    storage_gb     = number
  }))
  default = {
    initech_corp = { instance_class = "db.r6g.large", storage_gb = 100 }
    hooli_inc    = { instance_class = "db.r6g.xlarge", storage_gb = 250 }
  }
}

module "dedicated_tenant_db" {
  source   = "../../modules/rds"
  for_each = var.dedicated_tenants

  vpc_id             = module.vpc.vpc_id
  data_subnet_ids    = module.vpc.data_subnet_ids
  app_security_group = module.app_service.security_group_id
  instance_class     = each.value.instance_class
  allocated_storage  = each.value.storage_gb
  multi_az           = true                       # dedicated tier's SLA requires it (section 12)
  identifier         = "meeting-saas-${each.key}"
  environment        = "production"

  tags = {
    Tier     = "dedicated"
    TenantId = each.key                            # provisioning-time isolation record, cross-checked in section 9's audit
  }
}

output "dedicated_tenant_endpoints" {
  value = { for tenant, db in module.dedicated_tenant_db : tenant => db.endpoint }
}

Adding hooli_inc to dedicated_tenants and running terraform plan/apply provisions its dedicated database without touching any other tenant's resources — the same for_each-over-a-map isolation property Part 18.6's availability-zone example demonstrates, applied here to tenant onboarding instead of AZ expansion.

12. Scaling

The tiered architecture's key scaling benefit (Part 11.4, section 10): a large dedicated-tier tenant's own scale needs are fully isolated from the shared tier's capacity planning.

13. Cost Considerations

Dedicated-tier pricing should reflect its genuinely higher infrastructure cost (Part 10.5, section 12) — a direct architecture-to-pricing connection, verified via per-tenant cost tracking (Part 7.9/7.10).

14. Failure Scenarios

A dedicated-tier tenant's database instance failure is isolated to that tenant only (a direct benefit of the isolation tier). A shared-tier "noisy neighbor" (one tenant's heavy usage degrading others) is mitigated via per-tenant rate limiting (Part 7.9's gateway pattern, applied per-tenant, Part 11.4's section 8 exact mitigation).

15. Improvements

Add semantic caching (Part 7.8) for common cross-meeting query patterns, tenant-scoped per Part 9.6's cache-key discipline (Part 9.6's exact cache-leakage warning from its real-world example, deliberately avoided here from the start). The tenant-scoped build_tenant_scoped_cache_key utility (billing/cache.py) is built and regression-tested (section 7) ahead of the full feature specifically so the isolation discipline is never bolted on after a first, unscoped version ships — the remaining "improvement" is the semantic-similarity cache-matching layer on top of this already-safe key scheme.

16. Business Metrics

Search-feature adoption rate and its correlation with customer retention/expansion (a leading indicator, Part 15) — plus per-tier margin analysis confirming the dedicated tier's pricing actually recovers its higher infrastructure cost as designed.

---

Key takeaways

• Tenant isolation is this project's central, non-negotiable requirement — the test suite dedicated to it (test_tenant_isolation.py) deserves more engineering attention than any single feature.
• The tiered architecture from Part 11.4 extends naturally to a new feature (search) by applying the same tenant-tier-aware routing pattern, rather than requiring a separate isolation model per feature.
• Defense-in-depth verification (Part 4.5/9.6) at the retrieval layer is worth the modest extra engineering cost given how serious a tenant-isolation failure would be for this product category specifically.

Recommended next chapter

04-ai-workflow-automation.md