1.7 — Git

1. What is it?

Git is a distributed version control system: every clone of a repository is a full copy of its history, and commits form a content-addressed graph (each commit hashed by its content and parent(s)). For an AI FDE, Git is not just "how I save code" — it's how prompts, evaluation datasets, agent graph definitions, and infrastructure config are all versioned, reviewed, and rolled back in production AI systems.

2. Why does it exist?

Centralized version control systems (Subversion, CVS) required a network connection to the central server for most operations and made branching/merging expensive and rare. Git's distributed model (Linus Torvalds, 2005, built for Linux kernel development) made branching cheap and merging a first-class, frequent operation — which unlocked the workflows (feature branches, pull requests, trunk-based development) that essentially all modern software teams use today.

3. What problem does it solve?

For AI systems specifically, Git solves a problem tutorials rarely mention: prompts and agent behavior are code, and they need the same rigor — code review, rollback, blame/history — that application code gets. A prompt change that silently regresses an agent's behavior in production is a bug exactly like a code bug, and if it's not in version control with a clear diff, debugging "why did the agent start doing this yesterday" becomes guesswork.

4. How does it work internally?

The object model

commitparent + author + message + timestamp

treedirectory snapshot

blobfile content

blobfile content

Content-addressed by SHA — identical content anywhere in history is stored once.

Every commit points to a tree (a snapshot of the directory structure) and one or more parent commits. Each tree entry points to a blob (raw file content) or another tree (subdirectory). Content is addressed by SHA hash — identical file content anywhere in history is stored once. This is why Git can compute diffs, detect renames heuristically, and why a commit hash uniquely and verifiably identifies an exact snapshot of the entire repository at that point.

Branches are just pointers

A branch is nothing but a mutable pointer to a commit — git branch feature-x creates a new label pointing at the current commit; committing on that branch moves the pointer forward. This is why branching and switching is instantaneous compared to systems that physically copy files.

Merging and rebasing

• Merge creates a new commit with two parents, preserving the branch's history exactly as it happened.
• Rebase replays your branch's commits on top of a new base, creating new commits (different hashes) and producing linear history.

The trade-off: rebase gives cleaner, easier-to-read history at the cost of rewriting commit identity (never rebase commits that have been pushed and pulled by others, without explicit coordination) — merge preserves true history but can make it noisier.

5. Simple mental model

Git history is a tree of snapshots, not a list of diffs (even though it displays diffs to you). Every commit is a complete, addressable photograph of your entire project at that instant; Git computes the diff you see on demand by comparing two photographs, it doesn't store the diff as the primary representation.

6. Real-world example

An AI FDE team keeps their production system prompt in a versioned file (prompts/support_agent_v3.txt), not hardcoded inline in application code. When a customer reports the agent started giving worse answers on Tuesday, git log -p prompts/support_agent_v3.txt immediately shows what changed, when, and by whom — turning a vague "it got worse somehow" into a two-minute diagnosis and a one-line revert if needed.

7. Architecture diagram

commit C3feature/x

commit C2main · HEAD before merge

commit C1

Branches are pointers to a commit, not containers of commits.

8. Production considerations

• Protect main/production branches — require pull request review before merge, especially for changes to prompts, agent graph definitions, and evaluation thresholds.
• Tag releases (git tag v1.4.0) so a production incident can be tied to an exact, reproducible commit.
• Keep prompts, few-shot examples, and eval datasets in the same repository (or a tightly linked one) as the code that uses them — divergence between "the prompt in the repo" and "the prompt actually running in prod" is a common, painful source of debugging confusion.
• Use .gitignore deliberately for anything containing secrets or large binary artifacts (model weights, large datasets) — Git handles large binaries poorly; use Git LFS or external artifact storage instead.

9. Common mistakes

• Committing API keys or secrets — even one commit is enough, since Git history is permanent unless rewritten (and once pushed/shared, effectively never fully erasable).
• Force-pushing to a shared branch, silently discarding others' commits.
• Massive, unreviewable pull requests that bundle unrelated changes (a prompt change + a refactor + a dependency bump), making it impossible to isolate what actually caused a regression.
• Not tagging or branching before a risky change to a production prompt/agent config, making rollback slower than it needed to be.

10. Security considerations

• Treat any committed secret as compromised immediately (rotate it) — removing it from a later commit does not remove it from history without an explicit history rewrite (git filter-repo or BFG), and even then, anyone who already cloned/fetched has a copy.
• Enforce signed commits/tags in high-trust environments (e.g., regulated customer deployments) to establish provenance of changes to production AI configuration.
• Branch protection rules (required reviews, required status checks/CI) are a security control, not just a process nicety — they're what prevent a compromised or careless single commit from reaching production directly.

11. Performance considerations

• Large binary files bloat repository size and slow every clone/fetch forever (they live in history even after deletion) — use Git LFS or keep large artifacts (embeddings, model weights, datasets) out of Git entirely, referenced by external storage instead.
• Shallow clones (git clone --depth 1) speed up CI checkouts that don't need full history.

12. Cost considerations

• A bloated repository (accidentally committed large files, never cleaned up) increases CI checkout time and storage cost across every clone, every CI run, indefinitely — a small mistake with a long-tail cost.

13. When to use it

For essentially all code, configuration, prompts, and infrastructure-as-code in an AI system — Git is close to universal as the default choice today.

14. When NOT to use it

• Not a substitute for a dedicated dataset/experiment tracking system for large training datasets or model artifacts — pair it with DVC, LangSmith datasets, or similar for those (Git tracks the pointer/config, not the multi-GB blob).
• Not designed for concurrent editing of a single large binary file (e.g., a spreadsheet) — no meaningful diff/merge for such files.

15. Alternatives and trade-offs

Tool	Good for	Weak point
Git	Universal, distributed, cheap branching	Poor at large binaries by default
Mercurial	Similar model, historically simpler CLI	Much smaller ecosystem/adoption today
Perforce	Large binary assets (game dev), centralized locking model	Heavier, centralized, less common in AI/web engineering

16. Practical example

# A disciplined workflow for a prompt change with real production risk
git checkout -b prompt/support-agent-tone-fix
# edit prompts/support_agent_v3.txt
git add prompts/support_agent_v3.txt
git commit -m "Adjust support agent tone per customer feedback on 2026-08-28 escalations"
git push -u origin prompt/support-agent-tone-fix
# open PR, require review + a LangSmith eval run against the regression dataset before merge

17. Production-quality example

A pre-commit hook wired into CI that blocks a prompt change from merging without a passing evaluation run (conceptual, using GitHub Actions):

name: prompt-change-eval-gate
on:
  pull_request:
    paths:
      - "prompts/**"
jobs:
  eval:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run regression eval against changed prompt
        run: python scripts/run_eval.py --dataset regression_v1 --fail-below 0.9

This turns "did this prompt change regress the agent" from a manual, easy-to-skip step into an enforced gate — the same rigor a code change would get from a test suite (see Part 1.9 and Part 8 for the evaluation side of this).

18. Short exercise

You discover an API key was committed three commits ago and has since been pushed to a shared remote. Write out, in order, the actual steps you'd take (hint: rotating the key comes before, not after, any history rewrite).

19. Interview questions

• Explain why a Git commit hash is a valid way to make a production AI system's behavior fully reproducible.
• What's the practical difference between merge and rebase, and when would you mandate one over the other on a team?
• Why should prompts and evaluation datasets live in version control alongside application code?

20. FDE/customer scenario

Customer's engineering lead: "How do we know exactly what changed in the AI system between last week's version and this week's, since your team ships fast?"

This is answered cleanly if prompts, agent graph definitions, and config are all in Git with tagged releases and PR history — you can hand them a direct diff (git diff v1.3.0 v1.4.0) between two exact, reproducible states. If any of that lives outside version control (a prompt edited directly in a dashboard with no history), you cannot answer this question credibly, which is itself a trust problem in an enterprise engagement.

---

Key takeaways

• Git's content-addressed commit graph makes exact reproducibility of a system's state possible — use that for prompts and agent configs, not just code.
• Branch protection and PR review are security controls for production AI behavior, not just code style enforcement.

Things you should be able to explain

• The commit/tree/blob object model and why branches are cheap.
• The merge vs. rebase trade-off.

Things you should be able to build

• A CI gate that blocks a prompt change from merging without a passing evaluation run.

Common mistakes

• Committing secrets.
• Force-pushing shared branches.
• Keeping prompts outside version control.

Recommended next chapter

08-linux.md