2.2 — Deep Learning Basics

1. What is it?

Deep learning is machine learning using neural networks with many layers ("deep") of learned parameters. It's the specific branch of ML that produced the breakthroughs behind modern computer vision, speech, and — most relevantly here — every LLM you'll work with as an AI FDE.

2. Why does it exist?

Classical ML often required hand-engineered features (a human deciding "edge detectors" for images, or "TF-IDF scores" for text) because the algorithms themselves couldn't learn a useful representation from raw data. Deep learning's core insight — stack enough layers, and the network learns its own useful intermediate representations directly from raw data (pixels, raw text tokens) — removed most of that manual feature engineering, at the cost of needing much more data and compute.

3. What problem does it solve?

It solves representation learning: how do you turn raw, unstructured input (an image, a sentence) into something a mathematical model can usefully reason over? Every layer of a deep network transforms its input into a new representation, progressively more abstract — early layers in an image model might detect edges, later layers detect shapes, final layers detect "this is a cat." For text, this same principle is what lets a transformer go from raw tokens to something that captures meaning, syntax, and (functionally) reasoning.

4. How does it work internally?

The neuron and the layer

A single artificial neuron computes a weighted sum of its inputs, adds a bias, and applies a non-linear activation function:

output = activation(w1*x1 + w2*x2 + ... + wn*xn + b)

The non-linearity (ReLU, GELU, etc.) is not a minor detail — without it, stacking layers would mathematically collapse into a single linear transformation, no matter how many layers you stack. Non-linearity is what lets deep networks represent complex, non-linear functions.

Backpropagation

Given a loss (Part 2.1), backpropagation is the algorithm that computes how much each parameter in every layer contributed to that loss, using the chain rule of calculus, working backward from the output layer to the input layer. This gradient tells the optimizer which direction to nudge each parameter to reduce loss.

Forward pass:  input → layer 1 → layer 2 → ... → output → loss
Backward pass: loss → ∂loss/∂layer_n → ... → ∂loss/∂layer_1 (gradients)
Update:        each parameter -= learning_rate * its gradient

You will not implement backpropagation by hand in AI FDE work (frameworks handle it), but understanding that it exists — and that it's why training requires many passes over data, why learning rate matters, why very deep networks historically had training instability (vanishing/exploding gradients) — explains a lot of "why does training behave this way" questions that come up when customers ask about fine-tuning.

Why depth (and scale) matters

Empirically, deep learning models improve predictably with more data, more parameters, and more compute, following observed scaling laws — this empirical relationship (not a proof, but a consistently observed trend across many model families) is the direct justification for why LLMs got dramatically better by getting dramatically bigger, and is central to understanding why "just use a bigger model" is a real, evidence-backed lever, not hype — while also being an incomplete story (data quality, architecture, and training technique all matter too, and scaling has real diminishing returns and cost trade-offs, covered in Part 3.11).

5. Simple mental model

A deep network is an assembly line of increasingly abstract descriptions: raw material (pixels/tokens) goes in one end, and each station along the line refines the description a bit further (edges → shapes → objects; tokens → syntax → meaning → intent), until the final station produces the answer. No single station "understands" the whole picture — the capability emerges from the composition of many simple transformations.

6. Real-world example

A customer asks why their fine-tuned model "forgot" a capability it used to have after fine-tuning on their new dataset. This is catastrophic forgetting — a known deep learning phenomenon where training on new data can overwrite representations useful for old tasks, because gradient descent has no built-in mechanism to preserve unrelated prior capability. Explaining this correctly (versus vaguely saying "fine-tuning is tricky") is the difference between sounding credible and sounding like you're guessing.

7. Architecture diagram

Inputtokens / pixels

Layer 1low-level features

Layer 2mid-level features

Layer Nhigh-level features

Outputprediction / logits

Forward pass shown; gradients flow backward through every layer during training (backpropagation).

8. Production considerations

• Fine-tuning a deep model carries real risk of catastrophic forgetting and regression on capabilities you didn't intend to change — this is exactly why evaluation (Part 8) before/after any fine-tune is non-negotiable, not optional diligence.
• Training deep models (or fine-tuning large ones) requires GPU infrastructure with materially different cost/ops characteristics than typical backend services — most AI FDE work in practice consumes pretrained/hosted LLMs via API rather than training deep networks from scratch, which is precisely why Part 3 onward focuses there.

9. Common mistakes

• Assuming "deep learning" and "LLM" are interchangeable — LLMs are a specific, transformer-based application of deep learning to sequence modeling; not all deep learning is language-related (vision, audio, tabular deep learning all exist and are relevant for multimodal work, Part 3.10).
• Assuming more layers/parameters always helps — depth without enough data or the right architecture/normalization can make training harder, not better (vanishing gradients, overfitting on small datasets).
• Fine-tuning without a regression evaluation, then being surprised when an unrelated capability degrades.

10. Security considerations

• Membership inference and model inversion attacks (inferring whether specific data was in the training set, or reconstructing training examples) are active research areas relevant to any customer fine-tuning on sensitive data — a real consideration when a healthcare or financial customer wants to fine-tune on PII-containing records (Part 9.6 covers the broader PII/data-leakage picture).

11. Performance considerations

• Inference latency scales with model size and architecture — this is why model selection (Part 3.11) is a real engineering decision, not just "pick the smartest model," since a deeper/larger model is measurably slower and costs more per token.

12. Cost considerations

• Training/fine-tuning cost scales with model size, dataset size, and number of training steps — for most AI FDE engagements, using a pretrained/hosted LLM via API is dramatically cheaper and faster to get to production than training or fine-tuning a model from scratch, and this should be the default assumption unless there's a specific, well-justified reason otherwise (a narrow, high-volume, latency-critical task where a small fine-tuned model beats an API call on cost/latency at scale).

13. When to use it

Understanding deep learning fundamentals is essential background for reasoning about LLM behavior, fine-tuning trade-offs, and multimodal systems — even though you'll rarely train a deep network from scratch in typical FDE work.

14. When NOT to use it

Building and training a custom deep learning model from scratch is rarely the right first move for an enterprise AI FDE engagement — it's slow, expensive, and requires data/expertise most customer engagements don't have upfront. Default to pretrained/hosted models (Part 3) and only consider custom training/fine-tuning once a specific, measured gap justifies the investment (Part 3.11, Part 15 ROI framing).

15. Alternatives and trade-offs

Approach	Good for	Weak point
Pretrained LLM via API	Fast to deploy, no training infra, broad capability	Ongoing per-call cost, less control over exact behavior
Fine-tuned open model	Narrow task specialization, potentially lower per-call cost at scale, data stays in-house	Requires training infra, data, evaluation discipline, ongoing maintenance
Custom deep model from scratch	Maximum control, tailored architecture	Very high data/compute/expertise cost; rarely justified vs. the above for most business problems

16. Practical Python/code example

A minimal illustration of forward pass + non-linearity, to ground the concept (not production training code):

import torch
import torch.nn as nn


class TinyClassifier(nn.Module):
    """A minimal two-layer network illustrating the role of non-linearity between layers."""

    def __init__(self, input_dim: int, hidden_dim: int, output_dim: int):
        """
        Args:
            input_dim (int): Size of the input feature vector.
            hidden_dim (int): Size of the hidden representation.
            output_dim (int): Number of output classes.
        """
        super().__init__()
        self.layer1 = nn.Linear(input_dim, hidden_dim)
        self.activation = nn.ReLU()
        self.layer2 = nn.Linear(hidden_dim, output_dim)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        """
        Computes class logits for a batch of input feature vectors.

        Args:
            x (torch.Tensor): Input tensor of shape (batch_size, input_dim).

        Returns:
            torch.Tensor: Output logits of shape (batch_size, output_dim).
        """
        hidden = self.activation(self.layer1(x))
        return self.layer2(hidden)

Removing self.activation here would make this mathematically equivalent to a single linear layer, regardless of hidden_dim — the concrete, checkable version of "non-linearity is what makes depth meaningful" from section 4.

17. Production-quality example

A regression-evaluation harness for fine-tuning, directly addressing the catastrophic-forgetting risk from section 6:

from dataclasses import dataclass


@dataclass
class RegressionCheckResult:
    """Result of comparing a fine-tuned model's behavior against baseline on known tasks."""
    task_name: str
    baseline_score: float
    fine_tuned_score: float
    regressed: bool


def check_for_regressions(
    baseline_scores: dict[str, float],
    fine_tuned_scores: dict[str, float],
    tolerance: float = 0.02,
) -> list[RegressionCheckResult]:
    """
    Compares fine-tuned model performance against baseline across a fixed set of tasks,
    flagging any task that regressed beyond tolerance.

    Args:
        baseline_scores (dict[str, float]): Task name to score, before fine-tuning.
        fine_tuned_scores (dict[str, float]): Task name to score, after fine-tuning.
        tolerance (float): Allowed score drop before flagging a regression.

    Returns:
        list[RegressionCheckResult]: One result per task, flagging regressions explicitly.
    """
    results = []
    for task, baseline in baseline_scores.items():
        fine_tuned = fine_tuned_scores.get(task, 0.0)
        results.append(
            RegressionCheckResult(
                task_name=task,
                baseline_score=baseline,
                fine_tuned_score=fine_tuned,
                regressed=(baseline - fine_tuned) > tolerance,
            )
        )
    return results

18. Short exercise

Explain, in your own words, why removing the non-linear activation function from a deep network makes it mathematically equivalent to a single-layer model, no matter how many layers it has.

19. Interview questions

• Explain backpropagation at a conceptual level, without writing the calculus.
• What is catastrophic forgetting, and how would you detect it after fine-tuning a model?
• Why do scaling laws matter for how AI FDEs think about model selection and capability planning?

20. FDE/customer scenario

Customer: "We want to fine-tune an open-source model on our support tickets so it 'knows' our product better."

Before agreeing, an FDE would ask what specific capability gap fine-tuning is meant to close (tone/format consistency? domain vocabulary? factual knowledge?) — because factual knowledge is often better and more maintainably delivered via RAG (Part 3.5) than fine-tuning (fine-tuning bakes knowledge in at training time; it goes stale the moment the product changes, and re-fine-tuning for every product update is far more expensive and slower than updating a document store). This is one of the most common, valuable pieces of judgment an FDE provides: fine-tuning and RAG solve different problems, and conflating them leads to expensive, poorly-maintained solutions.

---

Key takeaways

• Non-linear activations are what make depth meaningful — without them, stacked layers collapse to one linear function.
• Backpropagation is how gradients (and thus learning signal) flow backward through a deep network.
• Fine-tuning risks catastrophic forgetting; evaluate before/after against a fixed regression suite.

Things you should be able to explain

• Why non-linearity matters in a deep network.
• The difference between fine-tuning for behavior/format vs. using RAG for knowledge.

Things you should be able to build

• A regression-check harness comparing baseline vs. fine-tuned model performance across tasks.

Common mistakes

• Fine-tuning to add factual knowledge instead of using RAG.
• No regression evaluation after fine-tuning.

Recommended next chapter

03-nlp.md