Skillversitas

PyTorch Overview

PyTorch is the deep learning framework the research and production both run on: a Tensor the N-dimensional array (the NumPy the GPU-backed, the operation the autograd-tracked), an nn.Module the model's container (the layer, the parameter, the forward), and the autograd engine (the gradient the backward pass computes, the no symbolic graph the static). The eager execution (the Python's, the step-by-step) is the debugger's friend - the print mid-forward works, the breakpoint lands - and the torch.compile (the 2.0+) adds the graph capture without the TorchScript sacrifice. This chapter covers what PyTorch is (the tensor, the module, the autograd), the eager model (the step-by-step, the debug), the install's shape (the CPU, the CUDA, the MPS), and the MNIST the running example the chapters build on.

Why PyTorch

# the eager: the Python's step-by-step (the debug
#   the natural, the the no graph's the no pre-compile)
import torch
import torch.nn as nn

model = nn.Linear(784, 10)          # the layer
x = torch.randn(32, 784)            # the batch
out = model(x)                      # the forward
loss = nn.functional.cross_entropy(
    out, torch.randint(0, 10, (32,)))
loss.backward()                     # the gradient
#   the print mid-forward works, the the
#   the breakpoint the
print(out.shape)   # the (32, 10) - the
                   #   the

The pitch is the eager execution: the model runs as the plain Python (the nn.Linear the call, the cross_entropy the loss, the backward() the gradient), so the print and the breakpoint work mid-forward (the debug the natural, the TensorBoard the trace) - the static graph (the TF1 era) compiles first and the debug is the session's inspect. The autograd is the engine behind: every operation records the gradient's path, and the backward() walks it reverse (the chain rule, the no manual derivative). The torch.compile (the 2.0+) captures the graph lazily (the first call the trace, the speedup the 1.2x to the 2x on the transformer) without leaving the eager's Python. The framework is the three: the Tensor (the next chapter), the nn.Module (the model), and the autograd (the gradient) - the MNIST's 784 input the 10 class the running example, and the install (the next chapter) is the CPU or the GPU.

The Install's Shape

# the install: the CPU (the default, the no GPU):
pip install torch torchvision

# the CUDA (the the NVIDIA's GPU, the
#   the):
#   the
#   the
pip install torch torchvision --index-url \
  https://download.pytorch.org/whl/cu121

# the MPS (the
#   the):
#   the
#   the
#   the

# the the CHECK (the):
python -c "import torch; print(torch.cuda.is_available(), torch.mps.is_available())"
#   the

The install is the one command: the pip install torch torchvision (the CPU build, the torchvision the dataset plus the transform the MNIST's), and the GPU is the index-url (the cu121 the CUDA version, the download.pytorch.org the wheel) - the torch.cuda.is_available() the check (the True the .cuda() the move, the False the .cpu()). The MPS is the Apple silicon's (the torch.backends.mps, the .to("mps")), and the device variable (the "cuda" if available else "cpu") is the pattern the model uses so the same code runs on the both. The rule is the one: the pip the CPU, the index-url the CUDA, the mps the Apple, and the device the model's target - the next chapter's tensor is the array the device holds, and the operation the autograd tracks.

The MNIST: The Running Example

# the MNIST (the 28x28 the digit, the the
#   the 60000 the train, the
#   the 10000 the test, the
#   the):
import torchvision
import torchvision.transforms as T

transform = T.Compose([
    T.ToTensor(),        # the
    T.Normalize((0.1307,), (0.3081,))   # the MNIST's
])
train = torchvision.datasets.MNIST(
    "data", train=True, download=True,
    transform=transform)   # the (32, 1, 28, 28) the
                           #   the batch, the
test = torchvision.datasets.MNIST(
    "data", train=False, download=True,
    transform=transform)
#   the
#   the

The MNIST is the dataset the chapters train on: the 60000 image the 28x28 digit (the train), the 10000 the test, the label the 0 to the 9 (the 10 class). The transform is the pipeline (the ToTensor the PIL to the (1, 28, 28) the 0 to the 1, the Normalize the MNIST mean/std the channel's) - the batch's shape is the (32, 1, 28, 28) (the 32 the batch, the 1 the channel, the 28x28 the spatial) and the DataLoader (the next chapter) the shuffle plus the batch. The model (the Linear the 784 to the 10, the CNN the later) takes the (N, 784) the flatten, and the cross_entropy the loss the 10 class - the example the small (the 1 epoch the 1 minute on the CPU), the pattern the real (the CNN the ImageNet the same loop). The next chapter installs the ecosystem (the torch, the torchvision, the device) and the tensor is the array the operation the autograd tracks.

The Working Rules

  • The eager is the Python's, the torch.compile the speedup. The step-by-step the debug (the print, the breakpoint mid-forward), the 2.0+ compile the graph capture (the 1.2x to the 2x, the no TorchScript), and the autograd the gradient the backward computes.
  • The nn.Module is the model, the Tensor the array. The layer the forward, the parameter the model.parameters(), the nn.Linear/Conv2d the build - the Module the container, the Tensor the data the device holds.
  • The install is the CPU default, the index-url the CUDA, the mps the Apple. The pip install torch torchvision the CPU, the cu121 the NVIDIA, the torch.backends.mps the Apple - the is_available() the check, the device the model's target.
  • The MNIST is the running example, the transform the pipeline. The 60000/10000 the 28x28 digit, the ToTensor plus the Normalize the (32, 1, 28, 28) the batch, the DataLoader the shuffle - the small the example, the pattern the real.

Summary

PyTorch is the deep learning framework: the Tensor the GPU-backed NumPy (the operation the autograd-tracked), the nn.Module the model's container (the layer, the parameter, the forward), and the autograd the engine (the gradient the backward pass, the chain rule). The eager execution is the Python's (the print mid-forward, the breakpoint the debug), and the torch.compile (the 2.0+) the graph capture the speedup without the TorchScript. The install is the pip install torch torchvision (the CPU), the index-url the CUDA (the cu121), the mps the Apple silicon, and the device variable the model's target. The MNIST is the running example: the 60000 the 28x28 digit, the ToTensor plus the Normalize the (32, 1, 28, 28) the batch, the DataLoader the shuffle, and the Linear the 784 to the 10 the first model. The next chapter installs the ecosystem and the tensor is the array the operation the autograd tracks - the Tensor the NumPy the device holds, the requires_grad the gradient's flag.