Skillversitas

The nn.Module Class

The nn.Module is the model's container: the __init__ the layer (the self.fc = nn.Linear), the forward the operation (the return self.fc(x)), and the parameters the optimizer's target (the model.parameters() the weight plus the bias). The Module is the recursive (the self.net = nn.Sequential the sub-module, the parameters the all), the to(device) the move (the weight the buffer), the train/eval the mode (the BatchNorm the Dropout's switch), and the state_dict the save (the weight the buffer, the no code). This chapter covers the Module (the __init__, the forward, the parameters), the Sequential (the no-forward), the sub-module (the recursive), the to/train/eval (the mode), and the state_dict (the save).

The Module: The __init__ and the forward

# the MODULE (the):
import torch
import torch.nn as nn

class DigitModel(nn.Module):
    def __init__(self):
        super().__init__()          # the
                                    #   the
                                    #   the
        self.fc1 = nn.Linear(784, 128)   # the LAYER
        self.fc2 = nn.Linear(128, 10)    #   the LAYER

    def forward(self, x):           # the
        x = x.flatten(1)            #   the the (N,784)
        x = torch.relu(self.fc1(x)) #   the
        return self.fc2(x)          #   the the (N,10)

model = DigitModel()
print(model)   # the
#   the
#   the

x = torch.randn(4, 1, 28, 28)
out = model(x)   # the (4, 10)
print(out.shape)
#   the

The Module is the two: the __init__ (the super().__init__() the required, the self.fc1 = nn.Linear the layer) and the forward (the x.flatten(1) the (N,784), the relu the non-linear, the fc2 the (N,10)). The model(x) is the forward (the __call__ the hook plus the forward), the print(model) is the structure (the Linear(784, 128), the Linear(128, 10)), and the model.parameters() is the generator (the fc1.weight, the fc1.bias, the fc2.weight, the fc2.bias) - the optimizer's target, the sum(p.numel() for p in model.parameters()) the count. The rule is the one: the __init__ the layer, the forward the operation, the parameters the optimizer - the next section's Sequential is the no-forward the simple, and the sub-module the complex.

The Sequential and the sub-module

# the SEQUENTIAL (the):
#   the
model = nn.Sequential(
    nn.Flatten(),
    nn.Linear(784, 128),
    nn.ReLU(),
    nn.Linear(128, 10),
)   # the
    #   the
    #   the

# the SUB-MODULE (the):
#   the
class Block(nn.Module):
    def __init__(self, dim):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(dim, dim), nn.ReLU())
    def forward(self, x):
        return self.net(x)

class Encoder(nn.Module):
    def __init__(self):
        super().__init__()
        self.blocks = nn.ModuleList([Block(128)
                                     for _ in range(3)])
        self.head = nn.Linear(128, 10)
    def forward(self, x):
        for b in self.blocks:      # the
            x = b(x)               #   the
        return self.head(x)
#   the
#   the
#   the

The Sequential is the no-forward (the nn.Flatten, the Linear, the ReLU, the Linear the list, the model(x) the order) - the simple the one-liner, the named_children the inspect. The sub-module is the complex: the Block (the self.net = nn.Sequential the sub), the Encoder (the self.blocks = nn.ModuleList the 3 the Block), and the parameters is the recursive (the blocks the all, the head the all) - the ModuleList the list the register (the no the plain list, the parameters the miss), and the to(device) is the recursive (the sub the all, the buffer the all) - the next section's train/eval is the mode, and the state_dict the save.

The train/eval and the state_dict

# the TRAIN/EVAL (the):
#   the
model.train()   # the TRAIN (the
                #   the
                #   the)
model.eval()    # the EVAL (the
                #   the
                #   the)
#   the
#   the
#   the

# the STATE_DICT (the):
#   the
sd = model.state_dict()   # the
#   the
#   the
#   the
torch.save(sd, "model.pt")   # the
model.load_state_dict(sd)    # the
#   the
#   the

The train/eval is the mode: the model.train() the BatchNorm the running (the batch the stat) plus the Dropout the on, and the model.eval() the BatchNorm the running (the no batch, the inference) plus the Dropout the off - the predict the eval plus the no_grad (the no graph), the train the train mode. The state_dict is the save: the model.state_dict() the OrderedDict (the fc1.weight, the fc1.bias, the buffer the BatchNorm's), the torch.save the file (the model.pt the weight only, the no code), and the load_state_dict the load (the new the same the structure, the strict=False the partial) - the next chapter's first-model is the Linear the MNIST's, and the training-loop the Module the run.

The Working Rules

  • The __init__ the layer, the forward the operation. The super().__init__() the required, the self.fc1 = nn.Linear the layer, the model(x) the forward (the __call__ the hook) - the parameters the optimizer's target, the numel the count.
  • The Sequential the no-forward, the sub-module the complex. The nn.Sequential the list (the simple), the Block plus the ModuleList the complex (the recursive the parameters, the ModuleList the register, the no the plain list).
  • The train/eval is the mode, the to the move. The BatchNorm the running (the train the batch, the eval the no), the Dropout the on/off, and the to(device) the recursive (the sub the all, the buffer the all).
  • The state_dict is the save, the load_state_dict the load. The OrderedDict the weight plus the buffer (the no the code), the torch.save the file, and the strict=False the partial (the transfer the pretrain) - the save-load chapter the detail.

Summary

The nn.Module is the model's container: the __init__ the layer (the super().__init__() the required, the self.fc1 = nn.Linear), the forward the operation (the flatten, the relu, the fc2), and the model.parameters() the optimizer's target (the weight plus the bias, the numel the count). The Sequential is the no-forward (the list the simple), the sub-module the complex (the Block plus the ModuleList the recursive, the ModuleList the register), and the to(device) the recursive move (the sub the all, the buffer the all). The train/eval is the mode (the BatchNorm the running, the Dropout the on/off, the predict the eval plus the no_grad), and the state_dict is the save (the weight plus the buffer, the no the code, the torch.save the file, the load_state_dict the strict=False the partial). The next chapter's first-model is the Linear the MNIST's, and the training-loop the Module the run - the zero_grad, the forward, the loss, the backward, the step the five the epoch the repeat.