Skillversitas

Saving and Loading Models

The save is the state_dict (the weight plus the buffer, the no the code): the torch.save(model.state_dict(), "m.pt") the weight only, the load_state_dict the new the same (the structure the match), and the strict=False the partial (the pretrain the head the no the match). The full (the torch.save(model)) is the quick (the code the pickle, the path the depend, the no the recommend), the checkpoint is the resume (the optimizer the momentum, the epoch, the best), and the Hub is the remote (the torch.hub the repo the load). This chapter covers the state_dict (the save, the load, the strict), the full (the pickle, the no), the checkpoint (the resume, the optimizer), and the Hub (the remote, the repo).

The state_dict: The save and the load

# the SAVE (the):
#   the
torch.save(model.state_dict(), "model.pt")
#   the
#   the

# the LOAD (the):
#   the
model = CNN()                        # the
model.load_state_dict(
    torch.load("model.pt"))          # the
model.eval()                         #   the
#   the
#   the

# the STRICT (the):
#   the
model = CNN()
model.fc = nn.Linear(3136, 100)   # the
model.load_state_dict(
    torch.load("model.pt"),
    strict=False)   #   the
#   the
#   the

The state_dict is the save: the model.state_dict() the OrderedDict (the fc1.weight, the fc1.bias, the BatchNorm the running), the torch.save the file (the model.pt the weight only, the no the code, the 100MB the CNN), and the load_state_dict the load (the new the same the structure, the device the map the torch.load(..., map_location="cpu")). The strict=False is the partial: the pretrain the head the no the match (the 1000 to the 10 the fc the skip), the missing plus the unexpected the return (the inspect the log), and the shape the no the match the error (the 10 to the 100 the strict the catch) - the next section's full is the quick the no the recommend, and the checkpoint the resume.

The checkpoint and the Hub

# the CHECKPOINT (the):
#   the
torch.save({
    "epoch": epoch,
    "model": model.state_dict(),
    "opt": opt.state_dict(),
    "best": best_acc,
}, "ckpt.pt")   # the

# the RESUME (the):
#   the
ckpt = torch.load("ckpt.pt")
model.load_state_dict(ckpt["model"])
opt.load_state_dict(ckpt["opt"])
start = ckpt["epoch"] + 1   #   the
#   the
#   the

# the HUB (the):
#   the
model = torch.hub.load(
    "pytorch/vision", "resnet50",
    pretrained=True)   # the
#   the

The checkpoint is the resume: the torch.save the dict (the epoch, the model the state_dict, the opt the state_dict (the momentum the Adam's 2nd), the best), and the load the resume (the start the epoch+1, the train the no the restart) - the epoch the save (the 1 the big the disk the balance), the best the keep (the test_acc the argmax), and the full is the quick (the torch.save(model) the code the pickle, the path the depend, the version the break, the no the recommend) - the Hub is the remote: the torch.hub.load("pytorch/vision", "resnet50", pretrained=True) the repo the load (the weight the download the auto, the transform the Hub the also), and the trust_remote_code=True the custom the allow (the HuggingFace the model) - the next chapter's regularization is the Dropout the weight decay the early stop the overfit's guard.

The Working Rules

  • The state_dict is the save, the weight only. The OrderedDict the fc1.weight plus the buffer (the no the code), the torch.save the file, and the load_state_dict the new the same (the structure the match, the map_location the device) - the 100MB the CNN, the weight the no the code.
  • The strict=False is the partial, the pretrain's. The head the no the match (the 1000 to the 10 the fc the skip), the missing plus the unexpected the return (the inspect the log), and the shape the no the match the error (the strict the catch) - the transfer the tool.
  • The checkpoint is the resume, the full the no. The dict the epoch plus the model plus the opt (the momentum) plus the best, the load the resume (the epoch+1), and the torch.save(model) the pickle (the code the path the depend, the version the break, the no the recommend) - the quick the debug, the no the ship.
  • The Hub is the remote, the repo the load. The torch.hub.load("pytorch/vision", "resnet50") the weight the download (the auto, the transform the also), and the trust_remote_code=True the custom (the HuggingFace) - the pretrain the no the code, the one-line the load.

Summary

The save is the state_dict: the model.state_dict() the OrderedDict (the weight plus the buffer, the no the code), the torch.save the file (the model.pt), and the load_state_dict the load (the new the same the structure, the map_location the device) - the strict=False the partial (the pretrain the head the no the match, the missing the return). The checkpoint is the resume: the dict (the epoch, the model, the opt the momentum, the best) the save the epoch, the load the resume (the epoch+1) - the full (the torch.save(model) the pickle, the path the depend) the quick the no the recommend. The Hub is the remote: the torch.hub.load("pytorch/vision", "resnet50") the weight the download (the auto, the transform the also), the trust_remote_code the custom (the HuggingFace). The next chapter's regularization is the overfit's guard: the Dropout the neuron the mask, the weight decay the L2, the early stop the val the stuck, and the data the augment the best.