Skillversitas

Convolutional Networks in Detail

The detail is the Conv2d's kernel, the stride, and the padding: the out = (in + 2*padding - kernel) / stride + 1 the shape the formula, the 3x3 the standard (the 7x7 the ImageNet's first, the 1x1 the channel the mix), and the weight the (out_C, in_C, kH, kW) the param the count. The receptive field is the spatial the neuron sees (the layer the grow, the scale the context), the weight share is the translation the invariant (the param the small, the edge the any the pos), and the 1x1 conv is the FC the channel (the ResNet's bottleneck). This chapter covers the shape (the formula, the track), the kernel (the 3x3, the 1x1, the 7x7), the receptive field (the grow), and the param (the count, the 1x1 the cheap).

The shape: The formula

# the SHAPE (the):
#   the
#   out = (in + 2*padding - kernel) / stride + 1
import torch.nn as nn

#   the
c1 = nn.Conv2d(1, 32, 3, stride=1, padding=1)
#   the (N,1,28,28) -> the (N,32,28,28) (the the
#   the)

c2 = nn.Conv2d(32, 64, 3, stride=1, padding=1)
#   the (N,32,28,28) -> the (N,64,28,28) (the the
#   the)

c3 = nn.Conv2d(64, 128, 3, stride=2, padding=1)
#   the (N,64,28,28) -> the (N,128,14,14) (the the
#   the)

p1 = nn.MaxPool2d(2, stride=2)
#   the (N,128,14,14) -> the (N,128,7,7) (the the
#   the)
#   the
#   the

The shape is the formula: the out = (in + 2*padding - kernel) / stride + 1 (the 28 the 3 the 1 the 1 the 28, the stride 2 the 14), and the track is the design (the pool the half, the fc the size the know). The kernel is the 3x3 (the standard, the 2 the 3x3 the 4x4 the receptive the same the param the less), the 1x1 (the channel the mix, the FC the cheap, the ResNet's bottleneck the 256 to the 64), and the 7x7 (the ImageNet's first, the GoogLeNet the 7x7 the 3x3 the split the param the cut) - the stride the down (the 2 the half, the no the pool the replace), and the padding the size the keep (the same the kernel//2) - the next section's receptive field is the context, and the param the count.

The receptive field and the param

# the RECEPTIVE (the):
#   the
#   the
#   the
#   the
#   the

# the PARAM (the):
c = nn.Conv2d(64, 128, 3)   # the
n = sum(p.numel() for p in c.parameters())
print(n)   # the the 73856 (the
           #   the
           #   the)
#   the 128*64*3*3 = the 73728 + the 128 (the the
#   the)

c1x1 = nn.Conv2d(256, 64, 1)   # the
n1 = sum(p.numel() for p in c1x1.parameters())
print(n1)   # the the 16448 (the
            #   the)
#   the
#   the

The receptive field is the spatial the neuron sees: the 3x3 the 1 layer the 3x3, the 2 layer the 5x5, the 3 layer the 7x7 (the layer the grow, the stride the jump) - the deep the large the context, the CNN's power the local the layer the stack. The weight share is the translation the invariant: the 3x3 the 64 channel the 576 param (the FC the 28*28*64 the 500K), and the edge the any the pos (the param the small, the invariant the learn). The param is the count: the Conv2d(64, 128, 3) the 73856 (the 128*64*9 plus the 128), and the Conv2d(256, 64, 1) the 16448 (the 1x1 the cheap, the channel the mix the spatial the no) - the resnet chapter the bottleneck the 1x1 the 3 the stack, and the rule is the one: the 3x3 the standard, the 1x1 the channel, the stride the down, the padding the keep - the next chapter's transfer-learning is the pretrain the fine-tune the real's path.

The Working Rules

  • The shape is the formula, the track the design. The out = (in + 2*padding - kernel) / stride + 1 (the 28 the 3 the 1 the 1 the 28), and the pool the half (the fc the size the know) - the design the shape the plan the before the code.
  • The 3x3 is the standard, the 1x1 the channel. The 2 the 3x3 the 4x4 the receptive the same the param the less, the 1x1 the FC the channel (the ResNet's bottleneck the 256 to the 64), and the 7x7 the ImageNet's first the 3x3 the split.
  • The stride is the down, the padding the keep. The stride 2 the half (the no the pool the replace), and the padding = kernel//2 the size the keep (the same) - the down the stride or the pool, the choice the model's.
  • The receptive is the grow, the param the count. The layer the grow (the 3x3 the 5x5 the 7x7), the weight share the translation (the 576 vs the FC's 500K), and the Conv2d(64,128,3) the 73856 (the 1x1 the 16448 the cheap) - the resnet the bottleneck the 1x1 the stack.

Summary

The detail is the Conv2d's three: the kernel (the 3x3 the standard, the 1x1 the channel the mix, the 7x7 the ImageNet's first), the stride (the 2 the half, the no the pool the replace), and the padding (the kernel//2 the size the keep) - the shape the formula (the out = (in + 2*padding - kernel) / stride + 1), the weight the (out_C, in_C, kH, kW). The receptive field is the grow (the layer the 3x3 the 5x5 the 7x7, the deep the context), the weight share is the translation the invariant (the 576 vs the FC's 500K), and the param is the count (the Conv2d(64,128,3) the 73856, the 1x1 the 16448 the cheap). The ResNet is the bottleneck (the 1x1 the 3 the stack, the 256 to the 64 the param the cut), and the next chapter's transfer-learning is the pretrain the fine-tune - the ImageNet's weight the your data the few the train, the real's path the from-scratch the alternative.