Skillversitas

Data Pipelines and Augmentation

The pipeline is the transform plus the DataLoader plus the worker: the Resize the target, the ColorJitter the light, the ToTensor the 0 to the 1, the Normalize the mean/std (the ImageNet the 0.485), and the worker the process (the GPU the feed). The augment is the regularize the data (the flip, the crop, the noise, the auto the RandAugment), the mixup the 2 the sample (the label the smooth), and the worker the collate (the padding the variable). This chapter covers the transform (the Compose, the Random, the Normalize), the augment (the RandAugment, the mixup), the worker (the num_workers, the pin_memory), and the collate (the padding).

The transform: The Compose and the Normalize

# the TRANSFORM (the):
import torchvision.transforms as T

train_tf = T.Compose([
    T.Resize((224, 224)),       # the
    T.RandomHorizontalFlip(),   #   the
    T.ColorJitter(0.4, 0.4, 0.4, 0.1),  # the the
    T.RandomRotation(15),       #   the
    T.ToTensor(),               #   the
    T.Normalize((0.485, 0.456, 0.406),
                (0.229, 0.224, 0.225)),  # the the
    T.RandomErasing(p=0.25),    #   the
])
test_tf = T.Compose([
    T.Resize((224, 224)),
    T.ToTensor(),
    T.Normalize((0.485, 0.456, 0.406),
                (0.229, 0.224, 0.225)),
])
#   the
#   the
#   the

The transform is the pipeline: the Resize the 224 (the ImageNet the input), the RandomHorizontalFlip the 0.5 (the 2x the data), the ColorJitter the light (the 0.4 the brightness/contrast/saturation, the 0.1 the hue), the RandomRotation the 15 (the angle), the ToTensor the 0 to the 1, the Normalize the ImageNet the mean/std (the 0.485 the pretrain the match), and the RandomErasing the 0.25 (the patch the mask, the attention the force) - the test the no the Random (the deterministic, the eval the fair), and the pretrain the match (the fine-tune the Normalize the same the ImageNet) - the next section's RandAugment is the auto, and the mixup the label the smooth.

The RandAugment and the mixup

# the RANDAUGMENT (the):
#   the
tf = T.Compose([
    T.Resize((224, 224)),
    T.RandAugment(num_ops=2, magnitude=9),
    T.ToTensor(),
    T.Normalize((0.485, 0.456, 0.406),
                (0.229, 0.224, 0.225)),
])
#   the
#   the

# the MIXUP (the):
#   the
def mixup(x, y, alpha=0.2):
    lam = torch.distributions.Beta(alpha, alpha) \
        .sample().to(x.device)
    idx = torch.randperm(x.size(0), device=x.device)
    x_mix = lam * x + (1 - lam) * x[idx]
    y_a, y_b = y, y[idx]
    return x_mix, y_a, y_b, lam
#   the
#   the
#   the

The RandAugment is the auto: the num_ops=2 the magnitude=9 (the 16 the operation the random the 2 the pick, the 9 the strength the 10), and the policy the search the no (the auto the hand the no, the CIFAR the 1% the gain) - the train the only (the test the no). The mixup is the label the smooth: the 2 the sample the lam the mix (the Beta(0.2, 0.2) the lam), the loss the lam * loss(y_a) + (1-lam) * loss(y_b) (the label the soft, the confident the over the guard), and the cutmix the patch the swap (the spatial the mix) - the worker is the feed (the num_workers the 4, the pin_memory the CUDA, the prefetch_factor the 2), and the collate is the padding (the variable the seq, the max the pad, the length the mask) - the next chapter's save-load is the checkpoint the resume the train's tool.

The Working Rules

  • The Compose is the pipeline, the Random the train. The Resize the 224, the Flip the 0.5, the ColorJitter the light, the ToTensor the 0-1, the Normalize the ImageNet (the pretrain the match), and the test the no the Random (the deterministic, the eval the fair).
  • The RandAugment is the auto, the gain the 1%. The num_ops=2 the magnitude=9 (the 16 the operation the random), the policy the search the no (the auto the hand), and the CIFAR the 1% the gain (the train the only, the test the no).
  • The mixup is the label, the cutmix the patch. The 2 the sample the lam (the Beta the 0.2), the loss the lam the mix (the label the soft, the confident the over the guard), and the cutmix the spatial the swap (the patch the replace).
  • The worker is the feed, the collate the padding. The num_workers the 4 (the GPU the feed), the pin_memory the CUDA (the page-locked), the prefetch the 2, and the collate the variable (the max the pad, the length the mask) - the RNN the seq the pad.

Summary

The pipeline is the transform plus the DataLoader: the Resize the 224, the RandomHorizontalFlip the 0.5, the ColorJitter the light, the ToTensor the 0-1, the Normalize the ImageNet (the pretrain the match), and the RandomErasing the 0.25 - the test the no the Random (the deterministic). The RandAugment is the auto (the num_ops=2 the magnitude=9, the CIFAR the 1% the gain), the mixup the label the smooth (the 2 the sample the lam, the loss the mix), and the cutmix the patch the swap. The worker is the feed (the num_workers the 4, the pin_memory the CUDA, the prefetch the 2), and the collate is the padding (the variable the seq, the max the pad, the length the mask). The next chapter's save-load is the checkpoint: the state_dict the weight, the optimizer the momentum, the epoch the resume, and the full the torch.save(model) the quick the no the recommend.