Deep Learning

Deep Learning is a subset of machine learning that uses neural networks with many layers (deep networks). It can handle complex data such as images, text, and audio.

Deep Learning vs Machine Learning

Deep Neural Network

from tensorflow import keras

model = keras.Sequential([
    keras.layers.Dense(256, activation='relu', input_shape=(784,)),
    keras.layers.Dropout(0.3),
    keras.layers.Dense(128, activation='relu'),
    keras.layers.Dropout(0.3),
    keras.layers.Dense(64, activation='relu'),
    keras.layers.Dense(10, activation='softmax')
])

model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['accuracy']
)
model.summary()

GPU Training

import tensorflow as tf

# Check for GPU
print(f"GPU available: {tf.config.list_physical_devices('GPU')}")

# Enable memory growth
gpus = tf.config.experimental.list_physical_devices('GPU')
if gpus:
    for gpu in gpus:
        tf.config.experimental.set_memory_growth(gpu, True)

Framework Overview

TensorFlow

import tensorflow as tf
print(f"TensorFlow version: {tf.__version__}")

PyTorch

# import torch
# print(f"PyTorch version: {torch.__version__}")

Keras

from tensorflow import keras
print(f"Keras version: {keras.__version__}")

Batch Normalization

from tensorflow import keras

model = keras.Sequential([
    keras.layers.Dense(128, input_shape=(100,)),
    keras.layers.BatchNormalization(),
    keras.layers.Activation('relu'),
    keras.layers.Dense(64),
    keras.layers.BatchNormalization(),
    keras.layers.Activation('relu'),
    keras.layers.Dense(10, activation='softmax')
])

Callbacks

from tensorflow import keras

callbacks = [
    keras.callbacks.EarlyStopping(patience=5, restore_best_weights=True),
    keras.callbacks.ReduceLROnPlateau(factor=0.5, patience=3),
    keras.callbacks.ModelCheckpoint('best_model.keras', save_best_only=True),
    keras.callbacks.TensorBoard(log_dir='./logs')
]

When to Use Deep Learning?

• Very large datasets (>10,000 samples)
• Unstructured data (images, text, audio)
• Manual feature extraction is difficult
• Computational resources are available (GPU)
• Complex problems with non-linear patterns

When to Use Traditional ML?

• Small-to-medium datasets
• Model interpretation matters
• Limited resources
• Features are already well defined
• Fast training is needed

Deep Learning Tips

• Start with a simple model
• Use transfer learning for small data
• Use data augmentation for images
• Regularization: Dropout, L2, early stopping
• Monitor validation loss, not training loss
• Use learning rate scheduling

Next chapter: CNN, an architecture designed specifically for image data.

Next: CNN →