Deep Learning Basics

Deep Learning (DL) is a subset of machine learning that uses neural networks with many layers. It excels at complex tasks like image recognition, speech recognition, and natural language processing.

What Is a Neural Network?

A neural network consists of layers of interconnected neurons. Each neuron receives inputs, applies weights, adds a bias, and passes through an activation function.

import torch
import torch.nn as nn

class SimpleNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(4, 10)
        self.relu = nn.ReLU()
        self.fc2 = nn.Linear(10, 3)

    def forward(self, x):
        x = self.relu(self.fc1(x))
        x = self.fc2(x)
        return x

model = SimpleNet()
print(model)

Activation Functions

Function	Formula	Use Case
ReLU	max(0, x)	Hidden layers
Sigmoid	1 / (1 + e^-x)	Binary classification output
Softmax	e^x / sum(e^x)	Multi-class classification output
Tanh	(e^x - e^-x) / (e^x + e^-x)	Hidden layers (alternative to ReLU)

Train with PyTorch

import torch.optim as optim
import torch.nn.functional as F

criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

for epoch in range(100):
    optimizer.zero_grad()
    outputs = model(X_train_tensor)
    loss = criterion(outputs, y_train_tensor)
    loss.backward()
    optimizer.step()
    
    if epoch % 20 == 0:
        print(f"Epoch {epoch}, Loss: {loss.item():.4f}")

Summary

You learned deep learning fundamentals and built your first neural network with PyTorch. Next, we'll explore neural networks in more depth.