Docker Logging

Logging in Docker enables monitoring and debugging of container activity.

Logging Driver Types

1. JSON File (Default)

# View container logs
docker logs container_name

# Real-time logs
docker logs -f container_name

# Logs with timestamps
docker logs -t container_name

# Last 100 lines of logs
docker logs --tail 100 container_name

2. Syslog

# Use the syslog driver
docker run -d --log-driver=syslog \
  --log-opt syslog-address=tcp://192.168.1.100:514 \
  myapp:latest

3. Fluentd

# Use the fluentd driver
docker run -d --log-driver=fluentd \
  --log-opt fluentd-address=localhost:24224 \
  myapp:latest

Use Case: Application Logging

version: '3.8'

services:
  api:
    build: ./backend
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"
    networks:
      - backend

  database:
    image: postgres:15
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "5"
    networks:
      - backend

  nginx:
    image: nginx:alpine
    logging:
      driver: json-file
      options:
        max-size: "20m"
        max-file: "3"
    networks:
      - frontend

networks:
  frontend:
  backend:

Log Analysis

# Search for errors in the logs
docker logs api 2>&1 | grep -i error

# Export logs to a file
docker logs api > api.log 2>&1

# Logs with grep
docker logs api 2>&1 | grep "ERROR"

# Logs with timestamps
docker logs -t api 2>&1 | tail -100

Centralized Logging

# ELK Stack (Elasticsearch, Logstash, Kibana)
version: '3.8'

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.10.0
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
    volumes:
      - es-data:/usr/share/elasticsearch/data

  logstash:
    image: docker.elastic.co/logstash/logstash:8.10.0
    volumes:
      - ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf

  kibana:
    image: docker.elastic.co/kibana/kibana:8.10.0
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch

  api:
    build: ./backend
    logging:
      driver: fluentd
      options:
        fluentd-address: localhost:24224

volumes:
  es-data:

Everyday Use Case

Case: A developer needs to debug an error in the application.

# View real-time logs
docker logs -f api

# Search for errors
docker logs api 2>&1 | grep -i "error\|exception"

# View logs from the last hour
docker logs --since=1h api

# View specific logs
docker logs api 2>&1 | grep "2024-01-15"

Best Practices

1. Use log rotation: Configure max-size and max-file
2. Send to centralized logging: ELK or similar
3. Use structured logging: JSON format
4. Monitor logs: Watch for errors and warnings
5. Back up logs: Store important logs

Use Case: Monitoring

# Export logs for analysis
docker logs api > api_$(date +%Y%m%d).log 2>&1

# Count errors
docker logs api 2>&1 | grep -i error | wc -l

# View the latest logs
docker logs --tail 50 api

Troubleshooting

Logs not appearing

# Check the logging driver
docker inspect --format='{{.HostConfig.LogConfig.Type}}' api

# Check the log location
docker inspect --format='{{.LogPath}}' api

Logs full

# Clean up old logs
docker system prune -f

# Delete logs manually
sudo truncate -s 0 /var/lib/docker/containers/*/*-json.log

Conclusion

Logging is essential for monitoring and debugging. Use log rotation and centralized logging for effective log management.