Face and Object Detection

Face detection in classical OpenCV uses cascade classifiers - Haar-like features evaluated in a boosted cascade, fast enough for real-time on a CPU. The modern path is the DNN detector, which is the subject of the next chapter; this one covers the Haar and LBP cascades shipped with OpenCV, how to load and run them, how to tune them, and the same pattern generalized to other objects (eyes, hands, smile) via the bundled XML files.

Haar Cascades

import cv2

# cascade XMLs ship with the package under data/
haar = cv2.CascadeClassifier(
    cv2.data.haarcascades + "haarcascade_frontalface_default.xml")
print(haar.empty())            # False if it loaded

cap = cv2.VideoCapture(0)
while True:
    ret, frame = cap.read()
    if not ret:
        break
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    gray = cv2.equalizeHist(gray)        # helps under uneven light

    faces = haar.detectMultiScale(gray, scaleFactor=1.1,
                                  minNeighbors=5, minSize=(30, 30))
    for (x, y, w, h) in faces:
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    cv2.imshow("Faces", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()

detectMultiScale slides a window across the image at shrinking scales and returns the boxes that pass the boosted classifier. The three knobs: scaleFactor (1.05-1.2, the ratio between consecutive window sizes - lower finds smaller faces but is slower), minNeighbors (how many overlapping detections must agree - higher = fewer false positives, the main tuning dial), and minSize (reject boxes smaller than this). Equalizing the histogram before detection materially improves recall in real lighting. The result is an array of (x, y, w, h) - the same annotation idiom from the drawing chapter renders them.

LBP and the Face-Eye Chain

import cv2

cascade_dir = cv2.data.haarcascades
face_c = cv2.CascadeClassifier(cascade_dir + "haarcascade_frontalface_default.xml")
eye_c = cv2.CascadeClassifier(cascade_dir + "haarcascade_eye.xml")

img = cv2.imread("group.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

faces = face_c.detectMultiScale(gray, 1.1, 5)
for (x, y, w, h) in faces:
    cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)
    # run the eye detector INSIDE the face ROI
    roi_gray = gray[y:y + h, x:x + w]
    roi_color = img[y:y + h, x:x + w]
    eyes = eye_c.detectMultiScale(roi_gray)
    for (ex, ey, ew, eh) in eyes:
        cv2.rectangle(roi_color, (ex, ey), (ex + ew, ey + eh), (255, 0, 0), 2)

cv2.imwrite("faces_eyes.jpg", img)

The pattern generalizes: detectors are cascaded, each stage running on the previous stage's region of interest. Eyes are detected inside the face box, a smile cascade inside the face, and the ROI is passed in the local coordinates the detector expects. This cascade-of-cascades structure is how multi-part detection works before you involve learned networks - the face detector is the gate that makes the eye search cheap.

Tuning and Failure Modes

Symptom	Cause / fix
Missed faces	low light, extreme angle - equalize first; lower `minNeighbors`; check the face is inside `minSize`
Many false positives	raise `minNeighbors` (5-8); tighten `minSize`
Duplicates over one face	raise `minNeighbors`; the cascade merges overlapping boxes as it does
Slow	downscale the frame; raise `scaleFactor` toward 1.2

Haar cascades are honest about their limits: they are frontal-face machines, they degrade sharply at profile angles and strong lighting gradients, and they detect faces but say nothing about identity or expression. For production-grade detection - profiles, occlusion, crowded scenes - the DNN detector in the next chapter is the upgrade; the Haar path remains the right choice when you want zero model files, zero dependencies, and a detector that runs on anything with a CPU.

Other Object Cascades

The haarcascades directory ships detectors beyond faces - upper body, full body, hand, smile, QR-code-like patterns - and the detectMultiScale call is identical for all of them. The same tuning logic applies, and the same ROI-cascade pattern chains them (full body -> face -> eyes). They are the same boosted-Haar engine with different training data, so expect the same strengths (fast, CPU-only, no download) and the same weaknesses (frontal bias, lighting sensitivity). For anything beyond faces, the DNN route is usually worth the model file.

Summary

Detect faces with a loaded CascadeClassifier and detectMultiScale, tuning scaleFactor, minNeighbors (the main false-positive dial), and minSize, and equalizing the gray frame first for real lighting. Chain detectors by running each stage inside the previous stage's ROI to build multi-part detection, and read the failure table - misses want pre-processing and lower neighbor counts, false positives want higher ones. Haar cascades are the zero-dependency baseline: fast and frontal, and the DNN chapter is the upgrade when accuracy across angles and scenes is the requirement.