Feature Matching

Feature matching connects the keypoints of two images - the same physical points seen twice - and the match quality determines everything downstream: the homography for stitching, the pose for AR, the confirmation that object A in frame 1 is object B in frame 10. The pipeline is detect -> describe -> match -> filter, and the filter is where accuracy is won: a raw matcher returns many false matches, and the ratio test plus RANSAC homography remove them.

Brute-Force Matching

import cv2

img1 = cv2.imread("left.jpg")
img2 = cv2.imread("right.jpg")
gray1 = cv2.cvtColor(img1, cv2.COLOR_BGR2GRAY)
gray2 = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)

orb = cv2.ORB_create(nfeatures=800)
kp1, des1 = orb.detectAndCompute(gray1, None)
kp2, des2 = orb.detectAndCompute(gray2, None)

# binary descriptors -> Hamming distance
bf = cv2.BFMatcher(cv2.NORM_HAMMING)
raw = bf.match(des1, des2)          # list of DMatch, best per point

# the ratio test: keep only matches whose best is clearly better than 2nd
good = []
knn = bf.knnMatch(des1, des2, k=2)
for pair in knn:
    if len(pair) == 2 and pair[0].distance < 0.75 * pair[1].distance:
        good.append(pair[0])
print(f"{len(raw)} raw -> {len(good)} after ratio test")

BFMatcher computes the distance between every descriptor pair; for binary descriptors (ORB) that is Hamming distance, for float descriptors (SIFT) use NORM_L2. match returns the single best per query; knnMatch(k=2) returns two, and the ratio test (good = best < 0.75 x second-best) is the classic false-match filter - a true match is usually far closer to its correct partner than to any impostor, while a random one has its top two close together. 0.75 is the canonical threshold from the Lowe paper.

FLANN: Fast Matching for Large Sets

import cv2

gray1 = cv2.cvtColor(cv2.imread("left.jpg"), cv2.COLOR_BGR2GRAY)
gray2 = cv2.cvtColor(cv2.imread("right.jpg"), cv2.COLOR_BGR2GRAY)
sift = cv2.SIFT_create()
kp1, des1 = sift.detectAndCompute(gray1, None)
kp2, des2 = sift.detectAndCompute(gray2, None)

# float descriptors -> L2, FLANN approximates the nearest neighbor
flann = cv2.FlannBasedMatcher({"algorithm": 1, "trees": 5},
                              {"checks": 50})
pairs = flann.knnMatch(des1, des2, k=2)

good = [m for p in pairs if len(p) == 2
        and p[0].distance < 0.75 * p[1].distance for m in [p[0]]]
print("good:", len(good))

FLANN (Fast Library for Approximate Nearest Neighbors) approximates the k-NN search so it scales to hundreds of thousands of descriptors - the regime where brute force gets slow. It is configured with an index type (algorithm: 1 is KDTree) and a checks parameter that trades speed for recall. For a few thousand points, BFMatcher is simpler and fast enough; FLANN earns its complexity at scale, such as matching a phone image against a large map of landmarks.

From Matches to Geometry: Homography + RANSAC

import cv2
import numpy as np

# assume kp1, kp2, good (ratio-tested matches) are ready
src = np.float32([kp1[m.queryIdx].pt for m in good]).reshape(-1, 1, 2)
dst = np.float32([kp2[m.trainIdx].pt for m in good]).reshape(-1, 1, 2)

# RANSAC: find the homography that fits the inliers, reject outliers
H, mask = cv2.findHomography(src, dst, cv2.RANSAC, 5.0)
inliers = mask.ravel().sum()
print(f"inliers: {inliers}/{len(good)}")

# keep only the inlier matches
good_in = [m for m, keep in zip(good, mask.ravel()) if keep]

img2_r = cv2.warpPerspective(img1, H, (img2.shape[1], img2.shape[0]))
vis = cv2.drawMatches(img1, kp1, img2, kp2, good_in, None,
                      flags=cv2.DRAW_MATCHES_FLAGS_NOT_DRAW_SINGLE_POINTS)
cv2.imwrite("matches.jpg", vis)

findHomography with cv2.RANSAC finds the 3x3 plane mapping that is consistent with the largest subset of matches, treating the rest as outliers - the last and most important filter. The inlier count is your confidence metric: a handful of ratio-passed matches that RANSAC rejects means the images do not share the plane assumption (e.g. the objects moved, or the camera rotated a lot). The accepted mask tells you which matches to keep for drawing or for the warp, and warpPerspective with H is the same call from the transforms chapter - matching is how you obtain its matrix automatically.

The Stitching Application

import cv2

# the library's full pipeline: detect, match, RANSAC, warp, blend
stitcher = cv2.Stitcher_create(cv2.STITCHER_PANORAMA)
status, pano = stitcher.stitch([cv2.imread("a.jpg"),
                                cv2.imread("b.jpg"),
                                cv2.imread("c.jpg")])
if status == cv2.Stitcher_OK:
    cv2.imwrite("pano.jpg", pano)
else:
    print("stitch failed:", status)

cv2.Stitcher wraps exactly this chapter's pipeline - feature matching plus homography RANSAC plus warped blending - into one call, and it is the reference implementation for understanding what the pieces do. When a custom stitch fails, the diagnostic path is the manual one: print the raw match count, the ratio-test survivor count, and the RANSAC inlier count; the stage where the number collapses is the stage that is wrong (poor features, wrong distance norm, or a geometry that is not a plane).

Summary

Match features with BFMatcher (Hamming for ORB, L2 for float descriptors) or FLANN at scale, always run the 0.75 ratio test on knnMatch(k=2) output to cut false matches, then finish with findHomography(..., cv2.RANSAC) - the inlier mask is the final quality gate and mask tells you which matches to trust. The three counts (raw, ratio-passed, RANSAC inliers) are the diagnostic of any matching failure. Above that layer, cv2.Stitcher is the whole pipeline as a product, and the same homography feeds warpPerspective for warping and alignment.