Skillversitas

Mouse Events and Callbacks

OpenCV windows can report mouse activity through a registered callback: setMouseCallback hands your function every event - button press, release, move, double-click - with the position and the event flags. The classic first interactive tool is the drag-to-select rectangle: press, drag, release, and you have a region of interest. The pattern generalizes to anything where the user points at the image: marking landmarks, choosing crop regions, painting masks.

Registering a Callback

import cv2

img = cv2.imread("photo.jpg")

def on_mouse(event, x, y, flags, param):
    # event is one of the cv2.EVENT_* constants
    if event == cv2.EVENT_LBUTTONDOWN:
        print(f"down at ({x}, {y})")
    elif event == cv2.EVENT_MOUSEMOVE:
        pass                          # moves can be high-frequency; keep it light
    elif event == cv2.EVENT_LBUTTONUP:
        print(f"up at ({x}, {y})")

cv2.namedWindow("Interactive", cv2.WINDOW_NORMAL)
cv2.setMouseCallback("Interactive", on_mouse)
cv2.imshow("Interactive", img)
cv2.waitKey(0)
cv2.destroyAllWindows()

The callback receives (event, x, y, flags, param): the event constant, the pixel position, modifier flags, and a user parameter you can pass to setMouseCallback (handy for state). The event constants cover the button lifecycle per button (EVENT_LBUTTONDOWN/UP, the R and M variants), EVENT_MOUSEMOVE, EVENT_DBLCLK, and wheel events. The window name in setMouseCallback must match the imshow/namedWindow name - the callback is per-window.

Drag-to-Select a Rectangle

import cv2
import numpy as np

img = cv2.imread("photo.jpg")
drawing = False
start = None

def on_mouse(event, x, y, flags, param):
    global drawing, start
    if event == cv2.EVENT_LBUTTONDOWN:
        drawing = True
        start = (x, y)
    elif event == cv2.EVENT_MOUSEMOVE:
        if drawing:
            view = img.copy()
            cv2.rectangle(view, start, (x, y), (0, 255, 0), 2)
            cv2.imshow("Interactive", view)
    elif event == cv2.EVENT_LBUTTONUP:
        drawing = False
        x1, y1 = x, y
        x0, y0 = sorted((start[0], x1))
        y0, y1 = sorted((start[1], y1))
        cv2.imwrite("selection.png", img[y0:y1, x0:x1])

cv2.namedWindow("Interactive", cv2.WINDOW_NORMAL)
cv2.setMouseCallback("Interactive", on_mouse)
cv2.imshow("Interactive", img)
cv2.waitKey(0)
cv2.destroyAllWindows()

The pattern: capture the start point on LBUTTONDOWN, redraw a copy of the image with a provisional rectangle on each MOUSEMOVE (draw on a copy - never accumulate on the original, or the rectangle smears), and finalize the region on LBUTTONUP. Normalize the coordinates (the release can be up-left of the press) before slicing, because NumPy slices assume start < stop. Showing a copy() each move is cheap enough for interactive use and keeps the source image pristine.

Keeping State Clean

  • Draw on a copy. The preview rectangle goes on img.copy(); the original only changes when the user commits.
  • Normalize corners. Sort the press/release coordinates so the ROI slice is always valid in either drag direction.
  • Use the param slot or a small object for anything beyond two variables - a dict or a tiny class beats a pile of globals as the interaction grows (multiple selections, a mode switch, a history).
  • Guard high-frequency events. MOUSEMOVE fires many times per frame; keep that branch to the redraw and move real work to the release.

What the Pattern Becomes

Once the drag-select works, the same callback structure drives: landmark tools (click to place a point, store a list), mask painting (left-drag paints, right-drag erases - two buttons, two operations), measurement (two clicks, draw the line, report the pixel distance), and calibration point pickers (click the checkerboard corners by hand). The callback is the input layer of any image-interaction tool, and every one of these is the drag-select skeleton with a different action on LBUTTONUP.

Summary

Attach setMouseCallback(window, fn) to handle (event, x, y, flags, param), and build interactions from the drag-select skeleton: start on EVENT_LBUTTONDOWN, preview on a copy during EVENT_MOUSEMOVE, commit and normalize the coordinates on EVENT_LBUTTONUP. Keep the move branch cheap, keep state explicit, and use the param slot as the interaction grows. The pattern generalizes to every pointing tool - landmarks, masks, measurement, calibration - which makes it one of the highest-leverage chapters for building your own vision utilities.