Vision API¶

vision ¶

Nectar SDK - Vision module.

AdaptiveHoughLinesP ¶

Bases: ILineEstimationMethod

Adaptive Hough transform with dynamic parameter tuning.

Automatically adjusts threshold and line length parameters based on image characteristics.

estimate `staticmethod` ¶

estimate(img_detect: ndarray, img_out: ndarray, offset: tuple, draw: bool = True, draw_color=None) -> Tuple[float, float, float, float, float, int, int, int, int, ndarray]

Detect line using adaptive Hough transform.

Parameters:

Name	Type	Description	Default
`img_detect`	`ndarray`	Binary image to detect lines in.	required
`img_out`	`ndarray`	Output image to draw on.	required
`offset`	`tuple`	Offset values for drawing.	required
`draw`	`bool`	Whether to draw the detected line.	`True`
`draw_color`	`tuple`	BGR color tuple for drawing.	`None`

Returns:

Type	Description
`tuple`	(center_x, center_y, angle, width, height, x, y, w, h, rotated_box)

Aruco ¶

Aruco(marker_dict: int, tag_size: float)

ArUco marker detector with pose estimation.

Detects ArUco markers and estimates their 3D pose relative to a calibrated camera.

Parameters:

Name	Type	Description	Default
`marker_dict`	`int`	ArUco dictionary size (e.g., 4, 5, 6, 7 for 4x4, 5x5, etc.).	required
`tag_size`	`float`	Physical size of the marker in meters.	required

Attributes:

Name	Type	Description
`camera_matrix`	`ndarray`	Camera intrinsic matrix from calibration.
`camera_distortion`	`ndarray`	Camera distortion coefficients.
`aruco_detect`	`Dictionary`	ArUco dictionary for detection.
`aruco_param`	`DetectorParameters`	Detection parameters.

total_markers `property` ¶

total_markers: int

int: Maximum number of markers in dictionary.

marker_dict `property` ¶

marker_dict: int

int: ArUco dictionary size.

tag_size `property` ¶

tag_size: float

float: Physical marker size in meters.

aruco_config ¶

aruco_config(marker_dict: int, tag_size: float) -> None

Reconfigure detector with new dictionary and tag size.

Parameters:

Name	Type	Description	Default
`marker_dict`	`int`	New ArUco dictionary size.	required
`tag_size`	`float`	New physical marker size in meters.	required

detect ¶

detect(img: ndarray, draw: bool = False)

Detect ArUco markers in image.

Parameters:

Name	Type	Description	Default
`img`	`ndarray`	Input BGR image.	required
`draw`	`bool`	Whether to draw detected markers on image.	`False`

Returns:

Name	Type	Description
`bbox`	`tuple`	Bounding box corners for each detected marker.
`ids`	`int or None`	ID of the first detected marker, or None if no markers found.

calculateYawFromCorners ¶

calculateYawFromCorners(bbox) -> float

Calculate yaw angle from marker corners.

Parameters:

Name	Type	Description	Default
`bbox`	`tuple`	Marker bounding box corners.	required

Returns:

Type	Description
`float`	Yaw angle in degrees (0-360).

pose_estimate ¶

pose_estimate(img: ndarray, draw: bool = False)

Estimate marker pose in camera frame.

Parameters:

Name	Type	Description	Default
`img`	`ndarray`	Input BGR image.	required
`draw`	`bool`	Whether to draw pose axes on image.	`False`

Returns:

Name	Type	Description
`id`	`int or None`	Detected marker ID.
`translation`	`ndarray or None`	Translation vector [x, y, z] in meters.
`yaw`	`float or None`	Yaw angle in degrees.

ColorDetector ¶

ColorDetector(mode: str = 'track', color: str = None, color_space: ColorSpace = HSV)

Detect and filter colors from images using configurable color spaces.

Supports two modes: - "track": Interactive trackbar-based calibration - "preset": Load pre-calibrated values from JSON file

Parameters:

Name	Type	Description	Default
`mode`	`str`	Operation mode ("track" or "preset").	`"track"`
`color`	`str`	Color name to load from calibration file (required for "preset" mode).	`None`
`color_space`	`ColorSpace`	Color space for detection.	`ColorSpace.HSV`

Attributes:

Name	Type	Description
`mask`	`ndarray`	Binary mask of detected color regions.
`result`	`ndarray`	Original image with non-matching regions masked out.
`color_values`	`list`	Current color range as [[min], [max]].

color_values `property` `writable` ¶

color_values

Current color range values [[min1, min2, min3], [max1, max2, max3]].

empty ¶

empty(a)

Trackbar callback placeholder.

initTrackbars ¶

initTrackbars() -> None

Create OpenCV window with trackbars for color adjustment.

Creates 6 trackbars for min/max values of each color channel based on the selected color space.

getTrackValues ¶

getTrackValues() -> list

Read current trackbar positions.

Returns:

Type	Description
`list`	Color range as [[min1, min2, min3], [max1, max2, max3]].

filterColor ¶

filterColor(img: ndarray) -> None

Filter image to isolate pixels within color range.

Applies color space conversion, thresholding, and morphological operations to create a clean binary mask.

Parameters:

Name	Type	Description	Default
`img`	`ndarray`	Input BGR image.	required

Notes

Updates self.mask and self.result attributes.

get_color_values ¶

get_color_values(color_name: str) -> list

Load color values from calibration JSON file.

Parameters:

Name	Type	Description	Default
`color_name`	`str`	Name of the color to retrieve.	required

Returns:

Type	Description
`list`	Color range as [[min1, min2, min3], [max1, max2, max3]].

Raises:

Type	Description
`ValueError`	If file not found, color not defined, or invalid JSON.

saveColorValues ¶

saveColorValues() -> None

Save current color values to calibration JSON file.

ColorSpace ¶

Bases: Enum

Supported color spaces for detection.

Attributes:

Name	Type	Description
`HSV`	`auto`	Hue-Saturation-Value color space.
`LAB`	`auto`	CIELAB color space (Lab*).

DistanceEstimator ¶

DistanceEstimator(model_type: Union[ModelType, str] = None, params_path: Optional[Path] = None, custom_params: Optional[Dict[str, Any]] = None)

Estimate distances from pixel measurements using calibrated models.

Parameters:

Name	Type	Description	Default
`model_type`	`Union[ModelType, str]`	Model to use for estimation. If None, uses default from config.	`None`
`params_path`	`Path`	Path to parameters YAML file. If None, uses default location.	`None`
`custom_params`	`Dict[str, Any]`	Override parameters for specific models.	`None`

Attributes:

Name	Type	Description
`model`	`EstimationModel`	Currently active estimation model.
`model_type`	`ModelType`	Type of the active model.

Examples:

>>> estimator = DistanceEstimator(model_type="polynomial")
>>> estimator.estimate(21.6)  # pixels -> cm
97.7

>>> estimator.compare_methods(21.6)
{'linear': 101.9, 'polynomial': 97.7, 'exponential': 97.8, ...}

model `property` ¶

model: EstimationModel

Currently active estimation model instance.

model_type `property` `writable` ¶

model_type: ModelType

Type of the currently active model.

estimate ¶

estimate(value: float, model_type: Optional[Union[ModelType, str]] = None) -> float

Estimate distance from pixel measurement.

Parameters:

Name	Type	Description	Default
`value`	`float`	Pixel measurement (e.g., object height in pixels).	required
`model_type`	`Union[ModelType, str]`	Override model type for this estimation. If None, uses default.	`None`

Returns:

Type	Description
`float`	Estimated distance in calibrated units (typically cm).

Examples:

>>> estimator.estimate(21.6)
97.7
>>> estimator.estimate(21.6, model_type="linear")
101.9

estimate_batch ¶

estimate_batch(values: ndarray, model_type: Optional[Union[ModelType, str]] = None) -> ndarray

Estimate distances for multiple pixel measurements.

Parameters:

Name	Type	Description	Default
`values`	`ndarray`	Array of pixel measurements.	required
`model_type`	`Union[ModelType, str]`	Override model type. If None, uses default.	`None`

Returns:

Type	Description
`ndarray`	Array of estimated distances.

Examples:

>>> pixels = np.array([15, 20, 25, 30])
>>> estimator.estimate_batch(pixels)
array([173.3, 112.5, 73.2, 53.4])

compare_methods ¶

compare_methods(value: float) -> Dict[str, float]

Compare estimates from all available models.

Parameters:

Name	Type	Description	Default
`value`	`float`	Pixel measurement to estimate.	required

Returns:

Type	Description
`Dict[str, float]`	Dictionary mapping model names to estimated distances.

Examples:

>>> estimator.compare_methods(21.6)
{'linear': 101.9, 'polynomial': 97.7, 'exponential': 97.8, ...}

get_model ¶

get_model(model_type: Union[ModelType, str]) -> EstimationModel

Get specific model instance.

Parameters:

Name	Type	Description	Default
`model_type`	`Union[ModelType, str]`	Model type to retrieve.	required

Returns:

Type	Description
`EstimationModel`	The requested model instance.

set_model_params ¶

set_model_params(model_type: Union[ModelType, str], params: Dict[str, Any]) -> None

Update parameters for a specific model.

Parameters:

Name	Type	Description	Default
`model_type`	`Union[ModelType, str]`	Model to update.	required
`params`	`Dict[str, Any]`	New parameter values.	required

available_methods `staticmethod` ¶

available_methods() -> list

Get list of available estimation methods.

Returns:

Type	Description
`list`	List of model type names (lowercase).

FitEllipse ¶

Bases: ILineEstimationMethod

Ellipse fitting line detection.

Fits an ellipse to the detected contour for curved line estimation.

estimate `staticmethod` ¶

estimate(img_detect: ndarray, img_out: ndarray, offset: tuple, draw: bool = True, draw_color=None) -> Tuple[float, float, float, float, float, int, int, int, int, ndarray]

Detect line by fitting ellipse to contour.

Parameters:

Name	Type	Description	Default
`img_detect`	`ndarray`	Binary image to detect ellipse in.	required
`img_out`	`ndarray`	Output image to draw on.	required
`offset`	`tuple`	Offset values for drawing.	required
`draw`	`bool`	Whether to draw the detected ellipse.	`True`
`draw_color`	`tuple`	BGR color tuple for drawing.	`None`

Returns:

Type	Description
`tuple`	(center_x, center_y, angle, width, height, x, y, w, h, rotated_box)

HoughLinesP ¶

Bases: ILineEstimationMethod

Probabilistic Hough transform line detection.

Uses cv2.HoughLinesP combined with cv2.fitLine for robust line estimation from multiple detected segments.

estimate `staticmethod` ¶

estimate(img_detect: ndarray, img_out: ndarray, offset: tuple, draw: bool = True, draw_color=None) -> Tuple[float, float, float, float, float, int, int, int, int, ndarray]

Detect line using probabilistic Hough transform.

Parameters:

Name	Type	Description	Default
`img_detect`	`ndarray`	Binary image to detect lines in.	required
`img_out`	`ndarray`	Output image to draw detected lines on.	required
`offset`	`tuple`	Offset values for drawing.	required
`draw`	`bool`	Whether to draw the detected line.	`True`
`draw_color`	`tuple`	BGR color tuple for drawing.	`None`

Returns:

Type	Description
`tuple`	(center_x, center_y, angle, width, height, x, y, w, h, rotated_box)

ILineEstimationMethod ¶

Bases: ABC

Abstract interface for line estimation strategies.

All estimation methods must implement the estimate() method to detect lines from binary masks.

estimate `abstractmethod` `staticmethod` ¶

estimate(img_detect: ndarray, img_out: ndarray, offset: tuple, draw: bool = True, draw_color=None) -> Tuple[float, float, float, float, float, int, int, int, int, ndarray]

Estimate a line in the image.

Parameters:

Name	Type	Description	Default
`img_detect`	`ndarray`	Binary image to detect the line in.	required
`img_out`	`ndarray`	Output image to draw on.	required
`offset`	`tuple`	Offset values (x, y) for drawing.	required
`draw`	`bool`	Whether to draw visualization.	`True`
`draw_color`	`tuple`	BGR color tuple for drawing.	`None`

Returns:

Type	Description
`tuple`	(center_x, center_y, angle, width, height, x, y, w, h, rotated_box) - center_x, center_y: Line center coordinates - angle: Line angle in degrees - width, height: Average line dimensions in pixels - x, y, w, h: Bounding box coordinates - rotated_box: Points of the rotated bounding box

LineDetector ¶

LineDetector(color='blue', estimation_method: ILineEstimationMethod = HoughLinesP, color_space=None)

High-level line detector with color filtering.

Combines color detection with line estimation methods for detecting colored lines in images.

Parameters:

Name	Type	Description	Default
`color`	`str`	Color name to detect (must exist in calibration file). If None, uses external_mask for detection.	`'blue'`
`estimation_method`	`ILineEstimationMethod`	Strategy class for line estimation.	`HoughLinesP`
`color_space`	`ColorSpace`	Color space for detection (HSV or LAB).	`None`

Attributes:

Name	Type	Description
`color_detector`	`ColorDetector`	Instance for color filtering.
`estimation_method`	`ILineEstimationMethod`	Active estimation strategy.
`prev_angle`	`float`	Previous angle for exponential smoothing.
`external_mask`	`ndarray`	External binary mask to use instead of color detection.

external_mask `property` `writable` ¶

external_mask: ndarray

External binary mask for line detection.

set_text_positions ¶

set_text_positions(positions_dict)

Set positions for text labels on output image.

Parameters:

Name	Type	Description	Default
`positions_dict`	`dict`	Dictionary with keys 'angle', 'center_x', 'color' and (x, y) tuple values.	required

detect_line ¶

detect_line(img, region=(0, 0), draw=True, draw_color=None)

Detect line in image using color filtering and estimation.

Parameters:

Name	Type	Description	Default
`img`	`ndarray`	Input BGR image.	required
`region`	`tuple`	Region of interest size (width, height). (0, 0) uses full image.	`(0, 0)`
`draw`	`bool`	Whether to draw visualizations.	`True`
`draw_color`	`tuple`	BGR color for drawing.	`None`

Returns:

Type	Description
`tuple`	(output_img, mask, center_x, center_y, angle, width, height) - output_img: Image with drawings - mask: Binary region mask - center_x, center_y: Line center coordinates - angle: Line angle in degrees - width, height: Line dimensions in pixels

ModelType ¶

Bases: Enum

Enumeration of available distance estimation model types.

Attributes:

Name	Type	Description
`LINEAR`	`auto`	Simple inverse linear model.
`POLYNOMIAL`	`auto`	Polynomial regression model.
`EXPONENTIAL`	`auto`	Exponential decay model.
`LOGARITHMIC`	`auto`	Logarithmic model.
`INVERSE_POWER`	`auto`	Inverse power law model.
`ROBUST_POLY2`	`auto`	Robust polynomial degree 2 model.

OpticalFlowConfig `dataclass` ¶

OpticalFlowConfig(method: str = METHOD_FARNEBACK, resize_to: Optional[Tuple[int, int]] = None, refresh_corners_below: int = 60, max_corners: int = 180, quality_level: float = 0.02, min_distance: int = 14, lk_win_size: int = 21, lk_max_level: int = 3, lk_arrow_scale: float = 4.0, lk_min_draw_px: float = 0.4, ema_alpha: float = 0.35, pyr_scale: float = 0.5, levels: int = 3, winsize: int = 21, iterations: int = 3, poly_n: int = 5, poly_sigma: float = 1.2)

Configuration for :class:OpticalFlowEstimator.

Parameters:

Name	Type	Description	Default
`method`	`str`	Backend algorithm. One of `"lucas_kanade"` or `"farneback"`.	`METHOD_FARNEBACK`
`resize_to`	`tuple of (int, int)`	Downsample frames to `(width, height)` before processing for performance. `None` keeps the input resolution. The output flow is rescaled back to the input frame coordinates so the numeric result and overlay stay consistent.	`None`
`refresh_corners_below`	`int`	Lucas-Kanade only. Re-detect Shi-Tomasi corners when the number of successfully tracked points drops below this threshold. Lower values reduce flicker (corners are kept longer) at the cost of fewer arrows over time.	`60`
`max_corners`	`int`	Lucas-Kanade only. Maximum corners returned by `cv2.goodFeaturesToTrack`.	`180`
`quality_level`	`float`	Lucas-Kanade only. Corner quality (0..1) passed to `cv2.goodFeaturesToTrack`.	`0.02`
`min_distance`	`int`	Lucas-Kanade only. Minimum spacing between corners (pixels). Larger values spread corners out and reduce visual clutter.	`14`
`lk_win_size`	`int`	Lucas-Kanade only. Pyramid window size.	`21`
`lk_max_level`	`int`	Lucas-Kanade only. Pyramid max level (0 = no pyramid).	`3`
`lk_arrow_scale`	`float`	Lucas-Kanade only. Visual amplification for arrows so sub-pixel flow stays visible (does NOT affect numeric output).	`4.0`
`lk_min_draw_px`	`float`	Lucas-Kanade only. Vectors shorter than this (in raw pixels) are not drawn. Filters out tracking noise on still frames.	`0.4`
`ema_alpha`	`float`	Smoothing factor in [0, 1] applied to the displayed mean flow and quality. Higher = more responsive, lower = smoother. Numeric fields of :class:`OpticalFlowResult` are NOT smoothed (raw per-frame values), only the on-image text readout is.	`0.35`
`pyr_scale`	`float`	Farneback only. Pyramid scale (must be < 1).	`0.5`
`levels`	`int`	Farneback only. Number of pyramid levels.	`3`
`winsize`	`int`	Farneback only. Averaging window size.	`21`
`iterations`	`int`	Farneback only. Iterations per pyramid level.	`3`
`poly_n`	`int`	Farneback only. Pixel neighborhood for polynomial expansion.	`5`
`poly_sigma`	`float`	Farneback only. Gaussian std for polynomial smoothing.	`1.2`

OpticalFlowEstimator ¶

OpticalFlowEstimator(config: Optional[OpticalFlowConfig] = None)

Compute optical flow and convert it to angular rate / velocity.

Parameters:

Name	Type	Description	Default
`config`	`OpticalFlowConfig`	Configuration. Defaults to Farneback with sensible parameters.	`None`

Examples:

>>> est = OpticalFlowEstimator(OpticalFlowConfig(method="lucas_kanade"))
>>> result = est.process(frame_bgr, focal_px=500.0, altitude_m=1.5)
>>> if result is not None:
...     print(result.velocity_m_s, result.quality)

reset ¶

reset() -> None

Drop accumulated state so the next call starts fresh.

process ¶

process(frame_bgr: ndarray, *, timestamp_s: Optional[float] = None, focal_px: Optional[float] = None, altitude_m: Optional[float] = None) -> Optional[OpticalFlowResult]

Compute optical flow for the current frame.

Parameters:

Name	Type	Description	Default
`frame_bgr`	`ndarray`	Input BGR frame.	required
`timestamp_s`	`float`	Capture timestamp in seconds. Defaults to `time.monotonic()`.	`None`
`focal_px`	`float`	Camera focal length in pixels. Required to compute `angular_rate_rad_s` and `velocity_m_s`.	`None`
`altitude_m`	`float`	Height above ground in meters. Required (together with `focal_px`) to compute `velocity_m_s`.	`None`

Returns:

Type	Description
`OpticalFlowResult or None`	`None` is returned on the very first call (no previous frame to compare against).

OpticalFlowResult `dataclass` ¶

OpticalFlowResult(method: str, dt_s: float, mean_flow_px: Tuple[float, float] = (0.0, 0.0), mean_flow_px_per_s: Tuple[float, float] = (0.0, 0.0), angular_rate_rad_s: Optional[Tuple[float, float]] = None, velocity_m_s: Optional[Tuple[float, float]] = None, quality: float = 0.0, n_vectors: int = 0, visualization: ndarray = (lambda: zeros((1, 1, 3), uint8))())

Per-frame output of :meth:OpticalFlowEstimator.process.

Attributes:

Name	Type	Description
`method`	`str`	Backend that produced this result.
`dt_s`	`float`	Seconds elapsed since the previous processed frame.
`mean_flow_px`	`tuple of (float, float)`	Mean displacement `(ux, uy)` over the frame in pixels.
`mean_flow_px_per_s`	`tuple of (float, float)`	Mean displacement converted to pixels per second.
`angular_rate_rad_s`	`tuple of (float, float) or None`	Mean displacement converted to rad/s using `focal_px`. `None` when no focal length was supplied.
`velocity_m_s`	`tuple of (float, float) or None`	Horizontal velocity in m/s assuming a downward-pointing camera. `None` unless both `focal_px` and `altitude_m` are given.
`quality`	`float`	Confidence in `[0, 1]`. For Lucas-Kanade it is the fraction of points successfully tracked; for Farneback it is a ratio between the mean and the spread of the magnitude field.
`n_vectors`	`int`	Number of vectors that contributed to the mean.
`visualization`	`ndarray`	BGR overlay with arrows / HSV field plus a text readout.

RansacLine ¶

Bases: ILineEstimationMethod

RANSAC-based robust line fitting.

Uses contour points with line fitting for outlier-resistant line detection.

estimate `staticmethod` ¶

estimate(img_detect: ndarray, img_out: ndarray, offset: tuple, draw: bool = True, draw_color=None) -> Tuple[float, float, float, float, float, int, int, int, int, ndarray]

Detect line using RANSAC-based fitting.

Parameters:

Name	Type	Description	Default
`img_detect`	`ndarray`	Binary image to detect lines in.	required
`img_out`	`ndarray`	Output image to draw on.	required
`offset`	`tuple`	Offset values for drawing.	required
`draw`	`bool`	Whether to draw the detected line.	`True`
`draw_color`	`tuple`	BGR color tuple for drawing.	`None`

Returns:

Type	Description
`tuple`	(center_x, center_y, angle, width, height, x, y, w, h, rotated_box)

RotatedRect ¶

Bases: ILineEstimationMethod

Minimum area rotated rectangle detection.

Fits a rotated rectangle to the largest contour for line orientation estimation.

estimate `staticmethod` ¶

estimate(img_detect: ndarray, img_out: ndarray, offset: tuple, draw: bool = True, draw_color=None) -> Tuple[float, float, float, float, float, int, int, int, int, ndarray]

Detect line using minimum area rotated rectangle.

Parameters:

Name	Type	Description	Default
`img_detect`	`ndarray`	Binary image to detect rotated rectangle in.	required
`img_out`	`ndarray`	Output image to draw on.	required
`offset`	`tuple`	Offset values for drawing.	required
`draw`	`bool`	Whether to draw the detected rectangle.	`True`
`draw_color`	`tuple`	BGR color tuple for drawing.	`None`

Returns:

Type	Description
`tuple`	(center_x, center_y, angle, width, height, x, y, w, h, rotated_box)

AbstractCam ¶

AbstractCam(name: str = 'camera')

Bases: ABC

Abstract base class for camera implementations.

All camera drivers must implement start(), get_frame(), and close() methods.

Parameters:

Name	Type	Description	Default
`name`	`str`	Camera identifier for logging. Default is 'camera'.	`'camera'`

Attributes:

Name	Type	Description
`name`	`str`	Camera identifier.
`is_running`	`bool`	True if camera is started and ready for capture.

name `property` ¶

name: str

Camera identifier.

is_running `property` ¶

is_running: bool

True if camera is started and ready for capture.

start `abstractmethod` ¶

start() -> None

Initialize and start the camera.

Must be called before get_frame(). Sets is_running to True on success.

Raises:

Type	Description
`RuntimeError`	If camera initialization fails.

get_frame `abstractmethod` ¶

get_frame() -> Optional[ndarray]

Capture and return a single frame.

Returns:

Type	Description
`ndarray or None`	BGR image as numpy array, or None if capture failed.

close `abstractmethod` ¶

close() -> None

Release camera resources.

Sets is_running to False. Safe to call multiple times.

CameraCalibration ¶

CameraCalibration()

Bases: Node

Unified camera intrinsic calibration node.

Supports both ChArUco and plain chessboard patterns, automatic or manual capture

Capture modes

auto Accepts a view automatically whenever the board is detected with at least min_corners_per_frame corners and auto_interval seconds have elapsed since the last capture. Stops after target_views. manual Key-driven from the preview window: c capture (when the board is good), u undo last, r reset all, Enter finish + calibrate, q abort. Requires a GUI window; falls back to auto when no GUI is available.

Output files (camera_matrix.txt, camera_distortion.txt) and the :meth:load_calibration API are shared by both patterns.

run ¶

run() -> None

Open the camera and start the capture loop.

calibrate ¶

calibrate() -> bool

Run cv2.calibrateCamera on the accumulated views.

Returns:

Type	Description
`bool`	True if calibration succeeded.

save_matrices ¶

save_matrices() -> None

Write camera_matrix.txt and camera_distortion.txt.

load_calibration `classmethod` ¶

load_calibration(output_dir: Optional[str] = None) -> Tuple[ndarray, ndarray]

Load calibration data from files.

Parameters:

Name	Type	Description	Default
`output_dir`	`str`	Directory containing `camera_matrix.txt` and `camera_distortion.txt`. Defaults to the calibration package directory, matching the save default.	`None`

Returns:

Type	Description
`tuple of np.ndarray`	Camera matrix (3x3) and distortion coefficients.

Raises:

Type	Description
`FileNotFoundError`	If calibration files do not exist.

CameraFactory ¶

Factory for creating camera instances from source identifiers.

Built-in drivers are loaded lazily on first use to keep the import cost of nectar.vision low (no eager pyrealsense2 / depthai / mediapipe loads).

Attributes:

Name	Type	Description
`_builders`	`dict`	Registry mapping source keys to builder callables or camera classes registered via :meth:`register`. Built-in drivers live in a separate internal registry.

register `classmethod` ¶

register(key: str, builder) -> None

Register a camera builder under key.

Parameters:

Name	Type	Description	Default
`key`	`str`	Source identifier (case-insensitive).	required
`builder`	`Type[AbstractCam] or callable`	Either a camera class instantiated as `builder(config)`, or a callable with signature `(config, node) -> AbstractCam`.	required

from_source `classmethod` ¶

from_source(source: str, *, config: Optional[CameraConfig] = None, node: Optional[Node] = None) -> AbstractCam

Create camera instance from source identifier.

Automatically detects source type: - File path: creates FileImageCam - ROS topic (starts with '/'): creates ROSCam - Registered key: creates corresponding camera

Parameters:

Name	Type	Description	Default
`source`	`str`	Source identifier. Can be file path, ROS topic, or registered key ('webcam', 'realsense', 'c920', etc.).	required
`config`	`CameraConfig`	Camera configuration. Auto-generated if not provided.	`None`
`node`	`Node`	ROS2 node required for ROSCam and RealsenseCam with ROS topics.	`None`

Returns:

Type	Description
`AbstractCam`	Configured camera instance ready for start().

Raises:

Type	Description
`ValueError`	If source type is unknown or config type mismatches.

DepthCam ¶

DepthCam(name: str = 'camera')

Bases: AbstractCam, ABC

Abstract base class for depth-capable cameras.

Extends AbstractCam with depth frame capture and distance measurement.

get_depth_frame `abstractmethod` ¶

get_depth_frame() -> Optional[ndarray]

Capture and return depth frame.

Returns:

Type	Description
`ndarray or None`	Depth image in meters (float32), or None if capture failed.

get_distance `abstractmethod` ¶

get_distance(u: int, v: int) -> Optional[float]

Get distance at pixel coordinates.

Parameters:

Name	Type	Description	Default
`u`	`int`	Horizontal pixel coordinate (column).	required
`v`	`int`	Vertical pixel coordinate (row).	required

Returns:

Type	Description
`float or None`	Distance in meters, or None if invalid/unavailable.

ImageHandler ¶

ImageHandler(image_source: str, image_processing_callback: Optional[Callable] = None, show_result: Optional[str] = None, *, config: Optional[CameraConfig] = None, camera: Optional[AbstractCam] = None, poll_interval: float = 0.01, frame_timeout: Optional[float] = None, executor: Optional[Executor] = None)

High-level camera interface backed by an internal ROS 2 node.

Manages camera lifecycle, frame polling, and optional image processing. A dedicated node is created at construction time and registered with the SDK runtime executor; the timer-based capture loop runs on that executor.

Parameters:

Name	Type	Description	Default
`image_source`	`str`	Camera source identifier. File path, ROS topic (starts with '/'), or registered key ('webcam', 'realsense', 'c920', 'imx219', 'oakd').	required
`image_processing_callback`	`callable`	Per-frame callback `(np.ndarray) -> Any`. The return value is propagated by :meth:`take_photo`.	`None`
`show_result`	`str`	OpenCV window name. `None` disables display.	`None`
`config`	`CameraConfig`	Camera-specific configuration. Auto-detected if not provided.	`None`
`camera`	`AbstractCam`	Pre-built camera instance. Bypasses factory creation.	`None`
`poll_interval`	`float`	Timer period in seconds for frame polling.	`0.01`
`frame_timeout`	`float`	Timeout for async frame waits. Defaults to 0.1 s.	`None`
`executor`	`Executor`	Executor to register the internal node with. Defaults to the shared :mod:`nectar.runtime` executor.	`None`

node `property` ¶

node: Node

Internal ROS 2 node owned by this handler.

open ¶

open() -> None

Build the camera (if needed) and start capture.

close ¶

close() -> None

Stop and release the underlying camera.

process ¶

process() -> None

Run the processing callback and optional OpenCV display on the current frame.

run ¶

run() -> None

Open the camera and start the timer-based capture loop.

take_photo ¶

take_photo(timeout_sec: float = 1.0, wait_for_new: bool = True) -> Optional[Any]

Capture a single frame and optionally run the processing callback.

Parameters:

Name	Type	Description	Default
`timeout_sec`	`float`	Maximum time to wait for a frame.	`1.0`
`wait_for_new`	`bool`	For async cameras, wait for a fresh frame instead of the buffered one.	`True`

Returns:

Type	Description
`Any or None`	Callback result when set, otherwise the raw frame. `None` on timeout.

Raises:

Type	Description
`RuntimeError`	If the camera has not been opened.

cleanup ¶

cleanup() -> None

Release the camera, destroy the timer, and unregister the internal node.

CalibrationResult `dataclass` ¶

CalibrationResult(model_type: ModelType, params: Dict[str, Any], rmse: float, r2: float, mae: float, aic: float, predictions: ndarray)

Results from fitting a single estimation model.

Parameters:

Name	Type	Description	Default
`model_type`	`ModelType`	Type of the fitted model.	required
`params`	`Dict[str, Any]`	Fitted parameter values.	required
`rmse`	`float`	Root Mean Square Error in distance units.	required
`r2`	`float`	Coefficient of determination (R-squared).	required
`mae`	`float`	Mean Absolute Error in distance units.	required
`aic`	`float`	Akaike Information Criterion for model selection.	required
`predictions`	`ndarray`	Model predictions on calibration data.	required

Attributes:

Name	Type	Description
`name`	`str`	Lowercase name of the model type.

name `property` ¶

name: str

Lowercase name of the model type.

FaceLandmarkRegion ¶

Predefined face landmark index groups.

Groups of landmark indices for specific facial regions. Useful for extracting specific facial features.

References

.. [1] MediaPipe Face Mesh Map https://github.com/google/mediapipe/blob/master/mediapipe/modules/face_geometry/data/canonical_face_model_uv_visualization.png

FaceMeshTracker ¶

FaceMeshTracker(config: Optional[FaceMeshTrackerConfig] = None)

Face mesh detection and tracking using MediaPipe.

Detects faces and provides 478 landmark points per face, including detailed mesh for facial features like eyes, lips, and contours.

Parameters:

Name	Type	Description	Default
`config`	`FaceMeshTrackerConfig`	Configuration object. If None, uses default settings.	`None`

Attributes:

Name	Type	Description
`detection_result`	`FaceLandmarkerResult or None`	Latest detection result from MediaPipe.
`is_running`	`bool`	Whether the tracker has been initialized.

Examples:

Basic usage with async detection:

>>> from nectar.vision import FaceMeshTracker, FaceMeshTrackerConfig
>>> config = FaceMeshTrackerConfig(num_faces=1)
>>> tracker = FaceMeshTracker(config)
>>> tracker.start()
>>> frame = cv2.imread("face.jpg")
>>> tracker.detect(frame, draw=True)
>>> cv2.imshow("Face Mesh", frame)

With context manager:

>>> with FaceMeshTracker() as tracker:
...     frame = cv2.imread("face.jpg")
...     tracker.detect(frame)
...     landmarks = tracker.get_landmarks()

Eye tracking example:

>>> from nectar.vision.algorithms.pose.face_tracker import FaceLandmarkRegion
>>> with FaceMeshTracker() as tracker:
...     tracker.detect(frame)
...     left_eye = tracker.get_landmarks(
...         landmark_ids=FaceLandmarkRegion.LEFT_EYE
...     )

References

.. [1] MediaPipe Face Landmarker https://ai.google.dev/edge/mediapipe/solutions/vision/face_landmarker

detection_result `property` ¶

detection_result: Optional[FaceLandmarkerResult]

Latest detection result.

is_running `property` ¶

is_running: bool

Whether tracker is initialized.

start ¶

start() -> None

Initialize the face landmarker detector.

Downloads the model if not available locally.

Raises:

Type	Description
`RuntimeError`	If model download fails or detector initialization fails.

close ¶

close() -> None

Release detector resources.

detect ¶

detect(frame: ndarray, draw: bool = False) -> ndarray

Detect face mesh in the image.

Parameters:

Name	Type	Description	Default
`frame`	`ndarray`	Input BGR image.	required
`draw`	`bool`	Whether to draw the face mesh on the image.	`False`

Returns:

Type	Description
`ndarray`	Image with mesh drawn if draw=True, otherwise original.

Raises:

Type	Description
`RuntimeError`	If tracker not started (call start() first).

draw_landmarks ¶

draw_landmarks(image: ndarray, draw_tesselation: bool = True, draw_contours: bool = True, draw_irises: bool = True) -> ndarray

Draw face mesh on image.

Parameters:

Name	Type	Description	Default
`image`	`ndarray`	Input BGR image to annotate.	required
`draw_tesselation`	`bool`	Draw the full face mesh tesselation.	`True`
`draw_contours`	`bool`	Draw face contours (outline, eyes, lips).	`True`
`draw_irises`	`bool`	Draw iris landmarks.	`True`

Returns:

Type	Description
`ndarray`	Annotated image.

draw_region ¶

draw_region(image: ndarray, landmark_indices: List[int], color: Tuple[int, int, int] = (0, 255, 0), radius: int = 1, thickness: int = 1, face_idx: int = 0) -> ndarray

Draw circles on specific landmarks.

Parameters:

Name	Type	Description	Default
`image`	`ndarray`	Input BGR image to annotate.	required
`landmark_indices`	`list of int`	Indices of landmarks to draw.	required
`color`	`tuple`	BGR color for the circles.	`(0, 255, 0)`
`radius`	`int`	Radius of the circles in pixels.	`1`
`thickness`	`int`	Thickness of the circles (-1 for filled).	`1`
`face_idx`	`int`	Index of the face to draw.	`0`

Returns:

Type	Description
`ndarray`	Annotated image.

get_faces ¶

get_faces() -> List[FaceResult]

Get detected faces as structured results.

Returns:

Type	Description
`list of FaceResult`	List of detected faces with landmarks.

get_landmarks ¶

get_landmarks(face_idx: int = 0, landmark_ids: Optional[List[int]] = None) -> list

Get face landmarks for a specific face.

Parameters:

Name	Type	Description	Default
`face_idx`	`int`	Index of the face.	`0`
`landmark_ids`	`list of int`	Specific landmark indices to return. If None, returns all 478.	`None`

Returns:

Type	Description
`list`	List of landmarks (normalized coordinates).

get_eye_aspect_ratio ¶

get_eye_aspect_ratio(eye: str = 'left', face_idx: int = 0) -> float

Calculate Eye Aspect Ratio (EAR) for blink detection.

EAR is the ratio of eye height to width. It decreases significantly during blinks.

Parameters:

Name	Type	Description	Default
`eye`	`str`	Which eye: "left" or "right".	`"left"`
`face_idx`	`int`	Index of the face.	`0`

Returns:

Type	Description
`float`	Eye aspect ratio. Typical values: ~0.25 open, ~0.05 closed.

References

.. [1] Soukupová, T., & Čech, J. (2016). Eye blink detection using facial landmarks. 21st Computer Vision Winter Workshop.

get_mouth_aspect_ratio ¶

get_mouth_aspect_ratio(face_idx: int = 0) -> float

Calculate Mouth Aspect Ratio (MAR) for mouth open detection.

Parameters:

Name	Type	Description	Default
`face_idx`	`int`	Index of the face.	`0`

Returns:

Type	Description
`float`	Mouth aspect ratio. Higher values indicate more open mouth.

FaceMeshTrackerConfig `dataclass` ¶

FaceMeshTrackerConfig(model_path: Optional[str] = None, num_faces: int = 1, min_detection_confidence: float = 0.5, min_presence_confidence: float = 0.5, min_tracking_confidence: float = 0.5, output_blendshapes: bool = False, running_mode: str = 'LIVE_STREAM')

Configuration for FaceMeshTracker.

Parameters:

Name	Type	Description	Default
`model_path`	`str`	Path to the face landmarker model file. If None, downloads the default model automatically.	`None`
`num_faces`	`int`	Maximum number of faces to detect.	`1`
`min_detection_confidence`	`float`	Minimum confidence for face detection (0.0 to 1.0).	`0.5`
`min_presence_confidence`	`float`	Minimum confidence for face presence (0.0 to 1.0).	`0.5`
`min_tracking_confidence`	`float`	Minimum confidence for face tracking (0.0 to 1.0).	`0.5`
`output_blendshapes`	`bool`	Whether to output face blendshapes (for expressions).	`False`
`running_mode`	`str`	Running mode: "IMAGE" for synchronous, "LIVE_STREAM" for async.	`"LIVE_STREAM"`

FaceResult `dataclass` ¶

FaceResult(landmarks: list, blendshapes: Optional[list] = None)

Result from face mesh detection.

Attributes:

Name	Type	Description
`landmarks`	`list`	478 normalized face landmarks.
`blendshapes`	`list or None`	Face blendshapes for expression tracking (if enabled).

HandLandmark ¶

Bases: IntEnum

MediaPipe hand landmark indices.

Reference: https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker

HandResult `dataclass` ¶

HandResult(landmarks: list, world_landmarks: list, handedness: str, confidence: float)

Result from hand detection.

Attributes:

Name	Type	Description
`landmarks`	`list`	Normalized hand landmarks (21 points).
`world_landmarks`	`list`	3D world landmarks in meters.
`handedness`	`str`	Hand type: "Left" or "Right".
`confidence`	`float`	Detection confidence score.

HandTracker ¶

HandTracker(config: Optional[HandTrackerConfig] = None)

Hand tracking and landmark detection using MediaPipe.

Detects hands in images and provides 21 landmark points per hand, along with handedness classification and gesture recognition utilities.

Parameters:

Name	Type	Description	Default
`config`	`HandTrackerConfig`	Configuration object. If None, uses default settings.	`None`

Attributes:

Name	Type	Description
`detection_result`	`HandLandmarkerResult or None`	Latest detection result from MediaPipe.
`is_running`	`bool`	Whether the tracker has been initialized.

Examples:

Basic usage with synchronous detection:

>>> from nectar.vision import HandTracker, HandTrackerConfig
>>> config = HandTrackerConfig(num_hands=2)
>>> tracker = HandTracker(config)
>>> tracker.start()
>>> frame = cv2.imread("hand.jpg")
>>> tracker.detect(frame, draw=True)
>>> results = tracker.get_hands()
>>> for hand in results:
...     print(f"{hand.handedness}: {hand.confidence:.2f}")
>>> tracker.close()

With context manager:

>>> with HandTracker() as tracker:
...     frame = cv2.imread("hand.jpg")
...     tracker.detect(frame)
...     fingers = tracker.raised_fingers()
...     print(f"Raised fingers: {sum(fingers)}")

References

.. [1] MediaPipe Hand Landmarker https://ai.google.dev/edge/mediapipe/solutions/vision/hand_landmarker

detection_result `property` ¶

detection_result: Optional[HandLandmarkerResult]

Latest detection result.

is_running `property` ¶

is_running: bool

Whether tracker is initialized.

start ¶

start() -> None

Initialize the hand landmarker detector.

Downloads the model if not available locally.

Raises:

Type	Description
`RuntimeError`	If model download fails or detector initialization fails.

close ¶

close() -> None

Release detector resources.

detect ¶

detect(frame: ndarray, draw: bool = False) -> ndarray

Detect hands in the image.

Parameters:

Name	Type	Description	Default
`frame`	`ndarray`	Input BGR image.	required
`draw`	`bool`	Whether to draw landmarks on the image.	`False`

Returns:

Type	Description
`ndarray`	Image with landmarks drawn if draw=True, otherwise original.

Raises:

Type	Description
`RuntimeError`	If tracker not started (call start() first).

draw_landmarks ¶

draw_landmarks(image: ndarray) -> ndarray

Draw hand landmarks and handedness labels on image.

Parameters:

Name	Type	Description	Default
`image`	`ndarray`	Input BGR image to annotate.	required

Returns:

Type	Description
`ndarray`	Annotated image.

get_hands ¶

get_hands() -> List[HandResult]

Get detected hands as structured results.

Returns:

Type	Description
`list of HandResult`	List of detected hands with landmarks and metadata.

raised_fingers ¶

raised_fingers(hand_idx: int = 0) -> List[int]

Detect which fingers are raised.

Uses world landmarks to determine finger positions relative to knuckle joints. Thumb detection accounts for handedness.

Parameters:

Name	Type	Description	Default
`hand_idx`	`int`	Index of the hand to analyze.	`0`

Returns:

Type	Description
`list of int`	List of 5 values (thumb, index, middle, ring, pinky). 1 = raised, 0 = lowered.

Notes

Returns empty list if no hands detected or invalid index.

get_landmarks ¶

get_landmarks(hand_idx: int = 0, landmark_ids: Optional[List[int]] = None) -> list

Get normalized landmarks for a specific hand.

Parameters:

Name	Type	Description	Default
`hand_idx`	`int`	Index of the hand.	`0`
`landmark_ids`	`list of int`	Specific landmark indices to return. If None, returns all 21.	`None`

Returns:

Type	Description
`list`	List of landmarks (normalized coordinates).

get_world_landmarks ¶

get_world_landmarks(hand_idx: int = 0) -> list

Get 3D world landmarks for a specific hand.

Parameters:

Name	Type	Description	Default
`hand_idx`	`int`	Index of the hand.	`0`

Returns:

Type	Description
`list`	List of world landmarks in meters.

find_distance ¶

find_distance(landmark1: int, landmark2: int, image: ndarray, hand_idx: int = 0, draw: bool = False) -> Tuple[float, ndarray, Tuple[int, int, int, int, int, int]]

Calculate pixel distance between two landmarks.

Parameters:

Name	Type	Description	Default
`landmark1`	`int`	Index of first landmark.	required
`landmark2`	`int`	Index of second landmark.	required
`image`	`ndarray`	Image for coordinate scaling and optional drawing.	required
`hand_idx`	`int`	Index of the hand.	`0`
`draw`	`bool`	Whether to draw the distance line on image.	`False`

Returns:

Name	Type	Description
`distance`	`float`	Euclidean distance in pixels.
`image`	`ndarray`	Image (with drawing if draw=True).
`coords`	`tuple`	(x1, y1, x2, y2, cx, cy) - endpoints and center point.

estimate_depth ¶

estimate_depth(hand_idx: int = 0, image_width: int = 640, image_height: int = 480) -> Optional[float]

Estimate hand depth using palm width heuristic.

Uses the distance between index finger MCP and pinky MCP landmarks to approximate distance from camera.

Parameters:

Name	Type	Description	Default
`hand_idx`	`int`	Index of the hand.	`0`
`image_width`	`int`	Image width for coordinate scaling.	`640`
`image_height`	`int`	Image height for coordinate scaling.	`480`

Returns:

Type	Description
`float or None`	Estimated depth in centimeters, or None if no detection.

Notes

This is a rough approximation based on typical hand sizes. Accuracy depends on individual hand size and calibration data.

HandTrackerConfig `dataclass` ¶

HandTrackerConfig(model_path: Optional[str] = None, num_hands: int = 2, min_detection_confidence: float = 0.5, min_presence_confidence: float = 0.5, min_tracking_confidence: float = 0.5, running_mode: str = 'IMAGE')

Configuration for HandTracker.

Parameters:

Name	Type	Description	Default
`model_path`	`str`	Path to the hand landmarker model file. If None, downloads the default model automatically.	`None`
`num_hands`	`int`	Maximum number of hands to detect.	`2`
`min_detection_confidence`	`float`	Minimum confidence for hand detection (0.0 to 1.0).	`0.5`
`min_presence_confidence`	`float`	Minimum confidence for hand presence (0.0 to 1.0).	`0.5`
`min_tracking_confidence`	`float`	Minimum confidence for hand tracking (0.0 to 1.0).	`0.5`
`running_mode`	`str`	Running mode: "IMAGE" for synchronous, "LIVE_STREAM" for async.	`"IMAGE"`

ModelCalibrator ¶

ModelCalibrator(data: List[Tuple[float, float]])

Calibrate and compare distance estimation models.

Parameters:

Name	Type	Description	Default
`data`	`List[Tuple[float, float]]`	Calibration data as list of (distance_cm, pixel_measurement) tuples.	required

Attributes:

Name	Type	Description
`pixels`	`ndarray`	Pixel measurements from calibration data.
`distances`	`ndarray`	Distance measurements from calibration data.
`results`	`Dict[ModelType, CalibrationResult]`	Fitted model results keyed by model type.

Examples:

>>> data = [(50, 32.2), (60, 28.5), (70, 24.2)]
>>> calibrator = ModelCalibrator(data)
>>> calibrator.fit_all()
>>> best = calibrator.best_model()
>>> print(f"Best model: {best.name} with R²={best.r2:.4f}")

fit_model ¶

fit_model(model_type: ModelType) -> CalibrationResult

Fit a single model type to the calibration data.

Parameters:

Name	Type	Description	Default
`model_type`	`ModelType`	Type of model to fit.	required

Returns:

Type	Description
`CalibrationResult`	Fitted model with parameters and metrics.

Notes

Computes the following metrics: - RMSE: Root Mean Square Error - MAE: Mean Absolute Error - R²: Coefficient of determination - AIC: Akaike Information Criterion

fit_all ¶

fit_all() -> Dict[ModelType, CalibrationResult]

Fit all available model types to the calibration data.

Returns:

Type	Description
`Dict[ModelType, CalibrationResult]`	Dictionary of results for all successfully fitted models.

Notes

Models that fail to fit (e.g., due to convergence issues) are silently skipped.

best_model ¶

best_model(criterion: str = 'aic') -> CalibrationResult

Get the best model according to specified criterion.

Parameters:

Name	Type	Description	Default
`criterion`	`str`	Selection criterion: "aic", "rmse", or "r2".	`"aic"`

Returns:

Type	Description
`CalibrationResult`	Best model result.

Raises:

Type	Description
`ValueError`	If no models have been fitted.

Notes

AIC: Lower is better (penalizes model complexity)
RMSE: Lower is better
R²: Higher is better

save_params ¶

save_params(path: Path, default_method: str = None) -> None

Save fitted parameters to YAML file.

Parameters:

Name	Type	Description	Default
`path`	`Path`	Output file path.	required
`default_method`	`str`	Default method to set in config. If None, uses best model.	`None`

Raises:

Type	Description
`ValueError`	If no models have been fitted.

print_results ¶

print_results() -> None

Print formatted comparison table of all fitted models.

plot ¶

plot(save_path: Path) -> None

Generate comparison plots and save to file.

Creates a 2x2 figure with: - Model fits overlaid on data points - Residual analysis - RMSE comparison (horizontal bar chart) - AIC comparison (vertical bar chart)

Parameters:

Name	Type	Description	Default
`save_path`	`Path`	Output image file path (e.g., "comparison.png").	required

Raises:

Type	Description
`ValueError`	If no models have been fitted.

C920Cam ¶

C920Cam(config: C920Config)

Bases: AbstractCam

Camera driver for Logitech C920/C920e webcams.

Parameters:

Name	Type	Description	Default
`config`	`C920Config`	Configuration with profile (resolution) and fallback device index.	required

Attributes:

Name	Type	Description
`C920_CTRL_MAP`	`dict`	Mapping of camera model names to v4l2 control parameters.

Notes

Requires v4l2-utils package for auto-detection. Profile settings: - 0: 640x480 - 1: 1280x720 (default) - 2: 1920x1080

start ¶

start() -> None

Detect device, apply settings, and start capture.

Auto-detects C920 via v4l2-ctl. Falls back to fallback_device_index if detection fails. Configures MJPG format and disables autofocus.

get_frame ¶

get_frame() -> Optional[ndarray]

Capture frame from webcam.

Returns:

Type	Description
`ndarray or None`	BGR image, or None if capture failed.

close ¶

close() -> None

Release webcam resources.

FileImageCam ¶

FileImageCam(config: FileImageConfig)

Bases: AbstractCam

Camera driver for static image files.

Loads an image from disk once and returns the same frame on every get_frame() call.

Parameters:

Name	Type	Description	Default
`config`	`FileImageConfig`	Configuration with image file path.	required

start ¶

start() -> None

Load image from configured path.

get_frame ¶

get_frame(wait_for_new: bool = False, timeout: float = 1.0) -> Optional[ndarray]

Return the loaded image.

Returns:

Type	Description
`ndarray or None`	BGR image, or None if file could not be read.

reload ¶

reload() -> bool

Reload image from disk.

Returns:

Type	Description
`bool`	True if reload succeeded, False otherwise.

close ¶

close() -> None

Mark camera as stopped.

IMX219Cam ¶

IMX219Cam(config: IMX219Config)

Bases: AbstractCam

Camera driver for IMX219 CSI cameras.

Parameters:

Name	Type	Description	Default
`config`	`IMX219Config`	Configuration with sensor ID, resolution, FPS, flip, and brightness settings.	required

Notes

Uses GStreamer pipeline with nvarguscamerasrc for hardware-accelerated capture.

Requires GStreamer and NVIDIA multimedia libraries. The pipeline uses NVMM memory for zero-copy GPU processing.

Brightness is applied via the videobalance GStreamer element and can be adjusted at runtime via :meth:set_brightness; the method restarts the capture pipeline to apply the new settings.

start ¶

start() -> None

Initialize GStreamer pipeline and start capture.

Raises:

Type	Description
`RuntimeError`	If camera cannot be opened (CSI not connected or configured).

set_brightness ¶

set_brightness(brightness: Optional[float]) -> None

Set brightness adjustment, then restart the pipeline.

Parameters:

Name	Type	Description	Default
`brightness`	`float or None`	Brightness offset in the range `[-1.0, 1.0]`. `0.0` is neutral; negative values darken, positive values brighten. Pass `None` to remove the `videobalance` element entirely.	required

Raises:

Type	Description
`ValueError`	If `brightness` is outside `[-1.0, 1.0]`.
`RuntimeError`	If the pipeline cannot be reopened after applying new settings.

get_frame ¶

get_frame() -> Optional[ndarray]

Capture frame from GStreamer pipeline.

Returns:

Type	Description
`ndarray or None`	BGR image, or None if capture failed.

close ¶

close() -> None

Release GStreamer pipeline resources.

OakdCam ¶

OakdCam(config=None)

Bases: DepthCam

Class to initialize the pipeline and define the parameters for accessing OAKD images

Note: all pipeline links must be configured before starting it. Some ready settings in this class prevent it, but for standalone applications you must pay attention to this.

OakdCam constructor: initializes the pipeline and configures cameras with their board sockets

:param config: OakDConfig instance with camera configuration

setup_camera ¶

setup_camera(cam_num: int, link_out: bool = True, set_control: bool = True) -> ColorCamera | MonoCamera | None

Function to set the camera type based on the camera number parameter

:param cam_num (int): index number for oakd cam - 1 for rgb, 2 for left monochrome cam, 3 for right monochrome cam :param link_out (bool): link the camera output with xout input, False to wait for another application :param set_control (bool): True for set initial control settings. False for otherwise

init_cam ¶

init_cam(usb2_mode: bool = False) -> Device

Initialize the device and return it

color_camera ¶

color_camera() -> ColorCamera

Link device to host, set resolution, Isp scale and fps to get the color camera

mono_camera ¶

mono_camera() -> MonoCamera

Link device and host to get the mono cameras

getQueue_CamType ¶

getQueue_CamType() -> DataOutputQueue

Gets an output queue corresponding to cam_type. If it doesn't exist it throws. Use this function to get the output queue from cam_type defined in the "setup()" scope

getQueue ¶

getQueue(stream_name: str, maxSize: int, blocking: bool = True) -> DataOutputQueue

Gets an output queue corresponding to stream name. If it doesn't exist it throws

:param stream_name (str): the stream name for output queue :param maxSize (int): max size of the queue :param blocking (bool): True for block the return of the function until arrive new msgs in the queue. False for otherwise

getFrame ¶

getFrame(queue: DataOutputQueue) -> Mat

Gets the cv frame from output queue frame wich is depthai.ImgFrame

:param queue (dai.DataOutputQueue): the output queue from getQueue function

get_stereo_depth ¶

get_stereo_depth() -> StereoDepth

Set stereo settings and get it to use.

post_processing_stereo_depth ¶

post_processing_stereo_depth(stereo: StereoDepth) -> None

Function to apply depth post-processing settings to the final map

https://docs.luxonis.com/software/depthai/examples/depth_post_processing/

configure_stereo_node_output ¶

configure_stereo_node_output(stream_names: list[str]) -> None

Set the stereo output requested

:param stream_names list[str]: stream names to configure ("disparity", "depth", "rectifiedLeft", "rectifiedRight", "syncedLeft", "syncedRight")

create_imu ¶

create_imu() -> IMU

Create an IMU sensor and return it.

Note: OAK-D-LITE doesn't have IMU

enable_imu_sensor ¶

enable_imu_sensor(sensor_name: str, rate: int) -> None

Enable the IMU sensor requested

:param sensor_name (str): the sensor name to enable ("accelerometer", "gyroscope") :param rate (int): the measurement frequency, in Hz, of the sensor. Max frequencies: accelerometer -> 512 Hz, gyroscope -> 1000 Hz

get_control_input_queue ¶

get_control_input_queue() -> DataInputQueue

Get the input control queue

set_control ¶

set_control(control: str, value: int = 0, mode=None) -> str

Set the variable control for OAK-D

:param control (str): the control to change the value ( brightness | contrast | saturation | sharpness | luma_denoise | chroma_denoise | ae_comp | autoexposure | anti_banding_mode | awb_mode | effect_mode | autofocus ) :param value (int): the integer value to set the control. Range -10 .. +10 for brightness, contrast and saturation. For sharpness, luma denoise and chroma denoise the range is 0 .. +4 For ae_comp -9 .. +9. Use this parameter if the control parameter needs an integer value. :param mode (control_mode): The mode connected to control passed (from dai.CameraControl..)

Return the result as a string

set_manual_control ¶

set_manual_control(manual_control: str, value: int) -> str

Set manual control for OAK-D

:param manual_control (str): The manual control name to set the value. ( exposure_time | sensitivity_iso | focus | white_balance) :param value (int): the integer value to set.

Accepted range:

    exposure_time -> 1 .. 33000
    sensitivity_iso -> 100 .. 1600
    focus -> 0 .. 255
    white_balance -> 1000 .. 12000

Return the result as a string

create_yolo_detection_network ¶

create_yolo_detection_network(model_path: Path | str, json_path: Path | str, sync_nn: bool = True, confidence: float = 1.0) -> None

Create a YOLO detection network. This function requires a blob model and a json file that you can get from a yolo model through the luxonis conversion tool available at https://tools.luxonis.com

:param model_path (Path | str): The path for the blob model :param json_path (Path | str): The path for the json file :param sync_nn (bool): True for a synchronous neural network, which keeps frame capture synchronous with the inference process :param confidence (float): the confidence for valid detections. Default is the confindence from json file.

depth_config ¶

depth_config(spatial_calculator: bool = False, calculation_algorithm: SpatialLocationCalculatorAlgorithm = MEDIAN) -> None

Configure depth settings for distance measurement applications. It defines the stereo, spatial location calculator (optional) and links the pipeline path.

:param spatial_calculator (bool): set spatial location calculator as input to stereo depth link. SpatialLocationCalculator can calculate spatials coordinates within a defined Region of Interest (ROI), and forwards the depth map through the passthroughDepth output for visualization or processing. :param calculation_algorithm (dai.SpatialLocationCalculatorAlgorithm): the algorithm that will be used to calculate spatial coordinates. Default is MEDIAN

spatial_calculator_config ¶

spatial_calculator_config(calc_algorithm: SpatialLocationCalculatorAlgorithm = MEDIAN) -> Tuple[SpatialLocationCalculator, XLinkOut, XLinkIn]

Define settings for Spatial Location Calculator for applications that requires a ROI definition.

:param calculation_algorithm (dai.SpatialLocationCalculatorAlgorithm): the algorithm that will be used to calculate spatial coordinates. Default is MEDIAN

update_spatial_calc_ROI ¶

update_spatial_calc_ROI(top_left: tuple[float, float], bottom_right: tuple[float, float]) -> None

Update the region of interest (ROI) in spatial calculator configs. It creates a dai.Rect to delimit the region using top left corner and bottom right corner past normalized.

:param top_left (tuple[float, float]): a tuple with the normalized top left points. [0.0 ... 1.0]. :param bottom_right (tuple[float, float]): a tuple with the normalized bottom right points. [0.0 ... 1.0].

create_spatial_detection_network ¶

create_spatial_detection_network(mn_model_path: Path | str, labels: list, confidence: float = 0.5, sync_nn: bool = True) -> None

Function to create a Mobile Net Spatial Detection Network using stereo node aligned with color camera. It defines stereo, spatial detection network, rgb camera and configure its settings.

:param mn_mnodel (Path | str): The path for the blob model :param labels (list): the list with the model labels :param confidence (float): the confidence for valid detections. Default is 0.5 :param sync_nn (bool): True for a synchronous neural network, which keeps frame capture synchronous with the inference process

OakdCameraResolution ¶

Bases: Enum

Enum class to color camera resolution types.

OpenCVCam ¶

OpenCVCam(config: OpenCVConfig)

Bases: AbstractCam

OpenCV-based camera driver using V4L2 backend.

Parameters:

Name	Type	Description	Default
`config`	`OpenCVConfig`	Camera configuration.	required

Notes

Supports two capture modes: - Synchronous (default): get_frame() blocks until frame is captured - Threaded: Background thread captures frames continuously

Use threaded=True when: - Timer-based polling causes frame drops - You need consistent FPS regardless of processing time - Multiple consumers need the latest frame

Use synchronous (default) when: - Simple single-consumer use case - Lower memory overhead preferred - Processing is fast enough to keep up with camera FPS

actual_settings `property` ¶

actual_settings: Dict[str, Any]

Return actual camera settings after initialization.

is_threaded `property` ¶

is_threaded: bool

Return True if using threaded capture mode.

start ¶

start() -> None

Open and configure the camera.

get_frame ¶

get_frame(wait_for_new: bool = False, timeout: float = 0.1) -> Optional[ndarray]

Get a frame from the camera.

Parameters:

Name	Type	Description	Default
`wait_for_new`	`bool`	Only used in threaded mode. If True, blocks until a new frame arrives. If False, returns the latest buffered frame immediately. Ignored in synchronous mode.	`False`
`timeout`	`float`	Max seconds to wait when wait_for_new=True. Default is 0.1s.	`0.1`

Returns:

Type	Description
`ndarray or None`	BGR image frame, or None if capture failed or timeout.

close ¶

close() -> None

Release camera resources.

RealsenseCam ¶

RealsenseCam(config: RealSenseConfig, node=None)

Bases: DepthCam

Camera driver for Intel RealSense depth cameras via pyrealsense2 SDK.

For direct hardware access when camera is not shared with other nodes. Use ROSDepthCam for ROS topic mode (e.g., when sharing with VSLAM).

Parameters:

Name	Type	Description	Default
`config`	`RealSenseConfig`	Configuration with resolution, FPS, and depth settings.	required

Raises:

Type	Description
`RuntimeError`	If pyrealsense2 is not installed.

Notes

Depth frames are returned in meters (float32). The align_to_color option registers depth to color frame for pixel-accurate RGB-D correspondence.

Examples:

>>> from nectar.vision.camera import RealsenseCam, RealSenseConfig
>>>
>>> config = RealSenseConfig(
...     color_res=(1280, 720),
...     depth_res=(1280, 720),
...     fps=30,
...     align_to_color=True,
... )
>>> cam = RealsenseCam(config)
>>> cam.start()
>>> rgb = cam.get_frame()
>>> depth = cam.get_depth_frame()

start ¶

start() -> None

Initialize and start the pyrealsense2 pipeline.

Raises:

Type	Description
`RuntimeError`	If pyrealsense2 is not installed.

get_frame ¶

get_frame(wait_for_new: bool = True, timeout: float = 0.1) -> Optional[ndarray]

Capture color frame from camera.

Parameters:

Name	Type	Description	Default
`wait_for_new`	`bool`	Ignored for SDK mode (always captures fresh frame).	`True`
`timeout`	`float`	Ignored for SDK mode.	`0.1`

Returns:

Type	Description
`ndarray or None`	BGR image, or None if capture failed.

get_depth_frame ¶

get_depth_frame(wait_for_new: bool = True, timeout: float = 0.1) -> Optional[ndarray]

Capture depth frame from camera.

Parameters:

Name	Type	Description	Default
`wait_for_new`	`bool`	Ignored for SDK mode (always captures fresh frame).	`True`
`timeout`	`float`	Ignored for SDK mode.	`0.1`

Returns:

Type	Description
`ndarray or None`	Depth image in meters (float32), or None if depth disabled or capture failed.

get_distance ¶

get_distance(u: int, v: int) -> Optional[float]

Get distance at pixel coordinates.

Parameters:

Name	Type	Description	Default
`u`	`int`	Horizontal pixel coordinate (column).	required
`v`	`int`	Vertical pixel coordinate (row).	required

Returns:

Type	Description
`float or None`	Distance in meters, or None if depth disabled, coordinates out of bounds, or no valid depth at that pixel.

close ¶

close() -> None

Release camera resources.

ROSCam ¶

ROSCam(node: Optional[Node] = None, config: Optional[ROSConfig] = None)

Bases: AbstractCam

Camera driver for ROS image topics.

Subscribes to sensor_msgs/Image or sensor_msgs/CompressedImage topics. When node is omitted, an internal node is created and registered with :mod:nectar.runtime; otherwise the provided node is used (this is the path taken when wrapped by :class:ImageHandler).

topic `property` ¶

topic: str

Return the subscribed topic name.

is_compressed `property` ¶

is_compressed: bool

Return whether using compressed image transport.

qos_reliability `property` ¶

qos_reliability: QoSReliability

Return the QoS reliability setting.

start ¶

start() -> None

Create subscription to configured topic.

Subscribes to either sensor_msgs/Image or sensor_msgs/CompressedImage based on the compressed configuration option.

get_frame ¶

get_frame(wait_for_new: bool = False, timeout: float = 1.0) -> Optional[ndarray]

Return the most recently received frame.

Parameters:

Name	Type	Description	Default
`wait_for_new`	`bool`	If True, blocks until a new frame arrives or timeout. If False, returns the current frame immediately.	`False`
`timeout`	`float`	Maximum time to wait for new frame (seconds).	`1.0`

Returns:

Type	Description
`ndarray or None`	BGR image (copy), or None if no frame received yet.

close ¶

close() -> None

Destroy subscription and release resources.

ROSDepthCam ¶

ROSDepthCam(node: Optional[Node] = None, config: Optional[ROSDepthConfig] = None)

Bases: DepthCam

Depth camera driver for ROS image topics.

Composes ROSCam for color stream and adds depth subscription with the same thread-safe pattern.

Parameters:

Name	Type	Description	Default
`node`	`Node`	ROS2 node for subscription creation.	`None`
`config`	`ROSDepthConfig`	Configuration with color and depth topic settings.	`None`

Examples:

Raw topics:

>>> config = ROSDepthConfig(
...     topic="/camera/color/image_raw",
...     compressed=False,
...     depth_topic="/camera/depth/image_rect_raw",
...     depth_compressed=False,
... )
>>> cam = ROSDepthCam(node, config)

Compressed topics:

>>> config = ROSDepthConfig(
...     topic="/camera/color/image_raw/compressed",
...     compressed=True,
...     depth_topic="/camera/depth/image_rect_raw/compressedDepth",
...     depth_compressed=True,
... )
>>> cam = ROSDepthCam(node, config)
>>> cam.start()
>>> rgb = cam.get_frame()
>>> depth = cam.get_depth_frame()
>>> distance = cam.get_distance(320, 240)

Notes

Thread-safe: all get_* methods return copies.
Depth frames are returned in meters (float32).
Topic paths must be explicit (include /compressed or /compressedDepth).

color_topic `property` ¶

color_topic: str

Return the subscribed color topic name.

depth_topic `property` ¶

depth_topic: str

Return the subscribed depth topic name.

is_compressed `property` ¶

is_compressed: bool

Return whether color topic uses compressed transport.

depth_enabled `property` ¶

depth_enabled: bool

Return whether depth is enabled.

start ¶

start() -> None

Create subscriptions to color and depth topics.

Subscribes to color topic via composed ROSCam and depth topic directly.

get_frame ¶

get_frame(wait_for_new: bool = False, timeout: float = 1.0) -> Optional[ndarray]

Return the most recently received color frame.

Delegates to composed ROSCam for thread-safe color frame access.

Parameters:

Name	Type	Description	Default
`wait_for_new`	`bool`	If True, blocks until a new frame arrives or timeout.	`False`
`timeout`	`float`	Maximum time to wait for new frame (seconds).	`1.0`

Returns:

Type	Description
`ndarray or None`	BGR image (copy), or None if no frame received yet.

get_depth_frame ¶

get_depth_frame(wait_for_new: bool = False, timeout: float = 1.0) -> Optional[ndarray]

Return the most recently received depth frame.

Parameters:

Name	Type	Description	Default
`wait_for_new`	`bool`	If True, blocks until a new frame arrives or timeout.	`False`
`timeout`	`float`	Maximum time to wait for new frame (seconds).	`1.0`

Returns:

Type	Description
`ndarray or None`	Depth image in meters (float32 copy), or None if depth disabled or no frame received yet.

get_distance ¶

get_distance(u: int, v: int, color_shape: Optional[tuple] = None) -> Optional[float]

Get distance at pixel coordinates.

Coordinates are automatically scaled when color and depth frames have different resolutions (common with RealSense ROS node).

Parameters:

Name	Type	Description	Default
`u`	`int`	Horizontal pixel coordinate (column) in color frame.	required
`v`	`int`	Vertical pixel coordinate (row) in color frame.	required
`color_shape`	`tuple`	Shape of color frame (h, w) for coordinate scaling. If None, coordinates are used directly without scaling.	`None`

Returns:

Type	Description
`float or None`	Distance in meters, or None if depth disabled, coordinates out of bounds, or no valid depth at that pixel.

close ¶

close() -> None

Destroy subscriptions and release resources.

T265Cam ¶

T265Cam(config: T265Config, node: Optional['Node'] = None)

Bases: DepthCam

Intel RealSense T265 tracking camera driver with stereo depth computation.

Parameters:

Name	Type	Description	Default
`config`	`T265Config`	Camera configuration.	required
`node`	`Node`	ROS2 node, required for ROS topic mode.	`None`

Raises:

Type	Description
`ImportError`	If pyrealsense2 is not installed (direct mode).

Notes

Stereo depth is computed on the host from the two fisheye cameras using Kannala-Brandt fisheye undistortion and StereoSGBM. The output resolution and FOV are configurable. Effective depth range is roughly 0.3-3m given the 64mm stereo baseline.

References

.. [1] librealsense t265_stereo.py example https://github.com/IntelRealSense/librealsense/blob/v2.53.1/wrappers/python/examples/t265_stereo.py

start ¶

start() -> None

Initialize camera and compute stereo rectification maps.

get_frame ¶

get_frame() -> Optional[ndarray]

Capture left fisheye frame as 3-channel BGR image.

Returns:

Type	Description
`ndarray or None`	Left fisheye image converted to BGR, or None if not available.

get_stereo_frames ¶

get_stereo_frames() -> Optional[Tuple[ndarray, ndarray]]

Get raw fisheye stereo pair.

Returns:

Type	Description
`tuple of (np.ndarray, np.ndarray) or None`	(left, right) grayscale fisheye images, or None if not available.

get_depth_frame ¶

get_depth_frame() -> Optional[ndarray]

Compute stereo depth from fisheye pair.

Uses pre-computed undistortion/rectification maps and StereoSGBM. Result is cached until new frames arrive.

Returns:

Type	Description
`ndarray or None`	Depth image in meters (float32) at stereo output resolution, or None if frames not available or stereo not initialized.

get_distance ¶

get_distance(u: int, v: int) -> Optional[float]

Get distance at pixel coordinates in the stereo depth output.

Parameters:

Name	Type	Description	Default
`u`	`int`	Horizontal pixel coordinate (column).	required
`v`	`int`	Vertical pixel coordinate (row).	required

Returns:

Type	Description
`float or None`	Distance in meters, or None if invalid.

get_pose ¶

get_pose() -> T265Pose

Get latest 6DOF pose from T265 tracker.

Returns:

Type	Description
`T265Pose`	Current pose with translation, rotation, velocity, angular velocity, tracker confidence, and timestamp.

get_rectified_frames ¶

get_rectified_frames() -> Optional[Tuple[ndarray, ndarray]]

Get undistorted and rectified stereo pair (cropped to valid region).

Useful for visualization of the stereo matching input.

Returns:

Type	Description
`tuple of (np.ndarray, np.ndarray) or None`	(left_rectified, right_rectified), or None if not available.

close ¶

close() -> None

Release camera resources.

T265Pose `dataclass` ¶

T265Pose(translation: ndarray = (lambda: zeros(3))(), rotation: ndarray = (lambda: array([0.0, 0.0, 0.0, 1.0]))(), velocity: ndarray = (lambda: zeros(3))(), angular_velocity: ndarray = (lambda: zeros(3))(), tracker_confidence: int = 0, timestamp: float = 0.0)

6DOF pose from T265 tracking camera.

ArucoNode ¶

ArucoNode()

Bases: Node

ROS2 node for ArUco marker pose estimation.

Detects ArUco markers and publishes their pose (translation, yaw).

Parameters (ROS)

image_source : str Camera source identifier. marker_dict : int ArUco dictionary size (default: 5). tag_size : float Physical marker size in meters (default: 0.2).

Publishes

/aruco/pose_estimate : ArucoTransforms Detected marker ID, translation vector, and yaw angle.

process_image ¶

process_image(img) -> None

Process frame and publish marker pose.

Parameters:

Name	Type	Description	Default
`img`	`ndarray`	Input BGR image.	required

cleanup ¶

cleanup() -> None

Clean up resources and shutdown.

CameraPublisherNode ¶

CameraPublisherNode()

Bases: Node

Generic camera image publisher node.

Captures frames from any supported camera driver via CameraFactory and publishes them to ROS image topics.

Parameters (ROS) — General

camera_source : str Camera source identifier (default: 'webcam'). Accepted values: 'webcam', 'opencv', 'c920', 'imx219', 'realsense', 't265', 'oakd', 'ros', 'ros_depth', 'file'. use_compression : bool Publish compressed JPEG images (default: True). jpeg_quality : int JPEG compression quality 0-100 (default: 80). log_fps_interval : float Interval in seconds to log FPS statistics (default: 5.0, 0 to disable). poll_interval : float Override frame-poll period in seconds (default: -1 = auto). frame_timeout : float Override new-frame wait timeout in seconds (default: -1 = auto).

Parameters (ROS) — OpenCV / Webcam

device_index : int Webcam device index (default: 0). fps : int Target frames per second (default: 30). width : int Frame width in pixels (default: 640). height : int Frame height in pixels (default: 480). fourcc : str FourCC codec code (default: 'MJPG'). buffer_size : int Camera buffer size 1-10 (default: 2). threaded : bool Use background thread for capture (default: True).

Parameters (ROS) — C920

profile : int C920 resolution profile 0-2 (default: 1). fallback_device_index : int Fallback V4L2 device index (default: 0).

Parameters (ROS) — IMX219

sensor_id : int CSI sensor id (default: 0). width : int Frame width in pixels (default: 1920) height : int Frame height in pixels (default: 1080) fps : int Target frames per second (default: 30). flip : int Image flip (default: 0). brightness : float Optional brightness setting (default: None, auto).

Parameters (ROS) — RealSense

color_width : int Color stream width (default: 640). color_height : int Color stream height (default: 480). depth_width : int Depth stream width (default: 640). depth_height : int Depth stream height (default: 480). fps : int Target frames per second (default: 30). align_to_color : bool Align depth to color frame (default: True). enable_depth : bool Enable depth stream (default: True). use_ros_topics : bool Subscribe to existing ROS topics instead of opening device (default: False). color_topic : str Color image ROS topic (default: '/camera/color/image_raw'). depth_topic : str Depth image ROS topic (default: '/camera/depth/image_rect_raw'). color_compressed : bool Color topic is compressed (default: True). depth_compressed : bool Depth topic is compressed (default: False).

Parameters (ROS) — OAK-D

cam_num : int OAK-D camera selector 1-3 (default: 1). 1: RGB, 2: left mono, 3: right mono. enable_depth : bool Enable depth stream (default: False).

Parameters (ROS) — ROS camera

topic : str Image topic to subscribe to (default: '/image_raw'). compressed : bool Topic carries compressed images (default: False). reliability : str QoS reliability: 'best_effort' or 'reliable' (default: 'best_effort'). durability : str QoS durability: 'volatile' or 'transient_local' (default: 'volatile'). history_depth : int QoS history depth (default: 1). encoding : str Image encoding (default: 'bgr8').

Parameters (ROS) — ROS Depth camera

topic : str Image topic to subscribe to (default: '/image_raw'). compressed : bool Topic carries compressed images (default: False). reliability : str QoS reliability: 'best_effort' or 'reliable' (default: 'best_effort'). durability : str QoS durability: 'volatile' or 'transient_local' (default: 'volatile'). history_depth : int QoS history depth (default: 1). encoding : str Image encoding (default: 'bgr8'). depth_topic : str Depth image ROS topic (default: '/camera/depth/image_rect_raw'). depth_compressed : bool Depth topic is compressed (default: False). depth_encoding : str Depth image encoding (default: 'passthrough'). enable_depth : bool Enable depth stream (default: True).

Parameters (ROS) — File

file_path : str Path to image file (default: '').

Publishes

image_raw : Image Raw BGR image (if use_compression=False).

cleanup ¶

cleanup() -> None

Release camera resources.

ColorCalibrationNode ¶

ColorCalibrationNode()

Bases: Node

Interactive color calibration node.

Combines click-to-sample and trackbar fine-tuning in a single window. Click colored regions to auto-compute HSV/LAB thresholds via flood fill, then adjust the trackbars for fine control. Named colors are saved to the shared calibration file consumed by ColorDetector(mode="preset").

Parameters (ROS)

image_source : str Camera source identifier (default: webcam). color_space : str Initial color space, hsv or lab (default: hsv). flood_tolerance : int Initial flood-fill tolerance for click sampling (default: 15).

Notes

Mouse: left-click to sample a region. Keys: q quit, r reset, z undo, s save, l load, c switch color space.

LineDetectionNode ¶

LineDetectionNode()

Bases: Node

ROS2 node for detecting lines using various estimation methods.

Processes images from a specified source, detects lines based on configured colors and estimation method, and publishes results.

Parameters (ROS)

line_colors : str Comma-separated list of colors to detect. method : str Line detection method (HoughLinesP, RotatedRect, etc.). image_source : str Camera source identifier. show_visualization : bool Whether to show OpenCV window. visualization_name : str Name for visualization window. spaces : str Comma-separated color spaces (hsv, lab). cap : int Webcam index for OpenCV.

Attributes:

Name	Type	Description
`estimation_methods`	`Dict[str, ILineEstimationMethod]`	Available estimation method classes.
`line_detectors`	`Dict[str, LineDetector]`	Detector instances per color.

initialize_color_detector ¶

initialize_color_detector(color: str, color_space: str)

Initialize line detector and publishers for a color.

Parameters:

Name	Type	Description	Default
`color`	`str`	Color name to detect.	required
`color_space`	`str`	Color space ("hsv" or "lab").	required

parameters_callback ¶

parameters_callback(params)

Handle runtime parameter changes.

Parameters:

Name	Type	Description	Default
`params`	`list`	List of changed parameters.	required

Returns:

Type	Description
`SetParametersResult`	Success/failure indication.

process_image ¶

process_image(img: ndarray) -> None

Process image frame and publish detection results.

Parameters:

Name	Type	Description	Default
`img`	`ndarray`	Input BGR image frame.	required

run ¶

run()

Start the line detection processing loop.

cleanup ¶

cleanup()

Clean up resources and shutdown.

ImageCalculus ¶

ImageCalculus(camera_offset: Dict[str, float] = None, camera_resolution: Dict[str, int] = None, pixels_per_degree: Dict[str, int] = None)

Utility class for image-based geometric calculations related to drones.

This class provides methods to convert pixel coordinates from an image into metric vectors in the drone reference frame, assuming a flat ground (Z = 0) and small-angle approximations for pitch and roll.

Coordinate frame convention: - X (forward): drone forward direction - Y (right): drone right direction - Z (up): positive upwards - Ground plane is located at Z = 0

Notes

All angles are expressed in degrees.
The calculations assume small pitch and roll angles (typically <= 5 degrees).
Camera offsets are defined in the drone body frame.

Initialize ImageCalculus object.

Parameters:

Name	Type	Description	Default
`camera_offset`	`Dict[str, float]`	Camera position relative to the drone center, in meters. Expected keys are: - 'forward': offset along the forward axis - 'right': offset along the right axis - 'up': offset along the up axis	`None`
`camera_resolution`	`Dict[str, int]`	Camera image resolution in pixels. Expected keys are: - 'width' - 'height'	`None`
`pixels_per_degree`	`Dict[str, int]`	Conversion factor from pixels to degrees. Expected keys are: - 'horizontal' - 'vertical'	`None`

camera_offset `property` `writable` ¶

camera_offset

Dict[str, float]: Camera offset relative to the drone body frame.

camera_resolution `property` `writable` ¶

camera_resolution

Dict[str, int]: Camera image resolution in pixels.

pixels_per_degree `property` `writable` ¶

pixels_per_degree

Dict[str, int]: Number of pixels corresponding to one degree of view.

calculate_vector_from_drone_to_ground ¶

calculate_vector_from_drone_to_ground(target_pixel: tuple[float, float], height: float, pitch: float, roll: float) -> tuple[float, float, float] | None

Calculate the vector from the drone to the ground intersection point.

This method projects a pixel from the image onto the ground plane (Z = 0), taking into account the drone height, camera offset, and small pitch and roll angles. The calculation uses small-angle approximations and is not intended for large attitude angles.

Parameters:

Name	Type	Description	Default
`target_pixel`	`tuple[float, float]`	Target pixel coordinates (x, y) in the image frame.	required
`height`	`float`	Drone height relative to the ground, in meters. Must be positive (drone above ground).	required
`pitch`	`float`	Drone pitch angle in degrees. Positive values indicate nose-up rotation.	required
`roll`	`float`	Drone roll angle in degrees. Positive values indicate right-wing-down rotation.	required

Returns:

Type	Description
`tuple[float, float, float] \| None`	A 3D vector (forward, right, down) from the drone to the intersection point on the ground, expressed in meters. The Z component is negative (downwards). Returns None if projection is invalid (height <= 0 or camera pointing above horizon).

Notes

Valid only for small pitch and roll angles (typically <= 5°).
Assumes flat ground at Z = 0.
Does not account for yaw rotation.

estimate_pixel_gps `staticmethod` ¶

estimate_pixel_gps(origin_lat: float, origin_lon: float, origin_row: int, origin_col: int, target_row: int, target_col: int, gsd: float, image_bearing: float)

Estimate the GPS coordinates of a target pixel in an image.

Based on a known GPS coordinate of a reference pixel (origin), the Ground Sampling Distance (GSD), and the image orientation (bearing).

Parameters:

Name	Type	Description	Default
`origin_lat`	`float`	Latitude of the origin pixel in decimal degrees.	required
`origin_lon`	`float`	Longitude of the origin pixel in decimal degrees.	required
`origin_row`	`int`	Row index (vertical axis) of the origin pixel.	required
`origin_col`	`int`	Column index (horizontal axis) of the origin pixel.	required
`target_row`	`int`	Row index (vertical axis) of the target pixel.	required
`target_col`	`int`	Column index (horizontal axis) of the target pixel.	required
`gsd`	`float`	Ground Sampling Distance (meters per pixel).	required
`image_bearing`	`float`	Orientation of the image in degrees (0° is North, increases clockwise).	required

Returns:

Type	Description
`tuple[float, float]`	A tuple containing the estimated latitude and longitude (in decimal degrees) of the target pixel.

calculate_offset_pixels `staticmethod` ¶

calculate_offset_pixels(offset_meters: float, height_meters: float, fov_degrees: float, image_pixels: int) -> float

Calculate the offset in pixels corresponding to a physical offset in meters.

Converts a physical offset from the camera to a reference point (e.g., the center of the drone), given the camera's height above the ground, the field of view (FOV), and the image resolution.

Parameters:

Name	Type	Description	Default
`offset_meters`	`float`	Physical offset from the camera to the reference point (meters).	required
`height_meters`	`float`	Height of the camera above the ground (meters).	required
`fov_degrees`	`float`	Field of view of the camera (horizontal or vertical) in degrees.	required
`image_pixels`	`int`	Number of pixels in the corresponding image dimension (width or height).	required

Returns:

Type	Description
`float`	Offset in pixels.

Notes

The calculation follows these steps:

Convert FOV to radians: fov_radians = fov_degrees * π / 180
Calculate ground span in meters: ground_span = 2 * height_meters * tan(fov_radians / 2)
Calculate meters per pixel: ground_span / image_pixels
Convert physical offset (offset_meters) to offset in pixels (offset_pixels)t: offset_pixels = offset_meters / meters_per_pixel

Vision API¶

vision ¶

AdaptiveHoughLinesP ¶

estimate staticmethod ¶

Aruco ¶

total_markers property ¶

marker_dict property ¶

tag_size property ¶

aruco_config ¶

detect ¶

calculateYawFromCorners ¶

pose_estimate ¶

ColorDetector ¶

color_values property writable ¶

empty ¶

initTrackbars ¶

getTrackValues ¶

filterColor ¶

get_color_values ¶

saveColorValues ¶

ColorSpace ¶

DistanceEstimator ¶

model property ¶

model_type property writable ¶

estimate ¶

estimate_batch ¶

compare_methods ¶

get_model ¶

set_model_params ¶

available_methods staticmethod ¶

FitEllipse ¶

estimate staticmethod ¶

HoughLinesP ¶

estimate staticmethod ¶

ILineEstimationMethod ¶

estimate abstractmethod staticmethod ¶

LineDetector ¶

external_mask property writable ¶

set_text_positions ¶

detect_line ¶

ModelType ¶

OpticalFlowConfig dataclass ¶

OpticalFlowEstimator ¶

reset ¶

process ¶

OpticalFlowResult dataclass ¶

RansacLine ¶

estimate staticmethod ¶

RotatedRect ¶

estimate staticmethod ¶

AbstractCam ¶

name property ¶

is_running property ¶

start abstractmethod ¶

get_frame abstractmethod ¶

close abstractmethod ¶

CameraCalibration ¶

run ¶

calibrate ¶

save_matrices ¶

load_calibration classmethod ¶

CameraFactory ¶

register classmethod ¶

from_source classmethod ¶

DepthCam ¶

get_depth_frame abstractmethod ¶

get_distance abstractmethod ¶

ImageHandler ¶

node property ¶

open ¶

close ¶

process ¶

run ¶

take_photo ¶

cleanup ¶

CalibrationResult dataclass ¶

name property ¶

FaceLandmarkRegion ¶

FaceMeshTracker ¶

detection_result property ¶

estimate `staticmethod` ¶

total_markers `property` ¶

marker_dict `property` ¶

tag_size `property` ¶

color_values `property` `writable` ¶

model `property` ¶

model_type `property` `writable` ¶

available_methods `staticmethod` ¶

estimate `staticmethod` ¶

estimate `staticmethod` ¶

estimate `abstractmethod` `staticmethod` ¶

external_mask `property` `writable` ¶

OpticalFlowConfig `dataclass` ¶

OpticalFlowResult `dataclass` ¶

estimate `staticmethod` ¶

estimate `staticmethod` ¶

name `property` ¶

is_running `property` ¶

start `abstractmethod` ¶

get_frame `abstractmethod` ¶

close `abstractmethod` ¶

load_calibration `classmethod` ¶

register `classmethod` ¶

from_source `classmethod` ¶

get_depth_frame `abstractmethod` ¶

get_distance `abstractmethod` ¶

node `property` ¶

CalibrationResult `dataclass` ¶

name `property` ¶

detection_result `property` ¶

is_running `property` ¶

FaceMeshTrackerConfig `dataclass` ¶

FaceResult `dataclass` ¶

HandResult `dataclass` ¶

detection_result `property` ¶

is_running `property` ¶

HandTrackerConfig `dataclass` ¶