Skip to content

Vision Module

Camera abstraction layer and image-processing algorithms for ROS 2. One CameraFactory opens any supported camera behind a common interface, an ImageHandler turns it into a callback-driven stream, and a set of algorithms (ArUco, color, line, distance, MediaPipe) run on the frames.

At a glance

import nectar
from nectar.vision.camera import ImageHandler

nectar.init()
ImageHandler("webcam", image_processing_callback=lambda frame: print(frame.shape)).run()
nectar.spin()
nectar.shutdown()

Single frame (no stream):

from nectar.vision import CameraFactory

cam = CameraFactory.from_source("webcam")
cam.start()
frame = cam.get_frame()
cam.close()

Side-by-side with OpenCV / ROS subscribers: With and without Nectar (with-without.md).

Documentation

Page Scope
Cameras CameraFactory, ImageHandler, AbstractCam/DepthCam, drivers, configs, T265, calibration, image geometry
Algorithms ArUco, color, line, distance regression, MediaPipe hand/face, optical flow
ROS 2 nodes ArUco, line detection, color calibration, and camera publisher nodes

Concepts

Two layers sit behind the module:

  • Cameras. CameraFactory.from_source(key) returns a driver implementing AbstractCam (start / get_frame / close); depth-capable cameras add DepthCam (get_depth_frame / get_distance). See Cameras.
  • Streaming + algorithms. ImageHandler wraps any camera in a worker-thread grab loop and calls your processing callback on each frame. OpenCV preview, if enabled, is pumped from nectar.spin() on the main thread. The callback runs an algorithm (ArUco, color, line, distance, MediaPipe), or you can run the ready-made ROS 2 nodes.

Both share the SDK runtime executor (see nectar.runtime), so vision composes with control and AI in one mission.

Examples

See examples/vision/ for complete working scripts:

Example Description
camera_example.py Basic camera capture and configuration
depth_example.py Depth camera visualization and distance (RealSense D4xx, OAK-D)
t265_example.py T265 fisheye + stereo depth + pose overlay + click-to-measure
optical_flow_example.py Sparse/dense optical-flow visualization
collect_photos.py Save frames at intervals for dataset creation

Extending the module

  • Camera drivers: add to camera/drivers/, inherit AbstractCam or DepthCam, add a config dataclass in camera/config.py, and register it with CameraFactory.register() in camera/factory.py.
  • Algorithms: add to algorithms/<category>/.
  • ROS 2 nodes: add to nodes/.
  • Export public symbols in __init__.py.