feat(kornia-slam-py): PyO3 high-level ORB-SLAM bindings

[edgarriba]

Summary

• New crate crates/kornia-slam-py: maturin-built Python extension that exposes the SLAM driver as a single class. Construct one OrbSlam, feed frames via slam.process(image) (numpy gray or RGB uint8), query the live map across calls.
• GIL released inside process() (via Python::allow_threads), so multi-threaded Python callers aren't blocked while SLAM crunches.
• Pipeline orchestration is duplicated from examples/orb_slam/src/ — the example crate is intentionally untouched. PIPELINE_SYNC.md documents the fork.

Python surface

from kornia_slam import OrbSlam, PinholeCamera, TrackingStatus

cam  = PinholeCamera(width=640, height=480, fx=500, fy=500, cx=320, cy=240)
slam = OrbSlam(camera=cam, n_keypoints=1000, enable_local_ba=True)

for img in frames:                                # (H,W) u8 gray OR (H,W,3) u8 RGB
    r = slam.process(img)
    # r.pose: ndarray (4,4) f64, r.status: TrackingStatus, r.frame_idx: int

pts = slam.map_points()                           # ndarray (N,3) f64 — active points only
slam.current_pose, slam.num_active_map_points, slam.current_keyframe_idx

Workspace-level fixes (necessary to build)

• crates/kornia-sensors/Cargo.toml: feat/slam-utils branch was deleted upstream → repointed to main.
• crates/kornia-slam/src/map.rs: two BaObservation literals were missing the new depth_meas / depth_sigma fields that upstream kornia-3d added. Set both to inert defaults (None, 0.0) — monocular, no metric depth.
• Root Cargo.toml: exclude = ["crates/kornia-slam-py"] so the binding crate has its own lockfile and pyo3/extension-module feature unification can't leak into other workspace members.

Design

Design spec at docs/superpowers/specs/2026-05-22-pyo3-orb-slam-bindings-design.md (gitignored under the repo's docs/ rule; available locally). Key decisions captured there:

Decision	Choice
Pipeline orchestration home	Duplicated in crates/kornia-slam-py
Image input	NumPy (H,W) u8 gray or (H,W,3) u8 RGB
Camera config	PinholeCamera Python class
ORB extraction	Inside slam.process(image)
process() return	TrackingResult { pose, status, frame_idx }
Map query (v1)	slam.map_points() → (N, 3) f64
GIL during process()	Released
Build tool	maturin
Min Python	3.9 (abi3)

Test plan

• cargo fmt -- --check (new crate)
• cargo clippy --all-targets -- -D warnings (new crate)
• cargo check -p kornia-slam -p orb_slam (existing workspace still builds)
• maturin build produces a wheel
• pytest tests/python/ — 15 tests passing (construction, gray/RGB input, frame-idx monotonic, map_points shape/dtype, error handling for wrong dtype / rank / channels, PinholeCamera distortion validation, TrackingStatus equality + repr)
• End-to-end script: construct OrbSlam, feed 8 frames, query map_points and current_pose across calls — runs cleanly
• Manual test on EuRoC (reviewer): pipe a real sequence through slam.process() and confirm map points / pose match what the Rust example produces

Out of scope (v1, explicit non-goals)

• Map point colors, descriptors, observation counts
• Keyframe pose array
• Trajectory save / load / serialization
• Reset / re-initialize without recreating the object
• Async / queue / multi-threaded camera ingest
• Stereo, RGB-D, IMU, multi-camera
• PyPI publishing / CI release workflow
• .pyi type stubs

Open implementation questions

• pyo3 0.22's #[pymethods] macro expansion emits .into() on PyResult<T> -> PyResult<T> on stable rustc 1.93+, which clippy flags as useless_conversion. Suppressed crate-wide via #![allow(clippy::useless_conversion)] in lib.rs until pyo3 upstream fixes it.
• Same crate is set to edition = "2021" (vs the rest of the workspace's 2024) because pyo3 0.22 generates code that trips Rust 2024's default unsafe_op_in_unsafe_fn lint.

Related upstream tickets

While writing this binding I had to hand-roll a couple of pieces that really belong upstream in kornia-rs. Filed for follow-up so future Python-facing crates don't redo the same work:

• [Feature]: Port kornia-algebra to PyO3 (Python bindings for matrices/vectors/Lie groups) kornia-rs#915 — Port kornia-algebra to PyO3 (matrices/vectors/Lie groups). Would replace the ad-hoc Pose3d -> ndarray(4,4) and intrinsic_matrix -> ndarray(3,3) marshalling in this crate.
• [Feature]: Port VideoReader to PyO3 with time + keyframe seek kornia-rs#916 — Port VideoReader to PyO3 with time + keyframe seek. Would let OrbSlam be driven directly from an .mp4 instead of pre-decoded numpy frames.

🤖 Generated with Claude Code

[edgarriba]

@cjpurackal would love your high-level feedback on the shape of this — the Python API surface (one OrbSlam class, process(image) returns TrackingResult, map_points() for queries), the decision to duplicate the example's Pipeline into the binding crate vs promoting it into the library, and the workspace touch-ups (sensors branch fix + BaObservation field bump). Haven't run it against a real EuRoC sequence yet — just the synthetic smoke tests — but will do that soon and report back.