Support for JAVA or OpenCV

[gasparramoa]

Hello,

First of all, thank you for the open-source library. This project is really something.

I used libcamera and picamera2 and was able to change properties such as the shutter speed (exposure) and analogue gain, which I really needed to solve my problem. I was able to work everything out with Python.

However, to incorporate everything with my application, I would need to have this camera support and control on JAVA. This can also be done through OpenCV, but I heard that there were a lot of problems (github1, github2, stack1, stack2, stack3 ) with that. I tried a lot of things, such as changing between OS versions (Bulleyes and Bookworm). There was even a Google summer code project that worked to fix this, but no solution was found.

In the end, for people who need to use opencv with libcamera, I followed this suggestion:

Until Libcamera is supported by OpenCV the only real solution is to use GStreamer which "comparatively" has quite a high CPU overhead. Example code to initialise VideoCapture;

import cv2
camera = "/base/soc/i2c0mux/i2c@1/imx219@10"
pipeline = "libcamerasrc camera-name=%s ! video/x-raw,width=640,height=480,framerate=10/1,format=RGBx ! videoconvert ! videoscale ! video/x-raw,width=640,height=480,format=BGR ! appsink" % (camera)
cap = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)

In my case, I am using camera module 3, and it even works with tuning files:

import cv2
camera = "/base/soc/i2c0mux/i2c@1/imx708@1a"
pipeline = f"libcamerasrc camera-name={camera} ! video/x-raw,width=640,height=480 ! videoconvert ! appsink"
tuning_file = "/home/tactonom/tuning-files/imx219_80d.json"
os.environ['LIBCAMERA_RPI_TUNING_FILE'] = tuning_file
cap = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)
ret,frame = cap.read()
(...) 

The problem is that with this solution, I cannot access AnalogueGain and Exposure, which is what makes these cameras interesting since you have so much control.

I want to ask the libcamera community if there is already any feasible solution for this problem.

Best,
Gaspar

[berndporr]

https://github.com/berndporr/libcamera2opencv here is a pure C++ / callback based openCV interface. Been using it for my robotics projects and realtime learning etc. Actually evolved from SimpleCam from this github here.

[kbingham]

@berndporr That's very cool to get opencv integration going again!

Just pasting some cross-referencing links of other things I know have happened:

• https://summerofcode.withgoogle.com/archive/2023/projects/lKttObKa
• https://github.com/advait-0/opencv/

But I don't htink that was ever completed or added to the opencv project.

It would be awesome to see something actually get done to add support directly to the open cv project!

[berndporr]

Thanks! I think the main issue is that openCV is basically still basically C-API style that it has a blocking I/O to capture frames in a loop with this terrible cv::wait while libcamera is callback based. The way it should be in 2026! So the google code project uses the libcamera's internal workings: grabFrame(): Retrieves a frame from the camera's internal queue so that the user can do again their blocking call to get a frame and then further down do their waitkey.
My feeling is that libcamera rather needs to nudge the openCV community gently to a callback based approach. They won't take that on board any time soon but libcamera could simply provide a std::span as a single linear buffer in its callback and then turning that into a Mat is just a single command basically just re-packaging it without any memory allocation.
https://github.com/berndporr/libcamera2opencv/blob/master/libcam2opencv_format_converter.cpp#L190
Obv if the format is not native it needs to be converted but openCV can handle that. At the moment in my code I use a hacked qcam converter but should be really done in openCV as it does it transparently for the capture, too!