Additional Camera Drivers #221
Replies: 8 comments 48 replies
-
|
Sure, I can add the drivers to the config later tonight. I took a quick look, and this seems about right. I will need to double-check it later. diff --git a/arch/arm64/configs/rockchip_linux_defconfig b/arch/arm64/configs/rockchip_linux_defconfig
index 34204c337788..3d69339ce528 100644
--- a/arch/arm64/configs/rockchip_linux_defconfig
+++ b/arch/arm64/configs/rockchip_linux_defconfig
@@ -4183,7 +4183,7 @@ CONFIG_REGULATOR_FIXED_VOLTAGE=y
# CONFIG_REGULATOR_USERSPACE_CONSUMER is not set
# CONFIG_REGULATOR_88PG86X is not set
CONFIG_REGULATOR_ACT8865=y
-# CONFIG_REGULATOR_AD5398 is not set
+CONFIG_REGULATOR_AD5398=y
# CONFIG_REGULATOR_ARM_SCMI is not set
# CONFIG_REGULATOR_DA9210 is not set
# CONFIG_REGULATOR_DA9211 is not set
@@ -4564,7 +4564,7 @@ CONFIG_VIDEO_RK_IRCUT=y
CONFIG_VIDEO_GC8034=y
# CONFIG_VIDEO_HI556 is not set
# CONFIG_VIDEO_IMX214 is not set
-# CONFIG_VIDEO_IMX219 is not set
+CONFIG_VIDEO_IMX219=y
# CONFIG_VIDEO_IMX258 is not set
# CONFIG_VIDEO_IMX274 is not set
# CONFIG_VIDEO_IMX290 is not set
@@ -4580,6 +4580,9 @@ CONFIG_VIDEO_GC8034=y
CONFIG_VIDEO_IMX415=y
CONFIG_VIDEO_IMX464=y
# CONFIG_VIDEO_IMX355 is not set
+CONFIG_VIDEO_IMX477=y
+CONFIG_VIDEO_IMX519=y
+CONFIG_VIDEO_IMX708=y
# CONFIG_VIDEO_IMX577 is not set
# CONFIG_VIDEO_IMX586 is not set
# CONFIG_VIDEO_JX_K17 is not set
@@ -4602,7 +4605,7 @@ CONFIG_VIDEO_OV4689=y
# CONFIG_VIDEO_OV50C40 is not set
# CONFIG_VIDEO_OV5640 is not set
# CONFIG_VIDEO_OV5645 is not set
-# CONFIG_VIDEO_OV5647 is not set
+CONFIG_VIDEO_OV5647=y
# CONFIG_VIDEO_OV6650 is not set
# CONFIG_VIDEO_OV5670 is not set
# CONFIG_VIDEO_OV5675 is not set
@@ -4677,6 +4680,7 @@ CONFIG_VIDEO_AK7375=m
CONFIG_VIDEO_AW8601=m
CONFIG_VIDEO_CN3927V=m
CONFIG_VIDEO_DW9714=m
+CONFIG_VIDEO_DW9817=m
CONFIG_VIDEO_DW9768=m
CONFIG_VIDEO_DW9800W=m
CONFIG_VIDEO_DW9807_VCM=m |
Beta Was this translation helpful? Give feedback.
-
|
Sorry to bother you. |
Beta Was this translation helpful? Give feedback.
-
that's quite a lot, do you have a company that tunes the sensors for the isp or how do you plan to use any of these sensors without the appropriate tuning files ? |
Beta Was this translation helpful? Give feedback.
-
|
I went ahead and added a SET_QUICK_STREAM ioctl call to my driver but I'm not sure if this was the cause of the problem or not. It turns out the camera module I was using was a dud and wasn't outputting any data and once I switched cameras everything is working! Now onto the tuning :)
I really hope there's a way to have the driver be compiled when needed rather than built-in as this would make development a whole lot easier. @Joshua-Riek do you have any ideas? |
Beta Was this translation helpful? Give feedback.
-
|
@peytonicmaster6 did you find a way to change resolutuion & fps of the camera? I found these commands in rockchip manual, but that does nothing but segfault of rkisp driver. If I change mode by recompiling camera driver, everything works fine, so that is not the driver issue. |
Beta Was this translation helpful? Give feedback.
-
|
@Monstrofil I have not been able to get around to changing resolution or framerate as I've been battling segmentation faults with getting the ISP to work in general. I had it working, then I updated ubuntu and the segmentation faults returned and every time I recompile the rkaiq_3A_server and librkaiq.so, I get a segmentation fault. I've tried v5.0x1.2-rc4 and v3.0x9.1 from firefly, v5.0x1.2-rc2 from JeffyCN (which looks to be where radxa builds rkaiq from for their linux distro), and the default one that ships with this ubuntu, but none of them have worked for me. When I first got it working, rkaiq_3A_server gave a detailed log of all the lines in the tuning file that were wrong, I fixed them, and then the segmentation faults went away. Do you have any idea of the debugging modes to use? I have tried From what I have read online, bad tuning files are the main cause of the segmentation faults, so I'd double check that you have the resolutions and frame rates you want to use in the tuning file. If you already have those resolutions in the tuning file and its giving segmentation faults still, I'm not sure where you'll want to look in the rkaiq code as their debugging is not great. For example in my case of not being able to get the rkaiq server running, if I do |
Beta Was this translation helpful? Give feedback.
-
Oh, that thing loves segfauls :) There are couple versions available, e.g. v3.0x9.1, v3.0x9.0, v3.0x8.9 and so on and those versions have incompatible json configs, so pay attention to what config you actually use. Unfortunately that is not my case. When I switch camera mode by just commenting out top one from supported_modes list in driver, everything works quite fine and camera starts giving me video in 40...30 fps. But when I switch mode using following commands, I either get non-working camera at all (streaming does not start), or everything works just fine, with 50/50 chance =/ Feels like I will just another kernel driver parameter and allow mode switching with reboot only, because this rockchip's isp becomes annoying. |
Beta Was this translation helpful? Give feedback.
-
|
I see. I use the tuning json files included with the source code I build librkaiq.so and rkaiq_3A_server with, but I still end up having problems. Could you share your librkaiq.so, rkaiq_3A_server, and the tuning files you have? I'd like to have a working version of the ISP so I can continue on with development (I'd like to get support for the V1, V2, and V3 raspberry pi cameras working at some point)? I'm also curious to look through your HQ camera tuning file as well. |
Beta Was this translation helpful? Give feedback.
-
|
libcamera is only usable on the raspberry pi isp, you can't just use this. |
Beta Was this translation helpful? Give feedback.
-
Determining names of the registers is the only first part of the yourney, as you still have to figure out what they really do (as they are not simple as exposure, etc), because they control things like internal dividers and multipliers, mipi lanes, and so on. I spent a lot of time trying to port 4k30fps mode from imx577 driver, but unfortunately that did not work and without 4000$ oscilloscope I'm blind and cannot analyze what is going on.
Yea, still in future plans. Motivation to work on a project was a bit ruined by many days spent in attempt to make 4k30fps mode work. |
Beta Was this translation helpful? Give feedback.
-
Right, I understand. A bit different from the basic temperature/humidity sensors I have written drivers for 😆 .
Very understandable. I can't believe it's this hard to get common hardware to work, why do they make it so damn hard to use sensors we've already bought. Have you come across the OpenIPC project? Maybe they might be able to offer some help or tips. |
Beta Was this translation helpful? Give feedback.
-
|
@Monstrofil have you seen this thread on the RPI forums for the frame rate calculation? The RPI IMX477 also says it's based on the IMX219 driver and that data sheet is leaked so it should be possible to get the 4k30fps configuration working. I'd be happy to help you out, I work with similar types of cameras for a living. Everyone, I've created a discord to give easier lines of communication and hopefully speed up the development process for cameras on rk3588 devices. Feel free to join here. |
Beta Was this translation helpful? Give feedback.
-
|
@peytonicmaster6 yes, I saw that thread, but pixel rate is only part of the settings. Proper configuration also needs e.g. these settings which determine clock speeds (example from imx577 datasheet): |
Beta Was this translation helpful? Give feedback.
-
|
I also wrote this simple tool to compare different sets of registers obtained from different vendors: Here is, for example, difference between 30 fps and 60 fps on imx577: I dropped common values, but as you can see there are much more differences then just pixel rate, |
Beta Was this translation helpful? Give feedback.
-
|
Hey, I am having trouble correctly loading the camera overlays in order to generate the /dev/videoXX files. In the official instructions for the orange pi OS (archlinux), the process is to load the overlay by adding something like the following line to /boot/extlinux/extlinux.conf: This would load the ov13850 overlay for the camera1 port. I don't see an equivalent .conf file anywhere in /boot/ so what would be the the correct way to load the rk3588-ov13850 / rk3588-ov13855 overlays at boot up? |
Beta Was this translation helpful? Give feedback.
-
|
Nevermind, I found it! I just have to add the overlay to the ubuntuEnv.txt file found in boot/firmware and it works perfectly. Example below: |
Beta Was this translation helpful? Give feedback.
-
|
Check these.
orangepi-xunlong/linux-orangepi@orange-pi-5.10-rk3588...Open360Camera:linux-orangepi:orange-pi-5.10-rk3588
https://github.com/Open360Camera/orangepi-build/actions/runs/5788110935
I did not have time to make proper description and finalize those changes
(e.g. only cam 2 overlay was added and driver is full of debug printk's),
but hopefully that helps.
Why do you need libcamera by the way? I mean, obviously it's the default
way to control camera on rpi, but it will take A LOT of time to rewrite
their IQ algorithms to work on rkisp3. If you just need CLI tools, I think
it's easier to create bash/python wrappers of v4l2 commands imitating
libcamera tools.
сб, 26 серп. 2023 р., 23:46 користувач peytonicmaster6 <
***@***.***> пише:
… Gotcha. I'm hoping I can make rkisp3 work with libcamera, but we'll see.
I've been following that thread for a while now and actually ended up using
your orange pi adapter to connect to my raspberry pi cameras! I'm currently
struggling with the overlays, and I can't seem to find anything in that
thread for the modifications needed with the overlay files. What changes
did you make to the rk3588-orangepi-5-camera{1,2,3} overlays and what do
your rk3588-imx477-c{1,2,3} drivers look like?
—
Reply to this email directly, view it on GitHub
<#221 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACCGX6H5MVESTHXYYUOEGMTXXJOC7ANCNFSM6AAAAAAZRUBUQQ>
.
You are receiving this because you commented.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
Thank you for those! Your overlays match mine so I don't think those are actually my problem, I'll double check the linux camera driver. I'm working on making a stereo vision setup with the goal of having sensor data and other information overlaid in real-time, along with some machine vision like face detection. I've been using libcamera on the raspberry pi and have gotten the base program and format set up. I was hoping to port everything over to the orange pi to use its more powerful processor, gpu, and npu, but I guess nothing is really forcing me to stay using libcamera. |
Beta Was this translation helpful? Give feedback.
-
|
@peytonicmaster6 out of curiosity, any luck with this project? |
Beta Was this translation helpful? Give feedback.
-
|
@mdurrani808 Not as far as I had hoped to be. I have gotten both the RPI V1 and V2 cameras working on an orange Pi 5B using the drivers and tuning files from Radxa. I'm currently working on porting the V3 driver from raspberry pi over, but I'm running into some issues. |
Beta Was this translation helpful? Give feedback.
-
|
@peytonicmaster6 @Joshua-Riek I am also working on the rpi imx708 camera driver, so far I am able to make it work on my custom RK3399 based board, using my custom iq files and refactored driver with rockchip support. Currently stuck with a greenish tint while capture the stream. The datasheet is not yet public, so cannot go any further than this. Let me know if you are interested it getting it working on other rockchip boards. |
Beta Was this translation helpful? Give feedback.
-
|
Well after a bit of debugging and reading SoC docs, it seems like rkisp cam-engine does not support json tuning files and uses legacy xml format. I will try to create an xml today using rockchip RKiqtool today, to see if anything gets improved. |
Beta Was this translation helpful? Give feedback.
-
|
Sadly, unable to create xml tuning file as the tool is not working on rk3399. @peytonicmaster6 can you try to create a custom tuning file using this tool ? |
Beta Was this translation helpful? Give feedback.
-
|
I was able to reduce some green tint by manually setting gain and exposure defaults, but it's still green. And one more thing is that I tried imx708 and IQ tuning file on radxa cm3-io, but the rkaiq server crashes with segmentation fault, the driver works, but the rkaiq server is not working properly. Do you have any idea what steps to take now?, the DT overlay is configured properly with |
Beta Was this translation helpful? Give feedback.
-
|
@UtsavBalar1231 Are you using the rkaiq camera engine or the rkisp camera engine? The rk3399 chips should be using the rkisp tools, rkaiq is for the newer chips. Also, since you're on the rk3399 platform, you should be able to use libcamera if that's of any interest! |
Beta Was this translation helpful? Give feedback.
-
|
I have a rk3399 custom board and a radxa cm3io (rk3566), cannot use rkisp engine on rk3399 because tuning file is not there, but on rk3566 rkaiq engine fails with segmentation fault error when running. Seems like an issue with improper iq tuning file |
Beta Was this translation helpful? Give feedback.
-
|
https://github.com/UtsavBalar1231/kernel_rockchip_linux/tree/imx708 This is my current base for imx708, will split the #!/bin/env python3
import cv2
cap = cv2.VideoCapture()
fourcc = cv2.VideoWriter_fourcc(*'NV21')
cap.open(0, apiPreference=cv2.CAP_V4L2)
cap.set(cv2.CAP_PROP_FOURCC, fourcc)
# 4608x2592@14, 1920x1080@66
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 720)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)
cap.set(cv2.CAP_PROP_FPS, 60.0)
while True:
ret, frame = cap.read()
cv2.imshow('frame', frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# After the loop release the cap object
cap.release()
# Destroy all the windows
cv2.destroyAllWindows()This is my script for testing |
Beta Was this translation helpful? Give feedback.
-
Did you succeed bro? I need a simular board in my project, and I ready to support this by whatever I can do. |
Beta Was this translation helpful? Give feedback.
-
I'm working on developing an adapter board to allow the use of the raspberry pi cameras with the orange pi, but I'm getting a little stuck on the driver side.
Could you enable the drivers to be built in the kernel for the raspberry pi cameras? There are default drivers for the sensors in the linux kernel, they just aren't being built at the moment. I can make the necessary changes to get the drivers to properly work with the cameras as I've done it before on the pi once they're loaded into the kernel.
The sensors I'm hoping to have enabled are the following:
OV5647
IMX219
IMX477
IMX519
IMX708
The accompanying lens drivers are the following:
AD5398
AK7375
DW9817
Beta Was this translation helpful? Give feedback.
All reactions