Unable to use RGB camera (Oak-D) for Feature-Tracking

[hatschiw]

Hello Spectacular AI Team,
I have an Oak-D Pro (which is basically an Oak-D S2 with an IR dot projector) and I am trying to use the RGB camera for Feature-Tracking.
The monochrome results are not usable. So Feature-Tracking with color-reference is needed.

If I use:

config = spectacularAI.depthai.Configuration()
config.useFeatureTracker = True
config.useColor = True

I get this Warning:

SpectacularAI WARN: Unsupported image format RGB888i
SpectacularAI ERROR: Cannot convert image to Spectacular AI format

Trying to change the Format for example with:

rgb_camera = pipeline.create(depthai.node.ColorCamera)
rgb_camera.setInterleaved(False)
rgb_camera.setColorOrder(depthai.ColorCameraProperties.ColorOrder.BGR)

...doesn't help because it seems that you create internally a depthai.node.ColorCamera instance.

Because I get this exception:

ColorCamera(3) - Specified camera board socket already used.

Is there anything I can do to get the RGB camera working with Feature-Tracking.

Also, I'd like to ask which is the lowest usable voxel-size? Thank you for your efforts!

Greetings to Finnland

[oseiskar]

Trying to change the Format for example with:

rgb_camera = pipeline.create(depthai.node.ColorCamera)
rgb_camera.setInterleaved(False)
rgb_camera.setColorOrder(depthai.ColorCameraProperties.ColorOrder.BGR)

...doesn't help because it seems that you create internally a depthai.node.ColorCamera instance.

Hi. You cannot create your own color camera node, but you can modify the camera node in the spectacularAI.Pipeline object. There's no example for this but see a related one, which shows how to modify the FPS by directly accessing DepthAI gray camera nodes.

However, this feature is not expected to work very well on most OAK models, especially the ones with a rolling shutter RGB sensor, see here for more info.

[hatschiw]

Got it! Thank you for the quick reply!