Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Containers cannot stream from /dev/video0 on Jetson Orin Nano #59

Open
costasvav opened this issue Dec 13, 2024 · 1 comment
Open

Containers cannot stream from /dev/video0 on Jetson Orin Nano #59

costasvav opened this issue Dec 13, 2024 · 1 comment

Comments

@costasvav
Copy link

I have a fresh install of JP6.1 on a Jetson Orin Nano dev kit using the SDK Manager, including all libraries for Computer Vision and Deepstream. When I attempt to use any streaming from an attached camera with an nVidia container, it does not work with the containers (NanoLLM, NanoOWL, etc). /dev/video0 and /dev/video1 exist with two different cameras. The cameras works fine outside the container after making adjustments to Jetson.io to switch the CSI pins to be compatible.

I assume some other steps are required to make the containers work with streaming video beyond a simple fresh install of JP with the SDK Manager. Any insight would be useful.

Below is the terminal output when attempting to run a container:

user@jetson-orin-nano:~$ jetson-containers run $(autotag nano_llm)   python3 -m nano_llm.agents.video_query --api=mlc     --model Efficient-Large-Model/VILA-2.7b     --max-context-len 256     --max-new-tokens 32     --video-input /dev/video0     --video-output webrtc://@:8554/output
Namespace(packages=['nano_llm'], prefer=['local', 'registry', 'build'], disable=[''], user='dustynv', output='/tmp/autotag', quiet=False, verbose=False)
-- L4T_VERSION=36.4.0  JETPACK_VERSION=6.1  CUDA_VERSION=12.6
-- Finding compatible container image for ['nano_llm']
dustynv/nano_llm:r36.4.0
V4L2_DEVICES:  --device /dev/video0
+ sudo docker run --runtime nvidia -it --rm --network host --shm-size=8g --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /var/run/dbus:/var/run/dbus --volume /var/run/avahi-daemon/socket:/var/run/avahi-daemon/socket --volume /var/run/docker.sock:/var/run/docker.sock --volume /home/user/jetson-containers/data:/data -v /etc/localtime:/etc/localtime:ro -v /etc/timezone:/etc/timezone:ro --device /dev/snd -e PULSE_SERVER=unix:/run/user/1000/pulse/native -v /run/user/1000/pulse:/run/user/1000/pulse --device /dev/bus/usb --device /dev/video0 --device /dev/i2c-0 --device /dev/i2c-1 --device /dev/i2c-2 --device /dev/i2c-4 --device /dev/i2c-5 --device /dev/i2c-7 --device /dev/i2c-9 -v /run/jtop.sock:/run/jtop.sock --name jetson_container_20241212_174730 dustynv/nano_llm:r36.4.0 python3 -m nano_llm.agents.video_query --api=mlc --model Efficient-Large-Model/VILA-2.7b --max-context-len 256 --max-new-tokens 32 --video-input /dev/video0 --video-output webrtc://@:8554/output
/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py:124: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
  warnings.warn(
/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Fetching 10 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 32413.48it/s]
Fetching 12 files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:00<00:00, 12719.65it/s]
17:47:49 | INFO | loading /data/models/huggingface/models--Efficient-Large-Model--VILA-2.7b/snapshots/2ed82105eefd5926cccb46af9e71b0ca77f12704 with MLC
17:47:54 | INFO | NumExpr defaulting to 6 threads.
17:47:54 | WARNING | AWQ not installed (requires JetPack 6 / L4T R36) - AWQ models will fail to initialize
17:47:55 | INFO | patching model config with {'model_type': 'llama'}
17:47:56 | INFO | device=cuda(0), name=Orin, compute=8.7, max_clocks=624000, multiprocessors=8, max_thread_dims=[1024, 1024, 64], api_version=12060, driver_version=None
17:47:56 | INFO | loading VILA-2.7b from /data/models/mlc/dist/VILA-2.7b/ctx256/VILA-2.7b-q4f16_ft/VILA-2.7b-q4f16_ft-cuda.so
17:47:56 | WARNING | model library /data/models/mlc/dist/VILA-2.7b/ctx256/VILA-2.7b-q4f16_ft/VILA-2.7b-q4f16_ft-cuda.so was missing metadata
17:47:58 | INFO | loading clip vision model openai/clip-vit-large-patch14-336
/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1142: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
17:48:02 | WARNING | disabling CLIP with TensorRT due to limited memory (falling back to Transformers API)
17:48:02 | INFO | loaded clip vision model openai/clip-vit-large-patch14-336
/opt/NanoLLM/nano_llm/vision/mm_projector.py:146: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  return torch.load(filename, map_location='cpu')
17:48:02 | INFO | mm_projector (mlp2x_gelu)  Sequential(
  (0): Linear(in_features=1024, out_features=2560, bias=True)
  (1): GELU(approximate='none')
  (2): Linear(in_features=2560, out_features=2560, bias=True)
)
17:48:02 | INFO | mm_projector weights:  dict_keys(['0.bias', '0.weight', '2.bias', '2.weight'])
┌────────────────────────────┬─────────────────────────────────────────────────────────────────────────────┐
│ _name_or_path              │ ./checkpoints/vila-sheared-llama-2.7b-256gpus-mmc4-coyo                     │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ architectures              │ ['LlavaLlamaForCausalLM']                                                   │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ attention_bias             │ False                                                                       │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ attention_dropout          │ 0.0                                                                         │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ bos_token_id               │ 1                                                                           │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ eos_token_id               │ 2                                                                           │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ freeze_mm_mlp_adapter      │ False                                                                       │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ hidden_act                 │ silu                                                                        │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ hidden_size                │ 2560                                                                        │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ image_aspect_ratio         │ pad                                                                         │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ initializer_range          │ 0.02                                                                        │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ intermediate_size          │ 6912                                                                        │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ max_position_embeddings    │ 4096                                                                        │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ mm_hidden_size             │ 1024                                                                        │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ mm_projector_lr            │                                                                             │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ mm_projector_type          │ mlp2x_gelu                                                                  │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ mm_use_im_patch_token      │ False                                                                       │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ mm_use_im_start_end        │ False                                                                       │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ mm_vision_select_feature   │ patch                                                                       │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ mm_vision_select_layer     │ -2                                                                          │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ mm_vision_tower            │ openai/clip-vit-large-patch14-336                                           │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ model_type                 │ llama                                                                       │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ num_attention_heads        │ 20                                                                          │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ num_hidden_layers          │ 32                                                                          │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ num_key_value_heads        │ 20                                                                          │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ pad_token_id               │ 0                                                                           │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ pretraining_tp             │ 1                                                                           │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ rms_norm_eps               │ 1e-05                                                                       │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ rope_scaling               │                                                                             │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ rope_theta                 │ 10000.0                                                                     │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ tie_word_embeddings        │ False                                                                       │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ tokenizer_model_max_length │ 4096                                                                        │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ tokenizer_padding_side     │ right                                                                       │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ torch_dtype                │ bfloat16                                                                    │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ transformers_version       │ 4.38.1                                                                      │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ tune_mm_mlp_adapter        │ False                                                                       │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ use_cache                  │ True                                                                        │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ use_mm_proj                │ True                                                                        │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ vocab_size                 │ 32000                                                                       │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ name                       │ VILA-2.7b                                                                   │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ api                        │ mlc                                                                         │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ mm_projector_path          │ /data/models/huggingface/models--Efficient-Large-Model--VILA-2.7b/snapshots │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ quant                      │ q4f16_ft                                                                    │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ type                       │ llama                                                                       │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ max_length                 │ 256                                                                         │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ prefill_chunk_size         │ -1                                                                          │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ load_time                  │ 12.43203760600045                                                           │
├────────────────────────────┼─────────────────────────────────────────────────────────────────────────────┤
│ params_size                │ 1300.8330078125                                                             │
└────────────────────────────┴─────────────────────────────────────────────────────────────────────────────┘

17:48:02 | INFO | using chat template 'vicuna-v1' for model VILA-2.7b
17:48:02 | INFO | model 'VILA-2.7b', chat template 'vicuna-v1' stop tokens:  ['</s>'] -> [2]
17:48:02 | INFO | Warming up LLM with query 'What is 2+2?'
17:48:07 | INFO | Warmup response:  '2+2 is a mathematical expression that represents the sum of two numbers. It is a simple yet powerful operation that is used in many areas of mathematics, including algebra, number theory, and statistics. The expression "2+2" is a shorthand way of writing the mathematical expression "2 \* 2", which represents the sum of two numbers.\n\n2+2 is a shorthand way of writing the mathematical expression "2 \* 2"\n\nThe expression "2+2" is a shorthand way of writing the mathematical expression "2 \* 2"</s>'
17:48:07 | INFO | plugin | connected PrintStream to on_text on channel 0
17:48:07 | INFO | plugin | connected ChatQuery to PrintStream on channel 0
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
        - Avoid using `tokenizers` before the fork if possible
        - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)

(gst-plugin-scanner:72): GLib-GObject-WARNING **: 17:48:07.759: cannot register existing type 'GstRtpSrc'

(gst-plugin-scanner:72): GLib-GObject-CRITICAL **: 17:48:07.759: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed

(gst-plugin-scanner:72): GLib-CRITICAL **: 17:48:07.759: g_once_init_leave: assertion 'result != 0' failed

(gst-plugin-scanner:72): GStreamer-CRITICAL **: 17:48:07.759: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed

(gst-plugin-scanner:72): GLib-GObject-WARNING **: 17:48:07.759: cannot register existing type 'GstRtpSink'

(gst-plugin-scanner:72): GLib-GObject-CRITICAL **: 17:48:07.759: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed

(gst-plugin-scanner:72): GLib-CRITICAL **: 17:48:07.759: g_once_init_leave: assertion 'result != 0' failed

(gst-plugin-scanner:72): GStreamer-CRITICAL **: 17:48:07.759: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed
sh: 1: lsmod: not found
sh: 1: modprobe: not found
[gstreamer] initialized gstreamer, version 1.20.3.0
[gstreamer] gstCamera -- attempting to create device v4l2:///dev/video0
[gstreamer] gstCamera -- didn't discover any v4l2 devices
[gstreamer] gstCamera -- device discovery failed, but /dev/video0 exists
[gstreamer]              support for compressed formats is disabled
[gstreamer] gstCamera pipeline string:
[gstreamer] v4l2src device=/dev/video0 do-timestamp=true ! nvv4l2decoder name=decoder enable-max-performance=1 ! video/x-raw(memory:NVMM) ! nvvidconv flip-method=0 ! video/x-raw ! appsink name=mysink sync=false
sh: 1: lsmod: not found
sh: 1: modprobe: not found
[gstreamer] gstCamera successfully created device v4l2:///dev/video0
[video]  created gstCamera from v4l2:///dev/video0
------------------------------------------------
gstCamera video options:
------------------------------------------------
  -- URI: v4l2:///dev/video0
     - protocol:  v4l2
     - location:  /dev/video0
  -- deviceType: v4l2
  -- ioType:     input
  -- codec:      unknown
  -- codecType:  v4l2
  -- width:      1280
  -- height:     720
  -- frameRate:  30
  -- numBuffers: 4
  -- zeroCopy:   true
  -- flipMethod: none
  -- sslCert     /etc/ssl/private/localhost.cert.pem
  -- sslKey      /etc/ssl/private/localhost.key.pem
------------------------------------------------
[gstreamer] gstEncoder -- codec not specified, defaulting to H.264
failed to find/open file /proc/device-tree/model
[gstreamer] gstEncoder -- detected board 'NVIDIA Jetson Orin Nano Developer Kit'
[gstreamer] gstEncoder -- hardware encoder not detected, reverting to CPU encoder
[gstreamer] gstEncoder -- pipeline launch string:
[gstreamer] appsrc name=mysource is-live=true do-timestamp=true format=3 ! x264enc name=encoder bitrate=4000 speed-preset=ultrafast tune=zerolatency key-int-max=30 insert-vui=1 ! video/x-h264 ! rtph264pay config-interval=1 ! application/x-rtp,media=video,encoding-name=H264,clock-rate=90000,payload=96 ! tee name=videotee ! queue ! fakesink
[webrtc] WebRTC server started @ https://jetson-orin-nano:8554
[webrtc] WebRTC server thread running...
[webrtc] websocket route added /output
[video]  created gstEncoder from webrtc://@:8554/output
------------------------------------------------
gstEncoder video options:
------------------------------------------------
  -- URI: webrtc://@:8554/output
     - protocol:  webrtc
     - location:  0.0.0.0
     - port:      8554
  -- deviceType: ip
  -- ioType:     output
  -- codec:      H264
  -- codecType:  cpu
  -- frameRate:  30
  -- bitRate:    4000000
  -- numBuffers: 4
  -- zeroCopy:   true
  -- latency     10
  -- sslCert     /etc/ssl/private/localhost.cert.pem
  -- sslKey      /etc/ssl/private/localhost.key.pem
------------------------------------------------
17:48:08 | INFO | plugin | connected VideoSource to on_video on channel 0
17:48:08 | INFO | mounting webserver path /data/datasets/uploads to /images/uploads
[gstreamer] opening gstCamera for streaming, transitioning pipeline to GST_STATE_PLAYING
17:48:08 | INFO | starting webserver @ https://0.0.0.0:8050
17:48:08 | SUCCESS | VideoQuery - system ready
 * Serving Flask app 'nano_llm.web.server'
 * Debug mode: on
Opening in BLOCKING MODE
[gstreamer] gstreamer changed state from NULL to READY ==> mysink
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter1
[gstreamer] gstreamer changed state from NULL to READY ==> nvvconv0
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter0
[gstreamer] gstreamer changed state from NULL to READY ==> decoder
[gstreamer] gstreamer changed state from NULL to READY ==> v4l2src0
[gstreamer] gstreamer changed state from NULL to READY ==> pipeline0
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter1
[gstreamer] gstreamer changed state from READY to PAUSED ==> nvvconv0
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter0
[gstreamer] gstreamer changed state from READY to PAUSED ==> decoder
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> v4l2src0
[gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer message new-clock ==> pipeline0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> nvvconv0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter0
[gstreamer] gstreamer message stream-start ==> pipeline0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> decoder
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> v4l2src0
[gstreamer] gstCamera -- end of stream (EOS)
17:48:08 | INFO | WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on https://127.0.0.1:8050
 * Running on https://192.168.5.118:8050
17:48:08 | INFO | Press CTRL+C to quit
[gstreamer] gstreamer v4l2src0 ERROR Internal data stream error.
[gstreamer] gstreamer Debugging info: ../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysink
[gstreamer] gstreamer message latency ==> mysink
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:11 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:13 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:16 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:18 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:21 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:23 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:26 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:28 | WARNING | video source /dev/video0 timed out during capture, re-trying...
17:48:28 | ERROR | Re-initializing video source "/dev/video0"
[gstreamer] gstCamera -- stopping pipeline, transitioning to GST_STATE_NULL
[gstreamer] gstCamera -- pipeline stopped
[gstreamer] gstCamera -- attempting to create device v4l2:///dev/video0
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Available Sensor modes :
Resolution: 3280 x 2464 ; Framerate = 21.000000; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 3280 x 1848 ; Framerate = 28.000001; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1920 x 1080 ; Framerate = 29.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1640 x 1232 ; Framerate = 29.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1280 x 720 ; Framerate = 59.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
[gstreamer] gstCamera -- found v4l2 device: NvV4L2 Argus PLugin
[gstreamer] v4l2-proplist, device.path=(string)/dev/video0, udev-probed=(boolean)false, device.api=(string)v4l2, v4l2.device.driver=(string)"0.99.3.3\ \(multi-NvV4L2\ Argus\ PLugin", v4l2.device.card=(string)"NvV4L2\ Argus\ PLugin", v4l2.device.bus_info=(string)platform:NV-ARGUS:1.000000, v4l2.device.version=(uint)0, v4l2.device.capabilities=(uint)2216693760, v4l2.device.device_caps=(uint)69210112;
[gstreamer] gstCamera -- found 2 caps for v4l2 device /dev/video0
[gstreamer] [0] video/x-raw, format=(string)NV12, width=(int)[ 48, 3280 ], height=(int)[ 48, 2464 ], framerate=(fraction)[ 0/1, 2147483647/1 ];
[gstreamer] [1] video/x-raw, format=(string)NV12, width=(int)[ 48, 3280 ], height=(int)[ 48, 2464 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)alternate;
[gstreamer] gstCamera -- couldn't find a compatible codec/format for v4l2 device /dev/video0
[gstreamer] gstCamera -- device discovery failed, but /dev/video0 exists
[gstreamer]              support for compressed formats is disabled
[gstreamer] gstCamera pipeline string:
[gstreamer] v4l2src device=/dev/video0 do-timestamp=true ! nvv4l2decoder name=decoder enable-max-performance=1 ! video/x-raw(memory:NVMM) ! nvvidconv flip-method=0 ! video/x-raw ! appsink name=mysink sync=false
[gstreamer] gstCamera successfully created device v4l2:///dev/video0
[video]  created gstCamera from v4l2:///dev/video0
------------------------------------------------
gstCamera video options:
------------------------------------------------
  -- URI: v4l2:///dev/video0
     - protocol:  v4l2
     - location:  /dev/video0
  -- deviceType: v4l2
  -- ioType:     input
  -- codec:      unknown
  -- codecType:  v4l2
  -- width:      1280
  -- height:     720
  -- frameRate:  30
  -- numBuffers: 4
  -- zeroCopy:   true
  -- flipMethod: none
  -- sslCert     /etc/ssl/private/localhost.cert.pem
  -- sslKey      /etc/ssl/private/localhost.key.pem
------------------------------------------------
[gstreamer] opening gstCamera for streaming, transitioning pipeline to GST_STATE_PLAYING
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Available Sensor modes :
Resolution: 3280 x 2464 ; Framerate = 21.000000; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 3280 x 1848 ; Framerate = 28.000001; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1920 x 1080 ; Framerate = 29.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1640 x 1232 ; Framerate = 29.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1280 x 720 ; Framerate = 59.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
[gstreamer] gstreamer changed state from NULL to READY ==> mysink
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter5
[gstreamer] gstreamer changed state from NULL to READY ==> nvvconv1
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter4
[gstreamer] gstreamer changed state from NULL to READY ==> decoder
[gstreamer] gstreamer changed state from NULL to READY ==> v4l2src1
[gstreamer] gstreamer changed state from NULL to READY ==> pipeline2
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter5
DEFAULT no IOCTL called
[gstreamer] gstreamer changed state from READY to PAUSED ==> nvvconv1
DEFAULT no IOCTL called
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter4
[gstreamer] gstreamer changed state from READY to PAUSED ==> decoder
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> v4l2src1
[gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline2
[gstreamer] gstreamer message new-clock ==> pipeline2
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter5
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> nvvconv1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter4
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> decoder
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> v4l2src1
[gstreamer] gstreamer message stream-start ==> pipeline2
[gstreamer] gstCamera -- end of stream (EOS)
[gstreamer] gstreamer v4l2src1 ERROR Internal data stream error.
[gstreamer] gstreamer Debugging info: ../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline2/GstV4l2Src:v4l2src1:
streaming stopped, reason not-negotiated (-4)
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysink
[gstreamer] gstreamer message latency ==> mysink
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:32 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:35 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:37 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:40 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:42 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:45 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:47 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:50 | WARNING | video source /dev/video0 timed out during capture, re-trying...
17:48:50 | ERROR | Re-initializing video source "/dev/video0"
[gstreamer] gstCamera -- stopping pipeline, transitioning to GST_STATE_NULL
[gstreamer] gstCamera -- pipeline stopped
[gstreamer] gstCamera -- attempting to create device v4l2:///dev/video0
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Available Sensor modes :
Resolution: 3280 x 2464 ; Framerate = 21.000000; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 3280 x 1848 ; Framerate = 28.000001; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1920 x 1080 ; Framerate = 29.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1640 x 1232 ; Framerate = 29.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1280 x 720 ; Framerate = 59.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
[gstreamer] gstCamera -- found v4l2 device: NvV4L2 Argus PLugin
[gstreamer] v4l2-proplist, device.path=(string)/dev/video0, udev-probed=(boolean)false, device.api=(string)v4l2, v4l2.device.driver=(string)"0.99.3.3\ \(multi-NvV4L2\ Argus\ PLugin", v4l2.device.card=(string)"NvV4L2\ Argus\ PLugin", v4l2.device.bus_info=(string)platform:NV-ARGUS:1.000000, v4l2.device.version=(uint)0, v4l2.device.capabilities=(uint)2216693760, v4l2.device.device_caps=(uint)69210112;
[gstreamer] gstCamera -- found 2 caps for v4l2 device /dev/video0
[gstreamer] [0] video/x-raw, format=(string)NV12, width=(int)[ 48, 3280 ], height=(int)[ 48, 2464 ], framerate=(fraction)[ 0/1, 2147483647/1 ];
[gstreamer] [1] video/x-raw, format=(string)NV12, width=(int)[ 48, 3280 ], height=(int)[ 48, 2464 ], framerate=(fraction)[ 0/1, 2147483647/1 ], interlace-mode=(string)alternate;
[gstreamer] gstCamera -- couldn't find a compatible codec/format for v4l2 device /dev/video0
[gstreamer] gstCamera -- device discovery failed, but /dev/video0 exists
[gstreamer]              support for compressed formats is disabled
[gstreamer] gstCamera pipeline string:
[gstreamer] v4l2src device=/dev/video0 do-timestamp=true ! nvv4l2decoder name=decoder enable-max-performance=1 ! video/x-raw(memory:NVMM) ! nvvidconv flip-method=0 ! video/x-raw ! appsink name=mysink sync=false
[gstreamer] gstCamera successfully created device v4l2:///dev/video0
[video]  created gstCamera from v4l2:///dev/video0
------------------------------------------------
gstCamera video options:
------------------------------------------------
  -- URI: v4l2:///dev/video0
     - protocol:  v4l2
     - location:  /dev/video0
  -- deviceType: v4l2
  -- ioType:     input
  -- codec:      unknown
  -- codecType:  v4l2
  -- width:      1280
  -- height:     720
  -- frameRate:  30
  -- numBuffers: 4
  -- zeroCopy:   true
  -- flipMethod: none
  -- sslCert     /etc/ssl/private/localhost.cert.pem
  -- sslKey      /etc/ssl/private/localhost.key.pem
------------------------------------------------
[gstreamer] opening gstCamera for streaming, transitioning pipeline to GST_STATE_PLAYING
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Available Sensor modes :
Resolution: 3280 x 2464 ; Framerate = 21.000000; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 3280 x 1848 ; Framerate = 28.000001; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1920 x 1080 ; Framerate = 29.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1640 x 1232 ; Framerate = 29.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

Resolution: 1280 x 720 ; Framerate = 59.999999; Analog Gain Range Min 1.000000, Max 10.625000, Exposure Range Min 13000, Max 683709000

DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
DEFAULT no IOCTL called
[gstreamer] gstreamer changed state from NULL to READY ==> mysink
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter7
DEFAULT no IOCTL called
[gstreamer] gstreamer changed state from NULL to READY ==> nvvconv2
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter6
[gstreamer] gstreamer changed state from NULL to READY ==> decoder
[gstreamer] gstreamer changed state from NULL to READY ==> v4l2src2
[gstreamer] gstreamer changed state from NULL to READY ==> pipeline3
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter7
[gstreamer] gstreamer changed state from READY to PAUSED ==> nvvconv2
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter6
[gstreamer] gstreamer changed state from READY to PAUSED ==> decoder
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> v4l2src2
[gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline3
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer message new-clock ==> pipeline3
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter7
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> nvvconv2
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter6
[gstreamer] gstreamer message stream-start ==> pipeline3
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> decoder
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> v4l2src2
[gstreamer] gstCamera -- end of stream (EOS)
[gstreamer] gstreamer v4l2src2 ERROR Internal data stream error.
[gstreamer] gstreamer Debugging info: ../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline3/GstV4l2Src:v4l2src2:
streaming stopped, reason not-negotiated (-4)
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysink
[gstreamer] gstreamer message latency ==> mysink
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:54 | WARNING | video source /dev/video0 timed out during capture, re-trying...
[gstreamer] gstCamera::Capture() -- a timeout occurred waiting for the next image buffer
17:48:57 | WARNING | video source /dev/video0 timed out during capture, re-trying...
@dusty-nv
Copy link
Owner 

[gstreamer] gstreamer v4l2src1 ERROR Internal data stream error.
[gstreamer] gstreamer Debugging info: ../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline2/GstV4l2Src:v4l2src1:
streaming stopped, reason not-negotiated (-4)

Hi @costasvav , I'm not familiar with which CSI cameras you are using or their drivers, but typically in this situation it's because the libargus/nvarguscamerasrc API should be used and not the V4L2 /dev/video raw interface.

Can you try creating the camera as csi://0 instead of /dev/video0 ?

Also, if you look through the log, you can see the gstreamer pipeline it constructs for the camera capture. If you continue having to debug this, just drop back to getting it to work from gstreamer on the command line first. Then once you know the edits required, you can make the changes back in the source. If a similar variant doesn't work with gst-launch then it's also unlikely to work in the app.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@costasvav @dusty-nv and others