intelrealsense / realsense_mipi_platform_driver Goto Github PK

Hello!
I'm trying to run RealSense D457 camera on AGX Orin.

My hardware environment are:

• Intel RealSense D457 camera * 1
• Nvidia Jetson AGX Orin DevKit (Jetpack5.0.2) * 1
• Leopard E3653-A03 board (MAX96712) * 1
• Leopard LI-FCB-4T1-SS-2M-NP-T1 cable * 1

In examples D457 is used with GMSL boards based on MAX 9296, but Leopard E3653-A03 board has MAX96712 inside.
Changing device tree is not enough to run D457+MAX96712, because there are some patches to MAX 9296 driver.

Is there any way to get the patch for MAX96712?
Kind regards

Is there any x86 platform support for the D457?

BG, Thomas

is Jetpack 5.1 supported?

Hi, I have a question about how the driver handles two D457 cameras and, more specifically, the max9295 serializer.

Looking at the device tree, if I have two D457, it seems like the driver would assign 0x42 and 0x60 to each max9295 respectively.
I presume that the max9295 on each D457 has the same initial i2c address? If that's true, and if both D457s were connected to one max9296, how then does the driver set an independent i2c address for each max9295? Wouldn't there be a bus conflict?

Thanks in advance!

I sometimes see the below Kernel log after restarting depth streaming capture while capturing metadata.
It happens on both JP4.6.1 and JP5.0.2. I used v4l2-ctl to test using these commands.

• Enable metadata catpture
  v4l2-ctl -d /dev/video1 --stream-mmap

• Enable depth stream capture:
  v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=720,pixelformat=UYVY --stream-mmap

• Kernel error log after some depth capture restarts

[  459.498562] ------------[ cut here ]------------
[  459.498746] WARNING: CPU: 3 PID: 8781 at /home/ANT.AMAZON.COM/raen/work/realsense_mipi_platform_driver-jp4.6/sources_4.6.1/kernel/kernel-4.9/drivers/media/v4l2-core/videobuf2-core.c:905 vb2_buffer_done+0x1f8/0x258
[  459.499093] Modules linked in: bnep fuse xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink xt_addrtype iptable_filter iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack br_netfilter zram overlay userspace_alert bluedroid_pm d4xx nvgpu ip_tables x_tables

[  459.499246] CPU: 3 PID: 8781 Comm: vi-output, DS5  Tainted: G        W       4.9.253-tegra #2
[  459.499252] Hardware name: Jetson-AGX (DT)
[  459.499258] task: ffffffc3d091e200 task.stack: ffffffc38ef18000
[  459.499266] PC is at vb2_buffer_done+0x1f8/0x258
[  459.499289] LR is at vi5_capture_dequeue+0x508/0x878
[  459.499298] pc : [<ffffff8008b2dbf0>] lr : [<ffffff8008b452e0>] pstate: 20c00045
[  459.499303] sp : ffffffc38ef1bd00
[  459.499308] x29: ffffffc38ef1bd00 x28: ffffffc3d78a6018 
[  459.499323] x27: 0000000000000000 x26: ffffffc3d7dd4048 
[  459.499337] x25: 0000000000000000 x24: 0000000000000000 
[  459.499352] x23: 0000000000000002 x22: 0000000000000005 
[  459.499366] x21: ffffffc3d78a6930 x20: 00000000000001da 
[  459.499378] x19: ffffffc3ce74bc00 x18: 000000000000153b 
[  459.499392] x17: 0000007f801fc530 x16: 0000000000000001 
[  459.499406] x15: 000000000000317b x14: 0000000000000000 
[  459.499419] x13: 000000000007fa4a x12: 0000000000006c06 
[  459.499433] x11: 0000000000000000 x10: 0000000000000a10 
[  459.499448] x9 : ffffffc38ef1bbd0 x8 : ffffffc3d091ec70 
[  459.499462] x7 : 0000000002e9f800 x6 : 0000000000006c06 
[  459.499475] x5 : 0000000000000400 x4 : 0000000000000000 
[  459.499488] x3 : 0000000000000000 x2 : 0000000000000000 
[  459.499501] x1 : 0000000000000005 x0 : 00000000ffffffff 

[  459.499519] ---[ end trace 49a7388e7c4f58fc ]---
[  459.499619] Call trace:
[  459.499632] [<ffffff8008b2dbf0>] vb2_buffer_done+0x1f8/0x258
[  459.499641] [<ffffff8008b452e0>] vi5_capture_dequeue+0x508/0x878
[  459.499651] [<ffffff8008b45908>] tegra_channel_kthread_capture_dequeue+0x90/0x1c0
[  459.499665] [<ffffff80080db0bc>] kthread+0xec/0xf0
[  459.499678] [<ffffff80080838a0>] ret_from_fork+0x10/0x30

Have you seen this issue before?

-Enrique

https://github.com/IntelRealSense/perc_hw_ds5u_android-jetson_tx2/blob/83166f3a7eb0364e2b5bfbbfb9da4972b0601927/build_all.sh#L29
This cause build failure as it try to access "/kernel/"

In the modification to the device tree the tegra_sinterface for device d4m1 is set to "serial_e" (line 267) but the corresponding CSI channel endpoint is set to port-index = <0> (line 451) which corresponds to "serial _a" on the Xavier.

How are the drivers receiving RGB data in this case?

Hi,
I am trying to use Intel D457 camera on AGX Xavier.

My hardware environment are:

• Nvidia Jetson AGX Xavier EVB * 1
• Leopard LI-JXAV-MIPI-ADPT-4CAM board * 1
• Leopard LI-GMSL2-IPX-DESER board * 1
• Leopard FAW-1233-03 cable *1
• Intel RealSense D457 camera * 1
• GMSL FAKA cable * 1

1. git clone the Intel D457 MIPI driver for Jetson AGX Xavier:
   $ git clone https://github.com/IntelRealSense/realsense_mipi_platform_driver

2. setup_workspace, apply_patches, build_all_deb on PC.
   $ ./setup_workspace.sh 5.0.2
   $ ./apply_patches.sh apply 5.0.2
   $ ./build_all_deb.sh 5.0.2

3. Flash JetPack 5.0.2 to the Jetson AGX Xavier EVB

4. Install Intel D457 mipi driver by .deb on AGX Xavier.
   $ sudo dpkg -i ./linux-image-5.10.104-d457_5.10.104-d457-1_arm64.deb
   $ sudo dpkg -i ./linux-headers-5.10.104-d457_5.10.104-d457-1_arm64.deb
   $ sudo dpkg -i ./linux-libc-dev_5.10.104-d457-1_arm64.deb
   $ sudo dpkg -i ./linux-image-5.10.104-d457-dbg_5.10.104-d457-1_arm64.deb

5. Edit /boot/extlinux/extlinux.conf on AGX Xavier.
   LINUX /boot/Image-5.10.104-d457
   FDT /usr/lib/linux-image-5.10.104-d457/tegra194-p2888-0001-p2822-0000.dtb

6. Create /etc/modules-load.d/d4xx.conf on AGX Xavier.
   $ echo "d4xx" | sudo tee /etc/modules-load.d/d4xx.conf

7. Install Intel Real Sense Viewer tool
   $ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE || sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE
   sudo add-apt-repository "deb https://librealsense.intel.com/Debian/apt-repo $(lsb_release -cs) main" -u
   $ apt-get update
   $ sudo apt-get -y install librealsense2-utils
   $ sudo apt-get -y install librealsense2-dev
   $ sudo apt-get -y install librealsense2-dbg
   $ reboot

8. But we can't see /dev/video0, /dev/video1, /dev/video2, /dev/video3 after reboot on AGX Xavier, and realsense-viewer also can't see any video stream.

My question is :
May I use this GitHub Intel D457 mipi driver for Leopard LI-JXAV-MIPI-ADPT-4CAM + FAW-1233-03 + LI-GMSL2-IPX-DESER + Intel RealSense D457 camera + AGX Xavier?

Hi @xzhangxa ,
JP5.0.2 Debian build failure, if you need more info please contact Dima.

the debians failed on dts step

make KERNELRELEASE=5.10.104-d457 ARCH=arm64 KBUILD_BUILD_VERSION=1 -f /home/dima/work/jetson-deb/perc_hw_ds5u_android-jetson_tx2/sources_5.0.2/kernel/kernel-5.10/Makefile intdeb-pkg
sh /home/dima/work/jetson-deb/perc_hw_ds5u_android-jetson_tx2/sources_5.0.2/kernel/kernel-5.10/scripts/package/builddeb
make[7]: *** No rule to make target 'debian/linux-image/usr/lib/linux-image-5.10.104-d457/../../../../../../hardware/nvidia/platform/t23x/concord/kernel-dts/tegra234-p3701-0000-p3737-0000.dtb', needed by '__dtbs_install'. Stop.
make[6]: *** [/home/dima/work/jetson-deb/perc_hw_ds5u_android-jetson_tx2/sources_5.0.2/kernel/kernel-5.10/Makefile:1401: dtbs_install] Error 2
make[5]: *** [/home/dima/work/jetson-deb/perc_hw_ds5u_android-jetson_tx2/sources_5.0.2/kernel/kernel-5.10/scripts/Makefile.package:87: intdeb-pkg] Error 2
make[4]: *** [/home/dima/work/jetson-deb/perc_hw_ds5u_android-jetson_tx2/sources_5.0.2/kernel/kernel-5.10/Makefile:1582: intdeb-pkg] Error 2
make[3]: *** [debian/rules:13: binary-arch] Error 2
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2

Need to apply this fix to JP 4.6.1.
(Jetpack-5.0.2 branch)

https://github.com/IntelRealSense/perc_hw_ds5u_android-jetson_tx2/blob/12034a267262b8c8685f2b750146ab9c0bc70dc0/kernel/nvidia/4.6.1/0073-D4XX-Klocwork.patch#L28