July 18, 2025

• No idea on which of the two networks the UGV was, so checked with map -sP 146.*.*.0/24. Only two machines on this network. Switched to the other network. Still on the old network.
• Looking at the other nodes from bringup_imu_ekf.launch.py. Yesterday I already looked at base_node.
• The ekf_node points to the package robot_localization. That package was actually not installed on the UGV rover. Added it to the manual. Reads the /odom_raw and published a filtere version. The documentation is quite rudimentary, but ugv_bringup used its own param/ekf.yaml, which indicates that it only uses the pose (x,y,yaw), and the x_vel and yaw_vel.
• Also the package imu_complementary_filter seems not installed. See paper (2015). No parameter-file, but 5-parameters are set in the launch-file. Those are the same as the settings in imu_complementary_filter/config/filter_config.yaml. Installed sudo apt install ros-humble-imu-tools on UGV Rover.
• Last important one is ldlidar package, which has its own ldlidar.launch.py. This points to ld19.launch.py, which has as 2nd node static_transform_publisher between the base_footprint and base_lidar_link, but with a null-vector (and the 2nd node is not launched). Check the robot-description if that contains a better transform between laser and base.
• Indeed, in the ugv_rover.urdf the vector between base_lidar_link_joint to base_lidar_link (xyz="0.04 0 0.04") to base_link (xyz="0.0 0.0 0.0165") are defined.
• Looked at the lines echo "eval "$(register-python-argcomplete ros2)"" >> ~/.bashrc. According to argcomplete documentation, this is needed when no global completion is activated. pip install argcomplete was already installed on UGV Rover, but the commmand activate-global-python-argcomplete could be found. Added manually python-argcomplete.sh in the PATH (~/.local/bin/)
• Anyway, started ros2 launch ugv_bringup bringup_imu_ekf.launch.py. Checking with ros2 node list. 6 nodes are running:
  /LD19
  /base_node
  /complementary_filter_gain_node
  /ekf_filter_node
  /transform_listener_impl_aaaabb524e10
  /ugv/robot_state_publisher
  /ugv_bringup
  
• Also checked the published topics with ros2 topic list: cmd_vel
  /diagnostics
  /imu/data
  /imu/data_raw
  /imu/mag
  /joy
  /odom
  /odom/odom_raw
  /odom_raw
  /parameter_events
  /rosout
  /scan
  /set_pose
  /tf
  /tf_static
  /ugv/joint_states
  /ugv/led_ctrl
  /ugv/robot_description
  /voltage
  Note that there two odom_raw topics are published. Note also the /scan and /ugv/robot_description
• Succes full launch ends with these two lines:
  [ldlidar_node-5] [INFO] [1752846995.480029725] [LD19]: ldlidar communication is normal.
  [ldlidar_node-5] [INFO] [1752846995.483222001] [LD19]: Publish topic message:ldlidar scan data.
• Started on nb-dual (native Ubuntu 20.04) the RoboStack environment with mamba activate ros_humble_env. Could start ros2 run rviz2 rviz2, which showed the laser-scan. Only fails on /ugv/robot_description, because package ugv_description couldn't be found.
• Did the first step of section 5.2 of the manual: clone the ugv_ws in my home.
• Did the first of the installs with install ros-humble-nav2-msgs ros-humble-map-msgs ros-humble-nav2-costmap-2d. That installs 8 packages:
  + ros-humble-bond 3.0.2
  + ros-humble-bondcpp 3.0.2
  + ros-humble-nav2-common 1.1.5
  + ros-humble-nav2-costmap-2d 1.1.5
  + ros-humble-nav2-msgs 1.1.5
  + ros-humble-nav2-util 1.1.5
  + ros-humble-nav2-voxel-grid 1.1.5
  + ros-humble-smclib 3.0.2
  
• mamba install ros-humble-image-geometry was not needed, already installed.
• mamba install ros-humble-rosbridge-suite installed 29 packages:
  + anyio 4.9.0
  + appdirs 1.4.4
  + automat 25.4.16
  + bcrypt 4.3.0
  + constantly 15.1.0
  + dnspython 2.6.1
  + h11 0.16.0
  + h2 4.2.0
  + hpack 4.1.0
  + httpcore 1.0.9
  + hyperframe 6.1.0
  + hyperlink 21.0.0
  + incremental 24.7.2
  + pyasn1 0.6.1
  + pyasn1-modules 0.4.2
  + pymongo 4.13.0
  + pyopenssl 25.1.0
  + ros-humble-rosapi 1.3.1
  + ros-humble-rosapi-msgs 1.3.1
  + ros-humble-rosbridge-library 1.3.1
  + ros-humble-rosbridge-msgs 1.3.1
  + ros-humble-rosbridge-server 1.3.1
  + ros-humble-rosbridge-suite 1.3.1
  + service-identity 24.2.0
  + service_identity 24.2.0
  + sniffio 1.3.1
  + tornado 6.5.1
  + twisted 25.5.0
  + zope.interface 7.2
  
• The command mamba install ros-humble-nav2-bringup installed 44 packages:
  + ceres-solver 2.1.0 + chrpath 0.16 + ghostscript 10.04.0 + glog 0.6.0 + gnuplot 5.4.5 + graphicsmagick 1.3.37 + metis 5.1.0 + mpfr 4.2.1 + ompl 1.5.2 + ros-humble-behaviortree-cpp-v3 3.8.2 + ros-humble-costmap-queue 1.1.5 + ros-humble-dwb-core 1.1.5 + ros-humble-dwb-critics 1.1.5 + ros-humble-dwb-msgs 1.1.5 + ros-humble-dwb-plugins 1.1.5 + ros-humble-nav-2d-msgs 1.1.5 + ros-humble-nav-2d-utils 1.1.5 + ros-humble-nav2-amcl 1.1.5 + ros-humble-nav2-behavior-tree 1.1.5 + ros-humble-nav2-behaviors 1.1.5 + ros-humble-nav2-bringup 1.1.5 + ros-humble-nav2-bt-navigator 1.1.5 + ros-humble-nav2-collision-monitor 1.1.5 + ros-humble-nav2-constrained-smoother 1.1.5 + ros-humble-nav2-controller 1.1.5 + ros-humble-nav2-core 1.1.5 py310h7c61026_3 robostack-staging/linux-64 19kB + ros-humble-nav2-dwb-controller 1.1.5 py310h7c61026_3 robostack-staging/linux-64 12kB + ros-humble-nav2-lifecycle-manager 1.1.5 py310h7c61026_3 robostack-staging/linux-64 164kB + ros-humble-nav2-map-server 1.1.5 py310h7c61026_3 robostack-staging/linux-64 304kB + ros-humble-nav2-navfn-planner 1.1.5 py310h7c61026_3 robostack-staging/linux-64 65kB + ros-humble-nav2-planner 1.1.5 py310h7c61026_3 robostack-staging/linux-64 281kB + ros-humble-nav2-regulated-pure-pursuit-controller 1.1.5 py310h7c61026_3 robostack-staging/linux-64 114kB + ros-humble-nav2-rotation-shim-controller 1.1.5 py310h7c61026_3 robostack-staging/linux-64 139kB + ros-humble-nav2-rviz-plugins 1.1.5 py310h7c61026_3 robostack-staging/linux-64 471kB + ros-humble-nav2-simple-commander 1.1.5 py310h7c61026_3 robostack-staging/linux-64 61kB + ros-humble-nav2-smac-planner 1.1.5 py310h7c61026_3 robostack-staging/linux-64 773kB + ros-humble-nav2-smoother 1.1.5 py310h7c61026_3 robostack-staging/linux-64 273kB + ros-humble-nav2-theta-star-planner 1.1.5 py310h7c61026_3 robostack-staging/linux-64 56kB + ros-humble-nav2-velocity-smoother 1.1.5 py310h7c61026_3 robostack-staging/linux-64 174kB + ros-humble-nav2-waypoint-follower 1.1.5 py310h7c61026_3 robostack-staging/linux-64 943kB + ros-humble-navigation2 1.1.5 py310h7c61026_3 robostack-staging/linux-64 14kB + ros-humble-ompl 1.6.0 py310he3f1553_3 robostack-staging/linux-64 3MB + ros-humble-slam-toolbox 2.6.4 py310hb344815_3 robostack-staging/linux-64 3MB + suitesparse 5.10.1 h9e50725_1 conda-forge/linux-64 3MB
• mamba install ros-humble-depthai-* couldn't be found, so only tried mamba install ros-humble-usb-cam. This fails on dependency:
  The following package could not be installed
  └─ ros2-distro-mutex 0.6* humble_* is requested and can be installed.
• Tried the --only-deps option, to no avail. Continued to the next package.
• Installing mamba install ros-humble-joint-state-publisher-* installed two packages:
  + ros-humble-joint-state-publisher 2.3.0
  + ros-humble-joint-state-publisher-gui 2.3.0
• Next is mamba install ros-humble-robot-localization (already installed).
• Last is mamba install ros-humble-imu-tools (doesn't exist).
• Starting to build. emcl2 failed, which can be skipped. Yet, also ldlidar fails, which is more a problem. Yet, probably less needed on the laptop side.
• At least the ugv_description now works under RoboStack:
  

July 17, 2025

• Found a solution for nvpmodel crash (nvpmode-SegFixed at the end).
• New problem, firefox will no longer start. Do a snap install firefox? No, firefox is already installed.
• Seems to be a SELinux protection added by Waveshare (see this discussion).
• Instead of installing SELinux protection, I tried instead to do snap remove firefox. firefox was actually connected with a lot of other snap-packages, such as camera and joystick. Trying snap install firefox again. Still same error.
• Installing via apt install firefox fails, because it is already installed via snap.
• Checking SELinux install. systctl status apparmor indicated inactive (dead) - start condition failed June 4, 2025.
• After installing SELinux it start relabeling all filessystems. Note that there is an user sddm with no passord. Relabbbeling costs a few minutes before the boot continues.
• Got a snapd-error, so followed this advice and disabled SELinux. Now firefox can bes installed, but a new error: cannot set capabilities: Operation not permited.
• Installed brave-browser instead. Gives a warning about libva, but works. Installing va-driver-all didn't help.
• Installed the nvpmode-SegFixed (via remote terminal). That works.
• Looking at the launch files in ugv_bringup. The bringup_imu_ekf.launch.py launches 10 nodes, bringup_imu_origin.launch.py and bringup_lidar.launch.py launches 9 nodes.
• The most basic seems to be the driver_node, which is an executable om the ugv_bringup package.
• Running ros2 run ugv_bringup ugv_driver & gives the following topics:
  /cmd_vel
  /joy
  /ugv/joint_states
  /ugv/led_ctrl
  /voltage
  
• Should look at pan_tilt_ros package.
• First looked at the ugv_driver.py code. The pan/tilt are controlled with joy-messages, drive-angle is axis 0, drive-speed is axis 1, pan is axis 3, tilt is axis 4 (inverted). axes 2 is not used. The pan-tilt can also be contorlled with the joint_state (pt_base_link_to_pt_link1 - pan) and (pt_link1_to_pt_link2 - tilt). The joint_state is given in radians.
• The driver subscribes to the voltage (and plays a sound when low), so in principal the ugv_bringup.py should be the first to be launched. It also publishes imu_data_raw, imu_mag and odom_raw.
• Was able to read out the voltage and drive the rover forward with /cmd_vel.
• Which node launches the pan-tilt camera and the depth camera. I know from June 26 that the depth-camera can be started with ros2 launch depthai_ros_driver camera.launch.py pointcloud.enable:=True.
• The base_node is node that reads in the imu_raw message and publishes odometry messages and when configured also the odom_frame transform.
• The ugv_vision package calls the usb_cam package with its own config/params.yaml. It also launches from image_proc pakage a RectifyNode. The two nodes are combined into a single container.
• The command ros2 launch ugv_vision camera.launch.py seems to work:
  [component_container-2] [INFO] [1752759640.742645655] [image_proc_container]: Load Library: /opt/ros/humble/lib/librectify.so
  [usb_cam_node_exe-1] [INFO] [1752759640.885099049] [usb_cam]: camera_name value: pt_camera
  [usb_cam_node_exe-1] [WARN] [1752759640.885346962] [usb_cam]: framerate: 30.000000
  [usb_cam_node_exe-1] [INFO] [1752759640.892700837] [usb_cam]: camera calibration URL: package://ugv_vision/config/camera_info.yaml
  [component_container-2] [INFO] [1752759640.907117633] [image_proc_container]: Found class: rclcpp_components::NodeFactoryTemplate
  [component_container-2] [INFO] [1752759640.907235493] [image_proc_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate
  [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/rectify_color_node' in container '/image_proc_container'
  [usb_cam_node_exe-1] [INFO] [1752759640.971137276] [usb_cam]: Starting 'pt_camera' (/dev/video0) at 640x480 via mmap (mjpeg2rgb) at 30 FPS
  [usb_cam_node_exe-1] [swscaler @ 0xaaab14a93020] No accelerated colorspace conversion found from yuv422p to rgb24.
  [usb_cam_node_exe-1] [INFO] [1752759641.007471467] [usb_cam]: Setting 'white_balance_temperature_auto' to 1
  [usb_cam_node_exe-1] [INFO] [1752759641.007589456] [usb_cam]: Setting 'exposure_auto' to 3
  [usb_cam_node_exe-1] [INFO] [1752759641.012967833] [usb_cam]: Setting 'focus_auto' to 0
  [usb_cam_node_exe-1] [INFO] [1752759641.520032053] [usb_cam]: Timer triggering every 33 ms
  
• Run bot ros2 run image_view image_view --ros-args --remap image:=/image_raw and image:=/image_rect. The rectification looks good. Adding image_transport:=theora and image_transport:=compressed didn't work.
  
• The distorion coefficients are [-0.208848, 0.028006, -0.000705, -0.000820, 0.000000]. The camera matrix is not exact in the center of the 640x480 image, (347.23, 235.67). The diagonal element is (289.11,289.75) The rectifcation matrix is the Identity-matrix, but the project+matrix has diagonal elements (196.86,234.53) and off-diagioanl (342.88, 231.54).
• According to ROS1 documentation the theora codec only works with 8-bit color or grayscale images. Not sure if the commment using transport "raw" is a default INFO from image_view. Remember that there was a bug in image_transport
• The Color Tracking example has a gimbal_track() function.
• Looked at the UGV rover in ~/git/ugv_jetson. The start_jupyter.sh is there, the noteboks are in the directory tutorial_en.
• Started with Tutorial #3 - pan-tilt control.
• Tutorial #14 fails on cv2.error: OpenCV(4.9.0) /io/opencv/modules/imgcodecs/src/loadsave.cpp:1121: error: (-215:Assertion failed) !image.empty() in function 'imencode'.
• Same for Tutorial #20, the one I liked to reproduce. Made color_tracking_node.py in ugv_custom_nodes, a copy of the line_following_node.py. Should only stear the pan-tilt instead of the wheels.

July 16, 2025

• Starting to use the UGV Rover at home. The small OLED screen doesn't show the Ethernet address anymore (as described at Waveshare documentation.
• Luckely the hotspot is still there, advertised as UGV. Yet, doesn't work with the password in the documentation.
• Connecting via the Jetson's ethernet connector is blocked by a Rover connector. Tried to connect via two USB-C to ethernet connectors, but don't see the ethernet lights up. Connecting with the suggested default IP address didn't work.
• Instead will connect the Display-port, and make the connection directly to my home access-point.
• Connecting to my home access-point with the suggested sudo nmcli d wifi list and connect worked.
• Could use the wifi-connection to ssh into the jetson with the IP checked with ifconfig
• Started with the suggested software update (including several ros-humble packages - which fails).
• Used the suggested temporary fix from this discussion (three weeks ago). Now 679 packages are updating.
• Looking for instructions to login to the UGV with USB, I found the battery and board instructions on youTube
• This video shows the UGV lower body. The thing blocking the internet port is indeed a wifi antenna. You can control the robot directly, without the Jetson, via JSON commands.
• Could login to UGV lower body, use the web-interface and even control the wheels.
• In the FAQ also an indication is made which 18650 batteries to use.
• In principal you can control the lower body also from the Jetson, you only have to know the serial interface (/dev/*?) of the Jetson PIN-40 interface.
• Watched a video on the Jetson PIN-overlays.
• According to this tutorial, the device is dev/ttyTHS0. Script didn't work, because base_ctrl wasn't installed.
• There is a setup.sh script according to Main Program Architecture, which should be part of ugv_pt_rpi.
• Indeed, base_ctrl.py is part of this repository.
• Yet, nor /dev/ttyACM0 nor /dev/ttyTHS1 changed the OLED (no warnings, so the JSON commands where succesfull send to these devices).
• Starting with colcon build --packages-select costmap_converter. Fails on abstract class. The compiler is g++ version 11.4.0, in CMakeLists.txt is specified CXX_STANDARD 14. These are the supported values. Setting it to 26 was too much. Start getting memory problems.
• Could look at jetson-min-desk.
• Tried CXX_STANDARD 23 and 20, but still a crash (although no error). Removed option -Wpedantic, and tried CXX_STANDARD 11, removed -Wextra and tried CXX_STANDARD 98, removed -Wall and tried CXX_STANDARD 17
• Removed thunderbird, but I have the feeling that the problem is memory, not storage-space.
• Looked at colcon build and used a single job with MAKEFLAGS=-j1 colcon build --executor sequentail. Now I get the error message again, and no complains about memory.
• Commented out the line in map_to_dynamic_obstacles/blob_detector.cpp
• Strange, this should be version 0.1.2, as I can see the blob_detecor in the humble version at github.
• This issue describes my problem, which is solved in this pull request.
• This patch solves the iisue. Still some warnings, but the package is built.
• Could do source ~/ugv_ws/install/setup.bash without warnings.

June 27, 2025

• Checked the Jetpack version with sudo apt-cache show nvidia-jetpack, which showed version 6.0+b106.

June 26, 2025

• OpenCV blog-post on edge-detection on a AI-generated image.
• The Knowledge Representation was very logic based, only watched the first unit.
• Also looked at the other MOOC course, Robots in Action. Looked at the first video of RoboEthics.
• Tried again to get the point-cloud from the RAE.
• Started docker again, went to /underlay_ws/install/depthai_examples/share/depthai_examples/launch/. Activated point_cloud_xyzi and metric_converter in rgb_stereo_node.launch.py script.
• Launched ros2 launch depthai_examples rgb_stereo_node.launch.py camera_model:=RAE. Could see the different topics.
• Started on WS9 rviz2, and added displays on different topics. Only received an image from /color/video/image, received nothing from /stereo/depth.
• Opened another terminal on the RAE. Outside the docker I see two processes active: depthai-device (31.8%) and rgb_stereo_node (15.8%). Not clear how I can see them outside the docker!.
• Inside the docker I only see rgb_stereo_node (15.8%).
• With ros2 node list I see:
  /container
  /covert_metric_node
  /launch_ros_76
  /point_cloud_xyzi
  '
• The only topic which gives me updates is /color/video/camera_info.
• Back to the ugv_rover. The oak_d_lite.launch.py starts the camera.launch.py from depthai_ros_driver. Inside the camera the pointcloud.enable is default False.
• The example_multicam.launch.py launches both camera and rgbd_pcl.launch.py.
• pointcloud.launch.py uses the PointCloudXyziNode plugin, and should publish /points.
• rgbd_pcl.launch.py starts camera.launch.py with pointcloud.enable True.
• rtabmap.launch.py also starts rtabmap_odom.
• Running ros2 launch depthai_ros_driver camera.launch.py pointcloud.enable:=True works, I could see the pointcloud2 (once I set the QoS to best_effort and the frame to oak_camera_frame):
  
• Received the invitation to preview Luxonis stereo-tuning-assistance
• The capture-package needed matplotlib, which conflicted with ros. Created a virtual environment, did the requirements install there and run ../capture_env/bin/python3.11 capture/oak_capture.py default /tmp/my_capture, which gives a X_LINK_ERROR (and a warning on no IR drivers and no RGB camera calibration). So, as long as I have not installed the RVC3_support in the venv, this doesn't work.
• Back to the ugv_rover. Tried ros2 launch ugv_slam gmapping.launch.py use_rviz:=false, but package slam_gmapping unknown.
• In ugv_ws/src/ugv_else/gmapping two directories can be found: openslam_gmapping and slam_gmapping. According to the README.md this can be started with ros2 launch slam_gmapping slam_gmapping.launch.py, but it seems that this directory is not included in the build.
• Had to be done in order. Did colcon build --packages-select openslam_gmapping followed by colcon build --packages-select slam_gmapping.
• To get rid of the source install/setup.sh I also did colcon build --packages-select explore_lite which worked. Unfortunately colcon build --packages-select costmap_converter failed on line 56 of blob_detector,cpp, which seems to be an C++-version problem (comment mentions compatibility).
• Running ros2 launch ugv_slam gmapping.launch.py use_rviz2:=false now works, in the sense that the LaserScan are visible. Could also select the Map inside RVIZ, but with the warning no Map received. Changing the Reliability QoS to Best effort solves that:
  
• Started ros2 launch ugv_bringup bringup_lidar.launch.py (does this means a double lidar_node?) to be able to do ros2 run ugv_tools keyboard_ctrl. The turning is a bit slow, could be increased by e. Going forward / backwards also helps.
• gmapping was able to make a map, although not perfect (two over each other). Can you reset the map without killing gmapping?
• Second try was much better, could not only map the small maze, but also going back to the door, followed by going left halfway the soccer field:
  

June 25, 2025

• Skipping two weeks of topics of the Robotics in a Nutshell MOOC (Force Control and Knowledge Representation).
• The 5th week is on Graph-SLAM. The description of Wolfram Burghard was a bit too high-level, the description of Giorgio Grisetti a bit too low-level. Minimizing all constraints is simple, but the least-square method is not explained in the best way, with a lot of math but without introducing all functions and operations. Maybe the chapter is better.
• The description of the manifold in Unit 2.4 is nice: "A manifold is a mathematical space that is not necessarily Euclidean on a global scale but can be seen as Euclidean on a local scale."
• Cyrill Stachniss presents the last Unit, with some reading material at the end. Most known, yet the video's on adaptive robust kernels are probably from the paper "Adaptive Robust Kernels for Non-Linear Least Squares Problems'. (April 2021)
• The supplemental material only show the overtake example, not the full mapping.
• I like the idea of replacing L1 or L2 with more robust kernels, to prevent sensitivity to outliers. Will this also work in scikit-learn:
  
• Started the RAE #1. Before I could ssh into the system, the RAE started already driving on one wheel. Once I started ros2 launch rae_bringup robot.launch.py enable_slam_toolbox:=false enable_nav:=false use_slam:=false in the docker the driving stopped.
• Connected to another shell into the docker. Looked with ros2 topic list. Several front and back images are published, yet no point-cloud.
• Inside the docker python3.10.12 is running.
• Inside python3 import depthai as dai doesn't work, so tried python3 -m pip install depthai. No module pip.
• Did sudo apt install python3-pip. Now I could isntall depthai-2.30.0.0.
• Also did sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg, because I had the ros gpg error again. At least 464 packages behind.
• Cloned depthai-python in /tmp/git and run python3 calibration_reader.py. Gave the following error:
  RuntimeError: No available RVC2 devices found, but found 3 non RVC2 device[s]. To use RVC4 devices, please update DepthAI to version v3.x or newer. .
• On October 31, 2024 I did something, directly playing with the depthai_ros_driver with DEPTHAI_DEBUG=1.
• Looked in /ws/src/rae-ros/rae_camera/src/camera.cpp. Code also depends on depthai (and depthai_bridge), but that seems to be cpp-library, not the python-module.
• For instance, /ws/src/rae-ros/rae_camera/stereo_node is an executable which loads /underlay_ws/install/depthai_bridge/lib/libdepthai_bridge.so, /usr/local/lib/libdepthai-core.so and /usr/local/lib/cmake/depthai/dependencies/lib/libusb-1.0.so.
• Tried ros2 launch rae_camera perception_ipc.launch.py, which conflicts with the already running nodes.
• Killed the other node, now I get several nodes running:
  /RectifyNode
  /battery_node
  /complementary_filter_gain_node
  /controller_manager
  /diff_controller
  /ekf_filter_node
  /joint_state_broadcaster
  /laserscan_kinect_back
  /laserscan_kinect_front
  /laserscan_multi_merger
  /launch_ros_1345
  /lcd_node
  /led_node
  /mic_node
  /rae
  /rae_container
  /robot_state_publisher
  /rtabmap
  /speakers_node
  /transform_listener_impl_55ba0cad70
  /transform_listener_impl_55ba4dfdb0
  /transform_listener_impl_55bbafe9d0
  
• Also now get the topic /scan
• Looked at my WLS-terminal with ROS-humble. The command ros2 node list gave a subset:
  /complementary_filter_gain_node /controller_manager /diff_controller /ekf_filter_node /joint_state_broadcaster /mic_node /robot_state_publisher /speakers_node /transform_listener_impl_55bbafe9d0
• Also the ros2 topic list is a subset:
  /battery_status
  /lcd
  /leds
  
• During the start of the perception_ipc script, I get the following startup information:
  [perception_ipc_rtabmap-1] [INFO] [1750858282.709590462] [rae]: Camera with MXID: xlinkserver and Name: 127.0.0.1 connected! [perception_ipc_rtabmap-1] [INFO] [1750858282.709810547] [rae]: PoE camera detected. Consider enabling low bandwidth for specific image topics (see readme). [perception_ipc_rtabmap-1] [58927016838860C5] [127.0.0.1] [1750858282.710] [system] [info] Reading from Factory EEPROM contents [perception_ipc_rtabmap-1] [INFO] [1750858282.741301485] [rae]: Device type: RAE [perception_ipc_rtabmap-1] [INFO] [1750858283.008338842] [rae]: Pipeline type: rae [perception_ipc_rtabmap-1] [WARN] [1750858283.062986091] [rae]: Resolution 800 not supported by sensor OV9782. Using default resolution 720P [perception_ipc_rtabmap-1] [WARN] [1750858283.082811496] [rae]: Resolution 800 not supported by sensor OV9782. Using default resolution 720P [perception_ipc_rtabmap-1] [WARN] [1750858283.134807345] [rae]: Resolution 800 not supported by sensor OV9782. Using default resolution 720P [perception_ipc_rtabmap-1] [WARN] [1750858283.154057327] [rae]: Resolution 800 not supported by sensor OV9782. Using default resolution 720P [perception_ipc_rtabmap-1] [INFO] [1750858283.945303168] [rae]: Finished setting up pipeline.
• Followed (after initialization of the IMU) with:
  [perception_ipc_rtabmap-1] [INFO] [1750858284.488374295] [rae]: Camera ready! [perception_ipc_rtabmap-1] [58927016838860C5] [127.0.0.1] [1750858284.597] [StereoDepth(11)] [info] Baseline: 0.074863344 [perception_ipc_rtabmap-1] [58927016838860C5] [127.0.0.1] [1750858284.597] [StereoDepth(11)] [info] Fov: 96.69345 [perception_ipc_rtabmap-1] [58927016838860C5] [127.0.0.1] [1750858284.597] [StereoDepth(11)] [info] Focal: 284.64175 [perception_ipc_rtabmap-1] [58927016838860C5] [127.0.0.1] [1750858284.597] [StereoDepth(11)] [info] FixedNumerator: 21309.232 [perception_ipc_rtabmap-1] [58927016838860C5] [127.0.0.1] [1750858284.597] [StereoDepth(11)] [debug] Using 0 camera model [perception_ipc_rtabmap-1] [INFO] [1750858284.806388901] [laserscan_kinect_front]: Node laserscan_kinect initialized. .
• Note that in /ws/src/rae-ros/rae_camera/config/cal.json can be found, for camera-model RAE (version 7).
• Not clear where all those nodes are started, the launch file starts one executable (perception_ipc_rtabmap) and the control.launch.py.
• The file /ws/build/rae-camera/perception_ipc_rtabmapi is a real exec, loading for instance: /opt/ros/humble/lib/librtabmap_slam_plugins.so
  /opt/ros/humble/lib/librtabmap_util_plugins.so
  
• Yet, looked into the code, looks like several nodes are started:
  executor.add_node(camera->get_node_base_interface());
  executor.add_node(laserscanFront->get_node_base_interface());
  executor.add_node(laserscanBack->get_node_base_interface());
  executor.add_node(merger->get_node_base_interface());
  executor.add_node(rectify->get_node_base_interface());
  executor.add_node(rtabmap->get_node_base_interface());
  
• The merger is ira_laser_tools::LaserscanMerger, rectify is image_proc::RectifyNode.
• There is a seperate branch of depthai-python rcv3_support
• Inside that branch I did python3 examples/install_requirements.py, which installed v2.19.1.0 from depthai. That gave:
  RuntimeError: No available devices (3 connected, but in use)
• Did ps -all and kill the ros2 process.
• Now python3 examples/calibration/calibration_reader.py in the rvc3_support branch gives the RAE calibration info (stored in examples/calibration/calib_xlinkserver.json).
• Also tried python3 examples/devices/list_devices.py, which gives the X_LINK info:
  [DeviceInfo(name=127.0.0.1, mxid=58927016838860C5, X_LINK_GATE, X_LINK_TCP_IP, X_LINK_RVC3, X_LINK_SUCCESS), DeviceInfo(name=192.168.197.55, mxid=58927016838860C5, X_LINK_GATE, X_LINK_TCP_IP, X_LINK_RVC3, X_LINK_SUCCESS)]
• So, started ros2 launch depthai_examples rgb_stereo_node.launch.py camera_model:=RAE.
• I see the following topics:
  /color/video/camera_info
  /color/video/image
  /color/video/image/compressed
  /color/video/image/compressedDepth
  /color/video/image/theora
  /joint_states
  /parameter_events
  /robot_description
  /rosout
  /stereo/camera_info
  /stereo/depth
  /stereo/depth/compressed
  /stereo/depth/compressedDepth
  /stereo/depth/theora
  /tf
  /tf_static
  
• Checking on WS9. I also see the topics there. The /color/video/image I can display. For video rviz2 complains that it cannot display stereo. That is a warning of rviz2, not on stereo-msgs.
• In rviz2 I also so a PointCloud, but it seems that the streaming just stopped (also no updates in the color-video). Restarting at the RAE didn't help.
• Looking in /underlay_ws/install/depthai_examples/share/depthai_examples/launch/.
• The rgb_stereo_node.launch.py should indeed also publish a point-cloud as topic /stereo/points.
• Note that rgb_stereo_node should also tries to launch rviz. Yet, this is commented out, together with the point_cloud_node
• I could try stereo_inertial_node.launch.py with depth_aligned=True, rectify=True, enableRviz=False.
• Yet, that fails:
  [stereo_inertial_node-2] [58927016838860C5] [127.0.0.1] [1750866665.675] [host] [debug] Device about to be closed...
  [stereo_inertial_node-2] [2025-06-25 15:51:05.681] [depthai] [debug] DataOutputQueue (depth) closed
  [stereo_inertial_node-2] [2025-06-25 15:51:05.681] [depthai] [debug] DataOutputQueue (preview) closed
  [stereo_inertial_node-2] [2025-06-25 15:51:05.682] [depthai] [debug] DataOutputQueue (rgb) closed
  [stereo_inertial_node-2] [2025-06-25 15:51:05.682] [depthai] [debug] DataOutputQueue (imu) closed
  [stereo_inertial_node-2] [58927016838860C5] [127.0.0.1] [1750866665.682] [host] [debug] Log thread exception caught: Couldn't read data from stream: '__log' (X_LINK_ERROR)
  [stereo_inertial_node-2] [58927016838860C5] [127.0.0.1] [1750866665.682] [host] [debug] Timesync thread exception caught: Couldn't read data from stream: '__timesync' (X_LINK_ERROR)
  [stereo_inertial_node-2] [2025-06-25 15:51:05.682] [depthai] [debug] DataOutputQueue (detections) closed
  [stereo_inertial_node-2] [2025-06-25 15:51:05.682] [depthai] [debug] XLinkResetRemote of linkId: (0)
  [stereo_inertial_node-2] [58927016838860C5] [127.0.0.1] [1750866671.207] [host] [debug] Device closed, 5531
  [stereo_inertial_node-2] [2025-06-25 15:51:11.210] [depthai] [debug] DataInputQueue (control) closed
  [stereo_inertial_node-2] terminate called after throwing an instance of 'nanorpc::core::exception::logic'
  [stereo_inertial_node-2] what(): Cannot use both isp & video/preview/still outputs at once at the moment (startPipeline)
  
• Instead commented out the point_cloud_xyzi and metric_converter
• On nb-dual WSL the /stereo/points is visible, but see no echo.
• Commented both out again. Try again tomorrow, after a reboot.

June 24, 2025

• The 2nd week of the Robotics in a Nutshell MOOC is on Image Formation and Calibration.
• I like the explanation of the Pinhole model, including the conservation fractions of the object heights and distances h'/h = l'/l:
  
• This is directly associated with the Thin Lense model, where the same fraction corresponds with the focal lens distances h'/h = l'/l = x'/f = f'/x:
  
• The 3rd unit of the 2nd week is a topic not often covered: Laser scanning with projection patterns. Scanning with two colored laser-planes, or providing a corner background I have not seen explained before. Nice that the unit starts with a single laser point.
• In the 4th unit impressive demonstration of the ICP algorithm in 3D is shown.
• Connected the OAK-D with the Thunderbolt cable to nb-dual, with native Ubuntu 20.04. No ROS2, so looked at ROS1 Noetic first.
• See no /opt/ros/noetic/share/depthai_ros_driver, so should install that one first.
• First had to reinstall the signature keys by using the third suggested option: sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg. 16 ros-packages are installed.
• If I look at this Medium tutorial, I should also install ros-noetic-depthai-examples (was already part of the pack).
• Yet, the command roslaunch depthai_examples rgb_stereo_node.launch camera_model:=OAK-1 fails on missing Intrinsic matrix available for the the requested cameraID.
• Known issue according to Luxonis post, should perform a Calibration.
• Started with python3 depthai_demo.py, which gives one frame and crashes on:
  depthai_sdk/managers/preview_manager.py", line 148, in prepareFrames packet = queue.tryGet()
  RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'color' (X_LINK_ERROR)'
• CONTROL-C restarted the connection, which gave:
  [DEVICEID] [3.8] [1.501] [StereoDepth(7)] [error] RGB camera calibration missing, aligning to RGB won't work
• Seems that I did the same with the RAE on December 12, 2024 and June 6, 2024.
• Should also look at this Luxonis discussion.
• The calibration files are written to the EPROM. Could first look at what is stored in the EEPROM with Calibration Reader.
• Moved to ~/git/depthai-python/examples/calibration. Running python3 calibration_reader.py gave the same error:
  RuntimeError: There is no Intrinsic matrix available for the the requested cameraID
• Looked at the calibration_flash_v5.py script. It tries to write ../models/depthai_v5.calib to the EEPROM. That file actually exists.
• Going back to Initial Connection.
• Cloned depthai-core.
• Run python3 examples/python/install_requirements.py, which instaleld PyYAML-6.0.2-cp38, opencv_python-4.11.0.86-cp37 and depthai-3.0.0rc2-cp38.
• Moved to ~/git/depthai-core/examples/python/Camera. Running python3 camera_all.py gave:
  RuntimeError: Device already closed or disconnected: Input/output error [2025-06-24 17:01:55.193] [depthai] [error] Device with id XXX has crashed.
• Switching back to WS9, look if it can work with the OAK-D or find calibration info at the UGV Rover OAK-camera.
• Running python3 calibration_reader.py for the WS9 and OAK-D gave the same error
• Moved to the ugv_rover. Cloned depthai-python, but python3 calibration_reader.py couldn't find module depthai. Running git submodule update --init --recursive didn't help. Doing python3 -m pip install . takes a while. Couldn't build the wheel at the end.
• Just did python3 -m pip install depthai.
• Now python3 examples/calibration/calibration_reader.py works, and gives the RGB Camera Default intrinsics, RGB Camera resized intrinsics... 3840 x 2160, 4056 x 3040, LEFT Camera Default intrinsics..., LEFT/RIGHT Distortion Coefficients..., RGB FOV, Mono FOV, LEFT/RIGHT Camera stereo rectification matrix..., Transformation matrix of where left Camera is W.R.T right Camera's optical center, Transformation matrix of where left Camera is W.R.T RGB Camera's optical center.
• Seems that the information is stored in calib_device_id.json file. The productName is "OAK-D-LITE".
• Tried oakctl. Could installed both on WS9 and the ugv_rover, but no (OAK4) devices found.
• Tried to install oak-viewer on WS9. Installation goes well, but launching oak-viewer gives:
  Viewer stderr: [PYI-3537627:ERROR] Failed to load Python shared library '/usr/lib/oak-viewer/resources/backend/viewer_backend/_internal/libpython3.12.so.1.0': /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.38' not found (required by /usr/lib/oak-viewer/resources/backend/viewer_backend/_internal/libpython3.12.so.1.0)
• Not the only one with this problem. There are many (old answers) on stackoverflow. The patchelf answer #14 looks promising.
• Close to my problem is the Proof of Concept at the end.

June 23, 2025

• Cyrill published a Graph-SLAM chapter in a new textbook Robotics Goes MOOC - Knowledge
• Actually, this is part of series of four textbooks:
  • Robotic Goes MOOC - Design
  • Robotic Goes MOOC - Interaction
  • Robotic Goes MOOC - Impact
• The corresponding MOOC is hosted by Federica.eu.

June 19, 2025

• Looking how to split the bringup_lidar.launch.py nicely in a rover and workstation part. For the rover there is the use_rviz=False, so could make a launch-script that launches the rviz (and description?) locally on the workstation.
• Strange enough, there is also a bringup-executable launched. What is in that executable?
• At the end, only selected three nodes for the workstation-side: use_rviz_arg rviz_config_arg, robot_state_launch.
• The executable seems to be ugv_bringup/ugv_bringup.py, which creates a number of publishers of the topics "imu/data_raw", "imu/mag", "odom/odom_raw", "voltage"
• There is also ugv_bringup/ugv_driver.py, which subscribes to topics "cmd_vel", 'joy', 'ugv/joint_states' (pan/tilt) and 'ugv/led_ctrl'. There is also a subscription to 'voltage' (can the voltage be controlled?). Looked in the code, if the voltage_value drops below 9 a low_battery sound is played.
• Made a rviz_lidar.launch.py with only three nodes, and added it to the install with colcon build --packages-select ugv_bringup. After setting the UGV_MODEL and LDLIDAR_MODEL I launced it with ros2 launch ugv_bringup rviz_lidar.launch.py use_rviz:=true
• Made two scripts start_lidar.sh (rover-side) and start_rviz_lidar.sh (workstation-side)
• Works, but I still have two robot_state_publishers, which also gives double /rf2o_laser_odometry, /transform_listener_impl_62d462ce14d0, /ugv/joint_state_publisher.
• Killed the workstation-side robot_state_publisher, still have a double rf2o_laser_odometry.
• Tried again, now without the robot_state_launch (2 nodes left). Starting start_rviz_lidar.sh gave 4 nodes:
  /rviz2 /transform_listener_impl_5c12fd746e70 /ugv/joint_state_publisher /ugv/robot_state_publisher
• Adding start_lidar.sh still gave many doubles
• Started a clean rviz2 with ros2 run rviz2 rviz2 -d ~/git/ugv_ws/install/ugv_bringup/share/ugv_bringup/rviz/view_bringup.rviz. Still two transform_listener_* and two rf2o_laser_odometry.
• Rebooted the jetson, still two rf2o_laser_odometries. The LD19 in ugv_else/ldlidar also launches a transform_listener_impl.
• Tried ros2 lifecycle set transform_listener_impl_636aff058900 shutdown, but get Node not found.
• Could try to reboot also ws9, but there are other users. Leave it here for the moment.
• Trying Tutorial 4: 2D Mapping Based on LiDAR.
• In the directory ~/git/ugv_ws/src/ugv_main/ugv_slam/launch there are three launches files, cartographer, rtabmap_rgbd and gmapping. Tutorial 4 is ussing gmapping.
• Started with activating the OAK-D camera with ros2 launch ugv_vision oak_d_lite.launch.py. Seems to work:
  [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/rectify_color_node' in container 'oak_container'
  [component_container-1] [INFO] [1750338883.796983921] [oak]: Starting camera.
  [component_container-1] [INFO] [1750338883.807265975] [oak]: No ip/mxid specified, connecting to the next available device.
  [component_container-1] [INFO] [1750338886.592273026] [oak]: Camera with MXID: 14442C10019902D700 and Name: 1.2.3 connected!
  [component_container-1] [INFO] [1750338886.593134048] [oak]: USB SPEED: HIGH
  [component_container-1] [INFO] [1750338886.639988351] [oak]: Device type: OAK-D-LITE
  [component_container-1] [INFO] [1750338886.642406675] [oak]: Pipeline type: RGBD
  [component_container-1] [INFO] [1750338887.683423674] [oak]: Finished setting up pipeline.
  [component_container-1] [WARN] [1750338888.471441275] [oak]: Parameter imu.i_rot_cov not found
  [component_container-1] [WARN] [1750338888.471589696] [oak]: Parameter imu.i_mag_cov not found
  [component_container-1] [INFO] [1750338889.080588298] [oak]: Camera ready!
  
• In RVIZ, I could display both the topic /oak/rgb/image_rect as /oak/stereo/image_raw as Image:
  
• When I look at the topics, only /oak/imu, /oak/rgb and /oak/stereo are published.
• Luxonis has launch files which also starts ROS depth processing nodes to generate a poincloud
• The oak_d_lite.launch.py script starts camera.launch.py from depthai_ros_driver, which starts the RGBD together with a NN.
• The installed driver files can be found at /opt/ros/humble/share/depthai_ros_driver/launch/. Didn't see the NN in spatial Mobilenet mode.
• Tried on the rover ros2 launch depthai_ros_driver rgbd_pcl.launch.py. Now there is also a topic /oak/points, of type PointCloud2.
• I could add PointCloud2 display in rviz2, but the node-log showed:
  New subscription discovered on topic '/oak/points', requesting incompatible QoS. No messages will be sent to it. Last incompatible policy: RELIABILITY_QOS_POLICY
• Looked with ros2 topic info /oak/points --verbose. The QoS profile of rviz and the driver match:
  Reliability: BEST_EFFORT
  Durability: VOLATILE
• Should be possible, at least with orbbec_camera.
• On November 7, 2024 I had point-cloud displayed for realsense camera.
• Tried an alternative approach. Attached my OAK-D directly to ws9. Had to install ros-humble-depthai-ros-driver, which also installed ros-humble-depthai-ros-msgs and ros-humble-ffmpeg-image-transport-msgs.
• Launching ros2 launch depthai_ros_driver rgbd_pcl.launch.py on ws9 failed on udev rules: Insufficient permissions to communicate with X_LINK_UNBOOTED device with name "1.1". Make sure udev rules are set
• Looked at Luxonis troubleshooting and did the udev update. Still going wrong, so should try another USB-cable (currently using a white USB-B to USB-C).
• Connected my Thunderbolt USB-C to USB-C cable. At least the camera_preview example from the python sdk works.
• I have feeling that it has something to do with the bootloader.
• Following the instructions of bootloader config. No devices found.
• Also tried device information, but also Couldn't find any available devices.
• Tried on ws9 ros2 run rviz2 rviz2 -d /opt/ros/humble/share/depthai_ros_driver/config/rviz/rgbd.rviz. Didn't work, nor for the OAK-D, nor the ugv_rover.
• On July 11, 2022 I had a working visualisation of the PointCloud in rviz.

June 16, 2025

• Looking at the github ugv_ws, but this workspace is at least 4 months old.
• The first step of the tutorial is starting rviz, which is a bit strange inside the docker (running on a remote station). Tried if I could start rviz on WSL with a Xserver running, but that gives:
  rviz2: error while loading shared libraries: libQt5Core.so.5: cannot open shared object file: No such file or directory
• So, try again, on WS9.
• Could connect via vislab_wifi to the rover.
• Cloned ugv_ws. Started ros-humbe. First command of build_first was:
  colcon build --packages-select apriltag apriltag_msgs apriltag_ros cartographer costmap_converter_msgs costmap_converter emcl2 explore_lite openslam_gmapping slam_gmapping ldlidar rf2o_laser_odometry robot_pose_publisher teb_msgs teb_local_planner vizanti vizanti_cpp vizanti_demos vizanti_msgs vizanti_server ugv_base_node ugv_interface
• This build failed because nav2_msgs package was not installed.
• Installation failed on:
  Failed to fetch http://packages.ros.org/ros2/ubuntu/dists/jammy/InRelease The following signatures were invalid: Open Robotics
• Solved it by adding the signature in a keyring, as suggested on askubuntu.
• Next to fail is again explore_lite, now on a missing map_msgs package.'
• This can be solved with sudo apt install ros-humble-nav2-msgs ros-humble-map-msgs ros-humble-nav2-costmap-2d.
• Next package (15/22) to fail is apriltag_ros, on package image_geometry.
• The package vizanti_server fails on rosbridge_suite. Now the first colcon_build is succesfully finished. To be sure did an intermediate source install/setup.bash.
• The second build is colcon build --packages-select ugv_bringup ugv_chat_ai ugv_description ugv_gazebo ugv_nav ugv_slam ugv_tools ugv_vision ugv_web_app --symlink-install .
• The ugv_nav fails on missing nav2_bringup. This is the only package that has to be installed. Did again a source install/setup.bash.
• In the README.md some additional humble-packages are mentioned, such as ros-humble-usb-cam and ros-humble-depthai-*.
• Started the first step of the tutorial, but ros2 launch ugv_description display.launch.py use_rviz:=true failed on missing UGV_MODEL.
• Looked in the launch-file and defined export UGV_MODEL=ugv_rover (name of urdf-file.
• Next exception is missing joint_state_publisher_gui package. That package was in the README.md, so did sudo apt install ros-humble-joint-state-publisher-*. The Joint-State Publisher window now pops up, only rviz2 is missing.
• That is part of apt install ros-humble-desktop-*, also in the README.md. That installs 385 packages.
• Now I could control the pan-tilt (both up/down and left/right), both in rviz and on the rover itself (after starting the driver):
  
• The robot doesn't respond on the wheel commands. Maybe I should also have defined the UGV_MODEL at the rover side. Tried again with UGV_MODEL defined, same behavior. Checked the topics, no camera images or depth-images are published (yet).
• Next is Tutorial 3 of the wiki.
• Launching ros2 launch ugv_bringup bringup_lidar.launch.py use_rviz:=true on ws9 fails on LDLIDAR_MODEL. That is not in ugv_bringup/launch/bringup_lidar.launch.py, but in ldlidar.launch.py called from there. So, LDLIDAR_MODEL should be ld06, ld19 or stl27l. So set export LDLIDAR_MODEL=ld19.
• Activated in RVIZ a LaserScan, selected the /scan topic, but nothing visible.
• Also launched ros2 launch ugv_bringup bringup_lidar.launch.py use_rviz:=false, which gave an error:
  [ugv_bringup-3] JSON decode error: Expecting value: line 1 column 1 (char 0) with line: z":1684,"odl":0,"odr":0,"v":1214}
• Try again with ld06? Actually, its a D500 Lidar. Yet, the D500 Lidar-kit is based on a STL-19P.
• Trying a 2nd time, now it works:
  [rf2o_laser_odometry_node-6] [INFO] [1750080862.481144040] [rf2o_laser_odometry]: Initializing RF2O node...
  [rf2o_laser_odometry_node-6] [WARN] [1750080862.495336634] [rf2o_laser_odometry]: Waiting for laser_scans....
  [rf2o_laser_odometry_node-6] [INFO] [1750080862.695500846] [rf2o_laser_odometry]: Got first Laser Scan .... Configuring node
  
• Could also drive around with ros2 run ugv_tools keyboard_ctrl (from ws9):
  
• I see the updates in RVIZ, but I don't see the scan topic in the topic list. A bit strange, because I see in with ros2 node list many nodes (several double, not ideal to use the same bringup for both). One of the nodes is /LD19:
  /LD19
  /base_node
  /base_node
  /keyboard_ctrl
  /rf2o_laser_odometry
  /rf2o_laser_odometry
  /rf2o_laser_odometry
  /rf2o_laser_odometry
  /rviz2
  /rviz2
  /transform_listener_impl_5b38e1b97df0
  /transform_listener_impl_5e016b673f00
  /transform_listener_impl_6337961ff800
  /transform_listener_impl_aaaaf1f50810
  /ugv/joint_state_publisher
  /ugv/joint_state_publisher
  /ugv/joint_state_publisher
  /ugv/joint_state_publisher
  /ugv/joint_state_publisher
  /ugv/joint_state_publisher_gui
  /ugv/robot_state_publisher
  /ugv/robot_state_publisher
  /ugv/robot_state_publisher
  /ugv/robot_state_publisher
  /ugv_bringup
  /ugv_driver
  
• When looking with ros2 topic echo /scan I see many .nan values:
  - 228.0
  - 232.0
  - .nan
  - 220.0
  - 209.0
  - 205.0
  - 201.0
  - 204.0
  - 212.0
  - 204.0
  - .nan
  - .nan
  - .nan
  - .nan
  

June 2, 2025

• Looking at the UGV-rover again. Tried to connected to the rover via the vislab-wifi. I also see the UGV network still active.
• Received Joey's labbook. The robot was connected vi LAB42, not the vislab_wifi. Able to login via ssh.
• The home-directory on the rover has several directories, including ugv_ws and ugv_jetson.
• Started with source /opt/ros/humble/setup.bash, followed by source ~/ugv_ws/install/setup.bash. That gives two warnings:
  not found: "/home/jetson/ugv_ws/install/costmap_converter/share/costmap_converter/local_setup.bash"
  not found: "/home/jetson/ugv_ws/install/explore_lite/share/explore_lite/local_setup.bash"
• Started ros2 launch ugv_custom_nodes line_follower.launch.py. Seems to work, only a few warnings:
  [v4l2_camera_node-1] [INFO] [1749735101.182860329] [v4l2_camera]: Driver: uvcvideo
  [v4l2_camera_node-1] [INFO] [1749735101.183075567] [v4l2_camera]: Version: 331656
  [v4l2_camera_node-1] [INFO] [1749735101.183090895] [v4l2_camera]: Device: USB Camera: USB Camera
  [v4l2_camera_node-1] [INFO] [1749735101.183096527] [v4l2_camera]: Location: usb-3610000.usb-2.2
  [v4l2_camera_node-1] [INFO] [1749735101.183101583] [v4l2_camera]: Capabilities:
  [v4l2_camera_node-1] [INFO] [1749735101.183105903] [v4l2_camera]: Read/write: NO
  [v4l2_camera_node-1] [INFO] [1749735101.183111183] [v4l2_camera]: Streaming: YES
  [v4l2_camera_node-1] [INFO] [1749735101.183125936] [v4l2_camera]: Current pixel format: MJPG @ 1920x1080
  [v4l2_camera_node-1] [INFO] [1749735101.183234419] [v4l2_camera]: Available pixel formats:
  [v4l2_camera_node-1] [INFO] [1749735101.183242515] [v4l2_camera]: MJPG - Motion-JPEG
  [v4l2_camera_node-1] [INFO] [1749735101.183246995] [v4l2_camera]: YUYV - YUYV 4:2:2
  [v4l2_camera_node-1] [INFO] [1749735101.183251315] [v4l2_camera]: Available controls:
  [v4l2_camera_node-1] [INFO] [1749735101.184699257] [v4l2_camera]: Brightness (1) = 0
  [v4l2_camera_node-1] [INFO] [1749735101.186169984] [v4l2_camera]: Contrast (1) = 50
  [v4l2_camera_node-1] [INFO] [1749735101.187917198] [v4l2_camera]: Saturation (1) = 65
  [v4l2_camera_node-1] [INFO] [1749735101.189416117] [v4l2_camera]: Hue (1) = 0
  [v4l2_camera_node-1] [INFO] [1749735101.189443318] [v4l2_camera]: White Balance, Automatic (2) = 1
  [v4l2_camera_node-1] [INFO] [1749735101.190926908] [v4l2_camera]: Gamma (1) = 300
  [v4l2_camera_node-1] [INFO] [1749735101.192166269] [v4l2_camera]: Power Line Frequency (3) = 1
  [v4l2_camera_node-1] [INFO] [1749735101.193665476] [v4l2_camera]: White Balance Temperature (1) = 4600 [inactive]
  [v4l2_camera_node-1] [INFO] [1749735101.194914213] [v4l2_camera]: Sharpness (1) = 50
  [v4l2_camera_node-1] [INFO] [1749735101.196162694] [v4l2_camera]: Backlight Compensation (1) = 0
  [v4l2_camera_node-1] [ERROR] [1749735101.196197767] [v4l2_camera]: Failed getting value for control 10092545: Permission denied (13); returning 0!
  [v4l2_camera_node-1] [INFO] [1749735101.196208903] [v4l2_camera]: Camera Controls (6) = 0
  [v4l2_camera_node-1] [INFO] [1749735101.196217671] [v4l2_camera]: Auto Exposure (3) = 3
  [v4l2_camera_node-1] [INFO] [1749735101.197664077] [v4l2_camera]: Exposure Time, Absolute (1) = 166 [inactive]
  [v4l2_camera_node-1] [INFO] [1749735101.199412059] [v4l2_camera]: Exposure, Dynamic Framerate (2) = 0
  [v4l2_camera_node-1] [INFO] [1749735101.200914146] [v4l2_camera]: Pan, Absolute (1) = 0
  [v4l2_camera_node-1] [INFO] [1749735101.202424394] [v4l2_camera]: Tilt, Absolute (1) = 0
  [v4l2_camera_node-1] [INFO] [1749735101.203911537] [v4l2_camera]: Focus, Absolute (1) = 68 [inactive]
  [v4l2_camera_node-1] [INFO] [1749735101.203945650] [v4l2_camera]: Focus, Automatic Continuous (2) = 1
  [v4l2_camera_node-1] [INFO] [1749735101.205413752] [v4l2_camera]: Zoom, Absolute (1) = 0
  [v4l2_camera_node-1] [WARN] [1749735101.206874879] [camera]: Control type not currently supported: 6, for control: Camera Controls
  [v4l2_camera_node-1] [INFO] [1749735101.207363787] [v4l2_camera]: Requesting format: 1920x1080 YUYV
  [v4l2_camera_node-1] [INFO] [1749735101.218292138] [v4l2_camera]: Success
  [v4l2_camera_node-1] [INFO] [1749735101.218322795] [v4l2_camera]: Requesting format: 160x120 YUYV
  [v4l2_camera_node-1] [INFO] [1749735101.229039172] [v4l2_camera]: Success
  [v4l2_camera_node-1] [INFO] [1749735101.236211424] [v4l2_camera]: Starting camera
  [v4l2_camera_node-1] [WARN] [1749735101.804031818] [camera]: Image encoding not the same as requested output, performing possibly slow conversion: yuv422_yuy2 => rgb8
  [v4l2_camera_node-1] [INFO] [1749735101.814271510] [camera]: using default calibration URL
  [v4l2_camera_node-1] [INFO] [1749735101.814428698] [camera]: camera calibration URL: file:///home/jetson/.ros/camera_info/usb_camera:_usb_camera.yaml
  [v4l2_camera_node-1] [ERROR] [1749735101.814617119] [camera_calibration_parsers]: Unable to open camera calibration file [/home/jetson/.ros/camera_info/usb_camera:_usb_camera.yaml]
  [v4l2_camera_node-1] [WARN] [1749735101.814656576] [camera]: Camera calibration file /home/jetson/.ros/camera_info/usb_camera:_usb_camera.yaml not found
  
• Checked on nb-dual (WSL) and saw the following topics:
  /camera_info
  /cmd_vel
  /image_raw
  /image_raw/compressed
  /image_raw/compressedDepth
  /image_raw/theora
  /joy
  /parameter_events
  /processed_image
  /rosout
  /ugv/joint_states
  /ugv/led_ctrl
  /voltage
  
• No topics like /scan and /odom yet, although Joey got that working at May 23 (ros2 launch ugv_custom_nodes mapping_node.launch.py.
• Looked what I tried as last command on February 6.
• Tried ros2 run image_view image_view --ros-args --remap image:=/image_raw inside WSL with VxSrv-Xlaunch running on the backgroud, but this gives:
  [INFO] [1749735895.930762600] [image_view_node]: Using transport "raw" terminate called after throwing an instance of 'cv::Exception' what(): OpenCV(4.5.4) ./modules/highgui/src/window_gtk.cpp:635: error: (-2:Unspecified error) Can't initialize GTK backend in function 'cvInitSystem'
• Tried again on ws9. Had to do unset ROS_DOMAIN_ID first. Still, ros2 topic list gives me only the two default topics, while I could ping and ssh to the rover, and see all topics there.
• ws9 was busy with a partial upgrade (asking for a reboot). Hope that this is not an update from Ubuntu 22.04 to 24.04.
• Installed sudo apt-get install ros-humble-image-view.
• Switched off all TurtleBot related settings in my ~/.bashrc
• Another user is running jobs on ws9, so could't reboot.
• This seems to happen at the LAB42 network, not on the vislab_wifi. Looking into Multiple Network sections from linuxbabe. That didn't work.
• Instead used the trick from stackexchange and looked up the UUIDs with nmcli c and switched to other network with nmcli c up uuid . The ssh-connection then freezes, because you have to build it up again via the other network.
• The two ethernet-connections also are in the 192.* domain, so had to unpluck them to connect to the robot.
• Now I see all topics. The image was black, but that was because the cover was still on the lens. Without cover the robot directly starts to drive (as expected from line-following).
• The images displayed with ros2 run image_view image_view --ros-args --remap image:=/image_raw have a delay of 10s:
  
• You can also request the processed_image with ros2 run image_view image_view --ros-args --remap image:=/processed_image:
  
• There are now 4 nodes running:
  /camera
  /driver
  /image_view_node
  /line_following_node
  
• The next step in the WaveShare tutorials is to control the leds (Tutorial 2).
• On ws9 Mozilla could open this web-page, chrome could.
• After starting the driver with ros2 run ugv_bringup ugv_driver, I could control the three led-lights with ros2 topic pub /ugv/led_ctrl std_msgs/msg/Float32MultiArray "{data: [255, 255]}" -1. With {data: [9,0]} only the two lower leds light up, with less brightness. Note that the ugv_driver is not the driver from the line-following. Yet, this seems only a difference in name (same executable) in ugv_custom_nodes/launch/line_follower.launch.py
• Time to go home.

March 4, 2025

• Interesting new SLAM paper with on LiDAR Situational Graphs (Feb, 2025), including code.

February 20, 2025

• Looking for nice images for the paper. Fig. 4 of Group 1 could be usefull to illustrate the first assignment.
• Also Fig. 6 would be a nice illustration.
• Yet, Group 1 chose the most centered line.
• Group 9 shows in Fig. 9 the benefit of using RANSAC compared to Canny edge.

February 18, 2025

• The RAE has a Robotics Vision Core 3, which is based on the Intel Movidius accelerator with code name (a href=https://www.intel.com/content/www/us/en/developer/articles/technical/movidius-accelerator-on-edge-software-hub.html>Keem Bay.

February 12, 2025

• An interesting arXiv paper on a Wheeled lab on the University of Washington.
• More details on their Project Website, including github code.
• They point to F1Tenth as educational platform, which is a website with both a learn and course section.
• Looked at Module E: Vision. System is also Nvidia Jetson based.
• The course is UPenn: ESE6150: RoboRacer Autonomous Racing Cars (Spring 2024).

February 6, 2025

• The ugv-rover has 4GB Jetson Orin Nano Kit. Documentation for the Orin can be found here. The documentation of the Rover can be found here.
• The rover comes without batteries. The instructions to load the batteries are not there yet.
• The battery compartment is actually below the rover. The screw-driver provided nicely fits. Placed three of the 18650 lithium batteries from the DART robot into the compartment and start charging (connector next to the on-off button).
  
• Connected the Display-port to my display and the USB-C to a mouse and keyboard. The ethernet-ip is displayed on the small black screen below the on-off button.
• Could login via the screen/keyboard. The Jetson is running Ubuntu 22.04.04.
• Switched off the hotspot (right top corner) and connected to LAB42.
• Tried to switch of the python programming started during setup. The kills seem to fail, but after a while no 10% python script popped up anymore. Looked at /home/jetson/uvg.log. Last line is Terminated.
• The docker scripts are actually in /home/jetson/uvg_ws. Didn't make two scripts mentioned in ROS2 Preparation executable. The ros2_humble script starts fine, and indicated a shell-server is started. Yet, I couldn't connect to the shell server, nor externally, nor via the docker-ip. Yet, doing a docker ps followed by docker exec -it CONTAINER_ID bash worked (no zsh in SPATH). Could find /opt/ros/humble/bin/ros2.
• Next tried RVIZ, but that failed because no connection to display localhost:12.0 could be made.
• The docker is a restart of container of a existing docker-image, not a fresh run.
• Tried to make a script that run from the image, with reuse of the host. Yet, out of battery before finished(charging is before USB-B connector). Switched to USB-C connector after the reboot.
• Starting the docker failed. Could be the background process. Tried docker run nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04. That works, but returns directly. docker run nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04. That works, but returns directly and gives a warning that no NVIDIA drivers are detected.
• Tried sudo docker run -it --runtime=nvidia --gpus all /usr/bin/bash. That works, entered the image. Only, no /opt/ros to be found, only /opt/nvidia. Strange, because I don't see any other image with docker image ls
• Looked if I could natively install the required ros-humble packages. sudo apt ugrade wants to downgrade 4 packages (nvidia-container-toolkit), so didn't continue.
• Installed apt install python3-pip. Next are the ros-packages, but I have to install the ros-repository first. Trying to install firefox, but that is snap-based. Used firefox for the ROS Humble Install instructions.
• Installed python3-colcon-argcomplete (11 packages)
• Installed ros-humble-base first (258 packages), followed by ros-humble-cartographer-* (360 new, 7 upgraded).
• Next ros-humble-joint-state-publisher-* (17 packages), followed by ros-humble-nav2-* (119 packages, 5 upgraded)
• Next ros-humble-rosbridge-* (22 packages), followed by ros-humble-rqt-* (61 packages)
• Next ros-humble-rtabmap-* (51 new, 1 upgraded), followed by ros-humble-usb-cam-* (11 packages).
• Last one is ros-humble-depthai-* (28 packages). Left the gazebo part for the moment.
• The code needed is available on github. Unfortunatelly, colcon build --packages-select doesn't work, so have do these three commands in each sub-dir of src/ugv_else: cmake -B build -DCMAKE_BUILD_TYPE=Release
  cmake --build build --target install
  sudo cmake --build build --target install
  
• There is not only /home/jetson/git/ugv_ws/install/setup.bash, but also the preinstalled /home/jetson/ugv_ws/install/setup.bash. Yet, running this script gave missing local_setup.bash for ugv_description, ugv_gazebo, ugv_nav and ugv_slam. Those local_setup.bash were logical links to /home/ws. Changing the link to /home/jetson (with sudo!) solved this. Now ~/.bashrc works.
• Yet, nor ros2 launch ugv_description display.launch.py use_rviz:=true nor ros2 run ugv_bringup ugv_driver works (package not found). Maybe a ros-dep init first, even better in a humble_ws/src/, with the different packages in that src-directory.
• At least ros2 run usb_cam usb_cam_node_exe worked. Could view the image with ros2 run image_view image_view --ros-args --remap image:=/image_raw. Only no permission to save image:
  

January 28, 2025

• This paper describes iMarkers, which allow to make unvisible ArucoTags. Yet, one need a polarizer sheet in front of one of the stereo-cameras.

January 27, 2025

• A student from Canada tested my VisualSfM - Ubuntu installation technical report.
• This student go an error of a missing siftgpu library when clicking "Compute Missing Matches" toolbar-icon.
• Looked at the SiftGPU/bin executables. I could start TestWinGlut “-i ~/src/vsfm/data/small_maze/frame99.jpg
• I downloaded the examples images from github, which are the two files processed when typing SimpleSIFT.
• The program MultiTreadSIFT started two threads on 100Hz, but than crashed after some time with a Killed signal.
• For me the "Compute Missing Matches" toolbar-icon (step 2 in manual) worked fine.

January 24, 2025

• Should check pyslam.

January 22, 2025

• Read the place recognition tutorial in RAM, which is accompanied with github page.
• Order the UGV Rover.

January 21, 2025

• Couldn't find reviews of the UGV beast.
• The software for the driver base for the UGV can also be found at ugv_base_ros github. (4 months old) - 2 discussions.
• The software on the Orin can be found on ugv_jetson github. Important issue (not answered yet): are there plans to support JetPack 6.
• The ugv_ws github doesn't have issues, only a pull request.
• An alternative could be WeCAR A.
• WeGo has several repositories. Yet, could only find three minimal udev_rules , odom and examples WeCAR repositories (3 years old).
• Much more expensive is Mecabot Plus TS - Orin Nano
• And the AgiLex Limo ROS - NUC.
• Or the RoboWorks RoboWorks RosBot 2 - Orin Nano.
• The TurtleBot-4 is less expensive.
• An Dutch alternative is the Avular Origin, although more a research platform. Examples github.
• There is a new book Ultimate Robotics Programming with ROS 2 and Pythonincluding github code. Based on Ubuntu 22.04 and ROS 2 Humble.
• Chapter 10 covers the Navigation stack, Chapter 13 covers Computer Vision (no code!).

January 20, 2025

• Looking for an anouncement of the next version of RAE, but according to this post, most of the developers are gone. We can ask for a refund.
• Last August there were still plans to make a RCV4 version. The release was planned for Q3-Q4 2025.
• According to this post the project was already deprecated in 2023. We could already ask for refunds in October 2024.
• Alternative could be UGV Rover, Nvidia Jetson Orin based. Twice the price. But including Lidar and depth cameras. ROS2 Humble based. I also see that it is Docker based?! The depth sensor is OAK-D-Lite, the Lidar is D500 DTOF.
• The UGV Beast documentation wiki.
• Assembly instructions: "Continues improvement"
• The three 18650 lithium batteries are not included.
• Most of the details are about the Jupyter notebooks.
• Starts to get interesting at ROS2 preparation. Again, ROS Humble is part of Docker container.
• The code of the Docker container is on github. Last update 3 weeks ago.

January 17, 2025

• Trying the foxglove startup. Without other processes running in the background, I only see have 4 topics.
• Did ros2 launch rae_bringup robot.launch.py, which gave the /cmd_vel.
• Yet, the docker crashed before I could drive around.

January 16, 2025

• Comparison of 4 RGB-D cameras. Overall the ZED 2 was the winner.

January 2, 2025

• Tested VisualSFM installation on Karlijn's old laptop. Found only one small typo.
• Note that the space-key no longer works on this laptop.

Previous Labbooks

• 2024
• 2023