August 17, 2026

• New ROS2 book: ROS 2: Zero to Robot. Should check Chapter 9 onwards.
• Continuing with Experimental
• Repairing the ASR-pipeline.
• In audio_capture() I added:
  audio_device = os.environ.get('AUDIODEV')
  if audio_device == 'None':
  audio_device = "plughw:2,0" # original setting of code
  print("Using as microphone device: ", audio_device)
  
  cmd = [
  "arecord",
  "-D", audio_device,
  "-f", "S16_LE",
  "-c", "1",
  "-r", "16000",
  "-t", "raw",
  ]
  
• In ~/ugv_dev_ws I did colcon build --cmake-args -Wno-dev --packages-select ugv_voice.
• Don´t forget source install/setup.bash
• Called AUDIODEV=plughw:1,0 ros2 run ugv_voice voice_ctrl --ros-args -p language:=en. Got a RuntimeError: No graph was found in the protobuf. This means that onnx model is missing again. It is still in /home/ws/ugv_dev_ws/src/ugv_main/ugv_voice/ugv_voice/models/asr/silero_vad.onnx, but code looked in /home/ws/ugv_ws/.
• Rebuild and restarted.
• Gave command ros2 topic pub /asr std_msgs/Bool "{data: true}" --once
• The ASR became active:
  [ASR] Start
  Using as microphone device: plughw:1,0
  [ASR] Run thread started
  [ASR] FINAL: THE LOSING
  voice reconize THE LOSING
  [ASR] FINAL: THE FOIL
  voice reconize THE FOIL
  [ASR] FINAL: START
  voice reconize START
  [ASR] FINAL: AND
  voice reconize AND - Said END
  [ASR] FINAL: IT WAS
  voice reconize IT WAS - Said DRIVE
  [ASR] FINAL: STOP
  voice reconize STOP
  
• The list of tokens.txt from ASR is quite long (502 lines).
• Code is based on this github, with documentation. The model is described here, although it is not clear how large the CORPUS should be.
• The command ros2 topic pub /tts std_msgs/String "{data: 'Hello robot'}" --once works directly, no modification of code is needed:
  [INFO] [1786950070.650875084] [voice_ctrl]: Speech request: Hello robot
  Audio saved to /home/ws/ugv_ws/tts_en.wav
  
• Next would be voice-chat, which needs a ollama server started.
• Installed ifconfig with sudo apt install net-tools.
• The ollama_llm_chat.py script uses an explicit ip. Could better use 127.0.0.1. But it just connect to the ollama-server, not starts it.
• Found some waveshare-ollama documentation for the UGV Rover PI ROS2.
• In the documentation the ollama-server was installed on a Windows machine, but curl -fsSL https://ollama.com/install.sh | sh also works for the arm64 architecture of the Rover:
  Downloading ollama-linux-arm64-jetpack6.tar.zst
  >>> Creating ollama systemd service...
  WARNING: systemd is not running
  >>> NVIDIA JetPack ready.
  >>> The Ollama API is now available at 127.0.0.1:11434.
  
• Indeed, just running ollama didn´t work, yet, ollama serve does.
• Tried ros2 run ugv_voice voice_chat \ --ros-args \ -p language:=en \ -p server_url:=http://127.0.0.1:11434/api/chat, complains that there is no /home/ws/ugv_dev_ws/install/ugv_voice/lib/python3.10/site-packages/ugv_voice/../config/voice_config.yaml. Indeed, the config is installed in /home/ws/ugv_dev_ws/install/ugv_voice/share/ugv_voice/config/voice_config.yaml
• Made the configuration loaded from the installed location with following code:
  from ament_index_python.packages import get_package_share_directory
  from pathlib import Path
  
  package_dir = Path(get_package_share_directory("ugv_voice"))
  config_path = os.path.join(package_dir, "config/voice_config.yaml")
  
• Now I got again a device error: pygame.error: ALSA: Couldn't open audio device: No such file or directory. This can be solved with a environment variable:
  AUDIODEV=plughw:1,0 ros2 run ugv_voice voice_chat --ros-args -p language:=en -p server_url:=http://127.0.0.1:11434/api/chat
• Made a symbolic link in ~/ugv_dev_ws/install/ugv_voice/share/ugv_voice/ to the models directory in src, and added models/llm/prompt.txt.
• Again a step further, it now fails in ASR on sherpa_ncnn/recognizer.py trying to load the Chinese model, while the language on the command-line was English.
• The LANG_CONFIG is nicely defined at the beginning of the code. For the asr_model for both languages the same model is used: "sherpa-ncnn-streaming-zipformer-bilingual-zh-en-2023-02-13"
• With sherpa-ncnn-streaming-zipformer-20M-2023-02-17 the program starts. Yet, don´t hear a wake_prompt.
• Send ros2 topic pub /kws std_msgs/Bool "{data: true}" --once which gave the wake_prompt "I'm listening".
• Starting KWS also started ASR:
  [INFO] [1786970904.096623368] [voice_chat]: /kws wake trigger
  Audio saved to /home/ws/ugv_ws/tts_en.wav
  Aborted by signal Terminated...
  arecord: pcm_read:2221: read error: Interrupted system call
  [ASR] Start
  Using as microphone device: plughw:1,0
  [ASR] Run thread started
  [INFO] [1786970906.083562505] [voice_chat]: Start ASR
  Recording raw data 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
  [ASR] FINAL: START
  [INFO] [1786970911.036434075] [voice_chat]: ASR: START
  [INFO] [1786970911.059365697] [voice_chat]: LLM: llm model request failed: 404 Client Error: Not Found for url: http://127.0.0.1:11434/api/chat
  [ASR] Stop
  Aborted by signal Terminated...
  arecord: pcm_read:2221: read error: Interrupted system call
  Using as microphone device: plughw:1,0
  [INFO] [1786970925.080179686] [voice_chat]: Wakeup flow finished
  
• From the ollama server-side I see indeed:
  [GIN] 2026/08/17 - 20:48:31 | 404 | 5.942008ms | 127.0.0.1 | POST "/api/chat"
• Installed brave-browser on the jetson (outside the docker).
• Looking for api/chat on the jetson with the browser gave a 405 error.
• Three ways out of this:
  • Install the qwen3:8b model
  • Make sure that the systemd is running inside the container
  • Run the ollama server runs on my laptop, instead of the jetson.
• Kept the ollama server running, and gave in another terminal the command ollama run qwen3:8b. That looks good, the ollama-server responded with ownloading a3de86cd1c13 in 16 326 MB part(s). (5.2 Gb)
• Yet, once installed, it gave the error:
  load_tensors: loading model tensors, this can take a while... (load_mode = dio)
  load_tensors: offloading 0 repeating layers to GPU
  load_tensors: offloaded 0/37 layers to GPU
  load_tensors: CPU model buffer size = 4977.62 MiB
  time=2026-08-17T21:16:24.747+08:00 level=INFO source=llama_server.go:1350 msg="waiting for llama-server to become available" status="llm server not responding"
  time=2026-08-17T21:16:25.469+08:00 level=INFO source=sched.go:641 msg="Load failed" model=/home/ws/.ollama/models/blobs/sha256-a3de86cd1c132c822487ededd47a324c50491393e6565cd14bafa40d0b8e686f error="llama-server process has terminated: signal: killed"
  

August 14, 2026

• OpenCV has now a OpenCV AI Competition 2026, with Visual SLAM as one of the suggested project areas. Should be Cloud-Optimized Vision with COOL.
• Looking at the voice_ctrl pipeline I have build up. The kws has a small keywords.txt, with wake words like 'Hi Google', 'Hey Siri', 'Alexa'.
• But first look if not only the speaker, but also the microphone work. Started voice_ctrl with SDL_AUDIODRIVER=alsa AUDIODEV=plughw:1,0 ros2 run ugv_voice voice_ctrl.
• Doing a 5 second recording with arecord -D plughw:1,0 \ -f S16_LE \usb_connected = False -r 16000 \ -c 1 \ self.usb_connected = True -d 5 \ 84,9 39% /tmp/mic-test.wav and playing it back with aplay -D plughw:1,0 /tmp/mic-test.wav worked fine.
• Looked into the code. In principle there are already prints when Keywords are spotted.
• Added a print-statement: KWS is not enabled!
• The voice_ctlr documentation indicates that sub-features can be controlled with ros2 topic pub /kws std_msgs/Bool "{data: true}" --once. Indeed, now I get [INFO] [1786718440.872718184] [voice_ctrl]: KWS start, directly followed by arecord: main:831: audio open error: No such file or directory.
• Looked into kws_sherpa_onnx.py. Here explicit the device "plughw:2,0" instead of "plughw:1,0" is used (or even better AUDIODEV)
• Implemented the AUDIODEV version. Now I get:
  [INFO] [1786719937.937962859] [voice_ctrl]: KWS start
  Using as microphone device: plughw:1,0
  Recording raw data 'stdin' : Signed 16 bit Little Endian, Rate 16000 Hz, Mono
  Keyword Spotting True
  
• Also gave 2nd signal ros2 topic pub /asr std_msgs/Bool "{data: true}" --once.
• Same problem:
  [INFO] [1786720128.253006937] [voice_ctrl]: ASR start
  [ASR] Start
  [ASR] Run thread started
  arecord: main:831: audio open error: No such file or directory
  
• Same solution will work (next time).
• pygame tries to write some log in my home-directory. The container doesn´t know the original user (1000). Changing it to root gives problems outside the container. Not sure if defining the original user in the container would work. The home-directory is now available, so pygame seems to use a subdirectory.

August 13, 2026

• Started again with Rover #2.
• Inside the container, there was a lot of double commands at the end.
• Added those 4 lines from October 10, 2025 at the end:
  export ROS_DOMAIN_ID=2
  
  export HOME=/home/ws
  source $HOME/ugv_ws/install/setup.bash
  source $HOME/.bashrc
  
• Still an error message:
  not found: "/home/ws/ugv_ws/install/teb_local_planner/share/teb_local_planner/local_setup.bash"
• Trying colcon build --packages-select teb_local_planner, which fails on missing g2o includes. Did sudo apt-get install ros-humble-libg2o from October 27, 2025.
• The build script failed, without explicit error. Tried the command colcon build --cmake-args -Wno-dev --packages-select ldlidar rf2o_laser_odometry robot_pose_publisher teb_msgs --executor sequential from build_common.sh.
• This fails on The build time path "/home/jetson/ugv_ws/install/costmap_converter_msgs" doesn't exist. in teb_msgs.
• Strange, no explicit reference to /home/jetson. Removed ~/ugv_ws/build/teb_msgs, but /home/ws/ugv_ws/build/teb_msgs/colcon_command_prefix_build.sh comes back.
• Starting with colcon clean workspace -y
• Next colcon build --cmake-args -Wno-dev --packages-select cartographer costmap_converter_msgs explore_lite --executor sequential : Summary: 3 packages finished
• Also colcon build --cmake-args -Wno-dev --packages-select openslam_gmapping slam_gmapping --executor sequential works.
• Now colcon build --cmake-args -Wno-dev --packages-select ldlidar rf2o_laser_odometry robot_pose_publisher teb_msgs --executor sequential also succeeds, colcon clean workspace -y did the trick.
• Now 11 packages: colcon build --cmake-args -Wno-dev --packages-select ugv_base_node ugv_interface ugv_bringup ugv_chat_ai ugv_description ugv_gazebo ugv_nav ugv_slam ugv_tools ugv_vision ugv_web_app --executor sequential Summary: 11 packages finished.
• First chmod +x ~/ugv_ws/src/ugv_else/vizanti/vizanti_server/scripts/*.py
• Now colcon build --cmake-args -Wno-dev --packages-select vizanti vizanti_cpp vizanti_demos vizanti_msgs vizanti_server --executor sequential
• Building gives every time WARNING:colcon.colcon_ros.prefix_path.catkin:The path '/home/ws/ugv_ws/install/apriltag' in the environment variable CMAKE_PREFIX_PATH doesn't exist.
• sudo apt install libopencv-dev=4.5.4+dfsg-9ubuntu4 from build_apriltag,sh was not needed, already installed.
• Last colcon build --cmake-args -Wno-dev --packages-select apriltag apriltag_msgs apriltag_ros --executor sequential Summary: 3 packages finished
• OK, trying bringup tutorial.
• Doing ros2 launch ugv_bringup bringup_lidar.launch.py use_rviz:=false.
• See warnings:
  [rf2o_laser_odometry_node-6] [WARN] [1786609668.755448906] [rcl.logging_rosout]: Publisher already registered for provided node name. If this is due to multiple nodes with the same name then all logs for that logger name will go out over the existing publisher. As soon as any node with that name is destructed it will unregister the publisher, preventing any further logs for that name from being published on the rosout topic.
  [robot_state_publisher-1] [WARN] [1786609668.764682008] [ugv.robot_state_publisher]: No robot_description parameter, but command-line argument available. Assuming argument is name of URDF file. This backwards compatibility fallback will be removed in the future.
  [rf2o_laser_odometry_node-6] [INFO] [1786609669.101472224] [rf2o_laser_odometry]: Got first Laser Scan .... Configuring node
  [ugv_bringup-3] JSON decode error: Extra data: line 1 column 2 (char 1) with line: 08,"gx":-9,"gy":9,"gz":-5,"mx":-139,"my":439,"mz":1267,"odl":0,"odr":0,"v":1190}
• Next tutorial: https://github.com/waveshareteam/ugv_ws/blob/ros2-humble-develop-251125/docs/teleoperation.md. In T1 (in container) I did: ros2 run ugv_tools keyboard_ctr. Could drive robot forward.
• Did the same from nb-dual, could drive forward.
• Also rviz2 started up nicely, already with the robot-description and LaserScan.
• Next action-commands from experimental
• Inside the container, created a ~/ugv_dev_ws by git clone -b ros2-humble-develop-251125 https://github.com/waveshareteam/ugv_ws.git (via a tmp-directory)
• Copied ugv_main/ugv_msgs to ~/ugv_ws and did colcon build --cmake-args -Wno-dev --packages-select ugv_msgs.
• Got several warnings on WARNING:colcon.colcon_ros.prefix_path.ament:
  The path '/home/ws/ugv_ws/install/yahboom_rosmaster'
  The path '/home/ws/ugv_ws/install/oakdlite_camera'
  The path '/home/ws/ugv_ws/install/marker_detection'
  The path '/home/ws/ugv_ws/install/line_follower'
  The path '/home/ws/ugv_ws/install/emcl2' - Strange, should be there
  The path '/home/ws/ugv_ws/install/costmap_converter' - Strange, should be there
  The path '/home/ws/ugv_ws/install/yahboom_rosmaster'
• Started in T1 ros2 run ugv_tools behavior_ctrl .
• Checked in T2 ros2 interface show ugv_msgs/action/Behavior OK
• Checked in T2 ros2 action list | grep behavior OK
• Yet, ros2 action send_goal /behavior ugv_msgs/action/Behavior \ "{command: '[{\"type\": \"drive_on_heading\", \"data\": 0.5}]'}" --feedback gives:
  Waiting for an action server to become available...
• Looking at the two behavior_ctrl.py in ugv_ws and ugv_dev_ws. Quite a difference, ugv_dev_ws has much more code.
• Moved the behavior_ctlr to ugv_ws (saving behavior_ctrl.old). Now the script gives:
  [INFO] [1786613825.139621589] [behavior_ctrl]: Migrated 2 points from /home/ws/ugv_ws/map_points.txt to /home/ws/ugv_ws/map_points.json
• Still no 'Waiting for an action server to become available...'
• Check if the server is actual there with ros2 service list --include-hidden-services | grep '/behavior/_action'
  /behavior/_action/cancel_goal
  /behavior/_action/get_result
  /behavior/_action/send_goal
  
• This was due to a wrong format of the command. An empty command ros2 action send_goal /behavior ugv_msgs/action/Behavior '{}' --feedback gave a reponse:
  Sending goal:
  command: ''
  
  Goal accepted with ID: 810b794684cd45dfbb1ebadf59e3875f
  
  Result:
  result: false
  message: 'Invalid command JSON: Expecting value: line 1 column 1 (char 0)'
• The send_goal documentation can be found here.
• Tried ros2 action send_goal /behavior ugv_msgs/action/Behavior "{command: '["type": "drive_on_heading", "data": 0.5]'}" --feedback, which gave:
  Sending goal:
  command: '[type: drive_on_heading, data: 0.5]'
  
  Goal accepted with ID: 691dd6ae4c67403aa3a0c4e28751b4da
  
  Result:
  result: false
  message: 'Invalid command JSON: Expecting value: line 1 column 2 (char 1)'
  
• Chat suggested ros2 action send_goal /behavior ugv_msgs/action/Behavior \ "{command: '{\"type\": \"drive_on_heading\", \"data\": 0.5}'}" \ --feedback, which indeed moved the robot:
  Sending goal:
  command: '{"type": "drive_on_heading", "data": 0.5}'
  
  Feedback:
  feedback: true
  status: queued
  progress: 0.0
  
  Goal accepted with ID: a86674f283804742844da11c8df33226
  
  Feedback:
  feedback: true
  status: drive_on_heading:0.00/0.50m
  progress: 0.0
  
  Feedback:
  feedback: true
  status: drive_on_heading:0.00/0.50m
  progress: 0.00999999139457941
  
  ...
  Feedback:
  feedback: true
  status: drive_on_heading:0.48/0.50m
  progress: 0.959850013256073
  
  Feedback:
  feedback: true
  status: done
  progress: 1.0
  
  Result:
  result: true
  message: ok
  
  Goal finished with status: SUCCEEDED
  
• Note that the rover was lifted from the table, so no lidar or IMU odometry, just turning of the wheels.
• The difference are the square brackets.
• The USB to Audio has arrived (13x).
• Unfortunately, with out 8Ω 5W Speaker.
• Actually, nor a speaker, nor a microphone is needed for the Jetson.
• Tested the speakers (Digital Output S/PDIF). Both Front Left and Right work. The two other options (Speakers - USB PnP Audio Device and Analog Output - Build in Audio) didn't gave any sound.
• There is also the Microphone from the USB-camera, which works.
• Also Digital Input (S/PDIF) - USB-camera works.
• Even Digital Input (S/PDIF) - USB PnP Audio Device works (USB-to-Audio not connected)
• Also Microphone - USB PnP Audio Device works (USB-to-Audio not connected)
• Only the Analog inpu - Build-in Audio doesn't work.
• Looking at the USB to Audio wiki, the microphone volume bar is not there. One of the FAQ.
• Going to section voice-control
• Voice control does NOT /cmd_vel.
• Inside ~/ugv_dev_ws I did colcon build --cmake-args -Wno-dev --packages-select ugv_voice. That is the only package build there (until now), so could do source ~/ugv_dev_ws/install/setup.bash.
• The command ros2 run ugv_voice voice_ctrl --ros-args -p language:=en can now be called, but fails on missing module pyttsx3.
• The module is in requirements.txt, which is called with build_first.sh. Note also the APT_PREFS=/etc/apt/preferences.d/block-gazebo-classic in build_first.sh. Also several function to purge_gazebo versions.
• Did simply python3 -m pip install pyttsx3==2.99.
• sherpa_onnx==1.12.28 was not available in the container, so did python3 -m pip install sherpa_onnx==1.12.28.
• Next python3 -m pip install sherpa_ncnn==2.1.15
• Also python3 -m pip install soundfile==0.13.1
• All packages now installed, but voice_ctlr.py loads relative its src directory, not its install: /home/ws/ugv_dev_ws/install/ugv_voice/lib/python3.10/site-packages/ugv_voice/../config/voice_config.yaml instead of install/ugv_voice/share/ugv_voice/config/voice_config.yaml.
• Corrected this, so the voice_config is loaded.
• Code now fails on line 31: pygame.mixer.init(): pygame.error: ALSA: Couldn't open audio device: No such file or directory
• The voice_config doesn´t specify the audio-device (yet see typo min_time_bewteen_play)
• Did sudo apt install alsa-utils, which was part of build_first.h
• Tested the different devices listed with aplay -l. speaker-test -D plughw:1,0 -c 2 -t wav works.
• Called AUDIODEV=plughw:1,0 ros2 run ugv_voice voice_ctrl --ros-args -p language:=en, which now breaks a few lines further:
  /home/ws/ugv_dev_ws/install/ugv_voice/lib/python3.10/site-packages/ugv_voice/../../../models/kws/sherpa-onnx-kws-zipformer-gigaspeech-3.3M-2024-01-01/tokens.txt does not exist
• The models are not copied, so should make a symbolic link to ./src/ugv_main/ugv_voice/ugv_voice/models/kws/sherpa-onnx-kws-zipformer-gigaspeech-3.3M-2024-01-01/README.md. Made the link, which is now loaded from correct relative path.
• Next fails line 132: self.keyword_spotter = _KeywordSpotter(keywords_spotter_config)
• This fails because the onnx files are still the Git LFS pointers, not the actual onnx-models. (files start with string "version https://...")
• Did sudo apt install git-lfs, followed by git lfs install.
• git lfs pull has to be done from a git-directory, in this case: ~/ugv_dev_ws/src/ugv_main/ugv_voice
• Now voice_ctrl.py fails later, on self.vad = sherpa_onnx.VoiceActivityDetector(
  RuntimeError: No graph was found in the protobuf.
• The model is not complete, the model.onnx is missing.
• The pretrained model can be found here. The TTS-model are based on 5 Chinese speakers. There is also English models, but VCTK has no dictionary.
• Downloaded wget https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/sherpa-onnx-vits-zh-ll.tar.bz2, voice_ctrl goes further.
• Had to install sudo apt install speech-dispatcher speech-dispatcher-espeak espeak-ng
• Get possitive message:
  [INFO] [1786637431.420254481] [voice_ctrl]: Audio USB connected
• Yet, I also get the negative message:
  Home directory not accessible: Permission denied
• That message is not from voice_ctrl.py, seems to come from pygame:
  pygame 2.6.1 (SDL 2.28.4, Python 3.10.12) Hello from the pygame community. https://www.pygame.org/contribute.html
• This seems to come from espeak, that tries to write into $HOME.
• namei -l "$HOME" showed that /home/ws was not owned by root, but user 1000. Did chown root /home/ws and chgrp root /home/ws. Warning is gone.
• Nothing written in $HOME, although I said some words.

August 12, 2026

• Checking Rover-12: it had batteries.
• Also put batteries in Rover-2. See if this rover has the same problem.
• Starting container with source ~/ugv_ws/ros2_humble.sh. When logging in with ssh -p 23 root@ROVER_IP I got several warnings I had not seen with Rover-12:
  -bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
  not found: "/home/ws/ugv_ws/install/teb_local_planner/share/teb_local_planner/local_setup.bash"
  -bash: /root/ugv_ws/install/setup.bash: No such file or directory
  
• Also vi doesn´t work :-)
• Tried to install vi, yet sudo apt update gave three warnings:
  sudo: unable to resolve host ubuntu: Name or service not known
  W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://mirrors.ustc.edu.cn/ros2/ubuntu jammy InRelease: The following signatures were invalid: EXPKEYSIG F42ED6FBAB17C654 Open Robotics
  W: Failed to fetch http://mirrors.ustc.edu.cn/ros2/ubuntu/dists/jammy/InRelease The following signatures were invalid: EXPKEYSIG F42ED6FBAB17C654 Open Robotics
  
• Had same problem on October 10, 2025.
• The unknown host is because /etc/hostname is still set to ubuntu, while /etc/hosts uses rover-02. Changed the /etc/hostname, no direct effect (should check after reboot).
• Uncommented 10 deb without space. The number of upgradable packages went up from 581 to 899.
• Downloaded the install_additional_ros_humble_packages.sh script. Selected again option 2 (keep current) for the sshd-server.
• Four packages have unmet dependencies:
  ros-humble-depthai-ros-driver-v3 : Depends: ros-humble-depthai-v3 (>= 3.7.1) but it is not installed
• Manually installing ros-humble-depthai-v3 failed on trying to overwrite '/opt/ros/humble/include/depthai/build/version.hpp', which is also in package ros-humble-depthai.
• Looked with dpkg -l | grep depthai. Both ros-humble-depthai-bridge (v2.12.2) and ros-humble-depthai-bridge-v3 (v3.3.0) were installed.
• In the install_additional_ros_humble_packages there is sudo apt-get install ros-humble-usb-cam ros-humble-depthai-* -y.
• Looked at documentation. The OAKD_Lite is rvc2-based. Seems to work also with v3 of the depthai-software. According to Humble documentation, I should use sudo apt install ros-$ROS_DISTRO-depthai-ros-v3
• Many conflicts between both version. Finally was able to remove old ros-humble-depthai.
• Also installed the kept back packages: sudo apt install libcudnn8 libcudnn8-dev libnccl-dev libnccl2. That has the following effect: npacking libcudnn8-dev (8.9.7.29-1+cuda12.2) over (8.9.6.50-1+cuda11.8)
• Also sudo apt-get install ros-humble-depthai-ros-v3 is now installed.
• The launch-file rgbd_pcl.launch.py looks interesting.
• Running ros2 launch depthai_ros_driver_v3 rgbd_pcl.launch.py
• Gives (inside container):
  /oak/imu/data
  /oak/rgb/camera_info
  /oak/rgb/image_raw
  /oak/rgb/image_raw/compressed
  /oak/rgb/image_raw/compressedDepth
  /oak/rgb/image_raw/theora
  /oak/rgbd/points
  /oak/stereo/camera_info
  /oak/stereo/image_raw
  /oak/stereo/image_raw/compressed
  /oak/stereo/image_raw/compressedDepth
  /oak/stereo/image_raw/theora
• The directory /opt/ros/humble/share/depthai_ros_driver_v3/launch/ has multiple launch files:
  calibration.launch.py __pycache__
  driver_as_part_of_a_robot.launch.py rgbd_pcl.launch.py
  driver.launch.py rtabmap.launch.py
  example_multicam.launch.py sr_poe_rgbd_pcl.launch.py
  example_segmentation.launch.py sr_rgbd_pcl.launch.py
  oak_t.launch.py stereo_from_rosbag.launch.py
  pointcloud.launch.py vio.launch.py
  
• The other two directories with launch files are: /opt/ros/humble/share/depthai_descriptions_v3/launch
  /opt/ros/humble/share/depthai_filters_v3/launch

August 11, 2026

• Updating the UGV Rover Manual.
• Our ugv_ws github is still 106 commits ahead of Waveshare's github.
• The only recent update is joy_ctrl.py
• There is also a Waveshare development branch, 31 commits ahead.
• One interesting to explore are the recommended combinations, like **Stronger obstacle avoidance** *(enough CPU)* | `… use_localization:=amcl use_localplan:=mppi` (or `teb` / `rpp`)
• Also should explore ugv_basics.md
• Also interesting to look at Installation manual, including the choice for Gazebo.
• Anyway, in this branch they have added 17 docs for 17 modules.
• Should try LiDAR intercation: demo.launch.py — includes bringup + one demo; exe:=lidar_follow, lidar_guard, or lidar_obstacle_avoidance
• Although experimental, maybe funny to start with behavior_ctrl, with or without voice-commands.
• Note that they also have ball-track and April-track examples in vision.
• Switched nb-real to Ubuntu-partition. Started up rover-12. The rover connect ed to ROBOLAB_VAR, but that access-point was not connected to the internet.
• Registered rover-12 to iotroam, to have internet-connection.
• Created a new directory ~/ugv_dev_ws/src and cloned the development-branch with git clone -b ros2-humble-develop-251125 https://github.com/waveshareteam/ugv_ws.git
• Started from the ugv_rover source ~/ugv_ws/ros2_humble.sh. Logged in at the docker-container with ssh -p 23 root@ROVER_IP. The rover had indeed ROS_DOMAIN_ID=12.
• Following Behavior Commmand Control, I started ros2 launch ugv_bringup bringup_lidar.launch.py use_rviz:=false.
• Startup fails on a KeyError (tried 3x).
• Had the same JSON decode error on November 5, 2025, with the recommendation to add some way to clean the JSON-buffers, or add some protection.
• Tried a reboot, still same KeyError.
• Bringup starts with an unexpected error: [ugv_bringup-3] [base_ctrl.feedback_data] unexpected error: device reports readiness to read but returned no data (device disconnected or multiple access on port?)
• Made /home/ws/script/check_is_jetson.py, and it is an ugv_jetson, which uses /dev/ttyTHS1.
• Checked permision on /dev/ttyTHS1. crw-rw---- 1 dialout . Root is part of group dailout.
• Gave additional o+rw permission, now ros2 launch ugv_bringup bringup_lidar.launch.py use_rviz:=false has no feedback error, but still fails on imu.
• Previously I started ros2 launch ugv_bringup bringup_imu_origin.launch.py use_rviz:=false, don´t know why. Trying to reboot again (inside the daemon). That wasn't enough. Trying again a full reboot.
• Instead, started ros2 run ugv_bringup ugv_driver (no output), ros2 run ugv_tools behavior_ctrl (output). Now ros2 action list | grep behavior showed /behavior, ros2 action info /behavior showed Action clients: 0; Action servers: 1. Yet, ros2 interface show ugv_msgs/action/Behavior failed, because ugv_msgs is only part of the development branch.
• Also cloned the ugv_dev_ws in the docker, copied the ugv_msgs into ugv_ws and did colcon build --packages-select ugv_msgs.
• Now ros2 interface show ugv_msgs/action/Behavior gives:
  string command
  
  ---
  bool result
  string message
  
  ---
  bool feedback
  string status
  float32 progress
  
• Sending command ros2 action send_goal /behavior ugv_msgs/action/Behavior \ "{command: '[{\"type\": \"drive_on_heading\", \"data\": 0.5}]'}" --feedback, but response is Waiting for an action server to become available.... Strange, because the action server seemed active.
• Checked the Data path, all three inputs are there, also the output (/cmd_vel is there.
• Tried ros2 topic pub /cmd_vel geometry_msgs/msg/Twist "{linear: { {x: 1.0}}" -1: no response.
• Tried ros2 launch ugv_bringup bringup_imu_ekf.launch.py use_rviz:=false, no error message. Last message:
  [robot_state_publisher-1] [INFO] [1786453218.437815102] [ugv.robot_state_publisher]: got segment right_up_wheel_link .
• Yet, no ros2 topic pub /cmd_vel geometry_msgs/msg/Twist "{linear: { {x: 1.0}}" -1 gave Waiting for at least 1 matching subscription(s)... . See with ros2 topic list: /cmd_vel. With ros2 topic info /cmd_vel I see one Subscription count: 1

August 6, 2026

• Lingbot-vision outperforms Dino v3. It needs as input RGBD, and outputs refined Depth and Refined point clouds. No ROS2-implementation.

July 24, 2026

• A page with SLAM Tutorials, including SLAM for Dummies, from 2013.

July 22, 2026

• Francisco Martin Rico wrote a new book on Software Architectures for Robots. Currencly only in available in Spanish, soon in English.
• Chapter 3 is focused on Perception and Representation.
• Chapter 5 is about Finite State Machines.
• Chapter 6 is about Behavior Trees.
• Chapter 7 starts with a description of NAV2, PlanSys2 and EasyNav. Nav2 works with Behavior Trees (BT), PlanSys with Planning Domain Definition Language (PDDL). EasyNav works mostly with plugins, for each part of the plan-decide-act cycle (Map, Localize, Plan, Control plugins). EasyNav is also developed at Intelligent Robotics Lab, Spain.
• The 2nd part of the book contains of practical tutorials.
• First chapter covers ROS2 basic commands.
• The software can be found at github, starting from the 2nd chapter (publisher / subscribe example).
• The 3rd chapter covers perception, based on both camera and laser observations. Concentrates on the code (C++, no python), the application (obstacle avoidance) less (for instance, no illustrations in this chapter).
• The 5th chapter is based on the Bump-and-Go example with Behavior Trees.
• The idea of the 2nd part that it gives directions what should be implemented, but would benefit from some illustration on 'this is how a solution could like'.
• Another remarkable fact that the 241 page book only uses two refences: the two versions of his previous 'Introduction to Robot Programming' book.
• Installing a GPU version of opencv-5 is not trivial. See this Tutorial. Key-facts: on Ubuntu 24.04, a pip install still gives version 4.6.0. Even on Ubuntu 26.04 you get a CPU-only version. A Jetpack first-source install is not verified yet.

July 20, 2026

• Looked at Autonomous Robotics course at JMU. The ROS2 Python video is using the Rocket Bot, with the following Python node code.
• The course has a whole github with teaching material. The final project is a rescue mission.

July 16, 2026

• This paper with code teaches a robot a visual map with the phone camera.
• Dataset can be explored with Rerun. No ROS-interface.

July 11, 2026

• Following the instructions from Francisco's textbook no longer works, because it is using the old method via the ros-archive-keyring.gpg.
• Followed the instructions from ROS2 Jazzy install and did it via the ros2-apt-source package.
• Installed ros-jazzy-desktop.
• Had to sudo apt install python3-rosdep before I could do rosdep install --from-paths src --ignore-src -r -y (after a sudo rosdep init ; rosdep update)
• Starting colcon build --symlink-install. The package br2_tracking fails on libopencv_video.so.3.4.4, on LIBAVCODEC.
• Checked br2_tracking. CMakeLists.txt only specifies that opencv is needed (no version), the ObjectDetector only inlcudes opencv2/opencv.hpp. < i> Checked the installed versions, which were in principle v4.0.6. Yet, in May 19, 2021 I created a number of symbolic links of libopencv_*.so pointing to version 3.4.4 in /usr/local/lib/. Moved all the links to libopencv_*.so.3, so should not overrule the latest libraries in /usr/lib/x86_64-linux-gnu/.
• Still same error, should define LD_LIBRARY_PATH with x86 earlier.

July 9, 2026

• Luxonis also introduces agent skills for their product.
• Should work both with Claude and Cursor. Skills are available on github.
• Looked at the TDP of GentleBots (2024), where the point for the 3D detection boxes with the Tiago to this github code - darknet3d. Example can be found in this ConstructSim interview with a 3D 3D 3D bounding box around Francisco himself.

July 8, 2026

• Francisco Martin Rico's A Concise Introduction to Robot Programming with ROS 2 (2nd edition) is still on my VitalSource Bookshelf.
• On September 19, 2025 I was reading Chapter 7 (Deep ROS2).
• Now starting with Chapter 1, which already contains interesting information such as the naming convention of resources in ROS2, although not for VAR.
• Fig. 1.5 already adds a LiDAR and RGBD to the Kobuki robot, which is Korea’s first robotic turtle (introduced in 2020).
• Yet, in the same chapter Tiago from PAL Robotics is mentioned as platform, which is a modern variant of PR2.
• Tiago is used to create two subsystems, the Behavior subsystem which controls the Head of the Tiago and a Navigation substem.
• In the github code with the book I only see a simultiaon of the Tiago, no Kobuki.
• The pal and tiago packages are loaded via third_parties.repos.
• The humble-branch (Ubuntu 22.04) has no status, for Ubuntu 24.04 both an active Jazzy and Rolling version are available.
• The Construct has a Tiago course, but ROS1 based.
• Finished with Chapter 1 of A Concise Introduction to Robot Programming with ROS 2.
• Chapter 2 of A Concise Introduction to Robot Programming with ROS 2 covers the Basics of ROS (the talker example). Quite usefull for the first seminar. No python examples so far as section 2.3. Section 2.3.1 introduces rqt_console tool to filter the log-messages in /rosout.
• Section 2.4 finally introduces the Simulated Robot Setup of the TIAGo. The TAIGo is one of the reference simulator models of ROS2 (Which are the others? At least the Turtlebot). Found no direct list (other than an old ROS1 URDF examples list). UvA AI chat only mentions Husky and Rosbot next to the Turtlebot3. The ROSbot3 from Husarion is quite compatible with the UGV Rover, only Raspberry Pi based and no PanTilt. Yet, it has a OAK-D Lite camra and a SLAMTEC C1 Lidar (and 4 wheels with indeependent quadrature encoders). Would be a good entree to explore its simulation tutorials, including SLAM toolbox + Navigation2.
• Learned that the option --no-arr prevents that you are overloaded with the contents of the array, i.e ros2 topic echo --no-arr /scan_raw
• Now at the end of Chapter 2 also rviz is introduced.
• Continue with Chapter 3. That is introducing Finite State Machines with Bump and Go problem. It is a virtual bumber, based on the laser measurements. This chapter has both a c++ and python implementation.
• Continue with Chapter 4. That introduces the TF subsystem. Maybe nice to highlight that in the paper. It also highlights the debugging tool Visual Markers, which is part of RVIZ. Instead of checking the distance to an object directly in the FSM, it introduces an obstacle_detector and obstacle_monitor pair. The monitor just points a marker to the object, doesn't react to it (yet).
• Chapter 5 will introduces Virtual Force Fields to navigate. It is all still LiDAR based, so it will be interesting to see if that could also be done with vision (obstacle avoidance as assignment 1).
• This chapter also introduces GoogleTest to check the C++-code. Nice, but a bit of a side-line.
• In section 5.2 the objects are tracked, that is where the camera in the head of the Tiago comes in (or the pan-tilt of the UGV Rover). The head is controlled with a PID-controller on the pan-tilt of the neck.
• Chapter 6 implements Bump and Go again, but now with a Behavior Tree. Here the testing is actually quite interesting, testing each of the BT nodes seperately.
• In section 6.3 a patrolling Behavior Tree is introduced, that makes use of Nav2 to find a charging system. Although demonstrated on the Tiago, it needs the ros-*-turtlebot3* packages together with the navigation2 packages.
• As assignment, it is suggested to publish the detected objects as 3D bounding boxes, as in Perceptor3D (Copyright GentleBots, a RoboCup@Home team (2021) from IntelligentRoboticsLabs of the Rey Juan Carlos University and University of León with uses the Tiago).
• Section 7 goes deep into Real-Time constraints of ROS2, although the Nao robot is mentioned, with both a CPU for the code in the head and a microcontroller the chest to control the motors.
• Also section 8 is interesting, but a bit off. Yet, section 6.3 nicely aligns with VAR, although well hidden.
• Let see if I can get to section 6.3 at WP9.
• I had already a directory ~/bookros2_ws. Checked with git rev-parse --abbrev-ref HEAD that I was on the humble-dev branch.
• The command vcs import < book_ros2/third_parties.repos showed the same, although several of the ThirdParty directories were a few commits behind.
• Note that pal_gazebo_worlds has several aruco_markermaps
• The command colcon build --symlink-install give some warnings and errors.
• The package launch_pal was already installed with apt-get. Add for the moment a COLCON_IGNORE in the directory.
• Both Groot and hey5_description had an old CMake version. Increased the cmake_minimum_required from respectively 3.2 and 3.4.0 to 3.5
• Same for pal_gazebo_worlds and several others. At the end all the packages could be build
• Run without problems Chapter 2: ros2 run demo_nodes_cpp talker and ros2 run demo_nodes_py listener. That is basic ros demos, not code from the book (although described in the book).
• Also ros2 run br2_basics logger works and could be monitored with ros2 run rqt_console rqt_console.
• The output of ros2 run br2_basics publisher_class could be inspected with ros2 topic echo /int_topic and received by ros2 run br2_basics subscriber_class.
• Also ros2 launch br2_basics pub_sub_v1_launch.py was there.
• Next tried ros2 run br2_basics param_reader --ros-args -p number_particles:=300.
• Most interesting is section 2.4
• Starting the simulator worked only with an additional parameter: ros2 launch br2_tiago sim.launch.py --ros-args --remap is_public_sim:=True. Even then, the play_motion_executer and arm_tucker failed. No gazebo world showed up.
• The moveit_config couldn't find the config/tiago.urdf.

July 7, 2026

• Installed brave-browser. That browser could load claude.ai. Using Sonnet
• Loaded the SKILL.md from print-system-info.
• Gave the prompt "“Read the SKILL.md from jetson-device-skills and generate a Python script that runs on Jetson Nano and calls these skills.”
• Script was created, which gave info on the Model, L4T release, Power mode, Kernel, Uptime and OS version.
• In the SKILL.md examples of the expected output are given.
• The jetson-package skill should suggest docker images especially made for the jetson.
• Running it in the generated_scripts folder doesn't work, because it calls scripts/artifact_hints.sh, which calls ../../jetson-diagnostic/scripts/detect_jetson.sh.
• Calling the scripts suggest a preferred_vllm_image, to use ghrc on Orin's with older releases the L4T r39 and a preferred jetson-ai-lab pypi.
• I added the preferred pypi for the jetson-user in ~/.config/pip/pip.config. Had some difficulties with finding the correct CUDA version. The default for Jetpack 6.0 is 12.6, but most methods to check (nvidia_smi, nvcc --version, cat /usr/local/cuda/version.txt, strings /usr/lib/aarch64-linux-gnu/nvidia/libcuda.so* | grep -i "CUDA Version") all failed. The only method that worked was dpkg -l | grep -i cuda, yet gave as version 36.3.0.
• Did pip install torch torchvision, which indeed checked the jetson-ai-lab.io first, but still cu126 packages were loaded, and cu13 packages. Maybe I should have selected the sbsa/cu130 subdirectory.
• Still, nvidia_smi and nvcc are not found.
• Tried the jetson-memory-audit. Running the script with drop-caches --mode 3 reclaimed 865 MiB (from 81 MiB free to 1183 MiB free, with 1423 MiB available).
• The jetson-llm-benchmark requires that jetson-llm-serve is called first, to start a vllm-server. I run out of my 5h window of Claude messages of my free plan.
• There is a release of SE3-LIO. As default LiDAR it uses livox, but also has a ouster-option. Best to test it on the Avular Origin, also because it is docker-based.
• I see some issues in the dockerfile. First, it is based on Ubuntu22.04 / Cuda 11.8.0 ?! Further I see a clone of Livox SDK2 but no option for the Ouster. For the rest it is mainly ros-humble-desktop ros-humble-pcl-ros based.
• I see the new Mid-360s at Livox as industrial Lidar, and the HAP as automotive Lidar.

July 6, 2026

• Experimenting with jetson-device-skills on the UGV-Rover #12.
• No ethernet, no wlan. Switched on the ROBOLAB_VAR in the Visualisation Lab. Now I have a connection.
• The device-skills are installed without problems, but I need an agent to work with this skills.
• AIchat suggest to start with Claude Code in a browser.
• Installing sudo apt-get install firefox first, because #12 didn't had a browser.
• The system is 563 outdated packages. The rover is running Tegra release R36, which is actually Jetpack 6.0
• Trying to update the outdate packages, but the 4 nvidia-container-toolkit packages will be downgraded?!
• Starting firefox gave an error, so trying if that is still the case after the upgrade.
• It is, so removed firefox again.
• Otherwise, sudo apt-get install falkon would be a more light-weigth option, although that requires many Qt-libraries. Installing 197 packages together with falkon.
• Falkon works, but not with claude.ai
• On July 17, 2025 I had the same error with firefox, and installed the brave-browser instead of firefox.
• Lets try tomorrow with brave.

July 4, 2026

• Read Autonomously Mapping with Visual SLAM blogpost from Michael Hart.
• Also read Deploying Ultralytics YOLO26 Using ROS 2, which indicates a numpy-version conflict, which forced Michael to install YOLO26 from source.
• His Getting Started with ROS 2 is from Jan 2025.

July 3, 2026

• The code of lingbot-map is impresive, tested on many datasets, but not ROS-based.

May 20, 2026

• The MuJoCo wrapper TAMPanda could also create MuJoCo environments for mobile robots, including Lidar, Odometry and IMUs.

May 12, 2026

• Alejandro used a carrot follower for a Unitree Go2-W. I found this this ROS1 code, which seems to correspond with the last module of the ROSi course from Estonia.

March 23, 2026

• Should check Nvidia's cuVSLAM on the UGV Rover.

March 19, 2026

• Would be nice to test the camera backprojection from this github.

February 18, 2026

• Technical details and features of the IMX335 are discussed this post. For instance, the IMX335 has high sensitivity thanks to backside-illumination.

February 13, 2026

• Start reading in ROS 2 Programming and Simulation (Jul 2025). The code can be found on github.
• The first chapter is quite communication oriented. Should continue with Getting Started with ROS 2 Programming.
• Next tutorial would be using the LeRobot example from RoArm M3 AI Kit. The code can be found on github
• Note that the example also mentions a USB-sound card.
• They are not part of the package. Waveshare actualy has two options: A hat for the Jetson Nano or USB to Audio.

February 11, 2026

• More details on the IMU ICM20948 can be found here
• This product is discontinued, but the same ICM20948 is used on the Waveshare control board. The only difference is the barometric pressure sensor BMP280, which gives the 10 measurement next to the 9 DOF from the IMU.
• There is still a IMU, which is ROS2 Humble compatible and with Demo code.
• The ROS Driver for Robots is the ESP32 board which is more modern than General Driver for Robots.
• Note that the laptop battery can maybe be mounted with the smartphone holder

February 4, 2026

• NASA's Ogma is a tool to generate runtime monitoring applications to detect anomalies. Nice for a SE-project.

February 2, 2026

• Received the DC-charging connectors (2.1x5.5x11). Connector seems to work.
• Checking the Lab0 introduction in Canvas. Several commands do not make sense (no ugv_driver.launch.py in ugv_bringup). Is this a missing pull in the github, or wrong documentation?
• Starting ros2 launch ugv_bringup bringup_imu_origin.launch.py use_rviz:=false, as described in my documentation, failed on Keyerrors of the IO-communication.

January 8, 2026

• Removed all extension cords to remove the random crashes of the pan-tilt camera.

Previous Labbooks

• 2025
• 2024
• 2023