July 30, 2024

• Tried to install the ROS2-rerun-integration at WSL of nb-dual.
• There is already an install directory, which also has a hook and a pythonpath.sh and ament_prefix_path.sh.
• Yet, running . install/setup.bash before the build fails on missing ./build/rerun_visualization/share/rerun_visualization/hook/pythonpath_develop.sh
• Started with python3 -m pip install -r requirements.txt (after sudo apt install python3-pip). Version 0.17.0 of the rerun-sdk is installed.
• The structure of the github is a bit strange, with rerun_visualisation twice defined, once in the root, once in src/rerun_visualisation, which only contains generic_cartographer. The generic_cartographer has a clear package.xml, which only depends on ament_make, so I skip the rerun_visualisation directory for the moment by doing touch rerun_visualization/COLCON_IGNORE. That is a trick from the colcon-tutorial, which also gives the install procedure sudo apt install python3-colcon-common-extensions
• The package generic_cartographer is succesfully build.
• The easy_install is now only a warning. Yet, now I get warnings that some files cannot copied from buiil to install, because they already exist.
• One of them is .//rerun_visualization/share/ament_index/resource_index/packages/rerun_visualization, which is a symbolic to something in /home/ubuntu. After moving that file and ./install/rerun_visualization/share/rerun_visualization/package.xml to /tmp the build works. I ma using v3.10.12 of python.
• Yet, running ros2 run rerun_visualization rerun_node fails on map_callback() function, but on the metadata. Looks like rerun is missing a plugin.
• Looks like something is wrong with the PYTHONPATH, because I can load rerun_visualisation from the git-root, but not from rerun_node. In the PYTHONPATH both ~/git/ROS2-rerun-integration/build/rerun_visualization as ~/git/ROS2-rerun-integration/install/rerun_visualization/lib/python3.10/site-packages are included.
• The command ros2 run rerun_visualization generic_cartographer also fails, because which is correct, because I don't see generic_cartographer in the PYTHON_PATH, only in the AMENT and CMAKE_PREFIX_PATH.
• Maybe I should check out the node-branch, instead of the main branch.

July 29, 2024

• Looking into ROS2-rerun-integration project.
• The first build command is using a depreciated setup.
• The setup command can be executed with sudo python3 setup.py install warning only, which creates /usr/local/lib/python3.10/dist-packages/rerun_visualization-0.0.0-py3.10.egg.
• Yet, the next command . install/setup.bash now fails with file ~/git/ROS2-rerun-integration/build/rerun_visualization/share/rerun_visualization/hook/pythonpath_develop.sh not found.
• Cleaned up with sudo python3.10 -m pip uninstall rerun_visualization.
• Instead, tried rerun-io/cpp-example-ros2-bridge.
• First installed pixi, but running the two pixi examples failed (client error, a key-agent crashed).
• The pixi task tries to make a new temporary humble_ws/ and install a number of packages in a virtual environment.
• Instead tried to do it in an existing workspace.
• First tried if my ~/turtlebot3_ws was still working. Had to unset COLCON_PREFIX_PATH, AMENT_PREFIX_PATH, CMAKE_PREFIX_PATH, because thos still pointed to ROS2-rerun-integration.
• Running colcon build --symlink-install failed on missing package rosidl_typesupport_c. Did sudo apt install ros-humble-rosidl-typesupport-c.
• Actually, I restarted the shell after installing pixi, so I should also restart ROS2 Humble.
• Tried the go2_ros2 first. Run out of disk-space, so removed miniforge, which was my conda initialize.
• Next it could not load the rosbag mcap-plugin, so did sudo apt-get install ros-humble-rosbag2-storage-mcap.
• Tried again, but now the rosbag2_player gave a warning on 'go2_interfaces' not found, so tried to install the official package.
• No official package, so looked into pixi.toml.
• That made the go2_interfaces.
• The example still fials, but now on the rerun-viewer itself. Did the suggested python3 -m pip install rerun-sdk==0.16.0, although 0.17.0 is the latest version. Yet, using the latest sdk gave a version conflict with the viewer (~/local/bin/rerun.
• A bit strange, because the viewer should be part of the sdk. Anyway, with version 0.16.0 it works:
  

June 10, 2024

• The CuRobo paper compares its IK with the open source Track IK, described in this paper

May 17, 2024

• Peter is missing the omni-kit-numpy. The omniverse has its own pip_archive, as explained on this page. It says that the Kit comes with the omni.kit.pip_archive extension that includes many commonly used packages, like numpy or PIL. To use them just add dependency [dependecies] "omni.kit.pip_archive" = {}.

May 14, 2024

• Looking for additional literature on the combination of RTAB-Map and the KUKA youBot.
• Experimenting with Zeta Alpha.
• Looking for something simular with Intelligent Autonomous Navigation of Robot KUKA YouBot didn't give much relevant suggestions.
• Directly searching with the query RTAB-Map youBot gave two good suggestions:
  • Simultaneous Localization And Mapping (SLAM)using RTAB-Map
  • b-it-bots RoboCup@Work Team Description Paper 2023
• The last one has a repository for three ROS version (Kinetic, Melodic, Noetic).
• On their platform the use a Lithium-ion e-bike battery. In the paper they still use 2x 12V LiFePO4 batteries.

May 13, 2024

• Juliette was able to perform HelloWorld on the KUKA youBot.

May 8, 2024

• Basille was able to access the Velodyne VPL-16, although he still has some problems with visualisation.

April 12, 2024

• It would be great to simulate and control the Mover5 robot with cuRobo.
• The installation of cuRobo requires to install Isaac Sim, which requires OmniVerse and CUDA 11.7 (which seems old).
• ws8 is Ubuntu 20.04.06. Checked the pytorch version with python3 -c "import torch; print(torch.__version__)", but no pytorch installed.
• There are two version of python3 available: python 3.8.10 (default) and 3.10.13. python 2.7.18 is also still available.
• Started with Omniverse. Asindicated , I get options to store the libraries and data, but for Linux that is in ~/.local/share/ov and ~/.cache/ov.
• On ws8, installations from cuda-11.7, cuda-12.1, cuda-12.4 can be found, although only the last seems complete. Cuda-11.7 only has a gds-directory.
• Found ISAAC SIM (version 2023.1.1) in Exchange. Note that version 2023.1.1 is one minor version higher than specified for cuRobo.
• Install succeeded, a post-installation script showed up that starts up the cache for the main.
• Next is cuda-11.7.
• Needed to reboot to get sudo rights.
• After that most of the instructions, although sudo add-apt-repository contrib fails and I used an explicit sudo apt-get -y install cuda-11-7.
• Installation seems to succeed, I only get warnings on the post-install instrutions, like:
  /sbin/ldconfig.real: /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_adv_train.so.8 is not a symbolic link
• Indeed, for each of those the libraries there seem to be a copy, instead of a link:
  51339832 Nov 10 12:36 libcudnn_adv_train.so.8.0.5
  51339832 Nov 10 12:36 libcudnn_adv_train.so.8
  51339832 Nov 10 12:36 libcudnn_adv_train.so
  
• Can easily be solved, but I don't need cuda 12.1 for the moment, so leave it for the moment. Keeps complaining about this for each installation, so repaired it anyway. Installed all remaining updates.
• New terminal to be sure that changes in ./bashrc are included. Command nvcc --version gives Build cuda_11.7.r11.7/compiler.31442593_0.
• Step 6 of the installation (building the source), succeeds, although I see a warning on the installed packages. Tried to solve this omni_python -m pip install usd-core==23.11, followed by omni_python -m pip install nvidia-srl-usd==0.13.0
• After this, step 7 succeeded.

March 8, 2024

• Both Basile and Juliette will come for an internship.
• The query on KUKA youBot 3D SLAM gave two hits:
  • Harikrishnan Suresh and Nitk Surathkal, SLAM and Navigation using LIDAR for a KUKA youBot, project report, Indian Institute of Technology Madras, July 2016
  • Freddy de Greef, A KUKA youBot simulation in USARSim, Bachelor thesis, Universiteit van Amsterdam, June 2015
• The first report is without code, although Harikrishnan Suresh is now a graduate student in the Robotics Institute at Carnegie Mellon University, so could be contacted.
• The thesis of Freddy has two interesting citations:
  • József Kuti and Péter Galambos, "Revitalizing KUKA youBot project for Research and Educational purposes: Architecture of a new C++ driver", 2023 IEEE 17th International Symposium on Applied Computational Intelligence and Informatics (SACI), Timisoara, Romania, 2023, pp. 605-610.
  • Alireza Mohseni, Path planning for mobile robots, Thèse de doctorat électronique, Montréal, École de technologie supréieure, Université du Québec, June 2021.
• The PhD-thesis from Montréal describes the KUKA youBot quite well in Chapter 3. The same robot is used for the experiments in Chapter 4. Both chapters use 2D-maps for the path-planning. This is a PhD-thesis without code.
• The Hungarian paper is a paper with code. The code is strange enough Windows-based, and indicates that it requires a firmware update of the TMCM-1610 firmware. Note that this project only build a new drivers for the arm, new drivers for the platform are in the futrue plans. The TMCM-1610 was also mentioned in youbot github project on gravity compensation of the arm, as described in his Master thesis from ETHZ (Sept. 2013).
• An interesting citation in the Hungarian paper is Intelligent Autonomous Navigation of Robot KUKA YouBot (Aug. 2019). Seems to be the result from a student project from Equador
• Another publication mentioning the TMCM-1610 is Low-Cost Automation for Gravity Compensation of Robotic Arm (May 2020).
• A good entry to the TCML programs can be found at analog.com.

February 28, 2024

• This is an implementation of Point-LIO algorithm for the Unitree Lidar L1.
• Point-LIO is a robust and high-bandwidth lidar inertial odometry (LIO) with the capability to provide accurate, high -frequency odometry and reliable mapping under severe vibrations and aggressive motions.

February 23, 2024

• There is a new challenge at the RoboCup this year: the ARM challenge.
• Challenge is to grasp with a UR5e robot.
• There is a github page with docker running gazebo.

February 21, 2024

• Installed Pulse-vpn on ws9, so I can directly update the labbook.
• Started the ./start_arm.sh again, this time the GUI starts up.
• Could be due to login / logout, could be that there is now no GPU job running in the background, could be that the system rebooted overnight. Also, I connected now the ZED-camera!
• Tried to run the camera-node only with ros2 run ros_interface_umi_rtx nodeCameraAPI (after sourcing /opt/ros/humble/setup.bash and ~/git/LAB42_RTX_control/ROS_ws/install/setup.bash
• Running ZED_Explorer, which is also not able to connect to the camera. Need a reboot, but Thomas is running a job.
• After a reboot, the camera can still not be found. On September 17, 2021 I had to wait 60 seconds, but this amount of time is already gone. Time to test the ZED_Diagnostic.
• The tool ZED_Diagnostic complained that no USB-3.0 connection was used. Moved to the blue USB-slots at the bag. Still a problem, but the tool suggested to flip the USB-C connection 180 degrees. The blue LED next to the text 'ZED M' is now also on. So it now works, although the firmware should be updated:
  
• According to this post a download option should become available in the ZED Explorer, but that notification doesn't show up.
• In the settings of the ZED Explorer the firmware version 1523 is displayed, so the update was already performed (in the background). Diagnostics is also OK now.
• Tried to see what is published by the node ros2 run ros_interface_umi_rtx nodeCameraAPI.
• When I do ros2 topic list I get:
  /depth_image
  /parameter_events
  /processed_image
  /processed_pose
  /rosout
  
• I tried to follow the instructions of StereoLabs. Started ros2 run rviz2 rviz2.
• I could add an Image plugin for /depth_image and processed_image. No PointCloud2 messages, no RobotModel topic:
  
• Didn't see the /processed_pose, although ros2 topic info /processed_pose that is of type geometry_msgs/msg/Pose and that rviz is a subscriber. The poses that are possible are /goal_pose and /initialpose (with confidence). Note that Rviz gives a warning: [rviz2]: topic '/processed_pose' has more than one types associated. rviz will arbitrarily use the type 'geometry_msgs/msg/Pose'
• Now also the ./start_arm.sh script launches the GUI including camera stream:
  

February 20, 2024

• Moved ws9 to the UMI-RTX, and checking if the code still works.
• Cloned Guillame's github, but that was a version developed for foxy.
• So, the umi-rtxd works fine, but the install_dependencies fails on trying to install ROS foxy on Ubuntu. There are two options:
  • Use robostack, although even there ROS foxy is no longer supported.
  • Upgrade the code to ROS humble, the current ROS2 version.
• Followed the instructions, installed conda and mamba. Did mamba create -n ros_foxy_env.
• Yet mamba install ros-foxy-desktop fails, because the package is no longer available.
• Some older packages are still available when I do conda config --env -add channels robostack. I could do mamba install ros-foxy-xacro. Yet, I didn't see ros-foxy-pinocchio (only ros-noetic-pinocchio).
• I could do mamba install ros-foxy-rviz-common, which installs many other packages (209 to be precise).
• Yet, pinocchio can be directly installed with mamba install pinocchio. Not clear if the ros-bindings are than also OK, but that is to be tested.
• Also installed ros-foxy-ros-core. Tried to install ros-foxy-base, but that gives conflicts.
• Created a new ros-foxy-base environment. where I could install ros-foxy-base, ros-foxy-xacro and ros-foxy-rviz-common. Left pinocchio for the moment.
• Moved to the LAB42 github into the ROS_ws. Tried colcon build, but colcon is not a command.
• Installation with mamba install colcon-core didn't work. Uninstalled the colcon-core and installed sudo apt install python3-colcon-common-extensions instead.
• Made a ros-humble-env, where I could install mamba install ros-humble-desktop. Had to do mamba install pinocchio (no ros-humble package in robostack-staging).
• Next package missing was ZED. Installed the Ubuntu 22.04 / cuda 12.1 ZED SDK version (v4.0.8).
• colcon complained about a conflict between envs/ros_humble_env/lib/libopenblas.so.0 (v0.3.26) and the same library in /usr/lib (v0.3.20). Renamed the one in ros_humble_env.
• The colcon build command works now without problems, but the install-directory stays empty.
• Deactivated mamba. New terminal. Sourced /opt/ros/humble/setup.sh
• Cloned a fresh version of the github.
• Build the ROS_ws. Now the robot starts moving. rviz is complaining that it cannot find a suitable GLXontext for ogre-v1.12.1.
• Continue tomorrow.

February 13, 2024

• When we have created a map, we could look at the ROS-plugin from PathBench

January 24, 2024

• Impressive navigation, manipulation and gripping by Meta with their OK-Robot. Loved their Fig. 4 in the paper:
  
• Had to do some browsing before I found the page back that inspired me on TRI’s Robots Learn New Manipulation Skills blog-post.

January 4, 2024

• Impressive manipulation learning at Mobile ALOHA from Stanford University.

January 2, 2024

• This new algorithm came along; FoundationPose - to track objects on the table. Not needed for action generation, but reliable tracking could help during execution.

Previous Labbooks

• 2023
• 2022
• 2021
• 2020
• 2019
• 2018
• 2017
• 2016
• 2015
• 2014
• 2013
• 2012
• 2011
• 2009
• 2008
• 2007