June 10, 2024

• The CuRobo paper compares its IK with the open source Track IK, described in this paper

May 17, 2024

• Peter is missing the omni-kit-numpy. The omniverse has its own pip_archive, as explained on this page. It says that the Kit comes with the omni.kit.pip_archive extension that includes many commonly used packages, like numpy or PIL. To use them just add dependency [dependecies] "omni.kit.pip_archive" = {}.

May 14, 2024

• Looking for additional literature on the combination of RTAB-Map and the KUKA youBot.
• Experimenting with Zeta Alpha.
• Looking for something simular with Intelligent Autonomous Navigation of Robot KUKA YouBot didn't give much relevant suggestions.
• Directly searching with the query RTAB-Map youBot gave two good suggestions:
  • Simultaneous Localization And Mapping (SLAM)using RTAB-Map
  • b-it-bots RoboCup@Work Team Description Paper 2023
• The last one has a repository for three ROS version (Kinetic, Melodic, Noetic).
• On their platform the use a Lithium-ion e-bike battery. In the paper they still use 2x 12V LiFePO4 batteries.

May 13, 2024

• Juliette was able to perform HelloWorld on the KUKA youBot.

May 8, 2024

• Basille was able to access the Velodyne VPL-16, although he still has some problems with visualisation.

April 12, 2024

• It would be great to simulate and control the Mover5 robot with cuRobo.
• The installation of cuRobo requires to install Isaac Sim, which requires OmniVerse and CUDA 11.7 (which seems old).
• ws8 is Ubuntu 20.04.06. Checked the pytorch version with python3 -c "import torch; print(torch.__version__)", but no pytorch installed.
• There are two version of python3 available: python 3.8.10 (default) and 3.10.13. python 2.7.18 is also still available.
• Started with Omniverse. Asindicated , I get options to store the libraries and data, but for Linux that is in ~/.local/share/ov and ~/.cache/ov.
• On ws8, installations from cuda-11.7, cuda-12.1, cuda-12.4 can be found, although only the last seems complete. Cuda-11.7 only has a gds-directory.
• Found ISAAC SIM (version 2023.1.1) in Exchange. Note that version 2023.1.1 is one minor version higher than specified for cuRobo.
• Install succeeded, a post-installation script showed up that starts up the cache for the main.
• Next is cuda-11.7.
• Needed to reboot to get sudo rights.
• After that most of the instructions, although sudo add-apt-repository contrib fails and I used an explicit sudo apt-get -y install cuda-11-7.
• Installation seems to succeed, I only get warnings on the post-install instrutions, like:
  /sbin/ldconfig.real: /usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_adv_train.so.8 is not a symbolic link
• Indeed, for each of those the libraries there seem to be a copy, instead of a link:
  51339832 Nov 10 12:36 libcudnn_adv_train.so.8.0.5
  51339832 Nov 10 12:36 libcudnn_adv_train.so.8
  51339832 Nov 10 12:36 libcudnn_adv_train.so
  
• Can easily be solved, but I don't need cuda 12.1 for the moment, so leave it for the moment. Keeps complaining about this for each installation, so repaired it anyway. Installed all remaining updates.
• New terminal to be sure that changes in ./bashrc are included. Command nvcc --version gives Build cuda_11.7.r11.7/compiler.31442593_0.
• Step 6 of the installation (building the source), succeeds, although I see a warning on the installed packages. Tried to solve this omni_python -m pip install usd-core==23.11, followed by omni_python -m pip install nvidia-srl-usd==0.13.0
• After this, step 7 succeeded.

March 8, 2024

• Both Basile and Juliette will come for an internship.
• The query on KUKA youBot 3D SLAM gave two hits:
  • Harikrishnan Suresh and Nitk Surathkal, SLAM and Navigation using LIDAR for a KUKA youBot, project report, Indian Institute of Technology Madras, July 2016
  • Freddy de Greef, A KUKA youBot simulation in USARSim, Bachelor thesis, Universiteit van Amsterdam, June 2015
• The first report is without code, although Harikrishnan Suresh is now a graduate student in the Robotics Institute at Carnegie Mellon University, so could be contacted.
• The thesis of Freddy has two interesting citations:
  • József Kuti and Péter Galambos, "Revitalizing KUKA youBot project for Research and Educational purposes: Architecture of a new C++ driver", 2023 IEEE 17th International Symposium on Applied Computational Intelligence and Informatics (SACI), Timisoara, Romania, 2023, pp. 605-610.
  • Alireza Mohseni, Path planning for mobile robots, Thèse de doctorat électronique, Montréal, École de technologie supréieure, Université du Québec, June 2021.
• The PhD-thesis from Montréal describes the KUKA youBot quite well in Chapter 3. The same robot is used for the experiments in Chapter 4. Both chapters use 2D-maps for the path-planning. This is a PhD-thesis without code.
• The Hungarian paper is a paper with code. The code is strange enough Windows-based, and indicates that it requires a firmware update of the TMCM-1610 firmware. Note that this project only build a new drivers for the arm, new drivers for the platform are in the futrue plans. The TMCM-1610 was also mentioned in youbot github project on gravity compensation of the arm, as described in his Master thesis from ETHZ (Sept. 2013).
• An interesting citation in the Hungarian paper is Intelligent Autonomous Navigation of Robot KUKA YouBot (Aug. 2019). Seems to be the result from a student project from Equador
• Another publication mentioning the TMCM-1610 is Low-Cost Automation for Gravity Compensation of Robotic Arm (May 2020).
• A good entry to the TCML programs can be found at analog.com.

February 28, 2024

• This is an implementation of Point-LIO algorithm for the Unitree Lidar L1.
• Point-LIO is a robust and high-bandwidth lidar inertial odometry (LIO) with the capability to provide accurate, high -frequency odometry and reliable mapping under severe vibrations and aggressive motions.

February 23, 2024

• There is a new challenge at the RoboCup this year: the ARM challenge.
• Challenge is to grasp with a UR5e robot.
• There is a github page with docker running gazebo.

February 21, 2024

• Installed Pulse-vpn on ws9, so I can directly update the labbook.
• Started the ./start_arm.sh again, this time the GUI starts up.
• Could be due to login / logout, could be that there is now no GPU job running in the background, could be that the system rebooted overnight. Also, I connected now the ZED-camera!
• Tried to run the camera-node only with ros2 run ros_interface_umi_rtx nodeCameraAPI (after sourcing /opt/ros/humble/setup.bash and ~/git/LAB42_RTX_control/ROS_ws/install/setup.bash
• Running ZED_Explorer, which is also not able to connect to the camera. Need a reboot, but Thomas is running a job.
• After a reboot, the camera can still not be found. On September 17, 2021 I had to wait 60 seconds, but this amount of time is already gone. Time to test the ZED_Diagnostic.
• The tool ZED_Diagnostic complained that no USB-3.0 connection was used. Moved to the blue USB-slots at the bag. Still a problem, but the tool suggested to flip the USB-C connection 180 degrees. The blue LED next to the text 'ZED M' is now also on. So it now works, although the firmware should be updated:
  
• According to this post a download option should become available in the ZED Explorer, but that notification doesn't show up.
• In the settings of the ZED Explorer the firmware version 1523 is displayed, so the update was already performed (in the background). Diagnostics is also OK now.
• Tried to see what is published by the node ros2 run ros_interface_umi_rtx nodeCameraAPI.
• When I do ros2 topic list I get:
  /depth_image
  /parameter_events
  /processed_image
  /processed_pose
  /rosout
  
• I tried to follow the instructions of StereoLabs. Started ros2 run rviz2 rviz2.
• I could add an Image plugin for /depth_image and processed_image. No PointCloud2 messages, no RobotModel topic:
  
• Didn't see the /processed_pose, although ros2 topic info /processed_pose that is of type geometry_msgs/msg/Pose and that rviz is a subscriber. The poses that are possible are /goal_pose and /initialpose (with confidence). Note that Rviz gives a warning: [rviz2]: topic '/processed_pose' has more than one types associated. rviz will arbitrarily use the type 'geometry_msgs/msg/Pose'
• Now also the ./start_arm.sh script launches the GUI including camera stream:
  

February 20, 2024

• Moved ws9 to the UMI-RTX, and checking if the code still works.
• Cloned Guillame's github, but that was a version developed for foxy.
• So, the umi-rtxd works fine, but the install_dependencies fails on trying to install ROS foxy on Ubuntu. There are two options:
  • Use robostack, although even there ROS foxy is no longer supported.
  • Upgrade the code to ROS humble, the current ROS2 version.
• Followed the instructions, installed conda and mamba. Did mamba create -n ros_foxy_env.
• Yet mamba install ros-foxy-desktop fails, because the package is no longer available.
• Some older packages are still available when I do conda config --env -add channels robostack. I could do mamba install ros-foxy-xacro. Yet, I didn't see ros-foxy-pinocchio (only ros-noetic-pinocchio).
• I could do mamba install ros-foxy-rviz-common, which installs many other packages (209 to be precise).
• Yet, pinocchio can be directly installed with mamba install pinocchio. Not clear if the ros-bindings are than also OK, but that is to be tested.
• Also installed ros-foxy-ros-core. Tried to install ros-foxy-base, but that gives conflicts.
• Created a new ros-foxy-base environment. where I could install ros-foxy-base, ros-foxy-xacro and ros-foxy-rviz-common. Left pinocchio for the moment.
• Moved to the LAB42 github into the ROS_ws. Tried colcon build, but colcon is not a command.
• Installation with mamba install colcon-core didn't work. Uninstalled the colcon-core and installed sudo apt install python3-colcon-common-extensions instead.
• Made a ros-humble-env, where I could install mamba install ros-humble-desktop. Had to do mamba install pinocchio (no ros-humble package in robostack-staging).
• Next package missing was ZED. Installed the Ubuntu 22.04 / cuda 12.1 ZED SDK version (v4.0.8).
• colcon complained about a conflict between envs/ros_humble_env/lib/libopenblas.so.0 (v0.3.26) and the same library in /usr/lib (v0.3.20). Renamed the one in ros_humble_env.
• The colcon build command works now without problems, but the install-directory stays empty.
• Deactivated mamba. New terminal. Sourced /opt/ros/humble/setup.sh
• Cloned a fresh version of the github.
• Build the ROS_ws. Now the robot starts moving. rviz is complaining that it cannot find a suitable GLXontext for ogre-v1.12.1.
• Continue tomorrow.

February 13, 2024

• When we have created a map, we could look at the ROS-plugin from PathBench

January 24, 2024

• Impressive navigation, manipulation and gripping by Meta with their OK-Robot. Loved their Fig. 4 in the paper:
  
• Had to do some browsing before I found the page back that inspired me on TRI’s Robots Learn New Manipulation Skills blog-post.

January 4, 2024

• Impressive manipulation learning at Mobile ALOHA from Stanford University.

January 2, 2024

• This new algorithm came along; FoundationPose - to track objects on the table. Not needed for action generation, but reliable tracking could help during execution.

Previous Labbooks

• 2023
• 2022
• 2021
• 2020
• 2019
• 2018
• 2017
• 2016
• 2015
• 2014
• 2013
• 2012
• 2011
• 2009
• 2008
• 2007