August 21, 2023

• Tested the 4.0.6 version of the zed-os2-wrapper. The code starts running, but fails as expected on the initialization when checking for a GPU (while a eGPU is available, which is recognized by the ZedDiagnistics:
  [zed_wrapper-2] [INFO] [1692606313.870871385] [zedm.zed_node]: ***** STARTING CAMERA *****
  [zed_wrapper-2] [INFO] [1692606313.870878806] [zedm.zed_node]: ZED SDK Version: 4.0.6 - Build 76651_9c03a9f5
  [zed_wrapper-2] [INFO] [1692606313.872226684] [zedm.zed_node]: *** CAMERA OPENING ***
  [zed_wrapper-2] [ZED][INFO] Logging level INFO
  [zed_wrapper-2] [ZED][INFO] [Init] Depth mode: ULTRA
  [zed_wrapper-2] [ZED][INFO] [Init] Camera successfully opened.
  [zed_wrapper-2] [ZED][INFO] [Init] Camera FW version: 1523
  [zed_wrapper-2] [ZED][INFO] [Init] Video mode: HD720@15
  [zed_wrapper-2] [ZED][INFO] [Init] Serial Number: S/N XXXXXXXX
  [zed_wrapper-2] [WARN] [1692606317.298657519] [zedm.zed_node]: Error opening camera: NO GPU COMPATIBLE
  [zed_wrapper-2] [ZED][ERROR] [Init] CUDA error : 209 while allocating memory.
  

August 18, 2023

• Removed the link in ~/zed_ws/src to the ~/git/zed-ros2-wrapper.
• Instead did a git clone --branch dev_4.0.6 --recursive https://github.com/stereolabs/zed-ros2-wrapper.git. Note the remark "Note: The zed-ros2-wrapper repository contains the repository zed-ros2-interfaces as a sub-module. zed-ros2-interfaces contains the definitions of the custom topics and services, and the meshes for the 3D visualization of the camera models on rviz 2. It is very important to use the command --recursive while cloning the repository to retrieve also the latest version of the sub-module repository." from the documentation.
• Will test the wrapper on Monday.

August 17, 2023

• Testing the installation procedure of LAB42 RTX control.
• The install script is missing a sudo apt update.
• The colcon build complained that /opt/openrobots didn't exist.
• Another complaint is a warning that the ament cmake policy is not set (run cmake_policy to repair).
• The build fails on missing rviz_common/tool_manager.hpp. With the install script I got a warning that tool_manager was not writable. The wget was missing a sudo.
• Running ./start_arm.sh fails on missing numpy. Solved it with sudo apt-get --reinstall install python3-numpy.
• Now the script starts, with the final message The daemon has been started. Also receive the error that ~/git/LAB42_RTX_control/logs/ doesn't exist.
• Created that file, so now the ./start_arm.sh starts but fails on downloadlib has problems connecting to UNIX socket: Connection refused.
• Tried my own drivers, but also here the daemon complains that (RTXD) Couldn't open '/dev/ttyUSB0'. Strange enough, the permission are set back crw-rw---- once I run the daemon. Maybe a conflict with running the other daemon in the background. That daemon is a ros2 service: ros2 daemon start.
• In June 27, 2016 I had the same permission error, not due to root-permission, but because another daemon was still running.
• Rebooted, and now I could use my rtxd and also start ./start_arm.sh (after manually starting ros2 daemon start.
• Trying to get the ZED camera working. The node crashes on OPENCV when no camera is attached. Yet, I have an input problem, needing both the eGPU, camera and UMI-RTX connected to robot
• with an USB-C adapter, the Diagnostic of ZED is OK (v3.8.2 of the SDK, using CUDA version 11.7). Yet, according to nvidia-smi I am using CUDA 12.2. Moved the camera to the USB-C enry of my Dell-adapter ring. Both ZED_Diagnostics and ZED_Explorer are OK
• Downloading ZED SDK 4.0.6 for CUDA 12.1. Yet, now the Diagnostics fails on missing CUDA 12.1. The ZED_Depth_Viewer still crashes.
• ALso downloaded and installed CUDA 12.1. nvidia-smi still reports 12.2, but ZED_Diagnostics now at least reports only a warning for the SDK (multiple version of CUDA detected).
• Tried the start_arm.sh again, but still an error umi_rtx/nodeCameraAPI: undefined symbol: _ZN2sl17RuntimeParametersC1ENS_12SENSING_MODEEbiiNS_15REFERENCE_FRAMEEB.
• Used the command apt list --installed | grep cuda | grep '12-2' to remove all other dependencies on cuda-12-2.
• Also removes cuda-11-7. According to sudo update-alternatives --config cuda there is only one version of cuda, although ZED_Diagnostics says diffently. Still ZED_Depth_Viewer is not working, as start_arm.sh.
• Had comparible problems on August 26, 2021.
• Moved to /usr/local/zed/samples/depth sensing/depth sensing/python. Running this example gives: NO GPU COMPATIBLE. Looks like this fails on status = zed.open(init).
• Made the cpp-example. This gives a bit more information:
  [ZED][INFO] [Init] Camera successfully opened.
  [Sample][Error] Camera Open | NO GPU COMPATIBLE : An NVIDIA GPU (Pascal or newer) is required to run this software (for Kepler and Maxwell GPU, please use CUDA 10.2 ZED SDK installers) Exit program.
  [ZED][INFO] [Init] Sensors FW version: 517
  [ZED][INFO] [Init] Camera FW version: 1523
  [ZED][INFO] [Init] Video mode: HD720@60
  [ZED][INFO] [Init] Serial Number: S/N XXXXXXXX
  [ZED][ERROR] [Init] CUDA error : 209 while allocating memory.
  [ZED][WARNING] NO GPU COMPATIBLE in sl::ERROR_CODE sl::Camera::open(sl::InitParameters)
  
• The actual crash only happens in viewer.init().
• Looking at the depth_export code. The grab fails on:
  This function call cannot be processed because of the current state of your Camera instance.
• Looked if I could build the ros2 wrapper, but officially I should use Ubuntu 22.04 / ROS2 Humble.
• In addition, there is a branch 4.0.6 (the SDK version) I use. Yet, building fails on missing zed_interfaces.
• For instance, it was missing ~/zed2_ws/src/zed-ros2-wrapper/install/zed_interfaces/share/zed_interfaces/package.xml. Time to go.

August 14, 2023

• Looked at the introduction of the Kinematics course of ConstructSim. The other content (9h) is only available with a subscription.
• A nice online-tool to convert 3D rotations is here

July 19, 2023

• Left the HDMI-micro connector on the right HDMI-connection of Turtle5. Too much stress on this connection, but it still works.
• Connected the ethernet cable from Turtle5 to the SPL-server. Now I have an internet connection. Did a sudo apt update, which shows 400+ packages to be upgraded.
• Yet, an upgrade gives a lock by process 2461 (unattended upgrade).
• Looked for some hints at askubuntu. Tried first sudo systemctl stop unattended-upgrades which also doesn't return. Did a CTR-C and tried sudo pkill --signal SIGKILL unattended-upgrades. That worked, but still the lock is there. Tried sudo dpkg-reconfigure unattended-upgrades, which created /etc/apt/apt.conf.d/20auto-upgrades with Unattended-Upgrades set to "0", but still the lock is there. The advice is to give it time to finish the current unattended upgrade. Yet, tried a reboot first. Even that gives the warning that a unattended-upgrade is in progress (don't turn off the computer). After a few minutes the unattended-uprades ends and the system reboots.
• Note that after reboot a start job is running for more than a minute on /sys/subsystem/net/devices/wlan1.
• The service Cloud.init reports that it needed 3 minutes to finish.
• Note that the Turtle5 works fine on getting power from the USB-C, instead of the battery (although under-power was detected, activating the power-safe options).
• A sudo apt upgrade now works, so updated a lot of packages (including several ros-foxy packages).
• Switched the Unattended-Upgrades on again with command sudo dpkg-reconfigure unattended-upgrades

July 13, 2023

• To connect a monitor to the Raspberry Pi of the Turtlebot 3, you need a Micro HDMI cable
• Connected an ehternet cable and connected it to nb-ros (left nb-dual at the map). Tried to find the address with nmap -sP 169.254.0-255.0-255. Scanned 4096 hosts, none up (IPv4 set to Link-Local, ethernet adapter shows traffic on its LED).
• Found a free sd-card of 64 Gb, which contained a Jetson image with ROS noetic and PAL DreamVU software.
• The Raspberri Pi didn't have a sd-card, so that explains that I didn't see a ping (or got a HDMI signal on its USB-C port). Checked on the Rasberry PI forum, the latter is not possible.
• Started to download the Rasberry Pi 4B ROS2 Foxy image from robotis.com
• Installed the Raspberry PI imager with sudo snap install rpi-imager (v1.7.5). Unzipped tb3_rpi4_foxy_20210825.zip
• Configured that I have a wifi-connection at home. Also changed the default ROS2_DOMAIN to 105.
• Should try to specify a static ip, as indicated askubuntu (which indicates that I should check /etc/cloud
• Note that Turtle5 has a LDS-02 laser scanner, so I have to also the steps of 3.2.8.

June 29, 2023

• DeepMind trains in RoboCat project to pick up fruit and solve the Tower of Hanoi with a wide diversity of robot arms, but no UMI-RTX!
• The accomponying paper is arXiv.
• The Modular Manipulation library on github contains actually only a model for the Sawyer robot, and no ROS-support!

June 27, 2023

• The UMI-RTX has two motor controllers (IP0 and IP1), both 8031 microcontrollers, with 4 I/O ports, two 16-bit timers/counters, on-chip oscillator and a serial port. The pin-layout of the 40 pins can be found here
• More details on the architecture of the 8031 microcontroller can be found in booklet (99 pages), including the data transfer instruction (section 2.10). In section 4.4 an example of a Stepper Motor Controller with a RS232 interface is given.
• Chapter 6 describes a simulator / emulator of the assembly code, including the serial and SPI interfaces.
• In 2012 this blog used a picture of UMI-RTX from the UvA on a story of the Jim’ll Fix It chair, which had a UMI-RTX arm:
  
• The chair of Jimmy Saville (Doctor Magic) was quite famous (the 4th doctor Who sat in it). It was upgraded to be robotic controlled by Kevin Warwick.
• This chair has the real 80 vibes:
  
• Jimmy also had a robot dog visiting:
  
• Anyway, the looks were futuristic:
  

June 16, 2023

• Found a 142-page bachelor thesis called 'The ToySim Project', by Anthony Rodriguez, Edith Cowan University from 1997, which describes a simulator for the RTX-UMI build with the DirectX/Direct3D libary under Windows95.
• This video shows a zoom in of the wrist-cover.
• The same pins to fixate the wrist-cover can be found in the shoulder-cover.

June 2, 2023

• locosim is a framework that also is build on top of Pinocchio, and provides a general interface to robot manipulators, shielding the robot description and hardware control. Yet, it is ROS1 based; there is no ROS2 version. Yet, from the manipulators only the UR5 is implemented, the other models are quadrupeds.

May 24, 2023

• KDL is part of the Orocus framework, which is not trivial to integrate with other ROS-packages.
• Pinocchio is developed at LAAS France, which has both a C++ and Python interface. I didn't see a ROS-interface.
• The Rigid Body Dynamics Library is in principal C++ based, although it has a python wrapper. Yet, I also don't see a ROS-interface.
• The IKFast Kinematics Solver is integrated in MoveIt, so fully ROS-integrated. IKFast was originally part of OpenRave (last version March 18, 2013).
• It seems that Bio IK was originally made for Unity, whereafter it was reimplemented for ROS Kinetic (Newer versions of ROS should work, but may need some adaptation).
• Sebastian Starke had in August 2020 his PhD-defence on Bio IK thesis.
• Remembered that early on already build a Virtual Robot. The looked at the Unreal Engine, but at the end build it in a Java Applet.
• Not based on a URDF.
• An URDF of the UMI-RTX can be found inside hsrdp_bridge of the Hackspace Robotics Development Platform, which contains both a scooter base and a UMI RTX arm.
• Note that the project also has hsdrp_moveit code, including a mesh of the left_finger.
• As extra, the UMI RTX matlab manual from the University of Edinburgh.
• The code is still available at Advanced vision course page (Dec 2005)
• Should first install matlab at nb-dual, before I can check if this code still works.

May 16, 2023

• The ikpy package is conveniently written in Python, but this package seems to be not very efficient.
• Other packages like KDL, Pinocchio, rbdl, ikfast or bio IK seems to be much faster.

May 10, 2023

• Used my ERF labbook to find the commands to control the Mover5.
• Checked the usb-connection with dmesg | tail:
  [ 6468.859424] can: controller area network core
  [ 6468.859599] NET: Registered PF_CAN protocol family
  [ 6468.864473] can: raw protocol
  [ 6655.238716] peak_usb 3-4.2:1.0 can0: setting BTR0=0x00 BTR1=0x1c
  [ 6655.271523] IPv6: ADDRCONF(NETDEV_CHANGE): can0: link becomes ready
  
• Started the can_interface with source ~/bin/startup_can_interface.sh.
• Initiated the work environment with source /opt/ros/noetic/setup.bash followed by source ~/catkin_ws/devel/setup.bash
• Started the GUI with roslaunch cpr_robot CPRMover6.launch. I got a warning that cannot launch node of type [robot_state_publisher/state_publisher], which is strange because roscd robot_state_publisher brings me to the /opt/ros/noetic/share/robot_state_publisher. Yet, it is a bit strange to use catkin with noetic (is noetic not colcon based?)
• Cloned the noetic-devel branch into catkin_ws and made it with catkin_make --only-pkg-with-deps robot_state_publisher. The executable robot_state_publisher can be found in ~/catkin_ws/devel/lib/robot_state_publisher, but the launch file starts state_publisher.
• Changed the type of node in the launch file to robot_state_publisher, now the GUI starts without warning (robot was not connected, so there was still a CAN interface warning):
  

May 4, 2023

• Should add INCLUDE = -I . -I ../../include/umi-drivers -I../daemon to the Makefile of the shell.

March 27, 2023

• Looking around for the code for the UMI-RTX.
• Found this github page, with a Java and Python API.
• The page also contains some useful resources, such as Ball State University - Honors Thesis - Katherine Weber, July 2001. ""Teaching an old robot new tricks". Nice title, but more a labbook than a thesis, with quite some low-level details to get a RTX R100 robot working with LabVIEW.

March 16, 2023

• Continued from September 22, 2022 and started the endless loop from nb-ros.
• The code can be found in ~/git/umi-rtx/bin, and started with sudo ./rtxd /dev/ttyUSB0 and sudo ./playall joints.txt, as partly indicated at README.md.

Previous Labbooks

• 2022
• 2021
• 2020
• 2019
• 2018
• 2017
• 2016
• 2015
• 2014
• 2013
• 2012
• 2011
• 2009
• 2008
• 2007