Started Labbook 2024.

December 18, 2023

• The ConstructSim just published a new course: Generative AI for Robotics
• Started with the demo. The navigator is waiting for a Human command. The simulation shows a RT-2, but not the rest of the room (yet).
• The RT-2 starts looking for a cat (drives into the kitchen). I see the simulation, but the Guacomole desktop doesn't give the option to see the navigation map.
• The requirements for this course are ROS2 basics, Python 3 and OpenCV Basics. The next section requires a subscription.
• The source consists of ai_bot, deepmind_bot, deepmindbot_demos. The last has both move_to_waypoints.py and waypoints_ai.py
• The last script calls a fake as stub, but in principal should load a ai/nav_waypoint_t5_model, which could be found at /home/simulations/ros2_sims_ws/install/demo_navigation_ai_pkg/share/demo_navigation_ai_pkg/ai/nav_waypoint_t5_model. The generation_config indicates that transformer version 4.32.0 is used.
• As environment variable export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp was set. This points to this github implementation.
• This cyclonedds is actually not the transformer, but the distributed system with a high-performance publish-subscribe messaging system.

October 26, 2023

• Meta's Fundamental AI Research team has published Habitat 3.0, which seems comparible with IBM's SPEAR.
• Maybe again a good challenge for CV2.
• The link to the code can be found at AI meta page.
• Next to their code, there also exist a ROS Noetic bridge (although for an older version of Habitat Sim).
• The Habitit agent seem to work with /action and /rgb, /depth, which could be easily bridged to the ROS environment:
  
• Meta's FAIR indicated two challenges: Follow Human and Replace items.

October 9, 2023

• DeepMind released the dataset for robot manipulation tasks RT-x.
• Paper can be found here
• Could be a possible graduation project, to replicate such a task ('put the apple next to the orange') for the UMI-RTX, Mover-5 and/or Kuka youBot.

September 25, 2023

• Looked if the microphone board of the Nao5 could replace the board of the Pepper, but the connection has far more pins:
   
  Nao5 robot (left), Pepper head (right)
• The difference in pins is to be expected, because the Pepper has more microphones.
• Also on the other side this not works, the data-signal wires are fixed to the square board (disconnected it to see the other side, but there is not much space). The analog connection to the microphone itself have quite standard connectors:
  
• It is also possible to remove the head, when peering inside:
  

September 12, 2023

• Looking if I can use an external microphone with the Pepper.
• Two options are USB and Bluetooth.
• The documentation on USB is rather short, the information on bluetooth is more complete. For pepper, no USB-connectivity is mentioned.
• Connecting with Choregraphe to *.*.*.37 still works.
• Did rfkill list, but only wlan showed up (no Bluetooth).
• Looked at this code, which mentions that the front microphone is selected (self.micFront)
• This paper does experiments comparing the internal microphone against two external ones (section V).
• This is python code to record sound with the internal microphones (selected with tuple [0,0,1,0]). Created a script in ~/audio/python. ALAudioRecorder example fails on AudioSndFile Open has failed. Yet, this was due that the directory /data/home/nao/recordings/microphones/ didn't exist yet. When it exist, a small test.wav file was created (44 bits). Added a time.sleep(10) to the code, which made the test-file bigger, but nothing was heard during the 9 seconds recording.
• Some technical documentation on the microphones can be found here
• Looked for the (depreciated) arduino example from humarobotics.
• The library could still be downloaded, which I unpacked in C:\Packages\Aldebaran\ChoregrapheSuite2.5\share\choregraphe\libraries\box\ArduiNao.
• Yet, even Choregraphe 1.14.5 didn't recognize the format. (Selecting the box-directory ArduiNaoCheck). Going two directories up (selecting directory ArduiNao) worked. Yet, executing the ArduiNaoCheck indicated that Module ArduiNao was not found (what is correct, that should be running on the robot).

July 4, 2023

• The IKPy was original made for the poppy humanoid, a project which now has three designs
• The Humaniod is 7560 euro, the torso around 4680 euro. The Ergo Jr. Kit is no longer for sale, but the list of materials is quite short. It mainly consists of 6 XL-320 motors and an dedicated Pixl board. Also this Pixl board is no longer available, but the design files of the PCB are still available. The robot-shop recommends OpenCM9.04 controller (also sold out).
• At the producer (Robotis), the control board is still available, including 485 expansion board which extents the TTL 3-Pin bus to Dynamixel 485 4-Pin bus.
• Looked at the portfolio of Robotis. The XL series is their compact budget range. Unfortunally, the 320 stands for the dimension. The XL-320 is still mentioned, but clearly discontinued. The smallest available now is the XL-330 (with two different gear-ratio's). The OpenRB starterkit comes with a 288:1 gear-ratio. The XL-320 had a gear-ratio of 238:1. The difference is in gear-ratio has to be updated in the control-code, but that seems doable. The larger dimensions of the six motors also means that the design of the 3D-printed parts has to be slightly upscaled, to build a Poppy 2.0.
• The PhD-thesis on Poppy can be found inria.

June 26, 2023

• The Pepper broke its microphone-array board. After resetting the fuse, the robot is further still functioning (starting Autonomous Life, following faces, turning around). It only doesn't react to sound anymore:
  
• Created a ticket at support.

June 23, 2023

• It seems that Pepper was flashed. I had to agree on the Aldebaran terms, and only the visualisation wifi was known.
• Added LAB42 again, unmutted the robot, and made a script for the AlphaGamma partners.

May 22, 2023

• Movement of the Pepper arms seems to be done with joint-control, Cartesian control is depcreciated according to this post from 2016.

May 1, 2023

• Read Toward Holistic Scene Understanding. Most interesting is their Table 1, where they show different intergrative robotic scene perception approaches.
• Most semantic approaches are quite old (2018, or earlier). Most promeissing seems Kimera, which is also the framework that emerged for the Robotic Vision Scene Understanding Challenge .

April 13, 2023

• Clearpath has released the design files of PR2 robot.

April 6, 2023

• Would be interesting to apply this segment anything algorithm in a @Home setting.

March 31, 2023

• Switched the sound off from Pepper, after connecting with Choregraphe to *.*.*.37. Don't forget to switch it on again.

March 16, 2023

• Activated the Open Campus day script on Pepper. Both laptop and robot are connected to the LAB42 network (146.*).
• Started Choregraphe 2.5.10. Switched off Automatic Life and did a wakeup (sun-button).
• After that the script (loaded from recent projects) works fine.
• The script also works when the robot is on the charger.

February 22, 2023

• This Wrapfi paper is an interesting middleware for Pepper.

February 13, 2023

• Read the RoboHouse self-stocking-robot interview

January 31, 2023

• Found the issue. Next to stop-button is a short of fuse. That one was loose. Pushing it back in brought Pepper back alive.

January 19, 2023

• Created ticket #3087 for the Pepper robot, because it still not starts up.

Previous Labbooks

• Labbook 2022
• Labbook 2021
• Labbook 2020
• Labbook 2019
• Labbook 2018