Labbook IMAV
Started Labbook 2012.
December 19, 2011
- Read SYSNAV - Parrot's article about the navigation algorithms on-board the AR.Drone. Speed estimation is based on Lucas-Kanade when there is not so much contrast and on the FAST feature algorithm when enough cornors are visible.
- The article is not cited yet, although the earlier article about the hardware archecture is already cited 6x.
October 21, 2011
- Read the fourth chapter of Multiple View Geometry. This is a short review of statistics. Specific for geometry, is that in a 2D projection four points are needed to estimate the transformation. The estimate is normalized (centered around the center of those points) and the Jacobian is estimated for the surface between the points. Outside the surface the covariance grows fast (square of radius).
October 18, 2011
- The discussion of different error measures for 2D Projective Transformation in Chapter 4 of Multiple View Geometry seems to be relevant for Generalized-ICP.
- Different measures are algebraic distance, geometric distance, reprojection error and Samson error. The later one projects points on both images to the same space and tries to find a surface between those points (projecting the points on the surface and optimizing the transformation).
- Finding the Transformation is an iterative process. Before a lineair solution is found, both datasets have to be normalized (center of point-cloud found, scaled by size point-cloud). After that the DLT algorithm can be applied. The DLT tries to match every point. RANSAC is a better alternative, because it ignores outliers. Another alternative is Least Median of Squares, which is an algorithm without parameters (estimates of the noise level), but fails when the number of outliers is larger than 50$. Yet, also RANSAC and LMS estimate a lineair solution only; it solves the low order of the Taylor expansion. Non-linear effects can be solved by minizing the Maximum Likelyhood using for the Levenberg-Marquardt algorithm.
October 17, 2011
Reread the example at the end of Chapter 10 of Geometrical Algebra for Computer Science and checked Fig. 11.10. Section 11.8.9 describes projective transformations in the homogenous model of Geometric Algebra. Leo Dorst has described absolute conics and quadrics in Geometric Algebra, but conics and quadrics have different properties in GM. This is something still to be researched.
October 16, 2011
- Read the third chapter of Multiple View Geometry, which concentrates on 3D geometry concepts. Points and planes are straight forward extensions of 2D geometry, lines are more difficult (why for instance Pl\"{u}cker coordinates are needed. A full homogeneous matrix can be build up as decomposition of the simularity (7 dof; 3 trans, 3 rot, 1 isotropic scaling), affine (+5 dof; scaling in every direction minus isotropic scaling) and elation part (+3 dof). The absolute dual qadric is equivalent with the absolute dual conic. The absolute dual quadric has 8 dof, those not realted to simularity. The plane at infinity is null-vector of the absolute dual quadric.
October 15, 2011
- Read the second chapter of Multiple View Geometry, which concentrates on 2D geometry concepts. A projective transformation is equivalent with an homogeneous matrix. Depended on which elements are zero (or symmetric) this could be a projective, affine, simularity or Eucledian transformation. A full homogeneous matrix can be build up as decomposition of the simularity (4 dof), affine (+2 dof) and elation part (+2 dof). The dual conic of the circular points collects the contribution of the affine and elation part (4 dof), which means that when the dual conic is known, the simulariry part can be reconstructed (which is metric (Eucledian + scale)). The circular points indicate the orientation of the Eucledian space.
October 14, 2011
- Read the first chapter of Multiple View Geometry. New concept for me was the absolute conic, which represents the fundamental matrix of a camera on the plane at infinity. When knowing the absolute conic, one can reconstruct a Eucledian space from a projective space (just as an affine space can be reconstructed if the plane at infinity is known) [Andrea Fusiello's notes on Elements of Geometric Computer Vision]. Should reread the example at the end of Chapter 10 of Geometrical Algebra for Computer Science and check Fig. 11.10.
October 13, 2011
- Looking at the ultimate optical flow reference. Robrecht points to several conference articles, where Lucas article from 1985 seems to be the most fundamental. Yet this article is only 31x cited. Lucas points to Horn & Schunck (1981) and Cornelius & Kanade (1983) as people that applied optical flow. 5681x cited it the article of Horn & Schunk (1981). Horn & Schunk refer to earlier work of Clocksin (1978) and Pradzy (1979). Pradzy did his thesis about this subject, in his 1983 paper he calls it a still unresolved issue (and points on the work of Marr and Ulman, which is mainly biologally inspired).
- Sparse optical flow is also difficult to find. Marco Zucchelli's thesis (2002) uses the term, because his algorithm is a hybrid between sparse and dense optical flow. Marco points to David's Nister (2001) thesis as related work (using corner and line features, match them with RANSAC, followed by projective reconstruction), which is cited 101x. Didn't see many important references to Marco Zucchelli's work. David Nister has a nice applied approach, implementing nearly every chapter of Hartley's and Zimmermans book. Saved a copy in \Reports. Interesting should be the correct analysis of project frame stitching in Section 7.10 and the generalization to colour of Canny edges in Section 5.3.
- Good entry to Kanade-Lucas-Tomasi feature tracker is Birchfield's page. A good recent paper is II-LK: A Real-Time Implementation for Sparse Optical Flow, which describes that Shi-Thomasi 'What is a good feature to track' is the extension which reduces the number of feature to sparse. According to Robrecht, it is the pyramidal implementation which make the LK-algorithm less sensitive to distance. According to Shi-Tomasi, they minize in the same way as LK (Newton-Raphson algorithm), only they use affine distance, and LK a translation model. According to Shi-Tomasi both models are valuable (translation when the inter-frame distance is small, affine when the distance is larger). With the affine model, the goodness criterion on the texture is more sensitive and can detect the feature is a moving edge (sensitive to occlusions and disocclusions).
- Nister's article about depth-maps is later published in a symposium paper.
September 14, 2011
- Cleaning up 64 bits version of project (seems also to influence 'Any CPU'). For UsarLib, there is a problem with an unknown DataTable (although I have all original references).
September 13, 2011
- Installed version 3 of Sander's ImageServer.
- Received the logfile of yesterday's last flight. Format is in yaml:
---
e: 1
t: 2.324000
alt: 212
or: [2561.000000, -2544.000000, 34884.000000]
accel: [-2.614730, 36.819786, -1016.263672]
vel: [0.000000, 0.000000, 0.000000]
---
The first line e: 1 indicates an measurement event. There are also control (0) and frame (2) events. Unfortunatelly, no frames were registered yesterday.
- Added 64bits gdal-1.8 version to 2010/assistance. Had to replace System.dll and other .Net references from C:\Windows\Microsoft.NET\Framework to C:\Windows\Microsoft.NET\Framework64. Still SharpZLib complains about mscore.lib. Couldn't find any direct reference.
Tools starts complaining about ambiguous definitions in the .Net namespace?! Removing the System.dll reference solves this issue.
- The post-build event from UsarCommander copies all libraries from GDAL to the build-directory. Changed this from GDAL-1.6 to GDAL-1.8 (had to change the location of SpatialReference in GeoTiff.vb). Still a crash on same location.
- Adding the non-csharp libraries in the post-build. Now the system continues, but complains about a ContextSwitchDeadlock:
- Also teleoperating over the map goes well. DownButton gives KenafDown action. Map is rotated differently from sensors_all.png.
- Created map.pgw. Map is too low. LowerLeft corner is at (-286, -254). while (-286, -180) should be at lowerleft corner of sensors_all.png (-286, -108). Changed 6th parameter from 92.5 to 170.5. Now the two maps are overlapping (center of building for the map a bit to the upperright of the sensors_all). (-229, -79) vs (-203, -64).
Map a little to the left of the building. (-266.7, -64.1) vs (-254.1,-69.8). Final pgw:
0
-0.25
-0.25
0
-115.5
150
- With the georeferenced map, the map and cameraimages overlap nicely
September 12, 2011
- Solved issue with IMUsensor. IMU is not an option for deadreckoning, because currently it is combined with GPS. Should look if I could use Nick's code to get a position estimate from the speed updates.
- The Kalman filter is at the beginning of Nick's slam.cpp.
- August 27, 2010 I could load sensors_all.png both in the GeoTiffViewer and UsarCommander. In October 15, 2008 I made the last modification (update of GDAL to FWTools 2.6.6.).
Same problem also occurs in 2011/competition code. GeoTiffViewer can load sensors_all.png.
- The gdal libraries for the GeoTiffViewer are from 2007, the gdal_fw library 2010.
- On September 6 I report that flying around in the image works fine. Code seems to be in SimulatedFlyActor. Also found CameraSensor.MoveTo, which is called from Agent.MoveGroundTruth.
- After running setfw.bat, dgalinfo on sensors_all.png works. GeoTiff:Load fails on Gdal.Allregister.
- FWTools seems to be outdated (and 32 bits), downloaded several 64bits distributions from gisinternals.
September 11, 2011
- Started with C:\Games\UDK\UDK-junebeta\Binaries\Win64\UDK.exe " RoboSoccer?game=USARBotAPI.ImageServerGame -log -d3d10", but still the games runs in dx9 (as reported by Sander).
- Tried to start with C:\Games\UDK\UDK-junebeta\Binaries\Win64\UDK.exe " RoboSoccer?game=USARBotAPI.UPISImageServer -log -d3d9", but localhost is not available without network?! At the camping a network is available, but could get no connection (tested with telnet),
both for the 32bits and 64bits variant.
- Switched to 2010/assistance branch, to look at warmup assignment. Added protection against no FlagSensor in VictimsLayer:RenderAgent. Added protection against no AerialOverviewIm in CameraSensor. Tried to load sensors_all.png with Apriori-button,
but GeoTiff gives a Load ERROR. Initial problem was due to wrong config (sensors_all.png on D-disk). Still GeoTiff error remains. In OSGeo4W gdalinfo sensors_all.png gives good information. Loaded sensors_all.png in uDig and saved it as GeoTiff. Also this file cannot be loaded as apriori (no error/warning in SynchronizeStanfordAerialImage, but agent stops)
- Checked for UDK AirRobot different messages:
- msg = "STA {Time 439.6753} {Battery 99999}": both parsed.
- msg = "ASTA {Name cam} {Link 0} {Value -0.0453}": new, not handled yet
- msg = "MISSTA {Time 439.6753} {Name cam} {Link 1} {Value -0.0453} {Torque 0}": MISSTA? is this a bug. Yet, message is handled (didn't check if it was parsed)
September 10, 2011
- The code of Sander's ImageServer is created in ..\..\Binaries\Win??\UserCode. This Binary directory should be copied into the UDK directory. Still, it seems that ImageServerGame.uc is missing in UsarBotAPI.
- Succesfully made Bas Terwijn's SmartSocket (after installing ant). AbstractLogger still fails on unknown variable MyLoggerFactory (after downloading log4j-1.2.16.jar and adding it to the classpath).
- ImageServerGame.uc is available in the Binary distribution from Sander. Changed imageserver port back from 80 to 5003.
- P3AT in Soccer2006 doesn't seems to have a camera. Tried AirRobot, but here the code fails on the IMUdata (singleStateSensor called with wrong arguments!?). At the end UDK hangs on request stop ImageServer.
- Commented IMUdata out. Still no images (legacy mode?!). Should also handle the ASTA messages from the AirRobot. Could use the P3AT in the RoboCup2011final.
- Could use the AirRobot with UDK 32bits and ImageServer in legacy mode. Added a request when the connection is idle. Now Sander's ImageServer works with UDK 64bits.
9 September, 2011
- Looked at reference for the SolvePNP algorithm for presentation. Only reference seems to be available at Learning OpenCV around page 391.
- On page 389 is claimed "The algorithm OpenCV uses to solve for the focal lengths and of sets is based on Zhangs
method [Zhang00], but OpenCV uses a different method based on Brown [Brown71] to
solve for the distortion parameters."
[Zhang00] Z. Zhang, A flexible new technique for camera calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence 22 (2000): 1330-1334.
[Brown71] D. C. Brown, Close-range camera calibration, Photogrammetric Engineering 37 (1971): 855866.
- Found a reference to Dementhon-Davis algorithm as a way to solve PNP in OpenCV, but this seems to point to one of the example codes.
- Dementhon-Davis algorithm is cited more recently.
Article claims that the iteration-part can be improved by using Lowe's non-linear method
(D.G. Lowe, Robust model-based motion tracking through the integration of search and estimation. Int. J. Comput. Vision 8 2 (1992), pp. 113122).
2 September, 2011
- Read to today Part II of Steve LaValle's Motion Planning Tutorial: Wild Frontiers. First section is about differential constraints (revelant for Nick's acceleration problems), second part is creating a navigation function (relevant for Martijn's work. Last part is about sensor uncertainty (very tempting to write Martijn's work with the concept of a history information state or I-state. Seems approperiate, because a number of actions and observations leads to success and failure (not a single state and also not a full history feedback)).
26 August, 2011
- Created start_spqrSoccer2006_x64.bat. World starts, but no AR.Drone appears. Maybe I should add the Unreal script and Textures to UsarSim. adrone_slam breaks in _CrtDbgBreak().
- adrone_slam default position is in HedgeMaze. Terrain3d crashes in CArrowRender.
- Created start_compWorldDay1_x64.bat. AR.Drone is now present in world, but doesn't respond to any commands (space and Q):
- Tried to ConnectToImageServer with AirRobot and UsarCommander. No cennection to 127.0.0.1:5003.
- Started 32bits version of altURI. With the 64bits version I get a view at http://127.0.0.1/image.jpg?h=300&w=400&x=100&y=200&q=50. Yet, no image on port 5003. Changed the ImageServer port to 80. Now the connection succeeds, but no images arrive (legacy mode?!).
- Looked at altURI_Cmd code. Mark has two ways to get an image. One is locally, via m_pfGetIplImage = (pfGetIplImage)GetProcAddress( m_altHookDLL, "GetIplImage" );, the other is via a m_SocketObjectImage.SendGetHTTPImage request.
18 August, 2011
- Followed the instructions from Chip OverClock. usb-flashtool is working, but the device is not visible in devmgr, so I can also not install the drivers. Try AddHardwareWizard, seems to work.
- According to MAPGPS, it is also possible to use the AR.Drone as USB Host by uploading the module dwc_otg.ko and loading this module into the AR.Drone kernel with the command insmod ./dwc_otg.ko. Only have to find the module dwc_otg and crosscompile it.
- The same user indicates at the ardrone-flyers forum the layout of the other wires not used for USB. The "USB" port also comes with a serial port (TTL level):
Pin 4: RX
Pin 6: TX
Pin 7: Ground
Also:
Pin 1: VBUS +5V (enabled after module dwc_otg.ko loaded)
Pin 2: +12V power
3 August, 2011
- Installed a custom selection of Visual Studio 2010 (didn't help much; 5.2 Gb instead of 5.5 Gb) to be able to compile Nick's ardrone_slam project.
- Only two directories have a VS10 project. terrain3d compiles, ardrone_slam has missing dependencies: ardrone_tool, opencv2, yaml.
- Added to the adrone_slam project the include and lib of opencv2.1, but it seems that Nick is using opencv2.3. Downloaded superpack 2.3.0. Compiles (after adding both include and include\opencv to include-path).
- Nick has an installation guide!
- Compilation goes well, but linking fails on opencv libraries (while those are in the library-path). Switched configuration from win32 to x64, which solves the opencv issue. All projects have to be reloaded and rebuild. Terrain3d has problems with missing D3DX libraries. Libary path was to x86-branch of to C:\Program Files\Microsoft DirectX SDK (June 2010)\lib. Setting this to x64 helped.
- Finally ardrone_slam works, but fails on DetourCreateProcessWithDll for UT2004. Time to port the AR.Drone to UDK.
18 May, 2011
- It is correct that the 3D inertia tensor only has 6 elements, because the inertia tensor is symmetrical. The off-diagonal elements are set to zero, the diagonal elements are calculeted with a formula equivalent with the one on wikipedia.
11 May, 2011
- Nick has made an abstract of three articles: Steder's 'Visual SLAM for flying vehicles', Durrant's 'Simultaneous localization and mapping: part I' and Davison's 'MonoSLAM: Real-time single camera SLAM'. At least Steder's abstract is usefull.
- Included part of the final report of the Visual Homing project of Maarten Inja & Maarten de Waard.
9 May, 2011
Related Labbooks