Motion Tracking
===============
In this lab exercise you will look at **normalized cross correlation**
for motion tracking. For the theory we refer to the :doc:`lecture
notes `.
Tracking an Object
------------------
In the following video you see a ball following the characteristic
parabolic path.
.. raw:: html
The Python code that follows downloads the video for you but in case
you have a need to do it yourself you can. The video can be
downloaded by right clicking on the video and selecting "Save video as
...".
The following piece of code will play the video in a matplotlib
figure. Note that matplotlib is not meant for this type of
display. The rendering pipeline in matplotlib is quite complex taking
care of different axes in windows, different scalings etc etc. So
don't expect real time performance.
.. literalinclude:: /../python/test/mp4player.py
Reading video's is not a trivial task. There are many different video
formats (encodings) and several file formats (yes these are two
different things...). OpenCV can read mp4 video files but the OpenCV
distributions for Linux do not always have the necessary code
compiled. For this reason i have downloaded and installed
``imageio``. This package needs ``ffmpeg`` or ``libav`` installed. I
have ``ffmpeg`` installed.
Exercise 1
..........
Write a tracker using the normalized cross correlation that is
available in skimage. In skimage it is called ``match_template``, its
documentation can be found `here
`_.
#. Manually select a template in frame 40 enclosing the ball.
#. Use that template to find the ball in subsequent frames.
#. With a marker (the ``scatter`` command in matplotlib) mark the
position of the ball.
The end result should be an image showing the last frame and all
markers drawn showing the patch the ball followed in its flight.
Exercise 2
..........
The template matcher from skimage is also based on the article
J.P. Lewis, "Fast Normalized Cross-Correlation", just like the text in
the lecture notes. Although the resulting expressions are a bit
different (the results the same if all is well...).
In this exercise you have to implement ``match_template`` yourself
based on the formula's in the :doc:`lecture notes
`.
Egomotion: Simulating an Optical Mouse
--------------------------------------
An optical mouse contains a camera that looks at the surface it is
moving on. The resolution of the camera is quite low but because of
that the frame rate is quite high. Therefore when moving the mouse,
frame i and frame i+1 have a significant overlap in area that is
depicted.
The video below shows what a mouse sees when moving over a table
surface (this is a simulation!)
.. raw:: html
Exercise 3
..........
You have to write a program that calculates what the path is that the
mouse travelled. Assume that in the first frame the center of the
image is at position (0,0). Pick a subset of image in frame 0 and use
that as a template to look for in frame 1. The position where you
found that in frame 1 is indicative of the movement of the mouse. Then
you select a template in frame 1 and look in frame 2 where it has
moved to. Etc, etc.
The result is a list of (dx,dy) tuples indicating the movement from
frame i to frame i+1.
Assuming a starting position you can calculate the list of positions
(x,y) for all frames. Make a plot of the path. At every 10th frame you
have to plot the frame index at the associated position (this helps us
in determining whether you have found the right path).
In this exercise you can either use your own cross correlation
template matcher or the one from skimage.