# 2.1. Optic FlowΒΆ

From Wikipedia:

Optical flow or optic flow is the pattern of apparent motion of objects, surfaces, and edges in a visual scene caused by the relative motion between an observer (an eye or a camera) and the scene.

The basic assumption used in most optic flow algorithms is that when a
point \(\v x\) in the image at time \(t\) moves to point \(\v x + d\v x\) in
the image at time \(t+dt\) its luminance does not change (*the constant
luminance assumption*):

Note that \(d\v x = \v v dt\) where \(\v v\) is the optic flow vector: the velocity vector at point \(\v x\) at time \(t\):

because \(dt\) is assumed to be infinitessimally small we may approximate the above equations in first order as:

or equivalently:

Note that given a video sequence \(f: (\v x, t) \mapsto f(\v x, t)\) we
can approximate the spatial gradient \(\nabla f\) and the temporal
derivative \(f_t\), but then we are left with just *one* equation and
*two* unknowns: the two elements of the optic flow vector \(\v v\).

The impossibility to calculate the optic flow vector in a point is
often called the *aperture problem*.

There are many known methods to circumvent the aperture problem (of
course you cannot *solve* the ‘problem’, in fact to call it a problem
is a bit of a misnomer). In these notes we consider only one: the
Lucas and Kanada method.

The Lucas and Kanada method assumes that around a point \(\v x\) a lot of points in the neighborhood of \(\v x\) have the same optic flow vector \(\v v\). Let \(\v x_1,\ldots,\v x_n\) denote \(n\) points from the local neighborhood of \(\v x\). Then for all \(i=1,\ldots,n\):

or equivalently (using \(\v v = (u\;v)\T\)

Stacking all equations for \(i=1,\ldots,n\) we get:

In case we have more then two points (\(n>2\)) this is overdetermined system and we may solve it using a least squares procedure, leading to:

The matrix \(A\T A\) is a \(2\times2\) matrix:

and is commonly known as the *structure tensor*. Note that matrix \(A\)
and thus also the structure tensor is dependent on the position in the
image (we have taken \(\v x_i\) for \(i=1,\ldots,n\) to be points in the
neighborhood of \(\v x\)).

When implementing this approach we have to consider:

- which neighborhood to choose and
- whether points in the neighborhood should be prioritized (weighted) when calculating the least squared error.

The Lucas-Kanade method to find the optic flow field (i.e. the flow vectors \(\v v\) for all positions in the image) is only valid for small discplacements (low velocity). For large displacements (more then say 2 pixels) the method cannot be used as such. However we could make an image pyramid such that large velocities are found from small displacements in subsampled images.