Projectivities
==============

When making images of the same 3D scene not all pairs of images can be
stitched together. There are two experimental setups that lead to a
set of images that can be stitched together:

* in case the scene is a plane in 3D (e.g. a large painting on the
  wall, or a large facade of a building), or

* in case the camera doesn't change its position (the optical center
  doesn't move) and only the orientation is changed (this is for
  instance---approximately---the case when you stand in one place and
  only rotate your camera when making pictures of the same scene.

A plane in 3D
.............

Consider two different views of the same plane in 3D. Without loss of
generality we assume that the plane we are looking at is the $Z=0$
plane in the world coordinate system.

The first camera looks at a point $\hv X$ and finds its projection in
the 2D point $\hv x \sim P \hv X$. If $\hv X$ is on the plane $Z=0$ we
may write:

.. math::
   \hv x \sim \matvec{ccc}{
     p_{11} & p_{12} & p_{14}\\ 
     p_{21} & p_{22} & p_{24}\\
     p_{31} & p_{32} & p_{34}}
     \matvec{c}{X\\Y\\1} \sim  H \matvec{c}{X\\Y\\1}

The second camera looking at the same point finds its projection in
$\hv x'\sim P'\hv X$ and we have

.. math::
   \hv x' \sim H' \matvec{c}{X\\Y\\1}

Combining these two expressions we find:

.. math::
   \hv x' \sim H' H\inv \hv x

Showing that there is indeed a $3\times3$ projectivity matrix
$H'H\inv$ that relates all points in both images.


The rotating camera
...................

Without loss of generality we assume the first camera is perfectly
aligned with the world coordinate system, i.e. the first camera has
translation vector zero and no rotation leading to:

.. math::
   \hv x \sim K\matvec{cc}{I&\v 0}\hv X

The second camera is only rotated with respect to the first camera and
while looking at the same point $\hv X$ this camera finds:

.. math::
   \hv x' \sim K'\matvec{cc}{R\T&\v 0}\hv X

Please not the we take a different internal matrix for the second
cam. This allows for zooming the camera!  Due to the zero last columns
in both camera matrices we may reduce both the above equations:

.. math::
   \hv x &\sim K\v X\\
   \hv x' &\sim K'R\T \v X

Note that now we use $\v X$ instead of $\hv X$. Combining the two
equations and eliminating the common $\v X$ (but unknown) in both
equations we arrive at:

.. math::
   \hv x' \sim K' R\T K\inv \hv x

Again showing that there is a projectivity that relates all points in
both images.