Projectivities
==============

When making images of the same 3D scene not all pairs of images can be
stitched together. There are two experimental setups that lead to a
set of images that can be stitched together:

* in case the scene is a plane in 3D (e.g. a large painting on the
  wall, or a large facade of a building), or

* in case the camera doesn't change its position (the optical center
  doesn't move) and only the orientation is changed (this is for
  instance---approximately---the case when you stand in one place and
  only rotate your camera when making pictures of the same scene.

A plane in 3D
.............

Consider two different views of the same plane in 3D. Without loss of
generality we assume that the plane we are looking at is the $Z=0$
plane in the world coordinate system.

.. figure:: /images/3dplane_projectivity.svg
   :align: right
   :figwidth: 50%

   **Observing a plane in 3D from two positions.** The left camera has
   its origin in $\v O$ and the right camera in $\v O'$. The optical
   axes of both camera's are drawn with dashed lines. The 3D point
   $\hv X$ is projected in point $\hv x$ in the left camera and in
   point $\hv x'$ in the right camera.


The first camera looks at a point $\hv X$ and finds its projection in
the 2D point $\hv x \sim P \hv X$. If $\hv X$ is on the plane $Z=0$ we
may write:

.. math::
   \hv x \sim \matvec{ccc}{
     p_{11} & p_{12} & p_{14}\\ 
     p_{21} & p_{22} & p_{24}\\
     p_{31} & p_{32} & p_{34}}
     \matvec{c}{X\\Y\\1} \sim  H \matvec{c}{X\\Y\\1}

The second camera looking at the same point finds its projection in
$\hv x'\sim P'\hv X$ and we have

.. math::
   \hv x' \sim H' \matvec{c}{X\\Y\\1}

Combining these two expressions we find:

.. math::
   \hv x' \sim H' H\inv \hv x

Showing that there is indeed a $3\times3$ projectivity matrix
$H'H\inv$ that relates all points in both images.


The rotating camera
...................


.. figure:: /images/rotatedcam_projectivity.svg
   :align: right
   :figwidth: 30%

   **Rotated Camera's at the same point.**
	      
Without loss of generality we assume the first camera is perfectly
aligned with the world coordinate system, i.e. the first camera has
translation vector zero and no rotation leading to:

.. math::
   \hv x \sim K\matvec{cc}{I&\v 0}\hv X

The second camera is only rotated with respect to the first camera and
while looking at the same point $\hv X$ this camera finds:

.. math::
   \hv x' \sim K'\matvec{cc}{R\T&\v 0}\hv X

Please not the we take a different internal matrix for the second
cam. This allows for zooming the camera!  Due to the zero last columns
in both camera matrices we may reduce both the above equations:

.. math::
   \hv x &\sim K\v X\\
   \hv x' &\sim K'R\T \v X

Note that now we use $\v X$ instead of $\hv X$. Combining the two
equations and eliminating the common $\v X$ (but unknown) in both
equations we arrive at:

.. math::
   \hv x' \sim K' R\T K\inv \hv x

Again showing that there is a projectivity that relates all points in
both images.


.. figure:: /images/pano_head.jpg
   :align: center
   :width: 50%

   **Rotating Camera.** For a real camera the optical center is
   somewhere in the camera (+ lens) and not exactly known. To allow
   for pan and tilt rotation around that point you need special
   equipment (a **pano head**).