1.4. Projectivities

When making images of the same 3D scene not all pairs of images can be stitched together. There are two experimental setups that lead to a set of images that can be stitched together:

• in case the scene is a plane in 3D (e.g. a large painting on the wall, or a large facade of a building), or

• in case the camera doesn’t change its position (the optical center doesn’t move) and only the orientation is changed (this is for instance—approximately—the case when you stand in one place and only rotate your camera when making pictures of the same scene.

1.4.1. A plane in 3D

Consider two different views of the same plane in 3D. Without loss of generality we assume that the plane we are looking at is the \(Z=0\) plane in the world coordinate system.

Fig. 1.32 Observing a plane in 3D from two positions. The left camera has its origin in \(\v O\) and the right camera in \(\v O'\). The optical axes of both camera’s are drawn with dashed lines. The 3D point \(\hv X\) is projected in point \(\hv x\) in the left camera and in point \(\hv x'\) in the right camera.

The first camera looks at a point \(\hv X\) and finds its projection in the 2D point \(\hv x \sim P \hv X\). If \(\hv X\) is on the plane \(Z=0\) we may write:

\[\begin{split}\hv x \sim \matvec{ccc}{ p_{11} & p_{12} & p_{14}\\ p_{21} & p_{22} & p_{24}\\ p_{31} & p_{32} & p_{34}} \matvec{c}{X\\Y\\1} \sim H \matvec{c}{X\\Y\\1}\end{split}\]

The second camera looking at the same point finds its projection in \(\hv x'\sim P'\hv X\) and we have

\[\begin{split}\hv x' \sim H' \matvec{c}{X\\Y\\1}\end{split}\]

Combining these two expressions we find:

\[\hv x' \sim H' H\inv \hv x\]

Showing that there is indeed a \(3\times3\) projectivity matrix \(H'H\inv\) that relates all points in both images.

1.4.2. The rotating camera

Fig. 1.33 Rotated Camera’s at the same point.

Without loss of generality we assume the first camera is perfectly aligned with the world coordinate system, i.e. the first camera has translation vector zero and no rotation leading to:

\[\hv x \sim K\matvec{cc}{I&\v 0}\hv X\]

The second camera is only rotated with respect to the first camera and while looking at the same point \(\hv X\) this camera finds:

\[\hv x' \sim K'\matvec{cc}{R\T&\v 0}\hv X\]

Please not the we take a different internal matrix for the second cam. This allows for zooming the camera! Due to the zero last columns in both camera matrices we may reduce both the above equations:

\[\begin{split}\hv x &\sim K\v X\\ \hv x' &\sim K'R\T \v X\end{split}\]

Note that now we use \(\v X\) instead of \(\hv X\). Combining the two equations and eliminating the common \(\v X\) (but unknown) in both equations we arrive at:

\[\hv x' \sim K' R\T K\inv \hv x\]

Again showing that there is a projectivity that relates all points in both images.

Fig. 1.34 Rotating Camera. For a real camera the optical center is somewhere in the camera (+ lens) and not exactly known. To allow for pan and tilt rotation around that point you need special equipment (a pano head).