Michael Sülzer

Student in Group Multimedia Information Processing

The topic of my diploma thesis is about the estimation of the global motion parameters of a moving camera in the context of generating panoramic images. After a definition of the main problems and limitations, a comparison of the actual methods  Manifold Mosaicing [PH97], Rotational Mosaicing with local and global alignment[SS97] and Topology Inference with local to global alignment [H97] is made. The second method of Shum and Szeliski is chosen for deeper analysis and implementation since it can handle all the requirements such as pairwise consistent alignment, global consistent alignment and elimination of ghosting artefacts.

In the step of pairwise alignment, a general alignment framework is used to register two subsequent images of a sequence. This framework consists of the two general techniques Differential [LK81] and Hierarchical [BAHH92] model based image registration. Two images are registered over an image pyramid beginning on the coarest level. In every level a gradient descent algorithm solves iterativly least-squares based normal equations to bring the two images into closer registration. The registration of a level is used as initialization of the next level. The last level gives the consistent pairwise alignment. Some methods are presented to increase speed and robustness of the pairwise alignment. Since the gradient descent algorithm can be stuck in local minima, two methods are suggested  to get a good initial motion estimate and to transfer a local or plane motion estimation to a model based estimation, which is close enough to the real motion and prevents the algorithm to stuck. These methods are Block Machting and Phase Correlation. Speedup techniques are patch-based alignment, a quadratic intensity model based on the standard linear model and gradient masks. The use of all methods together leads to much faster and more robust registration.

After the consistent pairwise alignment there may be accumulated local misregistrations, especially in longer sequences, which leads to blurring or ghosting artetacts or even gaps in the panorama. Simultaneous bundle block adjustment known from photogrammety is used two make the local aligned images global consistent. The images are divided into blocks and the block centers are used as pseudo feature points. Based on the pairwise alignment, for every block in every image the corresponding points in all overlapping images are calculated with block matching. These corresponding points lead to bundles of ray directions, which have different direction. The bundle block adjustment adjusts all ray directions so that they converge. The algorithm is very similiar to the gradient descent algorithm in the pariwise aligment. The result is a global consistent aligned image sequence.

But there can still remain local misregistration produced from motion parallax or lens distortion. The local motion is calculated in same way as is the global aligment and is interpolated with a tent function directly into the image. The result is a locally warped image, which is consists to all other images. This step is repeated with descending block sizes to reduce local misregistrations.

The main part of the work was spend on pairwise alignment and global alignment.

Here is a free-hand example of an image sequence with 8 images taken with a little consumer camera
 

1

2

3

...

8

Heres is a panorama of a synthetic scene  with 40 images on a fundus image with rotation

1

2

3

...

40

Here is a panoramic image of our working room

Recerences
 

[BAHH92]	J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani. Hierarchical model-based motion estimation. In Lecture Notes of Computer Science ECCV92, Springer Verlag, pages 237-252, 1992.
[LK81]	B.D. Lucas and L. Kanade. An iterative image registration technique with application to stereo vision. In Seventh International Joint Conference on Artificial Intelligence (IJCAI-81), pages 674-679, 1981.
[SHK98]	H.S. Sawhney, S. Hsu, and R. Kumar. Robust video mosaicing through topology inference and local to global alignment. In Lecture Notes of Computer Science ECCV98, Springer Verlag,  1998.
[PH97]	S. Peleg and J. Herman. Panoramic mosaics by manifold projection. CVPR97, pages 338-343, 1997.
[SS97]	H.Y. Shum and R. Szeliski. Panoramic image mosaics. Technical report, Microsoft Research, 1997.

Last modified 18. November 2000