A Geometry Reconstruction And Motion Tracking System Using Multiple Commodity RGB-D Cameras
Open AccessIn this dissertation, we contribute algorithms and frameworks for volumetric geometry and motion reconstruction in an integrated and scalable capture system. The system consists of a sparse set of commodity RGB-D cameras, which allows for fast and accurate scan of objects with multi-view inputs. We propose a robust and efficient tile-based streaming pipeline to fuse captured depth images into a truncated signed distance field (TSDF) for geometry reconstruction, which minimizes memory and calculation overhead. We then propose techniques to enhance the performance of high-quality TSDF rendering by using region index layers and adaptive depth estimation. These methods allow visualizing intermediate results of TSDF fusion at an interactive frame rate to provide instant feedback of the surface reconstruction. After the TSDF fusion, we apply a multi-grid warping method to address RGB image misalignment of both global structures and small details due to the errors in multi-camera registration, optical distortions and imprecise geometry reconstruction. In addition, we apply a global color correction method to reduce color inconsistency among RGB images caused by variations of camera settings. Moreover, when a static canonical model is obtained, we utilize our template-based non-rigid registration algorithm to address the misalignment problems in the frame-to-frame motion tracking. We analyze the deformation in the local coordinates of neighboring nodes and use this differential representation to formulate the regularization term for the deformation field in our non-rigid registration. The local coordinate regularization varies for each pair of neighboring nodes based on the tracking status of the surface regions. We propose our tracking strategies for different surface regions to minimize misalignment and reduce error accumulation. This method can thus preserve local geometric features and prevent undesirable distortions. We also introduce a geodesic-based correspondence estimation algorithm to align surfaces with large displacements and improve the convergence of the non-rigid registration during the motion tracking. Finally, we demonstrate the effectiveness of our proposed methods with detailed experiments.
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.