Cross-domain assessment of deep learning-based alignment solutions for real-time 3D reconstruction

Interesting deep learning solutions have been proposed recently to address different tasks along the 3D view alignment pipeline. However, a direct comparison among these technologies is still lacking, while their (possibly combined) potentials have yet to be extensively tested. This is especially true in cases where the focus is directed on diversified data and/or specific application requirements, such as the ones emerging in real-time 3D object reconstruction scenarios. This work is a first contribution in this direction since we perform an independent and extended comparison of the main deep learning-driven 3D view alignment solutions. We consider two relevant data types: data coming from commodity 3D sensors targeting indoor reconstruction applications, and denser data coming from a handheld 3D optical scanner, typically used for small-scale object reconstruction. While for the first scenario we refer to existing datasets, for the second setup we work on a new benchmarking dataset, namely DenseMatch. We run performance tests and extended comparisons, with different system configurations including model refinements, and we found solid evidence that the generalizability performance of deep learning systems for 3D alignment is critically linked to data features. Finally, we design and test the first integration of deep learning solutions into a baseline method for real-time 3D reconstruction, clearly demonstrating improved effectiveness in addressing and solving typical tracking and scan interruption issues arising in these demanding scenarios.