Specular Reflection Removal for Endoscopic Data with Applications to Medical Robotics
Open Access Deposited
The appearance of objects is significantly affected by the illumination conditions in the environment. Particularly with objects that have strong reflectivity as they suffer from more dominant specular highlights, causing information loss and discontinuity in the image domain. Many computer vision algorithms are vulnerable to errors in the presence of specular reflections because they violate the image consistency assumption and hinder the performance of many vision tasks, such as object recognition, tracking and surface reconstruction. This is more significant in vision-based medical applications where specularity imposes unendurable challenges. The negative effects of specularity are particulary dominant in medical practices that utilize endoscopic image sequences to offer diagnostic or assess with a medical task. The objective of this dissertation is to eliminate specular reflections in endoscopic data and achieve a good approximation of the damaged regions. To this end, we propose a novel fully-automated solution for endoscopic image restoration. Our approach is designed as a two-step scheme. The first is a fast and effective detection step that is statistically well-motivated and is based on the combination of a set of color variations and gradient information conditions. For the second step, we introduce an inpainting approach that exploits local context represented in graph data structures and emphasizes locality condition. We also introduce a variational solution that allows exploiting spatio-temporal correlations by representing prior data in a low-rank form. We demonstrate that our solution prevents major drawbacks of existing approaches while improving the performance in both detection accuracy and inpainting quality. In a collaborative work, we proceed to showcase the integration of our proposed solution in two open problems in medical robotics. The first being force feedback estimation and the second is cardiac motion prediction, both in the context of Robotic-Assisted Minimally Invasive Surgical systems. One of the main components of the frameworks developed for both applications is a vision-based \gls{3d} surface reconstruction step which was the focus of my contribution on both works. The problem was approached with a variational framework in which we integrated our specularity elimination solution to guarantees information availability and increase robustness in terms of partial occlusions. This integration led to significant improvement in the accuracy of the surface reconstruction task and in consequence, the overall performance of both frameworks on their respective goals.
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.