Active Perception Driven Tactile Modeling of Deformable Objects for Robots

Despite extensive research on tactile sensing, exploiting it to model and interpret the external world remains largely underexplored. We propose a unified framework that leverages vision-based tactile sensors to construct fine-grained object models, enhancing situational understanding. This frame work begins with a multi-task perceptual module that estimates comprehensive physical properties from tactile sequences, including contact geometry, force, material hardness, and dynamic behavior. Secondly, an active exploration module is developed to optimize sampling efficiency for large objects by tracking their boundaries and sampling their interiors. Finally, a visual tactile registration module is combined to map the indentation surface from the tactile sensor to a visual scale with the help of physical attributes, enabling globally consistent modeling even in deformable scenarios. Extensive experiments on diverse flexible and semi-transparent objects demonstrate the effectiveness of the proposed framework. Moreover, we demonstrate a proof-of concept application in underwater object reconstruction, where tactile modeling enhances surface detail and aids recognition under visual degradation. We hope this work provides a foundation for advancing robotic perception and manipulation capabilities, contributing to the pursuit of embodied intelligence in unstructured environments.