TangiAR: Markerless Tangible Input for Immersive Augmented Reality with Everyday Objects
Published in The 31st ACM Symposium on Virtual Reality Software and Technology (VRST ’25), 2025
Recommended citation: Neil Xu Fan, Xincheng Huang, and Robert Xiao. 2025. TangiAR: Markerless Tangible Input for Immersive Augmented Reality with Everyday Objects. In 31st ACM Symposium on Virtual Reality Software and Technology (VRST ’25), November 12–14, 2025, Montreal, QC, Canada. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3756884.3766028
Abstract: Tangible interactions with everyday objects have been shown to be fast, accurate, and natural, and have shown promise when combined with immersive augmented reality. However, implementing tangible controls presents considerable challenges. Previous works in the field either rely on additional tracking markers on objects, inadvertently shifting the difficulty to users, or are too computationally demanding for real-time operation on a head-mounted display (HMD). We propose TangiAR, a tangible control system which tracks everyday objects without the need for fiducial trackers, enabling them as passive controllers and virtual proxies in AR applications. TangiAR additionally enables hand and finger proximity interactions with tangibles, further expanding the interaction space. TangiAR can run on an unmodified Microsoft HoloLens 2, making it immediately practical. We evaluated the performance of TangiAR through a technical evaluation, including occlusion robustness and tracking accuracy tests, and a user study which examined the usability of our markerless object tracking system in various AR interactions.
Paper: Coming Soon :-)
Demo: