Interactive modelling in augmented reality with subdivision surfaces and advanced user gesture recognition

The paper discusses an integrated methodology to implement an interactive augmented reality 3D modelling environment with natural interaction, empowered by real-time gesture recognition. The methodology is developed from a geometry-sculpting algorithm based on the use of the subdivision surfaces approach to combine the ease and versatility of interactive modelling even of complex shapes, while maintaining high geometric continuity and smoothness. The interaction with the deformable elements of the geometry’s control cage to be divided uses an optimised version of the Grasp Active Feature/Object Active Feature algorithm developed from hand tracking and gesture recognition based on zero-invasive stereo-infrared techniques. Modelling, combined with an augmented reality environment, allows the modification of geometries having real objects as a reference and, in any case, a general spatial awareness during activities. The methodology was implemented and tested using an advanced mixed-reality headset, the Varjo XR-4, with hi-resolution pass-through and a second-generation Ultraleap for accurate and precise hand tracking.