Toward multi-category garments virtual try-on method by coarse to fine TPS deformation

Virtual try-on facilitates users to evaluate the wearing effect of garments on their bodies. As online clothing shopping develops, the category and style of garments constantly enrich. It is an issue to warp multi-category garments as the user shape without three-dimensional (3d) garment models. To tackle this issue, we propose a novel virtual try-on method toward multi-category garments by coarse to fine thin plate spline (TPS) deformation. To embody the user shape, 3d human body model is reconstructed with the garment pose. With the orientation and width classification criteria, the human body part mask is projected from 3d human body model, then it is adapted to the category and feature of garments. The spatial gradients with various scales are generated by comparing the shape difference between the garment mask and the human body part mask. To eliminate this shape difference, the coarse to fine TPS deformation is utilized to warp garment images from global to local. Ultimately, the warped garment images are worn on the virtual human body to preview the try-on effect. Experiments demonstrated that our method is robust to different human body shapes with different garments. Compared with state-of-the-art VITON methods, our method is superior in preserving the texture details and overall style in the virtual try-on for multi-category garments. The code is available at https://github.com/NerdFNY/MCG-VITON.