FitControler Fit-Aware Virtual Try-On Pipeline
Figure 3 : Overview of FitControler. The person image is first processed by (a) the garment-agnostic preprocessor to extract the mask and dense pose. These are concatenated with the noise map and garment image—along both channel and spatial dimensions as in CatVTON [ 6 ] —before being fed into (b) the fit-aware layout generator to produce a fit-sensitive segmentation map. The layout features are then delivered by (c) the multi-scale fit injector to VTON models via the ControlNet [ 47 ] interface. The layout generator is pre-trained and remains frozen when integrating FitControler into different VTON models, where only the fit injector requires model-specific training.
论文上下文
Paper title: FitControler: Toward Fit-Aware Virtual Try-On Abstract: Realistic virtual try-on (VTON) concerns not only faithful rendering of garment details but also coordination of the style. Prior art typically pursues the former, but neglects a key factor that shapes the holistic style -- garment fit. Garment fit delineates how a garment aligns with the body of a wearer and is a fundamental element in fashion design. In this work, we introduce fit-aware VTON and present FitControler, a learnable plug-in that can seamlessly integrate into modern VTON models to enable customized fit control. To achieve this, we highlight two challenges: i) how to delineate layouts of different fits and ii) how to render the garment that matches the layout. FitControler first features a fit-aware layout generator to redraw the body-garment layout conditioned on a set of delicately processed garment-agnostic representations, and a multi-scale fit injector is then used to deliver layout cues to enable layout-driven VTON. In particular, we build a fit-aware VTON dataset termed Fit4Men, including 13,000 body-garment pairs of different fits, covering both tops and bottoms, and featuring varying camera distances and body poses. Two fit consistency metrics are also introduced to assess the fitness of generations. Extensive experiments show that FitControler can work with various VTON models and achieve accurate fit control. Code and data will be released. Passages referencing this figure: Figure 1 : Illustrations of FitControler generations . Fig. 1 presents the qualitative results of FitControler on CatVTON [ 6 ] , and Fig.
