Flux-CatVTON Empowered Commercial-Grade Haute Couture Generation: Multimodal Closed-Loop Optimization with DeepSeek-Janus
This repository constitutes a proprietary AI-generated application deployed for evening gown and haute couture fashion presentation scenarios, encompassing ComfyUI workflows and external URL integration. The workflow is designed with multi-modal, multi-model collaborative optimization through a multi-level closed-loop architecture:
- Leveraging LLMs such as DeepSeek-R1 under prompt engineering guidance, achieving hierarchical optimization from inspirational element texts to design concepts, and from scene descriptions to model representations.
- Implementing cascade generation mechanisms through FLUX.1 model ecosystem components - utilizing Fill models (partial inpainting & image extension) with Redux adapters (image variation & style migration), coordinated with In-Context-LoRA contextual generation models to maintain thematic consistency.
- Employing Janus-Pro-7B's cross-modal comprehension integrated with Segment Anything Model (SAM) for secondary iterative refinement of garment details and automatic mask segmentation.
- Incorporating FLUX.1-Turbo-Alpha for multi-stage accelerated diffusion generation throughout the workflow pipeline, combined with CatVTON processing outputs to achieve high-efficiency, high-precision visual transformations.
The current implementation of CatVTON workflow still requires manual mask annotation (e.g., https://huggingface.co/spaces/xiaozaa/catvton-flux-try-on) for target area specification, which introduces inefficiencies and cognitive discontinuity in proprietary applications. To address these limitations, my workflow strategically integrates In-Context-LoRA with Redux modules to achieve fuzzy-consistent generation of primary garment elements. This dual mechanism ensures:
Cross-Image Coherence: Maintains structural and stylistic consistency across product variations through latent space alignment; Replaceable Zone Preservation: Embeds predefined replacement zones during initial generation phase for subsequent CatVTON operations; Automation Optimization: Mitigates region recognition errors in automated masking systems when processing heterogeneous object forms
By implementing this hybrid approach, can effectively decouple the manual mask dependency cycle while maintaining precise control over garment replacement parameters, thereby streamlining the iterative design process from conceptual generation to visual refinement.
This repository will publicly release the complete workflow architecture and prompt engineering framework documentation, including:
- Garment Generation Module
- Model Attire Projection System
- Integrated Production Suite
Privately deployable instances with external access URLs are provided to minimize deployment complexity and training overhead. For implementation details and commercial collaboration inquiries, please contact the project maintainer at lingy4251@163.com.
The code of this warehouse is open source according to Apache-2.0.
Thank you to the all authors of the open source components used in the project.