Generative Human Geometry Distribution
Xiangjun Tang, Biao Zhang, Peter Wonka
We introduce the first method that integrates geometry distributions into generative modeling.
Abstract
Realistic human geometry generation is an important yet challenging task, requiring both the preservation of fine clothing details and the accurate modeling of clothing-body interactions. To tackle this challenge, we build upon Geometry distributions—a recently proposed representation that can model a single human geometry with high fidelity using a flow matching model. However, extending a single-geometry distribution to a dataset is non-trivial and inefficient for large-scale learning. To address this, we propose a new geometry distribution model by two key techniques: (1) encoding distributions as 2D feature maps rather than network parameters, and (2) using SMPL models as the domain instead of Gaussian and refining the associated flow velocity field. We then design a generative framework adopting a two-staged training paradigm analogous to state-of-the-art image and 3D generative models. In the first stage, we compress geometry distributions into a latent space using a diffusion flow model; the second stage trains another flow model on this latent space. We validate our approach on two key tasks: pose-conditioned random avatar generation and avatar-consistent novel pose synthesis. Experimental results demonstrate that our method outperforms existing state-of-the-art methods, achieving a 57% improvement in geometry quality.
Introduces distribution-over-distribution model combining geometry distributions with two-stage flow matching for human 3D generation.
- Novel representation encoding geometry distributions as 2D feature maps rather than network parameters
- Two-stage training paradigm: first compresses distributions to latent space, then learns on latent space
- Achieves 57% improvement in geometry quality on pose-conditioned generation and novel pose synthesis
- Flow matching
- Diffusion models
- SMPL models
- Geometry distributions
- Latent space learning
Non-uniform sampling on target geometry surface due to varying number of target points per SMPL point
from the paperConstrained by diversity of training datasets; model generalizes well to various body types but cannot generate clothing styles absent from training data
from the paperUse of UV maps can cause seam artifacts in randomly generated results due to discontinuity segmentation
from the paper
Authors did not state explicit future directions.
Author keywords
- 3D Generation
- Human Generation
- Geometry Encoding
Related orals
Universal Inverse Distillation for Matching Models with Real-Data Supervision (No GANs)
RealUID provides universal distillation for matching models without GANs, incorporating real data into one-step generator training.
GLASS Flows: Efficient Inference for Reward Alignment of Flow and Diffusion Models
GLASS Flows samples Markov transitions via inner flow matching models to improve inference-time reward alignment in flow and diffusion models.
Neon: Negative Extrapolation From Self-Training Improves Image Generation
Neon inverts model degradation from self-training by extrapolating away from it, improving generative models with minimal compute.
Cross-Domain Lossy Compression via Rate- and Classification-Constrained Optimal Transport
Cross-domain lossy compression unifies rate and classification constraints via optimal transport framework.
NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scale
NextStep-1 achieves state-of-the-art autoregressive text-to-image generation by modeling continuous image tokens with lightweight flow matching instead of diffusion.