Vision-language, multimodal models, speech recognition, text-to-speech, audio generation.
BioX-Bridge enables parameter-efficient cross-modal knowledge transfer across biosignals using lightweight prototype-based bridge networks between foundation models.
PAPL aligns discrete diffusion training with planning-based inference via planned ELBO for improved text and protein generation.
LatentFT provides frequency-domain controls for generative music via diffusion autoencoder with latent-space Fourier transform enabling timescale-based manipulation.
Aggregates speech tokens into latent patches for efficient speech-text modeling with cross-modal alignment.
MetaEmbed uses learnable meta tokens with matryoshka training to enable test-time scaling for multimodal retrieval balancing quality and efficiency.
MASK aligns semantic knowledge between images and text using word embeddings as bridges to match out-of-distribution words in unpaired matching.
ScaleCUA scales open-source computer use agents with cross-platform dataset and dual-loop data pipeline.
UALM unified audio language model handles understanding, text-to-audio generation, and multimodal reasoning in single model with UALM-Reason for cross-modal generative reasoning.
Presents VibeVoice for zero-shot expressive long-form multi-speaker podcast generation using next-token diffusion.