The text-to-speech landscape has been dominated by discrete tokenization approaches for years 鈥?systems that convert text to discrete tokens before generating audio, then decode those tokens back into waveforms. VoxCPM2, developed by the OpenBMB team at Tsinghua University, shatters this paradigm with a tokenizer-free approach that produces more natural, expressive, and faithful speech synthesis while supporting 30 languages and 48kHz studio-quality output.
With over 10,000 GitHub stars, VoxCPM2 is rapidly gaining traction among researchers, developers, and companies seeking next-generation speech synthesis capabilities.
The Tokenizer Problem
Traditional TTS systems work by first converting text into discrete tokens 鈥?essentially breaking language into a fixed vocabulary that the model can work with. This process inevitably loses information. Pronunciation nuances, emotional inflections, regional accents, and subtle prosodic patterns often get flattened in the tokenization step.
VoxCPM2 takes a fundamentally different approach. It uses a diffusion autoregressive architecture that directly generates continuous speech representations, bypassing discrete tokenization entirely. The result is synthesis that preserves the full richness of natural speech.
30 Languages, One Model
VoxCPM2 was trained on over 2 million hours of multilingual speech data. The model directly supports 30 major world languages including Arabic, Chinese (with 9 regional dialects: Sichuan, Cantonese, Wu, Northeastern, Henan, Shaanxi, Shandong, Tianjin, and Southern Min), Danish, Dutch, English, Finnish, French, German, Greek, Hebrew, Hindi, Indonesian, Italian, Japanese, Khmer, Korean, Lao, Malay, Norwegian, Polish, Portuguese, Russian, Spanish, Swahili, Swedish, Tagalog, Thai, Turkish, and Vietnamese.
No language tag is required 鈥?the model automatically infers the language from input text and produces appropriately accented and prosodied output.
Voice Design: Create Voices from Text Descriptions
One of VoxCPM2 most innovative features is Voice Design. Instead of needing reference audio to create a specific voice, you simply describe the voice you want in natural language:
“A young woman, gentle and sweet voice”
“An older man with a deep, authoritative baritone”
“A cheerful teenage voice, slightly breathy”
No reference audio needed. The model generates a completely new voice based on your description, with controllable attributes for gender, age, tone, emotion, and pace.
Controllable Voice Cloning
When you do have reference audio, VoxCPM2 offers sophisticated cloning capabilities. Upload a short clip and the model clones the timbre 鈥?the unique vocal fingerprint that makes a voice recognizable. But unlike simpler cloning systems, VoxCPM2 allows style guidance to steer emotion, pace, and expression while preserving the original timbre.
Want a professional voice clone that sounds slightly more enthusiastic? Or a voice that maintains its character even when reading technical content? Style guidance makes this possible.
The Ultimate Cloning mode goes even further. By providing both reference audio and its exact transcript, the model can continue seamlessly from the reference, faithfully reproducing every vocal nuance 鈥?timbre, rhythm, emotion, and speaking style.
48kHz Studio Quality
VoxCPM2 outputs audio at 48kHz 鈥?the sample rate used in professional studio recordings. The system accepts 16kHz reference audio and uses AudioVAE V2 asymmetric encode/decode design with built-in super-resolution. No external upsampler is needed.
Real-Time Performance
Despite the high quality, VoxCPM2 is remarkably efficient. On an NVIDIA RTX 4090, real-time factor (RTF) reaches approximately 0.3 鈥?meaning one second of audio generates in 0.3 seconds of processing time. With Nano-VLLM acceleration, performance improves to approximately 0.13 RTF, enabling genuine real-time streaming synthesis.
Open Source and Production Ready
VoxCPM2 is released under the Apache-2.0 license, making it freely usable in commercial products. Weights are available on Hugging Face and ModelScope. A simple Python API makes integration straightforward:
pip install voxcpm
Then:
from voxcpm import VoxCPM
model = VoxCPM.from_pretrained("openbmb/VoxCPM2", load_denoiser=False)
wav = model.generate("Your text here.")
sf.write("output.wav", wav, model.tts_model.sample_rate)
Applications and Use Cases
The applications for VoxCPM2 are vast. Content creators can generate narration in multiple languages from a single voice. Game developers can create diverse NPC voice lines procedurally. Accessibility tools can generate personalized, natural-sounding speech for screen readers. Podcast producers can create multi-language versions without re-recording. Call centers can generate IVR prompts in any language with consistent branding voice.
The Broader Impact
VoxCPM2 represents a significant step toward democratizing high-quality speech synthesis. By open-sourcing a model that rivals commercial offerings in quality while adding unique capabilities like voice design and tokenizer-free architecture, the OpenBMB team is lowering barriers for researchers and developers worldwide.
Key Highlights:
- Tokenizer-free diffusion autoregressive architecture
- 30 languages plus 9 Chinese dialects
- Voice Design 鈥?create voices from text descriptions
- Controllable voice cloning with style guidance
- Ultimate cloning for maximum fidelity
- 48kHz studio-quality output with built-in super-resolution
- Real-time performance (0.3 RTF on RTX 4090, 0.13 with Nano-VLLM)
- Apache-2.0 license, commercially usable