AI voice lines to lip-sync: TTS, then mouth animation
Generate the voice line first, then drive the mouth from the audio. Which lip-sync tool fits depends on your art style and engine.
Use a plan that allows commercial use
- For example, ElevenLabs' free plan is for non-commercial use only, and paid plans allow commercial use. source
Generate and export the audio
- The ElevenLabs text-to-speech API defaults to MP3 (mp3_44100_128) and also offers PCM and other output formats. source
2D characters: Rhubarb Lip Sync
- Rhubarb Lip Sync analyses a recording and outputs mouth cues as TSV, XML, JSON or Moho .dat. It uses six basic mouth shapes (A–F) plus three optional extended shapes (G, H, X). source
Unity 3D characters: uLipSync
- uLipSync (MIT licence) computes MFCCs from the playing AudioSource and drives SkinnedMeshRenderer blend shapes. It works in real time or as pre-baked data that you can convert to an AnimationClip. source
Unreal MetaHumans: audio-driven animation
- MetaHuman Animator's Audio Driven Animation turns a SoundWave asset into facial animation, either in real time or offline. The offline mode has extra controls for head movement, blinks and mood. source
Other option: NVIDIA Audio2Face-3D
- NVIDIA publishes Audio2Face-3D models and tools on GitHub. Check each repo's own licence before you use it. source
Checked 2026-10-17. Tools and docs change, so follow the linked sources. Not affiliated with any vendor.
