Release 1.4.9 Oct 2023
Eng | 繁中 | 日本語 | 한국어 | 简中 |
AI Powered Voice Chat With Any Character
Overview
Now you can talk to your favourite characters face to face!
- You can choose to use ChatGPT or any LLM that you can run with OobaBooga WebUI.
- Completely local option available. If you want it totally free and private, you can run your own LLM and use it to power the chat.
- Built-in multilingo voice recognition.
- Built-in multilingo TTS engine. We’ve included a English model with over 900 different voices. Other language voices availble for download.
- Built-in native lipsync.
- Spacitial audio, you can tell where the speaker is without even looking.
- Easily customize character personality and description.
- You can save and replay chat messages.
- You can also use characters (in JSON format) from popular AI role play programs like TavernAI.
Current limitations
- Voice engine is Windows only. There are no voice option on Mac, Android and Quest for now.
- Running LLM along with DanceXR does require powerful GPU and large VARM. It is recommended to use remote setup.
Documentation
Useful links
- OpenAI https://platform.openai.com/
The famous ChatGPT. To use their model for AI chat, you’ll need to register and get an API key from them. Then enter that key in the Chat Config in DanceXR. The service is not free but is cheap enough. Don’t be confused with ChatGPT Plus though, they are different services. You’ll still be charged by usage even when you have a ChatGPT plus subscription.
- OobaBooga WebUI: https://github.com/oobabooga/text-generation-webui
This is a program that allows you to run LLMs (Large Language Model) locally.
- After installing WebUI, you need to download a model to run. Here are 2 uncensored models that we recommend.
https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GPTQ (7b, easier to run) https://huggingface.co/TheBloke/Nous-Hermes-Llama2-GPTQ (13b, smarter)
- Runpod: https://www.runpod.io/
This is a service that allows you to rent a GPU and run AI models. If you don’t want the hassle of installing your own WebUI, or don’t have powerful hardware to run LLMs, this is one of the choices.
- Voice download https://rhasspy.github.io/piper-samples/
The TTS engine we are using is called Piper. The above link has all the voices currently available. If you are interested you can also train your own voice models.
Model improvements
- Skirt and hair physics now have more options for you to finetune the physics effect.
- Refined collision control for boobs and softbody physics.
- You can now animate the transition effect of the Dressing System with Auto Update.
Improved Pico support
- Color passthrough available
- Bug fixes
Other fixes & improvements
- Improved FPS when there are many actors in the scene.