Project Introduction: MITSUHA
About The Project
MITSUHA is an innovative project that introduces a virtual assistant who interacts with users through voice communication, akin to a digital helper. This assistant is designed to revolutionize how people interact with technology by offering a more humanized interface. The virtual assistant, referred to as a "waifu," can engage in conversations, perform tasks, and manage devices within a smart home setup. Its unique functions include:
- Engaging in two-way communication using a microphone.
- Utilizing both short-term and long-term memory to improve interaction.
- Opening and managing applications.
- Operating at a higher cognitive level than the average user, ensuring smart predictions and responses.
- Being multilingual, with fluency in English, Japanese, Korean, and Chinese.
- Controlling smart home devices seamlessly, provided Tuya is set up correctly.
Built With
The project harnesses several cutting-edge technologies and platforms to ensure smooth and efficient functioning:
- Python: A primary programming language used for scripting and automation.
- Llama-cpp-python: Allows managing complex computations efficiently.
- Whisper: Provides speech-to-text capabilities.
- SpeechRecognition and PocketSphinx: Enable recognition of spoken words.
- VITS TTS: Facilitates text-to-speech functions.
- HyperDB: Acts as a vector database, aiding in conversational context retrieval.
- Tuya Cloud IoT: Manages and controls smart devices within the network.
Getting Started
To get started with the MITSUHA project, users need to follow several prerequisites to ensure a seamless setup:
- Install necessary software such as Python 3.10.11, GIT, CUDA if applicable, Visual Studio, and VTube Studio.
- Set up the environment for audio management using VB Cable Audio Drivers.
- (Optional) Setup Tuya cloud project for smart device integration.
It’s important to note that the project is still undergoing changes, and installing it might result in errors.
Roadmap
The MITSUHA project has ambitious future developments planned to enhance user experience further. These include:
- Integration with virtual, augmented, and mixed reality environments.
- Development of an animatronic body for more tangible interaction.
- Supporting additional languages such as Spanish and Indonesian.
- Creating a mobile version and compiling it into a single executable file.
- Improving ease of setup and localization efforts.
License
MITSUHA is distributed under the GNU General Public License v3.0, ensuring it's open-source and accessible for collaborative improvement.
Contact and Socials
Individuals interested in learning more, contributing, or providing feedback can contact the project team via email at [email protected]. They can also engage with the community on their Discord server or follow updates on YouTube.
Acknowledgments
The project benefits from contributions and support from several open-source initiatives and communities, including AI Waifu Vtuber and SchizoDev, underscoring the collaborative spirit behind MITSUHA.
This guide provides a comprehensive overview of MITSUHA’s capabilities and future potential, making it accessible and exciting for users and developers alike.