Creative ChatGLM WebUI
Welcome to the world of Creative ChatGLM, where you can engage in generating creative content using unique "Revision" and "Continuation" functionalities!
Features
- Continuation: Initiate content creation by providing a starting sentence, and let ChatGLM expand upon it with more.
- Revision: Modify the last sentence of ChatGLM's response to suit your needs better.
Environment Setup
There are several ways to set up the Creative ChatGLM environment depending on user needs and technical expertise.
Offline Packages
This method is ideal for:
- Non-developers, who do not need to write any code
- People with no experience in Python or setting up environments
- Those with poor internet connectivity, resulting in slow configurations and downloads
Here are some offline packages available:
| Package Name | Size | Details |
|---|---|---|
| Small VRAM Package | 5.3 GB | Requires 8GB of VRAM, suitable for ChatGLM2-6b-int4 |
| Large VRAM Package | 11.5 GB | Requires 16GB of VRAM, suitable for ChatGLM3-6b |
| Long Text Package | 11.5 GB | Also requires 16GB of VRAM, optimized for longer texts up to 128k |
| GLM4 Package | 16.98GB | VRAM requirement of 10GB, supports GLM-4-9B with INT4 loading |
| Environment Package | 2.6 GB | Excludes weight; weights download automatically post-setup for ChatGLM2-6B-int4 |
Download the desired package, extract it, and start the service by running start_offline.bat. To use the API, execute start_offline_api.bat.
Virtual Environment
This setup option is recommended for users who have Python installed and wish to isolate their environment from the system's Python installation.
- Start with the
setup_venv.batscript to set up the environment. - Launch the service using the
start_venv.batscript.
Python Development Environment
For developers working with Python, follow the instructions provided in the official ChatGLM-6B repository. Once set up, you can run app.py to start using the service or develop further using your preferred Integrated Development Environment (IDE).
Usage Guide
Continuation Feature
A user can begin a dialogue prompt like, "I am" and ChatGLM will creatively complete it, e.g., "I am 21 years old."
Revision Feature
Users can modify ChatGLM's response. For instance, turning "I am a ChatGLM-6B AI assistant" into "I am Yang Kaixin," allows further inquiries like "How old are you?" to elicit a modified response, e.g., "I am 15 years old."
Use Cases
- Reinitiating Dialogue: By repeating the input with different modifications, users can achieve satisfying conversational results.
Underlying Principles
The approach leverages modification of inference functions rather than training or altering released model weights:
- Continuation: Sets the user's input as part of the model's own output, allowing continuous inference using
stream_chat_continue. - Revision: Adjusts the input to modify the model’s output, skipping inference functions.
Offline Package Creation
To make an offline package:
- Prepare Python: Download the Windows embeddable package from the Python website and extract it to
./system/pythondirectory. Delete any.pthfiles to avoid dependency installation issues. - Prepare
get-pip.py: Download from the official source and save it in the same directory. - Install Dependencies: Use
setup_offline.batto set up dependencies. - Download Models: Use the provided scripts to download models, which are stored in both
~/.cacheand./models. You may need to relocate the models for offline use.
Adjust app.py to specify your desired model, then run start_offline.bat to test and ensure the service is operational.
License and Agreement
The code for the Creative ChatGLM project is open-sourced under the Apache-2.0 License. However, utilizing the model weights requires adherence to the Model License.