CharacterGLM-6B Project Overview
Introduction
CharacterGLM-6B is a cutting-edge conversational pre-trained model developed by Lingxin Intelligence and Tsinghua University's CoAI Lab. This model belongs to the ChatGLM2 series, an open-source model designed to maintain a balance between ease of deployment and conversational fluidity. CharacterGLM-6B aims to enhance the interactive experience by focusing on human-like characteristics that make AI interactions engaging and realistic.
Key Features
AI Characters Come to Life
To make AI characters more lifelike, CharacterGLM-6B emphasizes traits that define human textual expression. This involves focusing on both the 'attributes' and 'behaviors' of characters.
-
Attributes: These include seven categories that shape the content of language expression—identity, interests, viewpoints, experiences, achievements, social relations, and others.
-
Behaviors: These are shaped by dynamic elements like linguistic features, emotional expression, and interaction styles. For instance, older adults might prefer formal language, whereas teenagers might favor internet slang. CharacterGLM incorporates linguistic traits and personality to design character behaviors.
The Liveness Test for AI Characters
To exhibit human-like qualities genuinely, an AI character's expressiveness can be judged based on:
-
Consistency: The character should consistently exhibit stable attributes and behaviors throughout interactions to gain user satisfaction and trust.
-
Human-likeness: Characters need to interact naturally akin to human-to-human interaction, which enhances acceptance and facilitates more natural and engaging dialogues.
-
Engagement: The ability to captivate users and promote interaction is crucial. Making conversations interesting enough to carry on is a testament to the model's overall performance.
Methodology
Following these design principles, CharacterGLM developers collected role descriptions covering attributes and behaviors and crowdsourced a large, high-quality dialogue dataset. These descriptions were converted into natural language prompts, and models from 6B to 66B parameters were fine-tuned to create CharacterGLM. The training included online interaction data to ensure self-improving iterations of model performance.
Experimental Evaluation
Evaluation Criteria
The evaluations included metrics like:
- Quality: Measuring response fluency and context coherence.
- Safety: Ensuring that responses adhere to ethical standards.
- Correctness: Checking for hallucinations.
- Overall: General quality assessment.
Evaluation Setup
CharacterGLM was compared with ten Chinese-friendly mainstream LLMs. Ten annotators created two characters per model and conducted over 20 interaction rounds. Models were rated across six sub-dimensions and an overall rating ranging from 1 to 5, the higher the score, the better the performance.
Results
CharacterGLM demonstrated superior overall response quality compared to baseline models, with a significant advantage in many aspects though it did show lesser proactivity. However, its capability to develop narratives plays a crucial role in engaging users.
Practical Implementation
Installation
To get started:
-
Clone the repository:
git clone https://github.com/thu-coai/CharacterGLM-6B cd CharacterGLM-6B -
Install dependencies:
pip install -r requirements.txt -
Ensure using the correct versions of required libraries for optimal performance.
Usage
-
Web Demo: Start a web demo using Streamlit:
cd basic_demo streamlit run web_demo_streamlit.py -
Command Line Demo: Interact via terminal:
python basic_demo/cli_demo.py
Future Prospects
Currently, there is no support for model fine-tuning scripts, but they are in the plans to be made available soon.
Citing CharacterGLM
If CharacterGLM is useful for your work, please cite the following:
@article{zhou2023characterglm,
title={CharacterGLM: Customizing Chinese Conversational AI Characters with Large Language Models},
author={Zhou, Jinfeng and Chen, Zhuang and Wan, Dazhen and Wen, Bosi and Song, Yi and Yu, Jifan and Huang, Yongkang and Peng, Libiao and Yang, Jiaming and Xiao, Xiyao and others},
journal={arXiv preprint arXiv:2311.16832},
year={2023}
}
This structured introduction outlines the various elements that make CharacterGLM-6B a promising development in conversational AI, with a focus on realism, consistency, and engagement in human-like dialogue.