chat_templates - Diverse Chat Templates for Instruction-Tuned Language Models

Introduction to Chat_Templates Project

The Chat_Templates project is a repository designed to provide structured chat templates and input formats for instruction-tuned large language models (LLMs). These templates facilitate seamless integration with the 'transformers' library's innovative chat_template feature. For developers seeking to enhance or contribute additional templates, the project actively welcomes pull requests.

Recent Updates

The repository is frequently updated to support new models from major organizations, illustrating its commitment to staying current with the rapid developments in LLM technology:

July 2024: Added support for Meta's Llama-3.1 models.
June 2024: Integrated Google's Gemma-2 models.
May 2024: Included Nvidia's ChatQA models.
April 2024: Updated with Microsoft's Phi-3 and Meta's Llama-3 models.
February 2024: Enhanced support for Google's Gemma models and added guidance for using generation_configs.
January 2024: Supported Alibaba's Qwen2 models.

Repository Contents

Chat Templates: These are jinja files containing various chat templates applicable directly in Huggingface tokenizers.
Generation Configs: JSON configurations that control response generation. The stop_token_ids must be passed into the generate method through the eos_token_id argument.

Using the Repository

When utilizing the templates, it's crucial to include at least one user message in the messages. Omitting this may lead to unexpected outputs, as models are not tailored to handle scenarios with only system messages.

Example Usage

Meta-Llama-3-8B-Instruct: This example checks the correct implementation of a jinja file.
Mistral-7B-Instruct-v0.2: Highlights that Mistral models do not support system messages natively, promoting use of the correct chat template to prevent errors.
Vicuna-7b-v1.5: Discusses template formatting variations and how they can influence performance, advocating for the use of version 1.5 for better results.

Supported Models

The repository features a broad spectrum of model support, allowing users to tailor the templates to a variety of leading-edge LLMs:

Meta's Llama-3.1-Instruct and Llama-3-Instruct
Microsoft's Phi-3-Instruct
Google's Gemma and Gemma-2
Nvidia's Llama3-ChatQA
Other prominent models including Vicuna, Falcon, Alpaca, Saiga, and many more.

These entries provide relevant chat templates and generation configurations, detailed references, and sometimes user comments regarding model-specific behaviors.

Conclusion

For developers and researchers working with Hugging Face and similar language model libraries, the Chat_Templates repository offers essential tools and support to implement and optimize chat templates efficiently. Its open-source nature encourages community contributions, fostering an ecosystem of collaboration and continual improvement.