#Transformer-based models

Intel Extension for Transformers improves Transformer model efficiency across platforms such as Intel Gaudi2, CPU, and GPU. Offering seamless Hugging Face API integration for model compression and software optimizations, it enhances models like GPT-J, BLOOM, and T5 for faster inference. The toolkit includes a flexible chatbot framework and expands low-bit inference capabilities, offering robust support for developers working with GenAI/LLM technologies.

A solution for improving Transformer-based models with support for Baichuan, LLaMA, and other large language models through int8 quantization. Suitable for large models on a single GPU, it enables efficient processing of multi-modal and NLP tasks with enhanced performance via CUDA kernel optimization and innovative algorithms, and is easily integrable into Transformers and Fairseq.