#Generative Pre-training

PhoGPT is an open-source generative model designed for Vietnamese, comprising a base model and a chat-focused variant with 4 billion parameters. Trained on a vast dataset and fine-tuned with comprehensive conversational data, PhoGPT demonstrates improved performance for language tasks. Access technical details and guidelines for implementation. Note limitations in complex reasoning and safe question handling.

This PyTorch project offers an open-source implementation of Image GPT, focusing on image completion and quantization using generative pretraining techniques. Inspired by the 'Generative Pretraining from Pixels' paper, it supports efficient model training with limited computational resources. Key features include pre-trained model applications, k-means clustering for centroid computation, and generative pretraining strategies. Additional functionalities involve classification fine-tuning and example-based sampling for visuals. Future updates aim to introduce GPU-accelerated k-means, BERT-style pretraining, seamless OpenAI model integration, and efforts to replicate iGPT-S outcomes, thereby enhancing the exploration of datasets such as Fashion-MNIST and CIFAR10.