contrastors
Explore a comprehensive toolkit for contrastive learning that ensures efficient training with Flash Attention and multi-GPU features. Utilize GradCache to handle large batch sizes and delve into Masked Language Modeling pretraining. The toolkit includes Matryoshka Representation Learning for adaptable embedding sizes and supports CLIP and LiT models, along with Vision Transformers. Tailored for researchers with access to 'nomic-embed-text-v1' dataset and pretrained models, it enables effective training and fine-tuning of vision-text models. Engage with the Nomic Community for additional collaboration and insights.
