#cloud storage

Ente provides an open source, end-to-end encrypted cloud platform for secure data storage independent of third-party trust. The service includes Ente Photos, a cross-platform alternative to Apple and Google Photos with private sharing and on-device machine learning, and Ente Auth, a free authenticator app safeguarding data. The platform's security is verified by top cybersecurity firms such as Cure53 and Symbolic Software. Ente Photos offers 5GB of free storage, and there are opportunities for community contributions and support through their platforms.

Amazon EBS Autoscale is a daemon tool tailored for AWS EC2 instances to manage and expand storage automatically by monitoring filesystem mountpoints and integrating new EBS volumes. The project supports BTRFS or LVM with ext4, optimizing for Linux-based systems running upstart or systemd. This repository is deprecated and will be archived on May 17, 2024. Consider alternative AWS solutions like Mountpoint for Amazon S3 or Amazon EFS for ongoing scalable storage needs. Despite its upcoming archival, this repository remains a rich historical resource with automation scripts for storage configuration and installation.

The StreamingDataset offers efficient large dataset training from cloud storage with minimal delay. Designed for distributed multi-node environments, it optimizes data delivery and accuracy while integrating with PyTorch workflows. Supporting diverse data types like images, text, and video, it works with major cloud storage providers for simplified dataset handling. This system features deterministic sampling, fast mid-epoch resumption, high throughput, and balanced convergence, enabling reproducible model training on setups spanning from single GPUs to large clusters, ensuring efficiency and scalability.