LiteIO: An Overview
LiteIO is a sophisticated cloud-native service tailored for managing block devices with exceptional performance. At its core, it utilizes modern storage engines like SPDK (Storage Performance Development Kit) and LVM (Logical Volume Manager) to deliver efficient storage solutions. Designed for Kubernetes, LiteIO thrives in a hyper-converged architecture, enabling seamless block device provisioning throughout a cluster.
Features of LiteIO
-
High Performance: Built on SPDK, LiteIO employs the NVMe-over-Fabric protocol for direct connectivity between computing and storage nodes. This setup, coupled with streamlined protocols and backend I/O polling, ensures performance levels akin to local disks.
-
Cloud-Native Design: With full integration into Kubernetes via a CSI controller and driver, LiteIO offers a cloud-native interface, allowing users to dynamically manage volumes through Persistent Volume Claims (PVCs).
-
Simple Setup: Minimal dependencies, such as Hugepages, make it possible to install LiteIO with a single command line, facilitating rapid deployment.
-
Hyper-Converged Architecture: LiteIO supports hyper-convergence, enabling a single node to simultaneously act as both a frontend and backend. This eliminates the need for a minimum node count to establish a new cluster.
The Motivation Behind LiteIO
Traditionally, applications demanding intense data computation, such as databases, opt for local NVMe disks due to their superior performance and cost benefits over distributed disk services like Amazon's EBS. However, this preference can create resource allocation imbalances—such as a shortage of CPU or memory while storage resources remain abundant. LiteIO tackles this by aiming to harness these fragmented resources while maintaining performance levels comparable to local disks.
Architectural Breakdown
The LiteIO architecture comprises six principal components:
-
Disk-Agent: Deployed on every backend node, the Disk-Agent oversees the management of StoragePools on its node, interfacing with the data engine to create and remove volumes and snapshots. It also relays StoragePool statuses to central control and aggregates volume metrics for Prometheus monitoring.
-
Disk-Controller: This component has a comprehensive view of all StoragePools and Volumes, focusing on allocating requested volumes to appropriate StoragePools.
-
nvmf_tgt: Positioned as the data engine from SPDK, nvmf_tgt provides storage abstraction with functionalities like LVS and LVOL. While optional, it's essential for storage needs extending beyond local disks.
-
nvme-tcp: A kernel module for NVMe over fabrics using TCP transport, this must be present on computing nodes.
-
CSI-Driver: Implementing the Kubernetes CSI, it operates as a DaemonSet on computing nodes, linking to backend storage via nvme-cli tools.
-
CSI-Controller: This central service manages the creation and deletion of Persistent Volumes (PVs).
Collectively, these components foster a scalable and efficient cloud-native block storage solution accommodating diverse storage scenarios.
Quick Deployment
For exploring LiteIO locally in a Kubernetes setup, follow the Quick Start Guide or use Kubeadm for Kubernetes setup.
Performance Insights
LiteIO vs Native Disk & Others
When compared to native disks and alternative solutions like OpenEBS Mayastor, LiteIO demonstrates competitive IOPS and bandwidth performance metrics across various read/write test scenarios. Similarly, against ESSD-PL3, LiteIO exhibits superior performance at higher queue depths, particularly in mixed random read/write operations.
Optimal Use Cases
LiteIO targets specific use cases demanding high IO capabilities, akin to local disk performance, making it ideal for distributed databases or AI training tasks in Kubernetes environments. However, as of now, LiteIO does not support data replication. Thus, if your applications require such features, robust self-managing data replication or a tolerance for potential data loss might be necessary.
Looking Forward
Future enhancements on LiteIO's roadmap include Disk-Agent metric services and SPDK volume replication capabilities.
Join the Community
For further discussions and queries, consider joining LiteIO's WeChat group.
