Introduction to dstack
dstack is a user-friendly alternative to Kubernetes and Slurm, designed specifically for artificial intelligence (AI) applications. It simplifies the process of managing and running AI workloads in containers both on cloud platforms and on local servers. By doing so, dstack accelerates the development, training, and deployment of AI models. It is compatible with various cloud providers as well as on-premises servers, making it a versatile tool for AI development.
Key Features
Accelerator Support
One of the standout features of dstack is its built-in support for a variety of hardware accelerators including NVIDIA GPUs, AMD GPUs, and Google Cloud TPUs. This ensures that users can leverage powerful computing resources to enhance the performance and efficiency of their AI models right from the start.
Recent Updates and News
dstack constantly evolves to meet the needs of AI developers. Here are some of the latest updates:
- October 2024: The release of dstack 0.18.17 introduced new functionalities such as on-premises AMD GPU support and AWS EFA support.
- August 2024: Version 0.18.11 included enhancements like AMD support, encryption capabilities, and more.
- August 2024: The introduction of a Control Plane UI in version 0.18.10 improved user interface capabilities.
- July 2024: dstack 0.18.7 added features including Fleets, RunPod volumes, and a new dstack apply function.
- May 2024: Version 0.18.4 brought Google Cloud TPU support, and version 0.18.2 offered improvements in on-premises clusters and private subnet management.
Installation Process
For those interested in using dstack, setting up the platform involves a few steps:
Server Setup
Before using the Command-Line Interface (CLI) or application program interfaces (APIs), users must set up a dstack server. Once the server is running, users simply need to configure the CLI.
Backends Configuration (Optional)
If users plan to integrate dstack with their own cloud services, they should create a configuration file (~/.dstack/server/config.yml) to set up the necessary backends. This can also be accomplished through the control plane UI after starting the server. Alternatively, if running containers solely on local servers is the plan, backend configuration can be skipped.
Server Initialization
After the backends are configured, users need to start the server with the following commands:
$ pip install "dstack[all]" -U
$ dstack server
With this setup, the dstack server begins operation and provides an admin token for access.
CLI Configuration
Connecting the CLI to the dstack server requires configuration with the server's address, user token, and project name as follows:
$ pip install dstack
$ dstack config --url http://127.0.0.1:3000 \
--project main \
--token [Your-Admin-Token]
How dstack Operates
Configuration Definition
Users define configurations as YAML files within their development repository. These configurations include:
- Dev Environments: For interactive development using desktop Integrated Development Environments (IDEs).
- Tasks: For scheduling jobs, including distributed ones, or running web applications.
- Services: For deploying models and applications with features like auto-scaling and authorization.
- Fleets: For managing cloud and local server clusters.
- Volumes: For handling persisted data storage.
- Gateways: For managing network traffic and public endpoints.
Configuration Application
Configurations can be applied through the dstack CLI or programmatic APIs. Once applied, dstack handles provisioning, job queuing, auto-scaling, networking, and more, across both cloud environments and on-premises systems.
Additional Resources
For more details and practical examples, users can explore the following resources:
Contribution and Licensing
dstack welcomes contributions from its user community. Detailed information on how to contribute can be found in the project's contributing guidelines. The project is licensed under the Mozilla Public License 2.0, ensuring open access and collaboration.