dalai

Introduction to Dalai

Dalai is a versatile project designed to run the LLaMA and Alpaca models on your personal computer. This project makes it easy for developers and enthusiasts to explore these models locally without the need for extensive server resources.

Key Features

1. Cross-Platform Compatibility:

   • Dalai supports all major operating systems, including Linux, Mac, and Windows. This ensures a broad reach and versatility for users on different platforms.

2. Memory and Disk Space Requirements:

   • Dalai is optimized to run on most modern computers. Specific memory requirements are:

     • 7B model: ~4 GB
     • 13B model: ~8 GB
     • 30B model: ~16 GB
     • 65B model: ~32 GB

   • Disk space varies by model:

     • 7B Alpaca: 4.21GB
     • 13B Alpaca: 8.14GB
     • LLaMA models range from 31.17GB (full) and 4.21GB (quantized for 7B) to 432.64GB (full) and 40.88GB (quantized for 65B).

3. Model Management:

   • You can choose which models to install individually, allowing for efficient disk space use. Models are available in both full and quantized versions for better storage management.

4. Hackable Web Application:

   • Dalai comes with a customizable web application, making it easier to interact with the models via a graphical interface.

5. JavaScript and Socket.io API:

   • APIs are included to facilitate seamless integration into existing JavaScript projects, providing real-time, interactive capabilities.

Getting Started

Dalai provides simple steps for quick setup across different systems. Below is a brief guide for each platform:

Docker Compose

1. Ensure Docker is installed and running.
2. Use the following terminal commands to build and run Dalai:

   docker compose build
   docker compose run dalai npx dalai alpaca install 7B
   docker compose up -d
   

3. Access the application at http://127.0.0.1:3000/.

Mac Setup

1. Install Node.js (version 18 or above).
2. Install Models:
   • For Alpaca: npx dalai alpaca install 7B
   • For LLaMA: npx dalai llama install 7B
3. Run Web UI:
   • Launch the server with npx dalai serve and visit http://localhost:3000.

Windows Setup

1. Install Visual Studio with the necessary development options (Python, Node.js, C++).
2. Install Models using the command line (cmd):
   • Alpaca: npx dalai alpaca install 7B
   • LLaMA: npx dalai llama install 7B
3. Run Web UI:
   • Use npx dalai serve to start the server and visit http://localhost:3000.

Linux Setup

1. Install Dependencies:
   • Python (version 3.10 or lower)
   • Node.js (version 18 or above)
2. Install and Run Models similar to other platforms, using terminal commands.

Community and Support

Dalai has a strong community presence with platforms like GitHub, Twitter, and Discord, offering ample support and collaboration opportunities for users from all walks of life.

By providing accessible tools and a user-friendly setup process, Dalai empowers developers to experiment and create with ease, right from their local machines.