Hands-on LLMs Course: Building a Real-Time Financial Advisor
The "Hands-on LLMs" course is designed to guide learners through the end-to-end process of training and deploying a real-time financial advisor. Created by Paul Iusztin, Pau Labarta Bajo, and Alexandru Razvant, this course offers an in-depth exploration of large language models (LLMs) and how they can be used to develop robust financial advising systems.
1. Building Blocks
The course is structured around a three-pipeline architecture:
1.1. Training Pipeline
Learners are introduced to a training pipeline that focuses on:
- Loading a financial Q&A dataset.
- Fine-tuning an open-source LLM with QLoRA.
- Logging training experiments and storing the best model using the tools provided by Comet ML.
- Deploying the pipeline on Beam's serverless GPU infrastructure.
For those without the required hardware (4-core CPU, 14 GiB RAM, and a 10 GiB VRAM CUDA-enabled Nvidia GPU), instructions are provided for deploying the training pipeline on Beam's infrastructure.
1.2. Streaming Real-time Pipeline
This pipeline captures financial news from Alpaca, processes and transforms it into embeddings using Bytewax in real-time, and saves these embeddings in the Qdrant Vector DB. It is automatically deployed on AWS using GitHub actions. Basic hardware requirements are a 1-core CPU and 2 GiB RAM.
1.3. Inference Pipeline
Utilizing LangChain, this pipeline:
- Downloads the fine-tuned model.
- Accepts user input to query the Qdrant Vector DB.
- Enhances queries with financial news context.
- Provides financial advice through the LLM.
- Logs interactions on Comet ML's monitoring platform.
This pipeline is deployed as a RESTful API on Beam's serverless infrastructure and includes a user interface for easier interaction via Gradio.
1.4. Financial Q&A Dataset
The financial dataset used is created by fine-tuning with distillation from GPT3.5, targeting the development of models like Falcon 7B for specialized financial terminology and queries.
2. Setup External Services
Before embarking on the course, learners will need to set up and register for several external services, all of which offer free versions:
- Alpaca: To access financial news data.
- Qdrant: To manage vector DBs.
- Comet ML: For ML model tracking and storage.
- Beam: For serverless compute solutions.
- AWS: For deploying the feature pipeline.
3. Install & Usage
The project is divided into modules, each requiring its own setup as described in their respective README files. These modules include the dataset generator, training pipeline, streaming pipeline, and inference pipeline.
4. Lectures
A comprehensive series of video lectures and articles support the course, providing detailed walkthroughs and supplementary insights:
- Costs: All code and video lectures are free. Medium articles require a subscription if not already accessible.
- Ask Questions: Learners are encouraged to seek help through GitHub issues or connect with instructors via LinkedIn.
- Video Lectures: Topics range from course introductions to hands-on tutorials and system design discussions.
- Articles: These pieces delve into specific topics such as the importance of streaming pipelines and tips for LLM fine-tuning.
5. License
The course is open source under the MIT license, allowing learners to clone, fork, and utilize the code freely, provided that they credit the original authors.
6. Contributors & Teachers
The course is brought together by three main contributors:
- Pau Labarta Bajo: Senior ML & MLOps Engineer, primary instructor of the video lessons.
- Alexandru Razvant: Senior ML Engineer, key engineer behind the scenes.
- Paul Iusztin: Senior ML & MLOps Engineer, contributes to video lessons occasionally.
This Hands-on LLMs course offers an in-depth, structured approach for learners aiming to harness the power of LLMs in financial applications, complete with practical deployment and operational insights.