Advanced Solutions Lab - ASL-ML-Immersion Project
Overview
The Advanced Solutions Lab (ASL) team at Google Cloud has developed the ASL-ML-Immersion project, a comprehensive collection of Jupyter notebooks that are intended to be executed on Google's Vertex AI platform. Vertex AI is Google Cloud's robust AI Platform designed for advanced machine learning tasks. This project is designed to take software engineers, who may have no prior experience in machine learning, from novice to an advanced understanding of the subject.
The notebooks cover a vast range of model architectures including Deep Neural Networks (DNN), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), transformers, and specialized models like Spectral Normalized Gaussian Process (SNGP). These models target various data types such as tabular data, images, text, and time-series. The main implementations are conducted through TensorFlow and Keras. Additionally, the notebooks provide tools for effectively operationalizing models on Google Cloud's Vertex AI, using frameworks like TensorFlow, Scikit-learn, and PyTorch on a large scale. These include Vertex training, tuning, serving, as well as leveraging TFX and Kubeflow pipelines.
For those unfamiliar with machine learning or Vertex AI, it is recommended to start with the Introduction to TensorFlow.
Repository Structure
The repository is structured to facilitate learning and experimentation. The primary folders within the repository include:
- Kernels: Contains kernel scripts essential for some of the notebooks in the lab.
- Notebooks: This is the heart of the repository, containing labs and solutions categorized by different machine learning topics. Each category includes 'labs' for hands-on coding practice where users can complete tasks, and 'solutions' to verify their work.
- Sub-folders under notebooks include topics like BigQuery, production-level ML systems, Docker and Kubernetes, and more.
- Scripts: Houses setup scripts essential for setting up and enabling services on Vertex AI.
For a comprehensive view of the notebooks and their specific content, users can refer to this readme.
Environment Setup (Vertex AI)
To get started, users should begin by accessing Google Cloud's CloudShell and executing the following commands:
git clone https://github.com/GoogleCloudPlatform/asl-ml-immersion.git
cd asl-ml-immersion
./scripts/setup_on_cloudshell.sh
Following this, they should refer to the official documentation to set up a JupyterLab instance on the Vertex AI Workbench. While most labs do not need GPUs or TPUs, some notebooks might suggest their use for enhanced performance.
Once the Vertex Workbench Instance is set up and running, users can open a terminal within their JupyterLab instance and run:
git clone https://github.com/GoogleCloudPlatform/asl-ml-immersion.git
cd asl-ml-immersion
export PATH=$PATH:~/.local/bin
make install
Some specific notebooks may require additional setup steps, which will be outlined in the respective notebooks. Once the setup is complete, users can begin executing notebooks using the "Python 3" kernel.
Contributions
At the moment, contributions to this repository are restricted to Googlers. For more details about the contribution process, interested contributors can check the CONTRIBUTING.md.
Disclaimer & Licensing
It is important to note that this project is not an officially supported Google product, and using Google Cloud services will incur associated costs. More information on pricing can be found here.
The code within this repository is released under the Apache License, Version 2.0, full details of which can be accessed here. The code is provided "as is," without warranties or conditions of any kind.