gpt-j-api - Explore GPT-J Model for Efficient Text Generation and Classification

Introducing the gpt-j-api Project 🦜

The gpt-j-api project is a user-friendly and efficient application programming interface (API) for interacting with the GPT-J language model and its variants. Designed to simplify the engagement with this advanced AI model, it provides seamless access for both text generation and zero-shot text classification across multiple languages.

Ways to Access

The gpt-j-api offers two primary avenues for users to test and interact with the model:

Streamlit Web App: Accessible via http://api.vicgalle.net:8000/, this interface is designed for straightforward user interaction with the model.
Dedicated API: Comprehensive documentation is available at http://api.vicgalle.net:5000/docs, detailing how to implement and utilize the API effectively.

Open API Endpoints 🔓

The gpt-j-api includes public API endpoints which do not require authentication, making them readily accessible. Below are the main features:

GPT-J Text Generation 🤖

Generate Endpoint: POST /generate/ - This endpoint allows users to input parameters and receive generated text from the model.

Zero-shot Text Classification (Multilingual) 🌍

Classify Endpoint: POST /classify/ - This enables users to classify text across different languages without requiring pre-training on each language.

Utilizing the API 🔥

The API can be easily integrated into Python scripts or Bash commands. Here’s a quick setup guide:

Text Generation Using Python:

import requests
context = "In a shocking finding, scientist discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English."
payload = {
    "context": context,
    "token_max_length": 512,
    "temperature": 1.0,
    "top_p": 0.9,
}
response = requests.post("http://api.vicgalle.net:5000/generate", params=payload).json()
print(response)

Zero-shot Classification Using Python:

import requests
payload = { 
    "sequence" : "The movie started slow, but in the end was absolutely amazing!", 
    "labels" : "positive,neutral,negative"}
response = requests.post("http://api.vicgalle.net:5000/classify", params=payload).json()
print(response)

Using Bash Commands:

curl -X 'POST' \
  'http://api.vicgalle.net:5000/generate?context=In%20a%20shocking%20finding%2C%20scientists%20discovered%20a%20herd%20of%20unicorns%20living%20in%20a%20remote%2C%20previously%20unexplored%20valley%2C%20in%20the%20Andes%20Mountains.%20Even%20more%20surprising%20to%20the%20researchers%20was%20the%20fact%20that%20the%20unicorns%20spoke%20perfect%20English.&token_max_length=512&temperature=1&top_p=0.9' \
  -H 'accept: application/json' \
  -d ''

Deploying the API Server

To deploy your server, all you need is a TPU VM environment where the code has been tested on both v2-8 and v3-8 variants. Follow these steps:

Install Requirements and Weights:

python3 -m pip install -r requirements.txt
wget https://the-eye.eu/public/AI/GPT-J-6B/step_383500_slim.tar.zstd
sudo apt install zstd
tar -I zstd -xf step_383500_slim.tar.zstd
rm step_383500_slim.tar.zstd

Run the Server:
```
python3 serve.py
```

After setting up, head over to http://localhost:5000/docs to utilize the API.

Deploying the Streamlit Dashboard

For those interested in a more graphical interface, simply run:

python3 -m streamlit run streamlit_app.py --server.port 8000

Getting in Touch

For any inquiries or requests, feel free to reach out via email at vgallegoalcala at gmail dot com.

Sponsors and Acknowledgements

Special thanks to Aspie96 and the TPU Research Cloud for their generous support, which makes this project possible.