Project Overview: ReLLM
ReLLM, or Regular Expressions for Language Model Completions, presents an innovative way to extract structured data from large language models (LLMs). While some may joke that using regular expressions complicates problems, ReLLM smartly integrates them to provide more structured outputs from LLMs.
Core Functionality
ReLLM operates by utilizing regular expressions to define specific syntactic or semantic structures within the completions of language models. This tool can ensure outputs conform to certain templates, like JSON or XML formats, as well as more narrative constructs, such as sentences with blanks.
How ReLLM Works
The mechanism of ReLLM involves filtering out non-matching tokens before the generation of outputs. By testing possible completions against partial regex patterns, ReLLM can mask those that don’t fit the desired structure. This masking process aids the language model in avoiding the generation of mismatched tokens.
Hosted Version and Installation
For those interested in a hosted version of ReLLM, the Thiggle Regex Completion API is available at github.com/thiggle/api.
To get started with ReLLM, users simply need to install it using pip:
pip install rellm
The benefits of using ReLLM include improved completion quality, even when utilizing smaller language models, and the enhanced capability of programmatically parsing output. For practical examples, such as parsing a context-free grammar like JSON, visit the relevant examples provided.
Usage Example with Python
Here's a basic usage example using Python's transformers library:
import regex
from transformers import AutoModelForCausalLM, AutoTokenizer
from rellm import complete_re
model = AutoModelForCausalLM.from_pretrained("gpt2")
tokenizer = AutoTokenizer.from_pretrained("gpt2")
prompt = "ReLLM, the best way to get structured data out of LLMs, is an acronym for "
pattern = regex.compile(r'Re[a-z]+ L[a-z]+ L[a-z]+ M[a-z]+')
output = complete_re(tokenizer=tokenizer,
model=model,
prompt=prompt,
pattern=pattern,
do_sample=True,
max_new_tokens=80)
print(output)
This example will generate a structured output following the specified pattern.
Demonstrative Examples Using GPT2
ReLLM’s effectiveness is illustrated with a series of examples using the GPT2 model:
- Structuring Acronyms: Given a prompt to complete an acronym, ReLLM produces "Realized Logistic Logistics Model," conforming to the specified pattern, while an unconstrained model generates unrelated content.
- JSON Array Generation: ReLLM outputs a correctly formatted JSON array
["a", "b", "c"], contrasting with a model without regex constraints, which produces a more complex, less structured output. - Sentence Completion: In filling a sentence, ReLLM outputs a coherent and meaningful sentence, whereas the alternative generates anecdotal but unrelated content.
The examples highlight ReLLM’s proficiency in ensuring outputs that strictly align with predefined regular expression patterns, pushing the boundaries of structured data generation within the realm of language models.