Prompts Royale: An Overview
Introduction
Prompts Royale is a groundbreaking application designed to streamline the process of prompt engineering. This task often requires multiple iterations to perfect, and ensuring that a prompt delivers the best possible instructions can be both challenging and time-consuming. Prompts Royale simplifies this process by enabling users to generate diverse prompt candidates, test these against specific scenarios, and ultimately identify the most effective option.
Core Features
Automatic Prompt Generation: Prompts Royale can create prompt candidates automatically based on the user’s input description and the associated test case scenarios. Additionally, users have the flexibility to write their own prompts if desired.
Automatic Test Cases Generation: The application can generate test cases automatically from a user’s description, which helps to provide a starting point for evaluating prompt effectiveness.
Monte Carlo Matchmaking + ELO Rating: Utilizing the Monte Carlo method for matchmaking ensures maximum information is gathered with minimal effort. The ELO rating system then ranks prompt candidates by their win rates and the strength of their opponents.
Customizability: Prompts Royale offers a versatile settings page, allowing users to tweak any parameter of the application to suit their needs.
Local and Secure: All data is stored locally, and requests to language model APIs are handled by the user’s browser, enhancing security and privacy.
How to Use Prompts Royale
Users can access Prompts Royale at promptsroyale.com or choose to run it locally by cloning the repository. Running locally requires Node version 16 or higher, and it uses Bun for dependency management. Here’s a brief guide to get started:
-
Clone the repository:
git clone [email protected]:meistrari/prompts-royale.git -
Install dependencies using Bun:
bun i -
Run the server:
bun run dev
Practical Usage Example
Consider the task of writing a prompt to create a headline for a website. By providing different scenarios and expected outputs, such as a car dealership website or a data analytics SaaS, users can generate a range of prompt candidates. For instance:
- Scenario: Website of a car dealership
- Expected Output: "Find the car of your dreams at the best price"
The system then evaluates these prompts and determines the most effective one through a combat system.
How It Works
Prompts are modeled as normal distributions with an initial mean of 1000 and a standard deviation of 350, representing their ELO score. Through a series of duels, matched via a Monte Carlo sampler, prompts are tested and ranked. Each duel influences the ELO scores, with the following formulas used for updates:
- $\mu_{A}' = \mu_{A} + \frac{K}{N} \cdot (score - expected_score(A,B))$
- $\sigma_{A}' = \sigma_{A} \cdot LR$
Where the learning rate (LR) is crucial for determining how quickly the system converges to a winner, with a recommended value of 0.96.
Conclusion
Prompts Royale is an invaluable tool for anyone involved in prompt engineering, providing a systematic way to generate and evaluate prompt candidates. By combining automated processes with a customizable setup, it simplifies and enhances the task of delivering precise instructions to language models.
Credits
The creators of Prompts Royale drew inspiration from the work done by the team behind gpt-prompt-engineer, building upon their substantial contributions to the field.