OpenLLMetry: An Overview
Introduction
OpenLLMetry is an open-source solution designed to enhance the observability of applications using Large Language Models (LLMs). This project is built on top of OpenTelemetry, a widely-adopted framework for observability, allowing seamless integration with existing monitoring solutions like Datadog, Honeycomb, and more. OpenLLMetry is managed by Traceloop and is distributed under the Apache 2.0 license, ensuring it's free for anyone to use and modify.
Features
1. Comprehensive Observability
OpenLLMetry extends the capabilities of OpenTelemetry to provide detailed insights into LLM applications. By using standard OpenTelemetry data, it can be effortlessly integrated into your observability stack, offering robust insights into database operations, API calls, and more.
2. Extensive Instrumentation
The toolkit can instrument a wide range of services and frameworks. This includes well-known LLM providers and vector databases like OpenAI, HuggingFace, Pinecone, and Weaviate, among others. This ensures that developers can get visibility into nearly any component of their LLM application.
3. Easy Integration & Use
Getting started with OpenLLMetry is user-friendly. The project offers an easy-to-use SDK. By simply installing the package and initializing it in the code, developers can immediately start tracing their applications.
Getting Started
To begin using OpenLLMetry, you can install the SDK via pip:
pip install traceloop-sdk
Then, incorporate the following snippet into your application to initiate tracing:
from traceloop.sdk import Traceloop
Traceloop.init()
For immediate trace visibility, especially during local testing, you can disable batch sending:
Traceloop.init(disable_batch=True)
Supported Integrations
OpenLLMetry can export traces to a variety of destinations, including:
- Traceloop
- Dynatrace
- Datadog
- New Relic
- Honeycomb
- Grafana Tempo
- HyperDX
- SigNoz
- Splunk
- OpenTelemetry Collector
- IBM Instana
Each integration can be set up by following the detailed instructions in the official documentation.
Instrumented Technologies
LLM Providers: OpenAI, Anthropic, Cohere, HuggingFace, and more.
Vector Databases: Chroma, Pinecone, Qdrant, and others.
Frameworks: LangChain, LlamaIndex, Haystack, etc.
Community and Contribution
OpenLLMetry encourages contributions and offers various ways to get involved. Interested contributors can refer to the contributing guide or book a free pairing session with the team for assistance. Furthermore, community support is available through multiple channels:
- Slack: For real-time discussions.
- GitHub Discussions: For in-depth conversations and support.
- GitHub Issues: To report bugs or issues.
- Twitter: For the latest updates.
Acknowledgments
Special thanks to @patrickdebois for proposing the name that this project proudly carries today.
Contributors
The project's growth and success are thanks to numerous contributors, who have generously shared their expertise to improve OpenLLMetry.
For more details or to get involved, visit the official website or check the GitHub repository.