# OpenLLMetry review

> OpenLLMetry is an open-source set of OpenTelemetry extensions that adds observability to LLM applications, with instrumentations for LLM providers, vector DBs and frameworks.

Devtoolwatch Board, the week of Oct 25–31, 2026. Canonical page: https://index.devtoolwatch.com/tools/openllmetry/

## TL;DR

OpenLLMetry is a set of extensions built on OpenTelemetry that traces LLM applications. It is for Python developers who want LLM observability inside the stack they already use. Because OpenLLMetry outputs standard OpenTelemetry data, you can send traces to Datadog, Honeycomb, Grafana and other destinations.

## OpenLLMetry at a glance

- **Category:** [Evals and observability](https://index.devtoolwatch.com/categories/evals-and-observability/)
- **Maintained by:** traceloop from ServiceNow (organization, [@traceloop](https://github.com/traceloop) on GitHub)
- **License:** Apache 2.0
- **Language:** Python, Shell
- **Repository:** [github.com/traceloop/openllmetry](https://github.com/traceloop/openllmetry)
- **Website:** [www.traceloop.com/openllmetry](https://www.traceloop.com/openllmetry)
- **PyPI package:** [traceloop-sdk](https://pypi.org/project/traceloop-sdk/) 0.62.4
- **Latest release:** [0.62.4](https://github.com/traceloop/openllmetry/releases/tag/0.62.4) on GitHub, Sep 29, 2026
- **First published:** Sep 2, 2023

## What OpenLLMetry does

- OpenLLMetry instruments calls to LLM providers such as OpenAI, Anthropic, Bedrock, Cohere, Gemini, Mistral AI and Ollama.
- OpenLLMetry instruments vector databases including Chroma, Pinecone, Qdrant, Weaviate, Milvus, LanceDB and Marqo.
- OpenLLMetry supports frameworks such as LangChain, LangGraph, LlamaIndex, CrewAI, Haystack, LiteLLM and OpenAI Agents.
- OpenLLMetry instruments the MCP protocol, including client session spans.
- OpenLLMetry can also instrument everything OpenTelemetry already covers, such as databases and API calls.
- OpenLLMetry exports traces to tested destinations including Traceloop, Datadog, Honeycomb, New Relic, Splunk, Sentry and the OpenTelemetry Collector.
- OpenLLMetry provides decorators such as @workflow and @agent to name and group traced operations.

### How it works

OpenLLMetry runs inside your Python application. You install the Traceloop SDK and call Traceloop.init(), which instruments supported libraries and emits standard OpenTelemetry data. You can route that data to your existing observability backend. If you already use OpenTelemetry, you can add individual OpenLLMetry instrumentations directly without the SDK.

## How to install OpenLLMetry

**With pip**

```
pip install traceloop-sdk
```

### Requirements

- Python 3.10 or later, below version 4.

### First steps

1. Install the SDK with pip install traceloop-sdk.
2. Import Traceloop from traceloop.sdk and call Traceloop.init() in your code to start tracing.
3. When running locally, use Traceloop.init(disable_batch=True) to see traces immediately.
4. Follow the integration docs to connect your chosen observability destination.

## Who OpenLLMetry is for

- Teams that want LLM traces in an existing OpenTelemetry-based observability stack.
- Python developers building on LLM provider APIs, vector databases or agent frameworks.
- Projects already instrumented with OpenTelemetry that need LLM-specific instrumentations added.

### Limitations

- This repository covers Python. JavaScript and TypeScript use the separate OpenLLMetry-JS project.
- OpenLLMetry does not store or display traces itself. You need a destination backend to view them.
- Older SDK versions collected anonymous telemetry. Telemetry is removed from v0.49.2 onward, so upgrade to that version or later.

## Usage data (the week of Oct 25–31, 2026)

- Not ranked this week
- New GitHub stars per week (last 12 weeks): Aug 9: 15; Aug 16: 13; Aug 23: 23; Aug 30: 7; Sep 6: 18; Sep 13: 19; Sep 20: 18; Sep 27: 18; Oct 4: 19; Oct 11: 22; Oct 18: 8; Oct 25: 12

## OpenLLMetry alternatives

| Rank | Tool | Stars | Downloads/wk |
|---|---|---|---|
| #1 | [SkillSpector](https://index.devtoolwatch.com/tools/nvidia-skillspector/) | 23.7k | – |
| #2 | [Judgeval](https://index.devtoolwatch.com/tools/judgmentlabs-judgeval/) | 1.1k | 80.2k |
| #3 | [A.I.G (AI-Infra-Guard)](https://index.devtoolwatch.com/tools/tencent-ai-infra-guard/) | 7.2k | – |
| #4 | [Langfuse](https://index.devtoolwatch.com/tools/langfuse/) | 36.8k | 9.8M |
| #5 | [Opik](https://index.devtoolwatch.com/tools/opik/) | 23.1k | 709.1k |
| #6 | [promptfoo](https://index.devtoolwatch.com/tools/promptfoo/) | 26.7k | 1.2M |

## OpenLLMetry FAQ

### Is OpenLLMetry free and open source?

Yes. OpenLLMetry's source code is on GitHub (traceloop/openllmetry) under the Apache 2.0 license, so you can use it for free.

### How do I install OpenLLMetry?

Run `pip install traceloop-sdk`. The traceloop-sdk package needs Python 3.10 or newer, below 4.

### Is OpenLLMetry still maintained?

Yes. The traceloop/openllmetry repository was last updated on Oct 19, 2026, and the latest release, 0.62.4 on GitHub, came out on Sep 29, 2026.

### How popular is OpenLLMetry?

OpenLLMetry has 7.5k GitHub stars as of Nov 1, 2026. It gained 12 new stars in the week of Oct 25–31, 2026, against a 12-week average of 16 a week.

### What are the best alternatives to OpenLLMetry?

Other LLM evals and observability tools on the Devtoolwatch Board in the week of Oct 25–31, 2026, by momentum: SkillSpector (#1), Judgeval (#2), A.I.G (AI-Infra-Guard) (#3), Langfuse (#4), Opik (#5). The alternatives table above has their numbers.

### Does OpenLLMetry work with my existing observability tool?

OpenLLMetry likely works with your current tool, because it outputs standard OpenTelemetry data. The README lists tested destinations including Datadog, Honeycomb, Grafana, New Relic, Splunk, Dynatrace, Sentry, SigNoz and the OpenTelemetry Collector.

### Can I use OpenLLMetry without the Traceloop SDK?

Yes, OpenLLMetry works without the SDK. If you already have OpenTelemetry instrumented, you can add individual OpenLLMetry instrumentations directly. No telemetry is collected when you use the instrumentations without the SDK.

### Does OpenLLMetry collect telemetry?

No, current versions of OpenLLMetry do not collect telemetry. The project no longer logs or collects telemetry in the SDK or the instrumentations from v0.49.2 onward.

### How is OpenLLMetry related to OpenTelemetry semantic conventions?

OpenLLMetry's semantic conventions are now part of OpenTelemetry, according to the README. Recent releases also added GenAI semantic convention compliance for OpenAI Agents.

### Is there a JavaScript version of OpenLLMetry?

Yes, there is a JavaScript version. JS and TypeScript users should use OpenLLMetry-JS, a separate repository, with the @traceloop/node-server-sdk package.

## Sources

1. [traceloop/openllmetry on GitHub](https://github.com/traceloop/openllmetry)
2. [OpenLLMetry README](https://github.com/traceloop/openllmetry/blob/main/README.md)
3. [traceloop-sdk on PyPI](https://pypi.org/project/traceloop-sdk/)
4. [OpenLLMetry website](https://www.traceloop.com/openllmetry)
5. [Release 0.62.4 (Sep 29, 2026)](https://github.com/traceloop/openllmetry/releases/tag/0.62.4)
6. [Release 0.62.3 (Aug 10, 2026)](https://github.com/traceloop/openllmetry/releases/tag/0.62.3)
7. [Release 0.62.2 (Aug 9, 2026)](https://github.com/traceloop/openllmetry/releases/tag/0.62.2)
8. [Hacker News: Show HN: OpenLLMetry – OpenTelemetry-based observability for LLMs](https://news.ycombinator.com/item?id=37843907)
9. [Hacker News: Show HN: You don't need to adopt new tools for LLM observability](https://news.ycombinator.com/item?id=39371297)
10. [Devtoolwatch Board, the week of Oct 25–31, 2026](https://index.devtoolwatch.com/)