Langfuse vs Helicone vs Arize vs LangSmith: The 2026 AI Observability Landscape

Why the choice actually matters

Before comparing tools, be clear on what "AI observability" actually needs to do that regular APM doesn't. Chapter 1 of the AI Observability Playbook covers this in depth; the summary:

• Prompt and response capture. The 4KB prompt and 8KB response are the interesting data — and standard APM tools don't capture them.
• Token and cost tracking per call. Duration is not the unit of AI cost. Tokens are.
• Structured trace of multi-step workflows. An agent turn spans many model calls and tool executions — you need to see the causality, not just individual calls.
• Eval infrastructure. Both offline (golden datasets) and online (LLM-as-judge on production samples).
• Prompt versioning. Version control for the strings that determine your model's behavior.
• Dataset management. Building and iterating on eval datasets sourced from production.

Every tool in this comparison does at least the first three. The differentiators are in the last three — which is where teams end up picking different tools for different reasons.

How this comparison was built

The four tools compared in depth here — Langfuse, Helicone, Arize AI, LangSmith — are the ones with the largest production footprints and the clearest positioning against each other. Every one has been evaluated against the same six criteria:

1. Setup time — how long from signup to first useful trace
2. Prompt management — versioning, playground, deployment workflows
3. Eval infrastructure — golden datasets, LLM-as-judge, regression detection
4. Trace quality — how well nested agent/RAG workflows render
5. Pricing at scale — cost at 10M events/month, not just the free tier
6. Data residency — self-host options, region choice, deletion controls

Coverage of general-purpose APM options (Datadog LLM Observability, Weights & Biases, DIY OpenTelemetry) is in the honorable mentions section — they're viable choices for specific situations but don't fit the same product category.

Prices and features noted here reflect the state of each product as of publish date. All four products are actively developed; details change. Verify pricing on each vendor's site before committing.

The four contenders at a glance

Before the individual deep dives, here's how the four compare across the criteria that matter most for the choice:

The four sit in genuinely different positions. Below, each in depth.

Langfuse: the open-source contender

Positioning: The open-source-first observability platform. Comprehensive prompt management, strong eval infrastructure, own-your-data story that appeals to teams with data residency requirements or that want to avoid vendor lock-in.

What Langfuse does best

• Prompt versioning as a first-class feature. The prompt-management UI is arguably the best in this comparison. Version, deploy, roll back, and A/B test prompts without touching your code — the SDK fetches the "production" version at runtime.
• Dataset workflow that scales. Sample production traces into datasets. Iterate. Run evals against multiple prompt versions or models. This loop is smoother than any competitor here.
• Self-hosting that actually works. Apache 2.0 licensed, Docker Compose setup, well-documented Kubernetes deployment. Teams under HIPAA/financial compliance often end up on self-hosted Langfuse for this reason alone.
• LLM-as-judge evals with human review. The two-tier evaluation (fast automatic judge, then human review on flagged cases) is well-integrated into the trace UI. See Chapter 6 of the Observability Playbook for eval patterns.
• Growing OpenTelemetry support. Native OTel exporter means you can send traces to Langfuse alongside your existing APM without duplicate instrumentation.

Where Langfuse falls short

• Dashboards are functional, not beautiful. If you want polished executive-facing charts, Datadog or Arize will look better out of the box.
• Self-hosted maintenance is real work. "Free" if you don't count the eng time to run Postgres, ClickHouse, and the app tier reliably. Budget accordingly.
• Enterprise features (SSO, RBAC) require paid tier. Open source is genuinely open, but larger orgs will end up paying for what they need.
• No native alerting. You'll pipe metrics to Prometheus/Grafana or another tool for that.

Pricing model

Hobby (SaaS):    Free — 50k observations/month, 30-day retention
Pro (SaaS):      $59/mo base + $0.001/observation over 100k, 90-day retention
Team (SaaS):     $199/mo — SSO, RBAC, longer retention
Enterprise:      Custom — self-hosted with support, SLA
Self-hosted:     Free (open source), you pay infrastructure + eng time

Best for

Teams that value data ownership and want the strongest prompt-management workflow available. Also the natural choice for teams under compliance regimes (HIPAA, financial) that rule out most SaaS options.

Skip if

You want zero-config setup — Helicone will be faster to first-value. You need the highest-polish enterprise dashboards — Arize or Datadog will feel more finished.

Helicone: the zero-friction option

Positioning: The fastest possible path from "we should observe our LLM calls" to "we have observability." Proxy-based architecture means no code changes for basic observability — just change your base URL.

What Helicone does best

• Setup takes under 10 minutes. Change the base_url on your OpenAI/Anthropic client to Helicone's proxy, add an API key header, done. First traces flow in immediately.
• Generous free tier. 100,000 requests/month for free is enough for most side projects and early-stage startups to never pay.
• Built-in features beyond observability. Response caching (their own layer, not provider-native), rate limiting, retries, and user-level rate limits all configurable via headers. Genuinely useful for products that would otherwise build these themselves.
• Async logging option for zero-latency deployments. If you don't want to route through their proxy, an SDK sends logs asynchronously without adding request-path latency.
• Prompt experiments feature. A/B test prompt versions with real production traffic, aggregated results in the dashboard.

Where Helicone falls short

• Proxy adds a network hop. Even at low latency (usually 20-50ms added), some production stacks care. The async SDK option avoids this but loses features like response caching and rate limiting.
• Prompt management is basic. No dedicated prompt versioning UI — you version in code. Fine for small teams; awkward at scale.
• Eval features are less mature. LLM-as-judge exists but the workflow is less polished than Langfuse or Arize. Dataset management is functional, not comprehensive.
• Vendor concentration risk. Routing all your LLM traffic through a third-party proxy creates a hard dependency. Helicone's uptime is good, but any outage on their end is your outage.

Pricing model

Free:        100k requests/month, basic features
Pro:         $20/mo — unlimited requests, most features
Team:        $200/mo — SSO, priority support
Enterprise:  Custom — SLA, dedicated support
Self-hosted: Free (MIT license), infrastructure cost only

Best for

Teams that need observability yesterday and don't want to slow down the roadmap for a proper integration. Startups on the free tier stretch a long way. Products that would benefit from the extra features (caching, rate limits) get real bonus value.

Skip if

You have strict latency budgets that can't absorb 20-50ms of proxy overhead (and can't use the async SDK). You need enterprise-grade prompt management workflows. You need self-hosted deployment on your own infrastructure (technically possible but less polished than Langfuse's self-host story).

Arize AI: the enterprise choice

Positioning: Enterprise ML observability that also handles LLMs. Deepest drift-detection and monitoring capabilities in this comparison, plus Phoenix — a free open-source companion tool for local development and evaluation.

What Arize does best

• Drift detection is unmatched. If your production distribution shifts (users start asking different questions, embedding space moves, tool call patterns change), Arize will detect it earlier and with more precision than anyone else in this comparison. This capability comes from the team's roots in traditional ML monitoring.
• RAG-specific evaluation. Retrieval precision, retrieval recall, grounding rate — the metrics that specifically matter for RAG workloads — are first-class rather than bolted on.
• Trace visualization is best-in-class. Deeply nested agent workflows render cleanly, with the causality relationships easy to trace visually.
• Phoenix for local development. Free open-source tool for running the trace UI locally against your dev environment. Great for iterating on prompts without hitting production.
• Handles ML + LLM together. If your team runs both traditional ML models and LLM-powered features, Arize is the only tool in this comparison that treats them as first-class citizens equally.

Where Arize falls short

• Setup is heavier. Full production integration takes 1-2 hours, not minutes. Enterprise features add more.
• Pricing is not for small teams. No self-serve pricing on the site — you talk to sales. Real annual contracts start in the low five figures for most deployments.
• Prompt management is less mature than Langfuse or LangSmith. Arize's strength is monitoring what's already in production, not managing what goes into production.
• Overkill for simple use cases. If you just need to log traces and monitor cost, you're paying for capability you won't use.

Pricing model

Phoenix (open source): Free, self-hosted, local development
Arize AI (SaaS):       Enterprise contract only — talk to sales
                       Typical range: $30k-$200k+ annual
On-prem:               Available on enterprise contracts

Best for

Enterprise teams already running both traditional ML and LLM workloads. Products where RAG quality is a competitive differentiator and needs deep monitoring. Regulated industries needing on-prem deployment with vendor support.

Skip if

You're an early-stage startup — the pricing and complexity are wrong for your stage. You only run LLM workloads (no ML) and don't need the drift-detection depth. You want prompt-management workflows more than monitoring depth.

LangSmith: the LangChain-native tool

Positioning: Built by the LangChain team. Natively integrated with LangChain and LangGraph workflows, works with anything else via SDK. Strong evaluation infrastructure and playground UX.

What LangSmith does best

• LangChain integration is invisible. If you're on LangChain, you set one environment variable and every chain, tool call, and LLM invocation is automatically traced with full nesting. Nothing else touches this level of zero-config for LangChain users.
• LangGraph state visualization. Multi-step agent workflows built on LangGraph render as first-class state graphs — you can see the exact path taken through the graph for each turn. Uniquely valuable for LangGraph users.
• Playground and prompt hub. Test prompt changes against your dataset before deploying. Public prompt hub with community-contributed prompts (usable in your workflows via reference).
• Evaluator library. Pre-built evaluators for common tasks (QA correctness, hallucination detection, toxicity, format compliance) that work out of the box.
• Native dataset workflow. Sample traces to datasets, run experiments comparing prompt/model versions, view results in the same UI. Well-integrated.

Where LangSmith falls short

• Less polished for non-LangChain workflows. The SDK works, but every feature was designed assuming LangChain workflows. You'll see LangChain terminology in places that don't map naturally to your custom stack.
• Ties you to the LangChain ecosystem. If you're already skeptical about LangChain's abstractions (many teams are), LangSmith deepens that dependency.
• Self-hosted is paid-only. Unlike Langfuse (open-source self-host), LangSmith self-hosted is enterprise-tier pricing.
• Fewer alerting/monitoring features. Compared to Arize's monitoring depth, LangSmith is lighter — more focused on iteration, less on production observability.

Pricing model

Developer:   Free — 5k traces/month, 1 seat
Plus:        $39/user/month — 10k traces included per seat, extra @ $0.005 each
Enterprise:  Custom — self-hosted option, SSO, higher volume

Best for

Teams building on LangChain or LangGraph — the integration depth is genuinely different from any competitor. Also good for teams that value the prompt hub and pre-built evaluator ecosystem.

Skip if

You're not on LangChain and don't plan to be. Custom-built agent stacks work with LangSmith but you're paying for value you don't fully use — Langfuse or Helicone will feel less LangChain-centric.

The honorable mentions

Three more options worth naming, none quite in the same category as the four above but each right for specific situations.

Datadog LLM Observability

Add-on to Datadog APM. Adds LLM-specific dashboards, cost tracking, and prompt/response capture to the standard Datadog trace view. The only credible option if your organization is already deep on Datadog for the rest of your infrastructure — the cost of adding another observability vendor often outweighs the potential upside of a purpose-built tool. Pricing: adds to your existing Datadog contract, typically 15-40% on top of what you're already paying.

Best for: enterprises already committed to Datadog. Skip if: you're not on Datadog for other reasons.

Weights & Biases

Started in ML experiment tracking (training runs, hyperparameter sweeps) and expanded into inference observability. Strong for teams that do fine-tuning alongside inference — the same tool tracks your training experiments and your production inference. Less differentiated for pure-inference workloads. Pricing: free tier available, paid tiers scale with team size and storage.

Best for: teams doing serious fine-tuning + inference. Skip if: you're pure inference (Langfuse or Helicone will fit better).

DIY: OpenTelemetry + Grafana

Roll your own with OpenTelemetry semantic conventions for GenAI + Grafana Loki (logs) + Tempo (traces) + Prometheus (metrics). Total control, no vendor lock-in, standard tooling. But 3-6 months of engineering time to reach feature parity with any of the above, and ongoing maintenance forever.

Best for: teams with strong existing OpenTelemetry expertise and specific requirements no vendor meets. Skip if: you have any option that fits and any interest in shipping features instead of maintaining observability infrastructure.

The decision framework

The full decision framework in one table:

The default answer

For most teams asking this question for the first time: start with Helicone or Langfuse SaaS. Both have generous free tiers that will get you through the early exploration without a payment decision. Helicone if you want zero-config; Langfuse if you know you'll want prompt-management workflows later. You can migrate to something else in six months when you have real data about what you actually need.

The mistake to avoid: overthinking the choice at the start. The wrong choice at 50k traces/month is a weekend of migration work. The wrong choice at 50M traces/month is a quarter-long project. Pick something adequate now and let usage tell you what to switch to later.

The pricing reality check

Vendor pricing pages are optimized to look reasonable. Real pricing at production scale is different. Rough monthly cost at three realistic scales:

Numbers are approximate ranges — pricing tiers, retention choices, and extra features (SSO, longer retention, on-call support) all shift costs meaningfully. Get a formal quote before contract signing at any scale above the free tier.

Hidden costs to watch

• Retention. 30-day retention is often the default; 90-day or longer costs meaningfully more. Compliance workloads that need years of retention should model this into the choice.
• Seat pricing. LangSmith and some others price per user seat. Fine for small teams; adds up fast for larger orgs.
• Storage for prompt/response payloads. Full payload capture at 8KB average across 50M requests = 400GB/month — vendors will meter this.
• Eval judge model costs. LLM-as-judge on samples is a separate line item. At 5% sample rate on 50M events with a Haiku judge, that's 2.5M judge calls monthly.

Self-host cost reality

"Free open source" self-hosting for Langfuse or Helicone still has real costs. A production-grade self-hosted Langfuse deployment typically needs: Postgres cluster ($200-500/mo managed), ClickHouse cluster ($300-1,000/mo managed), the app tier (2-4 instances, $100-400/mo), plus 0.25-0.5 FTE of engineering attention for maintenance and upgrades. At 5M events/month, self-hosted is often comparable to SaaS on total cost once eng time is included. It's still the right choice when data residency requires it.

What switching between them actually costs

Switching observability tools is easier than switching most infrastructure but harder than vendors imply. Real cost breaks into three buckets:

1. Instrumentation code changes

If you instrumented with OpenTelemetry from the start, switching is close to config-level — export destination changes, most spans transfer. If you instrumented with a vendor SDK, expect 2-5 days of engineering work per major integration point (agents, RAG pipelines, batch jobs).

This is why instrumenting to OpenTelemetry semantic conventions is worth it even if you use a vendor SDK today. All the tools in this comparison accept OTel input — future you will thank present you.

2. Historical trace migration

You typically don't migrate historical traces. It's rarely worth it — the value of historical traces drops sharply after 30 days, and by the time you're switching tools you probably have new insights the old traces wouldn't help with. Plan for a clean cutover with maybe a week of overlap where you send to both.

3. Eval datasets and prompt versions

This is the sticky bit. Datasets you've built up in one tool (curated production samples, LLM-judge scoring history, human review annotations) don't transfer cleanly. Prompts you've versioned in one tool need re-importing.

Every tool in this comparison provides export APIs — Langfuse and LangSmith have particularly clean dataset export/import. Budget 2-5 days for eval infrastructure migration on top of the code changes.

The pragmatic switching playbook

1. Set up the new tool alongside the old one. Send traces to both for 2-4 weeks.
2. Migrate one non-critical feature first. Verify traces, evals, and dashboards work as expected.
3. Migrate remaining features one at a time, oldest to newest. Newer features are easier to reinstrument correctly.
4. Keep old tool running for 30-60 days after cutover as fallback (in case something's missing you didn't notice).
5. Cancel the old tool subscription only after a full month with no reason to look back at it.

Frequently asked questions

Common questions from teams evaluating AI observability platforms for the first time or considering a switch.