OVHcloud publishes reference architecture for self-hosted Langfuse on MKS with AI Endpoints

OVHcloud published a reference architecture on 23 July 2026 for deploying Langfuse, an open-source observability platform for LLM applications, on its Managed Kubernetes Service (MKS). The declared goal is to track token consumption, latency and cost per model for each call to AI Endpoints, OVHcloud's OpenAI-compatible inference service, without allowing prompts or responses to leave the customer-controlled infrastructure.

How the data flow works

OVHcloud Managed Kubernetes Service (MKS) removes the operational burden of managing the Kubernetes control plane: node pools, updates and high availability are handled by OVHcloud, whilst you retain full control over what runs on the worker nodes. It is the natural place to run a stateless application like Langfuse's web and worker processes, whilst stateful components live in OVHcloud managed services alongside the cluster. Every stateful dependency—PostgreSQL, Valkey, ClickHouse, object storage—is an external OVHcloud managed service accessed via TLS.

The technical point of interest for anyone deploying this pipeline to production is the separation between application calls and tracing: the OpenAI SDK with the Langfuse wrapper exports in the background, via OpenTelemetry, a batch of spans to the /api/public/otel/v1/traces endpoint without blocking the response already delivered to the application. The Langfuse worker then reads the queue from Valkey, retrieves the batch from object storage and writes trace and generation records to ClickHouse with model, token count, latency and pricing.

Cluster sizing

For the test, OVHcloud recommends two separate node pools: a system pool (np-system, 3 nodes B3-8 flavour) for Traefik and cert-manager, and one dedicated to application load (np-workload, 1 node B3-16 flavour) for Langfuse web and worker pods. To these add three managed databases—PostgreSQL 17 on Business plan, Valkey 8.1 on Business plan and ClickHouse 25.8 on Production plan—plus an S3-compatible bucket for object storage.

Model pricing via public API

The catalogue at https://catalog.endpoints.ai.ovh.net/rest/v1/models_v2 provides information and metadata on AI Endpoints models, including model lists, supported capabilities, pricing and model-specific metadata. Use of the Catalog API is required in this architecture to fetch the pricing block and register each model's price in Langfuse, which then automatically calculates call costs. The call requires no authentication and is designed to populate local price tables on a periodic basis, whilst the actual inference API remains the one used by the application for every request.

For those already running MKS clusters with AI Endpoints workloads, the complete guide with YAML manifests and installation scripts is available on the OVHcloud blog.