Langfuse

Langfuse is an LLM engineering platform that provides tools for tracing, prompt management, evaluation, dataset management, and metrics tracking to help teams build, debug, and improve complex LLM applications. The platform is designed with a focus on enterprise security, being SOC 2 Type II and ISO 27001 certified, as well as GDPR compliant.

Key features:
- Detailed production traces for faster debugging of LLM applications
- Collaborative versioning and deployment of prompts with low latency retrieval
- Playground within the Langfuse UI to test different prompts and models
- User feedback collection and evaluation function execution
- Dataset derivation from production data for model fine-tuning and testing
- Tracking of cost, latency, and quality metrics for performance optimization

How it works:
Users interact with Langfuse primarily through its web app, which provides an interface for managing and debugging LLM applications. They can view detailed traces, version and deploy prompts collaboratively, test prompts and models in the Playground, collect feedback and run evaluations, and derive datasets from production data to fine-tune their models.

Integrations:
Langfuse supports integrations with Python and JS/TS SDKs, as well as native integrations with popular libraries such as OpenAI SDK, Langchain, Llama-Index, LiteLLM, Haystack, Flowise, Langflow, Vercel AI SDK, Superagent, and Mirascope.

Use of AI:
Langfuse leverages generative artificial intelligence by integrating with various LLMs and providing tools to manage and optimize their performance.

AI foundation model:
The platform supports models from OpenAI and other popular LLM providers.

Target users:
- Development teams building and maintaining complex LLM applications
- Enterprises requiring robust security and compliance features
- Researchers needing detailed tracing and evaluation tools for LLMs
- Startups looking for scalable solutions to manage and optimize LLM applications

How to access:
Langfuse is available as a web app, API, SDKs for Python and JS/TS, and a self-hosted option for users who prefer to run the platform on their own infrastructure.