Chatter

Chatter is a comprehensive platform designed to streamline the development, evaluation, and versioning of large language model (LLM) deployments. It provides teams with a robust suite of tools to create complex model chains, perform automatic evaluations, and manage prompt versions, enabling efficient LLM workflows and high-quality model performance through iterative testing and collaboration.

Key features:

- Complex Chains: Build intricate chains using various models and configurations, with support for function calling, chaining, and data manipulation.
- Automatic Evaluations: Maintain a testing suite with automatic evaluations across multiple metrics, including LLM-based evaluation, semantic similarity, and regex matching.
- Prompt Versioning: Keep all prompts versioned and logged for easy comparisons and tracking.
- Collaboration Tools: Provide a separate viewer for non-technical users to facilitate stakeholder involvement in the testing process.
- SDK & Code Export: Separate iteration from the codebase for enhanced speed and prompt security.
- Jinja2 Templating Engine: Perform intermediate data transformations directly in prompts using Jinja2.
- Retrieval Augmented Generation (RAG): Quickly set up RAG pipelines from vector databases or other data sources, and build and maintain document repositories.
- API Key Vault: Centrally manage LLM API keys and track token usage and costs over time or per call.
- Analytics: Gain detailed insights into every call, including duration, token usage, cost, and performance.
- Observability: Examine every call within a chain to understand information flow and debug efficiently.
- Function Builder: Use a simple interface to build and maintain a library of function calls with easy configurability per API call.
- Routing: Support complex routing flows involving multiple sets of functions and system prompts for a single query.
- Chat Testing: Support multiple roles within a call and seamlessly import and evaluate specific chat messages.

How it works:

1. Users access Chatter through a web app interface.
2. They build complex model chains using the platform's tools.
3. Automatic evaluations are performed to test model performance.
4. Users can collaborate with team members, including non-technical stakeholders.
5. The platform provides analytics and observability features for monitoring and debugging.
6. Users can set up RAG pipelines and perform data transformations as needed.
7. API keys and costs are managed through the integrated Key Vault.

Integrations:

- Vector Databases
- API Management tools

Use of AI:

Chatter leverages generative artificial intelligence to enhance its evaluation and testing capabilities. The platform supports various LLMs and allows users to build complex chains and perform advanced data manipulations.

AI foundation model:

The platform's flexibility suggests compatibility with a wide range of models, though specific foundation models or LLMs are not mentioned.

Target users:

- Development Teams
- Data Scientists
- Non-Technical Stakeholders

How to access:

Chatter is available as a web app and offers an SDK for integration with other systems.