Mixtral-8x7b-Instruct-v0.1

Mistral AI's Mixtral 8x7B is a high-performance, open-source AI model that utilizes a Sparse Mixture-of-Experts (SMoE) architecture to efficiently process and generate text across multiple languages.

Key features:
- Sparse Mixture-of-Experts architecture: A router network selects two out of eight groups of parameters to process each token, increasing parameter count while controlling cost and latency.
- 46.7 billion total parameters, with only 12.9 billion parameters used per token for efficient processing.
- Supports English, French, Italian, German, and Spanish.
- Excels in code generation and instruction following, achieving a score of 8.3 on MT-Bench.
- Handles context up to 32,000 tokens.
- Demonstrates less bias on the BBQ benchmark and more positive sentiments on the BOLD benchmark compared to Llama 2.

How it works:
Mixtral 8x7B is a decoder-only model that uses a feedforward block to select from eight distinct groups of parameters. For each token, a router network chooses two groups (experts) to process the token, combining their outputs additively. This approach allows the model to use a fraction of its total parameters per token, balancing performance and cost.

Integrations:
- vLLM Project: Incorporates Megablocks CUDA kernels for efficient inference.
- Skypilot: Enables deployment of vLLM endpoints on any cloud instance.
- Mistral Platform: Available behind the endpoint mistral-small, currently in beta.

Use of AI:
Mixtral leverages generative AI through its sparse mixture-of-experts architecture, enhancing performance and efficiency. The model is pre-trained on open web data, with experts and routers trained simultaneously. It can be fine-tuned for specific tasks, such as instruction following and bias reduction.

AI foundation model:
Mixtral 8x7B is a foundation model pre-trained on open web data, which can be fine-tuned for various tasks.

How to access:
Mixtral 8x7B is an open-source model, licensed under Apache 2.0. It can be accessed through the vLLM project and deployed using Skypilot. The model is also available on the Mistral Platform behind the endpoint mistral-small, currently in beta.