Tulu-2-DPO-70B

Tulu V2 DPO 70B is an advanced language model designed to function as a helpful assistant. Developed by the Allen Institute for AI (AI2), this model is a fine-tuned version of Llama 2, utilizing a mix of datasets and Direct Preference Optimization (DPO) to enhance its performance and serve as a robust alternative to the Llama 2 70B Chat model.

Key features:
- Model Type: Flagship model of a suite of instruction and RLHF tuned chat models.
- Languages: Primarily English.
- License: AI2 ImpACT Low-risk license.
- Finetuned From: meta-llama/Llama-2-70b-hf.
- Training Data: Diverse mix of human-created instructions and synthetic dialogues generated by other LLMs.
- Alignment: Further aligned using a Jax DPO trainer on the openbmb/UltraFeedback dataset.
- Model Size: 70 billion parameters.
- Performance Metrics: MT-Bench Score of 7.89 and AlpacaEval Win Rate of 95.1%.

How it works:
1. Users input text in a specific format, including user and assistant tags.
2. The model processes the input based on its training data and fine-tuning.
3. It generates a contextually relevant and helpful response.
4. The output is provided to the user in a coherent and structured manner.

Integrations:
GitHub Repository, AlpacaEval, vLLM

Use of AI:
Tulu V2 DPO 70B leverages generative AI to provide instruction-following capabilities, engage in dialogue generation, and synthesize content from various sources. It excels at generating preferred outputs based on user feedback and rankings provided by GPT-4.

AI foundation model:
The model is built on the Llama 2 architecture and further enhanced through Direct Preference Optimization (DPO). This process improves the model's ability to generate outputs aligned with user preferences and high-quality standards.

Target users:
- Developers
- Businesses
- Researchers
- Educators

How to access:
Tulu V2 DPO 70B is available through web applications like Hugging Face, as an API for custom application integration, and as an SDK for developers. While not open source, it is accessible under the AI2 ImpACT Low-risk license, allowing for a wide range of applications while ensuring responsible use.

Training details:
- Learning Rate: 5e-07
- Total Train Batch Size: 32
- Optimizer: Adam with betas=(0.9, 0.999) and epsilon=1e-08
- LR Scheduler Type: Linear
- LR Scheduler Warmup Ratio: 0.1
- Number of Epochs: 3.0

Input format:
The model is trained to use the following format:
<|user|>
Your message here!
<|assistant|>
A newline after <|assistant|> is crucial for optimal generation quality.