OpenAssistant-Pythia-12B

The Open-Assistant SFT-4 12B model is an English language AI assistant that can engage in conversational interactions and provide helpful responses to user queries. It was developed by the Open-Assistant project using a supervised fine-tuning approach on a pre-trained Pythia 12B language model.

Key features:
- Transformer-based language model architecture
- Fine-tuned on human-written conversations demonstrating ideal assistant behavior
- Trained on data collected through the Open-Assistant human feedback platform
- Supports a wide range of conversational interactions and question-answering tasks
- Released under an open-source Apache 2.0 license

How it works:
The model generates assistant responses based on the conversational context provided in the input prompt. User and assistant turns are denoted by the special tokens <|prompter|> and <|assistant|> respectively, with each turn ending with <|endoftext|>. To generate an assistant reply, the input prompt should end with the <|assistant|> token.

AI foundation model:
The Open-Assistant SFT-4 model is fine-tuned from the EleutherAI Pythia 12B model, a large language model pre-trained on a massive corpus of online text data. The Pythia 12B model provides the base knowledge and linguistic capabilities that enable the Open-Assistant model to engage in open-ended conversation.

How to access:
The Open-Assistant SFT-4 model checkpoint and code are publicly available in the Open-Assistant GitHub repository. Detailed instructions for running the model can be found in the repository's documentation.