Dolly-v2 preview

dolly-v2 is an instruction-following family of large language model by Databricks. It is derived from EleutherAI's Pythia models and fine-tuned on a ~15K record instruction corpus generated by Databricks in capability domains from the InstructGPT paper, including brainstorming, classification, closed QA, generation, information extraction, open QA and summarization. dolly-v2 is not a state-of-the-art model, but does exhibit surprisingly high-quality instruction following behavior uncharacteristic of the foundation model on which it is based.

The Dolly model family has some limitations. In particular, dolly-v2-7b struggles with: syntactically complex prompts, programming problems, mathematical operations, factual errors, dates and times, open-ended question answering, hallucination, enumerating lists of specific length, stylistic mimicry, having a sense of humor, etc. Moreover, dolly-v2-7b does not have some capabilities, such as well-formatted letter writing, present in the original model.

The PyTorch based modeling codes are published in the Dolly repository on GitHub.

• Instruction following text-generation: taking as input text instruction like "Explain to me the difference between nuclear fission and fusion," the model generates an answer.

Get started

This model can be used in a notebook. Click Open notebook to use the model in Colab.

Dataset and training

The model is derived from EleutherAI's Pythia model suite and fine-tuned on a ~15K record instruction corpus generated by Databricks in capability domains from the InstructGPT paper, including brainstorming, classification, closed QA, generation, information extraction, open QA and summarization.

Model output

Taking as input text instruction like "Explain to me the difference between nuclear fission and fusion," the model generates an answer.

Best practices and limitations

The model struggles with: syntactically complex prompts, programming problems, mathematical operations, factual errors, dates and times, open-ended question answering, hallucination, enumerating lists of specific length, stylistic mimicry, having a sense of humor, etc. Moreover, dolly-v2-7b does not have some capabilities, such as well-formatted letter writing, present in the original model.

Making predictions

For a full list of possible sampling parameters, see https://docs.vllm.ai/en/latest/getting_started/quickstart.html.

Versions

Links

• Dolly 2.0 on Kaggle
• Dolly on GitHub