Vicuna

Vicuna is a series of LLaMA models fine-tuned on user-shared conversations collected from ShareGPT. It was released in March 2023 by the Large Model Systems Organization (LMSYS). In the follow-up paper "Judging LLM-as-a-judge with MT-Bench and Chatbot Arena" by Zheng et al (2023), evaluations using GPT-4 show Vicuna-13B outperforms LLaMA-13B in many tasks such as Fermi problems, role-play scenarios, and coding and math tasks.

Vicuna was created by fine-tuning a LLaMA base model using approximately 125K user-shared conversations gathered from ShareGPT.com with public APIs. The training recipe builds on top of Stanfordâs Alpaca by adjusting the training loss to account for multi-turn conversations and computes the fine-tuning loss solely on the chatbot's output. To enable Vicuna's understanding of long context, the max context length was expanded from 512 in alpaca to 2048.

• Text Generation: Takes as input text prompts, generates text responses.

Get started

This model can be used in a notebook and by clicking Deploy. Click Open notebook to use the model in Colab.

Deployment and inference

Deploying a model consists of three steps:

1. Creating an endpoint resource
2. Uploading the model
3. Deploying the model to the endpoint

A service account will need to be created with the Vertex AI User role for deploying models to Vertex AI endpoints.

Example deployment (Python)

Example batch inference (Python)

Example response (Python)

Best practices and limitations

• Vicuna models can be deployed using vLLM on Google Cloud to achieve high throughputs.
• Vicuna is released under the Llama 2 Community License Agreement.

Versions

Links

• Judging LLM-as-a-judge with MT-Bench and Chatbot Arena by Zheng et al (2023)
• Vicuna by LMSYS ORG
• Vicuna on GitHub