Claude 3 Haiku
Claude 3 Haiku is Anthropic's fastest vision and text model for near-instant responses to simple queries, meant for seamless AI experiences mimicking human interactions.Claude 3 Haiku is Anthropic's fastest vision and text model for near-instant responses to simple queries, meant for seamless AI experiences mimicking human interactions.
All Claude 3 models can process images and return text outputs, and feature a 200K context window.
Enable the Vertex AI API.
Authenticate with one of the standard mechanisms documented here.
Note: It is always recommended to utilize the pinned version of the model to ensure consistent behavior and avoid potential disruptions to existing workflows when model updates occur.
The following is a sample prompt to the model. To learn more about the possible request parameters, see the Claude Messages API Reference.
Note that the API for Claude on Vertex differs from the Anthropic API documentation in the following ways:
"vertex-2023-10-16"
.Also note that the Anthropic Text Completions API is not available on Vertex AI.
Request JSON body:
For the media type field, Claude 3 models support image/png
, image/jpeg
, image/gif
, or image/webp
. The data part of the image is the file bytes encoded to a base64 string (e.g. base64 -i my_image.png
).
Set stream
to true to incrementally stream the response using server-sent events. Streaming substantially reduces end-user perception of latency, because the response is returned incrementally as it's generated.
Save the request body in a file named request.json and then execute the following command in Cloud Shell or a local terminal window with the gcloud CLI installed. Authenticate and replace PROJECT_ID
with your Google Cloud project ID.
Note: It is always recommended to utilize the pinned version of the model to ensure consistent behavior and avoid potential disruptions to existing workflows when model updates occur.
There are Anthropic SDKs available for Python and TypeScript.
To install the Anthropic Python SDK:
Authenticate with one of the standard mechanisms documented here.
To use the SDK:
To use the SDK to stream messages:
To use the SDK to process images:
This model supports the following set of features:
Claude models are general purpose large language models. They use a transformer architecture and are trained via unsupervised learning, RLHF, and Constitutional AI (including both a supervised and Reinforcement Learning (RL) phase). Claude 3 is trained with hardware supplied by Amazon Web Services (AWS) and Google Cloud Platform (GCP). Core frameworks include PyTorch, JAX, and Triton.
Claude 3 models are trained on a proprietary mix of publicly available information on the Internet as of August 2023 (see the Anthropic Claude 3 model card appendix for details regarding Claude 3.5 Sonnet's training data cutoff), as well as non-public data from third-parties, data that Anthropic's users or companies hired to provide data labeling and creation services voluntarily create and provide, and data Anthropic generates internally. Anthropic employs several data cleaning and filtering methods, including deduplication and classification to filter data.
For data Anthropic obtains by crawling public web pages, Anthropic follows industry practices with respect to robots.txt instructions and other signals that website operators use to indicate whether they permit crawling of the content on their sites. In accordance with Anthropic's policies, Anthropic does not access password-protected or sign-in pages or bypass CAPTCHA controls when accessing data to include in training sets, and Anthropic conducts diligence on the data that it uses. Anthropic operates its crawling system transparently, which means website operators can easily identify Anthropic visits and signal their preferences to Anthropic.
See Anthropic's Claude 3 model card for more details.
All Claude models have been tested pre-deployment with a suite of evaluations. These include capabilities evaluations – which help measure the model's skills, strengths, and weaknesses across a range of tasks – as well as safety and alignment evaluations, which evaluate whether the model poses specific risks and the degree to which the model conforms to the ethical and behavioral expectations set for it.
Anthropic conducted a comprehensive evaluation of the Claude 3 family to analyze trends in their capabilities across various domains. Anthropic's assessment included several broad categories:
Anthropic's most intelligent models outperform their peers on most of the common evaluation benchmarks for AI systems, including undergraduate level expert knowledge (MMLU), graduate level expert reasoning (GPQA), basic mathematics (GSM8K) and more. They exhibit near-human levels of comprehension and fluency on complex tasks, leading the frontier of general intelligence.
Anthropic's academic benchmark evaluations cover reasoning, reading comprehension, math, science, grammar, and coding. Broadly Anthropic finds that Claude 3 models are stronger than previous models at coding and math, as evidenced by their scores in evaluations, often achieving SOTA in benchmarks such as the following, including undergraduate level expert knowledge (MMLU), graduate level expert reasoning (GPQA), basic mathematics (GSM8K) and more (bolded values are SOTA):
Benchmark |
Claude 3.5 Sonnet |
Claude 3 Opus |
Claude 3 Sonnet |
Claude 3 Haiku |
Undergraduate level knowledge MMLU |
90.4% 5-shot CoT 88.7% 5-shot |
86.8% 5-shot |
79.0% 5-shot |
75.2% 5-shot |
Graduate level reasoning GPQA, Diamond |
59.4% 0-shot CoT |
50.4% 0-shot CoT |
40.4% 0-shot CoT |
33.3% 0-shot CoT |
Grade school math GSM8K |
96.4% 0-shot CoT |
95.0% 0-shot CoT |
92.3% 0-shot CoT |
88.9% 0-shot CoT |
Math problem-solving MATH |
71.1% 0-shot CoT |
60.1% 0-shot CoT |
43.1% 0-shot CoT |
38.9% 0-shot CoT |
Multilingual math MGSM |
91.6% 0-shot |
90.7% 0-shot |
83.5% 0-shot |
75.1% 0-shot |
Code HumanEval |
92.0% 0-shot |
84.9% 0-shot |
73.0% 0-shot |
75.9% 0-shot |
Reasoning over text DROP, F1 score |
87.1 3-shot |
83.1 3-shot |
78.9 3-shot |
78.4 3-shot |
Mixed evaluations BIG-Bench-Hard |
93.1% 3-shot CoT |
86.8% 3-shot CoT |
82.9% 3-shot CoT |
73.7% 3-shot CoT |
Benchmark |
Claude 3.5 Sonnet |
Claude 3 Opus |
Claude 3 Sonnet |
Claude 3 Haiku |
Math & reasoning MMMU (val) |
68.3% |
59.4% |
53.1% |
50.2% |
Document visual Q&A ANLS score, test |
95.2% |
89.3% |
89.5% |
88.8% |
Math MathVista (testmini) |
67.7% CoT |
50.5% CoT |
47.9% CoT |
46.4% CoT |
Science diagrams AI2D, test |
94.7% |
88.1% |
88.7% |
86.7% |
Chart Q&A Relaxed accuracy (test) |
90.8% 0-shot CoT |
80.8% 0-shot CoT |
81.1% 0-shot CoT |
81.7% 0-shot CoT |
Claude can understand and output a wide variety of languages, such as French, Standard Arabic, Mandarin Chinese, Japanese, Korean, Spanish, and Hindi. Performance will vary based on how well-resourced the language is.
See Anthropic's Claude 3 model card for further details about Claude models.
Image & text input: With state of the art vision capabilities, Claude 3 models can process images and return text outputs to analyze and understand charts, graphs, technical diagrams, reports, and other visual assets.
Text output: Claude 3 models can output text of a variety of types and formats, such as prose, lists, Markdown tables, JSON arrays, HTML, code in various programming languages, and more.
Resource ID | Release date | Release stage | Description |
---|---|---|---|
claude-3-haiku@20240307 | 2024-01-29 | Preview Release | Prompt Caching |
claude-3-haiku@20240307 | 2024-12-17 | Preview Release | Token Counting |
claude-3-haiku@20240307 | 2024-03-19 | General Availability |
Google Cloud Console has failed to load JavaScript sources from www.gstatic.com.
Possible reasons are: