Model Cards & Prompt formats

Llama 3

Model Card

You can find details about this model in the model card.

Special Tokens used with Llama 3

<|begin_of_text|>: This is equivalent to the BOS token

<|eot_id|>: This signifies the end of the message in a turn.

<|start_header_id|>{role}<|end_header_id|>: These tokens enclose the role for a particular message. The possible roles can be: system, user, assistant.

<|end_of_text|>: This is equivalent to the EOS token. On generating this token, Llama 3 will cease to generate more tokens.

A prompt can optionally contain a single system message, or multiple alternating user and assistant messages, but always ends with the last user message followed by the assistant header.

Llama 3

Code to produce this prompt format can be found here.

Note: Newlines (0x0A) are part of the prompt format, for clarity in the example, they have been represented as actual new lines.

<|begin_of_text|>{{ user_message }}

Llama 3 Instruct

Code to generate this prompt format can be found here.

Notes:

Newlines (0x0A) are part of the prompt format, for clarity in the examples, they have been represented as actual new lines.
The model expects the assistant header at the end of the prompt to start completing it.

Decomposing an example instruct prompt with a system message:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a helpful AI assistant for travel tips and recommendations<|eot_id|><|start_header_id|>user<|end_header_id|>

What can you help me with?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

<|begin_of_text|>: Specifies the start of the prompt

<|start_header_id|>system<|end_header_id|>: Specifies the role for the following message, i.e. “system”

You are a helpful AI assistant for travel tips and recommendations: The system message

<|eot_id|>: Specifies the end of the input message

<|start_header_id|>user<|end_header_id|>: Specifies the role for the following message i.e. “user”

What can you help me with?: The user message

<|start_header_id|>assistant<|end_header_id|>: Ends with the assistant header, to prompt the model to start generation.

Following this prompt, Llama 3 completes it by generating the {{assistant_message}}. It signals the end of the {{assistant_message}} by generating the <|eot_id|>.

Example prompt with a single user message

<|begin_of_text|><|start_header_id|>user<|end_header_id|>

What is France's capital?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

System prompt and multiple turn conversation between the user and assistant

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a helpful AI assistant for travel tips and recommendations<|eot_id|><|start_header_id|>user<|end_header_id|>

What is France's capital?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Bonjour! The capital of France is Paris!<|eot_id|><|start_header_id|>user<|end_header_id|>

What can I do there?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Paris, the City of Light, offers a romantic getaway with must-see attractions like the Eiffel Tower and Louvre Museum, romantic experiences like river cruises and charming neighborhoods, and delicious food and drink options, with helpful tips for making the most of your trip.<|eot_id|><|start_header_id|>user<|end_header_id|>

Give me a detailed list of the attractions I should visit, and time it takes in each one, to plan my trip accordingly.<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Was this page helpful?

Yes

On this page

Llama 3

Model Card

Special Tokens used with Llama 3

Llama 3

Llama 3 Instruct

Table Of Contents

Llama 3

Model Card

Special Tokens used with Llama 3

Llama 3

Llama 3 Instruct