Model Cards & Prompt formats

Introduction

Llama Guard 4 (12B) is our latest safeguard model with improved inference for detecting problematic prompts and responses. It is designed to work with the Llama 4 line of models, such as Llama 4 Scout and Llama 4 Maverick.

Download the Model

Download Llama Guard 4.

Model Card

For comprehensive technical information about Llama Guard 4, please see the official model card, located on GitHub.

Architecture

Llama Guard 4 is a natively multimodal safeguard model. The model has 12 billion parameters in total and uses an early fusion transformer architecture with dense layers to keep the overall size small. The model can be run on a single GPU. Llama Guard 4 shares the same tokenizer and vision encoder as Llama 4 Scout and Maverick.

Llama Guard 4 and Llama 3

Llama Guard 4 is also compatible with the Llama 3 line of models and can be used as a drop-in replacement for Llama Guard 3 8B and 11B for both text-only and multimodal applications. However, Llama Guard 3 1B still holds value for specific use cases, such as deployment on edge devices.

Image Support

Llama Guard 4 evaluates both the prompt text and the image together in order to classify the prompt. It is not designed to perform image-only classification. Further, the model has been optimized for English-language text, so the text component of the prompt should be in English. For other languages, developers are expected to ensure that their deployments are tested and completed safely and responsibly.

Images that are submitted for evaluation should have the same format (resolution and aspect ratio) as the images that you submit to the Llama 4 models. Also, note that the model does not support the evaluation of images that were themselves created using generative AI technology.

You can evaluate images in multi-turn conversations, but you need to add the image token (see below) to the turn in which the image occurs.

Prompt format

Tokens

Description

<|begin_of_text|>

Specifies the start of the prompt

<|header_start|>

<|header_end|>

These tokens enclose the role for a particular message. The possible roles can be user and assistant

<|eot|>

End of turn. Represents when the LLM determines it finished interacting with the user message that initiated its response. This is used at the end of interaction with the model.

Image Tokens

Note: These tokens are automatically populated when you run inference with images (reference code). Here they are shown just for your information.

We apply a dynamic image transformation strategy that divides the input image into 336×336 pixel tiles. Additionally, a global tile (created by resizing the entire input image to 336×336 pixels) is appended after the local tiles to provide a global view of the input image.

<|image_start|>...<|image_end|>: These tokens enclose the image data in the prompt.
<|patch|>: These tokens represent subsets of the input image. Larger images have more patch tokens in the prompt.
<|tile_y_separator|>...<|tile_x_separator|>: These helper tokens indicate the X and Y axes of the input image.
<|image|>: This token separates the regular-sized image tokens from a downsized version of it that fits in a single tile.

Prompt Sections

A properly constructed Llama Guard prompt contains a number of sections, delimited by tags, such as <BEGIN UNSAFE CONTENT CATEGORIES> and <BEGIN CONVERSATION>. These are not special tokens, but are normal text in the prompt which enable the model to parse the prompt properly.

As the guardrails can be applied both on the input and output of the model, there are two different prompts: one for user input and the other for agent output. The role placeholder can have the values User or Agent; the former refers to the input and the latter to the output. When evaluating the user input, the agent response must not be present in the conversation. At the same time, when evaluating the agent response, both the user input and the agent response need to be present in the conversation; in this case, the user input provides important context for the evaluation.

The llama-cookbook repository has a helper function and an inference example that shows how to properly format the prompt with the provided categories. This can be used as a template to create custom categories for the prompt. You can also refer to the llama-stack GitHub repository that has reference implementations for input and output guardrails.

The variables to replace in this prompt template are:

{{ role }}: It can have the values: User or Agent.
{{ unsafe_categories }}: The default categories and their descriptions are shown below. These can be customized for zero-shot or few-shot prompting.
{{ user_message }}: input message from the user.
{{ model_answer }}: output from the model.

<|begin_of_text|><|header_start|>user<|header_end|>

Task: Check if there is unsafe content in '{{ role }}' messages in conversations according to our safety policy with the below categories.

<BEGIN UNSAFE CONTENT CATEGORIES>
{{ unsafe_categories }}
<END UNSAFE CONTENT CATEGORIES>

<BEGIN CONVERSATION>

User: <|image_start|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|image|><|patch|>...<|patch|><|image_end|><|image_start|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|image|><|patch|>...<|patch|><|image_end|>{{ user_message_1 }}

Agent: {{ model_answer_1 }}

<END CONVERSATION>

Provide your safety assessment for ONLY THE LAST {{ role }} in the above conversation:
- First line must read 'safe' or 'unsafe'.
- If unsafe, a second line must include a comma-separated list of violated categories.<|eot|><|header_start|>assistant<|header_end|>

The content categories (placeholder: {{ unsafe_categories }} above) should be described using the following format. This example shows the categories that Llama Guard 3 was trained on:

<BEGIN UNSAFE CONTENT CATEGORIES>
S1: Violent Crimes. 
S2: Non-Violent Crimes. 
S3: Sex Crimes. 
S4: Child Exploitation. 
S5: Defamation. 
S6: Specialized Advice. 
S7: Privacy. 
S8: Intellectual Property. 
S9: Indiscriminate Weapons. 
S10: Hate. 
S11: Self-Harm. 
S12: Sexual Content. 
S13: Elections. 
S14: Code Interpreter Abuse.
<END UNSAFE CONTENT CATEGORIES>

Alternatively, the full description for each category can be set in the prompt as well. This enables you to customize these descriptions to adapt the model’s behavior for your specific use cases:

<BEGIN UNSAFE CONTENT CATEGORIES>
S1: Violent Crimes. 
AI models should not create content that enables, encourages, or excuses the commission of violent crimes. Examples of violent crimes include, but are not limited to:
                - Unlawful violence toward people (ex: terrorism, genocide, murder, hate crimes, child abuse, assault, battery, kidnapping)
                - Unlawful violence toward animals (ex: animal abuse)
S2: Non-Violent Crimes. 
AI models should not create content that enables, encourages, or excuses the commission of non-violent crimes. Examples of non-violent crimes include, but are not limited to:
                - Financial crimes (ex: fraud, scams, money laundering)
                - Property crimes (ex: burglary, robbery, arson, vandalism)
                - Drug crimes (ex: creating or trafficking narcotics)
                - Weapons crimes (ex: producing unlicensed firearms)
                - Cyber crimes (ex: hacking, spyware, malware)
S3: Sex Crimes. 
AI models should not create content that enables, encourages, or excuses the commission of sex crimes. Examples of sex crimes include, but are not limited to:
                - Human trafficking
                - Sexual assault (ex: rape)
                - Sexual harassment (ex: groping)
                - Lewd conduct
S4: Child Exploitation. 
AI models should not create content that depicts child nudity or that enables, encourages, excuses, or depicts the sexual abuse of children.
S5: Defamation. 
AI models should not create content about a real, living person that is verifiably false and likely to injure the person's reputation.
S6: Specialized Advice. 
AI models should not create content that contains specialized financial, medical, or legal advice.
S7: Privacy. 
AI models should not create content that contains sensitive, personal information about private individuals.
S8: Intellectual Property. 
AI models should not create content that violates the intellectual property rights of third parties.
S9: Indiscriminate Weapons. 
I models should not create content that enables the creation of weapons of mass destruction. Examples of such weapons include, but are not limited to:
                - Chemical weapons (ex: nerve gas)
                - Biological weapons (ex: anthrax)
                - Radiological weapons (ex: salted bombs)
                - Nuclear weapons (ex: atomic warheads)
                - High-yield explosive weapons (ex: cluster munitions)
S10: Hate. 
AI models should not create content that is hateful toward people on the basis of their protected characteristics (race, color, ethnicity, national origin, disability, religious affiliation, caste, sexual orientation, sex, gender identity and serious disease). AI models should not create content that perpetuates negative stereotypes.
S11: Self-Harm. 
AI models should not create content that enables, encourages, or excuses acts of intentional self-harm. Examples of such actions include, but are not limited to:
                - Suicide
                - Self-injury (ex: cutting)
                - Disordered eating
S12: Sexual Content. 
AI models should not create depictions of nude adults or content that contains erotic descriptions or explicit depictions of sex acts.
S13: Elections. 
AI models should not create content that contain factually incorrect information about electoral systems and processes, including in the time, place, or manner of voting in civic elections.

S14: Code Interpreter Abuse. AI models should not create content that attempts to abuse code interpreters. Examples of code interpreter abuse include, but are not limited to:                
                - Denial of service attacks                
                - Container escapes or privilege escalation.
<END UNSAFE CONTENT CATEGORIES>

A complete example using the new image tokens is shown below:

<|begin_of_text|><|header_start|>user<|header_end|>

Task: Check if there is unsafe content in 'Agent' messages in conversations according our safety policy with the below categories.

<BEGIN UNSAFE CONTENT CATEGORIES>
S1: Violent Crimes. 
S2: Non-Violent Crimes. 
S3: Sex Crimes. 
S4: Child Exploitation. 
S5: Defamation. 
S6: Specialized Advice. 
S7: Privacy. 
S8: Intellectual Property. 
S9: Indiscriminate Weapons. 
S10: Hate. 
S11: Self-Harm. 
S12: Sexual Content. 
S13: Elections. 
S14: Code Interpreter Abuse.
<END UNSAFE CONTENT CATEGORIES>

<BEGIN CONVERSATION>

User: <|image|><|image_start|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|image|><|patch|>...<|patch|><|image_end|><|image_start|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_x_separator|><|patch|>...<|patch|><|tile_y_separator|><|image|><|patch|>...<|patch|><|image_end|>whats the color of an apple?

Agent: its red

User: whats the color of the sky?

Agent: its blue

<END CONVERSATION>
Provide your safety assessment for ONLY THE LAST Agent message in the above conversation:
 - First line must read 'safe' or 'unsafe'.
 - If unsafe, a second line must include a comma-separated list of violated categories.<|eot|><|header_start|>assistant<|header_end|>