Llama

Troubleshooting & FAQ

Get answers to Llama's most frequently asked questions

General

What if I want to access Llama models but I’m not sure if my use is permitted under the Llama Community License?

If you are a researcher, academic institution, government agency, government partner, or other entity with a Llama use case that is currently prohibited by the Llama Community License or Acceptable Use Policy, or requires additional clarification, please contact llamamodels@meta.com with a detailed request. We will consider requests on a case-by-case basis.

Can I use the output of the models to improve the Llama family of models, even though I cannot use them for other LLMs?

For Llama 2 and Llama 3, it's correct that the license restricts using any part of the Llama models, including the response outputs to train another AI model (LLM or otherwise). For Llama 3.1 however, this is allowed provided you as the developer provide the correct attribution. See the license for more information.

Do Llama models support other languages outside of English?

For Llama 2 and Llama 3, the models were primarily trained on English with some additional data from other languages. We do not expect the same level of performance in these languages as in English. Llama 3.1 however supports additional languages and is considered multilingual. See the Llama 3.1 model card for more information.

If I’m a developer/business, how can I access the models?

Details on how to access the models are available on our website link. Please note that the models are made available subject to the applicable Community License Agreement and Acceptable Use Policy. You should also take advantage of the best practices and considerations set forth in the applicable Responsible Use Guide.

Models are available through multiple sources but the place to start is at https://llama.meta.com/.

Models are also being hosted by Amazon Web Services, Databricks, Hugging Face, Groq and many others. See the Llama website for more information.

Can anyone access Llama models? What are the terms?

Llama models are broadly available to developers and licensees through a variety of hosting providers and on the Meta website and licensed under the applicable Llama Community License Agreement, which provides a permissive license to the models along with certain restrictions to help ensure that the models are being used responsibly. Hosting providers may have additional terms applicable to their services.

What are the hardware SKU requirements for deploying these models?

Hardware requirements vary based on latency, throughput and cost constraints. For good latency, we split models across multiple GPUs with tensor parallelism in a machine with NVIDIA A100s or H100s. But TPUs, other types of GPUs, or even commodity hardware can also be used to deploy these models (e.g.llama cpp, MLC LLM).

Do Llama models provide traditional autoregressive text completion?

Llama models are auto-regressive language models, built on the transformer architecture. The core language models function by taking a sequence of words as input and predicting the next word, recursively generating text.

Do Llama models support adjusting sampling temperature or top-p threshold via request parameters?

The model itself supports these parameters, but whether they are exposed or not depends on implementation.

What is the most effective RAG method paired with Llama models?

There are many ways to use RAG with Llama. See the developer documentation page for reference implementations.

Should we start training with the base or instruct/chat model?

This depends on your application. The Llama pre-trained models were trained for general large language applications, whereas the Llama instruct or chat models were fine tuned for dialogue specific uses like chat bots.

How can I fine tune the Llama models?

You can find examples on how to fine tune the Llama models in the Llama Recipes repository. See also the developer documentation page for reference implementations.

Am I allowed to develop derivative models through fine-tuning based on Llama models for languages other than those officially supported? Is this a violation of the acceptable use policy?

Developers may fine-tune Llama models for languages beyond English or other officially supported languages (e.g. Llama 3.1 supports 8 languages) provided they comply with the applicable Llama Community License Agreement and the Acceptable Use Policy.

Legal

Are Llama models open source? What is the exact license these models are published under?

Llama models are licensed under a bespoke commercial license that balances open access to the models with responsibility and protections in place to help address potential misuse. Our license allows for broad commercial use, as well as for developers to create and redistribute additional work on top of Llama models. We want to enable more innovation in both research and commercial use cases, but believe in taking a responsible approach to releasing AI technologies. For more details, our license can be found here.

Are there examples that help licensees better understand how “MAU” is defined?

“MAU” means “monthly active users” that access or use your (and your affiliates’) products and services. Examples include users accessing an internet-based service and monthly users/customers of licensee’s hardware devices.

Does the Critical Infrastructure restriction in the acceptable use policy prevent companies who have special critical infrastructure certification (e.g., a registered operator of “critical infrastructure” under the German BSI Act) from using Llama?

No, such companies are not prohibited when their usage of Llama is not related to the operation of critical infrastructure. Llama, however, may not be used in the operation of critical infrastructure by any company, regardless of government certifications.