Getting the models

Deploying Llama 3.1 405B: Partner Guides

Llama 3.1 405B is Meta's most advanced and capable model to date. To help you unlock its full potential, please refer to the partner guides below.

Our partner guides offer tailored support and expertise to ensure a seamless deployment process, enabling you to harness the features and capabilities of Llama 3.1 405B. Browse the following partner guides to explore their specific offerings and take the first step towards successful deployment.

AWS

Amazon Web Services (AWS) empowers you to harness the power of Meta Llama models, including the advanced 405B model, to build and scale generative AI applications effortlessly

Amazon Bedrock offer you a simple turnkey way to build and scale generative AI applications with Meta Llama models via a single API, with no infrastructure management required. Read the documentation and get started in the Amazon Bedrock console.

Amazon SageMaker JumpStart empowers you to leverage pretrained Llama models and MLOps controls with Amazon SageMaker features, and deploy models under your VPC controls, all accessible via SageMaker Studio or through the SageMaker Python SDK.

Azure

Microsoft is bringing Llama 3.1 suite of models to Azure AI Model Catalog in AI Studio. This addition enhances the catalog with advanced synthetic data generation and knowledge distillation capabilities. The Azure AI Model Catalog is a platform that allows you to discover, evaluate, fine-tune and deploy a wide range of AI models for prototyping, optimizing and operationalizing your gen AI applications.

Developers using Llama 3.1 models can also work seamlessly with other tools in Azure AI Studio, such as Azure AI Content Safety, Azure AI Search, and prompt flow to enhance ethical and effective AI practices. Learn more here with this documentation.

Databricks

Databricks Mosaic AI has partnered with Meta to support the Llama 3.1 model architecture across the full suite of products:

The full suite of Llama 3.1 models (8B, 70B, and 405B) is available using Foundation Model APIs. See Get started querying LLMs on Databricks. Production usage at scale is supported via provisioned throughput. See Provisioned throughput Foundation Model APIs on how to bring Llama to production.
Full customization support is available through Mosaic AI Model Training. See the getting started guide to start customizing with data from Unity Catalog

Databricks supports the full GenAI app development cycle, including through AI functions for large-scale application and AI Agents Framework and Evaluation for building production-ready agentic and RAG apps.

Dell

Developers can access the Llama 3.1 405B models by downloading an optimized container from the Dell Enterprise Hub on Hugging Face, specifically designed for on-premises deployment on Dell PowerEdge XE9680 infrastructure.

Fireworks AI

Fireworks AI offers the fastest and most efficient generative AI inference engine enabling developers to build and deploy applications using Llama 3.1 7B, 70B, and 405B models with unmatched latency and total cost of ownership.

Powered by the custom runtime, FireAttention, it can delivers 4x faster latency and 15x greater throughput than self-hosted alternatives, substantially lowering operational costs with up to 15x reductions in accelerator expenses. Fireworks AI is the preferred partner for enterprises seeking operational stability, scalability, and high-quality AI application development.

The full suite of Llama models are available for inference and fine-tuning. To begin using the Llama 3.1 7B, 70B, and 405B models on Fireworks AI, you only need to sign up for an account and start making API calls using the provided API Key.

Google Cloud

Leverage Google Cloud to effortlessly run Meta’s Llama models. Google Cloud offers a powerful AI ecosystem, featuring an extensive global infrastructure, custom-designed chips, advanced generative AI models, and a comprehensive development platform.

Vertex AI, Google Cloud's fully managed AI development platform, allows users to experiment with, customize, and deploy foundational models like Llama. Vertex provides various sizes of pre-trained, instruction-tuned Llama models in Vertex AI Model Garden, including Meta’s Llama 3.1 family of models. We offer Llama 3.1 405B as Model-as-a-Service within Vertex AI and all Llama 3.1 models in self-serve capacity. For details on fine-tuning and deploying Meta Llama models on Vertex AI, visit the website.

Groq

Experience Groq, the fastest AI inference for Llama 3.1, including 405B, 70B & 8B Instruct models. GroqCloud™, powered by LPU™ AI Inference Technology, provides record-setting inference speed to unlock a new class of AI applications and use cases. Enterprises benefit from not only fast AI inference, affordability, and energy efficiency, but the knowledge that Groq will continue to outperform with its unique software-first approach.

For developers it’s an easy three step process to get started with Groq. Simply replace your existing industry standard API key with a free Groq API key, set the base URL, and start building with Llama 3.1 on the Groq Developer Console.

NVIDIA

Experience the NVIDIA-optimized Llama 3.1 405B, Llama 3.1 70B, and Llama 3.1 8B NIM API endpoints with free NVIDIA cloud credits from ai.nvidia.com. Instructions to download and run the NIMs on your local and cloud environments are provided under the on each model page.

Access Llama models through the NVIDIA NGC catalog, where you can search, select, and download various versions, including base, instruct, and older models.

Search and download the models either from your browser, via WGET, or through the NGC CLI.
The models are licensed under the Llama 3.1 Community License Agreement.

Build custom Llama NIMs for enterprise applications with NVIDIA AI Foundry using NVIDIA NeMo, an end-to-end platform offering data curation, customization, model evaluation, retrieval, and guardrails, on NVIDIA DGX Cloud

IBM watsonx

Developers can access Llama 3.1 405B models on IBM watsonx an advanced platform designed for AI builders, integrating generative AI capabilities, foundation models, and traditional machine learning. It provides a comprehensive suite of tools that span the AI lifecycle, enabling users to tune models with their enterprise data. For detailed instructions, refer to the getting started guide and quick start tutorials.

Scale AI

Scale offers Llama 3.1 on Scale GenAI Platform to help enterprises build, evaluate, and deploy custom, production-ready Generative AI solutions that drive real business outcomes. With Scale GenAI Platform enterprises can customize and evaluate Llama 3.1 for enterprise use cases including financial services, legal, edTech, and more. Read more about how to evaluate Llama 3.1 in our blog.

Llama 3.1 is also now available to customers in Scale GenAI Platform to build custom GenAI apps, fine-tune, implement in RAG workflows, and evaluate against other models. To learn more, schedule a demo with Scale today.

Snowflake

Snowflake provides the option to instantly access and customize Meta’s collection of LLMs with serverless inference and fine-tuning using Snowflake Cortex AI as well as the option to run custom deployments using Snowpark Container Services. With LLMs running inside Snowflake, teams can use the same role-based access controls to secure and govern both models and data, ensuring cohesive security and governance.

To get started in minutes building enterprise-grade generative AI applications using state-of-the-art models in Snowflake, check out the get started guide for Llama in Cortex AI.

Together AI

Together AI offers the fastest fully-comprehensive developer platform for Llama models: with easy-to-use OpenAI-compatible APIs for Llama 3.1 and 3.2 models, as well as support for Llama Stack. The text-only models, which include 3B, 8B, 70B, and 405B, are optimized for natural language processing, offering solutions for various applications. The 3.2 vision+text models, such as the free 11B and 90B models, are specifically designed for multimodal tasks that combine text and image understanding. Together AI has made the Llama 3.2 11B model available for *free* development, experimentation, and personal applications.

Together Turbo and Together Lite endpoints for Llama models enable performance, quality, and price flexibility so enterprises do not have to compromise. Together AI applies cutting-edge research and innovations such as FlashAttention-3, enabling Together Inference Engine 2.0 to deliver up to 15x throughput improvement, while lowering operational costs.

Enterprises benefit from flexible deployment options, including Serverless or Dedicated Endpoints on Together Cloud or their own VPC environments using AWS or other cloud providers. Organizations retain full ownership of their fine-tuned Llama models and customer data, ensuring security and control at every stage of AI development.

Was this page helpful?

Yes

On this page