LLama

Llama 4 released April 5, 2025 with two production models: Scout (17B active parameters, 16 experts, 10M token context window, pretrained on 40T tokens — best for long-context and knowledge retrieval) and Maverick (17B active parameters, 128 experts, pretrained on 22T tokens — best multimodal performance, beats GPT-4o and Gemini 2.0 Flash on benchmarks). Behemoth, the 400B+ parameter teacher model, was still in training at the 2025 announcement. All Llama 4 models use Mixture-of-Experts (MoE) architecture that activates only a fraction of parameters per token, making inference more efficient than equivalent dense models.

Llama 4 uses Meta's custom Llama license, which permits commercial use for most companies. Key restrictions: companies with over 700 million monthly active users require a special license directly from Meta; EU-domiciled users and companies are currently prohibited from using or distributing Llama 4 models (this may change as Meta navigates EU AI Act compliance). For most commercial applications outside the EU, Llama 4 is free to use, fine-tune, and deploy. Always review the current license at llama.com before building commercial products.

vLLM is the production standard for Llama serving — continuous batching, PagedAttention for efficient KV cache management, tensor parallelism across GPUs, and an OpenAI-compatible API. For smaller deployments, Ollama provides one-command model management with REST API. Text Generation Inference (TGI) from Hugging Face is a vLLM alternative with strong streaming support. We recommend vLLM for throughput-critical production, Ollama for developer environments and edge hardware. Hardware minimum for Llama 4 Scout: 4× A100 80GB for FP16, 2× A100 for GPTQ 4-bit.

The break-even point depends on GPU rental costs vs OpenAI pricing. Rough estimates: at 50M tokens/month, GPT-4o-mini API (~$30) vs a shared GPU instance with Llama (infrastructure cost per month amortized). At 500M tokens/month, the math strongly favors Llama — GPT-4o at $2.50/1M input would cost $1,250/month in input tokens alone, while a single A100 instance running Llama can handle that volume. We model the exact break-even for your specific token volume, model tier requirements, and latency SLAs before recommending self-hosted Llama.

LoRA (Low-Rank Adaptation) adds small trainable matrices to Llama's attention layers — training only ~0.1% of the model's parameters while keeping base weights frozen. QLoRA combines LoRA with 4-bit quantization of the base model, enabling fine-tuning on a single A100 80GB GPU (vs multiple GPUs for full fine-tuning). Process: prepare domain data as JSONL instruction-response pairs, configure PEFT with rank and alpha hyperparameters, train on Hugging Face Trainer or Axolotl, evaluate on held-out test set, and either deploy as a separate adapter or merge weights. We run fine-tuning jobs and provide evaluation reports before production deployment.

Llama 4 Scout (active 17B params, FP16): 2× A100 80GB minimum for comfortable serving, 1× A100 with GPTQ 4-bit quantization. Llama 4 Maverick (active 17B params but larger total model, FP16): 4× A100 80GB recommended. For QLoRA fine-tuning: 1× A100 80GB is sufficient for Scout. Quantized versions (GGUF Q4) run on consumer hardware via Ollama: Mac M3 Pro (18GB unified memory) runs Scout at ~20-40 token/sec. Cloud GPU options: Lambda Labs A100 instances ($1.10/hour), CoreWeave, AWS P4d. We size GPU requirements and cost-model before infrastructure provisioning.

Llama Guard is Meta's open-source safety classifier for LLM outputs — a fine-tuned Llama model that classifies whether a response is safe or violates categories including violence, sexual content, criminal planning, hate speech, and privacy violations. It's used as a production safety layer: run Llama 4 for generation, then run Llama Guard to screen the output before returning it to users. We integrate Llama Guard into vLLM serving stacks as a post-generation filter for applications with safety requirements.

Yes — Llama 4 Scout and Maverick are natively multimodal. They accept image+text prompts and return text responses. Use cases: document scanning with embedded images, product photo analysis, form parsing from scanned documents, visual Q&A, and invoice OCR with layout understanding. Via vLLM, multimodal Llama 4 inference requires providing images in base64 or URL format alongside the text prompt. The vision capabilities are competitive with GPT-4o on MMMU benchmarks for Maverick.

Private Llama RAG architecture: (1) chunk documents, (2) generate embeddings (Llama's embedding model, nomic-embed-text, or sentence-transformers — all open-source), (3) store in pgvector, Qdrant, or Chroma, (4) retrieve top-k chunks at query time, (5) pass retrieved context to vLLM-served Llama for generation. The entire pipeline — embedding, vector storage, and generation — runs on your infrastructure with no external API calls. LangChain and LlamaIndex both have vLLM integrations via the OpenAI-compatible API. We build private RAG systems as a standard Llama engagement.

We provide Llama managed support covering model update evaluation (when Llama 5 or new Llama 4 variants release, we test against your evaluation set before recommending upgrade), vLLM performance tuning as traffic patterns change, quantization updates for new hardware, fine-tuning data refresh cycles, and infrastructure cost optimization. We also monitor the Llama license status for EU restriction changes and evaluate alternative open-source models (Mistral, Gemma, Phi) when use case constraints require.