Llama Stack

Quick Start | Documentation | Colab Notebook | Discord

• Overview
• API Providers & Distributions
• Resources
• Community

Overview

Llama Stack defines and standardizes the core building blocks that simplify AI application development. It provides a unified set of APIs with implementations from leading service providers. Get started instantly:

curl -LsSf https://github.com/llamastack/llama-stack/raw/main/scripts/install.sh | bash

• Unified API layer for Inference, RAG, Agents, Tools, Safety, Evals.
• Plugin architecture supporting local development, on-premises, cloud, and mobile environments.
• Prepackaged verified distributions for a one-stop solution in any environment.
• Multiple developer interfaces — CLI and SDKs for Python, Typescript, iOS, and Android.
• Standalone applications as examples for production-grade AI apps with Llama Stack.

API Providers & Distributions

Here is a list of the various API providers and available distributions. See the full list for details, including External Providers.

API Provider	Environments	Agents	Inference	VectorIO	Safety	Eval	DatasetIO
SambaNova	Hosted		✅		✅
Cerebras	Hosted		✅
Fireworks	Hosted	✅	✅	✅
AWS Bedrock	Hosted		✅		✅
Together	Hosted	✅	✅		✅
Groq	Hosted		✅
Ollama	Single Node		✅
TGI	Hosted/Single Node		✅
NVIDIA NIM	Hosted/Single Node		✅		✅
ChromaDB	Hosted/Single Node			✅
Milvus	Hosted/Single Node			✅
Qdrant	Hosted/Single Node			✅
Weaviate	Hosted/Single Node			✅
SQLite-vec	Single Node			✅
PG Vector	Single Node			✅
PyTorch ExecuTorch	On-device iOS	✅	✅
vLLM	Single Node		✅
OpenAI	Hosted		✅
Anthropic	Hosted		✅
Gemini	Hosted		✅
WatsonX	Hosted		✅
HuggingFace	Single Node						✅
NVIDIA NEMO	Hosted		✅	✅		✅	✅
NVIDIA	Hosted					✅	✅
Infinispan	Single Node			✅

A distribution (or "distro") is a pre-configured provider bundle for a specific deployment scenario — start with local Ollama and transition to production without changing your application code.

Distribution	Llama Stack Docker	Start This Distribution
Starter Distribution	llamastack/distribution-starter	Guide
Starter Distribution GPU	llamastack/distribution-starter-cpu	Guide
PostgreSQL	llamastack/distribution-postgres-demo	N/A
Dell	llamastack/distribution-dell	Guide

For the full list including Docker images see the Distributions Overview.

Resources

Full docs at llamastack.github.io/docs. Example apps at llama-stack-apps.

• Quick Start — start a Llama Stack server
• Getting Started Notebook — text and vision inference walkthrough
• Zero-to-Hero Guide — key components with code samples
• Server CLI Reference | Client CLI Reference
• Contributing | Adding a new API Provider | Release Process

Client SDKs — connect to a Llama Stack server in your preferred language:

Language	Client SDK	Package
Python	llama-stack-client-python
Swift	llama-stack-client-swift
Typescript	llama-stack-client-typescript
Kotlin	llama-stack-client-kotlin

Note: We are considering a transition from Stainless to OpenAPI Generator for SDK generation (#4609). The client-sdks/openapi/ directory contains the new tooling for local SDK generation.

Community

We hold regular community calls every Thursday at 09:00 AM PST — see the Community Event on Discord for details.

Thanks to all our amazing contributors!