Introduction

LlamaIndex started in 2022 as an indexing library for RAG. In 2026 that framing is obsolete. The 0.14.x line is an event-driven workflow framework with three agent classes, a hosted platform (LlamaCloud + LlamaParse), a production runtime (llama-agents + llamactl), and 300+ integration packages spanning readers, LLMs, embeddings, vector stores, and tools. It's still the sharpest tool for "documents are the hard part" workloads, and it's now a viable general agent framework alongside LangGraph.

This guide is the definitive reference for LlamaIndex as it exists today. It covers the package landscape, the Document/Node data model, ingestion pipelines with IngestionCache, the five index types, query engines and retrievers, response synthesizers, the event-driven Workflows API, the three agent classes (FunctionAgent, ReActAgent, CodeActAgent), multi-agent AgentWorkflow, and LlamaCloud/LlamaParse deployment.

Who this guide is for

The guide assumes you have:

• Built at least one working RAG or agent app — LlamaIndex, LangChain, or a hand-rolled pipeline
• Hit a case where the retrieval quality matters more than the agent loop
• Hit at least one ImportError, deprecation warning, or version mismatch that sent you searching

If that's you, read this end-to-end once to build a full map of the framework surface; return to specific chapters when you hit a related error. Each chapter links to the specific error-fix pages that answer narrower questions with tested code.

What LlamaIndex is (and isn't) in 2026

LlamaIndex is a data framework whose center of gravity is the document layer — Reader, Document, Node, IngestionPipeline, Index, Retriever, QueryEngine — plus an event-driven Workflows API and agent classes built on top. Where LangChain wraps provider SDKs and LangGraph is a stateful orchestration runtime, LlamaIndex is opinionated about the data path from a raw PDF to a grounded answer.

Practical implication: pick LlamaIndex when documents are the hard part (heterogeneous parsing, hierarchical indexing, hybrid retrieval, citation extraction). Pick LangGraph when control flow is the hard part (branching, multi-agent, long HITL). Pick the raw provider SDK when neither of those is your bottleneck. Many production teams run both — LlamaIndex for the retrieval layer, LangGraph (or a hand-rolled loop) for the agent orchestration.

The four things that changed in 0.11-0.14

1. Workflows replaced Query Pipelines. QueryPipeline was deprecated in 0.11 in favor of the event-driven Workflows API. Same underlying concept (chain steps together) with a cleaner mental model — steps decorated with @step, accepting and emitting typed Event objects, coordinated by a Context. Every non-trivial LlamaIndex app in 2026 uses Workflows.

2. Agents live on top of Workflows. FunctionAgent, ReActAgent, and CodeActAgent all extend BaseWorkflowAgent. AgentWorkflow composes multiple agents for handoff patterns. This means "the agent" and "the workflow" are the same runtime — you don't switch abstractions when moving from a simple agent to a complex multi-actor system.

3. The monolithic llama_index package is gone. Since the 0.10 split, everything installs from llama-index-core plus specific integration packages (llama-index-llms-openai, llama-index-vector-stores-postgres, etc.). The old llama_index monolith conflicts with the new packages — teams migrating from pre-0.10 code hit this immediately.

4. LlamaCloud + LlamaParse + llama-agents form a hosted stack. LlamaParse for messy enterprise document parsing (PDFs with tables, invoices, forms). LlamaCloud for hosted indexing pipelines. llama-agents for running Workflows as HTTP services. All optional — the framework works fully self-hosted — but the hosted layer is where the commercial energy sits in 2026.

What's not in this guide

• Deep LangChain / LangGraph coverage — see the LangChain pillar and LangGraph pillar
• Provider-specific model quirks — see the model pillars (Claude, OpenAI, Gemini)
• Vector database internals — LlamaIndex wraps them; for DB-side tuning see the vendor's docs
• Prompt engineering — framework-level content

The 2026 package landscape

Since the 0.10 split, LlamaIndex is not one package — it's llama-index-core plus a matrix of 300+ integration packages you install à la carte. Get the install pattern right and the framework works cleanly. Install the wrong combination — or leave the legacy monolithic llama_index package around — and imports break in confusing ways.

The five package families

Meta package: llama-index (no -core) installs a common starter set — core, OpenAI LLM, OpenAI embeddings, file reader. Fine for prototypes. For production, install llama-index-core plus the specific integrations you need. See llama-index-core vs integration import.

The legacy llama_index problem

Before the 0.10 split, everything lived in one llama_index package (note the underscore). That package still exists on PyPI at old versions and conflicts with the new packages. If you install both, imports get non-deterministic — some resolve to legacy, some to modern.

# Fix: uninstall the legacy package cleanly first
pip uninstall llama_index

# Then install the modern packages
pip install llama-index-core llama-index-llms-openai llama-index-embeddings-openai

Symptom: cryptic errors about missing modules that "should exist," or the same class imported from two paths behaving differently. See legacy monolith uninstall.

A working install for a real RAG app

# requirements.txt — pin exactly
llama-index-core==0.14.5
llama-index-llms-openai==0.4.2
llama-index-llms-anthropic==0.7.1
llama-index-embeddings-openai==0.3.1
llama-index-vector-stores-postgres==0.4.5
llama-index-readers-file==0.4.3
llama-index-postprocessor-cohere-rerank==0.4.1

# Add these when you use them
# llama-index-tools-tavily-research==0.4.0
# llama-parse==0.6.0
# llama-agents==0.2.0

Pin exact versions. Integration packages release independently and interface changes between minor versions do happen — LlamaIndex 2026 is more mature than in the 0.10 era but the risk isn't zero.

Import patterns after the split

Pattern: all core abstractions under llama_index.core.*; concrete integrations under llama_index.<family>.<name>. If an import at the old llama_index.<something> path fails, it moved to a specific integration package. See core vs integration import.

Settings — the global config

from llama_index.core import Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding

Settings.llm = OpenAI(model="gpt-5.6-terra", temperature=0)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-large")
Settings.chunk_size = 512
Settings.chunk_overlap = 50

Settings is the global configuration singleton. Set once at app startup; most LlamaIndex classes read from it if not passed an explicit LLM or embed_model. Prevents having to thread the LLM through every construction call. In multi-model apps, pass models explicitly to specific components instead of relying on Settings.

Documents, Nodes & ingestion

The data layer is LlamaIndex's home turf. Documents are raw content with metadata; Nodes are chunks of Documents that flow through indices and retrievers. Ingestion pipelines transform Documents into Nodes with configurable steps (chunking, extraction, embedding), optionally cached to avoid re-embedding unchanged content. Get this layer right and everything downstream becomes easier.

Documents — the input primitive

from llama_index.core import Document

doc = Document(
    text="The full text of the source document...",
    metadata={
        "source": "policies/refund-2026.pdf",
        "page": 3,
        "author": "Legal Team",
        "last_modified": "2026-08-01",
    },
    id_="policies-refund-2026-p3",   # stable ID for updates
)

Every Reader returns a list of Documents. Metadata flows through to Nodes and appears in retrieval results — critical for citations, filtering, and debugging. Set a stable id_ when you'll re-ingest updated content; the IngestionCache uses this to detect changes.

SimpleDirectoryReader — the workhorse reader

from llama_index.core import SimpleDirectoryReader

# Load a directory of files
documents = SimpleDirectoryReader(
    input_dir="./data",
    recursive=True,
    file_extractor={  # override extractor per file type
        ".pdf": PyMuPDFReader(),
        ".docx": DocxReader(),
    },
    exclude_hidden=True,
).load_data()

Handles common types (.txt, .md, .pdf, .docx, .csv, .json, .html) out of the box. For extended file support, install llama-index-readers-file. For messy enterprise documents (PDFs with tables, invoices), use LlamaParse (chapter 11). See file type not supported.

The 300+ readers in LlamaHub

Beyond SimpleDirectoryReader, the llama-index-readers-* family covers most enterprise data sources:

• llama-index-readers-notion, llama-index-readers-confluence, llama-index-readers-slack
• llama-index-readers-github, llama-index-readers-jira, llama-index-readers-google
• llama-index-readers-s3, llama-index-readers-web, llama-index-readers-database
• 200+ more via LlamaHub registry

Nodes — the chunk primitive

A Node is a chunk of a Document that flows through indices, retrievers, and query engines. Contains the chunk text, inherited metadata, embedding (once computed), and relationships to sibling/parent/child nodes:

from llama_index.core.node_parser import SentenceSplitter

parser = SentenceSplitter(chunk_size=512, chunk_overlap=50)
nodes = parser.get_nodes_from_documents(documents)

for node in nodes:
    print(node.text[:100])
    print(node.metadata)         # inherited from Document plus parser additions
    print(node.node_id)          # unique ID
    print(node.relationships)    # PREVIOUS, NEXT, PARENT, CHILD refs

Node parsers vary in strategy:

chunk_size and chunk_overlap are the two most-tuned parameters. Too small (128 tokens): loses context, retrieval brings back tiny fragments. Too large (2000+ tokens): dilutes semantic similarity, retrieval loses precision. Common starting point: 512 tokens with 50 overlap. See chunk_size mismatch.

Ingestion pipelines

from llama_index.core.ingestion import IngestionPipeline, IngestionCache
from llama_index.core.node_parser import SentenceSplitter
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.core.extractors import (
    TitleExtractor, QuestionsAnsweredExtractor, SummaryExtractor,
)

pipeline = IngestionPipeline(
    transformations=[
        SentenceSplitter(chunk_size=512, chunk_overlap=50),
        TitleExtractor(nodes=5),
        QuestionsAnsweredExtractor(questions=3),
        SummaryExtractor(),
        OpenAIEmbedding(model="text-embedding-3-large"),
    ],
    cache=IngestionCache(cache=SimpleCache(), collection="my-app-v1"),
)

nodes = pipeline.run(documents=docs)

Transformations run in order — parse into nodes, then extractors add metadata (title, questions the chunk answers, summary), then embedding. Every stage can be cached. Order matters: put cheap transformations early, expensive ones (embedding, LLM-based extraction) late so cache invalidation on cheap changes doesn't force re-running expensive stages. See transformation order.

IngestionCache — the reingest saver

IngestionCache hashes each transformation's input and output. On re-run:

• If input hasn't changed and cache has a hit, skip the transformation and use cached output
• If input changed, run the transformation and update cache
• Massive savings on re-embedding when only a fraction of documents changed

Common failure: change a transformation's parameters (bumped chunk_size, switched embedding model) but keep the same cache collection name. Cache hits still return old-parameter results — bug. Fix: version the cache collection when transformations change (collection="my-app-v2"). See IngestionCache hash mismatch.

Metadata handling — the citation foundation

Metadata on Documents flows to Nodes automatically. But not every field is visible to the LLM at query time — LlamaIndex separates metadata into two visibility categories:

• excluded_llm_metadata_keys — metadata keys hidden from the LLM prompt (still available for filtering and citations)
• excluded_embed_metadata_keys — metadata keys not included in the embedded text (affects similarity search)

Get these wrong and either the LLM sees noise (source paths, timestamps) or you lose useful signal from the embedding. See metadata not passed to Nodes.

Every LlamaIndex error category

LlamaIndex errors span ten categories, most tied to the framework's data-layer opinions (packages, documents, indices, workflows) rather than to underlying models. This chapter maps each category with links to specific error-fix pages.

1. Package & import errors

Fires at: Module load.
Common causes: Legacy llama_index monolithic package installed alongside modern packages; integration package not installed for a specific LLM/embedder/vector store; import from old paths (llama_index.readers.notion instead of llama_index.readers.notion.NotionPageReader); Python version below 3.9.

Related pages: core vs integration import · legacy monolith uninstall

2. Document & Node errors

Fires at: Reader.load_data() or NodeParser.get_nodes_from_documents().
Common causes: SimpleDirectoryReader file type not registered; Document metadata missing keys downstream code expects; chunk_size / chunk_overlap combination invalid (overlap ≥ size); NodeParser producing empty node lists on very short documents.

Chapter 3 covers Document and Node patterns. See file type not supported and chunk_size mismatch.

3. Ingestion pipeline errors

Fires at: IngestionPipeline.run() or during transformation.
Common causes: Transformation order wrong (embedding before splitter, extractors after embedding); IngestionCache hash collision when transformations change but cache collection name doesn't; embedding batch size too large timing out the provider API; missing async wrapper for async transformations.

See transformation order · cache hash mismatch · embedding batch timeout

4. Index errors

Fires at: Index construction or loading from persistence.
Common causes: VectorStoreIndex embedding dimension mismatch with existing vector store; SummaryIndex used on huge corpora (fits everything in prompt — token overflow); ComposableGraph child indices not persisted or not loadable; wrong index class for the workload.

Chapter 5 walks the index types. See dimension mismatch and index type choice.

5. Query engine & retriever errors

Fires at: QueryEngine.query() or Retriever.retrieve().
Common causes: similarity_top_k too low (missing relevant nodes); hybrid retriever setup incomplete (BM25 integration missing); response_mode incompatible with streaming; retriever returning empty results due to metadata filter mismatch.

Chapter 6 covers query engines. See similarity_top_k too low.

6. Response synthesizer errors

Fires at: Response synthesis stage.
Common causes: response_mode="tree_summarize" creating context overflow on very large retrieved sets; refine mode with too many nodes hitting rate limits from repeated calls; postprocessor reranker adding significant latency.

Chapter 7 covers synthesizers and postprocessors. See tree_summarize context overflow.

7. Workflow errors

Fires at: Workflow.run() or step execution.
Common causes: @step event type mismatch — step never fires because no event of the expected type is emitted; Context not passed as a step parameter; StopEvent never emitted (workflow hangs); async event loop conflict in notebooks.

Chapter 8 covers Workflows. See event type mismatch and StopEvent never emitted.

8. Agent errors

Fires at: Agent construction or agent.run().
Common causes: FunctionAgent used with a model that doesn't support native tool calling; ReActAgent output parser failing to parse the model's format; Pydantic v1 tool schema on a Pydantic v2 environment; CodeActAgent sandbox misconfiguration.

Chapter 9 covers agent classes. See FunctionAgent vs ReActAgent.

9. Multi-agent errors

Fires at: AgentWorkflow construction or handoff.
Common causes: root_agent not set (workflow doesn't know where to start); handoff tools not registered on the delegating agents; agents share state in ways that conflict.

Chapter 10 covers AgentWorkflow. See root_agent not set.

10. LlamaCloud, LlamaParse & deployment errors

Fires at: LlamaParse API calls or WorkflowServer deploy.
Common causes: LLAMA_CLOUD_API_KEY not set; LlamaCloud index sync errors from schema drift; WorkflowServer failing to import your workflow module; llamactl deploy configuration missing.

Chapter 11 covers the hosted stack. See LlamaParse API key.

Indices — Vector, Summary, Tree, KG, Composite

An index in LlamaIndex organizes Nodes for efficient retrieval. Five main types cover different retrieval patterns and corpus shapes. Picking the right one for your workload is the difference between a RAG app that scales and one that runs out of budget or misses everything relevant.

VectorStoreIndex — the default

from llama_index.core import VectorStoreIndex, StorageContext
from llama_index.vector_stores.postgres import PGVectorStore

vs = PGVectorStore.from_params(
    database="mydb",
    host="localhost",
    port=5432,
    user="postgres",
    password=os.environ["PG_PASSWORD"],
    table_name="my_embeddings",
    embed_dim=3072,   # ← must match your embedding model's dimension
)

storage = StorageContext.from_defaults(vector_store=vs)
index = VectorStoreIndex.from_documents(documents, storage_context=storage)

# Query
query_engine = index.as_query_engine()
response = query_engine.query("what is the refund policy?")

Embed nodes, store in a vector database, retrieve by similarity. The workhorse of RAG. Common failure: embedding dimension of the model doesn't match the vector store's embed_dim parameter — writes fail or (worse) silently corrupt the index. See dimension mismatch.

The vector store menu

LlamaIndex integrates with most major vector databases via llama-index-vector-stores-* packages:

• Postgres (PGVectorStore) — most production defaults; transactional, mature
• Pinecone, Weaviate, Qdrant, Milvus — managed or self-hosted vector-native DBs
• Chroma, LanceDB — lightweight embedded stores
• MongoDB Atlas Vector Search, Elasticsearch — hybrid DBs adding vector support
• Redis — fast, useful for cache-tier retrieval
• SimpleVectorStore — in-memory default, dev only

SummaryIndex — the small-corpus tool

from llama_index.core import SummaryIndex

# Stores nodes as a flat sequential list. Every query scans everything.
index = SummaryIndex.from_documents(documents)

# Every query iterates the full list
response = index.as_query_engine().query("summarize this")

SummaryIndex doesn't do similarity retrieval — every query touches every node. Great for:

• Small corpora (under ~50 nodes) where similarity search adds no value
• Summarization workloads where the LLM should see all content
• Cases where retrieval quality matters less than completeness

Terrible for: anything larger than what fits in your LLM's context window. Context overflow is the typical failure mode. See index type choice.

TreeIndex — hierarchical summarization

from llama_index.core import TreeIndex

# Builds a tree — leaves are nodes, internal nodes are LLM-generated summaries
index = TreeIndex.from_documents(documents, num_children=10)

Groups nodes into subtrees, generates a summary for each subtree, repeats up the tree. Query walks top-down or bottom-up. Trade-off: high ingestion cost (LLM calls to build the tree), lower query cost than SummaryIndex, retrieval quality can lag VectorStoreIndex.

Use case: hierarchical corpora with strong topical structure where the tree matches the document organization (book chapters, org docs by department).

KnowledgeGraphIndex — entity/relation retrieval

from llama_index.core import KnowledgeGraphIndex
from llama_index.graph_stores.neo4j import Neo4jGraphStore

graph_store = Neo4jGraphStore(
    username="neo4j", password="...", url="bolt://localhost:7687",
)

index = KnowledgeGraphIndex.from_documents(
    documents,
    graph_store=graph_store,
    max_triplets_per_chunk=10,
)

Extracts entities and relationships via LLM, builds a knowledge graph, retrieves by traversing the graph. Powerful for structured domains (legal, biomedical, finance) where semantic similarity misses relationship-based queries ("companies A and B are subsidiaries of the same parent").

Trade-off: expensive to build (LLM per chunk for extraction), retrieval quality depends on extraction quality. Overkill for most workloads; game-changing when the domain fits.

ComposableGraph — combining indices

from llama_index.core import ComposableGraph, SummaryIndex, VectorStoreIndex

# Child indices — one per doc collection
finance_index = VectorStoreIndex.from_documents(finance_docs)
legal_index = VectorStoreIndex.from_documents(legal_docs)
hr_index = VectorStoreIndex.from_documents(hr_docs)

# Compose: parent SummaryIndex routes queries to appropriate child
graph = ComposableGraph.from_indices(
    SummaryIndex,
    children_indices=[finance_index, legal_index, hr_index],
    index_summaries=[
        "Financial policies and procedures",
        "Legal contracts and compliance",
        "HR policies and employee handbook",
    ],
)

Multi-source RAG pattern: separate indices per topic, parent index routes based on query semantics. Cleaner than one giant index for large multi-domain corpora. Common gotcha: child indices must be individually persisted or the graph fails to reload. See child index not loaded.

Decision matrix — which index for what

Query engines & retrievers

A retriever fetches nodes given a query. A query engine wraps a retriever plus a response synthesizer to produce a final answer. Understanding the separation matters: for RAG-in-agent-tools you often want just the retriever; for user-facing Q&A you want the full query engine.

The layers

1. Retriever — takes query, returns list of Nodes
2. Node postprocessors — filter/rerank the retrieved nodes (optional)
3. Response synthesizer — takes nodes + query, generates the final answer
4. QueryEngine — bundles the three above into one call

Getting a retriever from an index

retriever = index.as_retriever(
    similarity_top_k=10,   # how many nodes to fetch
    vector_store_query_mode="hybrid",   # "default", "hybrid", "sparse", "mmr"
    alpha=0.5,   # hybrid weight (0=BM25, 1=vector)
    filters=MetadataFilters(
        filters=[MetadataFilter(key="source", value="policies", operator="contains")],
    ),
)

nodes = retriever.retrieve("what is the refund policy?")
for node in nodes:
    print(node.score, node.text[:200])

similarity_top_k is the most-tuned parameter. Too low (top 2-3): miss relevant nodes. Too high (top 50): drown the LLM in irrelevant context, waste tokens. Common starting point: top 5-10. Tune against your eval set. See similarity_top_k too low.

Hybrid retrieval

Pure vector search misses keyword matches (product names, error codes, IDs). Pure BM25 misses semantic paraphrases. Hybrid combines both. Requires a BM25-capable integration:

from llama_index.retrievers.bm25 import BM25Retriever
from llama_index.core.retrievers import QueryFusionRetriever

vector_retriever = index.as_retriever(similarity_top_k=10)
bm25_retriever = BM25Retriever.from_defaults(
    docstore=index.docstore, similarity_top_k=10,
)

hybrid = QueryFusionRetriever(
    retrievers=[vector_retriever, bm25_retriever],
    similarity_top_k=10,
    num_queries=1,
    mode="reciprocal_rerank",   # or "relative_score"
)

nodes = hybrid.retrieve("refund policy 2026")

QueryFusionRetriever combines results via reciprocal rank fusion (RRF) — merges rankings from multiple retrievers into one final ranking. Install llama-index-retrievers-bm25. See BM25 integration package.

Metadata filters

from llama_index.core.vector_stores import (
    MetadataFilters, MetadataFilter, FilterOperator, FilterCondition,
)

filters = MetadataFilters(
    filters=[
        MetadataFilter(key="year", value=2026, operator=FilterOperator.EQ),
        MetadataFilter(key="type", value="policy", operator=FilterOperator.EQ),
        MetadataFilter(key="department", value=["legal", "compliance"], operator=FilterOperator.IN),
    ],
    condition=FilterCondition.AND,
)

retriever = index.as_retriever(similarity_top_k=10, filters=filters)

Filter operators vary by vector store — Postgres and MongoDB support most; simpler stores support only EQ. Consult your vector store's docs. Common trap: retriever returns empty results because a metadata filter matches no nodes (typo, wrong casing, missing metadata field).

Query engines — retriever + synthesizer

from llama_index.core.query_engine import RetrieverQueryEngine
from llama_index.core.response_synthesizers import get_response_synthesizer

synthesizer = get_response_synthesizer(response_mode="compact")

query_engine = RetrieverQueryEngine(
    retriever=hybrid,
    response_synthesizer=synthesizer,
    node_postprocessors=[reranker],
)

response = query_engine.query("what is the refund policy?")
print(response.response)
for node in response.source_nodes:
    print(f"  [{node.score:.2f}] {node.metadata['source']}")

The response object has .response (the generated answer) and .source_nodes (the nodes used to generate it) — critical for citation UIs.

Streaming query responses

query_engine = index.as_query_engine(streaming=True, response_mode="compact")
response = query_engine.query("...")

for text in response.response_gen:
    print(text, end="", flush=True)

Not every response_mode streams. Compact and refine modes stream cleanly; tree_summarize and accumulate don't (they aggregate). If streaming=True with an incompatible mode, you get either an error or a non-streaming response. See response_mode vs streaming.

Sub-question query engine (for complex multi-hop)

from llama_index.core.query_engine import SubQuestionQueryEngine
from llama_index.core.tools import QueryEngineTool, ToolMetadata

tools = [
    QueryEngineTool(
        query_engine=finance_engine,
        metadata=ToolMetadata(name="finance", description="Financial documents"),
    ),
    QueryEngineTool(
        query_engine=legal_engine,
        metadata=ToolMetadata(name="legal", description="Legal contracts"),
    ),
]

sub_engine = SubQuestionQueryEngine.from_defaults(query_engine_tools=tools)
response = sub_engine.query("Compare our refund policy with our supplier contracts")
# LLM decomposes into sub-questions, routes each to the right engine, aggregates answers

Use when a single query needs answers from multiple distinct sources. Trade-off: multiple LLM calls per top-level query.

Response synthesizers & postprocessors

Once you have retrieved nodes, how you turn them into an answer matters. Response synthesizers implement different strategies — compact (fit into one prompt), refine (iterative), tree_summarize (hierarchical), accumulate (per-node). Node postprocessors run between retrieval and synthesis to filter, rerank, or transform retrieved nodes. Each has trade-offs on quality, latency, and cost.

The response modes

Compact mode — the default

Packs retrieved nodes into as few prompts as possible while staying under the model's context window. Almost always the right starting point.

Fails when: retrieved node text exceeds context window even after packing. Solution: reduce similarity_top_k, use a reranker to drop low-quality nodes before synthesis, or switch to refine mode.

Tree_summarize — powerful, expensive

synthesizer = get_response_synthesizer(
    response_mode="tree_summarize",
    use_async=True,   # parallelize the summarization pyramid
)

Groups retrieved nodes into subsets, summarizes each subset, then summarizes the summaries — a summarization pyramid. Handles arbitrarily large retrieved sets that don't fit in one prompt. Trade-off: log(N) LLM calls, latency grows with corpus size.

Common failure: intermediate summary levels themselves exceed context. Fix: reduce nodes at retrieval time, tune the LLM's context assumptions in the synthesizer config. See tree_summarize context overflow.

Node postprocessors — the reranker slot

from llama_index.postprocessor.cohere_rerank import CohereRerank
from llama_index.core.postprocessor import (
    SimilarityPostprocessor, KeywordNodePostprocessor,
)

# Cohere rerank — quality boost, latency and cost cost
reranker = CohereRerank(top_n=5, model="rerank-english-v3.0")

# Filter out low-similarity nodes
sim_filter = SimilarityPostprocessor(similarity_cutoff=0.7)

# Filter by keyword presence
kw_filter = KeywordNodePostprocessor(required_keywords=["policy"])

query_engine = index.as_query_engine(
    similarity_top_k=20,
    node_postprocessors=[sim_filter, reranker],
)

Postprocessors run in order after retrieval, before synthesis. Common pattern: retrieve loosely (top 20), let a reranker pick the top 5 most relevant, hand those to the synthesizer. Reranker cost is a per-query add — usually worth it for quality but measure. See reranker latency.

Available rerankers

Custom prompt templates

from llama_index.core.prompts import PromptTemplate

custom_qa_prompt = PromptTemplate(
    "Context information is below.\n"
    "---------------------\n"
    "{context_str}\n"
    "---------------------\n"
    "Given the context, answer the question. If the context does not contain "
    "the answer, say 'I do not have that information in the provided sources.'\n"
    "Question: {query_str}\n"
    "Answer:"
)

query_engine = index.as_query_engine(text_qa_template=custom_qa_prompt)

Customize the synthesizer prompt to enforce citations, ground more strictly, or match your brand voice. Default prompts are reasonable defaults but tuning them is often the highest-leverage single change you can make.

Workflows — event-driven orchestration

Workflows are LlamaIndex's answer to the "how do I chain multi-step logic?" question. Event-driven, step-based, async-first. Introduced in 0.11 and made the recommended orchestration path in 0.14. Steps decorated with @step accept and emit typed Event objects, and a per-run Context carries shared state. Every non-trivial LlamaIndex app in 2026 uses Workflows.

The minimum working workflow

from llama_index.core.workflow import (
    Workflow, step, Event, StartEvent, StopEvent, Context,
)

class MyEvent(Event):
    processed: str

class MyWorkflow(Workflow):

    @step
    async def process(self, ctx: Context, ev: StartEvent) -> MyEvent:
        query = ev.query
        result = f"processed: {query}"
        return MyEvent(processed=result)

    @step
    async def finalize(self, ctx: Context, ev: MyEvent) -> StopEvent:
        return StopEvent(result=ev.processed)

workflow = MyWorkflow(timeout=30)
result = await workflow.run(query="hello")
print(result)   # "processed: hello"

Five moves: define event types (Pydantic models extending Event); write step functions decorated with @step; each step's return type identifies which event(s) it emits; each step's parameter type identifies which event(s) trigger it; the workflow chains steps by matching event types; the workflow ends when a step emits StopEvent.

StartEvent and StopEvent

Two built-in events:

• StartEvent — the initial event that starts the workflow. Its fields come from kwargs passed to workflow.run(...).
• StopEvent — the terminal event. Emitting it stops the workflow and returns its result as the workflow's return value.

Every workflow needs at least one step that accepts StartEvent and at least one step that emits StopEvent. Missing either end and the workflow hangs (no start) or runs forever (no end). See StopEvent never emitted.

Event type matching

LlamaIndex routes events to steps by type. A step declared as async def process(ev: MyEvent) fires whenever any prior step emits a MyEvent. Return type declares which event(s) the step emits (Union types for multi-emission are supported).

Common bug: step never fires because no prior step emits its input event type. Symptom: workflow completes without visiting the step (or times out). Fix: verify the event type matches. See event type mismatch.

Context — the shared per-run state

class MyWorkflow(Workflow):

    @step
    async def store_data(self, ctx: Context, ev: StartEvent) -> ProcessEvent:
        await ctx.set("query", ev.query)
        await ctx.set("start_time", time.time())
        return ProcessEvent(...)

    @step
    async def use_data(self, ctx: Context, ev: ProcessEvent) -> StopEvent:
        query = await ctx.get("query")
        elapsed = time.time() - await ctx.get("start_time")
        return StopEvent(result=f"{query} took {elapsed:.2f}s")

Context is a per-run key-value store, async-safe. Use for state shared across steps that doesn't fit neatly in event payloads. Also exposes streaming (below) and workflow-run observation. If a step declares Context but you forget to add it as a parameter, the step fails at runtime. See Context not passed.

Multiple event returns and branching

from typing import Union

class ApprovedEvent(Event):
    action: str

class RejectedEvent(Event):
    reason: str

class MyWorkflow(Workflow):

    @step
    async def approve_or_reject(
        self, ctx: Context, ev: ReviewEvent,
    ) -> Union[ApprovedEvent, RejectedEvent]:
        if ev.score > 0.8:
            return ApprovedEvent(action=ev.action)
        return RejectedEvent(reason="Score too low")

    @step
    async def handle_approved(self, ctx: Context, ev: ApprovedEvent) -> StopEvent:
        result = execute(ev.action)
        return StopEvent(result=result)

    @step
    async def handle_rejected(self, ctx: Context, ev: RejectedEvent) -> StopEvent:
        return StopEvent(result=f"Rejected: {ev.reason}")

Union return types enable branching. The framework routes the emitted event to whichever step accepts it. Cleaner than an explicit routing step.

Streaming events

class ProgressEvent(Event):
    progress: int
    total: int

class MyWorkflow(Workflow):

    @step
    async def long_running(self, ctx: Context, ev: StartEvent) -> StopEvent:
        items = ev.items

        for i, item in enumerate(items):
            ctx.write_event_to_stream(ProgressEvent(progress=i, total=len(items)))
            await process(item)

        return StopEvent(result="done")

# Consume the stream
handler = workflow.run(items=my_items)
async for event in handler.stream_events():
    if isinstance(event, ProgressEvent):
        print(f"Progress: {event.progress}/{event.total}")

result = await handler

ctx.write_event_to_stream() emits events to the run's stream without triggering step routing. Consumers observe progress via handler.stream_events(). Useful for UI progress bars, real-time updates, or debugging long workflows.

Async and notebooks

Workflows are async-first. In a normal Python script:

import asyncio
result = asyncio.run(workflow.run(query="..."))

In a notebook (Jupyter/Colab), the event loop is already running. Just await workflow.run(...). Attempting asyncio.run() in a notebook raises about a nested event loop. See async in notebook.

Workflow validation

# Verify the workflow is well-formed before running
workflow._validate()
# Raises if steps don't chain correctly, event types unreachable, etc.

Call during development to catch structural issues (unreachable steps, missing StopEvent path). Skip in production if compile time matters.

Agents — FunctionAgent, ReActAgent, CodeActAgent

LlamaIndex ships three agent classes, all inheriting from BaseWorkflowAgent: FunctionAgent for models with native tool calling (GPT-5.6, Claude, Gemini), ReActAgent for any LLM via ReAct prompting, and CodeActAgent for code-generation-based reasoning. Picking the right one for your model determines whether the agent works, works but wastes tokens, or fails to parse its own output.

FunctionAgent — the default for modern models

from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.anthropic import Anthropic

def add(a: int, b: int) -> int:
    '''Add two integers.'''
    return a + b

def multiply(a: int, b: int) -> int:
    '''Multiply two integers.'''
    return a * b

agent = FunctionAgent(
    tools=[add, multiply],
    llm=Anthropic(model="claude-sonnet-4-6"),
    system_prompt="You are a math assistant.",
)

response = await agent.run("What is 12 + 8, times 3?")
print(response.response.content)

Uses the model's native tool-calling API — passes tools as structured schema, receives tool_call blocks, executes them, sends results back. Efficient (no prompt-based parsing) and reliable.

Tools can be plain functions (name and description parsed from the function signature and docstring), FunctionTool instances (explicit metadata), or MCP servers (chapter 11 covers MCP tools).

Requires a model that supports native tool calling. Not every LLM does — some open-weight models via HuggingFace, some older wrappers. On those, FunctionAgent raises about missing tool support at construction. See FunctionAgent vs ReActAgent.

ReActAgent — works with any LLM

from llama_index.core.agent.workflow import ReActAgent
from llama_index.llms.huggingface import HuggingFaceLLM

llm = HuggingFaceLLM(model_name="Qwen/Qwen2.5-Coder-32B-Instruct")

agent = ReActAgent(
    tools=[add, multiply],
    llm=llm,
    system_prompt="You are a math assistant.",
)

response = await agent.run("What is 12 + 8, times 3?")

Uses the ReAct (Reasoning + Acting) pattern via prompting: the LLM emits text like Thought: ... Action: add Action Input: {"a": 12, "b": 8} Observation: ... which LlamaIndex parses. Works with any LLM but:

• Slower (more tokens per turn due to Thought/Action verbiage)
• Fragile — small model that doesn't follow the format exactly crashes the parser
• Weaker tool-use quality — models fine-tuned for native tool calling do it better

Use ReActAgent when you need to run on models without tool-calling support. Prefer FunctionAgent everywhere else. See ReActAgent output parser.

CodeActAgent — reasoning through code

from llama_index.core.agent.workflow import CodeActAgent

agent = CodeActAgent(
    tools=[add, multiply],
    llm=llm,
    code_execute_fn=execute_python,   # sandbox executor
    system_prompt="Solve problems by writing Python code.",
)

Instead of calling tools directly, CodeActAgent generates Python code that uses the tools as functions. Model emits code, agent executes in a sandbox, feeds output back. More powerful for complex multi-step arithmetic, data manipulation, and iterative reasoning than tool-by-tool calling.

Requires a sandboxed code executor — you must provide code_execute_fn. Options: local subprocess (unsafe for user input), Docker container, cloud sandbox (E2B, Modal). See CodeActAgent sandbox.

Choosing between the three

Tools — the FunctionTool primitive

from llama_index.core.tools import FunctionTool
from pydantic import BaseModel, Field

class SearchArgs(BaseModel):
    query: str = Field(description="Search query")
    max_results: int = Field(default=5, description="Max results to return")

def search(query: str, max_results: int = 5) -> list[str]:
    '''Search the web for information.'''
    return do_search(query, max_results)

tool = FunctionTool.from_defaults(
    fn=search,
    name="web_search",
    description="Search the web for information",
    fn_schema=SearchArgs,   # explicit schema
)

Plain functions work (name and description parsed from signature and docstring). FunctionTool.from_defaults lets you override name/description or provide an explicit Pydantic schema. Pydantic v2 required — LlamaIndex 0.14 no longer supports v1 tool schemas. See Pydantic v2 required.

Query engines as tools

from llama_index.core.tools import QueryEngineTool, ToolMetadata

finance_tool = QueryEngineTool(
    query_engine=finance_index.as_query_engine(),
    metadata=ToolMetadata(
        name="finance_docs",
        description="Access financial policies and procedures documents",
    ),
)

agent = FunctionAgent(
    tools=[finance_tool, calculator_tool],
    llm=Anthropic(model="claude-sonnet-4-6"),
)

Wrap any query engine as a tool. Agent picks it when the query needs document context. This is the standard pattern for "RAG-in-agent" — the agent decides when to retrieve vs when to reason from parametric knowledge.

MCP tools

from llama_index.tools.mcp import BasicMCPClient, McpToolSpec

mcp_client = BasicMCPClient(command_or_url="my-mcp-server")
mcp_spec = McpToolSpec(client=mcp_client)
mcp_tools = await mcp_spec.to_tool_list_async()

agent = FunctionAgent(tools=mcp_tools + [my_local_tools], llm=llm)

MCP servers expose tools that any MCP-compatible agent framework can consume. LlamaIndex agents work with MCP tools the same as native tools. See the upcoming MCP Protocol pillar for depth on MCP.

Agent Context — carrying conversation across runs

from llama_index.core.workflow import Context

agent = FunctionAgent(tools=[...], llm=llm)

ctx = Context(agent)

response1 = await agent.run("My name is Sara", ctx=ctx)
response2 = await agent.run("What did I tell you my name was?", ctx=ctx)
# The agent remembers because ctx persists chat history

Context carries state across multiple agent.run() calls. For production, use a persistent Context (persist to disk or a database via ctx.to_dict() and Context.from_dict()).

Multi-agent with AgentWorkflow

AgentWorkflow composes multiple agents with handoff between them — the LlamaIndex pattern for supervisor-plus-specialist workflows, agent teams, and any scenario where one agent shouldn't handle everything. Same event-driven runtime as Workflows; different orchestration abstraction.

The minimum multi-agent workflow

from llama_index.core.agent.workflow import AgentWorkflow, FunctionAgent
from llama_index.llms.anthropic import Anthropic

llm = Anthropic(model="claude-sonnet-4-6")

def add(a: int, b: int) -> int:
    '''Add two integers.'''
    return a + b

def multiply(a: int, b: int) -> int:
    '''Multiply two integers.'''
    return a * b

adder = FunctionAgent(
    name="adder",
    description="Adds two integers together",
    system_prompt="You add numbers. Hand off to multiplier for multiplication.",
    tools=[add],
    llm=llm,
    can_handoff_to=["multiplier"],
)

multiplier = FunctionAgent(
    name="multiplier",
    description="Multiplies two integers together",
    system_prompt="You multiply numbers. Hand off to adder for addition.",
    tools=[multiply],
    llm=llm,
    can_handoff_to=["adder"],
)

workflow = AgentWorkflow(
    agents=[adder, multiplier],
    root_agent="adder",
)

response = await workflow.run("What is (12 + 8) times 3?")

Five moves: give each agent a name and description; declare can_handoff_to naming which other agents this one can delegate to; assemble in AgentWorkflow; specify a root_agent (where the workflow starts); run.

Why root_agent matters

Every AgentWorkflow needs a starting point. root_agent names which agent handles the first turn. Miss it and construction fails. Common bug on migration from a single agent: forgot to set root_agent. See root_agent not set.

Handoff mechanics

When you declare can_handoff_to=["multiplier"] on the adder, LlamaIndex injects a handoff tool into the adder's tool list. The tool is named after the target agent and, when called, transfers control (with any state/message context) to that agent.

Common pitfall: agent tries to hand off to another agent not in its can_handoff_to list. Handoff tool doesn't exist for that target; call fails. Fix: verify can_handoff_to matches the intended handoff graph. See handoff tool not registered.

Streaming multi-agent events

from llama_index.core.agent.workflow import (
    AgentInput, AgentOutput, ToolCall, ToolCallResult, AgentStream,
)

handler = workflow.run("...")

async for event in handler.stream_events():
    if isinstance(event, AgentStream):
        print(event.delta, end="", flush=True)   # token stream
    elif isinstance(event, ToolCall):
        print(f"\n[Tool call] {event.tool_name}: {event.tool_kwargs}")
    elif isinstance(event, ToolCallResult):
        print(f"[Tool result] {event.tool_output}")
    elif isinstance(event, AgentOutput):
        print(f"\n[Agent {event.current_agent_name} finished]")

response = await handler

AgentWorkflow emits typed events throughout the run — perfect for UI updates showing which agent is working, what tool got called, and streaming token output. This is where the event-driven runtime pays off.

The supervisor pattern

supervisor = FunctionAgent(
    name="supervisor",
    description="Routes user queries to the right specialist agent",
    system_prompt=(
        "You coordinate a team of specialist agents. "
        "For research queries, hand off to researcher. "
        "For writing tasks, hand off to writer. "
        "For code, hand off to coder. "
        "Reserve your own answer for coordination and summarization."
    ),
    tools=[],   # supervisor doesn't do work directly
    llm=llm,
    can_handoff_to=["researcher", "writer", "coder"],
)

researcher = FunctionAgent(
    name="researcher",
    description="Researches topics using web search and internal documents",
    tools=[web_search, doc_search],
    llm=llm,
    can_handoff_to=["supervisor"],   # hand back
)

writer = FunctionAgent(...)
coder = FunctionAgent(...)

workflow = AgentWorkflow(
    agents=[supervisor, researcher, writer, coder],
    root_agent="supervisor",
)

Supervisor delegates, specialists work, hand back. Same pattern as LangGraph's supervisor architecture (see the LangGraph pillar). Different abstraction, similar mental model. Pick based on framework preference.

Multi-agent gotchas

• Handoff loops — supervisor keeps re-delegating to the same specialist. Fix: specialists must update the context in a way the supervisor can observe as progress; add a turn counter as safety.
• Context bloat — every agent sees the full message history by default. Long multi-agent runs balloon token usage. Consider summarizing older turns.
• Specialist scope creep — an agent given too many tools starts trying to do everything itself. Keep specialist tool sets small and focused.
• Handoff graph too permissive — if every agent can hand off to every other agent, coordination breaks down. Prefer strict topology (specialists hand back to supervisor, not sideways).

LlamaCloud, LlamaParse & deployment

LlamaIndex 2026 ships a hosted stack: LlamaParse for parsing messy enterprise documents, LlamaCloud for managed indexing pipelines, and llama-agents + llamactl + WorkflowServer for running Workflows as HTTP services. All optional — the framework works fully self-hosted — but the hosted layer is where a lot of the framework's practical value sits, especially for teams whose documents don't parse cleanly.

LlamaParse — the document-parsing gap-filler

from llama_parse import LlamaParse

parser = LlamaParse(
    api_key=os.environ["LLAMA_CLOUD_API_KEY"],
    result_type="markdown",   # or "text", "json"
    parsing_instruction=(
        "Extract tables as markdown tables. "
        "Preserve document structure. Keep page numbers as metadata."
    ),
    num_workers=4,
)

documents = await parser.aload_data("./contracts/msa-2026.pdf")

LlamaParse handles what SimpleDirectoryReader can't: PDFs with complex tables, scanned documents (OCR), invoices, forms, financial statements. Uses vision-language models under the hood. Output as markdown (recommended for RAG) or structured JSON.

Common failure: LLAMA_CLOUD_API_KEY not set. LlamaParse is a hosted service — you need to sign up at cloud.llamaindex.ai and get an API key. See API key not set.

Cost model: per-page pricing, tier-based (basic mode cheapest, premium mode with vision LM more expensive but higher fidelity). For teams parsing millions of pages, LlamaParse is a real budget line; for teams with clean PDFs, SimpleDirectoryReader suffices.

LlamaCloud — managed indexing

from llama_index.indices.managed.llama_cloud import LlamaCloudIndex

index = LlamaCloudIndex(
    name="my-production-index",
    project_name="my-project",
    api_key=os.environ["LLAMA_CLOUD_API_KEY"],
)

# Ingest via LlamaCloud pipeline (data sources connected in the UI)
# Query as normal
retriever = index.as_retriever(similarity_top_k=5)
nodes = retriever.retrieve("...")

LlamaCloud provides:

• Managed vector store (no need to run Postgres/Pinecone yourself)
• Automatic ingestion pipeline with LlamaParse for parsing
• Connectors to common data sources (S3, Notion, SharePoint, Confluence, Slack)
• Scheduled sync jobs for keeping the index up-to-date
• UI for configuring pipelines, monitoring runs, browsing content

Trade-off: hosted convenience vs vendor lock-in and per-query cost. Great for teams without infra ops capacity; less compelling for teams already running their own vector database. Common issue: pipeline sync errors when source data schema drifts. See LlamaCloud sync error.

llama-agents — running Workflows as services

from llama_agents import WorkflowServer, WorkflowClient
from my_app.workflows import my_workflow

# Server side
server = WorkflowServer(workflows={"my_workflow": my_workflow})
server.serve(host="0.0.0.0", port=8000)

# Client side
from llama_agents import WorkflowClient

client = WorkflowClient(base_url="http://localhost:8000")

# Sync
result = client.run("my_workflow", query="hello")

# Async streaming
async for event in client.stream("my_workflow", query="hello"):
    print(event)

# Human-in-the-loop
run_id = client.start("my_workflow", query="review this")
# Later, when human decision arrives:
client.send_event(run_id, HumanFeedbackEvent(decision="approved"))

WorkflowServer exposes your Workflow definitions as HTTP endpoints. WorkflowClient lets external code call them, stream events, and send events (for HITL). This is what turns a locally-run Workflow into a deployable service.

llamactl — the deployment CLI

pip install llama-deploy

# Deploy a workflow
llamactl deploy ./deployment-config.yaml

# List running deployments
llamactl list

# Get logs
llamactl logs my-deployment

llamactl wraps deployment orchestration for llama-agents workflows. Options: deploy to LlamaCloud, deploy to your own K8s cluster with a manifest, run locally with Docker Compose. Config file (deployment-config.yaml) declares workflows, environment variables, resource limits, and integrations.

Deployment options in practice

Observability — tracing and evals

LlamaIndex integrates with OpenTelemetry via openinference-instrumentation-llama-index:

from openinference.instrumentation.llama_index import LlamaIndexInstrumentor

LlamaIndexInstrumentor().instrument()

# Now all LlamaIndex operations emit OTel spans
# Send to any OTel backend: Jaeger, Datadog, Honeycomb, Arize, Phoenix, Langfuse

Alternative: LlamaIndex's own traceAI observability layer (part of the LlamaCloud stack). Both work; OTel wins for teams already running OTel infrastructure. Traces show workflow steps, agent turns, LLM calls, tool executions, retrieval results — same picture as LangSmith gives for LangGraph, different backend.

Common deployment failures

• WorkflowServer import error — workflow module path wrong, or module imports fail at server startup. See WorkflowServer deploy error.
• Missing environment variables at build time — LLM API keys, LlamaCloud key needed at container startup for many workflows
• Cold start latency — first request after idle pays the model client init cost; consider warmup pings for latency-sensitive workloads
• Async event loop conflicts — some deployment platforms run their own event loops; Workflows need to integrate cleanly with the host loop

Production checklist

Twenty items to verify before your LlamaIndex application goes to production. Each corresponds to at least one incident from real 2026 deployments — with attention to the LlamaIndex-specific pitfalls (package split, ingestion caching, index type choice, agent parsing failures) that don't apply outside this framework.

Package & version management

1. Legacy monolithic llama_index package uninstalled — only modern llama-index-* packages installed
2. Every LlamaIndex package pinned to exact versions (==), not ranges
3. Python version verified against all installed packages' minimum requirements
4. Pydantic v2 used everywhere (v1 tool schemas break in 0.14)

Ingestion & data

5. Documents have stable id_ values for detecting updates across re-ingestions
6. IngestionCache collection name versioned when transformation parameters change
7. Metadata visibility (excluded_llm_metadata_keys, excluded_embed_metadata_keys) explicitly configured, not defaults
8. Embedding model choice locked in a config; dimension mismatch protection in place when re-embedding

Indices & retrieval

9. Index type chosen deliberately — VectorStoreIndex for general RAG, not SummaryIndex for large corpora
10. Vector store embed_dim parameter matches embedding model dimension
11. similarity_top_k tuned against an eval set, not left at default 2
12. Hybrid retrieval configured if BM25 keyword matching matters for the workload
13. Metadata filter operators verified compatible with the vector store

Workflows & agents

14. Every Workflow has a StopEvent path from every reachable step
15. Agent type matches the model — FunctionAgent for tool-calling models, ReActAgent only when necessary
16. AgentWorkflow root_agent explicitly set; handoff graph verified
17. Long-running workflows use streaming events for UI feedback, not blocking waits
18. Context persistence pattern in place if agents span multiple runs

Observability & deployment

19. Observability wired (openinference-OTel or LlamaCloud traceAI) — full workflow visibility, not just top-level logs
20. LLAMA_CLOUD_API_KEY (or self-hosted equivalents) present in every environment that needs it, verified before deployment

Frequently asked questions

The full library — 30 LlamaIndex error pages

Every LlamaIndex error-fix page on AI Error Hub, organized by category. Bookmark this section — when you hit a specific error, jump straight to the page that walks through it with tested code.