Gemini Context Caching Not Hitting / Discount Missing — Fix (2026) | AI Error Hub

GeminiCachingSeverity: MediumCost impact

Gemini Context Caching Not Hitting

You created cached_content but bills show full input token pricing — the cache isn't being applied. Common causes: TTL expired between calls, minimum 4096 tokens not met, content or model mismatch, or you forgot to reference the cache name in generate_content.

Detection: usage.cached_content_token_count is 0 despite cacheCategory: CachingCost impact: 75% missing discountLast tested: Nov 15, 2026

Quick Fix (TL;DR)

Explicit caching requires: (1) `create_cached_content` first to store, (2) pass `cached_content=cache.name` in `GenerateContentConfig`, (3) same model as cache creation, (4) at least 4096 tokens in cache, (5) TTL still valid. Verify hit via `usage.cached_content_token_count` in response.

The Symptoms You're Seeing

Symptoms & error variants

# Symptom 1: Cache created but not used
response.usage_metadata.cached_content_token_count = 0
response.usage_metadata.prompt_token_count = 45000  # full price

# Symptom 2: Cache expired
{"error": {"code": 404, "message": "Cached content not found: 
projects/*/locations/*/cachedContents/abc123", 
"status": "NOT_FOUND"}}

# Symptom 3: Minimum tokens not met
{"error": {"code": 400, "message": "Cached content must have at 
least 4096 tokens.", "status": "INVALID_ARGUMENT"}}

# Symptom 4: Model mismatch
{"error": {"code": 400, "message": "cached_content model 
'gemini-2.5-flash' does not match request model 
'gemini-2.5-pro'.", "status": "INVALID_ARGUMENT"}}

Cache Rules Reference

Rule	Value	Notes
Minimum tokens	4,096	Cache smaller and API rejects
Default TTL	1 hour	Configurable at creation
Max TTL	Up to 1 year (Vertex)	AI Studio caps lower
Model binding	Locked at creation	Can't reuse cache across model versions
Content match	Exact prefix	Cache is prepended verbatim to new prompt
Cost savings	~75% on cached tokens	Plus small storage fee per hour
Storage cost	Per token per hour	Small but adds up on long TTLs

What Actually Causes This Error

30%

TTL too short for usage pattern1hr default expires before repeat requests hit.

22%

Cache not referenced in request configCache created but generate_content omits cached_content field.

18%

Below 4096 token minimumSmall system prompts don't meet floor.

14%

Model mismatchCached on Flash, called on Pro (or vice versa).

10%

Content prefix driftSystem prompt changes slightly per request, breaks cache.

Wrong deployment pathCache created on Vertex, request goes to AI Studio.

Fixes That Work (Tested Nov 2026)

1Correct Explicit Cache Workflow

Python · create then reference
from google import genai
from google.genai import types
import datetime

client = genai.Client()

# Step 1: Create cached content — long system + doc
system_and_doc = "You are a legal analyst..." + very_long_contract_text

cache = client.caches.create(
    model="gemini-2.5-flash",
    config=types.CreateCachedContentConfig(
        contents=[types.Content(
            role="user",
            parts=[types.Part(text=system_and_doc)]
        )],
        system_instruction="Answer only from the contract text.",
        display_name="contract-analysis-v1",
        ttl="7200s",   # 2 hours — extend if repeat queries
    )
)
print(f"Cache name: {cache.name}, tokens: {cache.usage_metadata.total_token_count}")

# Step 2: Reference cache in each subsequent call
for question in user_questions:
    response = client.models.generate_content(
        model="gemini-2.5-flash",   # MUST match cache model
        contents=question,
        config=types.GenerateContentConfig(
            cached_content=cache.name   # ← key line
        )
    )
    
    # Verify cache hit
    cached_tokens = response.usage_metadata.cached_content_token_count
    assert cached_tokens > 0, "Cache didn't hit!"
    print(f"Cached: {cached_tokens}, new input: {response.usage_metadata.prompt_token_count - cached_tokens}")

2TTL Management & Refresh

Python · extend TTL, handle expiry
from google.api_core import exceptions

def ensure_cache_valid(cache_name, model, content_factory, ttl="7200s"):
    """Re-create cache if expired."""
    try:
        cache = client.caches.get(name=cache_name)
        # Optionally extend TTL
        client.caches.update(
            name=cache_name,
            config=types.UpdateCachedContentConfig(ttl=ttl)
        )
        return cache
    except exceptions.NotFound:
        # Recreate
        return client.caches.create(
            model=model,
            config=types.CreateCachedContentConfig(
                contents=content_factory(),
                ttl=ttl
            )
        )

# List active caches — housekeeping
for c in client.caches.list():
    print(c.name, c.model, c.expire_time, c.usage_metadata.total_token_count)

# Delete when done — stops storage billing
client.caches.delete(name=cache.name)

TTL math for cost

Storage costs vs discount: caching saves 75% on repeated input tokens but you pay per-token-per-hour storage. Break-even for a 50K token cache is usually 2-3 uses/hour. Below that, longer TTL costs more than the savings. Above it, extend TTL aggressively.

3Cache-Friendly Content Design

Python · keep prefix stable
# WRONG — timestamp in prompt breaks the cache prefix
prompt = f"Current time is {datetime.utcnow()}. {system_prompt}"
# Every request has different prefix → cache never hits

# RIGHT — put dynamic bits AFTER cached prefix
cached_prefix = system_prompt + long_doc  # cache this
dynamic_suffix = f"Current time: {datetime.utcnow()}. User question: {q}"

# Send: cached_content (system+doc) + new content (time+question)
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents=dynamic_suffix,
    config=types.GenerateContentConfig(cached_content=cache.name)
)

# Design heuristics for good caching:
# 1. Stable content (docs, KB, examples) → in cache
# 2. User-specific (name, ID, session) → post-cache
# 3. Tools schemas — usually stable, cache with system
# 4. Version stamps — put in a header, not per-request

Preventing This Error Going Forward

Verify hit via cached_content_token_count on every call in dev.
Keep cached prefix stable — no timestamps, IDs, per-request data.
Set TTL to match your access pattern. Analyze cache reuse rate.
Only cache 4096+ token content. Below floor won't cache.
Delete unused caches to stop storage billing.
Match model exactly between cache creation and request.
Log cache hit ratio; alert on drops. Indicates content drift.

Tested by Ahmed R.
Researcher · AI Error Hub

Last verified: Nov 15, 2026 · SDK: google-genai 0.8+

Frequently Asked Questions

For some models Google applies implicit prefix caching under the hood, but with no guarantees. For predictable savings use explicit caching (create_cached_content). Only explicit gives you the referenced cache name and reliable hit accounting.

On Vertex AI ~$1/M tokens/hour storage rate (varies by model tier), so 100K × 24 = $0.024. Trivial vs the ~75% discount on repeated calls. Only becomes painful for millions of tokens over long TTLs with few reuses.

Yes — tools count as part of cached content in most SDK versions. Especially useful for large tool arrays (agent frameworks). Confirm hit via cached_content_token_count; some SDK versions handle tools separately.

Vertex AI: caches are region-scoped. Cache created in us-central1 isn't accessible from europe-west3. Use consistent region per workload. AI Studio caches are global.

usage_metadata.cached_content_token_count reports total cached tokens hit. Combined with your explicit cache name reference, if the count is nonzero and matches your cache size, explicit hit worked. Implicit hits also populate this field.