Prompt caching

Anthropic

Anthropic models support explicit Prompt caching. On this platform, whether using the OpenAI chat/completions protocol or the Anthropic v1/messages protocol, you can use "cache_control": {"type": "ephemeral"} to specify content to be cached.

{
  "model": "claude-sonnet-4-5-20250929",
  "max_tokens": 4096,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "HUGE TEXT BODY",
          "cache_control": { "type": "ephemeral" }
        },
        {
          "type": "text",
          "text": "Name all the characters in the above book"
        }
      ]
    }
  ]
}

⚠️ cache_control is an extended field that is not included in the official OpenAI SDK protocol, so it must be explicitly added when making calls. You can verify cache creation/hit status through the response.

{
  "prompt_tokens": 7039,
  "completion_tokens": 650,
  "total_tokens": 7689,
  "prompt_tokens_details": {
    "cached_tokens": 7019,
    "cache_creation_input_tokens": 7019,  # 👈 cache created
    "cache_read_input_tokens": 0
  }
}
---
{
  "prompt_tokens": 7042,
  "completion_tokens": 572,
  "total_tokens": 7614,
  "prompt_tokens_details": {
    "audio_tokens": 0,
    "cached_tokens": 7019,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 7019 # 👈 cache read
  }
}

{"cache_creation_input_tokens":188086,"cache_read_input_tokens":0,"input_tokens":21,"output_tokens":393} # 👈 cache created

{"cache_creation_input_tokens":0,"cache_read_input_tokens":188086,"input_tokens":21,"output_tokens":393} # 👈 cache read

⚠️⚠️⚠️ For Anthropic models, the minimum Input Tokens required for Prompt caching are as follows:

Claude Opus 4.1, Claude Opus 4, Claude Sonnet 4.5, Claude Sonnet 4, Claude Sonnet 3.7: 1024 tokens
Claude Haiku 4.5, Claude Haiku 3.5, and Claude Haiku 3: 2048 tokens

OpenAI and OpenAI-Compatible Models

Typically, these models may support implicit caching. When users repeatedly access the same model with the same Prompt prefix, there is a chance of hitting the cache.

// Round 1
{
  "model": "gpt-4",
  "messages": [
    {
      "role": "system",
      "content": "HUGE TEXT BODY: Complete API documentation, code style guide, best practices (5000+ lines)"
    },
    {
      "role": "user",
      "content": "How do I authenticate API requests?"
    }
  ]
}

// Round 2 - Documentation cached
{
  "model": "gpt-4",
  "messages": [
    {
      "role": "system",
      "content": "HUGE TEXT BODY: Complete API documentation, code style guide, best practices (5000+ lines)"
    },
    {
      "role": "user",
      "content": "How do I authenticate API requests?"
    },
    {
      "role": "assistant",
      "content": "Use Bearer token in Authorization header..."
    },
    {
      "role": "user",
      "content": "What about rate limiting?"
    }
  ]
}

Below is a usage example of a cache hit:

{
  "prompt_tokens": 3003,
  "completion_tokens": 1564,
  "total_tokens": 4567,
  "prompt_tokens_details": {
    "cached_tokens": 2025 # 👈 cache hitted
  }
}

Gemini

Currently only implicit caching is supported. Implicit caching requires no manual setup or additional cache_control configuration. When users repeatedly access the same model with the same Prompt prefix, there is a chance of hitting the cache. Notes:

The average TTL (cache time-to-live) is 3-5 minutes, but it may vary (e.g., it could be only a few seconds)
Gemini 2.5 Flash requires a minimum input of 1024 tokens, and Gemini 2.5 Pro requires a minimum of 4096 tokens

Below is a usage example of a cache hit:

{
  "prompt_tokens": 2004,
  "completion_tokens": 1564,
  "total_tokens": 3568,
  "prompt_tokens_details": {
    "cached_tokens": 1994 # 👈 cache hitted
  }
}

For input examples, refer to the OpenAI and OpenAI-Compatible Models section above.

Getting Started

LLM API

Providers

Features

Integrations

Prompt caching

Anthropic

OpenAI and OpenAI-Compatible Models

Gemini

​Anthropic

​OpenAI and OpenAI-Compatible Models

​Gemini

Anthropic

OpenAI and OpenAI-Compatible Models

Gemini