API Documentation

Quick Start

Get started in under 60 seconds. Create an API key in the dashboard, then point your OpenAI client at SemaCache.

from openai import OpenAI

client = OpenAI(
    api_key="sc-your-key",
    base_url="https://api.semacache.io/v1"
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "What is semantic caching?"}
    ]
)

print(response.choices[0].message.content)

Authentication

All API requests require a SemaCache API key in the Authorization header. Create keys in the dashboard.

Authorization: Bearer sc-your-key

Upstream LLM Keys

SemaCache proxies requests to your LLM provider using your upstream API key. Two options:

curl -X POST https://api.semacache.io/v1/chat/completions \
  -H "Authorization: Bearer sc-your-key" \
  -H "x-upstream-api-key: sk-your-openai-key" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [...]}'

Chat Completions

OpenAI-compatible chat completions with three-tier caching: exact match → semantic match → LLM passthrough.

Request Body

Supported Models

Response

Standard OpenAI-compatible response format, regardless of upstream provider.

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "model": "gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Semantic caching stores responses..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 85,
    "total_tokens": 97
  }
}

Image Generation

Generate images with caching. Supports OpenAI GPT Image, Google Imagen, and xAI Grok Imagine. Generated images are rehosted to cloud storage for permanent URLs.

Request Body

response = client.images.generate(
    model="gpt-image-1",
    prompt="A cat wearing a top hat, oil painting",
    size="1024x1024"
)

image_url = response.data[0].url

Video Generation

Generate videos with Google Veo or xAI Grok Imagine Video. Videos are rehosted to cloud storage. Requires an upstream API key (Gemini for Veo, xAI for Grok).

Request Body

curl -X POST https://api.semacache.io/v1/videos/generations \
  -H "Authorization: Bearer sc-your-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "veo-2.0-generate-001",
    "prompt": "A drone shot over a mountain lake at sunrise",
    "duration_seconds": 8,
    "aspect_ratio": "16:9"
  }'

Cache Control

Control caching behavior per-request using HTTP headers. All headers are passed via the OpenAI SDK's extra_headers (Python) or headers (TypeScript) parameter.

Request Headers

Cache Duration (TTL) — Priority Order

The cache TTL (time-to-live) determines how long a cached response is stored before expiring. There are three ways to set it. The highest-priority source wins:

Examples

# Custom 1-hour TTL
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
    extra_headers={"x-cache-ttl": "3600"}
)

# Force fresh response (bypass cache read, still store)
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
    extra_headers={"Cache-Control": "no-cache"}
)

# Ephemeral — don't read or store
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Sensitive question"}],
    extra_headers={"Cache-Control": "no-store"}
)

Similarity Threshold

The similarity threshold controls how closely a new query must match an existing cached query to count as a semantic cache hit. A value of 1.00 means only exact semantic matches; a lower value like 0.85 accepts looser paraphrases, increasing hit rate but risking false positives.

Three Ways to Set It — Priority Order

The threshold is resolved per-request. The highest-priority source that is set wins:

Choosing a Value

Examples

# Set threshold globally for all requests on this client
client = OpenAI(
    api_key="sc-your-key",
    base_url="https://api.semacache.io/v1",
    default_headers={"x-similarity-threshold": "0.90"},
)

# Or override per-request
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
    extra_headers={"x-similarity-threshold": "0.98"},
)

# Combine with cache control
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
    extra_headers={
        "x-similarity-threshold": "0.85",
        "x-cache-ttl": "3600",
    },
)

Custom Models

Pro Enterprise

Register any OpenAI-compatible endpoint (vLLM, Ollama, Together AI, Groq, Fireworks, etc.) and SemaCache will cache responses the same way it handles built-in models.

Register via Dashboard

Go to Dashboard → Custom Models → Add Model. Provide the base URL, model name, and authentication details. Once registered, use the model alias in any request.

Register via API

Usage

After registration, use your custom model alias like any built-in model:

# Store your custom model's API key in the dashboard first
# Then use the alias like any other model
response = client.chat.completions.create(
    model="my-llama",
    messages=[{"role": "user", "content": "Hello!"}]
)
# SemaCache auto-detects it's a custom model and routes accordingly

Response Headers

Every response includes cache metadata headers so you can observe caching behavior.

response = client.chat.completions.with_raw_response.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}]
)

print(response.headers["x-semcache-match-type"])  # EXACT | SEMANTIC | NATIVE
print(response.headers["x-semcache-latency"])      # 4.2
print(response.headers["x-semcache-confidence"])   # 0.9712 (semantic only)

Error Handling

Errors follow the OpenAI error format for compatibility with existing error handling code.

{
  "error": {
    "message": "Monthly request limit exceeded. Upgrade to Pro for 50,000 requests/month.",
    "type": "rate_limit_error"
  }
}

Rate Limits

Rate limits are enforced at two levels: monthly quotas (total requests per billing period) and burst limits (requests per second / per minute) to prevent abuse and protect upstream providers.

Monthly Quotas

Burst Rate Limits

Burst limits use a sliding window to prevent short-duration spikes. These protect both you and the upstream LLM providers from runaway loops or accidental floods.

Rate Limit Response

When a rate limit is exceeded, the API returns 429 with a Retry-After header (for burst limits). Implement exponential backoff or respect the header value.

HTTP/1.1 429 Too Many Requests
Retry-After: 1
Content-Type: application/json

{
  "error": {
    "message": "Rate limit exceeded: 20 requests/second. Try again shortly.",
    "type": "rate_limit_error"
  }
}

Cache hits count toward your monthly request limit. Upgrade anytime from the billing page. Need higher limits? Contact us for custom enterprise pricing.