Cut your LLM API costs by up to 80%
SemaCache is an intelligent caching proxy for LLM APIs. It returns cached responses when a semantically similar query has been seen before — saving you money on every repeated question.
# Before — calling OpenAI directly
client = OpenAI(api_key="sk-...")
# After — just change the base URL
client = OpenAI(
api_key="sc-your-key",
base_url="https://www.semacache.io/api/v1"
)Try it yourself
Explore the dashboard and see the caching pipeline in action \u2014 no signup required. All data is mocked.
Overview
Monitor cache performance and usage.
898.3K tokens saved ($42.86)
1.3M total \u00b7 70% from cache
all time
5,621 exact · 3,372 semantic
LLM pass-throughs
898.3K tokens saved
Cache Hit Rate
8,993 / 12,847
Latency
Daily Requests
7dall requests
642K prompt \u00b7 642K completion
without cache: $61.28
Recent Queries
Latest cache interactions
| Query | Model | Type | Latency | Time |
|---|---|---|---|---|
| What is the capital of France? | gpt-4o-mini | EXACT | 4ms | 2:34 PM |
| Explain machine learning basics | gpt-4o | SEMANTIC | 19ms | 2:33 PM |
| How to sort a list in Python? | gpt-4o-mini | EXACT | 3ms | 2:31 PM |
| Benefits of cloud computing? | claude-sonnet-4 | NATIVE | 2.2s | 2:29 PM |
| Translate hello to Spanish | gpt-4o-mini | SEMANTIC | 21ms | 2:27 PM |
| SQL query to find duplicates | gpt-4o | EXACT | 5ms | 2:25 PM |
\ud83d\udcca Preview of the SemaCache dashboard with mock data. Sign up to see your real metrics.
You're burning $438/mo on repeat queries
Up to 40% of your API calls return the same or similar answers. SemaCache intercepts them before they hit your LLM — so you only pay once.
Configure your usage
~$0.02/request at average token usage
Your monthly savings
$175
That's $2.1k/year back in your pocket
40%
cost reduction
10K
free cache hits
2d
payback period
Stop leaving $2.1k/year on the table.
Pro pays for itself in 2 days. Cancel anytime. No risk.
Trusted by developers building with OpenAI, Claude, Gemini, Grok, and custom models. One line of code. Instant savings.
Features
Three tiers of intelligent caching
Every request flows through a fast pipeline: exact hash → semantic similarity → LLM passthrough. Each tier is cheaper and faster than calling the LLM directly.
Exact Match Cache
MD5 hash lookup in Redis. Identical queries return cached responses in under 5ms.
Semantic Match Cache
Gemini-powered embeddings with pgvector similarity search. Catches paraphrased queries automatically.
Multi-Provider Routing
One endpoint for OpenAI, Anthropic Claude, Google Gemini, and xAI Grok. SemaCache auto-detects the provider from the model name and routes accordingly.
Encrypted Key Storage
Store your LLM API keys securely in the dashboard. AES-256 encrypted at rest — keys never leave our servers.
Real-Time Analytics
Dashboard with cache hit rates, latency metrics, cost savings, and daily request volume per API key.
OpenAI-Compatible API
Drop-in replacement for any OpenAI SDK client. Works with Python, JavaScript, Go, and every other language.
Official SDKs on GitHub
MIT-licensed client libraries for every major language. Zero OpenAI dependency — each SDK talks directly to the SemaCache API and parses cache metadata from response headers.
How it works
From request to response in milliseconds
Your app sends a request
Point your OpenAI client at SemaCache. Your app sends requests as usual — no code changes needed beyond changing the base URL.
Exact match check
We hash the query and check Redis. If the identical query was asked before, the cached response is returned in ~5ms.
Semantic similarity search
If no exact match, we embed the query with Gemini and search our pgvector index. Paraphrased queries like "What's France's capital?" match "Capital of France?" with high confidence.
LLM passthrough & cache
On full miss, we route to the correct provider (OpenAI, Anthropic, Gemini, or Grok based on model name), return the response, and cache it for future hits.
Text, images, and video — all cached.
Every API call goes through the same three-tier pipeline. The first request generates and caches. Every repeat returns instantly — whether it’s a chat reply, a 4K image, or a generated video.
Measured end-to-end on production (Google Cloud Run), including full network round-trip. Chat: OpenAI GPT-4o Mini & Gemini 2.0 Flash. Image: OpenAI GPT Image 1 & Google Imagen 4.0. Video: Google Veo 2 & Veo 3. Same caching applies to xAI Grok and all other supported models.
Supported Models
Works with every major LLM provider
Built-in support for OpenAI, Anthropic Claude, Google Gemini, xAI Grok, Imagen, and Veo. Plus register any OpenAI-compatible endpoint as a custom model.
OpenAI
Chat Completions
Anthropic Claude
Chat Completions
Google Gemini
Chat Completions
xAI Grok
Chat Completions
Image Generation
OpenAI, Google, xAI
Video Generation
Google Veo, xAI
Bring your own model
Register any OpenAI-compatible endpoint — vLLM, Ollama, Together AI, Groq, Fireworks, or your own self-hosted model. SemaCache caches responses from custom models the same way it caches OpenAI, Claude, and Gemini.
- Register via dashboard or API — set base URL, model name, and auth
- Full three-tier caching: exact → semantic → passthrough
- Works with any provider that speaks OpenAI-compatible format
# Register "my-llama" in Dashboard → Custom Models
# Then use it like any built-in model
from openai import OpenAI
client = OpenAI(
api_key="sc-your-key",
base_url="https://www.semacache.io/api/v1"
)
response = client.chat.completions.create(
model="my-llama",
messages=[
{"role": "user", "content": "Hello!"}
]
)Use Cases
Built for every LLM-powered workflow
If your app calls an LLM API, SemaCache can save you money. Here are the most common patterns.
Customer Support Chatbots
Users ask the same questions hundreds of times: "How do I reset my password?", "What are your hours?", "How do I cancel?" SemaCache recognizes all variations and returns cached answers in milliseconds — cutting your API bill by 60–80%.
RAG & Knowledge Base Q&A
Retrieval-augmented generation apps often get the same questions about the same documents. SemaCache caches the LLM's synthesized answers so repeat queries skip the entire RAG pipeline — embedding lookup, context assembly, and LLM call.
E-Commerce Product Descriptions
Generating product descriptions, summaries, or recommendations with LLMs? The same products get described over and over. Cache the output once and serve it instantly on every page load.
Image & Video Generation
Marketing teams regenerate the same hero images and product videos repeatedly. SemaCache caches generated media — a cached image returns in <1s vs 10–15s from the provider. Cached video returns in <1s vs 30–60s.
Internal Tools & Dashboards
Internal apps that summarize reports, translate content, or classify tickets often process the same inputs. SemaCache eliminates redundant LLM calls across your team without any code changes.
Development & Testing
Running the same prompts during development and CI/CD burns through API credits. SemaCache returns cached responses instantly so your test suite runs in seconds, not minutes — and costs nothing on repeat runs.
Pricing
Start free, scale with confidence
Every plan includes multi-provider support and encrypted key storage.
Free
For experimentation and side projects
- 1,000 requests / month
- 1 API key
- Text + image caching
- 7-day audit logs
- Community support
Pro
For developers shipping to production
- 50,000 requests / month
- 5 API keys
- Text + image + video caching
- Custom model registry
- 30-day audit logs
- Email support
Enterprise
For teams at scale
- 500,000 requests / month
- Unlimited API keys
- Text + image + video caching
- Custom model registry
- 90-day audit logs
- Priority support
Ready to cut your LLM costs?
Get started in under a minute. No credit card required. Change one line of code and start saving.