PromptThin

> Reduce LLM API costs through caching, compression, and smart routing. Zero code changes.

PromptThin is a transparent proxy that sits between your AI agents and LLM providers. Two environment variables and you're done — every API call gets five compounding savings routes applied automatically.

Your app ──→ PromptThin ──→ OpenAI / Anthropic / Gemini / Groq

Where PromptThin works

PromptThin saves tokens when you control the API call — your own code, AI agents, or a self-hosted chat UI. It does not intercept calls made by managed chat interfaces like claude.ai, ChatGPT, or similar products; those platforms call LLM APIs internally and cannot be proxied.

> Tip for heavy claude.ai users: If you're hitting usage quota limits in the claude.ai chat, the fix is to use a self-hosted UI like Open WebUI or LibreChat pointed at the Anthropic API through PromptThin. You get the same chat experience with compression and caching reducing every turn's token cost.

Five savings routes

Recommended agent pattern:

# 1. Check savings estimate first (free)
estimate = call_tool("proxy_predict", model="gpt-4o", messages=messages)
# → "87% saving — compression + routing to gpt-4o-mini"

# 2. Send through PromptThin (savings applied automatically)
response = call_tool("proxy_chat", model="gpt-4o", messages=messages)
# → Returns answer + "[PromptThin] Tokens: 420 in / 85 out"

Per-request controls

Vision and image requests

The semantic cache fingerprints a request from the text portion of its messages only — image content blocks are never embedded. This means two requests with identical text but different images would otherwise hash to the same cache key and risk returning a cached answer about the wrong image.

To prevent this, PromptThin skips the semantic cache by default for any request containing image content — covering OpenAI/Anthropic-style image blocks (`image_url`, `image`, `input_image`) and Gemini-style inline/file image parts, in any message of the conversation, not just the latest one.

All other savings routes (compression, pruning, routing, thinking budget) are unaffected and still apply normally to vision requests.

If you have a workload where this is safe — for example, the image is decorative and the answer is fully determined by the text — you can opt back in for a single request:

curl -X POST https://promptthin.tech/v1/chat/completions \
  -H "X-API-Key: ts_your_key" \
  -H "X-Cache-Control: force-image-cache" \
  -H "Content-Type: application/json" \
  -d '{ ... }'

This is a per-request override, not a setting — each request containing images still needs `force-image-cache` explicitly to be cached.

Protecting parts of a prompt from compression

Prompt compression (Route B) compresses the entire text of your last user message. If a part of that message must survive byte-for-byte — JSON you're going to parse, code, an exact template, anything format-sensitive — wrap it in markers instead of disabling compression for the whole message:

Please summarize this:
>>
{"id": 123, "exact": "json"}
>>

How it works under the hood:

1. Before compression runs, each `>>...>>` block is extracted and replaced with a unique placeholder token.

2. LLMLingua-2 compresses the remaining text, with the placeholder tokens hinted as force-preserved.

3. After compression, PromptThin verifies every placeholder token survived intact. If even one was split, stripped, or altered by the tokenizer, the entire compression result for that message is discarded and the original uncompressed message is sent instead — this guarantees the protected content is never silently corrupted, at the cost of losing compression savings on that one message.

4. If the verification passes, the placeholders are replaced back with the original protected text and the markers are removed from the final message.

Malformed markers (e.g. unmatched start/end tags) are treated as plain text — the message compresses normally without raising an error.

This is a finer-grained alternative to `X-Compress-Control: no-compress`, which disables compression for the whole request rather than just a portion of one message.

Cache correctness

The semantic cache is only ever populated with responses that PromptThin can verify are well-formed. Before any response is written to the cache, it must pass all of the following checks:

• No transport or provider error — the upstream call must return HTTP 200. Timeouts, gateway errors, and provider-side error payloads are never cached.
• Non-empty, substantive content — responses with empty or near-empty text (e.g. a thinking model that returned nothing because its reasoning budget consumed the entire output) are rejected.
• No bad finish reason — provider-specific signals that the response was cut short or blocked are checked: OpenAI/Groq content-filter stops, Gemini `SAFETY` / `RECITATION` / `OTHER` / `BLOCKLIST` finish reasons, and Anthropic `stop_reason == "error"` all skip the cache.

These checks catch errors and malformed responses, not factual correctness — PromptThin has no way to verify whether a fluent, well-formed answer is actually *right*. If you're working with prompts where you don't want a possibly-imperfect answer cached for future similar requests by anyone, send `X-Cache-Control: no-cache` on that request. It skips both the cache read and the cache write, so that response is never reused.

For multimodal requests specifically, see Vision and image requests above — those are skipped by default regardless of response quality, because the risk there is a cache-key collision, not a bad response.

Pricing

Start free trial →

Security

• Provider keys encrypted with AES-256 — never in logs or responses
• Email verification required before making API calls
• Strong passwords enforced (8+ chars, upper, lower, number, special character)
• All traffic HTTPS only
• Keys stored in GCP Secret Manager

FAQ

Do I need to change my code?

No. Set two environment variables.

Does PromptThin slow down my requests?

Cache hits completely skip the LLM call — dramatically lower latency. Cache misses add <2ms overhead.

Does PromptThin reduce my claude.ai / ChatGPT chat quota usage?

No. Managed chat interfaces like claude.ai and ChatGPT control their own API calls internally — PromptThin cannot intercept those. PromptThin works when *you* control the API call: your own code, AI agents, or self-hosted chat UIs (e.g. Open WebUI, LibreChat). If you're hitting quota limits in a managed chat app, the fix is to use a self-hosted UI that calls the API directly through PromptThin instead.

What if I want to pass my provider key directly?

client = OpenAI(
    base_url="https://promptthin.tech/v1",
    api_key="ts_your_key",
    default_headers={"Authorization": "Bearer sk-your-openai-key"},
)

PromptThin detects the key prefix and uses it directly.

Can I use multiple providers?

Yes. Register keys for each provider. PromptThin routes to the right one based on the model name.

What happens after the 7-day trial?

Your card is charged $4.99 for the first month, then $11.99/month. Cancel anytime from the dashboard — no charge if cancelled within 7 days.

Contact

• Website: promptthin.tech
• Enterprise: kaztesla2025@yahoo.com
• Issues: Open an issue on this repository