README.md
Commit 01f96e7. All files · Download the tarball.
# LLM Spend Tripwire
Get a Slack or Discord message when your Anthropic API usage looks wrong.
LLM Spend Tripwire is a free, open-source (MIT) Cloudflare Worker that you run in your own Cloudflare account. Every 5 minutes it reads your organization's usage from Anthropic's Usage and Cost Admin API and posts to your Slack or Discord channel when:
1. **A key spikes.** One API key's busiest recent minute uses more than N times its average tokens per minute (default 5x over the last 6 hours), and more than a minimum you set (default 50,000 tokens in that minute).
2. **A key or model is new.** An API key or model shows up that Tripwire hasn't seen before, including a newly created key that hasn't been used yet.
3. **Keys are busy off-hours.** Usage happens outside working hours and days you set, in your time zone.
It can help you notice leaked or misused keys sooner. It doesn't block anything, revoke keys or change your Anthropic settings. You decide what to do, in the Claude Console.
[](https://deploy.workers.cloudflare.com/?url=https://github.com/swiftlabs-ops/llm-spend-tripwire)
> **Status:** Built to Anthropic's documented [Usage and Cost API](https://docs.anthropic.com/en/api/usage-cost-api) and [List API Keys](https://docs.anthropic.com/en/api/admin-api/apikeys/list-api-keys) endpoints, and tested against responses in the documented format. We have not yet run it against a live Anthropic organization. Try it with the sample replay below first, and tell us what breaks in this repository's Issues tab.
## What it does and doesn't do
| It does | It doesn't |
| --- | --- |
| Read usage in 1-minute buckets, grouped by API key and model | Read prompts, completions or any message content (the API doesn't expose them) |
| List your active API keys (names, IDs and the partial hints Anthropic shows) | Create, disable, rotate or delete keys |
| Post alerts to the one Slack or Discord webhook you configure | Send data to Swift Labs or anyone else |
| Estimate cost as tokens x list price | Show your actual bill |
| Work with Claude Console organizations | Work with OpenAI, Bedrock, Vertex AI, the claude.ai Team plan or Claude Enterprise |
## Before you start
- **A Claude Console organization and an Admin API key.** Anthropic says the Admin API "isn't available to organizations on the claude.ai Team plan or to individual Claude Console accounts." An individual Console account must first set up an organization on the [Organization page](https://platform.claude.com/settings/organization). Then a member with the admin role creates a key on the [Admin keys page](https://platform.claude.com/settings/admin-keys). It starts with `sk-ant-admin`.
- **Know what that key can do.** Claude Console Admin keys have no scopes to pick. Anthropic's docs say "every key has full access to the endpoints that accept it", so the key can also manage members, workspaces and API keys. Tripwire only makes `GET` requests with it, and the code is short enough to read (`src/anthropic.js`). Treat it like any admin credential: store it only as a Worker secret, and delete it in the Console if you stop using Tripwire. Anthropic suggests giving a service like this its own Admin key and setting an expiration when you create it. When the key expires, Tripwire posts an error alert; add a new key with `npx wrangler secret put ANTHROPIC_ADMIN_KEY`.
- **A Cloudflare account.** The Workers Free plan is enough for the defaults: one cron trigger, and one small D1 row read and written per check.
- **A Slack or Discord incoming webhook.** Slack: [create an incoming webhook](https://docs.slack.dev/messaging/sending-messages-using-incoming-webhooks/) (`https://hooks.slack.com/services/...`). Discord: Channel settings, then Integrations, then Webhooks (`https://discord.com/api/webhooks/...`). Tripwire refuses any other host.
## Setup (about 5 minutes)
**Option A: Deploy to Cloudflare button.** Click the button above. Cloudflare copies this repository into your GitHub or GitLab account, creates the D1 database, and asks for the secrets below.
**Option B: Wrangler.**
```sh
git clone https://github.com/swiftlabs-ops/llm-spend-tripwire
cd llm-spend-tripwire
npx wrangler login
npx wrangler secret put ANTHROPIC_ADMIN_KEY # paste your sk-ant-admin... key
npx wrangler secret put ALERT_WEBHOOK_URL # paste your Slack or Discord webhook URL
export TEST_SECRET=$(openssl rand -hex 32) # optional, keep it for the checks below
echo "$TEST_SECRET" | npx wrangler secret put TEST_SECRET
npx wrangler deploy # also creates the D1 database on first deploy
```
(`wrangler secret put` before the first deploy creates the Worker with the secret. If your Wrangler version asks you to deploy first, run `npx wrangler deploy` and then set the secrets.)
Set your time zone and working hours in `wrangler.toml` (for example `TIMEZONE = "America/New_York"` and `WORK_HOURS = "08:00-19:00"`), then deploy again.
**Check it works:**
```sh
# Send a test message to your channel:
curl -X POST -H "Authorization: Bearer $TEST_SECRET" https://llm-spend-tripwire.<your-subdomain>.workers.dev/test-alert
# Run a real check without sending or saving anything, and see what it would post:
curl -X POST -H "Authorization: Bearer $TEST_SECRET" https://llm-spend-tripwire.<your-subdomain>.workers.dev/dry-run
```
The first scheduled run posts "LLM Spend Tripwire is now watching your Anthropic organization." It records the keys and models used in the baseline window, plus your active keys, so they don't trigger "new" alerts.
## Try it without an Admin key (sample replay)
The repository includes a made-up organization (`src/sample.js`) whose usage responses follow Anthropic's documented format. Replay mode runs it through the same detection and formatting code as a real check. It uses an in-memory database and never calls Anthropic.
```sh
node scripts/replay.mjs # print the Slack payloads that would be sent
node scripts/replay.mjs --discord # print Discord payloads
node scripts/replay.mjs --post https://hooks.slack.com/services/... # post them to a test webhook
WORK_HOURS=08:00-15:00 node scripts/replay.mjs # try settings via environment variables
```
On a deployed Worker, `POST /dry-run?sample=1` returns the same payloads, and `POST /dry-run?sample=1&post=1` also posts them to your `ALERT_WEBHOOK_URL`. Both need `TEST_SECRET`.
Sample output (Slack payload, check run):
```text
LLM Spend Tripwire: 5 alerts for your Anthropic organization
- New API key: key "temp-test" (sk-ant-api03-Ef3...vW7X) [apikey_01SampleUnknown000000003] is active, created 2026-11-16T15:18:40Z by user user_01SampleUser.
- New model: claude-fable-5-2 was used (not seen before) by key "temp-test" (...).
- New model: claude-opus-5-5 was used (not seen before) by key "prod-backend" (...).
- Spike: key "prod-backend" (...) used 481,500 tokens in the minute starting 2026-11-16 15:24 UTC (11.6x its average of 41,500/min over the last 6h). Estimated cost of that minute: about $3.25.
- Spike: key "temp-test" (...) used 750,000 tokens in the minute starting 2026-11-16 15:22 UTC (no usage in the baseline window). Estimated cost of that minute: about $10.80.
Costs are estimates (tokens x list price), not your bill. Anthropic's cost data is daily; usage data usually lands within about 5 minutes. Check the Claude Console before acting.
```
## Configuration
Set these as `[vars]` in `wrangler.toml` (or in the Cloudflare dashboard). Bad values fall back to the default.
| Variable | Default | What it does |
| --- | --- | --- |
| `SPIKE_MULTIPLIER` | `5` | Alert when a key's busiest recent minute is more than this many times its average tokens per minute |
| `BASELINE_MINUTES` | `360` | How far back the average looks (60 to 10080). Idle minutes count as zero |
| `MIN_TOKENS_PER_MINUTE` | `50000` | Ignore spikes below this many tokens in one minute |
| `COOLDOWN_MINUTES` | `60` | Repeat a spike or off-hours alert for the same key at most this often |
| `TIMEZONE` | `UTC` | IANA time zone for working hours, e.g. `Europe/Berlin` |
| `WORK_HOURS` | empty (off) | e.g. `08:00-19:00`. Windows that cross midnight (`22:00-06:00`) work. Empty turns off-hours alerts off |
| `WORK_DAYS` | `1-5` | ISO weekdays (1 = Monday), e.g. `1-5` or `1,2,3,4,5,6` |
| `OFF_HOURS_MIN_TOKENS` | `1000` | Tokens in a minute before off-hours use counts |
| `ALERT_NEW_KEYS` / `ALERT_NEW_MODELS` | `true` | Turn "new key" / "new model" alerts on or off |
| `WATCH_KEY_LIST` | `true` | Also list active API keys each check, to catch new keys before they're used |
| `MAX_ALERTS_PER_MESSAGE` | `10` | Lines per message; the rest are summarized as "...and N more" |
| `PRICES_JSON` | empty | Extra or changed prices, USD per million tokens, e.g. `{"claude-sonnet-4-5": {"input": 3, "output": 15, "cache_read": 0.3, "cache_write_5m": 3.75, "cache_write_1h": 6}}` |
| `ANTHROPIC_VERSION` | `2026-10-21` | The `anthropic-version` header |
Secrets: `ANTHROPIC_ADMIN_KEY` (required), `ALERT_WEBHOOK_URL` (required), `TEST_SECRET` (optional; without it `/test-alert` and `/dry-run` return 404).
The cron schedule is in `wrangler.toml` (`*/5 * * * *`). Anthropic says its usage API "supports polling once per minute for sustained use", so `* * * * *` also works. Each check makes one usage request plus one key-list request (more only when results span several pages).
## How it works
- Each check requests `GET /v1/organizations/usage_report/messages` with `bucket_width=1m` and `group_by[]=api_key_id&group_by[]=model` for roughly the last 15 minutes. With `WATCH_KEY_LIST` on, it also requests `GET /v1/organizations/api_keys?status=active`.
- The first run seeds the baseline from hourly buckets (`bucket_width=1h`) covering `BASELINE_MINUTES`.
- Minutes older than 10 minutes are added to a per-key history of 10-minute sums. The baseline is that history's average tokens per minute, ending before the minutes being checked.
- Tokens counted: uncached input, cache writes, cache reads and output.
- Usage with no API key (Console Workbench, and most Claude Managed Agents sessions, per Anthropic's docs) is tracked as one "no API key" entry.
- State is one row of JSON in a D1 database in your account: the baseline sums, the key and model IDs seen, and when each alert last fired.
- If a webhook post fails, state isn't saved, so the next check finds the same alert again. If the Admin API returns an error, Tripwire posts that error at most every 6 hours.
### Cost estimates
Anthropic's Cost API reports cost only in daily buckets ("Only daily granularity (`1d`) is available"), so per-minute alerts use **tokens x list price** from a small table in `src/prices.js`. It was copied from [Anthropic's pricing page](https://platform.claude.com/docs/en/about-claude/pricing) on 2026-11-16 and covers Claude Fable 5.2, Opus 5.5, Sonnet 5.5 and Haiku 5.5 (prompts up to 100k tokens). Dated model IDs match by prefix. Unknown models are priced at Claude Fable 5's rate ($10 input / $50 output per million tokens) and marked in the alert. That's higher than every current model but lower than legacy Claude Opus 4 and 4.1 ($15 / $75), so those are underestimated. Batch, fast mode, long-context, data residency and priority pricing aren't applied. Treat every dollar figure as a rough estimate, not your bill.
## Privacy and security
- **Where data goes:** Tripwire talks to two places: `api.anthropic.com` (the base URL is fixed in code) and the Slack or Discord webhook you set. Nothing goes to Swift Labs. There are no analytics or telemetry. The Worker sends `User-Agent: LLMSpendTripwire/0.1.0`, as Anthropic asks integrations to do.
- **What alerts contain:** API key names, IDs and Anthropic's partial key hints, model names, token counts and estimated costs. Anyone in the channel sees them, so pick the channel accordingly. Full keys are never available to Tripwire, because the API returns only hints.
- **Webhook check:** only `https://hooks.slack.com/services/...` and `https://discord.com/api/webhooks/...` (plus `discordapp.com`, `ptb.` and `canary.` hosts) are accepted. Discord mentions such as `@everyone` are turned off, and Slack control characters are escaped.
- **Secrets** are Worker secrets. They aren't logged, returned in responses or included in error messages.
- **Endpoints:** `GET /` returns "running". `POST /test-alert` and `POST /dry-run` need `Authorization: Bearer <TEST_SECRET>`, compared in constant time, and return 404 when `TEST_SECRET` isn't set. `/dry-run` returns alert text from your real usage to whoever holds the secret.
- **Stored data:** one D1 row in your account (see How it works). To remove everything, delete the Worker and its D1 database, then delete the Admin key in the Claude Console.
## Limitations
- **Delay.** Anthropic says usage data "typically appears within 5 minutes of API request completion, though delays might occasionally be longer." With the default 5-minute schedule, expect alerts roughly 5 to 15 minutes after a spike starts.
- **Estimates, not bills.** See Cost estimates above. Code execution costs aren't in the usage endpoint at all, per Anthropic's docs.
- **Thresholds are simple.** A key with spiky normal traffic may need a higher `SPIKE_MULTIPLIER` or floor. Small or slow abuse can stay under any spike threshold. Google's Threat Intelligence Group describes attackers testing stolen keys with tiny requests (`max_tokens=1`) that slip past volume alerts. "New key" and off-hours alerts may still catch that, but nothing here catches everything.
- **Legacy endpoint.** `/v1/organizations/usage_report/messages` is part of what Anthropic now calls the legacy Usage and Cost API, which "remains supported". Anthropic recommends the newer Usage and Cost Analytics API for new integrations. Its data "typically refreshes within 4 hours" though, too slow for minute alerts, so Tripwire uses the legacy endpoint on purpose. If Anthropic retires it, Tripwire stops working until it's updated, and the error alert tells you.
- **Not covered:** the claude.ai Team plan, Claude Enterprise (Analytics API key), Claude Platform on AWS (the endpoints aren't available there), and Claude on Google Cloud, Bedrock or Microsoft Foundry. Per Anthropic's docs, if you turn on IP allowlisting for your organization, it also covers Admin API calls, and Cloudflare Workers don't call out from a fixed IP you can list.
- **Untested against a live organization.** See Status above.
- **Not affiliated with Anthropic.** "Claude" and "Anthropic" are Anthropic's names, used only to describe what this tool reads.
## Development
No dependencies besides Wrangler (to deploy). Tests use Node's built-in runner with mocked Anthropic responses in the documented format:
```sh
node --test test/*.test.mjs
npx wrangler deploy --dry-run
```
## License
MIT. Copyright (c) 2026 Swift Labs. No warranty: see [LICENSE](LICENSE).
Swift Labs. Not affiliated with Anthropic.