The agent pipeline behind a Chrome extension that trims wasted tokens from prompts inside ChatGPT and Claude.
- Role
- Agents & orchestration
- Timeline
- Aug 2026 – Present
- Team
- Capstone, CUNY Hunter
- Pipeline
- Auditor → Optimizer → Validator
Overview
Token Watching is a capstone project at CUNY Hunter: a Chrome extension that audits a prompt as you write it in ChatGPT or Claude and offers a shorter version with one click.
My part is the agent layer. I built the three agents that audit, rewrite and check each prompt, the orchestrator that chains them, and the shared token counting they rely on.
The problem
Prompts are full of tokens that don’t change the answer: greetings, hedging, repeated instructions. Every one of them is billed, on every call.
Cutting them is easy. Cutting them safely is not. A rewrite that drops a word limit, a negation or a line of code is cheaper and wrong, and one bad rewrite costs more trust than any number of tokens saved. The tool also can’t spend more tokens finding waste than it saves.
What I built
Auditor
Counts a prompt’s tokens the way the target model bills them, estimates its cost, and flags waste: filler words, an unusually high token-to-word ratio, runs of blank lines. Only a long prompt that passes every rule check gets a second look from Claude Haiku, which reports issues through a tool call so they arrive as a list.
Optimizer
Rewrites the prompt using the Auditor’s issues, through a structured-output schema (Pydantic) so the rewrite lands in its own field and no “Here’s your optimized prompt:” preamble can leak into what the user sees. On a representative prompt it went from 31 tokens to 8, a 74% cut, with the request intact.
Validator
Checks the rewrite against the original and returns a 1–10 score, a pass/fail verdict and a one-sentence reason. Before scoring, the model writes out READ, ANSWER, CITE and THINK steps: list the original’s requirements, check each one, quote anything lost, decide whether it matters. That trace is kept, so a verdict can be inspected rather than trusted.
Orchestrator
Runs the three agents in order and decides when to stop. A clean prompt never reaches the Optimizer. A rewrite that saves nothing is dropped before validation. A rejected rewrite goes back to the Optimizer with the Validator’s reason, up to three attempts, and if none pass the user gets their original prompt back.
Shared token counting
One module counts tokens for every agent: locally with tiktoken’s o200k_base encoding for ChatGPT, or through Anthropic’s count_tokens endpoint for Claude, with the message framing measured once and subtracted. Counts are cached, so the Auditor and Optimizer share one API call per prompt.
Design decisions
Don’t waste tokens to save tokens
Every stage takes the cheapest route first. Deterministic checks run before any model call, identical prompts pass validation without one, and most prompts finish the pipeline on rule checks alone.
Meaning over length
A rewrite the Validator rejects is never shown, even if it is shorter. Getting the same answer for more tokens beats a different answer for fewer.
Retries carry their history
Each failed attempt goes back to the Optimizer as the rejected rewrite plus the reason, and every earlier rejection is included, so the next try sees exactly what it wrote and what that broke.
Prompts are data
The prompt under review is wrapped in tags with an instruction to treat it only as text, so a prompt that says “ignore previous instructions” or “say none found” can’t steer its own audit or rewrite.
Failures fall back
If the Auditor’s Claude review hits a rate limit or network error, it logs a warning and returns the rule-based result. A refused or cut-off rewrite keeps the original prompt.
Difficulties & fixes
Most of the hard problems were about trusting a model’s output. These took the most care.
Almost every prompt was flagged
The token-to-word check fired above 1 token per word, but normal English averages about 1.3, so even “What is the capital of France?” was flagged and severity was almost never Low.
Raised the threshold to 1.6, just above the average, and turned the clean-prompt test from an expected failure into a passing test.
Every prompt cost $0.000
The cost estimate rounded to three decimals, and a single prompt costs a fraction of a cent, so anything under about 170 tokens showed as free.
The function returns the raw value and rounding moved to display. The comparison table shows cost per 1,000 runs, which is also closer to how real apps pay.
“Return it unchanged” wasn’t reliable
Asking the model to copy an already-concise prompt back exactly failed on one-word differences, which then counted as a rewrite.
Added an explicit unchanged flag to the schema, generated before the rewrite. When it is set, the original is kept no matter what else the model wrote.
Retries repeated the same mistake
The Validator’s reason reached the Optimizer as one more line in the issue list, without the rewrite it referred to.
Rejections are passed as (rewrite, reason) pairs in their own list, kept apart from the Auditor’s issues.
Every rewrite would fail validation
After the Validator switched to step-by-step reasoning, its 300-token reply limit cut it off before the SCORE line, so every rewrite parsed as score 1 and the pipeline would burn three attempts for nothing.
Raised the limit to 1,024 tokens. An unparseable reply still counts as a fail, never a pass.
Testing
The agents call a paid API, so the main test suite never does. Each agent and the orchestrator have offline pytest suites that swap in a fake client, checking what gets sent, how replies are handled, the order of calls between agents and when the retry loop stops. They run in CI on every pull request across Python 3.12 to 3.14, alongside a pylint quality gate.
To test the Validator’s judgment, not just its code, a separate live suite calls the real API with two known cases: a rewrite that only removes filler must pass, and one that drops a word limit and a JSON format must fail. It runs on its own workflow whenever the Validator changes.
View on GitHub ↗