CCAR-P Cost & Latency Optimization: Caching, Batch, Routing [2026]

Cost and latency questions on the Claude Certified Architect - Professional (CCAR-P) exam rarely ask you to name a feature. They describe a system that works, tell you it costs three times its budget or misses a latency SLA, and ask which lever an architect pulls first. This guide covers the mechanisms behind each lever: where tokens and milliseconds go, how prompt caching decides what it will reuse, why the Message Batches API buys price and throughput at the expense of waiting time, when model-tier routing holds accuracy, how context management keeps agent loops affordable, what streaming does and does not fix, and how structured output removes the retries that quietly double a bill. Two worked scenarios show how the levers combine into the answer CCAR-P scores as correct, and a table summarizes what each lever saves and costs.

Why CCAR-P treats cost and latency as architecture

Two domains carry this material. Claude Models, Prompting & Context Engineering (13%) covers selecting models on capability, cost, and latency trade-offs, optimizing context windows and token usage, and prompt reuse through caching, modular prompts, and Skills. Evaluation, Testing & Optimization (16%) covers optimizing token usage, latency, and cost-performance trade-offs, plus the logging and observability that make optimization measurable. Together that is 29% of the blueprint, and Solution Design (17%) leans on the same judgment whenever a stem names a cost ceiling or a performance SLA as the value pillar.

The tested principle is proportionality backed by evidence. A stem describes a workload profile (volume, complexity, latency tolerance, budget) and expects you to identify the dominant cost driver, pick the lever that addresses it without breaching the stated quality floor, and prove the change with an evaluation before rolling it out. Answers that reach for the most capable model everywhere, or the cheapest model everywhere, read as junior-level thinking on this exam, and so does any answer that optimizes before measuring.

Where the money and the milliseconds go

Every lever maps to one of five cost drivers, and reading a stem well means spotting which driver dominates.

Input tokens. Everything you send counts: tool definitions, the system prompt, conversation history, retrieved documents, and the user's message. The Claude API is stateless, so the whole context is resent and re-billed on every request. Input tokens are also what the model processes before its first output token, so they drive time-to-first-token as well as cost.

Output tokens. Output is priced higher per token than input on every model listed on the official pricing page. Verbose answers, unrequested explanations, padded JSON, and reasoning tokens (where a model's thinking is enabled, those tokens bill as output) all land here, and generation time scales almost linearly with output length.

Context length. In a multi-turn conversation or agent loop, each turn resends every earlier turn. Cost per turn and prefill latency both grow with history, so a session that starts cheap becomes expensive by turn twenty unless something bounds the context.

Retries. A parse failure, a timeout, or a validation error means sending the entire input again. Retries are the driver architects most often forget, because they hide inside application logic until you log per-request usage.

Tool round-trips. In standard tool use, each tool call is a full request: the model emits a call, your code runs it, and the result goes back with the whole context attached. An agent that makes eight tool calls pays for its context nine times and adds the tool results to it as it goes.

Prompt caching: what gets cached and when it pays

Prompt caching lets the API store the processed form of a stable prompt prefix so that later requests reuse it instead of reprocessing it. The prompt caching documentation describes the mechanism, and CCAR-P expects you to reason from it.

What is cached. The cache is a prefix match. Content is rendered in a fixed order (tools, then system, then messages), and you mark a breakpoint with a cache_control block on the last content block you want reused. Everything up to that breakpoint is eligible for reuse; everything after it is processed fresh. You get up to four explicit breakpoints per request, which lets you cache sections that change at different rates (tools that never change, a weekly system prompt, a per-session document set).

Prefix stability. Because the match runs on exact content from the start of the prompt, any change anywhere in the prefix invalidates everything after that point. The classic mistakes all put variable content early: a timestamp or user ID interpolated into the system prompt, tool definitions added, removed, or reordered between requests, per-user tool sets, and shuffled document order. Switching models also invalidates the cache, since caches are scoped to a model. The design rule is stable-first, variable-last: tools and shared instructions at the front, per-request content after the last breakpoint.

Lifetime and minimums. Cache entries expire after five minutes by default, with a one-hour option, and each read within the lifetime refreshes it. There is also a minimum cacheable length that varies by model (the documentation lists thresholds from 512 to 4,096 tokens depending on the model), and prompts below the minimum are processed without caching and without an error, which is why you verify caching from the response.

Verification. The response usage object reports cache_creation_input_tokens (written this request), cache_read_input_tokens (served from cache), and input_tokens (processed at full price). If cache reads stay at zero across requests that should share a prefix, the prefix is changing.

Economics. Per the prompt caching page, cache writes carry a premium over base input price (1.25 times for the five-minute lifetime, 2 times for the one-hour lifetime) and cache reads are billed at 0.1 times base input price. That shape tells you when caching pays: a long, stable prefix reused many times within the lifetime. Agent loops, high-traffic endpoints sharing one system prompt, and repeated questions over the same document set are the textbook cases. It does not pay for infrequent calls (the entry expires between them), short prompts (below the minimum, nothing is cached), or prompts that differ from the first token (there is no shared prefix). Caching also cuts latency, because a cached prefix skips the input processing that dominates time-to-first-token on long contexts.

Message Batches API: cheaper and asynchronous

The Message Batches API accepts many Messages requests in one submission, processes them asynchronously, and lets you poll for results. The batch processing documentation states the terms an architect designs around.

Batches are billed at 50% of standard API prices. A single batch is limited to 100,000 requests or 256 MB, whichever comes first. Most batches finish within an hour, but the guarantee is looser: results become available when every request completes or after 24 hours, and requests still unprocessed at 24 hours expire without being billed. Results are retained for 29 days and can come back in any order, so you match them to requests by the custom_id you assigned. Batch requests support the same Messages features as synchronous calls, including prompt caching, and the two discounts stack; because a batch can run longer than five minutes, the documentation recommends the one-hour cache lifetime for batches sharing context.

The design implication is the point CCAR-P tests. Batching is the right lever when latency does not matter: nightly classification runs, backfilling labels over an archive, bulk summarization, generating evaluation-set outputs, periodic reports. It is the wrong lever for anything user-facing or SLA-bound in seconds or minutes, because you cannot promise a response time inside a window that may stretch to a day. A stem that says "overnight," "bulk," or "no latency requirement" is pointing at batch. A stem that says "chat," "interactive," or "p95 under three seconds" rules it out regardless of the discount.

Model-tier routing

Anthropic's models overview positions its current lineup as a fastest tier, a balanced speed-and-intelligence tier, and a top-capability tier, with latency and price rising as capability rises. Exam questions describe tiers, and you should verify current model availability on the models page, since the lineup changes faster than any exam blueprint.

The routing rule is proportionality. High-volume, well-bounded tasks (classification, extraction, routing, simple lookups) belong on the fast tier. The balanced tier is the production default for most enterprise workloads. The top tier is reserved for complex multi-step reasoning, long-horizon agentic work, and high-stakes synthesis where a failure costs more than the model does. Two mechanics matter beyond the tier list. Where a model exposes a reasoning-depth control (an effort or thinking setting), treat it as a per-route dial: lower for routine work, higher where evaluations show reasoning failures. And because caches are model-scoped, switching models mid-conversation throws away the cached prefix; the pattern that preserves it keeps the main loop on one model and hands narrow sub-tasks to a cheaper model in a separate call.

Routing beats a single model at scale. A common architecture uses a fast-tier model to triage requests by difficulty and escalates the hard slice to a capable tier, so most traffic pays fast-tier prices while the accuracy floor holds. The evidence requirement is the same as everywhere on this exam: validate the fast tier on a golden set for that route before you move traffic, and monitor per-route accuracy afterward. Downgrading a tier because it is cheaper, with no evaluation, is a distractor pattern in Domain 2 and Domain 4 questions alike.

Context management: keeping the per-turn bill flat

Context management is the discipline of controlling what enters the model's context window and how long it stays there. Since every turn resends the whole context, it decides whether a long agent session stays affordable.

Prune tool output. Tool results are the fastest-growing part of most agent contexts. Return only the fields the model needs, truncate long results, and summarize large payloads before they enter context. Where the API supports it, composing several tool calls inside a code execution step (programmatic tool calling) keeps intermediate results out of the model's context and returns only the final output.

Compaction and context editing. Compaction summarizes older turns into a compact block when a conversation approaches its limit, preserving the thread while dropping the raw history. Context editing clears stale tool results and thinking blocks without summarizing them. Both keep the transcript lean and both lose detail, so the trade-off is tested with an evaluation that checks whether later turns still have what they need.

Summaries and memory. For long-running work, a rolling summary or a file-based memory the agent reads and writes gives you a bounded context per turn.

Progressive discovery. Loading tools and reference material only when a task calls for them (tool search, Skills that load on demand) keeps each request small. The exam pairs this with the capability-bloat failure mode from Domain 3: an agent with dozens of always-loaded tools pays a token tax on every request and routes worse. Adding tools mid-conversation normally invalidates the cache, which is another reason to prefer discovery mechanisms that append to the tool list.

Streaming and time-to-first-token

Streaming delivers output tokens as they are generated instead of after the full response completes. It changes when the user sees something, and it does nothing to the number of tokens billed or the total generation time. That split is the whole point of the topic. Time-to-first-token (TTFT, the delay before the first output token arrives) is dominated by input processing, so it falls with a shorter context or a cached prefix. Total time is dominated by output length, so it falls when you constrain output.

The design consequences follow directly. Stream any user-facing path so the interface feels responsive, and measure TTFT and end-to-end latency separately at p95 and p99, because they respond to different levers. Streaming also protects long generations from request timeouts, which is why the SDKs expect it for large output budgets. Streaming has no place in a batch: the two sit at opposite ends of the latency spectrum, and a stem that mixes them is testing whether you know it.

Structured output to cut retries

Structured outputs constrain the model's response to a JSON schema you supply, and strict tool use guarantees that tool call inputs validate against the tool's schema. The structured outputs documentation covers the mechanics; the architectural value is retry elimination. Every parse-failure retry re-bills the entire input and adds a full round of latency, and in a pipeline running millions of calls a small retry rate becomes a large cost line. Constraining the shape removes that failure class, shortens prompts (no more paragraphs of "respond only with valid JSON"), and deletes the repair code behind the call.

Two caveats keep this honest. Schema-valid output can still be wrong, so structured output does not replace accuracy evaluation. And the documentation notes a one-time schema compilation cost on first use, after which the schema is cached, so a pipeline generating a fresh schema per request gives that benefit back.

Measuring cost per successful task

The metric CCAR-P wants you to reason with is cost per successful task: total spend for a unit of work (model calls, retries, tool calls, escalations to a stronger tier, and human review) divided by the number of units that met the acceptance criteria. Cost per call is a trap. A cheaper model that fails more often can cost more per success once retries and review time are counted, and a caching change that saves tokens while hurting accuracy is a loss.

Instrumentation makes the metric possible. Log the usage fields on every response (input, output, cache write, cache read), tag each request with route, model, and prompt version, count retries and tool round-trips per task, and record latency percentiles and TTFT alongside. Then evaluate changes one variable at a time: an A/B test or shadow run that changes prompt version, model tier, or retrieval depth in isolation, scored against the same golden set. The CCAR-P evaluation strategy guide covers the metric selection and regression gates this loop plugs into.

The optimization levers side by side

Worked scenario 1: the support assistant that costs too much

A financial-services support assistant runs on the balanced tier. Every request carries a 6,000-token policy preamble plus the customer's conversation, traffic peaks at thousands of concurrent chats, the SLA requires a p95 first response under three seconds, and the monthly bill is roughly triple the forecast. Accuracy on the evaluation set is at target. The stem asks for the first optimization.

Walk the drivers. The dominant input cost is a large prefix identical across every request, and the same prefix drives TTFT because the model processes it before answering. Prompt caching addresses both: the preamble becomes a cached prefix, per-request cost drops to the cache-read rate for those tokens on hits, and first-token latency falls with it. Streaming the response keeps the interface responsive within the SLA. Neither change touches accuracy, which the stem says is at target.

Now the distractors. Moving to the Message Batches API halves the price and destroys the SLA, since a chat cannot wait inside a 24-hour window. Downgrading to the fast tier might work, but the stem gives no evaluation evidence for it, and choosing it first risks the accuracy that is currently fine. Rewriting the preamble to be shorter is a real technique, and it is the wrong first move here because it changes model behavior and would need re-evaluation, while caching changes nothing the model sees. The exam-correct choice is prompt caching on the stable prefix (with cache-read tokens verified in usage), streamed to the user, with tier experiments only after an evaluation supports them.

Worked scenario 2: nightly ticket classification on the top tier

An operations team classifies roughly 400,000 support tickets each night into 30 categories on the top-capability tier through synchronous calls. There is no latency requirement beyond finishing before the morning shift, the job runs three times over budget, and the accuracy target is 95% agreement with human labels. Select the two changes an architect should make.

The two drivers are model tier and pricing mode. Classification into a fixed label set is exactly the bounded, high-volume task the fast tier exists for, so the first change is a tier downgrade validated on a labeled sample against the 95% floor before the full cutover. The second is moving the workload to the Message Batches API, since a nightly job with an eight-hour window fits inside the batch processing window and takes the discount without changing outputs. Prompt caching adds a further gain if the shared instructions and label definitions form a long stable prefix (with the one-hour lifetime inside a batch), but on its own it is the smaller lever here.

The distractors keep the expensive configuration and trim around it: lowering the output token limit (classification output is already tiny), adding structured output (useful for reliability, marginal for this cost problem), or shortening the label descriptions without measuring the accuracy effect. The exam-correct pair is fast-tier routing with an evaluation gate and the Message Batches API.

How CCAR-P writes these questions

Optimization stems follow recognizable patterns, and the traps repeat.

• The flagship-everywhere trap. A design uses the most capable tier for every workload. The correct answer routes by task difficulty with evaluation evidence.
• The batch-for-chat trap. The discount is real, and the SLA still fails. Any interactive path rules batch out.
• The cache-that-never-hits trap. A design "enables caching" on a prefix that includes a timestamp, a user ID, or per-user tools. Prefix stability is the tested detail.
• The optimize-before-measure trap. Options that cut cost with no evaluation lose to options that cut cost behind an eval gate.
• The streaming-saves-money trap. Streaming changes perceived latency and timeout risk; it changes no token counts.
• The cheapest-per-call trap. A cheaper model with a higher failure rate can cost more per successful task once retries and review are counted.

The CCAR-P practice questions with explanations include optimization scenarios written at this altitude, and the cheat sheet condenses the caching and tier rules for final review.

Frequently asked questions

Next steps: work through the CCAR-P evaluation strategy guide to see how the evaluation gates in this article are built, then take a timed test from Preporato's CCAR-P practice exams to see whether your optimization instincts survive a multi-constraint stem. All six tests plus the flashcard deck are included in Preporato Pro; see pricing for current plans.

Sources:

• Anthropic: Prompt caching
• Anthropic: Batch processing (Message Batches API)
• Anthropic: Models overview
• Anthropic: Pricing
• Anthropic: Structured outputs
• Anthropic Partner Academy: CCAR-P certification