Medical information answers, with requirements added as you go

Calloway Therapeutics (fictional) sells 9 medicines in 10 countries: 9 in Europe and the US. Its Medical Information team of 190 specialists answers questions from doctors and pharmacists by phone, email and web form. Today a specialist spends about 22 minutes per inquiry: searching the document system, finding the approved document, drafting a reply and logging it. The team wants an assistant that drafts the reply with citations. The target is 12 minutes per inquiry.

Volumes:

• 4,000 inquiries a day. A specialist asks the assistant about 3 times per inquiry, so 12,000 queries a day. Assume 22 working days a month.
• The document system holds 41,000 approved documents: country labels, standard response documents, study summaries and FAQs. They average 12 pages of about 550 tokens each.
• Every document carries metadata: product, the countries where it is approved, language, version, status (approved, withdrawn, draft) and effective date.
• About 300 documents change each week.

The customer's rules:

• An answer may use only documents approved for the inquirer's country, and must cite the document, version and section.
• A question about a use outside the country's approved label (off-label) is flagged. The draft may only use the approved standard response for that topic. A specialist decides what to send.
• Any mention of a side effect or a product complaint in an inquiry must open a safety case for the pharmacovigilance team on the same business day.
• A specialist sends every reply.
• A withdrawn document must stop appearing in answers within 1 hour.
• EU inquiries are processed and stored in the EU, and US inquiries in the US.

Assume per query:

• Hybrid retrieval returns 50 candidates. A reranker keeps the top 10 chunks of 500 tokens each.
• A 3,000-token instruction block is the same on every query.
• About 600 tokens of question and conversation.
• About 700 output tokens (a draft reply with citations).

Assume these prices:

• Large model: $3 per million input tokens, $15 per million output tokens. Small model: $1 and $5. Cached input: 10% of the input price.
• Embeddings: $0.10 per million tokens. Reranker: $2 per 1,000 queries.
• Infrastructure per region today, as the customer's cloud team planned it: a search cluster of 3 nodes with 128 GB of memory at $1,100 a node a month, $800 a month of app and ingestion, and $400 a month of log and audit storage. Two regions: EU-1 and US-1. A 32 GB search node costs $300 a month.
• The model is available in EU-1, EU-2 and US-1.

Your tasks:

1. Design the base system: ingestion, chunking, the index and its filters, retrieval, generation with citations, the off-label and safety paths, and the specialist's screen.
2. How do you prove it works before specialists rely on it? Give the eval plan and gates.
3. What does a query cost, and what does a month cost?

The interviewer then adds requirements, one at a time:

1. "EU-1 goes down for 3 hours. EU specialists must keep working. The recovery time objective is 30 minutes and the recovery point objective is 15 minutes. EU data stays in the EU."
2. "Finance says the monthly run cost must halve."
3. "An inspection found a reply that cited a label withdrawn two days earlier. New rules: a withdrawn document must disappear from answers within 15 minutes, and we must be able to show, for 10 years, exactly what any answer was based on."

Write your base design in five boxes, the cost per query, and the size of the search index in GB. Then cover tasks 4 to 6 and answer them one at a time, as the interviewer would add them.