Step 1: Who may read a document
Kestrel Software is building an internal assistant over its knowledge base (docs.json): expense and
leave policies, engineering and sales guides, and a few documents that must stay closed, such as salary
bands, a restructuring plan and board minutes. Nine people in users.json will test it.
Permissions have to be enforced before the model sees anything. A model told "don't reveal confidential documents" can still be talked into it; a model that never receives them cannot leak them.
Each document's meta says who may read it:
allow: groups that may read it, or a single person as"user:<id>".deny: groups that may never read it, even if another of their groups is allowed. Deny wins.confidential: readable only by users withclearance, on top ofallow.
1. Write can_read(user_id, doc) with the three rules in this order: deny, then confidential, then allow
(by group or by user:<id>). Return True or False.
2. Run to print the access matrix, one row per user, and look at Dev (a contractor in the eng group) and Zoe (an exec without clearance).
access.py, the file you edit64 lines
"""Permission-aware retrieval for Kestrel's internal assistant. Users are ids from users.json ("ana", "ben", ...)."""
import hashlib
import harness
# ---------- Step 1: who may read a document ----------
def can_read(user_id, doc):
"""True if the user may read the document: deny wins, confidential needs clearance, then allow by group or user."""
# TODO (Step 1): user = harness.USERS[user_id], m = doc["meta"].
# 1. If any of the user's groups is in m["deny"], return False (deny wins).
# 2. If m["confidential"] and the user has no clearance, return False.
# 3. Otherwise True if the user shares a group with m["allow"], or "user:<user_id>" is in m["allow"].
raise NotImplementedError("Step 1: write can_read()")
# ---------- Step 2: which documents apply ----------
def in_scope(user_id, doc, today=harness.TODAY):
"""True if the document applies to the user today: their region (or "all"), already in force and not expired."""
raise NotImplementedError("in_scope() arrives in Step 2")
def visible(user_id, today=harness.TODAY):
"""Every document the user can read that is in scope for them today."""
return [d for d in harness.DOCS if can_read(user_id, d) and in_scope(user_id, d, today)]
# ---------- Step 3: search inside the fence ----------
def secure_search(user_id, question, k=3):
"""The k best documents for the question among those visible to the user."""
raise NotImplementedError("secure_search() arrives in Step 3")
# ---------- Step 4: answer from what the user may see ----------
REFUSAL = "I can't find that in the documents you have access to."
ANSWER_PROMPT = """You answer Kestrel employees' questions using only the documents below.
Cite the document id in square brackets after each fact, for example [d04].
If the documents do not answer the question, reply with exactly this sentence and nothing else:
""" + REFUSAL + """
Documents:
{documents}"""
def answer(user_id, question):
"""{"text": the model's answer, "sources": the ids of the documents it was given}."""
raise NotImplementedError("answer() arrives in Step 4")
# ---------- Step 5: a cache that cannot leak ----------
_answers = {}
def cache_key(user_id, question):
"""Same key only when the answer would be the same: same visible documents and same question."""
raise NotImplementedError("cache_key() arrives in Step 5")
def cached_answer(user_id, question):
"""answer(), reused whenever cache_key() matches an earlier call."""
key = cache_key(user_id, question)
if key not in _answers:
_answers[key] = answer(user_id, question)
return _answers[key]docs.jsonevaluate.pyharness.pyprobes.jsonquestions.jsonusers.json