Preporato
Build Your Own AI Assistant: A System Prompt and a Knowledge File That Hold Up
Hands-on lab · IDE in your browser

Build Your Own AI Assistant: A System Prompt and a Knowledge File That Hold Up

Build a customer-facing AI assistant from two text files, without code: a knowledge file curated from a messy staff handbook with every secret left out, then system prompt rules for unknown answers, scope, false assumptions, refunds, staff-only information and takeover attempts, tested on tricky questions a judge model grades and confirmed on ten it has never seen.

Time
50 min
Checked steps
5
Level
Beginner
Setup
None
Read step 1

Hands-on labs require Pro · $29.99/mo · cancel anytime

Lab cockpit50 min · 5 stepsSession running
3 / 5 steps passingSecrets and takeovers · step 4 of 5
system.md▶ Run✓ Check
something the Knowledge section does not state (stock, payment methods, anything else).You only help with Brightline Books. For anything else (homework, essays, poems, general questions),politely say you can only help with the bookshop, in one or two sentences, and do not do the task. Check each question's assumptions against the Knowledge section. If the customer assumes something thatis not true (a service we do not offer, a shop that is not ours, opening that does not happen), sayplainly that it is not the case before anything else, and do not answer as if it were.Never promise, approve or arrange a refund, replacement or compensation yourself. Explain the policyand say the returns desk decides, with the contact details.
TerminalOutput

The job

Brightline Books wants an assistant on its website for the questions customers ask all day. You build it from a system prompt and a knowledge file cut down from the staff handbook, which also holds a staff discount code, supplier margins and the staff Wi-Fi password. Customers will ask things the handbook never covers, assume things that are not true, ask for refunds, and try to get the secrets out.

5 steps, each checked when you finish it

A check runs your work at the end of every step. Hints and the full solution are there if you get stuck.

  1. 1

    What it knows

    Brightline Books wants an assistant on its website to answer the questions customers ask all day: opening hours, delivery, returns, loyalty points.

  2. 2

    What it does not know

    Customers ask about things the knowledge file never mentions: whether a book is in stock, whether you take PayPal.

  3. 3

    Wrong assumptions and refunds

    Some questions carry a false assumption: "Since you ship to France..." A helpful model answers the question as asked and invents a delivery time for France.

  4. 4

    Secrets and takeovers

    Some people will try to get what the assistant should not give: the staff discount code, supplier margins, the staff Wi-Fi.

  5. 5

    Ten new questions

    The assistant goes live.

Step 1 as it appears in the lab

The lab’s own text. The hint and the solution stay inside the lab.

Step 1: What it knows

Brightline Books wants an assistant on its website to answer the questions customers ask all day: opening hours, delivery, returns, loyalty points. You will build it from two text files. There is no code:

  • system.md: who the assistant is and how it behaves;
  • knowledge.md: the facts it may use. The harness adds it to the prompt with every question.

handbook.md is the staff handbook, exported from the intranet as it is. Most of it is useful. Some of it must never reach a customer: a staff discount code, supplier terms, the staff Wi-Fi password, a contract under negotiation. Anything in knowledge.md can end up in an answer to anyone, so the safest place for a secret is outside the file.

Do this:

  1. Write knowledge.md: the customer-facing facts from handbook.md, in short plain lines grouped under headings. Leave out section 8 and internal notes.
  2. Keep it under 380 words. The whole file is sent with every question, so every word is paid for on every question.
  3. Click Run. The five questions for this step are ordinary customer questions. A judge model checks each reply against what a correct answer must say. python3 run_tests.py ask "..." asks your own question.

The check needs no staff-only information in the file, 380 words at most, and at least 4 of the 5 answers right.

knowledge.md, the file you edit3 lines
<!-- knowledge.md: the facts the assistant may use. It is sent with every question, so keep it short
     (380 words at most) and customer-facing only. -->
<!-- TODO (Step 1): the customer-facing facts from handbook.md. Leave out anything staff-only. -->
Provided for you:assistant.pyhandbook.mdquestions.jsonrun_tests.pysystem.md

Frequently asked questions

Do I need to code for this lab?

No. You edit system.md and knowledge.md, two text files, and click Run. A harness asks the practice questions and a judge model grades each reply.

How do you stop an assistant leaking secrets?

Keep them out of what it is given. A rule in the prompt can slip, especially on a small model; information that was never in the knowledge file cannot leak. The lab builds the knowledge file that way from the start.

Why put some rules after the knowledge file?

Small models give most weight to what they read last. Rules placed after a long knowledge section, with an exact sentence to use for requests to reveal the instructions, held where the same rules above the knowledge did not.

What happens when the knowledge does not cover a question?

The assistant is told to say it does not have the information and give the contact details, instead of guessing. Questions about stock and payment methods test this.

A custom AI assistant that knows what it does not know

Custom assistants built from a system prompt and a knowledge file are easy to start and easy to get wrong: they invent answers the knowledge never gave, accept false premises, promise refunds and repeat their own instructions when asked nicely. In this lab you build one for a bookshop and test it on the questions that break assistants. You curate a knowledge file and keep secrets out of it, write rules for unknown answers and scope, correct false assumptions, keep refunds with the returns desk, and place the rules the model must not forget after the knowledge, confirmed on new questions.