Step 1: Name the columns
Every month Brightline's bookkeeper types about eighty supplier invoices
into a spreadsheet: supplier, invoice number, dates, the purchase order,
the amounts. The invoices arrive as text in every layout imaginable
(open a few in invoices/): neat blocks, chatty letters, tables, and
scans with broken lines.
You will get a language model to fill that spreadsheet, with no code:
you write the instructions in prompt.md, and a harness runs them on
23 invoices whose right values are known, writes the result to
invoices.csv, and checks every cell. prompt.md has your instructions,
the line ---DOCUMENT---, and below it the template each invoice goes
into.
The harness scores cells, and not all mistakes are equal:
| Cell | Cost |
|---|---|
| wrong value | 1 |
| left blank when the invoice has it | 1 |
| filled in when the invoice does not have it (invented) | 3 |
An invented purchase order number is worse than a blank: a blank gets noticed and looked up, an invented number gets paid.
1. Click Run with the starting prompt. The replies are prose, so no cell can be filled.
2. Replace the TODO: ask for one JSON object and nothing else, with
exactly these nine keys, each with a short description of what it holds:
supplier, invoice_number, invoice_date, due_date, po_number,
currency, subtotal, vat, total. JSON is the format the harness
reads; it looks like {"supplier": "...", "total": 12.5}.
3. Run again. Most cells now fill. Look at which columns are still wrong.
prompt.md, the file you edit10 lines
<!-- Notes between these arrows are for you; the harness removes them before sending. -->
You copy fields from supplier invoices into Brightline Books' accounts spreadsheet.
<!-- TODO (Step 1): ask for one JSON object and nothing else, and list the nine keys with a short
description of each: supplier, invoice_number, invoice_date, due_date, po_number, currency,
subtotal, vat, total. -->
Read this invoice.
---DOCUMENT---
<invoice>
{document}
</invoice>docs2table.pyholdout/inv-101.txtholdout/inv-102.txtholdout/inv-103.txtholdout/inv-104.txtholdout/inv-105.txtholdout/inv-106.txtholdout/inv-107.txtholdout/inv-108.txtholdout/inv-109.txtinvoices/inv-001.txtinvoices/inv-002.txtinvoices/inv-003.txtinvoices/inv-004.txtinvoices/inv-005.txtinvoices/inv-006.txtinvoices/inv-007.txtinvoices/inv-008.txtinvoices/inv-009.txtinvoices/inv-010.txtinvoices/inv-011.txtinvoices/inv-012.txtinvoices/inv-013.txtinvoices/inv-014.txtinvoices/inv-015.txtinvoices/inv-016.txtinvoices/inv-017.txtinvoices/inv-018.txtinvoices/inv-019.txtinvoices/inv-020.txtinvoices/inv-021.txtinvoices/inv-022.txtinvoices/inv-023.txtrun_tests.pytruth.json