Preporato
Classify 500 Spreadsheet Rows with AI and Measure the Error Rate
Hands-on lab · IDE in your browser

Classify 500 Spreadsheet Rows with AI and Measure the Error Rate

Sort a real-looking customer survey spreadsheet with a language model and no code: write a category prompt a program can read, batch 500 rows into a few calls, measure the error rate honestly with a blind hand-checked sample and its confidence range, fix the overlaps with the manager's house rules, and confirm the result on next month's survey.

Time
50 min
Checked steps
5
Level
Beginner
Setup
None
Read step 1

Hands-on labs require Pro · $29.99/mo · cancel anytime

Lab cockpit50 min · 5 stepsSession running
3 / 5 steps passingApply the manager's house rules · step 4 of 5
prompt.md▶ Run✓ Check
- delivery: late, lost, missed or misdelivered parcels; couriers and drivers- damaged: a book or gift that arrived damaged, or was faulty or misprinted- price: prices, discounts, loyalty points and value for money- website: the website or app: logging in, accounts, search, basket, checkout, card payments, speed- staff: how a member of staff treated the customer, in a shop, on the phone or on live chat- stock: a book or range that was out of stock, sold out, never restocked or cancelled for lack of stock- praise: praise with nothing to fix- other: questions, requests, and blank or meaningless answers
TerminalOutput

The job

Brightline Books' September survey came back with 500 free-text comments, and the customer experience manager wants each one in one of eight categories by Friday, with an honest figure for how many are wrong. You write the prompt, batch the rows so the job takes seconds, check a random sample by hand to estimate the error rate, turn the disagreements into house rules, and run next month's survey to prove the prompt still holds.

5 steps, each checked when you finish it

A check runs your work at the end of every step. Hints and the full solution are there if you get stuck.

  1. 1

    Sort 40 labelled rows

    Brightline Books' September customer survey came back with 500 free-text comments.

  2. 2

    Batch all 500 rows

    With batch_size: 1 every comment is a separate call, and every call resends your whole prompt.

  3. 3

    Spot-check the model by hand

    The manager will ask one question: how many are wrong?

  4. 4

    Apply the manager's house rules

    You sent the manager the rows where you and the model disagreed, and the answer is in house-rules.md.

  5. 5

    Run it on next month's survey

    The real test of a classifier is next month's data.

Step 1 as it appears in the lab

The lab’s own text. The hint and the solution stay inside the lab.

Step 1: Sort 40 labelled rows

Brightline Books' September customer survey came back with 500 free-text comments. The customer experience manager wants every comment in one of eight categories by Friday, with an honest figure for how many are wrong. Nobody has time to read 500 rows, so a model will sort them and you will measure it. There is no code to write: you edit prompt.md and settings.yaml.

Open categories.md for the eight categories. dev.csv holds 40 comments from August with the right category already filled in, so you can measure the prompt before you trust it.

The harness sends your prompt, then the comments numbered 1, 2, 3 ..., and reads your reply one line at a time looking for number: category. A reply the harness cannot read counts as a mistake. A spreadsheet needs a label in every row, so the reply format matters as much as the categories.

Do this in prompt.md:

  1. Replace the one-line prompt. List the eight categories with what each one covers, using the exact words from categories.md.
  2. Say exactly how to reply: one line per comment, in order, the number, a colon and the category, nothing else. Show two example lines.
  3. Click Run. It classifies the 40 dev rows and prints each mistake.

The check needs at least 32 of the 40 rows right.

prompt.md, the file you edit6 lines
<!-- prompt.md: the instructions the model gets. The harness adds the comments after it, numbered
     1, 2, 3 ..., and reads your reply one line at a time, looking for  number: category.
     Notes between these marks are removed before sending. -->
<!-- TODO (Step 1): list the eight categories from categories.md, with what each one covers, and say
     exactly how to reply: one line per comment, the number, a colon, the category. -->
Sort these customer comments into categories.
Provided for you:categories.mdclassify.pydev.csvrun.pysettings.yamlsurvey.csv

Frequently asked questions

Do I need to code for this lab?

No. You edit prompt.md, settings.yaml and a small CSV of hand labels, and click Run. A harness sends the rows to the model and writes the results to a CSV.

Why put many rows in one model call?

Every call resends the whole prompt. Batching 20 rows per call cuts the calls for 500 rows from 500 to 25 and most of the cost with them. Very large batches make a small model skip lines, which the lab lets you see.

How do you know the error rate without labelling everything?

Check a random sample by hand, without looking at the model's answers first, and count disagreements. The lab turns that count into a 95% range, then shows the true rate from an answer key so you can see how close a 30-row sample gets.

What are house rules?

Decisions the business makes about overlapping categories, such as whether a tracking-page complaint is delivery or website. Written into the prompt, they fix the errors that better wording alone cannot.

Classifying spreadsheet rows with an LLM, and knowing how many are wrong

Sorting free-text rows into categories is one of the most useful everyday jobs for a language model: survey comments, support tickets, product feedback. Getting labels is easy. Knowing how many are wrong is the part most teams skip, and it is the number a manager needs. In this lab you classify 500 survey comments with a small open model and no code. You set a reply format a program can parse, batch rows to cut calls and cost, hand-check a blind random sample and turn it into an error rate with a 95% range, fix category overlaps with written house rules, and confirm accuracy on a new month of data.