What an LLM actually is
To read the chat in the film, turn your phone sideways and tap full screen.
Read the film's narration instead
A customer called Leona writes to Cairnway Outdoor's support chat that her tent from order 5102 hasn't arrived. The bot answers her by name and asks when it was due. She says Saturday, and the bot apologizes. Then she asks the bot to have the courier leave the tent with her neighbor, and it asks for the order number she gave in her first message.
Each message Leona sends and the bot's reply to it make one turn, and on every turn the shop's server makes one HTTP request to the model's API. The request carries a body of JSON data that names the model and holds a list called messages.
Each entry in that list has a role and its text, which the API calls content. The shop's instructions to the bot have the role system, and developers call them the system prompt. What Leona typed has the role user, and the bot's earlier replies have the role assistant. The model reads that list, writes one reply, and keeps nothing once the call ends.
You run the shop's code, and you need to find out why the bot asked again. You could switch the bot to a larger model, tell it in the system prompt to remember order numbers, or open the request your server sent on turn 3. Pick the one you try first.
Decision 1: the right first step is to read turn 3's request body.
You open the request body your server sent on turn 3. It holds the system prompt, Leona's second message, the bot's second reply and her new question. Her first message and the bot's first reply, the two that held her name and order 5102, aren't in the list. With only those four entries to go on, asking for the order number was the right thing for the model to do.
Your server builds that list in a function called buildMessages(). To keep each request small, it sends the system prompt, the two messages before the new one, and the new message, so on turn 3 all of turn 1 is left out. Decide what each request should carry.
Decision 2: buildMessages() should send the system prompt, every earlier message from both sides, then the new one.
Your server now sends the system prompt, every message stored for this chat and the new one, on every turn. Turn 3 carries Leona's first message again, so the bot says it will ask ParcelLine to leave the tent with her neighbor, without asking which order.
Whichever API you use, the conversation has to reach the model on every call. Your code can send it, or some APIs keep it on their side and add it to each call for you. Either way the model reads every earlier message again, and you pay for all of that text each time, so a long chat costs more with every turn. When a bot forgets something, start with the request it received.
A large language model (LLM) is a model trained on a very large amount of text to continue text, and then tuned to follow instructions and hold a conversation: given what you send, it writes what plausibly comes next. "Large" refers to its parameters, the learned numbers inside it, and models in use today range from under a billion of them to about 2.8 trillion in Moonshot AI's Kimi K3, the largest model whose weights are public. Parameters are set during training, a separate offline process. The calls your application makes, called inference, only read them, so nothing a customer types changes the model.
The model reads one request, writes one reply and keeps nothing from the call for the next one, which is what engineers mean by stateless. With an API like the one in the film, your code stores the conversation. On every turn (one message from the person and the reply to it) it sends all of it again in a list called messages, where each entry has a role and its content: system for your instructions (the system prompt), user for the person and assistant for the model's earlier replies.
