Preporato
Structured Output & Function Calling with NIM
Hands-on lab · Runs in your browser

Structured Output & Function Calling with NIM

Get reliable machine-parseable data out of an LLM. Compare prompt-only JSON extraction with the function-calling API, chain two tools and measure the reliability gap.

Time
30 min
Checked steps
4
Level
Intermediate
Setup
None
ncp-aai
Read step 1

Hands-on labs require Pro · $29.99/mo · cancel anytime

Lab cockpit30 min · 4 stepsSession running
1 / 4 steps passingFunction calling with a JSON Schema · step 2 of 4
main.py▶ Run✓ Check
def extract_via_tools(text: str) -> dict:
TerminalOutput

4 steps, each checked when you finish it

A check runs your work at the end of every step. Hints and the full solution are there if you get stuck.

  1. 1

    Prompt-only JSON extraction (baseline)

    The easiest way to get structured data out of an LLM is to ask for JSON in the prompt.

  2. 2

    Function calling with a JSON Schema

    Instead of asking the model to write JSON in its reply, you tell it: *"here is a function, here is its schema, call it with the right arguments."* The model then returns a tool_call with arguments already validated against your schema — no regex, no json.loads.

  3. 3

    Chain two tools with validation

    Real agents use more than one tool per turn.

  4. 4

    Measure prompt-only vs tools reliability

    Prompt-only JSON extraction works until the message is messy.

Step 1 as it appears in the lab

The lab’s own text. The hint and the solution stay inside the lab.

Step 1: Prompt-only JSON extraction (baseline)

The naive approach

The easiest way to get structured data out of an LLM is to ask for JSON in the prompt. It mostly works — but mostly isn't good enough for production, and you'll quantify why in step 4.

The task

Extract structured contact info from a free-text message. Given an email like:

"Hi, this is Jane Chen from Acme Corp. My email is [email protected] and my phone is +1-415-555-2036. Thanks!"

Return:

{"name": "Jane Chen", "company": "Acme Corp", "email": "[email protected]", "phone": "+1-415-555-2036"}

Your task

Add to main.py:

  1. Create a NIM client.
  2. Define extract_via_prompt(text: str) -> dict that asks meta/llama-3.3-70b-instruct for a JSON object with keys name, company, email, phone.
  3. Parse the response with regex + json.loads. Return {} if parsing fails.
  4. Call it on a sample message and print the result.
main.py, the file you edit13 lines
from openai import OpenAI
import json, re

client = OpenAI(base_url="http://nim-proxy.labs.svc:8080/v1", api_key="nvapi-inject")
MODEL = "meta/llama-3.3-70b-instruct"

SAMPLE = "Hi, this is Jane Chen from Acme Corp. My email is [email protected] and my phone is +1-415-555-2036. Thanks!"

# TODO: extract_via_prompt(text) -> dict with keys name, company, email, phone
def extract_via_prompt(text: str) -> dict:
    ______

print(extract_via_prompt(SAMPLE))

Exam domains covered

Agent DevelopmentTool CallingNVIDIA Platform Implementation

Frequently asked questions

What's the difference between a tool call and a function call in the OpenAI schema?

They're the same concept; the schema is just layered. A chat completion that uses tools returns message.tool_calls, a list of objects each containing a type: "function" wrapper and a function object with name and arguments (a JSON string). "Function calling" is the older name — the API was originally one function per call — and "tool calling" is the current name that generalizes to multiple callable tools per turn. NIM's OpenAI-compatible endpoints expose both fields, and this lab uses the modern tool_calls shape throughout.

Why is function calling more reliable than prompting for JSON?

Because the schema is enforced on the server, not on the model's prose instincts. When you pass tools=[...] with a JSON Schema, the endpoint constrains generation to produce arguments that match the schema — required fields are present, types are correct, the output parses. Prompt-only extraction depends on the model having seen enough JSON in training to emit valid JSON for your specific shape, and it breaks on edge cases: trailing commas, unescaped quotes in names, markdown code fences, etc. Step 4 measures the gap concretely on a noisy test set.

What does tool_choice control and when should I set it?

tool_choice tells the endpoint how aggressively to call tools. "auto" (the default) lets the model decide whether to emit a tool call or a text reply. "required" forces a tool call. {"type": "function", "function": {"name": "save_contact"}} forces a specific tool. Use "auto" for agents that may or may not need the tool. Use "required" when your extraction pipeline must produce structured output, as in Step 2 — you don't want the model to chat back "sure, here's the info" in prose.

Why classify first, then pick a schema, instead of using one giant schema with all fields?

One fat schema with union types forces the model to reason about record type and field extraction in a single shot, and it makes your validation harder — you end up checking "if type == invoice, these fields should be present; if type == contact, these other fields." A two-step pipeline (classifier → specialized extractor) keeps each call focused on one job, lets you evolve each schema independently, and maps cleanly onto the router → specialist tool pattern real agents use. Step 3 walks you through both tools and the routing shim.

What counts as a "valid" extraction in Step 4's comparison?

A dict that parses cleanly and contains every required field for the record type. An extraction is broken if json.loads fails, if a required field is missing or null, or if a type doesn't match (e.g., phone came back as a number instead of a string). The Step 4 harness counts valid vs broken for both the prompt-only path and the tools path over the same messy input set. The expected outcome is that tools mode maintains a near-perfect valid rate while prompt-only degrades visibly on the awkward inputs.

Does every NIM model support function calling?

Most modern chat-completion NIMs do — the Llama 3.3 70B Instruct and Nemotron reasoning families used in these labs all accept tools and return tool_calls. But not every model in the NIM catalog exposes function calling: some vision-language models (for example google/gemma-3-12b-it) ignore a tool schema entirely and answer in prose, so a parser expecting JSON gets free text. The vlm-visual-qa lab explores that split explicitly; here, the contact-extraction models are all on the supported list.

What you'll build in this function-calling lab

Function calling is what separates LLM demoware from production — the moment you need machine-readable output that validates on the first try, prompting the model for JSON stops being good enough. This lab builds the same contact-and-invoice extraction pipeline two ways and measures the reliability gap: prompt-only JSON extraction as the naive baseline, versus schema-enforced function calling via the tools parameter on NVIDIA NIM endpoints we provision. You walk away with a structured-output pattern you can drop into any LangChain agent, intuition for when prompt mode is fine and when it actively fails, and a harness that gives you a concrete valid-vs-broken count on your own data.

The technical substance is the distinction between asking nicely for JSON and enforcing a schema at the API boundary. You define a save_contact tool with a formal JSON Schema, pass it via the tools=[...] parameter, set tool_choice to force the call, and read message.tool_calls[0].function.arguments — already valid against your schema, no regex, no markdown-fence stripping. You then compose two tools — a classifier that returns "contact" | "invoice" and a specialised save_invoice with its own schema — and see why a router-plus-specialist pipeline is cleaner than one fat union schema. The final step runs both strategies over intentionally awkward inputs (trailing commas, apostrophes in names, emoji, multi-line bodies) and surfaces the reliability gap quantitatively.

Prerequisites: Python with dicts or dataclasses, prior exposure to a NIM-backed agent (the react-agent-nim lab is the natural entry point), and a rough mental model of JSON Schema. The hosted environment ships with the OpenAI Python SDK pointed at our managed NIM proxy serving meta/llama-3.3-70b-instruct — same OpenAI-compatible tool_calls surface you'd use against the real API, no keys, no GPU provisioning. About 30 minutes of focused work, ending with a message-by-message report of which extractions parsed cleanly against the schema and which didn't — the same accounting real teams run before picking the default extraction mode for their pipelines.