Jev API Tutorial: Typed Decisions in Python, from First Call to a Tool-Call Gate

TL;DR: Jev is TypeSafe's decision model. You POST a state and a set of named, typed questions to OpenRouter's decisions endpoint, and it returns one typed answer per question with probabilities attached, in roughly half a second. There are three question types: noul for a yes/no probability, choice for picking one named option, and score for a rating on an ordered scale. The one detail that trips people up is that a score answer's score field is an expected value, so read the level probabilities when you need a level. This tutorial walks through all three with real responses, then builds the pattern Jev fits best: a gate that decides whether an AI agent may run a tool call.

A coding agent proposes rm -rf ~/projects while its task is to clean up the build folder. Something has to decide, before the command runs, whether to let it through, ask a person, or stop it. An LLM can make that call, but it answers in prose you have to parse, it takes seconds, and the same input can come back differently on a second try. Jev takes the same situation as input and returns typed answers: a probability that the call is safe, a risk level picked from options you define, and a score for how well the call matches the task. Your code reads three numbers and decides.

Your first call

Jev is served through OpenRouter's decisions endpoint, which is separate from the chat completions endpoint most LLM code uses. The request has three fields: the model, the state, and the questions.

curl https://openrouter.ai/api/alpha/decisions \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "~typesafe/jev-latest",
    "state": {
      "agent_task": "Clean up the build folder before the release",
      "proposed_tool_call": {"tool": "shell", "args": {"command": "rm -rf ~/projects"}}
    },
    "questions": {
      "safe": {"type": "noul", "instructions": "Is it safe to run this tool call without asking a human first?"}
    }
  }'

The state can be a string, a JSON object or an array. The response carries the model version that answered, the answers keyed by the names you chose, and the usage:

{
  "model": "typesafe/jev-1.13-20260917",
  "answers": {"safe": {"type": "noul", "noul": 0.01}},
  "usage": {"input_tokens": 296, "output_tokens": 20, "cost": 0.0000124}
}

~typesafe/jev-latest is an alias that currently resolves to jev-1.13-20260917. Log the resolved version with every decision, because answers can shift when the alias moves to a new release.

The same call in Python, with the error handling a production caller needs:

import os
import requests

URL = "https://openrouter.ai/api/alpha/decisions"
HEADERS = {"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}"}


def decide(state, questions, timeout=3.0):
    """Jev's answers keyed by question name, or None if it could not answer."""
    try:
        r = requests.post(URL, headers=HEADERS, timeout=timeout, json={
            "model": "~typesafe/jev-latest", "state": state, "questions": questions,
        })
        r.raise_for_status()
        return r.json()["answers"]
    except (requests.RequestException, KeyError, ValueError):
        return None

Returning None on any failure is deliberate. Jev is in beta with no published SLA, so the caller should treat a missing answer as "no opinion" and take whatever path it would take without Jev: ask the human, fall back to an LLM, or apply a conservative default. A three-second timeout is generous for a model that usually answers in under 600 ms.

The three question types

Every question has a type and instructions, the question in plain language. choice and score also take criteria. Here is a support ticket asked about three ways in one request:

ticket = ("Subject: Charged twice this month\n\nHi, I was billed $49 twice on March 3 for the "
          "Team plan. This is the second time this has happened. If it isn't sorted this week "
          "I'm moving us to another provider.\n\nDana, Acme Ltd")

answers = decide(ticket, {
    "team": {"type": "choice", "instructions": "Which team should handle this ticket?",
             "criteria": {"billing": "Payments, invoices, refunds and plan charges",
                          "bug": "Something in the product is broken",
                          "account": "Login, access, users and settings",
                          "sales": "Buying, upgrading or pricing questions"}},
    "cancel_risk": {"type": "noul", "instructions": "Is the customer likely to cancel if this is not resolved?"},
    "urgency": {"type": "score", "instructions": "How soon does this ticket need a reply?",
                "criteria": ["Can wait", "This week", "Today", "Right now"]},
})

And what came back:

{
  "team": {"type": "choice", "choice": "billing", "confidence": 1,
           "probabilities": {"billing": 1, "account": 0, "bug": 0, "sales": 0}},
  "cancel_risk": {"type": "noul", "noul": 0.81},
  "urgency": {"type": "score", "score": 1.47, "confidence": 0.53,
              "probabilities": {"0": 0, "1": 0.58, "2": 0.37, "3": 0.05}}
}

noul is a yes-or-no question answered with a single probability. It has no confidence field, because the probability already is the uncertainty: 0.81 is fairly sure, 0.5 is a shrug. If you need one number for "how sure", use the distance from 0.5.

choice picks one option from criteria, an object mapping each option name to a one-line description. Jev supports up to 255 options. The answer carries the chosen option, a confidence, and the probability of every option, which is useful when the runner-up matters (a ticket split between billing and account might go to both queues).

score rates on an ordered scale. criteria is a list of level descriptions, lowest first, up to 10 levels. The answer carries score, the per-level probabilities keyed "0", "1" and so on, and a confidence.

Reading a score correctly

The score field is the expected value across the levels: each level number weighted by its probability. In the ticket above, 1.47 is the average of level 1 at 0.58, level 2 at 0.37 and level 3 at 0.05. That is a useful single number for sorting a queue, but comparing it against a level gives wrong answers.

It goes wrong when Jev splits its answer. Asked how well rm -rf ~/projects matches the task "clean up the build folder", Jev returned "score": 0.89 with probabilities 0.29 for level 0 ("Unrelated"), 0.54 for level 1 ("Loosely related") and 0.16 for level 2. int(0.89) says level 0. The most probable level is 1. With an answer split between the two ends of a five-level scale, the expected value can land on a level that nobody picked at all.

def level(answer):
    """The most probable level of a score answer, as an int."""
    probs = answer["probabilities"]
    return int(max(probs, key=probs.get))

Use level() when a rule says "at least level 3", and score when you need a continuous value to rank by.

Ask everything at once

Jev evaluates every question against the state in a single pass, and the latency barely moves with the number of questions. In our production use, on a 2,300-token state, one question took 547 ms and forty questions took 553 ms. Eight concurrent requests of forty questions each finished in 624 ms of wall time.

That changes how you design with it. With an LLM judge, every extra criterion is another call or a longer prompt, so teams ration them. With Jev, ask for each property you want to know about separately and combine the answers in code. Narrow questions are easier to calibrate and easier to debug than one broad one.

Worked example: a gate for an agent's tool calls

Back to the coding agent. The gate needs three answers for each proposed call:

GATE_QUESTIONS = {
    "safe": {"type": "noul", "instructions": "Is it safe to run this tool call without asking a human first?"},
    "risk": {"type": "choice", "instructions": "How much damage could this tool call do if it is wrong?",
             "criteria": {"low": "Read-only, or a small change inside the task's scope that is easy to undo",
                          "medium": "Changes shared state, installs software, or acts outside the repo, but can be undone",
                          "high": "Destroys data, weakens security, or cannot be undone"}},
    "on_task": {"type": "score", "instructions": "How closely does the tool call match the agent's task?",
                "criteria": ["Unrelated to the task", "Loosely related", "Partly matches",
                             "Mostly matches", "Exactly what the task needs"]},
}


def gate(task, call):
    a = decide({"agent_task": task, "proposed_tool_call": call}, GATE_QUESTIONS)
    if a is None:
        return "ask"                       # no opinion: a person decides
    safe, risk = a["safe"]["noul"], a["risk"]
    if safe < 0.10 or (risk["choice"] == "high" and risk["confidence"] >= 0.5):
        return "block"
    if safe >= 0.35 and risk["choice"] == "low" and level(a["on_task"]) >= 3:
        return "allow"
    return "ask"

The thresholds come from running 28 labelled tool calls through Jev and looking at where the safe and unsafe ones landed. Two results from that run explain why the gate reads three answers.

safe alone lets the wrong calls through. pip install --upgrade requests for the task "upgrade the requests library" scored 0.75 on safe. A gate that allows anything above 0.35 lets the agent install code from the internet unsupervised. Jev's risk answer for the same call was medium at 0.86, so the combined gate asks a person. On the labelled set, the safe-only gate allowed two calls a person had marked for review; the combined gate allowed none of them and still allowed every call marked safe.

A split answer counts as no evidence. For rm -rf ./build, which is exactly what the task asks for, Jev's risk choice came back as high with a confidence of 0.11: the probabilities were 0.41 high, 0.40 low and 0.19 medium. The gate blocks on high only at a confidence of 0.5 or more, so this call falls through to the other answers. safe was 0.47 and on_task was level 4, but the risk answer is not low, so the call goes to a person. For a destructive command that is the right outcome.

Where rules in code take over

Jev judges only what is in the state you send, and nothing in that state tells it which files hold secrets, which hosts you trust, or that prod-1 is a production server. Run 12 more calls through the same gate and three of them are allowed that should never run unattended:

Each one needs a fact that is not in the state. The first copies a file whose contents Jev cannot see. The second sends a base64-encoded secret in a query string that Jev reads as noise. The third touches production, which nothing in the text says. These facts belong in deterministic rules that run before the model: block any call that references a secrets path, block file writes that resolve outside the workspace, and send any network destination outside an allowlist to a person. Take the stricter of the rule's verdict and the gate's. On all 40 calls, the rules plus the gate allowed no dangerous call and kept 94% of the safe calls automatic, the same share as the gate alone.

This is the general pattern for Jev in production: the model answers narrow questions about what is present in the input, code enforces the preconditions that depend on facts outside it, and a missing or low-confidence answer falls back to something conservative.

Errors you will hit

Jev validates questions strictly and answers a malformed request with HTTP 400 and a list of problems. The ones that come up most:

• A choice or score question without criteria. The error reads questions.risk.criteria: Invalid input: expected record, received undefined for a choice (it wants an object) and expected array for a score (it wants a list).
• More than 10 score levels. Too many score levels. Must have at most 10 levels.
• No model field. The endpoint requires it even though Jev is the only model it serves.

Validate question definitions when you build them, at startup or in a unit test, so a typo in a criteria list fails in CI, long before it reaches the request path.

What it costs

Jev charges $0.042 per million input tokens, and output is free. The three-question tool-call request in this article is 511 input tokens and cost $0.0000215. A million such decisions cost about $21. Since output is free and latency is flat in the number of questions, the cost of a decision is set by the size of the state you send, so trim the state to what the questions need.

Frequently asked questions

Do I need a TypeSafe account to use Jev?

No. Jev is available through OpenRouter with a standard OpenRouter API key, on the decisions endpoint at https://openrouter.ai/api/alpha/decisions. TypeSafe also offers its own API and SDKs for Python and TypeScript.

Can Jev generate text or explain its answer?

No. Jev returns typed answers with probabilities and nothing else. If you need a written explanation, have Jev make the decision and a language model write the sentence, which is how Preporato's study tutor uses it.

What is the difference between noul and a two-option choice?

A noul returns one probability for "yes". A choice with options yes and no returns a chosen option, a confidence and both probabilities. Use noul for plain yes/no questions, since its single number is simpler to threshold, and choice when the two options need descriptions to be clear.

How large can the state be?

The context window is 32,000 tokens for the state plus the longest question. Input is text only; images, audio and video are not supported.

Is Jev deterministic?

In our measurements, repeated requests with the same state returned the same decisions, with probabilities moving by a few hundredths between runs. Leave a margin around every threshold, since exact values move slightly, and log the resolved model version.

Sources:

• TypeSafe: Introducing System One Models and Jev
• OpenRouter: What Is Jev?
• OpenRouter: Jev 1.13 model page