TypeSafe AI for Beginners: A Deep-Dive Tutorial

Learn Jev, the first "System One" model — an AI that returns typed, structured decisions instead of generated text.

For developers who already know LLM APIs (OpenAI, Anthropic, etc.) — Python examples throughout.

If you've ever written json.loads() inside a try/except and prayed, this tutorial is for you.

We're going to learn TypeSafe — a new kind of AI that doesn't generate text. It returns typed answers, probability distributions, and confidence scores. No parsing. No prompt engineering. No hallucinated categories.

The one-sentence version: Jev is TypeSafe's flagship model and the first System One model — you send it content (state) plus typed questions, and it returns typed answers, probability distributions, and confidence. No text generation, no parsing.

How to Read This Tutorial

This is written for engineers who have already wired up a chat-completions call and parsed a JSON response. You know what a system prompt is. You've fought with "reply in valid JSON" instructions. That's exactly the pain TypeSafe removes.

You don't need any machine learning background. Every concept is introduced from scratch.

Table of Contents

• Part 0 — What is TypeSafe? (System One vs. the LLMs you know)
• Part 1 — The mental model: State, Questions, Answers
• Part 2 — Getting started: Playground, API, and your first Python call
• Part 3 — The three primitives in depth: Choice, Score, Noul
• Part 4 — The art of asking: atomic questions and batching
• Part 5 — Confidence: turning uncertainty into control flow
• Part 6 — Architecture patterns from the docs
• Part 7 — The Python SDK properly: sync, async, errors, retries
• Part 8 — Mini-project: the Resume Screener
• Part 9 — Porting an LLM workflow to TypeSafe
• Part 10 — Best practices, limits, and pitfalls
• Exercises (by part) · Solutions appendix · Cheat sheet · Further reading

Part 0 — What is TypeSafe?

The Problem Every LLM Developer Knows

Large language models are built to produce text for humans to read. That's a feature when your user is a human. It becomes a bug when the consumer of the output is your code.

Think about the last time you used an LLM to make a decision inside an application — classifying a support ticket, detecting toxicity, routing an intent. You did something like this:

1. Wrote a prompt saying "Classify this message. Reply ONLY with valid JSON: {"category": "billing|technical|sales"}"
2. Prayed the model didn't add a friendly sentence before the JSON.
3. Stripped markdown fences.
4. Called json.loads() in a try/except.
5. Handled the case where the model invented a category you never listed.
6. Retried on parse failure, maybe with a sterner system prompt.

Steps 2–6 exist because you're coercing a text-generation system into outputting structured decisions, then parsing the results back into something your code can depend on.

That is the exact mismatch TypeSafe was built to eliminate.

System One vs. System Two

The name comes from Daniel Kahneman's Thinking, Fast and Slow.

• System 1 thinking is fast and intuitive (a gut read).
• System 2 is slow and deliberate (step-by-step reasoning).

Current frontier LLMs — including the o-series and reasoning models — are System Two machines: powerful, but slow and expensive, because they generate long chains of text.

Jev is a System One machine. It's trained to make the fast, focused judgments a knowledgeable person could make in a few seconds given the right context:

• "Does this message convey urgency?" → yes, 0.98
• "Which team should handle this ticket?" → technical, 0.85
• "How frustrated is this customer, 0–2?" → 1.4

Like an LLM, Jev understands natural-language input. Unlike an LLM, it returns typed decisions and probabilities rather than generated text. It does not write replies, produce code, or explain its reasoning.

You define the space of possible answers; the model picks within it.

How the Training Differs (in One Minute)

The docs' AI primer frames three post-training paths for pretrained language models:

Calibration is the key word. In a calibrated model, probabilities are optimized against real outcomes: across many predictions, outcomes assigned 0.2 occur about 20% of the time, and outcomes assigned 0.8 occur about 80% of the time.

That makes uncertainty usable by software — you can threshold it, route on it, and escalate on it.

Calibration is measured across groups of predictions; it is not a guarantee that any single answer is correct.

RLHF also causes mode dropping: the model narrows toward the outputs humans preferred during training. Fine for chat; bad when you need honest probability distributions.

Fun fact: RLHF was co-invented by Diogo Almeida, cofounder of TypeSafe. Their bet is called Machine Native Intelligence — AI with software-like properties: structure, reliability, observability, testability, speed, consistency, and low cost.

Side-by-Side: What You Know vs. What You Get

The one-paragraph takeaway: Keep the LLM for generating language; use TypeSafe when code needs a judgment. They compose beautifully — Part 6 shows LLMs and Jev working in the same pipeline.

Part 1 — The Mental Model: State, Questions, Answers

Every TypeSafe request has the same shape. Three words to internalize:

one request → one response: state + questions → typed answers + probabilities + confidence

state + questions ──▶ TypeSafe AI model ──▶ typed answers (evaluates each + probabilities question against + confidence the state in parallel) │ ▼ your code branch · sort · route 

1.1 State — The Thing Being Judged

State is the content you want evaluated: a support message, a resume, a passage of text, the current state of your application. It can be:

The docs recommend an object for most requests: each part gets a descriptive name, and relationships stay clear.

Two input rules to remember:

1. Text only (for now). Jev evaluates strings, JSON objects, and arrays of text. Images, audio, and video are not supported (yet).
2. English is the primary training language. Other languages, including CJK scripts, are accepted but currently have lower accuracy.

1.2 Questions — The Judgments You Want

A question has four parts (only three are always required):

• ID — the key you pick, e.g. refund_requested. It labels the answer in the response. (The ID is for your code only — it is not sent to the model.)
• type — one of choice, score, or noul (the three primitives; Part 3).
• instructions — the actual question or statement to judge. This is where your evaluation logic lives. Write it as a complete, self-contained question, even if the ID seems self-explanatory.
• criteria — the possible answers: options for a Choice, ordered levels for a Score, or optional true/false definitions for a Noul.

A common beginner bug: Writing instructions: "refund" and relying on the ID refund_requested to carry the meaning. The model never sees the ID. Write the full question in instructions.

1.3 Answers — Typed Values, Not Prose

Each question returns one typed answer under the ID you chose. Two properties make these answers composable in ways LLM output is not:

1. Constrained: every answer is a probability distribution over your options or levels. The model cannot return a value outside them. There is nothing to parse and nothing to hallucinate outside the space you defined.
2. Independent: every question is evaluated in isolation against the state. One question's answer never becomes hidden context for another, so adding or removing questions never changes the others' results. This is the "no context-rot" guarantee.

1.4 The Panel-of-Experts Picture

The docs offer a mental model worth memorizing: you are briefing a panel of experts.

• The state is the case file you put on the table.
• Each question is one expert, asked one narrow question, who answers independently of the others.
• Your code is the chair of the panel: it collects the typed answers and decides what happens.

This picture immediately suggests the design rule that separates good TypeSafe systems from bad ones:

Make each question a gut-check judgment, and do the complex reasoning by composing answers in code.

That rule gets a full treatment in Part 4.

Part 2 — Getting Started: Playground, API, and Your First Python Call

You can get hands-on in three ways. We'll do all three, fastest first.

2.1 Try It: The Playground (Zero Code)

The quickest way to feel what "typed answers" means is the Playground at console.typesafe.ai/playground:

1. Open the Playground and log in.
2. Paste any text as the state, for example:

Hi, I've been trying to connect my Stripe account for 3 days and the integration keeps failing. I'm losing sales. Please help ASAP. 

1. Add a Noul question (a yes/no probability):

{ "urgency": { "type": "noul", "instructions": "Does this message express urgency?" } } 

1. Add more questions. Mix Noul, Choice, and Score in one call and watch all the typed answers appear at once.

Notice how the answers arrive: a number for the Noul, an option plus probabilities for the Choice, a position on your scale for the Score. No prose to parse anywhere.

2.2 Call It: The Raw HTTP API

Get an API key from the dashboard, then POST to a single endpoint:

POST https://api.typesafe.ai/v1/systemone Authorization: Bearer <API_KEY> Content-Type: application/json 

A complete cURL request — one state, three mixed questions:

curl -X POST https://api.typesafe.ai/v1/systemone \ -H "Authorization: Bearer $TYPESAFE_API_KEY" \ -H "Content-Type: application/json" \ -d @- <<'EOF' { "state": "Hi, I've been trying to connect my Stripe account for 3 days and the integration keeps failing. I'm losing sales. Please help ASAP.", "model": "jev-latest", "questions": { "department": { "type": "choice", "instructions": "Which team should handle this", "criteria": { "billing": "Payment or subscription issues", "technical": "Bugs or integration problems", "sales": "Pricing or account questions" } }, "frustration": { "type": "score", "instructions": "How frustrated the customer appears", "criteria": [ "Calm, just stating facts", "Frustrated but civil", "Very angry, strong language" ] }, "is_urgent": { "type": "noul", "instructions": "The message conveys urgency or time-sensitivity" } } } EOF 

And the actual response:

{ "model": "jev-1.13.0", "answers": { "department": { "type": "choice", "choice": "technical", "confidence": 0.78, "probabilities": { "technical": 0.85, "sales": 0.0, "billing": 0.15 } }, "frustration": { "type": "score", "score": 1.0, "confidence": 1.0, "legend": { "0": "Calm, just stating facts", "1": "Frustrated but civil", "2": "Very angry, strong language" }, "probabilities": { "0": 0.0, "1": 1.0, "2": 0.0 } }, "is_urgent": { "type": "noul", "noul": 1.0 } }, "usage": { "input_tokens": 392, "output_tokens": 65 } } 

Map this against what you know from chat APIs: there is a model field, a messages-analogue (state), a tools-analogue (questions), and a usage block counting tokens. The answers object is keyed by the question IDs you chose.

Notice what is missing: no choices[0].message.content, no markdown, no "As an AI..." preamble.

2.3 Code It: The Python SDK

Requires Python ≥ 3.10.

pip install typesafe-sdk # or: uv add typesafe-sdk export TYPESAFE_API_KEY="sk-..." # the client reads this automatically 

from typesafe_sdk import Choice, Noul, Score, TypeSafeClient client = TypeSafeClient() ticket = ("Hi, I've been trying to connect my Stripe account for 3 days " "and the integration keeps failing. I'm losing sales. Please help ASAP.") response = client.system_one( state=ticket, questions={ "department": Choice( instructions="Which team should handle this", criteria={ "billing": "Payment or subscription issues", "technical": "Bugs or integration problems", "sales": "Pricing or account questions", }, ), "frustration": Score( instructions="How frustrated the customer appears", criteria=[ "Calm, just stating facts", "Frustrated but civil", "Very angry, strong language", ], ), "is_urgent": Noul( instructions="The message conveys urgency or time-sensitivity", ), }, ) print(response.answers["department"].choice) # "technical" print(response.answers["frustration"].score) # 1.0 print(response.answers["is_urgent"].noul) # 1.0 

Three things to notice:

1. No prompt. The questions are the prompt. instructions is your evaluation logic; criteria is your output space.
2. No parsing. .choice, .score, .noul are typed fields on typed objects.
3. One call, three question types. They all evaluated in parallel against the same state.

2.4 Request/Response Anatomy, Annotated

For your mental model, every request you will ever write has this shape:

client.system_one( state=<string | dict | list>, # the thing being judged  model="jev-latest", # optional, this is the default  questions={ "<your_id>": Choice(instructions=..., criteria={...}), # → choice, probabilities, confidence  "<your_id>": Score(instructions=..., criteria=[...]), # → score, legend, probabilities, confidence  "<your_id>": Noul(instructions=...), # → noul (0–1)  }, ) # → response.answers["<your_id>"] (typed answer objects) # → response.usage (token counts) 

Tip for the impatient: The docs also offer an "agent skill" so coding agents (Claude Code, Cursor, etc.) can write TypeSafe integrations for you: claude plugin marketplace add typesafe-ai/skills then claude plugin install typesafe@typesafe-ai, or npx skills add typesafe-ai/skills --skill typesafe-ai. Handy once you know the basics yourself.

Part 3 — The Three Primitives in Depth: Choice, Score, Noul

TypeSafe calls its question types primitives — small, typed, composable building blocks, like software primitives. Each pairs a question shape with an answer shape:

Pick the type whose answer your code can act on directly: a Choice maps onto a match statement, a Score maps onto a threshold, a Noul maps onto an if.

3.1 Choice — One of a Known Set of Options

Use a Choice when the answer is one of a fixed set of options with no order between them: which team handles a ticket, which category a product belongs to, which language a code snippet is in.

"language": Choice( instructions="What programming language is this code written in?", criteria={ "python": None, "javascript": None, "typescript": None, "go": None, "rust": None, "other": "Anything else, including mixed or unclear snippets", }, ) 

Request fields: type: "choice", instructions, and criteria — a map of {option: description}. Descriptions are optional (None = just the label), but a good description sharpens the boundary between options.

Response fields:

{ "type": "choice", "choice": "technical", "confidence": 0.78, "probabilities": { "technical": 0.85, "sales": 0.0, "billing": 0.15 } } 

• choice — the selected option (always one of your keys).
• probabilities — the full distribution across every option, summing to 1.
• confidence — how peaked that distribution is, 0–1 (formula in Part 5).

Two practical rules from the docs:

1. Add an escape hatch. Include an other / none of the above option when your list might not cover every input, so the model is never forced to lie.
2. Low confidence often means your option list is ambiguous — two options overlap, or the state lacks the information needed to separate them.

3.2 Score — A Position on a Spectrum You Define

Use a Score when the answer is a position on an ordered spectrum and you can describe what each step means: bug severity, customer frustration, skill level, urgency.

"bug_severity": Score( instructions="How severe is the reported issue?", criteria=[ "Cosmetic; no impact to functionality", "Broken or degraded feature, but workaround exists", "Blocking issue; no workaround exists", ], ) 

Request fields: criteria is an ordered list of level descriptions, low end to high end. Minimum 2, maximum 10 levels. A level's number is its index in the array — the model sees the descriptions, not the numbers.

Response fields (real example from the docs, for the bug report "The export button crashes the settings page in Safari. It works in Chrome, but a few of our customers only use Safari."):

{ "type": "score", "score": 1.43, "confidence": 0.35, "legend": { "0": "Cosmetic; no impact to functionality", "1": "Broken or degraded feature, but workaround exists", "2": "Blocking issue; no workaround exists" }, "probabilities": { "0": 0.0, "1": 0.57, "2": 0.43 } } 

Read it like this:

• score: 1.43 is a position on the level line (0 to 2 here). It is the probability-weighted level: 0×0.0 + 1×0.57 + 2×0.43 = 1.43. It can land between two levels — the model is saying "between 'workaround exists' and 'no workaround', leaning toward workaround". This interpolation is a feature, not rounding error.
• legend maps level numbers back to your descriptions.
• confidence: 0.35 is low because the probability is split across levels 1 and 2 — exactly matching the report's ambiguity.

Designing good levels: make them one ordered dimension, mutually exclusive, and describe observable behavior ("strong language" rather than "quite upset"). If you cannot order the options, that is a Choice, not a Score.

3.3 Noul — The Probability That a Statement Is True

Use a Noul for a clean yes/no question where the probability itself is the useful signal: does this message contain PII, is the customer requesting a refund, does this resume mention distributed systems.

"is_human_escalation": Noul( instructions="Is the customer asking for a human agent?", ), "is_repeat_contact": Noul( instructions="Has the customer contacted support about this before?", criteria=NoulCriteria( # optional, defines yes and no  true="Mentions a prior attempt, ticket, or that they have asked before", false="No sign of any previous contact", ), ), 

Response: a single number.

{ "model": "jev-1.13.0", "answers": { "is_human_escalation": { "type": "noul", "noul": 0.99 }, "is_repeat_contact": { "type": "noul", "noul": 0.93 } } } 

Reading a Noul: the number is both the answer and the certainty in one value.

• Near 1 = strong yes.
• Near 0 = strong no.
• Near 0.5 = the model gives yes and no equal probability — it is telling you it cannot tell.

Real recorded values for is_human_escalation across different messages (from the docs):

"I need this sorted today" is urgent but never asks for a person → 0.26. "Are you a bot?" hints at wanting a human without asking → 0.40, an almost even split. Those middle values are exactly where your code should be making threshold decisions (Part 5).

There is no separate confidence field on a Noul — with only two outcomes, the single probability describes everything. (If you want a confidence-style number anyway, use |2p − 1|; see Part 5.)

3.4 The Classic Mix-Up: Noul vs. Score

This is the #1 conceptual trap, so the docs call it out explicitly. "Is this candidate strong in Python?" is a bad Noul question if what you want is a skill level.

• A Noul of 0.5 means "yes and no are equally likely" — not "medium skill". If "strong" is undefined, the probability is hard to interpret at all.
• Want a level? Use a Score with defined levels: no experience → some familiarity → daily use → deep expertise.
• Want a yes/no decision? Define the condition precisely: "Does the resume state that the candidate has used Python at work?" — then a Noul is perfect.

Rule of thumb: Noul judges a claim; Score measures a position.

3.5 Choosing Between Them — The Cheat Table

Part 4 — The Art of Asking: Atomic Questions and Batching

Getting the primitives right is half the skill. The other half is how you phrase questions and how many you ask per call.

4.1 Ask for One Snap Judgment per Question

System One models are built for the kind of judgment a knowledgeable person makes in a second given the right context:

✅ Good: "Does this message convey urgency?" · "Which team should handle this?" · "Does the resume mention Kubernetes?"

❌ Bad: "Analyze this message and determine the best course of action."

The bad one needs slow reasoning — that is a System Two job, or better: a decomposition problem. If your desired judgment weighs several independent factors, ask about each factor separately and combine the answers in code.

The docs' example: instead of "rate this startup pitch", ask separately about market size, technical feasibility, and differentiation — then weight them in your own formula. When priorities shift, you change a coefficient in code instead of rewriting a prompt. Try doing that with a monolithic LLM prompt.

4.2 Point at Parts of the State with Paths

Real states are structured objects. When a question targets one part of the state, name it with a dot-and-index path in backticks inside instructions:

state = { "ticket": { "subject": "Duplicate charge", "messages": [ {"from": "customer", "text": "I was charged twice for order A-104. Please refund the duplicate.Source：https://dev.to/aarun_soman_/-typesafe-ai-for-beginners-a-deep-dive-tutorial-1b38
My favorites

Related posts

Three dates that catch a hard-coded time-zone bug in a session clock

A session that starts at 08:00 local time does not necessarily start at the same UTC hour all year. Less obviously, the

0dev.to

Laravel N+1 Query Problem: How to Find and Fix It

When a Laravel page that used to load in half a second suddenly takes four, the cause is very often the same: the N+1 qu

0dev.to

LongCat-2.5 at $0.30/$1.20: Thinking Locks Temperature, 1M Context, and Vendor Benches Between Opus 4.8 and 5

Attributed compile + one small offline run on my box Primary source: Introducing LongCat-2.5 — Native Multimodality for

0dev.to

Shipped: two packages, a registry listing, and the first seals

What happened Post 1 ended with a list: the verify package inside Fiador — human-backed attestation for Discord bots — i

0dev.to

AI Agent Memory Needs Garbage Collection, Not Just Storage

TL;DR — Most long-term memory systems for AI agents only know how to write and append — never to forget, consolidate, or

0dev.to

Securing the Analytics Pipeline and Pivoting for Demo Day

Sprint Retrospective: Implementing Zero-Trust Analytics The conclusion of Sprint 11 was entirely focused on access contr

0dev.to