Help

Decisions shows what decision engines are good for. An engine reads some text, answers a question about it, and says how sure it is. Each demo asks a set of questions over a set of cases whose right answers are known, so you can see where an engine gets it right, where it does not, and how well its confidence tells you which is which.

Signing in is for Patchworks staff. The public home page explains the ideas; the demos, the engine comparison and your settings are behind sign-in.

Yes or no

A probability that a statement is true of the text, such as "the caller confirmed their identity: 0.93".

Choose one

A spread of probability over a fixed set of options, such as "which team should handle this claim".

Score a level

The level of a written rubric the text reaches, from lowest to highest, with a probability for each level.

Overview

How the questions did: how often the engine was right, how well its confidence matched what happened, which cases it would decide and which it would send to a person, and what the run cost.

Review

The cases held for a person, and a place to record your own label for each. Your labels are private to you.

Cases

Every case with the expected answer next to the engine's, and whether they agree.

Trace

One case in full: the question, what the engine saw, its probabilities and what the call cost.

Activity

The history of runs and changes to the demo.

Try one

Ask an engine a question about text you supply.

Compare

The engines side by side on the same cases.

The demos were built on Jev, a hosted model from TypeSafe. The open-source model Laya is shown on the same cases with precomputed results. OpenAI's Decisions API is listed as a preview and cannot be run. The Engines page has the detail.

Each engine reports its own kind of confidence number, so a value from one engine is not comparable with a value from another.

The Live call page replays three recorded calls to report a motor accident. As a call plays, the transcript grows and the supervisor's view updates: a coverage score, a conduct score, the checklist, a draft claim form and guidance for the agent.

What is recorded

Everything you see. The audio, the transcript and every answer the engine gave were recorded when each call was first run, one evaluation every five seconds of the call. The latency and token figures are the ones recorded then.

What is live

Nothing. The page asks no engine anything while it plays: it shows the recorded answer for wherever you are in the recording, so you can pause and move backward or forward freely.

What is not in this release

Live supervision of a real call. There is no call audio going in, no speech recognition and no engine request here; that would need a running service per call.

The three calls are one incident told three ways: a control, one where only the agent's turns differ, and one where only the caller's turns differ. If the audio is not loaded on a deployment, the replay runs on a timer with the transcript alone and says so.

• The demos, their questions, cases and precomputed runs cannot be edited. Editing needs your own copy of a demo, which is not available yet.
• When you review a case your label is saved for you alone and never changes the demo.
• Some demos carry a "labels unreviewed" mark: their expected answers have not been confirmed by their author.

• There is no live call supervision yet: the recorded calls are replayed, not run live.
• Each demo keeps only its latest run per engine. Older runs appear as history in the spend and metrics charts.
• Changes made outside the app do not appear, and there is no live refresh.
• There is no dark theme, and questions cannot be written as code.
• Live use of the Laya engine is optional and may be off; its precomputed results are always available.
• Trying an engine, running an evaluation and composing questions are limited per person and across the whole app each day; the limits and your usage are on the Settings page.
• There is no self-service password reset. If you forget your password, ask a Patchworks admin.

Your labels, settings and usage are private to your account: other people cannot see them. You can delete your account and everything stored under it from Settings.

Everything in the app is also available over a JSON API. Interactive reference docs are at /api/v1/docs; requests use your signed-in session.