How Jev decides

Where are the rules?

There aren't any. Nothing in this site says “a one-word reply after an argument means not fine.” That judgment comes from Jev's trained model. TypeSafe has not published how it works, beyond saying it was trained on synthetic data.Source: TechCrunch (press)

This is the whole request behind the 97% in the diagram:

client.systemOne({
  state: {
    // The exact text sent. "Jevlis ka?" is Marathi for "Did you eat?" and "Ho" means "yes".
    message: 'Jevlis ka? Ho. (after a small argument, said without looking up)',
  },
  questions: {
    really_fine: choice('Is the person who replied actually fine?', {
      fine: 'Genuinely fine',
      not_fine: 'Still upset despite the words',
      unclear: 'Cannot tell',
    }),
  },
});

And this is everything that came back:

{ choice: 'not_fine', confidence: 0.96,
  probabilities: { not_fine: 0.97, unclear: 0.02, fine: 0.01 } }   // 585 ms, jev-1.13.0

No reasoning, no paragraph, no rule it followed. Just one of the three options and how sure it is.

What you control

• The wording

  Words are the only lever. The same situation, described differently and asked alongside two other questions, came back at 88% instead of 97%. TypeSafe notes that two wordings of the same scale can behave differently.

  Source: TypeSafe: Score (vendor)

• The answers it may give

  Jev can only return one of the options you list, so it never invents a category. Always include a "can’t tell" or "none" option, or it is forced to pick something.

  Source: TypeSafe: API reference (vendor)

• What happens next

  Thresholds and the final decision live in your code. You decide what 97% is allowed to do, and when a person has to look.

  Source: TypeSafe: Confidence (vendor)

A lighter job for the person in the loop

With a chatbot, the reviewer reads a paragraph and has to decide whether to trust every line of it. With Jev, the reviewer checks one label and a probability against the source text. The confident cases can flow, and the unsure ones come to a person.

This comparison is my judgment from building with both. The evidence behind it:

• TypeSafe itself recommends sending answers between 30% and 70% to a person, and letting the clear cases through.Source: TypeSafe: Self-consistency cookbook (vendor)
• In our 50-case test, answers at 90% or higher were right 93% of the time; answers between 70% and 90% only 64%. That lower band is where a reviewer’s time pays off.Source: Our test results (our test)
• A community tool-call firewall built on Jev sent 21% of 1,369 decisions to a person. A useful planning number for review load, not a guarantee.Source: awesome-typesafe-jev (community project)

Keep the human honest

A lighter job can turn into rubber-stamping. Three habits prevent that:

• Show the reviewer the text that triggered the answer. Jev gives no reasons, and FDA guidance for clinical software turns on whether a person can review the basis for a recommendation.Source: Covington on FDA CDS guidance (legal)
• Spot-check the confident answers too. Fake evidence produced a 97% answer in our test, and Check Point broke Jev with it repeatedly.Source: Check Point Research (independent)
• Watch the volume. In an outside review of a hospital sepsis model, clinicians would have had to check 109 alerts to find one patient who needed action. Set thresholds for the workload people can handle.Source: JAMA Internal Medicine (peer reviewed)