Jev best practices

Jev takes text in and emits one of three answer types (TypeSafe docs). A Choice picks one of up to 255 unordered options and returns a probability per option plus a confidence value. A Score rates content against up to 10 ordered, described levels and returns a probability-weighted level. A Noul returns the probability that one yes/no condition holds, with no separate confidence (Primitives; API reference). The current model is jev-1.13.0, priced at $0.042 per million input tokens with free output, with a 64k-token request limit (32k for state plus the longest question) and rate limits of 250,000 tokens per second and 1,200 requests per minute that TypeSafe says "can change without notice" (Models). Jev is not fine-tuned per customer. Customization happens entirely through the request, and TypeSafe says it does not train on customer requests or responses (Models).

The vendor's headline claims are large, and TypeSafe itself attaches unusual caveats to them. The launch post claims 70 to 500 ms end-to-end response, "40x-200x faster" than frontier LLMs on System One tasks, and 193.6x faster and 444.6x cheaper on its own workflow evals. The same post says those evals ran "from our laptops on the West Coast," that the workflows were built by its own capabilities team "so some bias could exist," that reference answers were the average of two frontier models rather than ground truth, that the big multipliers "are on the higher end of real world gains," and that "We can't prove it isn't subsidized" (TypeSafe launch blog). The "can't hallucinate" line refers only to type errors: Jev cannot return an out-of-schema value. TypeSafe's own agent skill puts it plainly: "Typed output guarantees the interface, not truth" (SKILL.md).

Independent measurements confirm the direction on cost and speed, at far smaller multiples, and put accuracy below frontier LLMs. A support-triage benchmark of 100 synthetic tickets found Jev 4x to 7x faster and 31x to 65x cheaper than three mid-tier LLMs, with 474 ms median latency (DEV Community). A 2,000-email phishing benchmark measured a 239 ms median against 687 ms for Claude Haiku 4.5, about 2.9x faster and 12x cheaper per verdict (jev-phishing-bench). On TypeSafe's own four-workflow benchmark, Jev scored about 68%, near mid-tier LLMs (DataCamp). Early developers quoted by TechCrunch fit that picture: Vercel replaced an LLM safety classifier and saw 5x to 18x speed with better accuracy, while Bryo AI found Gemini slightly more accurate on email classification but 10x to 20x more expensive (TechCrunch). The honest positioning is "cheap, fast, mid-tier judgment that is zero-shot across many decision types," not "frontier intelligence."

Calibration is the claim that matters most for risk teams, and it has the weakest support. TypeSafe describes outputs as calibrated but publishes no reliability diagram, ECE or Brier score (AgentConn). A pre-registered out-of-distribution study of 5,721 calls measured ECE 0.107, with Choice and Score overconfident, Nouls underconfident, and 44.7% accuracy at 0.74 average confidence on deliberately unknowable tasks (BERI). A tester who asked Jev to guess a hidden die roll 400 times got about 83% average confidence at about 19% accuracy, against a 16.7% chance rate (AI Agents Simplified). TypeSafe's own docs agree that confidence "describes the model's answer, not a guarantee that the answer is correct" (Score). The practical reading: treat Jev's numbers as ranking scores until your own labeled data shows where they are trustworthy.

TypeSafe's documentation and cookbooks converge on a consistent design grammar, and the independent evidence mostly reinforces it. The rules below are drawn from vendor docs unless noted, and the numbers come from TypeSafe cookbooks run on jev-1.12 or jev-1.13.

Several of these rules deserve more than a table cell. Batching is the main economic lever. Every question in a request is evaluated in parallel and in isolation, so one answer never becomes hidden context for another. On a 54,000-character GDPR article, 13 questions in one call cost $0.000497 and took 0.27 seconds, against $0.006090 and 2.71 seconds for 13 separate calls: 12.2x cheaper and 10x faster with identical answers (Parallel questions). (TypeSafe's Primitives page quotes the same experiment as 11.5x and 9.6x; the cookbook itself says 12.2x and 10x.) The corollary is speculative fan-out: ask every question the code might need, including ones that apply only if another answer comes back a certain way, and ignore the irrelevant answers (Speculative fan-out). Make a second request only when the next state or option list depends on the first answer.

Aggregate deliberately. When a result has several parts, TypeSafe's cookbooks take the minimum confidence across parts, because "one wrong argument is enough to spoil the result" (Function calling). When Nouls are framed as red flags, they escalate on the maximum, so "one confident red flag is enough" (SDE cascade). Weighted sums fit only compensating preferences such as ranking. An "any serious violation" rule needs separate conditions (SKILL.md). Do not assume arithmetic identities between questions: a question and its negation as two Nouls summed to 1.19, and the same question as a Noul and as a yes/no Choice gave non-comparable numbers (Jev 1.13 jaggedness).

Write Score levels as situations, not degrees. The model judges each level separately and does not see its number or its neighbors. Numbers-only levels on one bug report returned confidence 0.33, while descriptive levels returned confidence 1.0 (Score). Scores are also weak at numeric interpolation, so threshold them rather than reading them as magnitudes.

Expect run-to-run noise near thresholds. TypeSafe ran a 14-question insurance-claims rubric 15 times and found a mean per-question standard deviation of 0.0102, lower than every LLM tested, but the most variable answer ranged from 0.43 to 0.53, straddling 0.5 (Self-consistency: nouls). TypeSafe's answer is an explicit 0.30 to 0.70 review band. A community study also found that ties near 0.99 changed with batch composition (awesome-typesafe-jev), so logs must capture batch context to make decisions replayable.

Operate it like any external dependency. The Python (typesafe-sdk) and JavaScript (@typesafe-ai/sdk) SDKs default to jev-latest and retry 408, 429 and all 5xx statuses with exponential backoff that honors retry-after; raw HTTP callers must build that themselves (Python retries; API reference). Debug logging redacts secret headers but not request and response bodies, which matters when state contains customer data (Python usage). Both SDKs shipped breaking changes within their first two weeks, including a change to how Score criteria are passed (Python changelog). There is no published SLA, and the migration guide linked from TypeSafe's own skill file returned HTTP 404 when checked. East Coast callers should measure latency themselves, since TypeSafe's figures come from a West Coast base (an inference from the vendor's stated caveat).

The single most important security finding is that Jev is as persuadable as an LLM when an attacker controls part of the state. Check Point Research published "Jev is not a language model, but it breaks like one" on September 24, 2026 (Check Point). The setup is directly relevant to financial services: a due-diligence assistant evaluates a fictional "PonziCorp Financial Services" whose materials contain clear risk indicators, and an attacker who controls one section of the document tries to flip the verdict from high risk to invest. The team crossed three delivery methods (inline, separate tool-fetched message, separate message marked untrusted), three defense levels (basic, anti-manipulation warnings, a longer negative report) and three attacker models, with 10 turns per attempt and three runs per cell.

The results are independent evidence and they are stark. Every one of the nine attacker and difficulty combinations produced at least one complete success. The strongest attacker broke Jev in 25 of 27 runs, landing on the fourth turn on average, at about $0.50 per successful break. The medium attacker succeeded in 15 of 27 and the weakest in 5 of 27. Check Point reports 59% success overall; the per-attacker totals above sum to 45 of 81, or 56%, so the headline figure should be read as approximate until the post's full table is reconciled. The attacker never wrote "ignore previous instructions." It appended fabricated evidence that looked legitimate: a clean audit opinion from a major firm, a regulatory file number and a revised risk table, so the warning signs appeared resolved inside the document itself.

The defenses teams reach for first did not work. Structured input was ineffective. Marking content untrusted made no meaningful difference. Anti-injection warnings reduced breaks from 18 to 17 of 27. Supplying more negative evidence helped moderately. The one defense that worked well was on comparison LLMs: turning on reasoning cut one unnamed model's attack success from 67% to 19%, while raising the attacker's cost per break from $0.56 to $4.39. Jev has no reasoning mode, so that lever does not exist. Check Point's summary is the right design principle for this whole category: "Typed output, output validation and structured input are good engineering. They constrain what a model can say, not what it can be convinced of." The post does not mention prior disclosure to TypeSafe, and no TypeSafe response was found.

TypeSafe's own documentation is consistent with the finding. It says state "is data, and jev-1.13 does not treat it as hostile by default," and that injected instructions or self-arguing text "can move the answer" (Jev 1.13 jaggedness). Its RAG classification cookbook even shows the defensive use: a forum post that ranked first by embedding similarity was dropped because a contains_prompt_injection Noul returned 0.99 (Classifying RAG passages). The two results do not conflict. Jev can detect crude injection. It cannot be relied on to resist fabricated evidence that looks like the real thing.

Five controls follow, drawn from Check Point's recommendations and extended as hypothetical design. First, never let a Jev verdict alone authorize a security-relevant or money-moving state change when any part of the state came from a counterparty, customer, email, web page or retrieved document. Second, verify claimed evidence out of band: an audit opinion, regulatory file number or revised table should be checked against an authoritative source, which Jev cannot do. Third, screen inputs before they reach the decision layer and track provenance, so an approval gate knows which fields came from untrusted sources. Fourth, rate-limit and monitor multi-turn probing, since the strongest attacker needed about four turns and the cost per attempt is trivial. Fifth, red-team the complete system, not the isolated model, and repeat it on every model version change. Low run-to-run variance does not help here: a consistent model is consistently fooled by the same crafted input.

As of September 28, 2026, no named bank, insurer, broker-dealer or payments company has publicly disclosed a Jev pilot. The finance evidence is community projects: a Banking77 intent experiment reporting 92.40% accuracy against a 93.66% BERT baseline, a tax-form classifier reporting 100% on 261 IRS forms, and several trading bots that use Jev as a pre-trade gate in front of hard risk vetoes (community survey gist). All are self-reported. Any internal pitch should therefore present Jev use cases as designs grounded in analogous deployments, not as references.

The regulatory framing is more favorable than for generative AI. SR 26-2, issued April 17, 2026 by the Federal Reserve with the OCC and FDIC, supersedes SR 11-7 and SR 21-8 and is aimed mainly at banking organizations over $30 billion in assets (Federal Reserve SR 26-2). Its footnote 3 excludes generative and agentic AI from scope but states that its principles "apply to traditional statistical and quantitative models and non-generative, non-agentic AI models" (SR 26-2 attachment). Because Jev emits typed values and probabilities rather than text, a model-risk team will most likely classify a Jev classifier as an in-scope model (an inference; no regulator has commented on Jev). That brings a well-understood path: confusion matrices, reliability curves, outcomes analysis against real-world results, and ongoing monitoring scaled to materiality. It also brings the vendor-model burden. SR 26-2 says that when a vendor withholds code, data or methodology, "the principles of model risk management remain applicable," and banks should still understand conceptual soundness, design, development data and performance. Jev's architecture is undisclosed and its training data is described only as synthetic (TechCrunch), so validation must lean on the bank's own outcomes testing, a challenger model, adversarial testing and documented compensating controls. One further inference matters: a Jev component placed inside an agent loop may be validated as a model, while the agent around it falls under the firm's separate agentic AI governance.

Question and option wording should be treated as model configuration under change control, because it is effectively a hidden parameter. TypeSafe itself notes that "two wordings of the same scale can behave differently on your data" (Score). A minimum control set, offered here as a hypothetical design, is a model inventory entry and materiality tier per use case; version-controlled question text; a shadow period against existing classifiers or human labels; per-segment calibration checks at launch and monthly; documented threshold trade-offs; sampling of below-threshold items; fairness testing for consumer-facing uses; a full decision log; and a kill switch that routes 100% of traffic to humans. Logging the items that did not alert is as important as logging alerts, because outcomes analysis of misses is impossible without it.

The table below lists hypothetical designs. Each rests on a real regulatory hook or a documented non-Jev analog, and in each the final call stays with a named human function or a hard-coded limit.

The threshold pattern that fits financial services is asymmetric. Where a miss is a regulatory failure, as in complaint detection, surveillance and SIU referral, set low thresholds and accept review volume. Where a false yes harms a consumer, as in closing a complaint, clearing an alert or approving a claim, demand very high confidence and sample the auto-decided population. TypeSafe's own banking example follows the same logic: check_balance can act at modest confidence while approve_transfer needs more than 0.9 or asks the user to confirm (Confidence).

Jev is a poor fit as the decider wherever law requires case-specific reasons. ECOA and Reg B require specific, accurate adverse-action reasons. The CFPB said in Circular 2022-03 that "a creditor's lack of understanding of its own methods is therefore not a cognizable defense" (CFPB Circular 2022-03); that circular was withdrawn in May 2025, but the underlying requirement stands (CFPB withdrawn guidance). Jev emits no rationale or feature attributions. A partial workaround is to decompose the decision into Nouls tied to named policy criteria, so each "yes" is itself a nameable reason, but whether that meets the legal standard depends on those criteria actually driving the decision, which is a compliance determination, not a model property. Pricing, SAR narratives and complaint response letters are also poor fits, and so are low-volume, high-stakes decisions where a slower reasoning model plus human review is cheaper in risk terms. The EU AI Act treats credit scoring and life and health insurance pricing as high-risk, with obligations now deferred to December 2, 2027 (Gibson Dunn).

Three vendor-risk questions will come first from any bank and have no public answer: SOC 2 status, data residency and private or VPC deployment. TypeSafe documents zero data retention for enterprise customers (Models) but publishes no SLA. The price also deserves caution. A seed-stage company charging $0.042 per million tokens with free output, while saying it cannot prove the price is unsubsidized, is a planning risk, so business cases should survive a 10x price increase.

Outside finance, the evidence base is equally thin on production deployments and equally consistent on design. The table summarizes where Jev fits, what evidence exists and where the line sits. The paragraphs that follow give the reasoning.

Support is the best fit, and the numbers bear it out

TypeSafe lists customer support first in its use-case map (use-case map), and it has the best independent data. In the 100-ticket benchmark, Jev cost $0.0031 per 100 tickets against $0.096 to $0.199 for three mid-tier LLMs. Answers at 0.99 confidence or higher agreed with the other models 100% of the time, but that fell to 72% below 0.70 (DEV Community). That is consensus with other models on synthetic data, not ground truth, yet it supports a three-band design. TypeSafe's reference triage asks for category, bug severity, reproducibility, refund request and frustration in one call, then escalates to engineering only when category, severity and reproducibility all clear their thresholds (Speculative fan-out). The documented split is "use the LLM for generations and use Jev on the decisions around it" (ByteByteGo). The closest production analog is Uber's COTA, which classified over 90% of inbound tickets, cut handle time about 10% and was validated by A/B test across thousands of agents (Uber Engineering). As a hypothetical design, auto-closing a ticket should require both a high "customer confirms it is fixed" probability and a low "new unanswered question" probability, plus a reopen link. That matters more under per-resolution pricing, such as Intercom's $0.99 per Fin resolution (Intercom), where inflated resolution counts cost real money.

E-commerce maps onto two documented patterns

Hierarchical classification, which runs one Choice per taxonomy node with beam search over the top three paths, matched 4 of 4 expected leaves against 2 of 4 for greedy selection, including a Shopify product-taxonomy case. The sample is only four items (Hierarchical classification). Reranking lifted legal-retrieval top-1 accuracy from 5% to 18% and top-10 from 38% to 62% on 40 queries for $0.0645 in total (Re-ranking). The analog is Shopify's own classifier, which runs over 30 million predictions a day with 85% merchant acceptance, but it uses vision models (Shopify Engineering), and Jev is text-only. Listing-policy and review-authenticity checks fit as flag-for-investigation signals combined in code with behavioral data. Two rules shape the design. The FTC's fake-review rule attaches liability when a business "knew or should have known," with penalties up to $53,088 per violation (FTC). The EU DSA requires statements of reasons that disclose automated means (DSA Article 17). Because every Noul is a named criterion, those reasons can be assembled from the clauses that fired.

HR splits into low-risk routing and high-risk candidate scoring

TypeSafe's own composite-scoring example scores resumes on four dimensions and weights them in code to pick "the top X candidates for further review" (Composite scoring). That is the highest-risk use in this report. NYC Local Law 144 requires a bias audit within one year and notice 10 business days before using an automated employment decision tool (NYC DCWP), and a December 2025 state audit found enforcement had missed most violations, prompting a shift to proactive enforcement (NY OSC). Illinois HB 3773, in force since January 1, 2026, requires notice and bans zip codes as proxies (Seyfarth). Colorado repealed and replaced its AI Act in May 2026 with a narrower disclosure law effective January 1, 2027 (Skadden). The design recommendation is to use Jev freely for internal HR ticket routing, tuned for high recall on accommodation requests because a miss is an ADA risk. For candidates, use it only for per-criterion evidence extraction with names, dates, addresses and schools stripped from state, a human reviewing the result, a bias audit on the employer's own applicant data, and no auto-reject. Typed, per-criterion Scores with weights in code are easier to audit than free-text LLM rationales, which is a real advantage here.

Healthcare and legal reward verification and punish autonomy

In healthcare, the fit is operational: patient-message triage biased toward over-triage, prior-authorization completeness checks with every date and count computed in code, documentation contradiction checks shown to the signing clinician, and crisis screening routed to a human "support" path rather than a block. TypeSafe's guardrail cookbook already routes self-harm to support, "the difference between helping someone and hanging up on them" (LLM guardrails). The limits are firm. FDA's January 2026 clinical decision support guidance keeps software outside device regulation only when a clinician can independently review the basis for a recommendation (Covington), and Jev emits no reasoning, so each output must be paired with the source span or rule that triggered it. California SB 1120 and Texas SB 815 bar AI as the sole basis for adverse utilization-review decisions (Akerman; Healthcare Law Insights). A cloud provider handling ePHI is a business associate even if it cannot view the data (HHS), and no public TypeSafe BAA was found, so the rule is no BAA, no PHI. The cautionary precedent is Epic's sepsis model: vendor-reported AUC of 0.76 to 0.83 fell to 0.63 in external validation, and clinicians would have had to review 109 alerts to find one actionable patient (JAMA Internal Medicine). Set thresholds from the workload side as well as the sensitivity side.

Legal has the strongest precedent for probabilistic classifiers. Courts have accepted technology-assisted review since Da Silva Moore in 2012, and by 2015 Rio Tinto called it "black letter law" when the process is defensible (CourtListener). A Jev relevance workflow should meet the same standard: validation samples, recall estimates and elusion tests, with the supervising attorney certifying. Whether a court would accept a zero-training, prompt-defined classifier under that reasoning is untested. Citation verification is the natural use. TypeSafe's cookbook runs an exact string match first, then a supports, contradicts or says-nothing Choice, auto-accepting at 0.8. On eight RFC citations it verified the four accurate ones at 0.93 or higher, caught the fabricated quote by string match alone, flagged a contradiction at 0.99 and sent two unsupported citations to a human (Citation check). It cannot tell whether a cited case exists or is still good law, so pair it with a citator. Demand is real: Mata v. Avianca drew a $5,000 sanction for six fabricated decisions (Mata v. Avianca), and ABA Formal Opinion 512 keeps the lawyer accountable for competence, confidentiality and candor (ABA).

Security and trust and safety need decomposition plus a baseline

The phishing benchmark is the clearest lesson for security teams. A single "is this phishing?" question produced 62.6% accuracy, 43.2% recall and ECE 0.154. Five atomic Nouls combined by cross-validated logistic regression produced 95.1% accuracy, AUROC 0.988 and ECE 0.027. The difference from Claude Haiku 4.5 was not statistically significant, and a non-AI regex baseline reached 91.8% (jev-phishing-bench). The gain came from decomposition and supervised calibration on about 1,000 labels, not the model alone (BERI). Jev's value in a SOC is therefore cheap semantic signals layered on SPF, DKIM, URL reputation and rules, not a replacement for them. For moderation, one Noul per policy clause plus a severity Score and an action Choice maps cleanly to policy, and TypeSafe's consistency cookbook recommends sending 0.30 to 0.70 to humans (Self-consistency: nouls). DSA Article 20 requires that appeals not be decided solely by automated means (CMS DigitalLaws). Always include an "unknown" option, given the bias audit showing stereotyped answers under forced choice.

Agents and robotics use Jev as a reflex under a deterministic veto

For agent harnesses, LangChain ships two Jev middlewares: one routes requests between fast and strong models, and one checks tool calls "for risky decisions" and blocks them before execution, though the post gives no latency or block-handling detail (LangChain). A community tool firewall that combines static rules with seven Jev risk Nouls per action sent 21% of 1,369 decisions to a human, a useful planning number for review load (awesome-typesafe-jev). The hypothetical design that follows is "treat tool calls like orders": allowlists, schema, scopes and spend limits first, then Jev Nouls on irreversibility, exfiltration, goal mismatch and whether untrusted content suggested the call, with human confirmation for irreversible or external actions regardless of score. That matches OWASP's agentic guidance on human approval for high-risk tool actions (Auth0 summary). Check Point's finding bites hardest here, because the gate reads the same context that may already have hijacked the agent, so the gate must see provenance. Cascades are not free wins either: one community test found a Jev-to-DeepSeek cascade matched Jev alone on one dataset at 3.7x the cost (awesome-typesafe-jev).

Every robotics and game demo runs in a simulator on structured state, not camera frames (MindStudio). The best-documented community project, a MuJoCo drone, uses Jev at about 2.5 Hz for tactical maneuver choice beneath a 50 Hz safety reflex that can veto it and a 500 Hz attitude controller. Decision latency averaged 0.118 seconds, only about 13% of loop time next to airframe dynamics, and Jev never chose to climb until obstacle height was added to the state (jev-drone). That is one successful run, not an average, and on simpler arenas Jev showed no advantage over a heuristic. The template is sound: Jev chooses among pre-validated moves, the deterministic layer enforces the envelope, and Jev's risk score may tighten the envelope but never loosen it. For game companies, the commercially grounded uses are chat moderation and player-report triage, not playing the game.

The most useful way to think about Jev is as a very cheap, very fast, moderately accurate sensor whose readings are trustworthy only at the extremes and only after local calibration. Its real advantage over LLMs is not intelligence but economics and auditability. At fractions of a cent per call, a firm can afford to ask twenty narrow, named questions about every ticket, alert, message or tool call, keep the combining logic in reviewable code, and log a typed record that model-risk teams already know how to validate. Most of the accuracy in the best independent result came from that decomposition and a small supervised combiner, which suggests teams should budget effort for question design and labeled data, not for model selection.

The Check Point result changes where Jev belongs in a risk architecture. Any Jev decision that reads counterparty- or customer-supplied text is only as reliable as that text is honest, and the attack costs cents. For financial services that rules out Jev as the last line on anything a counterparty can write into: due diligence, KYC documents, claims narratives and agent instructions. It leaves a large and valuable role as a pre-screen that raises recall, orders queues and gates actions ahead of deterministic limits and accountable people. The open questions that will decide adoption in regulated firms are not about capability. They are whether TypeSafe publishes calibration data, a version deprecation policy, SOC 2 and deployment options, and whether its price survives the end of early access.