ConvAI Open-Sources Laya, a 421M TypeSafe Jev Rival That Drops to 0.425 on Banking77

TL;DR

Laya is an open-weights answer to the System 1 decision model category that TypeSafe AI launched on September 16 with Jev. Three Apache 2.0 checkpoints, 421M and 322M parameters, pip install laya, weights on Hugging Face, 1,489 GitHub stars in under 48 hours. On its author's own board it edges Jev's published numbers on 4-label and 6-label tasks and returns a decision in 32.8 ms on a T4. On a 77-label stress test it scores 0.425 against Jev's 0.870, and the context window is 512 to 1,024 tokens against Jev's 32k state cap. Useful for high-volume routing with short option lists. Not a drop-in for everything Jev does, and the author says so on the page.

What actually shipped

The repo went up on September 18 and the Hacker News thread hit 1,137 points the next morning. Author Nandakishor Mukkunnoth, founder of ConvAI Innovations, published three checkpoints under Apache 2.0, all of them real downloadable weights rather than an API key and a waitlist.

• laya: 421M params on an ModernBERT-large backbone, English, 512-token context. 607 likes on Hugging Face at time of writing.
• laya-multilingual: 322M params on mmBERT-base, 100+ languages, 1,024-token context.
• laya-typed-decisions: 421M params fine-tuned from the English checkpoint for agent observability and customer service workflows, 1,024-token context.

The interface mirrors Jev's three primitives: choice (pick one option), score (ordinal rubric), and noul (probability that a statement is true). No text is generated, so there is no JSON to coerce and no rationale field to read. There is a ZeroGPU demo space and a PyPI package that shipped fourteen releases in two days, which is either momentum or a live stress test of somebody's release pipeline.

The clearest signal that this is a real artifact and not a blog post: within 24 hours a third party published receptron/laya, a Node and TypeScript port running the weights through ONNX Runtime with no Python at runtime. It calls itself Jev-compatible in request and response shape and claims it matches the Python reference to four decimal places, at roughly 140 ms per call on Apple-silicon CPU. A separate project, OpenDecision, went up the day before Laya chasing the same gap.

The accuracy curve bends at about 20 options

Here is the number that matters, and credit to the author for putting it on his own landing page under a heading called "Honest Limitations".

On AG News, four labels, Laya posts 0.950 to Jev's published 0.910. On DAIR Emotion, six labels, 0.595 to 0.480, with the note that Jev assigned zero probability on 16% of cases. On the project's own 2,000-decision typed-decisions set it hits 0.766 against Jev's 0.727, and above the 0.735 ceiling of the teacher distribution it was trained against, which is unusual enough to be worth its own writeup.

Then Banking77, a 77-way banking intent benchmark: 0.425 to Jev's 0.870. A 10-way support ticket routing task lands at 0.522. The recommendation on the page is blunt: keep choice schemas under 20 options.

Why the option count breaks it

This is not a data problem, it is a budget problem. The option-scoring head gets a fixed token allowance, stated as 192 to 256 tokens total, and every candidate option has to fit inside that same allowance. At 77 options that leaves roughly 3 to 4 tokens per candidate.

Picture a conference that prints all its name tags on one fixed sheet. Four guests get full names and job titles; seventy-seven get initials, and the greeter at the door starts guessing. That is the whole failure mode, and it is why the fix is schema design rather than a bigger model.

The practical move is hierarchical routing: a coarse choice with under 20 buckets, then a second pass inside the winning bucket. Two 32 ms calls still beat one API round trip by a wide margin.

Read the comparison column before you quote it

The head-to-head table is the part most coverage will lift verbatim, so read its footnote. The page states plainly that every Laya number is measured, while every Jev number is quoted from third-party posts and TypeSafe's own material. The typed-decisions model card goes further: "Jev figures are third-party published, not measured here, there is no TypeSafe API access in this project, and sample sizes and prompts differ."

That disclosure is more than most launches offer, and it also means the table is a cross-study comparison, not a bake-off.

Two more numbers deserve their exact context. The landing page's headline calibration win, expected calibration error 0.081 against 0.246, is Laya after temperature calibration. On the typed-decisions card, raw Laya lands at 0.213 against Jev's 0.144, and the card labels the model "still over-confident" there. Soft accuracy on the same set is 0.471 for Laya against 0.580 for Jev. Both tables are published by the same author; they answer different questions, and only one of them made the marquee.

The priority claim, and the paper trail

The launch post is titled around a grievance: that Mukkunnoth built non-autoregressive decision models a year before a well-funded lab called it a breakthrough. The evidence is partly real. SalesRLAgent, submitted March 30, 2025, does describe an RL-trained encoder that predicts a calibrated conversion probability in about 85 ms rather than generating text. That is the same shape of idea, applied to sales conversations.

The second citation is doing less work than the sentence around it. arXiv 2510.01237 is described in the post as schema-based decisions guided by reinforcement learning; the abstract is about a confidence-aware router that estimates uncertainty before generation, the same idea shipped as the hallunox package in September 2025. Useful work, different claim.

And the thread found the awkward part quickly. GLiNER, a bidirectional encoder doing parallel zero-shot extraction instead of sequential generation, was submitted in November 2023. GLiNER2 added schema-driven classification in July 2025. Prior art here goes back further than anyone in this week's argument, which is the usual outcome when a category gets a name and everyone checks their git log at once.

Should you run it

Yes, if your workload is short-context and short-schema: spam and phishing filters (0.993 and 0.980 on the project's own runs), ticket triage under 20 buckets, language routing, guardrail scoring. 421M parameters fit on almost anything, the ONNX port removes the Python dependency, and a Kaggle notebook on two free T4s fine-tunes your own checkpoint in roughly four hours.

No, if you need long state. 512 to 1,024 tokens is the hard wall, against a 32k state budget on Jev, and several commenters noted that ModernBERT itself supports 8,192, so the ceiling is this training run rather than the backbone. Validate on your own labels before you trust any number in this post, including the ones its author measured himself.

Key Takeaways

• Open weights exist now. Three Apache 2.0 checkpoints, 421M and 322M params, pip install laya, plus a Node and ONNX port that matches the Python reference to four decimals.
• Keep choice schemas under 20 options. The option head shares a 192 to 256 token budget, so 77 options leaves 3 to 4 tokens each and accuracy halves to 0.425. Route hierarchically instead.
• Context is the real constraint. 512 to 1,024 tokens against Jev's 32k state cap. That rules out whole-thread and long-document work today.
• The head-to-head is a cross-study table. Laya's numbers are measured, Jev's are quoted from elsewhere, and the author discloses it. The marquee calibration win is post-calibration; raw ECE is 0.213 to Jev's 0.144.
• Prior art predates the fight. GLiNER shipped bidirectional parallel extraction in November 2023, ahead of every artifact cited in this week's priority claim.
• The category is commoditizing fast. Two open competitors and a runtime port inside a week of Jev's launch is a strong hint about where this pricing goes.

Sources: Laya launch post, ConvAI Innovations, NandhaKishorM/laya on GitHub, convaiinnovations/laya model card, laya-typed-decisions model card, laya on PyPI, receptron/laya ONNX port, Hacker News discussion, arXiv:2503.23303 (SalesRLAgent), arXiv:2510.01237, arXiv:2311.08526 (GLiNER), arXiv:2507.18546 (GLiNER2), TypeSafe AI