How I built Laya

Most of the decisions a piece of software makes in a day are small. Which team does this support ticket belong to. Is this customer about to cancel. Is this message a refund request or a complaint. None of them need an essay. All of them, in 2026, are routinely answered by asking a large language model to write a sentence and then taking that sentence apart again with a parser.

That bothered me enough to spend a year on the alternative. This is how Laya happened, what it actually is, and the parts where it loses.

Problem statement

A generative model answers a yes or no question the same way it writes a poem. It reads your text once, which is the fast part, and then it generates, one token at a time, which is the slow part. For a one word answer you pay for the whole machine. Then your code has to read that answer back: parse the JSON, handle the case where it wrote prose instead, handle the case where it invented a label you never offered.

Worse, you cannot trust the number it gives you. Ask a generative model how confident it is and it will write a plausible looking percentage. It is text, not a measurement. If you branch on it, you are branching on a guess.

Overall solution

Every language model already does the cheap part: it reads your whole input in one pass. That is called prefill, and it is where the understanding happens. The generation loop that follows is a separate thing. So: keep the reading, throw away the loop, and attach small output heads that read the answer straight off the model.

No text is generated anywhere in the path. There is nothing to parse and nothing to hallucinate, because the model is never choosing words. It is choosing among the options you gave it, and the only thing it can return is one of them with a number attached.

Technical details

The base is an encoder, not a decoder. Laya is built on ModernBERT-large for English and a separate mmBERT checkpoint for everything else, which is how it covers more than 100 languages without a 7 billion parameter model in the loop. The checkpoints are 322M and 421M parameters. On a single T4 a question comes back in about 33 ms, and batched across ten questions it drops to around 7 ms per question.

Questions are typed, and there are only three types, because three is enough for almost everything: choice picks one of your labels, score returns a rating on a scale you define, and noul returns the probability that a statement holds. You hand it a state and some questions, you get back values your code can use directly.

Training against a scoring rule, not a teacher

The part I care most about is the calibration. A probability is only useful if 0.7 means roughly seven times out of ten. Getting that is not automatic, and copying a bigger model does not give it to you, because you inherit whatever overconfidence the teacher had.

So the training signal is a strictly proper scoring rule. The short version of what that means: the reward is designed so that the only way to maximise it is to report your honest probability. Shading the number up to look confident loses points. Shading it down to look humble also loses points. I called the setup RLCD, and it is a policy gradient over that reward rather than a cross entropy against someone else's labels.

The result you can check: expected calibration error of 0.081 after temperature fitting, against 0.246 for the hosted model I compared against. That is the number that decides whether you can write if confidence > 0.8 and mean it.

Three checkpoints and a router

One model that is good at everything is a nice idea that keeps not working. English text and Malayalam text want different tokenizers and different pretraining. So there are three checkpoints and a router that picks one per request, based on the input rather than on a setting you have to remember. Every published Laya number is what the router actually returned, not the best of three picked afterwards.

Where it wins and where it loses

Measured against Jev 1.13.0, on byte identical questions with a fixed seed. The Jev figures are third party published rather than measured by me, so sample sizes differ and you should read them as a direction, not a decimal.

Look at the Banking77 row, because it is the honest one. With 77 labels Laya scores 0.425 and loses badly. The reason is not mysterious: the options share a fixed token budget, so 77 labels get three or four tokens each and the label texts stop being distinguishable from one another. You can raise the budget at runtime, and there is a shortlist path that narrows the labels first, but out of the box, past about twenty options, the other model is the better tool. I would rather write that down than bury it.

On DAIR Emotion there is a detail worth more than the accuracy gap: the hosted model assigned zero probability to the true label on 16% of examples. Not low probability. Zero. Anything branching on confidence treats that as certainty, and it is certainly wrong.

The unglamorous half

The model was the interesting part and maybe a third of the work. The rest was everything that makes a thing usable by someone who is not me: a router that preloads sensibly, an HTTP server, an MCP server, LangChain and LangGraph nodes, a TypeScript port so it runs in Node and the browser, ONNX export, a GPU fast path, a fine-tuning notebook that runs on Kaggle's free two T4s, and a benchmark harness that regenerates every number in the README so I cannot quietly drift away from what I published.

Also roughly 1,800 commits, most of which are not features. They are the long tail of someone on a different operating system hitting a different error.

TLDR

I took the reading half of a language model, threw away the writing half, bolted typed output heads onto it, and trained those heads with a reward that only pays out for honest probabilities. The result answers a typed question in 33 ms on one GPU, works in more than 100 languages, calibrates three times better than the hosted model I measured against, loses clearly when you ask for more than twenty options at once, and is Apache 2.0 so you can read every line of it.

Source, package, and weights.