If you want a fast, fixed-output decision rather than generated text, Jev vs Luna is a real choice. On 200 product descriptions, Jev classified price buckets about four times faster than GPT-5.6 Luna, but cost more per call and needed two passes to match Luna's accuracy. The right pick depends on whether latency or unit cost drives your budget.
Jev vs Luna: what the 200-product test actually showed
In a 200-product price-estimation test run by the YouTube channel Edward Donner and published on 23 September 2026, Jev answered in roughly 238 to 264 milliseconds, about four times faster than GPT-5.6 Luna at just under a second, but Luna was cheaper and slightly more accurate in one pass. The test pulled 200 scraped product descriptions from Hugging Face, including items such as a guitar distortion pedal, and asked each model to estimate the retail price in US dollars from the description alone. Real prices were compared against the guesses.
The short version: the 193.6x faster and 444.6x cheaper figures in the launch announcement describe a different measurement scope than this test. On one product-pricing workload, Jev's advantage was real but about four times, not two orders of magnitude. The sections below cover what Jev is, how the three models were configured, the measured numbers, the pricing paradox, where Jev fits, and what the vendor has actually published.
What Jev is, and how it differs from autoregressive LLMs
Jev is a decision model built by TypeSafe AI, a startup founded by a former OpenAI researcher, that returns its output in a single query rather than streaming tokens. It does not generate text token by token the way ChatGPT and other autoregressive LLMs do, and it does not produce a typewriter-style stream. You give it a specification, similar in spirit to structured outputs, and it returns an answer that conforms to that specification.
The practical consequence is latency. A classification request such as assigning a product to a price bucket comes back as one bucket with one latency number, not a paragraph followed by a parse step.
TypeSafe AI describes the model as a system one model, a reference to the fast, instinctive mode of thinking in Daniel Kahneman's book Thinking Fast and Slow. That label is a positioning choice, not a benchmark result.
The technique the company says it used is RLCD, reinforcement learning from calibrated decisions. Because the model answers inside the spec it was given, it cannot drift into invented narrative text. It can still be wrong about the underlying fact, which the one-pass accuracy numbers below show.
How the test was set up: models, prompts, and configuration
The test compared three OpenAI models on identical input: ChatGPT.1 Nano (an older, small model), GPT-5.6 Luna with reasoning turned off, and Jev 1.1 accessed through OpenRouter. Each model saw the same 200 product descriptions and was asked to estimate the retail price.
ChatGPT.1 Nano and GPT-5.6 Luna generated text describing their price estimate. Jev was given a fixed set of price buckets running from 0 to 20 dollars up through 980 to 1,000 dollars and asked to classify each product into one bucket. To make the error figures comparable, the bucket midpoint was used as Jev's numeric answer.
This is a load-bearing detail. Jev's accuracy floor is set by bucket width, so the test design gave Jev an advantage that the text-generating models did not have. The author of the test says so directly in the video.
The runner used the OpenRouter endpoint openrouter.alpha.decisions.create with the model identifier typesafe/jev-1.1:free. Jev was available through OpenRouter at the time of recording; direct access was still on a waitlist.
Three configuration choices shape every number below: reasoning was off for Luna, the Jev run used ten price buckets, and a follow-up variant called Jev in two passes with finer-grained buckets.
The distortion pedal example
The first product was a guitar distortion pedal. ChatGPT.1 Nano guessed it cost $250; the real price was $219. Jev placed it in the $200 to $220 bucket, whose midpoint is $210, which is exactly the right bucket. The worked example is the fastest way to see the difference between the two approaches: the chat model writes a price, the decision model picks a band.
The measured numbers: error, latency, and cost per thousand calls
In one pass, ChatGPT.1 Nano was off by an average of 68 dollars, cost about 1.2 cents per thousand requests, and took roughly 879 milliseconds through OpenRouter. GPT-5.6 Luna was off by 54.91 dollars, cost about 3 cents per thousand requests, and took just under a second with reasoning off. Jev was off by 58.78 dollars, cost about 8 cents per thousand requests, and returned in 264 milliseconds.
The pattern is narrow and worth stating precisely. Jev beat the older small model on accuracy and roughly tripled its speed. Jev lost to Luna on accuracy by about 3.87 dollars of average error, lost to Luna on cost by roughly 2.7 times, and won on latency by about a factor of four.
| Model (configuration) | Avg error | Latency | Cost per 1,000 calls |
|---|---|---|---|
| ChatGPT.1 Nano | $68 | ~879 ms | ~1.2 cents |
| GPT-5.6 Luna, reasoning off | $54.91 | just under 1 s | ~3 cents |
| Jev 1.1, one pass, 10 buckets | $58.78 | 264 ms | ~8 cents |
| Jev 1.1, two passes, finer buckets | ~Luna level | 238 ms | almost 10 cents |
Running Jev in two passes with finer buckets changed the picture. Average error came down to roughly Luna's level, matching it, while latency fell slightly to 238 milliseconds and cost rose to almost 10 cents per thousand requests.
Cost figures here depend on the free-tier model identifier used and on OpenRouter's routing, so treat them as a snapshot from 23 September 2026 rather than a durable price sheet.
How to read the error chart
Each point on the published chart is one guess. The x-axis is the product's real price; the y-axis is the model's guess. Anything on the dashed line was guessed exactly right, and distance from the line is the size of the miss. The spread makes the failure mode obvious: one point sits at a guess of $1,200 for a product that cost $300. Jev's points stack in visible steps because bucket width, not pricing insight, decides how close it can land.
Why Jev was slower per call than the launch claim implied
The gap between 193.6x and roughly 4x comes from what each number measures. The launch post, quoted from the TypeSafe AI founder, described the model as "two orders of magnitude faster and more efficient" and "193.6x faster, 444.6x cheaper" at a point on the Pareto frontier.
A two-orders-of-magnitude speedup can be true for the workload it was measured on, and false for a different workload on different infrastructure. The 200-product test routed all three models through OpenRouter, and OpenRouter adds its own routing overhead to each call. That overhead is a fixed cost that compresses the ratio between a fast model and a slower one.
The bucket classification task gave Jev a favorable shape: one pass, fixed output, no parsing. That is close to the workload Jev is designed for. Even so, the observed speedup was around four times, not 193 times.
None of this disproves the vendor figure. It does mean the vendor figure has a scope, and that scope was not reproduced on this test.
The pricing paradox: decisions that cost more per call
A model that returns one bucket instead of a paragraph should cost less, because output tokens dominate inference cost. In this test it cost more than twice as much as Luna, and OpenRouter is the layer where the actual per-call bill is generated.
The likely explanation is that Jev was accessed through a free-tier model identifier during a launch week. Free-tier routing on OpenRouter is not the same as the vendor's production price. A 2.7x cost penalty measured on a free launch endpoint is not evidence that Jev is structurally more expensive than Luna.
The correct reading is narrower. At the moment of testing, on the routing used, Jev cost more per thousand calls than GPT-5.6 Luna with reasoning off. Anyone making a purchasing decision should re-measure on their own account and their own task before trusting either the 444.6x claim or the 2.7x penalty. The latency advantage, by contrast, was measured consistently across both Jev passes and is less likely to move as much with routing.
Where a decision model like Jev fits
Jev fits problems where the answer comes from a fixed set. Bucketing a product into a price band, classifying an email as spam or not spam, choosing whether an agent moves north, east, south, or west, or applying a guardrail decision: all of these want one label and one latency number.
The test author's own conclusion is that Jev is an important model and a step forward, with Luna-level performance at much lower latency, but that the launch framing overstates the size of the breakthrough. He expects more models of this kind, and more use of the RLCD technique behind it.
Fixed-output decisions are not the only place a model runs. Open-ended generation, multi-step reasoning, tool calling with chained results, and anything where the reader wants prose still favor a chat model. Jev's lack of streaming is a design property, not a defect, in the narrow case it targets.
The two-pass result matters most for practitioners. It shows that Jev's accuracy ceiling is adjustable, and that the cost of closing the accuracy gap to Luna is a higher price per call rather than more latency.
A practical checklist before switching to a decision model
Decide on your own workload, not on a launch post. The following steps mirror the test described above and produce numbers you can act on.
- Collect at least 200 labeled examples with ground truth, and keep them fixed across every model you compare.
- Convert the task into a fixed output spec where the answer genuinely comes from a closed set, then measure the bucket-width error floor before comparing it to a text model's average error.
- Run each model through the same routing layer with reasoning disabled on both sides, and record average error, p50 latency, and cost per thousand calls in the same units.
- Re-run the decision model with two or more passes and finer buckets, then compare the accuracy gain against the additional cost per call.
- Re-measure pricing once any free-tier or launch-period endpoint is replaced by standard billing, because that is the number your budget will actually carry.
The test in this article is a single task on a single day. Treat the ranking as a starting hypothesis for step 1, not as a result to copy. The code used in the video is available in the repo linked under it, so the run can be recreated end to end.
FAQ
Is Jev really 193.6x faster than ChatGPT?
The 193.6x figure comes from TypeSafe AI's own launch announcement and describes a vendor-measured workload, not a general comparison. In the 200-product test published on 23 September 2026, Jev ran about four times faster than GPT-5.6 Luna with reasoning off, through OpenRouter. Vendor benchmark scope and independent routing overhead both affect the ratio.
Was Jev more accurate than Luna?
No, not in one pass. Jev's average error was 58.78 dollars against Luna's 54.91 dollars on the same 200 products. When the test called Jev twice with finer price buckets, its error matched Luna's, at a higher cost per thousand calls. Bucket width sets Jev's accuracy floor, so the comparison depends on granularity.
Was Jev cheaper than Luna?
Not on the routing used in the test. Jev cost about 8 cents per thousand requests in one pass and almost 10 cents in two passes, against roughly 3 cents for GPT-5.6 Luna with reasoning off. Jev was accessed through a free-tier OpenRouter identifier during launch week, so a production price sheet may tell a different story.
Can Jev hallucinate?
TypeSafe AI's marketing says it cannot, meaning it will not drift into invented narrative text because every answer stays inside the specification it was given. It can still return a wrong label, just as any classifier can. The 58.78 dollar average error is a measured example of Jev being wrong about the underlying fact.
When should I use a decision model instead of an LLM?
Use a decision model when the answer comes from a fixed set and you need one latency number, such as price buckets, spam classification, or routing choices for an agent. Use a chat model when the output is prose, or when the task needs multi-step reasoning and chained tool calls.
How much did Jev cost to run?
About 8 cents per thousand requests in one pass, and just under 10 cents when the run used two passes with finer buckets. That was on the free model identifier typesafe/jev-1.1:free through OpenRouter, so it reflects launch-week routing rather than a standing price.
What was the real source of the 200 products?
The descriptions came from scraped product listing data downloaded from Hugging Face, 200 items in total, and the real prices were kept alongside them as ground truth. One of the items was a guitar distortion pedal that actually sold for $219.
What is System 1 thinking in this context?
TypeSafe AI borrows the term from Daniel Kahneman's Thinking Fast and Slow, where system one thinking is the fast, instinctive mode. For Jev that means answering immediately from the input instead of reasoning step by step, which is why the fixed-output classification case suits it best.
Does Jev stream its output?
No. Jev returns the whole result from a single query, so there is no typewriter effect and no partial output to render. If your interface depends on streaming text as it appears, Jev is the wrong shape for that screen, and a chat model such as GPT-5.6 Luna remains the right choice.
Turning a benchmark video into a written article
A benchmark like this one lives or dies on the setup details: which model identifier, reasoning on or off, how many buckets, which routing layer. Those details are easy to say out loud and easy to lose between the recorded run and the written summary. That is the exact gap Skalablog is built for. Paste a YouTube URL into Skala Blog, let it transcribe the video, and you get an article draft that keeps the measured numbers, the configuration, and the caveats together in one place. If you are the one who ran the test, that is the version worth publishing.
You can also read more Brazilian developer content at crazystack.com.br, and follow the work of Dev Doido do canal do youtube.
Fork this article
Start a new branch from the same video, shaped your way. You keep the credit; the original keeps the attribution.
A fork in another language is filed as a translation of this article, so the two pages point at each other. You can unlink it later from the editor.
0/240
You are creating
- Format
- For
- Language
- Source
- Your angle
You will be asked to sign in before it is generated.
Buy credits