Why I'm Ditching LLMs for System 1 Decision Engines (Jev, Laya & Building an Instant File Organizer)

2026-10-04T00:00:00+00:00

TL;DR: LLMs are great at writing essays, but using a 7B+ token-generating LLM to decide whether a file is an invoice or a resume is slow, expensive, and prone to hallucinations. I've been exploring non-autoregressive "System 1" decision engines like TypeSafe Jev and Laya, and built a sub-35ms local file organizer called laya-file-organizer</a>.

The Flaw in LLM-Based Routing & Classification</h2>
If you've built AI workflows or agent swarms recently, you've probably hit this wall:
To classify a ticket, route an agent call, or decide what folder a file belongs in, you end up making an API request to GPT-4o-mini or Ollama. You write a complex system prompt, define a Pydantic schema, and wait 500ms to 2 seconds while the model generates tokens letter-by-letter just to output {"category": "billing"}</code>.
It feels absurd:
Latency Overhead: You're waiting hundreds of milliseconds for token generation when you just needed a categorical decision.</li> Cost & Rate Limits: Every trivial routing decision eats into token budgets.</li>Hallucination Risk: Text generation models can fail schema parsing or output unexpected text.</li> </ol> Human brains don't work like this. When you look at an invoice, you don't draft a 500-word essay in your head to decide it belongs in your Expenses folder — you recognize it instantly. That's System 1 thinking. Enter Jev and Laya: Non-Autoregressive Decision Engines</h2> Recently, a new paradigm of decision models has emerged, pioneered by frameworks like TypeSafe Jev and Laya (built by NandhaKishorM</a>). Instead of generating text, these models work in a single forward pass (~30ms on CPU or GPU). You pass text in, and get calibrated typed decisions back: choice</code>: Categorical picks among discrete criteria (e.g. financial</code>, documents</code>, code</code>).</li> score</code>: Ordinal ratings (e.g. urgency 1 to 5).</li> noul</code>: Calibrated probability $P(\text{true})$ for yes/no statements.</li> </ul> Jev vs. Laya</h3> TypeSafe Jev: Established the benchmark for typed decision protocols, introducing schema-constrained classification without text generation.</li> Laya: Takes this concept further with a multilingual router, supporting 100+ languages natively. Laya runs on optimized transformer backends (ModernBERT and mmBERT) trained with Reinforcement Learning against Strictly Proper Scoring Rules (RLCD). It provides calibrated confidence scores and built-in abstention gates (min_confidence</code>), so the model tells you when it's genuinely unsure.</li> </ul> What I Built: laya-file-organizer</code></h2> To test Laya's capabilities in a real-world scenario, I built laya-file-organizer</a> — a local CLI utility that organizes messy directories (like ~/Downloads</code>) into structured folders automatically. Architecture</h3> File ──> Extract (Name, Extension, 1KB Sample) ──> Laya Router (1 Forward Pass ~33ms) │ Files Moved <── Sorted Category & Junk Check <───────────────┘ </code></pre> Context Sampler: Extracts the filename, file size, extension, and the first 1KB of text (for text, JSON, code, or CSV files).</li> Laya Predictor: Passes the context through laya.Router</code> to query two typed decisions simultaneously: category</code>: Financial</code>, Documents</code>, Code_and_Logs</code>, Media</code>, Archives_and_Installers</code>, Other</code>.</li> is_temporary_junk</code>: Yes/No probability check.</li> </ul> </li> Instant Execution: If junk probability > 70%, file moves to Junk_Candidates/</code>. If confidence < 50%, it moves safely to Uncertain/</code>. Otherwise, it routes directly to its target folder.</li> </ol> Performance & Experience</h3> Speed: Sorting 50 files takes ~1.5 seconds total on standard CPU.</li> Zero Cloud Leak: No API keys, no network calls after initial model load, 100% private.</li> Accuracy: Invoices with strings like "Invoice #402 due April 10"</code> get classified as Financial</code> with 0.96 confidence instantly.</li> </ul> What's Next?</h2> Replacing generative LLMs with dedicated decision engines like Laya for routing, triage, and filtering is one of the most practical AI speedups I've implemented. Check out the code on GitHub: 📦 Project: whysooraj/laya-file-organizer</a></li> ⚙️ Laya Engine: NandhaKishorM/laya</a></li> </ul>

whysooraj's Portfolio - jev

Why I'm Ditching LLMs for System 1 Decision Engines (Jev, Laya & Building an Instant File Organizer)

Jev vs. Laya</h3> TypeSafe Jev</strong>: Established the benchmark for typed decision protocols, introducing schema-constrained classification without text generation.</li>

Performance & Experience</h3> Speed</strong>: Sorting 50 files takes ~1.5 seconds total on standard CPU.</li> Zero Cloud Leak</strong>: No API keys, no network calls after initial model load, 100% private.</li>