Why I'm Ditching LLMs for System 1 Decision Engines (Jev, Laya & Building an Instant File Organizer)
TL;DR: LLMs are great at writing essays, but using a 7B+ token-generating LLM to decide whether a file is an invoice or a resume is slow, expensive, and prone to hallucinations. I've been exploring non-autoregressive "System 1" decision engines like TypeSafe Jev and Laya, and built a sub-35ms local file organizer called laya-file-organizer.
The Flaw in LLM-Based Routing & Classification
If you've built AI workflows or agent swarms recently, you've probably hit this wall:
To classify a ticket, route an agent call, or decide what folder a file belongs in, you end up making an API request to GPT-4o-mini or Ollama. You write a complex system prompt, define a Pydantic schema, and wait 500ms to 2 seconds while the model generates tokens letter-by-letter just to output {"category": "billing"}.
It feels absurd:
- Latency Overhead: You're waiting hundreds of milliseconds for token generation when you just needed a categorical decision.
- Cost & Rate Limits: Every trivial routing decision eats into token budgets.
- Hallucination Risk: Text generation models can fail schema parsing or output unexpected text.
Human brains don't work like this. When you look at an invoice, you don't draft a 500-word essay in your head to decide it belongs in your Expenses folder — you recognize it instantly. That's System 1 thinking.
Enter Jev and Laya: Non-Autoregressive Decision Engines
Recently, a new paradigm of decision models has emerged, pioneered by frameworks like TypeSafe Jev and Laya (built by NandhaKishorM).
Instead of generating text, these models work in a single forward pass (~30ms on CPU or GPU). You pass text in, and get calibrated typed decisions back:
choice: Categorical picks among discrete criteria (e.g.financial,documents,code).score: Ordinal ratings (e.g. urgency 1 to 5).noul: Calibrated probability $P(\text{true})$ for yes/no statements.
Jev vs. Laya
- TypeSafe Jev: Established the benchmark for typed decision protocols, introducing schema-constrained classification without text generation.
- Laya: Takes this concept further with a multilingual router, supporting 100+ languages natively. Laya runs on optimized transformer backends (ModernBERT and mmBERT) trained with Reinforcement Learning against Strictly Proper Scoring Rules (RLCD). It provides calibrated confidence scores and built-in abstention gates (
min_confidence), so the model tells you when it's genuinely unsure.
What I Built: laya-file-organizer
To test Laya's capabilities in a real-world scenario, I built laya-file-organizer — a local CLI utility that organizes messy directories (like ~/Downloads) into structured folders automatically.
Architecture
File ──> Extract (Name, Extension, 1KB Sample) ──> Laya Router (1 Forward Pass ~33ms)
│
Files Moved <── Sorted Category & Junk Check <───────────────┘
- Context Sampler: Extracts the filename, file size, extension, and the first 1KB of text (for text, JSON, code, or CSV files).
- Laya Predictor: Passes the context through
laya.Routerto query two typed decisions simultaneously:category:Financial,Documents,Code_and_Logs,Media,Archives_and_Installers,Other.is_temporary_junk: Yes/No probability check.
- Instant Execution: If junk probability > 70%, file moves to
Junk_Candidates/. If confidence < 50%, it moves safely toUncertain/. Otherwise, it routes directly to its target folder.
Performance & Experience
- Speed: Sorting 50 files takes ~1.5 seconds total on standard CPU.
- Zero Cloud Leak: No API keys, no network calls after initial model load, 100% private.
- Accuracy: Invoices with strings like
"Invoice #402 due April 10"get classified asFinancialwith 0.96 confidence instantly.
What's Next?
Replacing generative LLMs with dedicated decision engines like Laya for routing, triage, and filtering is one of the most practical AI speedups I've implemented.
Check out the code on GitHub:
- 📦 Project: whysooraj/laya-file-organizer
- ⚙️ Laya Engine: NandhaKishorM/laya