Portfolio showcasing projects and research. Zola 2026-04-11T00:00:00+00:00 https://blog.sooraj.fun/tags/celery/atom.xml 2026-04-11T00:00:00+00:00 2026-04-11T00:00:00+00:00 Unknown https://blog.sooraj.fun/projects/snorlax/ <p>An automated detective that filters out downstream log noise in microservices to isolate "Patient Zero" crash errors.</p> <h3 id="why-it-exists">Why it exists</h3> <p>Built for a national hackathon in Mumbai. The premise was simple: reading server logs at 2 AM is miserable, and microservice cascades make it worse. We wanted to build something that hands developers the smoking gun, not a stack of haystacks.</p> <h3 id="how-it-works">How it works</h3> <ul> <li>Ingests high-velocity JSON logs using <strong>Redis Streams</strong> as a high-speed buffer.</li> <li>Employs local ML embeddings (<strong>Sentence-Transformers</strong>) to vectorize log texts.</li> <li>Groups vectorized anomalies with <strong>DBSCAN clustering</strong> to isolate the root failure.</li> <li>Sends the isolated root cause to an LLM for a clean, one-sentence diagnosis and suggested fix.</li> </ul> <h3 id="what-happened">What happened</h3> <p>We took this to Mumbai, pitch-ready. The tech judges looked at it and declared: "This won't work in the real world." We didn't win, and the critique humbled us. But instead of throwing the code away, I went back home, rewrote the ingestion workers, patched the clustering bottlenecks, and cleared the edge cases that would have broken it in production. It works now.</p> <h3 id="what-i-d-change">What I'd change</h3> <p>If I did this again, I would swap DBSCAN for an incremental clustering algorithm to handle real-time streaming logs without periodically rebuilding the full vector database space.</p> <h3 id="links">Links</h3> <p><a href="https://github.com/whysooraj/snorlax">GitHub</a></p>