How I Built Semantic Discussion Clustering Without Embeddings (and Why It Was Good Enough)

A developer built a low-cost system to monitor and group discussions across Reddit, Hacker News, GitHub Issues, and forums.

Instead of embeddings, it uses tokenization, TF-IDF vectors, and cosine similarity to cluster related posts.

After testing several thresholds, 0.25 performed best for grouping discussion clusters.

The stack includes Node.js, Redis, BullMQ, PostgreSQL, and Groq for summaries, with more feeds and alerts planned.