Project: KnowsMore
My goal was simple: take the capabilities of ChatGPT, Perplexity, and NotebookLM and fold them into one local AI workspace where nothing leaves the device unless you explicitly allow it. Every pipeline stage is traced end-to-end and benchmarked against frontier models on the same tests, so "private" doesn't become a euphemism for "worse."
Application
New chat, private mode — local model, zero cloud exposure by default
Toolkit
I built the evaluation harness before I trusted the pipeline, so every claim would have receipts — every stage traced, every answer scored head-to-head against the same tests as Claude Sonnet 4.6 and GPT-4o.
The Context
ChatGPT, Perplexity, and NotebookLM make the same trade: send documents and questions to a third party, or do without them. KnowsMore runs the whole pipeline on local GPU infrastructure instead — multiple GPUs, multiple models running at once — with a real-time indicator showing what's about to leave the device before it does, gated behind explicit consent. Web search follows the same rule: private by default, cloud providers opt-in.
The Approach
I treated evaluation as a first-class workstream from day one. Every model call sits behind a provider interface — any local or cloud model runs behind the same pipeline — and every stage, from ingest to retrieval to generation, is traced through a self-hosted observability instance. That traceability turns "the local model is good enough" into a claim backed by real measurement, stage by stage.
What I Built
Ingest is structure-aware — parsing, chunking, and AI-enriched context before embedding — feeding a genuinely hybrid retrieval layer that blends keyword and vector search with a dedicated reranking pass before generation. It's production-grade retrieval discipline end to end: every stage traced, every stage swappable, built to the same standard as a hosted product from day one.
The Outcome
Benchmarked against Open WebUI across a real test set spanning multiple models. On the hardest category, questions that require pulling from multiple sources, KnowsMore's retrieval found the right context meaningfully more often than Open WebUI's, on every model tested. An AI judge graded both tools' answers side by side, swapping the order each time to rule out bias, and preferred KnowsMore's response more often than not.