or
How I built an open source deep research engine that costs a fraction of what OpenAI, Gemini, and others charge, while delivering significantly better results.
Greetings, dear LessWrong community, developers, team, and anyone else who is interested.
This is actually my first real post here, and I hope I live up to all the principles.
The problem:
We live in a fast-paced society where the value of knowledge and truth scales exponentially with our technological progress.
And especially in times of AI and fake culture, autonomously generated and factually verified knowledge is becoming increasingly important.
At the same time, we are all exposed to the stress of “effectiveness” and “productivity.” Who still has the time to conduct real in-depth research? To search for and validate information or establish facts? Virtually no one.
And that’s exactly why people use deep research engines. Google, Open AI, Perplexity, and others offer quick and “easy” ways to conduct deeper searches effectively and quickly.
But do they meet the demands of what we really need? I don’t think so. And here are the reasons :
Incorrect or hallucinated citations and sources. Tools such as Perplexity throw around long lists of sources that sound good—but when you click on them, you realize they don’t exist or are incorrect in terms of content.
False security, high-quality searches, and “cost throttling.” All providers make big promises here, but in the background, sources are “cut” or inferior models are used. Only with really expensive subscriptions do you get the full power.
Functional hallucinations. Open AI Deep Research, in particular, repeatedly generates false facts in that it thinks it can do certain things, such as generate things and use tools. This does not inspire confidence and unsettles users.
Gatekeeping of the truth. On the one hand, “subscription” constraints are created, and on the other hand, content censorship or censorship of sources is also created. A truly open search looks different.
Lack of transparency in methodology, source utilization, and processing. It’s all well and good that it looks great on the outside, but no one knows what’s really going on. Yet another black box.
In short: Today’s deep research tools are by no means bad per se. They fill a gap, but at the same time they are further away from what people want in a research tool.
Lutum Veritas Research Project -
But then there are always people working in research and development who think, “That’s not enough for me,” and I’m one of them. Martin. From Germany. 37 years old. Stubborn. Self-taught. Career changer in IT.
And that’s exactly how I felt: I want my own software now. And I want to publish it as open source because truth should not be hidden behind paywalls.And it was clear to me from the start what core ideas my software should represent:
1)No subscriptions, no paywall – bring your own key, pay only for usage. Done. No ifs, ands, or buts.
2)A source scraper and search mechanism worthy of its name that not only fetches what’s in AI-generated SEO dossiers, but also fetches the DIRT from the internet and the ESSENCE. That’s why Lutum Veritas—getting the truth out of the dirt.
3)No censorship. Search for what you want. And find answers. Without permission or compliance rules.
4)Open source and as deterministic as possible—transparency by design.
5)But above all: deeper, more detailed searches with results that go far beyond what the market has to offer to date.
Self-criticism
I am NOT claiming that my software is perfect. It isn’t. Nor am I claiming that it beats every other tool in every discipline worldwide. But I am claiming the following: I have built a standalone BYOK open source deep research tool that performs searches for a fraction of the cost of regular subscriptions or API deep research. It offers significantly deeper and more detailed analysis than any other tool. In addition to a regular mode, it has an “academic deep research mode” that provides analysis reports with unprecedented depth and evidence, often reaching over 200,000 characters. And I claim that because of this, and because of the way I have implemented context transfer, it recognizes significantly more “causal relationships” than the big players on the market.
There will be bugs. There will be things that don’t work perfectly yet. But I’m on it and constantly developing it further.
But further development requires testers and feedback. And that’s where you come in. I invite every developer, researcher, or anyone who is simply interested to test the software. Challenge it. Challenge me. So that I can make the best of it—on the one hand to meet my own standards, but also to provide the world with a tool that really delivers what it promises.
My last words? Call me narcissistic if you like. That’s what drives me, but I maintain that
as of today, I set the bar for deep research software.
———–> GitHub https://github.com/IamLumae/Project-Lutum-Veritas
Top comments (0)