We all know the grind of working with data, even with AI tools: every experiment starts with re-explaining everything, every iteration needs you to prompt, wait, review, correct, and repeat. And the moment you close the session, everything learned is gone.
It makes us the bottleneck, and this hinders human-AI collaboration...
So I built 𝐎𝐩𝐞𝐧𝐃𝐚𝐭𝐚𝐒𝐜𝐢, an autonomous agent purpose-built for DS/ML, and tested it on Kaggle. I enrolled in a recent competition, ran the agent with no hints, no guidance, while ironing my shirts.
In one shot, it landed AUC 0.95, a top-30% finish out of 3K+ teams and 36K+ submissions using hashtag#Anthropic's Claude Sonnet 4.6. (More on this in README)
The top-1 outperformed this agent by merely 0.004, but at the cost of massive manual effort even while using popular AI tools. The needed a dozen model families, deep learning, 400-feature notebooks, AutoML sweeps across many libraries, and 186 models ensembled carefully. Essentially a few weeks worth of effort and time!!
OpenDataSci abstracts away all the complexity and has so much to offer for DS/ML automation:
→ Owns the entire development lifecycle from EDA to final evaluation
→ Plans, codes, and executes autonomously in a secure local sandbox
→ Self-reviews and corrects before anything reaches you
→ Remembers your data across sessions, gets smarter each run
→ Runs parallel experiments and ensembles
→ Has advanced context management for token efficiency and quality
→ Ships with predefined skills for DS/ML, so it knows how to do things right
→ Bring your own knowledge: out-of-the-box support for custom skills
→ Works with any major LLM provider (hashtag#Anthropic, hashtag#OpenAI, hashtag#Bedrock, hashtag#VertexAI, hashtag#Ollama, hashtag#vLLM, and any OpenAI-compatible server).
This and so much more!! You set the goal. It does the work. No data science knowledge required.
🔗 https://github.com/f4roukb/open-data-sci
📦 pip install open-data-sci
Spin it up on your data and see what it achieves!
Top comments (2)
The number that jumped out at me isn't the AUC, it's the gap. Top-1 beat your agent by 0.004 but needed a dozen model families, 186 models stacked together, and weeks of work. That's a strong argument that an agent can get you most of the way fast, and the last sliver is where human effort still earns its keep. The one thing I'd hold loosely is that this is a single competition, so I'd be curious whether the same one-shot result holds on messier problems like time series or images before reading too much into it.
Thanks Nazar, I agree with you!
Unofficially, I've run the agent on many other datasets and got great performances.
Officially, I want to broaden the benchmark, but I also want to keep it affordable to run so that behavior-altering contributions can be evaluated to prevent regressions. Unfortunately, those gotta run on closed-source frontier models because that's what most users will be using or comparing to.
This and more on reproducible evals is on my roadmap for this project. Stay tuned!