Hello, Dev.to community!
How effective AI really is in software development, based on the latest METR study, the 2025 StackOverflow Survey, and my own experience. We’ll also talk about the new trend of vibe coding and how it's changing the development process.
Introduction
Nowadays, it’s impossible to ignore the influence of AI on software development. Instead of searching for information on Google, you can now simply ask ChatGPT, and what’s even cooler, you can make your request more specific and get more detailed information, instead of wasting time endlessly searching across the internet hoping to piece together the information like a puzzle. Honestly, I can’t even remember the last time I visited StackOverflow, while back in 2018 I used to have 10-15 active tabs open to find the information I needed or to help others by providing answers. Besides information, AI can help you analyze large logs you might have received from a server or find errors in a config file with 500+ lines, something that would take a human much longer.
AI can also assist in writing technical documentation. When I was working on my open-source library, ChatGPT wrote 80% of the XML documentation and also helped create a good documentation file for the GitHub repository.
But the most important thing is that AI can write code. For about six months now, I’ve been paying $20 a month for a subscription to Claude Code by Anthropic because it boosts my productivity and allows me to solve some routine tasks much faster. Additionally, I use ChatGPT for quick information searches or writing technical documentation.
According to the StackOverflow, about 51% of professional developers use AI tools on a daily basis, which is a pretty significant number.
Source: StackOverflow Survey 2025
Vibe Coding
Recently, the term "vibe coding" has become quite popular. It refers to a new style of programming where AI writes the code for you. It all started with using AI for solving algorithmic tasks, but it has evolved to the point where even people who don’t know how to program are trying to create something on their own. With the arrival of more powerful models and code editors like Cursor, the buzz around this trend is only growing. Now, with just one prompt, AI can generate a large amount of code, from design to business logic, and explain how and why it works.
According to a StackOverflow Survey, only 12-15% of developers are into vibe coding. While these numbers are still small, the direction is already formed and will continue to grow in the future.
When I started working on my startups, I often needed to write frontend code, which, to be honest, I didn’t really enjoy, except when working with Angular. In this case, Claude helped me by creating the basic layout, including mobile-responsive designs, and linking it with frameworks. After that, I would manually improve the components, adjust the appearance, and make everything work.
The risks here are minimal. AI doesn’t deal with business logic, databases, or payment systems. The worst thing that could happen is that AI doesn’t give me the expected result, and I waste a few hours.
Still, you should only engage in vibe coding if you understand what’s happening and are trying to optimize processes, not if you’re relying on AI to do something you don’t fully understand or can’t do yourself.
How Effective is AI?
A study published this summer by the AI research group METR questioned whether AI coding tools really help experienced developers be more productive.
In the study, METR had 16 experienced open-source developers complete 246 tasks on large code repositories. Half of the tasks allowed them to use AI tools like Cursor Pro, while the other half didn’t.
Before starting, the developers thought AI would help them finish their tasks 24% faster. But the results were surprising: "Using AI actually made them 19% slower" the researchers said.
Notably, only 56% of the developers in the study had experience using Cursor, the main AI tool offered in the study. While nearly all the developers (94%) had experience using some web-based LLMs in their coding workflows, this study was the first time some used Cursor specifically. The researchers note that developers were trained on using Cursor in preparation for the study.
These results raise doubts about whether AI tools will always make developers faster. The researchers believe that developers spend a lot of time asking AI for help and waiting for responses, which slows them down. Also, AI struggles with large, complex codebases, like the ones used in this test.
The study’s authors are careful not to draw any strong conclusions from these findings, explicitly noting they don’t believe AI systems currently fail to speed up many or most software developers. Other large-scale studies have shown that AI coding tools do speed up software engineer workflows.
Source: METR Research 2025
At the same time, on Reddit, developers share their experiences with AI. For example, in the post "How we vibe code at a FAANG", it’s mentioned that AI increased feature development performance by about 30%, not in a small startup, but in a large IT company. In addition to writing code, AI also helped write tests and sped up code reviews.
Source: How we vibe code at a FAANG
As you can see, using AI at work is not a magic pill that solves all problems. It’s primarily a tool that can both help and slow down even experienced developers. From my personal experience, AI is great for simple, localized tasks where no unique solution is required. But the more complex the context and the larger the task, the more it can slow you down.
Learning with AI
Back in 2017, when I was a student, my C++ teacher taught us to write code based only on our knowledge and memory. We didn’t write code on paper or in Notepad; at that time, we used Visual Studio 2015, where IntelliSense wasn’t as developed, and we didn’t know about plugins like ReSharper. You know what was a big discovery for us? Shortcuts. This is when, after typing a keyword (like "for" or "switch") and pressing "tab" the IDE would write the code for you – not exactly writing the whole thing, but generating a template structure to speed up development. We thought it was really cool, but we soon gave it up. Why? Because we were learning to write code on our own.
You can read books like CLR via C# by Jeffrey Richter or Code Complete by Steve McConnell, but unless you’ve written thousands (or even tens of thousands) of lines of code, you won’t learn how to program properly or will do it poorly. During our C++ studies, we wrote linked lists, binary trees, worked a lot with memory, and even created games like Tic-Tac-Toe and Fifteen in the Windows console. And you know how great it feels when all the code is written by you, without any help or AI?
According to the annual StackOverflow survey, about 70% of people aged 18-24, who are just learning programming, use AI.
In my opinion, AI can help explain material in simple terms, which is a huge plus, and you can talk to it like a mentor if you don’t have one. But you need to write code yourself and spend time understanding why something isn’t working. Only then will the knowledge stick and produce results. Therefore, try to minimize AI’s influence on your learning. Don’t rely completely on the code it provides. It can often give incorrect information and only confuse you, so make sure to consult other sources as well.
Once you learn to write code on your own, work with AI, and understand where to use it best, it will truly boost your productivity.
Conclusion
The appearance of tools like ChatGPT, Claude, Cursor, and others is a big plus for the software development industry. They help reduce routine tasks, speed up, and make development easier for programmers. However, there are still tasks that AI doesn’t handle well. So, first and foremost, you should rely on your knowledge and always keep learning.
Thank you for reading this article to the end. Share your thoughts about AI in the comments – where do you use it, and how much has it improved your efficiency?
Top comments (5)
Great read.
AI clearly boosts productivity in some cases, but slows in others. What feels missing is ownership: most models live behind closed APIs. Imagine if your AI ran directly on-chain; transparent, verifiable, and even monetizable.
That’s when AI stops being just a tool and becomes an asset.
Thanks!
You nailed the real story here: AI tools can absolutely shine for atomic tasks, like boilerplate, docs, config parsing, scaling down grunt work and keeping devs in flow. The METR study dropping a 19% slowdown for experienced devs is a hard wake-up call. In meta-level orchestration, the loop of prompt → validation → prod breaks when AI isn’t fed context, or sucks devs into cognitive tape loops.
I found the best way to get the most out of these tools is to treat AI outputs like intern drafts, not merge-ready code. Embed auto-tests or linting into the pipeline so hallucinations surface as build failures, not runtime surprises.
For complex codebases, isolate AI usage to confined modules. Don't let it roam free over core logic. Localized, test-backed components are way cheaper to contain than sprawling tech debt.
Explicitly throttle “vibe coding.” Let it prototype UI wiring or test scaffolding, but never in core flows that define consistency or security.
AI shines when it’s a tool, not an autopilot. Pilot how AI fits into your dev lifecycle: scaffold → validate → verify. That’s how “boost productivity” stays real, instead of burning time chasing hallucinations.
I wrote an article on this on my Substack after trying to see if Claude AI could help me with my app. Claude blew me away once I starting giving it oversight, bounds, and validation.
AI is incredibly useful and capable when you stop viewing it as a replacement tool.
Have you ever vibe-coded?
Some comments may only be visible to logged-in visitors. Sign in to view all comments.