(source for image)
This month on the 12th, the AI company Cognition AI showed to the world the first AI Software Engineer, Devin. Devin is the first AI that can not only have its own personal code editor and web browser but also does the process that a real software engineer would do. Seeing what Devin can do has fascinated me, but at the same time, I have also realized that I have heavily underestimated AI in the future and what that means for developers.
The Good...
Devin can be extremely helpful for developers in many ways.
1) Debugging
In one of the demos Cognition AI showed, a developer in the company showed how Devin can debug your code for you. They had a Git repository and had Devin come up with test cases and test the code for him. It turns out that the code had a bug and Devin debugged the code for him.
Devin debugs code by adding code that catches errors, for example, in Python, a print function could be used to detect any errors in Python code.
2) Self-Learning
Devin can self-learn, meaning for example, one developer trained Devin using a blog that she had found. Devin then learned from the blog and did the step-by-step process the blog had given. When Devin tested to finished code, it arrived to an error that the blog had. Devin debugged and fixed the code and outputted some pretty good results.
3) Performance
In the SWE-bench (Real World Software Engineering Performance) Devin has a huge (compared to other competitors) 13.86% (that means that this LLM (Large Language Model) can resolve real-world GitHub issues 13.86% of the time)! To put that into comparison, the AI in second place in terms if the SWE-bench is Claude 2 with only a 4.8% of solving a real-world Github issue.
...The Bad...
Everything that is first at something is going to be the worst in the area, what I mean is that Devin is at its worst, and will improve more and more over the years.
1) Speed
Depending on what Devin is coding, Devin can take minutes, hours, or even days! You could argue that a normal human software engineer could also take days, but compared to its competitors like ChatGPT and Llama, it's pretty slow.
2) Dependency on user's prompts
Like most AIs, Devin needs good prompts for good results. So giving Devin a 9-word sentence like:
Code an AI pet simulator in HTML and JS
Is not going to give you as good results if you would have given Devin a more detailed prompt.
3) Performance
Yes, Devin does have the best SWE-bench performance out of any AI out there, but it only solves 13.86% of real-world GitHub issues. What about the other 86.14%? Devin can not solve real-world Github issues 86.14% of the time, so your not going to be able to 100% automate your coding.
...and The Ugly
Devin right now might not be able to replace developers, but Devin does reveal a path we are heading towards that I never thought was possible.
AI taking jobs
I knew that AI has, is, and will take jobs from humans. Before, I thought developers didn't apply to this, after all, we are the ones who code the AI in the first place. But AI is already starting to replace developers. Developers are very dependent on AI; using it to debug their code, adding code snippets, or even completely coding for them.
Of course AI is definitely not perfect, so developers still need to code, but in the future, AI will be so advanced, I'm worried that not only will developers be so dependent on AI that AI is just doing developers’ jobs for them, but companies will start replacing developers with AI. After all, why pay over $100,000 per year per developer when you could pay $100 per year for unlimited AI.
ASI
Expert Ben Goertzel has predicted that AI will reach ASI (Artificial Super Intelligence) as early as 2027 (source). You could think of ASI as sentient AI or the singularity. Artificial Super Intelligence is where AI is so advanced that it has its own conciseness, its own emotions, and it surpasses human intelligence in all kinds of ways. In other words, ASI will be so advanced compared to humans, it would be a joke if you hired a human developer when ASI exist.
Conclusion
So in conclusion, Devin is awesome and is going to help developers, but it's not a major threat to developers right now. Devin has great potential, as it is the first AI software engineer, but we should as keep in mind because it's the first AI software engineer, it is also going to be the worst AI software engineer.
What Devin is, though, is a wake-up call to people who want to learn to code to do it now. 2024 is the year to learn how to code, and based on the exponential growth of AI and the abundant amount of junior and entry-level developers, this year might be your last chance. Read my article to learn more.
If developers don't want to get swept up in the AI tsunami, it's best that we, use AI to our advantage and instead of using AI to automate our coding we should use AI in our applications and websites, using APIs like OpenAI or Stability AI, or even creating your own like I did here.
Top comments (0)