Is generated code harder to maintain?

#ai #githubcopilot #productivity #discuss

Code quality and maintainability

I found an interesting article about code quality and maintainability of AI generated code.

This is the TL;DR:

GitHub Copilot usage has led to an increase in AI-assisted code, which in turn has created a downward pressure on code quality. Code written with AI is often harder to maintain and of lower quality as it’s often verbose or copy-pasted. However, GitHub Copilot has had a positive effect on developer productivity and satisfaction.

My conclusion

So it looks like, that you have to be careful when using Copilot to not trade higher initial productivity right now with higher maintenance cost in the future...

What's your opinion?

Photo by Mimi Thian on Unsplash

Top comments (7)

Ben Halpern • Jan 31

I think: Most likely yes.

I've already been guilty of sneaking in a comment that didn't make sense because it was generated. This happens enough time and you lose the point of the code.

I don't think it's an all-or-nothing thing, but I suspect a lot of generated code is going to be on the low-readability spectrum.

Steffen H. Nielsen • Jan 31

At my company, we are having big discussion on when to use generated code vs when not to, specifcly to test the maintainability.

We had to greate a custom data handler, that manipulates some Json data, stored in a SQL database, with basic CRUD operations.

I tried first to make the full implementation myself, and then using ChatGPT-4.

To then test the maintainability, one of my coworkers was tasked with updating the schema of the Json to support a new property. He then went and did this in both solutions.

The result was pretty clear. The generated code, was harder to alternate to support this new property, while the human created code was much easier.

To sum up, out findings:

Code structure was much different to what we normally do (naming, order of logic etc).
Generated code was faster to produce, but still required enough time to alter, that the time spent was almost the same at writing the code by human.
The generated code, was not able to utilize already existing methods and serives (to much duplicated code)

Our conclusion is therefore, that generated code is good for idea generation and getting some quick solutions, but it must never be fully copied.

Marcio Frayze • Feb 1

Very interesting experiment. Thanks for sharing it.

Volkmar Rigo • Feb 1

Thanks for sharing.

Kevin Naidoo • Jan 31

The way I see it, If you working on a professional project, I would say it's unethical to just generate code and paste it in.

I sometimes use generated code, but it's because I have written this code so many times in my career that it's become tedious for me to manually write. Still, I review every line to make sure it does what it is supposed to do, including tweaking it to follow coding standards.

You should not just check in code you don't understand or doesn't meet your company's coding standards.

Cognitive complexity is also another problem because ML-generated code can become overly complicated and verbose.

So long story short :-) Generated code is fine, so long as there's someone behind the wheel making the engineering decisions. This is why code generation is better left to experienced developers.