Matija Sosic for Wasp

Posted on Dec 11, 2023 • Originally published at wasp-lang.dev

AI code generation vs. coding by hand - what programming is going to look like in 202X 🤖 🤔

#programming #gpt3 #ai #webdev

We are working on a full-stack web framework for React & Node.js, that uses a simple configuration language to get rid of boilerplate. A number of times, we've been asked, “Why are you bothering creating a new framework for web app development? Isn’t ChatGPT / LLM X soon going to be generating all the code for developers anyhow?”.

This is our take on the situation and what we believe things will look like in the future.

Why do we need (AI) code generation?

In order to make development faster, we first came up with IDE autocompletion - if you are using React and start typing use, IDE will automatically offer to complete it to useState() or useEffect(). Besides saving keystrokes, maybe even more valuable is being able to see what methods/properties are available to us within a current scope. IDE's awareness of the project structure and code hierarchy also makes refactoring much easier.

Although that was already great, how do we take it to the next level? Traditional IDE support is based on rules written by humans, and if we, for example, wanted to make IDE capable of implementing common functions for us (e.g., fetch X using API Y, or implement quicksort), there would be just too many of them to catalogize and maintain by hand.

If there was only a way for a computer to analyze all the code we’ve written so far and learn by itself how to autocomplete our code and what to do about humanity in general, instead of us doing all the hard work ...

Delicious and moist cake aside, we actually have this working! Thanks to the latest advances in machine learning, IDEs can now do some really cool things, like proposing the full implementation of a function, based on its name and the short comment on top:

This is pretty amazing! The example above is powered by Github Copilot - it’s essentially a neural network trained on a huge amount of publicly available code. I will not get into the technical details of how it works under the hood, but there are plenty of great articles and videos covering the science behind it.

Seeing this, questions arise - what does this mean for the future of programming? Is this just IDE autocompletion on steroids or something more? Do we need to keep bothering with manually writing code, if we can just type in the comments what we want, and that’s it?

Support us! 🙏⭐️

If you wish to express your support for what we are doing, consider giving us a star on Github! Everything we do at Wasp is open source, and your support motivates us and helps us to keep making web app development easier and with less boilerplate.

⭐️ Thanks For Your Support 🙏

The Big Question: Who maintains the code once it’s generated?

When thinking about how ML code generation affects the overall development process, there is one thing to consider that often doesn’t immediately spring to mind when looking at all the impressive examples.

The question is - what happens with the code once it is generated? Who is responsible for it, and who will maintain and refactor it in the future?

Although ML code generation helps with getting the initial code for a specific feature written, it cannot do much beyond that - if that code is to be maintained and changed in the future (and if anyone uses the product, it is), the developer still needs to own and understand it fully. You can again use AI to help you, but in the end, you're the one responsible for it.

Imagine all we had was an assembly language, but code generation worked really well for it, and you could say “implement a function that sorts an array, ascending” and it would produce the required code perfectly. Would that still be something you’d like to return to in the future once you need to change your sort to descending 😅?

Or, to get closer to our daily lives, would it be all the same to you if the generated React code used the old class syntax, or functional components and hooks?

In other words, it means GPT and other LLMs do not reduce the code complexity nor the amount of knowledge required to build features, they just help write the initial code faster and bring the knowledge/examples closer to the code (which is really helpful). If a developer accepts the generated code blindly, they are just creating tech debt and pushing it forward.

Meet the big A - Abstraction 👆

If ChatGPT and the gang cannot solve all our troubles of learning how to code and understanding in detail how, for example, session management via JWTs works, what can?

Abstraction - that’s how programmers have been dealing with code repetition and reducing complexity for decades - by creating libraries, frameworks, and languages. It is how we advanced from vanilla JS and direct DOM manipulation to jQuery and finally to UI libraries such as React and Vue.

Introducing abstractions inevitably means giving up on a certain amount of power and flexibility (e.g., when summing numbers in Python, you don’t get to exactly specify which CPU registers are going to be used for it), but the point is that, if done right, you don’t need nor want such power in the majority of the cases.

What Uncle Ben actually meant: avoiding responsibility is the main benefit of abstraction! (Peter totally missed the point, unfortunately, and became Spiderman instead of learning how to code)

The only way not to be responsible for a piece of code is that it doesn’t exist in the first place.

Because as soon as pixels on the screen change their color it’s something you have to worry about, and that is why the main benefit of all frameworks, languages, etc. is less code == fewer decisions == less responsibility.

The only way to have less code is to make fewer decisions and provide fewer details to the computer on how to do a certain task - ideally, we’d just state what we want, and we wouldn’t even care about how it is done, as long as it’s within the time/memory/cost boundaries we have (so we might need to state those as well).

Let’s take a look at the very common (and everyone’s favorite) feature in the world of web apps - authentication (yaay ☠️ 🔫)! The typical code for it will look something like this:

import jwt from 'jsonwebtoken'
import SecurePassword from 'secure-password'
import util from 'util'

import prisma from '../dbClient.js'
import { handleRejection } from '../utils.js'
import config from '../config.js'

const jwtSign = util.promisify(jwt.sign)
const jwtVerify = util.promisify(jwt.verify)

const JWT_SECRET = config.auth.jwtSecret

export const sign = (id, options) => jwtSign({ id }, JWT_SECRET, options)
export const verify = (token) => jwtVerify(token, JWT_SECRET)

const auth = handleRejection(async (req, res, next) => {
  const authHeader = req.get('Authorization')
  if (!authHeader) {
    return next()
  }

  if (authHeader.startsWith('Bearer ')) {
    const token = authHeader.substring(7, authHeader.length)

    let userIdFromToken
    try {
      userIdFromToken = (await verify(token)).id
    } catch (error) {
      if (['TokenExpiredError', 'JsonWebTokenError', 'NotBeforeError'].includes(error.name)) {
        return res.status(401).send()
      } else {
        throw error
      }
    }

    const user = await prisma.user.findUnique({ where: { id: userIdFromToken } })
    if (!user) {
      return res.status(401).send()
    }

    const { password, ...userView } = user

    req.user = userView
  } else {
    return res.status(401).send()
  }

  next()
})

const SP = new SecurePassword()

export const hashPassword = async (password) => {
  const hashedPwdBuffer = await SP.hash(Buffer.from(password))
  return hashedPwdBuffer.toString("base64")
}

export const verifyPassword = async (hashedPassword, password) => {
  try {
    return await SP.verify(Buffer.from(password), Buffer.from(hashedPassword, "base64"))
  } catch (error) {
    console.error(error)
    return false
  }
}

And this is just a portion of the backend code (and for the username & password method only)! As you can see, we have quite a lot of flexibility here and get to do/specify things like:

choose the implementation method for auth (e.g. session or JWT-based)
choose the exact npm packages we want to use for the token (if going with JWT) and password management
parse the auth header and specify for each value (Authorization, Bearer, …) how to respond
choose the return code (e.g. 401, 403) for each possible outcome
choose how the password is decoded/encoded (base64)

On the one hand, it’s really cool to have that level of control and flexibility in our code, but on the other hand, it’s quite a lot of decisions (== mistakes) to be made, especially for something as common as authentication!

If somebody later asks “so why exactly did you choose secure-password npm package, or why exactly base64 encoding?” it’s something we should probably answer with something else rather than “well, there was that SO post from 2012 that seemed pretty legit, it had almost 50 upvotes. Hmm, can’t find it now, though. Plus, it has ‘secure’ in the name, that sounds good, right?”

Another thing to keep in mind is that we should also track how things change over time, and ensure that after a couple of years, we’re still using the best practices and that the packages are regularly updated.

If we try to apply the principles from above (less code, less detailed instructions, stating what we want instead of how it needs to be done), the code for authentication might look something like this:

auth: {
    userEntity: User,
    externalAuthEntity: SocialLogin,
    methods: {
      usernameAndPassword: {},
      google: {}
    },
    onAuthFailedRedirectTo: "/login",
    onAuthSucceededRedirectTo: "/dashboard"
  }

Based on this, the computer/compiler could take care of all the stuff mentioned above, and then depending on the level of abstraction, provide some sort of interface (e.g. form components, or functions) to “hook” in with our own e.g. React/Node.js code (btw this is how it actually works in Wasp).

We don’t need to care what exact packages or encryption methods are used beneath the hood - it is the responsibility we trust with the authors and maintainers of the abstraction layer, just like we trust that Python knows the best how to sum two numbers on the assembly level and that it is kept in sync with the latest advancements in the field. The same happens when we rely on the built-in data structures or count on the garbage collector to manage our program’s memory well.

But my beautiful generated codez 😿💻! What happens with it then?

Don’t worry, it’s all still here and you can generate all the code you wish! The main point to understand here is that AI code generation and framework/language development complement rather than replace each other and are here to stay, which is ultimately a huge win for the developer community - they will keep making our lives easier and allow us to do more fun stuff (instead of implementing auth or CRUD API for the n-th time)!

I see the evolution here as a cycle (or an upward spiral in fact, but that’s beyond my drawing capabilities):

language/framework: exists, is mainstream, and a lot of people use it
patterns start emerging (e.g. implementing auth, or making an API call) → AI learns them, offers via autocomplete
some of those patterns mature and become stable → candidates for abstraction
new, more abstract, language/framework emerges
back to step 1.

It’s the circle of (language) life, and it moves us all - Ingonyama nengw' enamabala, …

Conclusion

This means we are winning on both sides - when the language is mainstream we can benefit from AI-powered code generation, helping us write the code faster. On the other hand, when the patterns of code we don’t want to repeat/deal with emerge and become stable we get a whole new language or framework that allows us to write even less code and care about fewer implementation details!

Thanks for reading and hope you found this post informative! I'd love to hear if you agree with this (or not), and how you see the future of programming powered with the AI tools.

Top comments (37)

Ivan Zanev • Dec 11 '23

I asked ChatGPT if it can generate code entities, services and repositories for a school management system (students, curriculum, etc.). I also asked for it to generate HTML page where teachers can file a request to be hired. I added some requirements and asked for a generation of Laravel entities, services, repositories and everything required. Abstraction and continuous improvement (reworks) is also not a point - the AI can do it. The biggest problem is keeping continuity - if the AI assistant forgets the premises and constraints, you will have to start again. But it's not that big of a deal. Tell me something it can't do.

Matija Sosic • Dec 12 '23

Hey Ivan, thanks for joining the discussion! Sounds like you had a great experience building CRUD for your app with AI - that's one of my favorite use cases, too. Have you tried usemage.ai/?

Re what AI cannot do - it can't develop new abstractions, that never existed before (e.g., moving React from class-based components to hooks). That's one side of it - in my opinion, we don't want to freeze, e.g. React development, and stay on React 18 forever. We don't even want to stay with React as new, better/higher-level frameworks should emerge, just as React emerged after jQuery, Backbone, and others.

On the other side, and that's what I talked more about in the article, you care about the final code. Even if we assume that AI can generate everything perfectly, you still commit this code, other developers review it, and you want to be able to edit it. You need to understand it fully and be able to explain what each part does and why you implemented it the way you did. You want it to use the latest libraries/frameworks and the most up-to-date best practices.

That's why my point is that coding won't go away because of AI, however perfected it is - it will be faster and easier in some instances, but still the human is the one who holds the final responsibility for it.

Ivan Zanev • Dec 12 '23

Hi, Matija. I have never used AI before to generate a system but since reading your post I decided to try. The School Management System was the first thing that came to my mind. I just tried useimage.ai. Again, with the same test requirements. The app was generated in 1-2 minutes.

Your thinking is right about the social responsibility and authority. The AI cannot replace the fact that it is you. And there will be people who still come to you for solutions and decisions. No matter of the fact that an AI can do it, you will be expected to do it by using the AI.

However, my thinking is that the AI will reach the place where it will be an Automated Code Management solution. The example you gave about React will not be an issue since people wouldn't care about what happens underneath - the AI will upgrade and refactor as needed. Or it will re-create the system much like one of these guys that can recreate paintings.

Matija Sosic • Dec 12 '23

Now thinking about it, I think the main question do we want to be able to understand the code AI generates, or we don't care at all? I think we can't allow to have it as a black box, because that way we have no idea what AI actually did nor we can check it.

That means we want to be able to understand the code that is being generated. And to understand it easier, we probably want that code to be generated in the newer rather (e.g. React) rather than older technologies (e.g. jQuery or Backbone).

So yes, we don't need our code to be immediately using the latest version of React etc, (just like most of developers don't use it as well), but it still has to be new enough so we don't have a problem understanding it.

What was your thought about recreating paintings, that sounds interesting but I didn't get it?

Ivan Zanev • Dec 12 '23

The answer to the question "Do we care at all?" is "We do not care unless it breaks and then we have to call someone.". My question is whether there would be a self-correcting system. I will make the following analogy. In your home almost 100% there is a washing machine. As part of the industrial age, mechanisms were invented and utilities to automate effort. Nowadays we all have a washing machine in our homes. There is an argument that the washing machine does not do the job right but I don't know how many people will accept such an argument. The internals of the machine are not known and the machine can be repaired only by a technician. Nowadays there are washing machines hooked to an app that makes a quick diagnosis as to why the machine is not working. But there is no way that the machine can self-repair itself which is the premise of your post - that the systems we are building are not just software, but firmware and hardware and they cannot self-repaired.

But, supposedly, you don't code, just input the requirements and the AI generates the output system. If you can really capture the requirements of a system (guarantee there will not be anything hidden, additional or inexplicable to the AI), you can count on the AI to be able to regenerate the system in a way that the behavior will be the same. Such a system can be self-repairable, upgradable and no technicians will be required to maintain it - a robot.

I was saying about mavericks that can recreate a painting and it will look the same as the original. I can't say that code is a painting, but there was a saying that code is art, probably that's why I said it.

Saurabh Rai • Dec 12 '23

@matijasos You should have added the image of the whole code base. A small snippet really makes it look simpler.

Matija Sosic • Dec 12 '23

hah true! I should have added more screenshots, or stacked them one behind another :)

Igor Kotua • Dec 11 '23

Amazing article! Looking forward for the times when Copilot like models can be easily ran locally (I think we are quite close)

Matija Sosic • Dec 11 '23

Thanks Igor! Yeah, exciting times :)

Sattineez_SSC • Dec 15 '23

I use AI for explain to me some code of other languages I don't know, to show me examples and to teach me something. I don't think it will substitute programmers/devs, because it "feeds" on what programmers/dev upload (by GitHub, stackoverflow...), it can algorithmically speaking generate new code but it won't have the creativity to generate new point of view or how to approach the problem on another way I guess