I Tested How Good Is ChatGPT When Describing Code

A bunch of people is now using ChatGPT to write the code for them.

But I don't really care. I already know how to code.

So what do I do? The exact opposite. I'm using ChatGPT to explain complex blocks of code to me.

And today, I decided to push it to the limit. I decided to test Chat GPT and see how well it understands the code of various languages and complexities.

Ready to see the results?

ChatGPT Explaining Complex Code

Let's start from the easiest code snippet and work our way to the top.

I started with a simple React component.

import React, { useState, useEffect } from 'react';

interface Props {
  initialCount: number;
  step: number;
}

interface State {
  count: number;
}

const ComplexComponent: React.FC<Props> = ({ initialCount, step }) => {
  const [state, setState] = useState<State>({ count: initialCount });

  useEffect(() => {
    document.title = `Count: ${state.count}`;
  }, [state.count]);

  const handleIncrement = () => {
    setState(prevState => ({
      count: prevState.count + step
    }));
  };

  const handleDecrement = () => {
    setState(prevState => ({
      count: prevState.count - step
    }));
  };

  return (
    <div>
      <p>Count: {state.count}</p>
      <button onClick={handleIncrement}>Increment</button>
      <button onClick={handleDecrement}>Decrement</button>
    </div>
  );
};

export default ComplexComponent;

Explanation by ChatGPT

This code is a React functional component that takes in two props, initialCount and step, and uses the state hook useState and the effect hook useEffect to create a simple counter that increments and decrements based on the value of step and updates the document title to show the current count.

The component has two buttons, Increment and Decrement, which when clicked, will call the corresponding event handlers which will change the count state.

MUI Input

Let's step up our game a bit. Let's try to give it a source code for Input component of the famous material UI library.

The source code is quite big in size, around 400 lines of code.

Here's how ChatGPT explained it

This code is a React functional component called InputUnstyled that renders an input field and supports various props for customizing its behavior and appearance.

Python One-liner

So far, the model is doing qutie good. It can handle a lot of code. But can it also handle messy code?

For example, let's take a look at the following code. I have no idea what's happenig there.

(lambda f: f(f, 1))(lambda f, n: None if n > 100 else print((not n % 3 and "fizz" or "") + (not n % 5 and "buzz" or "") or n) or f(f, n+1)) 

ChatGPT didn't even flinch

This code is a lambda function that creates a recursive function that prints numbers from 1 to 100, but for multiples of 3 it will print "fizz" instead of the number and for multiples of 5 it will print "buzz" instead of the number.

If the number is both a multiple of 3 and 5 it will print "fizzbuzz" instead of the number. The recursive function will call itself until n > 100, then it returns None.

Something Crazy Now

Okay. So far, it's doing pretty good. But now, I wanted to try something crazy.

I literally searched for crazy Github gists. And this is what I found.

#!/usr/bin/env bash

${BASH_VERSION+shopt -s extglob lastpipe}

function doCompat {
    if [[ ${!KSH_VERSION} == .sh.version ]]; then
        builtin pids 2>/dev/null
        {
            eval function BASHPID.get "{ $(</dev/fd/$(($? + 3))) || exit 1; }"
        } <<-"3" 3<&0- <<-\4 4<&0-
        .sh.value=$(pids -f '%(pid)d')
3
        read -r .sh.value _ </proc/self/stat
4
    elif ! ${BASHPID+false}; then
        : # test for mksh or Bash
    else
        return 1
    fi
}

function let2 {
    typeset IFS=,
    printf %s $(($*))
    ((_))
}

function f {
    if [[ -p /dev/fd/0 ]]; then
        { (( $(</dev/fd/0) )) && f || return 1; } <<<"$(( $(<"$1") - 1 ))"
    elif [[ -f /dev/fd/0 ]]; then
        { let2 "$(</dev/fd/0) - 1" || kill -"$BASHPID"; } | f || return 1
    else
        echo 'unexpected input. (shouldnt happen).' >&2
        return 1
    fi
}

function main {
    if doCompat; then
        if [[ $1 == +([[:digit:]]) ]]; then
            set -m
            f <<<"$1"
        else
            echo 'First arg must be a digit.' >&2
            exit 1
        fi
    else
        echo 'Requires Bash, A recent ksh93, or mksh >= R41' >&2
        exit 1
    fi
}

FUNCNEST=10
main "$@"

# vim: set fenc=utf-8 ff=unix ft=sh:

As you might guess, ChatGPT didn't hesitated.

This code is a Bash script which contains several functions, including the doCompat, let2, f and main functions.

The script appears to check the type of shell that is running it and check for certain parameters passed to it. The script will exit with an error message if the shell is not compatible or if the first argument passed to it is not a digit.

If the shell is compatible and the first argument is a digit, it will execute the function f and pass the first argument to it. The function f appears to do some operations on the input, such as decrementing it and read it from a file descriptor.

The script also uses process ids, file descriptor and input/output redirection.

I'm Not Surprised

The power of AI is uncanny. It can write the code, but it can also help to understand the code.

I'm not surprised how good it performed on these various examples. And it's because of the simple fact.

I use GPT model to help me with code reviews each day.

In fact, I built a GitHub bot that automatically writes helpful description so I can review the code much faster.

In case you want to try it our yourself, here's the official website.