I firmly believe that, indeed, some programming teams may be smaller due to a single programmer's increased productivity with the assistance of a large language model (LLM) as a valuable tool. However, any company that replaces an entire department or team with an LLM fails to grasp the true nature of how LLMs function. A noteworthy yet often overlooked example is an amateur Go player who consistently defeated the world's most advanced Go AI by utilizing a basic concept of the game.
Essentially, the researcher managed to triumph over the 5000+ MMR Go AI approximately 96% of the time by employing what they termed the "double capture technique." They created multiple loops on the board to capture stones, but the AI neglected the inner loop. This is significant because it demonstrates that a Go program capable of defeating top Go players worldwide does not genuinely comprehend the game of Go. Even a novice Go player would know to evade such captures as they award numerous points to the opponent, yet this AI appeared indifferent.
Most importantly, these Go AIs employ the same methods used in all LLMs. Although these AIs may be able to pass the bar exam or compose a top-tier essay, they lack a true understanding of their actions. In the context of coding, this implies that these systems are approximating code writing rather than genuinely writing code like a programmer. They can mimic and identify patterns in existing code, but their limited understanding can also lead to hallucinations.
This is also the reason why these tools will always require the presence of an individual with domain expertise to fully leverage their capabilities. Without comprehension, these LLMs cannot operate independently. Even something like AutoGPT needs a chaperone to work properly.
For context, I am an engineer who has been working with GPT-3, GPT-4, and GitHub Copilot X for several months now. While these tools have saved me time in various instances, the experience is not akin to pair programming with a junior developer; it feels more like guiding a search engine to locate the information I need. Although it can generate code and retrieve information more rapidly than traditional engines, the resulting code is just as prone to flaws as any content found on GitHub or Stack Overflow.
This IMO is the biggest flaw with modern AI systems and until someone figures out a different way to build an AI or a way to cover these weaknesses, the current generation of AI won't be able to fully automate these more complex jobs without the help of a human.
I firmly believe that, indeed, some programming teams may be smaller due to a single programmer's increased productivity with the assistance of a large language model (LLM) as a valuable tool. However, any company that replaces an entire department or team with an LLM fails to grasp the true nature of how LLMs function. A noteworthy yet often overlooked example is an amateur Go player who consistently defeated the world's most advanced Go AI by utilizing a basic concept of the game.
Essentially, the researcher managed to triumph over the 5000+ MMR Go AI approximately 96% of the time by employing what they termed the "double capture technique." They created multiple loops on the board to capture stones, but the AI neglected the inner loop. This is significant because it demonstrates that a Go program capable of defeating top Go players worldwide does not genuinely comprehend the game of Go. Even a novice Go player would know to evade such captures as they award numerous points to the opponent, yet this AI appeared indifferent.
Most importantly, these Go AIs employ the same methods used in all LLMs. Although these AIs may be able to pass the bar exam or compose a top-tier essay, they lack a true understanding of their actions. In the context of coding, this implies that these systems are approximating code writing rather than genuinely writing code like a programmer. They can mimic and identify patterns in existing code, but their limited understanding can also lead to hallucinations.
This is also the reason why these tools will always require the presence of an individual with domain expertise to fully leverage their capabilities. Without comprehension, these LLMs cannot operate independently. Even something like AutoGPT needs a chaperone to work properly.
For context, I am an engineer who has been working with GPT-3, GPT-4, and GitHub Copilot X for several months now. While these tools have saved me time in various instances, the experience is not akin to pair programming with a junior developer; it feels more like guiding a search engine to locate the information I need. Although it can generate code and retrieve information more rapidly than traditional engines, the resulting code is just as prone to flaws as any content found on GitHub or Stack Overflow.
This IMO is the biggest flaw with modern AI systems and until someone figures out a different way to build an AI or a way to cover these weaknesses, the current generation of AI won't be able to fully automate these more complex jobs without the help of a human.
absolutely agree here, great text, thanks for sharing
youtu.be/l7tWoPk25yU
Exactly. Hadn't watched his video but I had read the research paper when it came out and it raised an eyebrow...
Literally this video