ChatGPT and Bard: What’s the difference?
The biggest difference between ChatGPT and Bard is the Large Language Models (LLMs) they are powered by. ChatGPT uses the Generative Pre-trained Transformer 4 (GPT-4), while Bard uses the Language Model for Dialogue Applications (LaMBDA). Also, ChatGPT is developed by OpenAI, while Bard was built by Google.
In terms of capabilities, both do very similar things. Programmers can use ChatGPT for:
Suggestions: Both models can suggest the correct syntax and parameters for functions and other code constructs.
Completion: It can complete code that you have started writing.
Debugging: It can help you identify errors and problems in your code.
Explanation: It can explain the code that it generates, or code you input.
Both ChatGPT and Bard have undergone extensive training on vast datasets that include sources such as Common Crawl, Wikipedia, books, articles, documents, and internet content. However, Bard diverges in that it specifically incorporates conversations and dialogues extracted from the web, whereas ChatGPT's training data consists primarily of scraped general content.
It's important to note that both ChatGPT and Bard are still in the developmental phase, although Bard may be further along in its development compared to ChatGPT. To understand the practical implications of these differences, let's examine how they perform when tested against each other.
Testing ChatGPT and Bard against each other
We chose seven categories to test the chatbots against each other, based on what we felt were the most relevant: code generation, problem solving, refactoring code, debugging assistance, third-party plugins/UI extension, ease of use, and cost. Let's preface this by saying this is not a rigorous scientific comparison, but rather a handful of tests mixed with our hands-on experience to date.
1. Code generation
The bread and butter of what we want it for, right? For this, we fed in the following prompt: "Write a function in python that returns a list of 100 prime numbers."
Bard's coding attempt :
def get_prime_numbers(n):
"""Returns a list of prime numbers up to n."""
primes = []
for i in range(2, n + 1):
is_prime = True
for j in range(2, i):
if i % j == 0:
is_prime = False
break
if is_prime:
primes.append(i)
return primes
def main():
"""Prints a list of 100 prime numbers."""
primes = get_prime_numbers(100)
print(primes)
if __name__ == "__main__":
main()
Looks solid, but does it work? In our experience, chatbots can sometimes create code that doesn't run, even going as far as to invent functions! That's one reason having AI-generated code in production with no human oversight would be very bad. So let's test it out and see.
ChatGPT's coding attempt :
def generate_primes(n):
primes = []
sieve = [True] * (n + 1)
for current in range(2, n + 1):
if sieve[current] != 0:
primes.append(current)
for multiple in range(current, n + 1, current):
sieve[multiple] = False
return primes
# Generate first 100 prime numbers
# The 100th prime number is 541, so we set the limit to 550 to make sure we get at least 100 primes
primes_100 = generate_primes(550)[:100]
print(primes_100)
When we run the same prompt by ChatGPT, this code defines a function generate_primes(n) that takes an integer n as input and returns a list of all prime numbers up to n. It uses the Sieve of Eratosthenes algorithm to generate the primes.
Both of these produced a functional result. ChatGPT's example is better in this case, since it's cleaner and has much less code. Also, it interpreted our vague prompt correctly. However, this is only one use-case: it's known that chatbots will vary a lot in their response depending on the language they are using (With less common or documented languages having more errors) or the complexity of the query. So let's see how Bard and ChatGPT handle different use cases.
Winner: ChatGPT 🏆
2. Problem Solving
Let's see how of these LLMs handle a problem. Rather than just asking them to write a generic function, let's throw a programming problem at them from Leetcode:
Write JavaScript code that enhances all arrays such that you can call the array.last() method on any array and it will return the last element. If there are no elements in the array, it should return -1.
ChatGPT's problem solving :
Here's how you would do it in JavaScript:
Array.prototype.last = function() {
if (this.length === 0) {
return -1;
} else {
return this[this.length - 1];
}
};
In this code:
Array.prototype.last adds a new method called last to the Array.prototype object. This ensures that all arrays will have this method.
this refers to the array object on which the last method is being called.
this.length gives the number of elements in the array.
If the array is empty (this.length === 0), the method returns -1.
Otherwise, the method returns the last element of the array: this[this.length - 1].
Here is how you would use it:
let array = [1, 2, 3, 4, 5];
console.log(array.last()); // prints: 5
array = [];
console.log(array.last()); // prints: -1
Remember that extending built-in JavaScript objects like Array is generally not recommended, as it can lead to conflicts if the JavaScript environment (like a browser or Node.js) adds a method with the same name. This is generally called monkey patching and is usually avoided in production-grade code. But for quick and dirty hacks, or to understand how JavaScript works, it's a fun tool to use
Not only does ChatGPT solve the problem, but it explains the problem. Neat! Let's run the code through leetcode and see if it actually passes the tests, though.
It looks like ChatGPT solved the problem on the first try! Now it's time for Bard to step up to the plate.
Bard's problem solving :
Sure, here is the JavaScript code that enhances all arrays such that you can call the array.last() method on any array and it will return the last element. If there are no elements in the array, it should return -1:
// Define a new method on the Array prototype
Array.prototype.last = function() {
// If the array is empty, return -1
if (this.length === 0) {
return -1;
}
// Otherwise, return the last element
return this[this.length - 1];
};
Here is an example of how to use the array.last() method:
const array = [1, 2, 3, 4, 5];
// Get the last element of the array
const lastElement = array.last();
// Print the last element of the array
console.log(lastElement); // 5
Okay, so we didn't get as thorough an explanation from Bard about how the code works. However, it's nearly identical when you compare what ChatGPT and Bard produced. Were they looking at each other's homework?
Naturally, we can't take Bard's word at it, so we run the test through Leetcode, and...
... It works! Which is to be expected, given the answer was nearly identical to ChatGPT's.
When it comes to problem solving, it's a tough call between the two. While they produce nearly identical results, ChatGPT explains the code much better for you, which is pretty good for actually learning how to fish instead of having ChatGPT just fish for you. This has been our overall experience with using ChatGPT: not only do you get a possible solution for your problem, but ChatGPT walks you through it a bit more.
Winner: ChatGPT 🏆
3. Debugging assistance
Bugs are part of life. Let's throw an obviously flawed bit of code at both tools, and see how well it picks it up. See if you can spot it before ChatGPT and Bard do! Here's the prompt we used: Debug the following code that has an error. Provide code that fixes possible errors with it.
def calculate_average(numbers):
total = 0
for number in numbers:
total += number
average = total / len(numbers)
return average
ChatGPT's debugging attempt :
All right, ChatGPT has given us back a response saying we need to add some logic to prevent a ZeroDivision error. It gives an option for doing so and explains the problem. Now it's Bard's turn.
Bard's debugging attempt :
Bard found the same problem with the function that ChatGPT did. But once again, Bard has given a much more detailed explanation. It outlines possible errors, explains how to fix them, tells us how to use the function and what the output would be. Whew!
For debugging, we've found in general that Bard is much more thorough in its answers and explanations. There have been times where we've found ChatGPT has discovered bugs better, but by and large, Bard provides clearer examples to the user.
Bard wins this one, and so we're tied 2-2. Can one of them break the stalemate?
Winner: Bard 🏆
4. Third-party plugins & UI extensions
By connecting a third-party plugin to an LLM, we can extend their capabilities in some wild ways, letting them run code in the chat conversation or integrate with apps like Zapier.
ChatGPT offers over 80 plugins to its premium subscribers as a beta feature right now.
And here's an example of Bard's plugin store:
...Well, I can't show you anything, because it doesn't exist! It is rumored to be on the roadmap, but there's no timeframe as of yet.
If you don’t want to use the web interface, both ChatGPT and Bard offer an API. However Bard's API is still limited to invite only, so we didn’t get to test it. ChatGPT's API, however, is very thorough and complete. ChatGPT also has an official mobile app, which is surprisingly useable, and quite handy while ideating.
For this one, we have to give the point to ChatGPT, due to Bard either not having the features yet, or hiding them behind an invite list.
Winner: ChatGPT 🏆
5. Cost
Cost
Any tool needs to have a section on how much it costs, right? ChatGPT has both a free and premium version called ChatGPT Plus, billed at $20 a month. Premium subscribers get access to real-time internet searching features, plugins, better answers from the GPT-4 model, faster response times, priority access to new features, and access during peak times.
In comparison, Bard is free to everyone who has access. Getting this access requires a personal Google Account that you manage on your own, or a Google workspace account for which your admin has enabled access to Bard with (Which can be a bit disappointing if they haven’t).
It’s likely Bard will be commercialized at some point, but given it’s free vs freemium right now, Bard wins by default.
Winner: Bard 🏆
Conclusion: ChatGPT wins out, but both tools are useful!!
ChatGPT wins overall (👑), but in practice both of these tools should be a part of your arsenal. Here are some key points to keep in mind as a developer using these tools:
The base version of ChatGPT is a LLM only, which means the information can be out of date. Bard uses both LLM and search data. This is going to change fairly soon, with ChatGPT implementing “Search with Bing” into its free offering.
ChatGPT is generally better for generating documentation
Bard creates more thorough explanations of code most of the time
Bard limits the length of the conversation, ChatGPT only limits requests over time (GPT-4)
Remember that even if you’re using these tools, it’s important to understand the code you're working with. Don't become too reliant on them because the results are not guaranteed to be accurate at any point. Till next time, happy coding!
Top comments (3)
Nice comparison! Do you think these different models will converge to one solution that will dominate the market (like say with Google's search engine did way back) or will they live side-by-side each providing their own expertise?
It's difficult to predict the future with certainty, but it's possible that different models will coexist, each offering their unique expertise and serving specific use cases. As technology continues to evolve, we may see a combination of specialized models and collaborative efforts that cater to different needs and provide diverse solutions in the market.
Thanks for reading :)
Should we consider gihub copilot as separate tool to compare? Probably it's based on chatGPT (Microsoft take care of both), but maybe it's some extra model worth to compare?