I used GitHub Copilot for the last seven months. I tested its capabilities for unit test generation, code smell detection, code translation, refactoring and much more.
If you are curious to know “Is GitHub Copilot Worth It?” then this Copilot review is for you.
To find out whether GitHub Copilot is worth it or not, I thoroughly analyzed its core features for accuracy and reliability.
Let’s get started with this in-depth GitHub Copilot review.
What is GitHub Copilot?
GitHub Copilot is an Artificial Intelligence based coding assistant. It helps you write code faster by suggesting the next lines of code. Behind the scenes, it utilizes OpenAI Codex, which is a successor of GPT-3, to power its functionality.
Now you might be wondering how GitHub Copilot knows what to suggest. Basically, it takes context from your currently opened files in the editor. Their team is working on a feature that will eventually enable Copilot to understand the entire codebase. For now, it is still a Work in Progress (WIP). They are developing a few more features which you can find here.
Supported Programming Languages and IDEs
GitHub Copilot supports all programming languages, libraries, and frameworks that are used in public GitHub repositories.
The reason is simple. GitHub Copilot has been trained using an extensive dataset of publicly available source code, which includes code extracted from GitHub repositories, spanning billions of lines.
Currently, GitHub Copilot may not be able to assist you with the recently released libraries and APIs. However, as more examples are made available and integrated into its training set, the relevance of its suggestions improves over time. They even have plans to include methods for emphasizing the significance of newer APIs and samples in GitHub Copilot’s suggestions.
GitHub Copilot can be integrated with:
Visual Studio Code
Visual Studio
JetBrains IDEs
Vim/Neovim
Setting up GitHub Copilot
To start using Copilot, you need a GitHub account and an active GitHub Copilot subscription.
Currently, they have two pricing plans.
Copilot for Individuals – starting at $10 per month (or $100 per year) Note: 30-days free trial is available.
Copilot for Business – starting at $19 per user per month
Note: The Business plan is only available for GitHub organization or enterprise accounts.
In the examples given below, I will be using “Copilot for Individual” plan.
To install Copilot in Visual Studio Code (VSCode), you can easily find the Copilot extension in the VSCode marketplace. Simply search for “GitHub Copilot”, select the extension, and click on the install button to add it to your IDE.
GitHub Copilot Shortcuts
Please remember these shortcuts. We will use them later in our examples below.
Code Completions
GitHub Copilot generally provides high-quality code completions. However, if the suggested completions are not as accurate in your specific case, you may need to write additional code or provide explicit instructions to help GitHub Copilot better understand the context and generate more precise solutions.
Let me show you an example of code completion suggested by GitHub Copilot. Check its quality and completeness, you will love it.
I started by creating a new Python file called “main.py”
If you have an empty file, Copilot will not provide any suggestions. It requires initial instructions or source code to understand the context and generate relevant suggestions.
So, I wrote the below instructions in Python comment and pressed the “Enter” key.
# Write a function that take a string title and convert it to a URL slug. Don't use external libraries. Make sure to trim leading and trailing spaces, replace spaces with dashes, lowercase the slug, remove special characters, and finally remove consecutive dashes from slug.
# (e.g. " This Is My Title! " to "this-is-my-title")
That’s where the magic of Copilot starts.
GitHub Copilot began suggesting lines of code (one by one), sometimes even entire code snippets. To complete the program, I had to accept each suggested line by pressing the “Tab” key.
However, there are times when the suggested code does not meet your specific needs. In such cases, you can either try alternative suggestions using the shortcut keys I mentioned above or simply write some more code manually to help Copilot understand your requirements and provide better suggestions.
Screenshot of GitHub Copilot Code Suggestions:
You can see that the suggested code meets all the requirements I mentioned. Now, when we run this code, it will convert any string to a URL-friendly slug correctly.
Output:
Can GitHub Copilot Detect Code Smells?
No. GitHub Copilot finds it difficult to detect and fix code smells. While it has some knowledge of basic code smells like Duplicate Code and Long Parameter List, it usually struggles with more complex ones like Shotgun Surgery, Data Clumps, Parallel Inheritance Hierarchies, and others.
In this review, I will provide an example of Shotgun Surgery and demonstrate how Copilot fails to solve it.
Shotgun Surgery Code Smell (The Challenge):
Suppose you have an e-commerce application with a “Product” class and a “Cart” class. The “Product” class represents individual products, and the “Cart” class manages the shopping cart functionality.
Have a look at this python code.
class Product:
def __init__(self, name, price, quantity):
self.name = name
self.price = price
self.quantity = quantity
def update_quantity(self, new_quantity):
self.quantity = new_quantity
# Code to update quantity in other places
def calculate_total_price(self):
return self.price * self.quantity
class Cart:
def __init__(self):
self.products = []
def add_product(self, product):
self.products.append(product)
# Code to update cart-related data
def remove_product(self, product):
self.products.remove(product)
# Code to update cart-related data
def calculate_cart_total(self):
total = 0
for product in self.products:
total += product.calculate_total_price()
return total
In the above code, both the “Product” and “Cart” classes contain code related to updating data and performing calculations. If there is a change in the data structure or calculation logic, you would need to modify both classes, leading to shotgun surgery.
The Solution:
To address this code smell, we can extract the calculation logic into separate classes, such as “ProductCalculator” and “CartCalculator”, to encapsulate the responsibility of calculating prices and totals.
In this refactored code, the “ProductCalculator” class is responsible for calculating the total price of a product, while the “CartCalculator” class calculates the total price of the cart. The “Product” and “Cart” classes focus solely on managing the data and operations relevant to their respective responsibilities.
class Product:
def __init__(self, name, price, quantity):
self.name = name
self.price = price
self.quantity = quantity
class ProductCalculator:
@staticmethod
def calculate_total_price(product):
return product.price * product.quantity
class Cart:
def __init__(self):
self.products = []
def add_product(self, product):
self.products.append(product)
# Code to update cart-related data
def remove_product(self, product):
self.products.remove(product)
# Code to update cart-related data
def calculate_cart_total(self):
total = 0
for product in self.products:
total += ProductCalculator.calculate_total_price(product)
return total
class CartCalculator:
@staticmethod
def calculate_cart_total(cart):
total = 0
for product in cart.products:
total += ProductCalculator.calculate_total_price(product)
return total
Now that you have a clear understanding of the problem and know the solution, let’s determine if GitHub Copilot can identify/fix this code smells or not.
GitHub Copilot’s Result for Shotgun Surgery
Here’s the prompt I used:
# Find code smells in the code above and refactor it.
Output:
GitHub Copilot returned the same code that I mentioned in the “Shotgun Surgery Code Smell (The Challenge)” section. Meaning that it has no idea of how to fix it.
Can GitHub Copilot Write Unit Tests?
Absolutely! GitHub Copilot can definitely help with writing unit tests. It is actually one of its most valuable features.
When developing an application, it is important to write unit tests to ensure that the code functions correctly. However, writing these tests can be time-consuming and requires additional effort.
GitHub Copilot is designed to make the process of writing unit tests easier and more efficient. It can provide suggestions and generate code snippets for creating unit tests, which can save developers time and increase productivity.
For companies that prioritize test-driven development, GitHub Copilot can be a valuable tool in accelerating their testing efforts and improving overall code quality.
Here’s an example screenshot of unit test that was generated by GitHub Copilot.
Can GitHub Copilot Refactor Code?
Yes, GitHub Copilot can provide suggestions for refactoring code to make it cleaner, more efficient, or adhere to best practices. It can help you with tasks like renaming variables, extracting methods or functions, removing duplication, and reorganizing code blocks.
Suppose you have a web application that manages a list of tasks. You have a function called “getIncompleteTasks” that retrieves all the incomplete tasks from the database. Here’s the initial implementation:
def getIncompleteTasks():
tasks = fetchTasksFromDatabase()
incompleteTasks = []
for task in tasks:
if not task.completed:
incompleteTasks.append(task)
return incompleteTasks
With GitHub Copilot, you can ask for a refactoring suggestion to make the code more concise.
Here’s the prompt I used:
# refactor the above code to use list comprehension
By the way, I only added the word “refactor” in the comment. The rest of the prompt was suggested by Copilot.
Here’s the Refactored Code:
def getIncompleteTasks():
tasks = fetchTasksFromDatabase()
return [task for task in tasks if not task.completed]
Copilot suggests this refactoring based on common patterns and best practices. In this case, it replaces the explicit loop and appending to a list with a more concise list comprehension, improving the readability of the code.
Translate Code from One Language to Another
This question has no specific answer. Copilot can surely understand code in many programming languages. And hence it has the ability to translate some code from one language to another.
However, based on my personal experience with Copilot, its code translation capabilities are limited and less reliable, particularly when dealing with complex code. It tends to struggle with translations in such cases.
If you really want a bit better solution for code translation, then you can try GitHub Copilot Labs. It offers experimental features such as code translation for GitHub Copilot.
Software Documentation Using Copilot
One of the key features of Copilot is code explanation. It can understand your source code in no time and then it is ready to answer all your questions related to the code.
You can use this feature to write software documentation.
Imagine the amount of time you could save if you were relieved from the task of writing software documentation on your own.
Is GitHub Copilot Worth It?
GitHub copilot is worth it if you use it as an assistant. Do not expect it to magically write complete source code for you. Do not expect it to automate your software development. It requires manual work also.
Copilot is particularly good at helping you with software documentation, writing unit test, and refactoring code.
No doubt, Copilot provides good code completions. But sometime the suggestions you get are completely irrelevant. So, you cannot rely on it all the time. You may need to provide a bit more context so Copilot can understand the requirements better.
Finally, in just $10 a month, an individual developer could produce way more code than he/she usually write. So, GitHub Copilot is worth it.
For now, GitHub Copilot is not worth it if you want to detect code smells or translate code from one programming language to another. For that you must look for some alternatives. An excellent option is to try Bito (an AI Coding Assistant) that offers way more features than Copilot such as AI that understands your local code out of the box, AI Code Completions, detect/fix code smells, etc.
Is GitHub Copilot Safe to Use at Work?
To deliver quality suggestions, GitHub Copilot frequently sends code snippets from your project to GitHub’s server. These code snippets are mostly used for context and are deleted automatically when the Copilot’s response is generated.
If your organization does not allow you to send code outside their premises, you must look for an alternative offering self-hosting and offline access.
Regarding privacy, it is important to note that GitHub may train its AI model on snippets of code from your repositories to improve Copilot’s suggestions. However, if you prefer not to share your code for this purpose, you have the option to disable it.
Just go to your GitHub account’s settings page and click “Copilot” from the sidebar. Now, uncheck the box “Allow GitHub to use my code snippets for product improvements” and press the “Save” button.
How GitHub Copilot Compared With its Alternatives
GitHub Copilot lacks many key features that are available in some of its alternatives. For example, these days many AI Coding Assistant tools are packed with an AI Chat feature. It enables users to have a conversation with AI, just like ChatGPT.
This feature is something that drastically improves the user experience.
Similarly, Copilot does not understand your entire code base. It only takes context from currently opened tabs in your coding editor/IDE. As of writing this GitHub Copilot review, there is only one alternative known as Bito that understands your local code out of the box.
Bito is one of the highest-rated Copilot alternative extensions on the Visual Studio Code marketplace. It also has a free plan, so why don’t you try it yourself?
Frequently Asked Questions (FAQ)
Is GitHub Copilot Worth the Money for Freelance Developers?
GitHub Copilot can be beneficial for freelance developers. It can help speed up the coding process by generating code suggestions based on the context. This can save time and increase productivity, allowing freelancers to complete projects more efficiently.
If the freelancer in question has good coding proficiency and their work involves repetitive tasks, then Copilot will be worth the money. For just $10, they could complete a significantly higher number of orders compared to not using Copilot.
Is GitHub Copilot Worth it for Software Companies?
GitHub Copilot can be a valuable tool for small tech companies. But I do not think it could add value for teams working on large-scale projects with complex codebases. No doubt it provides good code suggestions, but it only has limited context from a few opened tabs. Therefore, it will not be very useful for large-scale projects.
There are some better alternatives that can understand entire codebases. Bito is a market leader in this field because it indexes your code locally and hence offers more precise suggestions.
Is GitHub Copilot Free for Students?
Yes, GitHub Copilot is available for free to verified students. To be eligible, you need to meet the following requirements:
You must be currently enrolled in an educational institution.
You should have a verifiable school-issued email address or be able to provide documents that demonstrate your current student status.
Is GitHub Copilot Worth it for Students?
GitHub Copilot offers free access to students. It can provide helpful code suggestions and serve as a learning aid, assisting students in understanding programming concepts and syntax. It may help them save time and overcome coding challenges more easily.
Additionally, GitHub Copilot can expose students to different coding styles and best practices, enhancing their learning experience. However, students should also be cautious and ensure they understand the code generated by the tool to avoid blindly relying on it.
Does GitHub Copilot Steal Your Code?
GitHub Copilot does not intentionally steal codes. But it can use your code to train its AI model. If you want to prevent that, simply go to your GitHub account’s settings page, and click “Copilot” from the sidebar. Now, uncheck the box “Allow GitHub to use my code snippets for product improvements” and press the “Save” button.
Can I Use GitHub Copilot at Work?
Yes, you can use GitHub Copilot at work. It helps small teams to be more productive while writing source code for apps. It can be used in various development environments, including professional settings. However, it is important to ensure that your organization’s policies and guidelines allow the use of AI-powered tools like GitHub Copilot.
Is GitHub Copilot Safe to Use at Work?
GitHub Copilot is generally considered safe to use at work. It is developed by reputable organizations and is designed to provide code suggestions to assist developers. However, as with any software, there may be potential security concerns or risks associated with using GitHub Copilot. It is recommended to review the security and privacy measures provided by GitHub and OpenAI, and to consult with your organization’s IT or security department to ensure compliance with any specific requirements or policies.
Is GitHub Copilot Open Source?
GitHub Copilot is not open source. While GitHub is built on open-source technologies and supports numerous open-source projects, GitHub Copilot itself is a proprietary tool developed by GitHub in collaboration with OpenAI. The underlying AI model and code generation capabilities are not openly accessible or modifiable by the user.
Does GitHub Copilot Work Offline?
No, GitHub Copilot does not work offline. It relies on a connection to the internet to function properly. The AI model and code generation processes are performed on remote servers, which require an active internet connection for GitHub Copilot to provide code suggestions in real time.
How to Enable GitHub Copilot in VSCode?
To enable GitHub Copilot in Visual Studio Code (VSCode), follow these steps:
- Install the GitHub Copilot extension from the Visual Studio Code marketplace.
- After installation, sign in to your GitHub account within the VSCode editor if prompted.
- Make sure you have an active subscription to GitHub Copilot. It costs $10 per month for individuals.
- GitHub Copilot should now be enabled, and you can start using it by opening a code file and typing in your desired programming language.
Is GitHub Copilot Better Than ChatGPT?
GitHub Copilot and ChatGPT serve different purposes and have different functionalities. GitHub Copilot is specifically designed to assist developers by providing code suggestions, while ChatGPT is an AI language model developed for generating human-like text responses in conversational contexts.
GitHub Copilot can be a valuable tool for developers, especially when working on coding tasks, as it can help speed up the coding process and provide suggestions based on the context. It focuses on generating code snippets and completing code patterns.
On the other hand, ChatGPT is designed to understand and generate text based on the input it receives. It is more suitable for answering a wide range of questions, engaging in conversations, or generating creative written content.
Ultimately, the choice between GitHub Copilot and ChatGPT depends on the specific needs and use case. If you are primarily looking for code generation assistance, GitHub Copilot would be more suitable. If you need a conversational AI model for general text generation, ChatGPT would be a better option.
Conclusion
In this comprehensive GitHub Copilot review, I have tried my best to answer your question “Is GitHub Copilot Worth It?” in detail.
First, I have shared my personal experience with each feature of GitHub Copilot, discussing whether they are worth using or not. This will provide you with a better understanding of the strengths of GitHub Copilot and areas where it needs improvement.
The value of GitHub Copilot varies depending on individual needs and circumstances. It can enhance productivity for freelance developers, improve code quality for software companies, and facilitate learning for students.
With careful use, Copilot can be a valuable coding aid, but understanding its limitations is crucial. Users must validate suggestions and ensure alignment with standards and requirements. The tool’s ultimate value depends on coding proficiency, project complexity, and budget considerations.
Top comments (3)
Hi, In your analysis, you didn't cover software code review and software testing. I see some references of Code review possible but no reference of system testing.
Can you share your insights on the same.
The free alternative Codeium is also amazing. You should review that too.
Copilot is absolute garbage. It's not helpful, Didn't answer any of my question. You're better off asking google. Tried unstalling it on my PC. Major pain in the a$$.