Amit chakraborty

Posted on Nov 9 Originally published at amitchakraborty.dev

The AI tools panel · 2. Working with an agentic coding tool

#tutorial #aiengineering #theaitoolspanel #choosingtherighttool

The AI tools panel · Chapter 2 of 12 · AI Engineering · new chapter every Sunday afternoon

By the end of this chapter: Establish workflows that hold up on a real codebase.

The problem

Sofia, a senior backend engineer, is tasked with prototyping an internal tool to parse incoming support tickets, suggest a short response, and optionally generate a code snippet to reproduce a reported bug. She has access to several AI code assistants, including GitHub Copilot, Claude, OpenAI GPT-4-Turbo, and Code Llama. However, she is unsure which tool to use for this specific task and how to integrate it into her workflow. The prototype needs to be completed quickly, and the team is counting on her to deliver a working solution.

The symptom Sofia is facing is the uncertainty of choosing the right tool for the task, which is costing her time and potentially delaying the project. She has heard of the capabilities of each tool but needs guidance on how to effectively use them in a real-world scenario. The cost of not delivering a working prototype on time could be significant, potentially impacting the team's productivity and the company's reputation.

Sofia's experience with AI code assistants is limited to experimenting with them on small projects, and she is not confident in her ability to scale their use to a larger project like this one. She needs to establish a workflow that can handle the complexities of the task and deliver a reliable solution.

Before you start

To follow along with this chapter, you should have a basic understanding of the AI code assistants mentioned earlier, specifically GitHub Copilot, Claude, OpenAI GPT-4-Turbo, and Code Llama. You should also have a Python environment set up with the necessary dependencies installed, including the transformers library and the python-dotenv library for handling environment variables. The version of Python used in this chapter is Python 3.10.8.

To verify that your setup is working, run the following command:

python --version

This should output Python 3.10.8. If you are using a different version, you may need to adjust the code accordingly.

Choosing the right tool

When working with AI code assistants, it's essential to choose the right tool for the task at hand. In Sofia's case, she needs a tool that can parse incoming support tickets, suggest a short response, and optionally generate a code snippet to reproduce a reported bug. Based on the capabilities of each tool, GitHub Copilot and OpenAI GPT-4-Turbo are the most suitable options for this task.

GitHub Copilot is a popular choice for code completion and generation tasks, while OpenAI GPT-4-Turbo is a more general-purpose language model that can handle a wide range of tasks, including text generation and code completion. Claude and Code Llama are also capable tools, but they may not be the best fit for this specific task.

Integrating the tool into your workflow

Once you've chosen the right tool, you need to integrate it into your workflow. This involves setting up the tool, configuring it to work with your codebase, and establishing a workflow that can handle the complexities of the task.

For example, you can use the transformers library to integrate GitHub Copilot into your workflow. First, you need to install the library using pip:

pip install transformers

Then, you can use the following code to set up GitHub Copilot:

import os
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model_name = "github/copilot"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Set up the workflow
def generate_code(prompt):
    inputs = tokenizer(prompt, return_tensors="pt")
    outputs = model.generate(**inputs)
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

# Test the workflow
prompt = "Generate a code snippet to reproduce a reported bug"
code = generate_code(prompt)
print(code)

This code sets up GitHub Copilot and defines a workflow that takes a prompt as input and generates a code snippet as output.

Build it

To establish a workflow that holds up on a real codebase, follow these steps:

Choose the right tool: Based on the capabilities of each tool, choose the one that best fits your needs. For Sofia's task, GitHub Copilot and OpenAI GPT-4-Turbo are the most suitable options.
Set up the tool: Install the necessary dependencies and set up the tool according to the documentation. For GitHub Copilot, you can use the transformers library to integrate it into your workflow.
Configure the workflow: Configure the workflow to work with your codebase. This involves defining a function that takes a prompt as input and generates a code snippet as output.
Test the workflow: Test the workflow with a sample prompt to ensure it's working as expected.

When this breaks

When working with AI code assistants, things can break in various ways. Here are some common issues you may encounter:

Model not responding: If the model is not responding, check that you've set up the tool correctly and that the model is properly configured.
Generated code not working: If the generated code is not working as expected, check that the prompt is clear and concise and that the model is trained on a relevant dataset.
Model producing incorrect results: If the model is producing incorrect results, check that the training data is accurate and that the model is not biased.

To fix these issues, you can try the following:

Check the documentation: Check the documentation for the tool to ensure you've set it up correctly.
Test the model: Test the model with a sample prompt to ensure it's working as expected.
Fine-tune the model: Fine-tune the model on a relevant dataset to improve its performance.

What it costs

Using AI code assistants can have various costs, including:

Time: Setting up and configuring the tool can take time, especially if you're new to AI code assistants.
Resources: Running the model can require significant computational resources, especially if you're working with large datasets.
Accuracy: The model may not always produce accurate results, which can impact the quality of the generated code.

To mitigate these costs, you can try the following:

Start small: Start with a small project to get familiar with the tool and its capabilities.
Use pre-trained models: Use pre-trained models to reduce the time and resources required to train the model.
Test thoroughly: Test the generated code thoroughly to ensure it's accurate and working as expected.

In the interview

When discussing AI code assistants in an interview, you may be asked questions like:

What are the benefits and drawbacks of using AI code assistants?
How do you choose the right tool for a given task?
How do you integrate AI code assistants into your workflow?

A weak answer might focus on the benefits of AI code assistants without discussing the drawbacks or the challenges of integrating them into a workflow. A strong answer would discuss the trade-offs of using AI code assistants, including the potential costs and benefits, and provide examples of how to choose the right tool and integrate it into a workflow.

For example, a strong answer might say:

"I've used AI code assistants like GitHub Copilot and OpenAI GPT-4-Turbo on several projects. While they can be incredibly powerful tools, they also have their drawbacks. For example, they can be computationally expensive and may not always produce accurate results. To choose the right tool, I consider the specific requirements of the project and the capabilities of each tool. I also make sure to test the generated code thoroughly to ensure it's accurate and working as expected. In terms of integration, I've found that using pre-trained models and fine-tuning them on relevant datasets can be an effective way to improve the performance of the model."

Your tasks

To practice working with AI code assistants, try the following tasks:

Set up GitHub Copilot: Set up GitHub Copilot and use it to generate a code snippet for a simple task, such as a "Hello World" program.
Integrate OpenAI GPT-4-Turbo into your workflow: Integrate OpenAI GPT-4-Turbo into your workflow and use it to generate a code snippet for a more complex task, such as a web scraper.
Fine-tune a pre-trained model: Fine-tune a pre-trained model on a relevant dataset and use it to generate a code snippet for a specific task.

Done when you've completed each task and can demonstrate the generated code working as expected.

Your tasks this week

Do the exercises above before the next chapter. Reading a tutorial and doing
one are different activities and only one of them changes what you can build.

Stuck on any of them? Say so — describe what you tried and what happened:
tell me where you got stuck. I read every one, and the questions
that come back more than twice get answered in the next chapter.