AI coding assistant | API Costs

I am using the cptX VSCode extension to streamline code generation via ChatGPT and get the results right in the IDE.

Below is a screenshot of Azure accumulated costs over a month and a half (September 1 - October 15). Costs are associated with GPT-4 and GPT-3.5 API services.

The monthly cost can be estimated at around €6.62 or $6.97. Most of the time (~90%), I used the GPT-4x8k model.

Do I use the plugin a lot? I mostly use it for open-source projects. Here's a glimpse at my GitHub activities:

~380 commits over the past 1.5 months. My guess is that should I be coding daily and professionally, this usage wouldn't be much different. I consider AI-generated coding to be limited to smaller and simpler tasks AND unlikely to take up significant portion of produced code.

A bit of Context

The selection of AI coding assistants is not limited to GitHub's Copilot or Amazon's CodeWhisperer. There are many other options. For example, if you type "GPT" in the VSCode marketplace, you will get 250+ options, most of which are AI coding assistants of some sort:

Most of these extensions are wrappers around ChatGPT and require an OpenAI/Azure key.

We can outline two flavors of coding assistants by LLM (Large Language Model) kind:

1. Integrated/bundled model - such as Copilot or CodeWhisperer. In this case, the IDE plugins directly communicate with their back-end and use their own Large Language Model instances.
2. Bring-Your-Own-Model - such as CodeGPT or cptX, which require you to take care of getting a model endpoint (typically OpenAI) and configuring the IDE plugin (e.g. providing an API key).

Why have these BYOM assistants become so popular? There are many reasons; here are a few:

1. Early in 2023, ChatGPT blew minds with its capabilities. It was just better than Copilot. I remember several cases with colleague developers saying that they tried and stopped using Copilot, yet they used ChatGPT to copy and paste code snippets and talk to the AI bot.
2. Traditionally, Copilot was a text-completion service, where providing an instruction and getting the result was not straightforward (one could use workarounds such as writing instructions in comments).
3. Costs. With Copilot, you pay $10 a month. However, with the OpenAI API, you have a free trial quota, Azure provides a $50/month free budget for its services for the owners of MSDN subscriptions... And you can stay well below $10 a month, especially when using a cheaper (yet capable) GPT-3.5x16k model.

CoPilot X has certainly moved to a new paradigm of chatting. Yet to me it looks like a tool that is catching up with all those other instruction following AI assistants that rely on OpenAI APIs.