I've been using Copilot and Amazon Q daily for the past few months, both having the Claude Sonnet 4 model configured.
I know there are many pros and cons against writing quality code with these tools, so I had to convince myself: should I take the blue pill or the red one.
The projects I've been working on are using CloudFormation and Terraform for IaC and the deployment is done using GitLab pipelines.
In Amazon Q, I've also configured a few MCP servers like awslabs.cfn-mcp-server, awslabs.aws-documentation-mcp-server and terraform. In Copilot I wasn't able to configure any, due to company restrictions.
Now the facts:
1. Creating new repositories from scratch
I decided to use CloudFormation to create a new repo from scratch with Amazon Q and the generated templates needed tweaks in about 5-10% of the code. This was impressive.
On the other hand, when it came to deploying the stacks using GitLab, Amazon Q failed to suggest good validation syntax for things like "pipeline inputs". Every time the build failed with issues that weren't easy to fix, I turned to Copilot to get quick and valid fixes, while Amazon Q was a bit behind.
Each time I asked Copilot to scan the repo and provide improvement suggestions, it gave me a reply that I have a solid and mature solution, nothing major can be improved. On the same code, with the same request, Amazon Q delivered a solid number of suggestions which all made sense and in fact I adopted them.
Important: I initially "trusted" Q to use the agent mode and took things too far by committing and pushing new code to the main branch, without asking me first, while working on a feature branch. So that was the moment I decided that I need to manually approve all file changes...
In the end, I was satisfied with finishing the IaC way sooner than starting from scratch in a manual way.
2. Creating new repositories using as inspiration other working repositories
Simply said, this was a bad decision.
This time I used Copilot to start a new repo using Terraform, while instructing it to get inspiration from another Terraform repo which is deployed in production and has 90% the same resources. For the rest of the 10% resources, I instructed Copilot to check the repo I mentioned above, written in CloudFormation and write the equivalent Terraform code.
The result was a total mess-up. The code I instructed to be used as inspiration from the production repo was ignored, Copilot decided to write from scratch new custom Terraform modules, when the production repo was using the official Terraform ones.
I ended up cleaning and rewriting most of the custom modules, turning them into official ones, all with a lot of headaches and frustration.
I tried using Q to fix the repos Copilot created but there was just too much mess and the suggestions were just adding complexity, when I just needed simplicity by using an existing working repo.
Bottom line is that I found more helpful to start from scratch with a new repo, rather than trying to use existing ones and adapting them to my needs. This is exactly the opposite of what I was expecting so I'm curious how the newly Claude Sonnet 4.5 behaves, I'll test them both soon and update this blogpost with my findings.
If you need to choose from the 2, I would go with Q for IaC templates and code reviews but keep Copilot for GitLab pipeline syntax if needed.
Top comments (0)