DEV Community: Joe Giglio, Chief Remote Officer

Part 4 - Using Cursor and Claude to Create Automated Tests with Playwright

Joe Giglio, Chief Remote Officer — Tue, 05 Nov 2024 01:53:16 +0000

PART 4

Here is the Github source code repo for this project: https://github.com/joegiglio/cursor2-kb

This article focuses on using Cursor to create Playwright tests. If you are interesting in using Windsurf to create Playwright tests check out these videos: https://youtu.be/omJo9bDTIaY.

===
It’s been a while…

Last time we spoke, Cursor seemed to be breaking down and the index of the file structure had become corrupted. After waiting 23 days for a fix, I am happy to report that the problem does finally seem to be addressed! The UI is still quirky and it seems to slow down with some type of memory leak after a while which can be fixed with a restart. But at least now it is making changes to the correct files and I am not faced with a mysteriously corrupted file index.

While waiting for a fix, I noticed that many Cursor users were complaining about the instability of Cursor and/or Claude. Yes, it works sometimes and it is impressive when it does but it is a black box of moving parts, constant updates behind the scenes and inconsistent results.

I decided to put the Help Desk project to the side and wanted to see if I can build a Knowledge Base application instead. This might be a little simpler - no need for real time updates or complex integration points.

I used the same technology stack that I am most familiar with - Python, Flask and Bootstrap with a SQLite database. Everything was coming along swimmingly until I hit a snag with “database migrations”. Similar to the previous chapter, it generated constraint code that SQLite did not like and it threw an error:

Exception during DB migration:
 raise ValueError("Constraint must have a name")
ValueError: Constraint must have a name

That should have been easy enough for Claude to fix but it started spiraling and I ended up rebuilding my database from scratch. I will blame this on my inexperience with debugging flask-migrate but Claude was not able to clean up the mess without just wiping the database and starting over.

After things settled down, I ended up with a mostly functional knowledge base system, along with an Admin panel, dark / light mode, search (just using the SQL queries) and test data generator. It was built with Quill which is an open source text editor with support for formatting and images.

I say “mostly functional” because I purposely did not complicate it with authentication or mutli-tenancy. I kept the architecture simple with basic Flask routes and CRUD operations. No API yet… Perhaps I will revisit these features in phase 2.

As I was adding features, I kept noticing that Claude would fix one thing but break another - just like a real developer. Also, there was CSS and Javascript scattered all over the code. Sometimes, adding new articles would work but editing would not work and vice versa. Sometimes article formatting code would appear here and there and I had to ask Claude to clean it up. It was becoming whack-a-mole.

The project was getting complex enough that it needed test automation which was briefly discussed in the last chapter. In most companies, there is very little budget for QE and most Developers hate writing tests, so maybe AI can help?

I have a respectable amount of experience with Selenium and Cucumber but it has been a while. Playwright has been gaining traction and supposedly has better handling for locators and retries so I asked Claude to create some Playwright tests. The VS-Code Playwright extensions all seem to assume you are using Javascript and they do not have direct support for Python so I will need to run the tests from the Terminal, like an animal.

[PROMPT]

I would like to add end to end browser tests to this project using Playwright and Python. I would like the tests to use fixtures to seed the necessary data and then delete the data at the end of each test run. The tests need to clean up after themselves and not leave test data behind. If you can configure it to setup a test database while running the tests that is even better. Please let me know which libraries to install, create tests for the top 10 scenarios by looking at the main features of the product and let me know how to run these tests from within Cursor.

[/PROMPT]

The first attempt was pretty good, providing instructions on how to install the necessary libraries. The file structure for the tests also seemed accurate and it even added a test for responsiveness by changing the viewport size. Nice touch… But I did not like the initial set of tests it generated so I asked it for further refinement:

[PROMPT]

Authentication has not been implemented in this product yet. Please do not create tests for those features. Instead, focus on Admin functionality like creating, editing and deleting topics; adding, editing and deleting articles. Also, focus on end user functionality such as searching and viewing articles. Look at the code to figure out the top 10 features of the product and create tests for them.

[/PROMPT]

The next batch of tests looked a little better but still no Admin functionality.

[PROMPT]

Most of those tests look pretty good but I am not seeing anything for Admin functionality in the @admin.html file. We need to have tests for adding, editing, deleting topics and adding, editing and deleting articles.

[/PROMPT]

Now it looks like it got it right and created code to test the following app scenarios:

These look like great starting points. Now let’s see if the Claude's execution instructions are correct. Well, unfortunately not…

[PROMPT]

I followed your instructions for running the tests and I am running into a couple of problems.
I opened a new terminal and ran the commands “FLASK_ENV=testing flask run” followed by “pytest tests/e2e/”. I see the following problems:

When I navigate to the home page, I am seeing the same data I was seeing before. It did not seem to switch to the test database.

An exception: ImportError while loading conftest '/Users/joe/cursor2-kb/tests/e2e/conftest.py'. tests/e2e/conftest.py:12: in from models import Topic, Article ModuleNotFoundError: No module named 'models'

[/PROMPT]

Claude created a models.py file, duplicating the code in the app.py file and I questioned why…

The initial code for this application was all done in the app.py file, including the models. This works fine but can leave you with an unwieldy single file application. Best practice says to separate the models and routes from the config and this is where Claude was taking me, even though it took a long winded route:

[PROMPT]

The models.py file you are proposing looks like it is duplicating code from app.py. Would it make more sense to use the models in app.py instead?

[/PROMPT]

[RESPONSE]

Actually, it's the other way around - we should move the models out of app.py into models.py to follow the separation of concerns principle. This is a common pattern in Flask applications where:
app.py handles application configuration and initialization
models.py contains all database models
routes are often in separate blueprint files

[/RESPONSE]

Fair enough. Let’s go with it!

Well, not so fast. Upon closer inspection…

[PROMPT]

I like where you are going with separating the models.py file but you also deleted all the routes from the app.py file. I do not see them in any other file. They are just gone…

[/PROMPT]

Now Claude pulled all the routes out of the app.py file and moved them into a separate file for blueprints. It is making way too many architectural changes and I want it to revert… Beware the brilliant new developer that wants to rewrite everything! Although the suggested changes are considered best practices for a professional Flask codebase, I will not trust those changes until the tests are in place so we can verify they are working properly.

[PROMPT]

I understand that using blueprints is the preferred method for professional level Flask code. However, I am not comfortable making such refactors right now. I am first trying to get end to end tests in place. Then we can look at the project structure. Please do not move the routes out of the app.py file.

[/PROMPT]

Cursor attempted to revert but the merge UI was a sea of green and red and I was not comfortable with the change. New strategy: Let’s get some simple Playwright tests in place and then work on incrementally improving them. Maybe this will be better…

I stashed the proposed test list since I liked where that was going and then closed out of the Cursor Composer window and started a new Composer which I believe removes all content of the previous Composer.

Let’s try again with a simpler starting point. Iterative development is usually best anyway.

[PROMPT]

I would like to add end to end browser tests to this project using Playwright and Python. Please let me know which libraries to install, create tests for the top 10 scenarios by looking at the main features of the product and let me know how to run these tests from within Cursor.

[/PROMPT]

We had similar problems this time with Claude wanting to test functionality that does not yet exist. Time to scold Claude!

[PROMPT]

I see you are creating tests for authentication functionality. Although we have a login page, it is only a placeholder and does not actually function. It does not even have fields to allow anyone to login. Please review the project code again and come up with the top 10 test scenarios for Playwright.

[/PROMPT]

This caused Claude to apologize and create a bunch of test scenarios all around the Search functionality on the homepage. I was hoping not to hand feed the tests I wanted, but instead wanted it to find the features by looking at the code. No luck though, so I went back with:

[PROMPT]

This looks better but is focused too much on search. We need tests for the following features: search, create topic, edit topic, delete topics, create article, edit article, delete article, view article and some responsive tests to make sure everything looks OK on different screen resolutions.

[/PROMPT]

We are starting to spiral already… It created the basic framework for some more tests but it is making too many assumptions about data that will exist. Make no assumptions. This is why it is best to have a test database that is pre-seeded but we will try to get there eventually. For now…

[PROMPT]

Some of these tests make assumptions that certain links will be valid and certain content will exist. This may not be the case if articles or topics are deleted. Please make each test self contained by creating the data it needs and then deleting it when the test completes.

[/PROMPT]

So now Claude is right back to creating fixtures anyway! I guess there is no way around it if you want to do it right…

Another problem is using test strings like “Test Subject” and “Test Article” which may clash if we run parallel tests which is a common approach to speed up execution. I wanted the tests to use randomized data so I prompted with this:

[PROMPT]

Please use random data for the topic name, article name and article content. It would also be a good idea to add a test for foreign language characters to make sure they appear OK during browsing and searching.

[/PROMPT]

I have a feeling we are going to need a lot more tweaks but this is looking like a very reasonable start. Claude returned the following:

These changes were made:
Use Faker to generate random data in multiple languages
Add specific tests for multilingual content
Verify proper display of various character sets
Test search functionality with non-Latin characters
Verify no character encoding issues in display
Test content creation in multiple languages
The tests now verify that your application correctly handles:
Japanese (日本語)
Chinese (中文)
Korean (한국어)
Russian (Русский)
And regular English content
Each test creates its own random data and cleans up after itself, making the tests more reliable and independent.

These are the tests that were created:

Now to run them. First, I need to install the faker library which Claude instructed me to do…

I was given a bunch of different flags and options for running the tests. From previous experience, all of these instructions look right but I am going to use the pytest.ini file Claude created and pass in a few flags for verbose output, screenshots and a slowed down test execution so I can follow along visually…

And now to run pytest tests/e2e/ to actually run the tests!

All of the tests failed with this same error:

playwright._impl._errors.Error: Protocol error (Page.navigate): Cannot navigate to invalid URL

In pytest.ini, the base_url is set to http://localhost:5000 which looks right to me.

I couldn’t figure out why it was failing so I asked Claude. He made a few small tweaks and then suggested I move the pytest.ini file to the ROOT directory of the project and out of the tests/e2e directory where he originally placed it. I pushed back on this and he said it was fine where it is. OK then…

He also suggested I use absolute URLs in the tests instead of the URL set in the INI file and text fixtures. Wrong. I also pushed back on that and he agreed and fixed it. Losing faith here…

With these changes, I ran the tests again and now I see the browser open up and go to the base URL which is the homepage of the knowledge base but all tests are still failing. I went a few more rounds with Claude and it looks like the root page issues may be fixed but now we are failing with:

FAILED tests/e2e/test_admin.py::test_create_topic[chromium] - playwright._impl._errors.TimeoutError: Timeout 30000ms exceeded.

Which looks like more of a locator issue.

[PROMPT]

Now all the admin tests in test_admin.py are failing with this error: admin_page.fill("#topic-title", data["title"]). That is not the correct locator. Please fix.

[/PROMPT]

Claude fixed one test but the others are still failing with locator errors. I got more explicit:

[PROMPT]

Other than test_create_topic, the rest of the tests in @test_admin.py fail with locator errors. Please review all the locators in the tests and correct them.

[/PROMPT]

Claude made a few changes and now we have another locator issue…

[PROMPT]

Now all tests are failing with:
tests/e2e/test_admin.py::test_create_topic[chromium] - AssertionError: Locator expected to be visible

[/PROMPT]

It is obvious we will be here for a while, going round and round while Claude makes the necessary fixes. What is also obvious and unfortunate is that Claude does not have the proper context of the application and page elements without me having to be very explicit. Just asking Claude to “look at the code and create tests” is not good enough. You will need to be very familiar with feature functionality and page structure so you can nurse it along.

Claude was helpful in setting up the architecture of tests with pytest.ini and conftest.py but even so, it needed several tweaks for a basic test to run without an immediate failure.

This project uses Bootstrap on the front end along with Javascript confirmation and modal dialogs. Claude seems to have a very difficult time with these if there are multiple modals in the source code, where only one is visible, which is a common design pattern. After several iterations and guiding it with pseudo-code, we eventually came to an understanding and a passing test.

Throughout these tests, Claude made many assumptions about the names of locators, most of which did not exist anywhere in the source code. Only after explicitly telling it to look in certain files was it able to clean up but why make assumptions? Perhaps I did not feed it the correct files to begin with? I am still not sure how all of that is supposed to work.

Sometimes it seems to have access to the entire project while other times I need to be much more explicit. If it needs access to a file it should ask and not just create bullshit code. Remember, each back and forth costs you API credits.

Besides lots of test failures, I am also having issues with screenshots not being saved even though I am passing the flag in the pytest.ini file. Claude offered me a few permutations but none of them fixed the problem. I looked at the Playwright documentation and still no resolution. This seems like another one of those issues that “should be easy to fix” but you can easily burn a dozen Claude requests and get nowhere quickly. I will move on for now, using a workaround that Claude suggested.

After spending WAY TOO much time making tweaks with Claude, I now have only two tests that are passing and a third that starts out as PASS but ultimately ends up as FAIL due to some conflict between the test code and fixture code. It is becoming exhausting to debug and I feel like I am having circular conversations. I have gotten to a point where I just blindly merging suggested code changes and hoping for a miracle.

Earlier in this chapter, my intention was to build a test harness that seeded and removed test data along with reusable fixtures and functions to keep the data and test code clean. Those intentions are long out the window and now I just want the tests to pass and I will try to clean them up later. Let me know if any of that sounds familiar…

Even something seemingly as simple as maximizing the browser window when launched is proving to be way too difficult. Claude has made attempts with Python and Javascript and it just doesn’t work. He doesn’t know why and the geniuses on Stack Overflow have answered that question several times and their suggestions do not work either. My guess is there is some code conflict among the files but I don’t know the code well enough to figure it out right now. Where is AI when you need it!?

I found that some of the generated test code had more than one expect/assert. I believe it to be best practice to have only ONE assert per test but know that is not always a realistic case especially when the tests are new and you don’t trust them yet.

I found myself being a “prompt engineer” trying to figure out how to nudge things in the right direction. Since I did not write any of the product code myself I was not familiar with its inner workings and was hoping Claude could pick it up and run with it. After all, he wrote it but he seems to have a short memory. He gives you enough to get going but certainly not enough to ask once and collect the money.

What would make life easier is to have APIs to call for CRUD functions that create and delete data, leaving the end to end tests to only test their intended functionality. The shortcomings and missing features of this test project may sound all too familiar if you have ever written automated tests for a software application. You often end up having to work around product shortcomings before you actually get around to testing product functionality. You may recall way back in the first chapter that I intended to build the product API-First and that quickly spiraled out of control. Absent of APIs, perhaps direct SQL calls would have made more sense for setup and teardown.

If you have automated tests working in another project and are familiar with the code and page structure of a new project, copy/paste/tweak will likely get you further faster than the Claude Playwright approach outlined here. At least then you will have some familiarity with the code and can tweak if with some confidence. As of now (November, 2024), I find asking Claude to generate end to end tests with Playwright to be a painful exercise.

Claude20241022 Supplement

While writing this article, Anthropic released a new version of Claude Sonnet 3.5, dated 10/22/2024, which also became available in Cursor. They claim there are some important advancements in coding so I figured I would re-do my testing to see if there were any improvements.

After clearing up some virtual environment and dependency issues caused by changes Claude made to the requirements.txt file, I was on my way. I wiped out my database, created a new branch and went with the same prompt we started with earlier in the chapter:

[PROMPT]

I would like to add end to end browser tests to this project using Playwright and Python. I would like the tests to use fixtures to seed the necessary data and then delete the data at the end of each test run. The tests need to clean up after themselves and not leave test data behind. If you can configure it to setup a test database while running the tests that is even better. Please let me know which libraries to install, create tests for the top 10 scenarios by looking at the main features of the product and let me know how to run these tests from within Cursor.

[/PROMPT]

This resulted in Claude generating a tests/e2e directory and all the expected files but when I attempted to run the tests, they all failed. If I try to go to the website, all pages returned a 500. I noticed conftest.py has code that is supposed to switch the application to use a test database. This is great but it looks like it never initialized. The 'sqlite:///test.db' does not exist even though I see it referenced in the code.

If I restart the server, this also throws exceptions. I noticed all the tables and all my data has been removed from the original topics.db file. All my data and schema has been wiped without warning! Thankfully, this was just test data.

After complaining to Claude, he made some changes and created a Python file to initialize the database which I ran.

Now when I run the tests, they all throw an exception about missing constraints.

When I hit site manually, I get a database exception:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: topic
[SQL: SELECT topic.id AS topic_id, topic.name AS topic_name, topic.sort_order AS topic_sort_order, topic.description AS topic_description, article_1.id AS article_1_id, article_1.title AS article_1_title, article_1.content AS article_1_content, article_1.keywords AS article_1_keywords, article_1.topic_id AS article_1_topic_id, article_1.sort_order AS article_1_sort_order 
FROM topic LEFT OUTER JOIN article AS article_1 ON topic.id = article_1.topic_id ORDER BY topic.sort_order]

The act of running the tests has wiped my database again without warning! Something ain’t right around here….

[PROMPT]

I am still having some serious database issues: If I use the @init_db.py script, it restores my database tables and the app functions fine. However if I run the tests with pytest tests/e2e/ each test fails with: ERROR tests/e2e/test_app.py::test_reorder_topics[chromium] - sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) NOT NULL constraint failed: article.topic_id. Then when I try to use my main DB again, all the tables are removed again! It does not seem to properly setup the test database which also breaks my other database somehow!

[/PROMPT]

Claude made some changes. Next prompt:

[PROMPT]

We are still having issues. Now all tests fail with ERROR tests/e2e/test_app.py::test_create_article[chromium] - sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: topic andit is still siping out my other database!

[/PROMPT]

Claude made a bunch of changes and when I try to run the database initialization script again, it fails with:

(.venv) joe@Joes-MacBook-Pro cursor2-kb % python init_db.py Traceback (most recent call last): File "/Users/joe/cursor2-kb/init_db.py", line 1, in <module> from app import app, db File "/Users/joe/cursor2-kb/app.py", line 40, in <module> app = create_app() ^^^^^^^^^^^^ File "/Users/joe/cursor2-kb/app.py", line 34, in create_app db.init_app(app) ^^ NameError: name 'db' is not defined

I also can not start the server since it generates the same error.

TIME FOR A RESET!

Let’s simplify our initial prompt, again re-using one from earlier in this chapter. I restored the database manually, created a new branch, fixed the virtual environment and went with:

[PROMPT]

I would like to add end to end browser tests to this project using Playwright and Python. Please let me know which libraries to install, create tests for the top 10 scenarios by looking at the main features of the product and let me know how to run these tests from within Cursor.

[/PROMPT]

The initial list of tests look pretty good:

Two passed without any intervention, another positive sign:

There is an odd quirk though where passing the –-headed flag does not allow me to view the tests as they run. Claude was able to find a conflict between the pytest.ini and one of the conftest.py fixtures:

Claude instructed me to change the code here if I wanted to not run headless. I pushed back:

[PROMPT]

This change is working but I do not want to pass a headless directive here. I want it to be handled in the @pytest.ini file in the addopts section.
[/PROMPT]

Claude then recommended I make changes in two files every time I want to run headless. Why?

[PROMPT]

This is not correct. I want to control the head/headless flag in the @pytest.ini file. Your solution will force me to change this setting in two places. Please fix.

[/PROMPT]

We then ended up with this snippet of code which seemed odd to me. Why do we care about the –help flag?

This resulted in a new error which Claude was able to fix when prompted:

TypeError: argument of type 'bool' is not iterable tests/conftest.py:45: TypeError

[PROMPT]

Now the tests are running but I am still not seeing a visible browser when they run.

[/PROMPT]

The new Sonnet version seems to be stumbling like the previous version. I am sure I can keep plugging away at it and eventually it will tighten up but I am not seeing any dramatic differences for our purposes here.

I am frustrated with this experiment and will end it here. Next chapter, let’s see if Claude does any better with Selenium which has been around for much longer than Playwright. Perhaps there are more examples out there that Claude was able to consume. Playwright may be preferred these days but I find the Playwright / Claude pairing to be exhausting.

MARCH 2025 UPDATE:

I tried a similar experiment using Windsurf with Claude 3.5. Was it any better?

The only way to know for sure is to watch my video series, "The Truth about Vibe Coding with Windsurf… Better than Cursor?”

Part 1 | Part 2 | Part 3

ABOUT THE AUTHOR

Joe Giglio is a 25+ year veteran of the software industry. In his career, he has worked for a local dial-up ISP, startups of all sizes and a well established, slow moving enterprise. There were important lessons learned along every stop.

His recent projects include:

Remote work training and consulting: chiefremoteofficer.com

Kindle book "Making Remote Work, Work For You"

Remote company rankings. Think "Glassdoor for remote companies": remotescorecard.com

Joe’s guiding principles are: Remote first, quality driven, customer champion and lifelong learning. He was an introvert and practiced social distancing long before it was cool!

He currently resides in North Carolina and is always interested in new opportunities, meeting quality people and learning about exciting projects.

Contact Joe Giglio: joe -AT- joegiglio.org | Twitter | LinkedIn

Part 3 - Using Claude, Claude-Dev and Aider to Build a Ticketing System (Cursor too!)

Joe Giglio, Chief Remote Officer — Tue, 27 Aug 2024 18:00:40 +0000

Part 1 | Part 2 | Part 4

The project has been open sourced if you would like to play along at home: https://github.com/joegiglio/helpdesk-claude-dev-ai

PART 3 -

I would now like to add a couple of basic features to the Knowledge Base. Right now, we only have a skeleton UI. Let’s see if we can have Claude do something useful with it.

[PROMPT]
I would now like to add a couple of basic features to the Knowledge Base. Right now, the knowledge-base URL shows a placeholder UI but let’s start to make it do something useful. Let’s start with the following:
Add a link in the lefthand navigation bar for “Knowledge Base Settings”. This should link to a new template that contains these settings.
Inside the Knowledge Base Settings page, add a placeholder image and placeholder text.
Inside the Knowledge Base Settings page, add a link for “Topics”, which will be created by the end user. These topics will contain articles in a future step. Make the necessary database updates to support these changes.
[/PROMPT]

Claude made the necessary changes but had issues with missing libraries. It once again lost track of the virtual environment, .venv, which I needed to remind it of.

[PROMPT]
Please make sure you are using the virtual environment, directory named .venv
[/PROMPT]

We ended up with this UI which was missing an image, had a couple of different fonts and the Manage Topics button was not functional. I asked Claude to clean it up.

[PROMPT]
We need to make some changes to the UI of the knowledge base settings page.

The placeholder image is missing.
There are a couple of different fonts. Please pick one and stay with it.
The Manage Topics button does not do anything. It looks like there is no framework for it so let’s make it functional.
Update the database by creating a table to hold the Topics. It only needs one field for now, named Topic.
Create a page so a user can create and delete new topics. Confirm deletions. [/PROMPT]

Claude tried to use a local placeholder image again but I reminded it to use the via.placeholder.com service as discussed in part 2.

Watching the output, I noticed a DB migration error which Claude was able to fix:

Exception during DB migration:
raise ValueError("Constraint must have a name")
ValueError: Constraint must have a name

After some clean up, this was the UI, which I did not care for:

[PROMPT]
Please update the UI for the knowledge-base/settings page using the integrations/slack page as a guide. Improve the fonts and replace the placeholder image with a local image that represents a Knowledge Base - perhaps a book?
[/PROMPT]

API limits are tightening up!

429 {"type":"error","error":{"type":"rate_limit_error","message":"Number of request tokens has exceeded your per-minute rate limit (https://docs.anthropic.com/en/api/rate-limits); see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}

Watching the requests go by, I saw Claude using a cURL command to download an icon to represent the Knowledge Base. However, it was not loading. When asked, Claude says an SVG file was downloaded but saved as a PNG. That’s a new one. One more iteration and it was fixed, leaving me with:

I am not sure if I love the layout with the big Manage Topics button but it will do for now. Moving on to managing topics:

Not bad, but if I try to add a topic name that already exists, I hit an exception:

sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: knowledge_base_topic.name [SQL: INSERT INTO knowledge_base_topic (name) VALUES (?)] [parameters: ('topic 1',)]

This is the same constraint it had a problem creating earlier. Let’s see if Claude can fix it.

[PROMPT]
If I try to create a topic name that already exists, I get an exception. Please fix? sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: knowledge_base_topic.name
[SQL: INSERT INTO knowledge_base_topic (name) VALUES (?)]
[parameters: ('topic 1',)]
[/PROMPT]

…and another 429 error:

Claude must be getting hit very hard! Almost unusable tonight…

[PROMPT]
I am now blocked from creating multiple topics with the same name but there are a couple of problems: 1. Missing styling for the “error” flash message. The background color should probably be red. 2. The topic names should be case insensitive. If “topic” already exists, “I should not also be allowed to create “TOPIC”.
[/PROMPT]

Another 429 error. This is grueling…

I was able to sneak in one more request until I supposedly hit my DAILY limit:

429 {"type":"error","error":{"type":"rate_limit_error","message":"Number of request tokens has exceeded your daily rate limit (https://docs.anthropic.com/en/api/rate-limits); see the response headers for current usage. Please reduce the prompt length or the maximum tokens requested, or try again later. You may also contact sales at https://www.anthropic.com/contact-sales to discuss your options for a rate limit increase."}}

I did not do much with Claude tonight but quickly hit my token limit. I am not really sure how that is calculated but the API seems very restrictive tonight. All I can do is smile and wait til tomorrow.

Next day:

[PROMPT]
Let’s continue to improve the Knowledge Base topics UI by making the following changes: 1. The topic names should be case insensitive. If “topic” already exists, “I should not also be allowed to create “TOPIC”. 2. Put a limit of 10 on how many topics can be created and display this count in the UI. Disable the Create Topic button if there are 10 topics and display an appropriate message.
[/PROMPT]

Claude came up with some fancy class function code to check for number of topics and case insensitivity. Pretty elegant:

I noticed that a maxlength was not being enforced for the topic name, even though I tell Claude to do so in my system prompt which is supposed to apply to every prompt I send. Disappointing…

Even though the field in the database is configured to accept 100 characters, I was able to submit a topic name with thousands and nothing blew up. I am not sure how that all works but from where I come from, a lack of enforced limits leads to trouble.

[PROMPT]
It does not look like you are enforcing any length limit on a topic name. Is that correct?
[/PROMPT]

You're right, and I apologize for overlooking this important detail. Let's add a length limit to the topic names. We'll set a maximum length of 50 characters for topic names. I'll make the necessary changes to enforce this limit both in the backend and frontend.

Claude added some code but it was plain Javascript and I wanted jQuery which is already part of the project. I also have jQuery in my system prompt but it seems to ignore that too. Claude changed it to jQuery and even added a character counter (nice touch!). But… it doesn’t work!

[PROMPT]
I see you added a character counter but it doesn’t work. I do not see any errors in the web console. Please fix.
[/PROMPT]

Claude made a few changes to the order of script tags and now it works. Again, similar issues to what we had before.

[PROMPT]
Let’s make a couple more UI tweaks on the manage-topics page. 1. Sort the topics in alphabetical order, case insensitive. 2. Move the Topic count next to the “Existing Topics” header, removing it from below the Create Topic button.
[/PROMPT]

In the middle of this iteration, I hit my daily API limit AGAIN. Whatever Anthropic recently changed with “prompt caching” has done nothing to help me. It seems to have gotten worse.

And, of course:

400 {"type":"error","error":{"type":"invalid_request_error","message":"Your credit balance is too low to access the Claude API. Please go to Plans & Billing to upgrade or purchase credits."}}

This is the current UI. Not bad!

To the next day… Next, I want to create the code to add articles to the knowledge base.

[PROMPT]
Now I would like to create another feature where we can add articles to the knowledge base. Some requirements:
Each article must be in a topic.
If a topic has articles, the articles should also be deleted when its parent topic is deleted. The deletion confirmation should warn the user of how many articles will also be deleted. The user needs to type the word “delete” (case insensitive) to confirm the topic deletion, if it has articles in it.
Click a topic name to view the articles within it.

In the next prompt, we will build out the UI for the article creation page. For now, put together a placeholder page with placeholder images and text.

Please make the necessary changes to the files and database structure to support this feature.
[/PROMPT]

This is a pretty meaty prompt. Let’s see how Claude does…

There were a couple of issues with the database migration which seemed to be caused by Claude again not detecting the .venv
environment. I am going to add that to the System Prompt setting in Claude-Dev. Hopefully it helps.

After those issues were straightened out, I found the results to be pretty good. Of course it needs a few tweaks after some quick testing. I will pick these off in the next prompt:

[PROMPT]
We need a few tweaks to the article creation implementation. Please make these changes:
When deleting topics, we should always show a confirmation. However, only if a topic contains articles should we ask for a confirmation that requires the user to type the word “delete”, case insensitive.
Add a mechanism to delete articles and confirm these deletions.
On the main knowledge-base page, hide the search bar and placeholder messages if there are no topics. Add placeholder text that says there are no topics yet.

Add a more advanced article editor so users can format their articles during creation and editing. Protect against potentially dangerous characters by encoding and decoding article text upon saving and retrieval.
[/PROMPT]

I was curious to see which article editor Claude would select. It went with TinyMCE. However it did not appear.

[PROMPT]
Some more tweaks are needed:
The manage-topics page should show how many articles are in each topic.
“View All Articles” button should not appear if there are no articles in a topic. Instead, show text like “There are no articles in this topic”.
You attempted to add the TinyMCE editor for adding and editing articles but it does not appear. I see no errors in the console.
[/PROMPT]

TinyMCE is still not working. This feels like another websocket and JIRA ticket issue where Claude can’t fix the issue and I run out of requests. I hope not. More tweaks needed!

[PROMPT]
Some more tweaks are needed:
If there are no articles in a topic, the manage-topics shows both an article counter and the text “There are no articles in this topic”. There is no need to show both so remove the text “There are no articles in this topic”.

On the “knowledge-base” page, we are still displaying the “View All Articles” button if there are no articles in the topic. Instead, show text like “There are no articles in this topic”.
The TinyMCE editor still does not appear for adding or editing articles. I see no errors in the console.
[/PROMPT]

The TinyMCE UI is loading now but it is asking for an API key. I will ask Claude if there is a simpler, free editor available instead.

[PROMPT]
The TinyMCE editor is appearing now but it is asking for an API key. Is there another editor we can use that does not require a key? Basic formatting options should be fine for now. If you can find one, please replace TinyMCE with it.
[/PROMPT]

Claude replaced TinyMCE with CKEditor and now it loads! However, it is not without its issues. I am not able to upload an image or save the article. Errors in the dev console:

[PROMPT]
I am able to view CKEditor but I am having a couple of issues:
I can not save an article. Dev console shows this error: new:1 An invalid form control with name='content' is not focusable.
I can not upload an image. Dev console shows this error: ckeditorerror.ts:140 filerepository-no-upload-adapter
Please fix?
[/PROMPT]

Claude says:

I apologize for the issues you're experiencing. Let's address both problems:
The error "An invalid form control with name='content' is not focusable" is likely due to the CKEditor replacing the textarea with its own element, causing issues with form validation. We'll need to update our form submission process to handle this.
The "filerepository-no-upload-adapter" error occurs because we haven't configured an upload adapter for CKEditor. We'll need to implement a simple upload adapter to handle image uploads.

The first fix looked easy but Claude was adding some very complex Javascript code to accommodate file uploads. I pushed back:

[PROMPT]
That looks like some very complicated code for the upload adapter. Are you sure there is not a simpler solution, perhaps something built into CKEditor?
[/PROMPT]

Claude:

I apologize for the complexity. You're right, there is a simpler solution using CKEditor's built-in features. Let's simplify our approach by using CKEditor's Basic package, which doesn't include image uploads but provides basic formatting options. This should resolve both issues without requiring a complex upload adapter.

I want file upload capabilities but can live without it during this exercise. Unfortunately, I hit my daily API limit so that’s it for today’s activities. Such a momentum killer…

However, I did preview some of Claude’s fixes. I am able to save an article now but the preview is showing HTML code. There is also no way to actually view an entire article. An oversight on my part. Will need to fix!

Due to some shortcomings in my earlier prompts when building the UI, pages and functions intended for an Admin are getting mixed with those of an end user. I am going to have to clean that up one of these days. For now, let’s continue on with some fixes and tweaks… Here is another meaty one. I am about to run out of credits but here we go!

[PROMPT]
There are still some bugs and shortcomings that need to be fixed:
I still have an option to upload images when creating and editing an article. Trying to upload an image shows an error: ckeditorerror.ts:140. I would like to be able to upload images using the editor so please fix. We can use the /static/uploads/ directory to store uploaded images if necessary.

When I go to the topic/[topic number] page, I see article previews but they appear with visible HTML code. This page should act as an article preview and we should perhaps see a plain text version of the beginning of this article. We may have to tweak this later.
We need a new route to view an article, and the article should appear properly formatted. Right now, when you click an article it brings you to edit mode. We need to separate viewing from editing.

There should be a way to create a new article in each topic from the manage-topics page.
When creating a new article, there is a topic dropdown at the bottom of the page. This dropdown is not contextually aware of the topic the user is in. The dropdown should reflect the correct topic.
[/PROMPT]

While watching Claude build the code for the file upload feature, I noticed it was creating the same block of Javascript in a couple of different files.

[PROMPT]
It looks like there is some similar Javascript code in the new_article.html and edit_article.html files. Can we create this in a common file and share the code instead?
[/PROMPT]

Claude:

You're right, and that's a great suggestion. Let's create a common JavaScript file for the CKEditor configuration and use it in both new_article.html and edit_article.html.
First, let's create a new file called ckeditor_config.js in the static/js directory.

That was a lot of changes. While reviewing the code changes, I noticed the diff showed a bunch of code being removed from the app.py file and being replaced with the text “# … (rest of the existing code)”. I had never seen this before and though it was maybe how Claude dealt with a large file and it was not really going to delete existing code. Well, I was wrong!

When Claude completed these tweaks, I tried to hit the home page and it was 404. Lots of code and routes were removed and line 80 is literally "# … (rest of the existing code)”. Wow, that’s bad! Hopefully we can fix this but I smell API limits lurking!

[PROMPT]
Oh no! I seem to have lost a lot of code from the app.py file after your last change. Many of the routes are gone and have been replaced with "# ... (rest of the existing code)" on line 29 and 80 of app.py. Please fix!
[/PROMPT]

Claude:

I apologize for the oversight. You're absolutely right, and I'll fix this issue immediately. Let's restore the missing code in the app.py file.

Claude was able to fix the code but Wow - that’s a bad one. What the hell was that all about?

[PROMPT]
If I try to create the “New Article” button, I get this error: jinja2.exceptions.UndefinedError: 'csrf_token' is undefined
[/PROMPT]

Claude says we now need the Flask-WTF package and it will install it using pip. Let’s see if the system prompt to use the .venv directory will work…

No way! Claude did it again, deleting all the code during the code merge! You really need to pay attention and not just accept the generated code.

[PROMPT]
I can not accept this code! Line 31 “has # ... (rest of the existing code)” and you removed the rest of the code again!
[/PROMPT]

When Claude was done, the server tried to restart but failed with “ModuleNotFoundError: No module named 'flask_wtf'” which indicates that it was not using the .venv directory so I don’t believe the System Prompt is working. This may be an issue with the Claude-Dev extension.

[PROMPT]
I am getting the error "ModuleNotFoundError: No module named 'flask_wtf'" so please make sure you are using the .venv virtual directory when adding new libraries!
[/PROMPT]

Claude:

I apologize for the oversight. You're right, we need to install Flask-WTF in the virtual environment. Let's do that now.

And… like clockwork:

The good news is that Claude was able to use the correct virtual environment and the server started.

The bad news is that if I try to submit a new article, I am hit with another CSRF error. Wake up, Claude. You have some work to do!

[PROMPT]
Please fix the following bugs:
If I try to submit a new article, I get an error: “Bad Request. The CSRF token is missing.”
If I try to view an article, HTML tags are visible. The article should display formatted without the tags.

[/PROMPT]

Someone woke up in a bad mood… I am barely using it and I am getting hit with rate limits already.

[PROMPT]
I am still finding some bugs. These are all very basic and should not have made it into the code. Please check your code more carefully as these are costing my money each time you need to fix them! Perhaps the application is getting complicated enough that it is time to add some automated testing that will run on each code change? The issues I am finding:
When I try to view an article such as the one found at article/2, it is still showing HTML markup and not rendering the formatted article. Perhaps this is breaking when the article is saved? I am seeing lt; and gt; in the database and notice you have “content = html.escape(content)” on app.py line 416.
If I try to create a new topic, I am now getting “Bad Request. The CSRF token is missing.”
If I try to create a new article, I am getting “UndefinedError jinja2.exceptions.UndefinedError: 'form' is undefined”
[/PROMPT]

Watching Claude make code changes, I think it is melting down. Why is it deleting all this code again??

[PROMPT]
Why are you deleting all that code again? Line 35 has # ... (keep all the existing model definitions and helper functions) and all the other code is deleted? Be careful!
[/PROMPT]

Claude:

I apologize for the confusion. You're absolutely right, and I appreciate your attention to detail. Let's make the necessary changes while keeping all the existing code intact. We'll only update the specific parts that need to be modified.
Let's update the app.py file, focusing only on the parts that need to be changed

Sigh…

[PROMPT]
Lines 35, 37 and 46 still have those comments and you are deleting huge blocks of code. Why?
[/PROMPT]

And of course -

I can’t be the only one having these problems. Everyone else says they are making millions with Claude and I seem to be finding all the bugs! I opened an issue in the Claude-Dev Github project since I can’t live like this: Github link

After doing some more research, it does sound like my project is hitting some type of token limit which is documented here: Github issue. This really limits the complexity of the project you can work with and is the likely cause of so many API limit errors as the project became more complex.

If I understand the problem correctly, in order to increase code quality, Claude-dev sends the entire contents of the active file back and forth in requests. This eats up tokens and at some point, my app.py file became too large to manage. One suggestion is to break the code into smaller files. Way back in Part 1, this was the direction we were going but had some issues with circular imports and Uvicorn so here we are…

I guess I will put this project to bed for now and follow the Github thread for updates. This gives some hope that an upcoming Claude update will ease the pain: Github comment

Next up: Cursor? Last time I tried, I did not like the UX as much but a lot of people seem to have made the switch. Cursor seems to be the new kid in town for Developers that are using UI to increase their productivity. Because what we need right now is another IDE.

Round and round we go…

So, some conclusions?

Is Claude ready to replace software Engineers? Eh, not quite yet but it can make existing Engineers more productive. The API limits are restrictive for anything more than a small project, refactoring, bug fixes and use as a virtual pair programmer.

I have read that Claude can read complex codebases and fix bugs within it. In my experience, it ran into a couple of bugs it could not fix in a fairly simple project on code that it created itself so that did not give me the warm and fuzzies. How would it fare on an old codebase written by dozens of people with a complicated file structure? Claude would not have any business context and if asked to fix a UI bug, it did not do a great job at matching already existing UI and UX flow in this simple prooject. It needed several iterations to clean up after itself.

I find backend code “easier” than front end code so I would personally use it to help me build project structure and a skeleton UI. One of Claude’s strengths is “computer vision” which enables you to upload a file and ask Claude to recreate it in code. If you like the UI of a website, this should give you a great starting point. Perhaps this is the new “View Source”. Embrace and enhance, as they say.

Although one of my earliest prompts asked Claude to use Bootstrap to make a Responsive UI, I just checked the mobile view for the first time and found that it is not really functional in anything smaller than IPad Mini resolution and that doesn’t look great either. So make no assumptions. Claude’s code needs the same amount of testing as any other code.

In fact, it would make sense to have it build out an automated testing harness which can run prior to code check in. Nobody likes doing THAT work. Make Claude responsible for writing, maintaining and running tests. Definitely an avenue worth exploring.

I asked Claude to create some automated tests in one of my final prompts but the token and API limits put an end to this experiment before we ever got there.

QUICK ADDENDUM REGARDING CURSOR.AI

While I was working on this document, Cursor started heating up so I figured I would try it to see if it was any better. Well, it was at first.

They give you a free trial which includes 50 API calls and it was damn fast with no API limits. Great! I bought a monthly subscription and that's where the good news ends. Just like before, once the project becomes more than trivial, say your prayers. It's gonna be a long night...

I found that Cursor appeared to be making code changes but when I ran the project, the new code was not actually being applied to the correct files. I noticed lots of errors in the debug console about files missing and it was attempting to create directories that already existed. It seemed like the index had gotten corrupted somehow.

I posted this problem to the Cursor support forums, hoping for a quick workaround but never received one. Cursor developers have acknowledged this issue and at least one other person is seeing it too. They claim it will be fixed in the "next build". It has been 20 days and still no updates. I am crippled until then.

Part 2 - Using Claude, Claude-Dev and Aider to Build a Ticketing System

Joe Giglio, Chief Remote Officer — Mon, 19 Aug 2024 21:07:00 +0000

Part 1 | Part 3 | Part 4

The project has been open sourced if you would like to play along at home: https://github.com/joegiglio/helpdesk-claude-dev-ai

PART 2-
I wanted Claude to work on some simple integrations with popular, third party products. But first, I needed to clean up the UI a bit…

[PROMPT]
I don’t like how the “delete icon” looks in the ticket list view. It is just a small red box. Can you verify it looks as intended?
[/PROMPT]

After some back and forth, it was determined that the CSS file from Font Awesome was not loading correctly. Claude made the necessary changes and it was addressed. This did cost me another 20 cents and API credits though. Again, you have to pay for Claude to clean up after itself.

I added another $5 and wanted to switch gears towards Integrations.

[PROMPT]
I would like to make some changes to the Integrations page. 1. Add a fourth column for "Webhooks", with a placeholder image and short description text. 2. Make sure the height of all of the cards is equal. Right now, the Salesforce card is shorter than the others.
[/PROMPT]

Claude fixed it and came back with:

[PROMPT]
This looked better but the text in the cards are not aligned.
[/PROMPT]

I found Claude littering HTML files with inline styles like an animal. I asked it to create a styles.css file and move all CSS into it. I was surprised this was not done earlier in the project but it was quickly addressed.

I also did not like the image used for the Webhooks column since it was intended to represent a WAV file. Odd choice…

Image changes took a couple of tries to get right and I am paying for each request. I don’t want to come across as not being impressed by the power of this technology at my fingertips, so please don’t take it that way… But each request costs money and the daily API requests are limited so I hate having to waste them on follow-up clean up requests, especially on items that are seemingly “simple”.

Final result:

Now that we have a decent looking UI, I wanted to try to build simple integrations into a couple of these services.

It has been a while since I played with Slack Apps. From what I remember, the easiest way to get started was with a webhook. Slack assigns you a URL, you post JSON data to it and the text appears in a chat room. I did not want to waste my Claude credits yet so I spent some time refamiliarizing myself with Slack’s Admin and working with Gemini’s AI in a browser tab.

Turns out posting to a Webhook is still an option and you can find it among all the other bloated Slack app settings. Slack is looking more and more like a Salesforce product; death by 1,000 settings.

Now that I knew how to build the integration, I asked Claude to do the heavy lifting so I could critique his work…

[PROMPT]
I want to begin work on the Integrations feature of this application and want to start with Slack.

Whenever a new ticket is created in this application, I want to send it to a Slack chat room. I am not sure how to do this and need your guidance step-by-step. These steps will include configuration in Slack’s Admin. Let’s start there and then we will create the necessary code in our ticketing system to send data to Slack.
[/PROMPT]

Claude’s answer regarding the necessary set up in Slack’s Admin was correct:

Certainly! I'll guide you through the process of integrating Slack with your ticketing system to send notifications about new tickets. We'll start with the Slack configuration and then move on to implementing the necessary code in your application. Let's break this down into steps:
Step 1: Slack Configuration
To set up the Slack integration, we need to create a Slack app and configure it to receive incoming webhooks. Here's how to do that:
Go to the Slack API website (https://api.slack.com/apps) and sign in with your Slack account.
Click on "Create New App" and choose "From scratch".
Give your app a name (e.g., "Ticketing System") and select the workspace where you want to install it.
Once the app is created, go to "Incoming Webhooks" in the left sidebar and turn on "Activate Incoming Webhooks".
Scroll down and click on "Add New Webhook to Workspace".
Choose the channel where you want the notifications to appear and click "Allow".

You'll be given a Webhook URL. Keep this URL safe and secret, as we'll need it later in our code.
Step 2: Prepare the Application
Now that we have the Slack webhook URL, let's prepare our application to send notifications. We'll need to:
Install the requests library to send HTTP requests to the Slack webhook.
Store the webhook URL securely.
Modify the ticket creation process to send a notification to Slack.
Let's start by installing the requests library

I allowed Claude to proceed with making the necessary code changes to support this feature.

The code changes looked correct at first glance but I did not like that the WEBHOOK_URL variable was added to the app.py file. I thought it was more secure to add it to a separate file and remove that file from the code repo through .gitignore. Claude agreed, apologized for the oversight and cleaned up the code.

Attempting to start the server failed because it was trying to import the requests library which was never installed. It turns out that Claude was not aware that the project was using a virtual environment.

I prompted Claude to use the virtual environment named “.venv” but made a typo and told it to use “.env” instead which added more confusion. After requesting access to the top level of the project directory, it discovered the virtual environment at “.venv” and was able to install requests which cleaned up the problem.

I verified that creating a new ticket automatically posted it to the Slack room:

However, I would like the payload to also have a link to the ticket so whoever is using Slack can click the link to view the ticket in my system. This one is tricky because there is no mechanism yet to view an individual ticket other than the ticket edit screen. I wonder where this will take me…

[PROMPT]
This is working but I would like to change the message payload in the send_slack_notification function. I want to add a link to the new ticket so the viewer of the Slack message can click the link and view the ticket in this system.
[/PROMPT]

Claude indeed suggested linking to the edit_ticket URL…

Claude also took the liberty of improving the UI a bit, adding a View Ticket button. Nice job!

I noticed that the sort order for the All Tickets page was not specified so tickets were displayed as “oldest first”. I prefer “newest first” and asked Claude to fix this.

[PROMPT]
In the tickets function, please change the sort order to "newest first".
[/PROMPT]

The query to retrieve all tickets was correctly updated to

tickets = Ticket.query.filter((Ticket.deleted == False) | (Ticket.deleted == None)).order_by(Ticket.created_at.desc()).all()

But how far could I push this? I asked Claude to update the UI to offer a sort dropdown:

[PROMPT]
On the Tickets page, tickets are now being sorted by newest first. This works OK but I would like to add a sort feature to this page. Please add a dropdown list so the tickets can be sorted by Newest First, Oldest First, Assigned To, Priority, Status and Last Updated. Do the sorting client side to avoid having to reload the page. Retain the current default.
[/PROMPT]

Claude gave me a preview of the changes and it looked like some very complex Javascript that would be difficult to maintain. I fired back with:

[PROMPT]
This looks like it will work but also looks like complex code. Is there a way to achieve this functionality with built in features from Bootstrap or jQuery instead?
[/PROMPT]

Claude agreed:

You're absolutely right, and I apologize for suggesting an overly complex solution. Using Bootstrap and jQuery can indeed simplify this process and make the code more maintainable. Let's revise our approach using Bootstrap's built-in sorting functionality with the DataTables plugin, which works well with jQuery and Bootstrap.
First, we need to include the necessary CSS and JavaScript files for DataTables. We'll add these to the base.html template.

Its next iteration added some Bootstrap / jQuery plugins which added striping to the table but the sort dropdown was not visible.

[PROMPT]
I do not see any dropdown to sort the tickets. Please double check?
[/PROMPT]

Claude replies that it did not add a dropdown but instead added functionality to allow sorting by clicking the column heading which is a feature of Bootstrap’s Datatable. That is not what I asked for but it sounded like a potentially better solution. But it was not working!

[PROMPT]
If your intention was to allow sorting by clicking the column headers, I would be OK with that but that is not working either. Please add that functionality instead of the dropdown I originally requested.
[/PROMPT]

Claude thanked me for my flexibility, claimed to fix a few things and gave a nice summary of changes… but it was still not working!

[PROMPT]
I like these proposed changes but I am still not seeing them in the UI. There is no mechanism for sorting or searching. Please double check?
[/PROMPT]

In the middle of debugging this, I got hit with the dreaded:

Continuing on after a few minutes… Claude added some console.log statements and asked me to check the server console for these messages. In doing so, I noticed a script error which I reported back.

[PROMPT]
Viewing the console, I see "Script in tickets.html is running", but this is followed by an exception: tickets:160 Uncaught ReferenceError: $ is not defined at tickets:160:1
[/PROMPT]

Claude moved some script tags to the top of the base.html file and everything was working now with a much improved UI!

However, it left the debug statements in the console.

[PROMPT]
This is looking much better but you left the console.log debug statements. Please remove them.
[/PROMPT]

Claude apologized and removed the debug statements:

I apologize for overlooking the debug statements. You're absolutely right, and we should remove them for a clean production-ready code. Let's update the tickets.html file to remove the console.log statements.

I did not like the date and time format being used for the ticket list so I asked Claude to make these more human friendly. This seemed to break with the new Datatable component.

[PROMPT]
When viewing the ticket list, the date/time format used for the Created At and Updated At columns are not user friendly. Please use the format Month/Day/Year Hour:Minute AM/PM. We will also need to handle timezone considerations so I need you to suggest a fix for that too.
[/PROMPT]

Claude made the necessary changes to the ticket view, added the pytz library and is now displaying tickets in the Pacific time zone. Timezone should be relative to the viewer so we will revisit this when we get to the User Settings page, one of these days.

Just some more cleanup to the Tickets page before we go back to Integrations!

[PROMPT]
In the ticket listing, please update the search box to only search the "Title" column. Right now, all fields are being searched.
[/PROMPT]

Claude’s response:

The ticket list will now display dates in the format "Month/Day/Year Hour:Minute AM/PM" in the US/Pacific timezone, and the search box will only search the "Title" column of the tickets.
These changes provide a more user-friendly date/time format, address timezone considerations, and implement the requested search functionality improvement.

Nice work, Claude. Final UI from this segment:

Now let’s revisit Integrations. I had to put another $5 in the tip jar but it is well worth the small investment. I am learning a lot and the project is progressing.

[PROMPT]
I would like to do some more work on the Integrations part of this project. I think it would make sense for each Integration to have its own template page with its own settings that are stored in a new database table. For example, the Slack page should have a checkbox toggle for the user to decide whether to send new ticket data to Slack. It should also have a text field for the Slack Webhook URL, which should be used instead of the hardcoded URL that is now found in the config.py file.
The other Integrations - Salesforce, JIRA and Webhook should also have their own pages and settings, all stored in the database, which we will be building out once changes to the Slack page are satisfactory.
Each of these pages should inherit the lefthand navigation bar which should always remain visible during navigation.
[/PROMPT]

Claude made the necessary changes to the files and database and it worked fine. Nice job but it needed some UI tweaks.

[PROMPT]
This setting should be more descriptive, asking if the user would like to send newly created tickets to Slack. Also, enforce logic that if this setting is enabled, the URL field must have a valid URL before saving.
[/PROMPT]

Claude showed its proposed changes but I did not like that it was using plain Javascript. I would rather use jQuery and I asked for the change…

[PROMPT]
It looks like you are using plain Javascript. Please use jQuery for the Javascript portion.
[/PROMPT]

This change was working but I still wanted some more UI tweaks…

[PROMPT]
Those changes to the Slack integration page are working great but I would like some UI tweaks: 1. The spacing in the integrations/slack URL is too tight and needs to be improved. 2. Please use a flash message to show a success message when changes are saved.
[/PROMPT]

Claude showed its proposed changes but I did not like that it was putting the code for the Flash messages on the Integrations page. It should go in the base.html file.

[PROMPT]
That flashed_messages code might be better off in the base.html file so it can be used on any page that inherits it. Please confirm.
[/PROMPT]

Claude agreed…

[PROMPT]
That is working but can you use the type of Flash message that disappears automatically after a few seconds?
[/PROMPT]

Looking good. One more UI tweak and we should be there…

[PROMPT]
This is looking good but I would like to add a Slack logo to the integrations/slack page.
[/PROMPT]

Claude wants to make a new “static/images” directory with the file which makes sense to me. This is when I realized that the images being used so far were not stored locally. They were A HREFs pulling images from across the web. I want to use local images instead and requested this change…

[PROMPT]
This makes sense. I see there are images in the integrations.html file that are linked from the web and not local. All of those images should be stored locally in this new static/images directory. The code should be updated to use them. Please also make this change.
[/PROMPT]

Claude made the necessary code changes and ran a cURL command to copy the images locally. Nicely done!

[PROMPT]
Now that this is done, I want to follow the same format for the other integrations on the Integrations page - Salesforce, JIRA and Webhook. Please use the same Configure button which should link to each Integration page in the templates directory. Add the relevant logo and placeholder text since there are no settings for the other integrations yet.
[/PROMPT]

I hit an API limit while Claude was iterating so needed to take a break. Here is the UI from the final iteration:

Now on to JIRA integration. To keep it simple, I want to do the same as the Slack integration - namely create a new JIRA ticket using an API or webhook. The JIRA Admin and API docs are a maze of settings, so once again I asked Gemini for a head start:

[PROMPT]
You are an expert Python and Flask developer. I am developing an application in Python and Flask and I need to integrate with JIRA. I want to use the JIRA API to create new tickets. How can I do so? Is there a Python JIRA package or a webhook I should be calling? Please let me know what I need to enable in the JIRA Admin and give me some code examples.
[/PROMPT]

It gave me a couple of pointers on where to create an API key but I did not love this answer: “Depending on your JIRA instance (Cloud or Server), you might need to enable specific API permissions for your user or application. Refer to your JIRA administration documentation for details.”

If I was that smart, I wouldn’t need AI. 🙂

It did generate a Python code sample making use of the jira Python package. Let’s see what Claude has to say.

[PROMPT]
I would now like to create an integration into JIRA with this ticketing application. We need to create functionality similar to what was created for the Slack Integration piece, namely sending newly created tickets to JIRA.

How can I do so? Is there a Python JIRA package or a webhook I should be calling? Please let me know what I need to enable in the JIRA Admin and then make the necessary changes to this project.
[/PROMPT]

Claude made a bunch of code and database changes to accommodate my request. At first glance, they looked OK but the UI for the JIRA settings page removed the JIRA logo so I asked for it to be restored.

I also did not like the wording for the toggle, asking the user if he wanted to enable JIRA integration with no further context. I wanted it to match the wording on the Slack integration page:

[PROMPT]
I want the UX to match the one you created for the Slack integration page where the checkbox toggle reads "Send newly created tickets to Slack". Please do the same for the JIRA page.
[/PROMPT]

Claude made the change but each roundtrip costs money. A tweak like this could have easily been done manually but I am trying to see how much I can build with only AI prompts and I was curious to see if it would use existing UIs it already created as a guideline. It did not.

It also did not do a great job of telling me how to enable the API on the JIRA side whereas Gemini spelled out at least some of it.

Claude:

In the JIRA Admin panel:
Create a new project for your helpdesk tickets (note down the project key)
Create an API token for your JIRA account

I asked for clarification on how to create the JIRA API key but hit my daily API limit as it was coming up with an answer:

With some trial and error, I was able to create the API key and learned that the JIRA API URL is just the URL for your instance of JIRA. Easy enough. I put these entries in my system’s admin page and tried to create a ticket. Ticket creation worked but I saw this exception in the console:

response text = {"errorMessages":[],"errors":{"priority":"Field 'priority' cannot be set. It is not on the appropriate screen, or unknown."}}

The code generated by Claude was trying to set the Priority of the ticket in JIRA but that field does not exist in a default JIRA task list. I commented out this line, tried again and it worked! You can also add the field in the JIRA project settings as another possible fix. A more robust JIRA integration would need to have context of the JIRA project, available fields, etc.

I did see a new field was created in the database for “jira_issue_key” but it is always blank. It would be a nice touch to record this and have it visible in the view / edit ticket screen. I will try to clean this up once my daily API limits reset.

On to the next day. I hope Claude is feeling refreshed…

[PROMPT]
During the last task, you created the ability to send new tickets to JIRA. That is working now but it looks like the feature may be incomplete. You created a new database field in the ticket table named “jira_issue_key” but it is always blank. Can you fix this?
[/PROMPT]

Claude added some new code but it still didn’t work…

[PROMPT]
I just tried to create a new ticket in our system but the jira_issue_key field is still always blank in the ticket table.
[/PROMPT]

Claude is very slow today and is returning lots of 500 errors:

500 {"type":"error","error":{"type":"api_error","message":"Internal server error"}}

I went through about a dozen iterations with Claude trying to fix the problem. It added lots of logging and then some very complex database code that was way above my head. It was still not working.

[PROMPT]
Here is the latest log: INFO:main:New ticket created: #20
INFO:main:Slack notification sent for ticket #20
INFO:main:JIRA issue created: KAN-9 for ticket #20
INFO:main:JIRA issue key KAN-9 assigned to ticket #20
INFO:main:Before commit: Ticket #20 JIRA issue key: KAN-9
INFO:main:All changes committed for ticket #20
INFO:main:After commit: Ticket #20 JIRA issue key: KAN-9
ERROR:main:Failed to retrieve ticket #20 from database after commit And I have noticed that the ticket is no longer being created in the database although I am getting a Success Flash message. This is worse than the original problem. Before, I had an empty column but now I have an empty row and a false success message. Data is being lost without warning in the UI!
[/PROMPT]

I am probably going to run out of API requests soon and Claude can’t seem to solve this one and the code is in an ugly place now.

Thankfully I saved a commit as a restore point and if I disable JIRA I am still able to create tickets so the system is not completely crippled. I will put JIRA integration to bed for now and see if Claude can help me with some other features.

Or maybe not. Look who it is:

After looking at Claude’s code and the logs being generated, I vaguely remember having a similar problem in an old project. It would create a database entry and then needed to update the row immediately but Flask could not find the record. The problem was fixed with a db.session.flush() in my old project and I thought I would look like a genius by adding the fix here, but no luck. But I figured I would suggest it to Claude as a clue... Maybe he knows what to do with it.

The next day:

[PROMPT]
Yesterday, when we left off I was able to use this system to create a new ticket. However, if JIRA integration is enabled the jira_issue_key is always blank in the tickets table. You tried a bunch of different code and logging options but were not able to fix it. Having to try so many times has cost me some money with the API so I want to make sure we are going down the right path. In trying to debug it myself, I tried to use db.session.flush() before db.session.commit() but that also had the same problem. The ticket is created, but the jira_issue_key field is always blank. Can you try to fix this again? I can only allow 3 more tries though since this is getting expensive!
[PROMPT]

I hoped that would light a fire under his ass and keep him focused!

I told Claude he only had 3 tries but I gave him 8. The issue is still not fixed. I am stumped and he is stumped but I need to move on. This will likely be throwaway code anyway but issues like this drive me crazy!

I did find that if the table had an entry for the JIRA ticket ID, a button appeared that properly linked to the JIRA ticket which opened in a new window. So close, yet so far…

I would like to add a couple of new features: A knowledge base and “customer portal” where customers can submit tickets.

As a multi-channel ticketing system, customers should be able to submit support cases through email, webform, phone, Twitter, Facebook, SMS, etc. Webform via customer portal is the easiest so let’s start there…

[PROMPT]
I would like to create a new feature in the product and we will need to do it in several steps. The feature will allow end users to submit support tickets through a webform in our portal, neither of which exist yet. Please create new templates for customer facing pages: 1. A knowledge base with placeholder graphics and text. 2. This page should link to a webform where a user can submit a support case. The required fields are name, email address, subject and description. When a user submits the webform, it should create a new case in our system.
[/PROMPT]

While reviewing the proposed code, I noticed Claude was creating a local placeholder image. Normally I would be fine with this but other placeholder images throughout the product were done using via.placeholder.com. I asked Claude to re-use that service to keep the code consistent. I know that earlier I wanted Claude to use local images. This time I wanted to see if it would continue using the design pattern it started using earlier. It did not…

Claude was also creating some custom CSS for card sizing and I suggested he use the built in Bootstrap classes and he agreed that would be a better idea. His initial suggestions may work but they are not always best practices or consistent with the rest of the code base… and this is a simple codebase that Claude has created himself. Always double check the code before accepting it!

Here is the proposed UI for the Knowledge Base skeleton:

Even though that looked OK, I took a screenshot of a Knowledge Base UI that I liked and asked if Claude could build a UI similar to it. Computer Vision is supposedly one of Claude’s strengths and I have not used it yet. So I asked…

[PROMPT]
I would like to change the Knowledge Base UI found in the knowledge-base URL to be more like this image. Can you emulate this?
[/PROMPT]

This is what Claude came up with. Simple, clean and minimalistic… Not bad!

Moving on, I asked Claude to create a webform so users can submit new tickets. This worked fine but I wanted to modernize the ticket list UX so it shows new tickets without having to refresh. Off the rails we go!

[PROMPT]
I would like the tickets table to automatically refresh if the system receives a new ticket. Maybe we can use something like a websocket? New tickets should also flash yellow when they appear.
[/PROMPT]

Claude added the usual code for sockets-io, etc. The first attempt did not work - new tickets were created but the tickets Datatable did not update so Claude added some logging. Funny, I have this problem all the time with websockets too. I find it very confusing and it never works without some hair pulling.

Claude:

Thank you for providing the error message. It appears that there's an issue with the WebSocket connection. Let's try to resolve this by making some adjustments to our WebSocket configuration.
First, let's update the app.py file to ensure that CORS (Cross-Origin Resource Sharing) is properly configured for the WebSocket connection

API Request Failed

This API limit is going to kill me.

On to a new day...

We went round and round about a dozen times with websocket errors and logging but Claude can’t seem to fix this one.

The console logging suggest that it should be working fine but when I submit a new ticket, a Javascript alert pops up:

I asked Claude if he had access to view live pages on the web and suggested he review the following URLs for potential fixes:

https://datatables.net/forums/discussion/42814/requested-unknown-parameter-object-object
And http://datatables.net/tn/4

He reviewed the pages, changed the format of the data being passed to the Datatable but still no luck! Similar to the JIRA ticket issue, it looks like Claude is stumped, my API credits are about to run out and I am left with code that is littered with attempted fixes and logging. Again, good thing I committed code of the last good build.

Hey, guess who came to dinner:

API Request Failed

[SIDEBAR]

I asked Gemini if it could build a UI that emulated the same knowledge base UI that I fed to Claude. As a reminder, this way the image I used:

This was Gemini’s interpretation after several iterations. The styling is all off, everything is off center, etc. I am surprised that the results are so poor. I did not spend more than 15 minutes on it since it was not progressing much…

I am trying to use Gemini to fill in the gaps when Claude hits me with API limits. I guess its Computer Vision is not yet up to snuff though…

[/SIDEBAR]

Painful. To be continued when I regain my strength. 🙂

LINK TO PART 3

Part 1 | Part 4

ABOUT THE AUTHOR

His recent projects include:

Remote work training and consulting: chiefremoteofficer.com
Kindle book "Making Remote Work, Work For You"
Remote company rankings. Think "Glassdoor for remote companies": remotescorecard.com

Joe’s guiding principles are: Remote first, quality driven, customer champion and lifelong learning. He was an introvert and practiced social distancing long before it was cool!

He currently resides in North Carolina and is always interested in new opportunities, meeting quality people and learning about exciting projects.

Contact Joe Giglio: joe -AT- joegiglio.org | Twitter | LinkedIn

Part 1 - Using Claude, Claude-Dev and Aider to Build a Ticketing System

Joe Giglio, Chief Remote Officer — Tue, 13 Aug 2024 21:04:33 +0000

MARCH 2025 UPDATE:

I tried a similar experiment using Windsurf with Claude 3.5. How did it compare?

The only way to know for sure is to watch my video series, "The Truth about Vibe Coding with Windsurf… Better than Cursor?”

Part 2 | Part 3 | Part 4

PART 1 -

The project has been open sourced if you would like to play along at home: https://github.com/joegiglio/helpdesk-claude-dev-ai

AI is probably coming for all of us eventually. It promises to make companies “more efficient” but how far along is it right now? Can it really be used to replace software developers? Can you just pick a tech stack, tell AI what to write and put your feet up while the money rolls in?

I had tinkered with the usual LLMs - Copilot, ChatGPT, Gemini, Gab and Claude. I found them all to be capable of helping to create boilerplate code, adding basic unit tests and acting as a reliable pair programmer. The answers are usually pretty good but not without error. They give a solid starting point and prevent you from having to deal with all the geniuses on Stack Overflow.

I have seen demo videos of snake games and To-Do list web apps being created using only AI prompts without code and I wanted to take it further. Having a background in help desk and ticketing software, I wanted to push AI to create a ticketing system and see how far I can take it with integrations into third party apps such as Slack and Salesforce.

So what would my AI stack be? At the moment, the buzz seems to be around Claude, paired with VS Code extensions Claude-Dev and Aider so I figured I would start there. I believe the best tool for the job is the one you are most familiar with, so I would ask AI to use a tech stack of Python, Flask, SQLite, Bootstrap and jQuery. Keeping with languages I am familiar with will help me with the inevitable debugging and maintenance.

Using these plug-ins requires an API key but acquiring one is not as straightforward as you might expect. You create your account and login at https://claude.ai/ but need to obtain your API key from https://console.anthropic.com/ and there is no obvious link… or at least there was none during my initial draft but now there is. :)

Once I had my API key, I needed to add credits to my account or API requests would be rejected. Luckily, Anthropic is running a $5 credit promotion which appears in your dashboard. I accepted the offer, was texted a validation code which I entered and then…nothing. I did not see any credits. I was stuck in a loop of offer/validation/acceptance and could not get the credits to appear in my account. I sent them an email and figured I would try again the next day.

The next day came and there was no reply so I just decided to purchase $5 worth of credits and get on with the show. First up, I would try Aider which runs in a VS Code terminal window. I am not crazy about the UI but configuration was easy enough, it makes Github commits after each step and it gets great reviews on Youtube so it was worth a try.

Here is the initial prompt I used. I chopped it into a couple of pieces so I could check the work as we went along. Much like people, AI works best iteratively, giving you a chance to make sure things are working as intended at each milestone and giving you an opportunity to debug and clean up.

[START FIRST PROMPT]

I would like to create simple help desk software using AI to build the product. Instructions will be given iteratively so it can be built piece by piece.

You are an expert web developer with expertise in Python and Flask. All code must be secure to prevent SQL injection and other attacks. All fields must have reasonable size limits and maxlengths. Field lengths and data types are to be protected on the client and server side.

The technical requirements are:

Python
Flask
Fast-API
Bootstrap
jQuery
We will initially use SQLite for simplicity but will move to MySQL at some point so keep that in mind.

All pages must be responsive and work well on desktop, tablet and mobile screen resolutions. The product will be API-First with all CRUD operations working through the API layer using Fast-API. This includes fields for support tickets as well as administration such as users, preferences and other settings.
Integrations with other popular products such as Slack, JIRA and Salesforce and the ability to send data to a webhook URL will be important for adoption so keep that in mind while building the core of this platform.

[END FIRST PROMPT]

[START SECOND PROMPT]

Product requirements for the first iteration:
Build a lefthand bar with the following links:
Tickets
Integrations
Workflows
Team
Settings
Each of these links will be its own files in the templates directory.

When clicking one of the entries on the left navigation bar, the right side will refresh with the necessary data. Examples:
When you click Tickets, show a list of existing cases and a link to create a new case.
When creating a new case, display the following fields:
Ticket ID (Numeric, auto generated, read only but visible)
Customer First Name (field length 25, maxlength 50)
Customer Last Name (field length 25, maxlength 50)
Customer Email (field length 25, maxlength 50)
Subject (field length 50, maxlength 100)
Priority (Dropdown with values 1-Urgent, 2-Important, 3-Normal, 4-Low)
Assignee (Dropdown with a list of dummy data for now)
Details (Large textarea, maxlength 50,000)

There should be a save button so you can save the case.
When you click Integrations on the left navigation bar, show logos and placeholder data for Slack, JIRA and Salesforce.
When you click Workflows on the left navigation bar, show a placeholder header graphic and placeholder data.
When you click Team on the left navigation bar, show a placeholder header graphic and placeholder data.

When you click Settings on the left navigation bar, show a placeholder header graphic and placeholder data.

[END SECOND PROMPT]

After running the first part of the prompt, I checked the project structure and files. I was surprised and impressed to find that it generated a modular Flask file structure, with blueprints, separate routes files and Pydantic for strong typing which I believe is required for FastAPI.

Out of the box, Flask allows you to quickly crank out code and it is not strict about file structure or naming conventions other than the ‘static’ folder. This allows you to get up and running without a lot of ceremony which can be a benefit or drawback depending on the project. Aider, with Claude, was taking my project right into the deep end into somewhat unfamiliar territory but I liked what I saw so far!

However, the server would not start and dependency version conflicts took about 10 attempts to fix. Each request costs money and goes against your hourly and daily quota. Yes, it gives you the wrong code and yes, you have to pay for it to clean up and may be “put on timeout” if you ask it too many times while it works through the problem.

Although the server finally started and seemed to be working, it was using Uvicorn as its web server and it was not configured to detect changes to files other than Python. This meant every time I made a change to the HTML or CSS, I had to remember to manually restart the server. I found a few possible solutions from Claude and other websites but I could not get this to work. I put that annoyance aside and would go another round the next day.

Here is the initial UI Claude generated:

I spent some more time on the server restart problem the next day but still had no luck so I decided to add this requirement to the project requirement prompts along with some other tweaks to fix shortcomings from the first round. I created a new folder and went another round with Aider:

[START FIRST PROMPT]

I would like to create simple help desk software using AI to build the product. Instructions will be given iteratively so it can be built piece by piece.

You are an expert web developer with expertise in Python and Flask. All code must be secure to prevent SQL injection and other attacks. All fields must have reasonable size limits and maxlengths. Field lengths and data types are to be protected on the client and server side.

The technical requirements are:

Python
Flask
Fast-API
Bootstrap
jQuery
We will initially use SQLite for simplicity but will move to MySQL at some point so keep that in mind.
I am using VS Code as my IDE. Please make sure that changes to all files such as HTML templates and CSS files trigger an automatic restart of the web server.

All pages must be responsive and work well on desktop, tablet and mobile screen resolutions. The product will be API-First with all CRUD operations working through the API layer using Fast-API. This includes fields for support tickets as well as administration such as users, preferences and other settings.
Integrations with other popular products such as Slack, JIRA and Salesforce and the ability to send data to a webhook URL will be important for adoption so keep that in mind while building the core of this platform.

Let me know how to install all the necessary dependencies.

[END FIRST PROMPT]

[START SECOND PROMPT]

Product requirements for the first iteration:
Build a lefthand bar with the following links:
Tickets
Integrations
Workflows
Team
Settings

Each of these links will be its own files in the templates directory.

When clicking one of the entries on the left navigation bar, the right side will refresh with the necessary data and the left navigation bar will still appear. Examples:
When you click Tickets, show a list of existing cases and a link to create a new case.
When creating a new case, display typical fields that are found in ticketing software. Each should have a reasonable field length and maxlength.

There should be a save button so you can save the case to the database.
When you click Integrations on the left navigation bar, show logos and placeholder data for Slack, JIRA and Salesforce.
When you click Workflows on the left navigation bar, show a placeholder header graphic and placeholder data.
When you click Team on the left navigation bar, show a placeholder header graphic and placeholder data.
When you click Settings on the left navigation bar, show a placeholder header graphic and placeholder data.

[END SECOND PROMPT]

I ended up with a similar file structure as the first attempt but the server would not start due to circular import issues, a familiar pain point for Python developers when they try to modularize their code and break pieces into separate directories:

ImportError: cannot import name 'routes' from partially initialized module 'helpdesk_app' (most likely due to a circular import) (/home/joe/Documents/programming/python/helpdesk2/helpdesk_app/__init__.py)

Aider was able to fix this and similar issues after several attempts. Finally, I could get the server to start without issue but when I tried to hit the homepage, it generated a database exception which I fed back to Aider:

When I try to hit the homepage I am getting this error. It looks like a problem with the database: sqlite3.OperationalError: no such table: tickets

Aider itself immediately crashes with:

messages = self.format_messages() ^^^^^^^^^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/coders/base_coder.py", line 904, in format_messages messages_tokens = self.main_model.token_count(messages) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/models.py", line 532, in token_count return litellm.token_counter(model=self.name, messages=messages) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1987, in token_counter num_tokens += calculage_img_tokens( ^^^^^^^^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1834, in calculage_img_tokens resized_width, resized_height = resize_image_high_res( ^^^^^^^^^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1751, in resize_image_high_res if width <= 768 and height <= 768: ^^^^^^^^^^^^ TypeError: '<=' not supported between instances of 'NoneType' and 'int' (.venv) joe@mint:~/Documents/programming/python/helpdesk1$ aider Aider v0.49.1 Models: claude-3-5-sonnet-20240620 with diff edit format, weak model claude-3-haiku-20240307 Git repo: .git with 21 files Repo-map: using 1024 tokens VSCode terminal detected, pretty output has been disabled.

Trying again:

It looks like the database tables were never created or initialized. Can you create code to do so?

Traceback (most recent call last): File "/home/joe/Documents/programming/python/helpdesk1/.venv/bin/aider", line 8, in <module> sys.exit(main()) ^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/main.py", line 620, in main coder.run() File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/coders/base_coder.py", line 685, in run self.run_one(user_message, preproc) File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/coders/base_coder.py", line 723, in run_one list(self.send_message(message)) File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/coders/base_coder.py", line 945, in send_message messages = self.format_messages() ^^^^^^^^^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/coders/base_coder.py", line 892, in format_messages messages += self.get_files_messages() ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/coders/base_coder.py", line 592, in get_files_messages repo_content = self.get_repo_map() ^^^^^^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/coders/base_coder.py", line 564, in get_repo_map repo_content = self.repo_map.get_repo_map( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/repomap.py", line 105, in get_repo_map files_listing = self.get_ranked_tags_map( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/repomap.py", line 448, in get_ranked_tags_map num_tokens = self.token_count(tree) ^^^^^^^^^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/repomap.py", line 69, in token_count return self.main_model.token_count(text) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/models.py", line 542, in token_count return len(self.tokenizer(msgs)) ^^^^^^^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/models.py", line 528, in tokenizer return litellm.encode(model=self.name, text=text) ^^^^^^^^^^^^^^ File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/llm.py", line 20, in __getattr__ self._load_litellm() File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/aider/llm.py", line 27, in _load_litellm self._lazy_module = importlib.import_module("litellm") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1387, in _gcd_import File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 935, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 995, in exec_module File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/litellm/__init__.py", line 20, in <module> from litellm.proxy._types import ( File "/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/litellm/proxy/_types.py", line 10, in <module> from pydantic import BaseModel, ConfigDict, Extra, Field, Json, model_validator ImportError: cannot import name 'model_validator' from 'pydantic' (/home/joe/Documents/programming/python/helpdesk1/.venv/lib/python3.12/site-packages/pydantic/__init__.py). Did you mean: 'root_validator'?

Another exception - something Pydantic related? Way out of my league...

I was stuck and STILL had the web server restart issue. It was discouraging but I decided to simplify the initial application by removing the Fast-API requirements and sticking with a simpler Flask structure to see if I can get the basics to work and then push into the more advanced stuff iteratively.

These are the prompts I used next. There are some differences including instructions around the virtual directory, removal of FastAPI and the generating of a .gitignore file.

[START FIRST PROMPT]

I would like to create simple help desk software using AI to build the product. Instructions will be given iteratively so it can be built piece by piece.

You are an expert web developer with expertise in Python and Flask. All code must be secure to prevent SQL injection and other attacks. All fields must have reasonable size limits and maxlengths. Field lengths and data types are to be protected on the client and server side.

The technical requirements are:
Python
Flask
Bootstrap
jQuery

We will initially use SQLite for simplicity but will move to MySQL at some point so keep that in mind. Be sure to initialize the database.

Let me know how to install all the necessary dependencies. The virtual environment has already been created and is active. The directory name is “venv”.

Generate an appropriate .gitignore file for the project and include the venv directory in it.
I am using VS Code as my IDE. Please make sure that changes to all files such as HTML templates and CSS files trigger an automatic restart of the web server.

All pages must be responsive and work well on desktop, tablet and mobile screen resolutions.
Integrations with other popular products such as Slack, JIRA and Salesforce and the ability to send data to a webhook URL will be important for adoption so keep that in mind while building the core of this platform.

[END FIRST PROMPT]

[START SECOND PROMPT]

Product requirements for the first iteration:
Build a lefthand bar with the following links:
Tickets
Integrations
Workflows
Team
Settings
Each of these links will be its own files in the templates directory.

When clicking one of the entries on the left navigation bar, the right side will refresh with the necessary data and the left navigation bar will still appear. Examples:
When you click Tickets, show a list of existing cases and a link to create a new case.

When creating a new case, display typical fields that are found in ticketing software. Each should have a reasonable field length and maxlength.

There should be a save button so you can save the case to the database.
When you click Integrations on the left navigation bar, show logos and placeholder data for Slack, JIRA and Salesforce.
When you click Workflows on the left navigation bar, show a placeholder header graphic and placeholder data.
When you click Team on the left navigation bar, show a placeholder header graphic and placeholder data.

When you click Settings on the left navigation bar, show a placeholder header graphic and placeholder data.

[END SECOND PROMPT]

The server started right up and the file structure was much simpler:

Everything seemed to work OK and I was also impressed with what Claude did with the skeleton of the Integrations page:

I noticed that the .gitignore file was generated but it was not excluding the “.venv” directory which was my mistake. I originally instructed it to look for a “venv” directory instead of a “.venv” directory. AI is only as smart as what you feed it and it can’t read your mind. Yet.

I then prompted:

The basic frameworks seems to be working OK but I would like to add more fields for each ticket. Please add more field types that would be typical of a helpdesk ticketing application. Each field must be an appropriate length with maxsize enforced on the client and server side.
Also create the ability to edit each ticket from the tickets page. Clicking the ticket ID field would be the logical place to add this functionality.

This worked great and I thanked Claude and prompted him to take a break but I got this error message:

OK then… I let it sit for a while and wanted to see if Claude could add “ticket delete” functionality. I created a new branch and prompted:

Please now add the ability to delete a ticket. For now, let’s just mark a ticket as deleted and not physically delete it from the database. Add a delete option to each ticket in the listing of all tickets. Use an X icon. Confirm deletion. Hide deleted tickets from the list of all tickets. We will soon build a view to show only deleted tickets. Be sure to update the database schemas as appropriate.

This returned another rate limit error:

Womp, womp… I guess Claude will get the rest of the day off and I will see how far I can push it tomorrow.

The next day, I fed the same prompt and it returned a well thought out response to show that it understood the technical requirements:

To accomplish this task, we'll need to make several changes to our existing codebase. Let's break it down into steps and use the available tools to implement these changes. We need to update the database schema to add a 'deleted' column to the tickets table. We need to modify the app.py file to add a new route for deleting tickets. We need to update the tickets.html template to add a delete button with an X icon for each ticket. We need to modify the existing route that lists all tickets to exclude deleted tickets. Let's start by examining the current state of our app.py file and the tickets.html template.

After showing me the proposed change step-by-step, Claude reported back that it was done!

Task Completed The ability to delete tickets has been successfully implemented. Here's a summary of the changes: Updated the Ticket model in app.py to include a 'deleted' column (Boolean, default False). Modified the tickets route to only show non-deleted tickets using Ticket.query.filter_by(deleted=False).all(). Added a new route /tickets/<int:id>/delete for soft deleting tickets. Updated the tickets.html template to include a delete button with an X icon for each ticket. Added confirmation dialog before deleting a ticket. Ran database migrations to apply the schema changes. These changes allow users to mark tickets as deleted without physically removing them from the database. Deleted tickets are hidden from the main ticket list, and we can easily implement a view to show only deleted tickets in the future. To see the changes in action, you can run the Flask application and navigate to the tickets page. You should see the new delete buttons (X icons) next to each ticket. Clicking on a delete button will prompt for confirmation before marking the ticket as deleted.

I saw the new column in the database but now none of my tickets were appearing in the list:

Newly created tickets were appearing and this is the code generated to query for non deleted tickets:

def tickets(): tickets = Ticket.query.filter_by(deleted=False).all() # Only show non-deleted tickets return render_template('tickets.html', tickets=tickets)

Based on my experience, this is probably a zero vs. null issue. I asked Claude to fix it:

None of the existing tickets are appearing in the ticket list but newly created tickets do appear. The query for getting all tickets may be incorrect.

The response:

I apologize for the oversight. It seems there's an issue with the query for retrieving tickets. Let's fix this problem by adjusting the query in the app.py file. We'll modify the tickets route to correctly display existing tickets while still hiding the deleted ones.

Claude updated the code to:

def tickets(): tickets = Ticket.query.filter((Ticket.deleted == False) | (Ticket.deleted == None)).all() # Show non-deleted tickets and tickets without the 'deleted' attribute return render_template('tickets.html', tickets=tickets)

…and now it works as intended with a new column and deletion confirmation:

I am not too crazy about the delete icon but I am just about out of Claude credits. I will continue on this journey for the next part of this journal.

SIDE NOTE - While iterating on this project, I somehow ended up with an instance that requires user login for all of the pages. I did not explicitly ask for this and I am not sure how we ended up here. I had to reboot my system and lost all my Aider / Claude history and I am not sure if it can be retrieved. However, the first check in for the Master branch does show a “flask_login” import, followed by a bunch of fixes and refactors. Those refactors added a @login_required decorator to every page.

After all those fixes and refactors, each page of the UI is asking for a login but there is no mechanism for creating an account. I will need this feature at some point but not yet. Now it is just in the way. This goes to show that you should not blindly accept the code without reviewing it and you really need to understand what you are looking at. Claude took creative liberties and added a feature I did not ask for.

This was an interesting exercise for me and hopefully you found it insightful. I found that Claude has the same battles I would have once I push beyond a basic Flask setup - issues with circular imports, dependency version hell, etc. New day, new tool, same old problems.

Just like any of the “no code tools”, the audience that will benefit the most are the people who have had experience writing code from line one, understand programming concepts, are able to clearly explain requirements in the written form and have troubleshooting experience in the tech stack at hand.

I will be pushing this project further iteratively to see how far I can take it.

Next up: Integrations!

Part 2

Part 3 | Part 4

ABOUT THE AUTHOR

His recent projects include:

Remote work training and consulting: chiefremoteofficer.com
Kindle book "Making Remote Work, Work For You"
Remote company rankings. Think "Glassdoor for remote companies": remotescorecard.com

Joe’s guiding principles are: Remote first, quality driven, customer champion and lifelong learning. He was an introvert and practiced social distancing long before it was cool!

He currently resides in North Carolina and is always interested in new opportunities, meeting quality people and learning about exciting projects.

Contact Joe Giglio: joe -AT- joegiglio.org | Twitter | LinkedIn

Some images from Unsplash

The Importance of "Whole Company Support"

Joe Giglio, Chief Remote Officer — Wed, 29 Jan 2020 21:42:21 +0000

Teams do their best, and most valuable, work when they have a deep understanding of their products, domain and customers. If you are not working to better understand and serve your customers, what are you working on?

Many companies still consider "technical support" and "customer service" to be a burden, nuisance and cost center. I wholeheartedly disagree and take the opposite view. I believe the support team is one of the most important teams in a company. They understand the products at an intimate level and most importantly, understand your customer's use cases, demands, pain points and requirements.

Whole Company Support, also known as All Hands Support, has been gaining traction with companies that want to build a more well-rounded organization with deep levels of customer empathy.

Technical support, customer service, customer success - whatever you want to call it - these roles are vital to the health of your company. These are roles that require skill, tact, communication, constant learning and patience. Although not typically considered as "valuable" as "technical skills", these "soft skills" can often be the difference between a satisfied or disgruntled customer. Remember, the support team is often responsible for mending relationships and playing a liaison role between customers and the rest of the company.

Customer Support is a craft and not everyone is good at it. If there are team members in the organization that do not have the confidence, patience or communication skills to directly handle support requests, all is not lost! The goal of Whole Company Support is not necessarily to create more support agents, although that may be a positive side effect. Rather, the goal is to increase collaboration and build a company where everyone develops a deeper understanding of their products and empathy for their customers. Everyone can help the company meet these goals by partnering with a member of the Support team, reviewing support cases for common themes and committing to use their skills to help solve problems and improve the customer experience.

My first software job was as a Technical Support engineer. It was with a new startup company that did not have many customers so I was asked to also perform QA duties during support downtime. This turned out to be ideal since the roles complement each other... As QA, you learn every inch of the products. As Tech Support, you learn how real customers are using the product. The combined knowledge is helpful in helping drive product direction and influencing the Engineering team to focus on the right things.

On a personal note, this combined role kept me intimately involved with the Sales and Engineering teams and I won Employee of the Year awards twice. Not just because of the great job I did but also because everyone at the company knew who I was since I found solutions for prospective and existing customers. I straddled the line between customer advocate and technical “Product Management”.

A friend of mine works customer support for a major retailer and most of his calls fall into one of a few common themes. The number of phone calls and level of customer frustration could be drastically reduced if the company practiced Whole Company Support and was closer to the customer experience. Some examples:

Mailed gift cards - There is a bug with the gift card system that falsely reports the gift cards as delivered when they have been received by the post office. In addition, the tracking number in the email and the support system is often incorrect. This creates anxiety for customers who often believe the gift card is lost in the mail. The support agents are helpless in tracking the gift card and all the customers can do is “wait a few days” and hope the gift card shows up.
Third party vendor items - This company employs “drop shipping” for many items which is invisible to their customers. Once the customer pays for the item, the system simply has a status of “Processing”. Tracking information is not available and it is not possible to get further information or cancel an item without an agent sending a request that is handled by another team. This can take several days and creates anxiety for customers. If an item is taking too long to be delivered, customers often want to immediately cancel the order and it is not possible due to the lack of integration between the in-house and partner systems.
Lack of emailed updates - If an agent cancels or discounts an ordered item, the customer is not sent a confirmation email. This leaves doubt in the customer’s mind as to whether their request has been fulfilled.
Lack of “after call work” following phone calls - On busier days, the incoming call volume can be very heavy. In order to reduce hold times, the telephone system does not support “after call work” which means that calls are often wrapped up without agents entering any notes. If the customer calls in again, there is no record in the system of what was done for them and who handled the call.
Old orders disappear from the system - In order to relieve strain on the database, orders over a certain age are archived off the main system and are unavailable for agent querying. They are available on another system which can only be viewed by certain supervisors. Wait times for supervisor escalation can often exceed ½ hour so customers are frustrated having to wait so long.
Restrictive coupon codes - Customers are often emailed coupons that give them “25% OFF EVERYTHING AND FREE SHIPPING ()”. The asterisk () is a long screen of fine print that customers don’t read. The fine print encompasses the brand and merchandise restrictions, so “EVERYTHING” is never really on sale.
Shopping cart view/edit - Customers often call in about items that are still in their shopping cart. Because many items have the same description, agents often need a model number to assist the customers. If agents had the ability to view/edit the customer’s shopping cart, they would be able to provide a faster and personalized level of support. I have seen this strategy employed at Crutchfield and was always impressed with the technology.
No price protection - Prices may change from day to day and the company does not have any type of price protection policy. If the price drops on an item that you ordered yesterday for a higher price, too bad. Perhaps if the company employed an “automatic [X] day price protection” policy, customers would believe they are getting the best possible deal and would happily continue shopping there in the future.

After reading this list of grievances, hopefully the wheels are spinning in your head as to how you may address each one. These are all items that can be addressed by various teams in the company - IT, website, marketing, partnerships, etc. Some of these items can be fixed with code updates, others with revisions to outdated policies that may have once made sense but should be reviewed due to the impact on customer satisfaction.

Regardless of one's role, there are always benefits to working with the support team and being closer to the needs of your customers. Spending time with the support team should be part of onboarding training. I also recommend rotating support assignments for each team member. The schedule should be published far in advance so teams can adjust for a member's absence and lessen the impact on scheduled deliverables.

Give your support team the respect, tools, trust and empathy they need to keep your customers happy. It can be the difference between customer attrition and lifelong customers who happily recommend your company to their friends and social media followers.

About the Author

Joe Giglio is a 25 year veteran of the software industry. In his career, he has worked for a local dial-up ISP, startups of all sizes and a well established, slow moving enterprise. There were important lessons learned along every stop.

His guiding principles are: Remote first, quality driven, customer champion and lifelong learning.

He currently resides in North Carolina and is always interested in new opportunities, meeting quality people and learning about exciting projects.