DEV Community

Cover image for Don't fully trust AI in dev work! /yet
kaumnen
kaumnen

Posted on • Edited on • Originally published at akom.me

Don't fully trust AI in dev work! /yet

Let's talk about AI tools for code completion.

After spending quite a bit of time debugging my code, what's wrong with it, I've found the 'bug'. Not the ordinary one.

Don't fully rely on AI. Seems simple right? However, when you are doing some easy/repetitive stuff, you might fall under influence of just 'tabbing'.


What was the 'bug' anyway?

I was making a Twitter bot in NodeJS. Get some data from Reddit, and post it on Twitter bot account.

First off, I need a library, to simplify things about auth and whatever. A'ight, done.

npm install, make file structure, git init.

Here comes the fun part. I'm writing a constructor for a Twitter client. It needs 4 elements:

  • API key
  • API secret
  • Access token
  • Access secret

Okay, fair enough.


Constructing the constructor

First one, it's okay. TAB.

image.png

Seems good, TAB!

image.png

Awesome. GitHub Copilot is great!

image.png

It's so easy. TAB and there we go. Constructor finished!


Testing

Let's try to get a simple 'Hello World!' tweet out there.

401, Unauthorized.

Umm, what? Let's go to the .env file, let's check if the naming is correct. Copy, paste.

Check.

401, Unauthorized.

Okay, let's check for Twitter stuff. Check permissions, regenerate keys.

Check.

401, Unauthorized.

Now comes the part where I tried to debug code, try a different library that supports Twitter's V2 API. No success.


Debugging

Google, google, google. Try this, try that. Progress: NULL

I go for a walk, hoping for an 'Aha!' moment.

No luck. Back to work! Open Postman. Input parameters. Check headers, check body.

Send.

It worked!

Hmm, what could it be? Open postman, copy the curl request. Paste it in Notepad++.

I previously turned on debug mode in my code, copied the curl request from there too. Paste it in Notepad++.

Double-click one. Check.

Double-click another one. Check.

Another one. Check.

...

One is missing. How's that possible??

Now I went to the library docs.

Do you know what I did? Doesn't make sense, but let's try. Better than nothing.

Search for a method to input custom headers.

Nope. I did google (a lot) previously, about errors, and whatnot, but either there was no answer, or it was a typo in the .env file.


'Aha!' Moment

Maybe... It's the wrong naming of the constructor fields. So I check. Now I rely on IntelliSense and typing fields.

image.png

Seems good.

image.png

Keep on.

image.png

Okay. Onto the last one.

image.png

Wait, what? Wasn't it accessToken..? My literal reaction was: NO WAY! Are you kidding me?! 😂


GitHub Copilot:

image.png

IntelliSense:

image.png

GitHub Copilot’s AI was trained on the publicly available codebase. So, you have to take care of bugs others have written in their code!

GHC recommended me to use the accessTokenSecret field for a constructor, but the problem was that there isn’t a field with such a name!

The right one was accessSecret, recommended by IntelliSense engine.


There is always an opportunity to learn

So yeah… There are zero places that can teach you those things. You must experience them and unlock that kind of knowledge. 😄


Keep going!

These are my .02 cents. Don't let code completion AI tools rule your work. I'm sure there is a lot of funny stories like this. But we all make mistakes. Embrace them as learning opportunities.

I don't blame GHC for this. I blame myself. But whatever. At least I got some experience. There is always space for improvement.

Fun one for sure.

*Thank you for reading! *

_ktb

Top comments (6)

Collapse
 
ben profile image
Ben Halpern

I feel like the fundamentals of debugging sort of transcend the details of programming.

At least for me, I find the art of debugging is often about isolating variables and learning about a system. I think it's the most fun when you can tell that the problem is approachable, but hairy. Like, don't task me to debug bank software written in COBOL and enjoy it, but give me something reasonable and I'm going to methodically work my way through the problem by isolating parts of the system and testing them for variance.

So when we have this AI buddy helping write, debug, whatever... It still presents a system with variables that you can presumably isolate for success. If our AI partnership is to be successful, then there should be a capacity to either work with them or on them to isolate variables and solve problems. How this is done needs to evolve, but I really feel like the process is somewhat fundamental to any of these systems.

Collapse
 
kaumnen profile image
kaumnen

Agreed. Good points!

Collapse
 
andypiper profile image
Andy Piper

This was super interesting to read, thanks for sharing. So, which library did you end up using for the task?

Collapse
 
kaumnen profile image
kaumnen • Edited

Thanks, appreciate it!
I ended up using twitter-api-v2. It's even recommended on twitter docs too: developer.twitter.com/en/docs/twit...

npm link of mentioned package: npmjs.com/package/twitter-api-v2

github repo: github.com/kaumnen/RedditTweetz

Collapse
 
andypiper profile image
Andy Piper

Thanks, I wrote that page on the documentation site :-) so I am glad that you found it useful as a reference. We tried to list libraries that we knew about that covered most of the new API, although they are not "recommendations", just pointers.

Thread Thread
 
kaumnen profile image
kaumnen

Awesome, it was helpful, especially during debugging phase in order to narrow down a bug!