How good can be a security checklist generated by GTP-4?
I recently tested GPT-4 to see if it could generate a decent security checklist, so I made an app for that and I want to share my findings with you.
Check out the result for Ruby on Rails: https://checklist.devops.security/ruby-on-rails.pdf.
If you want to make your own, go to https://checklist.devops.security/
The good stuff
GPT4 can cover much more ground than GPT3 for security requirements. Now we can have security checklists much more specific than before and nicely grouped into sections. Compared to previously seen resources from industry standards, e.g., OWASP, I guess AI is helping us to step up.
The other side
GPT-4, like its predecessor GPT-3, isn't always 100% accurate or point to the latest libraries. Still better than what's out there. So, if you're using one of these checklists, it's still super important to have a human review it to make sure everything's on point.
And then there's the cost. GPT-4 API calls ain't cheap, especially when compared to GPT-3. So, if you're on a budget, this might not be the most practical solution. In the future things may change, though.
All in all, GPT-4-generated security checklists can be a handy resource, but make sure you double-check everything or ask someone to double-check for you.
Happy devsecops!
Top comments (0)