DEV Community


Discussion on: I wrote a crawler for the first time.

kevinhickssw profile image
Kevin Hicks

I had to write some crawlers when I first started. It definitely is one of those things you start out having no clue how to do it and thinking it's going to be extremely tough.

I like the simple approach you took to store the data in a JSON file and use Github Actions to update it. It's nice when we can keep things simple instead of spinning up databases and complex infrastructure.

Also, congratulations on getting over the intimidation of doing this. Us senior developers definitely do get afraid of tasks. We need to remember that we can always learn how to do something, just like how we didn't know how to write code before we were developers.

kaylasween profile image
Kayla Sween Author

Definitely! Keeping it simple was really important to me because I didn’t want to spend a long time getting it set up or potentially spending longer maintaining it.

Thanks! Yeah this is one that’s been haunting me for sure, so it gave me a good confidence boost! And that’s very true!

Thanks for reading!

Some comments have been hidden by the post's author - find out more