So before I get started on this blog, I must make a quick errata. My previous PR for 0.2.4 was rejected as it failed on a certain recipe. I tested it on several, alas it was not enough. Maybe I could add a testing module to this project down the road. Things to consider for 0.4. Anyway, long story short I went back and rewrote the entire PR using the main website's library, as opposed to using a scraper like I was before. Will report back with progress.
Continuing onwards, this week's blog post is about the second portion to assignment 3 (assignment 0.3.2) in my Open Source Development class. 0.3.2 is about continuing my progress as a novice developer to contribute once more to a project of my choosing; so I will return once again to Plain Old Recipe, a website that I love, written in a language that I love. What a combo.
I added three contributions (four if you include the one that I had to fix)- but two of them are nothing: one is an addition to the project's documentation about running it locally, and the other is about autofocusing the URL bar upon page load. They aren't much, I just wanted to add them because they were small. The main contribution is yet another recipe parser which I developed over the course of a day or two... which I... also might have submitted, and which came back as having failed some tests the developer did. I swear, I'm testing this stuff! I just hadn't predicted just how different one recipe could be from the next.
My first implementation made use of the website's internal scraping library which relies on BeautifulSoup4, this took me longer than I thought would have been necessary, as scraping a recipe website's list of instructions always proves harder than expected.
-
<div>- Every step
- Is linked to another step
- So you must first find the main div
- then iterate over the entire div's class.
- It sounds easy when spelled out like this but every website
- handles
lichildren a bit different, which requires - a different method for scraping
</div>
However as mentioned, my first implementation came back. Back to the drawing board. I've always wanted to draw out a system on a whiteboard, don't ask me why. I like to think that system engineering is where I'll end up in my career but we'll see. Something about it really interests me.
However, this time I went back with a recommendation to try using his internal JSON scraper- something which I'm not very familiar with (JSON, that is). This implementation worked out much better than the previous version did. I'm slowly amassing this weird specialization with the BS4 library (along with basic Python skills.) I particularly enjoy this elegant method I used to find all the children of an ol tag and iterating over them. The second attempt on this PR has yet to be accepted, and I'm hoping to hear back soon! And yet...
Regardless of how I put in 2 extra "PRs", or had to rework a previous PR, or even how I had to rework this PR, I still personally feel like this is not enough. Then I stumbled upon the real meat.
Someone on a previous issue had mentioned a great idea: a logging system which collects metrics about failed recipe attempts, thus giving the developer new ideas on what websites should be supported in the future. This could be awesome! There could be so much I could iterate on with something like this. That plus having just come back from a four two month co-op using Machine Learning, I'm always down to make pretty graphs or analyze some sort of data. Either way, I'm really excited to hear back from my suggestion on how this could be implemented (with the helpful sage-like wisdom from my Professor, who keenly suggested the idea of using a Google shreadsheet as a "cheaper" alternative to using a database.)
This experience using web technologies is priceless to me, and is exactly what I need as I could use as much web experience as possible for next coming term when I implement my capstone project.
See you next time, hopefully with some good news. This may or may not be the last update to this post, nor the last PR in 0.3. Depends on what happens. Stay tuned!
Top comments (0)