DEV Community

Amazing
Amazing

Posted on

Release 0.4: Progressing

#javascript #webdev #opensource #programming

A few updates here for you folks

Fixing the first requirement

Like what I thought the first requirement is quite simple. I use JSON pretty print to help me get a better and clean look at the returned JSON when testing. But not to mention this is my first time actually running the docker and start all the images by using npm services:start and every time you made some change to the code you will have to stop and run it again npm services:stop.

I hear that Emily has been working on the invalid feeds for a long time so I thinking about checking her commit would give me an idea on where to start it. I see Emily create some change in the src/api/posts/src/storage.js where we might get the info of the invalid feed through getInvalidFeeds. I went in and make use regex to chane all the \n to whitespace

reason: reason.replace(/\n/g, ' '),
Enter fullscreen mode Exit fullscreen mode

Then I restart the images and ran again the results the regex do the work

A lot of debugging for second requirement

For the second requirement, I would check all the common point of the 304 invalid and it is weird to me that some of them seem to work fine and show up in telescope where it should not

An invalid post would be a post that we would not be able to pass through the parser.

I put the break point at src/backend when the queue is failed we will call invalidateFeed

Image description

And to my prediction, the rss-parser fetcha a feed and gets 304, the 304 has no body so we considered it to be failed request and mark it invalid

I try to make change at src/backend/feed/processor.js

 if (!(response && response.status >= 400)) {
    info.shouldDownload = false;
    return info;
  }
Enter fullscreen mode Exit fullscreen mode

When I restart it, for a while there was no 304 psot which make me feel so exiciting about it but for a while the 304 keep showing up more and more so the problem has not been solved

Third fix

I actually found 2 resources which might be useful to help me in setting the ttl expire period for the key: Redis-doc and issue-100 and I wil be dig in to it in couple days to figure it out

Top comments (0)