DEV Community: Chris Pinkney

Curiosity, one of the most genuine of emotions

Chris Pinkney — Sat, 24 Apr 2021 00:56:08 +0000

Today marks the final day of my OSD700 experience. I can't believe it's already over, it feels like just yesterday when I was mashing my refresh button to ensure that I was one of the lucky few to get accepted into OSD600. Most of my friends weren't as lucky. Some may say it's through the power of fate, though personally I say it's through the power of fibre optic internet.

Chris, shortly after getting into OSD600

One of the things I've found myself ruminating on lately is just how much I've learned these past 8 months. Several questions have come up from friends asking me for advice, most of the time I the answer, or can at least yield some sort of guiding intuition. This intuition was solely gained from my time spent in these courses. I've found similar behaviour in some stackoverflow/reddit questions.

I've made friends too. I've had great experiences working with everyone in the project: from spending late nights working with Josue, to laughing at nonsense with Anton, Abdul, or Ilya, or talking about the benefits of coconut water with Tony, everyone has been a real joy to work with and get to know. I'm really going to miss everyone. My biggest regret is simply that this course couldn't have happened in person, I want my Telescope 2.0 pizza party damn it. Guess I'll just order a party size pizza anyway and send the bill to Dave.

Chris, shortly after finishing OSD700 (kidding of course)

Moreover, today also marks my final day as a Seneca student! My time at Seneca hasn't always been as wonderful as these two courses have been, in fact it's been mostly the opposite. I never considered dropping out or changing programs though, just that perseverance can often be a challenging endeavour in and of its own right. Although, OSD really turned things around for me. I always looked towards the lectures and to the weekly labs and assignments with an excited fervour. These courses ended up being so important to me whereas I would nearly say your time in CPA/CPD is wasted by not having the privilege of taking these courses.

But enough fluff from me. Let's discuss what went on this week with me.

The Maintainer

Sunday! I finally re-approved Tony's PWA PR after we got the final go-ahead approval from Dave. Interestingly, Tony reached out to the PWA's library creator to talk about how updates work when we deploy new code. The creator responded saying the PWA is updated! Because of this the PWA was given the go ahead.

I manually tested an update to a dependency with Doc Josue. We found both instances where the library was used and forced them to perform, like lions, tigers, and bears at a circus. Oh my. Always get excellent guidance from our resident Doc.

I also reviewed Abdul's PR which adds a health check to the posts service. Short, simple, to the point. My favourite type of PR to review.

Monday, bloody Monday. Off to a poor start apparently! I raised an issue but Josue told me that Abdul had beaten me to it. A funny coincidence, as Abdul had pointed out an issue with the users microservice, and was kind enough to throw a ticket up about it.

I also raised another issue to change the Users Microservice port to something that is unrestricted.

T-T-Tuesday. Approved a PR from Yuan which changed something that was bothering me since I created the about page using MDX. I also had the privilege of leading the final triage meeting, which was kind of sad for me. I always enjoyed leading the discussion, even if people were half asleep. I also "helped" by adding screenshot confirmations a few times in both Slack and GitHub to demo the Users Microservice being used live for the first time!

Thursday. Start off by approving Yuan's new slack icon fix. She brought in react-icons, a library I've used several times before and really enjoy. I also approved a neat addition to satellite which adds the ability to cleanly shutdown a satellite-based service. I also approved another one of Yuan's PRs which updates our dark mode palette. It removes a lot of magic strings we relied on in a few places and properly adds them to the MUI theme file.

Friday! A very sad day as it's our last release day, and the last day of class. I started it off by approving Anton's attempt to speed up CI which unfortunately doesn't seem to make a huge difference, though a lot of larger companies employ such techniques regardless. I then went onto approve Dave's PR which removes tests running prior to shipping a release. He also had an excellent write up to fully explain the process of this.

I then had a short meeting with Abdul to audit the use of createError in the Users Microservice. We came to the conclusion that it was fine as it was. I later approved his PR. Looks like Abdul has the honour of shipping the last piece of code to Telescope before 2.0 landed. Kudos!

And that was it. I had figured that this last Friday would have been more chaotic, instead it was more neutral. I won't complain about getting some more time to work on my blog post though!

The Microservice

Josue and I started the week off by working on converting the unit tests for the Users Microservice to proper e2e tests. We both got stuck about half way through when several responses seemingly stopped cooperating and refused to properly return what we were requesting. I started working on it the next day and found where the stumbling occurred, we either weren't parsing the JSON via await response.json() or weren't converting the response text via await response.text(). Silly little mistakes that we got eventually. With those out of the way we continued on our way and eventually had a lot of great tests written. Josue helped me out a lot with some really neat promise tricks, an area where I still struggle a lot. On another note, I ended up ripping out a lot of the code that checked for hardcoded error messages. More sage wisdom provided to me, as error messages generated by our dependencies may randomly change, leading to failing tests. I also got a little lesson from Sage Dave on the proper use the fat-arrow in JS, or rather how to clearly return a promise. I have a bad habit of saving the response to a variable, only to later return it without performing any logic. An unnecessary task which I changed in the final PR. We finally landed our updated tests on Tuesday.

From here we transitioned to attempting to implement a health check API for Firestore, both the production version and the emulated version. Both of us spent about an hour trying to research potential solutions to this, but at the current time it doesn't seem like this is possible. I even asked a (my first, actually) question on stackoverflow regarding this and was met with neutral responses. I updated the ticket regarding this issue and we moved onto something else.

Finally, we followed this up by trying to work on how a user becomes an admin in the users microservice:

We tried to post & put a user with a Telescope user token (via Postman) and were denied.
We tried to post & put a user with a Telescope admin token and were granted access.

Given the following, we both came to the conclusion that this work was already done for us. During our final meeting, both of us spoke to Dave about how exactly we're going to limit a user becoming an admin. We received some advice on how to proceed further, which was essentially 'lock down everything', something that I look forward to implementing with Josue. We'll give it another shot ASAP.

In the mean time, we decided to work on replacing the feed parser, which is currently being deprecated but used in a few places in our backend. We started out by curling the feed parser wiki, editing it to only have 3 blog urls (one atom, one rss, and one xml), and running our own web server via npx http-server, so instead of having our backend parse several hundred blog feeds, we'll only parse 3 of the possible known types of blog feeds. A really amazing idea from Doc. We continued on by doing some research into promise-based feed parsing libraries, which was eventually concluded with Doc suggesting that we replace feedparser-promised with rss-parser. From here it's just a matter of manipulating the shape of the data that we feed the new library we're transition too. We called it quits from here, but we'll continue working on it ASAP. Although that about does it with my microservice contributions for the week, I'll try my best to not to make this the last time I work on the microservice.

I'm exceedingly thankful to be working with Josue. He has both taught me a lot, and shown me proper ways of doing all sorts of things with both programming and soft skills. He is an exceedingly talented programmer worth his weight in gold, and more. Both he and Dave are two people that I can only aspire to emulate, surely either of them have forgotten more than I know. I truly wish Josue only the best in his future endeavours and hope that we keep in touch.

The Finale

I'm not sure what else needs to be said, I'm only sure that there's things left unsaid (if that makes sense.) The feelings and experiences from OSD cannot be expressed in a single post. I think my experiences as a digital carpenter in this course can be expressed in the following: I learned how to navigate and contribute to a medium-sized project in 4-8 months, and you can too!

Some big moments for me are as follows:

When I got my first code merged in OSD600
When I contributed (several times) to plainoldrecipe, a repo publicized by hacker news.
My Hacktoberfest shirt arriving!
Breaking Telescope! (which I think was my first Telescope contribution, naturally reviewed by Josue who was likely cursing my name)
Finally landing the users microservice
Learning tonnes of devops such as CI/CD & Docker
Realizing how many technologies I've touched in the last 8 months. I've worked on so many different, modern, and relevant tech stacks that I can really apply my knowledge to a lot of interesting places and careers.

What's next? Well, I'm not sure. I still haven't decided if I'm going to McMaster University to upgrade to a Bachelor's Degree, or if I'll simply seek a job. I do know one thing, I really need a better chair.

So it goes. See you later everyone, and thanks for all the fish.

Dawn of the Final Week (Again)

Chris Pinkney — Sat, 17 Apr 2021 01:00:48 +0000

The Maintainer

Like most weeks we started with panic, but this time with a little extra gusto as it's finals week.

I started off the week by "reviewing" one of Dave's PRs. I say "reviewing" as the tests for the PR are currently not yet passing, as such I was left to only review the code posted and not test the PR locally. As always, the code seemed fine. I was really excited to see this PR as the users microservice is finally starting to seriously come together.

I also approved a quick-fix PR from Pedro for a clean $20, as it was discovered that our landing page didn't properly display viewport heights. My favourite kind of PRs are always the ones that are <= 10 SLOC.

Also approved a really interesting Docker related PR from Doc Josue! I had never thought about how containers and volumes dispose of unused elements until now. I guess I just magically assumed they would disappear when the container closes.

Reviewed some changes to the frontend made by Royce, I liked where it was heading, just not how the names were duplicated. We discussed it during our Friday meeting as well, and everyone seemed to share a general consensus about the duplicated names but also agreed that the line was an excellent touch.

I approved Tony's PWA PR again since we have a bunch of testing now. I think it's adequate, however, Dave has expressed reluctance to add it as we're currently unsure how the PWA will behave when we deploy an update to prod. Tony has since reached out to the maintainer of next-pwa for answer. As a result, I have removed my approval and apologized to Tony. Hopefully we'll get a reply soon and can get some answers so we can move forward with this. Still one of my favourite PRs.

I added my thoughts to a comment that Tony left on an issue suggesting a share icon feature addition to Telescope. Tony expressed hesitation about working on this feature, but I think working in open source forces collaboration, expression, critique, and commentary, and shared my opinion as such. I'm curious to see what we can do with this. A share button would be cool, especially if it copied the link URL to the clipboard, then a user can just ctrl-v to share it to someone. Might be a nice feature for new comers to the project to tackle!

I also reviewed Royce's new SearchHelp PR which changes how the helpful search instructions are displayed on our search page. I pointed out a bug (that may or may not be related to his PR). Lots of great frontend fixes lately from Royce!

Finally, in other humorous news, I left a really dumb review on something that is either not finished, failing CI, or both. Heh. I'll have to take another look at it when it's ready. GitHub is great but either I'm not great, or the UI is occasionally un-intuitive.

The Microservice

I started working on converting the unit tests inside the Users Microservice to proper e2e type tests tests two days ago but ran into a few snags. For the life of both Josue and I just could not get the microservice to respond properly:

// takes a user object, performs GET using the user's email
const getUser = async (user) => {
  const response = await fetch(`${USERS_URL}/${hash(user.email)}`, {
    headers: {
      Authorization: `bearer ${createServiceToken()}`,
      'Content-Type': 'application/json',
    },
  });

  return response;
};

const response = await getUser(allUsers[0]); <-- array of user objects

response.body simply refused to co-operate with us, and returned a bunch of crap that we didn't want. Give us the user, curse you! ~~Curse you and all the halflings!~~

The solution ended up being const user = await response.json() which allowed us to parse the JSON returned and make assertions based on the user returned via the GETs body. I'm currently sitting at 6 tests rewritten out of the... who knows, 15 or so that I have to do? I'm hoping to have the rest of them done by Sunday.

With only 1 week left I made it a point to issue a call for a potential roadmap for the Users Microservice. We still have a lot of work to do, but fortunately there's a bunch of us working on this stuff and a few items won't take too long.

After these e2e tests go in to replace the current unit tests I'll be shifting gears to work on how users are declared administrators, followed up by some research, and concluded with adding some quotes. Then finishing up what isn't done by end of week!

In other more personal news:

My first final went okay I think... hopefully.
Think I might buy a cherry tree for my backyard garden this year. I have no idea how trees work or how to care for them so hopefully I learn a lot...
- A society grows great when aged people plant trees in whose shade they know they shall never sit. 🌲
Continuing with my Rust fascination from the previous blog post, I stumbled upon a Rust cargo library which adds support for various microcontrollers. I have a Circuit Playground Express sitting in my basement. I bought it last year with the hopes of designing an automated garden watering system. Maybe I'll try it in Rust sometime!
I've always wanted to get involved in something impactful. I remember a website that I bookmarked for this just occasion! Maybe it will help you, dear reader, too.

Human Murmuration

Chris Pinkney — Sat, 10 Apr 2021 01:41:38 +0000

Credulous at best, your desire to believe in
Angels in the hearts of men

The week started off not with a bang but with a whimper, as I opened an issue that Davedalf the White noticed during our weekly triage meeting. I've been trying to reproduce the bug, but like Pedro I just couldn't no matter how hard I tried. Software development can be strange like this sometime, but unfortunately if this can't be reproduceable fixing the invisible will post quite the challenge. (Edit: As of this Friday, during our weekly deployment meeting we're currently under the impression that this is no longer an issue.)

This was followed up by a quick microservice meeting to discuss the latest status and the massive changes required to fully land the Users Microservice. I'll talk more about that later but the clock is ticking, t-minus 2 weeks and counting from today's date.

Josue and I had a quick meeting with Pedro to discuss some typing issues related to his latest PR which updated the Dynamic Image component we use on Telescope's home page. The meeting was concluded by the suggestion to implement MUI Types. I kind of wish I had learned more about typescript and NextJS but I think I'm okay focusing mostly on the backend for now.

After the meeting, Josue and I reviewed a PR from Davedalf, following some excellent advice to not test this locally (as it's not really something that's testable locally), but instead read the code and the tests. It's hard to understate the significance of brief (yet detailed) code explanations.

I also reviewed Yuan's latest PR which adds a really that hides the navbar on the initial render of our website, but shows it as the user scrolls down. I pointed out an issue that (presumably) caused this PR to clash with another newly landed PR from Duke which adds (awesome) scroll snapping. Unfortunately I don't know enough to help beyond pointing out flaws. Living the dream.

On a similar note, I also left my thoughts (such as they are) on Huy's PR which touches up the author section of Telescope. I also reviewed another of Dave's PRs, yet again joined by Doc Josue, which is helpful given that I can barely read a lot of the code he's pushing out lately. Having someone to dumb things down for you is also helpful.

I also had a brief meeting with Dave regarding some issues he was running into when attempting to POST users to the Users microservice in some tests. Yet again Doc Josue was able to save the day, and a fix PR arrived shortly after. It turned out our backend was mocking our node-fetch requests, resulting in data not being sent to the microservice.

I finished off the night by leaving some thoughts about a weird code escaping bug, which was conveniently being caught by another blog post by yours truly. I've worked on our backend's sanitizer previously so fortunately I have a small bit of "insight" into what may or may not be causing the problem.

Friday, the week is over but we still have to deploy and land all our PRs before hand. Here's a flurry of PR's that I approved:

A PR which (finally) allows user authentication on our Vercel deployments. Something I've been waiting for for a while now.
and also ... a followup PR to fix something that I noticed just broke with the Vercel fix!
I also left some notes on Illya's search microservice.
Finally, I also approved a last minute fix from Doc Josue which fixed our latest prod deployment.

Users Microservice

We finally landed a fix for our paginated GET route (something I tried my best to review) which, admittedly, had me nervous for a few days. The problem we had previously was that the paginated GET route that I created only worked for users who had IDs starting from 0. Since we hash our user IDs, this obviously is not a solution. The solution is actually really clever: it relies on keeping track and embedding where we left off in the response's header so the subsequent request has all the information where to continue from in the response header:

const query = await db
    .collection('users')
    .orderBy('id')
    .startAt(userToStartAt)
    .limit(perPage)
    .get();

vs.

let query = db.orderBy(documentId()).limit(perPage);

// If we were given a user ID to start after, use that document path to add .startAfter()
if (startAfter) {
    query = query.startAfter(startAfter);
}

const snapshot = await query.get();
const users = snapshot.docs.map((doc) => doc.data());

// Add paging link header if necessary, so caller can request next page
addNextLinkHeader(res, users, perPage);

module.exports.addNextLinkHeader = function (res, users, perPage) {
    // If there aren't any results, there's no "next" page to get
    if (!users.length) {
        return;
    }

    // Similarly, if the number of users is less than the perPage size,
    // don't bother adding a next link, since there aren't going to be more.
    if (users.length < perPage) {
        return;
    }

    // Get the id of the last user in this page of results
    const lastUser = users[users.length - 1];
    const lastId = lastUser.id;

    // Construct the body of the header, giving the URI to use for the next page:
    // '<https://api.telescope.cdot.systems/v1/users?start_after=6Xoj0UXOW3&per_page=100>; rel="next"'
    const link = new LinkHeader();
    link.refs.push({ uri: `${USERS_URL}?start_after=${lastId}&per_page=${perPage}`, rel: 'next' });

    res.set('Link', link.toString());
};

Clever fixes like these are amazing.

Aside from going over this PR a few times, Josue and I wrote a tool to help export users from the Planet CDOT Feed List (a list of Telescope users and their blog information.) It... actually didn't turn out bad at all! The code is easy to read, maintainable, and best of all, short. It went through a few rounds of reviews (a new personal best for me.)

I also started initial discussions and research into proper e2e testing for the Users microservice, and implementing our own Redis cache. More on that to come by next Friday.

Overall a good week which I spent a lot of time reviewing, commenting on stuff, and having meetings.

In other more personal news:

Been playing a bit of Assassin's Creed Valhalla, it's a solid 6/10 which (in the few hours I've played) has been otherwise enjoyable.
Been thinking about getting into Rust a lot lately. All this JavaScript lately has been making me itch to get back into a "lower level" language.
- I really wish I had pushed myself more in OSD600 to try something new. I don't regret my time using Python at all (as it was also new to me), but I guess the grass is, in fact, always brighter on the other side. Or maybe it's always rustier? Who knows.
Still enjoying The Way of Kings a lot. Finally on part 3 after starting this book nearly 4 months ago. Highly recommend the graphic audio version if anyone else is an audio book fan (they also have a preview on their website which is awesome)
Wish me luck with my finals start next week.

This Horrifying Force (The Desire To Merge)

Chris Pinkney — Sat, 03 Apr 2021 17:53:40 +0000

The weekend crept around like a bad habit. I spent it working at my job and making progress on my two PRs which couldn't be landed in our Friday release due to ~~an acute programming deficiency~~ time and review constraints. Both PRs went through a rampant series of reviews before they could be landed, which I am consistently and eternally grateful towards. Telescope would look like a dog's breakfast were I put in charge of approving or disapproving code integrations. Plus it's always good to have someone better than you review your code:

But we'll get into the users micro-service stuff shortly. Switching gears to some Telescope maintainer work, I approved a PR which adds Portainer, a container management GUI (which I had explored a few days prior with Josue) which makes the Telescope admin's lives much easier. I used to host a lot of game servers when I was younger (still do) so I love to play around with software like this.

I also approved a small PR which fixes a typo we had in our CSS, and another fixing the way we display hyphens in Telescope users names. The latter PR had some pretty interesting discussion based around how to approach this issue, how should we display long names which contain hyphens? Should we break the name onto a newline, or split the name at the hyphen? Eventually we decided to simply split onto a newline after encountering a hyphen.

I suggested the use of Formik for our new user sign-up page PR. The sign-up page will eventually be sending post requests to the Users microservice to allow Seneca Students to create accounts and add their blog feeds/GitHub info to their profiles. Naturally it's imperative that we employ double-validation (a word which I totally didn't create just now), meaning we should be validating information that users can input in both the frontend and the backend.

I also lead our weekly triage meeting, with my co-pilot Royce. I think it went really well. I certainly didn't feel as nervous as I did the first time leading.

Users Microservice

One change of note to one of my PRs mentioned above, was an idea suggested by Head Wizard Davealdore to validate the query parameters used in the users micro-service even further. We currently have code in the Users microservice which parses each parameter passed in when executing a GET request to /?per_page=xxx&page=yyy to retrieve Telescope users. These values for the parameters (xxx and yyy above) can be between >= 1 and <= 100 for per_page, and >= 1 forpage:

const parsePerPage = req.query.per_page < 1 ? 100 : Math.min(parseInt(req.query.per_page, 10), 100);
const perPage = !parsePerPage || parsePerPage < 1 ? 100 : parsePerPage;

const parsePage = parseInt(req.query.page, 10);
const page = !parsePage || parsePage < 1 ? 1 : parsePage;

However, it was suggested to instead delegate some heavy lifting to Celebrate, a library we use in the backend as middleware to our routes, such that we don't have to entirely rely on weird hacky code like the above snippet. It looks something like this:

-    [Segments.QUERY]: {
-      per_page: Joi.number(),
-      page: Joi.number(),
+    [Segments.QUERY]: {
+      per_page: Joi.number().integer().min(1).max(100),
+      page: Joi.number().integer().min(1),

Not only does Celebrate now ensure that the parameters can ONLY be numbers (specifically integers), it ensures that they must be integers with a range between 1 and 100 for per_page and >= 1 for page. With this change also comes a much needed pruning of the spaghetti logic above, as since we're validating both page and per_page, we have no need to parse or specify the values, and the code can simply be changed to the follow:

-   const parsePerPage = req.query.per_page < 1 ? 100 : Math.min(parseInt(req.query.per_page, 10), 100);
-   const perPage = !parsePerPage || parsePerPage < 1 ? 100 : parsePerPage;
-
-   const parsePage = parseInt(req.query.page, 10);
-   const page = !parsePage || parsePage < 1 ? 1 : parsePage;
+   const { per_page: perPage, page } = req.query;

Awesome. I feel that every time I work with Celebrate I always come out feeling quite pleased with this library and with myself. When a JS library ups your self esteem you've probably been inside the house for too long. Sigh.

I also wanted to spend some time later in the week converting the unit tests in the Users microservice to e2e tests with Doc Josue, however a big PR from Head Wizard Davealdore was put up for review which changed a lot of the background and common files in the users microservice. Because of these changes I had to alter the order of my operations since a lot of files that I'd be touching are going to get changed after said PR lands. Thus I instead spent the night working towards finally landing my paginated PR instead, followed by a review to Davealdore's above mentioned PR. Also I think a certain head wizard needs a break... or a beer, or both, preferably both:

I also reviewed a PR about Jest, which added a watch mode to the e2e test runner. Constantly re-running tests was slowly driving me (further) into madness. I actually never thought that this had to be added, and instead figured it was just part of Jest and that I was, instead, being too lazy to google to run it.

This weekend I hope to finally get around to converting the users tests to proper e2e tests, submit more reviews to Dave's large PR, and ideally start working on adding our data to Redis as a cache between Firebase and Telescope. Happy Easter.

Hey Sailor

Chris Pinkney — Sat, 27 Mar 2021 02:43:01 +0000

I started off the week approving a simple but much needed PR from Yuan which shunk the title font size and added a link to the author's blog (my favourite part.) I then went onto approve YET ANOTHER pr from Miss Lee (who has been making some nice additions to our front end apparently) which re-adds our much needed admin buttons to our front end.

Next I set my sights on the ever polite Metropass (if that is his real name). I reviewed Mo's really cool PR and left my thoughts for him to digest. I had suggested that in addition to hardcoding how long we specify our Cache ages (i.e. how long the browser should cache a piece of data vs requesting a new piece of data all over again) the developer could alternatively pass a specific value to specify how long they want to cache their stuff (the ever technical word.)

The PR also reminded me of how switch cases finally got added to Python, I remember Googling how to do them in Python during OSD600 while working on link checker program, and since Python (at the time) didn't have them, I had to instead use if/else etc. This is kind of an ugly change if you ask me but not an entirely unwanted one.

I also threw some thoughts about a PR here, and finally I also reviewed Tony's PWA PR.

I remember talking to Tony near the beginning of the semester and us both agreeing to work on the PWA together (though we have since diverged paths massively since I'm currently obsessed with microservices) so I'm glad to actually see it being worked on. I have to say I'm really amazed at how simple it seemed to set this up. For some reason I was picturing doing something like React Native to get this work. Nope, simply import a library and Bob's you're uncle. Amazing. I even tested it on my phone and it worked beautifully. I was in shock, truly.

Finally, I gave my comrade Ilya a brief lesson on microservices (and satellite) since he's taking over managing a microservice. I'm really excited to see where it'll head to because I can finally talk and review microservices after my experience working on one for the last few weeks. Speaking of microservice...

Feeling undeservedly accomplished for now, I went back to touching (finishing?) up the Users Microservice. I had at least two goals that I wanted to accomplish this week: properly paginate the GET route, and fully set up the users microservice for prod. First thing was first so let's dive in:

I started off by working on paginating (a fancy word for saying "give me only a slice of the cake instead of the whole cake") the GET route for the microservice. After working on the issue for a while I stumbled upon a major issue: How can I request only n number of records and know where to start at when I don't have a point of reference? I can't just pump gas into my car and know when to stop, I need some sort of point of reference. Similarly I can't just request 20 records from the DB without saying where to start and stop from. How would the query know which 20 I'm requesting? The first 20? The second? The third? Etc. Can't I to request 1 page of 20 records, another page of the following 20, and a third page of another 20 records?

Generally in these situations there's something called an offset. I can request 20 records on the 5th page and simply offset which records I want by 20 * 5, thus ensuring that I get records 100-120. But not in Firestore! Another gotcya that's slowly pushing my away from the database that I once loved. The problem with this situation is that the offset method in Firestore requests ALL records in the DB as opposed to the few that I request. This is a problem when dealing with massive databases. If I have a database with 100,000 records, and I request 20, why should I pay for the bandwidth of requesting 100,000? (Probably so Google can charge you for it, but that's neither here nor there.)

I contacted Sage Dave and asked for some advice which left both of us in a stump. The solution I came up with is simply start from user 0 and work my way up from that when requesting n users. If a user has an id of 0, I can request 10 users on page 1 and 10 users on page 2, and since I know my starting point of reference I'll easily be able to request the first 20 users.

I finished my PR and threw it up for review. As with most of my code I'm getting good reviews with a lot of language-based semantic nitpicks. JS is not my forte. I mean, I have no forte, but if I did JS would not be it. I'm really starting to enjoy it though.

Next up is making sure that the Users Microservice is ready to be deployed on production. Since our code lives inside of Docker (with traffic managed by Traefik) I have to ensure that my microservice can both receive and send signals to the other microservices as required. The complicated part of this PR is differentiating between what environment the code is currently running in, and how to respond accordingly as a result.

When the microservice is running in dev mode, we have ensure that we're using the Firebase emulator and not the actual Firebase db (as to not incur usage charges when we're simply fixing code or adding features). How do you tell which code to run when though? This is a minor problem I struggled with a lot in this PR (I think mostly it's because my knowledge of Docker, Traefik, dev vs prod, is flaky at best). But my main challenge I faced with this PR was getting the emulator to work inside of Docker's dev environment (there's a lot of minute details and things to keep in mind with this issue, so I'll try to keep this brief.)

There's currently two dev versions to this microservice, a Docker version and a local version. Think of them as one in the same entity, just with a different coat of pain. The local version works flawlessly, so why doesn't the Docker version? I simply am not able to communicate with my microservice via Docker. WHY? It's maddening! I felt my sanity slip away while working on it. I explored every Google hit I could think of before relenting and asking for help from Doc Josue. After about 2 hours of us trying to figure this out, we came across the extremely obvious (in hindsight) solution.

You need a few things to ensure that the Firebase emulator functions properly:

You have to make sure that you specify a port and address in the firebase.json file.
You have to make sure that the projectIds match for both the emulator and the firebase config file.
You have to make sure that the FIRESTORE_EMULATOR_HOST environment variable is PROPERLY pointing to the emulated Firebase instance in question.

If you haven't guessed it, I was declaring the Docker address incorrectly: FIRESTORE_EMULATOR_HOST=localhost:8088 vs FIRESTORE_EMULATOR_HOST=firebase:8088. And it makes perfect sense too when you think about it. localhost does not exist to other Docker containers, thus saying "I want you (localhost:6666) to connect to Firebase at localhost:8088" is not applicable. localhost:8088 does not exist from one container to the next. Stupid. Very stupid of me. All we had to do was specify the Docker container's network address (via firebase:8088) and we were back in business. We also briefly tested deploying the microservice to prod using a real Firestore instance and I'm happy to report that everything works as expected!

Both PRs ended up taking much longer and being way more involved that I had thought either of them would be. I really happy that I stuck with it and managed to work through several blockers I had. I genuinely could not have done it without Doc Josue and Sage Dave as both issues required more pairs of eyes to finally figure out. Kudos to both of them. 🍻🍻🍻

In more personal news:

Currently listening to local Windsor band Woods of Ypres

I'm very excited that it's getting warming and I can finally start my garden up again. If anyone wants to request a specific fruit or veggie to grow now is the time, simply bring a 6 pack to share when you come to pick up the harvest. That or review my PRs. Preferably the former.
I finally got around to watching some of Dirty Money's season 2. It's just as good as season 1 so far.

Perseverance

Chris Pinkney — Sat, 20 Mar 2021 00:02:49 +0000

The week started off with me leading our weekly triage meeting, with Mo taking notes. There's not too much to talk about, as only a few students were wounded in the telescope attack. I decided to spend a good chunk of meeting time focusing on older issues, as seeing 100+ issues, some of them from 2019, makes my neat-freak skin crawl. I imagine that this is a project most medium sized projects deal with. I also imagine that larger scale projects simply don't. All in all I enjoyed leading the meeting but was surprisingly nervous throughout most of it.

I also lightly reviewed one of Sir Dave's PR as I happened to be playing musical chairs recently with various PORT values for the plethora of services we're all working on.

On Wednesday I continued to work on the excellent abundance of requested changes to my Users microservice by Mo, Abdul, Doc Josue, and Sir Dave. My all-time favourite requested change would have to be this one. Previously I had two functions which would create different users that I could save to my database (to test routes that return all users, for example) but this change lets me instantiate as many users as I'd like with one function, and simply pass in different information when required. Awesome. I had really wanted to do something like this but couldn't figure out how to syntactically get it down.

I also started working on a fix to the pathetically un-styled About page. I'm still not crazy about working on front-end stuff, but I'd still really like to get better at it, plus, in the immortal words of Doc Josue:

but you know, it's good to know a bit of everything

Although clearly he's never tried to ask three+ designers to all agree on a design of something, as I had several changes, both on the GitHub page and privately on slack, requested of me at the same time. Lots of fun! I did learn a lot about CSS in the process, and what I already knew was a big refresher for me as I'm not the greatest in styling stuff. I do however always enjoy doing CSS stuff... for about the first 30 minutes, then I just wish I had gone into medicine instead. Big thanks to Ilya who spent some time with me teaching me a lot about various CSS properties and breakpoints, and helping me get the PR to where it is today. Hopefully it'll land soon.

Thors-day! I finally finished my micro-service approved and landed! Many hours of hard work finally over, with more hours to come. I also got to use the shiny force button too!

I genuinely couldn't have done it without all the guidance and help given to me by my professor and peers. The one thing I love about open source work is the communication ability coupled with suggested changes. Learning from those better than you is incalculably valuable. I'm really glad that I stuck with it and got it done, although It didn't get completed in as timely of a manner as I'd have wanted.

Oh, and naturally my power went out half way-through my final rebase, which resulted in:

PS C:\Users\Chris\Documents\Code\Repos\telescope> git status
fatal: not a git repository (or any of the parent directories): .git

Lovely. I had to reclone my fork and continue on my way. Not the end of the world but definitely annoying.

So what's next? I'm not sure! Figuring out what to do is always part of the fun. I'd like to get this wired up to prod next, then do a test-run to see if everything works as expected. Then move onto addressing several security concerns. Finally, I'd like to start working on actually implementing the users micro-service!

Friday. I started out by reviewing a minor Husky fix by Josue. How strange that dependabot upgrading our web hook program actually caused it to stop working. I mean that genuinely not as a snarky comment. It seemed like functionality completely broke in between versions and none of us noticed it for a few weeks. Strange. Thankfully Doc patched us up and we're good to go.

I also very quickly approved this PR (generously explained to us during on Friday meeting call) which adds an authorized middleware to Satellite which ensures that users must be authorized to execute specific routes. I look forward to implementing this, as security is something I was concerned with.

Speaking of our Friday meeting, Royce spent some time working on a fix for inline img tags in Telescope which he was kind enough to explain to us. I didn't think a simple task like this would be so involved but given how different blog providers handle images differently than another, it makes sense that Telescope can't accommodate every blog provided equally. Royce proposed working on a fix to detect Blogspot (our current problem child blog platform) and handle those images differently from the rest. I'm curious to see how this will handle random RSS feeds (i.e. those not on medium/blogpost/dev/etc.) and if it'll be a problem.

Finally I also released a minor fix to my users micro-service's Dockerfile for prod. This is what I get for copying and pasting without looking through it. However the fix also fixes my OCD as I had just noticed that a few files were not in the same structure as the rest of the micro-services. Phew. Now I can finally sleep well at night.

An overall busy but quiet week for me. I'm really glad that I was able to get the microservice merged and I'm really excited to move further forward with it.

The Shippening: Part 1.8, Micro Madness

Chris Pinkney — Sat, 13 Mar 2021 00:56:50 +0000

Earlier this week I decided to pause progress of my microservice today to instead test @humphd's dockerized Firebase emulator. I needed to ensure this works for ~~three~~ two reasons:

~~To get a grade in OSD700 by testing a PR~~
To ensure that I could insert data into a local, traefik, dockeriz'd, offline, version of the Firebase Firestore.
To ensure a smooth integration into my User microservice.

Last week I participated in a hackathon and used Firebase as my db of choice. I ran into some issues that were preventing me from inserting data into my local version of the db. I had assumed that this was due to the fact that I had never specified a private API key to use. This is a big issue as having to specify a private API key to use the offline emulator means that we have to authenticate each and every developer who needs to use Firestore (this also ruins our potential CI pipeline.) Assuming the worst, I reported my grim findings during our triage meeting on Tuesday.

Evidently this is not the case as I did not specify one when testing this PR. Hopefully the fact that the emulated version now works is due to PEBKAC, RTFM, or both. I need to release a draft PR in order to get more devs to test my code.

I tested this PR by making a separate (i.e. bastardized) version of my current microservice work that I use to send data to my Firestore, and pointing it to the dockerized version. It sounds simple but there were a few things that I had to do first: I had to lighty reintroduce myself to Docker, fix my Ubuntu WSL2 install (god I need to get Ubuntu working with my fakeraid system...), and do some reading and catching up about Firestore Emulator. After some time I finally got it to work and reported by findings on the PR.

Anyway, testing this PR was a lot of fun and a headache at the same time. I imagine this feeling is a lot like raising children.

After finishing my review, I proceeded to continue working my microservice. It's finally nearing a point where I can release it to the wild for review. I needed (and am eternally grateful for) guidance, direction, and help to push this thing into a PR. The only thing that I lack now is time. Time, and brain cells mostly. It's getting late in the week and the last 10% of something always takes as much time as the 90% did. Well, maybe not 90% in this case, but it sure felt like it given all the work I've rushed into this in the last few days.

Here is my leftover todo list:

Finish up making some more tests, specifically ones that test Celebrate's validation rules a. Mostly done. I still could use several tests testing various smaller bits of code but those can always be added in later and spending too much time on this won't get a PR up sooner.
Migrate to https://www.npmjs.com/package/@firebase/rules-unit-testing because @firebase/testing is deprecated a. Done.
Create a basic README.md file a. Done. I'm rather proud of it actually. It does need some more work as I need to specify exactly how the service and docker + traefik work together to get data where it needs to go.
Implement a list of current deficiencies and issues to discuss when the draft PR goes up a. Done. Kind of. I've been keeping track of this but at lot of them have been fixed and a lot of things have changed in the last few days.
Implement Satellite a. Done. I was having some issues with this though, as after implementing Satellite all my tests were failing. My problem, as expected, was minor and silly: const { app } = require('../../index');, I wasn't de-structuring app from index, so all the function calls were failing as they didn't exist in app.
Dockerize the microservice a. Done. I'm a great admirer of Docker (and traefik) but it's still confusing to me. I just need more experience. Mostly I lack the context of how things are interconnected to each other via multiple Dockers + Traefik.
Add Firestore private key placeholders to env.production and env.staging a. Not done. I'm lacking both files so I'll need to iterate on this. Thanks, checklist!
~~Tidy up the package.json run commands using npm-run-all~~ a. Unnecessary since the entire microservice is run via Docker, having npm run commands are a thing of the past.
Create an update (put) route a. Done, along with accompanying tests.
Create a delete route a. Done, along with accompanying tests.
~~Add date/time users were created (mostly for funsies)~~ a. Not done. Scrapped for time.
~~Add date/time users were updated (again, for funsies)~~ a. Not done. Scrapped for time.
Migrate unnecessary package.json dependencies to dev-dependencies a. Done.
Finally, create a PR! a. Done!

So now that the microservice is finally up, I'm hoping for a lot of feedback. I still have things that I need to fix, though mostly minor things and documentation edits. I have a few tests that need to pass, and I'm hoping to make the time for it this week around my work schedule. I'd also like to expand on how exactly to test the PR on the PR page. Sometimes it's hard to translate what little goes on in my head, to fingers and keyboard.

Something else that needs answers that I didn't think about until now: How do I ensure my delete route can only be ran by admins? Any fellow proletariat can execute ~~the bourgeoisie~~ a delete route and delete a bunch of users. Questions, questions that need answers and riddles in the dark.

Minor list of things that did dun need doing:

Create a env.production and env.staging file
Clean up the readme
Fix the Dockerfile
Add a section on how to test the PR (on the PR page)
Fix the failing tests
- Expected value: {"displayName": "Galileo Galilei", "feeds": ["https://dev.to/feed/galileogalilei"], "firstName": "Galileo", "github": {"avatarUrl": "https://avatars.githubusercontent.com/u/7242003?s=460&u=733c50a2f50ba297ed30f6b5921a511c2f43bfee&v=4", "username": "galileogalilei"}, "id": 10001, "isAdmin": true, "isFlagged": true, "lastName": "Galilei"} vs.
- Received object: {"feeds": ["https://dev.to/feed/galileogalilei"], "firstName": "Galileo", "github": {"avatarUrl": "https://avatars.githubusercontent.com/u/7242003?s=460&u=733c50a2f50ba297ed30f6b5921a511c2f43bfee&v=4", "username": "galileogalilei"}, "id": 10001, "isAdmin": true, "isFlagged": true, "lastName": "Galilei"}
How are we going to do security?
Switch my port to 7777 since SOMEONE took my satanic panic port 6666, my disappointment is immeasurable and my day Is ruined.

Oh, and I just reviewed Tony's microservice! I unfortunately missed out on Abdul's and I kept my head buried in the sand for the last week or so trying to finish this up, so I'm glad I got to play around with someone else's. Got looks like a sweet library, that plus celebrate is really turning me towards NodeJS development.

Anyway that's it for me right now. Fun but busy week, and as always looking forward to the next one.

Hackathon update!

Chris Pinkney — Sat, 06 Mar 2021 03:34:42 +0000

Today marks the closing of my Hackathon week, aka the week where I neglected everything in the world, while hacking away on an app.

This week my partner and I participated in Seneca's Hackathon. It was an amazing experience that I highly recommend if you're looking to get humbled, to have a good time, or both. Preferably both. The hackathon featured keynote speakers, mentors, advice panels, and late night mixers!

There were several categories to participate in, but my partner and I decided to participate in the Digital Health/Vaccination Passport to Support category:

Enable organizations to verify health credentials for employees, students, customers, and visitors entering their site based on criteria specified by the organization. Privacy and integrity is central to the solution, and the digital wallet can allow individuals to maintain control of their personal health information and share it in a way that is secured, verifiable, and trusted. Participants can attempt to model processes to implement contact-tracing using digital health passports, to identify other individuals who accessed the same spaces/areas in a set period. Sponsors will provide instances and technical support for teams.

Every team in this category was given the following challenge (but given leeway to be as creative as possible while still maintaining the objectives outlined):

Your challenge here is to make it simple, safe and secure to check vaccination status of individual and help institutions such as your school or office to verify it, to ensure safety of the people returning to them.

Here is my mockup of the app I wanted to design:

Given that this is a category centered around privacy and integrity (two things which I value) naturally, this category spoke to me. It also seemed relatively easy given the uh... current climate of things around the world right now. My team's idea was to have a security and privacy centric app which allows users to quickly and easily display their vaccination and disease testing history with just a few clicks on a screen, which then spits out a QR code for easy reading and verification. We also wanted the app to also educate users on vaccines and diseases.

Luckily my wonderful girlfriend Hope curated information about various diseases (as advocated by the Ontario and Canadian government.) Another thing I wanted to focus on was life after COVID, so my team and I ensured the app also supported a wide variety of other diseases, such that the app doesn't succumb to a simple life of bit rot after this is all over. Over the three days of the hackathon I personally spent about 25-30 hours working on the app.

While we didn't win (and am too embarrassed to post the source code which is not hard to find) I really enjoyed my time and can't wait for the next one. My only two regrets is that it couldn't be held in person, and that I'll be graduating soon, meaning I'll be unable to participate in the next hackathon.

The hackathon was sponsored by Salesforce, Sightline Innovation, Microsoft, and others. I attended two keynotes speeches during my time, both very interesting, particularly the talk from a Microsoft CFO regarding the state of IOT and various industries facing integration and challenges . I have no experience in IOT but embedded devices are something that I've always wanted to get involved with. I asked a question during the keystone on how to get involved and was told to "pick up a raspberry pi and start hacking" (paraphrased.) So hey, maybe I'll do that?

10/10 experience, highly recommend. I'm excited to attend another hackathon outside of my school.

Insert catchy title here!

Chris Pinkney — Sat, 27 Feb 2021 01:52:30 +0000

I let this week get away from me due to all the chaos that comes from midterms. Ah well, so much for inbox 0.

Without further ado (i.e. excuses,) let's begin with what went down this week on the admin side of Telescope.

On Wednesday it was discovered that our new Telescope builds were failing to deploy to our staging server despite our CI being all green across the board:

Copying API_URL=https://dev.telescope.cdot.systems to NEXT_PUBLIC_API_URL
Using NEXT_PUBLIC_API_URL=https://dev.telescope.cdot.systems
info  - Creating an optimized production build...

Failed to compile.

./src/components/Posts/Post.tsx:101:54
Type error: Argument of type '{ month: string; day: string; year: string; }' is not assignable to parameter of type 'DateTimeFormatOptions'.
  Types of property 'year' are incompatible.
    Type 'string' is not assignable to type '"numeric" | "2-digit" | undefined'.
   99 |   const date = new Date(dateString);
  100 |   const options = { month: 'long', day: 'numeric', year: 'numeric' };
> 101 |   const formatted = new Intl.DateTimeFormat('en-CA', options).format(date);
      |                                                      ^
  102 |   return `Last Updated ${formatted}`;
  103 | };
  104 |

The failing file was Post.tsx, which strangely enough, was not touched for about 10 days. The builds were building just fine otherwise, it wasn't until we tried to build Telescope using Docker that we discovered the failure.

The fix was even stranger (old vs fix):

  const date = new Date(dateString);
  const options = { month: 'long', day: 'numeric', year: 'numeric' };
  const formatted = new Intl.DateTimeFormat('en-CA', options).format(date);

vs.

  const date: Date = new Date(dateString);
  const formatted = new Intl.DateTimeFormat('en-CA', {
    month: 'long',
    day: 'numeric',
    year: 'numeric',
  }).format(date);

I laughed about it with Josue before approving the fix 🤷 The lesson: have your CI deploy using the same tech as your prod server. Our CI does not build, run , and test using Docker and that's where the disconnect came from. One failed using Docker, and one did not.

Thanks Doc Josue!

I also minorly reviewed Tony's microservice. I didn't end up testing it locally, it's on my list of things to do tomorrow after work. It's great to see another microservice popping up that I can look at and borrow (~~steal~~) ideas from. Currently looking forward to more of this, plus Abdul's and Professor Dave's stuff.

User Microservice Update

Several updates happened this week to my microservice:

I finally got Firebase to work offline.
- It took a lot of trial and error, and blog posts mostly. I'm still not that great at reading technical instruction documentation.
This means I can implement testing!
- Since the db can now be manipulated offline, I can add and delete data to ensure that when we use the actual production version of the db, all our code that we've developed will work as intended.
- This also means that I can now start writing unit tests to ensure a solid relationship between Express and Firestore.
- The whole experience of this two pronged attack (offline mode + testing) has been really enlightening, it really displayed how quickly a prototype can be created, demoed, and implemented using some basic tooling. It doesn't matter if your db is on or offline, if you can test it then the code you created is guaranteed to work on a production version (sans outside interference from other tools like nginx, firewalls, etc.)
- I added several basic unit-tests using Supertest (after failing an initial attempt using Got)

I also got some really great feedback and direction from a little demo I presented to Professor X. Let's take a look (and laugh) at this week's goals from last week's blog post:

~~Get testing working with the emulator and Jest~~ Done!
Implement Satellite
Create some basic README.md files
~~Export my private key to .env (was having trouble with this last week)~~ Not possible in this form
~~Figure out how to utilize my User class/schema~~ Will likely be deleted
Create an update route
Create a delete route
Add date/time users were created (mostly for funsies)
Add date/time users were updated (again, for funsies)

Here is the updated todo list that I want to get done between now and our next big release (1.8 - March 12th 2021):

Finish up making some more tests, specifically ones that test Celebrate's validation rules
Migrate to https://www.npmjs.com/package/@firebase/rules-unit-testing because @firebase/testing is deprecated
Create a basic README.md file
Implement a list of current deficiencies and issues to discuss when the draft PR goes up
Implement Satellite
Dockerize the microservice
Add Firestore private key placeholders to env.production and env.staging
Tidy up the package.json run commands using npm-run-all
Create an update (put) route
Create a delete route
Add date/time users were created (mostly for funsies)
Add date/time users were updated (again, for funsies)
Migrate unnecessary package.json dependencies to dev-dependencies
Finally, create a draft PR!

I think that about does it for me this week. Hoping to get some more done next blog post to show off.

Other news:

I got an Aeropress on sale this week and some ground coffee at a local coffee shop, I'll never go back to my Kureig.
I think midterm week went okay for me. 🤞
I'm participating in Seneca's hackathon next week! It's my first and I'm hoping it'll be a lot of fun.

Il meglio è l'inimico del bene

Chris Pinkney — Sat, 20 Feb 2021 04:49:06 +0000

Another week done, another week in review. I'll talk first about my week then the progress I'm making with the User Microservice.

I started my Saturday off with a review on this minor PR here from Tony. MUI is really neat and I'd love to learn more about it, I love the idea of having a global theme and utilizing it throughout your entire web app. CSS libraries are spiffy.

Sunday Funday came around and I started it off by testing a PR from Mo which I concluded with adding instructions on how to test the changes locally. Unfortunately the PR was ultimately closed (as we no longer use Gatsby) but sometimes this is the nature in software development. Hell, most of Google's crap is removed several months after launching, I wonder how those engineers feel about that.

I concluded my Sunday reviews by testing this bug fix which was driving me up the wall every time I noticed it. Ilya does a lot of these little bug fixes which overtime increase the overall UX of our app. Love it.

Monday rolls around and I starting work on a pagebase that I can use for the About page that I designed using MDX.

I originally intended to dynamically render each and every doc in our app's doc folder using the dynamic routes feature of NextJS. With a similar layout (i.e. pagebase) each document will look the same when it's rendered. I want to get around to working on this but my primary focus right now is to finish the User Microservice.

I ran into an issue early on and couldn't for the life of me figure out how to make the .mdx file inherent the styles I passed but everything else is fine. I eventually had an idea to throw all the content into an about.md file and just import it in:

import MDXPageBase from '../pages/layouts/MDXPageBase';
import About from '../markdown-pages/about.md';
const AboutPage = () => {
  return (
    <>
      <MDXPageBase title='About Us'>
        <About />
      </MDXPageBase>
    </>
  );
};

export default AboutPage;

The pagebase's code (MDXPageBase) can be viewed here. I also might have reviewed my own PR:

I spent a bit more time working on this on Tuesday/Wednesday and finally made the PR on Wednesday. I also got some great help from Anton with a really neat use of destructuring to shorten some repeated code:

const {
    palette: {
      primary: { main },
    },
  } = theme;

I also discovered a new bug with out GitHub Contributors card which (thankfully) ol' reliable Ilya expressed interest in fixing.

Wednesday. Finished up pagebase for with some help from Tony (and a really great types link from Pedro.) I spent so long screwing around trying to get the theme working and just right in MDXPageBase so thank the gods for Tony, big shout out to him for the fix. Sent my PR and moved onto the next issue. Submitting a PR still makes me feel nervous.

Anyway, I then started working on my little SEO fix and jeez talk about a rabbit hole. This was supposed to be a 5 minute fix! It was anything but:

I started off by exploring all uses of meta tags in each and every component including the ones inside our _document file.
- I figured the smart play was to move the <Head> code (along with all its the accompanying <meta> tags) from _document to the SEO component because SEO renders <Head> and additional <meta> tags. This didn't work (see point 2.)
- I then tried another approach, removing all <Head> calls in SEO and returning only the <meta> tags, which would allow something like the following: <Head><SEO/></Head>. Also didn't work:
- TypeError: Cannot destructure property 'pathname' of 'Object(...)(...)' as it is null. Seems that _document doesn't like the use of const { pathname } = useRouter();. It only turned out then that I noticed the import for <Head> was different in _document and SEO... import Head from 'next/document' vs import Head from 'next/head'; So off to Google I went to discover the difference and if one or the other was unneeded (though at this point I started to panic, I started to feel as though there was no work around and we just have to have some sort of repeated code):
- For the curious reader: this link was super helpful, and this link was too. Turns out both <Head>s have separate purposes and as a result I think all the <meta> tags should stay as is and a minor tidy-up can take place in its stead.
- Disappointed! Thought I was on this really great refactor kick and now my PR is only going to contain... well, a 5 minute fix. I spent way longer on this than I should have but I learned some really interesting things about NextJS. I hung my head in shame, remade my branch, and started back refactoring SEO in place of all <Head> (next/head) calls, and made my PR.

Thor's-day. Tony and I discovered that our CI pipeline started to fail at around 11am or so:

npm ERR! code ETARGET
npm ERR! notarget No matching version found for @graphql-tools/merge@^6.2.8.
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.
npm ERR! notarget 
npm ERR! notarget It was specified as a dependency of '@graphql-tools/load'
npm ERR! notarget

We all discussed reasons for this and I did some digging around trying to find some info from GraphQL-Tools as to why and found this.

Also got some sage wisdom regarding this:

humphd  12:32 PM
it's a dep of a dep

humphd  12:32 PM
and npm is picking a patch release that doesn't exist

Anton Biriukov  12:33 PM
but if did not update our dep in the first place, how could've their dep got updated (which broke it)?
each package has to release a new version to update its own deps, right?

Abdulbasid Guled  12:34 PM
This is something I need to research, if dependencies within dependencies update on their own?

Anton Biriukov  12:34 PM
I was under impression that they should not

humphd  12:35 PM
they do with the lock file

Cool. TIL. A really bad day for someone on either the GraphQL team, or NPM, as GraphQL is pointing the finger at NPM for the reason their build failed. Pretty funny how a minor issue like this could affect potentially thousands of Gatsby and pages using GraphQL.

I also reviewed yet another MUI related PR from Tony.

Friday. Today. Our rush day I spent most of the morning prepping for our meeting, reviewing PRs, and catching up on the status of the project and our 1.7 release. Quick approvals in no particular order: This, this, this, and this. We all got a lot of work in this week and FINALLY shipped Telescope using only NextJS and TypeScript. Great for us, bad for all the students hard work that came before us. I can't wait until the next group of students hopefully does the same to us. We have a few minor bandaids to apply but for better or worse our mark remains on Telescope for the forseeable future.

The cycle of software continues evermore.

User Microservice

I tried to spend some time this week working on my User Microservice, though my primary focus being on getting my two other PRs in before 1.7.

Anyway, my focus for this week was to at least get Celebrate in and working as intended. I came across a really solid blog post by celebrate's creator here.

What is Celebrate?

celebrate is an express middleware function that wraps the joi validation library. This allows you to use this middleware in any single route, or globally, and ensure that all of your inputs are correct before any handler function.

A little backstory on the situation: I currently have a mini express server which will be making calls to a Firestore database, and receiving calls from a frontend containing data from users. Naturally, we have to ensure that the data being persisted is as clean as possible.

celebrate is perfect for this because it's flexible to utilize route validate either globally or specifically per route. I actually want to implement this globally in my capstone project as we use Passport and Google's OAuth ids, coupled with state to send and receiving data to our db, and for making payments via Stripe. Ensuring that a user currently has a valid ID in ALL backend calls can prevent anyone from using postman to persist garbage to our db.

Anyway, I spent a while working on celebrate and kept bashing my head against a wall as I just couldn't get the post route to cooperate with celebrate. I was doing everything right! Every post, every document, I even google'd other projects using celebrate. It looked fine! What was the issue?

Oversight. Oversight was the issue. It turns out the issue was router.use(bodyParser.urlencoded({ extended: true })); vs router.use(bodyParser.json());... for some reason I just assumed that the former contained that latter and thus I didn't need to specify bodyParser.json() Stupid. Really stupid. At least I got some wonderous sage wisdom (after wasting his time too in an attempt to help):

humphd 5:58 PM
don't stop your progress
just keep track of defects, write a test and add .skip() on it
perfect is the enemy of the good
the thing with how we do open source is that you acknowledge your bugs, and iterate to fix them
vs. only shipping when it's ready
ship all the time, and file then fix

One bug down, one to go:

router.get(
  '/user/:id',
  celebrate({
    [Segments.PARAMS]: {
      id: Joi.number().required(),
    },
  }),
  async (req, res, next) => {
    try {
      const userRef = db.collection('users').doc(req.params.id);
      const doc = await userRef.get();
      res.status(202).json(doc.data());
      }
    } catch (err) {
      next(err);
    }
  }
);

GET http://localhost:5555/user/123

yields the following error: Error: Value for argument "documentPath" is not a valid resource path. Path must be a non-empty string. which goes away when not using the celebrate middleware, and a user is retrieved successfully. I didn't let this slow me down and instead tried programming defensively for the time being:

router.get(
  '/user/:id',
  celebrate({
    [Segments.PARAMS]: {
      id: Joi.number().required(),
    },
  }),
  async (req, res, next) => {
    try {
      const userRef = db.collection('users').doc(req.params.id);
      const doc = await userRef.get();

      // placeholder if else until i can figure out celebrate
      if (!doc.exists) {
        console.log(`User data (id: ${doc.id}) was requested by ${req.ip} but could not be found.`);
        res.status(404).json({
          msg: `User data (id: ${doc.id}) was requested but could not be found.`,
        });
      } else {
        console.log(
          `User data (id: ${doc.id}) was requested by ${req.ip} and served successfully.`
        );
        res.status(202).json(doc.data());
      }
    } catch (err) {
      next(err);
    }
  }
);

This is a weird Firestore issue, my first instinct is that celebrate is converting the param 123 to a string or something funky. Unsure. It console logs as a number though so I'm not sure. Regardless, I'll actually listen for once and come back to this at a later time and date.

I also read a best practices doc from Google regarding Firestore which can be seen here.

I also wanted to write an update route for users, but it turns out that you can attach { merge: true } to a post request and send it off to Firestore to achieve very similar results. A found good link here explaining the difference, however I still think making a put / patch route is the way to approach this scenario as: update will update fields, but will fail if the document doesn't exist.

I also thought of exporting the celebrate schema that I'm using in both the get and post and throwing them into /config/celebrate.js file or something, so I can import it and make my requests a bit easier to read and understand (plus it'll look cooler):

import { celebratePost } from './config/celebrate';

router.post(
  '/user',
  celebratePost,
  async (req, res, next) => {
    try {
      await db.collection('users').doc(`${req.body.id}`).set(req.body, { merge: true });
      console.log(`Added user with id: ${req.body.id}:\n${JSON.stringify(req.body)}`);
      res.status(201).send(`Added user with id: ${req.body.id}:\n${JSON.stringify(req.body)}`);
    } catch (err) {
      next(err);
    }
  }
);

but this is just an idea I'm playing around with in my head. I'll probably forget about it right after I finish this blog post and never do it. We'll see.

I also started looking into Firebase Emulator. This major step is in its very early stages and I'm unsure how this will work with Jest, but I'm going to fart around with it all next week and hopefully have some progress to report back on. I also got Firestore to run locally, which is really sweet:

That about does it for me this week. It was busy and I think I got a lot done. I wanted to get more work into my microservice but c'est la vie. I'm hoping next week will be a major milestone for and I'll have a draft PR to present to the world.

Next week's microservice todo:

Get testing working with the emulator and Jest
Implement Satellite
Create some basic README.md files
Export my private key to .env (was having trouble with this last week)
Figure out how to utilize my User class/schema
Create an update route
Create a delete route
Add date/time users were created (mostly for funsies)
Add date/time users were updated (again, for funsies)

In Other News:

Found this really neat program here that converts a React Native frontend to a Windows/Mac app. I'm really interested in App development but have no experience in it at all. I want to pick up Java/Kotlin and Swift but figure React Native is the next best thing. And this is cool as hell.
Gojira released a new song on their upcoming album.
Found out about GitHub's skyline feature. Amazingly, you can export the render to an STL file, this making it 3d printable. I'll be trying to print it this week I think. Hopefully I can show it off next Friday.
We landed another fricken rover on Mars. I saw the livestream of it being launched around my birthday last year and set a reminder for this this week to watch it. Nasa even released the approximate time it would land on the Red Planet, and amazingly it was right. Isn't that insane?

Putting the 'Micro' in Microbrain

Chris Pinkney — Sat, 13 Feb 2021 03:33:43 +0000

Microbrain

...And we're back.

From outer space.

I just walked in to find you here with that sad look upon your face.

Monday-funday rolls around and I started the week off working on my About page issue. I'm finding this issue to be unexpectedly challenging, mostly due to the fact that my NextJS knowledge is not great, plus I'm trying to figure out how to style an MDX page using a combination of Pagebase and template at the same time, as our previous about page for Telescope was comprised of both pieces. In essence, my problem resolves around the fact that I can't just wrap plain markdown in an MUI theme. So right now I'm trying to import the about page content separately and style the entire about.mdx page. But this too is being problematic, yet also opportunistic. If I can solve this issue, I'll be able to simply import all of our other documents in our docs folder and render them accordingly.

However during our last triage meeting on Tuesday Royce raised a question asking if we can re-implement Pagebase, so He, Anton, Ilya and myself are going to have a meeting to discuss how to implement that. This might be a better alternative for me, and will probably be more elegant that the current spaghetti I'm cooking up. Mmmmm pasta.

Tuesday rolls around and and I'm still practicing away with my goal of GitHub inbox zero, all notifications clear and unfortunately all the fishy PRs have been hooked, lined, and sinker'd already. Hoping my pole snatches a few before the end of the week. In other news, Pedro's new design is looking REALLY nice and I'm really interested to see more.

We had a great triage meeting as usual, this week I was the desinated taker of ~~lives~~ notes, and I think I organized them pretty well all things considered.

Wednesday, everyone's least favourite day. Today I started working on a small meta tags issue that I raised a few days ago. I found that we're using <title>, <Head>, <meta>, both, neither, or using our SEO component, incorrectly. So I'm hoping to fix these smaller issues with this PR. As <meta> tags also relate to Accessibility, Anton's domain, I spoke to him about this as not to step on any toes, he gave me his blessing so I got to work. The way this week is looking I'll have the PR ready today or sometime later.

I'm also starting to review an older PR by Mo, which adds a neat frontend variable, a topic which I somehow always struggled with during the few times that I've tried to implement it. I made a comment on and I'll get back to it ASAP.

Thursday. I reviewed another PR from Mo and had to do some light research on what exactly legacy-peer-deps does. I also finally made the time to start some light work with my User Microservice (see below.)

Having a peer dependency means that your package needs a dependency that is the same exact dependency as the person installing your package. This is useful for packages like react that need to have a single copy of react-dom that is also used by the person installing it.

Source

Friday is here. I started off the morning by commenting on a PR that adds documentation, and you know how much I love me sum documentation. Moving on, I spent some time trying to test yet another one of Mo's PRs but for some reason I'm getting super strange errors. I followed up with Mo and everything looks fine on his end so I can only assume that it's something on my end. I was hoping to have a review for this up by today but Jest is not co-operating with me and I don't even know where to begin with these errors.

Our meeting was cool, I enjoy all of our meetings but the demos that some of us put on on Fridays always give me something to go ohhhhhh... ahhhhhhhh.... ouuuuuuuu at. We started off with Yuan and Anton demoing their respective works and investigations with Dependabot and Renovate. Both seem really cool. Dependabot is built into GitHub natively now, though Renovate seems more customizable (and more advanced.) I've always wanted to implement Dependabot to some repo, so maybe I'll do it for my capstone project. Looking forward to see more Dependabot in action.

Edit: I regret everything. Bot-sama plz.

Similarly, during the meeting I had a chance to demo my very (minor) work that I've done on the User Microservice the group, and got some really great feedback and direction which I'll discuss below.

I ended Friday with a little hangout session with Anton and Abdul where we spend an hour figuring out the best possible names for GitHub tags for Dependabot to use. It's very serious business.

User Microservice

For those who didn't read last week's blog post: I'm currently working with Anton on a way to migrate a large list of data (Telescope users names and blog URLs) to an external database. The database we agreed to work with is Google's Firebase.

Woohoo! I'm giving this current PR its own section because it's a lot to talk about (and also because I'm really excited about finally contributing something on the larger side to Telescope.) I had a meeting with Anton on Monday before our Triage meeting, which was a lot of fun, he's a great guy to work with. We partially scoped out the issue but had several questions we needed to raise during our meeting such as:

How does the current feed system work?
How does Seneca's authentication work, and what does it return (and how can we leverage this data for our issue)?
Since we're adding more data to the users (GitHub usernames & Avatar URLs) what should we do about our "legacy" users which won't have this information on file (until they sign into Telescope again?)
Do we want a user to store multiple feed URLs?
How should we parse users with multiple names?

Thankfully, all these questions (and more) got perfectly answered. We also got some feedback on what the user "schema" will look like (which you can see here.) I'm still trying to find my mix between brevity and verbosity, currently leaning more towards the latter as my weird anxious mind (imposter syndrome) makes me want to explain everything humanly possible (to the detriment of others, mind you.)

I'm very excited to start working on this, hopefully, if time permitting, I'll be starting on this later this week.

Feedback and Direction:

As mentioned after Friday's show-and-tell, I got some good comments, concerns and directions to take this feature in:

So Firebase requires a private API key in order to authenticate the developer's working on the DB. I'm currently storing this key in a .json file, which is pretty redundant given that we have a perfectly good .env file to utilize.
- After the meeting I went to try and implement the in the main .env file but ran into a few blockers:
- The JSON key is too large to fit inside of a single-lined string.
- Furthermore, I couldn't even import any environment values as they were all being listed as undefined. I wanted to look into this further but used enough time as it is so I dropped this issue for now. I'll ask for help soon*ish*.
~~Add some CRUD code to create a user to the db (with a randomly generated id.)~~
~~Add some CRUD code to read a user from the db (with a specific id.)~~
Figure out how to add unit tests, and work with the Firestore db offline (probably the most challenging.)
Implement middleware to handle data validation.

Things I'd like to do:

Implement pino logging
- We currently use pino in a few places in the project, I'd like to use it here too.

I hope to have a draft PR up for all of the above by Friday the 19th. Maybe a better approach is to implement a basic skeleton and create various issues to see if anyone is interested in trying their hand at this.

Other News:

Stanford made their iOS development course public on youtube which is awesome. Maybe I'll take a look into that sometime.
Attended a Seneca Honours Bachelor of Technology - Software Development Alumni where five graduates spoke about their history in the program and where they are now. Surprisingly two of the alumni gave shoutouts to David Humphrey and the OSD course! Again, extremely grateful that I'm here.

Blogpost #968-b

Chris Pinkney — Sat, 06 Feb 2021 01:32:09 +0000

TODAY THIS CODE UNIT WILL EXPLAIN THE GOING-ONS WITHIN THE LAST 168 HOURS [STOP]

IT WILL BE DRY AND BORING [STOP]

YOU HAVE BEEN WARNED [STOP]

UNCLOG YOUR EXHAUST PIPES AND PAY ATTENTION [STOP]

Actual picture of me, circa 2021

So I started this week off by helping Anton (a fellow Telescope contributor) test out a really neat addition to our CD pipeline: GitHub Action Deployment. Essentially, whenever we're ready to push a new version of the software out to prod we simply add a Git tag specifying the version of the software we're shipping, and push the tag to trigger the auto deployment routine. It also builds REALLY beautiful release notes. This is a really sweet addition that tickles my pipeline fancy (uhhh... wait). I worked with something similar to this in my OSD600 class, although my project was Python based as opposed to web based so it was great to learn more about this, the backend, and deployment.

Later in the week I reviewed a funny bug from Doctor Josue regarding our Netlify deployment: our deployments on Netlify were "failing" is the successive build had no major changes. This is annoying from a reviewer point of view, as each new PR which didn't require a rebuild of our site was automatically marked as failing our CI checks. But more importantly, our Netlify badge was INCORRECTLY LABELLING US AS FAILURES. We discussed this issue during our meeting, and this is a really great solution that I hadn't thought about. It just goes to show that sometimes the simplest solution is the best (even if it is just a temporary band-aid fix.) And now it proudly (forcefully?) displays our successful builds.

More excitingly, our SearchPage component landed, a component which was blocking my Search component, as it's dependent on SearchPage. I hastily reviewed it, and really appreciated the commentary in the code as it really helps a plebian such as I understand more about functional components and design decisions. Once day I'll be as good of a programmer as Calvin, but it is not today. As soon as it landed I got to work on finishing up my port of the Search page. In essence, Calvin's component does the actual searching in our database for user posts, and my component gives his work a page to be displayed on.

Although, I have to say I'm kind of conflicted on it. Calvin did such a great job on his SearchPage component that I don't see why we should even use my wrapper component at all. Moreover, we can just inject the SEO, Header, Banner, and Darkmode toggle button that my Search page renders using his PR. However, for completeness sake this mirrors how our Gatsby branch did it. Ah well. On a side note I think my descriptions and titles are getting to be pretty good. I also got a nice hint from my Professor which will lead me down a rabit-hole, my favourite place to be. I'll have to work on this with Anton later in the week as he's expressed interest in making our web-app more accessible-friendly and meta tags relates to this.

Speaking of Professor, I also got some really great reading material from a comment by the one and only Humphd about local storage. Coincidentally this is also something that I need to utilize for my on-going capstone project.

I also raised a new issue here about a warning we started to get recently as our React version is not on the latest version (v17.0) As NextJS depends on React, NextJS' next version (v11.0) depends on updating React. This lead me to read about the new features in React 17.0, which hilarious enough, add "nothing".

I also helped research (and discuss) a bug with our darkmode toggle button that Abdul and Anton both started noticing that I annoyingly couldn't reproduce on my end. Funnily enough, it was eventually discovered that the reason I couldn't reproduce this bug was due to the white darkmode toggle button blending in too well with our white background, essentially making it disappear. Even funnier, this issue only visible on widescreen monitors (my browser monitor is portrait.)

Lastly, I also find myself getting more and more interested in the world of Microservices. Ray and Professor Dave are starting a REALLY interesting discussion and I'm very interested to see where it goes.

Friday is finally upon us, and I started off my day by participating an a meeting with Doctor Josue, Anton, Ilya, Abdul, and Professor Dave. Unbeknownst to me, this meeting would span a whole five hours. (and this is how you know you truly love something: when five hours flies by and strangely enough you're still ready for more.) The purpose of the meeting was to help out Ilya, a fellow contributor, get his work in before our 1:30PM deadline. We made it in time and I learned some new git features (git cherry-pick), and some new TypeScript features by watching my fellows voraciously hack away at a new addition.

Our Friday triage meeting was as entertaining as always, although I disappointingly had to tell the group that I couldn't get my two issues in by our deadline. This week simply got away from me, but I think there's always a lesson to learn: plan better and don't over or under estimate the time cost of programming.

However, on a brighter note, the new feature that I took under my (broken) wing is looking positively amazing. The tl;dr is that I'm championing a new database addition to our project to keep track of our Planet Feed List, amongst other pieces of data. I'm not a database guy at all, but I do love to design new additions, and I love to play with APIs, so I get the best of both worlds here. I have a bit of basic experience using Firestore and I'm really looking forward to contributing something that I actually feel slightly confident working on. The best part about this feature is that we're planning on additional features which will built off this basework that I'll be doing! It's in its very immediate infancy right now so I'll discuss it further on next week's blog post, so keep an eye out for that. Or don't, I'm a robot not your mother.

That about does it from me this week. It excitingly exploded towards the end and I'm more and more eager to begin another week in the world of open source development.